mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-18 12:13:49 +00:00
36107ec8c9
* Fix ORPO text tokenization with processors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard ORPO tokenizer rewrite anchor * Resolve processor pad_token_id and preserve preference data collators for ORPO Two follow-ups so the text-only ORPO + VL processor path works end to end on top of the build_tokenized_answer and tokenize_row rewrites: 1. Add orpo_trainer_processor_pad_token to rewrite processing_class.pad_token_id in ORPOTrainer.__init__ to fall back to processing_class.tokenizer.pad_token_id when the processor itself has no pad_token_id (Qwen3-VL, Gemma-3, etc.). Without this, DPODataCollatorWithPadding(pad_token_id=processing_class.pad_token_id) raises AttributeError before training starts. 2. Stop the outer UnslothORPOTrainer.__init__ collator-swap from clobbering DPODataCollatorWithPadding when the tokenizer is a processor without .pad. The swap to TransformersDataCollatorForLanguageModeling is now only applied to LM-style collators, so ORPO/DPO/CPO/KTO keep their own prompt/chosen/ rejected handling. Otherwise the collator can't pad ORPO rows and raises "You should supply an encoding ... that includes input_ids" at train time. Verified with Qwen3-VL-2B-Instruct ORPO + text-only data (training completes to max_steps, no AttributeError, no collator error) and Llama-3.2-1B-Instruct ORPO (losses and grad-norms bit-exact identical to main, so the change is a true no-op for plain text tokenizers). Extends tests/python/test_orpo_processor_text_tokenizer.py with three new unit tests covering the pad_token_id rewriter. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| conftest.py | ||
| test_cross_platform_parity.py | ||
| test_dpo_vision_processor_passthrough.py | ||
| test_e2e_no_torch_sandbox.py | ||
| test_fast_sentence_transformer_redirect_lifecycle.py | ||
| test_flash_attn_install_python_stack.py | ||
| test_gpu_init_ldconfig_guard.py | ||
| test_install_python_stack.py | ||
| test_no_torch_filtering.py | ||
| test_orpo_processor_text_tokenizer.py | ||
| test_patch_trl_rl_trainers_defensive.py | ||
| test_studio_import_no_torch.py | ||
| test_tokenizers_and_torch_constraint.py | ||
| test_unsloth_run_tool_policy_resolver.py | ||