mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-24 08:13:59 +00:00
a335c00605
* Export ACCELERATE_MIXED_PRECISION when bf16/fp16 is set explicitly
In the default path the only branch that exports ACCELERATE_MIXED_PRECISION
is gated on `(not use_bf16 and not use_fp16)`, i.e. on the user having set
neither flag. Downstream readers resolve the autocast dtype from that
variable with a hardcoded 'fp16' default:
trainer._autocast_dtype = torch.float16 \
if os.environ.get('ACCELERATE_MIXED_PRECISION', 'fp16') == 'fp16' \
else torch.bfloat16
(unsloth_zoo/rl_replacements.py, and unsloth/models/rl_replacements.py)
So passing bf16=True left an initially unset variable unset, the reader
fell back to 'fp16', and a float16 autocast was wrapped around a bfloat16
model. GRPO then crashed on the first step inside matmul_lora:
RuntimeError: self and mat2 must have the same dtype,
but got Half and BFloat16
Paradoxically the default configuration worked (the automatic branch runs
and exports 'bf16'), while explicitly selecting bf16 -- the configuration
recommended in #4891 -- crashed. The same failure also reproduces with
16-bit loading, so 4-bit quantization is not required.
Add an `elif use_bf16 or use_fp16` branch that exports the resolved
precision, restoring the invariant the readers assume. Placed last so the
force_float32 and UNSLOTH_MIXED_PRECISION branches keep precedence.
Fixes #4891
* Add regression tests for explicit bf16/fp16 precision export
Covers the #4891 gap and the precedence that must not change:
- explicit bf16=True exports ACCELERATE_MIXED_PRECISION='bf16'
- explicit fp16=True exports 'fp16'
- force_float32 still wins over the explicit-flag branch
- UNSLOTH_MIXED_PRECISION='bfloat16' still wins over it
The two export tests fail against the pre-fix template and pass with it;
the two precedence tests pass either way. Uses the existing _decide()
harness, which executes the real template block extracted from rl.py.
* Merge main and tighten the mixed-precision comment for PR #7534
Names the actual root cause: transformers exported ACCELERATE_MIXED_PRECISION
itself until 5.x dropped the write.
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| conftest.py | ||
| test_bitsandbytes_kernel_readiness.py | ||
| test_change_system_message.py | ||
| test_conftest_bitsandbytes_preimport.py | ||
| test_construct_chat_template_validation.py | ||
| test_cpo_processor_text_tokenizer.py | ||
| test_cross_platform_parity.py | ||
| test_dpo_vision_processor_passthrough.py | ||
| test_e2e_no_torch_sandbox.py | ||
| test_fast_language_model_text_only.py | ||
| test_fast_model_config_passthrough.py | ||
| test_fast_sentence_transformer_embedding_parity.py | ||
| test_fast_sentence_transformer_redirect_lifecycle.py | ||
| test_flash_attn_install_python_stack.py | ||
| test_get_lora_parameters_bias_fp8_block_size.py | ||
| test_get_lora_parameters_fp8_block_size.py | ||
| test_gpu_init_ldconfig_guard.py | ||
| test_grpo_ddp_model_config.py | ||
| test_import_without_bitsandbytes.py | ||
| test_install_python_stack.py | ||
| test_install_uv_override_space.py | ||
| test_mlx_public_trainer_api.py | ||
| test_no_torch_filtering.py | ||
| test_orpo_processor_text_tokenizer.py | ||
| test_pad_token_fix.py | ||
| test_patch_trl_rl_trainers_defensive.py | ||
| test_remove_special_tokens_no_bos.py | ||
| test_studio_import_no_torch.py | ||
| test_to_sharegpt_optional_none.py | ||
| test_tokenizers_and_torch_constraint.py | ||
| test_torchcodec_torch_compat.py | ||
| test_unsloth_run_tool_policy_resolver.py | ||
| test_v100_fullft_precision.py | ||
| test_vision_lora_targeting.py | ||
| test_windows_arm64_python_choice.py | ||
| test_windows_git_gate.py | ||
| test_windows_no_torch_setup.py | ||
| test_windows_vcredist_download_tls.py | ||