mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-10 00:08:58 +00:00
03349d1e05
* feat: support text-only loading of Gemma 3 27B via FastLanguageModel (skip SiglipVisionModel) * test: instantiate text-only Gemma3 model and assert no vision tower Existing tests were AST source-introspection plus a config-resolves-to- text-config check; none actually instantiated a model from the text-only config. Add a small integration test that builds a shrunken Gemma3TextConfig (CPU-cheap), instantiates the matching CausalLM class, and asserts the resulting model exposes the LM head and has no vision_tower or multi_modal_projector attribute. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Deduplicate _get_text_only_config into _utils for PR #5816 * Fall back to full model when a VLM has no text-only class for PR #5816 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Preserve quantization_config and clarify warning for text-only loading for PR #5816 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only take text-only path when the VLM has its own text decoder for PR #5816 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Convert source string-match assertions to AST checks per Gemini review * Load real VLM text weights on transformers 5.x for text-only mode in PR #5816 transformers >=5 changed Gemma3ForCausalLM base_model_prefix from language_model to model, so a VLM checkpoint's text weights (gemma3: language_model.model.*, gemma3n: model.language_model.*) no longer auto-strip onto the text decoder and were silently initialized random. Add a version-gated key_mapping that remaps them onto the text keys, returning None on transformers <5 where the prefix still strips and a mapping would break the load. Apply the same family-guarded remap on the load_in_fp8 offline path and for direct FastBaseModel callers, and remap quantization llm_int8_skip_modules off the wrapper prefix after stripping. Add a regression test that loads real VLM checkpoint weights (the prior tests only instantiated a fresh model so they missed this) and drop the bitsandbytes dependency from the quantization-config test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Separate FP8 text-only cache and hoist the text-only guard for PR #5816 Address review of the text-only changes: (1) _offline_quantize_to_fp8 produced different artifacts for text-only vs full VLM but reused the same <name>-fp8-<mode> cache dir, so one mode could load the other's saved model; decide text-only before the cache name and add a -text-only suffix. (2) FastBaseModel.from_pretrained rewrote the VLM auto class to AutoModelForCausalLM before loading auto_config and before the family check, leaving is_vlm wrong for the fast_inference/vLLM block; hoist the family-guarded text-only decision above those checks and drop the redundant later block. (3) Wire the text-only regression test into the curated CPU pytest job so it runs in CI across the transformers matrix. * Trim text-only code comments for PR #5816 Shorten and de-duplicate the comments added for the text-only loading work; keep the non-obvious rationale (the transformers >=5 base_model_prefix change) and drop the obvious parts. Comments only, no code changes; AST-based tests still pass on transformers 4.57.6 and 5.4.0. * Make text-only loading opt-in via a public text_only argument for PR #5816 Rename the internal _force_text_only flag to a public text_only parameter on FastLanguageModel, FastModel and FastBaseModel (and the fp8 helper), defaulting False on all three. Text-only loading is now opt-in (text_only=True) instead of forced on by FastLanguageModel; the family guard and key remap are unchanged. Updated the AST tests for the new parameter and forwarding. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim text-only code comments for clarity --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| conftest.py | ||
| test_construct_chat_template_validation.py | ||
| test_cross_platform_parity.py | ||
| test_dpo_vision_processor_passthrough.py | ||
| test_e2e_no_torch_sandbox.py | ||
| test_fast_language_model_text_only.py | ||
| test_fast_sentence_transformer_redirect_lifecycle.py | ||
| test_flash_attn_install_python_stack.py | ||
| test_gpu_init_ldconfig_guard.py | ||
| test_install_python_stack.py | ||
| test_no_torch_filtering.py | ||
| test_orpo_processor_text_tokenizer.py | ||
| test_patch_trl_rl_trainers_defensive.py | ||
| test_studio_import_no_torch.py | ||
| test_tokenizers_and_torch_constraint.py | ||
| test_unsloth_run_tool_policy_resolver.py | ||