mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-30 10:04:02 +00:00
8b9ee5facb
* fix: keep offline GGUF export off the Hub for VLM tokenizers (#7481) Resolve cached snapshot directories before loading PreTrainedTokenizerFast during VLM processor fallback so transformers does not call is_base_mistral() -> model_info() when HF_HUB_OFFLINE is set. Also probe the local cache in _has_tokenizer_model instead of model_info when offline. Fixes unslothai/unsloth#7481 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: add real-cache offline GGUF integration checks for #7481 Download unsloth/gemma-3-270m-it-bnb-4bit (~430MB) and verify offline snapshot resolution and tokenizer load with network blocked. Full unsloth import tests remain GPU-gated. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: address Codex review on offline GGUF tokenizer paths (#7481) - Only rewrite Hub repo ids to cached snapshot dirs when offline - Copy tokenizer.model from cache offline in preserve_sentencepiece - Do not cache negative offline tokenizer.model probe results - Add regression tests for all three review items * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: probe HF cache before model_info for local-only GGUF saves (#7481) Always resolve tokenizer.model from the local Hub cache before calling model_info, and skip Hub metadata when the tokenizer was loaded with local_files_only or offline env vars. Fixes Codex review on PR #7482. * Fix lint blocker, false-green tests and offline defaults for PR #7482 Drop the two unused _env_says_offline imports that fail the Source lint import-hoist check. test_has_tokenizer_model_offline_skips_model_info and its local_files_only twin set model_info.side_effect = AssertionError, but _has_tokenizer_model wraps that call in "except Exception: return False", so the AssertionError was swallowed and both passed on the merge base with the fix absent. Assert model_info.call_count == 0 instead; both now fail on the base with assert 1 == 0. The real-cache integration tests called hf_hub_download and PreTrainedTokenizerFast directly, so they exercised plain huggingface_hub and passed identically on both trees. Route them through the resolver this PR adds, and gate the file at module level since importing unsloth needs a GPU host either way. _resolve_hub_repo_local_dir and _resolve_hub_repo_cached_file defaulted to local_files_only = False, so a helper named "resolve local dir" would download with backoff retries when called without the flag. Every caller already passes it explicitly, so default it closed. Use tempfile.gettempdir() rather than a hardcoded /tmp, which silently skipped both files on Windows, the platform in the bug report. Patch socket.socket connect rather than replacing the class, which broke isinstance checks. Wire the unit tests into the Bucket-A CI list; Repo tests (CPU) ignores tests/saving, so none of these ran anywhere. * docs: note transformers 4.57.2-5.5.4 window for local tokenizer resolve Name the version range where from_pretrained still probes model_info under local_files_only, and point at the 5.6.0 upstream fix so the helper can be removed once the supported floor moves past it. * fix: enable real-cache suite in offline GGUF integration runner Pass UNSLOTH_INTEGRATION_IMPORT=1 into the pytest subprocess so the documented runner actually executes the real-cache tests instead of reporting success after only the fake-cache unit file runs. * docs: note integration runner enables UNSLOTH_INTEGRATION_IMPORT Document that the runner sets the gate itself and still needs a host that can import unsloth. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep an explicit local_files_only load local-only at save time transformers takes local_files_only as an explicit from_pretrained parameter, so it never lands in tokenizer.init_kwargs, and _offline_aware_load restores HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE as soon as the load window closes. A VLM loaded with local_files_only = True but no offline env var therefore came back with the Hub repo id in name_or_path and nothing recording the request, so _tokenizer_wants_local_only returned False on the later save and _has_tokenizer_model fell through to HfApi.model_info - and then _preserve_sentencepiece_tokenizer_assets fetched tokenizer.model from the Hub with local_files_only = False. On a disconnected host that is a network wait before the export gives up. Stamp the load's local-only mode onto the returned processor and its tokenizer inside the forced-offline window, and honour that stamp in _tokenizer_wants_local_only, so the save path inherits the load's contract. Verified against a real hub-cache layout whose snapshot has tokenizer metadata but no tokenizer.model: before, one model_info call plus an hf_hub_download with local_files_only = False; after, zero model_info calls and cache probes only. Two tests added to tests/saving/test_offline_gguf_vlm_tokenizer_7481.py; both fail with the loader_utils hunk reverted and pass with it in place. * Carry the load's cache_dir through to saving for PR #7482 The local-only stamp added in e7b7400de preserved only the boolean. Saving still derived its cache from HF_HUB_CACHE or HF_HOME, which does not see a caller-supplied cache_dir, and FastBaseModel.from_pretrained threads one all the way down. So a local_files_only load against a custom cache missed on the probe, and the stamp then stopped the Hub fallback that used to cover it, and tokenizer.model was silently left out of the GGUF staging directory. Stamp the cache_dir alongside the local-only marker and prefer it at both sites in save.py that derive one from the environment. Reverting save.py alone, with the helper still present, fails the new test on behaviour. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Merge main and drop an unused import for PR #7482 Brings the branch up to date with main, which clears the stale Source lint blocker inherited from #7476 by taking studio/backend/utils/hardware/__init__.py out of this PR's changed-file set. pytest was imported in the new test file and never used, which the import-hoist check flags in its own right. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|---|---|---|
| .. | ||
| gpt-oss-merge | ||
| language_models | ||
| non_peft | ||
| text_to_speech_models | ||
| vision_models | ||
| run_offline_gguf_integration.py | ||
| test_compressed_export_schemes.py | ||
| test_export_api_surface.py | ||
| test_export_dispatch.py | ||
| test_fix_sentencepiece_gguf_robustness.py | ||
| test_fix_sentencepiece_tokenizer_guard.py | ||
| test_gguf_export_and_inference.py | ||
| test_gguf_single_pass_export.py | ||
| test_imatrix_export.py | ||
| test_is_gpt_oss_detection.py | ||
| test_llm_compressor_install_pin.py | ||
| test_normalize_tied_weights_keys.py | ||
| test_offline_gguf_real_cache_integration.py | ||
| test_offline_gguf_vlm_tokenizer_7481.py | ||
| test_patch_saving_none_tokenizer.py | ||
| test_preserve_tokenizer_eos_token.py | ||
| test_prewarm_base_model_hub_cache.py | ||
| test_quant_method_none_normalization.py | ||
| test_qwen3_5_vlm_full_finetune_key_remap.py | ||
| test_save_shell_injection.py | ||
| test_save_subprocess_utf8_encoding.py | ||
| test_torchao_remote_code_consent.py | ||
| test_unsloth_save.py | ||