mirror of
https://github.com/unslothai/unsloth.git
synced 2026-07-09 15:58:41 +00:00
b5aef63c03
* Studio: resolve the repo-root MTP drafter after the MTP/ GGUF rename The Gemma 4 QAT GGUF repos renamed the higher-precision MTP/ subdir copies from gemma-4-...-<quant>-MTP.gguf to mtp-gemma-4-...-<quant>.gguf, so their basenames now start with the same mtp- prefix as the small repo-root drafter (mtp-gemma-4-E4B-it.gguf). The drafter selectors filtered candidates by a mtp- basename prefix and took the first in sort order. With the new names the MTP/ copies also match, and because MTP/ (uppercase) sorts before the lowercase root file, selection flipped to the large BF16 copy under MTP/ instead of the root drafter both functions document they should pick. Restrict both selectors, and the companion byte estimate, to root-level mtp-*.gguf so the MTP/ copies stay explicit-selection only: - core/inference/llama_cpp.py _pick_mtp (loader auto-download) - hub/utils/gguf_plan.py preferred_mtp_sibling (Hub variant plans) - routes/inference.py _remote_gguf_companion_bytes (VRAM headroom) Also reuse a drafter already in the local cache before downloading, so a device that already holds a copy on disk does not re-fetch it. Old-scheme names keep working (they have no root-level mtp- sibling to mis-select). Adds regression tests for the new naming, both selection paths, and the on-disk reuse. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate MTP drafter cache reuse to offline mode Reuse the cached drafter only when HF is offline. Online, route back through _download_companion_gguf/hf_hub_download so the current revision is checked (etag) and a changed drafter is refetched, matching the offline-only cross-snapshot reuse already used for the main GGUF. This avoids pairing freshly downloaded weights with a stale cached draft. Make the reuse tests offline and add an online-skips-reuse test. * Studio: prefer a root MTP drafter across all cached snapshots Offline reuse scanned snapshots one at a time and returned the first snapshot that held any drafter, only preferring root within it. A newer partial snapshot with just the MTP/ copy could shadow the small root drafter in an older snapshot. Collect drafters across all snapshots and prefer any repo-root file before an MTP/ copy. * Studio: keep newest-first snapshot order when reusing cached drafters Collecting root candidates and sorting by absolute snapshot path could pick a drafter from an older snapshot. _iter_hf_cache_snapshots yields newest first and the main GGUF is resolved in that order, so preserve it (root still preferred over MTP/ copies) to avoid pairing a fresh main weight with a stale drafter revision. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| routes | ||
| schemas | ||
| services | ||
| storage | ||
| tests | ||
| utils | ||
| workers | ||
| __init__.py | ||
| dependencies.py | ||