mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-04 21:30:36 +00:00
43d3caf38b
* Studio: wire imatrix GGUF option and FP8/NVFP4 compressed export into the export UI GGUF export gains an importance-matrix toggle. When enabled it auto-downloads the upstream Unsloth imatrix for the base model (or uses a custom path), which unlocks the IQ low-bit quants iq2_xxs, iq2_m, iq3_xxs and iq4_xs. Merged export gains an FP8 / NVFP4 compressed-tensors precision selector that runs llm-compressor for vLLM. Backend threads imatrix_file through routes -> orchestrator -> worker -> export_gguf (both the local save and the hub push), and maps the new compressed format_type values onto the fp8/nvfp4 save_method, reporting the "<dir>-<suffix>" sibling output directory. Frontend adds the imatrix Switch on the GGUF card and a merged precision picker on the merged card, threaded through the export runtime store. Depends on unslothai/unsloth#6706 (save.py imatrix_file and compressed-tensors export) and unslothai/unsloth-zoo#839 (quantize_gguf imatrix flag). * Studio export: guard imatrix/compressed against older unsloth builds and force imatrix for IQ quants Addresses review feedback on the export wiring: - GGUF: pass imatrix_file only when set, so a plain no-imatrix export (e.g. Q4_K_M) no longer fails with an unexpected-keyword error against an unsloth build that predates the imatrix_file parameter. When imatrix is requested but unsupported, return a clear upgrade message instead of a TypeError. - Merged: gate FP8/NVFP4 compressed-tensors export on the installed unsloth actually supporting it, returning a clear message rather than a cryptic save_method failure. - Frontend: IQ quants (iq2_xxs, iq2_m, iq3_xxs, iq4_xs) are imatrix-only, so force the imatrix on when one is selected and lock the toggle, instead of submitting an IQ quant with no imatrix that llama.cpp would reject. Extends the backend tests for the new capability guards and the conditional kwarg wiring. * Studio: upload compressed merged models to the Hub without recompressing For an FP8/NVFP4 Hub export the model is already produced locally in the "<dir>-<suffix>" output. Uploading it directly with HfApi.upload_folder (mirroring export_base_model) avoids re-running the expensive compressed-tensors quantization a second time inside push_to_hub_merged, which for NVFP4 also re-runs calibration and risks OOM. Falls back to push_to_hub_merged when there is no local compressed output to reuse. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| export.py | ||
| orchestrator.py | ||
| worker.py | ||