mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-06 23:44:08 +00:00
7917c7828c
Some checks are pending
Core / Core (HF=default + TRL=default) (push) Waiting to run
Core / Core (HF=4.57.6 + TRL<1) (push) Waiting to run
Core / Core (HF=latest + TRL=latest) (push) Waiting to run
Core / llama.cpp build + smoke (push) Waiting to run
Cross-platform parity / parity (macos-latest) (push) Waiting to run
Cross-platform parity / parity (ubuntu-latest) (push) Waiting to run
Cross-platform parity / parity (windows-latest) (push) Waiting to run
Lint CI / Source lint (Python + shell + YAML + JSON + safety nets) (push) Waiting to run
MLX CI on Mac M1 / dispatch (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Security audit / advisory audit (pip + npm + cargo) (push) Waiting to run
Security audit / pip scan-packages :: extras (push) Waiting to run
Security audit / pip scan-packages :: studio (push) Waiting to run
Security audit / pip scan-packages :: hf-stack (push) Waiting to run
Security audit / npm scan-packages (Unsloth frontend tarballs) (push) Waiting to run
Security audit / workflow-trigger lint (pull_request_target / cache-poisoning) (push) Waiting to run
Security audit / pytest tests/security (push) Waiting to run
Security audit / npm provenance + new install-script diff (push) Waiting to run
Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Backend CI / (Python 3.10) (push) Waiting to run
Backend CI / (Python 3.11) (push) Waiting to run
Backend CI / (Python 3.12) (push) Waiting to run
Backend CI / (Python 3.13) (push) Waiting to run
Backend CI / Repo tests (CPU) (push) Waiting to run
Unsloth export capability / capability (macos-latest) (push) Waiting to run
Unsloth export capability / capability (ubuntu-latest) (push) Waiting to run
Unsloth export capability / capability (windows-latest) (push) Waiting to run
Frontend CI / Frontend build + bundle sanity (push) Waiting to run
Unsloth GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Unsloth GGUF CI / Tool calling Tests (push) Waiting to run
Unsloth GGUF CI / JSON, images (push) Waiting to run
Unsloth load-orchestrator CI / test (push) Waiting to run
Mac Studio API CI / Unsloth API & Auth Tests (push) Waiting to run
Mac Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Mac Studio GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio GGUF CI / JSON, images (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-14) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio UI CI / Chat UI Tests (push) Waiting to run
Mac Studio Update CI / Unsloth Updating Tests (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Windows Unsloth GGUF CI / Tool calling Tests (push) Waiting to run
Windows Unsloth GGUF CI / JSON, images (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / GPU prebuilt resolves without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 unit tests (VS 2026 / CMake guard) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2026) (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-2025-vs2026) (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
* feat(install): opt-in Vulkan llama.cpp backend and HIP gfx fallback (#7357) Add UNSLOTH_LLAMA_BACKEND=vulkan and --llama-backend vulkan to force the upstream Vulkan prebuilt on any host, persist llama_backend in the install marker, and re-assert it during Studio updates. On Windows AMD, auto-fallback to Vulkan when no detected gfx arch is in the upstream win-hip-radeon GPU_TARGETS set (e.g. gfx803 / RX 480). Mixed setups where at least one card is HIP-supported still default to HIP unless opted in. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(install): address Codex P2s on Vulkan gfx routing (#7357) Honor ROCm family tokens (gfx110X), include fork-supported gfx1103, require a known active gfx before auto-Vulkan, and base the HIP floor check on the visible-device target instead of every physical GPU in hipinfo. * Address Codex review: env namespace, physical-NVIDIA guard, test kwarg - llama_backend_from_env: stop reading UNSLOTH_LLAMA_CPP_BACKEND. That is a separate pre-existing setup variable meaning auto/cpu; setup.sh/setup.ps1 warn and ignore other values, so reading it here forced Vulkan behind that warning. Vulkan opt-in stays on UNSLOTH_LLAMA_BACKEND / UNSLOTH_FORCE_VULKAN. - _should_auto_vulkan_for_amd_windows: gate on not has_physical_nvidia (not merely has_usable_nvidia). A CUDA-masked NVIDIA card keeps has_physical_nvidia while has_usable_nvidia goes False; Vulkan ignores CUDA_VISIBLE_DEVICES and could enumerate the reserved card. Mirrors the Intel auto path. Explicit opt-in still overrides. - test fakes: validate_prebuilt_attempts/validate_prebuilt_choice gained a llama_backend kwarg; the four fake signatures in the fallback tests now accept it, clearing the TypeError that reddened Backend CI / Repo tests (CPU). Tests: UNSLOTH_LLAMA_CPP_BACKEND=vulkan no longer triggers Vulkan; hidden physical NVIDIA suppresses AMD auto-Vulkan while explicit opt-in overrides. * Keep gfx1034 on the ROCm path (fork gfx103X bundle covers it) The WINDOWS_HIP_PREBUILT_GFX_TARGETS allow-list omitted gfx1034, so _route_to_vulkan_prebuilt downgraded RX 6500/6400-class hosts to the upstream Vulkan prebuilt before published_rocm_choice_for_host could match the fork windows-rocm gfx103X bundle (whose members include gfx1034). Add gfx1034 to the allow-list and a regression test asserting it stays on the fork ROCm asset. * Fix auto-Vulkan stealing fork windows-rocm gfx908/gfx90a hosts for PR #7373 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Vulkan marker claiming a backend that was never installed for PR #7373 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the Vulkan backend routing comments for PR #7373 * Keep the visible-device-aware gfx when setup forwards --rocm-gfx setup.ps1 resolves the gfx arch from its own probe, and that pick is not fully visible-device aware: neither the hipinfo nor the amd-smi branch reads CUDA_VISIBLE_DEVICES, and the amd-smi branch matches a bare integer only, so a comma-separated HIP/ROCR mask such as 1,0 also falls back to GPU 0. The resulting arch was then forwarded through --rocm-gfx and replaced the arch detect_host() had already resolved for the runtime-visible GPU. On a mixed-AMD Windows host that flipped the auto-Vulkan decision: with GPU 0 gfx1100 and a masked-in gfx1010, the forward reinstated gfx1100, _should_auto_vulkan_for_amd_windows() saw a HIP-supported arch and the HIP bundle was installed for a GPU that cannot run it. Fold the forward in as a fill rather than a replacement: it still supplies the arch on amd-smi-only, driver-only and name-inferred hosts where the probe reports none, which is what --rocm-gfx exists for, but no longer overwrites a successfully detected active arch. An explicit UNSLOTH_ROCM_GFX_ARCH stays authoritative, since it is the documented manual override for hosts whose arch the probes get wrong. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope the Windows AMD Vulkan fallback per device and per repo Three follow-ups on the auto-Vulkan routing for #7357. Keep an explicit --rocm-gfx authoritative. The previous round stopped a forwarded gfx from replacing an arch detect_host() had already resolved, but --rocm-gfx is also the documented operator override for hosts whose probe is wrong or stale, and both arrive as the same argv. Narrow the advisory case to the two shapes setup can actually be describing: an arch the probe saw on this host (setup picked a different physical GPU of the same box), or a family label such as gfx110X, which is a bundle name the update path derives from the marker asset rather than a real GPU arch. Any other value is an override for an arch no probe reported and stays authoritative. Keeping family labels advisory also preserves the rule that an in-generation-but-unbuilt arch (gfx1033) is never upgraded into the gfx103X bundle. Do not auto-route to Vulkan from a HIP-only device mask. HIP_VISIBLE_DEVICES, ROCR_VISIBLE_DEVICES and CUDA_VISIBLE_DEVICES select the active arch, but the Vulkan runtime honours none of them: it enumerates through GGML_VK_VISIBLE_DEVICES and Vulkan ordinals in LlamaCppBackend._get_gpu_free_memory_vulkan. Masking down to a below-floor card therefore used to install a backend that could still enumerate the HIP-capable card the user deliberately hid, possibly one reserved for another workload. Require every physical AMD gfx to be below the floor, matching the has_physical_nvidia gate right above it. So the per-GPU list survives to that check, a forward that agrees with the probe no longer collapses rocm_gfx_targets to a single entry. Make the HIP support predicate repository-specific. The floor constant is a union of ggml-org's windows-hip gpu_targets and the fork's windows-rocm bundles, so it only answers "is this arch served" for the fork. With --published-repo ggml-org/llama.cpp, direct_upstream_release_plan() offers win-hip-radeon then CPU and never Vulkan, so the four fork-only archs (gfx908, gfx90a, gfx1034, gfx1103) were declared supported and fell through to CPU instead of the Vulkan bundle that would actually run. Add UPSTREAM_WINDOWS_HIP_GFX_TARGETS and select the set from the planned repo. * Keep probe-confirmed AMD GPUs in the physical list when a gfx is forwarded rocm_gfx_targets is the physical inventory _should_auto_vulkan_for_amd_windows() reads, so a forwarded --rocm-gfx that the probe never reported was deleting cards the probe had confirmed. On a mixed Windows AMD box whose active device is masked down to a below-floor card, a stale UNSLOTH_ROCM_GFX_ARCH or a name-inferred arch for the other GPU collapsed the list to that one arch, the floor check concluded no AMD GPU on the host reaches the Windows HIP prebuilt, and the install auto-fell back to Vulkan, which honours no HIP mask and would enumerate the reserved HIP-capable card. Add the forwarded arch to the list instead of replacing it: it selects the HIP target, it does not redefine what hardware is present. An empty probe still yields a single-entry list, so the driver-only Windows AMD host the forward exists for keeps its automatic Vulkan fallback, and an explicit --llama-backend vulkan is unaffected. * Do not auto-fall back to Vulkan when a HIP device mask filtered the probe hipinfo is itself a HIP application, and AMD documents HIP_VISIBLE_DEVICES as "only devices whose index is present in the sequence are visible to HIP", with that spelling recommended on Windows. Under a mask the Windows probe therefore enumerates the visible devices, so rocm_gfx_targets is what survived the mask rather than the physical inventory the auto-Vulkan floor check assumes. A masked-out gfx1100 next to a visible gfx803 made the check conclude that no AMD GPU on the box reaches the Windows HIP prebuilt and route the install to Vulkan, which honours none of these masks and would enumerate the reserved card. Decline to guess when a mask is set: the physical inventory is unknowable from a masked probe, so keep the HIP / fork / source path. This only ever turns the automatic fallback off, never on. The driver-only single-GPU host the fallback exists for sets no mask, an all-hiding "" / -1 mask is still handled as no active target rather than a partial view, and an explicit --llama-backend vulkan or UNSLOTH_LLAMA_BACKEND=vulkan is unaffected. Reading the physical inventory through an unmasked re-probe would also correct _pick_rocm_gfx_target, which indexes the token list by the mask value and so already assumes an unmasked probe. That is pre-existing behaviour on main and is left alone here. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Treat an all-hiding HIP device mask as suppressing the Vulkan fallback too The mask guard exempted an empty or -1 value on the grounds that the probe reports no active target under it, but that only holds for the probe: a forwarded --rocm-gfx still reconstructs an active arch, and setup infers that arch from the display-adapter name, which no HIP mask touches. A user who hid every AMD GPU from HIP could therefore still be auto-routed to Vulkan, which honours none of these masks and would then use all of them. That is the strongest form of the hazard the guard exists for, not an exemption from it. Presence of any of the three variables is now the whole test, which also removes the value parsing. An explicit --llama-backend vulkan or UNSLOTH_LLAMA_BACKEND is still unaffected. * Grant the fork-only Windows HIP coverage to the fork, not to every mirror The floor set is a union of the fork's windows-rocm bundles and only the fork is planned from its manifest: resolve_simple_install_release_plans() compares == DEFAULT_PUBLISHED_REPO and sends every other --published-repo through direct_upstream_release_plan(), whose AMD branch offers win-hip-radeon then CPU and never Vulkan. Exempting only the exact ggml-org spelling therefore told a mirror carrying upstream-standard assets that fork-only archs such as gfx1034, gfx1103 and gfx908 were HIP-served, landing them on HIP or CPU instead of the Vulkan bundle that would actually run. Gate on the fork instead. Matching the dispatch exactly, spelling included, also fixes a differently cased repo: that really does take the upstream path, so it must be answered with upstream coverage rather than the fork superset. An empty repo still defaults to the fork, as the resolver does. * Derive the Windows HIP gfx floor guard from the published manifest The guard compared WINDOWS_HIP_PREBUILT_GFX_TARGETS against a second hardcoded tuple in the same test file, so a windows-rocm arch newly published by the fork passed both. Affected hosts would then be routed off the hash-approved fork ROCm bundle onto an unhashed upstream Vulkan build with nothing failing. Read the fork's llama-prebuilt-manifest.json through the installer's own resolver instead, and assert the floor, the family labels, and the routing tuple all still cover what it publishes. The manifest ships only as a release asset, so an unreachable release skips with an explicit reason rather than flaking. Both literals match the manifest as published today. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Compress the Vulkan backend routing comments and docstrings for PR #7373 * Correct the family-label rationale in the Windows HIP coverage check The comment justified serving gfx103X / gfx110X against any repository by claiming upstream's windows-hip targets build every member of those families. The fork manifest maps gfx103X to gfx1030..1032 plus gfx1034 and gfx110X to gfx1100..1102 plus gfx1103, and UPSTREAM_WINDOWS_HIP_GFX_TARGETS carries neither gfx1034 nor gfx1103, so the stated reason is wrong even though the answer is right. State the real reason instead. A family label is a bundle name, not an arch, so the concrete GPU is unknown at this point; answering unsupported to cover the two uncovered members would move gfx1030..1032 and gfx1100..1102 off a working HIP build onto Vulkan for a card the label cannot identify. Those two archs still reach Vulkan through the concrete-arch branch below, which does answer per repository. Comment only. No behaviour change: the 5850-combination override sweep still reports 0 rocm_gfx_target changes, 0 auto_vulkan False to True flips and 680 True to False flips all backed by a probe-confirmed HIP GPU, and both the feature and override profile matrices are byte-identical. * Pin that a deliberate CPU install outranks Vulkan for PR #7373 UNSLOTH_LLAMA_CPP_BACKEND (setup.sh / setup.ps1, "auto" or "cpu") and UNSLOTH_LLAMA_BACKEND (this module, a backend name) are separate variables at separate layers, and both accept "cpu". setup translates its own =cpu into --force-cpu, which is what pins the CPU-only bundle on a GPU host and keeps Intel iGPU Vulkan crashes away (#7213), so no trigger this PR adds may outrank it. _route_to_vulkan_prebuilt already gets this right, since force_cpu short-circuits ahead of the forced, auto-Intel and auto-no-HIP triggers. Cover it so it stays that way: the matrix runs [Linux, Windows, macOS] x [NVIDIA, AMD, Intel, CPU only] x [unset, vulkan, hip, rocm, cpu] with the legacy UNSLOTH_FORCE_VULKAN set as well, and asserts the published bundle survives every one. WSL presents as Linux to this resolver, so it rides the Linux row. Also assert the guard is not vacuous: the same host still takes Vulkan once the CPU pin is gone, so the matrix cannot pass on a resolver that had simply stopped routing to Vulkan. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: LeoBorcherding <borchborchmail@gmail.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|---|---|---|
| .. | ||
| datasets | ||
| hardware | ||
| inference | ||
| models | ||
| paths | ||
| prebuilt | ||
| security | ||
| .gitkeep | ||
| __init__.py | ||
| _studio_release_build.py | ||
| api_errors.py | ||
| cache_cleanup.py | ||
| client_ip.py | ||
| coding_agents.py | ||
| cpu_threads.py | ||
| downsample.py | ||
| embedding_model_settings.py | ||
| helper_precache_settings.py | ||
| hf_cache_settings.py | ||
| hf_token_validation.py | ||
| hf_xet_fallback.py | ||
| hidden_models.py | ||
| host_policy.py | ||
| lifespan_shutdown.py | ||
| llama_cpp_freshness.py | ||
| llama_cpp_update.py | ||
| mlx_repair.py | ||
| native_path_leases.py | ||
| node_runtime.py | ||
| openai_auto_switch_settings.py | ||
| personalization_settings.py | ||
| preview_rate_limit.py | ||
| preview_sharing_settings.py | ||
| preview_token.py | ||
| process_lifetime.py | ||
| ssm_runtime.py | ||
| studio_version.py | ||
| subprocess_compat.py | ||
| training_runs.py | ||
| transformers_dtype.py | ||
| transformers_latest.py | ||
| transformers_version.py | ||
| update_status.py | ||
| upload_limits.py | ||
| utils.py | ||
| uv_path_safety.py | ||
| wheel_utils.py | ||
| whisper_cpp_freshness.py | ||
| whisper_cpp_update.py | ||