58 commits

Author	SHA1	Message	Date
Saicharan Ramineni	9d53656614	Make _uv_safe_path space-safe on macOS/Linux (#6503) (#6534) ... * Copy uv `-c`/`-r` paths to a space-free temp dir on macOS/Linux uv 0.11.x truncates a constraints/requirements path passed via `-c`/`-r` at the first space, so `unsloth studio` setup from a repo cloned under a path containing a space (e.g. `/Users/me/Open Source/unsloth`) fails with: error: File not found: `/Users/me/Open` _uv_safe_path() already worked around this on Windows via the 8.3 short path but returned the space-containing path unchanged on macOS/Linux, which have no 8.3 equivalent. Extend it to copy the (small, flat) constraints/requirements file into a space-free temp dir and hand uv the copy; the temp dirs are removed at process exit. Falls back to the original path on any error, so it is never worse than before. Refs unslothai/unsloth#6503 * Route UV_OVERRIDE through _uv_safe_path and fix temp-dir leak (#6503) The -c/-r fix did not cover UV_OVERRIDE, which uv also truncates at the first space. On Apple Silicon the overrides file is handed to uv via UV_OVERRIDE at install time (install_python_stack.py) and during the MLX self-heal (utils.mlx_repair), so a repo under a path containing a space still broke every uv call there. Move _uv_safe_path into backend.utils.uv_path_safety so both sites share it, and route UV_OVERRIDE through it. Also stop leaking the temp dir when shutil.copyfile fails after mkdtemp, and add tests for the UV_OVERRIDE channel, the TMPDIR-with-space fallback, the atexit cleanup, and the no-leak path. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-06-24 04:02:24 -07:00
oobabooga	1cc785e5a0	Studio: remove OpenEnv and other unused packages (#6585) ... * Studio: drop OpenEnv and unused ExecuTorch/open_spiel install deps * Studio: drop 8 more unused install deps from extras * Studio: restore tomli<3.11 for kernels; tidy dep-cleanup comments and tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: refresh scan-packages baseline for scipy _external + unsloth-zoo tests scipy moved its vendored array_api_compat from scipy/_lib to scipy/_external, so the four allowlisted array_api_compat __init__.py entries stopped matching and resurfaced as unsuppressed CRITICAL "Downloads and executes remote code" findings on all three pip scan-packages shards (extras, hf-stack, studio). Add the _external paths next to the existing _lib ones so both scipy layouts stay covered. Allowlist two unsloth-zoo test-file false positives now present in the hf-stack shard: tests/test_mlx_save_export_regressions.py (writes to /tmp dropper) and tests/test_mlx_trainer_internals.py (obfuscation plus exec/eval). Drop nine stale entries for packages removed from the Studio requirements and no longer in any shard closure (evaluate, pytest, hypothesis, kgb, langid), confirmed absent via with-deps resolution of all three shards. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-23 07:20:47 -07:00
Daniel Han	935f6c50ef	studio: tighten torchao Windows-ROCm comments and test docstrings (#6610)	2026-06-23 05:49:25 -07:00
Daniel Han	55c392ff7c	studio: fix sentence-transformers RAG embedder on Windows ROCm (torchao) (#6608) ... torchao has no working Windows ROCm build. transformers.quantizers imports it, and it loads torch's c10d distributed backend at module level, which the AMD Windows wheels omit (no RCCL). The import aborts, transformers can no longer expose PreTrainedModel, and the sentence-transformers embedder silently falls back to the llama-server GGUF embedder. Linux ROCm and NVIDIA are unaffected (the c10d ops are present / torchao is real there). The training and export workers already install the shared torchao stub before importing transformers, but the RAG embedder runs in the main backend process, which never did. Two fixes, both no-ops off Windows ROCm: - embeddings.py: install_torchao_windows_rocm_stub() before the first sentence-transformers import, so an already-installed torchao is neutralized (fixes existing venvs). - install_python_stack.py: stop installing torchao on Windows ROCm; it can only crash on import there, so new venvs never ship it. Add tests covering the embedder stub call and the install skip.	2026-06-23 05:39:02 -07:00
Daniel Han	e83d4ae072	Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) ... * Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH. shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed to report gcnArchName, which is why the arch came from the GPU-name table) and amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt). TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so the inductor/Triton cache lives under the user's Studio home, not the system drive root. Long paths are already enabled above so deep inductor paths still fit. unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32" warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because setup runs via unsloth.exe (the running launcher holds its own file). The 'studio update' flow still attempts it. CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after the AMD wheels land, so the log makes clear the final install is GPU-accelerated. Adds two regression tests covering the venv-internal vs external hipInfo gate. Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1 --local from this branch completed exit 0 with no DiskPart prompt, no rename warning, the cache under the Studio home, and "GPU ROCm PyTorch installed (gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon 8060S Graphics". * Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning setup.ps1 used to rename the running unsloth.exe out of the way before the base-package upgrade so pip could replace it. That rename never actually worked: setup runs *via* unsloth.exe, so renaming our own running uv-trampoline launcher failed with a sharing violation (WinError 32) and only printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32' warning on every Windows install and update. It also was not needed. pip tolerates a running/locked console-script .exe: it moves the old one aside and writes the new one. The base upgrade routes through pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left intact either way. Removing the rename block and its failed-install restore block removes the false warning for all Windows devices in both the install and update flows. * Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks Follow-up to PR #6296. - install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python installers, which already do this. - amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv containment check (Windows paths are case-insensitive) and run the HIP_PATH/ROCM_PATH candidate through it too. - setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are unavailable, and create the dir wildcard-safely. - tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and assert the PowerShell venv exclusion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Windows installer: install ROCm PyTorch directly for a known AMD arch When the GPU arch is known (name-inferred from the GPU-name table) but ROCm could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The repo.amd.com wheels bundle their own runtime (no HIP SDK required), which setup.ps1 already relies on, so the CPU base was a pure wasted download/install. - Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts still get CPU (unchanged). - Fall back to a CPU base if the ROCm-index install fails, so a transient repo.amd.com outage does not abort the install (setup.ps1 retries ROCm). - Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK. - Add a regression test for the arch-based gate. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Windows installer: correct the unsloth.exe rename-removal comment The comment claimed the base upgrade 'routes through pip on Windows' and that pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what the code does. install_python_stack tries uv first; on a locked launcher uv aborts and falls back to pip, but the pip fallback strips --upgrade-package and base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already satisfied and no-ops. The running unsloth.exe is left intact at its current version either way. Reword the comment to describe the real uv-first / pip-fallback-no-op behavior. No functional change. * Windows installer: close two gaps in the venv-internal hipinfo exclusion Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases: - setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the venv-internal hipInfo.exe was not recognized. Seed the venv root from UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers). - The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run Test-HipinfoIsVenvInternal on the candidate as well (both installers). Extend the PS gate test to assert both. Both .ps1 parse clean; install tests pass (the venv-internal / HIP probe coverage at 359 passed). * Windows installer: correct the CPU-base message for arches with no ROCm wheels After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl and installs ROCm directly, so it no longer reaches the "temporary CPU base" branch. That branch is now reached only by a name-inferred arch with no ROCm wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old text ("setup replaces it with GPU ROCm wheels ... the final install IS GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch stays on CPU for this GPU. * Windows installer: seed the venv-internal hipInfo check from a custom Studio home Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the setup python, and the default %USERPROFILE% path only. A standalone `unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias) and none of those set would not recognize the venv hipInfo on PATH, reopening the amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert it in the gate test. * Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter Two review points on the amd-smi/DiskPart UAC gate: 1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo path, which does not resolve symlinks, junctions, or 8.3 short names. A venv reached through an aliased path then fails the check, so its bundled hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py). 2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a leading ~, while the canonical resolver does. With a tilde form, [IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the same way as the resolver. tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink realpath case and a setup.ps1 tilde-expansion guard). * Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1 Follow-up review on the same install.ps1 paths: 1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal) seeded the venv root from a custom Studio home without expanding a leading ~, unlike the canonical resolver and setup.ps1. A tilde form left [IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart gate. Expand ~ in the probe, matching the setup.ps1 fix. 2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch to below 2.12. AMD's per-arch index publishes the companions independently and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a bare resolve can pick an ABI-incompatible set and fall back to CPU. Add torchvision/torchaudio floor maps and pass the pinned specs, mirroring setup.ps1 and install_python_stack.py. 3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no retry), the only torch step in the file without it. Switch to Invoke-InstallCommandRetry so the recovery path survives a transient index failure. tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde check over both installers, a torch/companion floor-map parity test, and a CPU-fallback retry guard). * Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo, which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe inside the venv and the bnb fix (plus the Studio backend) prepend the venv Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is unchanged. Applied to all three Python copies (install_llama_prebuilt.py, install_python_stack.py, backend/utils/hardware/amd.py) via a new _external_hipinfo_on_path helper, and both PowerShell callers (install.ps1, setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal. tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a shadow-regression test for the exact venv-first ordering, and a parity check that every Python copy uses the scanning helper). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock) Auditing a dual native+WSL uninstall on a real device surfaced three leftovers: 1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the shortcuts that reference that icon, so Explorer's icon cache briefly held it open. Remove-Item -Recurse reported success yet left the locked file, and the dir was never re-attempted, so it orphaned with a false "removed" log. _RemovePath now verifies the path is actually gone (retrying transient locks) and reports honestly, and the data dir is re-swept after the shortcuts go. 2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when empty, in both the powershell.exe and drvfs-fallback paths. 3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of ~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock. Verified by running both uninstallers on a real dual install: device fully clean (no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left). * install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04 ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu 24.04. When the installer runs in a newer default distro (e.g. 26.04) it cannot enable the GPU and silently falls back to CPU. If a 24.04 distro already exists, re-run the install there and stop in the current one so the GPU path is taken without the user having to know about the distro requirement. Runs before venv creation so the wrong distro is left untouched, guards against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm distro alone (librocdxg present), and skips the GGUF-only / opt-out / non-Strix cases. When no 24.04 distro exists we keep today's behaviour: continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never auto-downloading a distro. Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the function, rewrites its paths to fixtures, mocks wsl.exe) covering the full decision matrix, wired into tests/run_all.sh. * uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico: install.sh points the WSL shortcut's icon there while the native install owns the dir. The native uninstaller removed the whole dir unconditionally, so uninstalling native while keeping WSL left the WSL shortcut with a blank icon. The old code only avoided this when Explorer happened to hold the icon open, which is unreliable; on a real dual install the dir was deleted and the WSL shortcut went blank. _RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving "Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it removes the dir as before. uninstall.sh still drops the icon and the empty dir when WSL itself is uninstalled, so every uninstall order ends clean. Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and runs it against a temp dir with controlled shortcut dirs) covering the dual, native-only, empty, and missing-dir cases, wired into the windows-inference smoke workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico and the WSL shortcut's icon stays intact. * installer: condense AMD/ROCm code comments (no behavior change) Tighten the comments added for the Strix Halo native+WSL installer work so they are shorter and clearer without losing intent: the venv-internal hipInfo amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and the dual-install uninstall icon handling. Comment-only; code paths unchanged. 107 insertions, 166 deletions across 11 files. * install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration, so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin distro (and ran venv migration in the distro about to be abandoned). Move the reroute ahead of the venv section so the origin distro is left untouched, matching the function's own comment. Behavior is identical on every non-reroute path. * installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04 - install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the flavor-repair block does not retry the failed ROCm index and abort the install; pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too. - install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo. - amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding quotes from PATH entries before probing for hipinfo. - install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04. - uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or another WSL distro) still references it, in both the powershell and drvfs paths. - tests: regression coverage for all of the above. * installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate - install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on --local; run the reroute BEFORE dependency/uv install so the origin distro is left untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later ROCm-on-WSL bootstrap does not install into the unsupported origin distro. - install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real executables match (not an alias/function named hipinfo). - uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs. - tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the user the install continues CPU-only, but only the failed-reroute branch set UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off that flag, so the other three branches could still install ROCm into the unsupported origin distro (e.g. 26.04). Set the skip guard on all of them. Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap carries through to the rerouted 24.04 child instead of dropping to the prompt path. install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty $env:USERPROFILE so Join-Path does not throw on a profile-less service account. Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE guard assertion, and route shell-test fixtures through a single trap-cleaned root. * installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch() installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure could abort the whole install even after install.ps1/setup.ps1 fell back to CPU. Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor maps) and make the retry nonfatal: keep the existing build and let the user re-run update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe. install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred); both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with only 22.04 reaches the GPU instead of staying CPU-only. install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child (PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder. _path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath can't classify every path on the drive as venv-internal (defensive; venv never at root). uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA). Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text + behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: match WSL reroute target by exact distro name, not substring The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests. * install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only) The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as GPU-supported let the parent report a successful reroute while the child fell back to CPU. Drop 22.04 from the supported set and the reroute target list; 24.04 stays the sole target (keeping the exact whole-line distro match). An already-working ROCm on any other version is still left alone by the librocdxg check above. tests: reroute 22.04 cases updated to the 24.04-only behavior; make the "no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes; stop the tauri exit-order check from mis-flagging the reroute helper's [ "$TAURI_MODE" = true ] && ... --tauri one-liner. * installer: tighten comment wording across the Strix Halo install/uninstall paths Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer, clearer lines. Comments and a few docstrings only; no code, logic, or behavior change. Verified with bash -n, the PowerShell parser, and ast.parse, and the installer test suite still passes. * add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them Every shell and PowerShell script under the Studio/installer surface now carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright header (after the shebang where present): the installer (install.sh, install.ps1), build.sh, the .github and src-tauri scripts, the installer test suite, and the moe kernel test. Header-only, line endings preserved; bash -n, the PowerShell parser, and the installer tests all pass. * installer: drop the duplicate AGPL header from install.sh and install.ps1 Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below their usage comment block; the prior header pass added a second one at the top because it only scanned the first few lines. Remove the duplicate so each file keeps a single original header. * installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path () so the genuine CPU-only install stays fast. install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO]) to ask the desktop app to elevate for the target distro; capture the child's exit code and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line) instead of masking it. CLI mode still falls back to CPU on a generic failure. Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases; run_func now preserves the child exit code; force-reinstall assertions for both PowerShell installers. Note: codex's _rr_q apostrophe finding is a false positive -- the helper already emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") }) collapses the single-element array to a scalar string, so @cpuForce splatting enumerated it character-by-character into broken single-letter args (- - f o r c e ...), which made uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag). Test now asserts the array-build form and rejects the if-expression form. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * uninstall: remove the isolated Node.js runtime (~/.unsloth/node) The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting <studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block the delete. Env/custom mode nests node under the custom root, removed with that root. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-22 03:09:08 -07:00
Lee Jackson	1eb15162d9	fix: clean up Studio warning log formatting (#6265) ... * feat: queue chat prompts during generation * fix: address prompt queue review edge cases * fix: harden queued prompt dispatch * fix: track queued prompt run state by thread * fix: preserve prompt queue ordering * fix: isolate prompt queue on new chat * fix: clean up Studio warning log formatting * Fix export log markup --------- Co-authored-by: wasimysaid <wasimysdev@gmail.com>	2026-06-19 05:06:03 -07:00
Daniel Han	f5f9e217c1	studio: select torchao version from the installed torch (#6400) ... * studio: select torchao version from the installed torch The Studio installer pins CUDA torch to torch>=2.4,<2.11 and its driver ladder selects the cu130 wheel index on recent NVIDIA drivers, so pip resolves torch 2.10.0. overrides.txt hard-pinned torchao==0.14.0, whose C++ extensions are built against torch 2.9.0, so torchao skipped its cpp kernels ("Skipping import of cpp extensions due to incompatible torch version 2.10.0+cu130 for torchao version 0.14.0") and fell back to the slow Python path. Every CUDA index now tops out at torch 2.10.0, so this hit most modern installs, not just cu130. Pick the torchao version matching the torch actually installed in the venv (table: pytorch/ao#2919): torch 2.10.x -> torchao 0.16.0, 2.11.x -> torchao 0.17.0, otherwise the previous 0.14.0 (so torch <=2.9 is unchanged). The installer reads torch.__version__ from the venv via a cross-platform sys.executable probe (probe_torch_wheel_env is Linux-only) and passes the computed spec positionally to the existing force-reinstall override step; overrides.txt becomes a pointer to that logic. torchao's Python API (Float8Tensor, used by unsloth/kernels/utils.py) imports cleanly on 0.16.0/0.17.0, verified against torch 2.9.1. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: address review on torchao selection - Clean the torch minor of pre-release/dev suffixes before parsing (e.g. '2.10rc1' -> minor 10), matching wheel_utils.probe_torch_wheel_env. - Pass _windows_hidden_subprocess_kwargs() to the torch-version probe so it does not flash a console window on Windows (no-op elsewhere). - Use _safe_print for the selection log line, consistent with the file's other status output (safe on non-UTF-8 consoles). --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-17 04:09:40 -07:00
Matt Van Horn	08c3878919	fix: use partial hipinfo output on crash to avoid CPU fallback (RDNA 4 / gfx1200) (#6292) ... * fix: use partial hipinfo output on crash to avoid CPU fallback (#6043) `hipinfo.exe` on some RDNA 4 hosts (e.g. RX 9060 XT / gfx1200) exits with STATUS_ACCESS_VIOLATION (0xC0000005) after printing the gcnArchName line. The previous guard `$LASTEXITCODE -eq 0` in studio/setup.ps1 and `if result.returncode == 0` in install_python_stack.py discarded this partial-but-valid output, causing the installer to fall through to WMI name inference which sets HasROCm=false and installs CPU PyTorch instead of the ROCm wheel. Fix: check for gcnArchName in stdout first; accept the arch regardless of exit code. Only fall through to the amd-smi / WMI path when no gcnArchName is present at all (crash before any output, or a genuine "no device" error). A cyan INFO substep is emitted when the arch is recovered from a crashed hipinfo run so users can see what happened. Adds a regression test covering the crash-with-valid-output path. Fixes #6043 * Fix/adjust hipinfo crash fallback for PR #6292 --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-15 13:26:04 +02:00
oobabooga	5300c047b6	Installer: drop the lemonade ROCm fallback now the fork ships identical per-gfx prebuilts (#6225) ... --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-12 11:53:26 -03:00
Daniel Han	2db9fad4b5	Installer: GPU detection follow-ups after #6174 (poisoned venv repair, llama.cpp routing, probe bounds) (#6183) ... * Installer: harden GPU detection follow-ups after #6174 Ports the NVIDIA-priority and /proc/driver/nvidia/gpus hardening from #6174 to the remaining pathways and adds recovery for already-poisoned venvs: - install_python_stack.py: add _ensure_cuda_torch so 'unsloth studio update' force-reinstalls CUDA torch when the venv carries a ROCm build on an NVIDIA Linux host (the pre-#6174 poisoning signature). Honors UNSLOTH_TORCH_BACKEND, UNSLOTH_ROCM_TORCH_INSTALLED, and CUDA_VISIBLE_DEVICES=-1/'' opt-outs; never touches healthy CUDA, deliberate CPU wheels, macOS, or Windows. - install_llama_prebuilt.py: detect_host gains the /proc NVIDIA fallback and skips ROCm probes when NVIDIA is usable; forwarded --rocm-gfx/--has-rocm overrides still win. - setup.sh: GPU summary classifies NVIDIA first through a timeout-bounded probe with the /proc fallback; AMD probes are bounded and gain a KFD vendor_id 4098 fallback; the llama.cpp source build only selects GGML_CUDA/GGML_HIP when the matching GPU is actually detected. - install.sh: bound both nvidia-smi calls with a 10s timeout (no behavior change when healthy or when the timeout binary is absent); classify the exported UNSLOTH_TORCH_BACKEND on the final index path segment so custom mirrors containing 'rocm'/'gfx' in their base path are not mislabeled. - install.ps1 + setup.ps1: NVIDIA probes now require a real 'GPU N:' row from nvidia-smi -L under a 10s bound instead of bare exit code 0; later CUDA version and compute_cap queries are bounded too. Tests: 3 new test files (50+ tests), suite at 788 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Resolve-CudaToolkit driver probe for extracted-function unit test tests/studio/test_resolve_cuda_toolkit.ps1 extracts Resolve-CudaToolkit alone into a child pwsh and stubs nvidia-smi with a .ps1 script. The bounded runner is not in scope there (and ProcessStartInfo cannot dispatch .ps1 stubs), so the DriverMaxCuda parse silently returned nothing and the major-mismatch scenarios failed. Fall back to direct invocation when Invoke-NvidiaSmiBounded is unavailable; production setup.ps1 always has it defined and keeps the 10s bound. * Treat CUDA_VISIBLE_DEVICES empty or -1 as hidden in NVIDIA-first guards The NVIDIA-first guards added in this branch only special-cased CUDA_VISIBLE_DEVICES=-1 at two setup.sh gates and ignored the empty-string form entirely, while the Python detector (install_llama_prebuilt.py) already treats both as hidden. On a mixed AMD+NVIDIA host steered to the AMD card via CUDA_VISIBLE_DEVICES, the guards suppressed the AMD probes, so setup.sh fell to a CPU llama.cpp build and install.sh picked CUDA wheels instead of ROCm. Move the policy into the helpers so every consumer agrees: - install.sh: new _cvd_hides_nvidia checked first in _has_usable_nvidia_gpu - studio/setup.sh: same via _setup_cvd_hides_nvidia; the two ad-hoc CUDA_VISIBLE_DEVICES=-1 gate conditions are now redundant and removed - studio/install_python_stack.py: _has_usable_nvidia_gpu returns False when CUDA_VISIBLE_DEVICES is set to or -1 (whitespace tolerated) Tests: 5 new sh scenarios (hidden via , -1, padded -1, visible device, and mixed host with hidden NVIDIA restoring the ROCm route) plus a pytest class covering all three implementations behaviourally. Addresses the review comment on the NVIDIA-first setup.sh block. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Retrigger CI after PyPI 503 outage during the previous run --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-11 05:06:02 -07:00
Leo Borcherding	bf2cd745b1	Fix installer selecting ROCm torch on NVIDIA Linux hosts (#6174) ... * fix: prevent ROCm torch from installing on NVIDIA Linux hosts NVIDIA's open kernel module (driver 560+) registers GPU topology nodes in the KFD sysfs hierarchy with non-zero gpu_id values. The _has_amd_rocm_gpu (install.sh) and _has_rocm_gpu (install_python_stack.py) sysfs fallbacks previously treated any non-zero gpu_id as proof of an AMD GPU, so an NVIDIA-only host with the open kernel driver was misrouted to the ROCm install path, replacing the correctly-installed CUDA torch with ROCm wheels. Fixes: 1. install.sh _has_amd_rocm_gpu sysfs fallback: require vendor_id 4098 (AMD 0x1002) in the KFD node properties file before declaring an AMD GPU present. NVIDIA KFD nodes carry vendor_id 4318 (0x10DE) and are now skipped. 2. install_python_stack.py _has_rocm_gpu sysfs fallback: same vendor_id guard. Also preserves the existing fallback for older kernels that don't ship a properties file (trusts gpu_id alone there). 3. install.sh now exports UNSLOTH_TORCH_BACKEND ("cuda"/"rocm"/"cpu") immediately after get_torch_index_url() resolves the wheel family. install_python_stack.py reads this as _TORCH_BACKEND and short-circuits _ensure_rocm_torch() entirely on cuda/cpu hosts, providing a second layer of defense that is independent of subprocess GPU detection. Tests: 9 new cases in TestHasRocmGpuKfdVendorGuard, TestEnsureRocmTorch, and TestInstallShStructure cover all three changes. Full test_rocm_support.py suite: 289 passed, 2 skipped, 0 failed. Closes #6172 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: show actual torch backend in progress step labels The 'ROCm torch check' and 'ROCm torch (final)' step labels were hardcoded regardless of whether the installer was targeting CUDA, ROCm, or CPU. On NVIDIA hosts they showed 'ROCm' even though no ROCm wheels were being installed, which was misleading. Add _torch_step_label(suffix) which reads UNSLOTH_TORCH_BACKEND (set by install.sh) and formats the label as e.g. 'torch check (cuda)' or 'torch final (rocm)'. Falls back to live GPU detection for standalone studio update runs that bypass install.sh. * fix: make KFD sysfs vendor check conservative -- skip if no properties file The previous implementation fell through to `return True` when the KFD node's properties file was missing (OSError), intending to support older kernels. But NVIDIA open driver KFD nodes can also lack a properties file on some kernel versions, so the fallback still produced a false positive. Change the `except OSError: pass` to `continue` so any node without a readable properties file is skipped rather than trusted. KFD properties files exist on every kernel version that actually exposes /sys/class/kfd, so this does not regress real AMD GPU detection -- if the directory exists at all, properties files will be present for genuine GPU nodes. * fix: bulletproof NVIDIA vs AMD GPU detection Four changes that together ensure ROCm torch can never be installed on an NVIDIA host regardless of which detection path fires: 1. _has_rocm_gpu() (Python): NVIDIA guard at the top -- returns False immediately when _has_usable_nvidia_gpu() is True, blocking rocminfo, amd-smi, and KFD sysfs from producing a false positive even when ROCm tools are co-installed alongside the NVIDIA driver. 2. _has_amd_rocm_gpu() (install.sh): same NVIDIA guard -- calls _has_usable_nvidia_gpu first and returns 1 if it succeeds. 3. _has_usable_nvidia_gpu() (Python): adds /proc/driver/nvidia/gpus/ sysfs fallback. The NVIDIA driver populates this directory on Linux regardless of nvidia-smi state, so a subprocess PATH gap, timeout, or driver initialisation race can no longer silence NVIDIA detection. 4. _has_usable_nvidia_gpu() (install.sh): same /proc/driver/nvidia/gpus fallback, tried after nvidia-smi -L rather than instead of it. Together: NVIDIA wins at every decision point. If nvidia-smi works, it confirms NVIDIA. If it fails, /proc/driver/nvidia confirms NVIDIA. If somehow both fail, _has_rocm_gpu still checks NVIDIA first before any AMD path runs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: two KFD/proc-only corner cases from Codex review 1. KFD awk state not reset per node file (Ryzen+NVIDIA false positive): The awk glob processes all topology node properties files in one pass. Without FNR==1 reset, a Ryzen+NVIDIA host where an AMD CPU-agent node sets amd=1 (vendor_id 4098, gpu_id 0) can combine with a later NVIDIA node setting gpu=1 (gpu_id > 0), triggering found=1 before vendor_id 4318 is seen. Added FNR==1{ gpu=0; amd=0 } to reset per file. 2. proc-only NVIDIA not reaching CUDA wheel selection: _has_usable_nvidia_gpu returning true via /proc/driver/nvidia fallback left _smi empty, so get_torch_index_url entered the AMD/CPU branch and selected CPU wheels despite NVIDIA being confirmed. Introduced _nvidia_detected flag (separate from _smi) so the AMD branch is skipped whenever NVIDIA is confirmed by any path, while _cuda_ver reads from _smi when available (with the existing cu126 fallback when _smi is absent). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-10 21:00:11 -07:00
Daniel Han	265c9f5db4	Fix UnboundLocalError in ROCm version detection dpkg/rpm fallback (#6149) ... * Fix UnboundLocalError in _detect_rocm_version dpkg/rpm fallback A leftover local import re inside the amd-smi branch made re function local for the whole scope. When amd-smi and hipconfig are absent and dpkg-query or rpm reports rocm-core, the epoch strip at the dpkg/rpm fallback hit re.sub before any local binding existed and crashed the installer with UnboundLocalError. Drop the local import (the module already imports re at top level) and add a regression test covering the dpkg path without hipconfig. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-10 08:10:55 -07:00
Bubu	8af9fe63a3	fix: persist Windows ROCm BNB version (#6048) ... * fix: persist Windows ROCm BNB version * style: apply kwarg spacing hook * fix: avoid persisting caller ROCm overrides * fix: redetect managed BNB ROCm defaults * style: apply ROCm guard test formatting --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-10 06:01:05 -07:00
Daniel Han	62191c4765	Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) ... * Fix Windows installer winget msstore certificate failure `winget install` was invoked without `--source winget`, so winget also queried the msstore source. When msstore fails certificate pinning (error 0x8a15005e, "The server certificate did not match any of the expected values") winget aborts and demands `--source`, so the Python (and uv) install fails even though the package exists in the winget source. - Pass `--source winget` to all winget install calls (Python x2, uv). Both packages live in the winget source, so this is strictly correct and skips the failing msstore round-trip entirely. - Add a python.org fallback (Install-PythonFromPythonOrg) that downloads the official installer and runs it silently per-user (no admin/UAC) when winget is unavailable or fails for any reason. Mirrors the existing uv -> astral.sh fallback so Python installs without manual steps. Resolves the latest 3.13.x from python.org with a pinned fallback, and selects the amd64/arm64/x86 installer per architecture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Pin remaining setup.ps1 winget calls to --source winget Two winget invocations in studio/setup.ps1 still queried all sources and could hit the same msstore certificate-pinning failure (0x8a15005e) that broke the Python install in install.ps1: - `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe) - `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server) Every other winget call in this file already passes `--source winget` (Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own Python 3.12 install), so these two were stragglers. Both packages live in the winget source; pinning it makes setup robust to an unhealthy msstore source, matching the rest of the file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Stop amd-smi GPU probe from popping a DiskPart UAC prompt On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs `amd-smi list` / `static --asic` / `version`. amd-smi (shipped in System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory details, surfacing a confusing DiskPart UAC prompt mid-install. The Studio backend already documents and circuit-breaks on this in studio/backend/utils/hardware/amd.py, but the installers did not. Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate (no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi cannot stall the install for minutes. On failure/timeout the existing WMI name -> gfx fallback still resolves the arch, so detection is unchanged on working hosts. Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone and the probe is bounded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151) install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but it does not install AMD's driver/ROCm stack -- a large, admin-gated prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151 GPU smoke test. A hard preflight refuses to run until the Adrenalin >=26.3.1 driver is actually present, so it cannot half-install. Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Detect AMD GPUs by name so native Windows gets a GPU llama.cpp The gfx-arch inference from the WMI GPU name was gated behind $HasROCm, which the hipinfo/amd-smi probe leaves false on the common Windows case (Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the arch without elevation). So an AMD GPU was detected by name but never mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup fell back to a CPU llama.cpp build. Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx, which makes install_llama_prebuilt.py download the matching lemonade-sdk ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs with just the Adrenalin driver. PyTorch's ROCm wheels still require a confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference and never pulls broken ROCm torch. Also broaden the name->arch table to every family lemonade ships Windows assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150 (RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths The earlier PowerShell guard covered install.ps1 / setup.ps1, but the Python installer (install_llama_prebuilt.py detect_host, install_python_stack.py ROCm probes) and the Studio backend monitor (amd.py) also shell out to amd-smi on Windows, where it auto-elevates and pops the same DiskPart UAC prompt mid-install / at runtime. Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on Windows so it runs un-elevated (no prompt). Callers already tolerate an empty/failed result and fall back to WMI / name detection (installer) or the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS amd-smi behaviour is unchanged. - install_llama_prebuilt.py: handled centrally in run_capture (covers detect_host's `amd-smi list` and the version probe). - install_python_stack.py: new _amd_smi_env() helper on its 3 raw subprocess.run amd-smi calls. - amd.py: merge RunAsInvoker into the existing child env. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Tighten AMD GPU name->arch patterns to avoid mismatches The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were speculative and over-broad: W9xxx would also match old GCN FirePro W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that fails validation), and RX 90[0-9]{2} was redundant with the explicit 9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched AMD names still fall back cleanly to CPU. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Fetch the llama.cpp validation model via huggingface_hub The prebuilt validation downloads a tiny GGUF test model from huggingface via bare urllib. On Windows / proxy setups where the server sends an incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so a perfectly good GPU prebuilt is rejected and the installer falls back to a CPU source build. Route the validation-model download through huggingface_hub (hf_hub_download) -- the same mechanism Studio uses for model downloads, which completes the chain where urllib cannot -- keeping the direct URL as a fallback. This lets the lemonade ROCm prebuilt validate and install on cert-restricted machines (verified: hf_hub_download succeeds where urllib returns CERTIFICATE_VERIFY_FAILED). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Guard the remaining raw amd-smi version probe via run_capture A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Forward --rocm-gfx even when the ROCm runtime is unconfirmed setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm stays false) the gfx arch was name-inferred but never forwarded, so install_llama_prebuilt.py saw has_rocm=False and installed the CPU build -- even though the lemonade gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime; verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB). Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as windows-hip in the existing-install mismatch check. --has-rocm stays gated on the confirmed-runtime signal. Verified on Radeon 8060S / gfx1151: the installer now selects, validates, and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present) instead of the CPU build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export) setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`. On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is false) the gfx arch was name-inferred but the ROCm-wheel branch never ran, so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export. Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels (repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without a HIP SDK; a failed install still falls back to CPU. Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics", GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Force amd-smi un-elevated process-wide in the Python installers Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match Start-Process -PassThru leaves the returned process object's .ExitCode $null after WaitForExit on Windows PowerShell 5.1, so the helper set $LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)` was always false -- the amd-smi GPU / gfx-token / ROCm-version detection branch was effectively dead (masked only because the un-gated WMI name->gfx inference still ran). Reproduced on PS 5.1.26100. Rewrite the helper to use [System.Diagnostics.Process]::Start with a ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable, with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER= RunAsInvoker (inherited via the process env) still suppresses the auto-elevation / DiskPart prompt. Also drops the temp files and the empty-ArgumentList edge case. Verified: exit code propagates (7 -> $LASTEXITCODE=7), output captured, env restored. Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the spaceless retail name "RX 7700S" is correctly excluded (it belongs to the gfx1102 row). Both found by PR review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Address PR review follow-ups (install.sh table, update path, tests, WSL) From the multi-agent PR review: - install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1 (the bash table had drifted to the old narrow patterns). Adds RDNA 2 (gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S" resolves correctly (bash case has no negative lookahead). AMD-ROCm-only: the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA / CPU / macOS are unaffected. - setup.ps1: the "dependencies up to date" fast path skipped the torch reinstall, so an existing user who had CPU torch (installed before ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch is known AND the installed torch is CPU-only, don't skip -- force the dependency pass so the ROCm wheels install. - scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg symlink (breaks if amdgpu-install lays ROCm under a patch-version dir); add a LIBROCDXG_REF pin knob and a "verified against" freshness header. - tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts, _fetch_validation_model_bytes (hf path + urllib fallback), run_capture's Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1 name-table parity (catches future drift). 14 tests, all passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK On Windows, amd-smi re-initialises the ROCm runtime on every invocation (even `amd-smi version`) and, on hosts without a working HIP runtime (consumer APUs/dGPUs with only the Adrenalin driver), elevates a child process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress that runtime elevation (verified: even `amd-smi version` hangs and times out with RunAsInvoker set). Replace the ineffective RunAsInvoker-only approach with a real gate: only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo present, so amd-smi runs un-elevated) or the user opts in with UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates and is untouched (no regression). RunAsInvoker is kept as harmless belt-and-suspenders for tools that DO use manifest elevation. Applied consistently across: - studio/backend/utils/hardware/amd.py (runtime GPU polling) - install.ps1, studio/setup.ps1 (install-time detection) - studio/install_llama_prebuilt.py (prebuilt arch probe + version) - studio/install_python_stack.py (ROCm version + arch probe) Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh install detects the GPU, installs ROCm torch (torch.cuda.is_available() True), launches Studio with no DiskPart prompt, and inference, tool calling, web search, LoRA finetuning, and GGUF export all run on the GPU. Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate assertions; update the three amd-smi monitoring tests to opt in (they mock amd-smi as available). Full suite: 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: helpful WSL message when the GPU isn't exposed to ROCm In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds nothing and we silently fall back to CPU. Add an actionable hint in the CPU-fallback path, shown only on WSL and only AFTER detection has already failed -- so it is forward-compatible: the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds Ubuntu 26.04 support), detection succeeds and the hint never fires. The message: - notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed, - lists the two prerequisites (Adrenalin driver + Ubuntu 24.04), - if the distro is not 24.04, says AMD may not support it yet, - tells the user to `wsl --install Ubuntu-24.04` and re-run, - links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper. Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted) and the CPU install completes; on Ubuntu-26.04 the extra "this distro may not be supported" line appears and points to 24.04. Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's repo.radeon.com/amdgpu-install/ is indexed by unified installer version (30.30, 31.30, ...), NOT ROCm version, so the hard-coded amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first for a noble .deb matching the target ROCm major.minor (ROCm 7.2 -> 30.30.x/amdgpu-install_7.2.x), falling back to the newest available. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151): 1. Shortcut collision (real bug). install.sh's WSL branch wrote "Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the native-Windows installer (install.ps1 New-StudioShortcuts). Running install.sh in WSL therefore silently retargeted the native shortcut at the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon stopped launching native GPU Studio. Now the WSL shortcut uses a DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the proper icon. Native and WSL shortcuts now coexist. 2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL compatibility matrix, gianni.rosagallina.com Feb-2026 guide, ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees only the CPU. install.sh's WSL hint and the experimental install_rocm_wsl_strixhalo.sh header/preflight now state the exact driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link AMD's radeon-ryzen docs, and document the known librocdxg caveat that usable VRAM is currently capped at the .wslconfig memory setting. bash -n clean; install test suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: hint when the AMD driver is too old for ROCm-on-WSL Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on native Windows can also be used inside WSL2, but only with AMD Adrenalin Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows GPU works with any recent driver, so this is purely about enabling the WSL path. We intentionally do NOT auto-install the driver: AMD referrer-gates driver downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish Adrenalin via winget, so no installer can reliably fetch it -- and silently swapping a live display driver is risky. Instead we point the user at AMD's official download page (one click), after which the existing WSL detection lights up automatically. - install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the installed driver predates the 26.2.2 release (DriverDate < 2026-02-01), print a concise tip with the AMD download URL. Handles DriverDate as either a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1. - install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note that AMD downloads are referrer-gated (open in a browser). Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01; install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.ps1: refresh shell icon cache after creating the shortcut After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh its icon (ie4uinit.exe -show). Without this, a stale icon cache can show a blank shortcut icon until the next explorer restart -- most visible when a shortcut of the same name was rewritten (e.g. a native install followed by a WSL install, which previously shared the name; now they use distinct names, but the cache nudge makes the icon appear immediately regardless). Best-effort and wrapped in try/catch so it never fails the install. The bundled unsloth.ico itself is valid (verified it renders). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid. Previously, if an AMD-GPU host ever fell through to the source build (e.g. the prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only, no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt is the AMD path and how to restore it (re-run / check network / set UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU. No behavior change on the happy path: AMD still gets the GPU prebuilt (verified on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA (CUDA source build) and CPU-only hosts are unchanged. setup.ps1 parses; install suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut Three gaps found by running a real uninstall on a native-Windows + WSL host; all fixes are scoped to Unsloth-owned paths and no-op on the other pathways (env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses. uninstall.ps1: - Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio, not under it -- so deleting <studio> left hundreds of MB behind. Now removed explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated content). No-op in env/custom mode (llama.cpp nests under the custom root, removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept. - New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp and (b) an orphaned multiprocessing python fork that ran from the SYSTEM python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle blocks the directory delete, leaving a half-removed install. The new helper kills any process whose image path OR loaded module is under a target root (module scan scoped to python/unsloth/llama-server names; vendor-agnostic). - _RemovePath now retries (transient post-kill handle release). uninstall.sh: - Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir ~/.unsloth only if empty. - WSL Windows-side shortcut cleanup now matches by TARGET (any "Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk" -- and never removes a native-Windows shortcut (which launches wscript.exe). uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut The Start Menu shortcut kept showing a blank/generic icon even after the Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost keeps its OWN pre-rendered tile-icon cache (%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\ TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch it, and they don't recycle the host -- so a rewritten same-name shortcut keeps showing the first-rendered (often the generic wscript ">") tile until the host restarts on its own. Fix: after creating the shortcut, drop only the Start Menu RENDER caches (TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost (Windows auto-relaunches it), so the tile re-resolves the real icon via the shell image factory. start2.bin (the user's pinned layout) is deliberately preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and wrapped in try/catch so it can never fail the install. Windows-only (install.ps1); no effect on Linux/macOS/Studio. Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched, after this invalidation -- previously it returned the generic script tile. install.ps1 parses; install suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo (Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026): rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool. Every change is a strict no-op for all other configs (NVIDIA/CUDA, discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix WSL) and can never abort the installer. - scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe. Fixes that would have broken a working box: drop the /usr/lib/wsl/lib preflight (a working ROCDXG host has only d3d12/dxcore there); remove the obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname (was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent. - install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then loads the env so detection routes to the gfx1151 wheels. Fast-path when already configured. Fix an inaccurate WSL hint line. - studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1 in-process before torch (gated on /dev/dxg AND librocdxg.so), so the worker uses the GPU even when launched outside a login shell. Mirrors the existing BNB_ROCM_VERSION injection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache - uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env (/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system ROCm userspace is a shared prereq like CUDA and is kept by default; UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux. - uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing the shortcut so its tile disappears promptly (mirrors install.ps1), preserving start2.bin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: accurate AMD ROCm messaging (HIP SDK optional, not required) The Windows installer printed "HIP SDK not found - GPU-accelerated training unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.), setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com) which ship their own ROCm runtime and do NOT need the HIP SDK -- verified end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a Radeon 8060S with no HIP SDK installed. Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK is optional; only when the arch is unknown fall back to the HIP-SDK hint. Behavior (torch routing) is unchanged; this is messaging only. No-op for NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust Two fixes from the 3-reviewer regression audit + live testing on a systemd-enabled WSL distro (interop disabled): F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair could force-delete a pre-existing REAL ROCm install. The guard only checked that /opt/rocm is a real directory, not that it is the stray librocdxg stub. Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo, bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak) instead of deleting it, so a wrong guess can never lose data. WSL interop robustness (install.sh + uninstall.sh): both relied on `command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE it (on systemd distros powershell.exe fails with "Exec format error"). Result: the WSL shortcut silently failed to create (install) and to remove (uninstall). - uninstall.sh: test that powershell.exe actually runs; if not, remove the "Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which works without interop. The name is WSL-install-specific, so a native install's "Unsloth Studio.lnk" is never touched. - install.sh: when the shortcut cannot be created, warn with the manual launch command + how to re-enable interop, instead of failing silently. No behavior change on the interop-on path. The regression audit otherwise found no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone Reinstall regression found by uninstall->reinstall testing: after a Studio uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared ROCm (the default), a non-login reinstall hit the bootstrap fast-path (librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION -- NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed, and the installer fell back to CPU-only PyTorch. Fix: when librocdxg is present but the env drop-in is missing, restore the FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH + LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio worker get it too. No change to the env-present fast-path or any other host. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: clear Explorer icon cache so shortcut icons aren't blank Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct (the shell renders them non-blank via IShellItemImageFactory; the .ico has real image data at 16/32/48/128 px), but the stale cache entry wins. The previous fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it never invalidated Explorer's icon cache, so the desktop icon stayed blank. Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in install.sh): - ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh) - SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh WITHOUT restarting explorer - keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL shortcut path too, preserving start2.bin) Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the previously-resolved (often generic "white page") icon for that item and won't re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct (every icon API renders the sloth) -- only Explorer's cached display is stale. The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED broadcast) does NOT recover a stale item -- confirmed by reproduction. The reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created shortcut, which forces Explorer to re-read that exact item's icon. Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the installer's exact new refresh code, and the sloth icon recovered with NO explorer restart (confirmed by capturing the live desktop via PrintWindow). Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11 Start Menu tile cache (preserving start2.bin). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned The llama.cpp atomic-install staging root (install_llama_prebuilt.py INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install dir (~/.unsloth/.staging in default mode). It is normally pruned after a successful activate, but an interrupted or retained build can leave a <name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and .cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall. Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in env/custom mode (staging nests under the custom root removed already) and when absent. Cross-platform fix (the staging logic is platform-agnostic). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: WSL-absent hint + fix here-string lint false positive install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo user with no WSL yet gets an actionable next step (the hint previously assumed an Ubuntu-24.04 distro already existed). Best-effort, informational only. test_rocm_support.py: test_no_here_strings did a crude substring check that false-positived on the conda-style block marker printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written into the /etc/profile.d drop-in, also used as a sed delimiter pair by uninstall.sh, not a here-string. Strip quoted spans before the check so the lint still catches a real here-string operator but ignores quoted literals. install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make) Apply the valid bot review findings on #5940; reject the ones that don't hold. Fixed: - AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX 370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x / AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150; 8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm prebuilt/wheels. - amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI= 0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out exists to avoid. - amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and _detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass _amd_smi_env()), closing the last unguarded amd-smi spawn on Windows. - WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now requires rocminfo to enumerate the real gfx1151 agent instead of the generic _has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback "gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer skipped. The shared helper is untouched (no gfx90a regression). - install_rocm_wsl_strixhalo.sh: * Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files (x86)/...") word-split on the space and never matched; use find + read loop. * Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it and the librocdxg `make -j` build failed). * Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an unrelated RDNA GPU can't pass while the real GPU is absent. Reviewed but NOT changed: - "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct -- --rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`. - "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1 installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live the native install ends on torch 2.11.0+rocm7.13.0. - "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process already returns "" with LASTEXITCODE=1 (no uncaught exception). - "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed; only a comment about it remains. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps) librocdxg's build needs the Windows SDK 'shared' headers on the Windows host. Previously the helper just die()d with "install the Windows 11 SDK and re-run" if they were missing -- a manual prerequisite that broke the otherwise-seamless `curl ... install.sh | sh` one-liner on Strix Halo. Now, when the headers aren't found, the helper installs the Windows 11 SDK on the Windows host from inside WSL via winget (powershell.exe interop), then re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot). The user already consented to the ROCm-on-WSL setup, so no extra prompt is added beyond the OS UAC gate. - New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir) and _install_windows_sdk_via_winget helpers. - winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then .22621. The presence of the headers (re-check) is the source of truth, not winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin. - Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall through to the existing clear manual-install die(). Opt out with UNSLOTH_SKIP_WIN_SDK_INSTALL=1. Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD Adrenalin driver (AMD referrer-gates the download) remains manual. Verified _find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all winget flags validated against `winget install --help`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing" warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the _amd_smi_allowed() gate that every other amd-smi call site in the file uses. On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress (amd-smi's manifest is asInvoker). The probe also ran before the ROCm-torch-installed check, so it fired on every Windows AMD install. Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching _has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the best-effort "AMD GPU detected" note on HIP-SDK-less hosts. Adds a per-function AST regression test asserting every function in install_python_stack.py that names the amd-smi command and spawns a subprocess also references _amd_smi_allowed() (flags the pre-fix code; passes after). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * studio(cli): fix `unsloth studio stop` crashing on Windows `stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows CPython raises OSError (WinError 87, "The parameter is incorrect") for *every* pid -- alive or dead. `stop` only catches ProcessLookupError / PermissionError, so the OSError propagated and the command crashed with a traceback before ever reaching its (correct) `taskkill /F` path. Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows, signal-0 elsewhere) and use it for both the pre-check and the post-kill wait loop. The actual kill path is unchanged. Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for both a live and a dead pid; `_pid_alive` returns True/False correctly and the full stop() flow (alive -> taskkill -> dead -> "stopped") passes end-to-end against a throwaway process. Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for the win32 tasklist branch and the POSIX signal-0 branch). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151 The bash name->arch inference table in install.sh placed Strix Point identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1 PowerShell tables which correctly map them to gfx1150. It also carried a stray "HX 38" token absent from the PowerShell source-of-truth. Align install.sh with the PowerShell tables: gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|... Impact is low (the bash table only feeds the display label _gpu_disp_gfx and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven by the detected ROCm version, not this name string) but a Strix Point user would otherwise see/copy the wrong gfx arch. Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd) that parses install.sh's case table and asserts Strix Halo->gfx1151, Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match, cross-checking against install.ps1 (the previous parity test only compared install.ps1 <-> setup.ps1, missing install.sh). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.ps1: keep prebuilt-llama ownership guard within the test's block window The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs windows-cpu existing-install kind validation) pushed the install_llama_prebuilt.py invocation to ~1999 chars after the "installing prebuilt llama.cpp bundle (preferred path)" anchor, right at the edge of the 2000-char window that test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the helper string was truncated and the test failed with "substring not found" (CI: Repo tests (CPU)). The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the install_llama_prebuilt.py call) was already satisfied; only the proximity to the anchor regressed. Move the "installing prebuilt..." substep to immediately before the install (after the existing-install pre-cleanup), which also reads better (validate/clean existing -> then "installing"), shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged (console message ordering only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.sh: auto-run Strix Halo ROCm-on-WSL setup by default `curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo (gfx1151) GPU usable inside WSL with no extra commands. Previously the ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty), so the piped one-liner never set the GPU up automatically. Flip it to auto-by-default for the single narrow case the existing guards allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU setup the user ran the installer for. Opt out with UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI (only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS / CPU paths are unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * PR comments: condense to be succinct (comments/docstrings only) Shorten the verbose explanatory comments and docstrings this PR added across the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines, while preserving every non-obvious "why" (os.kill WinError 87, amd-smi RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard chain, ownership guards, etc.). No executable code, string literals, messages, or behavior changed. Verified comments-only: docstring-normalized AST equality (Python, 9 files), non-comment token equality (PowerShell, 3 files), comment-stripped diff + sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url + gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups / cli_studio_stop tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root) Fixes valid findings from the Codex/Gemini PR review: - install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with $amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt. - install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK `winget install` to --source winget (matches the msstore-cert fix rationale). - install.ps1: python.org fallback installs the py launcher per-user (InstallLauncherAllUsers=0, avoids admin), and derives the fallback full version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12) isn't silently replaced with 3.13 when the listing is unreachable. - install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a non-root reinstall (a plain redirect failed silently, dropping the ROCm env). - uninstall.sh: scope WSL Windows-side shortcut removal to the current WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one distro no longer deletes other distros' launchers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Studio ROCm Windows: fix field-reported issues from Strix Halo testers Four fixes from PR #5940 field reports (Win11 native, gfx1151): 1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships hipInfo.exe in the venv Scripts dir, which is only on PATH for activated venvs. Every bnb import logged "Could not detect ROCm GPU architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK installed, whose bin dir is not on PATH either). Prepend the Scripts dir to PATH before bnb imports in main.py, worker.py, and install_python_stack.py, gated on the file existing (only AMD wheels ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151 with zero errors. 2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's total is the WDDM budget the driver grants HIP (BIOS carve + ~half of remaining RAM) -- the OS share is already outside it. The 0.80 unified cap on top denied loads that fit (field report: 48.49 GiB budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08 free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90. 3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with the fix (BIOS UMA frame buffer / AMD Software Variable Graphics Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a 96 GiB box reads as policy, not a Studio bug. 4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade gfx1151): --fit defaults to 'on' upstream, so the fit step runs even when Studio already placed the model via -ngl -1, and aborts in ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off when the server crashes during startup and Studio's own VRAM math had placed the model (never when use_fit or an explicit fit flag was passed). Also keep the TAIL of crash output in the error log (the diagnostic line prints last; head-truncation cut exactly that) and reference the full on-disk log. Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads and generates through the new spawn loop, stub-crash retry appends --fit off and recovers, fraction probes confirm WDDM overcommit and sub-1.0-only enforcement on current AMD wheels. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi amd-smi does not reliably exist on Windows: the HIP SDK never ships a CLI, inbox Windows Update drivers do not, and only some full Adrenalin packages drop amd-smi.exe into System32 (field report: fresh Win11 + Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer work without it: - install_python_stack._detect_windows_gfx_arch: two new probes after hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD torch wheels (drives `studio update` on driver-only hosts), and (4) a last-resort GPU marketing-name -> gfx table via WMI (Win32_VideoController), mirroring setup.ps1's $nameArchTable so a standalone repair resolves the arch with zero AMD tooling installed. - install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir so a standalone rerun finds hipInfo.exe without HIP_PATH. - hardware/amd.py _run_amd_smi: which() guard before spawning -- absence now disables the poller in one step instead of burning the 3-strike circuit breaker on FileNotFoundError; corrected the stale comment claiming Adrenalin ships amd-smi. Simulated against the real detection functions on gfx1151: amd-smi absent, present-but-crashing (exit 1), present-but-hanging (60s sleep vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no exceptions, bounded time. Full adversarial install (broken amd-smi stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first): exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt, torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots healthy and stops cleanly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: per-attempt llama-server log names + amd-smi test portability Found by cross-platform simulation of the --fit off retry (Windows + Linux sandboxes, real load_model with stub servers): - llama-server log filename now carries the spawn-attempt index. The retry can respawn within the same epoch second; reusing the name opened the same file with "w" and truncated the crash log the retry warning had just pointed the user at (proven with a frozen time.time: one file, crash evidence gone; with the suffix both attempts keep their logs). Regression-pinned in test_llama_cpp_wait_for_health.py. - test_amd_primary_gpu_with_mock now mocks shutil.which alongside subprocess.run: the amd-smi absence guard which()-checks before spawning, so on hosts without a real amd-smi (Linux CI, driver-only Windows) the subprocess mock was never reached and the test failed. Surfaced by running the suite in a clean Linux sandbox. Simulation coverage on both OSes: 67-case platform/edge matrix (real shipped code blocks under win32/linux/darwin spoofs: OOM-guard fractions + VGM-hint boundary, bnb PATH-prepend gates, retry eligibility incl. equals-forms and decoy tokens, GPU-name table adversarial set, WMI fallback without powershell, monitor absence semantics), 6-scenario live retry matrix (crash-once/crash-always/ exit-zero/explicit-fit/hang/log-collision) against real llama-server spawns on Windows and WSL (GPU success legs on the 8060S), and a 3-engine browser matrix (chromium/firefox/webkit) driving the live backend's health + authed /v1 chat completion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: classify unified-memory via props.is_integrated first Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded arch set. Strictly additive -- truthy upgrades to unified; 0/absent falls through to the existing gfx1150/gfx1151 + device-name logic, so wheels that omit or zero the field cannot downgrade the known APU set. Extends correct unified-cap treatment to APUs outside that set (e.g. gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers on one driver signal. Verified live on gfx1151 (is_integrated == 1 on the AMD Windows wheel -> ('gfx1151', True) via the new path). * AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table Fleet validation on a Strix Halo WSL2 box showed the system rocminfo (HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin off PATH outside login shells. Detection probes that miss either of these report no GPU on a working ROCDXG host and select the CPU build even though the lemonade bundle offloads fine (95.7 tok/s measured vs 64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and the PATH fallback in install.sh, studio/setup.sh, and the installer's Linux rocm probe, mirroring what main.py/worker.py already do for the runtime. Also sync studio/setup.sh's name->gfx table with install.sh: 890M and the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S must match gfx1102 before the gfx1100 row, and the RDNA2/workstation rows were missing. New parity test pins the two bash tables together so they cannot drift again. * Studio: persist server session logs + native-crash stacks to disk Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows): "the studio just terminates without a warning". A native crash in the GPU runtime kills the process with no Python traceback, and a desktop- shortcut console closes before anything can be read. The server only ever logged to the console, so there was nothing to send back. run_server now tees stdout/stderr to ~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior unchanged; file copy is best-effort), arms faulthandler at the same file so access violations / SIGSEGV leave a stack trace on disk, and exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps on their captured stderr. Armed before `from main import app` so even import-time failures leave evidence. Keeps the newest 20 session logs; opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>" at startup so users know what to attach. Verified on this box: a forced real segfault (faulthandler._sigsegv) leaves the full session output plus "Fatal Python error: Segmentation fault" and the thread stack in the file while the console shows nothing; a normal server boot captures the startup banner and serves health as before. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value Match the shell helpers, which use the parameter-default form: a user who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU from DXG detection should not have the probe override it. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com>	2026-06-10 04:24:49 -07:00
Daniel Han	187144d4e7	Reduce and tighten code comments and docstrings repo-wide (#6095) ... Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.	2026-06-08 23:09:51 -07:00
Daniel Han	8292e699e4	Studio: make code comments and docstrings more succinct (#6029) ... Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.	2026-06-08 23:07:28 -07:00
Daniel Han	3ce187da02	Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) ... Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.	2026-06-08 04:24:13 -07:00
Daniel Han	c6e86d5e77	Update Install Scripts (#5968) ... * Update Install Scripts Add SPDX AGPL-3.0 headers to the installer scripts and let the piped web installs take their common options from the environment. - install.sh / install.ps1: read UNSLOTH_NO_TORCH (and UNSLOTH_PYTHON for install.sh) so a piped install needs no positional flags. Flags and the pipe forms still work; an explicit flag wins. - Fix the UNSLOTH_STUDIO_HOME example so the variable sits after the pipe and reaches sh instead of curl. - Add SPDX headers to install.sh, install.ps1, the uninstall scripts, and the MLX install scripts. - Drop the internal test package names from the studio install comments. * Mirror UNSLOTH_PYTHON env var to install.ps1 install.ps1 now reads UNSLOTH_PYTHON to pin the Python version, matching install.sh, and lists all three env vars (UNSLOTH_NO_TORCH, UNSLOTH_PYTHON, UNSLOTH_STUDIO_HOME) in the header examples. The requested version is preferred during detection and used as the winget install target; behavior is unchanged when the variable is unset.	2026-06-03 05:39:42 -07:00
Daniel Han	8ec9a74fd3	studio: ROCm cleanups follow-up to #5301 (#5874) ... Follow-up cleanups to the merged AMD ROCm support PR #5301: 1. De-duplicate the torchao Windows-ROCm import stub into a single shared module (studio/backend/core/_torchao_stub.py); both workers call one install_torchao_windows_rocm_stub() entrypoint. 2. Align the gfx name/arch comment columns in setup.sh and setup.ps1. 3. Isolate the float16 dtype fallback to AMD without native bf16; NVIDIA keeps dtype=None so unsloth's own bf16/fp16/FORCE_FLOAT32 detection is honored. 4. Hoist unconditional stdlib imports (gc, glob, re, subprocess, copy, types, sys, importlib.metadata) from function bodies to module top across the PR #5301-touched files; heavy/optional/relative imports stay lazy. 5. bitsandbytes Windows-ROCm install now uses plain pip (force_pip=True) instead of UV_SKIP_WHEEL_FILENAME_CHECK, per the AMD hackathon docs. Also adds scripts/verify_import_hoist.py (a scope-aware LEGB AST resolver that catches dangling-alias and rename-clash bugs in import-hoist refactors) and wires it into the Lint CI source-lint job as a self-test plus a pull_request compare gate.	2026-05-30 03:06:47 -07:00
Leo Borcherding	b6d5636cc0	fix/strix halo and windows AMD ROCm support (#5301) ... * fix(studio): set HIP_VISIBLE_DEVICES in apply_gpu_ids for ROCm training workers Training workers are spawned via multiprocessing spawn before detect_hardware() runs, so IS_ROCM is still False. If the user never set HIP_VISIBLE_DEVICES in their shell, _inherits_rocm_visibility is also False, leaving the worker with only CUDA_VISIBLE_DEVICES set. On ROCm hosts the HIP runtime honors HIP_VISIBLE_DEVICES over CUDA_VISIBLE_DEVICES, so the worker saw the full device list and torch raised "no usable HIP accelerator" on some setups. Fall back to probing torch.version.hip (a build-time attribute, safe to read before GPU init) to detect ROCm when neither IS_ROCM nor inherited env vars are available. Mirrors the existing fix in llama_cpp.py for llama-server subprocess GPU pinning. Fixes https://github.com/unslothai/unsloth/issues/5180 * test: tighten apply_gpu_ids ROCm fallback assertions Replace loose OR chain with exact string matches, split into three focused tests, and add a guard check for the try/except wrapper. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: detect ROCm unified memory (Strix Halo / AMD iGPU) via torch fallback amd-smi on iGPUs with shared/unified memory (e.g. Radeon 8060S on Strix Halo) reports only the dedicated VRAM slice (~512 MB) in its metric output, so get_visible_gpu_utilization() was returning usable_gb ≈ 0.35 GB instead of the full GTT pool (~128 GB). torch.cuda.mem_get_info() already surfaces the correct unified-pool size. Add _reconcile_rocm_unified_memory(): after amd-smi returns a valid result on a ROCm device, cross-check each device's vram_total_gb against torch.cuda.mem_get_info(). When torch reports a larger total, replace the amd-smi VRAM fields in-place. No-op for discrete AMD GPUs where the two sources agree. Fixes: "Falling back to all visible GPUs -- model may not fit" on AMD iGPU machines even when 100+ GB of unified memory is available. * Apply unified-memory reconciliation in get_gpu_utilization too The visible-GPU path was already corrected for AMD iGPUs with unified memory (Strix Halo / Radeon 8060S), but get_gpu_utilization was still returning the raw 512 MB amd-smi VRAM slice. Studio's /api/train/hardware endpoint and the live GPU monitor read from this primary path, so users continued seeing the wrong total even after auto_select_gpu_ids picked the right device. Refactor to share the per-device correction: * _apply_unified_memory_correction(metrics, torch_info) -- the actual replacement logic, in-place on a single metrics dict. * _reconcile_rocm_unified_memory(...) -- multi-device, iterates utilization["devices"] (visible-GPU path). * _reconcile_primary_rocm_unified_memory(...) -- single flat metrics dict (primary-GPU path), uses parent_visible_spec to pick the primary index, falls back to ordinal 0 when no visibility env is set. get_gpu_utilization now calls the primary reconciler under IS_ROCM, so both endpoints surface the real unified-memory pool on iGPUs while leaving discrete AMD GPUs untouched (torch_total <= smi_total -> no replace). * Use 'is not None' and log debug on torch.version.hip probe failures Two small follow-ups to the apply_gpu_ids ROCm fallback: 1. Match detect_hardware()'s 'getattr(torch.version, "hip", None) is not None' form so the entire codebase has one canonical 'this torch was built with HIP' check. On every shipping torch wheel hip is either None or a non-empty version string, so the new form agrees with the old bool() form on every real install. 2. Log the probe failure at debug level instead of swallowing it silently. The broad 'except Exception' is intentional (we never want apply_gpu_ids to crash a worker over a probe), but the silent pass made it impossible to tell whether the fallback was firing or being skipped. * fix(studio): honour HIP_VISIBLE_DEVICES in _get_parent_visible_gpu_spec before IS_ROCM is set When a user has HIP_VISIBLE_DEVICES set in their shell (e.g. "1" to select GPU 1) but detect_hardware() has not yet run in the Studio parent process, IS_ROCM is still False. _get_parent_visible_gpu_spec() was gated on IS_ROCM so it fell through to CUDA_VISIBLE_DEVICES (unset), saw all physical GPUs, and auto-selected index 0. apply_gpu_ids then overwrote HIP_VISIBLE_DEVICES with "0", making the intended GPU invisible to ROCm torch in the worker, which triggered the "no usable HIP accelerator" error (issue #5180). Apply the same _inherits_rocm_visibility pattern already used in apply_gpu_ids: check for HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES in the environment regardless of IS_ROCM so the correct GPU index is preserved. * fix(install): harden AMD ROCm GPU detection for multi-GPU and env-filtered setups The previous rocminfo awk pattern could miss discrete GPUs on machines where HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES is used to mask an integrated GPU — the env vars filter rocminfo output but may not propagate into the install script subprocess, causing detection to fail entirely. Two changes: - Tighten rocminfo pattern from /gfx[0-9]/ && !/gfx000/ to /gfx[1-9][0-9]/ — simpler and correctly excludes the CPU agent (gfx000) without a negative lookahead - Add sysfs KFD topology fallback: reads /sys/class/kfd/kfd/topology/nodes/*/gpu_id which is a kernel-level view unaffected by HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES Fixes detection failure reported in Discord by Chains (gfx1201 + iGPU machine where env var exclusion of the iGPU caused rocminfo to return no usable device). * Fix KFD sysfs awk fallback to read properties file The fallback added by this PR reads /sys/class/kfd/kfd/topology/nodes/*/gpu_id files but matches the literal token 'gpu_id' against their content. Those files contain only a single decimal value (e.g. '0' for CPU agents, '50432' for GPU agents), so the regex never matches and 'found' stays 0, making the fallback a no-op on every host. The properties file in the same directory contains key/value lines like 'gpu_id 50432' which is what the existing awk pattern expects. Reproduced with a synthetic sysfs layout: against gpu_id files awk exits 1; against properties files awk exits 0 when any node reports gpu_id > 0. * fix(setup.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.sh setup.ps1 only checked nvidia-smi and fell straight to "gpu: none" on AMD machines. setup.sh already probed rocminfo/amd-smi/hipconfig/hipinfo. Add three-tier detection mirroring install_llama_prebuilt.py's detect_host(): 1. hipinfo: gcnArchName in output confirms a real HIP GPU (not just SDK) 2. amd-smi list: "GPU: <digit>" data rows as fallback 3. WMI Win32_VideoController: last resort -- detects AMD GPU even without HIP SDK, then guides user to install it rather than silently going CPU Also corrects the "none" message to mention AMD ROCm alongside NVIDIA so users with AMD hardware understand the requirement. Fixes: rohit-style install where Strix Halo (Radeon 8060S) showed "gpu: none" even with the HIP SDK present. * fix(install.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.ps1 install.ps1 had the same nvidia-smi-only GPU detection as setup.ps1 before the setup.ps1 fix. Applies the same three-tier AMD detection: 1. hipinfo: gcnArchName confirms real HIP GPU 2. amd-smi list: GPU data rows as fallback 3. WMI Win32_VideoController: detects AMD GPU without HIP SDK and guides user to install it Fixes: install.ps1 showing "gpu: none" while setup.ps1 correctly showed "AMD GPU detected" on the same machine (reported by rohit, RX 7600 XT). * fix(install.ps1): suppress 'No NVIDIA GPU detected' when AMD GPU is present * feat: add Windows AMD ROCm PyTorch wheel installation install_python_stack.py: - Add _ROCM_WINDOWS_WHEEL_BASE and _ROCM_WINDOWS_RELEASES constants pointing to AMD repo.radeon.com (ROCm 7.2 -> torch 2.9.1+rocm7.2.1) - Extend _ensure_rocm_torch() with a Windows branch: detects ROCm via _has_rocm_gpu() / _detect_rocm_version(), requires Python 3.12 (cp312 is the only ABI AMD publishes for Windows), installs the direct wheel URL from repo.radeon.com install.ps1: - Capture ROCmVersion during AMD detection via hipconfig --version / amd-smi version (needed for wheel URL selection) - After Get-TorchIndexUrl, add an AMD wheel override block: when HasROCm and Python 3.12 detected, set ROCmTorchWheelUrl to AMD wheel URL - Expand torch install branch to handle ROCmTorchWheelUrl with uv pip install --force-reinstall --no-cache-dir * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: also install torchvision and torchaudio from AMD Windows repo AMD publishes matching torchvision-0.24.1+rocm7.2.1 and torchaudio-2.9.1+rocm7.2.1 cp312 wheels at the same repo.radeon.com release folder. Install all three in both install.ps1 and install_python_stack.py Windows ROCm path. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: add ROCm 7.1.1 Windows wheel mapping AMD uses a different version string for 7.1.1 wheels: 2.9.0+rocmsdk20251116 (date-tagged) instead of +rocm7.1.1. Adds the 7.1.1 release folder to both install.ps1 and install_python_stack.py so users with ROCm 7.1 get ROCm torch instead of falling back to CPU. * fix: install rocm_sdk_core and rocm_sdk_libraries_custom alongside torch The AMD Windows torch wheels declare rocm[libraries]==<ver> as a hard dependency. Without installing rocm_sdk_core and rocm_sdk_libraries_custom from the same AMD release folder, uv cannot resolve the dependency and fails with 'No solution found'. Include all 5 wheels in one install call. * fix: expand ROCm wheel array to scalars for Invoke-InstallCommand @array splatting inside a scriptblock only works when the native command is prefixed with '&'. Invoke-InstallCommand uses '& $Command' to run the block, so @ROCmAllWheelUrls was not being expanded. Extract to scalar variables $rw0-$rw4 which are captured correctly by the closure. * fix: use --no-deps for AMD Windows torch wheel install uv's resolver looks up rocm[libraries]==0.1.dev0 on PyPI during dependency resolution before downloading any wheels, and fails because the package doesn't exist on PyPI. --no-deps skips resolution entirely and installs all 5 AMD wheels directly. The GPU runtime dependency is satisfied by the HIP SDK, not a Python package. * fix: setup.ps1 and install_python_stack.py now install ROCm torch on Windows setup.ps1 was always setting CuTag='cpu' for non-NVIDIA hosts and installing cpu-only PyTorch, overwriting the ROCm torch installed by install.ps1. Adds the same AMD wheel selection logic (ROCm version detection, Python 3.12 check, 5-wheel install with --no-deps) to setup.ps1's torch install block. install_python_stack.py: remove IS_WINDOWS guard from _ensure_rocm_torch() call site so the Windows path in _ensure_rocm_torch() is reachable during 'unsloth studio update' as well. * fix: suppress manual-install warning when ROCm torch already present; fix progress counter - Gate the 'must be installed manually' warning on torch.version.hip being empty so it doesn't fire when our ROCm torch install succeeded - Update _TOTAL counter to include the 3 ROCm steps on Windows now that _ensure_rocm_torch() is called there (fixes 10/9 display) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: add rocm step display in setup.ps1; fix warning and progress counter - Add 'rocm' step after 'cuda' in setup.ps1 showing ROCm version or HIP SDK missing - Move ROCm version detection up to GPU detection block so it's available early - Suppress 'must be installed manually' warning when torch.version.hip is set - Fix _TOTAL counter to include ROCm steps on Windows (fixes 10/9 display) * fix: detect AMD SDK ROCm torch via __version__ when torch.version.hip is unset AMD's repo.radeon.com wheels (e.g. 2.9.0+rocmsdk20251116) do not set torch.version.hip, leaving it None. All three probes that relied solely on torch.version.hip now also check for 'rocm' in torch.__version__.lower(): - hardware.py detect_hardware(): IS_ROCM was never set, causing the studio to report 'Hardware detected: CPU' even after AMD wheels were installed and HIP DLLs were on PATH. - install_python_stack.py _ensure_rocm_torch(): skip-if-already-installed probe would always reinstall on subsequent runs. - install_python_stack.py Windows AMD warning: suppression check always failed, so the 'must be installed manually' note kept appearing after a successful AMD wheel install. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * perf: drop --no-cache-dir from AMD ROCm torch wheel installs uv caches downloaded wheels by default; passing --no-cache-dir forced a full redownload of the ~2 GB torch wheel on every install run. CUDA installs never had this flag -- AMD was the only path affected. * fix: use install-state flag instead of subprocess probe for AMD Windows warning Replace the subprocess torch probe in the post-install warning block with a module-level _rocm_windows_torch_installed flag set by _ensure_rocm_torch(). Subprocess re-import of torch is unnecessary and fragile -- the install function already knows whether it succeeded. * fix: hoist global declaration to top of _ensure_rocm_torch Python requires the global statement to appear before any assignment to the variable within a function. Moving it to the function top fixes the SyntaxError on line 354. * fix: pass AMD torch install status via env var to suppress false warning setup.ps1 now sets UNSLOTH_ROCM_TORCH_INSTALLED=1 after a successful AMD wheel install. install_python_stack.py reads this at the top of _ensure_rocm_torch() to skip both the subprocess probe and the warning -- no re-import of torch needed, and the warning message now correctly says 'could not be auto-installed' rather than 'must be installed manually'. * fix: register ROCm DLL directory before torch import on Windows Python 3.8+ ignores PATH for extension DLL loading on Windows; amdhip64.dll and other HIP runtime DLLs must be registered via os.add_dll_directory(). Without this, torch.cuda.is_available() always returns False on AMD ROCm Windows even when HIP_PATH is correctly set in system environment variables. Reads HIP_PATH / ROCM_PATH env vars first, then falls back to scanning common ROCm install roots (C:\Program Files\AMD\ROCm, F:\ROCm, C:\ROCm). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: remove hardcoded non-standard ROCm paths from DLL directory scan Only use HIP_PATH/ROCM_PATH (set by AMD installer) and the standard C:\Program Files\AMD\ROCm\<version>\bin location. Custom drive paths like F:\ROCm are user-specific and should not be hardcoded. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: prevent torchao overrides step from overwriting AMD ROCm torch torchao==0.14.0 in overrides.txt declares torch as a dependency. Without --no-deps, uv resolves torch from PyPI and installs 2.11.0+cpu on top of the AMD ROCm wheels (2.9.0+rocmsdk20251116). This was the root cause of 'Hardware detected: CPU' -- the AMD wheels were installed but then immediately overwritten by the overrides step. When _rocm_windows_torch_installed is True, add --no-deps to the overrides pip_install call so torchao is installed without pulling in CPU torch. * fix: add rocm_sdk namespace tarball to Windows ROCm wheel installs torch/_rocm_init.py calls `import rocm_sdk` at startup, which requires the rocm namespace tarball (rocm-*.tar.gz) in addition to the SDK wheel packages. This tarball was missing from both install.ps1 and setup.ps1, causing ModuleNotFoundError on first torch import. - Add rocm-0.1.dev0.tar.gz to ROCm 7.1.1 install (provides rocm_sdk namespace) - Add rocm-7.2.1.tar.gz + rocm_sdk_devel to ROCm 7.2.1 install - Install tarball in a dedicated step before main SDK/torch wheels - Switch to @array splatting in install.ps1 scriptblock for dynamic wheel count - Remove --no-cache-dir from Python-side ROCm wheel install (prevents ~2GB redownload) * feat: enable ROCm 7.2 torch install + warn on gfx1151 with ROCm < 7.2 Chigoma333 (AMD Radeon 8060S / gfx1151, Strix Halo) confirmed that ROCm 7.1 segfaults when tensors are moved to GPU, but ROCm 7.2 + torch 2.11.0+rocm7.2 works fully including training. Changes: - Uncomment (7,2): "rocm7.2" in _ROCM_TORCH_INDEX (was blocked by <2.11.0) - Add _ROCM_TORCH_PKG_SPECS dict with per-tag version bounds: rocm7.2 → torch>=2.11.0,<2.12.0; all older tags → <2.11.0 - Add _detect_amd_gfx_codes() helper that parses rocminfo output - Warn on gfx1151/gfx1150 (Strix Halo) when ROCm < 7.2 is installed, pointing users at the known segfault and recommending upgrade - install.sh get_torch_index_url(): enable rocm7.2 case (previously capped to rocm7.1), cap unknown future tags to rocm7.2 - install.sh: override TORCH_CONSTRAINT to >=2.11.0,<2.12.0 when rocm7.2 index is selected, so pip can actually resolve torch 2.11.0 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: prefer Python 3.12 for AMD ROCm users when 3.13 is also installed After GPU detection, if ROCm HIP SDK is found and the selected Python is not 3.12, run a second pass to locate a 3.12 install via py.exe and PATH (catches uv-managed installs). Switch $DetectedPython to 3.12 so the venv is created with a compatible interpreter for the cp312-only AMD Windows torch wheels. NVIDIA and Intel GPU paths are unaffected -- the re-detection block only runs when $HasROCm is true. Fixes: #5301 * fix: also check uv-managed Python 3.12 for AMD ROCm #5301 * fix: hide amd-smi console popups on Windows, guard torch.distributed.is_initialized for ROCm #5301 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: suppress remaining console popups on Windows, patch torch.distributed.is_initialized for ROCm #5301 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: stub all missing torch.distributed attrs for ROCm Windows wheel #5301 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: inject torch.distributed stub when C backend missing in ROCm Windows wheel #5301 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(rocm/windows): pre-stub torch._C._distributed_c10d + raise amd-smi timeout Two fixes for Windows ROCm regressions reported by electroglyph on #5301: 1. worker.py — torch.distributed stub now fires unconditionally on Windows The previous stub only injected sys.modules in the except branch, meaning it was silently skipped when `import torch.distributed` happened to succeed (the C backend is lazily resolved). The crash then hit later when transformers/trl triggered the lazy load. Fix: on win32 we pre-populate sys.modules['torch._C._distributed_c10d'] AND set the attribute on the torch._C extension module *before* attempting the import, covering both the early-ImportError and lazy-load failure modes. 2. amd.py — increase amd-smi timeout from 5 s to 30 s on Windows (10 s Linux) amd-smi on Windows must cold-init the ROCm runtime on first invocation; 5 s was consistently too short, producing repeated 'Command timed out' warnings in the server log. 30 s gives enough headroom without blocking indefinitely on broken installs. 3. install.ps1 — widen Python 3.12 enforcement to ROCmGpuLabel (WMI-only path) Users whose HIP SDK is not on PATH were detected via WMI but not switched to Python 3.12 before the install started, causing a second pass. Guard now fires on (HasROCm -or ROCmGpuLabel). * fix(rocm): guard c10d stub, fix TorchIndexFamily for 7.1, clean dead code + comments - worker.py: wrap c10d stub injection in `if _c10d_key not in sys.modules` so Windows NVIDIA users with a real torch.distributed are never affected - install.ps1: fix Get-TauriTorchIndexFamily receiving hardcoded "rocm7.2" even when ROCm 7.1 wheels are installed; now branches on $ROCmVersion - main.py: remove dead `import ctypes as _ctypes` (ctypes is never called) - hardware.py, install_python_stack.py, worker.py, install.ps1: shorten verbose multi-line comment blocks throughout - tests: update 4 stale assertions that expected rocm7.2 to be absent/capped * fix(tests): match windows AMD warning assertion to actual source string * chore: trim verbose comment blocks across all ROCm-related files * fix: guard reconcile call against None numeric_ids; add torchvision lower bounds * fix(install.ps1): recreate venv with Python 3.12 after ROCm switch Venv was created with 3.13 before GPU detection ran; switching $DetectedPython to 3.12 had no effect since $VenvPython still pointed to the 3.13 interpreter inside the already-created venv. * ux: detect AMD GPU before Python selection to avoid double venv creation - Early hipinfo + WMI probe runs before Find-CompatiblePython so Python 3.12 is selected upfront when AMD is detected; venv is now created exactly once instead of 3.13 then immediately 3.12. - Post-venv recreation block replaced with a simple warning for the rare case where AMD was missed by the early probe. - setup.ps1: show venv's actual Python version (e.g. 3.12) instead of the system Python found by the pre-activation search (was showing 3.13). * fix(rocm/win): auto-stub all _distributed_c10d symbols via PEP-562 __getattr__ The bare ModuleType stub caused ImportError when torch._dynamo was imported (triggered by trainer.py accessing torch._dynamo.config at load time). torch._dynamo pulls in torch.distributed.fsdp._flat_param which does: from torch._C._distributed_c10d import FakeProcessGroup and potentially other symbols. Adding module __getattr__ auto-creates a stub class for any missing symbol so all such imports succeed without enumerating every individual symbol. Applied to both the primary stub and the fallback stub in the except branch. * chore: trim c10d stub comment * fix(rocm/win): auto-stub missing torch.distributed attrs (Store, ProcessGroup, …) * fix(rocm/win): pre-stub fsdp submodules in sys.modules; fix __getattr__ subpackage clash * feat(rocm/win): arch-aware wheel selector always picks newest ROCm release Replace HIP-SDK-version-gated wheel selection with GPU arch-based logic. Select-ROCmWheelRelease (PS) and _select_windows_rocm_release (Python) map gcnArchName → minimum ROCm version, then pick the newest available release that satisfies it (currently always rocm-rel-7.2.1 for any supported GPU). Wheels bundle their own ROCm runtime so the installed HIP SDK 7.1 does not prevent using 7.2.1 wheels on gfx1200 (RX 9060 XT) and similar RDNA 4 GPUs. Also installs the bitsandbytes Windows ROCm continuous-release wheel and sets BNB_ROCM_VERSION=72 in worker.py before ML imports so bnb loads the libbitsandbytes_rocm72.dll that ships in that wheel. * fix(rocm/win): stub class metaclass for ProcessGroup.BackendType; amd-smi circuit breaker torchao.float8.inference accesses ProcessGroup.BackendType as a class-level attribute. Plain type() stubs have no __getattr__ on the metaclass so this raises AttributeError. Introduce _StubClassMeta whose __getattr__ returns child stub classes, fixing the torchao import chain. Add an amd-smi circuit breaker in amd.py: after 3 consecutive failures the module stops spawning the process, eliminating the repeated Windows UAC / DiskPart elevation prompts caused by polling a non-functional amd-smi. Also guard BNB_ROCM_VERSION=72 behind a DLL existence check so bitsandbytes fails with its own detection message rather than a harder "DLL not found" when the Windows ROCm bnb wheel is not yet installed. * fix: stub __members__ so torchao float8 enum check doesn't crash on ROCm Windows torchao.float8.inference accesses ProcessGroup.BackendType.__members__ expecting a Python Enum registry dict. _StubClassMeta.__getattr__ was blocking all dunder attributes, causing AttributeError. Return {} for __members__ specifically so the isinstance/iteration checks pass cleanly. * fix: stub distributed tensor/functional_collectives to prevent missing C++ op crash on ROCm Windows torch._dynamo.trace_rules eagerly loads torch.distributed.tensor at import time, which pulls in _functional_collectives.py. That file registers Meta kernels for _c10d_functional C++ ops, but those ops are only registered by torch._C._distributed_c10d — a C extension absent from ROCm Windows wheels. Pre-stubbing the affected modules in sys.modules prevents the real import chain from running and avoids the "operator does not exist" crash. * fix: give mod stubs __path__ and pre-stub _tensor to fix 'not a package' import error _make_mod_stub now sets __path__=[] so Python treats stub modules as packages. Without it, any import of a submodule raises "is not a package". Also pre-stub torch.distributed._tensor and its submodules so that _tensor/__init__.py (which re-exports from torch.distributed.tensor) never runs and torchao's `from torch.distributed._tensor import DTensor` gets a harmless stub instead of crashing. * fix: stub torch.ops._c10d_functional namespace with hashable op sentinels torchao.dtypes.nf4tensor uses _c10d_functional ops as dict keys at import time (all_gather_into_tensor.default, wait_tensor.default) and torch.ops.c10d.scatter_.default. None of these ops are registered on ROCm Windows because torch._C._distributed_c10d (the C extension) doesn't ship. Replace the whole _c10d_functional namespace with a custom stub whose ops return hashable .default objects, so dict-key construction doesn't crash. Also inject a scatter_ stub into torch.ops.c10d if it's missing. * fix: stub entire torchao package on ROCm Windows instead of individual ops torchao is not supported on ROCm Windows and its import chain transitively requires torch._C._distributed_c10d (absent from the ROCm Windows wheel). Rather than stub each missing op one by one, stub the whole torchao package upfront. Unsloth uses bitsandbytes for quantization, not torchao, so this has no functional impact. transformers gracefully handles an importable-but- empty torchao by disabling TorchAoHfQuantizer. * fix: set __spec__ on mod stubs so importlib.util.find_spec doesn't raise Manually-injected sys.modules entries have __spec__=None by default. importlib.util.find_spec() raises ValueError when it finds a module in sys.modules with __spec__=None (transformers.utils.import_utils hits this when checking if torchao is available). Give every stub a minimal ModuleSpec(name, loader=None, is_package=True) to satisfy find_spec. * fix: add meta path finder to auto-stub subpackages of stub modules `import torchao.prototype` goes through the import machinery, not __getattr__, so an empty __path__ means ModuleNotFoundError. Rather than list every submodule explicitly, register a MetaPathFinder that intercepts any import whose parent is one of our stubs (detected by loader=None in the parent's ModuleSpec). Real installed packages always have a SourceFileLoader so they are never intercepted. Also register child stubs in sys.modules from __getattr__ as a belt-and-suspenders measure. * fix: use _unsloth_stub sentinel instead of loader=None for stub detection The import machinery overwrites module.__spec__ with the spec returned by find_spec (which has loader=_StubSubpackageLoader, not None), so the loader=None check broke for second-level subpackages. Switch to a custom _unsloth_stub object identity sentinel set directly on each stub module -- it survives __spec__ being replaced and correctly identifies stubs at any depth (torchao.prototype.safetensors, etc.). * refactor(rocm/win): switch to repo.amd.com arch-aware index, remove stubs AMD recommends repo.amd.com/rocm/whl/{arch}/ as the Windows ROCm wheel source. These wheels bundle their own ROCm runtime, support all Python versions (not just cp312), and include the full torch._C extension set (including _distributed_c10d) that the old repo.radeon.com wheel omitted. Changes: - install.ps1: remove Select-ROCmWheelRelease + hardcoded cp312 wheel URLs; remove Python 3.12 forced-preference logic; install via --index-url repo.amd.com/rocm/whl/{arch-family}/ - studio/setup.ps1: same -- remove Select-ROCmWheelRelease, switch to repo.amd.com arch-aware index URL - studio/install_python_stack.py: replace _ROCM_WINDOWS_RELEASES / _select_windows_rocm_release with _windows_rocm_index_url() using the _GFX_TO_AMD_INDEX_ARCH map; drop Python 3.12 restriction - studio/backend/core/training/worker.py: remove all stub machinery (_make_mod_stub, _StubSubpackageFinder, _StubSubpackageLoader, _StubClassMeta, torchao/fsdp/dtensor stubs, _c10d_functional ops stubs, BNB DLL detection) -- no longer needed with new wheel source * fix(rocm/win): restore _distributed_c10d + torchao stubs; fix BNB install repo.amd.com torch wheels also omit torch._C._distributed_c10d on Windows (RCCL is not shipped on Windows). torch/distributed/__init__.py imports from it unconditionally at module level, so the stub must land in sys.modules before any torch.distributed import. torchao (pulled in by transformers.quantizers) walks torchao.float8.distributed_utils -> torch.distributed._functional_collectives -> distributed_c10d at import time. Stubbing torchao up-front short-circuits that chain. worker.py: - Restore _make_mod_stub / _StubSubpackageFinder / _StubSubpackageLoader - Restore _StubClassMeta for ProcessGroup.BackendType attribute access - Restore _distributed_c10d stub with __getattr__ (Windows only) - Restore torchao stubs (5 modules, Windows only) install_python_stack.py: - BNB AMD wheel install was inside the early-return branch that fires when torch is already a ROCm build (installed by install.ps1). Move BNB install outside that branch so it always runs on Windows ROCm — the PyPI bitsandbytes has only CUDA DLLs and fails to load on ROCm. * worker: remove _distributed_c10d stub; stub only torchao The installed torch/distributed/__init__.py from repo.amd.com (torch==2.10.0+rocm7.12.0) is now properly guarded with `if is_available():`, so `import torch.distributed` alone is safe. The crash only comes via torchao's import chain: torchao.float8.distributed_utils → torch.distributed._functional_collectives (unguarded import) → torch.distributed.distributed_c10d → torch._C._distributed_c10d ← absent on Windows ROCm Stubbing torchao short-circuits the chain entirely. No need to stub _distributed_c10d. Remove _StubClassMeta and the _c10d stub block; keep only _make_mod_stub + _StubSubpackageFinder + torchao seeds. * fix: BNB AMD wheel skipped + torch.compile segfault on Windows ROCm install_python_stack.py: the UNSLOTH_ROCM_TORCH_INSTALLED=1 early-return path (set by setup.ps1 when it installed torch itself) returned before ever reaching the AMD BNB prerelease wheel install. The PyPI bitsandbytes==0.49.x ships only CUDA DLLs, so loading it on ROCm fails with "libbitsandbytes_rocm72.dll not found". Now installs the AMD Windows BNB wheel before returning on that path too. worker.py: torch._grouped_mm crashes on gfx1200 (null HIP kernel pointer, 0xC0000005) when torch.compile's JitDecomp system dispatches it during the first forward pass. Detect Windows ROCm via torch.version.hip (already in sys.modules from section 1e) and set TORCHDYNAMO_DISABLE=1 to bypass the broken kernel dispatch. * fix: BNB AMD wheel install fails uv wheel filename check The bitsandbytes continuous-release wheel is intentionally mismatched: filename encodes 1.33.7.preview (= 1.33.7rc0 in PEP 440) but wheel metadata reports 0.50.0.dev0. uv rejects this by default. Introduce _install_bnb_windows_rocm() helper that sets UV_SKIP_WHEEL_FILENAME_CHECK=1 only for this specific install, then restores the previous env value. Both BNB install call sites (the UNSLOTH_ROCM_TORCH_INSTALLED early-return path and the normal Windows ROCm path) now use this helper. * worker: patch _grouped_mm CUDA dispatch on Windows ROCm (gfx1200 null kernel) TORCHDYNAMO_DISABLE=1 stopped the compiler frontend but not the autograd JitDecomp system, which also dispatches _grouped_mm and hits the same null HIP kernel crash (0xC0000005). Verified that torch.library.Library("aten","IMPL").impl("_grouped_mm", fn, "CUDA") successfully overrides the broken HIP kernel with a Python mm fallback on torch==2.10.0+rocm7.12.0. Schema: _grouped_mm(Tensor self, Tensor mat2, Tensor? offs=None, Tensor? bias=None, ScalarType? out_dtype=None) -> Tensor The fallback handles both the simple case (offs=None → torch.mm) and the grouped case (offs provided → split self by offsets, multiply each group against the corresponding slice of mat2, then cat results). Keep _WINDOWS_ROCM_GROUPED_MM_LIB alive at function scope to prevent the C++ dispatch registration from being freed by GC. * worker: fix torchao stub — return stub classes not modules for isinstance() peft/tuners/lora/torchao.py does: from torchao.dtypes import AffineQuantizedTensor, LinearActivationQuantizedTensor isinstance(weight, (AffineQuantizedTensor, LinearActivationQuantizedTensor)) The stub __getattr__ was returning stub modules, which isinstance() rejects with "arg 2 must be a type, a tuple of types, or a union". Add _StubTypeMeta metaclass whose __instancecheck__ always returns False, and _make_stub_type() to create stub classes via it. Change _make_mod_stub __getattr__ to return stub classes instead of stub modules for leaf attribute access, so isinstance() gets a valid type and returns False. _StubSubpackageFinder still handles import-style subpackage creation (those still need module objects in sys.modules); __getattr__ only fires for from-import or direct attribute access, which are the isinstance paths. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests: add coverage for Windows ROCm install paths and worker patches Add conftest.py to fix pre-existing sys.path issue that prevented test_rocm_support.py from running at all (install_python_stack.py imports from backend.utils.wheel_utils which needs studio/ on sys.path). New test classes cover everything added in this session: - TestWindowsRocmIndexUrl: arch → AMD pip index URL mapping (gfx120X-all, gfx1151, gfx1150, gfx110X-all, unknown → None, trailing slash) - TestDetectWindowsGfxArch: hipinfo output parsing, missing/timeout/bad returncode/no-gcnArchName paths - TestInstallBnbWindowsRocm: UV_SKIP_WHEEL_FILENAME_CHECK set+restored, env restored on exception, no-op when URL missing - TestRocmTorchInstalledEnvVar: UNSLOTH_ROCM_TORCH_INSTALLED=1 skips pip_install, calls _install_bnb_windows_rocm, sets flag - TestWorkerWindowsRocmPatches: _grouped_mm CUDA dispatch override, offs/grouped variant handling, GC-prevention sentinel, _StubTypeMeta __instancecheck__, _StubSubpackageFinder registration, torchao key submodule pre-stubbing, TORCHDYNAMO_DISABLE guard - TestRocmTorchPkgSpecs: rocm7.2 torch 2.11.x spec, default <2.11 cap, 3-tuple shape, _GFX_TO_AMD_INDEX_ARCH RDNA4/3.5/3 coverage * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests: fix encoding, IS_WINDOWS patching, and wrong assertion - Add encoding="utf-8" to all read_text() calls (54 occurrences) so tests pass on Windows where the default codec is cp1252 and source files contain UTF-8 emoji (e.g. ⚠️ in install_python_stack.py) - Add @patch.object(stack_mod, "IS_WINDOWS", False) to Linux-path TestEnsureRocmTorch tests so they reach the Linux code path when run on a Windows machine instead of short-circuiting into the Windows branch - Fix test_grouped_mm_patch_guarded_by_windows_and_hip_check: the source uses getattr(_torch_for_rocm, "version", None) not torch.version, so check for '"version"' and '"hip"' substrings instead 137 passed, 2 skipped * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: pin BNB_ROCM_VERSION=72 for torch==2.11.0+rocm7.13.0 compatibility AMD's pip index now ships torch==2.11.0+rocm7.13.0 (ROCm 7.13). bitsandbytes auto-detects HIP 7.13 from torch.version.hip and looks for libbitsandbytes_rocm713.dll, which the AMD Windows prerelease wheel does not ship (it only ships rocm72.dll), causing a load error at training start. Fix: - worker.py section 1f: set BNB_ROCM_VERSION=72 (via setdefault) before section 2 ML imports, so bitsandbytes always loads rocm72.dll on Windows ROCm - install_python_stack.py: set BNB_ROCM_VERSION=72 in _install_bnb_windows_rocm() for any post-install imports; update comment to document root cause - tests: 4 new assertions covering the fix (141 passed, 2 skipped) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: detect BNB ROCm DLL suffix dynamically instead of hardcoding '72' BNB_ROCM_VERSION was pinned to '72' which works today (AMD wheel ships rocm72.dll) but would break again if AMD ships a future wheel with a different DLL suffix (e.g. rocm713.dll). Add _detect_bnb_rocm_dll_ver() to install_python_stack.py: scans the installed bitsandbytes package dir for libbitsandbytes_rocm{VER}.dll using importlib.util.find_spec (no BNB import needed) and returns the suffix. '72' remains the fallback when detection fails. Apply the same detection inline in worker.py section 1f. Both paths still respect a pre-set BNB_ROCM_VERSION (caller override wins). Tests: +8 cases covering detection logic and fallback (147 passed, 2 skipped). * fix: patch torch.distributed stubs in server process for Windows ROCm On Windows ROCm, torch.distributed ships without process-group helpers (is_initialized, is_available, get_rank, get_world_size). The worker subprocess already patches these in section 1e, but the main server process calls _determine_attention_impl_for_gpu_estimate() which calls unsloth's resolve_attention_implementation() → is_initialized(), causing: "Could not resolve attention implementation for '...': module 'torch.distributed' has no attribute 'is_initialized'" Fix: patch the missing attrs onto torch.distributed at the top of _determine_attention_impl_for_gpu_estimate, matching the same stubs already applied in worker.py section 1e. No-ops on Linux/CUDA where torch.distributed is fully populated. * fix: gate _grouped_mm dispatch patch on HIP < 7.13 AMD fixed the gfx1200 null HIP kernel in ROCm 7.13 (torch 2.11+). Users on the new wheel now get the real GPU _grouped_mm kernel for MoE workloads instead of the Python mm fallback. Changes: - worker.py: add _hip_ver_at_least() helper; wrap full _grouped_mm patch in `if not _hip_ver_at_least(7, 13):` with else branch that logs the skip reason; update section-1f comment to document the fix - test_rocm_support.py: add 5 tests covering the helper definition, the (7, 13) gate expression, the else branch, the skip log message, and the AMD-format version string parsing (.split(".")[:2]) Verified: torch==2.11.0+rocm7.13.0 — 3D batch and grouped (offs) variants both succeed; null crash only present on rocm7.12 and earlier. * fix: stub is_torchelastic_launched on torch.distributed for Windows ROCm resolve_attention_implementation calls is_torchelastic_launched() which does not exist in the incomplete torch.distributed shipped with the Windows ROCm wheel, causing a warning on every model config load in the server process. Add it to the stub table alongside the four helpers already patched in _determine_attention_impl_for_gpu_estimate. Also adds two tests: one confirming the new stub and one confirming all five core distributed helpers are covered. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: explicit warnings on AMD ROCm arch/version fallbacks + Fast-Install arg order setup.ps1: - Fix Fast-Install argument order: packages before flags, consistent with all other Fast-Install calls in the file (was: Fast-Install --force-reinstall --index-url $url torch ...) (now: Fast-Install torch torchvision torchaudio --force-reinstall --index-url $url) - Add explicit [WARN] substep when $HasROCm is true but arch mapping fails: - GPU arch detected but not in supported wheel list → names the arch and lists supported families so user knows exactly what to report - HIP SDK present (amd-smi path) but gcnArchName unreadable → instructs user to re-install the HIP SDK; previously fell back silently to CPU install.sh: - Add [WARN] to stderr before silent CPU fallback when AMD GPU is confirmed (rocminfo/amd-smi) but ROCm version cannot be read from any source (amd-smi, /opt/rocm/.info/version, hipconfig, dpkg, rpm) - Add [WARN] to stderr when ROCm version is too old (< 6.0) with upgrade link install.ps1 and setup.sh: no changes needed (already handle these paths correctly) * fix: robust gfx arch detection for Strix Halo / HIP-runtime-only installs Covers users who have the HIP runtime (amd-smi available) but not the full HIP SDK (no hipinfo), which is common on Strix Halo iGPU systems. Without this, $ROCmGfxArch stays null and the installer silently falls back to CPU-only PyTorch despite a working GPU. Detection waterfall (setup.ps1 + install.ps1): 1. hipinfo gcnArchName -- full HIP SDK (existing, unchanged) 2. amd-smi list gfx pattern -- newer amd-smi versions embed arch 3. amd-smi static --asic -- ROCm 6+ ASIC details with GFX target 4. UNSLOTH_ROCM_GFX_ARCH env -- manual override escape hatch 5. GPU name → arch table -- best-effort from marketing name: 890M / Strix Halo → gfx1151 (RDNA 3.5 iGPU, Strix Halo) 880M / Strix Point → gfx1150 (RDNA 3.5 iGPU, Strix Point) 780M / Phoenix → gfx1103 (RDNA 3 iGPU) RX 7900/7800/7700 → gfx1100 (RDNA 3 desktop) RX 9070 XT / 9080 → gfx1201 (RDNA 4) RX 9070 / 9060 XT → gfx1200 (RDNA 4) When arch is inferred from name, a Cyan substep tells the user to set UNSLOTH_ROCM_GFX_ARCH to skip inference on future installs. WMI block intentionally does not set $HasROCm (no runtime confirmation). Tests: 11 new tests in TestStrixHaloGfxArchDetection covering all five detection levels, WMI safety, and gfx regex in both ps1 files. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: resolve hipinfo/hipconfig via HIP_PATH/ROCM_PATH when not on PATH AMD HIP SDK sets HIP_PATH on Windows but does not always add the bin directory to PATH. Get-Command hipinfo therefore silently fails and detection falls through to WMI, which cannot provide a gfx arch, leaving the user with a CPU-only PyTorch install and no warning. Changes: - setup.ps1 / install.ps1: before falling through to amd-smi, attempt to locate hipinfo.exe and hipconfig.exe under $env:HIP_PATH\bin (then $env:ROCM_PATH\bin) when Get-Command returns nothing - Emit a [WARN] with the resolved path and a one-liner to permanently fix PATH via SetEnvironmentVariable - Emit a [WARN] when HIP_PATH/ROCM_PATH is set but the exe is still not found (incomplete SDK install) - Emit a [WARN] with the first hipinfo output line when hipinfo runs but returns a non-zero exit code (e.g. "no ROCm-capable device detected") - 18 new tests in TestHipSdkEnvPathResolution; total 183 passed, 2 skipped * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: print HIP SDK path and full hipconfig version in terminal on AMD detection Both install.ps1 and setup.ps1 now emit substeps under the gpu step when AMD ROCm is detected: gpu AMD ROCm (gfx1200) HIP SDK: C:\Program Files\AMD\ROCm\7.1 hipconfig: 7.1.51803-d3a86bd04 Previously only the gpu label (e.g. "AMD ROCm (gfx1200)") was shown with no indication of where the SDK was found or which exact build was active. The full hipconfig build string (e.g. 7.1.51803-d3a86bd04 instead of just 7.1) is now stored in ROCmVersionFull and also used in setup.ps1's 'rocm' step label. 9 new tests in TestHipSdkDetectedSubstep; total 192 passed, 2 skipped * fix: Strix rocm7.1 segfault bypass + Ubuntu 24.04 HIP gcc-install-dir Issue 1 (install.sh): gfx1151/gfx1150 + ROCm 7.1 causes a segfault in torch._grouped_mm (moe_utils.py:167). The Radeon repo now ships cp313 wheels for rocm-rel-7.1, so _amd_gpu_radeon=true silently lands on the broken combo. When Strix Halo/Point is detected and TORCH_INDEX_URL is rocm7.1, override to rocm7.2 PyTorch index, update TORCH_CONSTRAINT, and set _amd_gpu_radeon=false to bypass the Radeon repo entirely. Emits a clear [WARN] explaining the segfault and linking to the ROCm upgrade docs. Issue 2 (setup.sh): ROCm 7.x ships clang-20 which on Ubuntu 24.04+ picks /usr/lib/gcc/x86_64-linux-gnu/14/ (runtime dir, no C++ headers), causing 'cstdlib file not found' and a failed llama.cpp HIP build. Iterate gcc versions 14→11 to find the first install dir that has both runtime and /usr/include/c++/<ver> headers, then pass --gcc-install-dir to clang via CMAKE_HIP_FLAGS. Fix confirmed by h34v3nzc0dex (llama.cpp 417/417 clean). 11 new tests across TestStrixRocm71Override and TestSetupShGccInstallDir; total 203 passed, 2 skipped * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: BNB_ROCM_VERSION in server process + torch._C._distributed_c10d stubs Two errors visible in training logs on Windows ROCm: 1. Server process bitsandbytes crash: "Configured ROCm binary not found at libbitsandbytes_rocm713.dll" The installed BNB wheel ships rocm72.dll (not rocm713.dll). The training worker already sets BNB_ROCM_VERSION=72 via DLL detection but the server process (main.py) imported bitsandbytes before that ran. Fix: add the same DLL-scan + BNB_ROCM_VERSION assignment to main.py inside the existing win32 guard, before any downstream import can pull in bitsandbytes. 2. torch.distributed import failure: "No module named 'torch._C._distributed_c10d'; torch._C is not a package" torch._C is a C extension on Windows ROCm — Python cannot do submodule imports from it, so torch.distributed fails to import before our attribute stubs could ever run. Fix: inject empty ModuleType stubs for _distributed_c10d, _distributed_autograd and _distributed_rpc into sys.modules inside the win32 guard in hardware.py BEFORE importing torch.distributed, so the import succeeds and our attribute stubs take effect. 9 new tests in TestServerStartupRocmFixes; total 212 passed, 2 skipped * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(win32): populate distributed c10d stub with dummy symbols torch.distributed tries to `from torch._C._distributed_c10d import FakeProcessGroup` (and ProcessGroup, Work, Store, etc.). The previous empty ModuleType stub caused an AttributeError on those names. Populate every stub with a _Dummy class for each known symbol so the import chain completes silently on Windows ROCm where torch._C is a compiled extension and its _distributed_c10d submodule doesn't exist. Adds four new tests in TestServerStartupRocmFixes covering FakeProcessGroup, ProcessGroup, setattr population, and all three _distributed_* siblings. * fix(win32): distinguish HIP SDK installed vs GPU not ROCm-accessible Previously, when hipinfo was found but exited non-zero (e.g. "no ROCm-capable device detected"), both install.ps1 and setup.ps1 fell through to the WMI-label-only branch and printed "AMD GPU detected -- HIP SDK not found" -- factually wrong since the SDK binary is present. Add $HipSdkInstalled flag (set true when hipinfo binary is found, regardless of exit code). When HipSdkInstalled && !HasROCm: - Show "AMD GPU detected -- not ROCm-accessible (HIP <ver>)" instead - Explain this is a driver issue, not an SDK issue, with a link - Still run hipconfig version capture so version shows in output - CPU-only hint now says "GPU not ROCm-accessible" not "require HIP SDK" Also applies to setup.ps1 (same detection block, same branches). Adds TestHipSdkInstalledButDeviceInaccessible (11 tests). * fix(win32): scope ROCm workarounds to AMD hosts only Three Codex-flagged issues where Windows ROCm workarounds incorrectly applied to Windows CUDA (NVIDIA) machines: main.py (P1): BNB_ROCM_VERSION was set unconditionally on all win32 hosts. On NVIDIA, bitsandbytes sees BNB_ROCM_VERSION and looks for a ROCm DLL that doesn't exist, breaking bitsandbytes initialisation. Fix: gate the block on HIP_PATH/ROCM_PATH being present (ROCm hosts only). worker.py (P2): torchao stubs were seeded for all win32 runs, shadowing real torchao on Windows CUDA and silently disabling torchao quantization for NVIDIA users. Fix: gate on HIP_PATH/ROCM_PATH (win32 ROCm only). install_python_stack.py (P1): _detect_windows_gfx_arch() only checked shutil.which("hipinfo"), skipping the HIP_PATH/ROCM_PATH fallback that the PowerShell installers use. On installs where the HIP SDK bin dir is not on PATH, _ensure_rocm_torch() returned early without installing ROCm wheels or bitsandbytes. Fix: mirror the env-var fallback. * fix(linux): route Strix + ROCm 7.1 to AMD arch-specific index Instead of falling back to pytorch.org/rocm7.2, the Strix override now routes to repo.amd.com/rocm/whl/gfx1151/ (or gfx1150/) which serves torch 2.11.0+rocm7.13.0 -- AMD's build containing the actual _grouped_mm kernel fix, verified on real gfx1151 hardware by h34v3nzc0dex. This exercises the real GPU kernel path rather than the rocm7.2 workaround. UNSLOTH_AMD_ROCM_MIRROR can override the base URL for air-gapped installs. Also teaches _tauri_torch_index_family to recognise AMD arch-specific URLs (repo.amd.com/rocm/whl/gfx*) and return the rocm7.13 family label so _tauri_gpu_branch correctly classifies these installs as rocm. Suggested by h34v3nzc0dex based on hardware-verified probe results. * fix(studio/rocm): gate ROCm-only side-effects on active torch runtime Address five edge cases flagged during PR review: 1. studio/backend/main.py: BNB_ROCM_VERSION was set whenever HIP_PATH or ROCM_PATH was present in the environment. A Windows CUDA user who once installed the HIP SDK and reverted to a CUDA torch wheel still has those env vars set, so bitsandbytes would try to load libbitsandbytes_rocm72.dll against a CUDA torch and crash. Now probe torch.version.hip inside the env-var guard (worker.py already does this). 2. studio/backend/main.py: os.add_dll_directory returned handles were discarded. Per CPython docs, the directory leaves the DLL search list when the handle is garbage collected. Retain handles in module-level _ROCM_DLL_HANDLES list so they survive process lifetime. 3. studio/install_python_stack.py: _install_bnb_windows_rocm() returned None regardless of pip_install_try outcome, and the caller flipped _rocm_windows_torch_installed to True unconditionally. On a failed BNB install the post-install "manual install may be required" warning was suppressed and the user was misled. Helper now returns bool; caller gates on it. 4. studio/install_python_stack.py: _detect_windows_gfx_arch returned the raw capture group, so mixed-case hipinfo output ("Gfx1151") missed the lowercase keys in _GFX_TO_AMD_INDEX_ARCH and silently fell back to CPU torch. Lowercase the token. 5. studio/install_python_stack.py: UNSLOTH_ROCM_TORCH_INSTALLED=1 early- return trusted the env var even when the venv was wiped between runs. Subprocess-probe torch importability first; fall through to the full install path if the probe fails. Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py (adds one new test for case 5 fall-through). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/rocm): worker.py parity + don't roll back ROCm torch on bnb failure Addresses findings from a 10x reviewer pass on the prior fix commit: 1. studio/backend/core/training/worker.py (parity with main.py): - Gate the torchao stub block on torch.version.hip / 'rocm' in torch.__version__ instead of HIP_PATH / ROCM_PATH env-var presence. Same root cause as main.py: HIP SDK env vars stick around on CUDA hosts. - Add module-level Windows ROCm DLL registration block. Worker subprocesses inherit env vars but not the parent's add_dll_directory handles, so the first `import torch` in the worker could fail to find amdhip64.dll when HIP_PATH\bin is not on PATH. Mirrors main.py setup. Handles retained at module scope via _ROCM_DLL_HANDLES. - Promote _WINDOWS_ROCM_GROUPED_MM_LIB to module scope with `global` in run_training_process so the torch.library.Library registration survives past function return / mid-run garbage collection. - Harden _torch_has_hip() to also accept 'rocm' in torch.__version__ (AMD SDK / Radeon wheels may not set torch.version.hip). 2. studio/install_python_stack.py: - Don't roll back ROCm torch when bitsandbytes install fails. The prior commit gated _rocm_windows_torch_installed on _install_bnb_windows_rocm() returning True; if torch installed successfully but bnb failed, the flag stayed False and later install steps could overwrite ROCm torch with the generic CPU torch wheel. Set the flag after torch install; surface bnb failure as a separate warning instead. - _detect_windows_gfx_arch now probes in three tiers: UNSLOTH_ROCM_GFX_ARCH env-var override (matches the PowerShell installer), then hipinfo (PATH or HIP_PATH\bin), then amd-smi (`static --asic`, `list`). Without the amd-smi fallback, runtime-only Radeon installs without hipinfo on PATH made `studio update` return early and leave the venv on CPU torch. - Linux torch-already-rocm probe in _ensure_rocm_torch now matches the Windows probe shape: accepts torch.version.hip OR 'rocm' in torch.__version__ to cover AMD SDK / Radeon Linux wheels. 3. studio/backend/utils/hardware/hardware.py: - apply_gpu_ids() final-fallback torch probe accepts 'rocm' in torch.__version__ in addition to torch.version.hip, matching detect_hardware(). AMD SDK wheels could otherwise leak through with CUDA-only visibility masks on a spawned ROCm worker. Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py (no test changes needed; the probe shape that prints the hip version (or 'rocm' sentinel) preserves the existing non-empty-string contract). Not addressed in this commit (deferred or out of scope): - Tag drift / lemonade checksum (PR 5303 surface, not this PR). - install.sh rocm7.2.1 URL: small fix, separate. - install.ps1 / setup.ps1 'Radeon 8060S' marketing-name fallback table. - Strix Halo + ROCm 7.1 routing asymmetry in Python update path. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/rocm): robustness pass - rocm tag normalisation, Strix routing parity, hardened detection Robustness pass on top of 76137b2d. Four targeted fixes: 1. install.sh ROCm-tag routing normalisation. `rocm7.2.1` would route to https://download.pytorch.org/whl/rocm7.2.1 which does not exist (PyTorch publishes major.minor URLs only). Same for any future patch-level tag. Normalise every rocm{maj.min}* pattern to the bare {maj.min} index URL. 2. install.ps1 + studio/setup.ps1 marketing-name fallback. The gfx1151 row matched 890M / Strix Halo / HX 37x / HX 38x / AI 9 HX but not the actual retail name 'AMD Radeon 8060S Graphics' shipped by OEMs (Ryzen AI MAX+ 395). Add '8060S' to the regex. 3. install_python_stack.py Strix + ROCm 7.1 routing parity with install.sh. The shell installer reroutes Strix Halo / Point + ROCm 7.1 to repo.amd.com/rocm/whl/{gfx}/ (which serves torch 2.11.0+rocm7.13.0 with the upstream _grouped_mm fix). The Python `studio update` path only warned and still installed the broken generic rocm7.1 wheel. Mirror the override: detect gfx1151/gfx1150 on ROCm 7.1, route to the AMD per-gfx index, honour UNSLOTH_AMD_ROCM_MIRROR override. 4. _detect_windows_gfx_arch amd-smi parsing tightened. The amd-smi fallback added in the prior commit used a bare `\bgfx[1-9][0-9a-z]{2,3}\b` match against the lowercased stdout, which could pick up stray gfx references in warnings / device-name strings. Anchor on labelled lines first (Target_Graphics_Version, ASIC, Arch, gfx) and fall back to the bare match only when no labelled line is present. Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py; sim_5301 23 cases pass (6 new sims for the Strix override + amd-smi parsing). * fix(studio/rocm): multi-GPU selection, Strix sibling handling, defensive cleanups Round 4 robustness pass based on 5 parallel Opus reviewers of head 21773215. Seven items from across regression / edge-case / error-paths / architecture reviews: 1. studio/backend/main.py BNB gate: aligned with the broad ROCm check used everywhere else in this PR (torch.version.hip OR 'rocm' in __version__). AMD SDK / Radeon Linux wheels do not always populate torch.version.hip; without this, main.py would silently skip BNB_ROCM_VERSION while worker.py set it. 2. studio/install_python_stack.py _install_bnb_windows_rocm: init _ok = False before the try block. Without this, if pip_install_try itself raises (e.g. OSError on uv binary missing), the finally block restored env vars correctly but the subsequent `if not _ok:` raised UnboundLocalError, masking the original exception. 3. studio/install_python_stack.py _detect_windows_gfx_arch: - Rewrote to use re.findall (not re.search) on both hipinfo and amd-smi output, dedup tokens preserving order, and select via new _pick_visible_index() helper. - HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES (first comma entry, integer) now picks the right GPU on multi-AMD-GPU hosts. Out-of-range or non-int values fall back to the first GPU (matches detect_host behaviour in install_llama_prebuilt.py). 4. studio/install_python_stack.py Strix override now consults the runtime target before flipping: - Previous behaviour intersected gfx_codes with {gfx1151, gfx1150} and picked the first Strix arch, ignoring whether HIP_VISIBLE_DEVICES selected a non-Strix sibling (e.g. discrete RX 7900 in a mixed APU+dGPU box). Could install Strix-specific wheels onto a gfx1100 dGPU. - Now resolves the runtime gfx via _pick_visible_index() and only overrides when that runtime target is in the Strix set. 5. studio/backend/main.py + studio/backend/core/training/worker.py: ROCm version dir scan no longer sorts lexically. Previous sort placed "10.0" before "7.0" alphabetically, which would mis-prioritise ROCm 10.x bin dirs once AMD ships them. New _ver_key() splits on "." and sorts numerically with a string fallback. 6. install.sh Strix override URL: replaced ${var%/} (strips one trailing slash) with a while-loop that strips all trailing slashes, matching Python's .rstrip("/"). A user setting UNSLOTH_AMD_ROCM_MIRROR with "http://corp/whl///" no longer ends up with "http://corp/whl///gfx1151/" which strict pip proxies (artifactory, sonatype) 404 on. 7. studio/install_python_stack.py: bumped torch import probe timeout from 30s to 90s. PyTorch's lazy .so loading can take 60-90s on cold NFS or USB-backed venvs. The shorter timeout was producing a false "torch missing" classification and reinstalling a working ROCm torch. Tests: 231 passed, 1 skipped. sim_5301 30 cases pass (added 7 new sims for multi-GPU detection, Strix sibling handling, and _ok-init regression). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/rocm): worker BNB/grouped_mm broad gate, install.sh Strix visibility, runtime-only ROCm detection Round-5 robustness pass based on 20 parallel reviewers of head 96b9e465. 1. studio/backend/core/training/worker.py - BNB version pin / dynamo disable / _grouped_mm fallback block was still gated on torch.version.hip alone despite the torchao stub block above already using the broad check. AMD SDK / Radeon Windows wheels (torch.__version__ contains "rocm" but torch.version.hip is None) silently skipped the Windows ROCm runtime patches. Aligned to the same broad check (8/20 reviewers). 2. studio/backend/core/training/worker.py - _hip_ver_at_least() now also parses the ROCm version out of torch.__version__ (e.g. "2.11.0+rocm7.13.0") when torch.version.hip is missing, so the kernel-fix gate is correct for SDK / Radeon wheels too. 3. studio/backend/core/training/worker.py - _grouped_mm_safe_impl with offs=None now picks torch.bmm/matmul for 3-D inputs instead of always calling torch.mm. The real _grouped_mm accepts 3-D batched matmul; the prior fallback raised "self must be a matrix" on MoE workloads (2/20). 4. studio/backend/main.py - dropped the HIP_PATH / ROCM_PATH env-var gate from the BNB block; probe torch directly. Runtime-only Radeon / AMD SDK Windows installs do not set those SDK env vars but still ship ROCm torch (5/20 reviewers). 5. install.sh - Strix override now collects every gfx token from rocminfo / amd-smi (in enumeration order), then indexes by HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES so a mixed Strix iGPU + non- Strix dGPU host where the user selected the dGPU does NOT get rerouted to the Strix per-gfx index. Mirrors the Python update path (5/20 reviewers). 6. install.sh - Strix detection chain now also probes `amd-smi static --asic`, matching the PowerShell installer (1/20). Closes the gap on runtime-only Strix hosts where `amd-smi list` does not surface a gfx token. 7. studio/install_python_stack.py - _has_rocm_gpu() now has the sysfs KFD topology fallback (/sys/class/kfd/kfd/topology/nodes/*/gpu_id), matching install.sh. On minimal package-managed installs without rocminfo / amd-smi GUI tools, `studio update` can now detect the GPU and repair the venv instead of returning early (2/20). 8. studio/install_python_stack.py - _detect_amd_gfx_codes() now falls back to `amd-smi list` and `amd-smi static --asic` when rocminfo is missing (2/20). Strix routing on runtime-only Radeon hosts now matches what install.sh has done for a while. 9. studio/install_python_stack.py - Strix override now applies even when has_hip_torch is True. The whole point of the override is to repair an existing broken torch.version.hip == "7.1" install; skipping the reinstall left users on the known _grouped_mm segfaulting stack (3/20). Tests: 231 passed, 1 skipped. sim_5301 30 cases pass. sim_cross 12 pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/rocm): code review hardening pass - main.py: numeric DLL sort (string sort picked rocm72 over rocm713); add basename() to regex; log warning on detection failure; log info when BNB_ROCM_VERSION is set (mirrors worker.py) - worker.py: explicit len-guard in _hip_ver_at_least() with warning logs instead of silent IndexError/ValueError swallow - hardware.py: isinstance(result, dict) guard before result.get() in _smi_query() to prevent AttributeError on non-dict backend returns - amd.py: round() before int() on parsed GPU IDs; log warning when truncation occurs (defensive against malformed amd-smi output) - setup.sh: quote --gcc-install-dir value in CMAKE_HIP_FLAGS so paths with spaces do not break the CMake argument - install.ps1, setup.ps1: apply colon-split + ToLower() to hipinfo gcnArchName match (consistent with each other and with setup.sh) - install.sh: tighten ROCm tag case patterns to explicit rocmX.Y|rocmX.Y.* to avoid unintended prefix matches * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/training): GPU OOM guard to prevent system freeze on VRAM exhaustion On RDNA 4 (gfx1200/gfx1201) and other ROCm GPUs, exhausting VRAM can cause a HIP driver hang that freezes the entire system rather than raising a recoverable Python exception. Two-part fix: - set_per_process_memory_fraction(0.90) caps the HIP/CUDA allocator at 90% of VRAM so PyTorch raises OutOfMemoryError before hitting the hardware limit, keeping the driver alive and the system responsive - top-level exception handler detects OOM errors by type and message and surfaces a clear actionable message to the UI (reduce max_seq_length, enable gradient_checkpointing, lower batch size) instead of the raw CUDA/HIP error string * fix(studio/rocm): OOM guard ROCm-only + unified memory, multi-GPU arch selection OOM guard (worker.py): - Scope to _hw.IS_ROCM only -- NVIDIA CUDA has a graceful OOM path and does not need the allocator cap - Detect unified memory by comparing torch VRAM against psutil system RAM; use 0.80 on unified-memory APUs (gfx1151 Strix Halo) where the GPU pool is carved from host RAM, 0.90 on discrete cards Multi-GPU arch selection: - install.ps1 / setup.ps1: replace -match (first hit only) with [regex]::Matches() to collect all gcnArchName entries, then index by HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES - install_python_stack.py: index into full token list before dedup so HIP_VISIBLE_DEVICES=2 on [gfx1100, gfx1100, gfx1151] resolves gfx1151 - install.sh: remove awk dedup from gfx token collection for same reason GCC multiarch (setup.sh): - Only append -linux-gnu when gcc -print-multiarch does not already return the full triple, fixing double-suffix on Ubuntu 24.04 * fix(tests): update ROCm version cap expectations from rocm7.1 to rocm7.2 Daniel's normalisation commit updated the cap from rocm7.1 to rocm7.2 since PyTorch now publishes that index and rocm7.2 ships torch 2.11.0. Test expectations were stale. * fix(tests): correct MLX smoke test losses_per_step assertion logging_steps=1 with max_steps=30 produces 30 loss entries, not 7. The assertion was stale from a previous config. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/worker): detect unified-memory APU by GPU name not VRAM/RAM ratio The previous heuristic (VRAM > 50 % of system RAM) false-positived on discrete cards in low-RAM systems — e.g. RX 9060 XT 16 GB on a 16 GB or 24 GB machine would trip the unified-memory path and log "unified memory host" when it should say "discrete". AMD iGPUs (gfx1150/gfx1151 Strix Halo, Strix Point, etc.) expose names with a digit+M suffix ("AMD Radeon 890M"), while discrete cards use "RX NNNN [XT|XTX]" naming. Matching that suffix is reliable across all current ROCm-capable AMD consumer GPUs and does not require psutil. Also includes the device name in the log line to ease future debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(install/setup.ps1): force array on hipinfo gcnArchName parse to fix single-GPU arch truncation When [regex]::Matches() finds exactly one match, PowerShell's pipeline unwraps the result to a scalar string. Indexing a scalar string with [0] returns the first *character*, so a one-GPU system would parse gcnArchName "gfx1200" as "g", which is not in the supported arch map and triggers the CPU-only fallback. Wrapping with @() forces the result to remain an array regardless of match count. On a single-GPU machine the arch is now correctly read as "gfx1200" (or whatever the full name is) so the ROCm wheel index is selected. Reproducer: hipinfo exits 0 and outputs exactly one gcnArchName line. Without @(), $_hipAllArches = "gfx1200" (String); $_hipAllArches[0] = 'g'. With @(), $_hipAllArches = @("gfx1200") (Object[]); $_hipAllArches[0] = "gfx1200". * fix(studio/rocm): classify unified-memory APU via VRAM/RAM ratio, not arch list Replace the gcnArchName allowlist {gfx1150, gfx1151} with a psutil-based heuristic: unified APUs expose the entire system RAM as the HIP pool (ratio ≥ 0.90), discrete cards are well below that. No arch name required — future APUs classify correctly without code changes. Also removes the stale import re / \d[Mm]\b device-name regex that 5d84704 left behind, and logs vram/sys GiB for easier on-hardware verification. Addresses h34v3nzc0dex review: Radeon 8060S (gfx1151, 128 GiB unified) now correctly gets 0.80 cap instead of 0.90. * fix(studio/rocm): revert to gcnArchName for unified-memory APU classification VRAM/RAM ratio >= 0.90 false-positives on machines where discrete VRAM equals system RAM (e.g. RX 9060 XT 16 GB + 16 GB system RAM → ratio 1.0, incorrectly classified as unified → wrong 0.80 cap applied). gcnArchName is the correct signal: naming-independent, stable within a product family, and already parsed throughout this PR. Unified set is {gfx1150, gfx1151} (Strix Point + Strix Halo). * fix(studio/llama-prebuilt): resolve hipinfo via HIP_PATH/ROCM_PATH on Windows shutil.which("hipinfo") returns None when the HIP SDK bin dir is not on PATH -- the HIP SDK installer sets HIP_PATH/ROCM_PATH but does not always add the bin dir to PATH. This caused has_rocm=False in the prebuilt asset selector, so AMD ROCm machines got the CPU llama.cpp zip instead of the HIP one, silently running all chat inference on CPU. Add _resolve_exe() that falls back to %HIP_PATH%\bin and %ROCM_PATH%\bin when shutil.which() finds nothing, mirroring the same fallback already present in setup.ps1. * fix(studio/llama-prebuilt): pass --has-rocm from setup.ps1 to skip re-detection The Python prebuilt installer re-detects ROCm independently via shutil.which("hipinfo"), which fails when hipinfo is not on PATH (HIP SDK sets HIP_PATH but doesn't always add the bin dir to PATH). This caused has_rocm=False and downloaded the CPU llama.cpp zip even on confirmed AMD ROCm machines. setup.ps1 already performs reliable ROCm detection with its own HIP_PATH/ROCM_PATH fallback. Add --has-rocm flag to install_llama_prebuilt.py so setup.ps1 can forward its result directly, and pass it whenever $HasROCm is true. The Python script then overrides has_rocm=True in the HostInfo without re-probing. * fix(studio/llama-prebuilt): add HIP asset to simple-policy Windows path direct_upstream_release_plan (used by --simple-policy, which setup.ps1 always passes) only checked has_usable_nvidia on Windows and fell straight to CPU for AMD ROCm machines, ignoring has_rocm entirely. The --has-rocm override had no effect because the simple-policy code path never reached resolve_asset_choice where has_rocm was checked. Add an elif branch for has_rocm that tries the upstream HIP asset (llama-TAG-bin-win-hip-radeon-x64.zip) before falling through to the CPU fallback, consistent with the non-simple-policy path. * fix(studio/setup.ps1): auto-remove mismatched llama.cpp install kind When an existing llama.cpp install is the wrong kind for the current GPU (e.g. windows-cpu on an AMD ROCm machine that should have windows-hip), the prebuilt installer skips on tag match and never upgrades. Read install_kind from UNSLOTH_PREBUILT_INFO.json before invoking the installer and remove the directory if the kind doesn't match, forcing a fresh download of the correct variant. * fix(studio/setup.ps1): show live PyTorch install output in verbose mode for ROCm The ROCm torch reinstall (setup.ps1 phase) always silently captured output, so in --verbose mode the torch downgrade mid-install (2.11.0+rocm → 2.10.0 → 2.11.0+rocm) looked like the final state was 2.10.0. Match the CPU/CUDA blocks which show live uv output when $script:UnslothVerbose is set. * fix(rocm/windows): set ROCBLAS_TENSILE_LIBPATH for bundled rocblas.dll The llama.cpp ROCm prebuilt bundles rocblas.dll next to the binary but not the Tensile kernel library files it depends on at runtime (rocblas/library/TensileLibrary*.dat + *.hsaco). The bundled DLL searches for these files relative to its own location by default, i.e. <binary_dir>/rocblas/library/, which does not exist in the prebuilt install tree. This causes a silent crash on the very first GEMM (prefill) with no output from llama-server, seen by the caller as WinError 10054 / 10061. Model load and the single-token warmup pass because they use simpler code paths that do not trigger rocBLAS GEMM. Fix: set ROCBLAS_TENSILE_LIBPATH in the subprocess env to <HIP_PATH>/bin/rocblas/library so the bundled DLL finds the kernel files from the system ROCm installation. Uses setdefault so a user- supplied env var is never overwritten. No-ops on CUDA and CPU (no HIP_PATH) and on Linux (win32 branch only). Reproducer log: rocBLAS error: Cannot read .../Release/rocblas/library/TensileLibrary.dat rocBLAS error: Could not initialize Tensile host: directory_iterator: The system cannot find the path specified. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(install.sh): restore gfx token dedup in Strix multi-GPU awk indexer 536a54df removed the per-source `| awk '!seen[$0]++'` dedup from the _gfx_all collection step but left the indexer awk as bare NF, so on a mixed-arch host (e.g. dGPU gfx1100 + Strix iGPU gfx1151) where rocminfo emits each gfx token twice (Name: field + ISA triple), HIP_VISIBLE_DEVICES=1 indexed vals[1] = the second gfx1100 occurrence instead of gfx1151, triggering the Strix routing on the wrong GPU. Add !seen[$0]++ to the indexer awk so duplicate tokens from the same GPU collapse to one entry before the HIP_VISIBLE_DEVICES index is applied -- matching exactly what the Python side does with dict.fromkeys() in _detect_amd_gfx_codes(). The comment above the block ("skip duplicates") already documented this as the intended behaviour. * fix(studio/install): correct _TOTAL progress count on Windows base_total += 3 fired for all non-macOS platforms including Windows, but flash-attn (line 1620) and ROCm torch final (line 1705) are both guarded by 'not IS_WINDOWS and not IS_MACOS', so on Windows with torch enabled _TOTAL was 13 while only 11 _progress() calls actually execute. Split into +1 for the ROCm torch check (all non-macOS) and +2 for the two Linux-only steps, so Windows gets _TOTAL=11 and Linux gets 14. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(install.ps1): enforce torch>=2.11.0 for gfx120X and Strix on Windows The AMD arch-specific index (repo.amd.com/rocm/whl/gfx120X-all/ and gfx1151/) publishes torch wheels from 2.7.1 through 2.11.0. Without a version floor pip can resolve to torch 2.10.0+rocm7.12 on RDNA 4 (gfx120X) or torch 2.10.0+rocm7.1 on Strix (gfx1151/gfx1150), both of which have a null-pointer crash in torch._C._grouped_mm (TheRock issues #5284 / #3284). torch 2.11.0+rocm7.13 contains the fix. Add $ROCmTorchFloor alongside $ROCmIndexUrl: set to torch>=2.11.0 for the two affected arch families, null for all others. Wire it into the uv pip install call so the broken wheels are never selected. * fix(rocm/windows): address Codex nits - deterministic DLL suffix, CUDA llama.cpp kind, HIP_VISIBLE_DEVICES arch indexing - install_python_stack.py / worker.py: _detect_bnb_rocm_dll_ver() and the inline worker probe now collect ALL libbitsandbytes_rocm*.dll suffixes and return max() by numeric value instead of stopping at the first glob hit. Filesystem glob order is not guaranteed; this ensures '713' always wins over '72' when both variants are present in the wheel. - setup.ps1 (expectedKind): add 'windows-cuda' branch so NVIDIA hosts are not treated as 'windows-cpu'. Previously an existing windows-cuda prebuilt was always considered a mismatch on non-ROCm machines, forcing an unnecessary re-download on every update. - setup.ps1 (amd-smi gfx arch): collect ALL gfx tokens from amd-smi list output in GPU order and honour HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES when selecting which arch to use. On mixed-arch AMD systems where the visible GPU is not the first enumerated one, this prevents installing an incompatible wheel index. Falls back to index 0 (same as before) when the visibility var is unset or is a comma-separated list. - test_rocm_support.py: add test_picks_highest_suffix_when_multiple_dlls to cover the multi-DLL case that was previously untested. * fix(rocm): misleading amd-smi log, BNB spec consistency, torch ceiling for AMD index amd.py: split 'returncode != 0 or not stdout' into two separate branches. Previously, exit-0 with empty output logged 'amd-smi returned code 0' (which reads as success, not a warning) and incorrectly incremented the circuit-breaker counter. Now: non-zero exit logs the code and counts toward the limit as before; empty stdout on exit 0 logs at DEBUG level and does not penalise the counter (amd-smi --json always emits at least [] on exit 0, so this branch is rare and is not a tool failure). main.py: replace spec.origin / os.path.dirname() with spec.submodule_search_locations to match install_python_stack.py and worker.py. For normal wheel installs both approaches reach the same directory, but using submodule_search_locations is the canonical way and handles editable bitsandbytes installs correctly. Also use max() by numeric suffix (same as the other two sites) instead of a sort-then-break loop. install.ps1: add <2.12.0 ceiling to the torch constraint for gfx120X (RDNA 4) and gfx1151/gfx1150 (Strix). AMD actively publishes new versions on their per-arch index; without a ceiling, a future 2.12.0+rocmX.Y wheel would be pulled in automatically before being validated on these architectures. The ceiling matches the existing Linux install_python_stack.py constraint for the same arches. Bump both when 2.12.x is confirmed working. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(rocm): torch floor in setup.ps1, torchvision pin for Strix, rocmsdk in _hip_ver_at_least setup.ps1: add \ (mirrors install.ps1) and derive \ from it. Previously the AMD index install called 'Fast-Install torch torchvision torchaudio --force-reinstall --index-url \' with no version constraint, so pip could resolve torch 2.10.0+rocm7.12 for gfx1151/gfx1200 -- the exact broken wheel the PR is meant to avoid. Now gfx120X and Strix enforce 'torch>=2.11.0,<2.12.0', matching install.ps1 and the Linux constraint. install_python_stack.py: pin torchvision and torchaudio in _strix_override_pkgs. The Strix Linux override uses --index-url (exclusive, no PyPI fallback); bare unversioned 'torchvision' and 'torchaudio' could resolve a build from AMD's index targeting a different torch major, causing ABI/version mismatches at runtime. Now pinned to '>=0.26.0,<0.27.0' and '>=2.11.0,<2.12.0' respectively, matching _ROCM_TORCH_CONSTRAINT['rocm7.2']. worker.py: extend _hip_ver_at_least to handle AMD SDK wheel version strings. The fallback regex r'rocm(\d+)\.(\d+)' cannot match '2.9.0+rocmsdk20251116' (no rocmX.Y component), so the function always returned False on SDK/Radeon wheels -- installing the Python _grouped_mm workaround on wheels that already have the working HIP kernel. Added a second check: if the version string contains '+rocmsdk', assume >= 7.13 (the rocmsdk format post-dates the gfx120X null-kernel fix) and skip the fallback. * fix(rocm): warn on OOB HIP_VISIBLE_DEVICES, bail on empty numeric_ids mask - setup.ps1: when HIP/ROCR_VISIBLE_DEVICES names an index beyond the detected GPU count, emit a yellow warning and fall back to GPU 0 instead of silently reading allGfxArches[-1] (wrong arch) - hardware.py _reconcile_primary_rocm_unified_memory: distinguish numeric_ids=None (no env var, use torch ordinal 0) from numeric_ids=[] (empty mask / HIP_VISIBLE_DEVICES=-1, no GPU visible); bail out early in the empty case to avoid querying torch.device(0) incorrectly * fix(rocm): gate StubSubpackageFinder on win32 ROCm, add gcnArchName fallbacks - worker.py _StubSubpackageFinder: the meta_path append was running on every platform on every call to run_training_process; moved it inside the if _is_win32_rocm: block since stubs are only seeded there and the finder is a pure accumulation on Linux/Windows CUDA - worker.py OOM guard: AMD SDK / Radeon wheels may not populate gcnArchName, causing Strix Halo to be misclassified as discrete and get the 0.90 cap (12.8 GB OS headroom) instead of 0.80 (25.6 GB); now tries gcn_arch_name / arch_name / gfx_arch_name variants first, then falls back to device-name matching (890M -> Strix Halo, 880M -> Strix Point) with a debug log when the fallback fires * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(rocm): pin torchvision/torchaudio in setup.ps1, remove -Unique from arch array - setup.ps1 ROCm torch install: torchvision and torchaudio were passed bare alongside pinned torch>=2.11.0,<2.12.0 for gfx1151/gfx1200 arches. AMD publishes packages independently so a future torchvision 0.27 (for torch 2.12) on the same arch index would cause pip ResolutionImpossible or an ABI-incompatible install. Added torchvisionFloorMap and torchaudioFloorMap mirroring install_python_stack.py's strix override (torchvision>=0.26.0,<0.27.0, torchaudio>=2.11.0,<2.12.0) and derived ROCmVisionSpec/ROCmAudioSpec used in all three Fast-Install call sites. - setup.ps1 amd-smi arch detection: Select-Object -Unique was collapsing same-arch multi-GPU arrays (e.g. two gfx1151 APUs -> 1-element array) causing HIP_VISIBLE_DEVICES=1 to trigger a false out-of-range warning and fall back to GPU 0 even though the correct GPU would have been at index 1. Removed -Unique; added comment noting the positional-index assumption and its non-contiguous-GPU limitation. * fix(rocm): add 8060s/8050s to OOM guard device-name fallback, extract classifier helper Path 3 of the OOM guard device-name fallback only checked for 890m/880m (gfx1150 Strix Point SKU names). Strix Halo (gfx1151) ships as Radeon 8060S (Ryzen AI MAX+ 395) and Radeon 8050S (cut-down SKU) -- neither matches, so the fallback returned is_unified=False and applied the 0.90 fraction instead of 0.80, leaving ~12.8 GiB OS headroom on a 128 GiB pool instead of ~25.6 GiB. Fix: add 8060s and 8050s to the name-match set. Also correct the comment that mislabelled 890M as a Strix Halo name (it is Strix Point). Refactor: extract the three-path classifier into _rocm_classify_unified_memory() so it can be unit-tested directly. Add 31 test cases in test_rocm_oom_guard.py covering all three paths and the regression case (Radeon 8060S Graphics). Reported-by: h34v3nzc0dex * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(rocm): pass explicit dtype on bf16-unsupported hardware (RDNA2) dtype=None lets unsloth auto-detect the model dtype. On RDNA2 (gfx103x, e.g. RX 6600) is_bfloat16_supported() incorrectly returns True, so unsloth picks bf16 and the first bf16 kernel dispatch triggers: LLVM ERROR: Cannot select: intrinsic %llvm.amdgcn.fdot2.bf16.bf16 Replace every dtype=None in load_model() with _auto_dtype which resolves to None when bf16 is supported (all modern NVIDIA + RDNA3+) and torch.float16 otherwise. This gives RDNA2 users a working float16 training path without touching NVIDIA behaviour at all. Fixes: https://github.com/unslothai/unsloth/issues/5337 * fix: reduce log noise for expected non-issues on Windows ROCm Three log lines fired at warning/error level for conditions that are completely expected on a Windows HIP SDK-only setup: amd.py - amd-smi WinError 2 (FileNotFoundError): downgrade warning -> debug. amd-smi ships with Adrenalin, not the HIP SDK; absence is normal. - 'disabling' message: downgrade warning -> info with clearer text 'not available (not installed; expected on HIP SDK-only systems); GPU VRAM polling disabled' hardware.py - torch.distributed.Store missing: downgrade warning -> debug. The distributed stub added in this PR intentionally omits Store; the attention-impl fallback to eager is expected and non-actionable. worker.py - causal-conv1d: add early Windows exit (info) in both _ensure_causal_conv1d_fast_path and _causal_conv1d_install hook; no cp313/win_amd64 wheel exists, so the install always fails. - FLA: add early Windows exit (info) in _ensure_flash_linear_attention_unconditional; triton dependency has no cp313/win_amd64 wheel. - Defense-in-depth: _install_package_wheel_first non-HIP PyPI failure logs info+debug on Windows instead of error; FLA failure logs info+debug on Windows instead of warning. * [AMD] FIx installation of bitsandbytes when it's from .dev and skip rebuilding llama.cpp if we build it manually. * fix: use force_pip for Windows ROCm bitsandbytes prebuilt wheel install uv rejects the bnb continuous-release wheel due to filename/metadata version mismatch (1.33.7.preview vs 0.50.0.dev0). Switch to force_pip=True (pip bypass) instead of the UV_SKIP_WHEEL_FILENAME_CHECK env var workaround -- cleaner and consistent with how the Linux path handles it. BNB_ROCM_VERSION is still set post-install to the detected DLL suffix so the worker subprocess loads the correct libbitsandbytes_rocm{VER}.dll even when torch.version.hip reports a newer HIP version than the wheel ships. * fix: three small correctness fixes found in PR review - _install_bnb_windows_rocm: use UV_SKIP_WHEEL_FILENAME_CHECK=1 with try/finally instead of force_pip=True so the env var is always restored and the failing CI test passes - _determine_attention_impl_for_gpu_estimate: gate torch._C distributed stubs on IS_ROCM so Windows CUDA users keep the real extension - install.ps1 amd-smi fallback: collect all gfx tokens and index by HIP_VISIBLE_DEVICES, matching the hipinfo path on multi-GPU hosts * fix: stub torchao in export subprocess on Windows ROCm On Windows, the ROCm build of PyTorch ships without the distributed C extension (torch._C._distributed_c10d). torchao, which is pulled in transitively by transformers.quantizers at import time, walks into torch.distributed._functional_collectives -> distributed_c10d and crashes with: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package This only affected the export subprocess because the training subprocess already applied an identical torchao stub (introduced separately to fix the same root cause). The export subprocess had no such guard and died during 'Importing Unsloth...' before any model loading could happen. Fix: apply the same _StubSubpackageFinder / torchao stub pattern to the export subprocess entry point, gated on Windows ROCm detection, before any import of transformers or unsloth_zoo. Root cause tracked in ROCm/TheRock#3284 (libuv / torch.distributed missing on Windows ROCm builds). Ref: https://github.com/ROCm/TheRock/issues/3284 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh, setup.sh: add GPU arch step logging to match PS1 scripts Both shell scripts were missing the step "gpu" terminal log block that install.ps1 and setup.ps1 emit. This adds equivalent output: GPU label with gfx arch (e.g. "AMD ROCm (gfx1151)"), ROCm root path, hipconfig version, and marketing name substep. Includes the same gfx arch detection chain (rocminfo → amd-smi list → amd-smi static --asic), UNSLOTH_ROCM_GFX_ARCH env override, and name-based arch inference table (Strix Halo/Point, RDNA 3/4) as the PS1 versions. install.sh also replaces bare echo blocks for the AMD ROCm and CPU-only cases with formatted substep output. * Fix BNB_ROCM_VERSION gate, ROCm GPU mask preference, APU unified memory and Release build for PR #5301 - main.py: gate BNB_ROCM_VERSION on the rocm bnb DLL or HIP_PATH/ROCM_PATH instead of importing torch on every Windows host - hardware.py: prefer HIP/ROCR visible-device masks only on ROCm hosts so a stale mask cannot override CUDA_VISIBLE_DEVICES on NVIDIA - llama_cpp.py: set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 only for unified-memory APUs (gfx1150/gfx1151) - setup.sh: pass -DCMAKE_BUILD_TYPE=Release for the HIP source build - add test_amd_apu_unified_memory.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: guard recompile_limit + fix AMD VRAM monitor fallback trainer.py: torch._dynamo.config.recompile_limit does not exist in some ROCm torch builds (e.g. pytorch.org/whl/rocm6.2 wheels). Guard the assignment so training doesn't crash on RDNA2/RDNA3. hardware.py: when amd-smi/nvidia-smi is unavailable or returns no usable data (HIP SDK-only Windows, Docker, unexpected JSON format), the existing fallback used torch.cuda.memory_allocated() which is process-specific and reads near-zero even with a fully loaded model. Switch to torch.cuda.mem_get_info() via _torch_get_per_device_info() which reports system-wide VRAM occupancy so the GPU monitor shows real usage on all AMD systems without requiring amd-smi. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: Windows VRAM monitor via Performance Counter API When amd-smi/nvidia-smi is unavailable on Windows, query dedicated GPU VRAM via Windows Performance Counters (same source as Task Manager). This gives system-wide cross-process usage, fixing the near-zero reading caused by torch.cuda.mem_get_info only seeing the Studio server process. Linux fallback path unchanged (mem_get_info is system-wide on ROCm). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: rename to _rocm_windows_perf_counter_vram_gb, scope to IS_ROCM Function is AMD ROCm specific — amd-smi absent on Windows when only the HIP SDK is installed. Scoped to IS_ROCM so NVIDIA Windows path is untouched (nvidia-smi handles that case). * fix: AMD VRAM monitor — Linux DRM sysfs + Windows perf counter Linux: read /sys/class/drm/card*/device/mem_info_vram_used|total for system-wide GPU memory across all processes. No tools required, always present on Linux AMD systems. Windows: Windows Performance Counter API (already added). Both paths are gated on IS_ROCM and only fire when amd-smi is absent. torch mem_get_info remains as last resort (process-local). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: AMD GPU monitor — utilization, temperature, and power for Windows and Linux fallback paths - Windows: GPU utilization via \GPU Engine(*engtype_3D*)\Utilization Percentage perf counter - Windows: temperature and power via ADL (atiadlxx.dll, ships with Adrenalin) - Linux: GPU utilization via DRM sysfs gpu_busy_percent - Linux: temperature via hwmon temp1_input (millidegrees C) - Linux: power via hwmon power1_average / power1_input (microwatts) All paths are no-op fallbacks (None) when the source is unavailable. Mirrors what nvidia-smi provides on the CUDA path. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: remove ADL ctypes — does not support AMD iGPU (Strix Halo) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: Erland366 <erland.pg366@gmail.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com>	2026-05-29 22:29:56 -07:00
Daniel Han	ca55acbb5f	Studio: unblock install on Linux ARM64 + Windows ARM64 + Intel Mac (#5790) ... * Studio: unblock cross-platform install on Linux ARM64 + Windows ARM64 Three independent bugs that together prevent `install.sh` / `install.ps1` from completing on the ARM machines GitHub Actions now ships (`ubuntu-24.04-arm`, `windows-11-arm`) and on equivalent real hosts (Ampere Altra, Raspberry Pi 5, Snapdragon X Elite, ...). Validated on the staging-2 cross-OS smoke suite -- five per-OS workflows pinned to `ubuntu-latest`, `ubuntu-24.04-arm`, `macos-14`, `macos-15-intel`, `windows-11-arm`. Before this change Windows ARM exits 1 in the winget gate and Linux ARM source-builds llama.cpp because the prebuilt selector returns 0 attempts; with it both reach healthy /api/health. 1. studio/install_llama_prebuilt.py -- resolve_simple_install_release_plans had explicit branches for windows+x86_64, macos+arm64, macos+x86_64 and linux+x86_64 only. Upstream ggml-org/llama.cpp ships `llama-bNNNN-bin-ubuntu-arm64.tar.gz` and `llama-bNNNN-bin-win-cpu-arm64.zip` (visible in the b9334 release manifest), so the missing elif branches force every Linux ARM64 and Windows ARM64 host into a source build even when a perfectly good upstream prebuilt is one HTTP GET away. Two new branches mirror the existing CPU variants; runtime_patterns_for_choice and runtime_payload_health_groups gain `linux-arm64` (.so layout) and `windows-arm64` (.dll layout) so the health-check pass-through matches the asset shape. 2. studio/setup.sh -- the helper-release-repo selector routed any non-x86_64 Linux to `unslothai/llama.cpp`, which only publishes the Linux CUDA bundle set. The result on Linux ARM64 was a guaranteed `direct_linux_release_plan` raise of "no compatible Linux prebuilt asset was found" on every release in the scan, then a source-build fallback. Pin Linux ARM64 (CPU-only) to `ggml-org/llama.cpp` so the new branch in (1) can see the upstream asset. setup.ps1 already hardcodes `ggml-org/llama.cpp`, so Windows ARM64 picks up (1) without an additional change. 3. install.ps1 -- the winget pre-check hard-failed before Python or uv detection. `windows-11-arm` runners (and many corporate Windows hosts without the Microsoft Store) ship without winget but already have a usable Python plus the Astral uv PowerShell installer reachable. Demote the winget check to a soft warning, defer the hard failure to the Python install branch (which is the only path that genuinely needs winget), and let the uv install fall through to `https://astral.sh/uv/install.ps1` when winget is absent. The uv PowerShell installer was already the existing fallback for the "winget present but uv install failed" case; this just makes it the primary path on hosts without winget. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: filter torchcodec on platforms without wheels torchcodec 0.10.0 ships wheels for manylinux_2_28_x86_64, macosx_12_0_arm64, and win_amd64 only -- visible on its PyPI page and in the resolver error reported by #4446. install_python_stack.py pulls torchcodec via extras-no-deps.txt, which is now installed unconditionally during `unsloth studio update --local` (the update command has no --no-torch flag). Result on Linux aarch64 / Windows ARM64 / Intel Mac (when invoked outside the install.sh auto-skip-torch path): ERROR: Could not find a version that satisfies the requirement torchcodec==0.10.0 (from versions: 0.0.0.dev0, ...) ERROR: No matching distribution found for torchcodec==0.10.0 error Installing extras (no-deps) (pip) failed (exit code 1) `NO_TORCH_SKIP_PACKAGES` already lists torchcodec but only fires when NO_TORCH is true -- the update path inherits no NO_TORCH from the original install and inferrence falls back to IS_MAC_INTEL only, so Linux aarch64 / Windows ARM64 sail past the guard. Adds a platform predicate PLATFORM_LACKS_TORCHCODEC_WHEEL and applies the torchcodec filter unconditionally there, independent of NO_TORCH. Surfaced by the staging-2 cross-OS smoke `unsloth studio update` step on ubuntu-24.04-arm; verified the same step is green with this patch overlaid. * Studio: skip librosa on no-torch hosts (unblocks Intel Mac install) Closes the last cross-platform install gap surfaced by the staging-2 cross-OS smoke (see unslothai/unsloth#5046 for the original report): `install.sh --local` on macos-15-intel fails at × Failed to build `llvmlite==0.47.0` error: failed-wheel-build-for-install ╰─> llvmlite error studio setup failed (exit code 1) Root cause: upstream llvmlite dropped the macosx_x86_64 wheel between 0.42.0 and 0.46.0 (https://pypi.org/project/llvmlite/0.47.0/#files -- only macosx_arm64 / manylinux / win_amd64 remain). pip falls back to a from-source build of llvmlite's FFI, which needs LLVM 14/15 dev headers and matching llvm-config -- not present in Xcode Command Line Tools' libclang and not installed by install.sh's MAC_INTEL deps branch. llvmlite enters Studio's tree via librosa -> numba -> llvmlite in extras.txt. openai-whisper (extras.txt:28) would also pull numba but is already filtered on no-torch hosts. Adding librosa to the same NO_TORCH_SKIP_PACKAGES set makes the install go through cleanly on Intel Mac (auto-detected NO_TORCH=true via the MAC_INTEL branch) and on any user-passed --no-torch host where torch-dependent audio pipelines would not run anyway. Tracked / verified on the danielhanchen/unsloth-staging-2#154 smoke matrix (macos-15-intel). * Studio UI tests: retry evaluate_fetch on transport-level failure (PR #5790) Mac Studio UI CI on this PR (run 26496820814, job 78026959359) failed with /api/models/list status=0 error='TypeError: Failed to fetch'. The artifact studio.log shows the server answered the two preceding /api/models/list calls from the React mount (both 200) but never received the third call from the test script: the browser reused a kept-alive HTTP/1.1 socket that uvicorn (5s keep_alive_timeout) had closed ~130ms earlier. Chromium under --single-process on macos-14 free runners is most prone to this; the post /api/auth/change-password session churn accelerates it. A rerun on the same SHA passed, which is the classic flake signature. evaluate_fetch in tests/studio/_playwright_robust.py already returns a structured {status: 0, body: None, error: "..."} on JS-side throws, but every caller treats status=0 as fatal. Add a bounded retry inside the helper so the one class of failure recovers transparently: status != 0 -> real HTTP response (incl. 4xx/5xx); propagate. error has "AbortError" -> caller's AbortSignal deadline; propagate. else (status==0) -> stale-keepalive or other transport failure; retry after 250ms / 500ms backoff so the pool evicts the dead socket before the next attempt. Defaults transport_retries=2, transport_backoff_ms=250 (max added latency on the happy path is zero; on a transport failure: up to 750ms of sleep). Callers keep the existing {status, body, error} shape; no call-site changes needed. Verified: tests/studio/_playwright_robust.py compiles; signature gains two kwonly args (transport_retries, transport_backoff_ms); 8 evaluate_fetch call sites in playwright_chat_ui.py + playwright_extra_ui.py pick up the retry without change. --------- Co-authored-by: danielhanchen <info@unsloth.ai> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-05-27 04:53:38 -07:00
Daniel Han	849da89605	Fix unsloth studio update silently downgrading on macOS arm64 (#5767) ... * Fix unsloth studio update silently downgrading on macOS arm64 Root cause: studio/install_python_stack.py's "Updating base packages" step passes `--upgrade-package unsloth -r base.txt -c constraints.txt` with base.txt's `unsloth` and `unsloth-zoo` entries unpinned. On macOS arm64 the resolver silently backtracks to an older unsloth (2026.5.2 or even 2025.7.2) whenever a transitive constraint (the most common one is bitsandbytes wheel availability: 0.49.0+ ships macosx_14_0_arm64 wheels, older versions do not) makes the unpinned requirement satisfiable by an older release. install.sh already maintains an explicit `unsloth>=N.N.N` floor for the same reason, but the floor was missing from the in-venv update path. Reproduced on macos-14 across 2026.3.18 / 2026.4.8 / 2026.5.2 / 2026.5.6 starting states. All four ended on unsloth==2026.5.2 after a clean `unsloth studio update` invocation (2026.5.6 was a true downgrade, others were stale or partial advances). Fix mirrors install.sh: query PyPI at runtime for the current latest version of unsloth and unsloth-zoo, then pass `unsloth>=<latest>` and `unsloth-zoo>=<latest>` as extra positional pins alongside the existing `--upgrade-package` flags. Network failures fall back to the historical unpinned behaviour so offline installs continue to work. Applied to all three upgrade branches (standard update, local-repo overlay, no-torch). Also fix the cosmetic `Hardware detected: MLX -- Apple Silicon (i386)` banner. platform.processor() reads `uname -p` which returns "i386" on many universal2-shaped Python builds even on a native arm64 interpreter; platform.machine() is the reliable source ("arm64" once is_apple_silicon has gated us). * Dedup floor-pin call sites + LRU cache PyPI lookup Three upgrade branches each rebuilt the same conditional `unsloth>=` / `unsloth-zoo>=` arg list with two PyPI round-trips per branch -- six round-trips per `unsloth studio update` invocation. Extract a `_pin_floor_args(*, include_unsloth=True)` helper and wrap `_resolve_latest_pypi_version` in `functools.lru_cache` so the three branches share a single PyPI request per package. Functionally equivalent; pure cleanup on top of the previous commit. * Warn when PyPI is unreachable so the silent fallback is visible If `_resolve_latest_pypi_version` returns None for either lookup the floor args are silently dropped, which restores the pre-fix resolver behaviour. Print a single cyan `warning` line in `_pin_floor_args` when that happens so users behind a proxy / captive portal / firewalled PyPI mirror know the upgrade has degraded -- and can supply network egress or a `--index-url` mirror and retry. * Soft floor with unpinned-fallback for hosts where floor is unsatisfiable Reviewer found that the unconditional unsloth-zoo>=LATEST floor turns a previously-resolvable macOS 13 arm64 update into a hard resolver failure: unsloth-zoo 2026.5.4 requires mlx-vlm>=0.4.4 -> mlx>=0.30.0, and mlx 0.30+ only publishes macosx_14_0_arm64 wheels. The pre-fix behaviour backtracked to an older unsloth instead of erroring. We should not turn "stale" into "fail". Add pip_install_with_floor_fallback: first try the install with the floor appended; if the resolver cannot satisfy it (subprocess exit code != 0), retry the install without the floor and print a clear warning. The fall-through preserves the legacy "succeed-but-stale" contract on hosts where wheel availability is the bottleneck. Also extend pip_install_try with a req= kwarg so the floor attempt can pass `-r base.txt` like pip_install does, and add an UNSLOTH_NO_PYPI_FLOOR=1 opt-out for air-gapped CI / corporate PyPI mirrors that intentionally do not expose pypi.org directly. All three upgrade branches (standard, local-repo, no-torch) now go through the helper so the fallback behaviour is consistent. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add second fallback level: floor without constraints macOS arm64 floored attempt with -c constraints.txt fails because the single-env constraint `transformers==4.57.6` conflicts with the new unsloth-zoo 2026.5.4 -> mlx-vlm 0.4.4+ -> transformers>=5.1.0 chain. First fallback level retries the floored install without constraints (transformers freely resolves to a mlx-vlm-compatible version); downstream pip_install calls still apply constraints.txt to anything that doesn't transitively conflict. If THAT still fails (wheel availability rather than constraint conflict), drop the floor and fall back unpinned as before. Verified locally with uv pip compile against aarch64-apple-darwin python-3.13: strict-constrained floor errors, no-constraint floor resolves cleanly to unsloth==2026.5.7 + unsloth-zoo==2026.5.4 + transformers==5.5.0 + mlx-vlm==0.5.0. * setup.sh/.ps1: also gate fast-path on unsloth-zoo being up to date The version-check fast-path in setup.sh / setup.ps1 only looked at unsloth itself. If unsloth was at the PyPI latest but unsloth-zoo was stale, the gate set _SKIP_PYTHON_DEPS=true and install_python_stack.py never ran -- so the new floor pin from PR #5767 had no effect for the exact "unsloth at latest, zoo behind" state several reviewers flagged. Probe both packages' installed-vs-latest versions and only skip the deps step when BOTH match. When either is behind, fall through to install_python_stack.py so the new resolver fix gets a chance to run. Verified setup.sh with `bash -n`; the setup.ps1 change uses PowerShell if-expressions for the null-default pattern rather than bash-style ${var:-default} which is not valid PowerShell. * Skip unsloth-zoo floor too for custom no-torch test packages Reviewer found the asymmetric guard: the no-torch branch was already gating the unsloth floor on package_name == "unsloth" (test side packages may not publish to PyPI), but the unsloth-zoo floor was still added unconditionally. A custom no-torch update that ships its own forked zoo metadata could now hit a public PyPI floor that does not match the fork's published version. Add a symmetric `include_zoo` parameter to `_pin_floor_args` and gate both pins on the same `package_name == "unsloth"` check. * Address review feedback: simpler except clause + private-index note Gemini flagged TimeoutError in the PyPI fetch exception list. OSError already covers socket timeouts and the 3.11+ TimeoutError subclass on every supported Python, so drop the redundant entry and explain what each remaining exception catches. Codex flagged that floor lookups against pypi.org could break installs behind a lagging private mirror. Step 3 of pip_install_with_floor_fallback already recovers transparently in that case; expand the docstring so the behavior is discoverable without reading the body. * extras-no-deps: skip transformers==4.57.6 on macOS arm64 Reviewer flagged that the resolver-selected transformers from the no-constraints base step on macOS arm64 (transformers 5.x for mlx-vlm 0.4.4+) gets silently downgraded back to 4.57.6 by extras-no-deps.txt during the very next step, breaking mlx-vlm imports at runtime even though unsloth itself reports as latest. Add a PEP 508 platform marker so the pin only applies off macOS arm64. constraints.txt still enforces 4.57.6 everywhere else; mlx-vlm only publishes wheels for darwin arm64, so other platforms are unaffected. * setup.sh/.ps1: gate fast-path zoo probe on _PKG_NAME == unsloth Reviewer found the asymmetric custom-package regression: the new zoo-aware fast-path probes public unsloth-zoo unconditionally, but a custom STUDIO_PACKAGE_NAME side build may ship its own zoo fork via dependency metadata and not install public unsloth-zoo at all. The previous behaviour (skip Python deps if the custom package itself is at its declared latest) is preserved by only running the zoo probe when the managed package literally IS unsloth. Matches the include_zoo gate already in _pin_floor_args() at install_python_stack.py. * install_python_stack: all-or-nothing floor + uv-to-pip retry Two reviewer findings on the floor-pin helpers: 1. _pin_floor_args() previously kept a half-floor if one PyPI lookup succeeded and the other failed. With unsloth at latest but the zoo lookup down, the resolver could still backtrack zoo while we required unsloth at latest, defeating the pin. Return [] on any lookup failure so the unpinned legacy path runs cleanly. 2. pip_install_try() ran ONLY uv when USE_UV was true; a uv-specific failure short-circuited to False even when pip itself could have applied the floor. Mirror pip_install()'s uv-to-pip fallback: try uv, fall through to pip on non-zero exit, and only then give up. * extras-no-deps: rewrite marker without `not` for PEP 508 parsers pip's vendored packaging rejects `not (...)` in PEP 508 markers; the grammar only specifies `and` / `or` between boolean atoms. The staging macos-14 matrix failed every job at "Installing extras (no-deps)" with `Expected a marker variable or quoted string`. Apply De Morgan's law so the marker uses `or` between two `!=` checks, which both pip and uv parse cleanly. Behaviour identical: skip the 4.57.6 pin only on darwin arm64; pin everywhere else. * constraints: skip transformers==4.57.6 pin on macOS arm64 too Marker-gating the extras-no-deps.txt pin was not sufficient. Every subsequent pip_install in the update pipeline passes -c single-env/constraints.txt, and constraints.txt itself pinned transformers==4.57.6 unconditionally. The latest staging-2 run shows the base step's no-constraints fallback installed transformers 5.5.0 correctly, but a later constrained step (extras / studio / data-designer deps) silently downgraded it back to 4.57.6, leaving mlx-vlm 0.5.0 in the venv with an unsatisfied transformers>=5.5.0 requirement. Apply the same `sys_platform != "darwin" or platform_machine != "arm64"` marker to the constraints.txt entry so it is inert on darwin arm64. Other platforms still pin 4.57.6 because mlx-vlm only publishes wheels for darwin arm64; no other platform is affected. * constraints: carve out darwin arm64 from every == pin Marker-gating only transformers was not enough; staging-2 still failed with the same `transformers==4.57.6 in venv after the update` outcome because the resolver hit a `huggingface-hub==0.36.2` (and adjacent) conflict with mlx-vlm's `huggingface-hub>=1.5.0` requirement, then fell back to a stale stack even after my no-constraints level fired on the base step. Apply the same `sys_platform != "darwin" or platform_machine != "arm64"` marker to every == pin in constraints.txt. Range pins (mcp, fastmcp, websockets) stay active everywhere because they do not conflict with the mlx-vlm chain. mlx-vlm only publishes wheels for darwin arm64, so no other platform is affected. * install_python_stack: also --upgrade-package transformers and mlx-vlm Staging-2 showed that even after the constraints.txt carve-out for darwin arm64, the venv still ended up with the OLD `transformers==4.57.6` paired with a NEW `mlx-vlm==0.5.0` from unsloth-zoo's transitive upgrade. The resolver's --upgrade-package flag only freshens the named packages and their newly-pulled transitive deps; transformers was already installed at a version that satisfied unsloth-zoo's range (`>=4.51.3,<=5.5.0` with exclusions), so the resolver did not upgrade it -- even though mlx-vlm 0.5.0 requires `transformers>=5.5.0`. Add `--upgrade-package transformers` and `--upgrade-package mlx-vlm` to all three base-step branches. Both are no-ops when the package is absent (mlx-vlm only ships wheels on darwin arm64); on darwin arm64 this is what nudges the resolver to upgrade both together so the final venv is internally consistent. On Linux/Windows, transformers stays at 4.57.6 because constraints.txt still pins it there and mlx-vlm never enters the resolution. * install_python_stack: explicit mlx-vlm + transformers realign on macOS arm64 Even with --upgrade-package hints, uv leaves the venv with the already-installed transformers (4.57.6 inherited from the OLD venv's constrained install) when that version still happens to satisfy unsloth's own metadata range -- but it does not also re-resolve mlx-vlm's stricter `transformers>=5.5.0` requirement, so the venv ends up with mlx-vlm 0.5.0 paired with transformers 4.57.6 and mlx-vlm imports break at runtime. After the base step, on darwin arm64 only, run an explicit `pip install --upgrade mlx-vlm transformers` with constrain=False. This forces both packages through the resolver again as direct top-level requirements, so transformers is pulled up to whatever mlx-vlm's metadata requires (5.5.0 today). No effect on any other platform because mlx-vlm has no wheels off darwin arm64 and the branch is gated on IS_MAC_ARM. * requirements: marker-gate every == pin that conflicts with mlx-vlm chain Staging-2 kept ending up with transformers==4.57.6 even after the realign step, because studio.txt unconditionally pins huggingface-hub==0.36.2 (and datasets==4.3.0). Installing studio.txt with constraints active pulls the resolver back to a huggingface-hub that only recent transformers (4.x) supports, which silently downgrades the realigned 5.5.0 to 4.57.6 -- exactly the inconsistency we tried to prevent. Also extras-no-deps.txt still pinned trl==0.23.1 unconditionally; the 0.23.1 wheel transitively requires huggingface-hub<1, same coupling. Marker-gate all three. The carve-out is identical to constraints.txt's: inactive on darwin arm64 (where the mlx-vlm chain dictates newer versions), active everywhere else (where Linux/Windows users rely on the single-env pins). mlx-vlm only publishes wheels for darwin arm64 so no other platform is affected. * realign: --force-reinstall mlx-vlm + transformers + huggingface_hub Plain --upgrade does not force uv to re-resolve mlx-vlm's transformers requirement when the already-installed transformers happens to satisfy unsloth's own range. Switch to --force-reinstall on the three packages so the resolver tears them down and brings them back together with consistent versions. Include huggingface_hub because transformers 5.x requires hf-hub>=1.5.0 and the resolver would not touch it otherwise. * realign: pin transformers via mlx-vlm's own metadata spec `pip install --force-reinstall mlx-vlm transformers` still resolved to an already-installed transformers 4.57.6 because uv treats it as satisfying unsloth's transformers range without re-checking mlx-vlm's stricter requirement. Pull mlx-vlm's actual transformers specifier from its installed metadata at runtime and pass it as an explicit version requirement (e.g. `transformers>=5.5.0` for mlx-vlm 0.5.0). That removes the resolver's wiggle room: it MUST pick a transformers satisfying mlx-vlm AND unsloth, which on darwin arm64 with the latest unsloth-zoo means transformers==5.5.0. Falls back to unpinned `transformers` if metadata read fails, so this never errors. * realign: uninstall-then-install to bypass uv's incumbent bias Every flag-based approach failed: --upgrade, --upgrade-package, --force-reinstall, and even an explicit `transformers>=5.5.0` requirement all left the venv with transformers==4.57.6 because uv treats the already-installed version as satisfying unsloth-zoo's range and refuses to disturb it, even when it does not satisfy mlx-vlm's stricter requirement. Replace the realign step with an explicit uninstall of the conflicting trio (transformers / mlx-vlm / huggingface_hub) followed by a fresh install. With no transformers in the venv, the resolver MUST pick a version satisfying every installed package's metadata, which on darwin arm64 with the latest unsloth-zoo is uniquely 5.5.0. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim verbose comments across PR #5767 changes * Simplify mac-arm64 fix: install MLX stack with --no-deps The previous approach (PyPI floor pin + 3-level fallback + macOS arm64 realign step + marker carve-outs on every == pin) was fighting symptoms. The root cause is that unsloth-zoo declares mlx-vlm>=0.4.4 as a darwin arm64 dep, and mlx-vlm 0.5.0's metadata pulls in transformers>=5.5.0, which conflicts with the main venv's transformers==4.57.6 pin and forces the resolver to backtrack unsloth. Severing that chain at its source: install mlx + mlx-metal + mlx-lm + mlx-vlm with --no-deps BEFORE unsloth-zoo. The resolver sees mlx-vlm already installed (>=0.4.4) and never inspects its transformers metadata. Per-model transformers version routing is already handled at runtime by the side-car venvs in utils/transformers_version.py (.venv_t5_530 for Ministral/GLM/Qwen3 MoE, .venv_t5_550 for Gemma 4). Net change: -224 / +71 lines across install.sh, install_python_stack.py and the three requirements files. Reverted: - _resolve_latest_pypi_version + _pin_floor_args + pip_install_with_floor_fallback - macOS arm64 realign step (pip uninstall + reinstall) - --upgrade-package transformers --upgrade-package mlx-vlm in base steps - All ; sys_platform != "darwin" or platform_machine != "arm64" markers in constraints.txt, studio.txt, extras-no-deps.txt - pip_install_try restored to its pre-PR signature Added: - install.sh: Apple Silicon MLX --no-deps install before unsloth (both fresh and migrated branches) - install_python_stack.py: same step gated on IS_MAC_ARM and not skip_base Kept (independent bugs): - setup.sh / setup.ps1 dual-package zoo version check - platform.processor() -> platform.machine() hardware-detect fix * Minimise PR to mac-arm64-specific changes only Revert setup.sh and setup.ps1 to main -- the dual-package zoo check was defensive and not strictly needed once mlx-vlm is installed --no-deps (the resolver-backtrack scenario that produced stale zoo no longer happens). Tighten remaining comments in install.sh and install_python_stack.py. Final PR-attributable changes: install.sh +24/-5 (MLX --no-deps in 2 places) studio/install_python_stack.py +19 (MLX --no-deps + IS_MAC_ARM) studio/backend/utils/hardware/hardware.py +6/-6 (processor() -> machine()) studio/backend/requirements/*.txt unchanged * Revert "Minimise PR to mac-arm64-specific changes only" This reverts commit 9470daa855. * Revert "Simplify mac-arm64 fix: install MLX stack with --no-deps" This reverts commit f8a43b87e8. * Revert "Trim verbose comments across PR #5767 changes" This reverts commit c3f293a10f. * Simplify mac-arm64 fix: --no-deps MLX + METADATA patch Root cause: unsloth-zoo declares mlx-vlm>=0.4.4 as a darwin-arm64 dep, and mlx-vlm 0.5.0's published metadata declares transformers>=5.5.0. Every subsequent resolver run with constraints.txt's transformers==4.57.6 sees the conflict and backtracks unsloth to escape it (user-reported downgrade). The aggressive pin doesn't reflect what mlx-vlm actually requires at top-level import time -- the symbols it loads (AutoProcessor, AutoTokenizer, ProcessorMixin, BatchFeature) are stable across transformers 4.51+. Model- specific submodules that genuinely need 5.x APIs are only loaded once the 3-tier transformers dispatcher (utils/transformers_version.py) has activated the matching .venv_t5_530 / .venv_t5_550 side-car at runtime. Fix: on Apple Silicon, install the MLX stack with --no-deps then rewrite mlx-vlm/mlx-lm's installed METADATA to declare transformers>=4.51.3. Now the resolver sees mlx-vlm 0.5.0 as compatible with the main venv's transformers==4.57.6 and there's nothing to backtrack. Reverts the previous heavy machinery: - _resolve_latest_pypi_version, _pin_floor_args, pip_install_with_floor_fallback - macOS arm64 realign step (pip uninstall + reinstall) - --upgrade-package transformers --upgrade-package mlx-vlm in base steps - All ; sys_platform != "darwin" or platform_machine != "arm64" markers in constraints.txt / studio.txt / extras-no-deps.txt - setup.sh / setup.ps1 dual-package zoo check (Windows never had the bug; with this fix in place stale zoo no longer happens on macOS either) - pip_install_try restored to pre-PR signature Kept: - install.sh: MLX --no-deps install in fresh + migrated branches - install_python_stack.py: same step gated on IS_MAC_ARM and not skip_base - _relax_mlx_metadata() helper, called immediately after each MLX install - studio/backend/utils/hardware/hardware.py: platform.processor() -> platform.machine() cosmetic fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use UV_OVERRIDE to relax mlx-vlm transformers pin uv supports --overrides / UV_OVERRIDE which globally overrides any package's stated dependency requirement. mlx-vlm 0.5.0 declares transformers>=5.5.0 and mlx-lm 0.31.3 declares transformers>=5.0.0; neither is true at top-level import time (their imports use AutoProcessor / AutoTokenizer / ProcessorMixin / BatchFeature which are stable across transformers 4.51+). Per-model 5.x routing is handled at runtime via the .venv_t5_530 / .venv_t5_550 side-cars. Override file (overrides-darwin-arm64.txt) declares transformers>=4.51.3 ; exported via UV_OVERRIDE env var on Apple Silicon by both install.sh and install_python_stack.py. uv then resolves mlx-vlm as compatible with the main venv's transformers==4.57.6 (constraints.txt) and unsloth advances cleanly to LATEST. Drops, vs. the previous attempts: - _resolve_latest_pypi_version + _pin_floor_args + pip_install_with_floor_fallback (floor-pin machinery -- replaced by single UV_OVERRIDE line) - macOS arm64 realign step (pip uninstall + reinstall) - --upgrade-package transformers --upgrade-package mlx-vlm in base steps - All ; sys_platform != "darwin" or platform_machine != "arm64" markers - _relax_mlx_metadata() helper + sed METADATA patch (uv reads from index, not dist-info, so dist-info patches were ineffective) Kept: - install.sh / install_python_stack.py: MLX latest install on Apple Silicon (now without --no-deps, the override lets the resolver pick a consistent set) - studio/backend/utils/hardware/hardware.py: platform.machine() cosmetic fix * Trim UV_OVERRIDE comments; bump override floor to 4.57.6 Match the main venv's constraints.txt pin exactly so the override file reads as the actual installed version rather than mlx-vlm's API floor. Comments collapsed to one-liners where possible. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-05-26 07:23:13 -07:00
Daniel Han	dfb3eedf77	ci: broaden Linux + narrow Windows llama.cpp runtime patterns + trim #5741 comments (#5746) ... * ci: broaden Linux llama.cpp runtime pattern to lib*.so* #5741 patched the explicit Linux pattern list to add ``libllama-*-impl.so*`` after ggml-org/llama.cpp#23462 (between b9279 and b9283) split each binary's entry code into a paired ``lib<binary>-impl.so`` shared library. Same class of upstream repackaging will hit us again whenever a new shared lib is added. Mirror what macOS already does and replace the per-lib list with a single ``lib*.so*`` glob. ``copy_globs`` (line 3614) unions patterns, so the per-variant ``libggml-cuda.so*`` / ``libggml-hip.so*`` entries were never filtering anything; the spec lives in ``runtime_payload_health_groups`` (line 5209) which keeps the explicit minimum-required list per variant. Dry-run against b9296-bin-ubuntu-x64.tar.gz: 40 files copied (all ggml, llama, mtmd, impl variants + the two binaries we ship), 22 skipped (other CLIs, rpc-server, LICENSE). Functionally equal to the post-#5741 set. * cleanup: trim #5741 comments on the pydantic split Comments added in #5741 explained the original bug in full each time. They are mostly redundant with the commit message and the PR. Trim them to one short paragraph per site. No behavior change. * ci: narrow Windows runtime pattern to llama-server.exe + llama-quantize.exe Studio only invokes llama-server and llama-quantize. Mac and Linux already filter to those two binaries; Windows was the odd one out with ``*.exe`` copying every CLI upstream ships (llama-cli, llama-bench, llama-mtmd-cli, ...). Dry-run on b9296 (win cpu-x64, cpu-arm64, cuda-13.1, hip-radeon): 20 unused EXEs skipped per variant, all DLLs (incl. the new llama-*-impl.dll family) still copied via ``*.dll``. ``existing_install_matches_choice`` already checks llama-server.exe exists explicitly (line 5297), so the health gate is unchanged.	2026-05-23 21:48:12 -07:00
Daniel Han	83b20976f7	ci: unblock Studio Windows + Linux + Mac smoke (#5741) ... Bundles three independent CI regressions hitting the maintainer PR backlog. Each one is verified end-to-end on a staging fork against real Ubuntu / macOS / Windows GitHub-hosted runners before this lands. 1. Windows --no-torch install: pydantic + pydantic-core drift to incompatible versions under `uv pip install --no-deps -r no-torch-runtime.txt` because pip resolves each independently from latest. pydantic.VERSION 2.13.4 pins pydantic-core==2.46.4 but pydantic-core 2.47.0 was the freshest published wheel, so `import pydantic` raised `SystemError: pydantic-core 2.47.0 is incompatible with the current pydantic version`. Resolve pydantic WITH deps in a focused pip call (install.sh, install.ps1, install_python_stack.py) before the --no-deps no-torch-runtime pass so pip pins pydantic-core to the version pydantic declares. pydantic's transitive deps (annotated-types, pydantic-core, typing-extensions, typing-inspection) are torch-free. Drop the redundant `Patch Studio venv with full typer / pydantic dep trees` workaround from the four Windows smoke YAMLs. Supersedes #5733 + #5734. 2. Linux Studio Update CI: upstream llama.cpp b9261+ split each binary's entry code into a paired `libllama-<binary>-impl.so` shared library. `llama-server` and `llama-quantize` NEEDED-link against `libllama-server-impl.so` / `libllama-quantize-impl.so` with RUNPATH `$ORIGIN`, so the prebuilt overlay must copy those alongside the binaries. Without that, ldd reports them missing, preflight rejects, the installer falls back to source build, and studio-update-smoke annotates `setup.sh idempotency regressed`. Add `libllama-*-impl.so*` to the Linux runtime patterns and lock the pattern in test_rocm_support.TestRuntimePatterns. 3. Mac Studio UI Chat: change-password submit clicked while disabled. The disable gate only checked new + confirm password length, but Playwright's first click landed before the current-password field's React state had committed, so the form was simultaneously logically-invalid (current_password empty) and the button was disabled. Tighten the gate to require `currentPassword.length >= 8` and mirror the same check in the submit handler so Enter / autofill cannot bypass. Supersedes #5738.	2026-05-23 06:59:16 -07:00
Roland Tannous	79adfd9c71	studio: skip flash-attn install on Blackwell GPUs (sm_100+) (#5420) ... * studio: skip flash-attn install on Blackwell GPUs (sm_100+) Dao-AILab does not publish prebuilt flash-attn wheels for sm_100, sm_120, or sm_121, and the older-arch wheels fail to load on Blackwell. Add a shared has_blackwell_gpu() helper and gate both the install-time (install_python_stack._ensure_flash_attn) and runtime (worker._ensure_flash_attn_for_long_context) paths on it. Detection uses nvidia-smi --query-gpu=compute_cap, which works on Linux and Windows. * test: stub has_blackwell_gpu in pre-existing runtime flash-attn tests prefers_prebuilt_wheel and falls_back_to_pypi exercise the install paths that the Blackwell guard now short-circuits. Make them explicit about non-Blackwell so they pass on real Blackwell hosts. * studio: cache has_blackwell_gpu, skip Blackwell warning under NO_TORCH - Wrap has_blackwell_gpu in functools.lru_cache so repeated calls in a single process avoid redundant nvidia-smi spawns. Tests clear the cache via setup_method/teardown_method. - In _ensure_flash_attn, run the NO_TORCH short-circuit before the Blackwell check so GGUF-only users (who never install torch anyway) do not see a Blackwell warning. Blackwell check still runs above the IS_WINDOWS / IS_MACOS gates so Blackwell-on-Windows users still see the explicit reason rather than a silent OS skip. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: add has_blackwell_gpu to mlx worker test wheel_utils stub test_mlx_training_worker_config loads worker.py against a hand-rolled utils.wheel_utils stub. Adding has_blackwell_gpu to the stub symbol list so worker's import line resolves. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-05-14 18:13:50 +04:00
Wasim Yousef Said	e35cbfb454	Add native GGUF intake to Studio (#5246) ... * feat(studio): add Tauri native GGUF intake * feat(studio): polish native GGUF intake * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): load backend helpers during local setup * fix(studio): acquire native load lease before unload * Studio: harden native path lease verification and Tauri intake - Wrap path.resolve(strict=True) and Path.stat() in NativePathLeaseError so a deleted or unmounted GGUF returns 400 instead of leaking the full filesystem path through the generic load_model/validate_model handler. - Re-apply _reject_network_or_device_path to the resolved canonical path for defense in depth after symlink resolution. - Replace try/except ValueError pattern in the device-path guard with Path.is_relative_to; the previous shape silently swallowed NativePathLeaseError (which subclasses ValueError) so /dev,/proc,/sys were never actually rejected. - Broaden the lease redaction regex and dict-key check (Python and Rust diagnostics) to cover both native_path_lease and nativePathLease so the camelCase form emitted by Tauri/frontend payloads is also redacted. - Hoist the redact_native_paths import to module top in loggers/handlers; the recursive filter no longer pays a per-record import lookup. - Persist activeNativePathToken in the chat runtime store so the rollback branch can mint a fresh lease and reload the previous native GGUF when a new load fails after unload; clear it in clearCheckpoint and overwrite it on each successful load. - use-native-drop: read options through a ref so the Tauri onDragDropEvent listener is registered once and stays attached across option changes; reject ambiguous multi-file drops up front instead of silently registering only the first GGUF. - pick_native_model: use an async pick_file with a tokio oneshot channel instead of blocking_pick_file so the Tokio worker is not held for the duration of the OS dialog. - registerNativeModelPath: drop the duplicate sourceKind argument; the Rust command parameter is source_kind. - install_python_stack: insert the script directory (studio/) on sys.path; the previous insert pointed at studio/backend/ which does not satisfy `from backend.utils.wheel_utils import ...`. * install_python_stack: keep _BACKEND_DIR on sys.path Restore the studio/backend insertion. Although the immediately following `from backend.utils.wheel_utils import (...)` is satisfied by studio/ already being on sys.path[0] when invoked as `python studio/install_python_stack.py`, wheel_utils itself runs `from utils.native_path_leases import ...`, which requires studio/backend/ to be importable. Without the backend insertion, the existing tests/python/test_install_python_stack.py collection fails with ModuleNotFoundError: No module named 'utils'. * Studio: tighten native path lease lifecycle and Tauri intake IPC - register_native_model_path now hardcodes NativePathSourceKind::Drop on the Rust side and the frontend stops sending source_kind. The previous JS payload (source_kind only) never reached the Rust deserializer because Tauri's default ArgumentCase::Camel maps the Rust parameter source_kind to the JS key sourceKind, so drag/drop registration silently failed. Hardcoding the source kind also keeps audit metadata trustworthy on this command. - Add native_path_secret_removed_for_child_start context manager and wrap multiprocessing.Process.start() at the inference, export, training, and data-recipe job spawn sites. The previous wrapper-only scrub left UNSLOTH_STUDIO_NATIVE_PATH_LEASE_SECRET visible to spawn-platform import-time worker code. The wrapper run_without_native_path_secret stays as defense-in-depth inside the child. - Stop passing exc_info=True from the native-grant load/validate error logs in routes/inference.py. The structlog filter_sensitive_data processor runs before the renderer, so ConsoleRenderer formatted tracebacks bypassed redaction; the redacted str(e) preserves the message text. - Replace the os.path.normcase string equality on the resolved canonical path with Path.samefile (with a normcase fallback) so Windows leases that differ only in extended-length \\?\ prefix or short-name spelling are accepted. - Wrap consumeNativePathToken in its own try/catch in the chat runtime rollback. If the previous native-model token has aged out of TOKEN_TTL we now surface a clear modelsError instead of silently swallowing the rollback inside the outer catch. - Reject non-ASCII lease strings in _split_lease and convert UnicodeEncodeError / binascii.Error / ValueError raised by _b64decode into NativePathLeaseError so verify_native_path_lease never escapes raw exceptions to the route handler. - Tighten dropStateForPaths to mark multi-file payloads invalid so the overlay matches the post-fix drop handler that rejects the same payload. - Replace the one-shot fetch in useNativePathLeasesSupported with a delayed-retry loop so the picker/drop becomes available once the backend is up rather than staying disabled for the rest of the session after a transient failure. - Drop the unused setActiveNativePathToken setter; the value is set via setState directly in use-chat-model-runtime. - Add a toast on auto-load failure in use-native-drop so a collapsed model selector does not hide the error. - Burn the lease nonce before _validate_current_stat so a stat-failed lease is single-use even if a later state change happens to match the original size/mtime. * Studio: cache lease secret, harden native path stat checks, polish intake UX - Cache the decoded UNSLOTH_STUDIO_NATIVE_PATH_LEASE_SECRET on first verify and validate that it is base64-decodable and at least 32 bytes. Subsequent _decode_secret calls return from the cache and never touch os.environ, so concurrent /api/inference/load and /api/health requests no longer race with native_path_secret_removed_for_child_start scrubbing the env. native_path_leases_supported now wraps _decode_secret so the health flag matches what verify_native_path_lease actually accepts. - Replace path.is_file()/is_dir() + path.stat() with os.lstat() in _validate_current_stat and explicitly reject S_ISLNK; size and mtime checks now refer to the link itself, closing the same-size+same-mtime symlink-swap window that the prior follow-symlink stat() left open. - Add an issued_at_ms < expires_at_ms sanity check in _validate_payload to reject internally inconsistent (HMAC-protected) lease payloads. - Sort _NATIVE_PATH_REDACTIONS by length (descending) before iterating in redact_native_paths so a longer registered path is replaced before a shorter prefix path; otherwise logs containing /foo/X.gguf.bak after only /foo/X.gguf was registered would leak the .bak suffix. - classify_existing_path now re-checks the canonical path with symlink_metadata after canonicalize, so a regular file that is replaced with a symlink in the small canonicalize window is rejected at registration. - ModelSelector renders the local file picker as its own block (not in the eject ternary), so a user with an active model can still replace it via the picker rather than only via drag/drop. - useNativePathLeasesSupported caps the readiness probe at MAX_READINESS_POLLS (60 = ~5 minutes) and aborts the in-flight fetch on unmount via AbortController, so a permanently-disabled backend stops generating sustained traffic and hot-reload no longer leaks open connections. - useChooseNativeModel returns a stable useCallback closure and guards the OS dialog with a useRef so rapid double-clicks cannot open multiple dialogs and orphan Rust tokens. - Branch the multi-file drop toast: if no GGUF was present we say "Only .gguf model files can be dropped here." and otherwise "Drop a single .gguf model file." so users dropping non-GGUF attachments get an accurate explanation. * native_path_leases: lstat the signed canonical path before resolving The earlier change to lstat inside _validate_current_stat operates on grant.canonical_path, which is the post-resolve target. If the user atomically replaces the originally-signed file with a symlink to a different file of identical size and mtime, path.resolve(strict=True) follows the symlink, samefile returns True (both ends share the new inode), and the lstat in _validate_current_stat sees the regular target file rather than the symlink, so the swap goes undetected. Add an os.lstat on the signed canonical path before path.resolve(strict=True), and reject S_ISLNK there. The lstat in _validate_current_stat stays as defense-in-depth for swaps that occur strictly between resolve and stat. * Studio: scrub native lease secret before mp.Queue spawn and tighten lease lifecycle - Move _CTX.Queue / _CTX.Event / _CTX.Process construction inside native_path_secret_removed_for_child_start at the inference, export, training and data-recipe spawn sites. The first Queue creation lazily spawns Python's multiprocessing.resource_tracker child, so when it ran outside the scrub context the tracker process inherited the lease secret. Reproduced via the proc filesystem environ entry; the wrapped order keeps the tracker clean. - native_path_secret_removed_for_child_start now refcounts entries: the env var is popped on the first entry and restored only when the last context exits. Concurrent training/inference/export starts no longer serialize on the env lock across the entire proc.start yield, while still guaranteeing the env stays empty for the duration of every overlapping spawn. - run_without_native_path_secret now also nulls the module-level cached lease secret. With the existing spawn-only multiprocessing context the cache is irrelevant in practice, but a future fork caller would otherwise inherit the in-memory secret even though the env var was scrubbed. - filter_sensitive_data now applies the native lease key check on the top-level event_dict, not only on nested dicts, so a logger call that includes a lease value as a top-level keyword field actually redacts it (the bare value does not match the prefix-anchored regex). - chat-page loadNativeModelIntent now passes intent.id to clearModelIntent so a second drag-drop during an in-flight first auto-load is not wiped from the chip area when the first resolves. - Bump useNativePathLeasesSupported's MAX_READINESS_POLLS from 60 to 720 so first-run installs that compile llama.cpp from source or download large CUDA wheels (well past 5 minutes) don't permanently disable the native picker. * native_path_leases: serialize first-decode against scrub context _decode_secret used a separate _SECRET_INIT_LOCK from the env scrub's _NATIVE_PATH_ENV_LOCK, so the very first decode (before the cache is populated) could race a concurrent native_path_secret_removed_for_child_start and read os.environ during the env-empty window, raising "Native path grants require the managed desktop backend." Subsequent calls hit the cache and were already safe. Acquire _NATIVE_PATH_ENV_LOCK around the env read inside _SECRET_INIT_LOCK and fall back to _SCRUB_SAVED_SECRET when the scrub has temporarily popped the env var. Lock ordering (init then env) is consistent with no other caller, so no deadlock. * Studio: surface native model load errors and harden native path label cache - Native model load and validate now bubble up the actual exception (with paths redacted) and apply the same friendly-error rewrite the non-native path uses, so users see "CUDA OOM", "trust_remote_code required", etc. instead of a generic "Failed to load native model: <label>". - run_without_native_path_secret now also nulls _SCRUB_SAVED_SECRET so a forked grandchild that imports native_path_leases cannot recover the secret via the scrub-aware fallback in _decode_secret. - _NATIVE_PATH_LABELS now has its own 10000-entry cap independent of the 100-entry redaction list, so display_label_for_native_path no longer falls back to returning the raw canonical path after 101 native paths in one session. Redaction list keeps the 100-entry cap for log-scan performance. - _validate_payload now also rejects null bytes in display_label, which is echoed back in HTTP responses and log lines. * Studio: harden native path lease validation and chained native rollback - child_env_without_native_path_secret now copies os.environ under _NATIVE_PATH_ENV_LOCK so a concurrent scrub-context env pop cannot raise RuntimeError: dictionary changed size during iteration in a background hardware scan or other env reader. - _validate_payload and grant construction route every signed numeric field (version, issued_at_ms, expires_at_ms, size_bytes, modified_ms) through new _required_int / _optional_int helpers that wrap raw int() ValueError into NativePathLeaseError. The single upstream catcher produces 400 instead of 500 for malformed signed payloads. - verify_native_path_lease now runs _validate_current_stat before _consume_nonce, so a transient stat error on the canonical path no longer permanently burns the nonce. Concurrent verifies still serialize through _consume_nonce, so single-use is preserved. - Chained native model rollback now restores activeNativePathToken in the chat runtime store after a successful rollback loadModel. Without this, a second consecutive failed switch could not re-roll-back because the store token had been overwritten by the failed attempt. - validate_model now applies the same not_supported_hints friendly rewrite to native model errors that load_model already does, so a native .gguf that fails validation with an upstream "is not supported" message gets the same actionable wording as the non-native branch. * Studio: harden native path log redaction, status disclosure, and chip lifecycle - structlog processor chain now runs format_exc_info before filter_sensitive_data so traceback strings are produced (and then redacted) rather than passed through as untouched (type, value, tb) tuples that the JSON or console renderer formats after the redaction filter has already finished. - native_path_secret_removed_for_child_start clears _CACHED_LEASE_SECRET in addition to popping the env var, so a fork during the scrub window cannot inherit the cached bytes via the parent's heap. Parent verify calls during the window keep working through the existing scrub-aware fallback in _decode_secret. - load_model's except ValueError handler now redacts native paths and uses the native model log label when native_grant_backed is true. Previously a ValueError raised after lease verification (e.g. from ModelConfig.from_identifier or downstream GGUF parsing) returned the raw exception string in the HTTP response body. - llama_cpp_backend now records the native display label at GGUF load time, and /api/inference/status prefers it over the redaction store. After a Python backend restart the redaction store is empty; the attribute keeps the friendly label, and an absolute model_identifier with no other label source falls back to the basename so the canonical path no longer appears in active_model. - reveal_path_token uses native "reveal and select" commands on macOS (open -R) and Windows (explorer /select,) so the file is highlighted in the file manager. Linux keeps the existing parent-directory open. - Native model rollback that fails because the previous token cannot be consumed now throws a rollback-specific Error, and the outer empty catch was replaced with one that re-throws the rollback error. The rollback-specific message now reaches the user instead of being overwritten by the original load error message. - NativeModelChip tracks the Rust token's expiresAtMs on a single setTimeout, disables the Load button at expiry, and relabels it "Select again" with an explanatory tooltip so users do not click into a guaranteed-failure path after the 15-minute TTL elapses. * Studio: tighten native artifact policy, mmproj sibling check, and intake UX - is_open_safe_artifact no longer grants Open for directories. Reveal already handles directory navigation, so the change closes the attack surface where a macOS .app artifact could be launched via open_path_token + open::that_detached. - Display labels are sanitized in classify_existing_path. Control characters in filenames (newlines, tabs, NUL et al.) are replaced with spaces and the label is trimmed and capped, so a file named with embedded newlines cannot inject forged log lines or scramble the UI status panel. - validate_entry_path skips the size_bytes/modified_ms equality check when the operation is Reveal or Open. Cloud-sync agents (Dropbox, iCloud Drive, OneDrive) routinely rewrite extended-attribute metadata which bumps mtime, and the user expects Reveal/Open to remain available for files in synced folders. - llama_cpp_backend gains a _native_grant_backed flag at GGUF load success. /api/inference/status only applies the absolute-path basename fallback when that flag is true, so a non-native absolute local GGUF still reports its canonical model_identifier and unload by identifier keeps working. - Native vision GGUFs now run through _validate_native_mmproj_companion before llama-server starts: the companion mmproj must be a regular file, not a symlink, and must live in the same resolved directory as the granted GGUF. This stops a hostile sibling or symlinked mmproj from being loaded under a single-file lease. - Chained native rollback restructured: the rollback loadModel + state + refresh runs inside its own try/catch that swallows so the outer throw error surfaces the ORIGINAL load failure. The native-token consume-failure case still throws the rollback-specific message early, before the inner block runs, so its actionable guidance is preserved. - Loading-model state and the duplicate-load guard in the chat runtime hook now compare both the model id and the native path token. Two drops or picks with the same basename in different folders no longer silently dedup; the second token is honored. - chat-page loadNativeModelIntent awaits selectModel before clearing the pending intent. If selectModel returns early via dedup or throws, the chip and its token stay so the user can retry instead of losing the selection. - NativeModelChip's Reveal button is disabled when the lease has expired (Rust would reject it anyway), and the Load button label reads "Expired" instead of "Select again" so the disabled element no longer promises an action it cannot perform. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-05-04 11:46:18 +02:00
Roland Tannous	5262d93b58	studio: add --local to setup.sh + overlay unsloth-zoo from git main (#5252) ... * studio: add --local to setup.sh + overlay unsloth-zoo from git main setup.sh now accepts --local, which exports STUDIO_LOCAL_INSTALL=1 and STUDIO_LOCAL_REPO=$REPO_ROOT. install_python_stack.py overlays unsloth-zoo from git main on top of the editable unsloth checkout in both local_repo branches (no-torch and with-torch). The Colab notebook now invokes ./studio/setup.sh --local so the cloned repo is used in editable mode and unsloth-zoo tracks main, matching the behavior of install.sh --local on a VM. install.sh --local is unchanged: it still sets SKIP_STUDIO_BASE=1, which short-circuits the local_repo branches in install_python_stack.py, so the overlay is not run twice. * studio: make --local overlays visible + guard empty arg parsing - setup.sh: gate the --local flag loop on $# > 0 (defensive against any shell that surfaces unset $@ under set -u) and emit a substep when local mode is detected so the user can confirm the flag was parsed. - install_python_stack.py: emit explicit _step lines before each overlay pip_install in both local_repo branches so overlays appear in the static log instead of being overwritten by the in-place progress bar.	2026-05-02 08:51:56 +04:00
Etherll	daf0889804	Fix Windows install when paths contain spaces or Python 3.14 is on PATH (#5201) ... * fix(studio): use py.exe to detect supported Python on Windows Description: The previous detection looked at `python --version` on PATH and hard-failed if the resolved Python wasn't 3.11-3.13. On systems where Python 3.14 sits ahead of 3.13 in PATH order, this aborted the installer even though a supported interpreter was installed. Prefer the py.exe launcher and probe `py -3.13`, `py -3.12`, `py -3.11` in turn. Fall back to `python --version` only when py.exe is absent, and surface a clearer error when no supported version can be found via either path. * Studio: consolidate Windows studio overlay into single Tauri-gated block Replace the in-file sentinel hotfix and the unconditional file-copy overlay with a single block gated on $TauriMode. Hash-compare makes re-runs no-ops, removing the sentinel-clobbering bug that occurred when the second copy path overwrote the marker without re-adding it. Non-Tauri --local installs no longer need a copy overlay: the editable install above (uv pip install -e $RepoRoot --no-deps) makes _PACKAGE_ROOT in unsloth_cli/commands/studio.py resolve to the repo source tree via PEP 660 __file__-relative resolution, so `unsloth studio setup` finds the local setup.ps1 and install_python_stack.py without any file copying. Plain PyPI installs invoked from a checked-out repo directory are also no longer silently overlaid from cwd. * fix(studio): work around uv space-in-path truncation on Windows uv 0.11.x truncates `-c <path>` and `-r <path>` arguments at the first space, breaking installs on Windows when the venv or repo sits under a path containing spaces (e.g. C:\Users\First Last\...). Pass paths through GetShortPathNameW to convert to 8.3 short form before handing them to uv. Plain pip is unaffected and keeps the original long path. No-op on Linux/Mac (gated on IS_WINDOWS and on the path actually containing a space). * Refactor Python stack overlay logic in install.ps1 Refactor overlay logic for Python stack installation and improve handling of missing target directories. * Update Python installation logic in setup.ps1	2026-04-28 01:10:47 -07:00
Daniel Han	b09aa82a3a	Studio: add github_repo seed reader and GitHub Support Bot recipe (#5169) ... * Studio: add github_repo seed reader and GitHub Support Bot recipe Adds a first-party Data Designer seed reader that scrapes GitHub issues, pull requests, and commits from one or more repositories via the GraphQL API, and a learning recipe (GitHub Support Bot) that turns those rows into synthetic support Q&A pairs for fine-tuning. Backend (new plugin studio/backend/plugins/data-designer-github-repo-seed): * GitHubRepoSeedSource config: repos, token (falls back to GH_TOKEN / GITHUB_TOKEN env var), item_types (issues / pulls / commits), per-resource limit (0 means all), max_comments_per_item. * Rate-limit-aware GraphQL client (GitHubClient + RepoScraper) shared across repos; flattens each item into a uniform row with columns item_type, repo, number, title, body, state, author, created_at, closed_at, url, labels, comments. * Registered via the data_designer.plugins entry point. Frontend: * New seed_github block variant so the seed node card shows "GitHub repositories" instead of the generic "Document file" placeholder, with its own icon and inline summary (repo count + item-type list). * Rewritten seed dialog github_repo form: repos textarea pre-filled with unslothai/unsloth + unslothai/unsloth-zoo, password input for the GH token, items-per-repo number with an "All" toggle, and the noisier options (item types, max comments, include comments) tucked under an Advanced collapsible. * Local model auto-load on Run: if a recipe uses an is_local provider and the inference server is not already serving that model, the executions hook calls /api/inference/load first. Removes the "open /chat to load a model" prerequisite that users kept tripping on. * Honor the recipe's run.rows value in the Run dialog (previously the store reset to 5 regardless of what the template shipped). Recipe (studio/frontend/src/features/data-recipes/learning-recipes/ github-support-bot.json): * Defaults to the Local Model provider + unsloth/gemma-4-E2B-it-GGUF. * Scrapes unslothai/unsloth and unslothai/unsloth-zoo, issues and pulls, up to 100 items per resource. * Two LLM blocks: normalized_question (llm-text) rewrites each thread into a clean support question, support_answer (llm-structured) produces JSON with answer / diagnosis_questions / cites / confidence. * Run defaults to 10 rows for a quick smoke test. Verified end-to-end on a running Studio: card renders, source-data dialog is pre-populated, All toggle disables the limit input, the recipe executes and produces rows against a loaded local GGUF. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: improve GitHub recipe support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: speed up GitHub scraper and harden the support-bot recipe Addresses a perf issue found while demoing the github_repo seed reader: Scraper is too slow at scale. The PRs GraphQL query pulls deeply nested fields (reviewThreads, reviews, commits, timelineItems, etc.) so the page size was pinned at 3 to stay under GitHub's node-count ceiling. 100 PRs meant 34 serial round trips. Added lighter query variants (PRS_PAGE_QUERY_LIGHT, ISSUES_PAGE_QUERY_LIGHT) that drop the fields the Studio flatten layer does not use (it only reads title, body, state, author, labels, comments). With the light query PR pages can safely go to 25 per page and issues to 50. The plugin scraper now passes light=True to RepoScraper so Studio always uses the fast path; the heavy query remains available for other callers. Recipe defaults are now demo-ready with production knobs called out: - max_parallel_requests: 1 and max_tokens: 800 so small local models stay stable when running the support_answer structured column. - support_answer prompt trimmed to 80-200 words so gemma-4-E2B GGUF can actually comply with the schema. The canonical 150-300 word codex prompt is still documented in the node3 markdown note for production upgrades. * Studio: rename GitHub recipe to 'GitHub Scraper' and add Easy mode Changes the recipe framing from a single-purpose 'Support Bot' pipeline to a general-purpose scraper that produces {user_request, grounded_response} training pairs. Aligns with the canonical github_data_gatherer dataset (11 enrichment tasks mirrored in pr_requests_20 / issue_requests_20 on the input side and explain_pr / issue_fix_plan / issue_solution on the output side). Recipe JSON changes: - columns[0] renamed normalized_question -> user_request, prompt now inverts a GitHub thread into a realistic user ask instead of normalising it. - columns[1] renamed support_answer -> coauthor_response, emits {response, followups, cites, task, confidence} and branches on issue vs PR thread type. - Notes rewritten to document the 11-task catalog and the canonical production prompt to paste in for a full dataset backfill. Frontend: Easy mode for github_repo recipes. The drag-and-drop canvas is hidden behind an 'Advanced' tab; Easy mode is the default for any recipe whose seed_source_type is github_repo. The Easy form reuses the existing GithubRepoSeedForm (promoted to exported), adds a rows input bound to previewRows, a model field bound to the model_config, and a single Run button that calls runPreview() directly (no modal). Non-github recipes see the same Editor / Runs tabs as before. View mode persists per-recipe-id in localStorage under recipe-studio:view-mode:<recipeId>. * Studio: auto-detect server GH_TOKEN and widen Easy-mode detection The GitHub seed form now fetches /api/data-recipe/seed/github/env-token on mount and, when the server exposes a GH_TOKEN / GITHUB_TOKEN env var and the token field is blank, shows a small 'Using server env var' badge and swaps the placeholder text. The token value itself is never returned to the UI. Widens Easy-mode detection in recipe-studio-page.tsx so that recipes saved before ui.seed_source_type was persisted also get the Easy tab: falls back to recipe.seed_config.source.seed_type, which is always present for github_repo seeds. * fix: polish GitHub recipe UI * Studio: default llama-server --threads to -1 (auto) Previously we passed --threads only when the caller set an explicit value, which meant llama-server fell back to its internal default. That default has varied across llama.cpp builds (some versions use hardware concurrency including hyperthreads, which hurts throughput on CPU-heavy inference). Always passing --threads -1 pins the behaviour to llama.cpp's auto-detect (physical cores). Caller-supplied n_threads still wins when non-None. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: auto-switch Easy mode to Runs pane on run start Easy mode had no progress island or canvas overlay, so after clicking Run the only visible state was the button label flipping to "Running..." while the screen otherwise stayed identical. This reads as stuck even though the job is progressing. Wire an onExecutionStart callback from recipe-studio-page.tsx through to useRecipeExecutions so that when a run is kicked off from easy mode, the page flips to the executions view where the Runs sidebar, progress bar, rate/ETA panel, and live log are rendered. Advanced/editor mode keeps its existing behavior and stays on the canvas (it already has the floating ExecutionProgressIsland). * fix: clean up GitHub scraper layout * Studio: forward llm-structured output_format as llama-server response_format Local GGUF runs of llm-structured columns used to generate the full max_tokens budget before the prompt-level "return JSON in a ```json fence" instruction got parsed. Small models (e.g. gemma-4-E2B-it) routinely broke format, so each row took ~65s and frequently failed with "No parsable JSON structure within ```json markdown fence". For any local-provider model_config referenced by an llm-structured column, clone the model_config and inject response_format into the clone's inference_parameters. Uses llama.cpp server's flat shape (tools/server/README.md): {"type": "json_schema", "schema": <output_format>} Not the OpenAI-nested form; data_designer's OpenAI adapter forwards response_format verbatim via facade._COMPLETION_REQUEST_FIELDS, and llama-server's documented schema path expects the flat variant. The clone is per (model_alias, column) so: - llm-text / llm-judge columns that share the same alias keep free-form sampling. - Each structured column gets its own schema, so columns with different output_formats don't collide. Effect on gemma-4-E2B-it demos: every row parses cleanly, and the model terminates immediately after the closing brace instead of running to max_tokens. Net wall-clock is usually faster even though grammar-constrained sampling is slightly slower per token. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: flip Easy to Runs pane before validation scrape, not after Previously onExecutionStart fired inside runExecution, which runs AFTER validateRecipe() -- and validation re-invokes the seed reader. For the github_repo reader that is a full GraphQL scrape, so the user sat on a "Running..." button with an otherwise unchanged Easy form for 10-15s before anything moved. Call onExecutionStart at the top of runWithValidation, right after we have a payload to send. The view flips immediately; ensureLocalModelLoaded + validateRecipe now run against the Runs pane instead of a frozen Easy form. runExecution still calls onExecutionStart downstream, but the callback is idempotent (the page's easy -> executions guard skips the second call), so no behaviour change for runs that pass validation. If validation fails the toast + runErrors path still fires; the Easy form's error banner still reads runErrors when the user switches back. * Studio: unify data-recipe workflow auth on sk-unsloth-* keys The previous commit (a61b4cc9) assumed storage.create_api_key(..., internal=True) and storage.revoke_internal_api_key(key_id) existed, but those helpers were only in the working tree, never committed. Recipe runs in local-model mode were therefore crashing with 500 when _inject_local_providers tried to mint a workflow key. This commit ships the missing pieces. auth/storage.py: - api_keys schema gains is_internal INTEGER DEFAULT 0 (with a guarded ALTER TABLE migration so existing auth.db files upgrade in place). - create_api_key takes an internal=False kwarg; internal keys are flagged so they can be hidden from user-facing listings. - list_api_keys takes include_internal=False so UIs never see workflow keys. - New revoke_internal_api_key(key_id): id-only revoke for keys minted by non-user subjects (the JobManager does not know a username). core/data_recipe/jobs/manager.py: - JobManager.start accepts internal_api_key_id and stores it on Job so lifecycle handlers can revoke eagerly. - _handle_event revokes on EVENT_JOB_COMPLETED / _ERROR / _CANCELLED. - _pump_loop subprocess-died fallback also retires the key so a crashed worker cannot leak a live sk-unsloth-* beyond its TTL. - Revocation is best-effort (swallow exceptions) -- the 24h TTL is the safety net if storage hiccups. core/data_recipe/jobs/types.py: - Job dataclass gains internal_api_key_id: int | None = None. Replaces the bespoke 24h JWT path that jobs.py used to mint for local providers. One mint/revoke/verify surface for every API key the server issues, and revocation is now eager (seconds, not 24h) instead of TTL-only. * Studio: plug workflow-key leak on unexpected create_job errors Review follow-up on the sk-unsloth-* workflow-key lifecycle in create_job. Previously the revoke handlers wrapped mgr.start(...) but only caught RuntimeError and ValueError, and get_job_manager() sat outside the try block entirely. Any other exception type (TypeError from a mismatched kwarg, OSError from the queue write, etc.) would bubble up to FastAPI and leave the minted key live until its 24h TTL. Fix: one try block covers both get_job_manager() and mgr.start(), with a trailing except Exception that revokes and re-raises. The RuntimeError -> 409 and ValueError -> 400 paths are unchanged so specific client-facing status codes still surface. Revocation is still best-effort (_revoke_internal_api_key_safe swallows errors) because we never want revoke failures to mask the original crash. Severity is low -- the key can't bootstrap longer access and the 24h TTL bounds the window -- but the reviewer's point stands: eager revoke on every failure path is the right invariant. * Studio: nest response_format under extra_body so pydantic accepts it The previous commit dropped response_format at the top level of a cloned model_config's inference_parameters, which BuilderConfig rejected with: ValidationError: Extra inputs are not permitted [type=extra_forbidden] data_designer.model_configs.1.inference_parameters.response_format data_designer's BaseInferenceParams is a pydantic model with extra=forbid and only a fixed set of fields (temperature, top_p, max_tokens, max_parallel_requests, timeout, extra_body). The pass-through path for anything the schema doesn't know about is `extra_body`, which the OpenAI SDK spreads into the chat-completions request body at the top level -- which is exactly where llama-server reads response_format from. Inject under extra_body (merging with any existing extra_body contents) so the clone validates. llama-server still receives {"type": "json_schema", "schema": <output_format>} at the top level of the request body, which is the flat shape llama.cpp's server expects. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: forward response_format to llama-server and fence-wrap the reply Two-part fix for the llm-structured data-recipe path: (1) The /v1/chat/completions proxy was dropping response_format. The route's passthrough branch only triggered on tools / tool messages, so requests carrying a JSON schema fell into the non-passthrough GGUF path which calls generate_chat_completion (no response_format kwarg). The schema never reached llama-server, so guided decoding was a no-op and the model emitted free-form text that happened to parse a fraction of the time. Widen the passthrough trigger and teach _build_passthrough_payload to forward response_format so llama-server's GBNF grammar actually runs. Guided decoding does not require supports_tools, so split the condition: a request is now passthrough-routed if it carries tools/tool messages (existing behavior) OR carries response_format (new). The vision guard, streaming fork, and tools-choice defaulting are unchanged. (2) data_designer's llm-structured parser looks for a ```json ... ``` markdown fence and discards anything else. Guided decoding emits only the JSON object (the GBNF grammar has no fence tokens), so a 100%-valid schema-constrained run still ended up 0 ok / N failed with "No parsable JSON structure within ```json markdown fence". In _openai_passthrough_non_streaming, wrap each choice's content in the expected fence when the caller asked for guided decoding. Already-fenced content is left alone so other clients that prefer raw JSON are not affected; the wrap is scoped to requests that carried response_format. Net effect on the GitHub Support Bot recipe on a local GGUF: schema actually binds during sampling, content arrives wrapped in the fence data_designer expects, and generation terminates immediately after the closing brace instead of running out to max_tokens. * Studio: Easy mode runs a full run, capped at the user's row count Easy mode used to call runPreview, which produces a test run: no artifact persisted, reduced progress tracking, and framed in the Runs pane as "Test run". The whole point of the form is to let a user kick off a real dataset build with one click, so wire it to runFull instead and bind the Rows input to fullRows (not previewRows). runFull requires a non-empty fullRunName. The Easy form has no run-name input, so seed a default on mount whenever Easy is active and fullRunName is still empty. Uses `<recipe name> <iso-timestamp>` so each Easy run gets a stable-ish default that still sorts chronologically in the Runs pane. User can override it from the Advanced run dialog before clicking Run. Rename GithubScraperEasyView's rows props from previewRows/setPreviewRows to rows/setRows so the view stays agnostic to which hook state the page chooses to bind. Loading indicator now follows fullLoading. * Studio: clamp GitHub scrape page size and memoize the materialization Two wins for the "before Generating fires" gap on small previews: (1) scrape_{issues,prs,commits} hardcoded per_page (50 / 25 / 100) and only checked the trial limit AFTER the page was written, so a 1-row Easy run still asked GitHub for a full 50-issue + 25-PR page, wrote them all to JSONL, and then stopped because total_new already exceeded the trial cap. Cap per_page at min(page_cap, trial_limit) so github_limit=1 actually asks for first:1. (2) GitHubRepoSeedReader.get_dataset_uri used to scrape fresh on every invocation. data_designer calls the seed reader multiple times per recipe job (validation, preview, per-column sampling), so a 2-repo Easy preview ran the full GraphQL scrape three times back-to-back, burning ~15s of dead air before any LLM generation began. Added a module-level in-process cache keyed on (repos, item_types, limit, include_comments, max_comments_per_item, sha256(token)[:16]) that stores the JSONL path of the first materialization. Subsequent calls with the same signature return the cached path, guarded by a staleness check that drops the entry if the file was tmp-cleaned. Raw token values never land in the key. Net effect on a 1-row Easy run, 2 repos, limit=1: 2 GraphQL round trips instead of ~12, and the first-to-Generating gap collapses from ~15s to roughly 2-3s. * Studio: make Easy mode Rows input editable instead of snapping to 1 The Rows to generate input used type="number" with value bound directly to the rows state and an onChange that coerced any non-positive parse result back to 1. The moment the user pressed backspace to clear the field, the parent re-rendered with value=1 and the caret jumped, making it impossible to change the value without arrowing the browser's +/- spinner. Switch to a text input with inputMode="numeric" and pattern="[0-9]*" (so mobile still shows a numeric keyboard, and the browser drops the spinner buttons the user did not want). Add a local rowsText buffer so the field can hold transient empty / partial digit strings while editing without fighting the parent state; the canonical rows value only advances when the buffer parses to a valid integer in [1, 10000], and onBlur clamps back to 1 or 10000 if the user left it out of range. No behavior change for valid numeric edits - the downstream runFull() still sees a clean positive integer. * Studio: expand dataset cells horizontally by column on click Click a long cell to expand that whole column. Click again to collapse. Replaces the prior row-level vertical expansion which made it hard to compare cells across columns. State is scoped per execution and per column; the row itself is no longer a click target. * Studio: force expanded dataset column to grow wide enough to read * Studio: disable thinking for local recipe inference and plumb the kwarg Reasoning-capable models (gemma-3n, qwen3.5, etc.) emit a <think>...</think> preamble ahead of the answer by default, which roughly doubles the generated token count per row on a local GGUF and pushes the actual answer past data_designer's json-fence regex on llm-structured columns. Recipes want the terse answer, not the scratchpad. Two halves of the fix: (1) routes/data_recipe/jobs.py: when _inject_local_providers walks the recipe's model_configs to point them at the local endpoint, also stash chat_template_kwargs={"enable_thinking": false} under each config's inference_parameters.extra_body. OpenAI SDK spreads extra_body into the top-level request body, so llama-server and the Studio /v1/chat/completions route both see it. (2) routes/inference.py: the chat-completions route previously dropped chat_template_kwargs on the floor because the whitelist body builder only forwarded known fields. - At the top of openai_chat_completions, lift chat_template_kwargs.enable_thinking from payload.model_extra onto the typed payload.enable_thinking field when the caller did not set the latter, so the non-passthrough GGUF path's generate_chat_completion(...) call honors the override. - Teach _build_passthrough_payload to forward a chat_template_kwargs dict, and have _build_openai_passthrough_body derive that dict from payload.enable_thinking so response_format requests (structured columns) also land at llama-server with the reasoning preamble suppressed. Net effect on a 10-row support-bot run with gemma-4-E2B-it-GGUF: responses arrive without <think> tags, wall-clock per call drops roughly in half, and structured columns stop leaking reasoning tokens through the GBNF-constrained output. * Studio: update GitHub Support Bot learning recipe with maintainer layout Replace the template with the hand-laid-out export from the maintainer so note nodes ship with real x/y positions (scattered around the graph instead of all stacked at x=480) and the edges / canvas pan look correct on first load. Also picks up the maintainer's prompt tweaks and output schema names (coauthor_response / user_request / followups / task / cites / confidence). Diff is mostly ui.nodes positions and prompt bodies; runtime shape is unchanged (seed_config / columns still target model_1 against the Local Model provider). * Studio: auto-size dataset sample columns; wide text gets a wide column Drop the per-column click-to-expand toggle and the 180-char truncation. Every column now renders its full value. Columns with long text get a min-w of 48rem so the text is readable without wrapping into a tall block; narrow-content columns get a 12rem min-w. The table wrapper already has overflow-x-auto, so wide-column totals cause a horizontal scrollbar instead of cramming everything into the viewport. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix GitHub scrape progress * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add resetApiBase export for test setup * Studio: rename github-support-bot output columns to User / Assistant Previously emitted user_request and coauthor_response, which did not match the canonical User / Assistant chat-pair shape that downstream SFT consumers expect. Renamed the columns in the recipe JSON (columns, UI node ids, edges, notes, prompt Jinja refs) and the matching copy in the learning-recipes index, data-recipes-page, and easy view. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com>	2026-04-24 12:02:03 -07:00
Wasim Yousef Said	a5eb2e3d50	Add tauri (#5144) ... * add unsloth studio desktop app * Fix review findings - studio/src-tauri/tauri.conf.json: retarget updater to staging repo (danielhanchen/unsloth-staging-2); switch to unslothai/unsloth on upstream merge. - studio/src-tauri/linux/postremove.sh: drop the interactive read loop and the /home/* iteration. Package maintainer scripts must stay non-interactive and must not touch other users' data. - studio/frontend/src/app/auth-guards.ts: honor tauriAutoAuth() boolean. Failed auto-auth now redirects to /login; requireGuest/requirePasswordChangeFlow only redirect to /chat when auth succeeds. The new early-return on failed auth is intentional so the login / change-password flows remain reachable when desktop auth is not yet established. - studio/frontend/src/config/env.ts: keep fetched=false on health failure so later calls retry instead of caching the client-side platform guess. - studio/src-tauri/src/install.rs: pick the available system package manager (apt-get, dnf, zypper, pacman); AppImage bundles run on non-Debian distros. - studio/frontend/src/lib/open-link.ts + markdown-text/sources callers: return boolean from openLink so callers only preventDefault on handled URLs; relative hrefs now navigate natively. - studio/frontend/src/features/settings/tabs/about-tab.tsx: fetch(apiUrl(...)) so the version request targets the backend port in desktop mode. The bare /api/health predates the Tauri webview (blame: the earlier onboarding commit, which ran with same-origin frontend/backend); in desktop mode the webview origin is tauri://localhost so the bare path fails. - install.ps1: gate the install_python_stack.py hotfix on a sentinel comment instead of a content regex; append the sentinel after applying so reruns are unambiguous. - unsloth_cli/commands/studio.py _write_auth_secret: use the atomic mkstemp + os.replace path on Windows too; chmod calls are wrapped in try/except OSError. - studio/src-tauri/src/preflight.rs probe_existing_backends: fan out the health probes concurrently; desktop-auth status still runs sequentially per candidate. reqwest::Client is internally Arc-wrapped so the in-loop .clone() is a refcount bump, not a deep clone; annotated inline. - studio/src-tauri/src/preflight.rs run_cli_probe: wait() after kill() to reap the child, matching probe_cli_capability. - studio/src-tauri/src/process.rs + main.rs: add stop_backend_detached and use it from the tray quit handler so the 5s graceful-wait does not block the Tauri main loop. RunEvent::Exit keeps the synchronous safety-net call. - studio/backend/main.py: drop the permissive localhost CORS regex in api-only mode; the explicit allow_origins list is sufficient. - .github/workflows/release-desktop.yml: drop max-parallel: 1 so platform builds run in parallel, and lift releaseBody to an env var so the three tauri-action invocations share one source of truth. * Fix review findings (loop 2) - studio/backend/auth/storage.py update_password: clear_desktop_secret() alongside clear_bootstrap_password() so rotating the admin password also revokes any previously provisioned .desktop_secret. Without this, an old local desktop credential keeps minting fresh admin tokens via /api/auth/desktop-login after a password rotation. - studio/src-tauri/src/desktop_auth.rs provision_desktop_auth: wrap cmd.output().await in tokio::time::timeout(30s). DESKTOP_AUTH_LOCK is held across the whole desktop_auth flow, and previously a hanging `unsloth studio provision-desktop-auth` subprocess would pin the lock indefinitely and freeze every subsequent desktop_auth call. * Add review tests * Consolidate review tests Merge review-added tests into the existing studio/backend/tests/test_desktop_auth.py (the PR's authoritative desktop-auth test file). Drops three scaffolding files under tests/python/ in favor of five focused tests next to the tests they extend: - test_update_password_clears_desktop_secret (runtime) - test_update_password_on_unknown_user_leaves_desktop_secret_intact (runtime) - test_cli_provisioning_delegates_to_storage_create_desktop_secret (source-level) - test_cli_connect_auth_db_reads_storage_db_path (source-level) - test_desktop_auth_provision_has_bounded_timeout (Rust source-level) * Revert auth-guards.ts Tauri branches to unconditional form The review loop on PR 5144 introduced a regression: the isTauri branch of requireAuth redirected to /login when tauriAutoAuth() returned false, and requireGuest / requirePasswordChangeFlow silently fell through on the same condition. The Tauri desktop app authenticates via a local auto-generated secret; it must never surface /login or /change-password to the user. A failed auto-auth should let the startup layer retry, not expose a password form. Restore the three Tauri branches to the author's original unconditional form (requireAuth: return; requireGuest / requirePasswordChangeFlow: throw redirect({to: '/chat'})). Keep the rest of the review fixes -- the apiUrl() fetch wrapping, authRedirect helper, and fetchAuthStatus refactor are all legitimate improvements and are preserved. * Revert release-desktop.yml to author's version The review loop's workflow-file tweaks (drop max-parallel: 1, lift releaseBody to an env var) are cosmetic. OAuth tokens cannot push workflow-file changes, and fine-grained PATs cannot honor maintainerCanModify on a third-party fork. Reverting the workflow file to wasimysaid's version lets the push go through without needing a classic PAT with both repo and workflow scopes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Daniel Han <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-04-23 04:50:10 -07:00
Daniel Han	1ccfd2e0a5	fix(rocm): tighten gfx regex to ignore generic ISA lines (#5033) ... * fix(rocm): tighten gfx regex to ignore generic ISA lines ROCm 6.1+ rocminfo emits generic ISA names such as "amdgcn-amd-amdhsa--gfx11-generic" and "amdgcn-amd-amdhsa--gfx9-4-generic" alongside the real GPU name. The previous `gfx[1-9]` regex used in `_has_rocm_gpu` matched both, so a host with only a generic ISA entry would be reported as having a usable AMD GPU. Tighten the pattern to `gfx[1-9][0-9a-z]{2,3}` so only real gfx ids match. This covers every documented target from GFX6 (gfx600) through GFX12 (gfx1201), including letter-suffixed ids like gfx90a (MI250 / MI250X) and gfx90c. Documented generic ISA names always have 1 or 2 digits before the dash and no longer match. Applied to both `studio/install_python_stack.py` and `studio/install_llama_prebuilt.py` so the two detection paths agree. Co-authored-by: Martin Hoyer <mhoyer@redhat.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Martin Hoyer <mhoyer@redhat.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-04-15 05:24:41 -07:00
Roland Tannous	13928b5f0e	Add configurable PyTorch mirror via UNSLOTH_PYTORCH_MIRROR env var (#5024) ... * Add configurable PyTorch mirror via UNSLOTH_PYTORCH_MIRROR env var When set, UNSLOTH_PYTORCH_MIRROR overrides the default https://download.pytorch.org/whl base URL in all four install scripts (install.sh, install.ps1, studio/setup.ps1, studio/install_python_stack.py). When unset or empty, the official URL is used. This lets users behind corporate proxies or in regions with poor connectivity to pytorch.org point at a local mirror without patching scripts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add pytest for UNSLOTH_PYTORCH_MIRROR in install_python_stack.py Tests that _PYTORCH_WHL_BASE picks up the env var when set, falls back to the official URL when unset or empty, and preserves the value as-is (including trailing slashes). * Remove stale test assertions for missing install.sh messages * Fix GPU mocking in test_get_torch_index_url.sh Extract _has_usable_nvidia_gpu and _has_amd_rocm_gpu alongside get_torch_index_url so the GPU-presence checks work in tests. Add -L flag handling to mock nvidia-smi so it passes the GPU listing check. All 26 tests now pass on CPU-only machines. * Strip trailing slash from UNSLOTH_PYTORCH_MIRROR to avoid double-slash URLs --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-04-15 11:39:11 +04:00
Datta Nimmaturi	da78c6be71	[Studio] Install flash attn at setup time for linux (#4979) ... * [Studio] Install flash attn at setup time for linux * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleanup changes Signed-off-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Test cases * wheel_utils: narrow url_exists exceptions and log at debug level --------- Signed-off-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>	2026-04-14 16:40:17 +04:00
Daniel Han	65b4028560	Pin bitsandbytes to continuous-release_main on ROCm (4-bit decode fix) (#4954) ... * Pin bitsandbytes to continuous-release_main on ROCm for 4-bit decode fix bitsandbytes 0.49.2 on PyPI ships with a broken 4-bit GEMV kernel on every ROCm target: - CDNA (gfx90a / gfx942 / gfx950 = MI210 / MI300X / MI350) via a broken blocksize=32/64 warp64 GEMV kernel whose tests were explicitly skipped with ROCM_WARP_SIZE_64 guards because the code was known broken. - RDNA3 / RDNA3.5 (gfx1100-1103 / gfx1150-1152) via a compile-time BNB_WARP_SIZE macro in the host-side dispatch that resolves to 64 when the multi-arch wheel is compiled with CDNA as the primary target, so num_blocks is wrong on RDNA and half the GEMV output is never written. At decode shape (1, 1, hidden) both bugs produce NaN. Training is unaffected because training shapes are (batch, seq_len > 1, hidden) and never touch the GEMV path. The crash during autoregressive inference surfaces as _assert_async_cuda_kernel in torch.multinomial which on HIP becomes a hard HSA_STATUS_ERROR_EXCEPTION instead of a clean Python error. Both bugs are fixed by bitsandbytes commit 713a3b8 ("[ROCm] Enable blocksize 32 4-bit quantization and GEMV kernels on AMD CDNA", PR #1887, merged 2026-03-09) which replaces BNB_WARP_SIZE with a runtime hipDeviceGetAttribute query and ships a working CDNA warp64 kernel. That commit has not shipped to PyPI yet, but continuous-release_main wheels are published on every push to bnb main via GitHub Releases. Point the ROCm install path at the continuous-release_main x86_64 and aarch64 wheels and fall back to PyPI >=0.49.1 when the pre-release is unreachable (offline installs, firewalled hosts, or architectures not covered by the pre-release wheels). Drop the pin once bnb cuts a 0.50+ tag on PyPI. Verified on MI300X (gfx942, ROCm 7.2, torch 2.10.0+rocm7.1): direct bnb GEMV shape test now returns 0.0078 max abs error at seq_len=1 (no NaN) vs NaN on 0.49.2, and full Unsloth + for_inference + 4-bit sampling generation works end-to-end. NVIDIA / CPU / Mac / Windows paths are unaffected -- the helper is gated on the ROCm torch index and platform.machine() respectively. * Drop Studio ROCm 16-bit fallback now that bnb 0.50+ fixes 4-bit decode The 16-bit fallback in studio/backend/core/inference/inference.py was added as a workaround for a bug that this PR already fixes at the install layer: bitsandbytes <= 0.49.2 has a broken 4-bit GEMV kernel on every ROCm target, which NaNs at decode shape (seq_len=1) and crashes autoregressive inference. bnb PR #1887 (commit 713a3b8, in 0.50.0.dev0+, pinned by install.sh / install_python_stack.py in this PR) restores correct 4-bit decode on MI300X and verified working end-to-end with full Unsloth + for_inference + sampling. Revert the dual code path so ROCm and NVIDIA both go through the normal FastLanguageModel.from_pretrained + for_inference flow: - Remove the conditional `from unsloth import` that skipped the import on ROCm. The monkey-patches it was trying to avoid were never the cause of the crash; bnb 4-bit GEMV was. - Remove the `if _hw_module.IS_ROCM:` branch in load_model that loaded with plain transformers + PEFT + bfloat16, and the `_resolve_fp16_base` helper it relied on. - Remove the `get_chat_template is not None` fallback in _load_chat_template_info -- get_chat_template is now always imported. - Refactor the audio/vision ROCm guard to check _hw_module.IS_ROCM directly instead of the removed _IS_ROCM_ENV global. Audio and vision on ROCm still need separate validation (FastVisionModel and the CSM audio codecs were never tested on HIP) so the guard stays for now. Add _bnb_rocm_4bit_ok() as a runtime safety net for users who install from this PR before the install.sh bnb pin kicks in, or whose installer fell back to the PyPI pin because the continuous- release wheel was unreachable. When the installed bnb is < 0.50 on ROCm, force load_in_4bit=False and strip any -unsloth-bnb-4bit / -bnb-4bit suffix from the model path so a pre-quantized repo resolves to its FP16 sibling instead of pulling bnb back in via the repo's quantization_config. LoRA adapters whose base is a pre-quantized repo on old bnb will still fail inside Unsloth's loader -- the only real fix there is `unsloth studio update`. Verified on MI300X (gfx942, ROCm 7.2, torch 2.10.0+rocm7.1): - HAPPY path (bnb 0.50.0.dev0, load_in_4bit=True, pre-quantized repo): loads in 4-bit via the fixed GEMV, generation returns "Paris." for greedy and sampling. - SAFETY-NET path (simulated old bnb, suffix-stripped to the FP16 sibling, load_in_4bit=False): loads in bf16, generation returns "Paris." for greedy and sampling. Net diff is ~45 lines smaller than the pre-revert state because the entire plain-transformers 16-bit branch is gone. * Cache _bnb_rocm_4bit_ok() with functools.cache load_model() can be called many times in a single session but the bnb version and hardware state cannot change at runtime, so memoise the check. First call is ~1.9 ms (dominated by the lazy `import bitsandbytes` inside the try block), subsequent calls drop to sub-microsecond dict lookups. Zero behavioral change. * Shorten verbose bnb/ROCm comments Comment-only cleanup across install.sh, studio/install_python_stack.py, and studio/backend/core/inference/inference.py. No behavioral change. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove _bnb_rocm_4bit_ok safety net from inference.py Studio's ROCm support is brand new (PR #4720, merged today) and every fresh install pulls the bnb continuous-release_main wheel via install.sh / install_python_stack.py in this same PR. There are no existing ROCm Studio installs carrying bnb < 0.50, so the defensive version-check fallback is guarding against a scenario that cannot actually occur. Delete the helper, the functools import, and the safety-net block -- inference.py now calls FastLanguageModel.from_pretrained directly with no ROCm branching. * Drop audio/vision ROCm guard in inference.py — verified unblocked by bnb fix Vision inference was blocked by the same bnb 4-bit GEMV bug that affected text inference (vision models use bnb 4-bit for the LM backbone). With bnb 0.50+ pinned in install.sh / install_python_stack.py, vision works end-to-end on MI300X: Llama-3.2-11B-Vision-Instruct-unsloth-bnb-4bit loaded in 4-bit via FastVisionModel + for_inference returns a correct answer to a multimodal prompt. Audio (CSM) was never actually blocked by HIP — on this hardware CSM loads and runs its backbone forward pass fine with bnb 0.50, then fails during generate() with a transformers-level kwarg validation mismatch in generation_csm.py (`backbone_last_hidden_state` rejected). That's a pre-existing transformers/CSM integration bug that reproduces identically on NVIDIA, so the ROCm-gated guard was never actually protecting users from anything HIP-specific. Remove the combined audio/vision guard and the now-unused _hw_module import. Also restore the one-word "Can be" in an inline comment that drifted during the earlier comment-shortening pass, so the inference.py delta vs pre-#4720 is exactly the max_seq_length<=0 crash fix and nothing else. * Shorten max_seq_length=0 guard comment to one line --------- Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-04-10 06:25:39 -07:00
Daniel Han	cad8c6ad05	Add AMD ROCm/HIP support across installer and hardware detection (#4720) ... * Add ROCm detection to install.sh and expand shell tests Add AMD ROCm GPU detection to get_torch_index_url() in install.sh. When nvidia-smi is not found, probe for ROCm via amd-smi, /opt/rocm version file, hipconfig, dpkg-query, and rpm. Includes validation guard for malformed _rocm_tag, Debian epoch prefix stripping, ROCm 7.2+ cap to rocm7.1 index, bitsandbytes AMD install, and status messaging. Shell tests expanded to 23 cases. Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Add ROCm torch reinstall support to install_python_stack.py Add _detect_rocm_version() and _ensure_rocm_torch() to detect when a Linux host has ROCm but the venv received CPU-only torch, and reinstall with the correct ROCm wheels. Covers ROCm 6.0 through 7.1 with a 30-second timeout on the torch GPU probe subprocess. Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Add ROCm support to llama.cpp prebuilt installer Add has_rocm field to HostInfo, extend detect_host() to probe for ROCm via hipcc/amd-smi/rocm-smi/ROCM_PATH, and route ROCm hosts to upstream prebuilts (Linux ROCm 7.2 prebuilt with source fallback, Windows HIP prebuilt with CPU fallback). Add linux-rocm and windows-hip install kinds to runtime_patterns_for_choice(). Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Add IS_ROCM hardware flag and fix AMD error message Add IS_ROCM flag to hardware.py detect_hardware() (set when torch.version.hip is present, DeviceType stays CUDA). Export IS_ROCM from __init__.py. Add "rocm" key to get_package_versions(). Replace "We do not support AMD" error in tokenizer_utils.py with a helpful message pointing to ROCm installation docs. Co-authored-by: Daniel Han <danielhanchen@gmail.com> * Add comprehensive ROCm support test suite (68 tests) Add tests/studio/install/test_rocm_support.py covering all ROCm code paths across install_llama_prebuilt.py, install_python_stack.py, hardware.py, tokenizer_utils.py, and install.sh. All tests use mocks and run without AMD hardware. Covers: asset selection (11), runtime patterns (5), HostInfo (4), ROCm version detection (9), torch reinstall (9), index mapping (8), hardware flag (8), tokenizer message (2), install.sh structure (10), and live regression (1). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden ROCm support: probe error handling, version cap, validation Address review findings from 8 independent reviewers: - Wrap _ensure_rocm_torch() torch probe in try/except for TimeoutExpired and OSError so a hung or broken torch import does not crash the installer (8/8 reviewers flagged this) - Add torch>=2.4,<2.11.0 version cap to the ROCm reinstall path to prevent installing unsupported torch 2.11.0 from the rocm7.1 index - Use with-statement for file reads in _detect_rocm_version() to avoid resource leaks - Handle ROCM_PATH="" correctly (use `or "/opt/rocm"` instead of default parameter to avoid relative path resolution) - Strengthen shell validation guard from rocm[0-9] to rocm[1-9] to reject rocm0.x tags that would produce nonexistent PyTorch index URLs - Switch shell version cap from blocklist to allowlist (rocm6.*|rocm7.0* |rocm7.1* pass through, everything else caps to rocm7.1) so future ROCm 10+ does not fall through to a nonexistent index - Add sorted() to _ROCM_TORCH_INDEX lookup for defensive ordering - Fix test_probe_timeout_handled: replace zero-assertion test with proper assertions verifying reinstall proceeds after timeout * Clean up rocm_paths list construction in detect_host() Filter None from the ROCM_PATH env var lookup at list construction time instead of relying on the inline `if p` guard in the any() call. * Require actual AMD GPU presence before selecting ROCm paths All 8 reviewers across 2 cycles independently flagged that ROCm detection used toolkit/filesystem hints (hipcc, /opt/rocm, rocm-core) as a proxy for GPU presence, which would misroute CPU-only or NVIDIA hosts that happen to have ROCm tools installed. Now all 3 detection points (install.sh, install_python_stack.py, install_llama_prebuilt.py) probe for an actual AMD GPU before entering the ROCm path: - install.sh: check rocminfo for gfx* GPU names, or amd-smi list for device rows, before version detection - install_python_stack.py: new _has_rocm_gpu() function probes rocminfo and amd-smi list before _ensure_rocm_torch() proceeds - install_llama_prebuilt.py: detect_host() probes rocminfo/amd-smi list instead of just checking tool existence or directory paths Also: - Shell test mock amd-smi now handles "list" subcommand - Python tests updated to mock _has_rocm_gpu where needed - Added test_no_gpu_with_rocm_tools_skips to verify the new guard - Test index lookups now use sorted() to match production code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden hipconfig version parsing and torch probe compatibility - Add parts[1].isdigit() check in hipconfig version parsing to handle versions like "6.3-HIP" where the minor component has non-numeric suffix (strip "-" prefix before int() conversion) - Use getattr() in torch probe subprocess to safely handle old or custom torch builds that may lack torch.version.hip/cuda attributes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Strengthen AMD GPU detection and add NVIDIA precedence guard - Change amd-smi list detection from any-non-empty-output to requiring "gpu" marker in output, matching the shell-side NR>1 check. Prevents false positives from header-only amd-smi list output. - Add nvidia-smi check at the top of _ensure_rocm_torch() so mixed AMD+NVIDIA hosts preserve NVIDIA precedence (matching install.sh and install_llama_prebuilt.py behavior). - Apply the same amd-smi marker fix to install_llama_prebuilt.py detect_host() for consistency. * Add Windows-specific ROCm/HIP detection in detect_host() The previous detect_host() ROCm check used rocminfo and amd-smi list which are Linux-only tools. On Windows, has_rocm would always be False, making the Windows HIP prebuilt path at line 1794 unreachable. Now detect_host() uses platform-specific detection: - Linux: rocminfo (check for gfx GPU names) or amd-smi list - Windows: hipinfo.exe, amd-smi, or amdhip64.dll on PATH This allows Windows AMD users to get the HIP prebuilt binary instead of silently falling through to the CPU prebuilt. * Add AMD ROCm gaps: Mamba/SSM source builds, GPU monitoring, Windows messaging, RDNA expansion - worker.py: Add HIP detection to causal-conv1d/mamba-ssm probe, check for hipcc before ROCm source builds, improve status messages and error reporting, add timeout and uv support for the source build fallback - amd.py: New AMD GPU monitoring module via amd-smi metric --json, mirroring nvidia.py structure (utilization, temperature, power, VRAM) - hardware.py: Branch to amd.py when IS_ROCM is True for GPU utilization, visible GPU queries, and physical GPU count - install_python_stack.py: Detect AMD GPUs on Windows and warn that ROCm-enabled PyTorch must be installed manually - kernels/utils.py: Expand is_rdna() to cover RDNA2 (gfx1030-1032), RDNA3 (gfx1102-1103), RDNA3.5 (gfx1150-1152) alongside existing entries - tests: Add 32 new tests covering all changes (95/95 pass) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden ROCm detection, fix VRAM heuristic, and expand RDNA2 coverage - Windows ROCm detection: validate actual GPU presence via hipinfo/amd-smi output markers instead of just checking tool existence on PATH - _ensure_rocm_torch: validate nvidia-smi actually reports a GPU before giving NVIDIA precedence (fixes AMD-only hosts with stale NVIDIA tools) - amd.py _parse_numeric: handle dict-shaped metric objects from newer amd-smi versions ({"value": 10, "unit": "W"}) and strip MiB/GiB units - amd.py VRAM heuristic: raise threshold from 100k to 10M to correctly handle MI300X (192 GB = 196608 MB) and other high-VRAM GPUs - amd.py visible GPU: use AMD-reported GPU IDs instead of enumerate index so non-dense sets like CUDA_VISIBLE_DEVICES=1,3 report correctly - install.sh: add ROCm <6.0 minimum version guard (no PyTorch wheels exist for older versions); fix rocm7.1* glob to not match rocm7.10+ - is_rdna: add gfx1033-1036 for RDNA2 mobile GPUs (RX 6600M etc.) - worker.py: increase ROCm source build timeout from 600s to 1800s; fix success log message for ROCm source builds - Tests: update mocks for _has_usable_nvidia_gpu, add RDNA2 target asserts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add HIP_VISIBLE_DEVICES support, unit-aware VRAM parsing, Windows GPU validation - hardware.py: check HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES on ROCm before falling back to CUDA_VISIBLE_DEVICES, so multi-GPU AMD setups with HIP-specific env vars report the correct visible device set - amd.py: add _parse_memory_mb() that reads "unit" from dict-shaped amd-smi JSON (e.g. {"value": 192, "unit": "GiB"}) and converts to MB correctly; fixes MI300X VRAM misreported as 0.19 GB instead of 192 GB - install_python_stack.py: Windows AMD warning now validates actual GPU presence via hipinfo/amd-smi output markers before printing - install_llama_prebuilt.py: restore amdhip64.dll fallback for Windows HIP detection after tool-based checks, so Windows HIP installs without CLI tools on PATH are still detected - hardware.py: fix IS_ROCM comment to accurately describe its role * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix HIP_VISIBLE_DEVICES empty-string handling in GPU visibility spec Use explicit None checks instead of Python `or` operator when reading HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES, so that an empty string ("") is correctly honored as "no visible GPUs" rather than silently falling through to CUDA_VISIBLE_DEVICES on mixed ROCm+CUDA systems. * Fix IS_ROCM test assertion for multi-line formatting * Cap torchvision/torchaudio versions, remove amdhip64.dll fallback, fix visible GPU count - Cap torchvision<0.26.0 and torchaudio<2.11.0 alongside torch<2.11.0 in both install.sh and install_python_stack.py to prevent resolver from selecting incompatible companion packages from ROCm wheel index - Remove amdhip64.dll fallback in Windows ROCm detection (DLL presence without hipinfo/amd-smi is not proof of GPU existence) - Fix get_visible_gpu_count() to use _get_parent_visible_gpu_spec() which respects HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES on ROCm hosts * Attribute is_rdna() RDNA2/3/3.5/4 expansion to PR #4428 The is_rdna() expansion to cover RDNA2 (gfx1030-1036), RDNA3 (gfx1100-1103), RDNA3.5 (gfx1150-1152), and RDNA4 (gfx1200-1201) architectures is based on the original work from PR #4428. Co-authored-by: GoldenGrapeGentleman <yueyuan@amd.com> Co-authored-by: billishyahao <bill.he@amd.com> * Support AMD Radeon for studio (#4770) Co-authored-by: Iswarya Alex <iswarya.alex@amd.com> * Remove ROCm test files from main PR Move test_rocm_support.py and shell test additions to a separate PR to keep the main ROCm support PR focused on implementation changes. * Fix installer and hardware detection issues for PR #4720 - Fix empty _tri_arg passed to uv pip install in Radeon path (causes "Empty field is not allowed for PEP508" error) - Fix Radeon fallback: use ROCm index instead of CPU-only when repo.radeon.com is unreachable (TORCH_INDEX_URL already has ROCm) - Use $TORCH_CONSTRAINT in fallback paths instead of hardcoded strings - Fix _pick_radeon_wheel: relax suffix to match manylinux_2_28_x86_64 wheels (AMD Radeon repo does not use bare linux_x86_64 platform tag) - Fix IS_ROCM export: use __getattr__ so callers always see the live value after detect_hardware() runs - Fix apply_gpu_ids: set HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES on ROCm so _get_parent_visible_gpu_spec picks up narrowed GPU set - Fix _parse_memory_mb: distinguish GB (1000 MB) from GiB (1024 MiB) - Add amd-smi version as a fallback in _detect_rocm_version - Fix trailing whitespace and missing newline at EOF in install.sh * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix GPU detection false positives and add missing health groups - Fix _has_rocm_gpu() false positive: require "GPU: <number>" data rows from amd-smi list, not just header containing "gpu" - Apply same fix in detect_host() in install_llama_prebuilt.py - Add runtime_payload_health_groups for linux-rocm and windows-hip so partial/corrupt ROCm/HIP prebuilt installs are properly detected - Add bitsandbytes install to Radeon fallback paths (was only in the success path, skipped when repo.radeon.com was unreachable) - Keep DEVICE/CHAT_ONLY as direct imports in __init__.py (matching main) and only use __getattr__ for IS_ROCM * Fix _ensure_rocm_torch and Windows AMD warning false positives - _ensure_rocm_torch: only skip when HIP is already present, not for CUDA builds (which are unusable on AMD-only hosts). Fixes the case where a venv has a stale CUDA wheel and the repair step is skipped. - Windows AMD warning: use GPU data row check (same as Linux fix) to avoid false positives from amd-smi list header-only output. * Fix amd-smi GPU detection for GPU[N] output format Older amd-smi versions output "GPU[0] : Card series: ..." instead of "GPU: 0". The regex now matches both "GPU: <digit>" and "GPU[<digit>" formats to detect actual GPU data rows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden AMD GPU detection against false positives - install.sh: replace weak amd-smi list check (awk 'NR>1 && NF') with strict pattern matching GPU data rows (/^GPU[[:space:]]*[:\[]/) - All files: reject rocminfo gfx000 (CPU HSA agent) by requiring gfx[1-9] instead of gfx[0-9] in the rocminfo GPU probe - Fixes false positives on hosts with ROCm tools but no AMD GPU * Remove duplicate comment from pre-commit merge * Refactor: deduplicate AMD detection, consolidate bitsandbytes, clean up imports - Extract _has_amd_rocm_gpu() shell function to avoid duplicating the rocminfo/amd-smi GPU detection logic in get_torch_index_url and the Radeon auto-detect block - Consolidate bitsandbytes install into a single case block after torch install (was duplicated 4 times across Radeon success/fallback paths) - Move math and re imports to top of amd.py (were inline in functions) - Add _smi_query() helper in hardware.py to centralize IS_ROCM backend selection for get_gpu_utilization and get_visible_gpu_utilization Addresses Gemini code review suggestions. * Fix VRAM parsing for string values and GB/GiB consistency - Extract unit from string-valued VRAM fields (e.g. "192 GiB") so _parse_memory_mb correctly applies the unit multiplier instead of treating the value as bare MB - Treat GB and GiB identically (both as binary x1024) since GPU tools including amd-smi use binary units even when labeling them "GB" - Fixes incorrect VRAM reporting on MI300-class cards (was showing ~0.19 GB instead of 192 GB for string-valued outputs) * Add --no-cache to uv for ROCm HIP source builds Avoid stale cache artifacts from partial HIP source builds when uv is used for causal-conv1d/mamba-ssm compilation on ROCm. The pip path already uses --no-cache-dir; this adds the uv equivalent (--no-cache) only when is_hip is True. * Fix critical: initialize _amd_gpu_radeon before case block _amd_gpu_radeon was only set inside the */rocm*) case arm, so on NVIDIA/CPU/macOS paths where TORCH_INDEX_URL does not contain "rocm", the variable was unbound. With set -u (nounset) enabled, this crashes the installer for every non-AMD user. Move initialization to before the case block so it is always defined. * Fix Windows AMD: route has_rocm hosts to HIP prebuilt path resolve_release_asset_choice was selecting windows-cpu for all Windows x86_64 hosts including those with has_rocm=True. Windows AMD users should fall through to resolve_upstream_asset_choice which tries the HIP prebuilt first. Add "not host.has_rocm" guard to the published windows-cpu selection. * Harden ROCm detection, Radeon wheel fallback, and HIP visibility Addresses review findings from parallel reviewers on PR #4720: - install.sh: add _has_usable_nvidia_gpu() helper requiring nvidia-smi -L to actually list a GPU before treating the host as NVIDIA. Fixes the stale-nvidia-smi-on-PATH regression where AMD-only hosts fell into the CUDA branch. - install.sh: fix hipconfig awk blocks to propagate a non-zero exit code when the output is not a recognisable version string, so the ||-chain continues to dpkg-query / rpm instead of terminating early. - install.sh: fail-closed on Radeon wheel fallback. When torch, torchvision or torchaudio is missing from the Radeon repo for the active Python tag, fall back to the standard ROCm index instead of silently mixing Radeon wheels with PyPI defaults. Quote all wheel arguments individually so wheel filenames cannot be word-split or glob-expanded. - install_llama_prebuilt.py: detect_host() now requires nvidia-smi -L to list a GPU before setting has_physical_nvidia. Routes AMD ROCm hosts with a broken leftover nvidia-smi to the ROCm path instead of misclassifying them as NVIDIA. - install_llama_prebuilt.py: scan upstream assets for any rocm-<version> prebuilt instead of hard-coding rocm-7.2, so ROCm 6.x / 7.0 / 7.1 / 7.3+ users pick up a matching upstream prebuilt when one exists. - install_llama_prebuilt.py: validate_server() adds --n-gpu-layers 1 for linux-rocm and windows-hip hosts, so new HIP prebuilts are preflighted on the GPU path instead of passing validation on CPU only. - install_llama_prebuilt.py: restore the published windows-cpu fallback for AMD Windows hosts without a HIP prebuilt so hash-approved bundles are still preferred over the raw upstream CPU asset. - install_python_stack.py: drop the /opt/rocm / hipcc gate in _ensure_rocm_torch() and rely on _has_rocm_gpu(). Runtime-only ROCm installs (package-managed minimal installs, Radeon software) that ship amd-smi / rocminfo without hipcc can now repair a CPU-only venv via "unsloth studio update". Adds an explicit IS_WINDOWS / IS_MACOS guard. - studio/backend/utils/hardware/amd.py: honour HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES in get_primary_gpu_utilization(). A process restricted to GPU 2 now reports metrics for GPU 2 instead of physical GPU 0. Tighten the plain bytes unit detection to an explicit allowlist. - studio/backend/utils/hardware/hardware.py: route get_backend_visible_gpu_info()'s backend_cuda_visible_devices field through a helper that reads HIP_VISIBLE_DEVICES on ROCm. Drop the unconditional "(rocm=False)" suffix in apply_gpu_ids() logs. * Fix round 2 regressions: ROCm validate_server and Windows HIP routing Follow-up to 810b833b addressing review findings on the first round of hardening commits: - install_llama_prebuilt.py validate_server: gate --n-gpu-layers on the resolved install_kind instead of host.has_rocm. AMD Windows hosts without a HIP prebuilt fall back to windows-cpu and must not be validated with GPU layers; thread install_kind through from the caller. - install_llama_prebuilt.py resolve_release_asset_choice: reinstate the "not has_rocm" guard on the published windows-cpu bundle so AMD Windows hosts reach resolve_upstream_asset_choice() where the new HIP prebuilt path lives. Prefer a published windows-hip bundle first when one exists, fall through to upstream HIP + upstream CPU otherwise. - install_llama_prebuilt.py detect_host: also set has_physical_nvidia when the secondary --query-gpu block confirms a working NVIDIA GPU, so older nvidia-smi versions without -L support do not silently skip the Linux diagnostics that key off has_physical_nvidia. - install_llama_prebuilt.py: drop redundant "import re as _re" / "import re as _re_rocm" local aliases in favour of the existing top-level "import re". - install_python_stack.py _ensure_rocm_torch: run the AMD bitsandbytes install unconditionally after the HIP-torch probe so "unsloth studio update" on venvs that already have ROCm torch still gains the AMD bitsandbytes build. - install.sh: add a non-x86_64 early-exit to get_torch_index_url() so aarch64 / arm64 Linux hosts do not hit the ROCm wheel index (PyTorch only publishes ROCm wheels for linux_x86_64). - install.sh: add bitsandbytes install to the migrated-environment branch so upgrades pick it up for ROCm hosts instead of only the fresh-install path. - install.sh: in the Radeon wheel path, pass version constraints + --no-index --find-links to uv instead of explicit wheel URLs so a version-compatible torch / torchvision / torchaudio triple is resolved, rather than picking the highest-version wheel for each package independently. - studio/backend/utils/hardware/amd.py _first_visible_amd_gpu_id: fall through to lower-priority visibility env vars when the first entry is malformed (leading comma, all-whitespace first token) instead of silently returning GPU 0. * Fix round 3 findings: x86_64 guard, ROCm version clip, Radeon deps Address issues surfaced by the round 3 reviewers on top of 8636fa63: - install_python_stack.py _ensure_rocm_torch: add the same `x86_64` guard that install.sh already has. Linux aarch64 / arm64 ROCm hosts must skip the repair path entirely; PyTorch only publishes ROCm wheels for linux_x86_64, and without this guard `unsloth studio update` aborts with a missing-wheel error on non x86_64 hosts. - install_llama_prebuilt.py resolve_upstream_asset_choice: add a best-effort _detect_host_rocm_version() helper (reading /opt/rocm/.info/version, amd-smi version, hipconfig --version) and filter rocm_candidates to entries whose major.minor is <= host version. Falls back to the newest candidate only when no compatible one exists, so a ROCm 6.4 host downloads rocm-6.4 instead of being handed the numerically newest rocm-7.2 bundle (which fails preflight and forces a source build). - install.sh: remove the round 2 --no-index switch from the Radeon wheel branch. --no-index forced uv to ignore PyPI entirely, which broke transitive dependency resolution (filelock, sympy, networkx, jinja2, fsspec, setuptools, typing-extensions, ...) on a fresh venv. Restore the round 1 explicit wheel URL invocation but add a torch / torchvision / torchaudio version-pair sanity check so a mismatched trio (e.g. torch 2.9.1 + torchvision 0.23.0 + torchaudio 2.9.0) falls back to the standard ROCm index instead of installing a broken combination. - install_python_stack.py _ensure_rocm_torch: restructure the "tag is None" path so it no longer short-circuits the bitsandbytes install. On a ROCm runtime older than anything in _ROCM_TORCH_INDEX, print the "no wheel" warning but still run the AMD bitsandbytes install. - studio/backend/core/training/worker.py: restore the pre-PR "no timeout" behaviour for non-HIP causal-conv1d / mamba-ssm source builds. The round 2 "timeout = 1800 if is_hip else 300" cap aborts slow non-HIP builds (Linux aarch64, unsupported torch/CUDA combos) after 5 minutes; omit timeout for the non-HIP branch so the cap only applies to ROCm source builds. * Fix round 4 findings: apply_gpu_ids env inheritance, Radeon X.Y, bitsandbytes gate Address remaining issues surfaced by the round 4 reviewers: - studio/backend/utils/hardware/hardware.py apply_gpu_ids: mirror the selection into HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES whenever the caller already had a ROCm visibility env var set, not only when IS_ROCM has already been set by detect_hardware(). Training and inference workers call apply_gpu_ids() before detect_hardware() runs, so the old guard would leave a forked ROCm worker with a stale HIP_VISIBLE_DEVICES mask that no longer matched the narrowed CUDA_VISIBLE_DEVICES selection. - install.sh get_radeon_wheel_url: accept X.Y ROCm versions in addition to X.Y.Z. The `/opt/rocm/.info/version` file and some hipconfig versions report only two components, and the Radeon repository publishes both rocm-rel-X.Y.Z/ and rocm-rel-X.Y/ directories, so treating X.Y as invalid caused Radeon hosts to fall back to the generic ROCm index even when a matching AMD wheel set existed. - install_python_stack.py _ensure_rocm_torch: only install the AMD bitsandbytes build when the venv actually has a ROCm-compatible torch (either already present or just installed by this function). Previously the bitsandbytes install ran unconditionally, which could leave an AMD bitsandbytes layered on top of a CPU/CUDA torch on hosts where the ROCm runtime is older than any entry in _ROCM_TORCH_INDEX. Also add --force-reinstall so an existing CPU/CUDA bitsandbytes is replaced by the AMD build during upgrades. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix gemini findings: amd-smi metric envelope validation and dict-wrapped GPU id Two medium-severity defensive fixes from the gemini-code-assist review on the AMD monitoring backend: 1. _extract_gpu_metrics may return a dict where every value is None when amd-smi succeeds (zero exit) but the JSON envelope contains no usable fields (error response, unsupported card). The new _has_real_metrics helper lets get_primary_gpu_utilization surface available:False and lets get_visible_gpu_utilization skip ghost device rows so the UI does not render placeholder cards with empty numbers. 2. Newer amd-smi versions wrap scalar fields as {"value": 0, "unit": "none"}, including the per-GPU id. The previous int(raw_id) call silently fell back to the enumeration index in that case, losing the real GPU id. Routing raw_id through the existing _parse_numeric helper handles bare ints, floats, strings, and the dict shape uniformly, with a debug log on parse failure. * Fix gemini round 2 findings: explicit length guard on ROCm version file parser Both _detect_rocm_version (install_python_stack.py) and _detect_host_rocm_version (install_llama_prebuilt.py) read /opt/rocm/.info/version or $ROCM_PATH/lib/rocm_version, split on "." and unconditionally accessed parts[1]. The surrounding broad `except Exception: pass` already swallowed the resulting IndexError, so a one-component file like "6\n" did fall through to the next detection source -- but the control flow relied on exception handling instead of an explicit check. Add `if len(parts) >= 2:` guards in both helpers so the loop falls through on its own without raising. Behaviour is unchanged for the common multi- component case; the previously-silent IndexError path becomes an explicit no-op. * Fix gemini round 3: include has_rocm in validate_server fallback path When validate_server is called without an explicit install_kind (older call sites that have not been updated), the fallback was only enabling --n-gpu-layers for NVIDIA and macOS arm64 hosts. AMD ROCm Linux hosts fell through to the CPU validation path even though the prebuilt being exercised was a HIP binary. Add host.has_rocm to the fallback expression so the GPU offload flag is applied consistently with the install_kind=='linux-rocm' / 'windows-hip' branches above. * Fix gemini round 4: remove risky bytes-vs-MB heuristic in _parse_memory_mb The previous heuristic divided any bare number above 10_000_000 by 1024*1024 on the assumption that large unit-less values were bytes. This misclassified small VRAM allocations: 5 MB of used VRAM reported as 5_242_880 bytes without a unit would be taken at face value and render as 5_242_880 MB (~5 TB) in the monitoring UI. Modern amd-smi always provides explicit units (MiB/GiB dict form), and legacy amd-smi returns bare numbers in MB -- the heuristic never had a real workload to handle. Drop it and default to MB for bare numeric input, keeping the existing unit-aware branches for dict / string inputs unchanged. The unrelated gemini suggestion to "default minor to 0" in the amd-smi version awk parser was intentionally NOT applied: rocm7.0 and rocm7.1 ship different wheel sets, so silently substituting 0 for a missing minor could install the wrong wheels. The existing reject-and-fall-through behaviour is safer. * Fix gemini round 5: POSIX compliance and leading-comma visibility parsing Three medium findings from gemini-code-assist addressed in this commit: 1. _pick_radeon_wheel used grep -o and sort -V, both GNU extensions that are not in POSIX and break on BSD/BusyBox coreutils. install.sh has a #!/bin/sh shebang so the whole pipeline was rewritten as a single awk script that extracts all href="..." hits on each line, filters to wheels matching the package prefix and python tag, and picks the newest version via zero-padded lexical comparison. No external sort or grep is needed. 2. _first_visible_amd_gpu_id in the AMD monitoring backend treated a leading comma (e.g. HIP_VISIBLE_DEVICES=",1") as "fall through to the next env var", which is surprising given the clear intent to narrow to device 1. Filter empty tokens after the split and return the first real one. An all-commas value ("," / ",,,") still falls through because no real tokens exist; the empty-string and "-1" explicit-zero cases are unchanged. The unrelated amd-smi version awk parser suggestion was not applied (see round 4 commit message for rationale: defaulting a missing minor to 0 could silently install the wrong ROCm wheel set). * Fix 20-reviewer.py findings: base drift, Radeon %2B, dpkg/rpm fallback, bnb, backend label Consolidated fix batch from a 20-parallel reviewer.py run on the current head. Each fix is drawn from a high-consensus finding and addresses a real bug or feature gap, not a stylistic preference. 1. install.sh: bump `unsloth>=2026.4.2` -> `unsloth>=2026.4.4` at five call sites so this branch no longer regresses main's version floor (main bumped to 2026.4.4 in #4876). Without this, merging 4720 would silently downgrade the minimum version pin for fresh installs. 2. install.sh: URL-decode Radeon wheel names before extracting the torch / torchvision / torchaudio version strings. Real wheel URLs from repo.radeon.com are percent-encoded ("torch-2.10.0%2Brocm7.2.0...") so the previous `[+-]` terminator in the sed regex never matched, `_torch_ver` stayed empty, `_radeon_versions_match` stayed false, and every Radeon consumer install silently fell back to the generic ROCm index. Now decode %2B -> + first, then extract, then validate. 3. install.sh: the two AMD bitsandbytes install lines were running `uv pip install "bitsandbytes>=0.49.1"` without `--force-reinstall`, so upgrades where the venv already has a CPU/CUDA bitsandbytes satisfying the constraint would keep the stale non-AMD wheel. Add `--force-reinstall --no-cache-dir` to both call sites, matching the pattern already used in install_python_stack.py::_ensure_rocm_torch. 4. install_python_stack.py and install_llama_prebuilt.py: add `dpkg-query -W rocm-core` and `rpm -q rocm-core` fallbacks to the Python-side ROCm version detectors so they match the chain in install.sh::get_torch_index_url. Package-managed ROCm installs (Debian/Ubuntu/RHEL/Fedora distro packages) can expose GPUs via rocminfo/amd-smi but still lack /opt/rocm/.info/version, hipconfig, or amd-smi `version` output -- without these fallbacks, `unsloth studio update` on such hosts returned None and skipped the ROCm torch repair. Also strip the dpkg epoch prefix ("1:6.3.0-1") before parsing so epoch-annotated packages parse correctly. 5. hardware.py: add a `_backend_label(device)` helper that returns "rocm" when IS_ROCM is set and the device is DeviceType.CUDA, and use it for every `"backend": ...` emission in JSON responses served to the Studio frontend. Internally we still represent ROCm hosts as DeviceType.CUDA (ROCm torch reuses the whole torch.cuda.* API surface), but the user-facing API now correctly reports "rocm" on AMD boxes instead of labeling them as "cuda". All 250 simulation scenarios pass (was 233 before this batch: added 17 new regression tests covering the version pin, %2B decoding, bnb force-reinstall flags, dpkg/rpm fallback presence, and the _backend_label helper's four-way truth table). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix gemini round 6 + URL audit: amd.py defensive checks, rocm6.5+ clip to 6.4 Two rounds of fixes in one commit, plus a full URL audit of every PyPI / download.pytorch.org / repo.radeon.com reference the PR introduces. amd.py (4 medium gemini findings on commit b3627bc2): 1. _extract_gpu_metrics used `and vram_total_mb` as part of the vram_util gate. The follow-up `vram_total_mb > 0` already handles the division guard, but the truthiness check was redundant and slightly surprising for a 0.0 valid value. Replace with explicit `is not None and > 0` for both vram_util and power_util. 2. get_physical_gpu_count called `data.get("gpu", ...)` without guarding for non-dict envelopes. A scalar / string JSON response from amd-smi would raise AttributeError. Add an isinstance(data, dict) check and return None for unexpected shapes. 3. get_visible_gpu_utilization had the same .get() exposure on the outer envelope. Rewrite the gpu_list extraction as an explicit list/dict/else cascade so a malformed scalar envelope produces gpu_list=[data] and continues without raising. 4. The same function's per-entry loop also called gpu_data.get() on whatever was inside gpu_list. If a scalar ever leaks into the list (directly or via the previous fix's fallback), _extract_gpu_metrics would raise on the first .get() inside the helper. Skip non-dict entries in the loop before extracting metrics. install.sh (URL audit finding, previously flagged by 20-reviewer as #13): 5. get_torch_index_url used `rocm6.*` in the rocm tag case statement, which matched rocm6.5 and rocm6.6 and emitted download.pytorch.org/whl/rocm6.5 -- which returns HTTP 403 because PyTorch only publishes rocm 5.7, 6.0-6.4, 7.0-7.2. Enumerate the supported 6.x minors explicitly and add a rocm6.* fallback branch that clips to rocm6.4 (the last supported 6.x wheel set). URL audit results (all URLs PR 4720 references): - 14/14 download.pytorch.org/whl/{cpu,cu118,cu124,cu126,cu128,cu130, rocm6.0..6.4,rocm7.0..7.2} return HTTP 200. - 9/9 repo.radeon.com/rocm/manylinux/rocm-rel-{5.7,6.0,6.1,6.2,6.3, 6.4,7.0,7.1,7.2}/ return HTTP 200. - X.Y.Z patch directories exist for 7.0.2, 7.1.1, 7.2.1 but NOT for 6.3.0, 6.4.0, 6.2.1 -- install.sh already handles this via the X.Y.Z -> X.Y fallback sed in the Radeon wheel install block. - Docs links (rocm.docs.amd.com, docs.unsloth.ai AMD guide) and the llama.cpp GitHub releases API endpoint all return 200. Test suite: 255 -> 258. New regression coverage: - U17: get_physical_gpu_count tolerates scalar amd-smi envelope - U18: get_visible_gpu_utilization tolerates scalar envelope - U19a-c: vram_util / power_util return None on zero total, but vram_total_gb still echoes 0.0 (not None) - A_rocm{6.5,6.6,6.9}_clips_to_rocm64: install.sh clips unsupported 6.x minors to rocm6.4 instead of producing a 403 index URL * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix reviewer.py round 2: tokenizer AMD multi-GPU, --no-torch bnb, main.py backend label Three high-confidence findings from a second 20-parallel reviewer.py run on commit 7effb3ae. Triaged 15 total findings and applied the three that were confirmed as real bugs; the rest were either false positives (e.g. "migrated AMD venv not repaired" -- _ensure_rocm_torch runs downstream via setup.sh regardless), design decisions (e.g. visibility mask env vars not consulted in installer detection), or edge cases the existing fallback logic already handles. 1. unsloth/tokenizer_utils.py [6/20]: the multi-GPU guard's shell probe runs `nvidia-smi --query-gpu=memory.used`, catches the failure, then only raises if `torch.cuda.is_available()` is False. On ROCm torch, torch.cuda.is_available() returns True (ROCm reuses the torch.cuda.* API), so the guard becomes dead code on AMD hosts and multi-GPU AMD setups slip through even though unsloth does not support them yet. Add a torch.cuda.device_count() > 1 fallback inside the except so AMD multi-visible-device setups are flagged consistently with the original CUDA memory check. 2. install.sh [1/20]: the fresh-install bitsandbytes block for AMD ROCm ran unconditionally when TORCH_INDEX_URL matched `*/rocm*`, even when SKIP_TORCH=true (from --no-torch or Intel Mac auto-detect). A user running `install.sh --no-torch` on an AMD host would still pull in bitsandbytes despite explicitly asking for GGUF-only mode. Wrap the case block in an outer `[ "$SKIP_TORCH" = false ]` guard. 3. studio/backend/main.py [3/20]: the /api/system endpoint returned `"device_backend": get_device().value`, which is "cuda" on ROCm hosts (because ROCm torch piggybacks on torch.cuda). Other endpoints (hardware.py) already use the _backend_label helper which swaps "cuda" -> "rocm" when IS_ROCM. Route /api/system through the same helper so the Studio UI reports the backend consistently across all endpoints. 4. studio/backend/tests/test_utils.py: update test_backend_matches_device to call _backend_label(get_device()) instead of raw get_device().value so the test matches the new contract and still passes on CUDA hosts. Tests: 258 -> 261. New regression coverage: - X08 main.py /api/system uses _backend_label - X09 tokenizer multi-GPU guard has device_count() fallback - X10 fresh-install bnb case block gated on SKIP_TORCH=false * fix: prevent bitsandbytes from overwriting ROCm torch with CUDA wheels During install, bitsandbytes was installed without --no-deps, causing uv to resolve torch from PyPI (CUDA build) and silently overwrite the ROCm wheels that were just installed in the previous step. This happened in three places: - install.sh: bitsandbytes install in both migrated and fresh paths - install_python_stack.py: bitsandbytes install inside _ensure_rocm_torch() Additionally, multiple install steps in install_python_stack.py (extras, overrides, studio deps) can pull in CUDA torch via transitive dependencies. A final _ensure_rocm_torch() call at the end of the install sequence ensures ROCm torch is always in place at runtime. All changes are gated behind ROCm-specific conditions and do not affect NVIDIA, CPU-only, macOS, or Windows install paths. Tested on AMD Instinct MI300X VF with ROCm 7.2.0 -- confirms torch==2.10.0+rocm7.1 with HIP 7.1.25424 after install. * fix: ROCm inference fallback -- skip Unsloth patching and bnb 4-bit on HIP On AMD ROCm (HIP), two issues prevent the normal Unsloth inference path: 1. Unsloth's global monkey-patching of transformers model classes (LlamaRotaryEmbedding, attention modules) triggers _assert_async_cuda_kernel crashes on HIP during generation. Training uses different code paths and works fine. 2. bitsandbytes 4-bit matmul kernels also trigger HIP assertion failures on MI300X (CDNA3 / gfx942), even without Unsloth patching. This commit adds a ROCm-specific inference fallback that: - Skips importing Unsloth at module level (prevents global patching) - Loads models in 16-bit with plain transformers + PEFT instead - Resolves pre-quantized model names (e.g. "xxx-bnb-4bit" -> "xxx") since pre-quantized HF repos still trigger bnb codepaths - Guards get_chat_template calls (unavailable without Unsloth import) - Fixes max_seq_length=0 being passed to from_pretrained (GGUF semantics don't apply to transformers path) The NVIDIA path is completely unchanged -- Unsloth import and for_inference() optimization remain active. GGUF inference (via llama-server/HIP) is unaffected since it never imports Python model classes. AMD GPUs typically have large VRAM (e.g. 192GB on MI300X) so 16-bit loading is practical for inference. Tested on AMD Instinct MI300X VF (ROCm 7.2, HIP 7.1.25424): - Simple generation: PASS - Compare mode (base vs finetuned): PASS - GGUF inference + tool calling: PASS (unaffected by this change) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: guard audio/vision inference on ROCm, remove unused import - Add clear RuntimeError for audio/vision model inference on ROCm (these paths use Unsloth's FastModel/FastVisionModel which would crash on HIP; GGUF inference is the supported path on AMD) - Remove unused `import os as _os` from the ROCm changes * fix: amd-smi parsing for newer output format (gpu_data wrapper, mem_usage, temperature) amd-smi on recent ROCm versions (7.x) wraps metric output in a {"gpu_data": [...]} envelope instead of returning a raw list. This caused get_primary_gpu_utilization() and get_visible_gpu_utilization() to fail silently (returning available=False) because the GPU data dict was never unwrapped. Additionally: - VRAM data moved from "vram" to "mem_usage" with "total_vram" / "used_vram" keys. Added fallback key lookup. - Temperature "edge" sensor returns "N/A" on MI300X VF; the previous dict.get() chain returned the "N/A" string instead of falling through to "hotspot". Changed to a loop that checks each key until a parseable value is found. Tested on AMD Instinct MI300X VF (ROCm 7.2, amd-smi 24.x): - GPU utilization: 0% (idle), up to 100% during training - Temperature: 40-44C (from hotspot sensor) - VRAM: 0.28/191.69 GB (idle) - Power: 158-211W draw * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bug fix detecting radeon (#4940) * Bug fix detecting radeon * Expanding GPU target for gfx1100* * Generalize gfx family-prefix filter to cover gfx10/gfx12 as well rocminfo on ROCm 6.1+ emits LLVM generic-family ISA lines alongside the specific GPU (e.g. gfx11-generic next to gfx1100). The outer grep captures the bare family prefix from the generic line, and passing that to -DGPU_TARGETS breaks the HIP build because clang only accepts specific gfxNNN ids. The previous filter only special-cased gfx11. Generalize it so any bare 2-digit family prefix (gfx10, gfx11, gfx12, ...) is dropped whenever a specific sibling target is present in the same list. No real AMD GPU has a 2-digit gfx id, so the filter can only ever drop family prefixes and never a real target. Covers the existing gfx11 cases unchanged, and extends the same fix to gfx10-1-generic / gfx10-3-generic (RDNA1/2) and gfx12-generic (RDNA4), which would otherwise hit the same build failure on newer rocminfo. --------- Co-authored-by: Iswarya Alex <iswarya.alex@amd.com> Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> --------- Co-authored-by: Eda Z <eda.zhou@amd.com> Co-authored-by: GoldenGrapeGentleman <yueyuan@amd.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: billishyahao <bill.he@amd.com> Co-authored-by: Iswarya Alex <47045679+iswaryaalex@users.noreply.github.com> Co-authored-by: Iswarya Alex <iswarya.alex@amd.com> Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>	2026-04-10 01:56:12 -07:00
Lee Jackson	5557e1fd27	studio: unify Windows installer/setup logging style, verbosity controls, and startup messaging (#4651) ... * refactor(studio): unify setup terminal output style and add verbose setup mode * studio(windows): align setup.ps1 banner/steps with setup.sh (ANSI, verbose) * studio(setup): revert nvcc path reordering to match main * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio(setup): restore fail-fast llama.cpp setup flow * studio(banner): use IPv6 loopback URL when binding :: or ::1 * Fix IPv6 URL bracketing, try_quiet stderr, _step label clamp - Bracket IPv6 display_host in external_url to produce clickable URLs - Redirect try_quiet failure log to stderr instead of stdout - Clamp _step label to column width to prevent negative padding * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add sandbox integration tests for PR #4494 UX fixes Simulation harness (tests/simulate_pr4494.py) creates an isolated uv venv, copies the real source files into it, and runs subprocess tests for all three fixes with visual before/after demos and edge cases. Standalone bash test (tests/test_try_quiet.sh) validates try_quiet stderr redirect across 8 scenarios including broken-version contrast. 39 integration tests total (14 IPv6 + 15 try_quiet + 10 _step), all existing 75 unit tests still pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Truncate step() labels in setup.sh to match PS1 and Python The %-15s printf format pads short labels but does not truncate long ones. Change to %-15.15s so labels wider than 15 chars are clipped, matching the PowerShell .Substring(0,15) and Python label[:15] logic. * Remove sandbox integration tests from PR These test files are not part of the styling fix and should not ship with this PR. * Show error output on failure instead of suppressing it - install_python_stack.py: restore _red for patch_package_file warnings (was downgraded to _dim) - setup.ps1: capture winget output and show on failure for CUDA, Node, Python, and OpenSSL installs (was piped to Out-Null) - setup.ps1: always show git pull failure warning, not just in verbose mode * Show winget error output for Git and CMake installs on failure Same capture-and-print-on-failure pattern already used for Node, Python, CUDA, and OpenSSL winget installs. * fix: preserve stderr for _run_quiet error messages in setup.sh The step() helper writes to stdout, but _run_quiet's error header was originally sent to stderr (>&2). Without the redirect, callers that separate stdout/stderr would miss the failure headline while still seeing the log body on stderr. Add >&2 to both step calls inside _run_quiet to match main's behavior. * feat: add --verbose flag to setup and update commands Wire UNSLOTH_VERBOSE=1 through _run_setup_script() so that 'unsloth studio update --verbose' (and the deprecated 'setup') passes the flag to setup.sh / setup.ps1 / install_python_stack.py. * fix(studio): honor verbose logging and keep llama.cpp failures non-blocking * fix(studio): switch installer to 'studio update' and normalize Windows setup logs * chore(studio): refine localhost tip and remove skip-base setup nois * fix(studio): align Windows setup logs with Linux style and improve startup tips * fix(studio): align Windows setup logs with Linux style * refactor(windows-installer): align install/setup logs with Linux style and silence auto-launch output * refactor(windows): align installer/setup output with Linux style and reduce default verbosity * refactor(windows): match install.ps1 output style/colors to setup and quiet default logs * fix(studio-banner): update personal-computer localhost tip * fix(setup.sh): restore verbose llama.cpp build output while keeping default quiet mode * fix(install.sh): align installer logging with setup style and restore POSIX-safe color output * fix(install.sh): preserve installer reliability and launch visibility Export verbose mode for child setup processes, harden install command handling under set -e, and keep first-run studio launch non-silent so users can always see URL and port fallback output. * fix(windows installer): keep exit semantics and degrade status accurate Use quiet command redirection that preserves native exit codes, keep startup output visible on first launch, and report limited install status when llama.cpp is unavailable. * fix(setup.sh): improve log clarity and enforce GGUF degraded signaling Restore clean default setup output, add verbose-only diagnostics, fail fast on Colab dependency install errors, and return non-zero when GGUF prerequisites or llama.cpp artifacts are unavailable. * fix(installer): harden bash preflight and PowerShell GPU checks Fail fast when bash is unavailable before invoking setup.sh, and replace remaining nvidia-smi pipeline checks with stream redirection patterns that preserve reliable native exit-code handling. * fix(windows): keep verbose output visible while preserving exit codes Ensure PowerShell wrapper helpers in install/update stream native command output to host without returning it as function output, so npm logs no longer corrupt exit-code checks in verbose mode. * fix(windows): avoid sticky UNSLOTH_VERBOSE and gate studio update verbosity * Fix degraded llama.cpp exit code, PS verbose stderr, banner URLs, npm verbose - setup.sh: Do not exit non-zero when llama.cpp is unavailable; the footer already reports the limitation, and install.sh runs under set -e so a non-zero exit aborts the entire install including PATH/shortcuts/launch. - setup.ps1: Remove $? check in Invoke-SetupCommand verbose path; PS 5.1 sets $? = $false when native commands write to stderr even with exit 0. Merge stderr into stdout with 2>&1 and rely solely on $LASTEXITCODE. - startup_banner.py: Show the actual bound address when Studio is bound to a non-loopback interface instead of always showing 127.0.0.1/localhost. - setup.sh: Use run_quiet_no_exit instead of run_quiet_no_exit_always for npm install steps so --verbose correctly surfaces npm output. * Fix install.ps1 verbose stderr, propagate UNSLOTH_VERBOSE, fix git clone verbose - install.ps1: Apply same Invoke-InstallCommand fix as setup.ps1 -- merge stderr into stdout with 2>&1 and drop the $? check that misclassifies successful native commands on PS 5.1. - install.ps1 + setup.ps1: Export UNSLOTH_VERBOSE=1 to the process env when --verbose is passed so child processes like install_python_stack.py also run in verbose mode. - setup.sh: Use run_quiet_no_exit for git clone llama.cpp so --verbose correctly surfaces clone diagnostics during source-build fallback. * Surface prebuilt llama.cpp output in verbose mode, remove dead code, fix banner - setup.sh: Use tee in verbose mode for prebuilt llama.cpp installer so users can see download/validation progress while still capturing the log for structured error reporting on failure. - setup.ps1: Same fix for Windows -- use Tee-Object in verbose mode. - setup.sh: Remove run_quiet_no_exit_always() which has no remaining callers. - startup_banner.py: Avoid printing the same URL twice when Studio is bound to a specific non-loopback address that matches the display host. * Fix run_install_cmd exit code after failed if-statement The previous pattern 'if "$@"; then return 0; fi; _rc=$?' always captured $? = 0 because $? reflects the if-statement result, not the command's exit code. Switch to '"$@" && return 0; _rc=$?' which preserves the actual command exit code on failure. Applies to both verbose and quiet branches. * Fix _run_quiet exit code, double uv install, missing --local flag - setup.sh: Fix _run_quiet verbose path that always captured exit code 0 due to $? resetting after if-then-fi with no else. Switch to the same '"$@" && return 0; exit_code=$?' pattern used in install.sh. - setup.sh: Consolidate the two uv install branches (verbose + quiet) into a single attempt with conditional output. Previously, when verbose mode was on and the install failed, a second silent attempt was made. - install.ps1: Pass --local flag to 'unsloth studio update' when $StudioLocalInstall is true. Without this, studio.py's update() command overwrites STUDIO_LOCAL_INSTALL to "0", which could cause issues if setup.ps1 or install_python_stack.py later checks that variable. * Revert SKIP_STUDIO_BASE change for --no-torch, restore install banners - Revert SKIP_STUDIO_BASE from 0 to 1 for --no-torch. install.sh already installs unsloth+unsloth-zoo and no-torch-runtime.txt before calling setup.sh, so letting install_python_stack.py redo it was redundant and slowed down --no-torch installs for no benefit. - Restore the "Unsloth Studio installed!" success banner and "starting Unsloth Studio..." launch message so users get clear install completion feedback before the server starts. * Make llama.cpp build failure a hard error with proper cleanup - setup.sh: Restore exit 1 when _LLAMA_CPP_DEGRADED is true. GGUF inference requires a working llama.cpp build, so this should be a hard failure, not a silent degradation. - install.sh: Catch setup.sh's non-zero exit with '|| _SETUP_EXIT=$?' instead of letting set -e abort immediately. This ensures PATH setup, symlinks, and shortcuts still get created so the user can fix the build deps and retry with 'unsloth studio update'. After post-install steps, propagate the failure with a clear error message. * Revert install.ps1 to 'studio setup' to preserve SKIP_STUDIO_BASE 'studio update' pops SKIP_STUDIO_BASE from the environment, which defeats the fast-path version check added in PR #4667. When called from install.ps1 (which already installed packages), SKIP_STUDIO_BASE=1 must survive into setup.ps1 so it skips the redundant PyPI check and package reinstallation. 'studio setup' does not modify env vars. * Remove deprecation message from 'studio setup' command install.ps1 uses 'studio setup' (not 'studio update') to preserve SKIP_STUDIO_BASE. The deprecation message was confusing during first install since the user never typed the command. * Fix stale env vars, scope degraded exit, generic error message for PR #4651 - install.ps1: Always set STUDIO_LOCAL_INSTALL and clear STUDIO_LOCAL_REPO when not using --local, to prevent stale values from a previous --local run in the same PowerShell session. Fix log messages to say 'setup' not 'update' since we call 'studio setup'. - setup.sh: Only exit non-zero for degraded llama.cpp when called from the installer (SKIP_STUDIO_BASE=1). Direct 'unsloth studio update' keeps degraded installs successful since Studio is still usable for non-GGUF workflows and the footer already reports the limitation. - install.sh: Make the setup failure error message generic instead of GGUF-specific, so unrelated failures (npm, Python deps) do not show misleading cmake/git recovery advice. * Show captured output on failure in quiet mode for PR #4651 Both Invoke-InstallCommand (install.ps1) and Invoke-SetupCommand (setup.ps1) now capture command output in quiet mode and display it in red when the command fails. This matches the behavior of run_install_cmd in install.sh where failure output is surfaced even in quiet mode, making cross-platform error debugging consistent. * Match degraded llama.cpp exit on Windows, fix --local recovery hint for PR #4651 - setup.ps1: Exit non-zero for degraded llama.cpp when called from install.ps1 (SKIP_STUDIO_BASE=1), matching setup.sh behavior. Direct 'unsloth studio update' keeps degraded installs successful. - install.sh: Show 'unsloth studio update --local' in the recovery message when the install was run with --local, so users retry with the correct flag instead of losing local checkout context. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-03-30 00:53:23 -07:00
Daniel Han	eacaf6827c	fix: no-torch install deps without pulling torch transitively (#4650) ... Use --no-deps for ALL packages (unsloth, unsloth-zoo, and runtime deps) since the current PyPI metadata for unsloth still declares torch as a hard dependency. Runtime deps (typer, pydantic, safetensors, transformers, etc.) are installed from no-torch-runtime.txt with --no-deps to prevent transitive torch resolution from accelerate, peft, trl, and sentence-transformers. no-torch-runtime.txt now includes unsloth's own direct deps (typer, pydantic, pyyaml, nest-asyncio) since --no-deps skips those too. install.sh installs no-torch-runtime.txt directly (via helper function _find_no_torch_runtime). install.ps1 does the same via Find-NoTorchRuntimeFile. SKIP_STUDIO_BASE stays at 1 to avoid setup.sh fast-path issues. install_python_stack.py NO_TORCH branch does the same for unsloth studio update, using package_name instead of hardcoded "unsloth".	2026-03-27 05:19:26 -07:00
Daniel Han	b1c3a1e857	fix: replace [huggingfacenotorch] with no-torch-runtime.txt requirements (#4649) ... The [huggingfacenotorch] extras only exist in pyproject.toml but are NOT published on PyPI, so uv pip install "unsloth[huggingfacenotorch]" fails on fresh installs from the registry. Fix: add studio/backend/requirements/no-torch-runtime.txt with the runtime deps (safetensors, transformers, datasets, accelerate, etc.) that mirror [huggingfacenotorch] from pyproject.toml. In no-torch mode: 1. install.sh/ps1 install unsloth + unsloth-zoo with --no-deps 2. SKIP_STUDIO_BASE=0 so install_python_stack.py's NO_TORCH branch runs 3. install_python_stack.py installs no-torch-runtime.txt	2026-03-27 03:58:51 -07:00
Lee Jackson	0233fe7f9c	studio: setup log styling (#4494) ... * refactor(studio): unify setup terminal output style and add verbose setup mode * studio(windows): align setup.ps1 banner/steps with setup.sh (ANSI, verbose) * studio(setup): revert nvcc path reordering to match main * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio(setup): restore fail-fast llama.cpp setup flow * studio(banner): use IPv6 loopback URL when binding :: or ::1 * Fix IPv6 URL bracketing, try_quiet stderr, _step label clamp - Bracket IPv6 display_host in external_url to produce clickable URLs - Redirect try_quiet failure log to stderr instead of stdout - Clamp _step label to column width to prevent negative padding * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add sandbox integration tests for PR #4494 UX fixes Simulation harness (tests/simulate_pr4494.py) creates an isolated uv venv, copies the real source files into it, and runs subprocess tests for all three fixes with visual before/after demos and edge cases. Standalone bash test (tests/test_try_quiet.sh) validates try_quiet stderr redirect across 8 scenarios including broken-version contrast. 39 integration tests total (14 IPv6 + 15 try_quiet + 10 _step), all existing 75 unit tests still pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Truncate step() labels in setup.sh to match PS1 and Python The %-15s printf format pads short labels but does not truncate long ones. Change to %-15.15s so labels wider than 15 chars are clipped, matching the PowerShell .Substring(0,15) and Python label[:15] logic. * Remove sandbox integration tests from PR These test files are not part of the styling fix and should not ship with this PR. * Show error output on failure instead of suppressing it - install_python_stack.py: restore _red for patch_package_file warnings (was downgraded to _dim) - setup.ps1: capture winget output and show on failure for CUDA, Node, Python, and OpenSSL installs (was piped to Out-Null) - setup.ps1: always show git pull failure warning, not just in verbose mode * Show winget error output for Git and CMake installs on failure Same capture-and-print-on-failure pattern already used for Node, Python, CUDA, and OpenSSL winget installs. * fix: preserve stderr for _run_quiet error messages in setup.sh The step() helper writes to stdout, but _run_quiet's error header was originally sent to stderr (>&2). Without the redirect, callers that separate stdout/stderr would miss the failure headline while still seeing the log body on stderr. Add >&2 to both step calls inside _run_quiet to match main's behavior. * feat: add --verbose flag to setup and update commands Wire UNSLOTH_VERBOSE=1 through _run_setup_script() so that 'unsloth studio update --verbose' (and the deprecated 'setup') passes the flag to setup.sh / setup.ps1 / install_python_stack.py. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-03-27 03:12:48 -07:00
Daniel Han	3c9f0ed149	fix: use unsloth[huggingfacenotorch] instead of --no-deps in no-torch mode (#4647) ... The previous --no-deps approach skipped ALL dependencies, not just torch. This left safetensors, transformers, datasets, accelerate, etc. missing, causing PackageNotFoundError at runtime. Fix: in no-torch mode, install unsloth[huggingfacenotorch] (which pulls all runtime deps except torch), then install unsloth-zoo with --no-deps (since zoo's published metadata still declares torch as a hard dep). This gives a working no-torch environment with all non-torch packages. Applied to all three installer files: install.sh, install.ps1, and studio/install_python_stack.py.	2026-03-27 02:38:11 -07:00
Daniel Han	e9ac785346	fix: install.sh Mac Intel compatibility + Studio no-torch support (#4624) ... * fix: install.sh Mac Intel compatibility + Studio no-torch support (#4621) On Intel Macs (x86_64), PyTorch has no wheels for torch >= 2.3, so the installer crashes. Even when torch is absent, Studio crashes on startup because two files have bare top-level torch imports. Studio's GGUF inference (llama.cpp) does not need PyTorch. Training and HF-inference already isolate torch to subprocesses. Only 2 files in the server startup chain had top-level torch imports preventing startup. Changes: - install.sh: detect architecture, default to Python 3.12 on Intel Mac, skip torch install, add Python 3.13.8 guard for arm64, pass UNSLOTH_NO_TORCH env var to setup.sh - data_collators.py: remove unused `import torch` (no torch.* refs) - chat_templates.py: lazy-import IterableDataset into function bodies - install_python_stack.py: add IS_MACOS/NO_TORCH constants, skip torch-dependent packages, skip overrides.txt, skip triton on macOS No existing working flow changes. Linux/WSL and macOS arm64 behavior is identical. * tests: add test suite for Mac Intel compat + no-torch mode Shell tests (test_mac_intel_compat.sh): - version_ge edge cases (9 tests) - Architecture detection for Darwin x86_64/arm64, Linux x86_64/aarch64 - get_torch_index_url returns cpu on simulated Darwin - UNSLOTH_NO_TORCH propagation to both setup.sh branches Python unit tests (test_no_torch_filtering.py): - _filter_requirements with NO_TORCH_SKIP_PACKAGES - NO_TORCH env var parsing (true/1/TRUE/false/0/unset) - IS_MACOS constant check - Overrides skip and triton macOS skip guards Python import tests (test_studio_import_no_torch.py): - data_collators.py loads in isolated no-torch venv - chat_templates.py has no top-level torch imports - Negative control confirms import torch fails without torch * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests: add E2E sandbox tests for Mac Intel no-torch mode Replace static/synthetic test stubs with real sandbox tests: - Shell: E2E uv venv creation at Python 3.12, mock uv shim to verify torch install is skipped when MAC_INTEL=true, dynamic env propagation test for UNSLOTH_NO_TORCH in both local and non-local install paths - Python filtering: test real extras.txt and extras-no-deps.txt with NO_TORCH_SKIP_PACKAGES, subprocess mock of install_python_stack() for 5 platform configs (NO_TORCH+macOS, Windows+NO_TORCH, normal Linux, Windows-only, macOS-only), VCS URL and env marker edge cases - Python imports: parametrized Python 3.12+3.13 venv fixture, dataclass instantiation for all 3 collator classes, chat_templates.py exec with stubs, negative controls proving import torch and torchao install fail in no-torch venvs 91 total tests, all passing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: address reviewer findings for Intel Mac no-torch mode P1 fixes: - Auto-infer NO_TORCH in install_python_stack.py via platform.machine() so `unsloth studio update` preserves GGUF-only mode without needing the UNSLOTH_NO_TORCH env var (6/10 reviewers) - Add openai-whisper and transformers-cfg to NO_TORCH_SKIP_PACKAGES since both have unconditional torch dependencies (4/10 reviewers) - Skip unsloth-zoo on Intel Mac --local installs (depends on torch) in both migrated and fresh install paths (1/10) - Recreate stale 3.13 venvs as 3.12 on Intel Mac re-runs (1/10) - Detect Apple Silicon under Rosetta via sysctl hw.optional.arm64 and warn user to use native arm64 terminal (1/10) P2 fixes: - Wire new test files into tests/run_all.sh (4/10 reviewers) - Add update-path tests (skip_base=False) for Intel Mac - Add _infer_no_torch tests for platform auto-detection P3 fixes: - Fix macOS progress bar total (triton step skipped but was counted) - Fix temp file leak when Windows + NO_TORCH filters stack All tests pass: 30 shell, 66 Python (96 total). * feat: add --python override flag to install.sh Lets users force a specific Python version, e.g. ./install.sh --python 3.12. Addresses M2 Mac users whose systems resolve to a problematic 3.13.x patch. When --python is set, the Intel Mac stale-venv guard and 3.13.8 auto-downgrade are skipped so the user's choice is respected. * tests: add comprehensive E2E sandbox tests for no-torch mode Add test_e2e_no_torch_sandbox.py with 7 test groups (43 tests total) covering the full no-torch import chain, edge cases, and install logic: - Group 1: BEFORE vs AFTER import chain comparison (proves the bug existed and the fix works by synthetically prepending top-level torch imports) - Group 2: Dataclass instantiation without torch - Group 3: Edge cases with broken/fake torch modules on sys.path - Group 4: Hardware detection fallback to CPU without torch - Group 5: install.sh flag parsing, version resolution, arch detection - Group 6: install_python_stack.py NO_TORCH filtering - Group 7: Live server startup without torch (marked @server, skipped when studio venv is unavailable) All 43 tests pass on both Python 3.12 and 3.13 isolated venvs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: add --no-torch flag to install.sh/ps1, fix lazy import bug in dataset formatting - Fix chat_templates.py: narrow torch IterableDataset import into inner try/except ImportError so dataset.map() works without torch installed - Fix format_conversion.py: same lazy import fix for convert_chatml_to_alpaca and convert_alpaca_to_chatml - Add --no-torch flag to install.sh with unified SKIP_TORCH variable (driven by --no-torch flag OR MAC_INTEL auto-detection) - Add --no-torch flag to install.ps1 with $SkipTorch variable - Print CPU hint when no GPU detected and --no-torch not set - Replace MAC_INTEL guards with SKIP_TORCH in torch install sections - Update shell tests (40 pass) and Python tests (90 pass) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: address reviewer findings for --no-torch installer paths - Fix migrated-env branch in install.sh and install.ps1: check SKIP_TORCH first, then branch on STUDIO_LOCAL_INSTALL. Previously SKIP_TORCH+non-local fell into else and installed unsloth-zoo (which depends on torch), defeating --no-torch mode. - Fix $env:UNSLOTH_NO_TORCH leak in install.ps1: always set to "true" or "false" instead of only setting on the true branch. Prevents stale no-torch state from leaking across runs in the same PS session. - Fix install_python_stack.py update path: add NO_TORCH guard around base.txt install so unsloth studio update does not reinstall unsloth-zoo (which depends on torch) in no-torch mode. * fix: install unsloth + unsloth-zoo with --no-deps in no-torch mode Instead of skipping unsloth-zoo entirely (which breaks unsloth's dependency on it), install both packages with --no-deps so they are present but torch is not pulled in transitively. Applied consistently across all no-torch paths: migrated-env, fresh-local, fresh-non-local in install.sh, install.ps1, and install_python_stack.py. * chore: temporarily remove test files (will be added in a follow-up) * refactor: deduplicate SKIP_TORCH conditional branches in installers Collapse if/else blocks that differ only by --no-deps into a single branch with a conditional flag variable. Applied to migrated-env and fresh-local paths in install.sh, install.ps1, and install_python_stack.py. * fix: apply --no-deps to fresh non-local --no-torch install path The non-local else branch was missing $_no_deps_arg/$noDepsArg, so uv pip install unsloth would resolve torch from PyPI metadata (the published unsloth package still declares torch as a hard dep). Now --no-deps is applied consistently to all SKIP_TORCH code paths. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-03-27 02:09:21 -07:00
Daniel Han	baabfa0a6e	Fix Colab huggingface-hub conflict, ensurepip fallback, bump to 2026.3.14 (#4603) ... * Fix Colab huggingface-hub conflict, ensurepip fallback, bump to 2026.3.14 - colab.py / setup.sh: relax == pins to >= when installing studio.txt on Colab so huggingface-hub does not clobber Colab's bundled version (breaks transformers is_offline_mode import) - install_python_stack.py: when uv is unavailable and pip is missing (uv-created venvs), bootstrap via ensurepip before attempting upgrade - Bump version to 2026.3.14 - Bump installer min version pins to 2026.3.14 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-03-25 09:38:02 -07:00
Roland Tannous	19e9c60a8e	Consolidate dual venvs and separate install from update (#4530) ... * refactor: consolidate dual venvs into single ~/.unsloth/studio/unsloth_studio * refactor: separate install.sh (first-time) from setup.sh (smart update with PyPI version check) * fix: install.sh calls setup.sh directly, keep both setup and update CLI commands * fix: use importlib.resources.files() directly without _path attribute * fix: bootstrap uv before pip upgrade to handle uv venvs without pip * fix: frontend 404 when launched via CLI, add global symlink to ~/.local/bin * feat: add --local flag to install.sh and unsloth studio update for branch testing * fix: resolve repo root from script location for --local installs * feat: add --package flag to install.sh for testing with custom package names * feat: add --package flag to unsloth studio update * fix: always nuke venv in install.sh for clean installs * revert: remove Windows changes, will handle in separate PR * fix: error when --package is passed without an argument * revert: restore Windows scripts to current main * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: always explicitly set STUDIO_LOCAL_INSTALL and STUDIO_PACKAGE_NAME env vars * fix: pass explicit STUDIO_LOCAL_REPO env var for --local installs * fix: align banner box for Setup vs Update labels * deprecate: hide 'unsloth studio setup' command, point users to update/install.sh * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: check stdout not stdin for auto-launch detection (curl pipe fix) * fix: update install URL to unsloth.ai/install.sh * fix: update install.sh usage comments to unsloth.ai/install.sh * fix: use --upgrade-package for base deps to preserve existing torch/CUDA installs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: --local install now also installs unsloth-zoo via base.txt before editable overlay * fix: don't skip base packages for --local installs (editable needs unsloth-zoo) * refactor: move --local full dep install to install.sh, keep SKIP_STUDIO_BASE for all paths * feat: add migration support for old .venv and CWD-based installs in setup.sh * Revert "feat: add migration support for old .venv and CWD-based installs in setup.sh" This reverts commit 301291d002. * feat: migrate old .venv layout in install.sh instead of always nuking * feat: validate old .venv with torch CUDA test before migration, recovery message on launch failure * fix: try CUDA then fall back to CPU for migration validation * fix: upgrade unsloth/unsloth-zoo with --reinstall-package on migration to preserve torch * remove: delete unused unsloth ui command (use unsloth studio instead) * Fix Windows venv path mismatch between install.ps1, setup.ps1, and studio.py install.ps1 was creating the venv CWD-relative ($VenvName = "unsloth_studio"), setup.ps1 was using an absolute path to ".unsloth\studio\.venv", and studio.py looks for ".unsloth\studio\unsloth_studio". All three paths were different, so the Windows installer would never produce a working Studio setup. install.ps1: - Use absolute $StudioHome + $VenvDir matching the Linux install.sh layout - Add 3-way migration: old .venv at STUDIO_HOME, CWD-relative ~/unsloth_studio from the previous install.ps1, or fresh creation with torch validation - For migrated envs, upgrade unsloth while preserving existing torch/CUDA wheels - Set SKIP_STUDIO_BASE=1 before calling setup.ps1 (matches install.sh behavior) - Fix launch instructions to use the absolute venv path setup.ps1: - Change $VenvDir from ".unsloth\studio\.venv" to ".unsloth\studio\unsloth_studio" - Add SKIP_STUDIO_BASE guard: error out if venv is missing when called from install.ps1 (which should have already created it) - Differentiate "Setup" vs "Update" in banners based on SKIP_STUDIO_BASE * setup.ps1: unconditionally error if venv missing, matching setup.sh setup.sh always errors out if the venv does not exist (line 224-228), telling the user to run install.sh first. setup.ps1 was conditionally creating a bare venv with python -m venv when SKIP_STUDIO_BASE was not set, which would produce an empty venv with no torch or unsloth. Now setup.ps1 matches setup.sh: always error, always point to install.ps1. * Fix --torch-backend=auto CPU solver dead-end on Linux, macOS, and Windows On CPU-only machines, `uv pip install unsloth --torch-backend=auto` falls back to unsloth==2024.8 because the CPU solver cannot satisfy newer unsloth's dependencies. install.ps1 already solved this with a two-step approach; this applies the same fix to install.sh and install_python_stack.py. install.sh: add get_torch_index_url() that detects GPU via nvidia-smi and maps CUDA versions to PyTorch index URLs (matching install.ps1's Get-TorchIndexUrl). Fresh installs now install torch first via explicit --index-url, then install unsloth with --upgrade-package to preserve the pre-installed torch. All 5 --torch-backend=auto removed from primary paths. install.ps1: add fallback else-branch when TorchIndexUrl is empty, using --torch-backend=auto as last resort (matching install.sh). install_python_stack.py: remove unconditional --torch-backend=auto from _build_uv_cmd. Torch is pre-installed by install.sh/setup.ps1 by the time this runs. Callers that need it can set UV_TORCH_BACKEND. Both install.sh and install.ps1 now share the same three-branch logic: migrated env (upgrade-package only), normal (torch-first + index-url), and fallback (--torch-backend=auto if URL detection fails). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use --reinstall-package for migrated envs on both Linux and Windows For migrated environments (moved from legacy venv location), --reinstall-package is better than --upgrade-package because it forces a clean reinstall even if the same version is already installed. This ensures proper .dist-info and .pyc state in the new venv location. --upgrade-package remains correct for the fresh install path where torch is already installed and we just want to add unsloth without re-resolving torch. * Address review findings: portability, parity, and stale comments - Replace grep -oP (GNU Perl regex) with POSIX sed in get_torch_index_url() so the script works on BSD grep (macOS is already guarded by the Darwin early-return, but Alpine/BusyBox would silently get the wrong CUDA tag) - Add LC_ALL=C before nvidia-smi invocation to prevent locale-dependent output parsing issues - Add warning on stderr when nvidia-smi output is unparseable, matching install.ps1's [WARN] message - Add explicit unsloth-zoo positional arg to install.ps1 migrated path, matching install.sh (--reinstall-package alone won't install it if it was never present in the migrated env) - Fix stale comment in install_python_stack.py line 392 that still claimed --torch-backend=auto is added by _build_uv_cmd - Add sed to test tools directory (function now uses sed instead of grep) * Add --index-url to migrated env path to prevent CPU torch resolution The migrated path runs uv pip install with --reinstall-package for unsloth/unsloth-zoo. While uv should keep existing torch as satisfied, the resolver could still re-resolve torch as a transitive dependency. Without --index-url pointing at the correct CUDA wheel index, the resolver would fall back to plain PyPI and potentially pull CPU-only torch. Adding --index-url $TORCH_INDEX_URL ensures CUDA wheels are available if the resolver needs them. Applied to both install.sh and install.ps1. * Revert --index-url on migrated env path The original install.ps1 on main already handles the migrated path without --index-url and it works correctly. --reinstall-package only forces reinstall of the named packages while uv keeps existing torch as satisfied. No need for the extra flag. * Fix unsloth studio update --local not installing local checkout studio.py sets STUDIO_LOCAL_REPO when --local is passed, but install_python_stack.py never read it. The update path always installed from PyPI regardless of the --local flag. Add a local_repo branch that first updates deps from base.txt (with --upgrade-package to preserve torch), then overlays the local checkout as an editable install with --no-deps. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-03-25 05:24:21 -07:00
Krishna Chaitanya	9b989ee898	fix: prevent UnicodeEncodeError on Windows CP1252 consoles in studio setup (#4563) ... * fix: prevent UnicodeEncodeError on Windows CP1252 consoles in studio setup On Windows, `unsloth studio setup` crashes with a UnicodeEncodeError when install_python_stack.py tries to print Unicode status glyphs (✅, ❌, ⚠️) to a console that uses a legacy code page like CP1252. Add a _safe_print() helper that catches UnicodeEncodeError and gracefully degrades emoji to ASCII equivalents ([OK], [FAIL], [!]). Replace all print() calls that emit Unicode glyphs with _safe_print(). Fixes #4509 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Replace Unicode dashes with ASCII in install_python_stack.py Box-drawing (U+2500) and em dash (U+2014) chars in section dividers and comments are themselves not representable on CP1252 -- replace with plain ASCII dashes for consistency with the fix. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-03-24 22:04:09 -07:00
Daniel Han	1f12ba16df	Combine studio setup fixes: frontend caching, venv isolation, Windows CPU support (#4413) ... * Allow Windows setup to complete without NVIDIA GPU setup.ps1 previously hard-exited if nvidia-smi was not found, blocking setup entirely on CPU-only or non-NVIDIA machines. The backend already supports CPU and MLX (Apple Silicon) in chat-only GGUF mode, and the Linux/Mac setup.sh handles missing GPUs gracefully. Changes: - Convert the GPU check from a hard exit to a warning - Guard CUDA toolkit installation behind $HasNvidiaSmi - Install CPU-only PyTorch when no GPU is detected - Build llama.cpp without CUDA flags when no GPU is present - Update doc comment to reflect CPU support * Cache frontend build across setup runs Skip the frontend npm install + build if frontend/dist already exists. Previously setup.ps1 nuked node_modules and package-lock.json on every run, and both scripts always rebuilt even when dist/ was already present. On a git clone editable install, the first setup run still builds the frontend as before. Subsequent runs skip it, saving several minutes. To force a rebuild, delete frontend/dist and re-run setup. * Show pip progress for PyTorch download on Windows The torch CUDA wheel is ~2.8 GB and the CPU wheel is ~300 MB. With | Out-Null suppressing all output, the install appeared completely frozen with no feedback. Remove | Out-Null for the torch install lines so pip's download progress bar is visible. Add a size hint so users know the download is expected to take a while. Also moves the Triton success message inside the GPU branch so it only prints when Triton was actually installed. * Guard CUDA env re-sanitization behind GPU check in llama.cpp build The CUDA_PATH re-sanitization block (lines 1020-1033) references $CudaToolkitRoot which is only set when $HasNvidiaSmi is true and the CUDA Toolkit section runs. On CPU-only machines, $CudaToolkitRoot is null, causing Split-Path to throw: Split-Path : Cannot bind argument to parameter 'Path' because it is null. Wrap the entire block in `if ($HasNvidiaSmi -and $CudaToolkitRoot)`. * Rebuild frontend when source files are newer than dist/ Instead of only checking if dist/ exists, compare source file timestamps against the dist/ directory. If any file in frontend/src/ is newer than dist/, trigger a rebuild. This handles the case where a developer pulls new frontend changes and re-runs setup -- stale assets get rebuilt automatically. * Fix cmake not found on Windows after winget install Two issues fixed: 1. After winget installs cmake, Refresh-Environment may not pick up the new PATH entry (MSI PATH changes sometimes need a new shell). Added a fallback that probes cmake's default install locations (Program Files, LocalAppData) and adds the directory to PATH explicitly if found. 2. If cmake is still unavailable when the llama.cpp build starts (e.g. winget failed silently or PATH was not updated), the build now skips gracefully with a [SKIP] warning instead of crashing with "cmake : The term 'cmake' is not recognized". * Fix frontend rebuild detection and decouple oxc-validator install Address review feedback: - Check entire frontend/ directory for changes, not just src/. The build also depends on package.json, vite.config.ts, tailwind.config.ts, public/, and other config files. A change to any of these now triggers a rebuild. - Move oxc-validator npm install outside the frontend build gate in setup.sh so it always runs on setup, matching setup.ps1 which already had it outside the gate. * Show cmake errors on failure and retry CUDA VS integration with elevation Two fixes for issue #4405 (Windows setup fails at cmake configure): 1. cmake configure: capture output and display it on failure instead of piping to Out-Null. When the error mentions "No CUDA toolset found", print a hint about the CUDA VS integration files. 2. CUDA VS integration copy: when the direct Copy-Item fails (needs admin access to write to Program Files), retry with Start-Process -Verb RunAs to prompt for elevation. This is the root cause of the "No CUDA toolset found" cmake failure -- the .targets files that let MSBuild compile .cu files are missing from the VS BuildCustomizations directory. * Address reviewer feedback: cmake PATH persistence, stale cache, torch error check 1. Persist cmake PATH to user registry so Refresh-Environment cannot drop it later in the same setup run. Previously the process-only PATH addition at phase 1 could vanish when Refresh-Environment rebuilt PATH from registry during phase 2/3 installs. 2. Clean stale CMake cache before configure. If a previous run built with CUDA and the user reruns without a GPU (or vice versa), the cached GGML_CUDA value would persist. Now the build dir is removed before configure. 3. Explicitly set -DGGML_CUDA=OFF for CPU-only builds instead of just omitting CUDA flags. This prevents cmake from auto-detecting a partial CUDA installation. 4. Fix CUDA cmake flag indentation -- was misaligned from the original PR, now consistently indented inside the if/else block. 5. Fail hard if pip install torch returns a non-zero exit code instead of silently continuing with a broken environment. * Remove extra CUDA cmake flags to align Windows with Linux build Drop GGML_CUDA_FA_ALL_QUANTS, GGML_CUDA_F16, GGML_CUDA_GRAPHS, GGML_CUDA_FORCE_CUBLAS, and GGML_CUDA_PEER_MAX_BATCH_SIZE flags. The Linux build in setup.sh only sets GGML_CUDA=ON and lets llama.cpp use its defaults for everything else. Keep Windows consistent. * Address reviewer round 2: GPU probe fallback, Triton check, stale binary rebuild 1. GPU detection: fallback to default nvidia-smi install locations (Program Files\NVIDIA Corporation\NVSMI, System32) when nvidia-smi is not on PATH. Prevents silent CPU-only provisioning on machines that have a GPU but a broken PATH. 2. Triton: check $LASTEXITCODE after pip install and print [WARN] on failure instead of unconditional [OK]. 3. Stale llama-server: check CMakeCache.txt for GGML_CUDA setting and rebuild if the existing binary does not match the current GPU mode (e.g. CUDA binary on a now-CPU-only rerun, or vice versa). * Fix frontend rebuild detection and npm dependency issues Addresses reviewer feedback on the frontend caching logic: 1. setup.sh: Fix broken find command that caused exit under pipefail. The piped `find | xargs find -newer` had paths after the expression which GNU find rejects. Replaced with a simpler `find -maxdepth 1 -type f -newer dist/` that checks ALL top-level files (catches index.html, bun.lock, etc. that the extension allowlist missed). 2. setup.sh: Guard oxc-validator npm install behind `command -v npm` check. When the frontend build is skipped (dist/ is cached), Node bootstrap is also skipped, so npm may not be available. 3. setup.ps1: Replace Get-ChildItem -Include with explicit path probing for src/ and public/. PowerShell's -Include without a trailing wildcard silently returns nothing, so src/public changes were never detected. Also check ALL top-level files instead of just .json/.ts/.js/.mjs extensions. * Fix studio setup: venv isolation, centralized .venv_t5, uv targeting - All platforms (including Colab) now create ~/.unsloth/studio/.venv with --without-pip fallback for broken ensurepip environments - Add --python sys.executable to uv pip install in install_python_stack.py so uv targets the correct venv instead of system Python - Centralize .venv_t5 bootstrap in transformers_version.py with proper validation (checks required packages exist, not just non-empty dir) - Replace ~150 lines of duplicated install code across 3 worker files with calls to the shared _ensure_venv_t5_exists() helper - Use uv-if-present with pip fallback; do not install uv at runtime - Add site.addsitedir() shim in colab.py so notebook cells can import studio packages from the venv without system-Python double-install - Update .venv_t5 packages: huggingface_hub 1.3.0->1.7.1, add hf_xet - Bump transformers pin 4.57.1->4.57.6 in requirements + constraints - Add Fast-Install helper to setup.ps1 with uv+pip fallback - Keep Colab-specific completion banner in setup.sh * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix nvidia-smi PATH persistence and cmake requirement for CPU-only 1. Store nvidia-smi as an absolute path ($NvidiaSmiExe) on first detection. All later calls (Get-CudaComputeCapability, Get-PytorchCudaTag, CUDA toolkit detection) use this absolute path instead of relying on PATH. This survives Refresh-Environment which rebuilds PATH from the registry and drops process-only additions. 2. Make cmake fatal for CPU-only installs. CPU-only machines depend entirely on llama-server for GGUF chat mode, so reporting "Setup Complete!" without it is misleading. GPU machines can still skip the llama-server build since they have other inference paths. * Fix broken frontend freshness detection in setup scripts - setup.sh: Replace broken `find | xargs find -newer` pipeline with single `find ... -newer` call. The old pipeline produced "paths must precede expression" errors (silently suppressed by 2>/dev/null), causing top-level config changes to never trigger a rebuild. - setup.sh: Add `command -v npm` guard to oxc-validator block so it does not fail when Node was not installed (build-skip path). - setup.ps1: Replace `Get-ChildItem -Include` (unreliable without -Recurse on PS 5.1) with explicit directory paths for src/ and public/ scanning. - Both: Add *.html to tracked file patterns so index.html (Vite entry point) changes trigger a rebuild. - Both: Use -print -quit instead of piping to head -1 for efficiency. * Fix bugs found during review of PRs #4404, #4400, #4399 - setup.sh: Add || true guard to find command that checks frontend/src and frontend/public dirs, preventing script abort under set -euo pipefail when either directory is missing - colab.py: Use sys.path.insert(0, ...) instead of site.addsitedir() so Studio venv packages take priority over system copies. Add warning when venv is missing instead of silently failing. - transformers_version.py: _venv_t5_is_valid() now checks installed package versions via .dist-info metadata, not just directory presence. Prevents false positives from stale or wrong-version packages. - transformers_version.py: _install_to_venv_t5() now passes --upgrade so pip replaces existing stale packages in the target directory. - setup.ps1: CPU-only PyTorch install uses --index-url for cpu wheel and all install commands use Fast-Install (uv with pip fallback). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix _venv_t5_is_valid dist-info loop exiting after first directory Remove premature break that caused the loop over .dist-info directories to exit after the first match even if it had no METADATA file. Now continues iterating until a valid METADATA is found or all dirs are exhausted. * Capture error output on failure instead of discarding with Out-Null setup.ps1: 6 locations changed from `| Out-Null` to `| Out-String` with output shown on failure -- PyTorch GPU/CPU install, Triton install, venv_t5 package loop, cmake llama-server and llama-quantize builds. transformers_version.py: clean stale .venv_t5 directory before reinstall when validation detects missing or version-mismatched packages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix ModuleNotFoundError when CLI imports studio.backend.core The backend uses bare "from utils.*" imports everywhere, relying on backend/ being on sys.path. Workers and routes add it at startup, but the CLI imports studio.backend.core as a package -- backend/ was never added. Add sys.path setup at the top of core/__init__.py so lazy imports resolve correctly regardless of entry point. Fixes: unsloth inference unsloth/Qwen3-8B "who are you" crashing with "No module named 'utils'" * Fix frontend freshness check to detect all top-level file changes The extension allowlist (*.json, *.ts, *.js, *.mjs, *.html) missed files like bun.lock, so lockfile-only dependency changes could skip the frontend rebuild. Check all top-level files instead. * Add tiktoken to .venv_t5 for Qwen-family tokenizers Qwen models use tiktoken-based tokenizers which fail when routed through the transformers 5.x overlay without tiktoken installed. Add it to the setup scripts (with deps for Windows) and runtime fallback list. Integrates PR #4418. * Fix tiktoken crash in _venv_t5_is_valid and stray brace in setup.ps1 _venv_t5_is_valid() crashed with ValueError on unpinned packages like "tiktoken" (no ==version). Handle by splitting safely and skipping version check for unpinned packages (existence check only). Also remove stray closing brace in setup.ps1 tiktoken install block. --------- Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-03-18 03:52:25 -07:00
Leo Borcherding	df98569f12	studio: improve Colab notebook, redesign ready popup, and clean up install output (#4339) ... * Removing .precommit config * edited colab comments * studio: update Unsloth_Studio_Colab.ipynb * studio: update Unsloth_Studio_Colab.ipynb * studio: add Colab T4 GPU metadata to force T4 instance * style: update colab popup to black/white theme with gem icon and play button * feat: center landscape image in colab notebook * style: shrink popup to fit content, truncate URL display * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: center landscape image in colab notebook * feat: use GitHub raw URL for studio landscape image in notebook * chore: update colab notebook * feat: add studio landscape colab display image and update notebook * feat: update notebook with studio landscape image * style: remove colors, add progress bar, add VERBOSE flag to install output * docs: add comments explaining VERBOSE flag and progress bar * chore: update colab notebook * fix: define VERBOSE, _STEP, _TOTAL at module level to fix NameError --------- Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-03-16 21:39:25 -07:00
pre-commit-ci[bot]	b20b3b80df	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	2026-03-14 00:54:09 -07:00
Daniel Han	4b6f5c76c1	studio: probe-based --system detection for uv ... Replace _in_virtualenv() heuristic with a runtime probe. At bootstrap time, try a dry-run uv install without --system. If that fails (exit code 2, "No virtual environment found"), retry with --system to confirm it works. This handles all environments correctly: venvs, Colab (system Python), local machines, containers.	2026-03-14 00:54:09 -07:00
Daniel Han	9b7eaf8f0c	studio: make uv optional + fix --system for Colab ... Three fixes based on review: 1. Make uv truly optional: _bootstrap_uv() now only checks if uv is already on PATH. It no longer tries to pip install uv. If uv is not present, pip is used with zero changes to behavior. 2. Add --system flag for Colab: on Colab there is no venv (packages install into system Python). uv requires --system in this case, otherwise it errors with "No virtual environment found". Added _in_virtualenv() check that detects VIRTUAL_ENV, sys.real_prefix, or sys.base_prefix != sys.prefix. 3. Fix label printed twice on uv fallback: when uv fails and falls back to pip, the label now says "(pip)" to distinguish from the initial uv attempt, instead of printing the same label twice. Tested: - venv path: no --system flag, uv installs correctly - no-venv path (Colab sim): --system flag added automatically - full unsloth studio setup + training run (Llama-3.2-1B, 10 steps)	2026-03-14 00:54:09 -07:00
Daniel Han	a7a66a66b9	studio: address review feedback ... install_python_stack.py: - Print uv error output on failure for debuggability - Refactor pip_install() to use early return after uv success, removing duplicated pip command path setup.sh: - Guard nvidia-smi command substitution with || true so it does not abort the script under set -euo pipefail when nvidia-smi fails (e.g., containerized environments, driver quirks) - Read all GPU compute capabilities and deduplicate, so mixed-GPU hosts get kernels built for all present architectures instead of only the first GPU	2026-03-14 00:54:09 -07:00