mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-24 08:13:59 +00:00
125 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
karan yadav
|
dc26127a42
|
Studio: propagate required backend version to repair pipeline (#8610) (#8670)
* Studio: propagate required backend version to repair pipeline (#8610) Fix second-launch infinite repair loop when installed backend version is outdated (#8610). When the Desktop App launches with an installed managed venv whose version is older than expected_backend_version() (e.g. 2026.8.4 < 2026.8.15), preflight flags the install as ManagedStale with desktop_backend_version_outdated. On auto-repair, unsloth studio update ran setup.sh/setup.ps1 from the old venv, which skipped python dependency installation because INSTALLED_VER == LATEST_VER on PyPI or PyPI timeout, leaving the venv unchanged. The installer fallback (install.sh/install.ps1) also lacked version floor pins on standard fresh paths, locking the user in a permanent repair error loop. Key changes: - Pass UNSLOTH_DESKTOP_BACKEND_VERSION from Tauri (update.rs & install.rs) to child process environments. - Force Python dependency pass in setup.sh & setup.ps1 when UNSLOTH_DESKTOP_BACKEND_VERSION is set and INSTALLED_VER < UNSLOTH_DESKTOP_BACKEND_VERSION. - Apply UNSLOTH_DESKTOP_BACKEND_VERSION floor constraint when updating core packages in install_python_stack.py. - Ensure standard fresh install paths in install.sh and install.ps1 use "unsloth>=2026.8.15". - Bump MIN_DESKTOP_BACKEND_VERSION in preflight/version.rs to "2026.8.15". - Add shell and Python unit tests for the fast-path escape and desktop backend version constraint. Fixes #8610 * Narrow desktop repair to version propagation * Handle version suffixes in repair fallback --------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> |
||
Daniel Han
|
0fb21e8a89
|
Revert "Studio: verify the installed version after the update pass (#8505)" (#8824)
This reverts commit
|
||
Nilay
|
fe79a5bbc2
|
Studio: verify the installed version after the update pass (#8505)
* Studio: verify installed version after the update pass * Studio update verify: PEP 440 compare, Requires-Python escape, LATEST_VER init * Studio update verify: accept only the newest interpreter-compatible release * Studio update verify: reuse the fetched PyPI response in the ps1 probe * Studio update verify: base64 the probe temp path (apostrophe-safe) * Studio update verify: filter compatible releases by wheel tags / sdist * Studio update verify: LiteralPath for the temp release table * Studio update verify: skip the strict check when a custom package index is active * Studio update verify: warn on older-but-successful, fail only when missing * Studio update verify: keep missing check on custom indexes, PEP 503 probe names * Studio update verify: run the missing-package probe even when PyPI is unreachable * Studio update verify: isolate probes from PYTHONPATH, generic kept message * Studio update verify: drop cwd from ps1 probe paths, non-fatal temp-table failures * Studio update verify: require loadable payload, keep venv site-packages in scrub * Studio update verify: drop the success manifest when the package is missing * Studio update verify: retry, invalidate, and surface a stuck success manifest * Studio update verify: keep the payload probe clear of the installer-helper guard The post-update probe matched a package initializer by its joined filename. The installer-helper guard in tests/test_installer_interactive_prompts.py scans every installer for filename-shaped tokens and resolves each one against the script's own directory, so that literal resolved to studio/__init__.py and the guard reported it as a helper the installers invoke but nobody scans. Parity and the CPU repo tests have been red on that since the probe landed. Match on stem and suffix instead. Same predicate, no filename token to resolve, and studio/__init__.py stays out of SCANNED_SCRIPTS where it does not belong. Verified on 3.10, 3.11 and 3.13 against fabricated venv trees: a dist-info whose payload was deleted still reports __MISSING__ through this branch on 3.10 and 3.11, and a RECORD-only install whose payload is intact still confirms. From 3.12 on, Distribution.files applies skip_missing_files, so this branch never sees a deleted path there and the top_level.txt branch above carries the check. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Nilay
|
1fb184deb4
|
Studio: self-repair sidecars whose extensions were built for another Python (#8705)
* Studio: detect ABI-mismatched sidecar extensions so self-repair fires after a Python upgrade * Studio: scan extension basenames and flag stable-ABI binaries on free-threaded builds Match the version tag on the file basename rather than the whole RECORD path, so a directory component carrying a wheel-style tag does not wipe the sidecar. Report .abi3 binaries only under a free-threaded interpreter, which cannot load them. Fix the stale-tag pick in the detection test so it stays stale under cp313t. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
9c69529705
|
Windows: stop compiling C# for colour on hosts that already render it (#8767)
* Windows: stop compiling C# for colour on hosts that already render it Enable-StudioVirtualTerminal is called unconditionally by install.ps1 and studio/setup.ps1, and it reaches Add-Type, which runs the C# compiler and drops a source file in %TEMP% on every install. An ANY.RUN submission of the shipped 0.1.701-beta Windows build captured that as two csc.exe processes and a "Suspicious source code drop". Under Windows Terminal there is nothing to enable: it always renders VT. Ask for that case first and skip the compile. All three conjuncts are load-bearing. WT_SESSION is inherited, so the desktop app's console-less spawn carries it into a pipe, and without the redirect check the Studio log panel would fill with escape sequences. $Host.UI.SupportsVirtualTerminal reports what the host CAN render, not whether this output buffer has ENABLE_VIRTUAL_TERMINAL_PROCESSING set, so it cannot carry the decision alone either. Nothing else moves. Outside this one function both scripts are identical to main line for line, and $script:StudioVtOk is the only value the function feeds, so the same verdict means the same bytes. The other compile stays. UnslothStudioFinalPathV2 feeds Get-StudioRuntimePathHash, which Python derives the same mutex name from byte for byte, so a managed fast path differing on case or an 8.3 name would let two installers each believe they hold the install lock. Guards: test_installer_av_shapes.py fails if the compile moves back ahead of the host check or loses a conjunct, and test_windows_setup_output_encoding.py runs this function beside the one it replaces on a real Windows host, with WT_SESSION forced set and forced empty, asserting the same verdict and the same banner bytes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Decide the redirected case without the compiler, not the Windows Terminal one Review caught that the WT_SESSION test was unsound. WT_SESSION is inherited, so a run launched from Windows Terminal into a NEW legacy console, which is what an elevated install gets, carries it with stdout not redirected and a buffer that has no ENABLE_VIRTUAL_TERMINAL_PROCESSING. SupportsVirtualTerminal reports host capability rather than the state of that buffer, so the branch would have claimed VT and printed literal escape sequences. There is no sound way to learn the current buffer's mode without GetConsoleMode, which is the compile. So decide the other direction instead: a redirected stdout is not a console, GetConsoleMode fails on a non-console handle, and the compiled path could then only return $false. Return it directly. This is provably identical rather than probably identical, and it covers the case that was actually measured: install.rs spawns install.ps1 with a pipe, so the desktop install is exactly where the compile was happening. Also drops the env plumbing from _run_console_less. It is lru_cached, so a dict argument would have raised TypeError before PowerShell was ever spawned, and the Windows parity job would have failed rather than proving anything. The parity case no longer needs it: the console-less probe IS the redirected case, so the early return is the branch under test rather than a bystander. * Reconstruct the exact merge-base function in the VT parity test The regex stripped only the guard and left the four comments above it behind, so the reconstructed predecessor was merge-base code plus comments rather than the merge-base function. Comments do not execute, so the comparison was still measuring the right thing, but a test that says it compares against the real predecessor should do that. Verified both files now reconstruct byte for byte. Also drops a stale WT_SESSION reference from an assertion message, left over from the design this PR replaced. * Tighten the comments this PR adds --------- Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
1b48147d8e
|
Windows: stop depending on the generated unsloth.exe console script (#8592)
* Windows setup: install uv from a pinned release instead of running remote script text
studio/setup.ps1 piped astral's install.ps1 straight into Invoke-Expression. That
download-and-execute shape is the single construct AMSI providers and cloud ML
scanners score hardest, and install.ps1 already replaced it with a pinned-SHA-256
archive download. Port the same implementation across.
Progress goes to the pipeline rather than the console, so the quiet path swallows
it exactly as it swallowed astral's installer output and the printed lines around
the call site are unchanged.
* Windows: stop pairing a hidden window with a bypassed execution policy
The Studio shortcut launched launch-studio.ps1 with -WindowStyle Hidden and
-ExecutionPolicy Bypass on the same command line. That pair is what Microsoft's
own detections key on, and studio/src-tauri/src/install.rs already refuses it for
the app's own launch of install.ps1.
The installer writes launch-studio.ps1 itself, so the file carries no
mark-of-the-web and RemoteSigned loads it. The hidden window is unchanged, so the
shortcut behaves exactly as before. The generated launcher's own child launch
moves to RemoteSigned for the same reason: it runs an inline -Command against an
executable, where no script file is loaded and the two policies are equivalent.
Also refresh a stale comment in studio/setup.ps1 that attributed the PSModulePath
fix to astral's uv installer, which no longer runs in-process.
* Installers: keep download-and-run command lines out of the shipped script text
AMSI scans install.ps1 in full before a single line of it runs, and generic
script classifiers read install.sh the same way inside the Linux bundle. Both
headers rehearsed the piped web one-liner five times over, plus a scriptblock
form and an execution-policy bypass, none of which anything in the scripts reads
and all of which the README already documents.
Point at the README instead and reword the in-body comments that quoted the
one-liner as shorthand. Every printed line is untouched: the remediation text the
installers show users still spells out the command in full.
Same treatment for scripts/uninstall.ps1's header.
* Windows: resolve process image paths with one Win32_Process query
install.ps1's venv-holder probe opened a handle to every running PID through
inline C# compiled at runtime. Opening a handle per process is a shape AV
heuristics score hard, and it bought nothing: Win32_Process reports
ExecutablePath for exactly the processes those handles could be opened against,
and answers for all of them in a single query instead of once per PID.
The remaining file-canonicalisation imports stay -- handle-based resolution of
linked ancestors has no faithful Windows PowerShell 5.1 equivalent, and it runs
on security-relevant paths.
Falls back to the per-process .Path when the query is unavailable, so a degraded
WMI repository degrades exactly as the old code did on a process it could not
open.
* Desktop: say who blocked the install when AMSI stops the script
PowerShell hands the whole top-level script block to AMSI while compiling it, so
a security product's verdict arrives as a parse error over the entire file before
install.ps1 runs a statement: no [TAURI:ERROR] marker, no phase log, and a stderr
tail the user cannot act on. unsloth#8523 shows what that looks like in the UI --
"Installation failed: + FullyQualifiedErrorId : ScriptContainedMaliciousContent".
Recognise the two stable error ids on either stream and append what the user
actually needs: nothing was installed, nothing was changed, it is a false
positive, update definitions and retry, do not turn off endpoint protection. The
raw id stays in the message, because the diagnostics report and any vendor
submission both need it.
Matches the id, never the message text, which is localized, and tolerates the
cmdlet suffix the Invoke-Expression form carries.
* Desktop: ship each bundle only the installer it can run
resolve_install_script picks install.sh on unix and install.ps1 everywhere else,
but the shared Tauri config bundled both into every target. The Linux AppImage
therefore carried 280 KB of Windows PowerShell it can never execute -- and it is
the largest script body a generic classifier walking the squashfs reads, which is
where Microsoft's Trojan:Script/Wacatac.B!ml verdict on 0.1.701-beta landed.
Move the resource map into the per-platform configs. The clean-machine job
already fails when a Linux bundle ships no install.sh; it now also fails when one
ships install.ps1, so the split cannot silently regress in either direction.
The .deb scanned clean with the same payload, so this is surface reduction rather
than a proven fix for that verdict.
* POSIX installers: install uv from a pinned release before falling back
install.sh downloaded astral's install.sh to a temp file, ran it and deleted the
file; studio/setup.sh piped it straight into a shell. Both are, shape for shape,
what a dropper does, and generic ML script classifiers score them accordingly --
the 0.1.701-beta Linux AppImage came back Trojan:Script/Wacatac.B!ml while the
.deb carrying the same scripts came back clean.
Fetch the pinned release archive and verify a hardcoded SHA-256 instead, matching
what install.ps1 already does on Windows. Only the four mainstream targets are
pinned: musl, armv7 and any host without a digest tool keep the path they have
today, because guessing a target triple wrong would break the install outright
and that costs far more than the heuristic score of the fallback.
Destination, PATH handling and every printed line are unchanged, so a host that
takes either path ends up in the same state it did before.
* tests: pin the installer shapes antivirus heuristics score
One file collecting what was removed, so it cannot drift back: no remote script
run in-process, no encoded or base64 payload, no hidden window paired with a
bypassed execution policy, no handle opened against another process, and no new
runtime-compiled native import outside an allowlist that carries a reason for
each entry that stays.
The last test is the other half of the contract. Hardening must not change what a
user sees, so the remediation lines the installers print -- which still spell out
the web one-liner in full -- are asserted verbatim. Removing the one-liner from
comments is the point; removing it from what the user is told to run would be a
regression.
Runs on the existing discovery-based pytest step, no workflow list to update.
* release: emit a false-positive submission packet for whatever gets flagged
The build job assembles a Microsoft submission packet, but only for the Windows
-setup.exe. The detection that actually arrived on 0.1.701-beta was
Trojan:Script/Wacatac.B!ml on the Linux AppImage, so nothing was produced for the
one asset that needed it.
The VirusTotal job already knows which assets were flagged and by which engines,
so put the packet there: hash, size and both portals, for every flagged asset
whatever platform it came from, with a note that clearance is per hash and per
vendor. Engine names are not repeated -- they are third-party text and already
appear escaped under Flagging engines.
The gate stays advisory; this only makes acting on it take seconds.
* Revert "Windows: resolve process image paths with one Win32_Process query"
This reverts commit
|
||
|
Daniel Han
|
99bcfd3d93
|
Keep the #8577 AMD peer guards message-only, and fix the table drift they exposed (#8689)
* Keep the peer guards message-only, and stop the shell Polaris arms over-matching Adversarial verification of the "message-only" claim found two ways this branch had started changing what gets installed, both introduced by my own review fixes. studio/setup.ps1: widening the adapter scan gate from $HasROCm to the arch let CIM fill $script:ROCmGpuLabels on the amd-smi path. That variable feeds $gpuNames and so the arch inference, and the existing unpinned "borrow another adapter's arch" rule then resolved an arch where none was resolved before. On a host where amd-smi confirms a runtime but reports no gfx token, an RX 9070 XT went from CPU torch to gfx1201, and an RX 5700 beside an RX 7900 went to gfx1100, with --rocm-gfx forwarded to the llama.cpp and whisper installers. install.ps1: the Get-WmiObject to Get-CimInstance swap had the same effect on PowerShell 7, where the old call threw and the catch swallowed it. A host CIM can see but the ROCm tools cannot went from CPU torch to a repo.amd.com index. Both scans are now exactly what they were before this branch, byte for byte, and the peer names live in their own variable that only the uncovered-card verdict reads: $wmiAmdNames in install.ps1, $script:ROCmPeerLabels in studio/setup.ps1. Neither block writes a label or an arch. The pwsh suite asserts that through the AST rather than asserting the gate, which is the property that actually matters; appending a label or arch assignment to either block fails it. The PowerShell 7 Get-WmiObject defect is therefore still there. Fixing it changes what every pwsh 7 AMD host without a HIP SDK installs, so it belongs in its own PR. Separately, the five copies of the unsupported table were not pinned to each other, only the supported ones were, and they had already drifted: the regex copies carry (?!0) so "RX 5800" is not Polaris, while the shell globs matched it through "RX 580". The shell arms now carry the same guard, and a new test compares all five tables by EVALUATING them in their own languages over a shared name corpus. Adding a row to one file alone fails it. Also pinned: a blank UNSLOTH_TORCH_INDEX_URL / _FAMILY must not read as a pin in the Python path, which dropping the .strip() previously passed. * Yield the peer suppression to an AMD visible-device mask The peer check walked every adapter without looking at HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES, so on a mixed host a masked-out supported card could silence the verdict about the card the user had actually selected. Both installers now skip the suppression under a mask, which is the rule studio/setup.ps1 already applies to its arch-borrowing branch: the mask names the card, so the verdict is about that one. HIP and ROCR only, deliberately, not Test-VisibleDevicesPinned's set: CUDA_VISIBLE_DEVICES masks NVIDIA devices and says nothing about which Radeon was chosen, and counting it fired the verdict beside a covered Radeon on every host that sets it, which two of the pwsh cases caught immediately. Three assertions per file pin it. Emptying the mask list fails one, adding CUDA_VISIBLE_DEVICES back fails another. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scan adapters with CIM and drop the masked verdict when it may name another card for PR #8689 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
5a5bf64130
|
Reduce antivirus false positives in the desktop installers (#8586)
* Windows setup: install uv from a pinned release instead of running remote script text
studio/setup.ps1 piped astral's install.ps1 straight into Invoke-Expression. That
download-and-execute shape is the single construct AMSI providers and cloud ML
scanners score hardest, and install.ps1 already replaced it with a pinned-SHA-256
archive download. Port the same implementation across.
Progress goes to the pipeline rather than the console, so the quiet path swallows
it exactly as it swallowed astral's installer output and the printed lines around
the call site are unchanged.
* Windows: stop pairing a hidden window with a bypassed execution policy
The Studio shortcut launched launch-studio.ps1 with -WindowStyle Hidden and
-ExecutionPolicy Bypass on the same command line. That pair is what Microsoft's
own detections key on, and studio/src-tauri/src/install.rs already refuses it for
the app's own launch of install.ps1.
The installer writes launch-studio.ps1 itself, so the file carries no
mark-of-the-web and RemoteSigned loads it. The hidden window is unchanged, so the
shortcut behaves exactly as before. The generated launcher's own child launch
moves to RemoteSigned for the same reason: it runs an inline -Command against an
executable, where no script file is loaded and the two policies are equivalent.
Also refresh a stale comment in studio/setup.ps1 that attributed the PSModulePath
fix to astral's uv installer, which no longer runs in-process.
* Installers: keep download-and-run command lines out of the shipped script text
AMSI scans install.ps1 in full before a single line of it runs, and generic
script classifiers read install.sh the same way inside the Linux bundle. Both
headers rehearsed the piped web one-liner five times over, plus a scriptblock
form and an execution-policy bypass, none of which anything in the scripts reads
and all of which the README already documents.
Point at the README instead and reword the in-body comments that quoted the
one-liner as shorthand. Every printed line is untouched: the remediation text the
installers show users still spells out the command in full.
Same treatment for scripts/uninstall.ps1's header.
* Windows: resolve process image paths with one Win32_Process query
install.ps1's venv-holder probe opened a handle to every running PID through
inline C# compiled at runtime. Opening a handle per process is a shape AV
heuristics score hard, and it bought nothing: Win32_Process reports
ExecutablePath for exactly the processes those handles could be opened against,
and answers for all of them in a single query instead of once per PID.
The remaining file-canonicalisation imports stay -- handle-based resolution of
linked ancestors has no faithful Windows PowerShell 5.1 equivalent, and it runs
on security-relevant paths.
Falls back to the per-process .Path when the query is unavailable, so a degraded
WMI repository degrades exactly as the old code did on a process it could not
open.
* Desktop: say who blocked the install when AMSI stops the script
PowerShell hands the whole top-level script block to AMSI while compiling it, so
a security product's verdict arrives as a parse error over the entire file before
install.ps1 runs a statement: no [TAURI:ERROR] marker, no phase log, and a stderr
tail the user cannot act on. unsloth#8523 shows what that looks like in the UI --
"Installation failed: + FullyQualifiedErrorId : ScriptContainedMaliciousContent".
Recognise the two stable error ids on either stream and append what the user
actually needs: nothing was installed, nothing was changed, it is a false
positive, update definitions and retry, do not turn off endpoint protection. The
raw id stays in the message, because the diagnostics report and any vendor
submission both need it.
Matches the id, never the message text, which is localized, and tolerates the
cmdlet suffix the Invoke-Expression form carries.
* Desktop: ship each bundle only the installer it can run
resolve_install_script picks install.sh on unix and install.ps1 everywhere else,
but the shared Tauri config bundled both into every target. The Linux AppImage
therefore carried 280 KB of Windows PowerShell it can never execute -- and it is
the largest script body a generic classifier walking the squashfs reads, which is
where Microsoft's Trojan:Script/Wacatac.B!ml verdict on 0.1.701-beta landed.
Move the resource map into the per-platform configs. The clean-machine job
already fails when a Linux bundle ships no install.sh; it now also fails when one
ships install.ps1, so the split cannot silently regress in either direction.
The .deb scanned clean with the same payload, so this is surface reduction rather
than a proven fix for that verdict.
* POSIX installers: install uv from a pinned release before falling back
install.sh downloaded astral's install.sh to a temp file, ran it and deleted the
file; studio/setup.sh piped it straight into a shell. Both are, shape for shape,
what a dropper does, and generic ML script classifiers score them accordingly --
the 0.1.701-beta Linux AppImage came back Trojan:Script/Wacatac.B!ml while the
.deb carrying the same scripts came back clean.
Fetch the pinned release archive and verify a hardcoded SHA-256 instead, matching
what install.ps1 already does on Windows. Only the four mainstream targets are
pinned: musl, armv7 and any host without a digest tool keep the path they have
today, because guessing a target triple wrong would break the install outright
and that costs far more than the heuristic score of the fallback.
Destination, PATH handling and every printed line are unchanged, so a host that
takes either path ends up in the same state it did before.
* tests: pin the installer shapes antivirus heuristics score
One file collecting what was removed, so it cannot drift back: no remote script
run in-process, no encoded or base64 payload, no hidden window paired with a
bypassed execution policy, no handle opened against another process, and no new
runtime-compiled native import outside an allowlist that carries a reason for
each entry that stays.
The last test is the other half of the contract. Hardening must not change what a
user sees, so the remediation lines the installers print -- which still spell out
the web one-liner in full -- are asserted verbatim. Removing the one-liner from
comments is the point; removing it from what the user is told to run would be a
regression.
Runs on the existing discovery-based pytest step, no workflow list to update.
* release: emit a false-positive submission packet for whatever gets flagged
The build job assembles a Microsoft submission packet, but only for the Windows
-setup.exe. The detection that actually arrived on 0.1.701-beta was
Trojan:Script/Wacatac.B!ml on the Linux AppImage, so nothing was produced for the
one asset that needed it.
The VirusTotal job already knows which assets were flagged and by which engines,
so put the packet there: hash, size and both portals, for every flagged asset
whatever platform it came from, with a note that clearance is per hash and per
vendor. Engine names are not repeated -- they are third-party text and already
appear escaped under Flagging engines.
The gate stays advisory; this only makes acting on it take seconds.
* Revert "Windows: resolve process image paths with one Win32_Process query"
This reverts commit
|
||
|
Daniel Han
|
d5a2160ef8
|
Say ROCm does not cover RDNA 1 instead of advising a fix that cannot work (#8577)
* Say ROCm does not cover RDNA 1 instead of advising an impossible fix An RX 5700 XT (Navi 10, gfx1010, RDNA 1) correctly lands on CPU PyTorch: AMD publishes Windows torch indexes for gfx103X, gfx110X, gfx1150, gfx1151 and gfx120X, and there is no gfx101X index. Because the name-inference table covers only arches that have wheels, the arch stayed null and the installer fell into the "arch unknown" arm, which tells the user to install the HIP SDK or set UNSLOTH_ROCM_GFX_ARCH. Neither can work: UNSLOTH_ROCM_GFX_ARCH=gfx1010 lands on the unmapped-arch path and returns CPU anyway. Add a separate name lookup for AMD generations ROCm PyTorch does not cover, read only to word the report. Product names come from LLVM's AMDGPU GFX10.1 processor table. The lookup never sets the arch the installers route on, so CPU fallback is reached by exactly the same path as before. Mirrored across install.ps1, studio/setup.ps1, install.sh, studio/setup.sh and studio/install_python_stack.py so every install path agrees. * Point pre-RDNA 2 AMD users at the Vulkan llama.cpp path The previous commit stopped at "ROCm does not cover this GPU", which is true and still a dead end. There is a working path: llama.cpp's Vulkan bundle drives these cards, which is how #8458's reporter got an RX 580 running and how LM Studio drives the same hardware. Nothing routes these users there automatically. _should_auto_vulkan_for_amd_windows opens with `active = _active_rocm_gfx_target(host); if not active: return False`, and a pre-RDNA 2 card resolves to no gfx target at all, so the Windows auto-Vulkan fallback structurally cannot fire for exactly the cards that need it. The environment variable is their only route, so the message now names it. Two things about that advice are load-bearing and both are tested: - The current spelling, UNSLOTH_LLAMA_CPP_BACKEND=vulkan. The legacy UNSLOTH_FORCE_VULKAN still works but force_vulkan_requested() resolves the new variable first and consults the legacy one only when the new one is absent or unparseable, deliberately, so =hip stays a real opt-out that a stale legacy variable cannot overrule. New text must not spread the legacy name. - WHEN to set it. The variable picks the llama.cpp bundle at install/download time; nothing reads it at runtime to choose a binary. #8458's reporter set it after installing, saw no change, and only a clean reinstall worked. Advice that names the variable without naming the moment is worse than none. Also adds Polaris 10/20/30 (RX 470/480/570/580/590, gfx803) to the messaging-only table so #8458's card gets the right message. gfx803 stays out of _GFX_TO_AMD_INDEX_ARCH and every supported-arch table, and routing is untouched; a test pins that directly. Polaris 11/12 (RX 460/550/560) is left out because gfx803 vs gfx804 could not be confirmed for that die, and this table is only worth having while it never guesses. "RX 570" is a prefix of "RX 5700" and "RX 550" of "RX 5500". Python and PowerShell carry (?!0) lookahead guards; POSIX `case` has no lookahead, so in install.sh and studio/setup.sh correctness rests on arm order and the RDNA 1 arms come first. That order is now documented and asserted, and the shipped arms are evaluated in a real shell rather than checked by eye. README leads with the current spelling, since the installer now names a variable and the README is where users check it. Finally, test_cpu_index_note_respects_explicit_pin asserted a pin check appeared within a character window before a note. That is a budget on intervening source, not the ordering property it is for, and this work had already pushed it from 400 to 1400. It now walks the enclosing if/elif chain by indentation, so it tests order and has no distance left to re-tune. * Stop the HIP SDK arm outranking the pre-RDNA 2 message, and fix its advice An RDNA 1 or Polaris user who already installed the HIP SDK never saw the new message: the $HipSdkInstalled arm sits earlier in the chain and told them the ROCm compute driver was missing, which is the impossible remediation this change exists to remove, and those users installed the SDK because the old advice said to. Guard that arm, plus the matching CPU-hint arm in install.ps1, on the unsupported arch. The Vulkan setter was printed as UNSLOTH_LLAMA_CPP_BACKEND=vulkan by the two PowerShell installers and by the Windows-only branch in install_python_stack.py. PowerShell parses that as a command name, so a user who pastes it sets nothing and the next install picks the same CPU bundle. Print $env:UNSLOTH_LLAMA_CPP_BACKEND = "vulkan" there, as the README already does. The arms also said PyTorch training runs on CPU on these GPUs. It does not: with no CUDA or XPU accelerator, unsloth raises NotImplementedError at import, which is why studio/setup.sh already tells its other CPU-torch hosts that training and GPU inference are unavailable. Say the same thing here. README: Vega 20 (Radeon VII, MI50, gfx906) is older than RDNA 2 and does have a ROCm PyTorch path (install.sh routes it to rocm6.3), so name Polaris and RDNA 1 instead of every pre-RDNA 2 AMD GPU. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard the README gfx906 carve-out against any spelling of the cutoff The ban was on one exact literal, so "every AMD GPU older than RDNA 2" passed while contradicting the Vega 20 carve-out two sentences later. Match the phrase family instead, and assert the group is named by its members (Polaris, RDNA 1) rather than by a generation cutoff. * Tighten the comments added by this PR Comments only, no code or user-facing string touched. Every reason a guard exists is kept, just said in fewer lines. * Keep the note on why the rationale sits above the arm * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: initialise the unsupported-arch state outside the AMD detection block The ROCm summary reads $script:ROCmUnsupportedGfxArch unconditionally, but the assignment sat inside `if (-not $HasNvidiaSmi)`, so an NVIDIA host never defined it and a caller's Set-StrictMode turned the summary into an aborting undefined-variable error. ROCmGfxArch beside it was always initialised at top level; this one was not. The guard asks the PowerShell parser whether any assignment is unnested rather than comparing line numbers: the file has three -not $HasNvidiaSmi blocks, so an ordering check picks the wrong one and passes for the wrong reason. * Name the remaining RDNA 1 boards, diagnose the KFD-only host, and keep mixed AMD hosts on the arch-unknown advice Adds the Navi 10 / Navi 14 professional boards LLVM's processor table omits (Radeon Pro W5700/W5700X -> gfx1010, Pro W5500/W5500M/W5300M and RX 5300/5300M -> gfx1012) to all five copies of the unsupported-name table. Each mapping comes from libdrm data/amdgpu.ids read against pci.ids and the kernel amdgpu PCI table, not from a guess; the tables still route nothing. studio/setup.sh's KFD sysfs fallback detects the GPU without rocminfo or amd-smi, so it left the marketing name empty and the report fell through to a plain AMD ROCm line on a host with no ROCm. It now reads lspci for that report only, never writing it back into the name the supported table and --rocm-gfx key on. install.ps1's WMI fallback classifies adapter 0 only, so a host pairing an RX 5700 with an RX 7900 was told nothing could enable ROCm, which is false there. The verdict is now withheld when another adapter is covered, leaving the arch-unknown advice that does apply. install_python_stack.py and studio/setup.ps1 already scored every adapter. * Scope the uncovered-arch verdict to the card it names, and name the boards it was missing A host is not one GPU. On a box pairing an uncovered card with one that has wheels -- an RX 580 beside an RX 7900 XTX, or beside an Instinct MI210 -- "setting UNSLOTH_ROCM_GFX_ARCH will not enable ROCm PyTorch" was false: masking to the other card and pinning its arch installs them, and install.sh routes exactly that host to gfx110X-all a few lines earlier. Every advice site now says what is true of the card it just named and claims nothing beyond it. Deciding it at runtime was tried and dropped. Reading "an AMD adapter neither table names" as a working peer misfires on the Vega-class iGPU (Raven through Cezanne, Mendocino) that sits beside the dGPU on most Ryzen desktops and has no ROCm torch path of its own, which would trade a correct dead stop for the open-ended errand this change exists to remove. Reading only the supported table misses the Instinct and V620 parts that are routable and appear in no name table at all. Neither rule is right often enough to speak for a host. Four real boards are added to all five copies of the message-only table: Radeon Pro 5700 / 5700 XT (pci.ids 7319 and 731b, Navi 10, gfx1010) are the only Navi 10 retail parts whose name carries neither "RX 5700" nor a W prefix, and Radeon Pro WX 7100 / WX 5100 (Ellesmere, gfx803) carry no RX number at all, so both fell through to the generic advice. Provenance from pci.ids as before, and the shell case arms are matched case-sensitively, which is now stated where only the arm ordering was. Tests: - The absolute host-wide phrasings are banned from all five sources, with the scoped replacements required per site so deleting the sentence cannot pass. - test_unsupported_arch_routing_guards_8529.py drives the real index resolvers rather than asserting table shape: install.sh's get_torch_index_url and _amd_arch_index_family_for_gfx under sh, the .ps1 family maps under pwsh (including writes after the declaration, and the -contains list), the Python resolvers on both platforms, the Strix per-arch reroute, and studio/setup.sh's report-only lookup. Positive controls throughout. - _assert_guarded_by_pin_arm matches the pin arm exactly rather than by prefix, binds the message's own enclosing arm, requires the pin arm to still say its note, ignores comment lines, and detects inline and re-indented chain closes. Mutation tested: 13 mutants covering a post-declaration map write in each .ps1, a single-quoted arch in _rocmWheelArches, a bypass inside get_torch_index_url, the Strix arm, feeding the unsupported lookup into _setup_gfx directly and one hop later, an inline-closed pin chain, a re-indented fi, and removing each new table row. All killed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Say whose wheels are missing: AMD ships RDNA 1 PyTorch, Unsloth does not install it "No ROCm PyTorch wheels exist for that arch" is no longer true of RDNA 1. AMD's TheRock lists device-gfx1010, device-gfx1011 and device-gfx1012 as installable torch extras on its multi-arch index, and SUPPORTED_GPUS.md marks all three Build Passing, Sanity Tested and Release Ready. gfx803 is absent from that table entirely, and no GCN4 family appears at all, so the Polaris half stands. The claim these installers can honestly make is about their own routing, not about ROCm at large: repo.amd.com publishes gfx103X/110X/1150/1151/120X and nothing for gfx101X or gfx80X, so UNSLOTH_ROCM_GFX_ARCH=gfx1010 still lands on the unmapped path and still returns CPU. Every site now says Unsloth has no wheels for the arch rather than that none exist, and the tables carry a note saying why the wording is scoped. Routing, the CPU fallback and the Vulkan advice are all unchanged. Verified by running the merge base and this branch side by side under identical stubbed hardware, and diffing: - POSIX shell, 127 simulated hosts x 2 blocks = 254 rows, each run in both trees. Selected torch index URL differed in 0 rows, exit code in 0 rows, get_torch_index_url stdout in 0 rows. The 32 rows whose end-of-run summary text moved are all AMD arch-unknown hosts resolving to gfx1010/1011/1012/803. Covers linux/wsl/macos/aarch64, NVIDIA at five CUDA levels, 15 supported gfx arches, multi-GPU lspci mixes, every override, dash and bash, set -eu, and lspci absent/failing/hanging. Asserted separately that the new WARN lines go to stderr, so TORCH_INDEX_URL=$(get_torch_index_url) is never polluted: 0 of 127 index rows had anything but one URL on stdout. - PowerShell, 232 cells over 116 adapter inventories x install.ps1 and setup.ps1. Resolved arch, index URL, arch family, routing flag and the gfx handoff are identical in every cell. 80 cells changed text, all RDNA 1 or Polaris. An RDNA 1 card beside an RX 7900 keeps the old wording and still resolves gfx1100, which is the covered-peer guard doing its job. Under Set-StrictMode the new code passes only because of the variable initialisation added outside the detection block; removing it fails. - Python stack, 23034 value comparisons over 5 platforms x 4 GPU classes x 38 arch inputs x 99 adapter names x masks x overrides x 6 mirror configs. 0 value differences. No unsupported arch produced an AMD index URL by any path in either tree, and no supported arch changed URL. A negative control that adds gfx1010 to the routing map reports 83 differences, so the zero is real. - Existing installs: a manifest written by the old code verifies identically under the new code and vice versa, over 12 write/read combinations; a manifest poisoned with gfx_arch and index-url keys changes no verdict, because nothing arch-shaped is persisted. The legacy UNSLOTH_FORCE_VULKAN resolves identically across all 90 combinations with UNSLOTH_LLAMA_CPP_BACKEND, including falsey values and =hip overriding a stale truthy legacy value. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Declare the unsupported-arch variable where its readers can see it $ROCmUnsupportedGfxArch was declared inside `if (-not $HasNvidiaSmi)` in install.ps1, but the arms that read it sit outside that gate, so on an NVIDIA host the read is of a variable that was never assigned. Its five neighbours (HasROCm, HipSdkInstalled, ROCmGpuLabel, ROCmVersion, ROCmGfxArch) are all declared above the gate, and studio/setup.ps1 already hoists its own copy for exactly this reason, so this one was the odd one out. Harmless as shipped, because Install-UnslothStudio runs with Set-StrictMode off. Under a caller's `Set-StrictMode -Version Latest` it is a hard stop: driving the extracted blocks under pwsh, an NVIDIA host that lands on the /cpu leaf (a pre-CUDA-11 driver, or UNSLOTH_TORCH_INDEX_URL pinned to cpu) throws on the read. Found by running this branch and its merge base side by side over 240 Windows cells; every resolved arch, index URL, arch family, routing flag and gfx handoff matched, and this was the only asymmetry that was not message text. A test pins the declaration above the gate in install.ps1 and at script scope in setup.ps1. Moving it back inside the block fails the test. * Let an identified uncovered card outrank the generic ROCm report, and teach export Two review findings, both reproduced first. amd-smi can report a GPU with no gfx token anywhere in `list` or `static --asic` and only a market name. That sets $HasROCm with no arch, so the generic `} elseif ($HasROCm)` arm fired and called an RX 5700 XT "AMD ROCm (AMD Radeon RX 5700 XT)" while the wheel note in the same run said gfx1010 has none. Driving the real detection and step chain under pwsh with a stubbed amd-smi reproduces it exactly, and the host is not hypothetical: amd-smi is only probed when the HIP SDK is present, which is what the #8529 and #8458 reporters installed because the old message told them to. Both scripts now carry the same `-and -not $ROCmUnsupportedGfxArch` guard the HIP SDK arm below already had. A supported gfx1100 host and an unmapped Instinct MI210 host are unchanged on the same harness, since the guard is a no-op when no arch was identified. The POSIX advice said to `set UNSLOTH_LLAMA_CPP_BACKEND=vulkan` and re-run the installer. A bare assignment is a shell variable, not an environment entry, so the installer subprocess never sees it and the user gets the CPU bundle again: $ sh -c 'UNSLOTH_LLAMA_CPP_BACKEND=vulkan ./installer' -> installer sees: [<unset>] $ sh -c 'export UNSLOTH_LLAMA_CPP_BACKEND=vulkan ./installer' -> installer sees: [vulkan] That is the #8458 failure mode reintroduced by the fix for it. The README block has always used export; the three POSIX message sites now agree with it. The PowerShell sites already used `$env:`, which is the process environment, so they were correct and are untouched. Tests pin both: the emitted POSIX setter is now `export ...`, and each generic ROCm arm must carry the unsupported guard. Dropping either guard fails. * Guard the ROCm summary chain the same way its two siblings are The summary at studio/setup.ps1 opens with a bare `if ($HasROCm)` rather than an `} elseif`, so it was missed when the other two chains were guarded. Its own third arm names the uncovered card, and that arm is only reached when nothing outranks it. On a host where amd-smi enumerates an RDNA 1 card with no gfx token, the "ROCm x.y" arm wins and the arm written for that card never runs, so the same run reports ROCm here and no wheels below. The existing check only matched arms opening with `} elseif ($HasROCm`, which is why it walked past this one. The new test finds the chain by its own body. * Keep banning the bare POSIX setter in the PowerShell sources Requiring `export` in the .sh advice was done by redefining _POSIX_SETTER, which is also the needle two PowerShell bans read. With the export folded in, a .ps1 that printed a bare UNSLOTH_LLAMA_CPP_BACKEND=vulkan no longer matched either ban: a mutant adding exactly that line passed all 261 tests in the file. Split the two. _POSIX_ASSIGNMENT is the bare form the Windows sources must never print, and _POSIX_SETTER stays the exported form the POSIX ones must teach. The same mutant now fails, and reverting either export still fails. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments on the unsupported-arch path Final pass over the comments this branch added. The five copies of the table header each repeated the same four points at four different lengths, so they are now one block of the same wording everywhere, and the provenance and scoped-claim notes are folded into it rather than trailing it. Same for the report arms and the test rationales: no reason removed, fewer lines to read. Comments, docstrings and wrapping only. AST-checked with comment_tools.py (4/4 code-unchanged), `bash -n` on both POSIX scripts, and the PowerShell parser on all three .ps1 files. Suites re-run: 1036 passed, 1 skipped, and the pwsh behavioural suite green. * Blame the right card in the CPU summary, and stop promising macOS Vulkan Two more review findings, both reproduced first. The end-of-run CPU summary calls the lspci lookup unconditionally, so unlike the arm in get_torch_index_url it is not covered by the empty-probe gate. On a host pairing an RX 5700 with an RX 7900, a CPU fallback caused by the 7900's ROCm being older than 6.0 was attributed to the 5700, replacing the "upgrade ROCm" advice with advice that is false for the card that actually caused it. Running the shipped guard under sh with a stubbed lspci, only that host moves: lone RX 5700 uncovered-card message -> unchanged lone RX 580 uncovered-card message -> unchanged lone RX 7900 generic message -> unchanged lone MI210 generic message -> unchanged RX 5700 + RX 7900 uncovered-card message -> generic message The summary now asks _infer_linux_amd_gfx_arch, which scans every display adapter, and stays quiet when any of them is covered. Same shape as the peer guard install.ps1 already carries. The README's Vulkan paragraph sat under the combined "macOS, Linux, WSL" heading. macOS has no Vulkan llama.cpp bundle: install_llama_prebuilt.py logs that the variable is ignored and installs the Metal build, and upstream ships no macOS Vulkan asset either (Metal is the default there, and Vulkan on macOS only exists through MoltenVK, which you have to build yourself). An Intel Mac carrying one of these very cards, the 16-inch MacBook Pro shipped the Radeon Pro 5300M/5500M/5600M, would follow the command and get nothing. The paragraph now names Linux and WSL and macOS gets its own sentence. The installer itself needed no change: get_torch_index_url returns before any AMD probe on Darwin and the summary is already gated on it, so the advice was never emitted there. Both guards are pinned by tests that fail when the guard is dropped. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments added in the last pass * Guard the Studio report's peer scan, explicit index pins, and Windows ARM64 Three more review findings, all reproduced first. studio/setup.sh had the same misattribution I fixed in install.sh last round, on the KFD path where neither rocminfo nor amd-smi answers and the lookup falls back to lspci. First match wins, so a host whose RX 5700 enumerates before an RX 7900 was told no override could help, which is false there. The supported name table is now a matcher, _setup_supported_gfx_from_name, with its arms byte-identical to before, so the scan can ask about a peer without touching $_setup_gfx. Same fixtures as the install.sh guard, run under sh: lone RX 5700 / RX 580 named -> unchanged RX 5700 + RX 7900, either order named -> quiet RX 580 + RX 7900 named -> quiet An explicit UNSLOTH_TORCH_INDEX_URL or _FAMILY reaches the ROCm install path for any gfx*/rocm* leaf, so "torch stays CPU-only and neither the HIP SDK nor UNSLOTH_ROCM_GFX_ARCH changes that" was false on a pinned run. install.sh's CPU note already skipped its guidance when pinned; install.ps1, studio/setup.ps1, studio/setup.sh and install_python_stack.py now agree with it. install.ps1's second site needed nothing, since it already sits behind -not $ROCmIndexUrl. studio/setup.ps1 throws on UNSLOTH_LLAMA_CPP_BACKEND=vulkan on Windows ARM64, where no Vulkan bundle is published, so the advice aborted the next update instead of enabling GGUF acceleration. Both PowerShell sites now branch on Get-HostMachineArch and point at a source build there. The advice-window test had to change with them: the claims now sit in if/else arms, and a fixed 8-line window either stopped mid-branch or spilled into the next arm, which is the failure its own docstring warns about. It walks to the end of the enclosing arm instead, capped. Both bash table-parity tests follow the matcher's new variable names and still compare the same rows. Each guard is pinned by a test that fails when it is dropped: the peer loop, the pin check, and the Get-HostMachineArch call were each mutated and killed. * Cut the comment lines that restated the message below them * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fill the Windows peer list on the amd-smi path, and three smaller corrections Four more review findings, all reproduced first. install.ps1's WMI scan sat behind `if (-not $HasROCm)`. amd-smi can report GPUs with no gfx token and only the first market name, which sets $HasROCm with no arch, so the scan was skipped and the peer guard added last round saw an empty list on exactly the multi-GPU host it exists for. It is now keyed on the arch, which is the condition under which the unsupported lookup can run at all: a host that already has an arch still does no WMI work here, and amd-smi's label still wins when it had one. The pwsh suite asserts the gate through the AST. studio/setup.sh returned the failure of a nonempty market name immediately, so a generic "AMD Radeon Graphics" from rocminfo ended the lookup before the lspci scan and the report fell back to the plain "AMD ROCm" line this change is meant to replace. It now returns only on a hit; a name that maps still short-circuits without touching lspci, and the peer guard still covers the mixed host. install_python_stack.py prints the same Vulkan advice as install.ps1 on the Windows WMI path, and the same ARM64 throw applies to it. Added _is_windows_arm64(), mirroring Get-HostMachineArch down to the PROCESSOR_ARCHITEW6432 case an emulated x64 Python needs. The setup.sh pin check treated a whitespace-only value as a pin, while get_torch_index_url trims both variables and treats a blank one as unset. It is trimmed the same way now. The variable also collided with the XPU block's _setup_pin, which is a global in POSIX sh, so it is renamed _setup_unsup_pin. One existing test asserted the behaviour the second finding calls a bug, that an unrecognised reported name claims nothing. Rewritten to the corrected intent, keeping a case that proves a covered card is still never claimed from lspci. test_windows_amd_gpu_scan_fallback.py extracts the WMI block by its literal gate, so its anchor follows the new condition. Each guard is pinned by a test that fails when it is dropped: all four were mutated and killed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover the Python emitter in the ARM64 Vulkan-offer guard * Fill the Studio peer list on the amd-smi path too studio/setup.ps1 carries the same WMI scan install.ps1 does, for its own name inference, and it had the same gate. amd-smi can report GPUs with no gfx token and only the first market name, which sets $HasROCm with no arch, so the scan was skipped, $script:ROCmGpuLabels held one name, $gpuNames was one entry, and a host pairing an RX 5700 with an RX 7900 was judged entirely on the 5700 even with HIP_VISIBLE_DEVICES=1 selecting the 7900. The scan is now gated on the arch, which is the condition the inference block below already runs under, so the two can no longer disagree about whether there is anything to infer from. A host that already has an arch still does no WMI work here, and amd-smi's label still wins when it had one: only the peer list is new on that path. The pwsh suite walks the AST from the $script:ROCmGpuLabels assignment to its enclosing if and asserts the condition names the arch and not $HasROCm, as it already does for install.ps1. Restoring the old gate fails it. tests/test_windows_amd_gpu_scan_fallback.py extracts that block with a regex anchored on the literal gate, so its pattern follows the new condition. * Run the covered-peer guard before the named hit too The guard added last round only covered setup.sh's lspci fallback, so a named hit still walked past it. amd-smi reports one market name, the first device's, so on a host whose RX 5700 precedes an RX 7900 the name IS the uncovered card and the false verdict came back through the other door. The scan now runs first, for both paths: lone RX 5700, named named -> unchanged lone RX 580, named named -> unchanged RX 5700 + RX 7900, named 5700 named -> quiet RX 5700 + RX 7900, no name quiet -> unchanged It must not become a silencer, so it applies only when lspci can answer: with no adapter list there is no peer to find, and the single-card host this report exists for still has to be told. A test drives the lookup with lspci off PATH and requires the verdict to survive. Putting the named hit back in front fails the two mixed-host cases while that no-lspci control still passes, so the ordering is what is pinned, not just the presence of the guard. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments added since the last pass * Ask CIM for the adapter list, not the cmdlet PowerShell 7 dropped install.ps1:3331 was the only live Get-WmiObject call left in the file; every other WMI query in it already asks Get-CimInstance, and the script handles PSEdition Core explicitly. Get-WmiObject was superseded by Get-CimInstance in PowerShell 3.0 and removed outright in PowerShell 7, so on pwsh the call threw, the block's own catch swallowed it, and $wmiAmdNames came back empty. That is the peer list the guard added two rounds ago reads, so on pwsh the guard could never fire and the amd-smi path went back to blaming the uncovered card. Same class, same properties, and the CIM cmdlets ship with 5.1 as well, so the swap costs nothing downlevel and matches what studio/setup.ps1 already does. The harness stubbed Get-WmiObject, so it was answering for a cmdlet the installer no longer calls. It now stubs Get-CimInstance and defines Get-WmiObject to throw, so a revert fails loudly instead of quietly returning nothing through that catch. Reverting the call fails four tests. * Tighten the two comments from the last pass --------- Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com> |
||
oobabooga
|
5426a78c39
|
Studio: switch llama.cpp backends from the UI (#8520)
* Studio: add llama.cpp backend selector * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix llama.cpp backend selection edge cases * Fix llama.cpp backend selection review issues * Fix remaining llama.cpp backend review issues * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix llama.cpp backend switch consistency * Re-pair whisper after llama runtime changes * Re-pair whisper when llama runtime identity changes * Unify llama backend selection invariants * Preserve llama backend fallback constraints * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Publish llama planning job state * Handle llama frontend job transitions * Simplify the llama.cpp backend selection contracts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address the open Codex findings on backend precedence and job ownership * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop the unused llama backend imports the hoist check rejects The Source lint job fails on studio/install_llama_prebuilt.py: scripts/verify_import_hoist.py reports INSTALL_KIND_BACKENDS and marker_backend as hoisted but unused. marker_backend is genuinely dead here, so it goes. INSTALL_KIND_BACKENDS is not: the installer is meant to share one vocabulary with the marker readers, and two tests assert that through this module. Give it a real use instead of a bare re-export by deriving VULKAN_INSTALL_KINDS from the map rather than spelling the same two names out again, which also removes a mirrored definition of the kind this file warns about elsewhere. * Treat an install marker that is not a JSON object as no marker read_install_marker returns whatever json.loads produced, so a marker holding [] or 123 reaches callers as something without .get. Every caller assumes a mapping, and the new backend picker adds one more: get_backend_status calls marker_backend(marker) and raises AttributeError, so GET /api/llama/backend answers 500 and Settings > System shows a load error for what is only a corrupt file. This is not new on this branch (get_update_status raises the same way on main), but the picker makes it reachable from a page users open. Guard it where the file is read, so the update planner, the picker and crash recovery all degrade to the source-build path together, exactly as they already do for unparseable JSON. * Disable Apply when the environment pins the backend The Select is disabled whenever env_backend is set, but Apply is gated only on dirtiness, and the two are computed independently. An automatic install whose detection has since drifted reports selection_applied false, so the row is dirty while the Select is disabled and Apply becomes the only live control. POST /api/llama/backend then refuses it with environment_override, which is correct, but the button should not have offered it. Also fills in the status shapes the payload tests did not reach: every unsupported reason, a macOS install reporting metal, and the terminal job states. * Run the setup.ps1 exit routing under pwsh instead of matching its text The Windows half of the fail-closed change was asserted by comparing source strings, which cannot catch a branch that reads the same and behaves differently. Extract the routing block the way the bash harness already does, run it under pwsh, and require the same decision from both: identical exit code, and identical answer on whether a source build was queued. Covers exit codes 0/1/2/3/4/5/137 against each explicit backend and both install states, 70 pairs, and pins the two branches the picker depends on: exit 5 fails closed everywhere, exit 2 stays the one automatic path allowed to fall back to a compile. Skipped where pwsh is absent, like the other PowerShell tests here. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Let a failed whisper re-pair be retried from the same selection The llama phase runs first and records the new backend, so a retryable whisper failure (a dropped download, an install that was busy) ends with llama.cpp on the requested backend and dictation still hardlinked to the old runtime. Retrying that selection is then already_selected, which skips the llama phase, and the whisper planner refused to run without one. The reported failure was unfixable except by switching away and back. Allow a repair-only job for that one refusal, gated on the pairing actually being stale so an ordinary already-selected request stays a refusal instead of becoming a no-op job that reports success. slim_pairing_is_stale exposes the comparison run_repair_phase already makes before it does any work. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
8502cf84c2
|
Detect the Radeon AI PRO R9700 (gfx1201): it carries neither 9070 nor 9080, so name inference found nothing (#8573)
* Detect the Radeon AI PRO R9700 (gfx1201) in every GPU-name arch table The name-to-gfx tables matched RDNA 4 Navi 48 on 9070|9080 only. The workstation card is branded Radeon AI PRO R9700, which contains neither token, so the first-match-wins table returned nothing, no arch was inferred and the installer fell back to CPU torch. On a host with the HIP SDK present the gcnArchName probe answers first and the gap is invisible, which is why the R9700 reporters in #7624 and #7307 never hit it. On a plain Windows 11 box with a single R9700 and no SDK, name inference is the only path left and the card is simply reported as not detected. Add R9700 to all six copies of the table (install.sh twice, install.ps1, studio/setup.sh, studio/setup.ps1, studio/install_python_stack.py). The token is R9700 rather than a bare 9700 so the 2002 ATI Radeon 9700 PRO cannot pick up RDNA 4 wheels. * Tighten the R9700 detection comments * Run the AMD GPU-name arch table test in cross-platform CI --------- Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> |
||
|
Daniel Han
|
d7fed06063
|
Windows: fix single-AMD-GPU hosts reading as "gpu none" and looping the installer (#8461)
* Windows: fix single-AMD-GPU hosts reading as "gpu none" and looping the installer
setup.ps1's WMI fallback built its adapter list with
$wmiGpus = if ($healthyGpus.Count -gt 0) { $healthyGpus } else { $amdGpus }
An unwrapped one-element branch unrolls to a bare WMI object on its way out of the
if, and a bare WMI object has no .Count in PS 5.1 (a string or hashtable does), so
`if ($wmiGpus.Count -gt 0)` never fired on a host with exactly one AMD adapter and
$ROCmGpuLabel stayed null. Setup reported "gpu none (chat-only / GGUF)", skipped the
name to gfx inference, and the stale-venv check then expected cpu torch against the
ROCm wheels install.ps1 had just placed:
Stale venv detected (torch rocm != required cpu).
[ERROR] The existing Unsloth environment needs repair.
The installer rolled back and the desktop app retried the same failure indefinitely.
Reproduced on a Radeon 8060S (gfx1151) Strix Halo laptop, where install.ps1 resolved
gfx1151 and setup.ps1 one second later saw no GPU at all.
Wrap the whole if in @(), the idiom the Intel scan below already documents.
install.ps1 also now exports the arch it resolved before invoking setup.ps1, so the
two never re-derive it independently. Setting UNSLOTH_ROCM_GFX_ARCH by hand was the
workaround for this bug, and doing it in the installer keeps any future divergence
between the two scans from turning into an unrecoverable rollback loop.
* Windows AMD: keep the arch handoff private, and fix the same unrolling bug in the name list
Follow-up to the single-AMD-GPU scan fix in this branch, from reviewing what it
could do to hosts other than the one it was reported on.
install.ps1 forwarded its resolved arch as UNSLOTH_ROCM_GFX_ARCH. That name is
the documented operator override, and install_llama_prebuilt.py reads it back as
_manual to decide whether a forwarded --rocm-gfx outranks its own probe, so
publishing an auto-detected value there disarmed that safeguard. install.ps1's
scan is also the weaker of the two: it takes the first AMD adapter with no
visible-device mask and no shadowing-iGPU repick, both of which setup.ps1
applies. On a 780M + RX 9070 XT host setup resolves gfx1201 today; the forward
made it take the installer's gfx1103 verbatim and hand llama.cpp the iGPU
bundle. It was also never restored, so it outlived the install in the caller's
shell on the documented irm | iex path.
It now travels as _UNSLOTH_ROCM_GFX_ARCH_HANDOFF, matching the _UNSLOTH_ prefix
the neighbouring handoffs use, saved and restored in the same finally block, and
consumed by setup.ps1 only after its own probes and inference come up empty.
setup.ps1's gpu name list had the same unwrapped if as the adapter scan: it
wraps each branch but not the if, so a single adapter name unrolls to a bare
String and $gpuNames[$nameIdx] indexes the name and yields "A". The
$nameArches[0] rescue covers that unless a visible-device mask is set, so a
pinned single-GPU host still inferred no arch and looped the same way. Audited
every '= if (' site across the .ps1 files; this was the only other one.
The tests asserted on .Count, which pwsh answers as 1 for a scalar because
PowerShell/PowerShell#5745 added that fallback in 6.1 and Windows PowerShell 5.1
never got it, so they passed against the unfixed source and guarded nothing.
They now assert the shape of the value, re-run each case against stubs carrying
Count = $null to reproduce 5.1's behaviour, cover the mask and multi-adapter
paths and the handoff lifecycle, and pin their own failure against the merge
base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the red/green check revision-independent, and add the AGPL header
The check resolved its pre-fix source through git merge-base against main. That
holds only until this merges: after it, the merge base is a commit that already
carries the fix, so the fixed source goes in as the before case and the three
assertions fail on every host with pwsh. Reaching for an older revision at all
also breaks in a shallow CI clone.
It now undoes just the two @() wraps in the shipped source in memory, which is
immutable, needs no git, and isolates the one thing under test since everything
else about the two sources is identical by construction. Verified both ways:
fed the fixed source in as before, the old assertion fails exactly as predicted;
with the wraps undone it reports scalar and no label as it should.
* Tighten the comments in the AMD GPU scan tests
* Do not consume the installer arch handoff under a visible-device mask
The inference above deliberately leaves $pickedName unset when a mask is set
and the selected adapter's name is not in the table, rather than borrowing a
peer's arch. The handoff then took it anyway: install.ps1 scans without the
masks and forwards the FIRST recognized adapter, so a host masking an unknown
discrete card while a 780M is listed first resolved gfx1103 and installed
wheels and prebuilts for the iGPU the mask hides.
ROCR_VISIBLE_DEVICES filters below HIP, so masked devices never reach the
runtime's enumeration at all, which makes targeting one strictly wrong rather
than merely suboptimal. Setup now skips the handoff whenever any of the three
masks is set, matching what its own inference and Resolve-ShadowingGfxPick
already do. UNSLOTH_ROCM_GFX_ARCH stays the escape hatch and still wins.
Confirmed both ways: with the guard removed the masked host resolves gfx1103,
with it in place it resolves nothing, and the unmasked gap-filling case the
handoff exists for is unchanged.
* Skip disabled AMD adapters in install.ps1's WMI fallback too
install.ps1 took the first AMD adapter WMI listed with no health check, while
setup.ps1 filters on ConfigManagerErrorCode. A disabled Radeon listed ahead of a
healthy unsupported one therefore resolved that dead card's arch. Because a
mapped arch installs ROCm wheels right there, the machine got wheels for a GPU
it cannot use while the live card went unserved, and setup, which discards that
adapter, disagreed and took the forwarded arch as its last resort.
Fixed at the source rather than by teaching setup to distrust the handoff:
rejecting it there would leave setup expecting cpu torch against the ROCm wheels
install.ps1 had already placed, which is the stale-venv rollback loop this
branch exists to end. Filtering here means the two scans start from the same
healthy set, so a forwarded arch can only ever name an adapter setup also kept.
Keeps setup's fallback for the case where the filter empties the list, since
code 45 is routine on a muxless laptop with a parked dGPU.
Confirmed both ways: before, the disabled card resolves gfx1201; after, the host
resolves nothing and lands on CPU exactly as setup does.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Daniel Han
|
e696d3328a
|
Windows installer: fix single-AMD-GPU detection and the unrecoverable "needs repair" loop (#8398)
* Windows installer: fix single-AMD-GPU detection and the "needs repair" loop Two defects that met on the same host and made the installer unable to converge on a single-Radeon Windows box (#8335). The WMI fallback picked between the healthy adapters and the full adapter list with a bare if-expression, which unrolls a one-element array to a scalar. A scalar's .Count is $null under Windows PowerShell 5.1, so a host with exactly one AMD GPU read as having none: no $script:ROCmGpuLabels, no inferred gfx arch, "gpu none" in the hardware report, and the installed +rocm venv judged stale against a required "cpu". Wrapped in @(), matching the Intel scan in the same file and in install.ps1. The stale-venv branch under $InstallerManagedSetup then aborted with "re-run install.ps1 so it can replace the environment safely with rollback". install.ps1 is the caller on that path and had already done exactly that earlier in the same run, and its failure path moves the previous environment straight back, so every attempt ended on the state it started from and the next attempt reached the same verdict. That abort has now been reported from four unrelated triggers. It is replaced by the in-place torch repair the index-pin and CUDA-family changes already use, which needs no delete: install.ps1 invokes setup through the venv's own unsloth.exe, so python.exe is locked by the process running the script. Also: - Test-VenvTorchIsRocm, the AMD counterpart of Test-VenvTorchIsXpu. A faulted Adrenalin or HIP runtime makes `import torch` raise at the DLL load or hang, and the venv then read as "torch could not be imported" and got deleted. version.py on disk still names the wheel, so trust it and point at the driver. - Invoke-BoundedPythonProbe keeps the stderr it used to drain and discard, in both installers, and the stale-venv message prints it. A dead driver, a half-written wheel and a missing torch were previously one sentence. tests/studio/test_amd_venv_repair_loop.ps1 covers the ROCm disk read, the probe error plumbing against a real child process in both installers, and the source shapes. pwsh 7 answers 1 to a scalar .Count, so it cannot reproduce the 5.1 half of #8335 and does not claim to; that leg is the windows-latest row of cross-platform-parity-ci. Closes #8335. * Windows installer: correct the 5.1 diagnosis, make the test reproduce it, and close the one path the repair loop fix made worse Validation follow-up to the previous commit. The stated root cause was over-general and, as written, wrong. "A scalar's .Count is $null under Windows PowerShell 5.1" is not true: a String or an Int32 answers 1 on 5.1, exactly as on 7, so anyone checking the claim that way concludes there is no bug. $null comes back only for objects whose PSObject carries no Count of its own -- [pscustomobject], which Microsoft documents, and CimInstance, which it does not. Get-CimInstance returns the second kind, which is what the WMI fallback assigns. Measured on windows-latest, PowerShell 5.1.26100.33158 against a real CIM instance, with pwsh 7.6.4 on the same runner: 'a' 5.1: 1 7: 1 [pscustomobject]@{...} 5.1: $null 7: 1 CimInstance (one instance) 5.1: $null 7: 1 @(if (...) { ... }) 5.1: 1 7: 1 The conclusion and the fix are unchanged; the reason is now the right one. That also means the test could always have been real, rather than a source shape apologising for pwsh. Win32_OperatingSystem returns exactly one instance on any Windows host, which reproduces "the machine has exactly one AMD GPU" with no AMD GPU present. The suite now adapts to its host: under 5.1 it reproduces #8335 and regression-tests the @() wrap for real, under pwsh the unroll still runs and the consequence is reported as a shape. 93 checks, green on both, verified on a windows-latest runner under powershell.exe 5.1.26100.33158 and pwsh 7.6.4. The claim that the 5.1 leg is covered by cross-platform-parity-ci was wrong: every step in that workflow is shell: pwsh, which is PowerShell 7. A shell: powershell step needs adding beside the pwsh one -- 5.1 is the runtime the CLI actually launches setup with (unsloth_cli/commands/studio.py). Left out of this commit only because the push token has no workflow scope; the exact step is on the PR. The in-place repair had one path it made worse than the abort it replaced. A venv directory with no Scripts\python.exe is incomplete, not stale, and has no interpreter to force-reinstall torch through. The abort used to catch it by catching every stale verdict. Without it the run reaches the activation -- a dot-source, which is NON-terminating at the "Continue" the pip section runs at -- so setup carried on with the venv unactivated, resolved every later python and uv pip against PATH, installed the whole stack outside the environment, and could still exit 0. Narrow to reach, since install.ps1 launches setup through that venv's own unsloth.exe, but silent when reached. Now refused, before the activation, without deleting anything. Also: - The surfaced probe error no longer indexes [0] into a pipeline that a whitespace-only stderr empties; fatal under a caller's Set-StrictMode, and studio/setup.bat does not pass -NoProfile. Confirmed to throw on 5.1 too. - $script:PinChangedForceReinstall is hoisted beside $installedTorchTag. Four install arms read it to decide --force-reinstall and a fresh install never reaches the assignment. - Test-VenvTorchIsRocm's comment claimed AMD's Windows wheels are labelled +gfx1151. They are not. repo.amd.com/rocm/whl/<arch>/torch/ publishes torch-2.11.0+rocm7.13.0-cp312-cp312-win_amd64.whl and keeps the arch in the URL only -- the same filename is a different binary under gfx1151, gfx110X-all and gfx120X-all. download.pytorch.org publishes the two-component +rocm6.4, Linux only. +rocm7.13.0 is what the #8335 reporter ended up on, and is now a case. The +gfx arm stays as deliberate defence. * Run the AMD repair loop tests under Windows PowerShell 5.1 as well Every step in cross-platform-parity-ci is shell: pwsh, so the whole job runs under PowerShell 7. 7 returns .Count = 1 for the single CimInstance that 5.1 returns $null for, which is the entire mechanism of #8335, so the existing pwsh step passes just as happily against the unfixed code. The suite had no leg on the shell the CLI actually launches setup.ps1 with. Adds the same test file under shell: powershell on the Windows row only. * Windows installer: keep the installer's GPU wheel through an in-place repair, refuse a venv with no activation script, and bound the AMD fast-path torch probe * Windows installer: keep the installer's GPU wheel through a GPU-to-GPU rescan, not just a CPU one * Windows installer: verify the venv activation actually took effect, not just that Activate.ps1 exists A present Activate.ps1 is not an activated venv. The script prepends the venv to PATH in its last statement, so a copy truncated by an interrupted or out-of-disk python -m venv runs to its last complete statement and returns without raising anything at all, and an unparseable one is a ParserError, which is non-terminating at the Continue the pip section runs at. Either way the dot-source succeeds with the ambient interpreter still first on PATH, which on a real install is the system or Microsoft Store python because install.ps1 keeps the venv Scripts directory off PATH on purpose. Fast-Install hands exactly that to uv pip install --python, every Exit-SetupFailure after it keys off an exit code from the wrong interpreter, and setup exits 0, at which point install.ps1 commits over its rollback copy. Assert the post-condition instead of adding a third existence check: the python now in effect must live under $VenvDir. VIRTUAL_ENV would not do, since Activate.ps1 sets it before the line that matters. Both sides are normalised through Get-Item, the same call Activate.ps1 uses to build the PATH entry, so a short 8.3 path, a substituted drive, a junction or a differently cased drive letter cannot read as outside, and every branch that cannot prove the interpreter is wrong returns rather than refusing a working install. * Windows installer: only preserve the GPU wheel install.ps1 actually selected The guard that keeps an installer-placed GPU wheel through a rescan treated any non-cpu wheel under UNSLOTH_INSTALL_ROLLBACK_MANAGED as the installer's own choice. That is not true on an upgrade: install.ps1's migrated-venv arm installs unsloth alone and never touches torch, and its flavor repair no-ops whenever its own expected tag is cpu or unrecognised, so a legacy ~/.unsloth/studio/.venv can hand setup a +cu118 wheel left by a previous install on different hardware. Preserving that pinned the whole dependency pass onto a cu118 index, and on a mapped AMD host the kept cu* tag also blocked the ROCm reroute, which needs $CuTag -eq "cpu" -- so the Radeon never got a ROCm wheel and setup still exited 0. install.ps1 now reports the family it settled on in UNSLOTH_INSTALLER_TORCH_TAG and setup preserves only a wheel that matches it. Absent or blank means the installer did not say (an older cached install.ps1, or --no-torch) and falls back to preserving, so an older installer keeps exactly its previous behaviour. A wheel the installer did not choose repairs in place, as it did before the guard existed; nothing aborts. * Windows installer: tighten the comments added across the AMD repair-loop rounds Four review rounds each re-explained the same mechanisms, so the exit-0 hazard, the @() unroll and the preserve guard were each written out in several places. Keep the full reasoning at one site per mechanism and cut the restatements. Comments and whitespace only; no executable change. * Scope ErrorActionPreference around the corrupt-activation child process The Windows PowerShell 5.1 leg of the parity workflow failed at the corrupt activation script case. Every assertion passed, then the run ended with a NativeCommandError. 5.1 wraps a native command's stderr in a NativeCommandError and applies $ErrorActionPreference to it, so the parse error the corrupt script writes to stderr became terminating under the Stop set at the top of the file. PowerShell 7.1 stopped applying the preference to native stderr, which is why the same script passes under pwsh and failed only on 5.1. Set the preference to Continue for the duration of the child invocation. The exit code is what the case actually judges and it is still read and asserted. --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> |
||
|
Daniel Han
|
5c6a01fa0c
|
Stop the PowerShell progress bar throttling installer downloads to 0.65 MB/s (#8476)
* Stop the PowerShell progress bar throttling installer downloads to 0.65 MB/s Windows PowerShell 5.1 redraws the Invoke-WebRequest progress bar on every read, and the redraw costs far more than the transfer itself. Measured on a windows-latest runner, the same 43 MB file takes 70.79s with the bar on and 0.34s with it off. The rate does not scale with the connection: the three installer downloads all land at 0.64-0.68 MB/s regardless of size or host, so the progress bar caps throughput at roughly 0.65 MB/s and a user on gigabit waits exactly as long as one on DSL. On the real URLs: python.org installer 41.34s -> 0.08s uv archive 26.82s -> 0.09s vc_redist.x64.exe 38.18s -> 0.29s setup.ps1 is spawned as powershell.exe by 'unsloth studio update', so 5.1 is the interpreter that actually runs it. -UseBasicParsing does not avoid this; only the preference does. PowerShell 7 is unaffected, so this is a no-op there. Both assignments are scoped. install.ps1 sets it inside Install-UnslothStudio with no scope qualifier, the same shadowing the prologue already relies on for PSDefaultParameterValues, because 'irm | iex' runs in the caller's own session and a global assignment would permanently disable their progress bars. setup.ps1 is a short-lived -NoProfile child process. Verified the bytes are unchanged: each URL was fetched with the bar on and off and the SHA256 matched. A full install with and without the change produced manifests differing only in six files that also differ between two unpatched installs (install id, timestamps), so 41284 of 41290 files are identical and the change accounts for none of the difference. * Cite the measurement from the call site each comment annotates Both comments quoted a 43 MB / 70.79s / 0.34s figure taken from a separate probe against the desktop release asset, not from the downloads they sit on, and setup.ps1 then gave two different sizes for the same file (43 MB in one sentence, ~25 MB in the next). Use each call site's own number instead: 27.8 MB / 41.34s / 0.08s for the python.org installer, 24.4 MB / 38.18s / 0.29s for the VC++ runtime, both from the same windows-latest run as the PR description, so the comment is checkable against it. Also drops 'PowerShell 7 is unaffected, so this is a no-op there'. True of the cost but not of the behaviour: on 7 the bar is still suppressed, there is just nothing to gain. 'PowerShell 7 never had the cost' says the intended thing. Comments only; the two assignments are untouched. --------- Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> |
||
|
Daniel Han
|
aaf994881b
|
Windows: verify the publisher of the installers we download and run (#8418)
* Windows: verify the publisher of the installers we download and run Both Windows fallbacks fetch an executable over HTTPS to the temp directory and Start-Process it immediately: the VC++ runtime in studio/setup.ps1 when winget is absent or fails, and the python.org installer in install.ps1. HTTPS vouches for the transfer, not for what arrived, and the process that runs it may be elevated. Neither URL can be pinned to a committed SHA-256 the way install_node_prebuilt.py pins the Node archives. aka.ms/vs/17/release is evergreen and its bytes change with every VS servicing update, and the python.org patch version is resolved at runtime from the directory listing. Verify the publisher instead. Checking the Authenticode status alone would not help, since any code-signing certificate from any trusted CA passes it. The signer subject is checked too, so the chain has to lead back to Microsoft or the Python Software Foundation. Both failure paths are the ones already there. The VC++ throw lands in the existing catch, which prints the same yellow line and falls through to the manual install instructions, and the python.org check returns $null exactly like the download failure two lines above it, so the caller still falls back to uv and astral.sh. * Treat an unreadable signature as a verification failure Get-AuthenticodeSignature can fail on the file itself rather than on its signature: antivirus quarantining the download before we inspect it, or the path becoming unreadable. install.ps1 sets $ErrorActionPreference = "Stop" at the top, so that error was terminating and escaped Install-PythonFromPythonOrg entirely, skipping the $null return the caller relies on for its fallback and leaving the downloaded executable in the temp directory. Confirmed under pwsh: the error propagates out of the function and the cleanup line never runs. Unreadable is unverified, so it now takes the same route as a bad signature: a yellow substep, remove the file, return $null. setup.ps1 already ran its check inside a try with a finally that removes the file, so only the install.ps1 path needed this. Also accept a quoted RDN value in both publisher checks. A subject can arrive as O="Microsoft Corporation", which the unquoted pattern would have rejected. * Tighten the comments on the two signature checks |
||
Wasim Yousef Said
|
f567ae8f39
|
Studio: skip redundant packaged frontend rebuilds (#8326)
* Desktop: skip frontend rebuild during updates * Tests: tolerate rustfmt in updater UTF-8 contract * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use packaged frontend for PyPI installs * Studio: keep the packaged frontend skip off source checkouts STUDIO_LOCAL_INSTALL records where the Python package came from, not which tree setup runs out of. An editable overlay separates the two: with UNSLOTH_CI_SOURCE_OVERLAY, or in a venv left editable by an earlier --local run, the mode stays 0 while SCRIPT_DIR is a checkout whose dist is a stale build artifact rather than a release one. The skip then serves that stale dist and a source change silently never reaches the browser, which is the outcome the overlay legs of clean-machine-install-ci exist to catch. A wheel ships no top-level files, so a pyproject.toml next to studio/ marks the tree as source. Require its absence before trusting the packaged dist; site-packages installs are unaffected and still skip. Also check the Tauri branch before the packaged one in setup.ps1 so a desktop update reports the same reason it reports on POSIX. Covered by new cases in tests/sh/test_packaged_frontend_skip.sh and tests/studio/test_node_decision.ps1. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
3a58fa5c41
|
Studio: apply base.txt on the install.sh and install.ps1 paths (#8195)
* Studio: apply base.txt on the install.sh and install.ps1 paths
install.sh and install.ps1 install unsloth and unsloth-zoo inline, then
export SKIP_STUDIO_BASE=1 so setup.sh / setup.ps1 do not install the same
two packages a second time. install_python_stack.py read that flag as
"skip base.txt" and short-circuited the whole step:
if skip_base:
pass
That was the same thing only for as long as base.txt held nothing but
those two names. Add a third, pinned entry to base.txt and it reaches no
fresh install on any platform: neither installer reads the file, and the
one branch that does was skipped. It would only land later, if the user
happened to run `unsloth studio update`.
Every install.sh and install.ps1 path was affected, on every platform:
CUDA, ROCm, XPU, CPU, macOS, local and non-local, fresh and migrated.
Keep skipping the two core packages, which is all the flag was ever
meant to avoid repeating, and apply whatever else base.txt asks for.
When base.txt holds only the core packages, as it does today, there is
nothing left to install and no extra subprocess runs. No-torch mode is
untouched: it has its own list in no-torch-runtime.txt, which the
installers do apply inline.
The core-package filter parses the project name rather than matching on
a prefix, so a future unsloth-<something> pin is not swallowed too.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reconcile base requirements with current main
* Preserve relative requirements includes across filters
* Fix filtered requirements test cleanup
* Separate core and shared base requirements
* Preserve shared base requirement resolution
* Keep the filtered-requirements and uv alias paths from aborting an install
The adjacent temp copy raised PermissionError on a read-only requirements dir, and a symlink failure handed uv back the spaced path it cannot read. Fall back to the temp dir and to a copy respectively, and stop the real-extras tests leaving filtered files in the tree.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Count the MLX slot and survive an unusable base.txt
Simulating every install path showed two gaps. base_total never counted the Apple Silicon MLX step, so `studio update` there ran 13 steps out of a declared 12 and recorded the wrong steps_total. And the new base.txt read happens before the manifest is dropped, so a missing or unreadable file aborted with a traceback where the old code reached pip; a BOM also read as content and scheduled an empty step. Progress coverage now spans both core paths on all four platforms.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the unreadable base.txt case independent of the mode bits
chmod(0o000) denies nothing as root, which containerized test jobs run as, and Windows does not implement POSIX modes at all, so the case asserted None against a file it could still read. Raise from a patched read instead.
* Tighten the comments this PR adds
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
|
||
|
Daniel Han
|
07df95079e
|
Studio: route every Windows installer line through the UTF-8 stdout sink (#8148)
* Studio: route every Windows setup line through the UTF-8 stdout sink The desktop setup log rendered "?? Unsloth Studio Setup" over a rule of replacement characters. Tauri spawns Windows PowerShell 5.1 with CREATE_NO_WINDOW (install.rs), so the [Console]::OutputEncoding setter throws and both entry scripts rebind [Console]::Out to a UTF-8 writer. step/substep already write only through that writer when stdout is redirected, so they came out right. Every other line did not: Write-Host is written by 5.1's console host with its own writer on the OEM code page, and U+1F9A5 has no OEM form while U+2500 becomes a bare 0xC4, which from_utf8_lossy turns into U+FFFD. The banner and the footer are not steps, so they kept arriving as mojibake, and install.ps1 had neither the IsOutputRedirected probe nor a mirror at all. Add Write-StudioLine above the first write in studio/setup.ps1 and install.ps1: console handle when redirected, Write-Host when interactive, since it is the only writer that colorizes. Rewrite 164 call sites in setup.ps1 and 155 in install.ps1 onto it, including install.ps1's own step/substep. Write-Host now survives only inside helpers that have already ruled out the redirected sink, and the launcher script install.ps1 generates keeps its own, since it runs as a separate process. No behaviour change for an interactive console user: same text, same colors, same single record per line. test_windows_setup_output_encoding.py gains byte-level coverage that the real banner and footer, sliced out of setup.ps1, survive both launch shapes as valid UTF-8 exactly once, plus a source contract that runs on Linux and names any file:line that reaches for Write-Host outside the allow-list. Studio.Setup.Output.Tests.ps1 covers Write-StudioLine in both modes and pins install.ps1's copy to setup.ps1's. Harnesses that splice these scripts apart now stub or dot-source Write-StudioLine: two PowerShell harnesses, one Python harness, and the VC++ redist leg of studio-windows-inference-smoke. pytest tests/python tests/test_installer_*.py: 1077 passed (2 pre-existing sandbox failures unrelated to this change). All 16 tests/studio harnesses and 57 Pester cases pass. Both scripts parse clean. * CI: spawn install.ps1 as a child process so its lines reach install.log * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stub the output sink in the llama.cpp backend PowerShell harness * Guard the console-less spawn on a Windows runner The byte-level cases in this file run with a console attached, and a GitHub runner gives a CREATE_NO_WINDOW child one, so the UTF-8 setter succeeds there and every version of these scripts emits a clean banner. Those cases cannot tell this fix from what preceded it. Add cases that call FreeConsole() in the child first, which is the state install.rs's own comment assumes CREATE_NO_WINDOW produces. There Write-Host has no screen buffer to query, throws, and takes the script down: 2 bytes of stdout and exit 1 rather than the banner. The probe is assembled entirely out of text sliced from the script under test and spawned with install.rs's own interpreter, flags and creation flags. No Windows job ran this file, so its byte-level half was only ever exercised under pwsh 7 on the Linux Backend CI leg, which is UTF-8 by default. Add it to the cross-platform parity matrix, which already has a windows-latest row and already triggers on install.ps1 and studio/setup.ps1. * Report skips in the parity step A platform-gated case that stopped running on the row it exists for still reports green with -q alone. * Slice the error preference too It is what turns the Write-Host throw into a dead script rather than a skipped line, so restating it would be assuming the result. * Say what the comments actually mean * Make the console-less cases fail on a lost banner, not just a mangled one * Stub the output sink in every harness that splices these scripts The Write-Host rewrite left four spliced-source harnesses reaching Write-StudioLine without defining it. An undefined command is a terminating error, so each one either aborted or was swallowed by the harness's own catch, and the test kept passing while no longer testing anything. - test_windows_python_venv_hardening.py, partial-rollback case: the five-line split-move warning was lost. The assertion that "both halves are named" only stayed green because $existing is a prefix of the rollback dir, so it matched the dir= line instead. Pin it to the warning text. - test_path_probe_access_denied.ps1, ownership guard: the catch scored the command-not-found as the intended failure and never reached Exit-SetupFailure. Pin the check to the EXIT-SETUP message. - test_windows_installer_concurrency_guard.py: the decision block prints before Exit-InstallFailure, so on Windows the active case aborted at exit 1 and never produced RESULT:blocked. - Studio.Setup.Vs2026.Tests.ps1: on a host without cmake, Ensure-BuildToolsForLlamaSourceBuild hits the sink first and the no-op case fails on the throw. Also stub the three remaining harnesses that splice sink-calling helpers but do not reach the sink on the paths they exercise today, so the next case added to them cannot reintroduce this. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
oobabooga
|
69d555b98e
|
Studio: check llama.cpp cache access before setup (#8032)
* Windows: preflight managed llama.cpp cache access before setup and update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix denied llama.cpp cache update errors * Trim canonical paths on both branches and probe listing on POSIX for PR #8032 Follow-ups from reviewing the denied llama.cpp cache preflight. Get-CanonicalDir only trimmed a trailing separator on the lexical branch. Resolve-Path preserves one, so "...\studio\" compared unequal to "...\studio" and Test-StudioHomeIsCustom called the default home custom. Get-ManagedLlamaCppDir then handed the preflight ...\studio\llama.cpp instead of the real cache, so a denied cache went undetected. The trim now runs after both branches, still guarded so a path root keeps its separator. setup.sh gated the prebuilt install on _studio_dir_unsearchable, which probes search (+x). Mode 111 passes that and still raises PermissionError inside install_llama_prebuilt.py, which lists the tree. Added _studio_dir_unreadable for call sites that list or replace rather than probe a known child. setup_fail only emitted [TAURI:ERROR] under UNSLOTH_TAURI_MODE. update.rs sets UNSLOTH_TAURI_UPDATE on every platform, so macOS and Linux desktop updates still showed "Update exited with code N" while Windows showed the reason. A blank USERPROFILE threw a raw binding error from the relocated resolver before Exit-SetupFailure could report it, so no [TAURI:ERROR] reached the app. Tests: the marker-readable, listing-denied case was asserted only as a literal substring, so one extra space reintroduced the early return with all 49 Python tests and all 125 PowerShell checks green. That assertion is now a regex, and the PowerShell suite builds the shape for real (icacls /deny :(RD) on Windows, mode 111 on POSIX) with a negative control. The shell suite gains the same mode 111 case and no longer compares empty grep output as an integer. * Fix the Tauri marker gate and narrow the USERPROFILE guard for PR #8032 Two defects in my previous commit, both found by simulating the changes rather than reading them. setup_fail joined both variables into one case subject, which is not an exact match. "*,1" matches any subject ending in ",1", so an unrelated UNSLOTH_TAURI_UPDATE=a,1 printed a stray [TAURI:ERROR] on a plain CLI run, and "1,*" did the same for a comma in UNSLOTH_TAURI_MODE. Testing each variable separately matches setup.ps1, which uses exact membership. Verified over the 144-case product of both variables: no false positives, no false negatives, exit codes preserved, and byte-identical to main for every value of UNSLOTH_TAURI_MODE when UNSLOTH_TAURI_UPDATE is unset or 0, so no existing CLI invocation changes. Added a regression test that fails on the old gate. The USERPROFILE guard used IsNullOrWhiteSpace, but Join-Path only rejects null and empty; it accepts a whitespace-only value. The guard therefore also stopped a run that previously completed, USERPROFILE=" " with a fully qualified UNSLOTH_STUDIO_HOME. IsNullOrEmpty keeps the clean message for null and empty, which is where the raw binding error was, and changes nothing else. * Run the llama.cpp access guard before the prebuilt and source branches The guard sat inside the prebuilt else-branch, so UNSLOTH_LLAMA_FORCE_COMPILE=1, a llama.cpp PR or source override, or anything else setting _SKIP_PREBUILT_INSTALL bypassed it. Those paths reach the phase 9 swap, which only probes access after `rm -rf "$LLAMA_CPP_DIR"` has already failed, so a denied cache stranded a completed source build instead of failing before it started. _assert_studio_owned_or_absent does not cover it either: it returns early unless the studio home is custom, so a denied default cache had no guard on that path at all. Verified by driving both helpers against a mode-000 tree: the ownership guard returns 0 on a default home while the access probe reports denied. Hoisted both checks, in the same order so the custom-home wording still wins, to just before the branch. The local-link paths are excluded because they already replaced or reused the tree. The late checks stay as defense in depth. * Use the read probe in both llama.cpp replace postconditions Mode 111 defeats `rm -rf` but stays searchable, so both postconditions fell through `_studio_dir_unsearchable` to the generic "could not be replaced" text and the user got no recovery guidance. Reproduced against a mode-111 tree: the rm fails, the search probe does not fire, and the run exits with the generic message. The local-link site sits above the hoisted guard, so it is the one that is reachable. The source-build site is only reachable when a tree becomes unreadable during the build, but that is the most expensive path to end with the wrong message, and it is the same one-word probe. Both now use `_studio_dir_unreadable` and print the permissions block. * Tighten the comments added by this PR --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <moonshotaisubstack@gmail.com> |
||
Michael Han
|
6f4cce68d1
|
Setup: clear stale WebView caches on install and update, keep user settings and data (#7361)
* Setup: clear stale WebView caches on install/update, keep user data
The desktop app's WebView caches (keyed by the Tauri bundle id
ai.unsloth.studio) hold copies of the previous frontend and keep serving
them after an update, so old styles linger even though the code on disk
is new. Clear cache-only paths on every install/update via setup.sh and
setup.ps1 (both fresh installs and 'unsloth studio update' route through
them).
Cleared: the HTTP/network caches, CacheStorage, service workers, and GPU
caches for the bundle id. Kept: LocalStorage, IndexedDB, cookies,
settings, models, and the studio database.
macOS: ~/Library/Caches/<bid> and WebsiteData/{CacheStorage,
ServiceWorkers,DiskCache}. Linux: XDG cache dir. Windows:
EBWebView\Default\{Cache,Code Cache,GPUCache,Service Worker}.
Adds tests/sh/test_setup_webview_cache_clear.sh covering both OS
branches, XDG overrides, and that user-facing storage survives.
* Address review: Linux data-dir caches, bash invocation, pre-webview clear
Three review findings, all verified:
1. wry keys the WebKitGTK base-cache dir to the app DATA dir (same as
base-data), so on Linux the stale frontend cache also lives under
~/.local/share/ai.unsloth.studio. Clear the cache-typed subdirs there
(WebKitCache, CacheStorage, serviceworkers) while keeping localstorage,
indexeddb, and cookies. Tests extended to 23 assertions.
2. run_all.sh invoked the new test with sh, but the extracted setup.sh
function uses bash arrays; dash aborts with a syntax error before any
assertion. Invoke with bash.
3. The in-app desktop update runs start_backend_update before
downloadAndInstall/relaunch, so setup.ps1/setup.sh clear caches while
the live WebView still holds them and silently fail. Clear the caches
from the Rust side in main() before the Builder runs: the config window
(and the WebView lock) exists by the time setup hooks fire, so this is
the one point where the profile is guaranteed unlocked. Same cache-only
path lists per OS; cargo check passes.
* Address review: gate the native clear on the app version, correct the macOS paths
* Tighten the comments added for PR #7361
* Serialize the WebView cache clear, stamp only a full clear, ignore relative XDG_DATA_HOME
* Take the profile lock before the stamp check, and detect dangling symlinks in the test
* Clear WebView caches only after the install-root override is validated
* Invalidate the app version stamp when setup clears the WebView caches
* Tighten the comments in the WebView cache clear
* Clear WebView caches after validating the override on Windows too
* Only clear WebView caches for a real Studio installation
* Tighten the comments added since the last pass
* Clear the WebView cache after single-instance arbitration
Resolve the profile through dirs::data_local_dir, the same call Tauri's
PathResolver makes for BaseDirectory::LocalData, so a launch with HOME or
LOCALAPPDATA stripped still finds the profile Tauri is about to use.
Move the clear into a plugin registered directly after single-instance.
Plugin setup hooks run inside Builder::build() in registration order and the
config window is only created later from App::run(), so a duplicate launch
has already exited by then and the WebView does not exist yet.
Make the no-venv fixture hermetic: report no system node/npm and opt out of
the isolated install so setup.sh skips Node provisioning and the frontend
build, and assert it still aborts at the venv check.
* Shorten the comments added in the last commit
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
|
||
|
Michael Han
|
89206b63a8
|
Studio: cap llama.cpp source build parallelism by RAM (#8061)
* Studio: cap llama.cpp source build parallelism by RAM The source build passed the logical core count straight to cmake -j. On a 20-thread 16 GB machine that is -j20, and each nvcc translation unit peaks near 2 GB, so the build oversubscribed memory and left the machine unresponsive with no way to stop it. Budget the job count from total physical RAM instead: reserve 2 GB for the OS, allow 2 GB per job, and clamp to the core count. RAM is only ever a ceiling, so machines with headroom keep every core. When RAM cannot be read the job count falls back to the old core count rather than guessing low. UNSLOTH_LLAMA_BUILD_JOBS overrides both. setup.ps1 carries its own copy because it cannot source setup.sh; the reserve and per-job constants are pinned to matching values by the test. * Address review: honour the cgroup memory limit when capping jobs /proc/meminfo is not namespaced, so inside a memory-limited Docker or Kubernetes container it reports the host's RAM. A 4 GB container on a large host therefore still budgeted from the host and would be OOM-killed during a source build, which is the same failure this cap exists to stop. Read the cgroup v2 then v1 memory limit and take the lower of it and MemTotal. v2's 'max' and v1's unlimited sentinel are not limits: the former does not parse, the latter loses the comparison on its own. Windows has no cgroups, so setup.ps1 is unchanged. * Address review: resolve the binding cgroup, and budget from available memory Two follow-ups on the cap. The first cgroup reader only looked at the hierarchy root. That is the binding limit inside a container with a private cgroup namespace, but under Slurm, systemd slices or --cgroupns=host the limit sits on the process's own path or an ancestor's, and the root reads 'max'. Those environments kept the original OOM behaviour. Mirror unsloth/dataset_num_proc.py instead: read /proc/self/cgroup, walk innermost-first to the root, pair each limit with the usage of the directory that set it, and take the smallest remaining allowance. memory.high binds as much as memory.max. MemTotal also ignored what the machine is already doing, so a 16 GiB box with 8 GiB resident still got a 14 GiB compile budget, which is the thrashing this change exists to stop. Budget from MemAvailable, which counts reclaimable page cache, falling back to MemTotal on pre-3.14 kernels. Windows has no cgroups but the same installed-versus-available gap, so it reads AvailableMBytes from the raw perf class, which is not localized the way Get-Counter paths are, and keeps installed RAM as the fallback. The readers take the cgroup root and /proc/self/cgroup as arguments, so the tests drive real trees rather than faking /sys. * Address review: resolve the cgroup mounts instead of assuming their paths The v1 branch assumed the memory controller sits at <root>/memory. It usually does, but a co-mounted or relocated hierarchy was skipped even when /proc/self/cgroup named a memory controller, and the budget then fell back to host memory, which is the silent revert this cap exists to prevent. The v2 branch assumed the unified mount is the root, which systemd hybrid mode breaks the same way. Resolve both from /proc/self/mountinfo: match fstype cgroup2 for the unified mount, and the memory controller by super option for v1, so a co-mount is found by name rather than by path. The conventional layout stays as the fallback when mountinfo is unreadable. unsloth/dataset_num_proc.py still assumes <root>/memory for v1; this is deliberately the more thorough of the two rather than a drift. * Address review: map the process path through the cgroup mount root _cg_mount kept mountinfo field 5 and discarded field 4, the filesystem root that mount exposes. A bind-mounted subtree, which is what Docker without a cgroup namespace and systemd slices produce, shows a mount root like /slice while /proc/self/cgroup still reports the host-absolute /slice/job, and the files are then at <mountpoint>/job. Joining the two unmapped walked a path that does not exist, so the walk settled on an outer limit instead of the binding one and over-estimated the allowance. Return the mount root alongside the mount point and strip it from the process path before walking. A process outside the mount's root gets only that mount's own limit, since nothing below it describes that process. * Address review: pick the cgroup mount that contains the process A hierarchy can be mounted more than once, and _cg_mount stopped at the first match. An unrelated subtree listed ahead of the binding mount meant the process path did not map into the chosen mount, so the hierarchy was dropped and the budget fell back to host memory. Collect every matching mount and choose one whose root contains the path from /proc/self/cgroup, most specific first, keeping the first seen when none of them does. The process path is now read once, before mount selection, since the choice depends on it. * Address review: decode mountinfo escapes, keep colons in cgroup paths mountinfo represents a space, tab, newline or backslash in a path as an octal escape, and both path fields were used verbatim, so a hierarchy mounted under such a path resolved to a directory that does not exist. Decode field 4 and field 5 before use. strtonum is a gawk extension, so the conversion is plain arithmetic that also runs under mawk and BSD awk. /proc/self/cgroup was split with -F: and read as field 3, which truncates the path at the first colon inside it. A systemd unit name may contain one. Only the first two colons are delimiters, so parse those and keep the remainder, for both the v2 line and the v1 controller lines. * Address review: keep zero available memory, and stop the newline transports The mount point is an arbitrary directory and may contain a newline, which mountinfo escapes as \012. Decoding it where the fields are read put that newline back into the reader's own line-oriented output, splitting one mount record into two, and _cg_dirs then walked a line-delimited list of paths that could be split the same way. Both records now travel escaped and NUL-delimited, and a single path is decoded once it is in hand. On Windows, an AvailableMBytes of 0 is a reading, not a failure. Falling back to installed RAM there handed a machine with nothing left its full core count, which is the case this cap exists for. Unreadable is now -1. The suite's host-memory assertions compared a cached MemAvailable against a second live read, which raced on Linux. _usable_ram_mb takes its meminfo path like every other reader here, so the numbers are pinned. * Pin the trailing-newline sentinel on the decoded mount path * Address review: budget macOS from available memory too The macOS branch still read hw.memsize, which is installed RAM, so a 16 GiB Mac with 8 GiB resident was handed a 14 GiB compile budget, the same gap MemAvailable closed on Linux. macOS has no MemAvailable. free + inactive + speculative + purgeable is the reclaim-aware equivalent, and the page size comes from the vm_stat header rather than being assumed 4096, which is wrong on Apple Silicon. Installed RAM stays the fallback when the output does not parse. Verified on a 64 GiB Apple Silicon host: 65536 MiB installed, 33157 MiB usable, 15 jobs rather than the bare core count. * Address review: keep a failing read from aborting the install setup.sh runs under `set -euo pipefail` and NCPU=$(_llama_build_jobs) is on the install's critical path, so a helper that returns non-zero does not degrade the job count, it aborts the install at the build step. The old NCPU line could not fail; this one calls ten helpers. _cg_read ran `head | tr`, and `[ -r "$1" ]` does not rule the pipeline failing out: a directory passes it, and a cgroup can be torn down between the test and the open. It now reads with the builtin and strips with an expansion, so there is no pipeline and no subprocess left to fail. The two awk calls in _cg_mounts and _cg_unesc are guarded for the same reason. The suite sourced the helpers into a plain `bash -c`, which is why none of this was visible. It now also drives them with the real shell options on, including the two AND-lists in _llama_jobs_for that are the classic footgun. * Guard the remaining reads, and pin them under POSIX mode bash applies errexit to a failing assignment in POSIX mode, which it does not do by default, so `m=$(awk ...)` inside a helper aborts the install for anyone with POSIXLY_CORRECT exported or bash invoked as sh. The meminfo and cgroup reads are guarded like the others, and the suite now drives the helpers under --posix as well, which is what makes those guards observable. Also pins the two properties the earlier guards left untested: _cg_read returns rather than blocking on a FIFO, which is what the -f test is really for, and a padded value is still parsed, which is what the whitespace strip is for. * Pin the read guards in the form that distinguishes them The POSIX assertions passed the reader as an argument, where the failure status is discarded, so they held with the guards removed. The distinguishing form is the assignment, which is what the real call site uses, so they assign now. Removes the guard on the _cgroup_free_mb read: that function is guarded internally and ends in `return 0`, so it cannot fail, and a guard no test can make fire is worse than none. * Ground the per-job budget in a measurement The constant was justified as "an nvcc translation unit peaks near 2 GiB". Measured against llama.cpp master with CUDA 13.1, the heaviest units are the flash-attention template instances at ~400 MiB, flat in arch count, and a full CUDA build at -j20 peaks at 8.2 GiB in aggregate across ~30 concurrent compiler processes. The number does not change: 2048 still has to cover the process fan-out, MSVC and hipcc which are not measured here, older toolkits which were far heavier on the same files, and the link step. But the comment now says what was measured and why the budget sits above it, rather than asserting a figure the current toolchain does not produce. * Bring the macOS reader under the same guards, and make its test run on Linux Two things on the vm_stat commit, both the same shapes already fixed on the Linux side of this change. _vm_stat_avail_mb ends in an unguarded awk, so under POSIX mode (POSIXLY_CORRECT in the environment, or bash invoked as sh) a vm_stat that does not parse aborts the install rather than falling back to installed RAM. And "_usable_ram_mb prefers vm_stat over hw.memsize" says it runs on Linux runners, but the branch it targets is an elif on sysctl succeeding, and Linux sysctl has no hw.memsize, so the branch was never entered and the assertion held vacuously off a Mac. It stubs sysctl now, which is what actually reaches it, and a companion assertion covers the fallback when vm_stat gives nothing. Mutation-checked: unguarding the awk fails 2, ignoring vm_stat fails 3, assuming a 4096 page size fails 4, counting active and wired fails 3. * Restore the vm_stat awk guard Belongs with the previous commit; the guard was dropped when the tree was reset between the mutation checks and the commit. * Keep a zero vm_stat reading rather than treating it as a parse failure Get-UsableMemoryMb returns 0 when AvailableMBytes is 0 and reserves -1 for an unreadable one. The shell reader did not make that distinction: a Mac with nothing reclaimable read as unparseable and fell back to hw.memsize, which is installed RAM, so the machine got its full core count. That is the oversubscription this PR exists to remove, on the one input where it matters most. Zero is a reading now; only a missing page size is not. * Treat a zero cgroup limit as a limit _cg_limit rejected a zero-byte limit as though it were the absence of one, so a cgroup that permits no further memory budgeted from host MemAvailable and took the full core count, which is the oversubscription this cap exists to remove. Only memory.high can be observed at zero: it throttles rather than killing (cgroup-v2.rst, "Going over the high limit never invokes the OOM killer"), so a process really does run under systemd's MemoryHigh=0. memory.max of 0 invokes the OOM killer, so nothing survives to read it. Zero is now a reading here, the same as on the Windows and macOS sides. Mutation-checked: restoring the -gt 0 test makes the fixture report its ancestor's 4 GiB instead of 0, and fails the new assertion. * Inspect every containing cgroup mount, and make the suite hermetic A limit above the narrower mount's root is invisible through that mount and visible through the broader one, so picking only the most specific hid it. The same hierarchy really is mounted twice with different subtree roots: rootless podman inside rootless podman leaves a host-derived bind mount beside a namespace-scoped one (containers/podman#21376). _cg_pick_mounts now yields every mount whose root contains the process path and the walk takes the smallest allowance across all of them, which is order-independent and can only lower the budget, never raise it. Three test-isolation defects alongside it, each of which would have gone red on a runner rather than on this box: - _usable_ram_mb hardcodes /sys/fs/cgroup, so inside a memory-limited container the assertions about HOST memory received the container's allowance instead of the fixture. In a 4 GiB cgroup they return 1 where they expect 5 and 20. The clean-machine install CI runs in containers, so this was reachable. Anything not testing the cgroup reader itself now stubs it out. - The FIFO case ran wherever mkfifo exists, but stock macOS ships no GNU timeout, so it exited 127 and failed a correct reader. It requires both now. - The suite inherited an exported UNSLOTH_LLAMA_BUILD_JOBS into every direct helper call, so a developer with the override set saw unrelated failures. It is unset once at the top; run_jobs still sets it explicitly per call. Mutation-checked: most-specific-only fails 2, and the isolated zero-limit fixture now fails 2 rather than 1 when the -gt 0 test is restored. * Stop double-counting macOS reclaimable pages free + inactive + speculative + purgeable overstates what can be reclaimed, because neither of the last two is disjoint from the first two, and overstating available memory buys back exactly the oversubscription this cap removes. speculative is a subset of free, stated outright in xnu osfmk/mach/vm_statistics.h: "NB: speculative pages are already accounted for in free_count, so speculative_count is the number of free pages that are used to hold data that was read speculatively from disk". purgeable is an attribute of a page rather than a queue it sits on, so a volatile page is already counted on whichever of the active or inactive queues holds it; the disjoint partition is free + active + inactive + wired + throttled + compressor. The sum is free + inactive. That under-counts by the purgeable pages on the active queue, which are reclaimable but in neither term, and that is the safe direction here: it costs build time on a busy Mac rather than the machine. The fixture keeps non-zero speculative and purgeable counts so that adding either back is visible. Mutation-checked: restoring speculative fails 6, purgeable fails 6, both fails 6, and dropping inactive fails 6. * Tighten the build job cap comments --------- Co-authored-by: shimmyshimmer <133493246+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
d8effae0d5
|
Studio: fix the Windows desktop setup log mojibake and double-printed steps (#8083)
* Studio: fix the Windows desktop setup log mojibake and double-printed steps
The desktop 'Getting things ready...' log rendered as:
?? Unsloth Studio Setup
<52 replacement chars>
gpu
none (chat-only / GGUF)
gpu none (chat-only / GGUF)
Encoding. studio/setup.ps1 never set [Console]::OutputEncoding, so Windows
PowerShell 5.1 encoded redirected output with the OEM code page while the
desktop app decodes the pipe as UTF-8 (String::from_utf8_lossy in
src-tauri/src/install.rs). That corrupts two different ways: the sloth U+1F9A5
has no OEM representation so PowerShell substitutes one '?' per UTF-16
surrogate, and the rule U+2500 does have one, so it becomes a bare 0xC4 byte
that is invalid UTF-8 and surfaces as U+FFFD. Both entry scripts now set the
console encoding, $OutputEncoding, PYTHONUTF8 and PYTHONIOENCODING before the
first write, and Refresh-Environment can no longer reload the two Python vars
back over ours mid-run. The patch is ASCII-only: these files are UTF-8 without
a BOM and 5.1 parses those as ANSI.
Duplication. step/substep wrote through Write-Host AND a console-handle mirror.
The mirror's comment assumed Write-Host does not survive the process chain; it
does, because the CLI spawns setup.ps1 as -Command "& '...' *>&1"
(unsloth_cli/commands/studio.py), which merges the Information stream into
stdout deliberately. The sink is now resolved once and exactly one is used:
redirected writes to the console handle, interactive writes to Write-Host.
Splitting. step composed one logical line from two Write-Host calls using
-NoNewline, and a redirected consumer turns each Information record boundary
into a line break. Both scripts now emit one composed record; install.ps1 needs
this most, having no mirror to fall back on.
Rust children on Windows get PYTHONUTF8/PYTHONIOENCODING too, since install.rs,
update.rs and process.rs all decode their output as UTF-8. The readers stay
lossy on purpose -- strict decoding would turn display corruption into an
installation failure.
Tests: a Pester suite auto-discovered by the existing pester job, and a pytest
byte-level probe that runs real PowerShell in both the -File and -Command
launch shapes and asserts on raw bytes. Verified to fail against the unfixed
tree (9 Pester and 13 pytest failures) rather than merely passing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Bind a UTF-8 writer with no console, and pass -X utf8 to the isolated child
Two holes in the previous commit, both on the exact path the desktop app takes.
[Console]::OutputEncoding P/Invokes SetConsoleOutputCP, which needs a console
handle. Under CREATE_NO_WINDOW there is none, so it throws, and it drops the
cached writer BEFORE throwing while assigning OutputEncoding only after. Console
.Out therefore rebuilt on the old code page. Swallowing the exception was not
enough once redirected step/substep use Console.Out as their only sink, so the
catch path now binds an explicit UTF-8 StreamWriter over OpenStandardOutput.
build_update_command launches Python with -I, which implies -E, so that process
ignores every PYTHON* variable and PYTHONUTF8/PYTHONIOENCODING never reached it.
Pass -X utf8 as a switch instead. The env vars stay for its descendants.
https://docs.python.org/3/using/cmdline.html#cmdoption-I
* Bind the UTF-8 writer to stderr as well when there is no console
The no-console fallback repaired Console.Out only. Tauri pipes stderr through
the same lossy UTF-8 decode (install.rs) and emits it to the same UI log, and
InstallFailureContext builds the user-facing failure message from those lines,
so a PowerShell error carrying a non-ASCII path still arrived as U+FFFD.
install.ps1 also writes its Clear-TauriInstallError markers there.
* Tighten the comments added by this PR
Comments only, no code change. Verified with the PowerShell AST tokenizer for
both .ps1 files and the Pester suite (token streams identical with Comment and
NewLine excluded), comment_tools.py for the Python test, and a code-only diff
for update.rs.
* Update the Windows command assertion for the added UTF-8 flags
windows_update_command_uses_python_not_replaceable_console_stub asserts the
exact argument vector, so adding -X utf8 broke it. The Windows cargo test job
in studio-tauri-smoke.yml runs it; the Linux job skips it under cfg(windows),
and cargo check type-checks tests without running them, so neither the org
Linux run nor the staging check caught it.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
Tai An
|
0c12d6473f
|
fix(studio/rocm): don't install for a shadowing iGPU on mixed AMD hosts (#7776) (#7778)
* fix(studio/rocm): don't install for a shadowing iGPU on mixed AMD hosts (#7776) On a board with both an AMD APU and a discrete Radeon, HIP enumerates the iGPU first, so _detect_windows_gfx_arch picked index 0 and the installer pulled the iGPU's wheel family -- a gfx1036 Raphael iGPU shadowing a gfx1200 RX 9060 XT, leaving the discrete card unused until the reporter set HIP_VISIBLE_DEVICES=1 by hand. When no visible-device mask is pinned and more than one distinct arch is enumerated, skip a leading shadowing APU arch so the discrete card decides the wheel family, and print which GPU was chosen plus the HIP_VISIBLE_DEVICES override. An explicit HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES value still wins verbatim. The Strix arches (gfx1150/1151/1152) are deliberately excluded from the skip set: they are first-class unified-memory training targets, so their selection is unchanged. Signed-off-by: Tai An <antai12232931@outlook.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/rocm): mirror the dGPU repick in setup.ps1 and honour CUDA_VISIBLE_DEVICES Addresses both review findings on #7778. 1. setup.ps1 resolved the gfx arch itself (hipinfo and amd-smi paths) and built $ROCmIndexUrl from it *before* invoking install_python_stack.py, and _ensure_rocm_torch() returns early once UNSLOTH_ROCM_TORCH_INSTALLED=1 -- so a fresh Windows install on a gfx1036 + gfx1200 host still received gfx103X-all wheels and never reached the Python-side repick. Resolve-ShadowingGfxPick mirrors _dedup_pick() and is applied at both PowerShell pick sites. 2. HIP honours CUDA_VISIBLE_DEVICES with the same semantics as its own masks -- _pick_rocm_gfx_target in install_llama_prebuilt.py already resolves all three identically -- so a ROCm install launched with only CUDA_VISIBLE_DEVICES set was treated as unpinned and could be overridden by the iGPU skip. It now counts as a pin on both sides ("" / "-1" still mean "no mask"). Tests: CUDA_VISIBLE_DEVICES pin + empty-is-not-a-pin cases, a setup.ps1 <-> Python parity check on the shadowing-arch list (the list now exists in two places), and the pre-existing shadowing tests now clear CUDA_VISIBLE_DEVICES so CI runners that export it cannot flip the assertions. Signed-off-by: Tai An <antai12232931@outlook.com> * fix(studio/rocm): keep a supported APU over a discrete card with no Windows wheels The shadowing-iGPU preference returned the first non-integrated arch in the enumeration regardless of whether AMD ships Windows wheels for it. On an unpinned gfx1036 + gfx1010 host that deposed a supported Raphael APU for a discrete card absent from _GFX_TO_AMD_INDEX_ARCH, so _windows_rocm_index_url resolved to None and the install fell back to CPU -- strictly worse than the shadowing the preference exists to undo. Only prefer the discrete arch when it actually has an index, unless the integrated pick has none either, in which case the swap costs nothing and the discrete card still wins. Both directions are covered: gfx1036+gfx1010 keeps the APU (fails without this change, returning gfx1010), gfx1013+gfx1010 still yields to the discrete card so the guard is not over-tightened. Signed-off-by: Tai An <antai12232931@outlook.com> * fix(studio/rocm): close two setup.ps1 gaps in the shadowing-iGPU preference Both halves of the #7776 preference existed in install_python_stack.py but only half of it in the PowerShell mirror, which resolves the arch and builds $ROCmIndexUrl itself before the Python installer ever runs. - Resolve-ShadowingGfxPick deposed a supported APU for any discrete arch, even one AMD ships no Windows wheels for (gfx1036 + an older gfx1010): the repick resolved to no index at all and dropped the host to CPU, strictly worse than the shadowing it undoes. It now consults $archFamilyMap, mirroring the _pick_has_wheels guard in _dedup_pick(). The map moves to script scope so detection can read it; contents are unchanged, so the four-way parity test still sees the same 18 entries. - The WMI fallback took the first AMD adapter before name -> arch inference, so an Adrenalin-only host listing a 780M ahead of an RX 9060 XT still inferred gfx1103 and installed gfx110X-all. It now keeps every AMD adapter, infers an arch for each, and runs the same preference over the list. Regression tests fail on the previous revision and pass on this one. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio/rocm): index the enumeration with CUDA_VISIBLE_DEVICES too _visible_devices_pinned() treats CUDA_VISIBLE_DEVICES as a pin, but _pick_visible_index() only read HIP/ROCR. On the probes that enumerate every GPU regardless of the masks (amd-smi, WMI), CUDA_VISIBLE_DEVICES=1 on a gfx1036 + gfx1200 host therefore suppressed the shadowing skip *and* resolved to index 0, installing the iGPU's wheels for the device the user masked away. Same mismatch in setup.ps1's $_hipVisIdx and $visGpu picks. All three masks are now read at every site, matching _pick_rocm_gfx_target in install_llama_prebuilt.py. Also corrects the _detect_windows_gfx_arch docstring, which still claimed the first GPU always wins without a mask. Signed-off-by: Tai An <antai12232931@outlook.com> * Share one mask resolver across every ROCm pick site for PR #7778 The shadowing-iGPU preference was correct, but each pick site still resolved HIP/ROCR/CUDA_VISIBLE_DEVICES with its own inline expression and they disagreed, so a mask the pin check honoured could resolve to a different GPU than the one the user asked for. On a mixed host that lands on index 0, which is the iGPU the preference exists to skip. setup.ps1 - Add Resolve-VisibleGpuIndex and use it at all four pick sites (hipinfo, amd-smi list, amd-smi static --asic, WMI name inference). Previously the hipinfo expression rejected " 1 " and the amd-smi one rejected "1,0". - The static --asic branch now collects every gfx token and runs the repick instead of taking the first regex match. - WMI inference indexes the adapter list rather than the inferred arch list, so an unrecognised name cannot shift a mask onto the wrong physical card, and it only repicks when every adapter mapped: an unknown name may itself be the discrete card. - Filter WMI adapters on ConfigManagerErrorCode so a disabled or driver-errored Radeon cannot depose a working iGPU. Get-CimInstance to match the rest of the repo. install_python_stack.py - _pick_visible_index now skips "" and "-1" and reads the next mask, matching _visible_devices_pinned. Before, HIP_VISIBLE_DEVICES="" with CUDA_VISIBLE_DEVICES=1 counted as pinned while the index resolved to GPU 0. - Out-of-range and unparseable masks warn instead of silently using GPU 0. - Strip the ":sramecc+:xnack-" suffix from gcnArchName like setup.ps1 does; a suffixed token matched neither the wheel table nor the skip set. - Prefer a wheel-backed candidate whenever one exists, not only when the picked arch has wheels, so gfx1036 + gfx1010 + gfx1200 no longer stops at gfx1010 and drops the host to CPU torch. Arch list - Drop gfx1037: it is not an AMDGPU target in LLVM, so no Windows tool emits it. - Add gfx1033 (Van Gogh) and gfx1153 (Krackan Point 2), both APUs. gfx1033 has a wheel family, so leaving it out let it act as the "discrete" card. - gfx1013 is Cyan Skillfish, not Van Gogh. Both advisories now tell the user to setx HIP_VISIBLE_DEVICES so the chosen GPU is used at runtime, not just at install time: the wheels alone do not change which device HIP enumerates first. Tests - TestSetupPs1ShadowingBehaviour actually executes Resolve-ShadowingGfxPick and Resolve-VisibleGpuIndex under pwsh, slicing them out by AST. The existing parity class only greps text, so a rename failed it while a semantic bug passed. - Regression tests for each fix above. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Warn on a pinned wheel-less GPU and silence a bogus range warning Both found by simulating the change across the OS x GPU-vendor product rather than by reading it. - _dedup_pick now says so when an honoured pin selects a GPU AMD ships no Windows wheels for while another enumerated GPU has them. The pin is still honoured verbatim, but the install drops to CPU torch and the mask is the reason, which was previously invisible. - _pick_visible_index takes warn=False for callers whose list is deduplicated. The Linux Strix reroute indexes _detect_amd_gfx_codes(), which collapses duplicates, so a dual same-arch box (two gfx1151) has a 1-element list and a perfectly valid HIP_VISIBLE_DEVICES=1 read as out of range. That printed a false "out of range" warning on a healthy Linux host. The Windows arch-selection path still warns, where the index space really is devices. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Align the visible-device masks with the ROCm runtime, and two repick fixes Three review items on the last round, all confirmed against the code at head. Mask semantics (mine to fix: 6d0ac82 got this wrong). "" and "-1" do not mean "no mask", they select no GPU at all, so falling through to the next variable was wrong. The ROCm runtime stores an explicitly empty var as " " (clr flags.cpp), then picks the HIP mask whenever its first byte is not NUL (paldevice.cpp on Windows, rocdevice.cpp on Linux), so an empty HIP_VISIBLE_DEVICES shadows CUDA_VISIBLE_DEVICES rather than deferring to it; parseRequestedDeviceList surfaces zero devices for " " and "-1", which ROCR states outright in amd_filter_device.h. _visible_devices_pinned and _pick_visible_index are now first-set-wins and treat any set value as a deliberate selection, matching _pick_rocm_gfx_target in install_llama_prebuilt.py and PyTorch's own _parse_visible_devices. Resolve-VisibleGpuIndex and Resolve-ShadowingGfxPick mirror it. Three tests asserted the old premise and now assert the runtime's. Resolve-ShadowingGfxPick did not prefer wheel-backed cards when the APU has no wheels either. The predicate went vacuously true and took the first non-integrated arch, so gfx90c,gfx1010,gfx1200 picked gfx1010, left $ROCmIndexUrl null and installed CPU torch despite the supported gfx1200. Now mirrors _dedup_pick's `_withWheels or (...)`. The Python WMI probe listed disabled adapters. setup.ps1 filters on ConfigManagerErrorCode but `(Get-CimInstance Win32_VideoController).Name` did not, so on a driver-only laptop a disabled RX 9060 could depose a working 780M and pull wheels for a GPU Windows never exposes. Same filter both sides. * Stop double-applying the mask to hipinfo, and two selection fixes hipinfo is itself a HIP application, so under a mask the runtime filtered and renumbered its device list before we ever read it. Indexing that output again applied the mask twice: with HIP_VISIBLE_DEVICES=1,0 on a gfx1036 + gfx1200 host, HIP exposes [gfx1200, gfx1036] and the second lookup landed on the iGPU, installing its wheel family for the card the mask put first. _dedup_pick now takes mask_resolved for the hipinfo probe and setup.ps1 reads $_hipAllArches[0]; amd-smi and WMI list every GPU regardless of the masks, so they keep the explicit index. The repo already stated this in _hip_visible_device_mask_set: "hipinfo, itself a HIP application, so under a mask it enumerates the VISIBLE devices, not the physical ones". The advisory hard-coded device 1. On gfx1036,gfx1010,gfx1200 the selected card is device 2, so following the message exposed the gfx1010 the installed wheels do not target. Both messages now name the selected arch's real index. The WMI path substituted another adapter's arch when the selected one had an unrecognised name. Unpinned that is the point (the #7776 iGPU has no entry in the name table, so the named discrete card should decide), but under a mask it installed wheels for a GPU the user masked away. The fallback is now gated on Test-VisibleDevicesPinned, which also replaces the inline pin loop in Resolve-ShadowingGfxPick so both sides share one definition. Two existing tests described a host that cannot exist: unfiltered hipinfo output under a mask. They now model the filtering HIP actually performs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep setup's dGPU repick for llama.cpp, and index WMI by adapter * Reinstall Windows ROCm torch when the wheel family changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the WMI arch probe silent on non-AMD adapters * Read the active ROCm family from the rocm meta-package * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve the visible-device mask against devices, not deduplicated arches * Harden the WMI probe and the PowerShell index parse * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the platform in the two new Linux reroute tests * Pin the arch too in the new Linux reroute tests * Parse rocminfo per agent and honour ROCR filtering on Linux * Tighten comments --------- Signed-off-by: Tai An <antai12232931@outlook.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
oobabooga
|
f8730f4339
|
Installer: select CUDA wheels that cover the host's GPUs (#7814)
* Installer: select CUDA wheels that cover the host's GPUs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Windows venv wipe and warning dedupe for PR #7814 - Windows pins torch<2.11, whose cu128 still ships sm_70, so capping a Volta to cu126 there rewrote a working family. The stale-venv check then read that as drift and deleted the venv on a direct "unsloth studio update", which cannot recreate it. Make the pre-Turing floor per-family (70 for cu128). - Repair an unpinned cu* -> cu* move in place instead of rebuilding the venv. - Decide the cu126 advice before deduping the uncovered-host warning: the host facts are release invariant but the artifact list is not, so the release walk-back let an unhelpful release swallow the remedy. - Gate the new coverage repair and the cu126 advice on x86_64, matching the cap. - Add tests/studio/test_pre_turing_cap.ps1: the parity test only greps for the call spelling, so neither PowerShell copy had behavioural coverage. * Tighten comments for PR #7814 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com> |
||
benj
|
6a58ea0f0e
|
Add Intel Arc GPU detection and XPU PyTorch install to Windows installer (#7706)
* Add Intel Arc GPU detection and XPU PyTorch install to Windows installer The installer's GPU detection chain (NVIDIA -> AMD ROCm -> else) has no Intel Arc/SYCL/XPU branch, so Intel Arc GPUs fall into the "none (chat-only / GGUF)" branch and get CPU PyTorch despite PyTorch publishing XPU wheels at download.pytorch.org/whl/xpu. This adds: - WMI-based Intel GPU detection (Arc, Iris, UHD, HD Graphics) - Torch XPU availability check for migrated/upgraded environments - An XPU PyTorch install path with the whl/xpu index - CPU fallback with a pointer to the Intel oneAPI docs when XPU isn't available - Updated messaging from "NVIDIA or AMD ROCm" to include Intel Arc The XPU wheels ship their own oneAPI runtime (intel-sycl-rt et al.) so no Intel oneAPI Base Toolkit is required for GPU training. Tested on: Windows 11, Intel Arc 140V GPU (8GB), PyTorch 2.9.0+xpu Co-authored-by: CommandCodeBot <noreply@commandcode.ai> * Fix Intel XPU detection and install path for PR #7706 The XPU index selected during GPU detection was overwritten by Get-TorchIndexUrl before the install branch read it, so Intel hosts still got CPU PyTorch while being told XPU wheels were being installed. - Move the XPU reroute after Get-TorchIndexUrl, and let an explicit pin win - Detect via Get-CimInstance (Get-WmiObject is absent in PowerShell 7) - Match only Arc / Data Center GPU, so UHD / HD / Iris Xe are not promised XPU - Split Intel GPU present from XPU-capable so the CPU fallback hint works - Bound the XPU torch trio like every other index (bare names resolved torch 2.13.0 + torchaudio 2.11.0 and pulled unsloth back to an old release) - Clear the XPU state after a CPU fallback, mirroring the ROCm path - Teach the index family, GPU branch and torch flavor helpers about xpu * Keep install.sh diagnostics in parity with the install.ps1 xpu family install.ps1 now classifies an /xpu index leaf as family xpu / branch xpu, so mirror the same two cases in _tauri_torch_index_family and _tauri_gpu_branch. These feed the [TAURI:DIAG] line only, and a Linux user can already reach the xpu index via UNSLOTH_TORCH_INDEX_FAMILY, where it previously reported auto/unknown. Linux Intel auto-detection is not added here. * Tighten the Intel XPU comments in install.ps1 Comment and whitespace only, no code change. * Address Codex review on the Intel XPU path - Run the Intel scan before the GPU report chain instead of inside its final else. A WMI-named-only AMD adapter set ROCmGpuLabel and took that chain, so a discrete Arc card next to an AMD CPU's integrated Radeon was never detected. The scan is gated on no usable NVIDIA or AMD, and the Intel branch ranks above the two AMD-present-but-unusable branches, so a usable AMD host is unaffected. - Let a migrated env's torch veto the hardware match only when it is itself an XPU build. A CPU build reports torch.xpu.is_available() False for lacking XPU support, not for unsuitable hardware, and was blocking the CPU to XPU upgrade. - Detect Intel in studio/setup.ps1 too. It only knew NVIDIA and AMD, so every successful Intel install printed none (chat-only / GGUF) right after install.ps1 reported a usable Arc GPU. Self-contained so studio update works. * Address the second Codex round on the Intel XPU path - Reset $script:IsIntelXpu at the start of each invocation. Under the documented irm | iex path $script: is the caller's session scope, so a second run in the same session inherited a stale true, skipped the scan on a now-NVIDIA host and still rerouted to the xpu index. Reproduced in pwsh before fixing. - Gate the Intel scan on whether AMD actually gets a wheel, not on whether an AMD arch was seen. An arch missing from the family map has no ROCm wheels and lands on CPU torch, so it must not outrank a usable Arc card. The map is hoisted above the scan and consumed unchanged by the AMD reroute. - Select the XPU index in studio/setup.ps1, not just report it. Previously setup printed Intel GPU detected and then installed CPU torch, so studio update never migrated an Arc box off CPU. Adds a bounded XPU install with a CPU fallback, teaches the stale-venv check about +xpu, and mirrors the wheel-aware AMD gate so the two files agree instead of wiping the venv on every update. * Address the third Codex round on the Intel XPU path - Force the dependency pass on an Arc host whose torch is not XPU-capable, the Intel counterpart of the existing AMD escape. Without it the fast up-to-date path skipped the install block, so the xpu index selection was never reached and a CPU venv never migrated. - Confirm a working XPU runtime before treating an xpu venv as stale. If CIM is unavailable or returns an Intel name outside the Arc match, the expected tag fell through to cpu and a valid XPU environment was rebuilt and lost. - Force-reinstall the XPU trio only when the installed wheel is not already +xpu, or the pin changed. It was unconditional, so a fresh install re-fetched multiple GB immediately and again on every update. - Warn when torch.xpu.is_available() is false after installing XPU torch, naming the Intel driver floor. Otherwise the installer promised GPU training while unsloth raised NotImplementedError at import on a stale driver. - Stop the detection probe vetoing the hardware match. Its cpu fallback could not displace the installed +xpu wheel, so it only mislabelled a capable GPU as unusable; the driver warning covers that case honestly, and setup.ps1 agrees. * Bound the XPU probes, repair xpu pins in install.sh, and floor bitsandbytes on the Intel path install.sh: teach _torch_flavor_tag, _expected_torch_flavor_tag and _torch_index_repairable about the xpu leaf. The diagnostic already reported gpu_branch=xpu, but an xpu pin fell to the custom arm so a migrated env kept its CPU wheel. The +xpu flavor arm is required alongside, otherwise a correct 2.10.0+xpu wheel reads as cpu and gets force-reinstalled every run. install.ps1 / studio/setup.ps1: route every torch probe through a new bounded Invoke-BoundedPythonProbe (ProcessStartInfo, both streams drained async, WaitForExit, kill on timeout). A hanging Intel driver init is exactly what these probes detect, and an unbounded one would hang the installer instead of reaching the warning. Timeouts read as not-available. Get-InstalledTorchTag now shares the helper rather than carrying a second copy of the pattern. install.ps1: install bitsandbytes>=0.50.0 on the XPU path. unsloth's floor is >=0.45.5, so a migrated venv keeps a pre-0.49 wheel with no XPU library and 4-bit QLoRA silently turns off. Same floor the AMD paths use, since <=0.49.2 NaNs at 4-bit decode and an Arc card can sit next to a Radeon. * Floor bitsandbytes on the Studio XPU migration and on an explicit xpu pin studio/setup.ps1: `unsloth studio update` migrating a CPU venv to XPU replaced only the torch trio. install_python_stack.py then upgrades unsloth and unsloth-zoo alone, so an installed bitsandbytes 0.45.x kept satisfying the base floor while carrying no Windows XPU kernels, and 4-bit QLoRA silently turned off. Adds the same bitsandbytes>=0.50.0 --no-deps pass install.ps1 got, placed after the stack so it is the last word, gated on $XpuIndexUrl (the CPU fallback clears it, no-torch never sets it) and still inside the -not $SkipPythonDeps block so the up-to-date escape does not reach it. install.ps1: key the bitsandbytes pass off the index leaf instead of $script:IsIntelXpu. An explicit UNSLOTH_TORCH_INDEX_FAMILY=xpu pin on a non-Intel host skips the XPU branch but still installs the trio from the xpu index, so torch is +xpu and needs the same floor. The CPU fallback rewrites $TorchIndexUrl, so a failed XPU install reads as cpu and stays quiet. * Tighten the Intel XPU comments across the three installers Comment-only pass now that the review has settled: several blocks grew over successive rounds and were restating the code or narrating the review. Net 36 lines removed, with the load-bearing facts kept -- why ProcessStartInfo rather than the call operator, why both probe streams drain async, why the helper is defined above the Intel scan, the 0.50.0 bitsandbytes floor and why not the curated extra, and why PEP 440 means a migrated env can confirm but never veto the Intel match. Also records why the Studio bitsandbytes pass must stay above the ErrorActionPreference restore: Fast-Install needs EAP=Continue or PS 5.1 turns pip stderr into a terminating error. No code tokens changed; verified with a PowerShell token-stream diff of install.ps1 and setup.ps1, and by hand for install.sh. * Bound the Intel WMI scan, bound the stale flavor probe, and stop CUDA Triton shadowing XPU studio/setup.ps1: the stale-venv flavor probe read StandardOutput.ReadToEnd() before WaitForExit, so the timeout was unreachable and a wedged import torch hung studio setup forever; stderr was never drained either. Routed through Invoke-BoundedPythonProbe, which already drains both streams and kills on timeout. A timeout now reads as unreadable flavor, so the venv rebuilds. install.ps1 / studio/setup.ps1: bound the Win32_VideoController query and add a registry fallback. -ErrorAction suppresses errors but bounds nothing, and -OperationTimeoutSec is not enforced for the local COM session this uses, so a degraded WMI repository blocks forever. install_llama_prebuilt.py already runs this query out of process for the same reason and documents an Arc A770 being misrouted by it. The registry class key answers in-process; it is the fallback rather than the fast path because a stale driver config can outlive the hardware, and here a false positive would install XPU torch on a host with no Arc. studio/setup.ps1: replace triton-windows with torch's own XPU triton after the stack. Both distributions own the top-level triton package, sharing 151 paths including __init__.py and _C/libtriton.pyd, so an in-place cu-to-xpu repair leaves the CUDA build shadowing the XPU one. Removing it alone would delete the shared files the XPU wheel overwrote, and unsloth declares triton-windows as a win32 dependency so an earlier removal is reinstalled by the stack: uninstall and reinstall, after the stack, only while triton-windows is present. The spec is read from the installed torch, since the name changed from pytorch-triton-xpu to triton-xpu in torch 2.10. * Tighten the comments added with the bounded scan and Triton replacement Comment-only pass over the previous commit's additions, which had not been through one: 15 lines removed across the two bounded-scan headers, the two registry-fallback headers and the Triton block. Kept the facts that cost measurement: -OperationTimeoutSec not being enforced for a local COM session, Ok being false on an empty answer because a Windows host always has an adapter, the registry class key being fallback rather than fast path here, the 151 shared Triton paths, and why the uninstall has to be paired with a reinstall after the stack. No code tokens changed; verified with a PowerShell token-stream diff of both files, which also confirms the two helper copies stay identical. * Stage the Triton replacement behind a download so the uninstall cannot strand the venv The replacement uninstalled triton-windows and then installed the XPU triton from the index. A failure between the two left the venv with a partially deleted triton, since the uninstall drops the paths shared with the XPU distribution, and the warning made that look like a skipped optional repair. The uninstall cannot go last, because it removes the paths in triton-windows' own record and those are the shared ones. So fetch first: pip download the wheel, confirm one is actually on disk (exit 0 alone is not enough, an sdist-only mirror satisfies that), and only then uninstall and install the local file. A local wheel installs with the network refused, so nothing after the destructive step depends on the index. A failed fetch leaves triton-windows in place, which is the pre-existing shadowing rather than a broken venv, and says so. Past that point only disk or permissions can fail, so restore triton-windows if the local install does, leaving a triton that imports. If both fail the message is loud and carries the repair command, with the index URL redacted since a mirror pin can carry a token. pip only: uv has no pip download (astral-sh/uv#3163). * Windows: harden the Intel registry fallback and declare the XPU install state up front Get-IntelRegistryAdapterNames wrapped the whole enumeration in a single try, so one unreadable subkey discarded every adapter found before it. windows_intel_gpu_in_registry(), the in-process Python probe over the same class key, skips per subkey and continues; the PowerShell copy now does too. It also matched on the PCI vendor id but returned DriverDesc, which the callers re-filter on "Intel", so a localized or OEM-branded Arc was found here and dropped there. Both installers carry the same copy and a test asserts they stay identical. setup.ps1 read $installedTorchTag and $XpuIndexUrl from outside the blocks that assign them. Unset and $null are both falsy so behaviour is unchanged, but a caller running with Set-StrictMode -Version Latest turned those reads into terminating errors, and install.ps1 is documented as irm | iex into the caller's own session. Two comment corrections: 0.48.2, not 0.49.0, is the first win_amd64 bitsandbytes wheel carrying libbitsandbytes_xpu.dll, and the triton package overlap is version-dependent rather than a fixed 151 paths. The new test drives the shipped helper with the registry cmdlets mocked rather than reading a hive, so it runs on Linux and macOS as well as Windows. * Studio: show the Intel XPU runtime row in the About tab hardware.py has always emitted versions["xpu"], but HardwareInfo only ever declared cuda and rocm. On an Arc host both of those are null, so the runtime row disappeared entirely while the GPU name and VRAM rows still rendered, leaving a host that looks half detected. That was unreachable on Windows until the installer learned to select XPU wheels, which is what makes it worth fixing here. The three-way choice is lifted into a helper at module scope: inlining it pushes AboutTab past the cognitive-complexity ceiling. The label is a proper noun, so every locale carries the same literal. * Windows: reach Intel XPU through a localized name, a stale fast path and an old wheel Four holes in the XPU paths, all found by driving the shipped code rather than reading it. The registry fallback only ran when the CIM scan failed. When it succeeds and returns a localized adapter name, which on non-English Windows carries no ASCII "Intel", the filter dropped the adapter and the host went to CPU torch. The registry now re-labels an adapter WMI already reported, matched by name so an entry naming nothing WMI listed stays ignored: a driver record outliving its card still cannot promote a host WMI answered for. The XPU trio accepted torch 2.4 and 2.5, which unsloth/models/_utils.py rejects at import for an XPU device. An xpu mirror carrying only an older wheel produced an install that reported success and then failed on the first import, and an existing 2.5+xpu venv was kept because it satisfied the range. The floor is 2.6 on the XPU paths only; the CPU fallback keeps 2.4. The "package is up to date" fast path escaped for an Arc host on CPU torch, but not for one already on XPU torch whose bitsandbytes predates the XPU kernels or whose triton-windows still shadows the XPU Triton. Those two live in the dependency pass, so a venv that reached +xpu without them, an explicit pin or an update whose first pass ran the pre-XPU setup.ps1, never got them on any later update either. An unreadable version reads as stale. install_python_stack.py writes its completion manifest immediately before returning, so an interrupt between the triton-windows uninstall and the XPU wheel install left a venv with no triton that the next update read as complete. The manifest is now held aside across the swap and restored only once a triton is importable again. * Windows: move the install manifest across the Triton swap instead of rewriting it Two problems with the hold added in 2603fc809, both on the restore side. Reading and rewriting the file cannot survive a manifest carrying a non-ASCII path. Windows PowerShell 5.1 writes Set-Content in the ANSI code page by default, and its -Encoding utf8 emits a BOM that install_manifest.read_manifest's json.load rejects outright ("Unexpected UTF-8 BOM"); Get-Content is ANSI on a BOM-less file too, so the read lost bytes before the write got a chance to. The manifest is now MOVED into the wheel's temp directory and moved back, so no encoding is involved at either end. That directory is already removed in the finally, which is what keeps an unrestored manifest gone. A manifest that would not move left the old valid one in place for the whole destructive window, since the failure only cleared the saved copy and carried on into the uninstall. That is the case the hold exists for, so it now skips the swap entirely and says so: triton-windows keeps shadowing the XPU Triton, which costs torch.compile on the GPU and is repairable on the next run, rather than risking a venv with no Triton that reads as complete. * Windows: confirm the install manifest actually moved before the Triton swap Move-Item across volumes is a copy followed by a delete, and it reports success when only the delete fails, leaving the original exactly where it was. So the guard added in af928dd88 could believe it had set the manifest aside while a valid one sat there for the whole destructive window, which is the case that guard exists to prevent. Found by modelling the manifest in the setup.ps1 scenario matrix, which this had no coverage for: with the parent directory read-only the swap still ran, and the locked scenario passed for the wrong reason. The move is now confirmed by testing the source path afterwards, and a manifest still standing aborts the swap like any other failure to move it. Four new scenarios cover it: the swap keeping a byte-identical manifest, a swap where neither Triton reinstalls correctly leaving it gone, a failed fetch never touching it, and a manifest that cannot move aborting the swap. * Windows: key the XPU fast-path remediation off the installed wheel, not just the GPU scan $HasNvidiaSmi suppresses the Intel scan, so on a mixed NVIDIA + Intel box under an explicit xpu pin $script:IsIntelXpu stays false while the pin still lands the venv on a +xpu wheel. The staleness check added in 2603fc809 was gated on that flag alone, so those hosts kept taking the fast path and never reached the bitsandbytes floor or the Triton replacement. This is the same gating mistake the bitsandbytes pass had in round 4, where the fix was to key off the index leaf rather than the scan. The leaf is not resolved yet at the fast path, but the installed flavor tag is, and whatever put the venv on a +xpu wheel the two remediations still apply. The runtime probe above stays on the scan: reinstalling XPU torch is only right where an Intel GPU was actually found. A pure NVIDIA host on a cu wheel never runs the probe, which the matrix asserts alongside the two new mixed-host rows. * Windows: reconcile Intel names for hybrid GPUs, and stop the XPU escapes firing where XPU is unreachable Five fixes from a review of the XPU work so far. The registry reconciliation was gated on "no ASCII Intel name present", so a hybrid laptop reporting its Intel UHD alongside a localized Arc stopped at the UHD and left the Arc unrecognised. It is now gated on the absence of an XPU match, and the regex behind both that gate and the classification is defined once so they cannot drift. The two fast-path escapes cleared $SkipPythonDeps for any Intel host, but the XPU install and its two remediations are all gated on $XpuIndexUrl, which an explicit cpu / rocm / custom-leaf pin never sets, and no-torch mode has no torch pass at all. Those hosts ran the whole dependency pass, installed nothing new, and re-fired the identical condition on every later update. Both escapes now require XPU to be reachable. The manifest path was learned by a subprocess whose output parsing could not work: `& python` returns one array element per line, interpolating that joins on $OFS, a SPACE, so splitting on newlines yields a single element and a banner ahead of the answer arrives glued to the path. Any such failure then skipped the hold silently and swapped anyway, which is the window the hold exists to close. manifest_path() is venv_root()/MANIFEST_NAME and venv_root() is sys.prefix, which is $VenvDir here, so it is assembled like Get-PersistedNoTorch already does. A test asserts the literal still matches MANIFEST_NAME. The uninstall's exit code was discarded. A triton-windows that will not uninstall, which on Windows means Studio is running and holding libtriton.pyd open, still shadows the XPU Triton, so installing over it achieved nothing and restored the manifest onto a venv this pass was supposed to have changed. The restore had no verification and an empty catch, while the finally deletes the held copy either way, so a failed restore lost the manifest with nothing on screen. * Windows: keep the WMI adapter list an array so the Intel re-label appends instead of concatenating `$_gpuNames = if (...) { @(...) } else { @(...) }` wraps each branch, and a one-element array unrolls on its way out of the if, so on any single-adapter host $_gpuNames was a String. The `+=` that re-labels a localized adapter then concatenated two strings rather than appending a name, and the GPU reported to the user came out doubled: Intel(R) UHD Graphics 620Intel Intel(R) UHD Graphics 620 No install decision changes. The re-label only appends a registry name that already contains the WMI name, so the concatenation matches the Arc / Data Center regex exactly when the registry name alone would, and every scenario in the matrix records the same verdict either way. It is the displayed adapter name that was wrong. Widened by the previous commit: gating on the absence of an XPU match rather than of any Intel name brought ordinary single Intel iGPU hosts into the re-label for the first time. @() now wraps the whole if in both installers, with a test asserting it stays that way. * Windows: give pin-only XPU installs the 2.6 floor, and treat an unreadable dependency probe as stale The XPU install branch required $script:IsIntelXpu as well as an xpu index leaf, so an explicit FAMILY=xpu or URL pin on a host whose Intel scan never ran -- a mixed NVIDIA box, where $HasNvidiaSmi suppresses it -- fell through to the generic branch and its torch>=2.4. Against a mirror carrying an older +xpu wheel that installs a torch unsloth rejects at import. Keyed off the leaf alone now, which is what the bitsandbytes gate below it already does and says in its own comment. install.sh had the same gap from the other direction: its xpu leaf is reachable only by an explicit pin and kept the generic floor, so it gets the same 2.6 trio. The fast-path dependency probe treated "did not answer" as "nothing to do". A timeout, or a malformed .dist-info making distributions() raise, then left the fast path intact and an XPU migration never reached the bitsandbytes floor or the Triton replacement on any later update either. It now clears the fast path, the same direction an unparseable version already took. Two install.ps1 rows move, both FAMILY=xpu pins on non-Intel hosts, both onto the XPU branch. The CPU fallback after a failed XPU install keeps its 2.4 floor. * Windows XPU: probe the preserved venv, drop torchaudio on ARM64, and give POSIX XPU the bitsandbytes floor Three fixes to the Intel XPU paths. install.ps1, migrated-runtime probe: a rerun over an existing install moves the old venv to $script:StudioVenvRollbackDir and creates an empty one in its place, both before this probe runs, so it always asked an interpreter with no torch and answered "no XPU". Ask the preserved environment when there is one, which is the migrated runtime the fallback exists for. install.ps1, Windows on ARM: no win_arm64 torchaudio wheel exists on any index. Keying the XPU branch off the index leaf alone routes an arm64 interpreter into a branch that hardcoded the trio, so the install aborted. Ask the interpreter for its platform tag, as the generic path already does, and drop that one pin on arm64. The CPU fallback below it gets the same treatment. install.sh, XPU pins: bitsandbytes ships XPU kernels (libbitsandbytes_xpu2025.so and _xpu2026.so) from 0.50.0 on manylinux, and nothing on the POSIX side raised the floor for them, so a migrated environment kept a pre-XPU build and lost 4-bit QLoRA on a torch that otherwise works. Matches what the Windows XPU pass already installs. * Studio: stop the xpu label test from forbidding a partial locale check-parity.ts states the contract plainly: "Locale files may be partial; missing keys must fall back to English." The new test required every overlay to carry the xpu label, which contradicts that and breaks on the next locale anyone adds. It already did: it.ts landed on main after this branch, so the merged tree fails on all three runners even though nothing about the label is wrong there. The label is a proper noun, so the English fallback is byte-identical to a translation and the requirement bought nothing. Assert what actually renders wrong instead: en.ts must carry the key, because it is the fallback every locale resolves to, and no overlay may define a value that disagrees with it. Both halves were checked against a merged working tree, and both still fail when the condition they guard is broken. * Linux XPU: hoist the bitsandbytes pass out of the fresh-install arm It sat inside `elif [ -n "$TORCH_INDEX_URL" ]`, which a migrated environment never enters because the `_MIGRATED` arm above it wins, so the one environment the pass existed for was the one that skipped it. The AMD passes handle this by existing twice, once per arm; this gate needs nothing branch-specific, so it moves past the chain instead and both arms reach a single copy. tests/sh/test_xpu_bitsandbytes_reachable.sh guards both halves: the block must be placed where every arm reaches it, and it must still fire only on the xpu leaf. 25 checks over [migrated, fresh] x [xpu, mirrored xpu, cuda, rocm, cpu, none] x [torch, no-torch], run against the block and the leaf parser extracted from install.sh. Moving the block back inside an arm fails it. * Report the XPU runtime before the hardware summary, and show every runtime in About setup.ps1: the hardware report runs ~1300 lines before the torch.xpu.is_available() check that keeps an XPU environment, so a host the WMI scan and the registry fallback both miss (wedged CIM service, an Intel part outside the Arc|Data Center regex) was told "none (chat-only / GGUF)" and then watched setup keep the XPU venv. Ask the same question before printing, so the report and the decision cannot disagree. A free disk read gates the interpreter launch: torch/version.py carries the local label, so a CPU-only host never pays for an `import torch` on every `studio update` just to be told it has no Intel GPU. The dist-info name cannot be used for this -- pip normalises the local label out of it (torch-2.9.1.dist-info for a +cu128 wheel). The promotion carries its own try: it must still run when the scan threw, which is the case it exists for, and a junk UNSLOTH_STUDIO_HOME would otherwise abort setup from Join-Path. about-tab.tsx: hardware.py reads versions["cuda"] off torch.version.cuda and sets versions["xpu"] from an independent torch.xpu.is_available() probe, and UNSLOTH_FORCE_XPU=1 is a supported configuration where CUDA is present but XPU is selected. Both are non-null there, so returning the first match hid the XPU row on exactly the host it was added for. Collect every reported runtime instead. tests/studio/test_setup_xpu_runtime_prereport.ps1 covers the two new helpers with the filesystem mocked, so it runs on all three runners: override precedence, ~ expansion, the four wheel flavours, a missing or unreadable version.py, and wiring assertions that the promotion precedes the report and that the cheap read gates the probe. The About-tab test gains a case that fails if the picker returns early again. * POSIX: recognise a working XPU runtime, and raise the bitsandbytes floor on the update path The hardware summary tested NVIDIA, AMD and Apple Silicon and then fell through to "none (chat-only / GGUF)", so a Linux host running the +xpu wheel install.sh had just installed was told training needs an NVIDIA or AMD GPU. Added an arm ranked below both, matching setup.ps1. The bitsandbytes floor was also unreachable on the route an existing XPU user actually takes. `unsloth studio update` runs this file, never install.sh (see the note at the top of setup.sh), and neither this file nor install_python_stack.py had an XPU floor, while unsloth's own dep floor is 0.45.5 -- which a pre-XPU wheel satisfies indefinitely. So 4-bit QLoRA stayed unavailable on a torch that otherwise works. One detection serves both, but they read different signals on purpose. The floor keys on the WHEEL (+xpu, read off torch/version.py) and the summary keys on the RUNTIME (torch.xpu.is_available()): a +xpu wheel installs fine on a host whose driver never initialises, and that host should still get the kernels while no GPU is claimed for it. The disk read gates the interpreter launch, so a CPU-only host pays nothing per update. tests/sh/test_setup_xpu_posix_summary.sh builds real venv trees, version.py files and stub interpreters rather than mocking, so the disk read and the runtime probe genuinely execute: 13 checks over the four wheel flavours, working/dead/missing runtime, no venv, and the arm's rank. Removing the arm fails four of them. * POSIX XPU: make the bitsandbytes step nonfatal, bound the probe, and act on an XPU pin Three defects in the POSIX XPU code from the previous commit. run_quiet routes failure to setup_fail and exits, so the best-effort bitsandbytes upgrade could abort an otherwise fine `studio update` over a transient download, and the warning after it was unreachable. run_quiet_no_exit is the nonfatal wrapper. The runtime probe had no timeout. A stalled Intel driver wedges inside `import torch`, which is exactly the host this probe classifies, so it could hang every update forever. Bounded at 60s rather than the 10s the smi probes use: a cold `import torch` takes seconds by itself and a short bound would read a healthy host as having no GPU. Systems without coreutils timeout keep the previous behaviour rather than losing detection. An explicit XPU pin was protected but never acted on. An xpu leaf names no family the cuda/rocm repair helpers know, so _explicit_unknown_family_torch_index_url makes both skip it, and `unsloth studio update` never runs install.sh -- so switching a CPU install to UNSLOTH_TORCH_INDEX_FAMILY=xpu left the CPU wheel in place indefinitely. The fix goes in install_python_stack.py, which already parses the pin, rather than setup.sh, which has no pin awareness at all: _ensure_xpu_torch mirrors the existing _ensure_cpu_torch, the xpu leaf is classified so the backend is no longer unknown, and the ROCm helper skips an xpu backend so it cannot treat the pin as an AMD host. Windows is excluded because setup.ps1 owns torch there and installs the trio itself. That put the XPU trio in a third file, so tests/sh/test_xpu_torch_spec_parity.sh asserts the floors match across install.sh, install_python_stack.py and install.ps1 plus the wiring. Each of its four structural guards was mutation-tested: a drifted floor, a lost classification, wiring at one call site instead of two, and the ROCm skip removed all fail it. The POSIX summary suite gains checks for the nonfatal wrapper and the bound. * Linux XPU: swap generic Triton, gate the pin repair on the version, and escape the fast path Three defects in the XPU code from the previous commit. _ensure_xpu_torch returned on the +xpu tag alone, so a migrated 2.5+xpu venv was left in place even though unsloth/models/_utils.py raises at import for an XPU device below 2.6. It now returns only when the flavour and the supported range both match. That repair was also unreachable on the route it was written for. setup.sh skips install_python_stack entirely when the package version is current, and that pass is the only thing that acts on an XPU pin, so a CPU install switched to the xpu family stayed CPU. Added a third fast-path escape beside the anyio and incomplete-manifest ones. Generic triton and torch's pytorch-triton-xpu / triton-xpu both own the top-level triton package, and resolving unsloth against a pinned +xpu torch pulls both -- uv reports pytorch-triton-xpu 3.5.0 alongside triton 3.7.1 -- so the CUDA-oriented build lands last and torch.compile loads the wrong library on an Intel GPU. This is the POSIX half of the Windows swap: the spec is read from torch's own metadata, so the pytorch-triton-xpu to triton-xpu rename at torch 2.10 needs no hardcoding, and the fetch happens before the uninstall because the uninstall drops the shared paths from generic triton's own record. test_torch_installs_do_not_use_deprecated_index_url forbade --index-url on "$TORCH_INDEX_URL" anywhere in install.sh. That rule is about uv, which deprecated the flag in favour of --default-index; pip never had --default-index, so the pre-fetch legitimately uses it. The assertion is now per occurrence and exempts pip download only, and it joins backslash continuations first, since the flag and its command are routinely on different physical lines. Both a same-line and a continuation-line uv offender were mutation-tested and are still caught. tests/sh/test_xpu_triton_swap_posix.sh asserts the swap by execution -- ordering, the rename, no generic triton, torch wanting CUDA triton, non-xpu index, no-torch, empty index, and a dead mirror that must warn without removing anything. * XPU: move the Triton swap where both routes reach it, and bootstrap pip for it Five defects in the XPU code from the previous commits. The Triton pre-fetch could never have run. `uv venv` is created without --seed, so a fresh venv has no pip and `python -m pip download` fails with "No module named pip" every time, leaving the swap a no-op that only ever warns. My shell test missed it because its stub interpreter answered pip commands. install.sh already bootstraps pip this way before its pre-release bitsandbytes wheel. The swap also never ran on `unsloth studio update`, which runs setup.sh and never install.sh. Both fixes fall out of moving it: install.sh runs setup.sh, which runs install_python_stack.py, so that module is the one place both routes pass through. The install.sh copy is deleted rather than duplicated, and the shell test is replaced by tests/studio/test_xpu_triton_swap.py, which covers the no-pip case and asserts install.sh carries no second copy. The fast-path pin match missed authenticated and fragmented mirrors (https://mirror/whl/xpu?token=...), which read as "no XPU pin" and skipped the repair; query and fragment are now stripped before the leaf test. That escape also launched an interpreter, which a wedged Intel driver hangs inside. It now reads the local label out of torch/version.py instead: nothing to bound, and a CPU-only host pays nothing per update. setup.ps1's fast path asked only whether XPU was available. A 2.5+xpu build answers yes and is still rejected by unsloth/models/_utils.py at import, so it now checks the supported range too, via Test-TorchXpuVersionSupported. The POSIX suite is up to 22 checks; the three new guards were mutation-tested by removing the query strip, the fragment strip, and by making the escape launch an interpreter. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * XPU: make a failed Triton swap unsurvivable, and widen the fast-path escapes Five defects in the XPU code from the last two commits. The Triton uninstall ignored its return code. A read-only or locked venv leaves generic triton registered, so installing over it lets a later upgrade of that distribution delete the shared files again, and every dependency pass repeats the swap. A failed uninstall now changes nothing at all. Past the uninstall the venv has no triton, because the uninstall takes the shared top-level files with it, so a warning there let the caller write a completion manifest over a venv whose torch.compile is broken -- and the next update fast-paths straight past it, since no generic distribution is left to trigger on. That install is now fatal. _ensure_xpu_torch returned when the probe timed out. On this path a wedged `import torch` is evidence rather than noise: the usual cause is a stalled Intel driver under an unsupported +xpu wheel, which the resolver keeps because it satisfies the base range. An authoritative pin now repairs on an inconclusive probe. This deliberately differs from the CPU counterpart, where a wedge has no such likely cause. The fast-path pin match stripped one trailing slash, so a ".../whl/xpu//" pin still read as no pin. It now strips them all, like the shared leaf parsers. Moving the Triton swap into the Python stack left the fast path with no reason to run it: a migrated environment with supported +xpu torch and a leftover generic triton kept the CUDA-oriented build forever. A stale generic triton now forces the dependency pass too, detected from the dist-info name so no interpreter is launched. The POSIX suite is up to 26 checks and the Triton tests to 16. Two of the guards were rebuilt after their own negative controls found them vacuous: the stale-triton check matched the detection loop rather than the branch that acts on it, and a fixed line window had drifted off the code it was meant to cover, so it is now anchored on the block. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not require the XPU pin again after install Three Intel paths still assumed the pin was still in the environment, or that every XPU host looks like x64 Linux. install_python_stack.py: the generic-Triton swap returned unless UNSLOTH_TORCH_INDEX_URL / _FAMILY was set. That pin is one-shot -- a user who ran UNSLOTH_TORCH_INDEX_FAMILY=xpu ./install.sh has nothing left in the environment by the next plain `unsloth studio update`, yet that update's dependency pass can pull generic triton back in and shadow torch's XPU build again. The installed +xpu wheel is the durable signal (setup.sh already raises the bitsandbytes floor off it), so fall back to it and to the default xpu index. The label is read off disk: importlib.metadata drops the local version label, and `import torch` loads the SYCL runtime, which can wedge. install.ps1: the flavor repair built its own XPU trio including torchaudio, which has no win_arm64 wheel on any index. A migrated ARM64 venv skips the fresh XPU branch and takes this path, so the repair failed outright before setup.ps1 could reach its ARM-aware fallback. One builder now serves both sites, since the two copies drifted the moment only one learned about ARM. install.sh: adding the xpu tag made the final flavor guard reachable on an Intel pin, and it probes with an unbounded `import torch`. On a host whose driver initialization wedges that hangs the installer, with no timeout anywhere before setup.sh's bounded probes. The xpu path reads torch/version.py off disk instead; every other family keeps the interpreter read unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Key the setup XPU paths on the installed wheel, not the pin Follow-up to d07c179f: install_python_stack now treats the installed +xpu wheel as the durable signal, but the two callers upstream of it did not. setup.sh fast path: the escape only ran under `case $_setup_pin in *xpu`, so after a one-shot UNSLOTH_TORCH_INDEX_FAMILY=xpu install every later `studio update` saw no pin, kept _SKIP_PYTHON_DEPS=true and never reached the Triton swap at all -- generic triton kept shadowing the XPU build forever. The disk read now happens unconditionally and the swap escape keys on the wheel. The pin leaf is also compared exactly, like the shared index parsers: a custom mirror ending in -xpu was classified as the curated family, which cleared the skip flag on every up-to-date run while _ensure_xpu_torch declined to act. setup.ps1: bounding the flavour probe turned a timeout into "rebuild", and the host most likely to time out inside `import torch` is an Arc box whose compute driver stalled -- where torch/version.py still names a good +xpu wheel. With no currently exported pin the stale path then deleted the venv. It now falls back to the same disk check and warns about the driver. Other families still rebuild on an unreadable flavour. setup.sh summary: a +xpu wheel whose runtime will not initialise fell through to "none (chat-only / GGUF)", telling an Arc owner their hardware is unsupported and hiding the driver update that fixes it. It gets its own arm. * Stop the XPU paths from stranding or wiping a venv Four ways the Intel paths could still leave a user worse off than before they ran anything. setup.ps1 stale check: on a hybrid NVIDIA + Arc host the XPU promotion is gated on -not $HasNvidiaSmi, so a pinless `unsloth studio update` expects a cu* tag, calls the working Arc venv stale and DELETES it -- then exits, because only install.ps1 creates venvs. A direct update now keeps any +xpu venv and says to re-run install.ps1, which rebuilds with a rollback copy. setup.ps1 Triton swap: when the staged XPU wheel failed to install after triton-windows was removed AND the generic restore also failed, the branch only printed. $stackExit stayed 0, so setup reported success and install.ps1 committed a venv with no importable triton over its rollback. It now carries the real failure code into the existing handler. install_python_stack: the `pip download` that stages the XPU Triton wheel inherited the user's pip index environment. PIP_NO_INDEX makes pip ignore --index-url outright, and PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS are consulted in addition to it, so the fetch could fail (leaving generic Triton shadowing the XPU build) or serve the wheel from an index the pin never named. It now takes the same _install_env_for_cmd scrub every other pinned install gets. setup.sh runtime probe: the arm taken when coreutils `timeout` is absent ran the probe with no deadline, on exactly the stalled-driver host the bounding exists for. The deadline now lives inside the probe as signal.alarm, which terminates the process even while the driver blocks in C. * Keep a preserved XPU venv on the XPU index Follow-up to 10ba6e31c, which stopped a direct update wiping a +xpu venv on a hybrid NVIDIA + Arc host but left the rest of the pass believing the host was CUDA. The index chain prefers NVIDIA over Intel, and the CUDA arm does not --reinstall-package torch, so uv left the +xpu wheel in place as satisfied while installing triton-windows over torch's XPU triton -- and with $XpuIndexUrl null nothing swapped it back. A half-converted venv is worse than either end state, so the preserved case now selects the xpu leaf, ahead of the NVIDIA arm and behind an explicit pin. The hardware report is untouched: there really is an NVIDIA GPU in the machine. install_python_stack: an inconclusive XPU probe was always read as a flavour mismatch, but on a stalled Intel driver under a SUPPORTED wheel that is two 90-second hangs and two force-reinstalls of the whole trio on every update, repairing nothing. The disk answers what the probe cannot, so a supported wheel now yields the driver warning and an unsupported or missing one still repairs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Lowercase the setup.sh pin leaf like every other index parser install.sh's _torch_index_url_leaf, setup.ps1's Get-TorchIndexLeaf and install_python_stack's _torch_index_leaf all lowercase before classifying. This copy did not, so UNSLOTH_TORCH_INDEX_FAMILY=XPU (or a URL ending in /XPU) left the leaf uppercase, the equality test against "xpu" failed, and the fast path stayed on. Those same classifiers call that pin XPU once they are reached, so the wheel was never migrated and the update silently repaired nothing. The comment above the line already claimed to match the shared parsers; now it does. Three cases added to tests/sh/test_setup_xpu_fastpath_escape.sh (FAMILY=XPU, FAMILY=Xpu, a URL ending /XPU), plus one that lowercasing must not widen the match: a custom leaf like PRIVATE-XPU stays an unknown family. All three fail against the previous line and pass now. * Do not promise CPU training when the XPU runtime will not start The unavailable-runtime arm said training and GPU inference run on CPU until the driver is fixed. They do not: with neither CUDA nor XPU available, get_device_type() in unsloth/device_type.py raises NotImplementedError, so importing unsloth fails outright rather than falling back. llama.cpp is unaffected, which is what chat and GGUF actually run on, so say that instead. The drift guard added with it needed two passes to be worth anything. Anchoring the arm on the flag name alone matched the bitsandbytes block instead, whose own "4-bit QLoRA may be unavailable" warning made both assertions pass on any wording; and the arm's explanatory comment quotes the phrase it must not use, so comment lines have to go before the grep. Restoring the old message now fails both checks. * Let an explicit non-XPU pin migrate off an XPU wheel Two halves of the same gap: asking for CUDA/ROCm/CPU on a host already running +xpu did nothing. setup.sh: the fast-path escape fired only when the pin itself was xpu, or when a stale generic triton shadowed the build. With an up-to-date install, a +xpu wheel and the pin switched to another family, neither arm matched, install_python_stack never ran, and the authoritative pin was ignored. Added an arm for that case, digit-gated like the shared classifiers so a custom verbatim leaf (rocm-current, cu-private) stays UNKNOWN and does not force a pass that repairs nothing. install_python_stack: _ensure_cpu_torch classifies the installed build and returns early on "already a CPU build". Its probe tested hip, rocm, cuda and +cu<digits>; an XPU wheel sets neither torch.version.cuda nor .hip, so it read as CPU and an explicit CPU pin over it did nothing at all. Keyed on the +xpu local label, since torch.version.xpu is None on some builds. Additive: +cu128 and +rocm still read gpu, +cpu and untagged still read cpu. The escape suite's extractor stopped after the second _SKIP_PYTHON_DEPS assignment, so adding a third arm truncated the block and the new cases failed while the old ones passed. It now stops at the next outer arm and asserts exactly three arms extract, so a future arm fails loudly instead of disappearing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop a wedged Intel driver from blocking the paths that repair it Both of these are fallout from making the CPU repair XPU-aware: it now has to classify an XPU wheel, and every route to that classification went through `import torch`, which loads the SYCL runtime and blocks on the exact host these paths exist to rescue. install_python_stack: the classifier probe times out after 90s and the except branch returned, so an explicit CPU pin over a wedged +xpu venv stayed a no-op. Classify off disk on timeout via _installed_torch_label_on_disk (find_spec, no interpreter) and fall through to the repair. Gated on a GPU label so a slow but healthy CPU-only host does not force-reinstall torch every update. install.sh: the rollback preservation probe read torch.__version__ through the interpreter at venv-replacement time, ahead of every bounded probe in setup.sh, so a hang there took the whole installer with it. It now reads torch/version.py, the same source _installed_torch_version_for_tag already uses for this reason. The interpreter stays as the fallback for a layout without one, where torch is absent and the import fails fast. The install.sh test executes the block against a fake venv whose stub interpreter records being called, so "read off disk" is proven by the interpreter never running rather than by reading the source. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match torch index families exactly, and judge the XPU fast path on the wheel studio/setup.sh classified a pin as a known non-XPU family with prefix globs (cu[0-9]*, rocm[0-9]*), so cu128-private, cu128rc1, cu128.1, rocm7.2-private, rocm7. and rocm7.2.1 all read as known while install_python_stack calls every one of them UNKNOWN and runs no repair: the fast path was cleared and the dependency pass that followed applied nothing, every update. It now matches exact families like install.sh _is_pip_rocm_family_leaf and install_python_stack _is_cuda_family_leaf: cpu, cu<digits>, rocm<digits>[.<digits>], gfx<digit>... (gfx stays a prefix on all three sides, since gfx120x-all is a real Radeon index leaf). studio/setup.ps1 keyed the same escape on torch.xpu.is_available(), which is also false for a supported +xpu wheel on an old or wedged compute driver. No dependency pass can repair a driver, and the pass force-reinstalls nothing when the flavour already matches, so each studio update repeated the bounded probes and a full resolution just to reach the warning Assert-XpuRuntimeReady already prints. The escape now asks Test-VenvTorchIsXpuSupported, which reads torch/version.py off disk and applies the same 2.6 <= v < 2.11 window, matching what setup.sh does on POSIX and removing the last import torch from a path an Arc host with a stalled driver is most likely to hit. Its only caller gone, Test-TorchXpuVersionSupported is removed. Tests: the escape test now also asks install_python_stack itself about a 28-leaf corpus and asserts the shell predicate agrees leaf for leaf, so the two cannot drift again (75 checks; 11 fail against the previous globs). The pre-report test covers the new helper and asserts the fast-path escape names no readiness probe and launches no interpreter. * Trim comments across the Intel XPU detection changes * Normalise setup.ps1 line endings before the wiring regexes A Windows checkout returns CRLF, so the fast-path escape pattern, which is anchored on a literal \n, matched nothing on windows-latest: the region came back empty, "the escape was found" failed, and the two -not checks inside it reported PASS with nothing to look at. Cross-platform parity caught it on windows-latest with 3 failures. $setupText is now normalised to LF once at the read, which covers both literal newline patterns in the file, and a new check asserts the raw CRLF form does NOT match the same pattern, so it is the normalisation rather than luck that makes this work. Verified against a CRLF copy of setup.ps1: the previous test fails there with exactly those 3 checks and the new one passes. * Run the Triton swap after every torch migration, not between two of them _ensure_xpu_triton keys off the installed +xpu label when no explicit XPU pin is set, and it ran ahead of _ensure_cpu_torch. So an existing +xpu venv updated with an explicit CPU pin had generic triton removed and XPU triton installed, and only then did _ensure_cpu_torch replace torch with the CPU build: a CPU environment whose top-level triton package is the XPU implementation, with the generic triton its own dependency set declares now gone. The CUDA and ROCm repairs already ran ahead of the swap, so their pins left the label correct by the time it read it; CPU was the one migration that did not. Moving the swap to the end of both repair blocks fixes it for every family at once and removes the ordering assumption entirely. The new test asserts the order on the AST at both call sites, so a reflow cannot fake it; against the previous order it fails on the first assertion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bound the wedged-driver probe without GNU timeout macOS ships no GNU timeout (Homebrew coreutils installs it as gtimeout), so on the macOS parity leg the `timeout 30 python3 ...` line exited 127 the instant it was called. The test reads only the exit code, and 127 is non-zero with an elapsed time of 0, so both assertions passed without python ever starting: the alarm behaviour they exist to prove was never exercised on macOS. Replaced with the script's own background watchdog, which behaves the same on every platform, and added a lower bound on the elapsed time. The alarm is 2s, so a run that returns instantly did not execute the probe, which is precisely how the missing-timeout case looked. Verified by shimming `timeout` to exit 127: the previous test still reports 37 passed, and by shimming python3 to return instantly: the previous test still reports 37 passed while this one fails on the deadline check. * Trim comments in the Intel XPU detection changes --------- Co-authored-by: CommandCodeBot <noreply@commandcode.ai> Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com> |
||
|
Daniel Han
|
cc0185fbb1
|
Windows: guard the source-build and whisper.cpp probes on an unreadable install tree (#7757)
* Windows: guard the source-build and whisper.cpp probes on an unreadable install tree Follow-up to #7735, which routed the prebuilt llama.cpp probes through three-state path probing but left two gaps. Phase 4 read $LlamaServerBin with a bare Test-Path under "Stop". A forced compile, a pinned PR or a custom llama source skips Phase 3.4 entirely, so on those routes this was the first probe inside the tree and a denied build\ aborted with the raw "Test-Path : Access is denied" the merged PR set out to remove. It now probes three-state, and the CMakeCache.txt read below it is guarded too: a listed file can still deny the read, which the probe cannot see. The probe is skipped for a linked UNSLOTH_LOCAL_LLAMA_CPP_DIR, where it would read through the junction into the user's own checkout, and the denial reports -OwnershipUnverified under a custom home, where nothing on this route has proven the tree is ours. The whisper.cpp phase promises failure is never fatal, but under a custom UNSLOTH_STUDIO_HOME an unreadable tree exited the whole run, taking llama.cpp inference down with it. Assert-StudioOwnedOrAbsent gains a -NonFatal mode that hands the denial back instead; an unowned tree still stops. The check stays behind the installer-exists gate it used to sit inside, so a tree without install_whisper_prebuilt.py remains the no-op it was. Backend: _is_runnable let Path.is_file() propagate EACCES. Now that setup leaves a denied whisper.cpp in place, that turned into a 500 out of /api/inference/audio/stt/status, the one endpoint reporting both dictation engines, so the setup message promising Transformers dictation still works was not true. It reads as engine-unavailable instead. * Harden the denial contract tests against surviving mutations Mutation testing found six ways to reintroduce the bugs this branch fixes while the tests stayed green. Assert-StudioOwnedOrAbsent: the -NonFatal returns were counted, not ordered. Moving one below its Exit-PathAccessDenied makes it dead code and the whisper phase fatal again; hoisting one above the custom-home gate reports a fresh install as unreadable. Each return is now pinned immediately above the exit it pre-empts, with no unpaired return allowed. The whisper denial branch had no assertion scoped to its own body. Both phrases it was checked for already occur elsewhere in the phase, so the branch could be turned back into an Exit-SetupFailure and stay green. The branch is now sliced out and checked for step/Yellow, both phrases, and the absence of any exit. The installer gate was checked for presence, not for being a conjunct, so -or-joining or negating it reopened the installer-less tree the test is named for. The denial subject was unpinned, so it could name llama-server.exe and tell the user to move aside one file instead of the tree. Slice terminators are now asserted through one helper: an unasserted terminator does not fail, it silently widens the window to end-of-file and makes everything inside it near-vacuous. The whisper binary probe test gated its only behavioural case on geteuid() == 0, which silently drops it in any root container. It probes for a real denial instead. Two pre-existing exact counts in the ownership guard tests become floors: this branch consumed the last of their headroom, so the next legitimate route added there would break two tests that say nothing about it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the -NonFatal negative control for Windows ACL semantics The control probed a marker file that did not exist. Windows reports a missing child of a denied directory as absent rather than throwing, so the control read as "this host cannot deny" and failed the suite on windows-latest while passing under chmod on Linux. It now probes a file that exists inside the locked tree, matching the control the suite already uses. That difference also splits the routes by platform for a tree with no ownership marker, which is the fresh custom-home case: Linux catches it on the marker probe, Windows has to catch it on the adoptable-state read. Added a case that accepts either route and rejects anything but Denied, so the Windows one is exercised for the first time. * Detect a denied tree that has no ownership marker on Windows Staging CI on windows-latest caught this. Get-StudioAdoptableState decided "denied" only from probes of two marker files inside the tree, but Windows reports a MISSING child of an unreadable directory as absent rather than throwing. A denied tree holding neither marker therefore returned "No", and Assert-StudioOwnedOrAbsent fell through to "path is not an Unsloth-owned install" and exited: the wrong cause, and fatal, on the only platform any of this runs on. It also defeated the whisper -NonFatal path, since an unowned tree is still fatal by design. Listing the directory itself distinguishes "no markers here" from "cannot look", so that is the fallback when neither probe reported a denial. A readable tree with no markers still returns "No" as before, and the catch swallows anything that is not a denial because this helper must not throw. This also corrects the message a denied custom-home llama.cpp tree produced on Windows, which reported the same wrong cause. chmod 000 blocks the child probes outright, so it never reached the new code. chmod 111 allows stat of a named child while forbidding a listing, which is exactly the Windows shape, so the test now covers both and the negative control fires only on the 111 case. * Tighten comments in the denied-tree and whisper install changes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Michael Han
|
ebfefcf84e
|
Windows: do not abort setup on an unreadable llama.cpp install (#7735)
* Windows: do not abort setup on an unreadable llama.cpp install
Test-Path raises UnauthorizedAccessException instead of returning $false
when an ACL denies the probe. setup.ps1 runs under $ErrorActionPreference
= "Stop", so the bare probe of UNSLOTH_PREBUILT_INFO.json in the llama.cpp
prebuilt phase killed setup with a raw "Test-Path : Access is denied" and
exit code 1. The desktop app had nothing but [TAURI:ERROR_DEFAULT] to fall
back on, so it showed "unsloth studio setup failed (exit code 1)".
~/.unsloth/llama.cpp sits beside the app, not inside it, so reinstalling
reused the unreadable folder and hit the same line again, including a
reinstall to a different drive.
Add Get-PathState (Present / Absent / Denied) plus Test-PathQuiet, route
the probes that read inside install trees we do not own through them, and
report a denied llama.cpp install through Exit-SetupFailure so the reason
and the recovery steps reach the desktop UI.
Reported in unsloth-test/unsloth-test#9
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Stop on every denied path, and split the recovery commands
Review follow-ups:
- Assert-StudioOwnedOrAbsent treated a denied root as absent and returned,
so the caller could go on to replace a tree it cannot read. Probe the
root three-state and stop on Denied, still gated on $StudioHomeIsCustom
so default-home behaviour is unchanged.
- The source-build .git probe treated a denied checkout as "no checkout"
and cloned a replacement. The swap that follows recursively removes the
original and moves the temp tree over it under "Continue" and unchecked,
so a denied child could leave a half-deleted install. Stop instead.
This path already treated denied as absent before the previous commit
(that probe runs under "Continue", so it printed an error and took the
false branch), so the hazard is older than this branch, but it is in
scope for the same reason.
- Probe $LlamaCppDir itself three-state, so an unreadable parent is
reported rather than dying on the bare probe under "Stop".
- takeown and icacls were printed joined by "then", which is not a
PowerShell separator: takeown would swallow the rest as arguments and
icacls would never run. Print them on separate lines.
Fold the repeated guidance into Exit-PathAccessDenied so all five denial
routes report the same thing.
* Harden the denial reporting path, found by simulation
Ran the real decision blocks against simulated filesystems (denied file,
denied parent, traverse-only and list-only dirs, symlinks, dangling links,
wildcard and unicode paths, 3000 random paths) plus PSScriptAnalyzer's
5.1/6.2/7.0 syntax check. Two things came out of it:
- Get-PathDenialDetail threw a parameter-binding exception on an empty
path. It runs while a failure is being reported, so it would have
replaced the actionable message with a raw binding error at exactly the
wrong moment. Null and empty are now accepted and return no detail.
- The link-target lookup used an empty catch, which PSScriptAnalyzer flags
and which hid the intent. It assigns $null explicitly now.
Both are covered by new checks. Also promoted the strongest invariant from
the simulation into the suite: Get-PathState must agree with a bare
Test-Path on every probe that did not throw, and Denied may only appear
where the old probe threw, so no path that worked before can take a
different branch now.
Verified: PSUseCompatibleSyntax reports nothing for 5.1, 6.2 and 7.0; the
Python contract tests pass on 3.10 through 3.13 in separate uv venvs; the
tauri install:: unit tests pass (17), which is the code that prefers the
[TAURI:ERROR] line over the generic exit-code message.
* Trigger the Windows PowerShell tests when they change
studio-windows-inference-smoke.yml runs six PowerShell unit tests out of
tests/studio, but its pull_request paths filter matched none of them, and
no other workflow runs them. A PR touching only one of those tests never
ran it. Five predate this branch; the sixth is the ACL test added here.
Scope the filter to tests/studio/*.ps1 rather than tests/studio/**, so a
python-only change under that directory does not pull in the GGUF smoke
jobs. This matches what the other two workflows already do: parity-ci
lists its .ps1 test outright and update-smoke uses a scoped glob.
Guard it in test_ci_shell_suite_coverage.py, which exists for this exact
failure (tests/sh had the same hole): every tests/*.ps1 a workflow invokes
must be matched by that workflow's paths filter, and must exist. The
GitHub glob matcher it needs has its own table-driven test, since a wrong
matcher would make the guard pass on everything.
Verified by reverting the one-line filter change: the guard then names all
six unrun tests.
* Make the Windows PowerShell test step fail when a test fails
Verifying the path-filter fix turned up a second hole in the same step. A
`shell: pwsh` step inherits only the LAST command's exit code, and this
step ran five tests as five bare commands, so only the last one could fail
the build. test_resolve_cuda_toolkit.ps1 has been printing
FAIL exits non-zero (scenario 2, forced source build)
FAIL exits non-zero (scenario 6, no toolkit, forced)
2 check(s) FAILED
on every Windows run, exiting 1, and the job reported success. Confirmed
on main (run 30723608191, sha
|
||
Etherl
|
1faa0377e5
|
Fix Windows llama.cpp prebuilt setup with inaccessible PATH entries (#7696)
* Fix Windows prebuilt fallback on inaccessible PATH * Fix inaccessible inherited Windows PATH entries * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Windows PATH test on POSIX * Keep transient prebuilt failures on the source-build path Narrowing the source build to helper exit 2 made every other nonzero exit fatal, but the helper reports a rate-limited or unreachable api.github.com as EXIT_ERROR: fetch_json raises a bare RuntimeError for HTTP 403, and the fork branch of the release resolver has no wrapper of its own where the ggml-org branch does. Unauthenticated GitHub API calls are limited to 60 an hour per IP, so a shared runner or a NAT'd network hits this routinely, and a source build clones over git rather than the API, which is why falling back used to recover. This PR's own macos-14 kill@torch run failed exactly that way. Classify release-listing failures on the install path as PrebuiltFallback so they exit 2 again. Scoped to install_prebuilt rather than the resolver: --resolve-prebuilt turns PrebuiltFallback into a successful {"prebuilt_available": false} payload and update_flow caches any exit-0 answer for RESOLVE_TTL_SECONDS, so wrapping there would pin a transient 403 as "no prebuilt" for 24 hours. Also close the paths that stayed fatal: - collect_system_report ran inside the PrebuiltFallback handler, so a probe that raised replaced the in-flight fallback with EXIT_ERROR. - python_runtime_dirs and windows_runtime_dirs stat sys.path entries and %ProgramFiles% outside dedupe_existing_dirs, so skip_unusable could not protect them and a denied entry still aborted Windows discovery. - binary_env still required inherited LD_LIBRARY_PATH and DYLD_LIBRARY_PATH to be readable, the same thing the Windows branch stopped requiring. - sync_marker_force_cpu and sync_marker_llama_backend guarded the marker read but not the write, so a read-only marker on an otherwise up to date install failed setup. - runtime_libs.python_runtime_dirs, the serve-time copy, had the same unguarded sys.path stat; the sidecar turns the raise into an empty dir list and loses every CUDA wheel dir. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only reclassify transient resolver failures, and persist the reuse marker Three follow-ups from cross-platform validation. The resolver wrapper caught Exception, so a TypeError or AttributeError from a bug in host-conditional resolver code (per-gfx ROCm, Windows arm64, the macOS walk-back) also exited 2 and bought a source build, logged under a message that blames the network. Narrow it to OSError, RuntimeError and ValueError, which covers every transient shape by MRO: URLError, HTTPError, SSLError and TimeoutError are OSError, JSONDecodeError is a ValueError, and fetch_json raises a bare RuntimeError for HTTP 403. Code defects go back to EXIT_ERROR, with a test pinning that direction. Guarding the marker write stopped a read-only marker from failing setup, but it also let a deliberate --force-cpu go unrecorded while setup reported success, which is how the updater re-routes a CPU user onto a GPU bundle (#7213). Write through atomic_write_bytes first, which swaps in a sibling temp file and so lands on a read-only marker in a writable dir, and warn loudly when it genuinely cannot be recorded. Skip the setup.sh routing test on Windows: setup.sh is the POSIX installer, Windows runs setup.ps1, and driving a POSIX script through Git Bash with Windows paths proves nothing about either. The PowerShell branch of the same routing stays covered by the platform-independent textual test. * Make Python runtime discovery optional, and never truncate the reuse marker Guarding only the search root left the strict dedupe on python_runtime_dirs' own return to raise, so a readable site-packages with a denied torch/lib or nvidia/*/lib child still aborted Windows discovery. That is the same shape as the bug this branch is about, one level down: the parent lists fine and the entry underneath is denied. These candidates are optional CUDA wheel dirs found by globbing, and one that cannot be stat'd could not have served DLLs to the loader either, so skip them like the serve-time copy already does. Drop the in-place retry in the marker rewrite. It opened a valid marker with truncation, so an ENOSPC or I/O error mid-write would strand a partial UNSLOTH_PREBUILT_INFO.json and later updates would stop recognising the install. The atomic path already covers the case the retry was there for, a read-only marker in a writable dir; when it fails, leave the old marker alone and warn. * Preserve the marker mode across the atomic refresh os.replace keeps the source file's mode and NamedTemporaryFile is 0600, so refreshing a shared install's marker left UNSLOTH_PREBUILT_INFO.json readable only by whoever ran setup, and other users could no longer recognise or update that installation. Reproduced: a 0444 marker came back 0600. Build the temp file here instead of calling atomic_write_bytes, so the original mode is restored before the swap rather than after, leaving no window where the marker is private. Clean up the temp file if the replace fails, so a failed refresh strands nothing next to the marker. * Claim only transport failures, and never strand a temp marker A plain OSError is not evidence of a network problem. EMFILE after file descriptor exhaustion, ENOMEM, or a local EACCES reading TLS configuration were all being converted to EXIT_FALLBACK, so setup started the resource heavy source build that this branch otherwise refuses for unexpected helper failures, and a build needs more descriptors and more memory, not fewer. Name the transport shapes instead: URLError covers HTTPError and the socket/DNS errors urllib wraps, plus SSLError, ConnectionError, TimeoutError, and the RuntimeError and ValueError that fetch_json raises for a 403 and for an undecodable payload. ENOSPC still reaches EXIT_NO_SPACE, now through __main__'s classifier rather than install_prebuilt's, and the test asserts it the same way __main__ decides it. The marker temp file was only unlinked when the chmod or the replace failed. A write, flush or fsync that raised, which is the ENOSPC case this is built to tolerate, jumped straight past the cleanup and left a partial UNSLOTH_PREBUILT_INFO.json.tmp-* beside the valid marker, one per attempt on a full volume. Track it across every failure path instead. Also restore the original owner and group on the replacement where the caller is permitted to, since os.replace installs the temp file's ownership and a shared marker would otherwise pick up the invoking user's primary group. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments added by this change --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
2d61c3aa04
|
Studio setup: report a missing llama.cpp in Tauri mode instead of aborting the install (#7720)
* Studio setup: report a missing llama.cpp in Tauri mode instead of aborting the install setup.sh ends by exiting non-zero when the llama.cpp prebuilt did not produce a usable server and it was called from install.sh, so the installer can report the GGUF failure after finishing PATH and shortcut setup. In Tauri mode that non-zero exit is not "report", it is "abort": install.rs turns it into "Installation failed". So one transient prebuilt download failure, and a single HTTP 403 rate limit is enough, fails the entire first-launch install of the desktop app, moments after setup.sh's own footer said Installed. Everything except GGUF inference is in fact working, and whisper.cpp in this same script already degrades rather than failing for exactly this case. Tauri mode now emits a [TAURI:STEP] line naming what is missing and how to get it back. Every other caller keeps the non-zero exit unchanged. * Use the progress channel for the notice, and degrade on Windows too Two problems with the first version, both found in review. The notice used [TAURI:STEP], which is the wrong channel twice over. install.rs maps it to install-step, and use-tauri-backend.ts counts those without ever storing the payload, so the text was thrown away. Worse, install.sh already emits exactly seven unique STEP markers against the seven-entry INSTALL_STEPS list, so an eighth pushed the counter past the end and rendered 'Step 8 of 7'. [TAURI:PROGRESS] becomes install-progress-detail, which the installing screen renders verbatim. Windows had the identical defect and was not covered. install.ps1 sets both SKIP_STUDIO_BASE=1 and UNSLOTH_TAURI_MODE, then turns any non-zero setup.ps1 status into Exit-InstallFailure, which install.rs reports as 'Installation failed'. setup.ps1 now gates on UNSLOTH_TAURI_MODE the same way Exit-SetupFailure already does, and still fails for every other caller. Tests: 8 cases, adding one that the notice never emits [TAURI:STEP] and a setup.ps1 block check for the Windows parity. |
||
|
Daniel Han
|
479b46e8a9
|
studio setup: let Windows PowerShell load its own Security module (#7692)
* studio setup: let Windows PowerShell load its own Security module
`unsloth studio update` failed on Windows when launched from a PowerShell 7
prompt:
installing uv package manager...
The 'Get-ExecutionPolicy' command was found in the module
'Microsoft.PowerShell.Security', but the module could not be loaded.
and stopped there with exit 1 and no further output. setup.ps1 installs uv by
running astral's install.ps1, which calls Get-ExecutionPolicy, and
Invoke-SetupCommand makes a failure there fatal.
_run_setup_script spawns powershell.exe, which is Windows PowerShell 5.1, and
the child inherits the caller's PSModulePath. From a pwsh 7 prompt that path
leads with PowerShell 7's module directories, which ship their own
Microsoft.PowerShell.Security. 5.1 finds that copy first and cannot load it.
The problem is precedence, not absence, so the system directory is prepended.
Two weaker variants were tested on a windows-latest runner and both still
failed: dropping PSModulePath so 5.1 rebuilds its default (the machine-level
value on that image also leads with PS7), and appending the system directory
(the PS7 copy is still found first).
Confirmed by a patched/unpatched pair on the same runner and the same PyPI
install, differing only in this file:
patched update exit 0, Security module error: False
unpatched update exit 1, Security module error: True
Guarded on PSEdition so the block is inert under PowerShell 7 itself, and on
Test-Path so a non-standard SystemRoot is a no-op.
pwsh is what Windows Terminal's default profile and the `pwsh` command give a
modern user, and there was no Windows coverage of `unsloth studio update` in
CI, so this went unnoticed.
* Apply the same PSModulePath fix to install.ps1, which has the same exposure
install.ps1:1493 runs astral's uv installer in-process the same way setup.ps1
does, and that installer calls Get-ExecutionPolicy out of
Microsoft.PowerShell.Security. The desktop app reaches it as
Tauri -> Rust -> powershell.exe (studio/src-tauri/src/install.rs:326), and
PowerShell rewrites PSModulePath only for a direct pwsh -> powershell.exe hop,
so the Rust process in between leaves Windows PowerShell 5.1 leading with
PowerShell 7's module directories. PowerShell/PowerShell#18681 is this exact
chain through an intermediate process.
The --shortcuts-only path returns at install.ps1:1104 before reaching the uv
install, so the `unsloth studio update` shortcut refresh was never exposed.
scripts/uninstall.ps1 loads no Security cmdlet and spawns no shell, so it needs
nothing.
Also guards on $env:SystemRoot, since Join-Path throws on an empty or null Path
under ErrorActionPreference Stop and this runs before anything else, and
corrects the comment about why the failure is fatal: the uv call is wrapped in
try/catch, so what ends the run is that Invoke-Expression executes the installer
in this process.
* Record why PSModulePath is not restored
The reordering is process scope, so it outlives the script in an
interactive console. That is a deliberate trade, not an oversight, and
the reasoning was only in the PR discussion. Comment only.
* Keep PSModulePath out of the registry refresh
Refresh-Environment reloads every Machine and User variable except Path,
which put the broken machine-level PSModulePath back over the
normalization at the top of the file. Eight of its call sites run before
the uv installer at setup.ps1:3080, and that installer is what loads
Microsoft.PowerShell.Security, so the fix was undone exactly where it has
to hold. PSModulePath now joins Path as an exception.
Adds tests/studio/test_psmodulepath_normalization.ps1, which pins both
halves: the system directory is prepended rather than appended in both
entry points, and Refresh-Environment leaves PSModulePath alone. It fails
against the previous commit.
|
||
|
Daniel Han
|
07fb20f5b0
|
Route the explicit Vulkan setup failures through the setup failure helpers (#7645)
#7188 added three bare exits to studio/setup.sh and three to studio/setup.ps1 for the explicit-Vulkan paths. Bare exits skip setup_fail / Exit-SetupFailure, so the [TAURI:ERROR] line that #7529 added never reaches Studio desktop and a Vulkan install failure shows up there with no actionable context. tests/sh/test_tauri_retry_failure_context.sh already asserts that each setup script has exactly one explicit exit and that it lives inside the helper. That assertion was failing, but the failure was invisible: Backend CI skips the Shell installer tests step whenever the auto-discovered pytest step fails first, and it had been failing on an unrelated encoding assertion until #7642. Exit codes are unchanged: setup_fail 1 and Exit-SetupFailure both exit 1, and the Tauri line is still emitted only when UNSLOTH_TAURI_MODE is set. |
||
Ayushman
|
dbfae7d92c
|
Studio: support Vulkan GPU selection and explicit Vulkan installation (#7188)
* feat(studio): GGUF host-residency memory mode + Vulkan gpu_ids pinning
Narrowed to complement #6414 (merged), which already provides the GGUF GPU
device picker, gpu_memory_mode (auto/manual), gpu_layers, n_cpu_moe, and
tensor_split. This adds only what #6414 lacks:
1. Host-residency `gguf_memory_mode` (auto/pinned/resident) mapping to
llama.cpp --mlock / --no-mmap. Distinct from #6414's gpu_memory_mode (that
controls VRAM/layer offload; this controls host-RAM residency). Default auto
is a no-op: an omitted value launches identical argv/env to main. First-class
field shadows and scrubs inherited --mmap/--no-mmap/--mlock and
LLAMA_ARG_MLOCK/NO_MMAP/MMAP so a stale value can't leak. Backend/API-driven
(mirrored from /status), no new UI control.
2. Vulkan-ordinal gpu_ids hardening. #6414 rejects gpu_ids on a Vulkan build
with HTTP 400; this replaces that with real pinning: gpu_ids are treated as
ggml Vulkan ordinals (matched against the probe directly, never remapped
through CUDA_VISIBLE_DEVICES), pinned via --device Vulkan<i>, with conflicting
user --device / inherited LLAMA_ARG_DEVICE stripped under a pin, and the
training coexistence guard budgeting conservatively against the least-free
card (ordinals can't map to physical free-VRAM).
Backend: gguf_memory_mode field + validator (LoadRequest / ValidateModelRequest,
echoed on responses); _memory_mode_flags / _canonical_memory_mode; strip_memory_mode
+ strip_device in llama_server_args; is_vulkan_build / assert_requested_gpu_ids_resolvable;
memory-mode reload-dedup in _already_in_target_state and the route matcher
(device-stripped under a pin, mirroring the launch); gpu_ids_are_vulkan_ordinals
budgeting in training_vram. Frontend: a read-only activeMemoryMode mirror wired
into the existing reload flow (no dropdown). #6414's gpu_ids picker is untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve inherited memory flags and reject GPU pins on CPU-only llama.cpp builds
- strip_shadowing_flags: default strip_memory_mode to False (opt-in, matching
strip_offload / strip_tensor_split / strip_device). The inherit resolver now
strips an inherited --mlock/--mmap/--no-mmap only when the request supplies
gguf_memory_mode, so a same-model Apply that omits the field keeps a user's
pass-through memory flag instead of silently dropping it.
- /load and /validate: on the CUDA path, reject explicit gpu_ids when the
llama.cpp build ships no cuda/hip/vulkan ggml lib (CPU-only). Such a build
ignores CUDA_VISIBLE_DEVICES, so the pin would run on CPU while the API
reports it active. Mirrors the non-CUDA resolvable check.
- Document that gpu_ids are ggml Vulkan ordinals on a Vulkan build (enumerated
independently of CUDA_VISIBLE_DEVICES), not CUDA physical indices.
- Add regression tests: CPU-only route reject, inherit preserve/strip of memory
flags, and the flipped strip_memory_mode default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Detect versioned Vulkan libs and sync memory mode across all load paths
- _is_vulkan_backend now matches versioned Vulkan sonames (libggml-vulkan.so.0)
via a shared _lib_dir_has_ggml_backend matcher, unified with
_backend_lacks_gpu_lib so Vulkan detection and the CPU-only-build check agree.
A distro/split-lib Vulkan install without the dev-only unversioned symlink is
now detected as Vulkan and gets the --device Vulkan<i> pin instead of being
misread as CUDA. This also keeps the Chat Settings picker reliably disabled on
Vulkan builds (main.py gates gguf_gpu_ids_supported on this detection), so the
UI never emits physical indices into the Vulkan-ordinal route.
- Frontend: fold activeMemoryMode into loadedGpuMemoryFields so every successful
load path (primary, rollback, compare-load, GGUF/non-GGUF/MTP auto-load) resets
it from the LoadResponse, not just the primary path. Prevents a stale pinned
mode from an earlier model being re-sent by an immediate same-model Apply.
- Add test_is_vulkan_backend_matches_versioned_soname.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pin get_device in gpu_ids GGUF route test for GPU-less CI
test_inference_route_validates_gpu_ids_for_gguf patches resolve_requested_gpu_ids
to exercise the CUDA physical-ID resolver, but did not pin the device. On a GPU-less
CI runner get_device() returns non-CUDA, so the route took the non-CUDA/Vulkan
deferral branch whose 'no GPU backend detected' 400 contains the substring 'not
supported', tripping the test's assertNotIn. Pin get_device to CUDA so the test
deterministically covers the path it intends.
* fix(tests): simplify ordering assertions and tighten GPU validation error message
* fix(studio): add GGUF variant check before reusing draft extras; apply aggregate headroom to Vulkan multi-GPU pins
- inference.py /load path: check extra_args_source and gguf_variant before
reusing a loaded server's extra_args for draft-device validation, so a
different quant of the same repo no longer inherits stale --spec-draft-device.
- inference.py /validate path: same variant-aware check before inspecting
stored backend extras for draft-device rejection.
- training_vram.py: Vulkan multi-GPU pin now also enforces the aggregate
_MULTI_GPU_OVERHEAD (0.85) check instead of returning early on per-GPU
min_free alone, preventing a chat load from starting with too little
protected headroom and OOMing an active training run.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): harden Vulkan probe/budget and diffusion classification for GGUF placement
- Use the versioned-soname matcher in the Vulkan free-memory probe too, matching
_is_vulkan_backend: a split-library install that ships only libggml-vulkan.so.0
is now sized correctly instead of returning no devices and rejecting gpu_ids.
- Budget the Vulkan multi-GPU aggregate over the least-free N pinned cards, not all
visible GPUs: with the ordinal to physical mapping unknown, summing every visible
card could approve a pin that no N-card placement can actually hold.
- Classify a GGUF from its local header before the name heuristic: a normal
llama-server GGUF whose path or repo merely contains "diffusion" is no longer
rejected for gguf_memory_mode, since the loader routes on the decoded header.
- Compute the Vulkan-ordinal flag before diffusion_gpu and gate diffusion_gpu off
when it fires, so an unclassified GGUF pinned on a Vulkan build is budgeted as
ordinals instead of the single-device CUDA path sizing the wrong physical card.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): gate XPU gpu_ids rejection to non-Vulkan; skip GGUF memory-mode reject for non-GGUF
- /load and /validate paths now detect the Vulkan build before the XPU
rejection so Vulkan-backed Intel GPUs can use gpu_ids via --device
Vulkan<i> ordinals instead of being blocked by the torch-xpu device check.
- _reject_diffusion_memory_mode now returns early for non-GGUF configs so
a stale or shared payload with gguf_memory_mode cannot block unrelated
non-GGUF loads.
* fix(studio): skip llama.cpp GPU-lib check for diffusion; resolve versioned Vulkan libs in probe
- /load and /validate paths now skip _backend_lacks_gpu_lib for confirmed
DiffusionGemma GGUFs, since the diffusion runner bypasses llama-server and
handles gpu_ids via --gpu/DG_GPU independently of the llama.cpp build.
- _vulkan_probe.py now resolves versioned sonames (libggml-vulkan.so.0 etc.)
via a directory scan so split-lib installs that lack the unversioned symlink
are no longer rejected with no visible devices.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test(studio): cover diffusion gpu_ids skip + versioned Vulkan soname; mirror is_vulkan_build on draft-device stubs
- Add is_vulkan_build to three draft-device test stubs so they mirror the real
backend: the gpu_ids validation now reads it before the draft-device reject.
- test_cpu_only_llama_build_skips_reject_for_diffusion_gguf: a diffusion GGUF is
served by the visual-server runner, so a CPU-only llama.cpp build does not
reject its gpu_ids (the flow reaches the id resolver past the skipped reject).
- test_versioned_only_vulkan_soname_is_probed: a split-library install shipping
only libggml-vulkan.so.0 is still classified Vulkan and probed, not rejected.
* fix(studio): route confirmed diffusion gpu_ids through the CUDA path on a Vulkan build
A confirmed diffusion GGUF is served by the visual-server runner, which takes a
CUDA physical id via --gpu/DG_GPU (it remasks CUDA_VISIBLE_DEVICES), never a
Vulkan ordinal. The gpu_ids preflight previously sent it to the Vulkan-ordinal
branch whenever the installed llama-server was a Vulkan build, so a valid
physical pick could be rejected (no matching Vulkan ordinal) or an ordinal
validated that the runner then applies to the wrong CUDA GPU. Route a confirmed
diffusion GGUF through the CUDA resolver at both /load and /validate even on a
Vulkan build, matching the training guard's existing diffusion_gpu handling.
* Keep llama.cpp fit headroom
* Adapt host memory modes to llama.cpp load mode
* feat(studio): expose GGUF host memory controls
* Clarify GGUF host memory semantics
* Expose Vulkan ordinals in the GGUF GPU picker
* Show Vulkan hardware names in the GPU picker
* Clarify the llama.cpp Vulkan backend selector
* Keep explicit Vulkan choices authoritative
* Preserve requested and effective GGUF GPU pins
* Align GGUF dedupe fixture with requested GPU pins
* Restore Vulkan diffusion pin guard
* Close GPU backend integration gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Simplify GGUF placement state and validation
* Trim GGUF placement comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF placement review findings
* Address GGUF placement review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reject invalid remote diffusion placement before teardown
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Correct Vulkan GGUF placement boundaries
* Simplify Vulkan metadata handling
* Preserve deferred GPU index namespaces
* Preserve deferred GPU and memory settings
* Preserve host memory in compare loads
* Handle DiffusionGemma memory settings
* Use semantic preset summary font size
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Enforce host memory environment overrides
* Keep host memory default implicit
* Clarify host memory settings
* Clarify host memory choices
* Clarify Host RAM choices
* Explain Host RAM behavior
* Limit Host RAM wording change
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard GPU selections by backend namespace
* Correct Host RAM placement semantics
* Restore the GGUF fit target
* Remove GGUF Host RAM controls
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Correct GPU placement comments
* Fix DiffusionGemma GPU selection capabilities
* Make GPU selections own main device placement
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim GGUF placement regression coverage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove orphaned memory policy and close Vulkan gaps
* Correct draft-device recovery guidance
* Fix Vulkan source and training device handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align automatic Vulkan device pools
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail closed on explicit Vulkan no-space, refresh the GPU picker on probe recovery
setup.sh and setup.ps1 both dispatch the prebuilt installer's exit status
before checking whether Vulkan was explicitly requested, so status 4 (no disk
space) took the no-space branch and never reached the strict-backend check in
the else. With an older CUDA, ROCm or CPU llama-server already installed,
_has_local_llama_server kept _LLAMA_CPP_DEGRADED false, so setup exited 0 and
Studio carried on using the backend the user asked to replace. Driving the
real dispatch block with a stubbed installer shows the same explicit request
exiting 1 on a generic failure and 0 on a no-space failure. Apply the same
check to the no-space path in both scripts.
useGpuDevices fetched /api/system once from an effect with no dependencies and
subscribed to nothing. When the Vulkan probe is unavailable, main.py answers
gguf_devices as an empty list rather than omitting it, so the "?? devices"
fallback does not apply and the picker starts hidden. useInferenceGpuInfo then
retries every 3 seconds and refreshes the shared cache, but nothing told the
mounted device hook, so the picker stayed hidden until it was remounted.
Notify subscribers when a refetch replaces the cache and have useGpuDevices
subscribe, comparing by value since each refresh builds a fresh array and an
unconditional set would re-render on every retry tick.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Vulkan placement state and installation
* Use published Vulkan bundles
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Consolidate llama.cpp backend selector
* Specify UTF-8 for file operations
* fix(studio): harden GGUF GPU placement
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): align backend selection contracts
* fix(studio): align MTP draft device placement
* fix(studio): address Codex P2s on legacy Vulkan env var and cold-cache GPU pins
- llama_backend_from_env() (and setup.sh/setup.ps1's shell-level mirror) falls
back to the legacy UNSLOTH_LLAMA_BACKEND when UNSLOTH_LLAMA_CPP_BACKEND is
unset, so an existing UNSLOTH_LLAMA_BACKEND=vulkan environment keeps forcing
Vulkan across the setup.sh/setup.ps1 consolidation instead of silently
reinstalling CUDA/ROCm/CPU.
- loadedGpuMemoryFields() no longer drops a just-applied gpu_ids pin to null
when cachedPinnableGpuIndexKind() returns undefined (cache cold / Vulkan
probe not ready yet). That state is deferred, not rejected: the pin is kept
so a reload/rollback during that window still sends gpu_ids instead of
letting llama.cpp fall back to every device.
The third P2 (force-compile skipping the Vulkan-source-build rejection) was
already fixed by an earlier commit on this branch -- verified
_NEED_LLAMA_SOURCE_BUILD is set before the guard runs in both setup.sh and
setup.ps1, and added a regression test locking in the ordering.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix Vulkan backend env fallback and GGUF preflight token
Three narrow correctness fixes on the Vulkan GPU selection path.
setup.sh / setup.ps1 consulted the legacy UNSLOTH_LLAMA_BACKEND only when
UNSLOTH_LLAMA_CPP_BACKEND was empty, while install_llama_prebuilt.py's
llama_backend_from_env() consults it whenever the new var is not an explicit
backend name. With UNSLOTH_LLAMA_CPP_BACKEND=auto beside a legacy
UNSLOTH_LLAMA_BACKEND=vulkan the shell dropped the request, leaving
_explicit_vulkan_backend and _explicit_vulkan_source_build false while the
installer it launches still planned Vulkan, so a Vulkan install that could not
be satisfied silently degraded to a CUDA/ROCm/CPU build instead of failing
closed. Mirror the Python rule in both scripts; an unrecognized value with no
legacy backend is still preserved so the warning can name it.
The new GGUF diffusion preflight in the chat load path sent the raw stored HF
token to /validate before validateModel/loadModel could run
prepareHfTokenForUse. The Hub rejects an invalid Authorization header with 401
even on a public repo, so a stale saved token aborted the whole load instead of
offering the existing continue-anonymously/replace-token recovery. Prepare the
token the same way the compare path already does.
The training guard handed can_load_chat_during_training an empty Vulkan
free-VRAM map for an unclassified GGUF on a Vulkan build. That reads as no free
VRAM anywhere, so every uncached remote GGUF was refused with 409 while
training ran. Only an explicit pin is unbudgetable (the ordinal belongs to one
namespace and neither can stand in for the other); automatic placement has no
ordinal to mis-map, so it keeps the torch view.
Each fix has a regression test that fails without it.
* Let the diffusion runner take physical ROCm indices
device_backend is a display label: _backend_label swaps CUDA for "rocm" when
IS_ROCM is set, but ROCm reuses torch.cuda.* and the DiffusionGemma runner
selects by the same physical index either way. Gating on "cuda" alone marked
every physical ROCm device unpinnable, so the picker vanished on multi-GPU
ROCm hosts and status hydration could discard a valid selection.
* fix(studio): keep unavailable Vulkan inventory out of CUDA fallback, prepare autoload HF token, restore legacy hip/rocm opt-out
Three P2s from the latest Codex review round on PR #7188:
- toGpuDevices(): a confirmed-Vulkan inference backend with a still-cold or
transiently-empty device probe no longer falls through to the torch/CUDA
inventory. That fallthrough was marking physical CUDA/ROCm devices
diffusionPinnable, exposing a GPU picker for DiffusionGemma that sent IDs
the backend rejects outright whenever is_vulkan_build is true. Now returns
no devices until the Vulkan probe actually succeeds.
- loadAutoLoadCandidate(): the startup auto-load path's diffusion-classifying
fetchGgufStagedMetadata probe (fired when a cached GGUF has saved gpu_ids)
sent the raw stored HF token, bypassing prepareHfTokenForUse. Unlike
validateModel/loadModel (which prepare internally), fetchGgufStagedMetadata
does not, so an expired token could 401 even a public repo and abort the
candidate before the anonymous/replace-token recovery flow got a chance to
run. Now prepares the token first, mirroring performLoad's identical guard.
- _normalized_llama_backend(): dropped hip/rocm entirely during the backend-
selector consolidation, despite force_vulkan_requested()'s own comment
("so =hip is a real opt-out a stale UNSLOTH_FORCE_VULKAN cannot overrule")
and _route_to_vulkan_prebuilt()'s ("an explicit hip/cpu is the opt-out")
describing behavior it no longer delivered. An existing
UNSLOTH_LLAMA_BACKEND=hip/rocm environment on an unsupported HIP arch read
as "no backend named" and silently routed to Vulkan (auto-fallback, or a
stale UNSLOTH_FORCE_VULKAN=1). Restored hip/rocm recognition (canonicalized
to "hip"), matching the pre-consolidation mapping.
Regression tests added for all three.
* Cover the auto-load HF token preflight behaviourally
The contract test added alongside the fix asserts on the source text. This
runs the real classification block out of chat-adapter.ts under node with a
stubbed Hub that 401s any non-null Authorization value, so it fails on the
token value that actually reaches /api/inference/validate rather than on a
symbol being present.
Reverting only the source (leaving the import in place) reddens it with the
simulated 401, not an ImportError.
* Converge Vulkan backend selection and review fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Stop the structlog stub shadowing the real package for PR #7188
The bare setdefault parked an empty placeholder before anything imported
the real structlog, so every later module calling structlog.get_logger at
import time raised AttributeError. It only bit when this file was collected
first, which is why the 8 hardware-dispatch cases passed alone and failed
under pytest tests/studio. Only stub when the package is genuinely absent.
* Prepare model settings GGUF token preflight
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the established llama.cpp backend selector
* Preserve GPU namespace while Vulkan inventory recovers
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix final PR 7188 review findings
* Normalize llama.cpp device flag aliases
* Converge PR 7188 review findings
* Honor backend opt-outs for Intel Vulkan routing
* Retry cold system discovery for GPU selection
* Probe structlog with find_spec instead of a bare import
The availability check imported the module purely for its side effect, so
the import-hoist verifier flagged it as an added-but-unused import and
failed Source lint. find_spec answers the same question without binding a
name.
* Brace the PowerShell variable in the setup.ps1 backend-choice probe
* Leave an existing structlog entry alone in the availability probe
find_spec raises ValueError on a module already in sys.modules whose
__spec__ is None, which is what a bare types.ModuleType stub is. Check
sys.modules first so anything already present, real or stubbed, is left
untouched and only a genuinely absent package gets stubbed.
* Consolidate duplicated Vulkan selector tests without changing behaviour
Three of the tests added for the llama.cpp backend selector asserted on the
same two helpers with the same inputs, and two more differed only in whether
the backend arrived from the environment or from --llama-backend. Fold them
into the parametrized cases that already covered those helpers:
- test_llama_cpp_backend_env_requests_vulkan,
test_llama_cpp_backend_auto_does_not_trigger_vulkan and
test_hip_backend_env_opts_out_of_vulkan become rows on
test_force_vulkan_requested_accepts_public_selector_and_legacy_alias, which
now also asserts llama_backend_from_env() alongside force_vulkan_requested().
The unset case and the whitespace-padded HIP/ROCM values keep their coverage
as new rows.
- test_explicit_non_vulkan_backend_suppresses_intel_auto_route and
test_non_vulkan_backend_argument_suppresses_intel_auto_route become one test
parametrized over the env and argument paths.
- test_unclassified_gguf_without_pin_keeps_the_torch_budget and
test_unclassified_gguf_with_pin_refuses_to_budget shared the same four
patches and differed only in the pin, so they become subtests of one case.
No source changes and no assertion is dropped. Each behaviour was re-checked by
reverting the source hunk it guards and confirming the surviving test fails on
a value.
* Keep the standing GPU mode on diffusion loads and the ARM64 Vulkan fallback for PR #7188
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the Intel auto-route guard test able to fail and stop the ps1 harness doubling --force-cpu
test_non_vulkan_backend_suppresses_intel_auto_route ran on Linux x86_64, where
dropping the explicit_backend guard changes only a log line, so the test passed
either way. Move it to Linux ARM64 + Intel GPU, the host where the guard decides
the routed repo, and assert that value.
_run_ps1 composed the --force-cpu snippet twice: it is already inside the
normalized block the helper extracts. The substring assertion hid the duplicate.
Compose it once and compare the whole argv. setup.ps1 appends the flag once and
is unchanged.
* Dot-source Get-HostMachineArch in the setup.ps1 Pester suite
#7549 taught Test-VCRedistInstalled to consult the host architecture before
trusting the System32 DLL. The Pester suite dot-sources a fixed list of
functions out of setup.ps1 one at a time, and that list did not gain the
helper, so the two clean-box cases (registry miss, and an old sub-14.20
redist) throw "Get-HostMachineArch is not recognized" instead of returning
false. The registry hit returns early, which is why the other cases pass.
#7597 made exactly this fix to the sibling list in the VC++ round-trip job
but left the Pester suite alone. Verified with pwsh 7.6 + Pester: 3 failed
before, 31 passed 0 failed after.
* Drop the branch's Pester Get-HostMachineArch fix now that #7606 landed
This branch carried a local fix for the same Pester dot-source gap that
#7606 has since merged to main (
|
||
|
Daniel Han
|
9bfa18cdb0
|
Windows: unblock the consumer install on clean and no-winget machines (#7549)
* Windows: unblock the consumer install on clean and no-winget machines Four independent things stop a clean Windows box today. git was a hard Exit-SetupFailure in setup.ps1, justified as required by pip for git+https:// deps and by npm. Neither holds on the consumer path: the unsloth-zoo git+https URL is only used under STUDIO_LOCAL_INSTALL, node is a pinned nodejs.org prebuilt that never touches system npm, and the frontend lockfile has no VCS dependencies. It stays fatal for --local, where it really is needed. Ensure-VCRedist was winget-only, so on hosts without winget (LTSC, Server, managed corporate images) it silently did nothing while the install reported success, and torch then failed to import on a missing VCRUNTIME140.dll. Adds a direct aka.ms/vs/17/release/vc_redist.<arch>.exe download with /quiet /norestart, accepting exit codes 0 and 3010. The redistributable stays required: it is the runtime the prebuilt llama-server and torch link against, not the MSVC compiler, which is already detection-only. Windows on ARM has no PyTorch at all. Measured with uv against download.pytorch.org/whl/cpu and PyPI for aarch64-pc-windows-msvc / cp313: torch, torchvision and torchaudio all resolve to nothing, wheels exist only for win_amd64 and the manylinux targets. The installer burned three uv retries on an unsatisfiable resolution and reported a bare 'Failed to install PyTorch (exit code 1)'. Now it says what is actually wrong and points at --no-torch, which works because llama.cpp does publish windows-arm64-cpu. install_node_prebuilt.py hit '[WinError 5] Access is denied' on os.replace of the freshly extracted directory during a FRESH install, which is a scanner or indexer holding handles for a moment. Retries only winerror 5, 32 and 145 with capped exponential backoff; any other OSError still raises immediately. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give the ARM64 dead end a recovery that works for web installs The only remedy printed was .\install.ps1 --no-torch, but the documented path is irm | iex, where no file exists and flags cannot be forwarded. Name the env var the script already honours at line 145. * Windows on ARM: drop torchaudio, do not abort the install The fail-fast was based on a wrong premise. Counted against download.pytorch.org/whl/cpu: torch has 42 win_arm64 wheels and torchvision 60; only torchaudio has none. PyTorch has shipped Arm-native Windows builds since April 2025, so aborting blocked a platform that mostly works. Drop the one unsatisfiable pin instead. Decide from the interpreter uv will resolve for, not the PowerShell host: an x64 CPython under emulation gets working win_amd64 wheels on an ARM64 box, and powershell.exe inherits PROCESSOR_ARCHITECTURE from its parent. * Carry the ARM64 torchaudio omission into studio setup Dropping it from the first PyTorch command was not enough: install.ps1 then runs studio setup with SKIP_STUDIO_BASE=1 and setup.ps1 reinstalls the bare trio from the CPU index, so the ARM64 path still aborted. Apply the same interpreter-based test there. An unreadable platform keeps the full trio. * Build the torch spec list outside the verbose branch The ARM64 guard landed inside `if ($script:UnslothVerbose)`, so on the default path $_torchTrio was never assigned and the splat expanded to nothing: uv ran as `uv pip install --index-url ...` with no package, exit 2, straight to Exit-SetupFailure. That broke the ordinary Windows install. Hoist it above the branch and use substep, which prints on both paths. Realign the two parity guards to the splat form; they asserted the pre-refactor literal command and were the actual cause of the red parity legs. Both halves are still checked: the bounded list is built, and it reaches the install. * Tighten the comments on the Windows install path * Windows install: honour the ARM64 torchaudio skip everywhere and keep git for source builds Hoist the venv-interpreter platform probe above every torch branch in studio/setup.ps1 so the win_arm64 torchaudio omission applies to the ROCm, CPU and CUDA/custom paths. A pinned index whose leaf is not cpu routed an ARM64 host into the CUDA/custom branch, which still asked for torchaudio. Require git again when a llama.cpp source build is opted into up front (UNSLOTH_LLAMA_FORCE_COMPILE, UNSLOTH_LLAMA_PR / PR_FORCE, a non-upstream source). Those paths git clone in phase 4, so setup used to report git as not required, install the build toolchain, then fail at the clone. A local llama.cpp dir overrides them, and the automatic source fallback after a failed prebuilt download stays non-fatal. Also tighten the comments across the changed install paths. * Install the x64 VC++ runtime unconditionally in the direct-download fallback The winget branch always installs Microsoft.VCRedist.2015+.x64, but the direct-download fallback picked the package from PROCESSOR_ARCHITECTURE, which reports the architecture of the running PowerShell process rather than the interpreter that will load the DLLs. Find-CompatiblePython in install.ps1 selects an interpreter on version and non-Conda status alone, with no architecture predicate, so a native ARM64 shell can settle on an emulated x64 Python whose win_amd64 torch and prebuilt llama-server need the x64 runtime, while the fallback had just installed the ARM64-only package. Ensure-VCRedist also runs well before the venv exists, so the interpreter cannot be probed at that point. Microsoft ships the x64 redistributable as an Arm64X superset that carries both ARM64 and x64 binaries, so it is correct on both machines and the manual instruction printed on failure already pointed at it. * Windows on ARM: prefer an x64 Python interpreter An ARM64 host cannot complete the install with a native ARM64 interpreter. pyarrow, pulled in by unsloth -> datasets, has never published a win_arm64 wheel on any version, and neither has hf-transfer, a direct dependency. Both therefore fall back to a source build: pyarrow dies in scikit-build-core CMake configuration and hf-transfer dies in openssl-sys for want of perl, several minutes into a run that looked healthy. torch and torchvision are not the problem, they have win_arm64 wheels and install fine. Windows 11 on ARM runs x64 binaries under emulation and both packages ship win_amd64 wheels, so an x64 interpreter installs cleanly. Find-CompatiblePython accepted an interpreter on version and non-Conda status alone. It now ranks candidates by architecture on ARM64 hosts and returns an x64 one when present, asking each interpreter for its own sysconfig.get_platform() rather than guessing from its path. Host architecture comes from PROCESSOR_ARCHITEW6432 and OSArchitecture as well as PROCESSOR_ARCHITECTURE, which describes only the current process and reads AMD64 in an emulated shell. This is a preference, not a requirement. If only ARM64 is found, x64 is bootstrapped through winget --architecture x64 or the python.org fallback, and if neither works the installer names pyarrow and hf-transfer up front instead of failing later on a CMake or Rust error. The ARM64 torchaudio skip stays live for that path. Non-ARM hosts return on the first match exactly as before, with no extra interpreter probing. * Windows install: three correctness fixes on the ARM64 and git-less paths Ensure-VCRedist never reached its x64 download on an ARM64 machine that already had the arm64 redistributable: Test-VCRedistInstalled accepted System32\vcruntime140_1.dll regardless of architecture, and there that file can be the pure-ARM64 package. An ARM64 PE cannot load into an emulated x64 process, so the x64 Python this branch now prefers would have been left without a usable runtime. The x64 registry entry is the only x64-specific proof, and Microsoft registers Runtimes\{x86|x64|arm64} per architecture, so vc_redist.x64.exe still writes Runtimes\x64 on an ARM64 host and the check cannot loop. The DLL probe stays for x64 hosts. Phase 1 demanded git for any non-blank UNSLOTH_LLAMA_PR_FORCE, but the promotion that actually turns it into a source build requires a positive integer, so PR_FORCE=0 or a non-numeric value aborted a git-less consumer install for a build that never runs. Both sites now use the same predicate. The automatic fallback after a failed prebuilt llama.cpp download reached git clone with no git check anywhere in between, and Invoke-SetupCommand returns 0 for a command-not-found, so a git-less host did not stop there: it continued into an empty directory and reported a cmake configure failure instead. Git is now resolved where the source build is decided, with a last winget attempt, and a missing git degrades exactly like a missing cmake rather than aborting, since the opt-in source triggers already required git in Phase 1. Also tightened the comments across the changed Windows install code, keeping the reasons on the guards that prevent a specific failure. * Rank ARM64 Python candidates by minor version before architecture The x64 preference filtered the whole candidate list on architecture, which outranks the version preference the candidates were collected in. With UNSLOTH_PYTHON=3.12 on a Windows ARM64 box holding an ARM64 3.12 and an x64 3.13, it returned the x64 3.13: the explicit pin was silently broken, and because a x64 interpreter was found the caller never ran Install-X64Python to fetch an x64 3.12. With no pin it was worse still, since an x64 3.11 outranked a newer ARM64 3.13 and defeated the newest-first fallback. Walk $minors in order and take the x64 build of the best minor available, falling back to that minor's ARM64 build so the caller bootstraps x64 for the version actually requested. x64 still wins within a minor, and non-ARM hosts are untouched. * Windows install: see every registered Python, order git before the toolchain Find-CompatiblePython only ever probed `py -3.X`, which runs the launcher's preferred build for that minor. On an ARM64 box that is the native ARM64 interpreter, so a same-minor x64 install that is registered with the launcher but neither preferred nor on PATH never became a candidate. The x64 preference then lost to ARM64, and Install-X64Python re-downloaded an x64 CPython that was already on the machine; when that download is unavailable the install continues on ARM64 and source-builds pyarrow and hf-transfer, which publish no win_arm64 wheels. Enumerate `py -0p` on ARM64 hosts and probe each listed path. The `-3.12-64` suffix cannot be used for this: it has meant "not 32-bit" since 3.11 and does not distinguish arm64 from amd64. studio/setup.ps1 ran Ensure-BuildToolsForLlamaSourceBuild before checking git in Phase 4. That helper calls Exit-SetupFailure when Visual Studio Build Tools cannot be installed, so on a clean no-winget box the git degraded path added by this PR was unreachable and a standalone update aborted instead of finishing in limited mode; where winget does exist it spent a multi-GB Build Tools download on a clone that could never run. Check and install git first, skip the toolchain helper when git is still missing, and report the git branch before the cmake branch so the message names the real cause. _swap_into_place retried the forward rename for about 16 seconds but rolled back with a bare os.replace. A scanner holding the backup for the same WinError 5/32 then left no install_dir at all and stranded the working runtime in .old-*, and its exception replaced the original failure. The rollback now uses the same backoff and logs instead of masking the error it is recovering from. * Installer: use an already installed x64 Python on ARM64 when none can be downloaded Find-CompatiblePython ranks x64 within one minor and returns the native build when that minor is ARM64-only, leaving Install-X64Python to bootstrap x64. On an offline or winget-less box that bootstrap fails, and the retry went through the same resolver, so an x64 build of a lower-priority supported minor already on the machine was never picked up and setup continued on ARM64 Python, where pyarrow and hf-transfer have no wheels. Add an -X64Only mode that returns the best installed x64 interpreter or nothing, and call it as the last resort in Install-X64Python. The version-first preference is unchanged: x64 of the requested minor is still bootstrapped first. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in the Windows ARM64 installer changes * Setup: require Git for a source build behind an unbuilt local llama.cpp dir UNSLOTH_LOCAL_LLAMA_CPP_DIR only overrides the source-build opt-ins once the directory holds a reusable llama-server.exe. Pointing it at the canonical install location with nothing built there falls through to the normal install, so the Phase 1 gate now probes the same layout candidates as the Phase 4 reuse check before dropping the requirement. * Setup: require Git when UNSLOTH_LLAMA_TAG=master forces a source build * Tighten comments in the Windows installer changes * Setup: negotiate TLS 1.2 for the direct VC++ runtime download --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
|
Daniel Han
|
767f2f36fb
|
Windows setup: route the stale-manifest failure through Exit-SetupFailure (#7569)
The manifest-removal guard added in #7492 exits with a bare 'exit 1', so in Tauri mode the installer never emits the [TAURI:ERROR] line and the desktop UI falls back to a generic failure instead of naming the cause. Every other failure path in studio/setup.ps1 goes through Exit-SetupFailure, and tests/sh/test_tauri_retry_failure_context.sh asserts that invariant, so 'Repo tests (CPU)' has been red on main since that merge. Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
Lee Jackson
|
d7594ec10f
|
Fix Windows no-torch setup (#7511)
* Fix Windows no-torch setup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix no-torch env normalization on Windows * Accept on for Windows no-torch mode * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep no-torch mode across studio update on Windows Guarding the direct torch/Triton install made `install.ps1 --no-torch` actually produce a torch-free venv, which then broke the next `unsloth studio update`. That path exports no UNSLOTH_NO_TORCH, so $NoTorchMode was false, the stale-venv check read the missing torch as a broken venv, and setup tried to delete the venv it was running out of: [ERROR] Could not remove stale venv: Access to the path 'python.exe' is denied. That teardown can never succeed there, because setup.ps1 runs via unsloth.exe out of that same venv. The same gap also let the shared dependency pass reinstall torch from PyPI, unpinned, into a GGUF-only environment. install_python_stack.py now records the mode in the install manifest and setup.ps1 reads it back when no env var is exported, then re-exports a canonical value for the dependency pass (setup.ps1 drops the manifest before invoking it, so the child cannot repeat the lookup). The key is additive and MANIFEST_SCHEMA is unchanged, so existing manifests stay valid and a missing key keeps today's behaviour. Also: - read_manifest() caught only OSError, but UnicodeDecodeError is a ValueError. That is now on the installer's import path, so a manifest re-saved as ANSI or truncated mid-write would abort every install. - The env predicate now trims surrounding whitespace, matching the Python side. - The Windows update smoke workflow asserts the update leaves the venv GGUF-only, which is what would have caught this. Known follow-up, pre-existing: an install killed between the manifest drop and the dependency pass leaves no recorded mode, so a later update still walks the stale-venv path. Closing that needs a marker the installer never drops. * Persist no-torch mode in a marker the dependency pass cannot drop The install manifest alone was not enough. Both setup.ps1 and install_python_stack.py remove it before every dependency pass, and it is only rewritten on success, so a no-torch install interrupted in between left nothing recording the mode. The next update then resolved no-torch as false, read the expected missing torch as a stale venv, and tried to delete the environment whose python.exe was running it, which leaves the install unrepairable from the CLI. Add .unsloth-no-torch next to the existing .unsloth-studio-owned marker, written before the pass and cleared when torch is wanted. setup.ps1 writes it as soon as the mode resolves, so the window between the manifest drop and its own torch install is covered too. Read order stays manifest key first, then marker, so migrating out of no-torch is never blocked by a marker an earlier run left behind. Neither present still reads as "install torch", so nothing changes for installs made before either existed. Also adds the AGPL-3.0 header the new test file was missing. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
|
Daniel Han
|
1781770bee
|
Studio: detect an interrupted dependency install instead of launching a backend that cannot import (#7492)
Some checks are pending
Unsloth GGUF CI / JSON, images (push) Waiting to run
Mac Studio UI CI / Chat UI Tests (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Windows Unsloth GGUF CI / Tool calling Tests (push) Waiting to run
Windows Unsloth GGUF CI / JSON, images (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / GPU prebuilt resolves without Visual Studio (push) Waiting to run
Unsloth GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio API CI / Unsloth API & Auth Tests (push) Waiting to run
Mac Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Mac Studio GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio GGUF CI / JSON, images (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-14) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio Update CI / Unsloth Updating Tests (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 unit tests (VS 2026 / CMake guard) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2026) (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-2025-vs2026) (push) Waiting to run
* Studio: detect an interrupted dependency install instead of launching a backend that cannot import An installer killed part-way leaves a venv with a working CLI but without studio.txt's dependencies. Nothing recorded that, so three separate places all reported it healthy: - the desktop preflight probed only `unsloth -h` (typer + rich) and a hardcoded desktop-capabilities dict, neither of which touches studio.backend, so it returned ManagedReady and spawned a backend that died on `import structlog`; - setup.sh's fast path compared the installed unsloth version against PyPI, which matches on a half-built venv because unsloth is installed early, so `unsloth studio update` printed "up to date" and repaired nothing; - start_managed_repair calls that update and then re-checks with the same blind probes, so Repair reported success without fixing anything. install_python_stack.py now clears a completion manifest before the dependency pass and writes it only after the final step. `unsloth studio verify-install` and desktop-capabilities' new studio_install_ok field read it, the preflight turns a false answer into ManagedStale so auto-repair runs, and setup.sh / setup.ps1 gain an escape hatch next to the existing anyio one. Separately, the wheel ships studio/ and studio.backend* but declared none of their dependencies, so `unsloth train`, `export`, `chat`, `inference` and `studio` all ended in a rich traceback after a plain pip install. structlog is the only hard module-level import that chain reaches once starlette's annotation-only import moves under TYPE_CHECKING, so it becomes a core dependency and the rest of the server stack becomes a [studio] extra mirroring studio.txt. The CLI import sites now report missing dependencies as a sentence with two remedies. Fixes #4701, #5260, #7147 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match the trimmed comments merged on the pip branch * Put the install manifest in the preflight fingerprint for PR #7492 The capability cache keyed the venv on pyvenv.cfg, uv.lock, requirements.txt, the interpreter and site-packages/unsloth_cli/commands/studio.py, none of which a repair touches when it only reinstalls studio.txt. So an entry cached while the install was healthy stayed valid after the manifest was dropped, and the probe returned Ready on exactly the half-built venv this is meant to catch. * Address the review findings on PR #7492 Fail the install when the completion manifest cannot be written, instead of exiting 0 without the record every later check requires, which is a repair loop by construction. Compare the version of the package the manifest names, so `studio update --package X` does not read as a permanent version change. Read the manifest from the venv that owns it when the CLI runs outside the managed venv, and drop the dependency verdict in that case: the walk ran against the wrong interpreter and says nothing about that venv. Name the import that actually failed. `unsloth train` reaches torch through the same guard, and the studio extra does not carry it, so recommending that extra alone left the command failing in the same place. * Declare click, which typer stopped providing, for PR #7492 unsloth_cli/commands/start.py imports click at module scope and unsloth_cli/__init__.py imports that module, so every unsloth command needs it. typer carried click through 0.19 and dropped it in 0.27, and the declared floor is typer>=0.12.0, so a fresh resolve gets no click. On the published wheel it still arrives because huggingface_hub requires click<9,>=8.4.2, which is luck rather than a declaration. A wheel built from this branch's dependency list has neither, and every command dies at import. Verified: before, `unsloth --help` on a fresh venv raised ModuleNotFoundError for click; after, it exits 0. The drift test now covers it. * Keep a running backend from the previous app version manageable The manageability bump gated two unrelated things through one constant. For the managed CLI probe 2 is right: a CLI reporting 1 cannot answer studio_install_ok. For a RUNNING backend it is wrong, because a process already started cannot change what it reports, so bumping studio/backend/main.py in lockstep does not help one the previous app version spawned. That backend is proven ours by root id and ownership token, but lifecycle_control_block_reason returned Unmanageable, and that branch never calls adopt_verified_backend. has_owned_backend() stays false, so Repair falls into block_external_conflict, which finds the same process and refuses: the app could no longer stop a backend it owns the token for. The same regression in backend.rs turned a terminal-launched same-root server from AttachedReady into ExternalConflict. Split the constant: DESKTOP_BACKEND_MANAGEABILITY_VERSION = 1 for the two live-backend probes, DESKTOP_MANAGEABILITY_VERSION = 2 for the CLI probe. Every real gate (protocol, auth, ownership, desktop-login, MIN_DESKTOP_BACKEND_VERSION) is untouched, so an old backend still reaches OwnedStale, adopt, stop, repair. Also stop the installer when the stale manifest cannot be removed. Windows raises on a read-only or locked file, and the pass would then run behind a marker that still names this version and these digests, so a run killed part-way would verify as complete. * Answer for the managed venv, not the one the CLI happens to run in The guard matched ModuleNotFoundError.name, an import name, against missing_requirements(), which returns distribution names. So a missing PyJWT printed 'pip install jwt', and jwt, docx and fitz are each a real but unrelated PyPI project (fitz is a neuroimaging workflow tool), so following the advice installed the wrong package and left the backend just as broken. Map the import to its distribution before deciding, and never offer the import itself. install_state() verified the caller's own prefix. The wheel ships studio/, so a CLI installed outside the managed venv always finds its own copy of the helper first, and a healthy managed install reported studio_install_incomplete with a missing list copied from the wrong venv. Selecting the root is not enough: _installed_version() reads the running interpreter and req_root defaults to the caller's studio.txt, so both checks still answered for the wrong venv. Hand verify_install() that venv's own metadata, enumerated through Distribution.discover(context = ...path), which does not fall back to sys.path. The candidate order is untouched, so shadowed-tree detection is unchanged. setup.ps1 replaces pip, torch and triton before install_python_stack.py runs, so the manifest it drops is not dropped before the first mutation. A run killed in between kept a marker that still verifies while torch was half-replaced; drop it at the top of the dependency pass instead. setup.sh is unaffected, the stack is the first thing its pass runs, and a test now pins both. pip uninstall rewrites nothing that was fingerprinted, and cache_matches re-reads the cached studio_install_ok rather than re-checking, so a venv that lost a studio.txt package kept being served the healthy verdict. Fold a sorted hash of the installed dist-info names into the marker hash. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * A missing manifest helper is a torn install, not an old one studio/install_manifest.py ships in the same wheel as _studio_deps.py, so nothing legitimately has one without the other: a CLI predating both never reaches this code, and the desktop already calls such a CLI stale on desktop_manageability_version. Returning ok=true there reported a healthy install for a tree the package update had half replaced, and the preflight then launched a backend whose own run.py could be just as absent. Report it incomplete so repair runs. * Tighten comments across the install-detection changes * Validate Studio dependency readiness --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com> |
||
|
oobabooga
|
01c856c6c5
|
Surface actionable installer failures in Studio desktop (#7529)
* Studio: surface actionable installer failures * Correct installer failure attribution * Preserve desktop installer failure context * Use explicit setup failure attribution * Preserve package manager failure details |
||
|
Nilay
|
ae6b96ba93
|
Studio: fail fast on out-of-disk instead of a doomed llama.cpp source build (#7420)
* guard llama.cpp prebuilt against out-of-disk instead of doomed source build * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * address review comments on out-of-disk guard * keep reusable installs and Windows parity in the out-of-disk guard * preserve the ENOSPC cause when re-raising fallback errors * catch out-of-disk before the attempt loop and accept all llama-server layouts * Fix out-of-disk detection gaps and false positives for PR #7420 Follow-ups found while testing the guard against a real ENOSPC (LD_PRELOAD shim returning errno 28 under a path prefix, real network, real release): - hydrate_source_tree retried the next mirror after an ENOSPC and only raised on the last URL. Both source fallbacks 404 for the published mix commit, so the reported cause was HTTP 404 and the run fell through to the source build exactly like before the guard. Stop at the first environment-fatal error. - The 5 GB preflight rejected hosts that install fine. A full CUDA install peaks at 0.87 GB, the largest published bundle is 0.77 GB and macOS is 0.01 GB, so at 3 GB free the install succeeded before and exited 4 after, with the source-build fallback suppressed too. It is now advisory, and a real ENOSPC still exits 4. This also drops the case where an install matching an older release plan was rejected before its reuse check. - ENOSPC raised inside shutil.copytree arrives as shutil.Error with errno None and no __cause__ or __context__, so it was never classified. That path covers the hydrated source tree, the runtime overlay and the activation fallback copy. - _causal_chain followed __context__ even when __suppress_context__ was set, so `raise ... from None` over an unrelated ENOSPC reported disk full and wrongly suppressed the source build. - TemporaryDirectory now ignores cleanup errors: an rmtree failure on the way out replaced the in-flight SystemExit and lost EXIT_NO_SPACE. - setup.sh skips the arm64 CPU last resort after exit 4; it re-ran the same disk-rejected installer and buried the hint under a second error dump. - The in-app updater turns exit 4 into a readable message instead of "installer exited 4" plus a log tail. Adds tests/studio/install/test_llama_prebuilt_no_space.py covering the classifier, the advisory warning and the exit codes. * Fix Python 3.9 breakage and Windows disk-full detection in the out-of-disk guard Found by running the guard across the whole supported interpreter range (requires-python is >=3.9,<3.15) and a spoofed [Linux, WSL, macOS, Windows] x [NVIDIA, AMD, CPU] host matrix. - TemporaryDirectory(ignore_cleanup_errors = True) is 3.10+, so the previous commit raised TypeError at install time on 3.9 and turned a working install into a hard failure. Replaced with a scratch_dir() contextmanager built on mkdtemp plus rmtree(ignore_errors = True), which behaves the same on every supported version. - getattr(exc, "winerror", None) crashed on 3.9. urllib's HTTPError is an OSError that proxies unknown attributes to a wrapped file object and raises KeyError, which getattr does not swallow, so any mirror 404 during an install would have blown up inside the classifier. Read it defensively instead. - Classify Windows disk-full by winerror as well as errno. CPython's PC/errmap.h maps ERROR_DISK_FULL (112) to ENOSPC but has no case for ERROR_HANDLE_DISK_FULL (39), which arrives as EINVAL, so a Windows os.replace() onto a full disk read as an ordinary failure and fell through to the source build. Tests cover both winerror codes, a non-disk winerror, and HTTPError alone and wrapped in a PrebuiltFallback. 116 simulation cases pass on 3.9 through 3.14. * Classify quota, flattened Windows and validate-install out-of-disk for PR #7420 - EDQUOT counts as out of space: a quota'd home has free blocks this user cannot have, so the source build is just as doomed. Reported separately so df does not mislead. Confirmed end to end with a real kernel EDQUOT: the installer went from 6 retries then a source build (exit 2) to exit 4. - Match the flattened Windows disk-full text. copytree stringifies each per-file OSError, and OSError.__str__ returns early on winerror, so the text reads [WinError 112] and never [Errno 28]. Captured on a real NTFS volume. Markers are bracketed so WinError 112 does not match WinError 1120. - --validate-install now exits 4 on a full disk. It caught PrebuiltFallback and exited 2 before the classifier ran, and setup.sh answered 2 by deleting the GPU build that had just succeeded and starting a CPU rebuild that needs more of the space that ran out. Both halves are needed: the call site only tested nonzero. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in the llama.cpp out-of-disk guard --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
Leo Borcherding
|
3ea6d14c39
|
AMD: CI coverage for recent fixes, plus three wrong gfx ids (#7431)
* ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite
Three merged ROCm fixes shipped without tests, and the CI wiring that
would have run them was gated on files the fixes do not touch.
Tests added (113):
tests/studio/install/test_rocm_arch_table_parity.py (27)
diffs the four duplicated gfx -> AMD pip-index tables across
install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py,
plus the GPU-name -> arch tables and the torch 2.11 pin allowlist.
tests/studio/install/test_rocm_native_linux_lib_dirs.py (26)
covers #7233: system-ROCm lib dirs prepended ahead of bundled
libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env
var, root resolution order, and source parity between the two copies.
studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46)
covers #7292: registration on the CUDA dispatch key, grouped and
ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 +
RDNA4 name gate, executed from the shipped source rather than a copy.
tests/studio/test_ci_shell_suite_coverage.py (14)
fails if either shell runner goes back to a hardcoded list or skips
a file without a recorded reason.
CI wiring:
studio-backend-ci.yml: add install.sh / install.ps1 to the path filter
(the suites it runs assert against those two files, so install-only
changes -- the shape most AMD/ROCm routing fixes take -- skipped it),
and replace the 13-file hardcoded shell list with directory
discovery. That list had fallen seven files behind, including
test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm
WSL reroute, which had never run on a PR.
tests/run_all.sh: same discovery loop so local and CI agree.
* Test review fixes: assert on outcomes, not on the code under test
Self-review of the previous commit found four tests that passed for the
wrong reason.
1. The arch-table parity test pinned expected gfx ids copied out of the
shipped tables, which enshrined three upstream inaccuracies as
correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is
gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's
ROCm compatibility matrix. The expectation is now the AMD pip index
leaf -- the thing the tables exist to produce, and what a wrong
answer costs the user. The three known drifts are listed explicitly
with a test asserting they stay cosmetic, i.e. that the wrong and
right ids still map to the same wheel index. That test turns red the
day one of them starts routing users to the wrong wheel.
2. The RDNA4 device-name test extracted the regex from worker.py and
then matched with it, so it could not fail. Widening the pattern --
the dangerous edit, since it forces the slow Python mm fallback onto
RDNA3 users -- would have been silently accepted. It now reads the
live pattern and checks it against fixed cases, plus asserts the
name match stays guarded by `not _lin_arch` and that the name is
lowercased before matching.
3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml,
so reindenting the step would fail the build while a real regression
to a hardcoded list could slip past a reformat. It now parses the
YAML, finds the step by name, and asserts on the glob plus the
absence of individual filenames. The path-filter test likewise reads
the parsed trigger instead of scanning raw text.
4. A set comprehension in the parity helper had a ternary whose branches
were identical.
Mutation-tested: widening the RDNA4 regex, desyncing one copy of the
name table, dropping install.sh from the path filter, and re-skipping
the ROCm WSL shell suite each fail at least two tests. Verified on
Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass.
* Fix three wrong gfx ids in the GPU-name arch tables
The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on
three entries. Corrected against the "Radeon GPU" list at
rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html:
RX 9070, RX 9070 GRE gfx1200 -> gfx1201 (Navi 48, same die as the XT)
RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101 (Navi 32, not Navi 31)
PRO W7700 gfx1100 -> gfx1101
PRO V710 gfx1102 -> gfx1101 (Navi 32, not Navi 33)
No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all
and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies
of the index-family map. That collapse is why the errors survived being
copied into six places -- the leaf-level tests could not see them.
It was not purely cosmetic, though. install.sh's second copy feeds
"Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the
printed advice exported gfx1100 and made a wrong id authoritative for
every later run. It would also have become a real misroute the moment AMD
split a family across index leaves, as they already do for gfx1151/gfx1150.
Fixed in all six places, which is two more than the table's own "kept in
sync with" comments claim exist:
install.sh _infer_amd_gfx_arch_from_gpu_name
install.sh case "$_gpu_disp_mkt" (banner + env tip; undocumented)
studio/setup.sh
install.ps1
studio/setup.ps1
studio/install_python_stack.py
Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell
copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the
PowerShell copies keep the (?!S) lookahead.
Test changes:
- test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx
ids transcribed from AMD rather than from the tables. Agreement between
six copies proves nothing when all six were transcribed from the same
mistake, so the ground truth has to come from outside. Verified it
catches the bug: against the pre-fix tables it fails 6 tests.
- The parity check now covers all six copies. It had four; the two
install.sh copies were being treated as one, and
_WIN_GPU_NAME_ARCH_TABLE was not checked at all.
- test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong
ids as expected values; updated, and extended with a 9060 XT and a
7900 XTX case so each RDNA3/4 die is represented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard against unregistered copies of the GPU-name arch table
Counting the copies by hand is what let them drift: the in-code "kept in
sync with" comments claimed four, the arch-id fix found six, and scanning
the tree turns up a seventh.
TestNoUnregisteredArchTable rediscovers the copies from the source tree
instead of trusting a hand-maintained list. A table line is one that names
a card and gives its arch; real tables score 9-17 such lines and the only
other hits in the repo are two single-line prose comments, so the
three-line threshold is not load-bearing. A companion test asserts the
scan still finds the known copies, so the heuristic cannot go blind and
pass by finding nothing.
The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests
build their fake AMD host from. It states the mapping backwards (gfx ->
the name torch should report), which makes it an independent witness: it
had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six
installer copies were wrong, and nothing compared the two. Now they are
round-tripped against each other.
RX 6700 XT is pinned as a known divergence rather than normalised. AMD's
compatibility matrix documents no consumer RX 6000 card and no gfx1031 at
all, the installer arm is commented "gfx103X family", and gfx1031 appears
only as an index-family key, never as a value a name table emits. With no
external source to correct against, changing shipped behaviour would be
guesswork. A test fails if the divergence ever disappears, so the
exemption cannot go stale.
Also adds the reverse of the AMD-matrix check: a documented card that
matches no arm anywhere is a silent CPU fallback rather than a wrong id.
This cannot detect hardware nobody transcribed, which would need a live
fetch of AMD's matrix and a non-hermetic suite; the docstring says so
rather than implying coverage that is not there.
Verified on Linux: 478 passed, plus all five new guards mutation-tested
to confirm each fails when its invariant is broken.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Docstring said six copies; the list under it now has seven
* tests: run discovered shell tests with bash, not sh
tests/run_all.sh discovered tests/sh/ instead of listing files, but still
invoked each one with sh. Every file there declares a bash shebang, and on
Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh,
test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh
fail on bashisms under dash and pass under bash. The old hand-written list
happened to name only dash-clean files, so switching to discovery is what
surfaced it. Backend CI already used bash, so this was a local-only break.
Guarded by a new test asserting both runners invoke tests/sh/ with bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index
The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU
strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table
(src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152.
Unlike the three ids already fixed here, this one is not wheel-neutral:
repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with
separately built torch wheels, so these laptops were installing wheels
built for a different LLVM target. gfx1152 was absent from the codebase
entirely, so it needed the index-family maps, the torch 2.11 floor lists
(same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the
Windows arch allowlist as well as the seven name tables.
The parity test added in this PR did not catch it because its AMD-matrix
expectations stopped at 890M/880M. Added the APU rows, so the case that
actually changes a wheel is now covered: reverting the tables fails 9
tests naming 860M, 840M and Krackan.
gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153
wheel family, so there is nothing to route it to.
Verified: bash -n on both shell installers, PowerShell AST parse on both
.ps1 files, python ast.parse on all touched modules, install suite 1334
passed with no new failures against main, shell suite 20 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gfx1152 to unified-memory classifiers, make parity allowlist set-based
Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and
shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and
Strix Halo (gfx1151), but only the installers knew about it. The two
runtime classifiers still had two-element arch sets, so a Krackan laptop
got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp
without GGML_CUDA_ENABLE_UNIFIED_MEMORY.
- worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set,
and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M
cannot collide there: the function is only reached under _hw.IS_ROCM.
- llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set.
- Tests for both, including the :sramecc-:xnack- suffix form.
TestGfx211AllowlistParity compared four hardcoded allowlist strings, so
adding gfx1152 to all four installers correctly turned three assertions
red without any installer actually disagreeing with another. Each test
now extracts the set its installer holds and compares it to one EXPECTED
constant. Order and spacing are free, membership is not, and the next
leaf is a one-line edit instead of four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
|
||
|
Michael Han
|
d5cf96d628
|
Studio: add local speech-to-text dictation engine (#7095)
* Studio: add Voice settings tab (dictation, dictionary, read aloud) New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR. * Studio: drop the single option STT engine select, rename TTS option The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer. * Studio: harden Voice settings against edge cases found in simulation Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery. * Studio: address Voice settings review feedback Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior. * Studio: use the chat mic icon in Voice settings for consistency The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere. * Studio: address second round of Voice settings review feedback Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race. * Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item * Studio: guard dictation mic lifecycle in Voice test and Compare composer Release a microphone opened after the component unmounts, and stop Compare dictation on a permission or security failure instead of silently recording from the default device, matching the main chat adapter. * Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings - Join final dictation chunks with a space so recorded transcripts do not merge words - Ignore a stale recognizer onend so a quick stop then restart is not torn down - Use previewingRef so a double click on TTS preview does not orphan the first request - Keep the read-aloud stop control visible when a new run starts while a message is spoken - Stop the dictionary remove button from deleting an adjacent entry on a blur then click race * Studio: trim redundant Voice settings comments * Studio: fix Voice preview and Compare dictation edge cases - Only cancel the shared speechSynthesis for a system-voice preview, so stopping a Studio preview no longer stops an unrelated chat read-aloud - Release the Studio preview audio and its WAV data URL on normal completion - Iterate every finalized result in Compare dictation so batched phrases are kept - Cap persisted recent dictations to the last 20 on hydration * Studio: use clipboard fallback for recents and release failed preview audio - Copy recent dictations via the copyToClipboard helper so the execCommand fallback works in Safari and insecure http LAN contexts - Release the Studio preview audio when play() rejects, not just on ended/error * Studio: add local speech-to-text dictation engine Add an offline dictation engine that transcribes with a local faster-whisper model, alongside the existing browser (Web Speech) engine. The browser engine streams audio to Apple or Google speech services and needs internet; the new engine runs on the server, works offline, and drives any chat model without evicting it (it loads in the backend process, separate from the model subprocess). It also gives Firefox dictation, which has no Web Speech support. Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper is torch-free, so this does not disturb the existing model stack. Frontend: a Dictation engine setting (browser or local model), a curated model picker with sizes, and MediaRecorder capture posted to the transcribe route. The model warms automatically when the engine is selected, with live status. * Studio: stream local STT transcription as you speak Local dictation showed nothing until you stopped, because the whole clip was transcribed once on stop. Now the growing recording is re-transcribed on a fast pass every second and emitted as live interim text, with an accurate final pass on stop. Partial recordings decode fine, and the model refines earlier words as more audio arrives. Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast preview pass; the final stop uses the accurate path. * Studio: make local dictation stop instant and reliable Stopping local dictation waited for a final network transcription before the session ended, so the stop button did not flip and a second click ended the session early and dropped the text. Now stop commits the live transcript immediately, releases the mic at once, and ignores a second stop while finalizing. Previews run more often so the committed text is current. * Studio: record local dictation in short clips for reliable streaming Re-transcribing a growing buffer every second got slower as it grew, flooded the backend, showed stale words, and could leave the stop button stuck waiting on a backlog. Record short independent clips instead and transcribe each once, appending the text as you speak. Work per clip is bounded, so stopping is prompt (with a hard timeout as a safety net) and long dictations stay smooth. * Studio: dictate then transcribe once on stop, ChatGPT style Local STT dictation streamed by re-transcribing the growing clip, which was quadratic and saturated the backend (multi-second lag), and stop only halted the recorder without releasing the mic, so it kept recording. Record the microphone continuously, release it the instant the user stops, and transcribe the whole clip once. Stopping is immediate and the transcript lands in about a second. Also add the tiny model for the fastest option. * Studio: surface dictation and read-aloud failures instead of failing silently - Compare dictation reports microphone and speech-recognition errors via toast, reusing the main chat adapter's describeMediaError and describeSpeechError - Read-aloud toasts genuine model or synthesis failures while ignoring cancellations * Studio: ChatGPT-style recording bar for dictation Clicking the mic now drops the composer into a dedicated recording bar with a live waveform, a discard (X) and a confirm (tick), instead of a plain stop button. The tick stops recording and transcribes the clip; the X throws the recording away and keeps whatever text was already in the composer. The model adapter taps the mic with an analyser to drive the waveform, and the router tracks the live session so the X can cancel it without transcribing. * Studio: transcribe dictation while speaking, ChatGPT layout Match ChatGPT's recording layout: the bar now renders in place of the input with the left plus button kept, the waveform in the middle, and the discard and confirm buttons together on the right. Cut the post-confirm delay by transcribing in the background as the user talks. The audio is split at natural pauses (voice-activity detection off the same analyser that drives the waveform) and each clip is transcribed as it is cut, so confirming only has to finish the short final tail. The model is also warmed when recording starts so the first run never pays a cold load. * Studio: ChatGPT waveform, hide tools while dictating, faster STT Make the recording UI read like ChatGPT: the waveform is now a dense row of round dots that rise into thin centered bars, and while dictating only the plus button shows, with the mode badge and tool toggles hidden so the bar is just the waveform and controls. Speed up transcription: decode greedily (beam_size=1), which is several times faster on CPU with negligible accuracy loss on short dictation clips, and cap background segments at 6s so the final tail after confirm stays short. * Studio: finish ChatGPT voice bar and low-latency STT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: full-width waveform with a timer that freezes on stop Use the full-width waveform for the recording bar: brighter, bigger bars that advance on a fixed cadence (keeping peaks between advances) so they glide instead of racing by, inset from the composer edges. Keep a visible timer and the green confirm button, matching the ChatGPT reference, and freeze the timer and waveform the moment the user confirms. * Studio: fix multilingual local dictation * Studio: speed up dictation and release local STT * Studio: harden dictation finalization and STT decoding * Studio: restore Firefox dictation fallback * Studio: add dictation history manager * Studio: manage speech model downloads * Studio: remove em dash from voice model label * Studio: move dictation history into Voice * Studio: source local STT from Unsloth Whisper models Point the dictation STT sidecar and its Model Hub download entries at Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3) and run them through Transformers, so Studio only ever downloads Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs repos; keep the Model Hub as the only download path via local_files_only, and keep PyAV for audio decoding. Device selection uses float16 on CUDA and float32 on MPS and CPU, since Whisper's decoder is unstable in float16 on MPS and repeats tokens. Shorten the model picker labels to name plus download size and update the STT tests for the new backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: smooth dictation waveform and keep pill height * Studio: align STT model dropdown width and tidy voice copy * Studio: guide to local engine when browser dictation is offline * Studio: clarify voice section and STT model copy * Studio: keep STT warm with training-aware eviction * Harden STT lifecycle and browser compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model discovery test lint * Harden cross-browser microphone errors * Harden cross-browser microphone errors * Surface voice test recognition errors and fall back to Studio TTS - Voice test now toasts non-abort speech-recognition failures instead of ending silently, matching the main and Compare dictation paths. - Read-aloud routes to the backend model when the runtime lacks Web Speech synthesis (audio-only WebView), so it no longer errors immediately. * Fix reviewed STT lifecycle races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix read-aloud fallback controls * Guard read-aloud stop when deleting a non-speaking message aui.message().stopSpeaking() throws unless this message is the one being read aloud, so calling it unconditionally rejected the delete handler before the message was removed. Only stop speech when this message is speaking. * Cap recent dictation transcript length before persisting Recent dictations only limited entry count, so a long transcript stored the full text in the persisted voice settings and a few could exceed the localStorage quota, throwing synchronously from the uncaught dictation cleanup path. Truncate each entry on save and on hydration, matching the dictionary cap. * Studio: keep dictation mic clickable and guide to local model Register the dictation adapter unconditionally so the mic stays enabled for any engine and starts working right after switching to the local model on an already-open thread. When the browser engine cannot run (Firefox, Brave, non-secure origins), clicking the mic shows a toast that points to the local speech-to-text model instead of leaving a disabled button. The toast stacks its action below the text with a fully rounded button. * Studio: add bottom padding below the dictation guidance toast button * Studio: increase bottom padding under the dictation toast button * Studio: add bottom padding inside the dictation toast button * Studio: add five Whisper defaults and custom model search Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end. Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary. Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use public Unsloth Whisper repositories Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests. * Studio: update Whisper download sizes Reflect the cleaned public Tiny and Base repositories in the curated model labels. * Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes - Show the download size on the right of each model row so long names like Whisper Large v3 Turbo no longer hide it - Update curated Whisper sizes to the safetensors weights actually downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB - Drive the model list scroll from a wheel handler so the mouse wheel scrolls it inside the Settings dialog, not just the scrollbar - Add a search icon and shorten the placeholder to Search model * Studio: do not search when a dictation model is picked, shrink repo label - Treat the filled-in model text as a selection, not a query, so choosing a model no longer kicks off a Hugging Face search - Make the repository line under each model name smaller * Studio: tighten dictation model and local engine descriptions * Studio: keep model display on pick instead of the query, shrink row text - Guard the combobox input so selecting a model shows its name and does not echo the typed query back or start a search - Map the item label to the friendly display so picks fill the field - Reduce the model name and size text in each row * Studio: show only the model name in the dictation field, shrink size label - Drop the download size from the search field; the name alone is shown once a model is selected, with sizes kept in the dropdown list - Reduce the size label text in each row * Studio: clarify the dictation model description * Studio: drop Hugging Face from the dictation model description * Studio: move the dictation dictionary to its own Manage subpage - Replace the inline entry list with a Manage row, matching Dictation history, so a long dictionary no longer crowds Voice settings - Add a DictationDictionaryView subpage that holds the entry editor * Studio: match STT field font, use best voice for System default - Bump the dictation model field text to text-sm so it matches the engine dropdown next to it - Resolve the System default read-aloud voice to the top curated voice instead of the browser default, which is a robotic legacy voice on macOS * Studio: rerank read-aloud voices and drop duplicate voice entries - Rank by vendor quality, then the user's locale, then a preferred list of natural voices, so the best voice leads instead of the first alphabetically - Collapse voices that macOS reports twice under one name and language * Studio: fold dictionary and recents into the dictation section - Drop the separate Dictation dictionary and Recent dictations headings; their Manage rows now sit under Dictation, split by the row divider - Shorten the custom spellings description * Studio: add search and sort to dictation history - Filter saved dictations by text with a search field - Sort by newest, oldest, or A to Z; show a no-matches message - Keep Clear all available regardless of the current filter * Studio: settle cancelled STT loads before training and fix dictation review items Wait for a cancelled STT load to exit and release its memory before reporting it freed for training, so the loader cannot still be inside from_pretrained()/.to(device) holding VRAM when the training subprocess starts. A load that finishes before observing the cancel now gets unloaded so the memory is actually reclaimed. Clear the accelerator cache before the CPU fallback in load() so a failed CUDA/MPS load does not strand reserved VRAM once the sidecar is marked CPU-resident. Send the saved Hugging Face token when polling STT download progress so a gated or private repo resolves and shows the correct Load/Downloaded state instead of reporting missing. Mark the composer Dictate button as type="button" so clicking it does not also submit the draft when the composer already has text or attachments. * Studio: pin dictation settings per session and close STT startup races Capture the STT model and language when a dictation session starts and pass them to every queued segment and the warm-up load, so changing the model or language mid-recording no longer transcribes the same clip with the wrong model or a model that is not downloaded. Check the local runtime at the top of transcribe(), before the model cache lookup and the bounded audio decode, so a server missing PyTorch or Transformers returns 501 up front instead of decoding a long clip first. Treat the training startup window as active for STT device selection. start_training frees VRAM in before_spawn but only assigns _proc later, so a concurrent STT load could take the GPU that was just cleared. A startup flag now reports training active from the free until the process is live, forcing those loads to CPU; a finally clears it on every exit. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stub the STT runtime check in transcribe orchestration tests transcribe() now verifies the local runtime up front, so the unit tests that exercise transcription orchestration must treat the runtime as present to keep passing where PyTorch, Transformers, and PyAV are not installed. Stub ensure_stt_available in the shared fixture and restore the real check in the availability and load-rejection tests. * Harden custom Whisper dictation models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add whisper.cpp dictation engine with per-engine downloads and history rework Engines - New GGML STT sidecar that runs a managed whisper-server subprocess with idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh) - Dictation engine picker now offers Browser, Local transcription (whisper.cpp), and Local transcription (Transformers) - Both local engines serve the same five curated Whisper models and download them directly with byte-level progress reported by /audio/stt/status - Models auto load on selection and when their download finishes - Unload and training admission account for both engines Benchmarks (Apple Silicon, greedy, warm, same checkpoints) - whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in about 0.45s vs 0.86s for Whisper Small - whisper.cpp GGUF path is unchanged by the Transformers addition (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s) Voice settings UI - Plain curated model select replaces the searchable combobox - Single download progress bar with transfer rate for both engines - Dictation history now stores every dictation with Show more pagination, a top Clear history action, and links back to the chat it was spoken into - Archived chats dialog gets the same pagination - Delete dialog offers deleting a dictation together with its chat Tests: 88 backend STT tests pass, including new snapshot download coverage. Frontend typecheck, lint, i18n parity, and production build pass. * Merge local engines into one option and source GGML models from unslothai Engine selection - The dictation engine dropdown is back to two choices: Browser and Local transcription. The selected model decides the backend: curated ids run GGML checkpoints through whisper.cpp, searched Hugging Face repositories run safetensors through Transformers - Model picker lists the curated models and searches Hugging Face for other Whisper repositories, validating them before selection. The trigger is a plain button so the selection never renders inside a text input - /audio/stt/status accepts a model query param so downloaded state works for custom repositories; the engine param on load, transcribe, and download routes is derived from the model everywhere Model source - Curated GGML checkpoints now download from the Unsloth-hosted unslothai/whisper-*-GGUF repositories (one repo per model) instead of ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob tracking are per-model Fixes - Voice settings and dictation history were not persisting: the quota-safe localStorage wrapper was declared after the store that uses it, so the persist storage factory failed silently. Every settings write also threw mid-click, which kept the model picker popover from closing on selection - is_model_downloaded now verifies config, preprocessor config, and real weight files instead of trusting an offline snapshot lookup, so a partial download left by an aborted fetch shows the Download button instead of failing to load - Removed whisper.cpp mentions from user-facing text: the ready status shows Loaded instead of the runtime name, picker rows show the source repository, and runtime error messages say local transcription runtime Verified with automated browser sessions and live API checks: selection closes the picker with no page errors, persisted settings hydrate on reload, a stale partial snapshot triggers download then loads on MPS and transcribes, and curated models download from the unslothai repos. 88 backend STT tests, typecheck, lint, i18n parity, and build pass. * Skip the duplicate source line for custom models in the STT picker A custom repository's display name is its id, so search results and the appended current selection rendered the same string twice. The source line now only renders when it differs from the name; curated rows keep their name, unslothai source repository, and download size. * Verify every shard of a sharded checkpoint in the downloaded check A snapshot holding one of N shards (or a corrupt shard index) passed the downloaded check and then failed at load. When model.safetensors.index.json exists, every shard in its weight map must now be present. Found by simulation; covered by a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename stale _starting references in the pump resilience tests The startup flag on TrainingBackend was renamed to _spawn_in_progress but two tests added alongside it still asserted on the old name, failing the Python 3.11 to 3.13 CI jobs. * Make the selected model row clearly highlighted in the STT picker The current selection was a faint background tint. It now uses the accent background with a medium weight name. Two line rows use a small corner radius; single line custom repo rows keep the pill shape. * Address review feedback on STT snapshot checks, VRAM release, and dictation UX Verify snapshot completeness in the load preflight so a partial download fails before the audio is decoded, for curated and custom repos alike. Drop the failed accelerator traceback before the CPU retry so the cache clear can actually release that memory. Keep unloading the GGUF sidecar after cancelling an in-flight Transformers load; both engines can hold memory at once. Allow Auto language with English-only .en checkpoints, matching the backend which sends no forced language. Keep the discard button usable while a transcription is pending so a slow or hung request cannot trap the composer in dictation mode. Stop linking Compare and settings test dictations to the unrelated active single chat thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move the CPU retry out of the exception handler On Python 3.10 the interpreter exception state keeps its own reference to the traceback, so dropping it from the caught exception was not enough to release the failed accelerator load during the retry. Leaving the handler before clearing the cache works on every supported version. * Address review feedback on session handoff, chat pinning, and server lifetime Starting a dictation from a second entry point now cancels the session it replaces, so the old recording cannot keep the microphone open or save a transcript with no discard button pointing at it. The linked chat is pinned when recording starts, so switching threads while a transcription finalizes cannot relink the transcript to the newly opened chat. whisper-server is now bound to Studio's lifetime like the other long-lived children: PDEATHSIG on Linux, the parent job object on Windows, and pid adoption so the shutdown sweep reaps it; before this it survived a Ctrl+C exit as an orphan still holding the model. * Remove the dictation mic test from Voice settings The composer dictate button covers the same check, so the test row, its transcript panel, the unsupported fallback row, and their strings and search entry are gone. * Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits GGUF (whisper.cpp) sidecar: - Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed. - Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind. - Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription. - Reject a missing model before decoding audio, matching the Transformers download preflight. Voice settings: - The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it. Dictation dictionary: - Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fix curated GGUF whisper filenames to match hosted repos The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin, not ggml-<id>.bin, so every curated dictation download and cached-path lookup 404'd and the whisper.cpp engine could never load a model. Point GGML_STT_MODELS at the real filenames and guard the naming with a test. * Studio STT: validate a custom dictation repo before downloading it The Transformers STT engine accepts an arbitrary owner/model repo, but the download route handed it straight to snapshot_download, pulling a possibly large non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper checkpoint first with the existing metadata-only validate_remote_model (no weights); curated ids short-circuit and the GGUF engine (curated-only) is unaffected. A non-Whisper repo now 422s before any download. * Studio STT: preempt a still-loading GGUF server for training admission A whisper-server still in its startup window binds accelerator memory but has no loaded_model yet, so training admission could miss it and launch into an OOM. Make the GGUF startup cancellable (cancel_pending_load signals an abort event and terminates the starting process without the load lock; _wait_for_server observes it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock until the killed server is reaped), and always fold the GGUF sidecar into the resident-STT summary so a resident Transformers model cannot mask a loading GGUF server. free_stt_model_for_training now cancels an in-flight load and waits for it to settle before training claims the memory. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fall back to Transformers when whisper-server is absent A curated dictation model (including the default small) hard-pinned the GGUF engine, but standard installs do not ship whisper-server, so every recording 501'd instead of using the Transformers engine that serves the same checkpoint -- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine: a GGUF request for a curated id (the only ids GGUF accepts, all Transformers- servable) downgrades to Transformers when whisper-server is unavailable, applied consistently to download, load and transcribe (not unload, which targets a specific engine). The Voice tab likewise falls back to the Transformers status so the model is not shown unavailable and download is not blocked. * Studio STT: hide custom Whisper caches from the legacy model pickers The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with only the owner/model id, which cannot reach the config-based Whisper check, so a downloaded custom (non-curated) Whisper checkpoint was still offered as a chat model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo config and hides it, matching the discovery route. * Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction - Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat model inventory and pickers, backend and frontend. Only their Transformers safetensors companions were hidden; the GGUF repos use a different org and a -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked into chat pickers. - Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the Transformers sidecar. transcribe() holds self._lock across the whole inference call, so /audio/stt status polls and training admission previously blocked behind an in-flight transcription. - stt_unload resolves through the serving resolver: a "gguf" pick on a host without whisper-server is served by the Transformers fallback, so unload must target that engine or the resident model is never freed. Unload also attempts every engine even if one raises, so a failure freeing one backend no longer skips the other. - free_stt_model_for_training frees the Transformers and GGUF sidecars under independent exception boundaries so a failure unloading one no longer skips the other before training claims the memory. Adds tests/test_stt_review_fixes.py covering all four. * Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness - The model dictation adapter sent the raw setting (the literal "auto") to the backend, while the browser engine resolves Auto via resolveDictationLanguage. A batch of non-English voice notes came back mostly English on Auto. Add resolveModelDictationLanguage: only the literal "auto" is resolved to a concrete locale, gated so it becomes a language the model AND Whisper can honor (mirroring the backend's known-whisper-languages set); an explicit language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire it into both adapter call sites. - GgmlSttSidecar._process_alive() read self._process twice; a concurrent unload() nulls it under the lock while loaded_model/device read lock-free, so a null between the two reads called None.poll(). Snapshot once. Adds a deterministic regression test. * studio: tighten comments and docstrings in the dictation modules * studio: harden dictation model downloads, GGML readiness, and recording paths Address review findings on the STT dictation feature: - build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom Studio home unless it carries the Studio ownership marker, matching the setup.sh policy, and marks trees it creates - _snapshot_is_complete validates every shard of a sharded PyTorch (pytorch_model.bin.index.json) checkpoint like the safetensors path, and requires tokenizer assets (tokenizer.json or vocab.json + merges.txt) - custom-repo downloads pin the revision resolved at validation time and restrict snapshot_download to the model/tokenizer/config/preprocessor file classes Studio loads - the GGML sidecar holds its port reservation until just before spawning whisper-server and only accepts readiness from a responder that both looks like whisper.cpp's server and belongs to the still-running managed child, probing twice, so mic audio cannot be posted to a foreign local process - the recording adapter transcribes every non-empty segment; the RMS meter only shapes segment boundaries and can no longer discard quiet speech - Compare-pane dictation can cancel a pending transcription on second click, with the button relabeled while finalizing - localStorage quota recovery halves the dictation history until the save fits, so small histories shrink too - the System default TTS voice resolves to the platform default voice - new dictation UI imports go through the chat and hub feature barrels Regression tests cover the build-script gate, sharded PyTorch and tokenizer completeness, revision pinning and allow patterns, and the whisper-server readiness probe. * Fix STT download and voice picker follow-ups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add dictation button regression coverage * Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294) * Studio STT: add prebuilt whisper.cpp (whisper-server) installer New install_whisper_prebuilt.py downloads a per-platform whisper-server bundle published by the unslothai/whisper.cpp prebuilt CI into the managed whisper.cpp dir (build/bin/whisper-server) so local dictation needs no compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py: host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the trust anchor, staging + install lock + atomic swap, traversal-safe extract, co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired into setup yet; the pins ship empty so every asset fails closed until the first fork release is published and its digests are reviewed in. * Studio STT: install prebuilt whisper.cpp during setup and update Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so `unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server into the managed whisper.cpp dir the sidecar discovers. It skips a user-set WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL, forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence. * Studio STT: harden whisper-server child env + WSL ROCm detection - Sidecar spawns whisper-server with a scrubbed child env that prepends the binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child. - find_whisper_server_binary now requires an executable, not just a file. - Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only; gfx parsing skips the gfx000 CPU agent and generic ISA lines. - Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the executable check, and the WSL rocm detection. * Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can detect and install a newer whisper-server release from inside the app: - backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json and compare the installed release against the newest unslothai/whisper.cpp release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a (major, minor, patch, serial) key with a strict downgrade guard; 24h cache; fail-open. - backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch and atomically swap the newest bundle, unloading the warm GGUF sidecar first. - backend/routes/whisper.py mounted at /api/whisper (update-status + update). - pyproject: add whisper_prebuilt_pins.json to studio package-data so the installer's trust anchor ships in the wheel (it is a data file, not a .py module, so package discovery alone does not include it; node_prebuilt_pins.json is listed for the same reason). Without this a pip-installed wheel had no pins and the prebuilt install aborted to Transformers STT. Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade guard, marker layouts, stale decision, fail-open). * Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust model: instead of a committed whisper_prebuilt_pins.json, verify every download against the release's own whisper-prebuilt-sha256.json checksum index, fetched from the same GitHub release. - parse_release_checksums / fetch_release_checksums / expected_sha256_for replace the pins layer. The index is validated for schema/component and that its release_tag matches the resolved release; an asset absent from it, a release that does not publish it, or a manifest sha256 that disagrees with it all fail closed to a source build. - resolve_release_tag now resolves the newest published release at runtime (or an explicit --published-release-tag), matching llama and the freshness check; removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in. - Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data entry (nothing to ship now, same as llama which has no committed pins). - Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on uncovered asset, tampered-manifest guard, newest-release resolution). This is a same-origin checksum (integrity, not authenticity), identical to the llama.cpp installer; pair releases with GitHub artifact attestations for provenance. * Resolve whisper prebuilt release via the download host (no GitHub API) Mirror install_llama_prebuilt.py's fast path: resolve the release tag from the releases/latest redirect and fetch the manifest + checksum index from constructed releases/download URLs, so the common install path makes zero api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour per IP; the download host is not). Fall back to the GitHub API only on a 404, malformed asset, or tag mismatch. * Studio STT: coverage-aware whisper prebuilt selection via a shared core whisper's select_artifact returned the first os/arch/backend manifest match and ignored the SM-coverage fields the release manifest already carries, so a Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks cuda13-newer. Extract the coverage-aware selection into a shared, component-agnostic core under studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and generalised over a normalised artifact. whisper's HostInfo now records the GPU compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and select_artifact routes CUDA/ROCm through the shared selector: every visible SM must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes, and "already matches" contract are unchanged. On the B200 the installer now resolves cuda13-newer, matching llama. * Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship libcudart/libcublas -- they load the same runtime the host already has. So the driver's advertised CUDA version is only an upper bound: a cuda13 bundle still needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the shared core and intersect it with the driver-compatible lines in select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g. torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13 one; a host with no CUDA runtime at all falls back to CPU. Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator truthiness, not a match) that made every major report present; add a real filesystem test that exercises the scan. * studio: harden shared prebuilt core to full llama parity Apply the review findings on the shared coverage-aware prebuilt-consumer core so whisper.cpp selection is exactly equivalent to the llama.cpp path. hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an index/UUID selector now reports has_usable_nvidia False instead of staying usable, via supports_explicit_visible_device_matching plus the physical / explicit-match branches, and _select_visible_rows now matches rows the way llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus fallback and has_physical_nvidia. Adds parse_macos_version. runtime_libs.py: the Linux on-disk scan now requires the exact libcudart / libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare versioned file without the SONAME symlink no longer counts as loadable. Hardens the ldconfig parse against an empty left-hand side. selection.py: fix the Blackwell/torch reordering so it keys on the covering runtime lines (falls through to the torch preference when the covering lines were filtered out), matching linux_cuda_choice_from_release. Corrects the compatible_runtime_lines_for_driver docstring: the bundles do not ship the CUDA runtime, so the driver version is only an upper bound and the caller must intersect with the on-disk scan. install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new HostInfo.macos_version) so a bundle that cannot load on the host OS version is dropped. Keep resolver stdout to only the JSON line by leaving logs on stderr in --resolve-prebuilt mode, and map an unexpected probe failure to prebuilt_available False instead of a traceback. Tests: new host-probe suite for the visible-device logic, exact-SONAME runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON, exit-code mapping, and the repo key. * studio: fix whisper prebuilt selection + launch parity gaps from review A parallel review surfaced integration defects where the whisper path could select or launch a bundle that cannot run on a concrete host. Each is fixed to match install_llama_prebuilt.py. macOS min_os: the manifest labels macOS requirements as macos-<version> (e.g. macos-14.0), which the version parser could not read, so the guard was a no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the platform prefix before parsing. ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact ROCm matching treats that token as the active GPU, a mixed APU + dGPU host (gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU sections and honors the visibility vars (empty / -1 -> no AMD GPU). --rocm-gfx override: recording the arch without setting has_rocm left the host on its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies has_rocm and clears NVIDIA state, like llama's _apply_host_overrides. CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so on a host whose CUDA runtime lives only in the PyTorch wheels the selection would gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch runtime dirs to the child loader path for CUDA bundles (bundle dir still first), mirroring binary_env. Also normalize a manifest artifact's supported_sms defensively (parity with llama's parser) and document that blackwell_min_toolkit_for_caps is retained for the Phase B llama Windows path. Not changed (verified parity, not defects): Linux/Windows min_os is enforced nowhere in llama (macOS only); the resolver is optimistic about the checksum index and the install path verifies. * studio: tighten prebuilt-core code comments * studio: lift shared prebuilt installer core out of the whisper installer * studio: reuse the llama.cpp prebuilt installer machinery for whisper * studio: unify llama and whisper prebuilt installers on a shared descriptor core * studio: consolidate prebuilt installer tests into the shared core suite Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every component-agnostic behavior runs against both descriptors: the full seven profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle stability, missing SM metadata, dotted SM normalization, no-driver fallback policy), the ROCm gfx family matrix, macOS min_os gating and its helper, backend resolution incl. cpu-fallback precedence and Intel-mac auto detect, checksum-index non-object and plain-lookup cases, the tar symlink/hardlink extraction guards moved from the llama suite, and the compute-cap, visible device, runtime-line and Blackwell helper value tables moved verbatim from the llama characterization suites. Delete only tests whose exact behavior the master now asserts for the same component: 40 pure-alias helper cases in test_selection_logic.py (replaced by value-identical master tables plus an alias-identity pin), 6 extraction moves and the master-absorbed zip-symlink case in the llama logic suite, 3 routing twins in test_rocm_support.py already pinned byte-for-byte in test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by the master whisper parameterization. Wrapper wiring pins, the llama release plan dialect, fingerprints and every llama-only behavior stay untouched. * studio: dedupe sidecar and update helpers into the backend prebuilt package * studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow * studio: consume paired slim whisper prebuilts via the llama ggml runtime * studio: serve every whisper backend from slim prebuilts * studio: drop the whisper fat per-accelerator selection chain unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan selection glue; keep slim selection + pairing, link_ggml_runtime, and one legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim release. Exit 2 now reads as prebuilt unavailable (whisper never source builds); setup already treats it that way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire libomp runtime DLL alongside ggml in slim whisper installs llama's clang-built windows-arm64 ggml-base.dll imports libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL. Without it next to whisper-server.exe the loader fails with STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64 was affected. The empty-runtime guard still requires a real ggml library; libomp alone is not a pairing. * studio: drop whisper-side fat-selection support structure Slim whisper bundles are selected per os/arch only; all accelerator capability comes from the installed llama.cpp prebuilt, whose installer already did the coverage-aware selection. Remove the machinery that only existed to pick among fat per-accelerator whisper bundles: - prebuilt_core: delete the generic CUDA/ROCm coverage selection (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters, detected_cuda_runtime_lines, the exact-SONAME linux probe) that no shipped component routes through; llama keeps its own selection chain and whisper shadows select_artifact with the slim-only version. select_artifact is now a plain os/arch/backend first-match. - install_whisper_prebuilt: drop the HostInfo CUDA fields (compute_caps, driver_cuda_version, torch_runtime_line) and the torch runtime probe that populated them; nothing reachable reads them, and the resolver payload sources runtime_line from the artifact. - whisper_cpp_update: delete the standalone start_update job worker; whisper applies only run as the chained phase of the combined llama+whisper update. The status payload keeps its job field (idle). - routes/whisper: drop the progress logger that could never fire. - tests: remove tests of the deleted paths and tests duplicating the descriptor-parameterized core suite or the llama freshness suite. Contracts unchanged: resolver JSON keys, exit codes, marker fields, pairing logs, and the pinned pre-slim fat CPU escape hatch. * Address review feedback on the whisper prebuilt update and install paths - Pin the chained whisper phase to the release the freshness check offered, so the download-host latest pointer cannot reinstall an older build in a loop - Wire the whisper prebuilt install into setup.ps1 (Windows setup previously skipped it entirely) - Treat a non-executable server or missing wired ggml libraries as a broken install instead of reporting already matches - Keep whisper sidecar reloads out of the job-level reload flag and resync chat state after a partial chained update that unloaded llama - Repoint home and profile vars for the whisper-server subprocess at a managed scratch dir and drop credential-store pointers - Clear the prebuilt marker before the opt-in source build overwrite - Write the prebuilt marker with explicit utf-8 encoding * Tighten comments in the whisper prebuilt consumer * Harden the Windows whisper setup phase and the chained update edges - setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH / UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard before the atomic install, and forward the release-tag pin and ROCm hints like setup.sh - sidecar: a cpu-selected install launches whisper-server with --no-gpu (slim wiring links every llama backend, so the flag is what keeps a deliberate CPU choice off the GPU) - chained update: leave whisper unpinned on macOS (the llama phase can walk back there, and a newest-tag pin could be an impossible pairing on every retry) and treat installer exit 2 as kept-existing-runtime instead of failing the combined job - job.to_tag now comes only from the llama phase, so a whisper-only round cannot report a llama update that never ran * Fix slim whisper runtime follow-ups * Address remaining whisper update reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address remaining prebuilt update reviews * Fix remaining chained update reviews * Fix remaining whisper runtime review edges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Daniel Han
|
2c492c8d9b
|
Recognize Radeon 8065S (Gorgon Halo / Ryzen AI Max 400) as gfx1151 (#7290)
* Recognize Radeon 8065S (Gorgon Halo / Ryzen AI Max 400) as gfx1151 * Classify Radeon 8065S (Gorgon Halo) as unified memory in ROCm OOM guard * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
35f887d795
|
Installer: enable ROCm torch on RDNA2 (gfx1030-1036) on Windows (#7277)
Some checks failed
Mac Studio GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio GGUF CI / JSON, images (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-14) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio UI CI / Chat UI Tests (push) Waiting to run
Mac Studio Update CI / Unsloth Updating Tests (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Windows Unsloth GGUF CI / Tool calling Tests (push) Waiting to run
Windows Unsloth GGUF CI / JSON, images (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / GPU prebuilt resolves without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 unit tests (VS 2026 / CMake guard) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2026) (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-2025-vs2026) (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Unsloth export capability / capability (ubuntu-latest) (push) Has been cancelled
Lockfile supply-chain audit / lockfile supply-chain audit (push) Has been cancelled
Unsloth export capability / capability (windows-latest) (push) Has been cancelled
Unsloth export capability / capability (macos-latest) (push) Has been cancelled
* Installer: enable ROCm torch on RDNA2 (gfx1030-1036) on Windows repo.amd.com publishes a gfx103X-all wheel family with win_amd64 torch 2.9.1/2.10.0/2.11.0+rocm7.13.0 (cp310-313), but both Windows allowlists omitted RDNA2, so RX 6000 cards (gfx1030/1032, etc.) fell back to CPU-only torch. Map gfx1030-1036 to gfx103X-all in install.ps1 ($archFamilyMap) and install_python_stack.py (_GFX_TO_AMD_INDEX_ARCH). No torch floor (mirrors gfx110X-all: newest wheel, no _grouped_mm bug on RDNA2). NVIDIA/Mac/CPU and Linux paths untouched; gfx906 stays CPU (no wheels published). * Sync studio/setup.ps1 RDNA2 (gfx1030-1036) allowlists for PR #7277 |
||
|
Daniel Han
|
3ab8dce97a
|
install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection (#6692)
* install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection
get_torch_index_url (and the studio-update mirror _detect_cuda_torch_index_url)
chose the torch wheel family solely by probing the host GPU, with no override.
In a headless / container / CI build the host driver is visible via the
/proc/driver/nvidia/gpus fallback but nvidia-smi cannot report a CUDA version,
so the function fell back to its cu126 default and installed the wrong wheels
(e.g. a cu128 image got cu126 torch).
Add an explicit override checked before any probing, in both the shell installer
and the Python studio-update path:
- UNSLOTH_TORCH_INDEX_URL full index URL, used verbatim (wins)
- UNSLOTH_TORCH_INDEX_FAMILY family (cpu, cu128, rocm6.4, ...) appended to the
mirror base (UNSLOTH_PYTORCH_MIRROR still honoured)
This matches how the published GPU images select CUDA -- vLLM and SGLang take the
CUDA version from an explicit build ARG rather than detecting it, and the Unsloth
Docker base image already pins the cu128 index directly. Desktop installs are
unchanged: with no override set, detection runs exactly as before.
Adds test_get_torch_index_url.sh cases for the override (family, full URL,
precedence, mirror base, trailing-slash strip, empty-ignored).
* install: make the torch-index override authoritative across ROCm paths
Address review feedback on the override added in this PR so a pinned index is
honoured everywhere, not just in get_torch_index_url:
- Skip the WSL ROCm bootstrap (root privilege + large downloads, probes
/dev/dxg) when UNSLOTH_TORCH_INDEX_URL / _FAMILY is set; it previously ran
before the override was consulted.
- Skip the Radeon/Strix rerouting (which re-probes the GPU and overwrites the
resolved URL with repo.radeon.com / repo.amd.com) when the index is pinned, so
an explicit ROCm override (e.g. UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4) is kept.
- install_python_stack.py: derive _TORCH_BACKEND from the override when
UNSLOTH_TORCH_BACKEND is unset (standalone studio update), so _ensure_rocm_torch
/ _ensure_cuda_torch repair to the requested family instead of re-detecting.
- Strip ALL leading/trailing slashes in the shell override to match the Python
side (avoids 404s on strict pip proxies).
Adds test cases for double-slash and leading/trailing-slash overrides.
* install: honor pinned torch index in CUDA/ROCm repair paths
Follow-up to the override work in this PR: the get_torch_index_url / install.sh
reroute already respect a pinned UNSLOTH_TORCH_INDEX_URL / _FAMILY, but the
Python repair helpers in install_python_stack.py still re-probed the GPU and
could overwrite the pinned family. Make the pin authoritative there too:
- _ensure_cuda_torch: an explicit cu* pin commits to CUDA wheels, so repair a
ROCm-poisoned venv even when no NVIDIA GPU is visible here (headless /
container / CI cross-install), instead of bailing on the GPU-presence gate.
- _ensure_rocm_torch: skip the AMD per-gfx (Strix) reroute when a ROCm index is
pinned, and in the generic reinstall path install from the pinned URL verbatim
rather than re-detecting the host ROCm version. gfx*/rocm7.2 indexes serve
torch 2.11+, so select the 2.11 package specs for a gfx leaf.
- install.sh: raise the torch constraint to 2.11 for */gfx* indexes too, matching
rocm7.2, so a pinned full-URL/family override that returns early keeps a valid
constraint.
Add _explicit_torch_index_url / _explicit_rocm_torch_index_url helpers and tests
covering the no-GPU CUDA pin repair and the explicit gfx index honored verbatim.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: honor torch-index override on the Windows installers too
The pinned-index work landed for install.sh and install_python_stack.py, but the
Windows installers still picked the wheel index from GPU probing. Extend the same
UNSLOTH_TORCH_INDEX_URL / _FAMILY contract so a pinned index wins on every platform:
- install.ps1: Get-TorchIndexUrl returns the pinned URL/family before nvidia-smi
probing; the AMD ROCm reroute is skipped when the index is pinned, so an explicit
cpu/cu* pin on an AMD host is not overwritten.
- studio/setup.ps1: add shared Get-PinnedTorchIndexUrl / Get-TorchIndexLeaf helpers;
the stale-venv check, the install selection and the AMD reroute all honor the pin,
and the CPU/CUDA install pulls from the resolved index URL.
- tests: parity test that all four installers read both override vars and the two
Windows installers gate the AMD reroute on the pinned flag.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: complete pinned-index handling for ROCm/Windows edge cases
Follow-ups to the override work flagged in review:
- install.ps1: a pinned gfx*/rocm>=7.2 index previously skipped the AMD reroute
that sets the torch>=2.11 floor, so the generic install used torch>=2.4,<2.11
and could resolve the known-bad _grouped_mm wheel. Route a pinned ROCm index
through the ROCm install path with the 2.11 floor + companions, and guard the
companion-spec lookup so a skipped reroute block cannot null-deref.
- studio/setup.ps1: the stale-venv check compared the installed flavor (cuXXX/cpu,
with +rocm misread as cpu) against the raw pinned leaf (gfx1151 / rocm6.4), so a
correct pinned ROCm venv was always marked stale. Classify +rocm wheels as the
generic 'rocm' flavor and normalize a pinned rocm*/gfx* leaf to 'rocm' before
comparing (cu* stays specific so cu126-vs-cu128 still rebuilds).
- install_python_stack.py: _ensure_cuda_torch now also reinstalls from a pinned
CUDA index when the venv carries a CPU wheel (headless CPU-venv-to-CUDA
cross-install via 'studio update'), not only when it finds a ROCm build.
- tests: parity assertions already cover all four installers honoring the override.
* install: finish pinned ROCm/CUDA edge cases on Windows + repair path
Follow-ups to the previous round:
- studio/setup.ps1: a pinned gfx*/rocm>=7.2 index now routes through the ROCm
install path with the 2.11 floor + companions (it previously fell through to the
CUDA branch with bare torch/torchvision/torchaudio against the ROCm index). The
CPU/CUDA fallback index is forced to the CPU wheel index when a ROCm index is
active, so a failed pinned-ROCm install does not retry the ROCm mirror.
- studio/setup.ps1: the stale-venv check no longer treats an unrecognized pinned
URL leaf (e.g. a PEP 503 mirror ending in /simple) as a torch flavor tag, which
was marking a correct venv stale; cu*/cpu/rocm/gfx leaves are still compared.
- install.ps1: the post-failure CPU fallback uses an explicit CPU index instead of
, which for a pinned ROCm index was the ROCm mirror itself (so the
'fallback' just retried the failing index and aborted the installer).
- install_python_stack.py: _ensure_cuda_torch now also reinstalls when the venv's
CUDA family differs from a pinned one (installed cu126 vs pinned cu128), not only
CPU->CUDA; the probe reports the installed cuXXX tag for the comparison.
* install: keep the ROCm to CPU fallback install inside the retry-helper window
The pinned-ROCm CPU fallback computes an explicit CPU index, but the comment
explaining why it cannot reuse $TorchIndexUrl pushed the actual
Invoke-InstallCommandRetry / --force-reinstall call more than 600 chars past the
"ROCm PyTorch install failed" message, so test_pr5940_followups's window check
no longer saw the retry helper. Move the CPU-index computation and its comment
above the failure substep so the retrying force-reinstall stays adjacent to the
message. No behavior change: same explicit CPU index, same retry, same
--force-reinstall.
* install: address #6692 review round 5 (ROCm/CPU pin edge cases)
setup.ps1:
- Stale-venv check: treat an AMD/ROCm host (HasROCm or a resolved gfx arch) with
no explicit pin as expecting "rocm", not "cpu", so a healthy +rocm venv is not
flagged stale (which made installer-managed setup exit and direct update rebuild).
- Pinned-ROCm install failure now routes into the force-reinstall CPU branch:
CuTag stays the rocm/gfx leaf on failure, so the condition also checks
ROCmCpuFallback; otherwise the CUDA branch installed from the CPU index without
--force-reinstall and kept the partial ROCm torch.
- Explicit ROCm pin compare no longer collapses gfx*/rocm* to a generic "rocm":
it compares the +rocmX.Y version (and the torch 2.11 line for gfx pins) so
changing the pinned family (e.g. rocm6.4 -> gfx1151) rebuilds and applies it.
install_python_stack.py:
- _ensure_rocm_torch: an explicit ROCm wheel-index pin now bypasses the
NVIDIA-present / no-AMD-GPU / unreadable-ROCm gates (headless/container/CI
cross-install), mirroring the explicit-CUDA-pin bypass in _ensure_cuda_torch.
- Add _ensure_cpu_torch: an explicit CPU pin (FAMILY=cpu or /cpu URL) now has a
repair path that reinstalls CPU torch over an existing CUDA/ROCm build on a
standalone update (which skips install.sh's flavor enforcement).
install.sh:
- Pin torchvision/torchaudio companions alongside torch for the rocm7.2 / per-gfx
index and the Strix reroute (those AMD indexes publish companions independently
and a bare name can resolve a torch-2.12-built wheel, an ABI mismatch).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* torch-index override: classify CUDA pin by leaf; trim blank shell overrides
_ensure_cuda_torch only overrode the NVIDIA-presence gate for *any* pinned index,
so a non-CUDA mirror URL (or a ROCm/CPU pin) on a non-NVIDIA host with ROCm torch
could force a CUDA reinstall over a working ROCm venv. Add
_explicit_cuda_torch_index_url() (leaf cu*), matching the ROCm/CPU helpers, and
gate on it instead.
install.sh::get_torch_index_url treated a whitespace-only UNSLOTH_TORCH_INDEX_URL
/ _FAMILY as authoritative (yielding an invalid index), unlike the Python .strip()
and PowerShell IsNullOrWhiteSpace paths; trim leading/trailing whitespace first.
* install: honor pinned torch index over CVD/GPU gates and fix leaf-based ROCm classification
- install_python_stack.py: an explicit cu* pin now clears the CUDA_VISIBLE_DEVICES
empty/-1 hide gate as well as the NVIDIA-presence gate, so
CVD=-1 UNSLOTH_TORCH_INDEX_FAMILY=cu128 studio update repairs to CUDA wheels
(parity with install.sh's get_torch_index_url override, which skips all GPU
probing). Unpinned CVD=-1 still skips.
- install_python_stack.py: _ensure_cpu_torch installs the bounded _CPU_TORCH_PKG_SPEC
instead of a bare torch/torchvision/torchaudio trio; the /cpu index now also
serves torch 2.11+, which is outside the supported <2.11 range.
- install.sh: the torch>=2.11 constraint case matches the index leaf (rocm7.2|gfx*)
instead of the whole URL, so a mirror base path containing a gfx/rocm7.2 segment
with a cu*/cpu family is not false-matched onto the 2.11 line.
- setup.ps1: the stale-venv check expects rocm torch only for arches the install
path maps to a repo.amd.com wheel index; an unmapped/unreadable arch installs
CPU, so a correct CPU venv is no longer marked stale.
- Tests for each of the above.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten pinned torch-index override edge cases
- install.sh: trim whitespace-only UNSLOTH_TORCH_INDEX_URL/_FAMILY before the
_torch_index_pinned guard, matching get_torch_index_url, so a blank override no
longer skips the WSL bootstrap and Radeon/Strix reroutes while detection still
picks the normal index.
- install.sh / install.ps1 / setup.ps1 / install_python_stack.py: force the torch
2.11 floor only for the gfx families with the <2.11 _grouped_mm bug (gfx120X-all,
gfx1151, gfx1150). A pinned override to gfx110X-all/gfx90a/gfx908 stays on the
default range, matching the automatic AMD path.
- install_python_stack.py _ensure_cuda_torch: treat an untagged CUDA build under a
CUDA pin as a family mismatch (reinstall), and match cuXXX pins narrowly (cu +
digits) so a custom/current mirror leaf no longer forces CUDA over a CPU/ROCm venv.
- install_python_stack.py _ensure_rocm_torch: reinstall when an explicit ROCm pin
names a different ROCm family than the already-installed ROCm torch (the ROCm
analogue of the CUDA cuXXX mismatch repair).
Adds tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: fix second-order edge cases in pinned torch-index ROCm/CUDA handling
Parse the ROCm torch probe positionally so an empty HIP marker is kept:
CPU/CUDA torch no longer reads as HIP, so the ROCm reinstall is not skipped.
Emit one "<marker>|<version>" line (like the CUDA probe) for a robust parse.
Limit the gfx torch 2.11 expectation to the install allowlist
(gfx120X-all/gfx1151/gfx1150). A pinned gfx110X-all/gfx90a/gfx908 index stays
on the default <2.11 specs, so a correct 2.10+rocm wheel is no longer judged a
mismatch and force-reinstalled every update.
Distinguish an AMD per-arch wheel (three-part +rocmA.B.C) from a generic
pytorch.org wheel (two-part +rocmA.B): a gfx per-arch pin over a generic 2.11
wheel now reinstalls the per-arch wheel, while an already-installed per-arch
wheel is not re-flagged (no reinstall loop).
Mirror all of the above in setup.ps1 via new Test-RocmGfx211Leaf /
Test-CudaFamilyLeaf / Get-RocmPinStaleTags helpers, reused by both the
install-spec path and the stale-venv check so they cannot diverge again.
Require a digit after "cu" (^cu[0-9]) in setup.ps1, install.ps1 and install.sh
so a mirror leaf like /custom or /current is not branded CUDA and does not
rebuild the venv every run.
Add tests: CPU/CUDA probe -> has_hip_torch False; gfx110X-all pin + 2.10 wheel
not stale; gfx1151 pin + generic 2.11 wheel stale; gfx1151 pin + per-arch wheel
not stale; /custom and /current not CUDA; plus cross-language allowlist and
cu-digit parity guards, and a PowerShell unit test for the new setup.ps1 helpers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix ROCm/gfx pin case normalization, ROCm-tag requirement, and CUDA-leaf classification
Normalize torch-index leaves to lowercase before the gfx*/rocm*/cu* allowlist
matches so the canonical gfx120X-all (capital X) gets the torch 2.11 floor in
install.sh (leaf, flavor and repairable helpers). Require an installed +rocm
local tag before a rocmX.Y or non-2.11 gfx pin is judged satisfied in
setup.ps1 Get-RocmPinStaleTags and the Python _rocm_pin_family_mismatch, so an
untagged CPU/CUDA wheel never leaves the pin unapplied. Classify a leaf as CUDA
only via ^cu[0-9]: the Python _TORCH_BACKEND derivation now uses
_is_cuda_family_leaf, and install.sh brands cuda only on cu[0-9]* (unset on an
unknown /current /custom mirror leaf) so the stack probes the GPU instead of
skipping ROCm repair. Add bash, Python and PowerShell tests for capital
gfx120X-all floor, current/custom not-cuda, and untagged-wheel ROCm pins.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: converge torch-index pin detection via a per-venv marker
Introduce a torch-index MARKER that records the exact wheel --index-url used
after each successful torch install, so `unsloth studio update` / repair makes
the "did the pinned index change?" decision by an EXACT string compare rather
than inferring it from the wheel +rocm/+cu version tag. The tag cannot encode
the AMD per-arch gfx family (two 2.11 gfx indexes both install +rocm7.13.0), so
the tag heuristic missed a gfx1151 -> gfx120X-all switch and a custom-URL swap.
Marker path is per-venv (.unsloth-torch-index), one line = the resolved index
URL, written atomically (temp + rename). Path, format and normalization are
shared across all four installers (install.sh, install_python_stack.py,
setup.ps1, install.ps1).
- Reapply gfx pins on a per-arch target change: the marker's exact compare
reinstalls when the pinned index differs, even when both wheels share a tag.
- Honor custom ROCm URL pins during repair: an explicit index whose leaf is not
rocm/gfx/cu/cpu (e.g. simple, current) now reinstalls torch VERBATIM from the
pin when it differs from the marker ("URL wins verbatim").
- Align the KNOWN-2.11 rocm/gfx set to exactly rocm7.2 plus the gfx allowlist
gfx120x-all/gfx1151/gfx1150 in every language; stop treating an unknown newer
rocm (rocm7.3, which does not exist) as the 2.11 line speculatively.
Backward compatible: with no marker (old venvs, torch installed out-of-band) the
existing +rocm/version-tag heuristics still decide, and a matching marker never
reinstall-loops. A cu128 CUDA pin stays a CUDA pin; custom and current leaves are
not CUDA. Adds marker tests (py/sh/ps) plus cross-installer parity checks.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: keep the torch-index marker additive to flavor validation
Three narrow fixes in the marker-based stale-venv detection:
- setup.ps1: a matching marker no longer overwrites the detected installed
flavor. The marker compare is now an additional rebuild trigger, so a stale
wheel (torch swapped to a +cpu build while the marker still records a cuXXX
pin) is still caught by the flavor check instead of being masked as up to date.
- setup.ps1: a supported AMD arch carrying CPU torch is no longer marked stale
and wiped. The downstream AMD Windows ROCm override upgrades CPU torch to ROCm
in place, so wiping first would delete the venv and abort with "Virtual
environment not found". Only a genuinely wrong CUDA wheel still rebuilds.
- install.sh: the Radeon --find-links path records its repo.radeon.com base in
the marker instead of the generic pytorch.org ROCm fallback index, so a later
pin to that generic family correctly reinstalls rather than comparing equal.
Mirrors install.ps1/setup.ps1, which already record the real AMD index.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: honor custom pins and repair pinned venvs in place
Four follow-ups to the torch-index marker work:
- install_python_stack.py: _ensure_cuda_torch/_ensure_rocm_torch now bail when an
explicit custom-index pin names no known torch family, so a verbatim URL override
(a private/simple mirror) is not clobbered by auto-detected CUDA/ROCm wheels
before _ensure_verbatim_torch_index applies it.
- install_python_stack.py: the ROCm marker is additive, not a substitute -- a
matching marker still runs the family/version check so a wheel swapped after the
marker was written is caught. Mirrors setup.ps1.
- setup.ps1: a stale venv under an explicit pin, whose torch still imports, is
repaired in place (force-reinstall torch from the pin in the dependency pass)
instead of wiped. The wipe path only delegates to install.ps1, so on a direct
update it stranded the user at "Virtual environment not found" instead of
applying the new pin. A broken venv or unpinned drift still wipes/delegates.
- install.ps1: when a pinned ROCm install fails over to a CPU base, the marker now
records the CPU index actually used instead of the ROCm pin, so the next managed
setup does not see CPU torch under a ROCm pin and abort as stale.
* setup.ps1: keep the ROCm CPU-fallback force line the pr5940 test guards
5c93ffd4 folded the pin-change force-reinstall into the ROCm CPU-fallback
condition on one line, so the exact literal that test_pr5940_followups.py checks
(if ($ROCmCpuFallback) { $cpuForce = @("--force-reinstall") }) no longer appeared
and the test failed. Split the two conditions into separate if lines: the ROCm
fallback line is restored verbatim and the pin-change force is its own line. Both
still set $cpuForce to the array, so @splat passes one arg.
* install: honor exact CUDA/custom index URL pins in the torch-index marker
Address three Codex review findings on the torch-index marker mechanism:
- install.sh: after the ROCm CPU repair reinstalls torch from the generic
$TORCH_INDEX_URL, record that as the marker source. A Radeon --find-links
install set _TORCH_MARKER_INDEX_URL to its repo.radeon.com base earlier, so
leaving it made the marker misreport Radeon wheels and a later Radeon pin would
compare equal and skip a needed reinstall.
- install_python_stack.py: _ensure_cuda_torch now consults the exact-URL marker
(_marker_pin_mismatch) when the installed +cuXXX tag matches the pinned leaf,
so a same-leaf CUDA mirror change (official cu128 to an internal cu128 mirror)
is reinstalled and re-recorded instead of skipped.
- _normalize_index_url / _normalize_family_leaf (install.sh, setup.ps1,
install_python_stack.py): lowercase only KNOWN wheel-family leaves (rocm/gfx/
cpu/cuXXX) so gfx120X-all still matches gfx120x-all, while a custom
(unknown-family) leaf keeps its case so a verbatim URL pin like /Current does
not compare equal to /current. Tests updated to assert the refined behavior.
* install: fix 3 torch-index marker edge cases (CPU mirror pin, Radeon leaf, migrated venv)
Addresses three review findings on the torch-index override path:
1. CPU index URL change on an already-CPU venv. _ensure_cpu_torch returned
early whenever torch was already a CPU build, so a standalone update that
moved the pin (official /cpu -> a private UNSLOTH_PYTORCH_MIRROR /cpu, same
+cpu tag) never reinstalled. It now consults the exact-URL marker and
reinstalls only when _marker_pin_mismatch reports a different index,
mirroring the CUDA/ROCm same-family handling. A matching marker (or none)
still leaves CPU torch untouched, so there is no reinstall loop.
2. Radeon find-links directory misclassified as a pip ROCm family. A
repo.radeon.com/.../rocm-rel-7.2.1 leaf starts with "rocm" but is a
find-links listing, not a pip --index-url. The old startswith(("rocm",
"gfx")) test routed it into a --index-url reinstall that fails against
find-links. New _is_pip_rocm_family_leaf gates on ^rocm\d / gfx (matching
install.sh's rocm[0-9]* and setup.ps1's ^(rocm[0-9]|gfx)), so a Radeon URL
routes to the verbatim/marker path instead.
3. Migrated venv rewriting its marker to a pin it did not install. install.sh
and install.ps1 write the marker unconditionally, so a migration that
preserves existing torch recorded the newly requested pin and a later
update then found a matching marker and skipped the reinstall the pin
needs (e.g. a per-arch gfx1151 -> gfx120X-all switch, identical +rocm tag).
Both now track _TORCH_INSTALLED_THIS_RUN and write the marker only when
torch was actually installed or repaired this run.
Also add Get-NormalizedFamilyLeaf to the setup.ps1 helper-extraction list in
test_torch_index_marker.ps1 (it was added to setup.ps1 and the shell test in an
earlier round but missed here) and add two unit tests covering findings 1 and 2.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: keep pinned torch repairs on the pinned index
Two fixes for explicit index pins (UNSLOTH_TORCH_INDEX_FAMILY / _URL):
1. install_python_stack.py's repair paths ran uv without clearing the
inherited uv index env vars. uv resolves the default index (--index-url
or --default-index) at the LOWEST priority, so a UV_INDEX or
UV_EXTRA_INDEX_URL mirror in the environment won for any package it
served: a cu128-pinned repair could install torch from the mirror and
then record the cu128 marker it never used. Verified empirically: with
UV_EXTRA_INDEX_URL=.../cu126 exported, uv pip install torch
--index-url .../cu128 resolves torch 2.13.0+cu126. Strip the four uv
index env vars for pinned-index commands only, mirroring the gate
install.sh, install.ps1 and setup.ps1 already have; non-pinned installs
keep the user's mirror.
2. install.ps1 routed any pinned leaf matching rocm* through the ROCm
--default-index path, so a custom find-links leaf like rocm-rel-7.2.1
was treated as a PEP 503 ROCm index and could silently fall back to CPU
torch on resolution failure. Require a digit after rocm, matching
install.sh's rocm[0-9]* and install_python_stack.py's ^rocm\d.
Adds parity + unit tests for both (11 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: keep pinned repairs off UV_TORCH_BACKEND and narrow setup.ps1's rocm pin match
Round 2 of the pinned-index hardening:
1. _build_uv_cmd converted UV_TORCH_BACKEND into --torch-backend before the
new env isolation could act, and uv's torch backend redirects torch
resolution to its own per-backend index even when --index-url is given
(verified: a cu128-pinned dry run with UV_TORCH_BACKEND=cpu resolves
torch 2.13.0+cpu). Pinned-index commands now never receive the flag and
UV_TORCH_BACKEND joins the stripped env vars, so uv cannot re-read it.
2. setup.ps1's pinned reroute had the same bare rocm* glob install.ps1 had:
a custom find-links leaf like rocm-rel-7.2.1 was routed through the ROCm
--index-url path instead of the verbatim unknown-pin path. Now requires
a digit after rocm, matching install.ps1, install.sh and
_is_pip_rocm_family_leaf.
3. The marker test's case-normalization checks used -eq, which is
case-insensitive in PowerShell, making them vacuous, and the unknown-leaf
expectation was written lowercased while the implementation deliberately
preserves custom-leaf case. Tightened to -ceq with the case-preserving
expected value.
Adds unit + parity tests for 1 and 2 (5 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: extend the pinned-index guards to every remaining surface
Round 3 of the pinned-index hardening, closing the same holes on the
surfaces the earlier rounds missed:
1. install.sh's pinned-install env scrub now clears UV_TORCH_BACKEND (uv's
torch backend redirects torch resolution to its own per-backend index
even against --default-index), and both PowerShell wrappers clear it in
their pinned-install scrubs, matching install_python_stack.py.
2. setup.ps1's marker stale check still classified any rocm* leaf as a
PyTorch ROCm family while the install selection is digit-gated, so a
custom rocm-current / rocm-rel-7.2.1 pin stale-compared as
not-rocm vs rocm and force-reinstalled on every studio update. The
stale check now uses the same ^rocm\d gate.
3. install_python_stack.py's pinned-command scrub also strips
PIP_EXTRA_INDEX_URL for the pip fallback: pip adds the env extra index
in addition to --index-url, so an inherited mirror could satisfy torch
off the pin while the marker recorded the pinned URL. PIP_INDEX_URL
needs no strip since the explicit --index-url flag overrides it.
Parity + unit tests extended (4 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: scrub find-links and carry the pinned scrub through pip fallbacks
Round 4 of the pinned-index hardening:
1. UV_FIND_LINKS joins every pinned-install scrub (install.sh, install.ps1,
setup.ps1, install_python_stack.py): uv's --find-links locations can
satisfy torch off the pinned index the same way an extra index does.
2. setup.ps1's Fast-Install restored the scrubbed vars in its finally
BEFORE the pip fallback ran, and never touched the pip env vars at all,
so a failed uv attempt fell back to python -m pip with an inherited
PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS able to win over the pinned
--index-url. The scrub now wraps the whole function (uv attempt + pip
fallback) and includes the pip vars; restore happens after both.
3. install_python_stack.py's scrub also strips PIP_FIND_LINKS for its own
pip fallback, completing the PIP_EXTRA_INDEX_URL fix from round 3.
Parity tests extended (2 new tests).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: digit-gate rocm leaves in marker normalization and ROCm side effects
Round 5 of the pinned-index hardening (three custom-rocm-leaf edge cases):
1. _normalize_family_leaf lowercased every leaf starting with rocm, so a
custom mirror leaf like rocm-Current compared equal to its lowercase form
and a case-only pin change was skipped. URL paths can be case-sensitive.
The rocm prefix is now digit-gated (rocm[0-9]*, matching
_is_pip_rocm_family_leaf) in install.sh, setup.ps1 and
install_python_stack.py, so only true family leaves (rocm7.2) are
lowercased; a custom rocm-* leaf keeps its case.
2. setup.ps1 Test-MarkerPinMismatch compared normalized URLs with -ne, which
is case-insensitive in PowerShell, so a case-only marker change (Simple
vs simple) was treated as matching and the reinstall skipped. Now -cne.
3. install.sh gated the AMD bitsandbytes install and the "repair ROCm torch"
--default-index reinstall on a bare whole-URL rocm glob, so a custom
CPU/CUDA/private index whose leaf merely starts with rocm (rocm-current)
was force-repaired from the wrong ROCm-only path whenever torch.version.hip
was empty. Both now gate on _torch_index_is_rocm_family, computed once from
the digit-gated leaf (rocm[0-9]*/gfx*).
Tests: 4 new parity assertions plus 2 case-sensitivity marker checks.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: apply an explicit custom torch-index pin on the first update
Round 6: an explicitly-set custom (unknown-family) UNSLOTH_TORCH_INDEX_URL
was silently ignored on the first `studio update` of a venv that predates
the marker feature, on both platforms, because the no-marker case was
treated as "do nothing" and the version-tag heuristics cannot judge an
unknown leaf.
1. install_python_stack.py _ensure_verbatim_torch_index now reinstalls
verbatim when the marker is ABSENT (None), not only when it differs, and
short-circuits only when the marker already records this exact pin. It
then writes the marker, so every later update is a no-op. A user who did
not set the override gets pin=None and is untouched, so an out-of-band
torch install is never clobbered.
2. setup.ps1: for an unknown-family pin on a marker-less venv the stale-venv
check now sets PinChangedForceReinstall so the torch block reinstalls in
place from the pin. It deliberately does NOT set shouldRebuild, which
would wipe the venv and strand a direct `studio update`.
3. setup.sh (the Linux `studio update` entry point) skipped
install_python_stack.py entirely when unsloth was already current, so the
marker-driven reinstall (both the verbatim custom pin and the cu/rocm
flavor and family-change repair, e.g. gfx1151 to gfx120X-all) never ran.
It now forces the dependency pass when a torch-index pin env var is set;
the pass is idempotent and no-ops when the marker already matches. This
mirrors setup.ps1's stale-venv pre-check.
Tests: 3 new parity assertions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: expect first-update reinstall for a no-marker custom index pin
Follow-up to d671d8fb2: _ensure_verbatim_torch_index now applies an
explicit unknown-family URL pin verbatim on the first update when the
marker is absent (instead of no-op), so the old
test_verbatim_custom_url_no_marker_is_noop assertion was stale. Rewritten
as test_verbatim_custom_url_no_marker_reinstalls_once: asserts the one
verbatim reinstall from the pinned URL, that the marker is written, and
that a second call with the pin still set is idempotent (no reinstall
loop).
* install: gate the pinned update pass on the marker and record a pin baseline
Round 8, two follow-ups to the round-6 first-update pin fix:
1. setup.sh forced the full dependency pass on EVERY `studio update` while a
torch-index pin stayed exported, even after the marker already recorded the
same pin, turning quick updates into the expensive pass every time. It now
probes install_python_stack.py --torch-pin-needs-apply (which reuses the
exact marker normalization) and forces the pass only when the pin is not yet
applied (marker absent or different); an already-applied persistent pin keeps
the fast path. A probe error fails safe toward running the pass. setup.ps1
gets the same probe in its fast path for parity.
2. A known-family full-URL pin on a venv predating the marker (e.g. an installed
cu128 build and UNSLOTH_TORCH_INDEX_URL pointing at a same-family mirror) left
the marker absent forever: the _ensure_* helpers deliberately do not force a
multi-GB reinstall of identical-family wheels on an old venv, so nothing
recorded the pin and every update re-entered the pass. _record_torch_index_pin_baseline
now records the resolved pin as a baseline after the ensure sequence when the
family already matches and no marker exists, so the pin is tracked (a later
genuine change is detected and applied) and the update loop is broken, without
the redundant reinstall.
Tests: 3 new baseline unit tests, 4 new parity assertions, and the CLI probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.sh: keep the pin probe's exit 1 from killing the update under set -e
The --torch-pin-needs-apply probe deliberately exits 1 for the common
steady-state answer (pin already recorded, keep the fast path), but it ran
as a bare command under set -euo pipefail, so the whole studio update
aborted before the exit code was even captured. Absorb the status with
|| _PIN_NEEDS_APPLY=$? and pre-seed 0 so all three outcomes route as
documented: 0 runs the pass, 1 keeps the fast path, anything else fails
safe into the pass. Parity test asserts the guard.
* install: strip pin credentials, disable uv config discovery, bound verbatim installs
Four verified fix groups from a 12-reviewer audit of the torch-index
override feature, each reproduced before fixing:
1. Credential persistence: all four marker writers stored the raw pin URL,
so an authenticated pin (https://user:token@mirror/simple) persisted its
credentials in .unsloth-torch-index (mode 0644 under a default POSIX
umask) and install_python_stack.py printed pin URLs verbatim in repair
messages. Userinfo is now stripped before persisting and in every
log/substep that interpolates a pin, via lockstep helpers
(_strip_index_url_credentials in install.sh / install_python_stack.py,
Remove-IndexUrlCredentials in install.ps1 / setup.ps1). The three
normalizers strip too, so an OLD marker that already carries credentials
still compares equal to the same pin: no reinstall loop on upgrade.
Query strings deliberately stay in the marker; two indexes distinguished
only by query must not compare equal.
2. uv configuration discovery beat the explicit pin: with a discovered
uv.toml declaring torch-backend = "cpu" or a [[index]] entry, uv 0.10.12
resolves torch 2.13.0+cpu against an explicit --index-url/.../cu126 pin;
UV_NO_CONFIG=1 restores +cu126 (reproduced both ways). The pinned-install
scrub in all four installers now sets UV_NO_CONFIG=1 and drops
UV_CONFIG_FILE.
3. The verbatim custom-index update path installed a bare, unconstrained
torch trio while fresh installs from the same unknown-leaf pin apply the
supported range; _ensure_verbatim_torch_index now installs the bounded
trio spec, closing the fresh-vs-update asymmetry.
4. Query-bearing pins (.../cu128?token=x) classified by raw leaf split and
force-reinstalled on every update (the installed cu128 never equals
cu128?token=x). Query/fragment are now stripped before leaf
classification in all four implementations; the marker comparison keeps
the query per (1).
Rejected after verification (no change): the pin-baseline record cannot
produce a wrong later decision (every pin change still mismatches and
reinstalls from the new pin); the venv temp-file symlink scenarios require
an attacker who already owns the environment; pathological inputs like
" / cu128 / " have no realistic caller and fail loudly.
Parity, stack, rocm-support, marker (sh + ps1), pin-stale, index-url and
flavor suites all pass (455 python + full shell/ps1 batteries).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: harden custom-pin repair against clobber, broken torch, and pip config
Four follow-ups to the pinned-index audit fixes:
1. setup.ps1 routed an unknown-leaf custom pin through the CUDA branch with
a bare torch trio while install.ps1 (fresh) and the Python verbatim path
bound the supported range; the pinned unknown-leaf route now applies the
same torch>=2.4,<2.11.0 bound. Known cu* leaves and unpinned runs are
unchanged.
2. The final torch safety pass could not repair a clobbered unknown-family
pin: intermediate dependency steps can pull torch from PyPI (the pass
exists for exactly that reason), but the verbatim helper short-circuited
on marker==pin and no flavor tag exists to probe. The helper now keeps a
per-run snapshot of the installed trio (taken after a verbatim reinstall
or on the first matching-marker pass) and reinstalls from the pin when
the final pass sees the trio drifted. Probe failure skips the
comparison; a reinstall refreshes the snapshot, so no loop.
3. _record_torch_index_pin_baseline could freeze a known-family pin as
applied on a venv whose torch is missing or broken (every family helper
returns without reinstalling when its probe fails), making
--torch-pin-needs-apply report done forever. The baseline now probes the
installed flavor and records only on a match: a cuXXX pin requires the
matching +cuXXX tag, cpu requires a cpu build, rocm/gfx requires hip;
probe failure records nothing.
4. The pinned pip fallback stripped PIP_* env vars but user/site pip config
files still applied (a configured global.extra-index-url can satisfy
torch off the pin). PIP_CONFIG_FILE is now pointed at the null device
for pinned commands (pip loads no config files then), in
_install_env_for_cmd and setup.ps1's Fast-Install pinned scrub.
install.sh / install.ps1 have no pip fallback (uv-only), verified.
Tests: 7 new rocm_support tests (snapshot reset fixture), 1 stack test,
2 parity tests. Full battery green (464 python, sh and ps1 suites).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: complete the pin-repair coverage across the fast path and platforms
Three cross-platform follow-ups to the round-2 pin-repair fixes:
1. The --torch-pin-needs-apply probe only compared marker==pin, so a torch
trio clobbered to the wrong family (a cpu wheel replacing cu128 via a
later pip install) with a still-matching marker reported "already
applied" and the _ensure_{cuda,rocm,cpu} repair never ran on the Linux
fast path. The probe is now a testable _torch_pin_needs_apply() that also
checks the installed flavor against a known-family pin (via a shared
_torch_flavor_matches_pin() helper, so the baseline and the probe cannot
drift). An unknown-family pin has no flavor to validate and a failed
probe cannot prove drift, so both keep the fast path.
2. macOS ARM (real CPU/MPS torch, not NO_TORCH) never applied an unknown-
family custom pin on update: both the verbatim path and the baseline
returned on IS_MACOS while fresh install.sh honors the pin, so the marker
was never written and setup.sh forced the dependency pass on every update
forever. The guards are now IS_MAC_INTEL (Intel mac is already NO_TORCH),
and the final pass applies the pin on macOS ARM.
3. The round-2 final verbatim repair sat in the step-13 sequence guarded
not IS_WINDOWS, so on Windows a dependency step that clobbered torch after
the pin was applied was masked by the matching marker (setup.ps1 does not
re-validate the main venv's torch after calling this script -- verified).
Step 13 now runs the verbatim snapshot-drift repair on Windows and macOS
ARM too; the Linux-oriented cuda/rocm/cpu family helpers stay Linux-only.
Tests: 13 new rocm_support cases (flavor drift, macOS ARM, Windows repair),
parity updates. Full battery green (475 python, sh and ps1 suites).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: strip query tokens from the marker and tighten the pin-drift probe
Four follow-ups to the round-3 pin-repair fixes:
1. The credential stripper feeding the torch-index marker and the logged repair
messages dropped only user:pass@ userinfo, so a private feed that carries its
auth token in the query string (.../simple?token=SECRET) persisted the token
in the world-readable marker (mode 0644 under a default umask) and printed it
in substep output. All four strippers (install.sh, install.ps1,
studio/setup.ps1, install_python_stack.py) now drop the query and fragment
before building the sanitized URL. A query is not part of a PEP 503 index's
identity, so this also stops a rotated token from spuriously mismatching the
marker and forcing a needless reinstall.
2. The --torch-pin-needs-apply fast-path probe accepted an untagged CUDA build
(no +cuXXX local tag) under a specific cuXXX pin, but _ensure_cuda_torch
reinstalls exactly that build to enforce the pin. The probe was more lenient
than the repair, so the repair pass was skipped on the fast path.
_torch_flavor_matches_pin now reports a mismatch for an untagged build under a
cuXXX pin, forcing the pass.
3. The probe's ROCm branch accepted any HIP build for a rocm/gfx pin, while
_ensure_rocm_torch decides a reinstall with the per-arch
_rocm_pin_family_mismatch predicate (a generic +rocm7.2 wheel under a per-arch
gfx pin, or a wrong ROCm version, is a mismatch). The probe now reuses that
predicate, so it is as strict as the repair. This needs the installed torch
version, so _probe_torch_flavor now returns (marker, cutag, version) and
_torch_flavor_matches_pin takes the pin URL (extracting the leaf internally).
4. On Windows a known-family cu*/cpu pin is applied to the main venv by setup.ps1
before install_python_stack.py runs; a later dependency step can clobber it,
and the GPU-aware _ensure_{cuda,cpu}_torch self-skip on Windows while the
verbatim helper handles only unknown-family pins, so nothing repaired the
clobber (setup.ps1 does not re-validate the main venv's torch afterward,
verified). New _ensure_pinned_known_family_torch reinstalls a drifted cu*/cpu
pin in the step-13 Windows/macOS-ARM branch; rocm/gfx per-arch specs stay owned
by setup.ps1, unknown-family by the verbatim helper.
A speculative ROCm 2.11 floor was also raised but is unreachable: the rocm7.2
index publishes no 2.x wheel below 2.11.0, and an unknown newer rocm is not
floored speculatively.
Tests: query/fragment strip cases in the sh + ps1 marker suites and the Python
strip/marker tests; the tri-state helper and the probe/baseline harnesses moved
to the (marker, cutag, version) flavor with matching versions; new probe cases
(untagged CUDA, generic-rocm-under-gfx) and 8 _ensure_pinned_known_family_torch
tests; a four-way query-strip parity assertion. Full battery green (1150 python,
sh 26/26 marker, ps1 marker/flavor/pin-stale).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: reinstall markerless gfx pins and cap custom-index updates at torch 2.11
Two follow-ups from the pin-marker audit:
1. A markerless venv with a gfx per-arch 2.11 pin trusted the wheel version
tag, which is byte-identical (+rocm7.13.0) across gfx120X-all / gfx1151 /
gfx1150. A pre-marker install holding one gfx arch's wheel that is now
pinned to a DIFFERENT gfx index was therefore never switched:
_rocm_pin_family_mismatch returns no-mismatch for any three-part +rocm
2.11 wheel, and _ensure_rocm_torch's absent-marker branch fell through to
that heuristic. _ensure_rocm_torch now forces a one-time reinstall when the
marker is absent AND the pin leaf is a 2.11 gfx per-arch index; the reinstall
writes the marker, so the next update compares exactly and does not loop
(the correctly-pinned no-reinstall guarantee then comes from the exact marker
compare, not the ambiguous tag). Non-gfx-2.11 pins (rocmX.Y, non-2.11 gfx)
stay on the tag heuristic -- their tags are distinguishable.
2. The verbatim custom-index update path used _CUDA_TORCH_PKG_SPEC (torch
<2.12.0) while a FRESH install of the same unknown leaf caps torch at
<2.11.0 (install.sh's default TORCH_CONSTRAINT, and setup.ps1's custom-pin
branch), so a private /simple mirror publishing torch 2.11 could upgrade a
`studio update` to a state the fresh installer never produces. Added
_CUSTOM_INDEX_TORCH_PKG_SPEC (torch>=2.4,<2.11.0), used only by the verbatim
path; companions stay pinned for the same exclusive --index-url ABI reason
as _CUDA_TORCH_PKG_SPEC (a bare name could pull a torch-2.12-built
torchvision). _CUDA_TORCH_PKG_SPEC is unchanged (known-family cu/cpu repair
correctly tracks install.sh's widened cu ceiling).
Tests: 2 new markerless-gfx cases (one-time reinstall + marker write + no-loop
second run, and the rocmX.Y absent-marker no-op), the pre-existing markerless
gfx no-reinstall test flipped to assert the one-time reinstall (it had encoded
the old tag-trusting behavior), and the custom-index bound assertions. 488
passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: a matching marker must not mask a broken, clobbered, or misclassified torch
Four round-6 follow-ups, all closing cases where a matching torch-index
marker wrongly vouched for a torch that is not actually the pinned one:
1. _is_cuda_family_leaf matched cu+digits by PREFIX (^cu[0-9]), so a custom
mirror leaf like cu128-private classified as CUDA family; the flavor check
then compared the installed cu128 tag to the whole leaf cu128-private and
forced a reinstall on EVERY update (never converging). The cu family is
now matched EXACTLY (re.fullmatch cu[0-9]+), so a cu-suffixed custom leaf
routes through the verbatim/unknown path with a stable marker. Mirrored in
install.sh (_normalize_family_leaf: strip cu, require an all-digit
remainder) and setup.ps1 / install.ps1 (^cu[0-9]+$).
2. _torch_pin_needs_apply returned False on a failed torch probe (missing or
unimportable) under a matching marker, so setup.sh kept the fast path and
a broken torch was never repaired. A failed probe now forces the pass: the
marker cannot vouch for a torch that does not import, forcing is idempotent,
and once torch imports again the probe succeeds and the forcing stops
(self-resolving). Reverses the round-4 conservative choice for this case.
3. _ensure_verbatim_torch_index snapshotted the installed trio on the first
pass with a matching marker and treated an unimportable torch (snapshot
None) as "no drift, skip", so a torch clobbered to a broken state before
the run was masked. A None snapshot now reapplies the pin. A torch
clobbered to a WORKING-but-wrong build under an unknown-family pin remains
undetectable from metadata (no flavor tag; reinstalling every update would
be the loop this avoids) and is documented as a known limitation.
4. The step-13 Windows final repair reran only the verbatim (unknown-family)
and known-family cu*/cpu paths, so a clobbered explicit rocm/gfx pin (the
wheel setup.ps1 installed from AMD's per-arch index) was left in place. The
branch now also runs _ensure_rocm_torch on Windows for an explicit rocm/gfx
pin; it has a Windows path and no-ops when torch already links HIP, so it
only reinstalls a genuinely clobbered ROCm venv (loop-safe).
Tests: the round-4 failed-probe-trusts-marker test flipped to force the pass;
new cases for the cu-suffix no-loop, the broken-torch verbatim reinstall, and
the Windows rocm final-repair structure; item-2 exact-cu parity assertions.
490 passed. sh/ps1 marker + flavor + pin-stale suites all green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: repair Windows ROCm pins from the pinned URL and honor NO_TORCH
Four round-7 review items, two of them regressions in the round-6 work:
1. _torch_pin_needs_apply ignored UNSLOTH_NO_TORCH. With a torch-index env
var set and no marker, the failed-probe branch forced the dependency pass
on every `studio update`, and the pass (which also honors NO_TORCH) never
installs torch or writes a marker, so nothing could ever stop the forcing.
It now returns False immediately under NO_TORCH: the pin only matters once
torch is actually installed.
2. The step-13 Windows final repair (round-6) restored a clobbered explicit
rocm/gfx pin by calling _ensure_rocm_torch, whose Windows path reinstalls
from the arch AUTO-DETECTED via hipinfo, not from the pin. A user pinning a
different gfx family or a private mirror was restored from the wrong source
(and the wrong marker written), and a headless box was skipped entirely
(the arch probe returns nothing). The repair now goes through
_ensure_pinned_known_family_torch, which reinstalls from the PINNED url with
the same per-arch floor setup.ps1 uses (2.11-line gfx leaves) or a bare trio
(older arches, rocmN mirrors). It is gated on IS_WINDOWS since macOS ARM has
no ROCm, and the existing flavor check keeps it loop-safe (a matching HIP
wheel is left alone).
3. _ensure_verbatim_torch_index's broken-torch check (round-6) used
"_installed_trio_snapshot() is None", but that helper reports a REMOVED torch
as "torch==absent" (a non-None tuple) and a broken import as the stale
on-disk version, so a missing or unimportable torch under a matching marker
was read as "no drift" and skipped. The matching-marker path now confirms
torch health with an import probe (_probe_torch_flavor): a torch that does
not import reapplies the pin, while a healthy torch keeps the snapshot-based
intra-run drift detection.
4. A unit test for _ensure_cpu_torch did not pin NO_TORCH False like its
siblings, so a suite run with UNSLOTH_NO_TORCH=1 in the environment made the
guard return early and the reinstall assertions fail spuriously.
Tests: the round-6 broken-torch verbatim test re-encodes the non-None
"torch==absent" snapshot case (the exact state the old "is None" check missed);
new Windows-ROCm pinned-repair cases (reinstall from the pin, per-arch floor vs
bare spec, matching-wheel no-op, off-Windows no-op); a NO_TORCH fast-path probe
case; the parity test now asserts the Windows final branch does not auto-detect
the ROCm index and that the helper reinstalls from the explicit pin. 494 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: floor the rocm7.2 index in the Windows pin repair; isolate marker tests
Three round-8 review items, two of them downstream of the round-7 changes:
1. _ensure_pinned_known_family_torch gave a rocm<d> index leaf a bare
torch/torchvision/torchaudio trio while flooring only gfx* leaves, so a
Windows venv clobbered under an explicit rocm7.2 pin could reinstall an
unbounded or ABI-mismatched trio from that exclusive --index-url. It now
mirrors the spec the initial ROCm paths pin: the rocm7.2 floor for 2.11-line
gfx leaves and rocm<d> leaves that serve torch 2.11, the <2.11 default for
older rocm versions, and a bare trio only for older gfx per-arch leaves
(which publish no floor), matching _ROCM_TORCH_PKG_SPECS / _ensure_rocm_torch.
2. test_verbatim_custom_url_no_marker_reinstalls_once called
_ensure_verbatim_torch_index twice; the second call now hits the
matching-marker health probe, and with pip_install mocked torch never becomes
importable, so in a no-torch environment _probe_torch_flavor returned None and
forced another reinstall, failing the idempotence assertion. The test now pins
a healthy flavor so the idempotence check is about the marker, not ambient
torch.
3. The TestEnsureRocmTorchMarker fixture patched os.environ per test but not
_TORCH_BACKEND, which install_python_stack.py computes once at import from
UNSLOTH_TORCH_BACKEND. A runner starting with a cuda/cpu backend made
_ensure_rocm_torch early-return and skip the mocked repair these tests
exercise. The fixture now neutralizes _TORCH_BACKEND so the marker tests are
independent of the caller's installer-pin environment.
Tests: the Windows floor-spec test now asserts a rocm7.2 mirror pin uses the
rocm7.2 floor (not bare), plus a new rocm7.1 case that must fall back to the
<2.11 default; the marker suite passes under a hostile
UNSLOTH_TORCH_BACKEND=cuda / UNSLOTH_TORCH_INDEX_URL env. 495 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: apply same-flavor pin repoints, keep ROCm fallback nonfatal, bound custom companions
Four round-9 review items, two of them regressions in the round-7 pin helper:
1. _ensure_pinned_known_family_torch returned as satisfied whenever the installed
flavor matched the pin, so a same-flavor SOURCE change (one /cpu or /cu128
mirror to another, or a gfx1151 -> gfx120x-all per-arch switch, both carrying
the same wheel tag) was never applied, while _torch_pin_needs_apply kept forcing
the pass on the marker mismatch forever. It now also reinstalls when the marker
records a DIFFERENT index of the same flavor, rewriting the marker so the next
update matches (no loop), exactly as the Linux _ensure_{cuda,cpu}_torch helpers
do. An absent marker on an already-matching venv is still left to the baseline
recorder (no forced reinstall of a correct pre-marker venv).
2. That helper reinstalled a Windows ROCm pin with the FATAL pip_install, so when
setup.ps1 had taken its CPU fallback (the pinned AMD index unavailable), the
final repair re-hit the same missing index and aborted the whole install. The
ROCm reinstall is now nonfatal (pip_install_try): on failure it leaves the CPU
base in place and writes no ROCm marker, so the install completes -- matching
_ensure_rocm_torch's Windows path. cu*/cpu pins stay fatal (authoritative source).
3. install.sh left torchvision/torchaudio bare for a pinned custom/unknown-leaf
index (a private /simple mirror), unlike the Python update path's
_CUSTOM_INDEX_TORCH_PKG_SPEC, so a mirror also exposing newer companion wheels
could resolve a torch-2.12-built torchvision against the capped <2.11 torch. It
now bounds the companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0)
for a custom leaf, gated on an empty _expected_torch_flavor_tag so known families
keep their curated bare/floored companions.
4. install.sh's _expected_torch_flavor_tag matched cu[0-9]* by prefix, so a custom
leaf like cu128-private classified as the cu128 family and force-reinstalled a
correct +cu128 wheel on every run. It now requires exact cu+digits (routing the
suffixed leaf to the custom path), matching the Python re.fullmatch(cu[0-9]+) and
PowerShell, and feeding item 3's custom-leaf detection.
Tests: new cases for the same-flavor marker-change reinstall, the nonfatal ROCm
fallback (no marker on failure), the rocm7.2/older-rocm floor selection now split
across the nonfatal path, cu-suffixed custom leaves in test_torch_flavor.sh, and the
custom-leaf companion bounds in test_torch_constraint.sh. 497 python + 143 shell
assertions pass; the marker suite still passes under a hostile
UNSLOTH_TORCH_BACKEND=cuda env.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: bound custom-pin companions on the Windows setup path; isolate pin-probe tests
Two round-10 review items:
1. setup.ps1's custom/unknown-leaf pin branch capped only torch ($cudaTorchSpec)
and still asked the exclusive index for bare torchvision/torchaudio, so a
private mirror that also serves newer companion wheels could install a
torch<2.11 wheel alongside a torchvision>=0.26 / torchaudio>=2.11 built for a
newer torch ABI, after which the marker records the pin as applied. It now
bounds the whole trio (torch>=2.4,<2.11.0 / torchvision>=0.19,<0.26.0 /
torchaudio>=2.4,<2.11.0) for a pinned non-cu-family leaf, matching install.sh,
install.ps1's fresh pinned install, and install_python_stack.py's
_CUSTOM_INDEX_TORCH_PKG_SPEC. This completes the companion-bounds fix across all
three installers; known cu* leaves keep bare specs (the family index bounds them).
2. The _torch_pin_needs_apply probe tests did not pin NO_TORCH False, so a test
process launched with UNSLOTH_NO_TORCH=1 short-circuited the probe (the round-7
guard) and returned False for cases that expect the pass to run. The _needs_apply
helper now patches NO_TORCH (default False) around the call, and the dedicated
no-torch case passes no_torch=True explicitly.
Tests: the cross-platform parity test now asserts setup.ps1 bounds the full trio
(not just torch) for a custom leaf; the pin-probe suite passes under a hostile
UNSLOTH_NO_TORCH=1 environment. setup.ps1 parses clean; 497 python + shell suites
green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: bound custom rocm-* pins, redact diag tokens, snapshot custom pins before base update
Three round-11 review items, all reproduced before fixing:
1. install.sh's custom-index companion bounds gated on _expected_torch_flavor_tag
returning empty, but that helper returned "rocm" for ANY rocm* leaf, so a custom
mirror whose leaf starts with rocm but is not a pip family (a private rocm-current
mirror, a Radeon find-links rocm-rel-7.2.1) escaped the bounds and installed bare
torchvision/torchaudio. It now digit-gates rocm to rocm[0-9]* (matching the Python
_is_pip_rocm_family_leaf ^rocm\d), so those custom leaves return "" and the <2.11
companion caps apply; real rocm7.2 / gfx per-arch indexes still classify as rocm.
2. _tauri_torch_index_family classified by the raw last path segment, so a pinned URL
carrying auth in the query (.../rocm7.2?token=SECRET) had the token echoed verbatim
into the emitted [TAURI:DIAG] line. It now strips query/fragment before classifying
(mirroring the marker/log credential stripping), so no token reaches the diagnostic
output; as a side effect .../cu128?token=x now classifies as cu128 instead of auto.
3. On studio update, the core package step (a newer unsloth can require a torch the
custom pin does not satisfy, pulling a default PyPI trio) runs BEFORE the step-2b
verbatim check, which then recorded the already-clobbered trio as the baseline for a
matching marker and left the pin unapplied. A new _capture_verbatim_baseline() records
the pre-clobber trio before the core step, so the verbatim pass detects the drift and
reapplies the pin. Captures only for a matching custom pin with importable torch; a
mismatched/absent marker or broken torch is left to _ensure_verbatim_torch_index.
Tests: _expected_torch_flavor_tag rocm-current / rocm-rel cases; _tauri_torch_index_family
token/fragment redaction with a no-leak regression guard; _capture_verbatim_baseline
record/skip cases plus an end-to-end clobber-detection scenario; a structural guard that
the capture runs before the core step. 501 python + shell suites pass; install.sh bash -n
clean, shellcheck unchanged from base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: match rocm family leaves exactly, enforce the rocm7.2 torch line, repair a broken pinned torch
A pinned index is a pip ROCm --index-url family only when its leaf is an exact
rocm<digits> / rocm<digits>.<digits> (rocm7.2) or a gfx* per-arch leaf. The prior
^rocm[0-9] prefix match also caught suffixed private-mirror leaves (rocm7.2-private,
rocm7-current), routing them through the ROCm/companion-family path instead of the
verbatim pin: the companion bounds were skipped and, on a pre-marker venv with a
compatible +rocm wheel, the pin was never applied. Match the family exactly through one
shared helper at every site:
- install_python_stack.py: _is_pip_rocm_family_leaf (re.fullmatch), plus the two other
loose gates it feeds (_normalize_family_leaf, _torch_flavor_matches_pin).
- install.sh: a new _is_pip_rocm_family_leaf routes _expected_torch_flavor_tag,
_torch_index_repairable, _normalize_family_leaf and the ROCm side-effect gate.
- setup.ps1: a new Test-PipRocmFamilyLeaf routes Get-NormalizedFamilyLeaf and both
pinned reroutes; install.ps1 anchors its reroute regex.
_rocm_pin_family_mismatch (and its setup.ps1 mirror Get-RocmPinStaleTags) compared only
the ROCm version, so a +rocm7.2 wheel whose torch release drifted off the 2.11 line
(2.12/2.13 from an out-of-band upgrade or a custom rocm7.2 mirror) satisfied the family
check while violating _ROCM_TORCH_PKG_SPECS['rocm7.2'] (torch>=2.11,<2.12). Flag it stale
so the repair reinstalls to floor; >=2.11 alone is not enough, so the release is compared
exactly against the 2.11 line for a KNOWN-2.11 rocm pin.
_ensure_pinned_known_family_torch returned on a failed import probe, but
_torch_pin_needs_apply forces the dependency pass on that same failed probe: a broken
torch under a known-family pin was left in place and the pass was forced on every update.
Treat an unimportable torch as drift and reinstall the pinned trio (the spec and marker
derive from the pinned leaf, not the absent flavor); once it lands the probe succeeds and
the fast path returns.
Tests: exact-match cases across test_torch_flavor.sh, test_rocm_support.py,
test_cross_platform_parity.py and the two .ps1 helper suites; the rocm7.2 release-line
and broken-probe-reinstall cases; extraction lists updated for the new helpers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: anchor the PS pinned-ROCm floor gate and bound install.ps1 custom-pin companions
Round 12 made every family CLASSIFIER exact, but the Windows install-flow floor gate reads
$_pinRocm211 directly from the raw pinned leaf with an unanchored -match '^rocm(\d+)\.(\d+)'
BEFORE any exact classification runs. A suffixed custom leaf (rocm7.2-private) matches that
rocm7.2 prefix, so it takes the 2.11-floor branch and is force-routed through the ROCm
install path before the exact-match elseif can send it to the verbatim install. Anchor the
match ($) in both install.ps1 and setup.ps1 so only an exact rocmX.Y leaf is floored; a
suffixed or newer-suffix leaf falls through to the verbatim path. The Python floor
selection is already exact (dict lookups gated on _is_pip_rocm_family_leaf), so only the two
PS scripts needed this.
install.ps1's custom (non-cu-family) pinned-torch install bounded torch>=2.4,<2.11.0 but
left torchvision/torchaudio bare, so a private mirror serving newer companions could pull a
wheel built for a newer torch ABI while the marker records the pin as applied. Bound both
companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) when the leaf is not a
cu<digits> family index (a cu index bounds its own resolution), matching setup.ps1's
Test-CudaFamilyLeaf gate and _CUSTOM_INDEX_TORCH_PKG_SPEC.
Tests: parity guards for the anchored floor gate in both PS scripts and for install.ps1's
bounded custom-pin companions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten comments in the torch-index-override paths
Collapse the verbose comment and docstring blocks added across the installer
scripts and their tests to fewer, clearer lines without changing behaviour.
Remove a duplicated CUDA-spec comment block. Comments/docstrings only; no code
changes (AST-verified).
* install: repair a broken pinned torch on Linux, strip trailing slash in tauri family, count the final step
_ensure_cuda_torch / _ensure_cpu_torch returned on a failed import probe (torch present but
unimportable). With an explicit CUDA/CPU pin, _torch_pin_needs_apply forces the dependency
pass on that same failed probe, and the base package update does not force-reinstall an
already-installed torch distribution, so the broken torch was left in place and the pass
reran every update without repairing it. Treat a failed probe under a pin as drift and
reinstall from the pinned index (the reinstall rewrites the marker and the next probe
imports, so no loop). This is the Linux counterpart of the known-family repair fix.
_tauri_torch_index_family stripped the query/fragment before classifying but not a trailing
slash, so a token-authenticated pin like .../cu128/?token=x collapsed to .../cu128/ and fell
through the exact-suffix */cu128 and */cpu arms to "auto". Strip a trailing slash too,
mirroring _torch_index_url_leaf.
The Windows / macOS-ARM final torch-repair step (_ensure_pinned_known_family_torch) runs a
progress step that base_total never counted (the final-step increment was gated to Linux),
so _STEP ran one past _TOTAL on those platforms. Add the missing increment.
Tests: broken-probe reinstall for the CUDA (family and URL pins) and CPU paths; trailing
slash / slash+token cases for _tauri_torch_index_family; a full-flow progress-count guard
asserting _STEP == _TOTAL on Windows and Linux.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten comments in the torch-index-override paths
* install: harden the torch-index pin across all four installers
Redact index-URL credentials from captured install logs before they print on
failure. uv/pip failure text embeds the failing --index-url verbatim, so a
user:token@ or ?token= secret could leak into the console. Add a shared
redaction pass (_redact_install_output / Redact-InstallOutput) wired into the
error-output dump in install.sh, install.ps1, setup.ps1 and
install_python_stack.py. Verbose mode still streams live uncaptured output, so
it is intentionally left unredacted (developer opt-in).
Trim trailing slashes on the PATH only for a verbatim UNSLOTH_TORCH_INDEX_URL
override, preserving a ?query/#fragment token. A whole-URL rstrip corrupted a
base64 token ending in "/", and a single-slash strip left .../cu128//
classifying as an empty leaf. Add _trim_index_path_slashes /
Trim-IndexPathSlashes and route the override through it; strip ALL trailing
slashes in the backend-branding leaf classifier so a double slash still yields
the real leaf.
Reject a trailing-dot ROCm leaf (rocm7.) in the bash family validator so it
matches Python re.fullmatch(rocm\d+(?:\.\d+)?) and the PowerShell regex: both the
major and the minor must be non-empty digits, so rocm7. is a custom verbatim pin,
not a pip ROCm family.
Scrub PIP_NO_INDEX and PIP_INDEX_URL for a pinned install in the two installers
that have a plain-pip fallback (install_python_stack.py, setup.ps1):
PIP_NO_INDEX=1 makes the fallback ignore every index including the pinned
--index-url, and PIP_INDEX_URL replaces it. install.sh and install.ps1 install
via uv --default-index (which ignores pip config/env), so they are unaffected.
Add unit tests (bash, Python, PowerShell) and cross-platform parity tests
covering credential redaction, path-only slash trimming, the rocm7. validator,
the double-slash leaf, and the PIP_NO_INDEX/PIP_INDEX_URL scrub.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: redact captured torch-install output and warn on a failed pinned ROCm repair
Close a redaction gap the earlier pass missed: setup.ps1's direct
`Fast-Install ... | Out-String` branches (ROCm from $ROCmIndexUrl, CPU/CUDA from
$TorchInstallIndexUrl, plus the Triton and T5 sub-venv installs) printed the
captured $output verbatim on failure, bypassing Redact-InstallOutput. A private
index carrying userinfo or a ?token= in the pin could leak into Windows Studio
setup logs. Route every `Write-Host $output` through Redact-InstallOutput.
Warn on a failed pinned Windows ROCm reinstall in
_ensure_pinned_known_family_torch: the branch printed "reinstalling from it" then
called pip_install_try, but had no else, so a failure continued silently and left
the user believing the pin was applied while the old CPU/wrong torch survived.
Mirror the auto-ROCm Windows path and warn, telling the user to retry.
* install: redact captured output on the pip fallback and optional-install failure paths
The uv install path already redacted its captured output, but pip_install's pip
fallback runs through run(), which printed result.stdout verbatim on failure, and
_print_optional_install_failure did the same. A pinned --index-url carrying
userinfo or a ?token= could still leak there when uv is unavailable or the pip
fallback also fails. Route both through _redact_install_output. The verbose
pip_install_try path stays raw (developer opt-in), matching the other installers.
* install: split the survive-updates marker subsystem into a follow-up
The torch-index override PR grew a persisted per-venv marker plus repair
machinery (stale-pin detection, verbatim re-apply, update-time reinstall
triggers) that roughly doubled it. That subsystem is orthogonal to the core
feature and is being reworked in a follow-up (versioned/hashed marker,
full-URL pin baseline), so it moves there wholesale instead of shipping
twice.
What this PR still does: UNSLOTH_TORCH_INDEX_URL / UNSLOTH_TORCH_INDEX_FAMILY
pick the torch wheel index at install time in all four installers, with the
exact rocm/gfx/cpu/cu leaf classification, the torch 2.11 floor for the
per-arch AMD indexes, bounded companions for custom leaves, credential
redaction of captured installer output, path-only slash trimming, and the
uv/pip index env scrubs. Flavor-based repair keeps honoring the pin: a wrong
family under an explicit pin still reinstalls from the pinned URL, and
setup.ps1 repairs a pinned stale venv in place instead of wiping it.
What moves to the follow-up: the .unsloth-torch-index marker file and its
writers/readers/normalizers, exact-URL pin-change detection on update
(same-tag gfx switches, custom-mirror repoints), the verbatim trio snapshot
and clobber re-apply, the pin-baseline recorder, and the
--torch-pin-needs-apply fast-path probe in setup.sh / setup.ps1. Their tests
(the marker sh/ps1 suites, the stale-pin suite, and the marker classes in the
rocm/cuda/parity suites) move with them; the removed code is preserved on a
local archive branch to seed that PR.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: re-apply a ROCm pin over an existing HIP wheel via the version tag
The subsystem split left an explicit ROCm/gfx pin unenforced on `studio
update` whenever the venv already imported ANY ROCm torch: the pinned
reinstall lived inside the `elif not has_hip_torch` branch, so a rocm6.4 to
rocm7.2 switch, a gfx1151 pin over a generic +rocm7.2 wheel, or a broken
2.12+rocm7.2 drift never re-applied the pin.
Restore the markerless half of that detection: _rocm_pin_family_mismatch
compares the pinned leaf against the installed wheel tag (exact rocmX.Y
compare, the 2.11 gfx per-arch allowlist, the untagged-wheel rule), the HIP
probe emits "<hip_marker>|<version>" again so the installed tag is available,
and _ensure_rocm_torch reinstalls from the pinned URL when the tag mismatches
even though HIP torch is present. setup.ps1 mirrors it: the stale-venv check
routes a pinned rocm/gfx leaf through Get-RocmPinStaleTags instead of
collapsing it to a generic "rocm" flavor, and the existing pinned in-place
repair (no wipe) applies the change.
What still waits for the follow-up marker PR, by design: pin changes the
wheel tag cannot see -- a per-arch switch between two 2.11 gfx indexes
(identical +rocm7.13.0 tag), a custom-mirror URL repoint under the same
family leaf, and unknown-family verbatim pins. Those need the persisted
index record.
Tests restored with the code: the _rocm_pin_family_mismatch table, the five
update-path cases (older-rocm reinstall, gfx-over-pre-2.11 reinstall,
matching-pin no-reinstall, non-2.11 gfx no-reinstall, gfx-over-generic-2.11
reinstall), the "|" probe-format guards, and the AST-extracted
Get-RocmPinStaleTags suite for setup.ps1.
* install: compare major-only rocm pins, redact URL fragments, bound pinned CPU trio
Three review fixes on the restored pin-repair path.
The family classifier accepts a major-only rocm<d> leaf (rocm7), but the
mismatch comparators only parsed rocmX.Y, so a rocm7 pin fell through to the
2.11-line fallback and INVERTED both verdicts: an installed +rocm6.4 wheel
compared as satisfied (pin never re-applied) while a matching +rocm7.2 wheel
compared as stale (reinstall loop). Major-only pins now compare on the major
alone in _rocm_pin_family_mismatch and Get-RocmPinStaleTags: rocm6.x under a
rocm7 pin is a mismatch, any rocm7.x satisfies it, an untagged wheel never
does, and a bare +rocm tag with an unreadable version is accepted (matching
the existing lenient unreadable fallback).
The output redactors scrubbed userinfo and ?query= values but not #fragments,
so a pin like https://mirror/whl/cu128#token=secret leaked the secret in
captured uv/pip failure text -- inconsistent with the URL handling itself,
which already treats fragments as sensitive. All four redactors gain a
URL-anchored fragment rule (anchored so a bare "# comment" line in tool
output is never touched).
setup.ps1's CPU branch installed a bare torch/torchvision/torchaudio trio;
fine for the unpinned host default, but a PINNED cpu index routes through the
same branch and the /cpu index serves newer torch, so a fresh pinned CPU
install could land an unsupported trio that _ensure_cpu_torch then keeps
(it accepts any CPU build). Under a pin the branch now installs the bounded
trio mirroring _CPU_TORCH_PKG_SPEC (torch>=2.4,<2.12.0 and matching
companions); the unpinned path is unchanged.
Tests: major-only rows in the Python mismatch table and the AST-extracted
setup.ps1 suite; fragment + query-plus-fragment + bare-hash-comment cases in
all four redactor suites; a parity check that the pinned CPU trio bounds
exist, are gated on the pin, and mirror the Python repair spec.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tighten comments in the torch index override paths
* tests: track the moved pass-through inheritance in the gguf order check
Main moved the llama_extra_args pass-through inheritance out of the
GGUF branch into _resolve_inherited_extra_args, which runs before it,
so the source-order assertion's "if request.llama_extra_args is None"
anchor no longer exists inside the branch and the check failed after
the main merge. The test now asserts the same property in the current
shape: inheritance before the GGUF branch (a carried --no-mmproj still
shapes the hub guard's companion requirement), and marker, hub guard,
unload in order within the branch. Full file passes (32 tests).
* tests: anchor the inheritance order check on the call, not the definition
source.index("_resolve_inherited_extra_args(") matched the function
definition, which always precedes the endpoint, so the ordering
assertion was vacuously true. Anchoring on "= _resolve_inherited_
extra_args(" pins the first call site inside the load endpoint (line
4505), which is the statement whose position relative to the GGUF
branch the test is meant to guard. 32 tests pass.
* tests: align the gguf order test with main
Main fixed the stale ordering assertion in PR 7252; adopting its
version verbatim removes this file from the branch diff entirely and
avoids a conflict on the next main merge. 32 tests pass.
* install: bound the companion constraints to torch's window everywhere
A full platform x vendor validation matrix over this branch surfaced a
real trio mismatch on the cpu/mac paths: torch is capped <2.11 (installs
2.10.0+cpu) but the bare torchaudio companion resolves 2.11.0+cpu,
because torchaudio 2.11 dropped its exact torch pin. Reproduced in a
sandboxed end to end cpu install. torchvision still exact-pins torch and
self-corrected.
The default companion constraints are now bounded to torch's window
(<0.26 / <2.11) and widen together with the cu* torch window (<0.27 /
<2.12), so every leaf resolves a paired trio. Verified with uv dry-runs
on the cpu, cu130, and rocm6.4 leaves (2.10.0/0.25.0/2.10.0,
2.11.0/0.26.0/2.11.0, 2.9.1/0.24.1/2.9.1) and a rerun of the sandboxed
cpu install, which now lands torch 2.10.0+cpu with torchaudio
2.10.0+cpu.
The Strix WSL reroute now also forwards UNSLOTH_TORCH_INDEX_URL and
UNSLOTH_TORCH_INDEX_FAMILY into the rerouted 24.04 distro; dropping
them silently reverted the child install to auto-detection, defeating
the pin this branch introduces.
test_torch_constraint.sh updated: the bounded companions must appear at
the defaults and the custom-leaf block, no bare companion may remain,
and the cu* widen must carry the companions with it.
* install: harden the override path against reroute drift and credential leaks
Review sweep focused on default-path idempotency found no defects on the
unset path; these fixes cover the override path and failure reporting.
install.sh:
- The early WSL Strix Halo distro reroute now honors an explicit index
pin (UNSLOTH_TORCH_INDEX_URL / _FAMILY): the pin is used in the current
distro instead of probing the GPU and re-entering another distribution,
matching the contract of the later Radeon and Strix guards. Whitespace
only values do not gate, in parity with get_torch_index_url.
- Verbose mode now streams installer output through the credential
redactor; it previously bypassed the redaction the quiet path applies.
The exit code survives the pipe via an rc file since the script runs
under plain sh with no pipefail.
- The kept-release fallback warning now strips credentials from the
index URL before printing it.
install.ps1:
- Bounded torchvision and torchaudio next to every capped torch install
(custom pin, ROCm CPU fallback, CUDA flavor repair). torchaudio 2.11
dropped its exact torch pin from the wheel metadata, so a bare
companion beside torch<2.11 can resolve a mismatched 2.11.0 build,
cu family indexes included. Mirrors the install.sh companion bounds.
studio/install_python_stack.py:
- The verbose failure path now redacts index URLs in pip and uv output
before printing, matching every other output site in the file.
All sh, ps1 and python installer test suites pass (the host-defaults
suite has a known pre-existing failure unrelated to this change).
* install: redact verbose Windows installer output and repair the parity tests
Follow-ups to the override-hardening commit, from review:
- install.ps1 Invoke-InstallCommand and setup.ps1 Invoke-SetupCommand now
pipe verbose output through Redact-InstallOutput per record, and the
three verbose Fast-Install torch call sites (ROCm, CPU, CUDA) do the
same: uv and pip echo the pinned index URL, credentials included, in
their errors, and verbose mode previously bypassed the redaction the
quiet paths apply. ForEach-Object and Out-Host leave $LASTEXITCODE
untouched, verified with a native command exiting 7 behind the pipe.
- test_cross_platform_parity.py: the install.ps1 companion-bounds
assertion now matches the implemented behavior (bounds on every index,
no cu-family exemption, since torchaudio 2.11 dropped its exact torch
pin) instead of requiring the removed $_pinCuLeaf gate.
- test_rocm_support.py: the WSL reroute guard test slices the whole
function body to its closing brace instead of a fixed 1200-character
window, which the new pin-gate preamble had outgrown.
428 tests pass across the parity, install stack and rocm support suites;
the sh and ps1 installer suites pass unchanged.
* install: tighten comments in the torch-index and ROCm/CUDA repair paths
* install: digit-gate the gfx family leaf and honor ROCm pins in the Windows repair
Two review follow-ups on the override path:
- The pip ROCm family predicate accepted ANY gfx-prefixed leaf, so a
custom verbatim pin like /gfx-private classified as a ROCm family and
enabled the ROCm-only side effects (AMD bitsandbytes, ROCm torch
repair) on a mirror that may serve CPU/CUDA wheels. gfx now requires a
following digit (gfx90a, gfx1151, gfx120X-all), consistently in
install.sh, install_python_stack.py, install.ps1 (family gate and
expected-flavor classifier) and setup.ps1, matching the strictness the
rocm side already had (rocm7.2-private stays verbatim). The broader
backend BRANDING globs are unchanged on purpose: radeon repo leaves
(rocm-rel-X.Y) must still brand the rocm backend without being
force-repaired as a family.
- The Windows branch of the ROCm torch repair always installed from the
public per-arch index, ignoring an explicit ROCm-family pin: after a
pinned setup.ps1 install failed to a CPU base, the repair retried
repo.amd.com instead of the pinned index. The branch now resolves
_explicit_rocm_torch_index_url() first, uses it as the install index
when set, and mirrors the Linux pin contract by skipping the NVIDIA
and gfx-detection gates a pin is documented to override.
Source-assertion tests updated to the tightened predicate and the new
repair label. 1165 tests pass across the parity, install stack and
studio install suites; the sh and ps1 suites pass; both PowerShell
installers parse clean.
* Remove scratch archives accidentally committed with the comment pass
The temp/ archive copies of installer and test files were working
scratch, not PR content, and inflated the diff by about nine thousand
lines.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
Naitik Pal
|
cf912cbd88
|
feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var to force CPU fallback #7213 (#7228)
* test(studio): add e2e test for cpu-fallback overriding vulkan * feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var * feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var * Preserve UNSLOTH_LLAMA_CPP_BACKEND=cpu across llama.cpp updates for PR #7228 The in-app updater rebuilt the installer command without --cpu-fallback and only re-asserted Vulkan, so accepting a llama.cpp update after forcing CPU on an Intel iGPU host re-ran host detection and routed back to the crashing Vulkan bundle (#7213). Record install_kind in the prebuilt marker and re-assert --cpu-fallback on update when the installed bundle is CPU. Also make setup.sh's UNSLOTH_LLAMA_CPP_BACKEND check case-insensitive to match setup.ps1, and add tests for the updater CPU preservation and the setup.sh flag plumbing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim and validate UNSLOTH_LLAMA_CPP_BACKEND, warn on unknown values for PR #7228 Trim surrounding whitespace and lowercase the value in both setup.sh and setup.ps1, so values like ' cpu ' or 'CPU' still force the CPU-only prebuilt. An unrecognized value (e.g. 'gpu') now prints a warning instead of silently falling back to auto. Extend test_setup_llama_cpp_backend.py to cover both scripts, including trimmed, empty and unknown values. * Preserve arm64 CPU installs on update and honor CPU override in Windows prune for PR #7228 The update-path CPU preservation only matched install_kind ending in -cpu, so arm64 CPU bundles (linux-arm64, windows-arm64) were re-routed to a GPU or source build on update. Match the full set of CPU-only kinds instead. Persisting install_kind also activated the previously inert Windows mismatch-prune in setup.ps1: on a GPU host with UNSLOTH_LLAMA_CPP_BACKEND=cpu it saw the windows-cpu marker as mismatched and deleted it every rerun. Normalize the override once and make CPU expected so a deliberate CPU install is kept. Extend the tests to cover both. * Document legacy llama.cpp markers keep heal-to-GPU on update for PR #7228 Legacy prebuilt markers written before install_kind was persisted intentionally do not force --cpu-fallback on update: the in-app updater lets them re-resolve (heal to a GPU bundle) per the existing behavior from #6097, and only markers that explicitly record a CPU install_kind are pinned to CPU. Add a comment and a regression case documenting the boundary. * Tighten llama.cpp CPU-fallback comments for PR #7228 * Fix Windows install-prune to keep valid Intel/fallback bundles for PR #7228 Persisting install_kind activated the setup.ps1 mismatch-prune, whose expectedKinds was incomplete: the non-NVIDIA/non-AMD branch omitted windows-vulkan (the Intel auto-route) and the GPU branches omitted the windows-cpu/windows-arm64 fallback the installer uses when a GPU prebuilt is missing. That made every setup rerun delete and re-download a valid Intel Vulkan (or CPU-fallback) install. List all kinds the installer can produce per host so only a bundle the host cannot run is pruned. Cover the full matrix in tests. * Persist force_cpu marker flag so only forced CPU installs re-assert on update for PR #7228 * Add --force-cpu for deliberate CPU installs and warn on macOS for PR #7228 * Record force_cpu when reusing a matching CPU bundle for PR #7228 * Accept force_cpu keyword in installer test validator fakes for PR #7228 --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Michael Han
|
6d8c18cd1a
|
Replace standalone Studio wording with Unsloth (#7221)
* Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error. |
||
|
Leo Borcherding
|
91a0df9514
|
Studio: make the Cloudflare tunnel opt-in (off by default) (#7046)
* Studio: make the Cloudflare tunnel opt-in (off by default) A wildcard bind (`-H 0.0.0.0`) auto-started a public trycloudflare.com tunnel, so exposing Studio on the LAN also published it to the public internet. Flip the default so the tunnel is opt-in. - `--cloudflare` is now tri-state (Optional[bool], default None = off), mirroring the existing --enable-tools/--disable-tools handling. Pass --cloudflare to expose a public HTTPS link for a wildcard bind; --secure still implies the tunnel. - --secure + --no-cloudflare is still rejected as a contradiction. - Update the parent-command guard, re-exec forwarding, startup-banner wording, the colab comment, README, and tests. * Studio: update installer/setup launch hints for opt-in Cloudflare The post-install launch hints only mentioned --secure for a public link. Now that the tunnel is opt-in, clarify that -H 0.0.0.0 exposes the raw port on the LAN (not a public URL), and surface --cloudflare as the explicit opt-in for a public HTTPS link (--secure keeps the raw port private). Applied to install.ps1, install.sh, and studio/setup.sh. * Studio: address review - keep cloudflare tri-state + harden run re-exec Two review points from the bots: - Gemini: keep `cloudflare` as Optional[bool] in run_server instead of casting None -> False, so the startup banner can distinguish "OFF (default)" (unset) from "OFF (--no-cloudflare)" (explicit). `_cloudflare_flag` and the banner branch now carry the tri-state. - Codex (P1): `unsloth studio run` re-execs the studio venv's console script, which can be an older build whose --cloudflare defaulted on; omitting the flag let it re-enable the tunnel. That path now forwards the default polarity explicitly (--no-cloudflare, or nothing under --secure since --secure implies the tunnel). The plain `unsloth studio` path runs the same-version in-tree run.py (resolved via _find_run_py), so it keeps forwarding only an explicit polarity and still shows the accurate "(default)" banner. Tests updated for the tri-state banner labels, the None gate cases, and the new re-exec forwarding. * Studio: forward --no-cloudflare on plain re-exec too (mixed install) Codex follow-up: _find_run_py falls back to STUDIO_HOME/.../studio/backend/ run.py when the package copy is absent, so the plain `unsloth studio` re-exec can land on an older run.py whose --cloudflare defaults on. Forward the default polarity explicitly there too (--no-cloudflare, or nothing under --secure), matching the run subcommand. The common in-venv launch skips the re-exec and still shows the tri-state "(default)" banner. * Studio: fix launch hint - --cloudflare needs the wildcard bind Codex P3: the launch hint listed --cloudflare next to the loopback `unsloth studio -p 8888` command, but the tunnel only starts for wildcard binds, so `--cloudflare` alone on 127.0.0.1 does nothing. Show `-H 0.0.0.0 --cloudflare` in the hints (install.ps1, install.sh, studio/setup.sh) and clarify the same in the README. * Studio: cross-platform masked terminal password prompt helper Per-keystroke '*' echo (POSIX termios cbreak / Windows msvcrt.getwch), backspace editing, Ctrl-C abort, EOF handling, confirmation loop with re-prompt on mismatch or policy failure. Pure should_prompt gate for the --secure/--cloudflare exposure paths. * Studio CLI: force a terminal password change before public tunnel exposure When a launch will start the Cloudflare tunnel (--secure, or --cloudflare on a non-api-only wildcard bind) and the admin account still has its seeded bootstrap password, prompt for a new password in the terminal (masked with '*', confirmed, re-prompting until valid) before any re-exec or server exists. The change is committed in the parent so it never crosses argv or the environment and older studio-venv children see it immediately. Without a terminal, warn and fall back to the backend bootstrap shutdown timer. Mirrors backend update_password semantics in one transaction: rehash, rotate the JWT secret, clear must_change_password, revoke refresh tokens, drop the desktop secret, then remove the stale credential files. * Studio: terminal password gate before the public tunnel (backend backstop) Never publish a trycloudflare URL while the seeded admin password is active: run_server now runs a terminal password-change gate after the tunnel decision and strictly before start_studio_tunnel. Interactive refusal fails closed (shutdown + exit 1, mirroring the secure gate); without a tty it warns and keeps the bootstrap deadline. Success applies the same effects as the change-password route (update_password + revoke_user_refresh_tokens) and drops the stale app.state.bootstrap_password. MIN_PASSWORD_LENGTH centralised in auth/storage.py and referenced by the HTTP schema. terminal_prompt.py carries the pure gate helper (interactive loop stubbed; supplied by the masked-input module). Also migrates the studio/setup.ps1 launch footer that still showed the bare wildcard hint. * README: reconcile remote-access section with opt-in Cloudflare tunnel * Studio: harden the terminal password gate after review - run.py: run the gate BEFORE the uvicorn socket binds. On a wildcard --cloudflare launch the served HTML injects the bootstrap credential for first login, so a pre-gate listener would hand the default password to anyone who reaches the raw port while the operator is still typing. The gate now also seeds the admin row itself (it can run before lifespan startup). - Headless launches that nothing would protect now fail closed: the bootstrap deadline never arms for api-only serving and UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT=0 disables it, so warn-and-proceed would have promised a shutdown that never comes. Both the CLI and the backend refuse to publish in that case; the ordinary headless path still warns and relies on the 1h deadline, and no longer auto-fills the default credential into HTML served on a public URL. - storage.update_password gains revoke_refresh_tokens to delete the user's refresh tokens in the SAME transaction as the password commit; the change-password route and the backend gate use it (a separable follow-up delete could fail after the commit and leave a stale refresh token able to mint access tokens under the rotated secret). - clear_bootstrap_password is best-effort: a locked/undeletable file must not surface as a failed password change. - CLI masked reader: disable ISIG like the backend so Ctrl-Z cannot suspend the process with the shared terminal stuck in no-echo mode; handle Ctrl-C/Ctrl-Z as characters; treat stream EOF mid-line as an abort instead of submitting a partial password. Both readers restore terminal attrs from a SIGTERM/SIGHUP handler since a finally block cannot run when a default-disposition signal terminates the process. - Backend reader: decode byte-at-a-time through an incremental UTF-8 decoder so multi-byte characters split across read boundaries are no longer dropped; isatty checks tolerate closed/None streams. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: persist bootstrap suppression through lifespan startup The pre-bind password gate nulled app.state.bootstrap_password, but the FastAPI lifespan runs after it and re-reads the bootstrap password into app.state on both admin paths, so a headless public launch could still serve the injected credential in HTML. Carry a persistent suppress_bootstrap_injection flag that the lifespan honors instead. Also drop the quoted Tuple annotation on _terminal_password_gate that tripped the import-hoist lint (the typing import looked unused). * Studio CLI: keep the pre-exec auth DB private (0700 dir, 0600 db) On a fresh install the pre-exposure password gate creates auth/ and auth.db through the CLI before the backend ever runs, and sqlite3.connect leaves the DB 0644 under a 022 umask. Mirror backend storage.get_connection's chmod so the committed password hash and JWT secret are never world-readable, even if the launch aborts before the backend applies its own modes. * Tighten pre-exposure password gate comments * Studio: delete seeded bootstrap password before headless public re-exec The headless warn-and-proceed path returns with the default admin password still active, then re-execs a child Studio process. An old studio-venv child (mixed-version install) predates the pre-bind gate and its injection-suppress flag, so its lifespan reads .bootstrap_password and injects the seeded credential into the public HTML for up to the bootstrap deadline. A CLI-flag handshake cannot fix this uniformly: the studio run path uses ignore_unknown_options and an old in-venv child runs in-process, so it would never reject the flag. Delete the seeded .bootstrap_password file in the parent before re-exec so a fresh child of any version reads None and never serves it. This covers both re-exec paths and both child versions. must_change_password stays set, so the login page still forces a change and the bootstrap shutdown timer still arms; only the plaintext-on-disk copy is removed. Recovery is via a terminal-attached run or reset-password. Backend gate and CLI warnings updated to match. * Studio: commit the seeded admin before headless public re-exec The headless-warn path deletes the seeded .bootstrap_password so a re-exec'd child cannot inject it, but _ensure_cli_default_admin's INSERT was never committed and rolled back on conn.close(). On a fresh STUDIO_HOME an old studio-venv child then found no admin, regenerated a fresh bootstrap password + file, and injected THAT into the public page, defeating the deletion. Commit the seeded admin right after _ensure_cli_default_admin so any re-exec'd child sees the existing account and does not regenerate. Regression tests cover both re-exec paths on a fresh (unseeded) DB. * Studio: fail closed when the bootstrap password file cannot be removed On the headless public path, deleting .bootstrap_password is the protection against an old re-exec'd child injecting the seeded credential. If unlink fails (locked file, read-only auth dir) the file is still on disk, so warning and proceeding would still leak it for the bootstrap-timeout window. Abort with a clear error instead. Regression test covers the unlink-failure fail-closed path. * Studio: hold no-echo for the whole password line, not per keystroke The POSIX masked reader set cbreak/no-echo inside _getch_posix and restored the terminal to echo-on in a finally after every single keystroke, because _read_password calls _getch once per character. Between one char returning and the next call re-entering cbreak, ECHO was on, so a keystroke arriving in that window echoed the password in cleartext. Move the terminal mode into a _prompt_raw_mode context that _read_password holds around the entire line (mirroring unsloth_cli/commands/_password_prompt.py, which already did this), restoring once when the line completes or aborts. _getch_posix now only reads, since the mode is held by the caller. The context is a no-op when stdin is not a real terminal, keeping the _getch test seam. Add a regression test asserting the raw-mode context wraps the read exactly once and every keystroke is read while it is active. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: strip the seeded bootstrap password when the auth DB check fails The pre-exposure gate returned early on two auth-DB inspection failures and proceeded to re-exec without removing the seeded .bootstrap_password: - _connect_auth_db() failure: a seeded credential from a prior run may still be on disk. - the must_change_password read-back failure: worse, _ensure_cli_default_admin had already seeded the admin and the code committed it (writing .bootstrap_password) right before the failing SELECT. In the mixed-version case (a new outer CLI re-execing an old studio-venv child that predates the pre-bind gate), that child would read the file back and inject the default admin credential into the public Cloudflare page. The sibling headless branch already deletes the file for exactly this reason, so these returns were an inconsistent gap. Factor the delete-or-fail-closed logic into _strip_seeded_bootstrap_password_or_exit and call it on both inspection failures (and reuse it in the headless branch): strip the seeded file first (version-independent protection), failing closed if the removal itself fails. must_change_password stays set, so the login page still forces a change and the bootstrap shutdown timer still arms. Add tests for both new paths (connect failure and post-commit read-back failure strip the file and proceed; a failed strip fails closed). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fail closed when the seeded admin cannot be committed before exposure The pre-exposure gate wrapped _ensure_cli_default_admin (the INSERT), its conn.commit(), and the must_change_password read-back in one try, and the except recovered by stripping .bootstrap_password and proceeding to re-exec on the assumption the admin was already committed. That assumption only holds when the failing statement is the SELECT. When the INSERT or the commit itself fails (e.g. a write lock held past the busy timeout on a fresh install), no admin row is committed: it rolls back on conn.close(), and a re-exec'd old studio-venv child (no pre-bind gate) then finds no admin, regenerates a fresh bootstrap password + file, and serves that default credential on the public Cloudflare page. Stripping the file cannot stop a regeneration. Split the seed+commit into its own try that fails closed (refuse the public launch, best-effort removing any half-written seed file) since we cannot prove a committed admin; keep the separate read-back failure on the strip-and-proceed path, where the admin is committed so an old child finds it and will not regenerate. Add a test for the seed-commit-failure path. * Studio: decode the CLI masked password reader with errors="replace" The CLI reader read keystrokes with text-mode sys.stdin.read(1), which raises UnicodeDecodeError on a pasted non-UTF-8 password (e.g. Latin-1 bytes), or under PYTHONUTF8 yields a lone surrogate that later crashes the pbkdf2 encode -- either aborts the launch with a traceback. The backend mirror (terminal_prompt.py) already reads raw bytes through an incremental decoder with errors="replace". Mirror that here: read with os.read and an incremental decoder so invalid bytes map to U+FFFD, iterating over each emitted char (one byte can complete a replacement plus the next char). * Studio: resolve the child launcher before the pre-exposure gate The gate strips the seeded .bootstrap_password on a headless public launch, and it ran before the re-exec launchability check (studio venv / run.py / console script present). So a headless launch with an incomplete studio setup would seed the admin, delete the bootstrap password, then abort because the child could not be found, leaving the admin at must_change_password=1 with no password ever shown or injectable: locked out until `unsloth studio reset-password`. Resolve and validate the child launcher first, in both `studio` (studio_default) and `studio run`, and only then run the gate, so an unlaunchable setup exits before anything is stripped. Add a regression test that a missing venv exits without removing the seeded file. * Studio: fail closed when the auth DB cannot be opened before exposure The connect-failure branch of the pre-exposure gate stripped .bootstrap_password and proceeded, on the assumption a committed admin from a prior run made an old child find it and not regenerate. But on a fresh public launch whose _connect_auth_db() itself fails (transient lock during the schema/seed step, or an unwritable home), no admin is committed, so a mixed-version re-exec child that predates the backend gate can find no user, generate a fresh bootstrap password, and serve it on the public Cloudflare page. Stripping a file we cannot vouch for cannot stop a regeneration. Make this branch fail closed like the seed/commit failure path: we only continue past the DB inspection once a committed admin is confirmed. The existing file is left untouched so a retry (after a transient lock clears) can still prompt. Update the connect-failure test to assert fail-closed, and give the in-venv --secure flag test a real STUDIO_HOME with an already-changed admin so the gate is a no-op rather than relying on a DB-open failure. * Studio: invalidate seeded bootstrap files before deleting auth.db on reset reset-password deleted auth.db first, then best-effort unlinked the seeded .bootstrap_password and desktop secret. unlink() only ignores FileNotFoundError, so a locked or read-only file (Windows AV, read-only auth dir) survived while auth.db was gone. The next server start then re-seeded from that stale plaintext and re-validated the exact credential the reset was meant to revoke. Invalidate the credential files first, truncating any that cannot be unlinked, then delete the DB, so a surviving file can never carry a reusable secret. clear_bootstrap_password now truncates on unlink failure for the same reason, and its warning says the contents were cleared rather than claiming the stale password is already invalid. * Studio: require a servable frontend before the pre-exposure gate can strip the seeded password A headless public launch strips the seeded .bootstrap_password before the re-exec'd child starts. If the child then cannot serve the login page (the only in-band way to change the seeded password) the admin is locked out (must_change_password=1, no file, no UI) until reset-password. Add _require_servable_frontend_or_exit and call it before the gate on both `unsloth studio` and `unsloth studio run` public launches: fail closed if a non-api-only public launch has no built frontend dist, before anything is stripped. A user-supplied --frontend is validated to contain index.html so a bad path cannot silently bypass the check; an auto-resolved dist is trusted (_find_frontend_dist already requires index.html) and forwarded to the child. Model-load aborts on `studio run` remain a residual: the parent must strip for mixed-version safety (an old studio-venv child has no pre-bind gate) and model loadability cannot be proven before exec, so that path stays recoverable via reset-password. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden reset-password ordering and validate the in-venv backend before the strip Three follow-ups to the pre-exposure hardening: reset-password now deletes auth.db FIRST and proves it is gone before touching the seeded credential files. If the DB cannot be removed (a running Studio or Windows holds it open, or a read-only auth dir) it aborts with the credential files untouched, so a forgotten-password reset is not left half-done with the recovery credentials deleted while an un-resettable must_change_password=1 DB survives. After the DB is gone it invalidates the stale credential files (unlink, else truncate) and fails closed if a file can be neither removed nor truncated, since a surviving plaintext would be re-seeded by generate_bootstrap_password() and re-validate the revoked password. The in-venv (in-process) launch path had no analogue of the re-exec launcher check: a headless public launch would seed the admin and strip the seeded .bootstrap_password in the gate before _load_run_module() later failed on a broken/partial venv, leaving must_change_password=1 with no password to log in. Add _validate_inproc_backend_before_strip, called on the in-venv path (both `unsloth studio` and `unsloth studio run`) before the gate on the headless public path, so a broken backend fails cleanly before anything is stripped. It is scoped to the headless path so an interactive prompt is not delayed behind a full backend import. * Studio: validate the frontend and tunnel before the strip on every public path Five follow-ups closing the remaining pre-exposure-strip lockouts: The in-venv (in-process) paths of both `unsloth studio` and `unsloth studio run` validated the backend but not the frontend before the gate, so a headless public launch with a missing/bad dist would strip the seeded .bootstrap_password and then abort in run_server() during frontend setup, leaving must_change_password=1 with no login page. Both now validate a servable frontend before the strip (cheap check first, backend import after) and serve the resolved dist in-process. The `studio run` re-exec discarded the dist that satisfied the pre-strip check and only forwarded a user-supplied --frontend. In a shadowed install where the parent finds a built dist the child cannot, it stripped and exec'd without the path, and the child aborted during frontend setup. It now forwards the resolved dist, matching `unsloth studio`. On a headless --secure launch the bind is loopback, so the Cloudflare tunnel is the only public exposure. If cloudflared is provably unavailable (found nowhere and undownloadable) the tunnel cannot start, so stripping the recovery credential would just lock the user out with no public URL ever served. Add _tunnel_binary_confirmed_unavailable and, on --secure only, refuse the launch with the credential preserved rather than strip. Wildcard --cloudflare binds 0.0.0.0 publicly regardless of the tunnel, so it still strips; any uncertainty (helper not loadable) also still strips, since a possible credential leak outweighs a recoverable lockout. clear_bootstrap_password no longer claims it cleared the file's contents when both unlink and truncate failed; it now reports the stale password is still on disk and asks the user to remove it manually. * Studio: fix cloudflared probe path and skip the bootstrap strip for a self-suppressing child Two follow-ups to the --secure pre-exposure hardening: The cloudflared availability probe loaded cloudflare_tunnel by file path but not its backend deps: ensure_cloudflared() -> _cache_path() lazily imports utils.paths.storage_roots, which only resolves when studio/backend is on sys.path. From the outer CLI it is not, so the probe saw ensure_cloudflared() return None (cache unresolvable) and wrongly treated the tunnel as unavailable, refusing --secure even when cloudflared was cached or downloadable. Add the backend dir to sys.path for the probe (and remove it after) so the cache path resolves as it will in the child. A headless --secure launch stripped the seeded .bootstrap_password before the child proved the tunnel could actually connect, so a cloudflared that is present but cannot establish the tunnel (blocked connectivity, Cloudflare outage) left must_change_password=1 with no recovery credential. But the strip is only needed when the re-exec'd child is an OLD studio-venv backend with no pre-bind suppression: this install's own run.py sets app.state.suppress_bootstrap_injection before binding and never serves the seeded credential publicly. Add _child_self_suppresses (true in-process, or when the re-exec target is this install's own run.py by path identity) and skip the strip in that case, keeping .bootstrap_password as a local recovery credential; the strip stays fully in force for the studio-venv console-script path and any venv-fallback run.py, where an old child is actually possible. * Studio: reword the pre-exposure terminal password prompt * Studio: warn when -H is overridden by --secure; align pre-exposure prompt wording - --secure/--secure run: emit a Note (not an error) when -H is a non-loopback host, since --secure forces the loopback bind and would otherwise discard -H silently. - Reword the pre-exposure terminal prompt to 'exposed on the public internet' in both the backend gate and the CLI mirror. - Align the CLI success line with the backend ("Password updated for '<user>'."). - Tests for the new -H warning (present when overridden, absent on loopback). * Studio: add non-interactive --password to set the initial admin password Headless hosts (CI, containers, systemd units) have no TTY, so the forced first-exposure password change could not be completed unattended. Add a non-interactive way to set the INITIAL admin password before the server binds: - --password <value>, the UNSLOTH_STUDIO_PASSWORD env var, or --password - (read one line from stdin). Off by default; unset falls back to the normal interactive terminal prompt / browser setup. - Applies on any launch (public --secure/--cloudflare or a headless -H 0.0.0.0 bind), only when the account still has its seeded bootstrap password. An already-set password is a hard error, never an override; an invalid value (too short, or equal to the bootstrap) fails closed before bind. - The CLI applies the change in the parent, never forwards --password to the re-exec child, and strips UNSLOTH_STUDIO_PASSWORD from the child env so the secret never crosses to the child. run.py does the same on the direct path and strips the env var so spawned subprocesses (cloudflared, llama-server, tools) cannot inherit it. Mirrors resolve_supplied_password across the CLI and backend, documents the option in the README (including the argv-visibility caveat), and covers all flows (env/stdin/literal, fail-closed cases, no-forward, env-strip, reset-password roundtrip) in the CLI, backend, and unit suites. * Studio: truncate the stale bootstrap file when unlink fails on a CLI password change The post-change cleanup in _cli_update_password only warned when .bootstrap_password could not be unlinked but was still writable (locked file, read-only auth dir), leaving the old plaintext on disk. If auth.db is later reset or removed, generate_bootstrap_password() reads that file back and re-validates the revoked bootstrap password. Truncate the file on unlink failure so its stale plaintext cannot be re-seeded, mirroring the backend clear_bootstrap_password(); the password change is already committed, so this never rolls it back. The warning now states truthfully whether the contents were cleared or the file must be removed manually. * Studio: tighten comments --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
c570180a32
|
Tighten Studio instruction-file cleanup boundaries (#7097)
Some checks failed
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio UI CI / Chat UI Tests (push) Waiting to run
Studio Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Studio UI CI / Chat UI Tests (push) Waiting to run
Studio Update CI / Studio Updating Tests (push) Waiting to run
Windows Studio API CI / Studio API & Auth Tests (push) Waiting to run
Windows Studio GGUF CI / VC++ runtime detect + install round-trip (windows-2025-vs2026) (push) Waiting to run
Windows Studio GGUF CI / VC++ runtime detect + install round-trip (windows-latest) (push) Waiting to run
Windows Studio UI CI / Chat UI Tests (push) Waiting to run
Windows Studio Update CI / Studio Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Mac Studio API CI / Studio API & Auth Tests (push) Waiting to run
Mac Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Mac Studio GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio GGUF CI / JSON, images (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-14) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Update CI / Studio Updating Tests (push) Waiting to run
Windows Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Windows Studio GGUF CI / Tool calling Tests (push) Waiting to run
Windows Studio GGUF CI / JSON, images (push) Waiting to run
Windows Studio GGUF CI / Studio install + inference without Visual Studio (push) Waiting to run
Windows Studio GGUF CI / GPU prebuilt resolves without Visual Studio (push) Waiting to run
Windows Studio GGUF CI / setup.ps1 unit tests (VS 2026 / CMake guard) (push) Waiting to run
Windows Studio GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Studio GGUF CI / real-VS detection (VS 2026) (push) Waiting to run
Cross-platform parity / parity (windows-latest) (push) Has been cancelled
Cross-platform parity / parity (macos-latest) (push) Has been cancelled
* Handle linked instruction files in Bash cleanup * Limit instruction cleanup to managed dependencies * Make Bash cleanup test portable * Run junction cleanup regression on Windows * Keep instruction cleanup CI focused |
||
|
Daniel Han
|
9e77c1e663
|
Studio: remove AGENTS.md and CLAUDE.md from install artifacts (#7096)
* Studio: remove AGENTS.md from install artifacts * Studio: prune CLAUDE.md from install artifacts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Studio instruction cleanup edge cases * Trim Studio cleanup comments * Make Studio cleanup safe on PowerShell 5.1 * Fix Studio cleanup ownership boundaries * Simplify Windows link detection --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
alkinun
|
216a1fad33
|
Fix Windows installer torch index override (#6972)
* Fix Windows installer torch index override * Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898) * Harden setup.ps1 index-var clearing to truly remove vars (#6898) * Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898) * Neutralize all uv index env vars for pinned torch installs (#6898) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
oobabooga
|
fcb1152c76
|
Studio: source CPU llama.cpp prebuilts from unslothai/llama.cpp (#6311)
* Studio: source CPU llama.cpp prebuilts from the unslothai fork * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt * Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt * Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments * Studio: correct stale fork-routing comments and --resolve-prebuilt help * Refresh stale ggml-org routing comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |