70 commits

Author	SHA1	Message	Date
karan yadav	dc26127a42	Studio: propagate required backend version to repair pipeline (#8610) (#8670) ... * Studio: propagate required backend version to repair pipeline (#8610) Fix second-launch infinite repair loop when installed backend version is outdated (#8610). When the Desktop App launches with an installed managed venv whose version is older than expected_backend_version() (e.g. 2026.8.4 < 2026.8.15), preflight flags the install as ManagedStale with desktop_backend_version_outdated. On auto-repair, unsloth studio update ran setup.sh/setup.ps1 from the old venv, which skipped python dependency installation because INSTALLED_VER == LATEST_VER on PyPI or PyPI timeout, leaving the venv unchanged. The installer fallback (install.sh/install.ps1) also lacked version floor pins on standard fresh paths, locking the user in a permanent repair error loop. Key changes: - Pass UNSLOTH_DESKTOP_BACKEND_VERSION from Tauri (update.rs & install.rs) to child process environments. - Force Python dependency pass in setup.sh & setup.ps1 when UNSLOTH_DESKTOP_BACKEND_VERSION is set and INSTALLED_VER < UNSLOTH_DESKTOP_BACKEND_VERSION. - Apply UNSLOTH_DESKTOP_BACKEND_VERSION floor constraint when updating core packages in install_python_stack.py. - Ensure standard fresh install paths in install.sh and install.ps1 use "unsloth>=2026.8.15". - Bump MIN_DESKTOP_BACKEND_VERSION in preflight/version.rs to "2026.8.15". - Add shell and Python unit tests for the fast-path escape and desktop backend version constraint. Fixes #8610 * Narrow desktop repair to version propagation * Handle version suffixes in repair fallback --------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-08-19 19:12:22 +01:00
Daniel Han	6e57736ad9	Stop blaming the recipe for a turn that never came back (#9266) ... * Stop blaming the recipe for a turn that never came back `connection (codex)` fails on main and on unrelated branches, and the two lines it prints contradict each other: ##[warning] ... judging the turn on its assertions instead of calling it guide drift. ##[error] [guide drift] agent=codex: the documented launch command exited non-zero (rc=124) ... so the documented flow in start.py drifted. Neither is right. run_timed printed the warning unconditionally, but three callers (connection, resume, attribution-ab) have no assertion that can rescue a partial turn and treat a cap as fatal on purpose. So it promised the opposite of what was about to happen, for exactly the callers most likely to hit it. It now states the fact and leaves the verdict to the caller. The error is worse, because it sends the reader to the one file that is not implicated. The transcript shows codex launching perfectly -- correct provider, correct model, correct profile -- and then sitting on ERROR: Reconnecting... 1/5 for the full 600s. Nothing about the documented flow had drifted; the model server never answered. A cap means the launch command was fine and the turn never came back, which is a different failure with a different owner. Still fatal. Waiving a cap here would report "connection OK" for a recipe that printed a banner and then blocked on a headless prompt, which is the failure this job exists to catch, and assert_reply cannot tell a finished reply from a startup banner. Only the attribution changes. A non-zero exit from the launch command is still reported as drift, which is the case guide_fail is right about. Four static guards, since the script needs five agent CLIs and a live model server to run. All four mutants (run_timed speaking for its callers again, the timeout routed back through guide_fail, the timeout waived instead of fatal, a non-zero exit no longer reporting drift) were reintroduced and confirmed red. Wired into workflow-trigger-lint, the only job with no paths filter. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the waiver guard enforce its own sentence, not a magic number The shell suite went red on this branch: 'exactly the expected TIMED_OUT sites (got 4, want 3)'. Worth deciding which of the two was wrong rather than moving the number. The heading is 'only file-edit turn 1 rescues a soft timeout', and that is the invariant worth having. The assertion under it counted every consultation of TIMED_OUT and pinned the total at 3 -- so it conflated the single waiver with the fatal checks beside it, and its own comment admitted as much ('turn 1's guard, PLUS the resume and attribution-ab fatal checks'). The fourth site this branch adds makes a cap MORE explicitly fatal at connection. Failing a guard named for waivers because a fatal check was added is the assertion being over-specified, not the change being wrong. The neighbouring semantic assertions agree: 'connection guard has no TIMED_OUT escape' passes. So the two shapes are counted separately. The waiver is the || form and is the only one that lets execution continue past a cap; everything else consults TIMED_OUT to stop. That enforces the sentence rather than a number, and keeps all the power: a waiver added anywhere still fails. Mutation-tested against the repaired guard -- a waiver added to the connection guard, resume's fatal check turned into a waiver, and the connection check deleted outright are all caught. 44 pass, 0 fail. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-19 03:21:20 -07:00
Daniel Han	c87fe20e32	Repair the Tauri retry test against the download-marker pipeline (#8996) ... * Repair the Tauri retry test against the download-marker pipeline Shell installer tests has been failing on main, and failing in the worst way: the file printed its header and exited 1 with no FAIL line, because two separate things broke and both of them are silent. #8805 added the uv download markers, which put the wrapped command inside `{ ...; } | _uv_download_markers ...`. The verbose path was already a pipe; the quiet path, which is the one this file exercises, was not. Two consequences: 1. This file hand-copies five functions out of install.sh and stubs five more. It never learned about _uv_download_markers, so the last stage of that pipeline was "command not found", every wrapped command looked like it exited 127, and `set -e` ended the run. Extract the real function rather than stub it, so the test still exercises the pipe the installer actually runs. 2. The command now runs in a subshell, so _test_command's `_test_attempt` increments were discarded. It failed on every attempt instead of succeeding on the second, which is precisely the retry-recovers path the first assertion is about. Move the counter into a file. Both are needed; each alone still fails, at a different assertion. #8805 is right on both counts. Streaming output through a marker filter requires a pipe, and real installs run external commands, so a subshell costs them nothing. It was the test's harness that had gone stale. Also add `|| true` to the two `grep -c` calls that gate the first assertion. grep -c exits 1 on a zero count, which under `set -e` aborts with no output. That is why this read as silence rather than as a failure, and the same guard is already applied to the grep -c calls later in the file. tests/sh: 47 files, all passing. * Keep the attempt file in the final EXIT cleanup The second trap replaces the first, so naming the attempt file only in the first one leaked it: one stray temp file per run, measured. * Name the remaining silent aborts in the Tauri retry test My earlier commit message claimed the grep -c guard was already applied to the rest of the file. That was wrong: lines 231, 326 and 368 were still unguarded, so a zero count there aborts under set -e before the FAIL line it gates. Worse, the guard I did add only covers the case where a wrapped command succeeds and emits the wrong markers. The dominant path is the command itself returning non-zero: under set -e that aborts at the call, before any assertion runs. A broken installer retry therefore still reproduced the exact header-then-silence symptom this file was fixed for. Guard the three grep -c sites, and give the four commands that must succeed an explicit failure message instead of letting set -e end the run mutely. Each path mutation checked, silence to a named failure: retry never recovers -> FAIL: recovered retry returned non-zero rollback_substep renamed -> FAIL: successful Unix rollback output ... UNSLOTH_TAURI_MODE renamed -> FAIL: Unix installer does not pass Tauri mode tests/sh: 47 files, all passing, no temp files leaked.	2026-08-16 07:40:19 -07:00
Daniel Han	e227792740	Let the uv signal handler exit without going through setup_fail (#8744) ... * Let the uv signal handler exit without going through setup_fail tests/sh/test_tauri_retry_failure_context.sh asserts that studio/setup.sh contains exactly one top-level exit and that it is setup_fail's. 5a5bf6413 added a second one inside _setup_uv_on_signal, the HUP/INT/TERM handler that cleans up a partial pinned-uv install, so the whole shell step of Repo tests (CPU) has been red on main since then. The assertion exists so that no failure path can leave the desktop with a bare exit code instead of a [TAURI:ERROR] line. A signal is not one of those paths. install.rs cancels an install by SIGTERMing the process group and setting intentional_stop, and install.sh's own _on_install_signal answers a signal by cleaning up and exiting 128+signal with no error context. A cancel routed through setup_fail would set the sticky explicit_error the app promotes over everything else, so the user would be shown an installer failure for something they asked for. The handler is right, so this changes the test. The count is not loosened, it is split. setup_fail must hold exactly one exit and the signal handler exactly one, their sum must equal every exit in the file, and the two lines must still read exit "$exit_code" and exit "$1". A third exit anywhere, or a second one inside either, still fails. The handler also has to stay reachable only from its own three traps at 129/130/143, since a call from ordinary control flow would make it exactly the unrouted failure exit the count forbids. Adds the behavioural half too, which the file had no coverage for: a stubbed pinned-uv install is interrupted for real in Tauri mode and has to report 143, remove the work directory and both staging files, and print no [TAURI:] output. Removing the cleanup call, emitting an error line on cancel, adding a third exit, or calling the handler directly each fail the file. * Trim the comments on the setup exit assertion Same assertions, same output; only comment and blank lines move. The reasons a future reader needs are kept: setup_fail is the sole failure exit so the desktop gets a [TAURI:ERROR] line, the signal handler re-raises 128+signal because install.rs cancels by SIGTERMing the process group with intentional_stop, and the count is split rather than loosened. --------- Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-13 17:16:18 -07:00
Daniel Han	5a5bf64130	Reduce antivirus false positives in the desktop installers (#8586) ... * Windows setup: install uv from a pinned release instead of running remote script text studio/setup.ps1 piped astral's install.ps1 straight into Invoke-Expression. That download-and-execute shape is the single construct AMSI providers and cloud ML scanners score hardest, and install.ps1 already replaced it with a pinned-SHA-256 archive download. Port the same implementation across. Progress goes to the pipeline rather than the console, so the quiet path swallows it exactly as it swallowed astral's installer output and the printed lines around the call site are unchanged. * Windows: stop pairing a hidden window with a bypassed execution policy The Studio shortcut launched launch-studio.ps1 with -WindowStyle Hidden and -ExecutionPolicy Bypass on the same command line. That pair is what Microsoft's own detections key on, and studio/src-tauri/src/install.rs already refuses it for the app's own launch of install.ps1. The installer writes launch-studio.ps1 itself, so the file carries no mark-of-the-web and RemoteSigned loads it. The hidden window is unchanged, so the shortcut behaves exactly as before. The generated launcher's own child launch moves to RemoteSigned for the same reason: it runs an inline -Command against an executable, where no script file is loaded and the two policies are equivalent. Also refresh a stale comment in studio/setup.ps1 that attributed the PSModulePath fix to astral's uv installer, which no longer runs in-process. * Installers: keep download-and-run command lines out of the shipped script text AMSI scans install.ps1 in full before a single line of it runs, and generic script classifiers read install.sh the same way inside the Linux bundle. Both headers rehearsed the piped web one-liner five times over, plus a scriptblock form and an execution-policy bypass, none of which anything in the scripts reads and all of which the README already documents. Point at the README instead and reword the in-body comments that quoted the one-liner as shorthand. Every printed line is untouched: the remediation text the installers show users still spells out the command in full. Same treatment for scripts/uninstall.ps1's header. * Windows: resolve process image paths with one Win32_Process query install.ps1's venv-holder probe opened a handle to every running PID through inline C# compiled at runtime. Opening a handle per process is a shape AV heuristics score hard, and it bought nothing: Win32_Process reports ExecutablePath for exactly the processes those handles could be opened against, and answers for all of them in a single query instead of once per PID. The remaining file-canonicalisation imports stay -- handle-based resolution of linked ancestors has no faithful Windows PowerShell 5.1 equivalent, and it runs on security-relevant paths. Falls back to the per-process .Path when the query is unavailable, so a degraded WMI repository degrades exactly as the old code did on a process it could not open. * Desktop: say who blocked the install when AMSI stops the script PowerShell hands the whole top-level script block to AMSI while compiling it, so a security product's verdict arrives as a parse error over the entire file before install.ps1 runs a statement: no [TAURI:ERROR] marker, no phase log, and a stderr tail the user cannot act on. unsloth#8523 shows what that looks like in the UI -- "Installation failed: + FullyQualifiedErrorId : ScriptContainedMaliciousContent". Recognise the two stable error ids on either stream and append what the user actually needs: nothing was installed, nothing was changed, it is a false positive, update definitions and retry, do not turn off endpoint protection. The raw id stays in the message, because the diagnostics report and any vendor submission both need it. Matches the id, never the message text, which is localized, and tolerates the cmdlet suffix the Invoke-Expression form carries. * Desktop: ship each bundle only the installer it can run resolve_install_script picks install.sh on unix and install.ps1 everywhere else, but the shared Tauri config bundled both into every target. The Linux AppImage therefore carried 280 KB of Windows PowerShell it can never execute -- and it is the largest script body a generic classifier walking the squashfs reads, which is where Microsoft's Trojan:Script/Wacatac.B!ml verdict on 0.1.701-beta landed. Move the resource map into the per-platform configs. The clean-machine job already fails when a Linux bundle ships no install.sh; it now also fails when one ships install.ps1, so the split cannot silently regress in either direction. The .deb scanned clean with the same payload, so this is surface reduction rather than a proven fix for that verdict. * POSIX installers: install uv from a pinned release before falling back install.sh downloaded astral's install.sh to a temp file, ran it and deleted the file; studio/setup.sh piped it straight into a shell. Both are, shape for shape, what a dropper does, and generic ML script classifiers score them accordingly -- the 0.1.701-beta Linux AppImage came back Trojan:Script/Wacatac.B!ml while the .deb carrying the same scripts came back clean. Fetch the pinned release archive and verify a hardcoded SHA-256 instead, matching what install.ps1 already does on Windows. Only the four mainstream targets are pinned: musl, armv7 and any host without a digest tool keep the path they have today, because guessing a target triple wrong would break the install outright and that costs far more than the heuristic score of the fallback. Destination, PATH handling and every printed line are unchanged, so a host that takes either path ends up in the same state it did before. * tests: pin the installer shapes antivirus heuristics score One file collecting what was removed, so it cannot drift back: no remote script run in-process, no encoded or base64 payload, no hidden window paired with a bypassed execution policy, no handle opened against another process, and no new runtime-compiled native import outside an allowlist that carries a reason for each entry that stays. The last test is the other half of the contract. Hardening must not change what a user sees, so the remediation lines the installers print -- which still spell out the web one-liner in full -- are asserted verbatim. Removing the one-liner from comments is the point; removing it from what the user is told to run would be a regression. Runs on the existing discovery-based pytest step, no workflow list to update. * release: emit a false-positive submission packet for whatever gets flagged The build job assembles a Microsoft submission packet, but only for the Windows -setup.exe. The detection that actually arrived on 0.1.701-beta was Trojan:Script/Wacatac.B!ml on the Linux AppImage, so nothing was produced for the one asset that needed it. The VirusTotal job already knows which assets were flagged and by which engines, so put the packet there: hash, size and both portals, for every flagged asset whatever platform it came from, with a note that clearance is per hash and per vendor. Engine names are not repeated -- they are third-party text and already appear escaped under Flagging engines. The gate stays advisory; this only makes acting on it take seconds. * Revert "Windows: resolve process image paths with one Win32_Process query" This reverts commit 7897865c9. tests/python/test_windows_installer_concurrency_guard.py bans Get-CimInstance and $process.Path from Get-RunningStudioVenvProcesses outright, and requires the native image-path lookup. That contract came out of #7764, which closed a set of races where the installer inferred "in use" from something other than a confirmed executable identity and blocked installs that should have proceeded. Win32_Process.ExecutablePath does answer the same question, but a wrongly blocked install costs far more than the heuristic weight of three native imports. Record the imports in the AV-shapes allowlist with that reasoning instead, and keep the ban on the process-memory APIs, which the installer has no use for. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments added by this branch Opening comment-reduction pass over the PR diff: same intent, fewer lines. Cut hardest on the prose that restated the PR description rather than explaining the code next to it. Comments and docstrings only, verified with comment_tools.py check --strip-docstrings across every Python file in the diff. * Drop an unused helper from the uv pinned-release test * Fix three review findings on the installer hardening Stray-resource check aborted the step it was meant to assert. grep exits 1 when it selects nothing, and under this step's set -o pipefail plus the runner's bash -e that kills the assignment outright, so every correctly split .deb failed clean-machine CI before reaching the check. Both lookups take || true now: no match is the passing case for the stray one, and for install.sh it was swallowing the explicit annotation in favour of a bare exit 1. studio/setup.sh skipped astral's XDG_DATA_HOME/../bin destination tier, which install.sh, install.ps1 and studio/setup.ps1 all honour. A host that configured an XDG location got uv under ~/.local/bin instead, where no later shell looks for it. The session PATH prepend hid it at install time. The AMSI guidance claimed nothing was changed even when the block landed on the nested studio/setup.ps1, which install.ps1 launches through the same inherited pipes after the venv, PyTorch and the packages are already on disk. Split the wording on whether a [TAURI:STEP] marker has been seen: a pre-start block produces none, so the reassurance is only given where it is true. * Key the submission packet on the flagged count, not the engine list stats and results are separate fields of the same VirusTotal response, so an asset can carry a flagged count with no readable results map. The summary table reports that asset and the packet skipped it, which is exactly the one that needs a packet. Select on stats.flagged and keep the engine list for the Flagging engines section, which is correctly keyed on having engines to name. * Drop the bundle stray-resource assertion from clean-machine CI That job downloads a published release, never a bundle built from the branch, so asserting the new resource split there turns every run red until a release ships with it. The split is a property of the Tauri config, and tests/studio/test_tauri_installer_resource_contract.py already enforces it at the right layer. The || true on the install.sh lookup stays: it is what lets the explicit annotation print instead of the step dying on grep's exit 1 under pipefail. * Cover the uv host matrix and repeat application in the pinned-release test The pinned path picks an archive per host triple, and a wrong pick installs a binary that cannot execute, which is worse than not installing at all. Drive _uv_pinned_asset over 20 host combinations and require each one to return its own triple or decline to the fallback. Also run the installer three times over one HOME and require an identical tree, and require a stale uv at the destination to be replaced rather than joined by a second copy: the installer is re-run on every upgrade and every repair. * Pick the pinned uv archive off a positive libc check, not the absence of musl An independent audit pass found the Linux selector accepts any host whose ldd output does not say musl. That is not the same question astral's installer asks: it checks a minimum glibc and drops to its musl-static archive below it, so three hosts that worked before this branch now get a GNU binary that cannot exec, and the helper reports success so the fallback never runs. aarch64 with glibc below 2.28 (Ubuntu 18.04) x86_64 with glibc below 2.17 (RHEL 6) a musl image with no ldd at all, where the probe simply finds nothing Read the version instead, from ldd or getconf, and require it to clear astral's floor for the triple. Anything unreadable declines to the fallback. Also ask the userland for its bitness rather than trusting uname on a 64-bit kernel running a 32-bit userland, and follow astral in reading hw.optional.arm64 so a translated shell under Rosetta 2 still gets the native macOS build. Three more from the same pass: Report success only when the destination uv is executable. A copy onto a busy or read-only destination could leave a file that is not, and reporting success there skipped the fallback. Nothing is unwound on the failure path on purpose: the fallback installs over whatever is at the destination, and deleting there would take out a working uv the host already had. Clear the mark of the web on the launcher we author. WriteAllText replaces the unnamed data stream and leaves other NTFS streams alone, so a launch-studio.ps1 that somehow carried one would keep it across the rewrite, and RemoteSigned refuses a marked unsigned script. Store the security-block kind and resolve its wording in message(). stdout and stderr are read by independent threads, so a [TAURI:STEP] written before a block can be observed after it, and freezing the wording at observation time could tell a user nothing was changed on a run that had already installed PyTorch. Also require the error id to appear as the value of a FullyQualifiedErrorId field, so a scanner log that merely names it cannot attach antivirus guidance to whatever fails next. The host matrix in the shell test grows to 28 rows covering every case above, and removing the new gate fails 8 of them. install.rs gains two tests: 37 pass. * Replace a symlinked uv destination instead of writing through it Three from the review on the previous head. cp onto a destination that is a symlink follows the link, so installing over `~/.local/bin/uv -> /opt/homebrew/bin/uv` rewrote the Homebrew binary in place and left the link pointing at a file another package manager owns. Stage next to the destination and rename over it: rename replaces the link itself, and it is atomic, so a concurrent reader never sees a half-written uv either. The staging file is removed when the rename fails, so a failed run leaves no debris. Verify the Windows copy the same way the shell scripts now do. Copy-Item is non-terminating under the caller's ErrorActionPreference, so a locked or ACL-denied destination let execution reach `$haveUv = $true` and the function reported success over whatever was already there. Compare the destination against the archive we just verified, so a stale uv.exe cannot pass for the one we meant to install. install.ps1 carried the same shape and gets the same treatment. Point the header links at the heading that exists. The README has no "Install Unsloth Studio"; it is "Unsloth Studio (web UI)", whose anchor is #unsloth-studio-web-ui. Three test cases cover the symlink: the file behind the link is untouched, the link itself is replaced, and no staging file survives. Reverting the fix fails two of them. * Fail the build when the uv pin drifts from a version floor Before the pin, astral's endpoint always delivered the newest uv, so raising UV_MIN_VERSION was safe on its own. It is not any more: a floor above the pin means a host with no uv gets 0.12.1 installed and then judged too old by the same script that installed it, on the one path where the pin is what runs. Two checks. All four installers must name the same uv, or which version a machine ends up with depends on which script reached it first. And the pin must clear every floor in the tree (UV_MIN_VERSION, UV_OFFLINE_MIN_VERSION, $UvMinVersion). Raising a floor past the pin fails the first, bumping one installer's pin alone fails the second. * Write the shell profile entry the pinned uv path no longer gets for free The P1 here is a real regression and it took a second look to see why. install.sh decides whether to add ~/.local/bin to the user's shell profile with `case ":$PATH:"`, near the end of the run. By then this process has prepended that directory twice, once for the uv bootstrap and once for the venv, so the guard answers yes for a login shell that would answer no and the profile line is never written. That was survivable while astral's installer ran, because it wrote its own profile line and its env file. The pinned path writes neither, so on a fresh account whose login PATH lacks ~/.local/bin the install succeeds, the current shell works, and the next terminal cannot find `unsloth` or `uv`. Snapshot the inherited PATH before anything prepends to it and test the guard against that. Two more from the same review. Honour a configured uv mirror exclusively. UV_INSTALLER_GHE_BASE_URL and UV_INSTALLER_GITHUB_BASE_URL already win outright in both PowerShell installers and in astral's own; the shell path ignored them and tried the public hosts first. A restricted network sets one precisely because those hosts are unreachable, and download() has no timeout, so it would hang rather than reach the fallback. Do not let the twin of an earlier clear erase a later AMSI verdict. Clear-TauriInstallError writes one logical clear to BOTH streams (install.ps1:198) and independent threads read them, so a block observed between a clear and its own twin was discarded by the twin. Ignore a clear identical to the one just processed; a genuine later recovery carries different text and still clears. Two tests cover both directions, 39 install tests pass. * Stage the uv copy under a per-process name An audit pass reproduced a race I introduced with the symlink fix. Both POSIX helpers staged through a fixed destination-side name, so two installers targeting one directory shared it: A finishes copying the staging file B opens the same path with truncation A renames that inode into place as uv B keeps writing through its open descriptor, which is now the published uv The published uv was observable at zero bytes until B resumed, which makes the claim in the comment about a concurrent reader flatly wrong. install.ps1 is covered by its named mutex, but nothing serialises the POSIX helpers, and studio/setup.sh runs standalone on every studio update. mktemp in the destination directory instead. Each rename then publishes a file no other process can still be writing, which is what the atomicity argument needed all along. The loser cleans up its own staging file and declines, so the caller falls back rather than reporting a success it did not achieve. * Keep a default install as quiet as it was when a uv mirror misbehaves Two console regressions from the audit pass, both on paths the install still recovers from. download() runs curl -LsSf, and -S deliberately prints its own errors. The fallback ran under run_maybe_quiet, so a failed download printed nothing before; the pinned attempts run outside that wrapper, so an unreachable mirror now put two curl: (N) lines on the console of a default install that then succeeded. Redirect stderr on the speculative attempts only, leaving download() untouched for every other caller. [TAURI:WARN] is a marker level install.sh has never emitted, and the app forwards unknown markers to its progress UI verbatim (install.rs:639), so a digest mismatch would have surfaced as raw text in the desktop window. Make it a verbose only stderr line: the next mirror or the fallback still runs, so a default install has nothing to say here. Printed-string diff against the merge base is back to additions inside $(...) capture plus that one verbose-gated line, with nothing removed or changed. * Ask the installed uv whether it runs before skipping the fallback The libc gate reads a glibc version from ldd or getconf and treats that as proof a GNU binary will execute. It is not. A stripped NixOS-derived image without nix-ld reports a glibc version through getconf while its loader lives in the Nix store, so the pinned x86_64 uv asks for /lib64/ld-linux-x86-64.so.2 and gets nothing. Every static check passed, so the helper reported success, the astral fallback was skipped, and the first real uv call failed with No such file or directory. astral's installer fails its own glibc probe on that host and ships the fully static musl archive, which runs. The user went from a working uv to none. The archive is digest-verified astral uv by the time it is placed, so ask it: run --version and require it to succeed. One exec closes the whole class rather than this one host, covering a wrong triple, a loader that is not where the binary looks, and a destination we could not really write. A test drives an archive whose uv cannot execute and requires the helper to decline; removing the exec check fails it. * Pair every clear with its twin, not just the previous one install.ps1 clears after each recovered step, so a lagging reader can be several clears behind when a block lands. With clears A then B on one stream and A's twin arriving on the other after the verdict, asking only whether this is the message just seen answers no, and the delayed twin discarded the verdict the guidance exists to explain. Each logical clear emits exactly two markers, so count unpaired ones by message: the first sighting is the clear, the next pairs with it. A test drives the A, B, verdict, A', B' ordering; 40 install tests pass. * Validate the staged uv before it replaces a working one My own exec check was on the wrong side of the rename. The sequence that bites: a host has a uv good enough for UV_OFFLINE_MIN_VERSION but below UV_MIN_VERSION, so the block runs with _uv_present_before true; the pinned path renames over that working binary; the --version check then fails because the loader is missing or the destination is mounted noexec; the fallback download also fails. The installer neither restores the old uv nor reports that none is available, and every later command runs the broken one. Test the staging file instead, before the rename. It sits on the destination filesystem, so it answers the noexec question too, and a binary that cannot run here never gets to replace one that could. Two tests: a working incumbent uv survives an archive whose uv cannot execute, and the rejected staging file is cleaned up. Moving the check back after the rename fails the first. * Stop the AMSI guidance claiming more than it knows Two of these are honesty defects in text a blocked user reads. "nothing was changed on this machine" is false. Rust starts a diagnostics attempt and its phase log before PowerShell is ever spawned, and spawn_script can create ~/.unsloth first, so a pre-start block has already written to disk. The honest claim is that no installation step ran. "This is a false positive" is not something the classifier can know. It proves the output carries a PowerShell error id and nothing about the script's integrity, and install.ps1 can sit in a user-writable directory, so a locally modified copy can earn a genuine verdict. Telling someone to report a correct detection to their vendor is worse than telling them to reinstall from an official package first and only escalate if an unmodified copy is still blocked. Two smaller ones from the same pass. The matcher tested for the field name and the id independently, so a line naming both in prose qualified; it now requires the id to follow the colon and end at a comma or whitespace, which is what the comment always claimed. And the clear-pairing map is bounded: legitimate producers use a small fixed label set, and child output must not be able to grow it without limit. 42 install tests pass. * Honour astral's download override, and stop Unblock-File asking Three from the second audit round. Unblock-File declares SupportsShouldProcess at the default Medium impact, so a profile that sets $ConfirmPreference to Medium or Low gets a prompt from the line I added, even for a launcher that never carried the stream. -ErrorAction does not suppress a ShouldProcess prompt, and a noninteractive host turns it into an error that skips shortcut setup entirely. -Confirm:$false. UV_DOWNLOAD_URL and its older alias INSTALLER_DOWNLOAD_URL outrank the mirror variables in astral's installer, and the merge-base path inherited that because it ran astral's script. All four implementations now honour them first and exclusively. My earlier comment argued they point at a version the pin would reject, but that reasoning had it backwards: a host sets one because it cannot reach the public endpoints, so ignoring it meant public egress first and, with no timeout on the download, a hang instead of a fallback. The pin still applies, so a source serving a different build fails the digest and the caller falls back to astral's installer, which honours the same variable. chmod 0755 on the staging file rather than +x. cp gives it the umask default and +x then adds execute only where the umask allowed read, so a umask of 077 left uv unusable for every other account on a shared machine. astral ships them 0755. Four checks pin the override precedence across all four installers and the mode across both shell ones, with the behaviour verified against a stubbed downloader. * Validate uv before it replaces an incumbent on Windows, and bound the probe install.ps1 and studio/setup.ps1 copied the extracted uv.exe straight over the destination and only asked whether it ran afterwards. A host with a working older uv and a policy (AppLocker, WDAC, endpoint protection) that refuses the new one was left with neither. Run the extracted binary where it landed first, then keep a copy of the incumbent across the publish and restore it if the published copy will not run, since Windows has no atomic replace for a file that may be open. The probe itself is bounded: Start-Process with a 20s WaitForExit and redirected streams, and on POSIX no stdin plus a 20s ceiling where timeout exists. A binary this installer just downloaded must not be able to hang an unattended install by prompting or by never exiting. install.sh and studio/setup.sh also published the pinned uvx after rejecting the pinned uv, leaving a pairing that is never built or tested. A uv that fails to stage, copy or run now abandons the whole placement. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Verify each uv mirror, and persist PATH when the account has no rc file Both PowerShell installers checked the archive digest once, after the download loop had already broken out. A captive portal or a proxy answering 200 with its own body is a successful download by every measure Invoke-WebRequest has, so the first mirror consumed the only attempt and the second, healthy one was never tried. The digest now decides whether a mirror counts as served. install.sh picked a shell profile from .zshrc, .bashrc or .profile and did nothing when none existed. A fresh account has none: astral's installer used to create its own PATH setup there, the pinned path does not, so the next terminal resolved neither unsloth nor uv. Fall back to creating ~/.profile, which every POSIX login shell reads. The existing content guard keeps it written once. * Remove the install.sh a Windows upgrade would otherwise keep forever Windows bundles now carry only install.ps1, but NSIS writes the current resource manifest and deletes nothing, and the uninstaller deletes only what is in that manifest. An in-place upgrade from a release that bundled both installers left install.sh in $INSTDIR permanently, which also made the non-recursive RMDir "$INSTDIR" fail at uninstall. The pre-install and pre-uninstall hooks now delete it, so the population most likely to upgrade actually gets the split. Also silence the speculative mktemp -d in the pinned uv path: its failure falls back to astral's installer, so an unusable TMPDIR printed a line the user could not act on and that the merge base did not print. * Remove the pinned uv temporaries when an install is interrupted The pinned path unpacks a 40 MB archive into a work directory and stages the binary next to the destination, but only cleaned both up when the helper returned normally. A Ctrl-C in between left the archive behind and left a staging file inside a directory that is on PATH. Both paths are now published to the exit and signal traps as they are created and cleared when the helper releases them. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Persist the PATH the way each shell actually reads it, and fail a half-published pair Four follow-ups from review: A uvx that the archive carried but that could not be staged or renamed left uv published next to a stale or missing uvx and still reported success, skipping the fallback that would have installed both. Either half failing now fails the placement, in install.sh and studio/setup.sh. studio/setup.sh had none of the interrupt cleanup install.sh gained: a Ctrl-C left the unpacked archive behind and a staging file inside a directory on PATH. It now owns HUP, INT, TERM and EXIT for the duration of the pinned install and hands them back on the way out. fish sources none of the POSIX rc files, so the ~/.profile fallback was a no-op for a fish user. The persistence helper writes a conf.d drop-in with fish_add_path there, and honours ZDOTDIR for zsh. UV_INSTALL_DIR, UV_UNMANAGED_INSTALL, XDG_BIN_HOME and XDG_DATA_HOME can put uv somewhere other than ~/.local/bin, and astral's installer wrote a PATH line for whichever it picked. The pinned path now persists its own destination too, with UV_NO_MODIFY_PATH honoured as astral honours it. * Make the Windows uv publish a real transaction, and quote persisted paths The companion copies ran bare: under install.ps1's Stop preference a locked or ACL-denied destination threw past the rollback and left a mismatched set with the backups still on disk, and under setup.ps1's Continue preference it kept a stale companion and reported success. Both now copy under -ErrorAction Stop inside the transaction, so any failure unwinds like the others. A failed restore also used to delete the backup anyway, which is the one path in this block that could leave the host with less than it started with: the two things that make a restore fail, an open incumbent and a denied ACL, are the same two that made the replace risky. The backup is now kept and named. fish takes an unquoted path with a space as two directories, neither of which exists, so the drop-in single-quotes it; and the rc line is written inside double quotes, so a uv directory holding a dollar or a backtick is escaped. The second test caught a doubled backslash in the escaper itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments added by this branch Comments only, no code touched: 161 comment lines become 106 across install.sh, install.ps1, studio/setup.sh, studio/setup.ps1 and the NSIS hooks. Each one keeps the reason it was written for, said once. Verified with the PowerShell AST parser, sh -n and bash -n, the 50-check uv pinned release suite and 114 installer tests, and by confirming the diff contains no non-comment line. * Tighten the install.rs comments too Comments only: 35 lines become 27, each keeping the reason it was written for. 42 install tests pass and the diff contains no non-comment line. * Abort on a companion that cannot be backed up, and pair clears by stream A uvx.exe that could not be copied aside, because it is locked or its ACL denies reads, was skipped and the new uv.exe published anyway, so the function reported success with a mismatched pair and the fallback never ran. Any backup failure now fails the placement and runs the rollback, in install.ps1 and studio/setup.ps1. The ERROR_CLEAR pairing keyed only on the message, so two real clears of one label on one stream were taken for a clear and its twin. That happens: _install_torch_default_index emits its recovery during the install and again during the ROCm repair. A verdict landing between them was then erased by the genuinely later clear arriving on the other stream. The map is keyed by stream as well, so only the opposite stream's copy can consume a pending marker. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not fail an install because the uv probe could not get an answer Three clean-machine CI legs that pass on main failed on this branch: arm64 and two Windows containers, all three with winget unavailable, which is the only condition under which the pinned fallback runs. Each downloaded the right asset, passed the digest, and then failed the probe. Start-Process -NoNewWindow with redirected streams does not behave in a container or on the arm64 image the way it does in a desktop session, and a boolean probe reported that as a broken binary and aborted the install. The probe is now tri-state. Only the binary answering non-zero is a failure. A launch that throws or a wait that times out is inconclusive, and since the digest already proved the bytes are astral's pinned release, an inconclusive probe publishes as the pre-pin code did. Every path prints why, with the captured stderr and the exit code, so the next occurrence is not opaque. Also from review: the POSIX path now stages both binaries and publishes them together with the incumbents saved aside, so a failed uvx rename restores the uv it replaced instead of leaving a new uv beside a stale uvx; the Windows rollback records the destination before the copy that can truncate it; and UV_UNMANAGED_INSTALL suppresses the profile write, as it does for astral. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give setup.sh the same pair publish and PATH persistence as install.sh studio/setup.sh published uv and then uvx one after the other, so a failed uvx rename left a new uv beside the host's stale one, and the remote fallback can be unavailable. It now stages both, validates uv, and publishes the two renames back to back with the incumbents saved aside, restoring them if the second fails. setup.sh is also run directly for local and Colab setup, where astral's installer used to write the profile line for whichever destination it chose. Without one the PATH export died with that shell and every later run reinstalled uv. It now persists its own destination, with fish handled on its own terms and both of astral's opt-outs honoured. * Treat an empty uv exit code as no verdict, not as a failure The arm64 clean-machine leg still failed on the tri-state probe, and the diagnostic that came with it said why: "uv --version exited ." with no number. WaitForExit(ms) can return before the exit code is cached, so ExitCode was empty and an empty value is not 0, which read a working uv as broken. The parameterless WaitForExit settles it and returns at once because the process has already exited, and a code that is still missing is inconclusive rather than a failure, which is the same rule the launch and timeout paths already follow. Verified against pwsh that a real non-zero exit and a real launch failure still classify as failed and unknown respectively. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Eight review fixes across the uv publish and PATH persistence The fish escaper in studio/setup.sh reached sed as an invalid expression, so a fish user running setup directly would have had setup killed under set -e right after uv was published. It now matches the one in install.sh, and the test runs both escapers rather than reading them. An incumbent that cannot be hard-linked or copied cannot be restored either, so publishing over it would be a one-way move. Both shells now decline. Writing that test turned up that my own rollback deleted both incumbents when nothing had been published, since the no-predecessor branch cannot tell the two cases apart; the rollback is now reached only after a publish was attempted. A rollback with no predecessor removes the binary it published, rather than leaving half a pair the host never had. A signal between the two renames left the undo copy as the only reference to the incumbent, and the handler deleted it. It restores it now, in both shells. setup.sh prepended ~/.local/bin unconditionally after a successful pinned install, so a stale uv there could shadow a custom UV_INSTALL_DIR destination and the rest of setup would run the wrong one. That prepend is now only for astral's installer, which is what writes there. PATH entries are compared literally rather than as case patterns, so a destination holding *, ? or [ is not mistaken for an unrelated entry. On Windows, a .unsloth-old left behind by a failed restore is the only copy of a working uv, and the next run reused that exact name. It takes a distinct one. * Keep the pinned uv first on PATH, and only count an active profile entry install.sh prepends ~/.local/bin after the uv bootstrap, and astral's env file does too, so a custom UV_INSTALL_DIR destination was pushed behind a stale uv sitting in the home directory and every bare uv below picked the wrong one. The pinned destination goes back in front. setup.sh had the same shape and was fixed in defd2292a. The profile check treated any occurrence of the destination text as proof the PATH entry was already there, so a commented-out old export, or /opt/uv-old when the destination is /opt/uv, suppressed the write and left the next shell without uv. Comments are stripped and the directory has to appear as a whole entry. * Gate the NSIS tidy-up, and remove an orphan uv on signal The pre-install hook runs before the user can still cancel, and $INSTDIR can be a directory they picked in the GUI, so deleting install.sh there could take a file that was never ours. Both hooks now only act where our own executable already is. A signal between the two renames restored a predecessor but did nothing when there was none, leaving a 0.12.1 uv beside whatever uvx the machine had. It now removes what it published, which is what the ordinary rollback already does. * Write the uv PATH entry to every startup file astral's installer wired astral's uv installer wires ~/.profile, each of .bashrc, .bash_profile and .bash_login that exists, .zshrc or .zshenv under ZDOTDIR, and a fish drop-in under ~/.config. Replacing that installer with a pinned archive meant the PATH entry only reached the one file for whichever shell happened to be running, so a bash user whose .bash_profile does not source .bashrc, a /bin/sh login, or anyone who later switched shells would have no uv on PATH where they used to. Both POSIX installers now write the same set, once each, with the existing whole-entry check keeping a re-run idempotent. Files that do not exist are not created, apart from ~/.profile, which astral creates too. * Cut the uv publish back to what the common case needs The rollback machinery that grew over the review rounds covered cases a user is very unlikely to meet: an incumbent that cannot be hard-linked, a signal landing between two renames, a restore that itself fails, a second installer racing the first. It was 281 net lines, and every finding in the last two rounds was in it rather than in the hardening. What stays is what the common case needs. POSIX stages both binaries, runs the staged uv, and publishes the pair with two renames; a failure anywhere before them leaves the destination untouched, and the caller falls back to astral's installer exactly as before. Windows probes the extracted uv.exe before touching the destination, then copies the three under -ErrorAction Stop and re-checks the digest at the destination. The staging files are still removed on a signal, since they live in a directory that is on PATH. 64 shell checks and 114 installer tests cover the rest. * Match the exact fish entry, and let a UNC launcher load The fish drop-in is the only thing that puts uv on a fish user's PATH, since fish reads none of the POSIX files, and its check treated any occurrence of the directory as proof: /opt/uv-old suppressed /opt/uv. It now matches the exact fish_add_path line it would write. A launcher on a UNC share is a remote script to PowerShell, and RemoteSigned refuses an unsigned one, so a roaming profile got a shortcut that exits without starting Studio. That case, and only that case, uses Bypass, and drops -WindowStyle Hidden with it so the pair the detections key on never appears. * Wire every startup file on a DEFAULT install too, and give setup.ps1 a fallback The all-profile PATH write was gated on the uv destination differing from ~/.local/bin, which is exactly where a normal install puts it, so every ordinary machine still got the single-file write the shim path has always done. Three independent audits found this. The gate is gone, and the idempotency check now also matches the $HOME-relative spelling the shim block writes, so the default case does not end up with two lines for one directory. studio/setup.ps1 replaced astral's installer with the pinned archive and had nothing to fall back to. A failed pinned install therefore left UseUv false and silently ran torch, bitsandbytes, Triton and the rest through pip: a different resolver, not just a different download. winget is the fallback, as install.ps1 already does, rather than the remote script this branch exists to remove. * Read the pinned install's real result, and three narrower publish guards The winget fallback I added last round read Invoke-SetupCommand's return value, which is [int]$LASTEXITCODE rather than the function's $true, so it fired on every run: a redundant managed install, and a second copy on a machine that asked for UV_UNMANAGED_INSTALL. The function records its own success on the script scope and the fallback reads that. A directory named uv at the destination looked like a published binary: mv moves into it and reports success, and a searchable directory passes -x, so the install reported success and the first later uv call failed instead. Both shells refuse a directory target, as the installer already does for its own shim. The PATH idempotency pattern escaped only part of the ERE metacharacter set, so a destination holding + ( or | did not match itself and every reinstall appended another block to every profile. * Restrict the profile duplicate check to PATH lines, and cover mapped drives for PR #8586 * Match only PATH-setting lines in the profile duplicate check for PR #8586 * Tighten the installer comments added by PR #8586 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>	2026-08-13 07:02:18 -07:00
Wasim Yousef Said	72ab966221	Installer: suppress macOS uv developer tools dialog (#8479) ... * Desktop: stop the loopback client following redirects `loopback_http::client` is the client that posts `.desktop_secret` to /api/auth/desktop-login, and it was built without a redirect policy. reqwest follows up to 10 redirects by default, and its cross-host protection strips headers rather than bodies, so a responder answering 307 (which preserves the method and the body) would carry the secret to whatever the Location header names, after the loopback URL had already been checked. Its sibling `streaming_client` already refuses redirects for exactly this reason: "Redirects are refused so a loopback URL cannot be bounced off-host after the check." Give `client` the same policy. No behaviour change for any real backend, which never redirects these routes. * Suppress macOS uv developer tools dialog * Update workspace guard for uv wrapper * Stop the installer raising the macOS command line developer tools dialog On a Mac without the Command Line Tools, /usr/bin/git, lipo, install_name_tool and friends are libxcselect shims. Executing one resolves no developer dir and posts to com.apple.dt.CommandLineTools.installondemand, which draws the 'requires the command line developer tools' dialog naming the tool. Resolving the path does not; only execution does. Two call sites execute a shim on the consumer path: _has_working_git ran 'git --version' to decide whether git works, so on a clean Mac the probe raised the dialog it exists to detect. It now answers from the resolved path when that path is exactly /usr/bin/git and no toolchain is selected. Deliberately narrow: a Homebrew, MacPorts or Xcode.app git earlier on PATH is a real binary and is still probed by executing it, so a Mac with a working git but no CLT selected behaves exactly as before. An earlier version of this gated on 'no CLT implies no working git' and broke that case, which the existing test caught. xcode-select -p only asks which toolchain is selected and never prompts. The venv arch probe called lipo first and fell back to file -L. lipo is a shim; 2>/dev/null hides its stderr but not a GUI dialog. file is base system and always answers, so the order is swapped. Both spellings feed the same case below, against 'Mach-O 64-bit executable arm64' or 'universal binary ... [x86_64] [arm64]' rather than lipo's 'arm64' / 'x86_64 arm64', so the branch taken is unchanged. clean-machine-assert.sh already made this same swap for its own use. The cctools binaries were missing from the clean machine CI tool list, so none of this was visible: trace mode generated no wrapper and the absent list never checked them. install_name_tool, lipo, otool, objdump, vtool, strip and nm are added, which is what makes these fixes regression testable. test_macos_clt_gate.sh gains two cases pinning the contract: with a shim git and no toolchain selected the probe answers no WITHOUT executing it, proven by a stub that records execution into a marker file, and with a real git elsewhere on PATH the stub IS executed. The first assertion passed vacuously when written (wrong temp path meant the marker could never be created) and was fixed by making its pair fail first. 18 to 23 passing. * Preserve working git on Intel macOS * Keep the git shim guard on under Rosetta --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-11 12:19:30 -07:00
Daniel Han	3fc67dcad6	Linux: resolve the ROCm version from every source, not the first that answers (#8414) ... * Linux: resolve ROCm version from every source, not the first that answers install.sh took the ROCm version from the first source that responded, in the order amd-smi, /opt/rocm/.info/version, hipconfig, dpkg rocm-core, rpm rocm-core. Debian 13 (and Linux Mint on top of it) packages hipconfig at 5.7.31921-0 next to a 6.1.x rocminfo/HSA runtime, so detection settled on rocm5.7, the "PyTorch ROCm wheels require ROCm 6.0+" gate fired, and a working RX 7900 XTX got torch 2.10.0+cpu. The same install log then reported "AMD ROCm (gfx1100) ROCm: /opt/rocm", so the installer held evidence against its own conclusion. Detection now reads every source and takes the highest reading. A source reporting lower than another on the same host is stale packaging, not a downgrade, so the highest is the one that describes the runtime the GPU uses. The inline || chain is split into one helper per source plus a highest-wins resolver; each helper returns 0 unconditionally, so the case where every source is missing still reaches the actionable no-version warning instead of dying under set -e. The sub-6.0 warning also names a way forward. It now says the reported version was the highest any source gave, and names UNSLOTH_TORCH_INDEX_FAMILY and UNSLOTH_TORCH_INDEX_URL, both of which return from get_torch_index_url before any GPU probing and are forwarded through the WSL reroute, so they are a real override for a host with split ROCm packaging. Tests: tests/sh/test_rocm_version_source_disagreement.sh covers the reported Debian 13 shape, a stale reading in each source position, a genuine ROCm 5.x host still falling back to CPU, every source missing not killing the installer under set -e, and the unchanged tag normalisation (patch levels, 6.5+ clip, 7.3+ cap). test_rocm_support.py's set -e guard is rewritten against the new shape and gains a per-position highest-wins check. * Linux: stop a removed-not-purged rocm-core from choosing the ROCm wheels Resolving the ROCm version by highest reading fixed the undershoot in #8402 and opened the symmetric question: a source reading HIGHER than the host now picks the wheel family instead of being shadowed by its position in the chain. Most of that is fine and was already bounded. PyTorch's ROCm wheels vendor their own ROCm userspace into torch/lib, so what they need from the host is the amdgpu/KFD driver, and AMD documents driver/userspace compatibility as +/- 2 releases, widening to about a year's span from 6.4 on. The normalisation below can only emit an index leaf PyTorch actually publishes. A host that reads high because it genuinely carries a newer ROCm tree is therefore usually right, or at worst harmless. What is not bounded is a reading taken off something that is not installed at all, and dpkg could produce one. `dpkg-query -W` lists every package in the status database except purged ones, so a rocm-core taken out with `apt remove` and never purged sits in state "deinstall ok config-files" and still reports the version it had. A host that ran ROCm 7.0, went back to 6.1 and never purged therefore offered a 7.0 reading off a package that is gone. Fourth in the old first-answer chain it was usually shadowed; under highest-wins it beat every live source. Detection now asks for ${Status} alongside ${Version} and takes the version only when the status word is "installed". ${Status} is a documented showformat field with no dpkg version floor, and dpkg-query renders an unrecognised field as empty rather than failing, so on a dpkg that lacked it this source would go silent rather than over-report. The other four were audited for the same class of state and none has one. rpm has no equivalent of config-files: `rpm -e` drops the header record, and `rpm -q` then exits 1, which the helper already treats as no answer. amd-smi resolves librocm-core.so and hipconfig reads its own tree's version file, so both report a ROCm that is installed. /opt/rocm/.info/version describes whatever tree /opt/rocm resolves to. What those three cannot do, on a host carrying several ROCm trees, is say which runtime a process will actually load; none of them knows, and neither does this installer. So disagreement is now on the record. When the sources do not agree, resolution names every reading and which one won. That is the cheap part of the answer to the multi-version case: when a too-high pick does bite it surfaces as an HSA error at the first HIP call, or as a GPU silently missing from torch, and neither points back at wheel selection without the readings in the install log. Agreeing sources stay silent. The call site stops discarding stderr, which only ever carried this line: each source's own noise is suppressed inside the resolver. Tests: the shell suite gains the config-files 7.0 entry on a 6.1 host, its installed twin at the same versions so the assertion cannot pass on the version alone, the other non-installed dpkg states, and a stale-high reading in each of the five source positions. Its dpkg-query mock now renders the showformat string it is handed, so the tests cover how install.sh asks for the version and not only how it parses the answer; test_rocm_support.py gains the same mock and the config-files/installed pair. Both new cases fail against the previous commit and pass here. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop amd-smi reporting the amdgpu driver version as the ROCm version amd-smi prints one pipe-delimited line, so the ROCm field is followed on the same line by the amdgpu driver version. Stripping every non-digit from the field glued the two together, and when amd-smi cannot detect a ROCm userspace it prints "ROCm version: N/A", which left only the driver version behind: ... | ROCm version: N/A | amdgpu version: 6.10.10 | ... -> rocm6.10 While detection took the first source that answered this was mostly hidden, since a later real reading could not be reached anyway. Under highest-wins a fabricated reading competes, and this one wins: on a host with a genuine ROCm 6.1 the N/A line resolved rocm6.4 instead of rocm6.1. The field is now cut at the separator and required to start with digits, so an N/A or empty field contributes nothing instead of a wrong number. * Extract the ROCm version helpers in the no-version endpoint test The test builds a shell script out of get_torch_index_url plus the two gfx helpers, but the version chain now lives in its own helpers, so the call to _detect_rocm_version_tag hit a command that does not exist. The guarded assignment swallowed the 127 and the no-version endpoint was reached for the wrong reason, which made the test pass whatever the resolver did. Mutation confirms it: returning a real tag from _detect_rocm_version_tag left the test green before this change and fails it after. The /opt/rocm/.info/version rewrite moves onto the helper text for the same reason. The read moved out of get_torch_index_url, so rewriting the function body alone had quietly become a no-op. * Bound the rpm probe highest-wins made unconditional Resolving the ROCm version from every source instead of the first that answers moved `rpm -q rocm-core` from last in a first-answer-wins chain to a probe that runs on every AMD host. On RHEL/SLES with a normal ROCm install /opt/rocm/.info/version answered at position two, so rpm was never invoked; now it always is. rpm -q is not a lock-free read. On the BerkeleyDB backend (rpm < 4.16, so RHEL 8 and SLES 15, both supported ROCm platforms) a leftover /var/lib/rpm/__db.00* from any killed rpm or yum leaves plain queries stuck in futex until the files are removed, and rpm 6.0.x reintroduced a read lock that deadlocks rpm --query against a running dnf transaction. Route it through the existing _run_bounded helper so a wedged rpmdb makes the source decline to answer instead of hanging the installer. _run_bounded no-ops where timeout is absent, so this adds no new dependency. The other four sources are left alone: amd-smi ran first in the old chain so it is not newly reachable, the version file is a plain read, hipconfig --version reads a version file and is not linked against HIP or HSA, and dpkg-query takes no dpkg lock at all. * Tighten the ROCm version resolution comments --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-11 05:55:17 -07:00
Wasim Yousef Said	f567ae8f39	Studio: skip redundant packaged frontend rebuilds (#8326) ... * Desktop: skip frontend rebuild during updates * Tests: tolerate rustfmt in updater UTF-8 contract * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use packaged frontend for PyPI installs * Studio: keep the packaged frontend skip off source checkouts STUDIO_LOCAL_INSTALL records where the Python package came from, not which tree setup runs out of. An editable overlay separates the two: with UNSLOTH_CI_SOURCE_OVERLAY, or in a venv left editable by an earlier --local run, the mode stays 0 while SCRIPT_DIR is a checkout whose dist is a stale build artifact rather than a release one. The skip then serves that stale dist and a source change silently never reaches the browser, which is the outcome the overlay legs of clean-machine-install-ci exist to catch. A wheel ships no top-level files, so a pyproject.toml next to studio/ marks the tree as source. Require its absence before trusting the packaged dist; site-packages installs are unaffected and still skip. Also check the Tauri branch before the packaged one in setup.ps1 so a desktop update reports the same reason it reports on POSIX. Covered by new cases in tests/sh/test_packaged_frontend_skip.sh and tests/studio/test_node_decision.ps1. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-11 03:02:17 -07:00
BardiaKoopah	9856d99bce	macOS: stop the Xcode CLT dialog during venv creation (#8380) ... * macOS: stop the Xcode CLT dialog during venv creation Before downloading the managed build it was asked for, uv walks PATH and executes each interpreter it finds to read its version. On a Mac with no Command Line Tools, /usr/bin/python3 is Apple's xcode_select tool shim (byte-identical to /usr/bin/git and /usr/bin/clang), so that probe pops the "command line developer tools" dialog naming python3, for a toolchain this install never uses. These three call sites already request an explicit managed build (cpython-X-macos-aarch64-none), so --python-preference only-managed only stops the search. It changes nothing about which interpreter is chosen. Left alone deliberately: the non-arm64 branch a few lines down takes _python_request, which passes through a user's --python override or an explicit interpreter path, and only-managed would ignore an interpreter they asked for. First install only. Once uv has a matching managed CPython cached it does not probe PATH at all, so a re-run was already quiet either way. Note the ordering this removes: _check_macos_deps at 2233 already prints "no Xcode Command Line Tools (not required)" before uv then prompts for them anyway. * Keep the offline system-Python fallback for PR #8380 only-managed also disables uv's system-interpreter fallback, so an offline Mac with no cached managed build went from a working venv to a fatal 'No interpreter found ... in managed installations'. Route the three arm64 sites through _uv_venv_arm64, which retries the request unflagged when the managed-only attempt fails, and add a regression test. * Harden the macOS venv fallback for PR #8380 Simulated the venv block across 90 platform combinations (OS x arch x --python x pre-existing venv) and confirmed the uv argv is byte-identical to main everywhere except macOS arm64, where only the flag is added. Two more configurations join offline in losing the install without a fallback: UV_PYTHON_DOWNLOADS=never and UV_NO_MANAGED_PYTHON=1. Note why the flag is spelled --python-preference (accepted since uv 0.4.30, vs 0.8.16 for --managed-python), and guard the Studio installer stream: the recovery has to emit ERROR_CLEAR or install.rs keeps reporting a failure it recovered from. * Tighten the comments added for PR #8380 Comment-only pass, verified: with comment and blank lines stripped, all three changed files hash identically to 4687bac18. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-11 02:34:05 -07:00
Daniel Han	8080c93c9e	Studio: install sd.cpp under the Studio home, not beside it (#8226) ... * Studio: install sd.cpp under the Studio home, not beside it A custom UNSLOTH_STUDIO_HOME put the managed stable-diffusion.cpp tree at the home's parent, unlike llama.cpp, whisper.cpp and node, which all install under the home. For a relative home that parent collapses to the working directory, so a stable-diffusion.cpp checkout sitting there was picked up as the managed install and install_sd_cpp_prebuilt refused to run: the target was a pre-existing non-empty directory without the ownership marker. Derive the root as "studio home"/stable-diffusion.cpp from an absolutised home, in both the installer and the engine's finder, matching default_managed_llama_dir. The legacy default home ~/.unsloth/studio still maps to ~/.unsloth/stable-diffusion.cpp, and a tree an older build installed beside the home is still discovered and still repairable, gated on the ownership marker so an unrelated checkout is never adopted. * Stop the sd.cpp server under a custom root before uninstall deletes it Moving the managed tree under the Studio home left the uninstaller listing only the old sibling location as an owned root. The tree itself still goes, because the custom root is removed wholesale, but a resident sd-server survives unlinking its binary, so it kept running and holding its port while its install disappeared underneath it. List both locations, each still gated on the ownership marker so a checkout the user keeps at either path is never signalled. The Windows script needs no equivalent change: its handle scan already walks every known root by prefix, which now contains the tree. Comments in both scripts updated to describe where the install actually is. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Read the sd.cpp accelerator record from the root the binary is in The accelerator upgrade check read the record from the current managed root while the finder may hand back a binary from the tree an older build installed beside the Studio home. That root holds no record, unrecorded reads as a mismatch for a GPU target, and the matching bundle already on disk gets downloaded again on every load. owning_managed_root() returns the marker-owned root a binary actually lives under, and the check reads the record from there. is_managed_binary is now a thin wrapper over it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Find a legacy install beside a symlinked home, and never serve a stale legacy server after an install - The compatibility lookup resolved the home before taking its parent, while the old installer took the lexical parent. For a home that is itself a symlink the tree an older build created sits next to the LINK, so the lookup missed it: a needless re-download, and the old install left orphaned from the uninstaller too. Lexical first, resolved after, both still marker-gated. - After an install that ships no sd-server, the finder's legacy probe could hand back the old server built for another accelerator, and ensure_sd_server_binary returned it without rechecking. A forced CUDA load then ran the legacy CPU server. It now returns None there, so the router uses the sd-cli the install just landed. * Apply the kwarg-spacing formatter to two files it had not been run over * Clear the legacy sd.cpp beside a symlinked Studio home, and stop reinstalling a serverless bundle Uninstall derived the legacy sibling from the canonicalized custom root, so for a Studio home that is itself a symlink it looked beside the link's TARGET while an older build had installed beside the LINK: that tree was never stopped and never removed. Take the lexical parent as an extra marker-gated candidate, matching what the finder already does. And once the current managed root holds a completed install for the accelerator being asked for, a mismatched sd-server still sitting in that legacy tree is not a reason to install again: the bundle simply shipped no server. Without this, every model load re-downloaded it. * Make the serverless suppression evidence-based, and close two gaps in the uninstall stop pass Four follow-ups on the review of the previous commit. The serverless guard fired on the accelerator record alone, which reads a server that was deleted by hand (or by the runnability repair) as a bundle that never had one, and then suppresses the very reinstall that would put it back. install() now records ships_server, taken off the archive member list, and the guard needs that recorded false plus a genuine mismatch on the legacy side. An unrecorded install stays unknown and keeps its old behavior, and a legacy server that matches the accelerator being asked for is still preferred over the one-shot CLI. The guard also has to run before _accelerator_changed, which reports "unchanged" while the managed tree is in use, so a load starting during a generation was handed the mismatched legacy server. In uninstall.sh, the lexical sibling reached the string-based deny list without being resolved, so a home carrying ".." could aim a removal at a protected tree; canonicalize a copy for that check. And the nested <root>/stable-diffusion.cpp is now stopped even when unmarked, provided the root is a Studio root this run deletes: the current-root finder can select an unmarked binary there, and the marker gate belongs to the paths that survive when unowned, not to a tree that goes regardless. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Memoise the bundle's server capability with the accelerator, not separately An unwritable install record already kept the accelerator in process memory, so a serverless install whose record could not be written read back as server-capable and the load that finds a mismatched legacy server went on reinstalling. Remember both together, or neither is trustworthy. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give the Windows stop scan the physical path of a linked Studio home The backend resolves the home before deriving the nested sd.cpp tree, so on a junction or directory symlink the native binaries run out of the target. _CustomStudioRoots only normalizes the string (System.IO.Path.GetFullPath is lexical and never follows a reparse point), and the scan matches Win32_Process.ExecutablePath by prefix, so the running server was never matched and survived an uninstall that deleted its tree. Add the reparse target to the stop scan only; the deletes still refuse to chase a link out of the expected location. * Scope both uninstall stop paths to trees this run actually owns The lexical sd.cpp sweep skipped the ownership check the canonical loop makes first, so a stale or mistyped UNSLOTH_STUDIO_HOME pointing at a path that was never a Studio could take the marked legacy sibling of a different, valid install. Apply _is_studio_root there too. On Windows the stop scan was handed the whole physical target of a linked home. The delete leaves that target standing, so anything there that is not ours is neither locking nor being removed; pass the Studio-managed subtrees underneath it instead, and only for the homes, since a component dir can itself be a link onto a shared runtime. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-10 07:03:20 -07:00
Nilay	54462b80c7	Fix two macOS installer bugs (#8279) ... * Installer: stop planting a self-referential symlink inside the macOS launcher bundle * Uninstaller: keep the desktop app's data when that app is still installed * Cover the uninstaller's app-data gate in tests * Find the app that owns the bundle id instead of guessing its path for PR #8279 A packaged app kept in a renamed bundle or a subdirectory such as /Applications/AI & ML/Unsloth.app missed all three hard-coded candidates, so the uninstaller still wiped its settings, cookies and WebView storage. Match on CFBundleIdentifier via mdfind plus a scan of /Applications and ~/Applications, excluding install.sh's launcher, which shares the id but only opens a browser. * Drop the depth cap on the bundle scan for PR #8279 A bundle filed deeper than three levels, say /Applications/Development/AI/Local/ Unsloth.app, fell outside -maxdepth 3, so the uninstaller wiped the installed app's data. Prune at each .app instead: the walk never enters a bundle, which is what the cap was avoiding, so any nesting depth is reachable and free. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-09 22:40:14 -07:00
Long Yixing	c7ffee7bcf	Resolve latest mlx-vlm and route its new processors (#8251) ... * fix(installer): prepare Apple Silicon dependencies * fix(studio): route mlx-vlm processors to transformers 5	2026-08-09 15:39:34 +03:00
Daniel Han	9f37302e2e	Report a denied UNSLOTH_LOCAL_LLAMA_CPP_DIR instead of aborting on errexit (#8143) ... * Report a denied UNSLOTH_LOCAL_LLAMA_CPP_DIR instead of aborting on errexit * Report a denied ancestor and stop on a denied install parent Two gaps in the first pass, both reproduced. A build under an unsearchable ancestor cannot be stat'd, so `[ ! -d ]` read it as missing and setup said "does not exist", sending the user to correct a path that was already right. `_report_denied_ancestor` walks up to the deepest ancestor it can stat and reports that one, so the message names the directory that actually needs the permission change. A genuinely absent path still reports as missing. Keeping the textual path when the install parent could not be canonicalized only moved the abort: `rm -rf` fails, `[ -e ]` is false because the parent cannot be stat'd, and the following `ln -sfn` then aborts under errexit with no [TAURI:ERROR]. Nothing can be written under a parent we cannot search, so report there instead of continuing. * Follow symlinks and terminate dirname options when locating a denied ancestor * Strip trailing slashes before probing for a symlinked denied ancestor * Tighten the denied-ancestor comments in setup.sh and its test --------- Co-authored-by: danielhanchen <moonshotaisubstack@gmail.com>	2026-08-08 01:18:24 -07:00
oobabooga	69d555b98e	Studio: check llama.cpp cache access before setup (#8032) ... * Windows: preflight managed llama.cpp cache access before setup and update * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix denied llama.cpp cache update errors * Trim canonical paths on both branches and probe listing on POSIX for PR #8032 Follow-ups from reviewing the denied llama.cpp cache preflight. Get-CanonicalDir only trimmed a trailing separator on the lexical branch. Resolve-Path preserves one, so "...\studio\" compared unequal to "...\studio" and Test-StudioHomeIsCustom called the default home custom. Get-ManagedLlamaCppDir then handed the preflight ...\studio\llama.cpp instead of the real cache, so a denied cache went undetected. The trim now runs after both branches, still guarded so a path root keeps its separator. setup.sh gated the prebuilt install on _studio_dir_unsearchable, which probes search (+x). Mode 111 passes that and still raises PermissionError inside install_llama_prebuilt.py, which lists the tree. Added _studio_dir_unreadable for call sites that list or replace rather than probe a known child. setup_fail only emitted [TAURI:ERROR] under UNSLOTH_TAURI_MODE. update.rs sets UNSLOTH_TAURI_UPDATE on every platform, so macOS and Linux desktop updates still showed "Update exited with code N" while Windows showed the reason. A blank USERPROFILE threw a raw binding error from the relocated resolver before Exit-SetupFailure could report it, so no [TAURI:ERROR] reached the app. Tests: the marker-readable, listing-denied case was asserted only as a literal substring, so one extra space reintroduced the early return with all 49 Python tests and all 125 PowerShell checks green. That assertion is now a regex, and the PowerShell suite builds the shape for real (icacls /deny :(RD) on Windows, mode 111 on POSIX) with a negative control. The shell suite gains the same mode 111 case and no longer compares empty grep output as an integer. * Fix the Tauri marker gate and narrow the USERPROFILE guard for PR #8032 Two defects in my previous commit, both found by simulating the changes rather than reading them. setup_fail joined both variables into one case subject, which is not an exact match. "*,1" matches any subject ending in ",1", so an unrelated UNSLOTH_TAURI_UPDATE=a,1 printed a stray [TAURI:ERROR] on a plain CLI run, and "1,*" did the same for a comma in UNSLOTH_TAURI_MODE. Testing each variable separately matches setup.ps1, which uses exact membership. Verified over the 144-case product of both variables: no false positives, no false negatives, exit codes preserved, and byte-identical to main for every value of UNSLOTH_TAURI_MODE when UNSLOTH_TAURI_UPDATE is unset or 0, so no existing CLI invocation changes. Added a regression test that fails on the old gate. The USERPROFILE guard used IsNullOrWhiteSpace, but Join-Path only rejects null and empty; it accepts a whitespace-only value. The guard therefore also stopped a run that previously completed, USERPROFILE=" " with a fully qualified UNSLOTH_STUDIO_HOME. IsNullOrEmpty keeps the clean message for null and empty, which is where the raw binding error was, and changes nothing else. * Run the llama.cpp access guard before the prebuilt and source branches The guard sat inside the prebuilt else-branch, so UNSLOTH_LLAMA_FORCE_COMPILE=1, a llama.cpp PR or source override, or anything else setting _SKIP_PREBUILT_INSTALL bypassed it. Those paths reach the phase 9 swap, which only probes access after `rm -rf "$LLAMA_CPP_DIR"` has already failed, so a denied cache stranded a completed source build instead of failing before it started. _assert_studio_owned_or_absent does not cover it either: it returns early unless the studio home is custom, so a denied default cache had no guard on that path at all. Verified by driving both helpers against a mode-000 tree: the ownership guard returns 0 on a default home while the access probe reports denied. Hoisted both checks, in the same order so the custom-home wording still wins, to just before the branch. The local-link paths are excluded because they already replaced or reused the tree. The late checks stay as defense in depth. * Use the read probe in both llama.cpp replace postconditions Mode 111 defeats `rm -rf` but stays searchable, so both postconditions fell through `_studio_dir_unsearchable` to the generic "could not be replaced" text and the user got no recovery guidance. Reproduced against a mode-111 tree: the rm fails, the search probe does not fire, and the run exits with the generic message. The local-link site sits above the hoisted guard, so it is the one that is reachable. The source-build site is only reachable when a tree becomes unreadable during the build, but that is the most expensive path to end with the wrong message, and it is the same one-word probe. Both now use `_studio_dir_unreadable` and print the permissions block. * Tighten the comments added by this PR --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <moonshotaisubstack@gmail.com>	2026-08-07 22:33:42 -07:00
Michael Han	6f4cce68d1	Setup: clear stale WebView caches on install and update, keep user settings and data (#7361) ... * Setup: clear stale WebView caches on install/update, keep user data The desktop app's WebView caches (keyed by the Tauri bundle id ai.unsloth.studio) hold copies of the previous frontend and keep serving them after an update, so old styles linger even though the code on disk is new. Clear cache-only paths on every install/update via setup.sh and setup.ps1 (both fresh installs and 'unsloth studio update' route through them). Cleared: the HTTP/network caches, CacheStorage, service workers, and GPU caches for the bundle id. Kept: LocalStorage, IndexedDB, cookies, settings, models, and the studio database. macOS: ~/Library/Caches/<bid> and WebsiteData/{CacheStorage, ServiceWorkers,DiskCache}. Linux: XDG cache dir. Windows: EBWebView\Default\{Cache,Code Cache,GPUCache,Service Worker}. Adds tests/sh/test_setup_webview_cache_clear.sh covering both OS branches, XDG overrides, and that user-facing storage survives. * Address review: Linux data-dir caches, bash invocation, pre-webview clear Three review findings, all verified: 1. wry keys the WebKitGTK base-cache dir to the app DATA dir (same as base-data), so on Linux the stale frontend cache also lives under ~/.local/share/ai.unsloth.studio. Clear the cache-typed subdirs there (WebKitCache, CacheStorage, serviceworkers) while keeping localstorage, indexeddb, and cookies. Tests extended to 23 assertions. 2. run_all.sh invoked the new test with sh, but the extracted setup.sh function uses bash arrays; dash aborts with a syntax error before any assertion. Invoke with bash. 3. The in-app desktop update runs start_backend_update before downloadAndInstall/relaunch, so setup.ps1/setup.sh clear caches while the live WebView still holds them and silently fail. Clear the caches from the Rust side in main() before the Builder runs: the config window (and the WebView lock) exists by the time setup hooks fire, so this is the one point where the profile is guaranteed unlocked. Same cache-only path lists per OS; cargo check passes. * Address review: gate the native clear on the app version, correct the macOS paths * Tighten the comments added for PR #7361 * Serialize the WebView cache clear, stamp only a full clear, ignore relative XDG_DATA_HOME * Take the profile lock before the stamp check, and detect dangling symlinks in the test * Clear WebView caches only after the install-root override is validated * Invalidate the app version stamp when setup clears the WebView caches * Tighten the comments in the WebView cache clear * Clear WebView caches after validating the override on Windows too * Only clear WebView caches for a real Studio installation * Tighten the comments added since the last pass * Clear the WebView cache after single-instance arbitration Resolve the profile through dirs::data_local_dir, the same call Tauri's PathResolver makes for BaseDirectory::LocalData, so a launch with HOME or LOCALAPPDATA stripped still finds the profile Tauri is about to use. Move the clear into a plugin registered directly after single-instance. Plugin setup hooks run inside Builder::build() in registration order and the config window is only created later from App::run(), so a duplicate launch has already exited by then and the WebView does not exist yet. Make the no-venv fixture hermetic: report no system node/npm and opt out of the isolated install so setup.sh skips Node provisioning and the frontend build, and assert it still aborts at the venv check. * Shorten the comments added in the last commit --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-07 06:35:01 -07:00
Michael Han	6712c7513d	Uninstall: remove WebView runtime data so reinstalls don't serve a stale frontend (#7360) ... * Uninstall: remove WebView runtime data (stale frontend after reinstall) The desktop app's WebView creates runtime data keyed by the Tauri bundle id (ai.unsloth.studio) at first launch, not at install time, so the uninstallers never removed it. A leftover WebKit/WebView2 cache then serves a stale frontend bundle to the next install, showing old styles after a supposedly clean reinstall. uninstall.sh: remove ~/Library Caches/WebKit/Application Support/ HTTPStorages/cookies/saved state/prefs on macOS and the XDG cache/data/ config/state dirs on Linux; stop the desktop app binary first. uninstall.ps1: remove LOCALAPPDATA (EBWebView profile) and APPDATA dirs for the bundle id; stop the desktop app and any msedgewebview2.exe helper holding handles on them first. Adds tests/sh/test_uninstall_webview_data.sh running the full script against a fixture HOME for both OS branches. * Test: never enter the real WSL cleanup from the webview-data test Stubbing uname alone is not enough on a WSL host: the script's 'grep -qi microsoft /proc/version' probe still fires and the Linux test cases would run the real WSL cleanup against the host's /mnt/* shortcuts and /etc profile. Add a PATH-stubbed grep that fails only the /proc/version probe (delegating everything else to the real grep via an absolute path so the stub can never self-exec), plus no-op powershell.exe and sudo stubs as defense in depth. * Test: sandbox XDG_RUNTIME_DIR and drop the GNU-only mktemp -p * Tighten the comments added for PR #7360 * Final comment pass for PR #7360 * Ignore relative XDG home overrides when removing WebView data * Final comment pass for PR #7360 * Scope the Studio process stop to the caller's session for PR #7360 * Scope the Studio app kill to the target user for PR #7360 * Scope by owning account, run defaults as the home owner, and de-vacuum the kill assertion for PR #7360 * Resolve a symlinked home and stop claiming removal succeeded for PR #7360 * Report failed removals in the PowerShell summary and across the custom-root subshell for PR #7360 * Count a deny-listed custom root as incomplete removal for PR #7360 * Remove the deep-link handler .desktop entry on uninstall * Only claim chat history is gone when a studio.db was actually removed * Refresh the desktop database in the directory the handler was removed from * Count a deny-listed custom root as incomplete cleanup in uninstall.ps1 * Harden the uninstall summary markers and the WebView2 helper sweep * Keep the uninstall summary conservative when it cannot account for a removal * Confirm the database is really gone before the summary says so * Stop claiming the uninstall removed provider API keys * Resolve a symlinked studio.db without relying on readlink -f * Scope the signed-out claim and anchor relative database links * Stop the legacy-named desktop process on Windows too * Anchor a relative install-root link before testing for the database * Tighten the comments in the WebView cleanup paths --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-07 06:08:28 -07:00
Michael Han	89206b63a8	Studio: cap llama.cpp source build parallelism by RAM (#8061) ... * Studio: cap llama.cpp source build parallelism by RAM The source build passed the logical core count straight to cmake -j. On a 20-thread 16 GB machine that is -j20, and each nvcc translation unit peaks near 2 GB, so the build oversubscribed memory and left the machine unresponsive with no way to stop it. Budget the job count from total physical RAM instead: reserve 2 GB for the OS, allow 2 GB per job, and clamp to the core count. RAM is only ever a ceiling, so machines with headroom keep every core. When RAM cannot be read the job count falls back to the old core count rather than guessing low. UNSLOTH_LLAMA_BUILD_JOBS overrides both. setup.ps1 carries its own copy because it cannot source setup.sh; the reserve and per-job constants are pinned to matching values by the test. * Address review: honour the cgroup memory limit when capping jobs /proc/meminfo is not namespaced, so inside a memory-limited Docker or Kubernetes container it reports the host's RAM. A 4 GB container on a large host therefore still budgeted from the host and would be OOM-killed during a source build, which is the same failure this cap exists to stop. Read the cgroup v2 then v1 memory limit and take the lower of it and MemTotal. v2's 'max' and v1's unlimited sentinel are not limits: the former does not parse, the latter loses the comparison on its own. Windows has no cgroups, so setup.ps1 is unchanged. * Address review: resolve the binding cgroup, and budget from available memory Two follow-ups on the cap. The first cgroup reader only looked at the hierarchy root. That is the binding limit inside a container with a private cgroup namespace, but under Slurm, systemd slices or --cgroupns=host the limit sits on the process's own path or an ancestor's, and the root reads 'max'. Those environments kept the original OOM behaviour. Mirror unsloth/dataset_num_proc.py instead: read /proc/self/cgroup, walk innermost-first to the root, pair each limit with the usage of the directory that set it, and take the smallest remaining allowance. memory.high binds as much as memory.max. MemTotal also ignored what the machine is already doing, so a 16 GiB box with 8 GiB resident still got a 14 GiB compile budget, which is the thrashing this change exists to stop. Budget from MemAvailable, which counts reclaimable page cache, falling back to MemTotal on pre-3.14 kernels. Windows has no cgroups but the same installed-versus-available gap, so it reads AvailableMBytes from the raw perf class, which is not localized the way Get-Counter paths are, and keeps installed RAM as the fallback. The readers take the cgroup root and /proc/self/cgroup as arguments, so the tests drive real trees rather than faking /sys. * Address review: resolve the cgroup mounts instead of assuming their paths The v1 branch assumed the memory controller sits at <root>/memory. It usually does, but a co-mounted or relocated hierarchy was skipped even when /proc/self/cgroup named a memory controller, and the budget then fell back to host memory, which is the silent revert this cap exists to prevent. The v2 branch assumed the unified mount is the root, which systemd hybrid mode breaks the same way. Resolve both from /proc/self/mountinfo: match fstype cgroup2 for the unified mount, and the memory controller by super option for v1, so a co-mount is found by name rather than by path. The conventional layout stays as the fallback when mountinfo is unreadable. unsloth/dataset_num_proc.py still assumes <root>/memory for v1; this is deliberately the more thorough of the two rather than a drift. * Address review: map the process path through the cgroup mount root _cg_mount kept mountinfo field 5 and discarded field 4, the filesystem root that mount exposes. A bind-mounted subtree, which is what Docker without a cgroup namespace and systemd slices produce, shows a mount root like /slice while /proc/self/cgroup still reports the host-absolute /slice/job, and the files are then at <mountpoint>/job. Joining the two unmapped walked a path that does not exist, so the walk settled on an outer limit instead of the binding one and over-estimated the allowance. Return the mount root alongside the mount point and strip it from the process path before walking. A process outside the mount's root gets only that mount's own limit, since nothing below it describes that process. * Address review: pick the cgroup mount that contains the process A hierarchy can be mounted more than once, and _cg_mount stopped at the first match. An unrelated subtree listed ahead of the binding mount meant the process path did not map into the chosen mount, so the hierarchy was dropped and the budget fell back to host memory. Collect every matching mount and choose one whose root contains the path from /proc/self/cgroup, most specific first, keeping the first seen when none of them does. The process path is now read once, before mount selection, since the choice depends on it. * Address review: decode mountinfo escapes, keep colons in cgroup paths mountinfo represents a space, tab, newline or backslash in a path as an octal escape, and both path fields were used verbatim, so a hierarchy mounted under such a path resolved to a directory that does not exist. Decode field 4 and field 5 before use. strtonum is a gawk extension, so the conversion is plain arithmetic that also runs under mawk and BSD awk. /proc/self/cgroup was split with -F: and read as field 3, which truncates the path at the first colon inside it. A systemd unit name may contain one. Only the first two colons are delimiters, so parse those and keep the remainder, for both the v2 line and the v1 controller lines. * Address review: keep zero available memory, and stop the newline transports The mount point is an arbitrary directory and may contain a newline, which mountinfo escapes as \012. Decoding it where the fields are read put that newline back into the reader's own line-oriented output, splitting one mount record into two, and _cg_dirs then walked a line-delimited list of paths that could be split the same way. Both records now travel escaped and NUL-delimited, and a single path is decoded once it is in hand. On Windows, an AvailableMBytes of 0 is a reading, not a failure. Falling back to installed RAM there handed a machine with nothing left its full core count, which is the case this cap exists for. Unreadable is now -1. The suite's host-memory assertions compared a cached MemAvailable against a second live read, which raced on Linux. _usable_ram_mb takes its meminfo path like every other reader here, so the numbers are pinned. * Pin the trailing-newline sentinel on the decoded mount path * Address review: budget macOS from available memory too The macOS branch still read hw.memsize, which is installed RAM, so a 16 GiB Mac with 8 GiB resident was handed a 14 GiB compile budget, the same gap MemAvailable closed on Linux. macOS has no MemAvailable. free + inactive + speculative + purgeable is the reclaim-aware equivalent, and the page size comes from the vm_stat header rather than being assumed 4096, which is wrong on Apple Silicon. Installed RAM stays the fallback when the output does not parse. Verified on a 64 GiB Apple Silicon host: 65536 MiB installed, 33157 MiB usable, 15 jobs rather than the bare core count. * Address review: keep a failing read from aborting the install setup.sh runs under `set -euo pipefail` and NCPU=$(_llama_build_jobs) is on the install's critical path, so a helper that returns non-zero does not degrade the job count, it aborts the install at the build step. The old NCPU line could not fail; this one calls ten helpers. _cg_read ran `head | tr`, and `[ -r "$1" ]` does not rule the pipeline failing out: a directory passes it, and a cgroup can be torn down between the test and the open. It now reads with the builtin and strips with an expansion, so there is no pipeline and no subprocess left to fail. The two awk calls in _cg_mounts and _cg_unesc are guarded for the same reason. The suite sourced the helpers into a plain `bash -c`, which is why none of this was visible. It now also drives them with the real shell options on, including the two AND-lists in _llama_jobs_for that are the classic footgun. * Guard the remaining reads, and pin them under POSIX mode bash applies errexit to a failing assignment in POSIX mode, which it does not do by default, so `m=$(awk ...)` inside a helper aborts the install for anyone with POSIXLY_CORRECT exported or bash invoked as sh. The meminfo and cgroup reads are guarded like the others, and the suite now drives the helpers under --posix as well, which is what makes those guards observable. Also pins the two properties the earlier guards left untested: _cg_read returns rather than blocking on a FIFO, which is what the -f test is really for, and a padded value is still parsed, which is what the whitespace strip is for. * Pin the read guards in the form that distinguishes them The POSIX assertions passed the reader as an argument, where the failure status is discarded, so they held with the guards removed. The distinguishing form is the assignment, which is what the real call site uses, so they assign now. Removes the guard on the _cgroup_free_mb read: that function is guarded internally and ends in `return 0`, so it cannot fail, and a guard no test can make fire is worse than none. * Ground the per-job budget in a measurement The constant was justified as "an nvcc translation unit peaks near 2 GiB". Measured against llama.cpp master with CUDA 13.1, the heaviest units are the flash-attention template instances at ~400 MiB, flat in arch count, and a full CUDA build at -j20 peaks at 8.2 GiB in aggregate across ~30 concurrent compiler processes. The number does not change: 2048 still has to cover the process fan-out, MSVC and hipcc which are not measured here, older toolkits which were far heavier on the same files, and the link step. But the comment now says what was measured and why the budget sits above it, rather than asserting a figure the current toolchain does not produce. * Bring the macOS reader under the same guards, and make its test run on Linux Two things on the vm_stat commit, both the same shapes already fixed on the Linux side of this change. _vm_stat_avail_mb ends in an unguarded awk, so under POSIX mode (POSIXLY_CORRECT in the environment, or bash invoked as sh) a vm_stat that does not parse aborts the install rather than falling back to installed RAM. And "_usable_ram_mb prefers vm_stat over hw.memsize" says it runs on Linux runners, but the branch it targets is an elif on sysctl succeeding, and Linux sysctl has no hw.memsize, so the branch was never entered and the assertion held vacuously off a Mac. It stubs sysctl now, which is what actually reaches it, and a companion assertion covers the fallback when vm_stat gives nothing. Mutation-checked: unguarding the awk fails 2, ignoring vm_stat fails 3, assuming a 4096 page size fails 4, counting active and wired fails 3. * Restore the vm_stat awk guard Belongs with the previous commit; the guard was dropped when the tree was reset between the mutation checks and the commit. * Keep a zero vm_stat reading rather than treating it as a parse failure Get-UsableMemoryMb returns 0 when AvailableMBytes is 0 and reserves -1 for an unreadable one. The shell reader did not make that distinction: a Mac with nothing reclaimable read as unparseable and fell back to hw.memsize, which is installed RAM, so the machine got its full core count. That is the oversubscription this PR exists to remove, on the one input where it matters most. Zero is a reading now; only a missing page size is not. * Treat a zero cgroup limit as a limit _cg_limit rejected a zero-byte limit as though it were the absence of one, so a cgroup that permits no further memory budgeted from host MemAvailable and took the full core count, which is the oversubscription this cap exists to remove. Only memory.high can be observed at zero: it throttles rather than killing (cgroup-v2.rst, "Going over the high limit never invokes the OOM killer"), so a process really does run under systemd's MemoryHigh=0. memory.max of 0 invokes the OOM killer, so nothing survives to read it. Zero is now a reading here, the same as on the Windows and macOS sides. Mutation-checked: restoring the -gt 0 test makes the fixture report its ancestor's 4 GiB instead of 0, and fails the new assertion. * Inspect every containing cgroup mount, and make the suite hermetic A limit above the narrower mount's root is invisible through that mount and visible through the broader one, so picking only the most specific hid it. The same hierarchy really is mounted twice with different subtree roots: rootless podman inside rootless podman leaves a host-derived bind mount beside a namespace-scoped one (containers/podman#21376). _cg_pick_mounts now yields every mount whose root contains the process path and the walk takes the smallest allowance across all of them, which is order-independent and can only lower the budget, never raise it. Three test-isolation defects alongside it, each of which would have gone red on a runner rather than on this box: - _usable_ram_mb hardcodes /sys/fs/cgroup, so inside a memory-limited container the assertions about HOST memory received the container's allowance instead of the fixture. In a 4 GiB cgroup they return 1 where they expect 5 and 20. The clean-machine install CI runs in containers, so this was reachable. Anything not testing the cgroup reader itself now stubs it out. - The FIFO case ran wherever mkfifo exists, but stock macOS ships no GNU timeout, so it exited 127 and failed a correct reader. It requires both now. - The suite inherited an exported UNSLOTH_LLAMA_BUILD_JOBS into every direct helper call, so a developer with the override set saw unrelated failures. It is unset once at the top; run_jobs still sets it explicitly per call. Mutation-checked: most-specific-only fails 2, and the isolated zero-limit fixture now fails 2 rather than 1 when the -gt 0 test is restored. * Stop double-counting macOS reclaimable pages free + inactive + speculative + purgeable overstates what can be reclaimed, because neither of the last two is disjoint from the first two, and overstating available memory buys back exactly the oversubscription this cap removes. speculative is a subset of free, stated outright in xnu osfmk/mach/vm_statistics.h: "NB: speculative pages are already accounted for in free_count, so speculative_count is the number of free pages that are used to hold data that was read speculatively from disk". purgeable is an attribute of a page rather than a queue it sits on, so a volatile page is already counted on whichever of the active or inactive queues holds it; the disjoint partition is free + active + inactive + wired + throttled + compressor. The sum is free + inactive. That under-counts by the purgeable pages on the active queue, which are reclaimable but in neither term, and that is the safe direction here: it costs build time on a busy Mac rather than the machine. The fixture keeps non-zero speculative and purgeable counts so that adding either back is visible. Mutation-checked: restoring speculative fails 6, purgeable fails 6, both fails 6, and dropping inactive fails 6. * Tighten the build job cap comments --------- Co-authored-by: shimmyshimmer <133493246+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-08-07 04:12:10 -07:00
Daniel Han	c2a91dbe6d	Revert "Add option to launch Studio without opening the default browser (#7016)" (#8040) ... This reverts commit 3860d69ab2.	2026-08-06 08:39:06 -07:00
Michael Han	3860d69ab2	Add option to launch Studio without opening the default browser (#7016) ... * Add no-browser launch option for the Studio desktop launcher The generated launchers (launch-studio.sh / launch-studio.ps1) always opened the default browser once the server became healthy. Add a --no-browser launcher flag, the UNSLOTH_STUDIO_NO_BROWSER env var, and a persisted installer preference (studio.conf / baked into the ps1 launcher) with an interactive install prompt. When auto-open is off the launcher still starts or attaches to the server and prints the URL, for users who run Studio as a browser PWA or app window. The --shortcuts-only refresh run by studio update preserves the choice. * Make UNSLOTH_STUDIO_NO_BROWSER falsy check case-insensitive in the shell launcher Simulation testing caught that False or Off disabled the browser in launch-studio.sh while the PowerShell launcher's -notin treats them as falsy case-insensitively. Lowercase the value before matching so both launchers agree, and pin the behavior in the launcher test. * Run the launcher no-browser shell test in CI The shell installer test step runs a hardcoded list, so the new tests/sh/test_launcher_no_browser.sh was only bash -n parsed by lint and never executed. Add it to the list; it only reads install.sh and install.ps1 and writes to mktemp sandboxes, so it fits the step's no-writable-tree constraint. * Keep the saved browser preference when the reinstall prompt is accepted An interactive reinstall over an install that had persisted STUDIO_OPEN_BROWSER='0' would flip it back to 1 when the user pressed Enter, because the prompt default was hardcoded to yes and the answer then overrode the preserve logic. Seed the prompt default from the existing preference (studio.conf on macOS/Linux/WSL, the value baked in launch-studio.ps1 on Windows) and flip the hint to [y/N] accordingly. Explicit y/n answers still override. * Silence SIGPIPE noise in the launcher no-browser test grep -q exiting on first match SIGPIPEs the echo feeding it when the haystack is the whole installer, spamming 'write error: Broken pipe' in the CI job log. Feed grep from here-strings instead. * Address review feedback: reroute flags, curl EPIPE, post-install browser open Three fixes from PR review and field testing: - Forward an explicit --no-browser/--browser choice into the WSL Strix Halo reroute so the rerouted install honors the flag. - Drain piped stdin before the --shortcuts-only early exit so curl | sh -s -- --shortcuts-only no longer dies with curl error 23. - Open the browser after the installer's own foreground launch when the preference is on: a background watcher polls /api/health, verifies the per-install studio_root_id so a different Studio on the port is never opened, then opens the URL once. Mirrored in install.ps1 with a Start-Job watcher. When the preference is off nothing changes; the server already prints its URL. * Use selected port for post-install browser watcher * Gate the browser prompt on skip-autostart, stub the WSL openers in its test Addresses the two open review items, plus one leak found while confirming them. UNSLOTH_SKIP_AUTOSTART is documented for automated installs, but the new browser prompt did not check it: under `curl ... | sh` only stdin is the pipe, so [ -t 1 ] is still true and the prompt blocked an install that previously ran unattended. Both installers now gate it exactly as they gate the launch prompt below it. Reproduced under a real pty before and after, and pinned by two assertions that are absent at the previous head. tests/sh/test_launcher_no_browser.sh stubbed only open and xdg-open, but _open_browser prefers powershell.exe then cmd.exe on WSL, so running the suite on a WSL machine opened a real browser window and then failed on the empty recorder. All four openers are stubbed, PATH is pinned to them plus coreutils, and /proc/version is a fixture, so one Linux runner now covers every rung of the WSL ladder and the no-opener fallback. The post-install watcher also outlived the installer: its pid was never kept, so Ctrl-C on the foreground server left it polling out its 120s deadline and it could open a browser after the install had visibly finished. It is now reaped once the server exits, and the PowerShell job is stopped and removed in a finally, matching what install.ps1 already does for its other Start-Job. The watcher no longer treats a missing studio_install_id as "any backend will do". --------- Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>	2026-08-06 04:51:05 -07:00
Daniel Han	976a1152ad	Ask uv for a Python that can import torch, and skip the one that cannot (#7830) ... * Ask uv for a Python that can import torch, and skip the one that cannot Fixes #7803. CPython 3.13.8 carries python/cpython#139783: inspect.getsourcelines() drops a function body when a decorator is followed by a comment. That is the shape of the @_overload_method blocks in torch 2.11's nn/modules/rnn.py, which are parsed at import time, so `import torch` dies with IndentationError. 3.13.9 was an expedited release carrying only that fix. install.sh asked uv for a bare "3.13" and let it choose the patch. Measured with uv 0.9.2 and only 3.13.8 present: --python 3.13 -> 3.13.8 then import torch: IndentationError --python >=3.13.9,<3.14 -> 3.13.12 then import torch: 2.11.0+cpu OK So the request is the fix. PYTHON_SKIP names the releases that cannot run the stack and _python_request turns a bare 3.13 into the range; a venv left on a skipped interpreter by an earlier run is recreated, on any platform, which the previous check could not do because it was gated on macOS arm64. UV_MIN_VERSION also moves to 0.9.3, the first uv whose bundled manifest carries 3.13.9. That is belt-and-braces rather than the fix, since the range resolves on 0.9.2 too. Raising it pulls every 0.8.16-0.9.2 host into the refresh block, so an existing uv in that range is no longer fatal when the network is unreachable. Windows reaches such an interpreter differently: uv is handed a resolved path, never a version, so it cannot pick the patch, but Find-CompatiblePython matches on the minor version and would return an already-installed 3.13.8. Remove-SkippedPython turns that into "not found" so the caller installs $PythonFallbackFullVersion (3.13.13). The uv floor is left alone there, since the uv-managed Python path is not taken on Windows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Screen the skipped patch inside the resolver, and never delete the only venv Windows: Find-CompatiblePython matched on the minor, so the screen sat on its result and every other caller of the resolver -- the two install helpers among them -- could still be handed the interpreter the first call had just rejected. Nulling the result also ended the search, so a host with 3.13.8 and a healthy 3.12 failed instead of using the 3.12. The patch is already in the string the minor comes from, so screening during enumeration costs no extra subprocess and keeps the fallback ladder. install.sh: the legacy-layout migration moves the old environment into $VENV_DIR without arming the rollback, so a plain rm -rf before a recreate that then fails leaves the machine with nothing. Move it aside through the existing rollback machinery instead. Ask uv for the series minus the skipped patches rather than for a floor above them: an offline host, or a uv whose manifest predates 3.13.9, can still have a good cached 3.13.7 that a floor would refuse. Measured with uv 0.10.7, only 3.13.7 and 3.13.8 installed, --offline: "3.13" gives 3.13.8, ">=3.13.9,<3.14" errors, ">=3.13,<3.14,!=3.13.8" gives 3.13.7. Hoist the install.ps1 extractions out of the f-strings: a backslash in an f-string expression is a syntax error before 3.12 and the repo is 3.9+. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Run the resolver driver only where its fake launcher can execute The launcher is a /bin/sh script, and Windows has neither a shebang nor a PATHEXT entry for an extensionless file, so Get-Command never finds it and Find-CompatiblePython reports "none" whatever versions the tree holds. Two cases failed on a real windows-latest runner and the third passed for that reason rather than on merit. Skip the three on Windows, where the rest of the file still covers the screen, and pair the negative case with a positive control so a harness that cannot run the launcher fails instead of quietly agreeing. * Do not screen the interpreter for an install that never imports torch Every entry in the skip list is there for one reason: it cannot import torch. A --no-torch/-NoTorch install never does, so refusing the machine's only 3.13 would send a locked-down GGUF-only host into a download it may not be able to complete, over a package it will not install. Also turn away anything that is not a plain X.Y before the arithmetic: a relative --python path like 3.13/bin/python survived the globs, and dash aborts the whole install with "Illegal number" rather than reaching uv. And keep the uv version probe alive on an image with no awk, which is precisely the host the offline exception around it exists to protect -- the pipeline exits 127 there and set -e was killing the install before the code could treat the version as unreadable. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-05 03:03:39 -07:00
Daniel Han	1301e16004	Keep the file-edit turn 2 prompt to one instruction, and stop waiving its cap (#7846) ... * Keep the file-edit turn 2 prompt to one instruction, and stop waiving its cap * Tighten the turn 2 comments	2026-08-04 23:47:42 -07:00
oobabooga	df5f139bac	Studio: add image generation, editing workflows and LoRA training with Unsloth GGUFs (#6763) ... * Tighten comments in the image stack tests and scripts * Close video single-file, training reservation, and image mount-resume gaps Route on-device single-checkpoint video folders through the single_file loader: a bare local .safetensors directory (no model_index.json) is advertised as a pipeline with no filename, so validation rejected it before it could load. Reinterpret the pick as a single_file load of the sole checkpoint, mirroring the image load route. Treat a reserved-but-not-yet-spawned LLM training start as active in is_training_active() so /images/load, /video/load, and /diffusion/start cannot race the reserved run for VRAM during the pre-spawn free window. Mirrors the diffusion training service reservation. Resume an in-flight image generation on the Images page mount: probe generate-progress, re-enter the poll loop, and refresh the gallery on completion so a run started elsewhere is reflected and its saved image appears without a manual refresh. Seed resident image defaults from the resolved base_repo rather than a possibly path-shaped repo_id so the first resident generation uses the right recipe. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Publish image generation active state before pre-denoise setup generate() assigned self._gen only at the pipe() call, after deferred compile, LoRA resolution/application, and ControlNet download/build had run. Across that setup window generate_progress() reported inactive even though _generate_lock was held, so a reloaded page's mount probe showed idle and let a second generate queue behind the first. Publish an active step-0 _GenState the moment the generation lock is acquired, before the setup work, and clear it in the outer finally so a setup-time error cannot leave the UI stuck active. Mirrors the video backend's queued phase and the training start guard. * Studio: fix diffusion install ownership, dataset upload atomicity, gallery pagination, and teardown races install_sd_cpp_prebuilt: only write the .unsloth-studio-owned marker when the install created the target directory or it was empty. Adopting a pre-existing, unowned, non-empty directory (a user's own stable-diffusion.cpp checkout) made it eligible for the uninstaller's recursive delete. routes/training upload: make the multi-file promotion transactional. Back up each displaced original and roll every destination back on any failure, so a mid-loop rename error can no longer partially overwrite the live dataset. routes/training _resolve_dataset_folder: reject a symlinked dataset directory and prove the resolved folder stays under the datasets root, so image read/caption/delete cannot escape the root through a link. routes/training delete: escape glob metacharacters in the thumbnail filename so deleting an image named like [ab].png removes only its own thumbnails. image_gallery / video_gallery listing: filter records against the response schema inside the pager via a valid callback, so offset/limit/has_more all count over accepted records. A leading schema-invalid record no longer returns an empty page with has_more=true and stalls infinite scroll at offset 0. image_gallery / video_gallery save: publish via a temp file plus atomic rename (the sidecar is the video pair's commit marker) and clean up on failure, so a partial write never surfaces a truncated PNG or strands an orphan MP4. diffusion_train_common discovery: treat an empty caption sidecar as a metadata tombstone that still falls through to the dreambooth instance prompt, so clearing every metadata caption no longer fails with no captioned images found. diffusion backend unload: wait for an in-flight denoise to exit before tearing down process-wide patches and state, mirroring the load path. diffusion_engine_router: serialize the whole check/unload/publish transition so a concurrent selection cannot return the engine being unloaded. uninstall.ps1: gate the default sd.cpp process stop on the owner marker so a user's own sd-server is not terminated for a directory we then keep. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject native batch seeds outside the JSON-safe range * Refuse sd.cpp install into unowned non-empty target dir When the install target already exists, is non-empty and lacks the .unsloth-studio-owned marker (a user's own stable-diffusion.cpp checkout, or unrelated files beside a custom Studio root), install() previously still extracted the release into it. Skipping the ownership marker only stopped the uninstaller from deleting the directory; extraction still merged binaries into the user's working tree and could overwrite same-named files. Fail up front with a clear message pointing the user at a fresh/empty location before any download or extraction, leaving their directory untouched. Update the ownership test suite to assert the refusal. * Studio: gate dataset uploads on the symlink check and surface local video single-file checkpoints * Tighten comments and docstrings added by the image-generation fixes * Studio: close arbiter load-registration race and surface native progress + local pipeline folders Publish native sd.cpp generate progress (_gen) before LoRA resolution so a reload probe reads active during setup, matching the diffusers path. Register the diffusion/video GPU load under the arbiter lock (acquire_for now takes a register callback) so a competing acquire cannot evict an owner before its load is marked in-flight and let two loaders allocate VRAM at once. Admit local diffusers pipeline folders (root model_index.json, weights in component subdirs) in the local model scan so they reach task tagging and the On Device picker. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: include marker-owned custom sd.cpp roots in the uninstall stop scan; harden Pester install against the nuget.exe PSGallery bootstrap * Studio: surface local pipeline scan roots, tag single-file checkpoints by filename, mark companion-only pipelines partial - _scan_models_dir: admit a scan folder that is itself a diffusers pipeline (root model_index.json, weights in transformer/ vae/ subdirs). _is_model_directory rejects such a root, so the child scan would list the component subdirs as bogus models and hide the real pipeline; treat the root as one model via _local_pipeline_index. - _local_is_diffusers / _local_model_task: include the sole checkpoint filename in the family-detection needles (_local_family_needles, resolved via resolve_local_single_file). A generically named folder holding one loadable qwen-image-*.safetensors / ltx-*.safetensors identifies its family only from the filename; the load route already resolves that file, so tag it or the task-scoped picker (which rejects task=null) hides the on-device model. - list_cached_models: mark a companion-only base snapshot partial. A GGUF image load prefetches the base repo's VAE / text-encoder / model_index.json but skips the transformer (the GGUF supplies it); the snapshot has a pipeline manifest yet is not a loadable BF16 pipeline, and _cached_repo_partial misses it. _repo_pipeline_missing_denoiser flags a pipeline snapshot whose transformer/ or unet/ component carries no weight, so the picker drops it instead of advertising it as fully on-device. * Studio: preserve foreign gallery files, force safetensors on remote ControlNets, and close dataset/seed/GPU gaps Gallery clear/delete now scope to Studio-owned files: image_gallery and video_gallery skip PNGs / MP4s without a readable recipe (a hand-dropped or orphan file the listing already hides), so clear() and a guessed-id delete no longer destroy files the gallery never surfaced. Remote ControlNets now force use_safetensors: a bare owner/name reaches from_pretrained without the base trust gate, and the Hub scan fails open when unavailable, so requiring safetensors closes the pickle deserialization vector. POSIX uninstall now stops resident sd-server / sd-cli under an owned sd.cpp root before removing the tree (marker-gated), mirroring the Windows stop-before-delete scan; a live native server no longer survives unlinking its binary. Diffusion dataset containment: the training-start read path and the discovery picker route bare names through the protected resolver, so a symlinked dataset is rejected / not advertised like the caption/delete routes already do. Uploads gain the inference decode guard (oversized real images 400 before OOMing the trainer) and dataset upload/caption/delete/import are blocked with 409 while a diffusion run is active. JSONL readers (trainer + routes) tolerate non-object JSON and invalid UTF-8 instead of raising AttributeError / 500. LoRA family compatibility is enforced in the shared resolver, not only the picker, so a direct API client cannot apply a mismatched-family adapter. GPU arbiter gains release_if so the image/video unload idle-check and release are atomic against a concurrent same-owner load's registration. Native batch recipes persist the base batch_seed and restore replays from it, so a native batch_index>0 image no longer advances its seed twice. FLUX.2-klein selects its sd.cpp text encoder by variant (4B -> Qwen3-4B, 9B -> Qwen3-8B) instead of the single family default. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: track the loaded GGUF filename so native companion resolution reproduces the load identity * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate local-pipeline image tagging on a real family; validate video sidecars before delete/clear * Studio: tighten image-generation fix comments and docstrings * Studio: gate gallery serve/export on ownership; keep image progress active until persisted; reserve diffusion training before the dataset scan * Studio: restore Reapply target on async image/video load errors; recheck training state before dataset commit; reclaim partial sd.cpp installs on retry Images/Video: a background model load that fails AFTER starting (error/eviction during download) leaves the previous pipeline resident, but handleLoad had already overwritten lastLoad.current with the failed pick, so "Reapply to loaded model" reloaded the failed model. Carry the prior Reapply target into the poller and restore it on the async error/null paths, mirroring the quant rollback. Training: an in-flight diffusion dataset upload passed _require_diffusion_dataset_mutable() at entry but could still commit files after a concurrent /diffusion/start reserved the training slot, mutating the dataset underneath the trainer. Re-check the interlock immediately before the commit phase; a 409 there leaves the staged temps for the finally to clean. sd.cpp install: an interrupted extraction (disk full, killed process, a raising post-extract cudart fetch) left the target non-empty with no owner marker, so the next lazy install tripped the "not a Studio-managed directory" refusal and wedged native install. Write the ownership marker before the partial writes when the target is reclaimable, so a retry recognises the debris as ours and re-extracts. * fp8 DiT quant: floor the dynamic activation scale with activation_value_lb An all-zero activation token row makes the dynamic per-row fp8 scale 0, which turns the quantized data to NaN and the render to black frames on torchao's plain-torch kernel path. The fused fbgemm/mslk quantize kernels clamp zero rows internally, so the bug only reproduces on machines without them, which is most user environments. Zero rows are real inputs, not a corner case: Wan 2.2 zero-pads its text conditioning, and Hunyuan-1.5 and Qwen-Image regenerate zero rows inside their transformer blocks every step. Pass activation_value_lb=1e-12 to Float8DynamicActivationFloat8WeightConfig whenever the installed torchao supports the kwarg (Float8Tensor rework, 0.13+), checked via inspect.signature so older torchao keeps exactly the current behaviour; the existing Float8MMConfig fallback chain is unchanged. Verified on GPU: with the forced plain-torch kernel path a zero-row input NaNs without the floor and stays finite with it, and end to end on HunyuanVideo-1.5 fp8 goes from a solid black frame (LPIPS 1.00) to a normal render (LPIPS 0.225); on Wan the floor matches the condition_embedder exclusion (LPIPS 0.211 vs 0.206). Same-seed renders with fused kernels present are unaffected, and pre-quantized fp8 checkpoints stay valid since weight scales are untouched. * Wire hosted pre-quantized DiT checkpoints into the image families Point prequant_repos for flux.1, flux.2-klein, flux.2-dev, qwen-image (int8 only there; fp8 is family-denied), z-image and krea-2 at the unsloth/<Model>-FP8 Hub repos carrying gate-validated int8 and fp8 transformer checkpoints, so the fast quant path loads the small pre-quantized file instead of materialising the dense bf16 transformer and quantising on device. Measured on FLUX.2-dev int8: build peak drops from 60.7 GB (dense + quantize) to 30.7 GB (hosted prequant), identical 30.7 GB resident after either path since loading a checkpoint is bit-identical to on-the-fly quantisation. The hosted repos name files <Model>-<SCHEME>.pt, so resolve_prequant_source now derives that model-name filename from the repo id (scheme suffix stripped case-insensitively) and carries the legacy transformer_<scheme>.pt as a fallback the resolver tries when the primary 404s, keeping older repos loadable. Wiring a repo also exposed a fallback hazard: with a prequant source present, the dense-fit preflight used to be skipped entirely, so a failed prequant download would fall through to the dense bf16 load the memory plan never budgeted, OOMing after eviction. The preflight now always runs and gates an allow_dense_fallback flag through _load_dense_quant_pipeline: a dense misfit still skips the fast path when no prequant exists, but with one it proceeds and a prequant failure raises to the GGUF build instead of loading dense. The same flag is set when the auto-policy replans an offloaded GGUF against a prequant-sized transient. Tests updated to the new filename convention plus new coverage for the derivation and the legacy-name fallback; the prequant-skips-refit test now asserts the re-check runs and forbids the dense fallback. Verified end to end on GPU: z-image int8 resolves the hosted repo, downloads the model-name file and renders (6.8s load, 5.9 GB peak). * Route krea-2 through its per-component loader on the transformer-quant fast path _assemble_pipe used Pipeline.from_pretrained for every family, but the krea repo ships transformers-5.x configs and no top-level tokenizer files, so the tokenizer dies with vocab_file=None. The pre-quantized checkpoint loaded fine and then the assembly crashed, dropping the load to the GGUF build, which krea-2 cannot take (Krea2Transformer2DModel has no from_single_file). Assemble per-component via load_krea2_pipeline like the pipeline-kind and single-file paths already do. Verified live: Krea-2-Turbo int8 and fp8 hosted prequant loads now assemble and render through the Studio images tab. * Keep Qwen-Image's text-stream linears bf16 on int8 (short prompts break torch._int_mm) Qwen-Image's MMDiT runs every text-stream Linear at M = actual prompt tokens: the Qwen2.5-VL embeds are not padded to a fixed length like FLUX's 512-token T5. A short prompt (13 tokens) or the near-empty negative prompt drives torch._int_mm below its M > 16 floor and the first denoise step raises 'self.size(0) needs to be greater than 16, but got 13' (measured on B200 through the Studio images tab). Add per-family int8 exclusions (txt_in, add_q/k/v_proj, to_add_out, txt_mlp) for qwen-image and qwen-image-edit, threaded through exclude_tokens_for_scheme(scheme, family) and the prequant checkpoint validation, so a checkpoint baked under the old token list is rejected and re-quantised instead of loaded crashing. The text stream runs at M = tens vs the image stream's M ~ 4k, so the exclusion costs nothing; the rebuilt hosted checkpoint gates 28/28 PASS with LPIPS mean 0.057 (was 0.069). * Harden the diffusion memory plan against transient free-VRAM undercounts A cold FLUX.2-dev int8 load on an idle 183 GB B200 planned offload=model (companions exceed budget) and silently served the GGUF as-is; the identical retry went resident and engaged the hosted prequant. The plan arithmetic was byte-identical across both loads (required 90,228 MiB, resident needs free of about 124 GB); the only divergent input was torch.cuda.mem_get_info, which is device-wide and instantaneous: a transient foreign CUDA context briefly held about 100 GB at the first snapshot, and the planner trusted that single read. Three changes: - settled_snapshot_device_memory: on cuda, synchronize + empty_cache (best-effort) and take the MAX free over up to 3 spaced reads. A transient can only shrink free, so the max rejects transient undercounts while a persistent tenant still caps every read. _plan_memory now uses it. - plan_fits_total_capacity + one replan retry: when the dense/prequant candidate fits TOTAL device capacity under the standard reserve and the 0.85 resident margin, an offload verdict can only stem from the free reading, so the loader re-snapshots and replans once before declining the fast path. Explicit balanced/low_vram modes skip the retry (they offload by mode). - diffusion.transformer_quant_declined log line with required/budget/free and the plan reasons, so the next decline is diagnosable from the server log (previously silent). Verified: cold FLUX.2-dev int8 first load in a fresh server now engages the hosted prequant resident (offload=none). * Add FLUX.2 Klein and FLUX.2-dev DiT LoRA training Register flux.2-klein and flux.2-dev in the DiT trainer following the upstream DreamBooth references: latents train patchified and batch-norm normalized from the VAE posterior mode, the packed forward reuses step-invariant position ids, and the guidance vector (3.5) is gated on the variant's guidance_embeds config. Conditioning stacks load per variant (Mistral via Flux2Pipeline for dev, Qwen3 via Flux2KleinPipeline for Klein) and are encoded and freed before the transformer lands on the device. The fused single-stream to_qkv_mlp_proj joins the attention projections in the LoRA targets; the single-stream out projection stays dense because its to_out suffix would also match the double-stream ModuleList container. Wire both families through the training registry (family set, labels, VRAM notes, rank 16 / lr 1e-4 defaults, bf16-only preflight), mark them trainable with train base repos in the family registry, add FLUX.2-dev to the gated-repo token check, and trust both official bases for training downloads. Verified on B200: 30-step klein int8 (19.6s) and nf4 (20.9s) and dev int8 (52.0s) runs train with finite decreasing loss and the saved adapters apply on the bf16 base pipeline (weight 0 reproduces the base image exactly, weight 1 visibly restyles it). * Support LoRA adapters on torchao int8/fp8 quantized image pipelines Adapters are baked at load time: they attach to the dense transformer, then quantize_ converts only the frozen base linears (the lora_ side path is excluded by name), then the loader compiles. Post-quant PEFT injection is not possible on a manually quantized module, so the prequant shortcut is skipped for a baked load and the memory plan is sized for the dense build (force_dense on the quant candidate). At generation time the baked topology is frozen: weight tweaks and disabling (scale 0 reproduces the quantized base exactly) go through set_adapters, while adding or removing adapters returns a clean 400 telling the client to reload with the new selection. supports_lora now returns True for int8/fp8 diffusers loads (checked before the gguf-kind early return, since the quant fast path keeps the picker kind); nvfp4/mxfp8 and GGUF-via-diffusers stay blocked. The load request model takes an optional loras list, threaded through begin_load on both engines (native ignores it and keeps applying LoRA at generation). Verified end to end on GPU: Z-Image GGUF picker + int8 + trained adapter loads through the API, bake marker logged, weight 1.0 vs 0 renders differ visibly, weight 0.5 accepted live, unknown adapter rejected as 400. Affected suites: 296 passed. * Add FLUX.1 Krea dev to the image model catalog Krea's guidance-distilled FLUX.1-dev finetune keeps the exact dev layout, so it runs under the existing flux.1 family unchanged. Wire it up end to end: - Catalog group with the gated official bf16 pipeline and the open QuantStack GGUF quants; the gated artifact is skipped on auto-routing when undownloaded. - Trust the official repo for non-GGUF from_pretrained loads, next to the other black-forest-labs bases. - Generation defaults: 28 steps at guidance 4.5 per the model card. The generic "krea" defaults key (Krea-2-Turbo's 8-step no-CFG recipe) used to swallow the id, which would have produced garbage output; the new flux.1-krea key precedes it on both the backend table and the images page table. - The flux.1 prequant checkpoints are schnell-based; the loader's baked base_model_id validation refuses them for the Krea-dev base, so int8/fp8 requests dense-quantize instead (covered by existing prequant tests). * Resolve pre-quantized checkpoints per base variant One family entry covers several published variants whose weights differ (flux.1: schnell, dev, Krea-dev), but prequant resolution was keyed on (family, scheme) alone, so only the default base could ever be served: the loader's baked base_model_id validation correctly refused the schnell checkpoint for dev and Krea-dev bases and every such load paid the dense download plus on-the-fly quantise. Add an optional prequant_variant_repos table on DiffusionFamily as (base_repo, scheme, repo_id) triples and thread the resolved base repo through resolve_prequant_source / usable_prequant_source and their three call sites (load fast path, memory-plan probe, auto-policy candidate). A base without its own entry keeps returning the family default, preserving the existing refuse-then-dense behavior exactly. Wire the flux.1 variants: the gate-validated unsloth/FLUX.1-dev-FP8 checkpoints (built in the earlier campaign but never reachable) and the new unsloth/FLUX.1-Krea-dev-FP8. * Add the Lumina Image 2.0 family to the image catalog Alpha-VLLM/Lumina-Image-2.0 is a 2.6B single-stream DiT with a Gemma2-2B encoder and a standard 16-channel VAE, all transformers-4.x-compatible, so the generic from_pretrained pipeline path loads it as a new lumina-2 family: - Family entry (Lumina2Pipeline / Lumina2Transformer2DModel), aliased to lumina-image-2.0 / lumina-image-2 / lumina2. No bare lumina alias: Lumina-Next checkpoints are a different arch and must stay unknown rather than crash mid-load. bf16-only upstream, so the fp16 fallback stays off like z-image. - Trust the official repo for non-GGUF loads; bf16 component table entry (ships fp32, ~5.2 GB transformer + 5.2 GB encoder bf16-resident). - Generation defaults 50 steps / guidance 4.0 per the model card, and the generate call passes the card's cfg_trunc_ratio=0.25 itself (family-gated, signature-gated): the pipeline default (1.0) runs the CFG double-forward on every step and oversaturates output. - Catalog group with the single ungated bf16 pipeline artifact (11 GB resident) plus routing assertions; images page defaults row. - No GGUF artifact: none exists upstream (only finetune/LLM quants), so the dense transformer_quant fast path (GGUF-kind-only) stays unreachable for now. Offline probes of the future prequant campaign: int8 and fp8 both engage and render cleanly (fp8 LPIPS 0.11 vs bf16, int8 0.33 from 50-step trajectory drift with intact quality), so neither scheme is family-denied. * Wire the hosted Lumina Image 2.0 int8/fp8 checkpoints Gate-validated against same-seed bf16 renders (28/28 pairs per scheme, zero failures): int8 LPIPS mean 0.146 / SSIM 0.937, fp8 LPIPS mean 0.116 / SSIM 0.946. Uploaded to unsloth/Lumina-Image-2.0-FP8 following the existing checkpoint repo conventions. * Add the HunyuanImage 2.1 family to the image backend The hunyuanvideo-community diffusers mirror carries the full stack in standard layout: a 17B dual-stream DiT (32.5 GB bf16), a Qwen2.5-VL text encoder, a ByT5 glyph encoder, the 32x HunyuanImage VAE, and guider/ocr_guider components (AdaptiveProjectedMixGuidance) that diffusers 0.39 loads natively, so the generic from_pretrained pipeline path covers everything with no per-component assembly. Family notes: - The call's guidance knob is distilled_guidance_scale (there is no guidance_scale kwarg), so cfg_kwarg routes the UI value there; real CFG runs inside the repo's guider at its baked scale. Defaults follow the card recipe: 50 steps, 3.25. - 2K-native: verified live at both 1024 and 2048. - Coexists with the HunyuanImage-3.0 structured exclusion (3.0 has no diffusers pipeline and stays excluded with its stated reason). - int8/fp8 dense quantization verified live (LPIPS 0.186 both vs same-seed bf16); a short prompt does not trip the int8 torch._int_mm minimum on this arch, so no family exclude entry is needed. - bf16 component table for the memory planner: (32.5, 16.3, 0.8) GB. * Surface HunyuanImage 2.1 in the image model catalog Catalog group with the open bf16 mirror pipeline (~50 GB resident, so a bare click on a consumer card routes to the QuantStack GGUF quants, which load and render through the generic GGUF path, verified live) plus the images page defaults (50 steps, guidance 3.25 feeding distilled_guidance_scale). * Add the HiDream-I1 family to the image backend A 17B MoE DiT (16 double + 32 single layers, 4 routed experts) with four text encoders, on HiDreamImagePipeline (diffusers 0.39). One family covers the open Full / Dev / Fast repos (same arch); per-variant generation defaults follow the upstream inference recipes (Full 50 steps at guidance 5, the distilled Dev 28 and Fast 16 guidance-free). The repos name a Llama-3.1-8B text_encoder_4 in their model_index but do not ship its weights; the official example passes the gated meta-llama repo in by hand. The loader instead assembles the component from the open unsloth mirror (byte-identical weights, already inside the non-GGUF trust gate), injected at the three pipeline from_pretrained sites, with output_hidden_states matching the official example. Memory planning counts the assembled TE4: 34.2 GB DiT + 28.8 GB encoders, ~63 GB bf16-resident. * Surface HiDream I1 in the image model catalog One catalog group with the three official bf16 pipelines (Full, plus the Dev and Fast distillations as labeled artifacts) at their ~63 GB resident size, so auto-routing keeps this a datacenter-GPU pick. city96's GGUF is deliberately not wired: the GGUF path would need the same Llama TE4 assembly for very small demand. Images-page defaults mirror the backend table with the variant keys ahead of the generic hidream key. * Pin the measured HiDream quant verdict in tests int8 and fp8 both engage and render cleanly on this family, including short prompts on int8: the routed MoE expert Linears only ever see the concatenated image+text stream (M far above the torch._int_mm minimum), so no deny entry and no family exclude tokens are warranted. Assert that so a future table edit cannot silently regress the measured behavior. * Wire the hosted HunyuanImage 2.1 int8/fp8 checkpoints Verified bit-identical to on-the-fly quantize: all 1264 state dict tensors (456 quantized) dequantize equal between the loaded checkpoint and a fresh quantize_ pass, so quality matches the runtime Dtype path exactly. Same-seed LPIPS suite means (0.35 int8 / 0.28 fp8) blend trajectory divergence with this family's own run-to-run nondeterminism (identical weights and seed reproduce a 17/255 mean pixel delta through the 50-step guider pipeline); per-case hard checks pass and the drift is compositional, reviewed visually. Uploaded to unsloth/HunyuanImage-2.1-FP8. * Fix silent LoRA drop and wasted transformer prefetch on GGUF quant loads Two live-test findings on the images load path: - transformer_quant with baked LoRAs, when the dense quantized build is declined for memory or fails: the load completed as a plain GGUF with the adapters silently dropped (HTTP success, supports_lora=false after the fact) -- wrong output with no signal. The load now fails with the recovery options (drop the adapters, free VRAM, or pick a smaller model). Weight-0 adapters still count as no bake request, and the plain no-LoRA decline keeps its silent GGUF fallback. - A fresh GGUF load on a small GPU prefetched the base repo's full bf16 transformer shards (~47 GB on Qwen-Image) because the dense-quant prefetch widening only checked scheme viability, not whether the device could ever hold the candidate resident. Gate the widening on total device capacity (reserve + 0.85 margin, the plan_fits_total_capacity bar) so a card that is certain to decline the dense build never pays the download; capable devices keep the prefetch. * Fix video progress under-reporting during load and generate Two live-test findings on the video progress endpoints: - load-progress downloaded_bytes froze mid-download: the counter used scan_cache_dir, which skips in-flight *.incomplete blobs, so it sat at the last completed blob for the whole multi-GB shard pull while the disk kept filling. Count the repo's cache directory directly (completed plus incomplete blobs, snapshot symlinks skipped so nothing is double-counted). - generate-progress reported total_steps=null / fraction=0 while step advanced: the video API only carried the native total field while the image API exposes total_steps and fraction, so one poller could not work against both. Derive the image-compatible aliases in generate_progress and declare them on the response model; the native total stays for back-compat. * Wire the hosted HiDream I1 int8/fp8 checkpoints Gate-validated: all 28 per-case pairs pass per scheme (LPIPS suite means 0.291 int8 / 0.278 fp8, in the 50-step trajectory-divergence band; CLIP delta means 0.007-0.008), and the int8 checkpoint is verified bit-identical to on-the-fly quantize across all 1615 state dict tensors (1073 quantized, max abs diff 0.0). Uploaded to unsloth/HiDream-I1-Full-FP8. * Add a pre-cast text-encoder loader for the layerwise fp8 scheme The runtime text_encoder_quant=fp8 path downloads the full bf16 text encoder and layerwise-casts it in place on every fresh load. For the heavyweight encoders (LTX's Gemma3-27B ~50 GB, FLUX.2-dev's Mistral-24B ~48 GB, Qwen-Image's Qwen2.5-VL ~16.6 GB) that download dominates load time on a fresh machine. diffusion_te_prequant.py loads a pre-cast fp8-storage state dict instead: meta-init the encoder skeleton from the checkpoint's te_class, load_state_dict(assign=True), rebuild on CPU if non-persistent buffers stay on meta, then re-apply the same layerwise cast to install the upcast hooks. The cast is a deterministic storage transform, so the loaded encoder is bit-identical to dense-load-then-cast by construction. v1 hosts the layerwise fp8 storage scheme only: its state dict is plain tensors (torch.load(weights_only=True), no pickle execution). The dynamic-compute schemes (fp8_dynamic, int8, nvfp4) build torchao subclass wrappers at runtime and are deliberately not hosted. Checkpoints validate format, scheme, component and base_model_id before use and any problem falls back to the dense download and cast. Local path overrides reuse the DiT prequant allowlist env var. Families opt in via a new te_prequant_repos (scheme, component, repo_id) field on both DiffusionFamily and VideoFamily; the field defaults empty so nothing changes until a gate-validated artifact is wired. * Inject hosted pre-cast text encoders during pipeline assembly Wire te_prequant_pipe_kwargs into the three pipeline assembly sites: the diffusion full-pipeline branch, the diffusion transformer-only and GGUF branch (where the companion TE is the big remaining download), and the shared video assembly path before the pipeline/component split. Injection is gated exactly like the runtime cast (mode normalized to fp8, device supported, family not denied), so it can never engage where quantize_text_encoders would not; the later quantize_text_encoders call re-applies the cast idempotently and keeps status reporting truthful. With no hosted checkpoint configured the call returns {} and assembly loads the dense encoder as before. * Add the pre-cast text-encoder checkpoint builder Applies the runtime layerwise fp8 storage cast to a model's dense text encoder once and saves the cast state dict with baked metadata (format tag, base_model_id, family, scheme, component, te_class, versions) in the layout diffusion_te_prequant.py validates. Resolves the encoder class from the checkpoint's config.architectures so the recorded te_class matches what the pipeline instantiates. CPU-runnable: the cast touches storage dtypes only. * Test the pre-cast text-encoder load path Hermetic CPU coverage for diffusion_te_prequant: the checkpoint filename convention, family-table resolution by scheme and component with malformed entries skipped, resolution priority (path override, hosted repo, none) and the fp8-only scheme gate, the checkpoint validation matrix (wrong format, missing state_dict, wrong scheme, wrong component, wrong or missing base_model_id) with base case folding, the local-path allowlist refusal and missing-file fallback, and the assembly injection gating (mode, hosted entry, device support, family deny, load failure, successful injection). Also pins the te_prequant_repos field on both family dataclasses and that no family ships a hosted TE checkpoint until the campaign wires one. * Fix pre-cast TE checkpoint loading and engagement reporting Two bugs found while building the hosted checkpoints: - The builder recorded torch.__version__ (a TorchVersion object) in the checkpoint metadata, so torch.load(weights_only=True) rejected every artifact and the loader silently fell back to the dense download. Record plain strings. - Re-applying the layerwise fp8 cast to an injected pre-cast encoder raised on the duplicate hook registration, making quantize_text_encoders report the engaged cast as failed (status showed no TE quant while the encoder ran fp8). _cast_fp8 now returns early when the hooks are already installed. Also corrects the LTX TE size note: Gemma3-12B stored fp32 (~49 GB), not 27B. * Wire the hosted pre-cast fp8 text encoders qwen-image and flux.2-dev (diffusion) and ltx-2 (video) now resolve a hosted pre-cast fp8 text encoder from their unsloth -FP8 repos: - unsloth/Qwen-Image-FP8: Qwen2.5-VL-7B, 16.6 GB dense -> 8.8 GB - unsloth/FLUX.2-dev-FP8: Mistral-Small-24B, 48.0 GB dense -> 24.7 GB - unsloth/LTX-2-FP8: Gemma3-12B, 48.7 GB fp32 store -> 13.2 GB Every checkpoint verified bit-identical to dense-load-then-cast (729 / 585 / 1066 tensors, zero mismatches) and smoke-tested through the real backends with the repo engagement marker. Tests cover the wired entries, the resolver filenames, builder metadata weights_only survival, and the idempotent re-cast. * Report the compute dtype on fp8-cast encoders and inject the pre-cast TE on the dense fast path Two more findings from the hosted-TE GPU smokes: - Module.dtype reports the first floating parameter, which after the layerwise fp8 cast is the fp8 STORAGE dtype. Flux2 derives its prompt embed and latent dtypes from encoder.dtype and feeds them to randn_tensor, which has no fp8 kernel, so ANY flux.2 load with text_encoder_quant=fp8 crashed at generation (pre-existing, runtime cast included). The cast now swaps in a subclass whose dtype property reports the compute dtype; forward behaviour is unchanged. - The dense transformer_quant fast path assembles companions through _assemble_pipe, which never received the pre-cast TE injection, so the hosted encoder engaged on full-pipeline and GGUF builds but not on the fast path. Threaded through like the other two branches. Verified live on B200: qwen-image (full pipeline), flux.2-dev (GGUF picker with int8 DiT prequant), ltx-2 (video backend) all engage the hosted TE, render non-black, and report text_encoder_quant=fp8 truthfully. * Key the fp8 cast idempotency on an explicit completion marker Hook presence alone cannot distinguish a legitimately pre-cast text encoder from leftover hooks after a cast that failed mid-pass, so the early return now requires the completion marker _cast_fp8 sets once the hooks are fully installed. Leftover partial state keeps failing closed. Also tolerates non-Module encoder doubles in the hook probe and the dtype override. * Extend the fp8 TE quant to HiDream's Llama text_encoder_4 The generic quantize_text_encoders pass only covers text_encoder.._3, so HiDream's HEAVIEST encoder (Llama-3.1-8B TE4, 16.1 GB bf16) always stayed dense. TE4 is assembled separately (hidream_te4_kwargs), so the fp8 path now lives there: when the requested TE quant is layerwise fp8 and the device/family qualify, TE4 prefers the hosted pre-cast checkpoint (unsloth/HiDream-I1-Full-FP8, 8.6 GB) and falls back to dense-load-then- cast; a mid-pass cast failure reloads a fresh dense encoder instead of shipping partial state. The pre-cast loader and builder gain config_subfolder/config_overrides for standalone encoder repos whose config sits at the root and whose pipeline needs forward flags (output_hidden_states/attentions). Verified on B200: bit-identity 291 tensors (225 fp8, 0 mismatches), hosted checkpoint engages through the real backend (marker + status fp8), load 24.3 s vs 48.0 s dense, LPIPS 0.133 mean over 3 same-seed pairs vs the dense-TE render (gate 0.25), non-black frames. * Correct the ltx-2 resident TE estimate to the bf16 cast size The memory plan's bf16_components_gb held 50.4 GB for the LTX text encoder, which is the fp32 hub store of Gemma3-12B (~49 GB download), not what sits on device: the pipeline loads it torch_dtype=bf16, ~24.4 GB resident. The 26 GB over-estimate pushed the auto plan toward offload on cards that fit the real footprint. Comments and the size-table test now pin the resident semantics. * Host pre-cast fp8 text encoders for four more families Round 2 of the hosted TE set, each bit-identical to dense-load-then-cast and gated through the real backend (marker + status fp8 + same-seed LPIPS vs dense TEs): - FLUX.1 T5-XXL (text_encoder_2): 9.52 -> 5.90 GB, one artifact for schnell/dev/Krea-dev (T5 shards byte-identical across all three, verified sha256). 220 tensors, 144 fp8, LPIPS 0.109. - Lumina Gemma2-2B: fp32 hub store 10.46 -> 3.20 GB (3.3x download cut). 288 tensors, 182 fp8, LPIPS 0.041. - Z-Image Qwen3-4B: 8.04 -> 4.41 GB. 399 tensors, 252 fp8, LPIPS 0.112. NOT shared with flux.2-klein-4B: klein retrained layer 35's MLP (verified tensor diff, maxdiff 0.86), so klein hosts no entry. - Krea-2 Qwen3-VL-4B: 8.88 -> 4.83 GB. 713 tensors, 460 fp8, LPIPS 0.082. The constructor-assembled krea pipeline takes the encoder directly (load_krea2_pipeline text_encoder kwarg); the loader remaps 5.x rope_parameters and re-ties weights after assign so the rebuilt encoder matches the builder's structure. HunyuanImage 2.1 reuses the Qwen-Image artifact outright: its Qwen2.5-VL text encoder is byte-identical (every shard sha256, 16,584,414,544 bytes), recorded in the new component-level base-equivalence table the checkpoint validator consults. The injection loop now covers text_encoder.._3 so a family can host several components. Live check: LPIPS 0.123 vs dense. * Report the fp8-cast compute dtype without swapping the encoder class The dtype override swapped encoder.__class__ to a dynamic subclass, which breaks transformers' kwargs-based output recording: a fp8-cast Qwen3VLModel stopped returning hidden_states and every krea-2 generation with text_encoder_quant=fp8 crashed at encode_prompt (regression from the HiDream TE4 change; caught by the krea hosted-TE live smoke). The override is now a property shadowed on the ORIGINAL class that prefers a per-instance compute-dtype attribute, so class identity is preserved and uncast instances keep the stock behaviour. The idempotency test now pins exact class identity and the uncast-sibling fallback. * Pass the calibrated distilled sigma curve to LTX-2.3 8-step runs The 22B distilled DiT was trained against ltx_core's fixed DISTILLED_SIGMA_VALUES, but the diffusers scheduler derives 8-step spacing from resolution-shifted flow matching and lands far off at every reachable mu (second sigma 0.945-0.981 vs 0.99375, tail 0.37-0.61 -> 0.1 vs 0.725 -> 0.42 -> 0). At the distilled default step count the backend now passes the list verbatim, neutralising the scheduler's dynamic shift and terminal stretch for the call (they distort even explicit sigmas) and restoring them afterwards. Other step counts and the dev/base DiT keep the scheduler's own spacing. Live-verified on B200 through the video branch backend: the scheduler holds the exact curve after an 8-step distilled GGUF generation, config restored, healthy clip. Also reword the transformer_quant resolved reason to the measured reality: quant halves resident weights and hosted checkpoints cut load time, while per-step speed is roughly bf16 parity. * Pin the fp8 weight-quantize kernel against silent MSLK switching torchao's Float8Tensor KernelPreference defaults to AUTO, which switches the weight-quantize kernel to MSLK whenever an mslk package is importable on sm90+. Measured on B200: that changes fp8 scale rounding bitwise (8/8 FLUX matrices differ, scales ~55 percent of bytes), so a box that merely gains mslk would break the hosted-prequant bit-identity invariant; the mslk path is also slower under torch.compile (opaque extern call blocks inductor's quantize fusion, FLUX.1 fp8 e2e 1.149 to 1.624 s). Pin KernelPreference.TORCH explicitly, matching current no-mslk behaviour bit for bit; signature-gated for older torchao. GPU-smoked (finite, rel err 0.037) and pinned by test. * Shift Qwen-Image training sigmas to the inference distribution Qwen-Image's scheduler skips its static shift under use_dynamic_shifting, so the DiT trainer was drawing UNSHIFTED uniform-schedule sigmas for it (mean sigma 0.50) while inference always runs the exponential mu = log 3 shift plus the shift_terminal 0.02 stretch. Add a flow_shift config lever: "auto" (the new qwen-image default) rebuilds the training sigma table through the scheduler's own time_shift and stretch_shift_to_terminal so the draw matches the inference distribution exactly (mean sigma 0.72); a numeric value applies the standard linear shift s*u/(1+(s-1)*u); 1.0 keeps the historical identity behavior and stays the default for FLUX, Z-Image and Krea 2. The model timestep conditioning follows the shifted sigma, gathered in fp32 so bf16 rounding never skews it. Also wire two opt-in levers with off defaults: cfg_dropout (per-sample empty-prompt conditioning dropout, encoded alongside the captions before the text encoders are freed) and weighting_scheme="bell" (bsmntw-style mid-schedule Gaussian loss weighting normalized to mean 1). Verified with two 80-step rank-8 bf16 LoRA runs on Qwen/Qwen-Image (identity vs auto, same seed): both converge with finite decreasing loss and produce coherent same-seed previews. Unit tests cover the exact transform, the shifted sampling distribution, per-family defaults and config plumbing. * Add LoRA EMA, a persistent conditioning cache, and aspect bucketing helpers diffusion_train_extras hosts the opt-in training extras: LoRAEMA shadows only the trainable adapter params (warmup-ramped decay, default 0.99, exported as a second adapter under output_dir/ema), PersistentConditioningCache stores latent posterior stats and caption embeddings as safetensors keyed by content hash + family + resolution, and the aspect-ratio bucketing helpers group mixed-aspect datasets into same-area divisor-snapped shapes. The DiT trainer wires the first two behind config flags that default to the current behavior: ema_decay (0 disables) and cond_cache_dir (None disables). A fully warm cache skips loading the VAE and text encoders entirely; a cache hit is bit-identical to a fresh encode, including the per-channel qwen latent normalization. Also fixes the stale _gather_sigmas call in the perf test that still passed the scheduler instead of the sigma table. * Tighten torchao configs and note the FSDP2 design for the DiT trainer nf4 loads now enable double quantization (~0.4 bits/param off the frozen base scales at no fidelity cost), fp8 training uses the rowwise recipe when the torchao build ships it (per-row scaling confines the DiT activation outliers that a tensor-wide scale collapses), and the inference quant filter gains a per-scheme GEMM-tiling divisibility floor (16 for scaled_mm, 32 for MX blocks) so one ragged Linear cannot crash the first denoise after a clean quantize pass. plans/fsdp2_diffusion_design.md records the multi-GPU design: bf16/fp8 over FSDP2 with per-block units, LoRA attached before sharding, int8 out of scope (DTensor over the quantized subclass is undefined), per-family notes. * Batch diffusion inference with per-image seeds, an inference conditioning cache, and GGUF loader fixes Batched generation: /images/generate takes a prompts list (one image per prompt, txt2img only) or a seeds list (one prompt, one image per seed); the legacy batch_size path derives per-image seeds base..base+n-1 like the native engine. Every image gets its own torch.Generator so any batch member replays alone from its gallery recipe; the whole list runs as one forward by default with OOM backoff that halves a failed chunk, and an explicit batch_size caps images per forward. Validated 10-22x over serial engines on 32-image suites with LPIPS deltas within 0.002. Conditioning cache on the inference path: UNSLOTH_DIFFUSION_COND_CACHE_DIR (the inference sibling of the trainers' cond_cache_dir, same persistent store) wraps encode_prompt so repeated prompts skip the text-encoder forward entirely; verified bit-identical outputs. Bypassed while LoRA adapters are attached; tensor-argument calls pass through uncached. Compile cache: GGUF loads fingerprint their own bundles (quant=gguf, a different compiled graph than the dense family) and batched calls register every distinct (w, h, batch) chunk shape they ran, so the heavy GGUF batched warmups (~159 s at batch 32 on 12B-class, ~655 s on 20B CFG-batched) are paid once ever. GGUF loader: strip the sd.cpp model.diffusion_model. container prefix in the single-file converter; diffusers' FLUX.2 converter KeyErrors on it and the Qwen-Image identity mapping strands the model on meta. * Correct batched seed-replay docs to match measured behavior Same-seed images at the same batch shape are bit-identical; a solo regeneration with the recorded seed matches its batched rendition up to batch-size-dependent kernel numerics (mean abs pixel delta about 2.5/255, LPIPS delta under 0.002), not bit-exactly. The previous wording overclaimed bit-identity across batch shapes. * Note that batched bit-identity assumes a settled compiled graph The first generation issued while the deferred compile is still in flight can deviate transiently (observed once on a cold fp8 build: mean abs pixel delta 0.063/255); once the graph is settled, same-seed same-batch-shape images are bit-identical across runs. * Studio sidebar: Image03/FlimSlate icons, More flyout, Train row, New pills - Images uses Image03Icon and Video uses FlimSlateIcon. - New "More" row (MoreHorizontalIcon) opens a right-side flyout on click or hover holding Video, Recipes and Export; the close is delayed 180ms so the pointer can cross the gap. Its SidebarMenuButton deliberately takes `title` rather than `tooltip`: with `tooltip` the button returns a Tooltip root and DropdownMenuTrigger asChild would hand its ref to a non-DOM node. - Dropped the "Train" section heading; Train is now a top-level row between Images and More. data-tour="navbar" moves to the surviving nav group so the product tour keeps its anchor. - "New" pill beside Images and (inside the flyout) Video, via NavBadge. * Studio sidebar: match flyout rows and New pills to the existing scales - More flyout rows dropped their sidebar-row typography and size-icon override, which fought DropdownMenuItem's own scale (text-sm, gap-2.5, px-3 py-2 and size-4 icons) and rendered oversized glyphs and text next to the nav. - New pill reuses the brand "beta" badge recipe (nav-badge font, --ui-font-scale sizing, nav token colours) rather than hardcoded 9px values. - The More row's native title tooltip (an OS box on hover) is replaced by the app's Tooltip, wrapped around DropdownMenuTrigger so both triggers compose onto the same button, and shown only on the collapsed rail like other nav rows. * Settings: pin and reorder the sidebar navigation Adds a "Sidebar navigation" section to Settings -> Appearance, above the existing profile-menu customizer, with the same drag-to-reorder + switch UI. - New sidebarNav preference: one { id, pinned } entry per navigable row (projects, hub, images, train, video, recipes, export), array order = render order. Defaults match the shipped layout, so an untouched install is unchanged. - Unpinning moves a row into the More flyout rather than hiding it, so no page becomes unreachable. New chat and Search stay fixed as actions. - app-sidebar now renders from one navRows descriptor map, so a pinned row and its flyout counterpart cannot drift; the More row appears only when something is unpinned and highlights off whatever it actually holds. - Mirrored in the backend PersonalizationCustomization: without it the model's extra="ignore" would drop the field, and because sync replaces local state with the server's copy once customization is saved, the user's pin order would reset on the next sync. The validator dedupes and back-fills like sidebarMenu but preserves the client's order, since here order is meaningful. Frontend typecheck, i18n parity and catalog checks pass; 32 personalization tests pass, including a round-trip asserting a reordered list survives a save. * Sidebar customizer: drop the Search row, skip More for a lone item - Search is reached from the top bar, so it is no longer previewed as a fixed sidebar nav row; New chat stays. - More now appears only when it would hold two or more rows. A single unpinned row renders inline in its saved order position instead: a flyout wrapping one item costs a click and earns nothing. The customizer's More preview follows the same threshold. * Sidebar settings: hide a lone unpinned tab, match New chat icon, rename Profile menu - With exactly one tab unpinned, both More and that tab are dropped, so nothing is drawn for it (previously it rendered inline). The page stays reachable by URL. - The customizer's New chat preview uses PencilEdit02Icon, the icon the real row renders; Edit03Icon was a different glyph. - "Sidebar menu" is now "Profile menu", described as the shortcuts behind your name at the bottom of the sidebar, so it no longer reads as a second name for the navigation section above it. * Tighten comments in the new sidebar and delete-guard code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio sidebar: keep the More row highlighted while its panel is open Moving the pointer into the flyout left the row unhighlighted while the panel stayed open. The row now carries data-menu-open, added to the nav hover selector list. Not data-state: the tooltip and menu triggers both write that attribute, so whichever lands last wins. * Images: use the shared pill toggle for Create/Train and pad the panels - Create/Train was the only segmented control on its own Tabs styling. It now uses PillTabs, the same control as the model picker and Hub toggles, pinned to the header row's 34px. PillTabs takes an icon per tab, so the inline-span workaround for TabsTrigger goes away. - pt-3 on both the Create and Train panels, which sat flush against the model selector row. * Images: make the workflow picker a dropdown instead of a 7-up strip Seven workflows in a 340px rail left ~48px each, so the labels crowded and the hints were only reachable as title tooltips. The strip is now a dropdown: the trigger shows the current workflow and its hint, and each row carries its own description. A row the loaded model can't run is disabled and shows the reason in place of the hint, so the gating explains itself. Adding a workflow no longer shrinks the others. * Images: workflow icons, hint under the trigger, more top room, unclipped Train cards - Each workflow carries an icon, shown on the closed trigger and on every row. - The trigger is one line (icon plus name). The selected workflow's description moved below it, where it reads like the Field hints further down the rail. - pt-6 instead of pt-3 on both Create and Train, so the cards clear the model selector row. - The Train right column scrolls while its cards use ring-1, which draws outside the box and was clipped at the scroll edges. p-px gives the ring room. * Images: one-line workflow rows, stronger trigger fill, roomier mode tabs - Dropdown rows are icon plus name only. The selected row's description already shows under the trigger, and a disabled row keeps its reason as a title. - Trigger fill moves to the bg-foreground/[0.07] dark:bg-foreground/[0.12] pair the hub cards use, so it reads against the card in both modes. - Description under the trigger goes from text-ui-10 to text-ui-11p5. - More horizontal padding on the Create / Train tabs. * Images: drop card borders for the composer shadow, keep scrollbars inside, use app controls in Train - Cards lose ring-1 for .panel-soft-surface: the composer's shadow in light, flat in dark, matching .chat-composer-surface and the menus. - Both rails now clip (overflow-hidden) with the scroller inside, so the scrollbar can't ride over the rounded corner. Same shape video-page already uses. - Train's 9 native selects become the app Select, so they no longer open an OS-native menu, and the native file input is hidden behind a Choose images button that reports the count. - Image previews use explicit 8-10px radii: this theme sets --radius to 1.1rem, so rounded-md was 15.6px and the thumbnails read as circles. * Images: one card for controls and preview, chat sliders, wider softer shadow - Controls and preview were two floating cards; they now share one card split by a divider. The Advanced dock stays separate since it toggles. - SliderField wraps Chat's ParamSlider, so the sliders match Chat (label row with the value, full-width neutral track) instead of a green track with a spin box. All 14 call sites keep their props. - panel-soft-surface goes from 0 2px 8px -2px /0.16 to 0 4px 22px -6px /0.10: lighter, spread wider. * Images: flat Create and Train panes, hover-only scrollbars, tidier Train dataset step Both Images tabs now sit on the page background like the Hub: no card, no shadow, no bounding box. A single rule divides the controls rail from the preview canvas (Create) and from the run area (Train), and the settings and previous-runs sections read as panes rather than nested cards. Also: - Scrollbars in these panes use the existing hover-scrollbar recipe, so the thumb only shows while the pane is hovered. - Workflow rows explain themselves with a tooltip after a short hover, which also works on disabled rows, and the descriptions are much shorter. - Training images rows are name plus image count; the license stays on the example card. - The upload step loses its dashed box, the buttons match the sizes around them, and Upload only appears once files are picked. - The empty preview uses the same icon as the Images nav item. * Images: full-height panes, wider settings rail, Create/Train offset from the selector The rule between the panes now runs the whole page height (the row drops its bottom padding and each pane pads its own content), the settings rail is wider on both Create and Train, and the Create/Train switch sits further right of the model selector. * Images: put both tabs on the Hub's centered measure Top bar and content now share mx-auto max-w-1100 with px-5 / sm:px-8, so Create and Train sit at the same width and position as the Hub instead of running edge to edge. * Images: restore the top bar position, drop the panes lower under it * Images: center the mode switch, flip the arrow with the orientation, app tooltips everywhere The Create/Train switch is centered on the page instead of trailing the model selector, with wider buttons. The flip control's arrows now rotate with the orientation and its label says which way the flip goes. Every native title tooltip on the page is now the app's tooltip, so they all get the rounded surface instead of the OS box. * Images Train: plainer field text, no green buttons, columns that stop colliding - The dataset name, trigger prompt, adapter name and custom base fields now say what they are in plain words instead of leaning on example values. - Import, Upload, Back, Back to settings and Train another are outline buttons, not green ones. - Example thumbnails are landscape tiles, so photos are not cropped to chunky squares. - Settings cells get min-w-0 and the select value truncates, so a long option like the nf4 label no longer widens its column into the next one. - The number stepper sits a little further in from the field edge. - Create and Train are wider. * Images Train: roomier example cards with Import on the thumbnail row * Video: same treatment as the Images tabs - No cards: the rail and the canvas sit on the page background, divided by a rule that runs the full page height, on the Hub's centered measure. - Wider rail, chat's sliders, hover-only scrollbars. - Every native title tooltip is now the app's tooltip, including the clip cards. - Reapply and Cancel are outline buttons, the empty state uses the Video nav icon, and the clip tiles are less rounded. * Images and Video: narrower generation rail, matching Train headings Create and Video rails go from 392px to 368px. Train a LoRA and Training settings are now the same size and both in the heading font: the h2 already picks it up from the base rule, so the settings header opts in with font-heading and the weight that rule pins. * Images Train: shorter copy throughout Family notes, example descriptions, precision labels and every helper line are trimmed so they stop wrapping to three lines and colliding with the next column. The nf4 label now fits its select without truncating. * Images Train: a little more spacing between field groups * Images and Video: tighten code comments * Fix training start NameError, the load-order guard test and CPU-only diffusion tests - start_training forwards resume_source_run_id to _start_training_impl, which reads it. Without it every start raised NameError. - Restore main's anchor in the load-marker order test: the file now has an earlier `if config.is_gguf:`, so indexing the first one compared the wrong branch. - The two diffusion tests that reach diffusers now skip when it is absent, matching the CPU repo-test env. - The UI smoke finds nav rows that live in the sidebar's More flyout. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Treat a null metadata caption as no caption str(None) stored the literal "None" as the caption, so a null row counted as captioned and would have trained on that text. Also drops an unused import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix invalid-UTF-8 500s, the flat Canny map and the dropped DiT knobs read_text raises UnicodeDecodeError, which is not an OSError, so one bad caption sidecar or video sidecar 500d the info, upload and gallery routes. A flat image now yields the all-black edge map instead of its own luminance, and the four DiT loss knobs the trainer implements are declared so model_dump keeps them. * Use the ui font-size tokens instead of raw px text utilities text-[11px] and friends ignore the UI font size preference, which the repo's font-scale contract test enforces. Same rendered size at the default scale. * Fix diffusion dataset 500s, the dropout-1.0 no-op run and the reset base pick Four correctness fixes on the training side: - The labeling grid read caption sidecars under except OSError, but a non-UTF-8 sidecar raises UnicodeDecodeError (a ValueError), so one bad file 500d /diffusion/dataset/{name}/images and the grid could not be opened to repair it. Read it as no caption, matching the info summary. - An image past Pillow's own hard limit raises DecompressionBombError, which derives straight from Exception and so escaped the upload guard's (OSError, UnidentifiedImageError, ValueError) and returned 500 instead of the intended 400. - lora_dropout accepted 1.0, which makes PEFT build nn.Dropout(p=1.0): lora_A and lora_B receive no gradient and the run saves an untrained adapter while reporting normal progress. Bound it below 1.0, matching the LLM request schema. - The train panel re-seeded the base repo on every dataset refresh because the family object identity changes on each info fetch, so an upload or caption save silently replaced the user's chosen base and the run started on a different model. Track the pick and only re-seed on a real family change. * Show the retained failure when a video page mounts after a failed job Mount-time recovery handled only phase=completed, so reloading the page after a multi-minute generation failed left an idle view with no diagnosis: the backend keeps the terminal failed record only until the next job, and nothing else survives the reload. Surface it the same way the poll does, filtering the cancelled sentinel. * Fix batched generation crashes, cache keying and unreplayable recipes Four bugs in the batched inference path, all found by review: - A mixed-prompt batch sent a scalar negative prompt against a prompt list. Z-Image asserts on the length, and Qwen-Image, Krea 2 and FLUX true-CFG encode a batch-1 negative against batch-N latents and fail in the transformer's text/image concat. Broadcast it to match the batch. - The FBCache step-cache reset sat above the chunk loop. diffusers only resets that state at the end of a successful call, so a forward that raised (the OOM the backoff is meant to recover) left its own residual behind and the halved retry died on a shape mismatch. Reset before every forward instead. - The conditioning cache keyed on the checkpoint alone, but a GGUF or single-file load takes its text encoders from the companion base, so the same checkpoint against a different base reused the previous base's embeddings. Key the base too. - Gallery records stored the base seed and the requested batch size even when a prompts/seeds list drove the run, so restoring the second image of seeds=[5, 99] replayed seed 5. List-driven outputs now record as single-image recipes on their own seed. Also bound strength above 0: every img2img pipeline derives its step count from it, so 0 leaves zero denoising steps and either raises or, on SDXL, crashes on empty latents. * Fix quantized-load LoRA bake, prequant family exclusions and outpaint canvas Six review findings across the Images page and model scanning: - The quantized (int8/fp8) load path can only attach LoRA adapters before quantization, but the frontend load request had no loras field, so every generation after such a load was rejected and each reload repeated it. Send the selection with the load. - build_prequant_checkpoint passed no family to the scheme exclusions while recording the family in metadata, so a Qwen int8 artifact baked the short-M text-stream linears and was then rejected wholesale by the loader's family-keyed check. - Registering a bare single-file checkpoint directory produced no On Device row even though the images loader can load it; only its parent worked. Admit that shape when nothing else matched. - Unload left the Reapply target set, so the repair path was skipped and Reapply reloaded the ejected model. Clear it, as the video page does. - Both FLUX.2 bases were trusted for training but not inference, so Deploy to Create rejected every FLUX.2 adapter. - Outpaint allocated the grown canvas before downscaling, exceeding the browser canvas area cap on a large photo; an over-cap canvas is unusable, so Extend silently posted a fully transparent image and mask. Scale the source first. * Send the picked GGUF filename with the quant so diffusion loads fire The variant expander emitted only the quant label, and nothing else in the frontend set ggufFilename, so the Images and Video pages could never take their GGUF branch: both gate it on meta.ggufVariant and meta.ggufFilename, then fall through to the single-file path, which returns because the id is a repo id and not a .gguf name. Every quant pick was a silent dead click, with no load request reaching the backend. The filename was already on the variant row (the picker keys its list on it, and the variant validator requires a non-empty string), so thread it through the click handler. The chat path is unaffected: it reads ggufVariant and never needed the filename. * Version the conditioning cache key and reject non-finite flow_shift Two correctness fixes: - The cache keyed the checkpoint and its companion base by name only, so a Hub repo advancing to a new commit, or a local directory updated in place, kept returning embeddings from the previous text encoder. Pair both with a revision marker: the locally resolved commit sha for a Hub repo, config plus text-encoder file stats for a directory. Neither loads the encoders, so a warm run still keeps them off the GPU. - flow_shift only checked positivity, but JSON accepts 1e309, which floats to inf, and inf <= 0 is False while NaN fails every comparison. The sigma table then evaluates s * u / (1 + (s - 1) * u) as NaN, which poisons every sampled sigma and saves a corrupted adapter while progress looks normal. Require a finite value. * Keep curated models listed, guard the video companion repo, pin diffusers Three review findings: - The picker filtered every catalog member out of Recommended and Hub search on the way to canonical group rows, but nothing renders those rows yet (catalogGroupFitsDevice and groupMatchesQuery are imported and unused). A task-scoped picker's models list is catalogToModelOptions(), i.e. group members exclusively, so both lists came back empty and no curated model could be discovered or downloaded. Keep the artifacts listed until the grouped UI exists. - The video delete guard compared only repo_id, so deleting the companion base of a loaded GGUF video model was allowed even though it supplies the VAE and text encoders. Compare base_repo too, matching what the images guard already does for its companions. - diffusers was declared unversioned while the diffusion stack requires 0.39 (Krea2Pipeline, the cache_context child registries, the Flux2 and Z-Image pipelines), so an upgrade could keep an older release and selecting an advertised model failed until the user upgraded by hand. * Namespace the trainer conditioning cache per checkpoint, bound the learning rate - The trainer keyed its persistent conditioning cache on family and resolution only, while the keys themselves carry just the caption or image content and crop variant. One cache directory reused for two checkpoints, or for the same repo at a new revision, let a warm run skip loading its encoders and train on the other model's embeddings and latent statistics. Namespace on the base checkpoint and its resolved revision as well. The revision helper now lives beside the cache in diffusion_train_extras and the inference wrapper delegates to it, so the two cannot disagree about what counts as the same source. - The diffusion learning rate only checked positivity, but 1e309 floats to inf and satisfies gt, so the route evicted the resident models and started AdamW with an infinite rate: the first step destroys the adapter while progress looks normal and the result is saved. Bound it below 1.0, matching the LLM schema, which rejects inf for the same reason. * Fix GGUF image model picks doing nothing, and pick the train base in the top bar The quant rows never forwarded the .gguf filename, so every hub GGUF pick on Images/Video fell through to a silent return. On Train the top bar now picks the training base instead of a generation model, which is GGUF-only and untrainable. * Pin diffusion and video loads to the live HF cache root Both read huggingface_hub's import-time HF_HUB_CACHE, which changing the cache folder does not update: progress counted the old root while the download wrote to the new one, and from_pretrained could split one model across both. * Add the diffusion download plan endpoint Reports the repos and exact files a pick needs so the download manager can stage them with the loader's own file scope. A plain snapshot would add the packaged root single, transformer shards and fp16 twins the loader never opens. * Add a file-scoped flavour to the Hub download job Lets a consumer that reads a deliberate subset of a repo stage it through the normal download manager. Keyed as "@scope" so it never collides with a quant or with the repo's full snapshot, and the file list rides the registry so an XET to HTTP retry respawns the same scoped job. * Stage image and video downloads through the Hub download manager They downloaded inline inside the load, so they had none of the manager's disk preflight, manifest verification, resume or panel progress. Picks now stage as scoped jobs carrying the loader's own file list, then load from a warm cache. * Fetch staged GGUF checkpoints as scoped jobs, and stop calling diffusion models unsupported A GGUF entry went out as a full snapshot, whose ignore list drops *.gguf: the job finished at once having fetched only docs, and the repo landed on device unloadable. Every entry is scoped now. The Hub also no longer tags image/video models as unsupported (they run on their own pages), and those pickers name what they select. * Apply the picker task filter to local model sections LM Studio, ./models and custom-folder rows ignored it, so the Images picker listed chat GGUFs that 400 on a diffusion load. The backend already tags every local model with a task for this purpose. * Route a chat pick of a diffusion model to the Images or Video page Chat cannot load one, so it was either hidden or failed on load. The unfiltered picker now lists on-device diffusion models and navigates to the page that runs them, passing the repo and quant so that page loads it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Route the real GGUF filename, keep non-GGUF curated models, key scoped downloads by file set Five review findings, four of them ways a click did nothing or fetched the wrong thing: - A chat pick of a diffusion model routed ggufVariant (a label like Q4_K_M) in the search param the target page uses verbatim as the GGUF filename, so the load asked for a file that does not exist. Route ggufFilename; no filename means a curated non-GGUF pick, loaded as a pipeline. - The task-scoped pickers kept only GGUF repos, so the catalog's bf16, bnb-4bit and single-file fp8 artifacts could not be discovered or downloaded on the Images and Video pages even though loadSpecFor knows how to load them. Keep curated artifacts whatever their format, in Recommended and in Hub search. - Both pages deduplicated routed selections on the model alone, and they now stay mounted, so picking the same repo again -- another quant, or the same one after chat evicted it -- returned early without loading or clearing the query string. Key on model and quant. - Every scoped image download shared one @diffusion job key regardless of the requested files, so switching quant mid-download adopted the running job: the UI waited on the first file set, then loaded a file that was never fetched. Include a digest of the file set in the key. - A scoped plan silently dropped requested files missing from Hub metadata, and snapshot_download succeeds when an allow pattern matches nothing, so the job reported completion and triggered a load with required files absent. Fail the job instead. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the scoped download key derivable, and stop the hidden page hijacking a route Four review findings, the first a regression from my own last commit: - Keying scoped download jobs by a digest of the file set broke the download manager: it builds that key client-side (it polls and cancels before any response tells it a key), so it watched and cancelled a key no worker owned and never fired its ready callback. Keep the derivable "@scope" key and refuse the second request instead when a live job on the slot is fetching a different file set -- decided inside the registry claim, under the lock, so a concurrent claim cannot slip past it. The manager records the file set on the job as well, so a sibling quant's transfer is not adopted locally either. - Both diffusion pages read the route query through a loose useSearch and both stay mounted once visited, so the hidden one consumed the other's ?model=: it navigated back to its own route and tried to load, say, an image checkpoint as a video model. Only the visible page consumes it. - The staged download plan was built without the configured HF token or the Advanced values the load itself sends. The token matters most: the backend's Hub metadata lookup is best-effort, so a gated base silently planned no companion entry and the load pulled those multi-GB files inline, outside the manager. The memory/quant controls decide whether the base transformer/ shards are needed at all, and the route dropped memory_mode, cpu_offload, the prequant path and the LoRA selection before asking for the plan. - The video preview kept playing after leaving the page: the keep-alive layout only hides it, and display:none does not pause a media element, so a clip the user unmuted kept its audio going over the next page. Pause on the active transition and do not auto-replay while hidden. Also completes the hand-built request bodies in the hub download tests: the scoped-files field this branch added to the route read as an AttributeError against them, failing five tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Serialize the GPU handoffs, gate DiT training on a GPU, and keep 3.9 installable Six review findings, three of them evict-then-fail orderings: - The chat load reclaimed the GPU without telling the arbiter it existed. A chat load holds no llama-server process until its GGUF has downloaded, which is minutes, so a competing Images/Video acquire in that window found nothing to cancel, took the GPU, and the chat load then spawned onto the same device. It now registers an in-flight marker through acquire_for's register hook (under the arbiter lock, as the image and video loads do), the evictor cancels a marked load, and the route undoes itself if ownership moved while it loaded. - The Hub-download conflict check ran after that handoff, so a GGUF the download manager already owns destroyed the resident Images/Video pipeline and then 409'd, having loaded nothing. It moves above the handoff, together with the marker it handshakes with. - The image load released the engine router's transition lock before registering the load, so a second load choosing the other engine could unload the still-idle engine this one captured; the load then landed on a deactivated engine, where generate, status, unload and the arbiter's evictor can no longer reach it. Registration now happens under that lock and refuses if the engine changed. - Training a DiT family on a host with no GPU was accepted: nf4 is not a CPU fallback, its 4-bit load goes through bitsandbytes, which requires CUDA, XPU or MPS. The start unloaded the working Images pipeline, pulled the text encoders, and only then died in the child. Rejected before the teardown now, and /info stops advertising a precision that always 400s. SDXL keeps its documented fp32-on-CPU path. - Both diffusion pages kept the routed-pick marker forever, so re-picking the same checkpoint (after chat evicted it) neither loaded nor cleared the query string. The marker is released once the query is gone. The Images key also carried a stray NUL byte, which made the file read as binary to grep and other tooling. - diffusers dropped Python 3.9 in 0.38, so the unconditional >=0.39.0 pin left pip no candidate at all on 3.9 and made every install that composes the huggingface extras unresolvable there. The floor is conditional now. Also fixes tests that were already red on the branch: two hand-built request fakes had gone stale against fields this branch added, and the handoff-ordering test only failed on a host with fewer than two GPUs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the GGUF variant contract test against the merged handler signature The assertion pinned the exact single-line call handleVariantClick(v.quant, v.downloaded, expectedBytes, v.filename), but the handler takes (quant, filename, downloaded, sizeBytes) and prettier wraps the call across lines, so the mandatory repository test job failed on every push. Match the call structurally and assert the filename really is forwarded in the handler's argument order. * Stop a background page and a stale record taking the GPU or a download with them Five fixes from a review pass over the diffusion work. delete-finetuned rmtree'd a model the Images or Video engine was holding: every guard on that route is chat-only, and Images loads any local path, so deleting a local diffusion model under the storage root pulled the weights (and the companion VAE / text encoders sd.cpp re-reads each generation) out from under a live pipeline. The cached-model route already refuses this; the trained/exported one now does too, matching by path rather than repo id, and failing open on a chat-only install so it cannot block ordinary deletes. A staged download finishing while its page was hidden loaded the model and evicted whatever the user was actually using: both diffusion pages stay mounted behind the router and a load takes the GPU unconditionally. The pick is now held until its page is on screen again, which is also what chat does. A scoped download could report success having fetched nothing. With Hugging Face metadata unavailable no manifest is written, so verification is a no-op, and snapshot_download returns an existing snapshot folder without downloading when its own repo_info call fails. A repo already on disk from a full snapshot job (which ignores *.gguf) therefore completed with no weights and auto-loaded against them. The requested file list needs no network, so it is checked against the disk directly. The XET to HTTP retry reclaimed the job slot without the scoped file list, and that claim overwrites the stored record, so a later identical scoped start compared an empty list against the real one and 409'd instead of adopting the running download. The DiT accelerator gate probed torch.mps.is_available(), which only exists from torch 2.5 while the supported floor is 2.4. All three probes shared one try/except, so on torch 2.4 the AttributeError read as 'no block' and a CPU-only host still evicted the resident pipeline, downloaded the encoders and died in the child. Each accelerator is probed on its own now, through torch.backends.mps. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stage what an LTX-2.3 load reads, keep a routed file's load kind, drop an unbakeable LoRA Three from the latest review. The video download plan always asked for the wide base file list, so an LTX-2.3 pick staged the 2.0 base's VAEs, vocoder and connectors that the checkpoint supplies itself, while the companion files the 2.3 assembly does read were left out of the plan and pulled inline at load, outside the panel's progress, cancel and disk preflight. The plan now recognises a 2.3 pick by name (the load keeps the authoritative header probe, and under-guessing only falls back to the load-time pull), narrows the base list, and stages the extras in the same entry as the checkpoint so one repo stays one scoped job. A pick routed from the chat picker arrives as ?model= and ?quant= with no picker metadata, so a bare local .gguf or .safetensors was loaded as a pipeline: an explicit model_kind wins over the backend's filename sniffing, so it evicted the resident model and then failed on the missing model_index.json. Both pages now derive the load kind from the path, the same way their own picker handlers do. A torchao int8/fp8 build takes adapters only at load time. Switching artifact inside one family keeps the LoRA selection, since the family did not change, but the load did not bake it, so the next generation was rejected with 'reload the model with the adapter selection' while the picker still showed the adapter as active. The selection is now dropped once per resident build, with a message saying to pick and load again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cancel an evicted safetensors load, spare the arbiter for CPU-only chat, fetch clips lazily Four fixes from the latest review round: - The GPU arbiter's chat evictor only cancelled the llama.cpp side. The orchestrator publishes active_model_name once its worker reports success, so an in-flight safetensors load was visible only as an entry in loading_models and finished onto the GPU after ownership had transferred. Cancel every pending load, and give the safetensors branch the post-load ownership recheck the GGUF branch already had. - A manual gpu_layers=0 GGUF load runs on the CPU with the GPUs hidden from the child, yet it took the arbiter unconditionally: it cancelled a running image or video generation for a model needing no VRAM, then held CHAT ownership so the next GPU workload unloaded it for nothing. Gate the acquire on the same predicate the launch-time CPU-only mask uses, as the image and video loaders gate on their resolved device. - The staged-download hook subscribes per repo, not per job, so another job on the same repo advanced the staged queue (starting a load whose scoped files were still downloading) or wiped a queue that was still running. Compare the variant each callback carries, like the chat page's auto-load does. - The video gallery fetched every record of a page into an object URL that lives until the page closes: 50 clips at tens to hundreds of MB each, for cards the user may never scroll to. Fetch a clip as its card nears the strip's edge, plus the selected one the player needs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim the comments across the diffusion backend Comment-only pass over the Python this PR touches: drop what the code already says, collapse multi-line explanations that still read on one line, and keep the reasoning that is not recoverable from the code. No code, docstring semantics or behaviour changes; verified with an AST comparison against the previous revision, and the backend suite is unchanged (same 37 environment failures as before: the API integration tests that need a live keyed server, the flash-attn install hooks, and the GPU memory fields). * Invalidate latents on a VAE swap, keep a cut-off generation, surface the EMA adapter - source_revision() scanned the checkpoint root plus text_encoder/tokenizer but not vae, so swapping or fine-tuning the VAE in place left the conditioning cache namespace unchanged and a warm run trained against latents from the old checkpoint. Include the vae directory, like any other component the cached tensors come from. - /images/generate answers only when the images are saved, and secure mode's tunnel caps an origin response near 100 seconds, which a native CPU or a high-step run passes routinely. The page reported failure while the work kept running, and a retry would duplicate it. A lost response (fetch rejection or a gateway status the origin never answered) is now told apart from a refusal: the page waits out generate-progress and reloads the gallery, so the run it started still lands. - The trainer emits the EMA adapter's path with the terminal event, but the state update dropped it, so neither the run history nor either response schema carried it and an enabled EMA left nothing discoverable. Keep it, and show it next to the primary adapter. - weighting_scheme advertised a choice of timestep sampling; sampling is always logit-normal and the flag only selects the bell loss weights. Describe what it does. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Hide unloadable cached rows, hold the dataset interlock, bound a GIF export - The cached-model listing tagged any repo with a model_index.json as text-to-image, so a community pipeline the image loader's trust rule refuses still got a row in the Images picker, and a detected-but-untrusted video repo fell through to that same tag. Gate the image tag on the load path's rule and hide an untrusted video repo outright. - A routed diffusion pick only carries a GGUF filename, which is all the chat picker has, so a curated single-file artifact arrived with no quant and was loaded as a pipeline: from_pretrained on a repo with no model_index.json. Pass the page's own catalog spec into the route pick, so a routed pick resolves to exactly what a direct pick on that page resolves to. - The dataset mutation endpoints checked is_active() and only then handed their filesystem work to a thread, so a start reserving in that gap changed captions or removed images underneath the preflight or the running trainer. The interlock is now registered for the whole request under the lock reserve() uses, and a start refuses while a mutation is open rather than waiting on it. - GIF export held every kept frame as a paletted image before encoding; a clip may be 2048x2048 for 1024 frames, and at the 12 fps target the step is 1, so one export click could allocate over 4 GB and take the backend down. Downscale past 720 px and widen the step to keep at most 300 frames. - seed accepted any Python int, so an out-of-range one passed every preflight, evicted the resident models, spawned the trainer and only then died in torch.manual_seed. Bound it to torch's 64-bit range in the request and config. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the accelerator probes in the DiT family-metadata tests Six tests read family_train_infos() (or a start preflight) without pinning the host probes, so they only held on a machine with a bf16 accelerator: on a GPU-less runner the DiT gate empties precision_modes, turns supports_compile off, and replaces any other preflight message with the no-accelerator note, and all six failed there. A conftest fixture pins both probes for exactly those tests, so they assert the family metadata they are about on every host. The gate's own CPU-only behaviour keeps its dedicated tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Carry the pipeline task into the hub inventory the pickers read - The task-scoped pickers filter On Device rows on a task, and the chat picker routes a diffusion pick by the same field, but those rows come from the /api/hub inventory, which never carried one: the Images and Video pickers listed nothing on device and the routing never fired. Both cached scans and the local listing now tag rows with the classifiers the models API already uses, the schemas and the frontend adapter carry it through, and a row the backend classified as a generation task is exempt from the chat-only guard that was also dropping it. - The local routing map was keyed by model_id while the row click passes id (a filesystem load id for a models_dir or LM Studio entry), so the lookup missed and the pick fell through to the chat loader. Key both. - A staged download whose start answered "error" left its head in place, where the effect never re-runs and onReady never fires, so the pick was stranded until the user reselected. Clear the queue and say so. - Every scoped pick in a repo shares the @diffusion variant, so the variant alone cannot tell two file sets apart: restaging while the first job finished let its completion pass for the new pick and load a checkpoint that had not downloaded. Bind the callbacks to the repo + file set they started, and to the staging generation. - A rejected generate POST does not say whether it reached the backend, so an immediately idle progress read was ambiguous and a submission that never landed looked like a finished image. Require evidence: progress seen active, or a gallery record that was not there before the POST. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the OpenAI image URL fetchable, keep WebM audio, stream example imports Four review items on the diffusion Studio work: - response_format=url returned the bearer-gated gallery route, which a standard image client downloads with no Authorization header, so the default response format was unusable. Mint a short-lived HMAC link instead (the shape RAG already uses for pdf.js) served by a signed route, and leave the gallery route itself bearer-only. - A manual gpu_layers=0 load carrying speculative_type="off" -- a value the UI persists and sends -- read as GPU-bearing, so it took the GPU arbiter and evicted a resident image/video pipeline even though the launcher hides the GPUs for it. Canonicalize the mode and exempt "off". - The curated example import prepared the whole split before the loop stopped at the 10-100 image cap; m1guelpf/nouns is 49,859 rows / 328 MB. Stream instead, with the prepared load kept as a fallback for a repo that cannot stream. - WebM export dropped the audio track an LTX-2 clip carries, silently, on the format offered for web embeds. Mux it as Opus through a resampler + FIFO, and keep exporting the video alone on a build without libopus. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not stub out triton on a GPU host when the Xet backend fails to import The lazy loader retries `import unsloth_zoo.hf_xet_fallback` under UNSLOTH_ZOO_DISABLE_GPU_INIT=1 whenever the first attempt raises. That flag makes unsloth_zoo take its MLX/CPU path, which injects triton and bitsandbytes STUBS into sys.modules for the rest of the process. On a working GPU box whose first import failed for an unrelated reason (a bitsandbytes/CUDA mismatch, say) the retry succeeds, so Studio boots looking healthy and then dies at the first CUDA-only kernel: a GGUF or compiled diffusion generation hits the stub and returns NotImplementedError: Unsloth: 'triton.tools.experimental_descriptor.enable_in_pytorch' was called on Apple Silicon / MLX, where triton is stubbed out. so every image generation 500s with an Apple-Silicon message on a Linux CUDA host, while the load reports success. Found by loading Z-Image-Turbo GGUF through the API on a box where bitsandbytes could not initialise. Gate the retry on the host genuinely having no accelerator. The Xet stall watchdog is optional and already degrades with a warning; a process whose triton is stubbed out is not recoverable. The warning now says why it did not retry. * Fix the lost-generation proof set, the settle timeout and the hub inventory's diffusion gates Seven fixes from the latest review round on the Images page and the hub cache inventory. Images page: - The lost-POST settle path built its "already seen" gallery id set inside the catch, after the request failed. By then the earlier runs of the same batch had already prepended their records, so run 2 could accept run 1's image as proof that its own request reached the backend. The set is now captured once before the first POST and grows with every record the batch produces. - settleLostGeneration fell out of its SETTLE_MAX_MS loop and returned normally, so a wedged generation was counted as done and the next run started against a busy backend. It now throws on timeout. - Restoring a recipe cleared the ControlNet selection but left the workflow tab and the init / mask / reference images pointing at whatever was loaded, so the next Generate conditioned on an unrelated image. It now clears all of them and returns to Create. - The download plan omitted the adapter selection the load itself bakes in. A baked LoRA forces the dense build path, so the plan described a different file set than the load that followed and the rest was pulled inline, outside the download manager. Both now derive the list from one helper. Hub cache inventory: - A download for a repo an Images or Video load is staging was allowed to start: only the llama.cpp loader was consulted. Both diffusion backends already expose loading_repo_ids for the delete guard, and the download guard now reads them too. - A companion-only prefetch (pipeline manifest plus VAE and text encoder, no transformer) passed the snapshot-partial check, since every file its manifest expected did arrive, and was advertised as on-device although from_pretrained cannot load it. - The single-file flag never reached the picker through the hub inventory path, so a checkpoint-only diffusion repo read as a full pipeline and failed after the handoff. The two pipeline-shape helpers now live in hub/utils/inventory_scan.py so /api/models/cached and the hub inventory classify the same repos the same way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop adopting an unknown scoped download, leaking raced blobs and resurrecting deleted clips Three items from the latest review round. A scoped download job carries a deliberate file subset, and every file set of one repo rides the same "@scope" slot. A client that adopts a live job from the backend had no file list to compare against: the active-downloads response never carried one, so an adopted job's set was unknown and any later scoped request for the same repo read as "already started". Selecting a different checkpoint then waited on the wrong transfer and tried to load a file nobody fetched. The response now publishes the scoped file list, adoption records it, and an unknown set no longer satisfies a scoped request. A gallery record can be deleted while its blob is still downloading. The delete revokes the URL present at that moment, so the fetch that lands afterwards inserted a fresh object URL for a record no card renders and nothing can revoke: a full MP4, tens to hundreds of MB, pinned for the rest of the session, and once per raced fetch. Both galleries now discard a blob whose record went away, with an epoch covering the video page's Clear all. The video backend keeps the last completed job until the next one starts, and the Video page merges that record on mount to cover a job that finished after the gallery fetch. Deleting the clip left the record in place, so every reload prepended a ghost card whose file request 404s until another generation replaced it. Deleting the clip, or clearing the gallery, now clears the matching terminal record, and the page skips a record it deleted itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve revisions from the live cache, serialize dataset imports, and stop pinning every gallery blob Five more items from the review round. The conditioning-cache revision marker read huggingface_hub's import-time HF_HUB_CACHE constant. Studio can move its cache during a session and loading follows the live setting, so after a move the marker went unresolved (or pointed into the previous root) and pulling a new revision of the same checkpoint no longer invalidated the cache: a warm run could reuse the old encoder's embeddings and the old VAE's latents. It now looks in the active Studio cache first and keeps the environment and the library constant as fallbacks, which the trainer subprocess still needs. The dataset interlock counts mutations rather than excluding them, so two imports of different examples into the same empty name both got past the emptiness check. The winner promoted its staging directory atomically; the loser found the folder non-empty, fell back to a per-file move, and merged its images and captions into the winner's dataset. Imports now take a per-folder lock, a second one is refused with 409, and the emptiness check is repeated under the lock. On Windows the sd.cpp asset resolver filtered only by accelerator token, so a Windows arm64 host matched an x64 zip, downloaded and installed it, and failed later when the binary would not run. It now filters by architecture the way the Darwin and Linux branches do. Every gallery page fetched every PNG up front and kept the object URL for the session, so scrolling a large gallery grew memory without bound for tiles the user may never look at. The Images strip now fetches a tile as it nears view, like the Video strip, and keeps the eager path only where IntersectionObserver is unavailable. A 503 carrying a JSON body comes from the application, not a proxy, so it is surfaced as the error it is instead of entering lost-response settlement and being reported as a request that never reached the server. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Name the class of a failed generation instead of a bare "Image generation failed." Found on a macOS runner: the native renderer aborts inside its own text encoder there, and the page showed only "Image generation failed." with the sd-server backtrace left in the server log, so nothing about the failure reached the user. The failure is now classified into fixed text, out of memory and native-process death, so the message says what happened and what to try. None of the engine's own output is echoed, since a native tail carries local paths and argv; that stays in the log, and an unrecognised failure keeps the original literal. * Treat an undecodable caption sidecar as the tombstone the trainer sees Uploads store .txt and .caption sidecars as raw bytes, so one can hold invalid UTF-8. The trainer treats any existing sidecar, decodable or not, as an empty tombstone and never falls back to the metadata row for that image. The labeling grid and the dataset summary read an undecodable sidecar as absent instead, so both showed a metadata caption that the run would silently replace with the instance prompt, and counted the image as captioned. Both now track sidecar presence separately, so what the user reviews is what the run trains on. * Keep the reason a native server died, not just its backtrace A ggml abort prints its cause first and then a stack trace, so reporting the last twenty captured lines gave twenty addresses and nothing about the failure: on the macOS runner the native server died on an unimplemented Metal op and the message carried only frame pointers. The captured tail now leads with the lines that name a cause and keeps recent context after them, for both the startup failure and the mid-request death. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop staging the dense text encoder for an fp8 video load Two halves of the same gap, found while measuring the LTX-2.3 download plan: - The video download plan and the scoped pre-download never saw text_encoder_quant. An fp8 request loads a hosted pre-cast encoder, so asking for one still staged and downloaded the base repo's dense Gemma3 (48.79 GB of Lightricks/LTX-2 on the 2.3 distilled pick) that the pipeline then never opened. The plan now drops those shards and stages the pre-cast checkpoint instead; their configs stay, since the pre-cast loader still meta-inits the encoder from the base repo's component config. - The LTX-2.3 assembly builds every component itself, so pipe_kwargs (which carries the pre-cast encoder for from_pretrained) never reached it and an fp8 request silently loaded the dense encoder anyway. It is passed across explicitly now. The dense skip is earned, not assumed: only a pre-cast checkpoint that resolves on the Hub lets the plan drop the dense shards, and only one already fetched to disk lets the pull drop them, so an unpublished or gated artifact leaves both exactly as they were. If injection still fails after that, the load tops the dense weights back up rather than handing from_pretrained a snapshot with no encoder in it. Measured against the real Hub on the 2.3 distilled Q4_K_M pick: 67.24 GB before, 18.92 GB with a 0.43 GB stand-in for the pre-cast artifact (the base entry drops from 24 files / 48.79 GB to 13 files / 0.04 GB). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match the file's typing idiom for flow_shift models/training.py annotates with typing constructs throughout (105 Optional[...], no Union), and flow_shift was the one place using PEP 604. Union[] reads like the rest of the file, and it also drops the runtime evaluation that would raise on Python 3.9. * Bound the gallery blob cache, and three interlock fixes Four review findings, all reproduced first: - The gallery object-URL caches were unbounded. A clip runs from a few MB to a few hundred, both pages stay mounted after their first visit, and entries were only dropped on delete, so scrolling pinned everything for the session. Both pages now share a byte-budgeted LRU (512 MB video / 192 MB images) keyed off the visibility signal the near-viewport fetching already provides. On-screen media, the selected clip or image, and the item just fetched are never evicted, so eviction is invisible and a single item larger than the whole budget cannot evict itself into a refetch loop. - The image, video and chat load guards ran two independent training probes but returned early when the FIRST one raised, so an unreadable LLM backend disabled the diffusion interlock and a load could proceed straight into an active diffusion trainer on the same GPU. The probes are independent now. - An engine switch swallowed a failed teardown and published the new engine anyway, which is exactly the leak the unload exists to prevent: the arbiter's evictor, /images/unload and the next load all resolve through get_active_diffusion_engine(), so the still-resident pipeline (or a live sd-server) became unreachable and the next load allocated on top of it. The switch now fails and leaves the old engine published, so it stays reclaimable. - The native generation timeout was 30 minutes while the Images page waits up to 6 hours (SETTLE_MAX_MS), so slow-but-progressing CPU jobs died deterministically at the deadline. Measured on GPU-less runners, a 512x512 4-step Q2_K generation took 900 s on Linux and 1465 s on Windows, so larger images or step counts clear half an hour easily. The ceiling now matches the page's window and applies to the whole request: chunks of a split batch share one deadline instead of each getting a full budget. Cancellation is unchanged. Declined: gating the huggingfacenotorch extra off Python 3.9 over the conditional diffusers marker. The marker is deliberate and its comment says why: diffusers dropped 3.9 in 0.38, so pinning >=0.39 outright leaves pip no candidate and the whole extra unresolvable there. The pipelines it names live in studio/backend, which cannot install on 3.9 anyway (studio.txt pins matplotlib==3.10.9 and fastmcp>=3.0.2, both requires_python >=3.10), and the extra is the general core one, so the alternative drops 3.9 for library users who never touch Studio. * Close the load-versus-training-start race, and two picker fixes - The image and video load guards read is_active() and only then selected an engine, acquired the arbiter and registered the load. A /train/diffusion/start reserving inside that window freed residents the load had not registered yet, so the trainer came up beside a brand-new pipeline. The service already had exactly the right pattern for this in dataset_mutation, so gpu_load_admission mirrors it: reserve() refuses while an admission is open, an admission refuses once a start is reserved, both decided under the one lock. The span is only the registration, since begin_load returns as soon as the load is registered and _free_gpu_for_diffusion_training preempts an in-flight load from that point. Chat is deliberately not covered: its load spans an eviction plus a multi-minute GGUF load, and it admits models that fit beside training by design, which is a different contract from the diffusion pipeline's all-or-nothing one. - Hugging Face gives the LTX-2 family the image-to-video pipeline_tag (both Lightricks/LTX-2 and unsloth/LTX-2.3-GGUF report it), so a text-to-video-only filter dropped the flagship audio family out of Video Hub search while the rest of the app routed it to Video. - Task-scoped quant fit sized picks against the LARGEST visible device while resolve_diffusion_device_target returns a bare "cuda" and torch places on the current one. On a heterogeneous host that recommended a checkpoint sized for the bigger card and then loaded it onto the smaller one. Fit now uses the device the load actually lands on; identical on a homogeneous host. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Expose the persistent conditioning cache in the start schema DiffusionLoraConfig has carried cond_cache_dir for a while and the DiT trainer acts on it, but DiffusionTrainingStartRequest omitted the field, so Pydantic dropped it silently and every API-driven run fell back to the in-memory cache that is rebuilt from scratch each time. The warm path skips loading the VAE and the multi-GB text encoders on a rerun whose images, captions and resolution are unchanged, so this was a real capability that could not be reached. Contained like output_dir rather than left to the trainer subprocess's cwd, since it is another directory the trainer writes to. Blank or omitted still means the in-memory cache, so it must not resolve to the outputs root. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix diffusion policy and classification issues from review fp8 auto precision defaulted to precise accumulate on any non-consumer GPU, which made fp8 2.05x slower than int8 on RTX 6000 Ada and slower than not quantising at all. NVIDIA's professional whitepapers do publish equal FP8 rates for both accumulate modes there, so the hardware premise held, but the cost is in the cuBLAS path rather than the published rate. Default to fast accumulate: measured on B200 the flag is a no-op (4096^3 _scaled_mm at 3023.8 vs 3041.8 TFLOP/s, bitwise-identical output, 1.213 s vs 1.230 s end to end), so it is a large win where it bites and free where it does not. Precise accumulate stays available via transformer_quant_fast_accum. Z-Image's DiT is a Lumina2 derivative, so unsloth/Z-Image-GGUF and unsloth/Z-Image-Turbo-GGUF both declare general.architecture = "lumina2" and the whole line was tagged image-diffusion-unsupported and hidden from the Images "On Device" list, though validate_load_request loads them. Resolve shared archs from the repo/file name like bare "wan" already does, with a test asserting the picker and the loader agree for every family. The sage attention on-demand install ran an unpinned `pip install sageattention`, but PyPI's newest wheel is 1.0.6 and diffusers refuses anything below 2.1.1: the install always "succeeded", wrote an unusable version into the running venv, and was rejected on the next line. Carry the dispatcher's floor so pip resolves nothing instead. The dense-quant disk gate sized the download from the bf16-RESIDENT table. The fp32 families download twice that (Z-Image: 23,479 MiB against a 21,970 MiB gate), leaving a window where the check passed and the download filled the disk; Ideogram 4 ships fp8 and was overcharged the other way. Size the gate by published bytes, verified against HF sibling metadata for all 12 families. Patch installs went through unsloth_zoo, which refuses to import unless UNSLOTH_IS_PRESENT is set, and that is set by unsloth itself. The server imports unsloth at boot so it never showed there, but any other process ran silently unpatched with every install returning False, which is 13 test failures on a clean environment. Import unsloth and retry once, memoised per process. Also: the GGUF+LoRA refusal pointed at the native engine without saying a GPU host only selects it under UNSLOTH_DIFFUSION_ENGINE=sd_cpp, so the suggestion was unreachable; the gallery recipe recorded loras from the generate request alone, losing a load-time bake; load-progress claimed "40.07 GB downloaded" for a fully cached load; and pickers.tsx imported three catalog-group helpers it never used. Reported by oobabooga. * Keep the sd.cpp text encoder on CPU under Metal macos-14 loads FLUX.2-klein-4B Q2_K natively on mps and then dies on the first generation with exit code -6: ggml_metal_op_encode_impl: error: unsupported op 'RMS_NORM' -> ggml_abort LLMEmbedder::encode_prompt -> LLMRunner::compute -> GGMLRunner::compute ggml's Metal backend gates RMS_NORM on contiguous rows and aborts the process when that does not hold, with no per-op CPU fallback, so any LLM text encoder (Qwen3 for FLUX.2 and Z-Image, T5 for FLUX.1) takes sd-server down. The encoder runs once per prompt while the DiT runs every step, so pinning only the encoder keeps Metal for the part that matters. UNSLOTH_DIFFUSION_SD_CPP_METAL_TE_GPU=1 opts back in once ggml grows the kernel. * Gate the unsloth retry in the diffusion patch backend The retry added for the clean-environment patch failures is not free: importing unsloth pulls torch in behind it, which costs ~940 MB of RSS measured in a process that had neither, and on a host with no accelerator it fails anyway. A cross-platform CI job that had generated fine at ~900 s later died 19 s in with SIGTERM and every 'if: always()' step skipped, which is the runner being torn down rather than a step failing. Retry only when torch is already imported (true of the server and of anything patching a real module, and the condition that stops the retry from being what loads torch), unsloth is installed but not yet imported, and the first failure was the ImportError the sentinel guard raises. The clean-environment case it was added for still passes 29/29. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only retry the unsloth import where it can succeed The gate still let the retry run on hosts unsloth does not support, which is where it is most harmful: a 7 GB macOS runner lost the Studio server 26 s into a load, and the Linux runner was torn down mid-generation. Neither MPS nor plain CPU can complete the import, so the retry there pays the cost and fails anyway. Require an accelerator unsloth actually supports (CUDA/ROCm via torch.cuda, or XPU), with UNSLOTH_ALLOW_CPU as the documented override, and hoist the predicate to module level so it is tested directly rather than through the import system. On a CPU-only host the retry no longer fires at all; on CUDA the clean-environment case it was added for still passes 29/29. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard old diffusers, stream video exports, record conditioned recipes Three fixes from review. The 0.39-only pipeline classes (Flux2Klein, Z-Image, Krea 2, LTX-2, HunyuanImage) were resolved by getattr deep in the load, so on the older diffusers that packaging still allows on Python 3.9 -- diffusers dropped 3.9 in 0.38 and this project still supports it, so the 0.39 floor has to be conditional or the extra becomes unresolvable -- an advertised model failed with a bare AttributeError after its checkpoint had already been downloaded. Krea 2 already guarded itself this way; assert_pipeline_class_available now runs the same check for every image and video family from validation, before any fetch, and names the version and the fix. WebM export accumulated the whole VP9 output in a BytesIO and returned it as one bytes object that the response held again. The request caps allow 2048x2048 for 1024 frames, so an export runs to hundreds of MB and concurrent clicks could exhaust the process, while the MP4 route beside it already streamed from disk. transcode_to_file encodes to a temp file and the route returns a FileResponse with a background unlink, so nothing large is resident. A conditioned generation's recipe carried only the txt2img fields, so the gallery presented an inpaint or upscale result as a complete Create recipe and restoring it replayed an unrelated text-to-image request. The images themselves are still not persisted (user uploads with their own lifetime), but the workflow and its scalars are, restore reapplies them, and the toast now names the inputs that have to be supplied again instead of silently landing on Create. Reported by Codex. * Per-load video cancel event, family-gated image picker, cond cache refusal A cancelled video load could resume: begin_load cleared the shared cancel event, and unload() drops _loading without waiting for the worker, so the next load cleared the very object the cancelled worker was watching and its multi-gigabyte pull ran on alongside the replacement until the token check at the end. Each load now gets its own threading.Event, passed down through _fetch_te_prequant and _predownload_base, so a cancelled worker stays cancelled. A cached repo with a model_index.json was advertised as text-to-image on the trust rule alone, but validate_load_request also requires a detected image family, so a trusted pipeline of an unsupported class produced a picker row that deterministically 400s. The picker now applies both gates, mirroring the video branch. cond_cache_dir was accepted for sdxl and then ignored: only the DiT trainer reads it, while the SDXL trainer builds a per-run in-memory latent cache, so the promised cross-run reuse never happened. The route now refuses it with a 400 that names the families which do support it, checked against the resolved family so an omitted model_family with an SDXL base is caught too. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the frontend build broken by the gallery blob cache tsc -b failed on the branch head, so npm run build produced no dist and every platform job fell back to --api-only: blob-url-cache.ts(29,15): TS1294: This syntax is not allowed when 'erasableSyntaxOnly' is enabled dataset-labeling-grid.tsx / dataset-showcase.tsx: Argument of type '{ url: string; bytes: number; }' is not assignable to parameter of type 'string' The cache took its budget as a constructor parameter property, which the project's tsconfig forbids, and fetchGalleryObjectUrl now returns the blob size alongside the URL for that budget, which the two dataset thumbnail components still consumed as a bare string. Declare the field explicitly and destructure the URL at both call sites. tsc -b is clean and vite build emits dist again. * Recover from a ggml unsupported-op abort by restarting on the CPU backend ggml checks every node against the device's supports_op and calls GGML_ABORT when one is not implemented, because a single-backend graph has nowhere else to put it: there is no per-op CPU fallback. The whole sd-server dies with SIGABRT mid-generation and the user gets "the native image renderer stopped unexpectedly" with no way forward. Seen on macos-14 arm64 with FLUX.2-klein-4B Q2_K through the cross-platform CI: the text encoder is already pinned to CPU, and the abort moved into the denoise loop instead. ggml_metal_op_encode_impl: error: unsupported op 'MUL_MAT' -> ggml_abort StableDiffusionGGML::sample -> sample_k_diffusion A retry on the same backend would abort identically, so the load is restarted once with --backend cpu (the only flag that changes which backend executes the graph; --offload-to-cpu moves parameters, not compute) and the generation is re-submitted. The same checkpoint then renders slower rather than not at all. Strictly bounded: the signature must carry both the unsupported-op line and ggml_abort, the device must not already be CPU, and it happens once per load, so an OOM kill or a genuine crash still surfaces as itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix two tests that only fail in a full-suite run The 3.10 CI leg resolves PyAV 17, where av.container.OutputContainer is an immutable C type, so the no-libopus export test died on "cannot set 'add_stream' attribute of immutable type" before it asserted anything. Inject the refusal by wrapping the container av.open() returns instead; modules stay patchable on every build. Removing the injection makes the test fail again, so it still covers the branch it is named for. The Xet shim's degraded-path tests drop utils.hf_xet_fallback from sys.modules and import a throwaway copy. Restoring only the sys.modules entry left the utils package attribute bound to the throwaway, and the two disagreed for the rest of the process: a later monkeypatch of the dotted target patched one copy while the code under test imported the other, so the patch did nothing and test_fetch_te_prequant_only_reports_what_it_downloaded reached the real Hub and got a 401. Restore both bindings. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop an ignored-cancel sd-server, guard deletes during diffusion training, repair unusable managed binaries sd-server does not interrupt an in-flight job, so when it ignores a cancel the grace branch abandoned the poll and reported cancellation while the native job kept a core (or the GPU) busy to completion and held the server's job slot. The comment said the caller stops the server, but only unload does that immediately: a superseding load stops it after its multi-gigabyte download, and a load that then fails never gets there. Stop it here, as the deadline branch already does. DELETE /api/models/delete-finetuned checked only the LLM trainer, so it could rmtree the output directory a live diffusion LoRA run was about to write its adapter into. Consult the diffusion training service too, like the dataset mutation and model-load routes. find_sd_*_binary only checks is_file(), so an interrupted extraction (or a prebuilt for the wrong CPU) left a present-but-unrunnable binary the installer never retried: every load probed it, fell back to diffusers, and native inference stayed off until the directory was deleted by hand. Probe it and reinstall, but only for a copy under the installer-owned root -- SD_CLI_PATH, UNSLOTH_SD_CPP_PATH, an in-tree build and anything on PATH are the user's. * Plan the pre-cast text encoder, and make the cross-trainer GPU admission atomic An fp8 text-encoder request loads a hosted PRE-CAST checkpoint, but the image download plan never received text_encoder_quant, so the manager staged the base repo's dense encoder (FLUX.2-dev's Mistral-24B is ~48 GB, Qwen-Image's Qwen2.5-VL ~16.6 GB) and the load then pulled the pre-cast file inline, outside the manager's progress and disk preflight. The plan now takes the field, resolves the hosted artifact with the same resolver the injection uses, stages that file, and drops only those components' dense weight shards. The load's own prefetch takes the same treatment, since it paid the same cost. Only a checkpoint that really resolves on the Hub earns the drop, so a gated or renamed artifact still stages the dense encoder the load will fall back to. The two trainers admitted each other with independent check-then-act guards: the diffusion route checks the LLM backend several network-bound preflights before it reserves, and the LLM route checks the diffusion service well before it spawns, so two near-simultaneous starts could both pass and train on one GPU. reserve() now re-tests the LLM backend under its own lock, and the LLM route holds the diffusion service's gpu_load_admission across its spawn, so exactly one of the two wins. Both halves fail open, so a chat-only install still trains. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the Advanced options a staged download planned against Staging does not set busy, so while a multi-gigabyte download runs the user can still change precision, memory mode, speed or the baked LoRA selection. The pending record held only the repo and artifact, and the completed download fired a load that read the CURRENT state: the staged file set could then be missing files that load needs (fetched inline, with no progress and no disk preflight) or hold gigabytes it no longer uses. One snapshot of every Advanced control is now taken when the plan is built, and it travels with the pending record into the load, so the load that runs is the one the download was planned for. * Do not advertise a family the installed diffusers cannot build The newer families (Z-Image, Krea 2, FLUX.2, LTX-2, HunyuanImage) exist only from diffusers 0.39, and 0.39 cannot be installed on Python 3.9 at all -- diffusers dropped 3.9 in 0.38, so the requirement is conditional or the whole extra becomes unresolvable. On such an environment the picker still offered those rows, every pick failed deterministically, and the error's advice to run pip install -U diffusers could not fix it without also upgrading Python. The cached-repo picker now applies the same availability check validate_load_request does, which is keyed on the pipeline class actually present rather than on the Python version, so it is also right for an intentionally pinned older diffusers on 3.10+. Fails open when diffusers cannot be imported at all: that is a different problem and the load path reports it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the diffusion engine selection after each router test The active engine is module state, and several tests set it by plain assignment because what _activate does to it is the thing under test, so monkeypatch could not undo it. A leaked ENGINE_SD_CPP left get_active_diffusion_engine() handing back the sd.cpp backend for the rest of the process, and every later route that reads the active engine then saw an unloaded model: eight tests in test_openai_images_generations_route.py returned 503 in a full-suite run while passing on their own. The autouse fixture now snapshots and restores it. * Stream gallery clips, and close three races around them Four fixes from the latest review pass. The video gallery downloaded each clip into a blob before it could play, so playback waited on the whole file (tens to hundreds of MB), seeking was limited to what had arrived, and every viewed clip stayed pinned in the webview. The file route already streams and serves ranges; it just could not be a <video src> because it is bearer-gated. Mint a short-lived signed link instead (its own HMAC secret, 12 hour TTL, separate from the image links) and hand it to the element, which then fetches only the ranges it plays. That removes the blob budget, its LRU and every revoke on this page. The sd.cpp readiness probe accepted any process answering on the port, so a foreign server that grabbed the port between the bind check and the spawn was adopted as ours. Confirm the listener is our child before reporting ready, and stay best-effort (psutil missing, an unknown owner, or any probe error still passes) so the check can only reject a definitely foreign process. Dataset import held its lock for the extract but not for the upload path, so two concurrent uploads into the same folder interleaved; take the same lock and return 409. And reject Windows device names (CON, NUL, COM1..9, LPT1..9, with or without an extension) plus trailing periods in dataset names, which are unopenable on Windows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Build the image download plan for the engine that will load /images/download-plan always asked the diffusers backend, while /images/load picks the engine per host: a GGUF pick on a machine with no usable GPU routes to native sd.cpp, which reads a single-file VAE plus text encoders and never opens the base repo's sharded components. Measured on unsloth/FLUX.2-klein-4B-GGUF (Q2_K): the plan staged 7.66 GB of FLUX.2-klein-4B components the native load discards, and the 7.80 GB sd-cli actually needs was then fetched inline by the loader, outside the download manager's progress and its disk preflight. Z-Image-Turbo is the same shape. The plan now asks whichever engine the load will select. predict_engine() applies the selection policy without any side effect: it activates nothing (staging a download must not unload the resident model) and only locates the binary rather than installing it, but still counts an installable binary as available, since that is what the load does on a fresh host. The native backend gains a download_plan built from the same _asset_specs the loader fetches, returning the same envelope, so the manager stages exactly the files sd-cli opens. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not let a queued generation outlive the model, and three scan fixes Five items from the latest review; four were real. An unload or arbiter eviction only cancels the generation holding _generate_lock. A second request queued behind it holds no cancel event yet, and Python locks are not FIFO, so it could take the lock the instant the active denoise released it, still see a loaded pipeline, and run a whole new denoise after the model was told to go away: the eviction then waits minutes for it and an image lands after the eject. Unload and a superseding load now raise a fence under _lock before they queue, and a generation that wins the lock while one is pending refuses instead. The cached-model scan judged pipeline completeness across every revision, so a repo holding an older complete snapshot plus a newer companion-only one read as complete while the snapshot from_pretrained actually opens has no transformer. Both scans now look at the revision the loader will open. Deleting a dataset image deleted its caption sidecar unconditionally, which for cat.jpg alongside cat.png removed the caption the survivor still resolves to. The sidecar now goes only with the last image of that stem, matching what the thumbnail cleanup beside it already did. Importing an example into a folder that holds no images but does hold files fell back to promoting the staging dir one file at a time, so an interruption left a partial dataset that the image_count check accepts as complete on retry. Those files are folded into the staging dir instead and the promotion stays a single atomic rename. The MPS generator report does not apply: torch.Generator(device="mps") has worked since PyTorch 2.0 (pytorch/pytorch#91348) and the studio installer pins torch>=2.4. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten diffusion comments Collapse the multi-line comment blocks across the image, video, sd.cpp and diffusion-training code to one or two lines each, and drop comments that only restate the statement below them. Comments only, no code or behaviour changes. * Tighten diffusion comments (second pass) Collapse the remaining multi-line comment blocks in the video page, training routes and service, sd.cpp server and installer, memory and speed planners, and the shared request models. Comments only, no code or behaviour changes. * Tighten diffusion comments (third pass) Collapse the remaining multi-line comment blocks in the attention, cache, LoRA, prequant, precision and compile-cache modules, the sd.cpp arg builder and engine, the video routes, the Ideogram 4 assembly, the model picker, and the diffusion test suites. Comments only, no code or behaviour changes. * Restore the dataset when an example import cannot be promoted Promotion folds the folder's pre-existing entries into the staging dir so the swap is one atomic rename. Every failure after that fold left the user with nothing: the 409 path and an os.replace error both fell through to 'finally: shutil.rmtree(staging)', which deleted the entries that had just been moved in there, while the response said 'Nothing was written'. A same-named entry was also unlinked outright before the promotion was known to succeed. Park superseded same-name entries in a rescue dir instead of deleting them, record every move, and restore all of them if any step of the promotion fails. The fold loop itself is covered too: renaming a non-writable directory raises EACCES on POSIX, which previously escaped as a 500 after the earlier entries had already been moved out. A failed rename now maps to the same retryable 409 as the rmdir conflict. Verified with the reported trigger (a non-empty mode-500 directory whose name collides with an imported file): the folder listing is now identical before and after the failed import. * Drain the teardown fence on a failing unload, give each load its own cancel event Two independent leaks on the image path, both already solved elsewhere in the same file. unload() incremented _teardown_waiters, ran _unload_locked() and decremented, with no try/finally, while the superseding-load path used a finally for the same pair. _unload_locked ends in clear_gpu_cache(), whose CUDA branch calls synchronize/empty_cache/ipc_collect unguarded, and a sticky CUDA fault makes those raise. The count then never drained, so every later generation was refused as cancelled for the life of the process, a fresh load included, since begin_load's own increment and decrement are symmetric. unload() is reached from the chat/video GPU handoff, the engine router and two training routes, so one fault during an ordinary handoff wedged image generation until restart. Release it in a finally. The image and native backends each cleared one shared cancel Event on a new load. unload() sets that event to cancel an in-flight multi-GB download and drops _loading in the same breath, so a replacement load is admitted while the cancelled worker is still inside the fetch, and its clear() re-enabled the very object that worker was watching: the cancelled download resumed and ran alongside the replacement. Take a fresh Event per load and thread it to the worker, as the video backend already does, and set it under the lock since begin_load now rebinds the attribute. * Name utf-8 on the diffusion text I/O and the sd.cpp subprocess pipes tests/test_text_io_encoding.py failed on five files this branch adds. Text I/O without an explicit encoding falls back to the Windows ANSI codepage, so a non-ASCII path or manifest value round-trips corrupted, and the three sd.cpp pipes decode the child's UTF-8 output as ANSI on Windows despite already passing errors = 'replace'. Eleven read_text() / write_text() sites across diffusion_compile_cache, diffusion_ideogram4 and diffusion_krea2, plus text = True on the sd-cli version probe, the sd-cli run and the sd-server pipe. * Record the load-time build on a gallery image's recipe A gallery record documents itself as the image's full generation recipe and is embedded in the PNG, but the only load-related field it carried was the repo id. A GGUF repo holds many quants, so that does not say which one made the pixels, and it says nothing about an adapter baked in at load time. The fallback meant to cover the baked case could never fire: with no loras on the request _adjust_baked_loras zeroes every baked adapter and _active_lora_pairs drops zero-weight entries, so active_loras was always empty. A baked-and-disabled build is not the same pipeline as a never-baked one, so the recipe could not reconstruct the image once the model was rebuilt. Persist model_kind, gguf_filename, transformer_quant and the baked adapter names, read off the load state rather than the request, and show them in the recipe popover. The new fields are optional with defaults, which matters because list_gallery_images drops any record that fails validation, so a required field would have emptied every existing gallery; a regression test pins that. * Drop eleven duplicated comment tails, restore the mxfp8 denial note The comment passes collapsed several wrapped blocks onto one line without deleting the last physical line of the original wrap, leaving the tail of each sentence repeated as its own comment underneath. Two of the eleven were re-worded rather than byte-identical, so a strict suffix match missed them. 6e16ad16f also dropped the line justifying the qwen-image mxfp8 denial while that rule stayed live in _FAMILY_SCHEME_DENY, under a header that then documented only fp8 and nvfp4. Restored. Comments only; verified with the AST gate. * Only let the dense-quant fallback use shards the prefetch actually staged The prefetch skips the base repo's transformer/ shards whenever a prequant checkpoint is expected, since that checkpoint replaces them. But a prequant fetch can fail for reasons the planner cannot see: an unpublished, gated or renamed artifact, a hub 5xx, a proxy, a checkpoint the validator rejects. The loader then fell through to from_pretrained(subfolder = 'transformer') and pulled those shards inside the load lock during 'finalizing', after the previous pipeline was already evicted, where the cancel event has no reach, load_progress has already reported bytes_downloaded == bytes_total, and the cache-disk gate had only reserved the small prequant checkpoint. That is verbatim the situation _dense_quant_prefetch_needed's own docstring exists to prevent. Gate the in-loader dense fallback on the shards being staged, read off the returned file list rather than the request so a failed size estimate closes it too, and let the GGUF build take over otherwise, which is what the prequant-sized replan already does one branch over. It is also the invariant the text-encoder path already enforces: only a component that really resolves may have its dense weights dropped from a plan or a prefetch. Adds the missing coverage for the gate's prequant arm, which the existing disk-gate tests never reached. * Refuse a training output_dir that resolves to the outputs root resolve_output_dir strips a leading 'outputs' and drops '' / '.' segments, so '.', './', './.', 'outputs', 'outputs/outputs' and ' . ' all clean away to nothing and land on the outputs root itself rather than a run directory under it. The DiT trainer then writes pytorch_lora_weights.safetensors flat into the root, where scan_trained_models and scan_checkpoints cannot see it (both filter is_dir()), and a second such run overwrites the first. The UI only checks the field is non-empty, so 'outputs' is one plausible run name away. Refuse it with a 400 that says what to do instead. cond_cache_dir collapses the same way but has an honest 'off' to fall back to, so a root-resolving value now means the in-memory cache, which is what the comment above it already promised: otherwise a run drops one flat safetensors per cached latent and caption into the directory trained models live in. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Repair three defects the main merge left behind Git merged all three files without a conflict, but the result was wrong in each case. The tests only started failing once the merged tree was run. routes/inference.py duplicated the GGUF load block: this branch had moved the gguf_load_in_flight marker and the _hub_download_blocks_gguf_load guard under "if config.is_gguf and config.gguf_hf_repo", and the conflict resolution re-added main's copy at the old position, so both ran. Dropped main's copy; the earlier placement is the deliberate one, so a 409 from the hub guard cannot tear down a resident Images or Video pipeline. test_gpu_selection.py still called _hf_offline_if_dns_dead, which main renamed to _hf_offline_if_unreachable_for (#7591). Disjoint edits, so no conflict, but four route-error tests referenced a function that no longer exists. test_gguf_load_cache_reuse.py anchored its ordering assertion with rindex over "if config.is_gguf:", taking the last one before the load marker. That only held while _resolve_inherited_extra_args sat above every such line; main has since hoisted it above the gpu_ids preflight, so the anchor landed between the call and the marker and the assertion compared against an unrelated later call site. The ordering it checks is a property of _load_model_impl as a whole, so it now anchors on the function. The ordering itself is intact: _resolve_inherited_extra_args, then the gguf_load_in_flight marker, then the hub guard, then the chat handoff, then unload_model. * Stop a real sd.cpp install from breaking its own discovery tests The five "nothing is installed" assertions in test_sd_cpp_engine.py cleared SD_CLI_PATH and UNSLOTH_SD_CPP_PATH and patched Path.home, which covers hops 1, 2 and the fallback half of hop 3. It leaves two hops live. Hop 3 goes through managed_install_root(), which honors UNSLOTH_STUDIO_HOME and STUDIO_HOME and resolves to the stable-diffusion.cpp directory beside the studio home. That is the documented way to run side-by-side Studios, so anyone who has one set gets a real sd-cli back from a finder the test expects to return None. Hop 4 is the in-tree developer build, which does the same for anyone who built sd.cpp inside the checkout. Isolated with an autouse fixture rather than another helper call, because reaching the failure needs no fixture: SdCppEngine(binary = None) runs the finder from its constructor, which is why test_generate_raises_when_binary_missing failed too. The fixture points the studio home and the in-tree root at an empty tmp tree, so every hop is answered by the test rather than by the box. The in-tree root moves into a named in_tree_install_root() so it can be pointed somewhere empty; behaviour is unchanged, including the OSError and IndexError guard on an unexpected layout. * Make the sd.cpp uninstall test actually extract the code it tests The two sed ranges anchored the production fragments at column 0, but both blocks sit inside the main removal function and are indented, so each range matched nothing and LOOP_FILE / DEFAULT_FILE came out empty. Sourcing an empty file is a no-op, so the suite reported 4 passed / 7 failed: the seven removal assertions failed because nothing ran, and the four that passed were 'kept' assertions that pass trivially when nothing runs. The shell job auto-discovers tests/sh/test_*.sh and runs each under set -e, and this file is not in its skip list, so it was a deterministic red. Anchor on optional leading whitespace and fail loudly on an empty or _remove_path-less fragment, so a future reshuffle of uninstall.sh cannot make the suite vacuous again. Now 11 passed, 0 failed, against the real removal loop and the real default-mode block. * Agree on what an engine can build, and on what the installer owns Two gates, each half-applied. The unbuildable-family gate had one caller, the image branch of the cached repo picker. The GGUF classifier, the local-model classifier and the video branch had none, so on a diffusers too old for a family the picker still offered its GGUF and the load then failed. Meanwhile the loader asserted the diffusers pipeline class before engine selection, so a GGUF this host routes to native sd.cpp, which instantiates no pipeline class at all, was refused with an upgrade instruction that could not help. Both are wrong one-sidedly: hide a family only when NEITHER engine can build it, and demand the diffusers class only when diffusers is what will load it. One predicate, family_buildable_here, now answers both, so the picker and the loader cannot disagree. The population is real on Python 3.9, whose diffusers ceiling is 0.36: Flux2KleinPipeline, Krea2Pipeline and LTX2Pipeline are all absent there, and FLUX.2-klein GGUF is a repo in this PR's title. That assertion also raised RuntimeError, which /images/load maps to 409, the status that otherwise means a load is already in progress, and which escaped /images/download-plan (it catches ValueError and FileNotFoundError) as a bare 500 with the message lost. It is an unloadable pick like every other, so it raises ValueError and both routes answer 400 with the text intact. The managed-binary repair used a path test for ownership while the installer uses a marker. On a managed root without the marker, which is any install predating it, the repair deleted sd-server and the reinstall was then refused, permanently, because the surviving sd-cli keeps the directory non-empty so the marker can never be claimed: the user went from an unrunnable binary to no binary and no way back. Require the marker before discarding, which is the same definition of ours that uninstall.sh already uses to keep a user's own stable-diffusion.cpp checkout. A genuinely interrupted extraction still self-heals, since install() writes the marker before it extracts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add gguf and av to the studio extra so the wheel install matches studio.txt This branch added both to studio/backend/requirements/studio.txt, which is what install.sh uses, but never to the studio extra in pyproject.toml, which is what 'pip install unsloth[studio]' uses. Nothing else keeps the two in sync, so tests/studio/install/test_studio_extra_matches_requirements.py failed deterministically, and the CI job runs the whole tests/ tree. The drift is not only a red test: gguf backs diffusers' GGUFQuantizationConfig and av does the MP4 encode and audio mux, so a wheel install got a Studio that cannot read a GGUF or export a video, which is most of what this PR adds. * Fix two test-suite defects: the CPU-only patch gate and a popped module The patch backend reaches unsloth_zoo's helpers only through an ~940 MB 'import unsloth', which a CPU-only host cannot complete, so every patch install returned False and nine arch/eager/compile tests failed there -- exactly the runners the retry was narrowed to protect. unsloth_zoo only wants UNSLOTH_IS_PRESENT in the environment, which costs nothing and needs no accelerator, and the conftest already sets its sibling UNSLOTH_ALLOW_CPU. Set it there too, as run.py and main.py already do at module scope, so no real server ever took the expensive route. CPU-only goes from 9 failed to 24 passed, 1 skipped; the GPU run is unchanged at 29 passed. test_setup_cache_env_hf_home popped utils.hf_cache_settings to model a fresh process and never restored it, so a later import built a second module object and rebound it on the utils package. test_hf_cache_settings then wrote its setting into one object while core.inference.diffusion read the other, the same split-module failure the xet shim already had. Restore both bindings in teardown: 21 pass together, and each file still passes alone. * Restore the setup_fail assertions the main merge reverted A fourth instance of the class 9541cc535 fixed: the merge took main's studio/setup.sh, which #7644 changed to abort through the setup_fail helper so desktop mode still emits [TAURI:ERROR], but kept this branch's older copy of the test, which still asserted the literal 'exit 1'. setup.sh is byte-identical to main here and the only diff in this file was the reverted assertions, so take main's version. 60 passed. * Fence the video teardown, reject non-finite clipping knobs, stop a test installing sd.cpp The video backend never had the teardown fence the image backend documents. Its unload and its superseding-load path both signalled the active generation, then did 'with self._generate_lock: pass' and tore the state down with the lock free. A generation queued behind that barrier holds no cancel event yet, so the signal cannot reach it, and Python locks are not FIFO: it won the lock the instant the barrier released it, read a still-loaded state and denoised a whole clip against the pipeline being freed. Reproduced on both paths, where the queued generation returned a finished MP4. Mirror the image backend: count waiters, refuse a generation while one is pending, and tear down inside the barrier with the counter released in a finally. _teardown_state becomes _teardown_state_locked since the caller now holds both locks. max_grad_norm and snr_gamma were bounded on one side only, so 1e309 floated to inf and passed. clip_grad_norm_ then computes an infinite clip coefficient, clamps it to 1.0 and scales nothing, and min(snr, inf) / snr makes every min-SNR weight 1.0. The run starts, reports normal progress and trains with the requested knob silently disabled. Measured both. NaN already failed the bounds; allow_inf_nan makes that explicit. learning_rate and flow_shift already guard this exact vector. test_load_routes_to_sd_cpp_on_cpu stubbed ensure_sd_cpp_binary but not ensure_sd_server_binary, which select_and_activate_engine probes first with installs enabled, so the unit test downloaded and unpacked 108 MB into the developer's real ~/.unsloth when UNSLOTH_STUDIO_HOME was unset. Now 0 bytes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Key the video schedule reset on the resolved defaults, not the repo id A GGUF repo holds several variants, so another client swapping a distilled build for the base one from the same repo changes the steps and guidance the backend reports while repo_id stays put. modelChanged stayed false, the page kept the previous schedule, and later generations ran an 8-step distilled setting on a model that expects roughly 40 steps and CFG 4. Include the reported defaults in the key. They only move when the resident artifact does, so a manual steps change still survives a status poll. * Retire the release-lag pin now that 2026.7.6 carries the relaxed gates The virgin Windows container row that installs from PyPI on purpose was tolerated with continue-on-error, because a Server Core container has no Microsoft Store and so no winget, and the released studio/setup.ps1 hard-stopped on a winget-only git gate and reached for winget again for the VC++ runtime. #7549 relaxed both, but it landed on 2026-07-28 and the newest wheel was 2026.7.5 from the 23rd, so the row could not get past it. The pin carried its own tripwire for exactly this moment, and it has fired: unsloth 2026.7.6 shipped 2026-07-29 and the released wheel now installs end to end, so the step reported success and the assertion errored with "delete this pin and drop continue-on-error from the Install step so this row gates". Doing that. Both rows now gate unconditionally, and the assertion that enumerated the accepted failure signatures goes with the pin it protected, since there is no longer an accepted failure for it to describe. The other three release-lag mentions in this file are descriptive rather than tolerated-failure pins: they explain why the fedora and ubuntu-nonroot legs run with overlay: true, and both already refuse a triton failure as a regression rather than accepting it as lag. Left alone. * Pin the hosted pre-quant onto the plain-torch fp8 kernel A Z-Image GGUF pick at the DEFAULT speed mode, once the hosted fp8 pre-quant is actually reachable, dies at generate with an HTTP 500: torch._dynamo.exc.Unsupported: Operator does not support running with fake tensors. Developer debug context: unsupported operator: mslk.f8f8bf16_rowwise.default _fp8_config already pins KernelPreference.TORCH when it BUILDS a config, precisely because the default AUTO switches to the MSLK kernel wherever an mslk package is importable (sm90+). A hosted checkpoint escapes that pin completely: the preference is serialized per Float8Tensor, and all 239 weights in the published Z-Image-Turbo-FP8 checkpoint carry AUTO. Loading it re-arms the very kernel the pin exists to avoid, mslk.f8f8bf16_rowwise has no fake impl, and the first compiled generate therefore cannot be traced. Isolated away from the product to be sure of the mechanism: quantise one Linear three ways on this box and compile each. KernelPreference.TORCH eager ok, compiled ok KernelPreference.AUTO eager ok, compiled FAILS on mslk.f8f8bf16_rowwise library default eager ok, compiled FAILS the same way So the pin is correct and necessary, and the only gap is that the hosted path never got it. _validate_checkpoint checks scheme, granularity, base model, min_features, exclude tokens and fast_accum, but not this. Rewriting the preference on load is safe: it selects a matmul kernel, it is not weight data, so the tensors stay bit-identical and the checkpoint's own state_dict_sha256 still describes them. It is also the faster path compiled, since the opaque extern call blocks inductor quantize fusion. Why this went unnoticed: the pre-quant repos are private, so nothing that could not read them ever took this path. It becomes the default the moment they are readable. Verified end to end at 1024x1024, 8 steps, speed_mode default, on the published checkpoint: before HTTP 500 at generate, diffusion.generate_failed after pinned 239 weights to the plain-torch fp8 kernel, load 10.0 s, transformer_quant fp8 with compiled engaged, cold 7.9 s, warm 0.87 0.87 0.87 0.86 0.87 s That warm number also beats the 1.4 s recorded for this shape on 2026-07-26. 159 prequant and transformer-quant tests pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover the fp8 kernel pin so the 500 cannot come back quietly The hosted pre-quant re-arming MSLK was only visible as an HTTP 500 on a compiled generate, which needs a GPU, a readable private repo and an importable mslk to reproduce. None of that is available in CI, so the pin would rot unnoticed. Three hermetic cases on _pin_kernel_preference instead: AUTO weights are rewritten and already-TORCH ones are left alone (counting only what changed), a weight that refuses the assignment does not sink the whole load, and with no torchao enum available the checkpoint is left exactly as saved. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: tighten comments across the image generation, video and training code Rewrite the comments added by this branch to be shorter and clearer: collapse multi-line explanations into one or two lines, drop comments that only restate the code, and keep the rationale that explains why a choice was made. No code changes. * Studio: let the pipeline-class guard survive a host with no diffusers Backend CI installs the CPU-only dependency set, which has no diffusers, and runs without a GPU. 13 tests failed there for three separate reasons, one of which is a real product bug rather than a test-environment gap. assert_pipeline_class_available did a bare "import diffusers", so on any host without it the call raised ModuleNotFoundError instead of the ValueError its own contract promises. /images/download-plan catches only ValueError and FileNotFoundError, so that escaped as a bare 500 with the message lost, which is the exact failure the guard exists to prevent. It is reachable in production: the native sd.cpp engine serves GGUF picks on a CPU or Apple host where diffusers is never installed. Absent diffusers now returns, since the check answers "is the installed diffusers new enough for this family" and there is no version to judge; a pick that genuinely needs diffusers still fails in the loader with its own message. The rest were tests asserting through gates unrelated to what they cover: - Three cond_cache_dir route tests drive a DiT family, which the start route refuses without an accelerator. They now take the existing dit_train_host fixture, written for this case, so they keep testing the schema on every host. - test_download_plan_forwards_the_load_time_controls stubs the diffusers planner but sends a GGUF pick, which routes to the native planner on a GPU-less host. It now pins the engine, so it tests kwarg forwarding rather than the hardware. Engine selection keeps its own tests. - The pipeline-class guard test needed a real diffusers only for its sweep over every shipped family. That half is split out and skips on its own; the stub-driven refusal needs no diffusers and still runs. Added a test for the absent-diffusers contract above. 409 pass on a GPU host with diffusers; 301 pass and 1 skips with diffusers blocked and no GPU. * Studio: cut the image generation comments down further Second pass over the comments this branch adds. Collapse the multi-line blocks that still read as paragraphs, rewrite the longest one-liners so they say the same thing in fewer words, and drop a stale comment that had drifted away from the constant it described. No code changes: only comments and whitespace. * Studio: workflow rail, sidebar Video row, and Train settings polish Sidebar - Pin Video under Images by default; persist store bumped to v3 so an untouched install adopts it while a custom arrangement is left alone. - Add "Customize sidebar" to the end of the More flyout, opening Settings > Appearance scrolled to the sidebar nav section. Images - Replace the workflow dropdown with an icon rail down the left edge: all seven workflows visible, keyboard nav, tooltips carry the labels. - Workflows stay selectable with no model loaded, so one can be set up before picking a model. Generate is already gated on a loaded model. - Video moves out of the Create/Train strip into a link at the far right, with a matching Images link on the Video page. - Wider gutters around the settings column, headings matched to Train. Train - Field guidance moves into "i" tooltips; only state that limits a control stays on the page. Steps, LoRA rank and the rest gain hints. - /info reports params, qlora_vram_gb, gated and note as fields, so the family note renders as chips. vram_note is rebuilt from them unchanged for older clients. Shared - Number steppers appear on hover or focus. - Lighter control border (#e9e9e9) on Images and Video, light mode only. * Studio: move Advanced inline on Images and Video - Replace the top-bar toggle and right-docked Advanced panel with a disclosure under Seed, so load-time tuning sits with the settings it affects and opening it no longer shrinks the preview canvas. - Video uses the bordered variant, since offload and memory decide whether a model fits at all. - Open state persists per page in localStorage. - Tighten the Steps unit trigger: 14px chevron, less right padding. * Studio: full width media pages and layout polish - Drop the 1100px cap on Images Create, Images Train and Video, so the preview canvas grows with the window instead of sitting in a band. - Align the workflow rail to the model selector label, widen the gap before its divider, and restore the divider itself. - Advanced: more room above and below, icon and label sized to the slider rows, and the same quiet row on Video as on Images. - Give Video a pane heading and description, matching Images. - More space above Seed on both pages. - More no longer takes the active style when the current page is one of its own rows. * Studio: fix the Extend side toggles The resting outline used a ring, and index.css blanks the ring on the button holding mouse focus, so the side you just clicked lost its border until focus moved. Use a border for the outline and leave the ring to focus-visible. Also separates the two states properly (the off state had no surface of its own), drops the border in dark mode as the inputs do, and adds the aria-pressed and focus-visible styling the buttons were missing. * Studio: drop the border on the Extend side toggles Light mode should carry no border either, so the fill alone marks the state: a muted surface when off, a primary tint when on. Matches the borderless treatment dark mode already had. * Studio: move the Images workflows into the sidebar The workflow switcher was a vertical icon rail on the page. It now lives in the sidebar under Images, so the page keeps that width for the canvas. - Workflows list under the Images row, with a chevron to fold them away. Hovering the row peeks them in a flyout on the standard menu surface. - Clicking Images while already there toggles the list instead of navigating. - The listed workflow carries the highlight, so the Images row drops it. - Create takes its own icon: it was sharing the New chat pencil. - Model hub moves above Projects. Persisted layouts bump to v4 and only adopt the new order where the stored one is still a shipped default. - Images and Video content both start at 32px, clear of the sidebar. * Studio: simplify the Images workflow submenu Dropping the hover flyout: two ways to reach the same seven workflows, one of them overlapping the list right below it, read as clutter. - The flyout is gone. The workflows are rows under Images and nothing else. - On the Images page they are always open, since they are that page's switcher. Elsewhere they stay folded, and the row's chevron opens them. - Create takes the sparkles icon. - Create and Train both pad their settings column to 40px a side, so the two tabs line up with each other and with the model selector. * Studio: dock the generate action and tidy the Create controls The primary action sat at the foot of a long scroll, so it was off screen until you scrolled for it. - Generate, Video's Generate and Train's Start training float at the bottom of their settings column. No bar behind them: hover lightens the fill rather than thinning it, and the disabled state is opaque, so the controls underneath never show through. - Aspect ratios read as names: Square (1:1), Widescreen (16:9) and so on. - Width and height replace their sliders with two compact boxes. Type a size or pick one from the menu; the value still snaps into range and still drives the locked ratio. - Negative prompt gets a quiet disclosure under the prompt. It was there before but only above guidance 0, which is not the default, so it never showed. - The info "i" is smaller across the settings UI. * Studio: reveal Images and Video field hints on hover The "i" next to every field label sat there permanently, which made a column of settings read as busier than it is. Scoped to the diffusion pages by CSS rather than per component: the hints come from the page's own Field and SelectRow, the train panel's FieldLabel, and the chat ParamSlider, which is shared and should not change for chat. Reveals when the pointer is inside the field, on focus-within for keyboard, and stays inert while hidden. InfoHint grows a data-slot so the rule has a stable hook, rather than keying off its aria-label. * Studio: one-line sliders, edge fades, shared negative prompt - Steps, Guidance and the rest put label, track and value on one row. ParamSlider is shared with chat, so this is an opt-in prop and chat keeps the stacked layout. - The settings columns fade at whichever edge they run past, as the sidebar and model picker do, instead of cutting off. New useScrollFades hook drives all three. - Negative prompt moves to a shared component and Video picks it up, so both pages collapse it the same way. - Right gutter sits closer to the rule, Steps takes a bigger break above it, and the Advanced rule sits between its neighbours rather than up against the field above. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add the API monitor to the sidebar nav Sits under More, below Export. Points at the monitor page, not the API keys dialog the profile menu opens. No store version bump: an id missing from a stored layout is appended with its default pinned state, so existing sidebars gain the row at the end and keep their own order. * Studio: take main's copy of the sidebar spinner test The merge kept this branch's older version, which pins the Recents row to h-[33px]. Main already relaxed that to any height, since row density moves independently of the trailing column the test is about. * Studio: fix FLUX.2 klein size resolution, HunyuanVideo tier routing, and the personalization save Five fixes found while auditing the model registries against the live Hub. FLUX.2 klein 9B loaded against the 4B config. One family covers both klein sizes and defaults to 4B, relying on the base_model card tag for the real base, but the trust allowlist only had klein-4B, so the correct tag was discarded and a 9B GGUF (inner_dim 4096) was loaded against a 4B config (3072). That surfaced as a bare shape mismatch from inside the GGUF quantizer naming neither the file nor the repo. Adds the missing allowlist entries and a header-level size check that fails early with a legible message; the check is fail-open, so an unreadable file, a non-FLUX.2 family, or an unmapped base leaves the load exactly as it was. The sd.cpp text encoder rule matched the literal "klein-9b", so klein-base-9B was handed the 4B encoder. HunyuanVideo-1.5 720p checkpoints routed to the 480p family. Only the literal 720p_t2v path was aliased, so 720p_i2v and every GGUF repack fell through to the generic token and inherited the 480p base repo, which also supplies the VAE and text encoder. The tier is baked into the weights (transformer target_size 640 vs 960, scheduler shift 5.0 vs 9.0) and the two bucket lists are disjoint, so this ran the whole pipeline off-tier. Resolution presets that the checkpoints were never trained for. Wan2.2 TI2V-5B is 720P-only upstream (SUPPORTED_SIZES is exactly 704x1280 and 1280x704, asserted in generate.py), and the HunyuanVideo 480p square preset 624x624 is not a bucket of its tier; 640x640 is. Personalization could never be saved. The frontend ships eight sidebar nav ids and refills every missing one on each save, but the backend Literal had seven and no "api", so every PUT to /api/settings/personalization returned 422. Aligns the server defaults and the Literal with the shipped layout. A catalog row pointing at a file that does not exist. unsloth/Qwen-Image-2512-FP8 holds torch prequant .pt checkpoints, not qwen-image-2512-fp8.safetensors, and fp8 is denied for this family anyway because it renders black, yet the row was the auto-route target on any GPU above 40 GB. Also marks FLUX.1-schnell, Krea-2-Turbo and the two Ideogram repos gated: all four are gated on the Hub today, and schnell being Apache-2.0 does not make it ungated. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stop a case-only duplicate upload destroying an image, and size the load against the right GPU Two fixes from the review round. A dataset upload containing two names that differ only by case silently loses one on Windows and macOS. The in-batch duplicate check is an exact string compare, so "Cat.png" and "cat.png" both pass, but on a case-folding filesystem they are one destination: the commit step moves the first staged part aside, writes the second over it, then deletes the backup, while the response still reports both as uploaded. Captions collapse the same way and never even reach the image branch. No single folder can hold such a pair, but the Windows and macOS open dialogs flatten search and Recents results across folders into one multi-selectable list, which is a normal way a LoRA dataset gets assembled. Rejecting the pair everywhere would regress Linux, where the two really are different files with their own sidecars, so the check probes the filesystem once per process instead of keying off sys.platform: macOS also ships case-sensitive APFS volumes, and a Linux host can keep its Studio home on an exFAT or NTFS mount. A failed probe answers "case-sensitive", which leaves today's behaviour untouched. The quantization fit budget was sized against the wrong card under a reordering CUDA_VISIBLE_DEVICES. A bare "cuda" load lands on visible ordinal 0, but the reducer took the lowest PHYSICAL index, which the nvidia-smi path reports as index_kind "physical". Under "3,1" that sizes against GPU 1 while the pipeline loads onto GPU 3. The hook's own interface doc already promised the lowest visible ordinal. Ranks by visible_ordinal, falling back to index only for an older backend that omits it. * Studio: rework the diffusion dataset panel and align the pane gutters Upload: - one Upload button beside the dataset name, with an upload icon - picking files uploads them, no second confirm click - Add button beside the Training images dropdown, so images can go into a set that already exists - fall back to the upload form when a selected dataset no longer resolves Labeling grid: - two columns at any width; sm:grid-cols-3 crushed the tiles in a fixed width column - pin the caption size so the Textarea's md:text-sm does not outsize a tile - name the tile hover group; a bare one revealed every tile's Remove at once - Remove is an icon button on the image, not a word over the artwork - drop the amber tile fill, keep the No caption label - match the header, toggle and status text to the section's type scale Gutters: - the run area and the preview canvas now sit 40px off the rule, the gutter the settings column has off the page edge * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: give the dataset preview per-image hover and remove, and wrap it onto rows - each thumbnail is its own button, so it can carry an X that removes just that image; the strip was one button and could not nest a second control - hover lightens the image instead of drawing a border - wraps onto more rows rather than scrolling sideways, which also keeps the bottom corners rounded - samples 12 thumbnails, up from 8, now that a second row is available - a delete refreshes the panel's dataset counts * Studio: show the workflow icon beside the Images and Video headings Images reads it off WORKFLOW_TABS, so it is the same icon the sidebar submenu shows and it follows the active workflow. Video's heading is also Create, so it takes the Create icon rather than the nav row's film slate. * Studio: reject a duplicate LoRA id on the diffusion load path too DiffusionGenerateRequest already refuses a repeated adapter id, with a comment naming the hazard: _resolve_lora_set suffixes colliding adapter names, so the same id resolves the SAME adapter twice and set_adapters stacks both copies past the per-adapter weight bound. DiffusionLoadRequest bounds only the list length, and it matters more there. Generation-time stacking spoils one image; on the load path the adapters are baked into the quantized build before compilation, so the unintended combination rides every image until a reload. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: teach the GGUF reuse test double about holds_no_vram The resident-GGUF fast path consults llama_backend.holds_no_vram before asserting CHAT ownership, but two tests hand it a SimpleNamespace built before that attribute existed, so they raised AttributeError and returned a 500. A real LlamaCppBackend exposes it as a property; the doubles now carry it too. Product code is unchanged: these pass on main and were the branch breaking its own test, not a defect in the guard. --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: michaelhan <michaelhan2050@gmail.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>	2026-08-04 08:11:01 -07:00
Daniel Han	cf4acbcce5	Make the Colab oracle tripwire able to fire, and stop blaming start.py for a hung agent CLI (#7838) ... * Make the Colab oracle tripwire able to fire, and stop blaming start.py for a hung agent CLI * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep a connection-mode cap fatal, and do not read 137 as a timeout * Note that the opencode non-exit is intermittent * Close three gaps the waiver opened: turn-2 side effect, attribution-ab, and the guard tests * Bound a TERM-resistant CLI again, and judge a capped turn 2 on an artifact * Gate both timeout statuses on the clock, keep the cron lint reachable, and make refresh --all atomic * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-04 07:42:42 -07:00
oobabooga	f8730f4339	Installer: select CUDA wheels that cover the host's GPUs (#7814) ... * Installer: select CUDA wheels that cover the host's GPUs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Windows venv wipe and warning dedupe for PR #7814 - Windows pins torch<2.11, whose cu128 still ships sm_70, so capping a Volta to cu126 there rewrote a working family. The stale-venv check then read that as drift and deleted the venv on a direct "unsloth studio update", which cannot recreate it. Make the pre-Turing floor per-family (70 for cu128). - Repair an unpinned cu* -> cu* move in place instead of rebuilding the venv. - Decide the cu126 advice before deduping the uncovered-host warning: the host facts are release invariant but the artifact list is not, so the release walk-back let an unhelpful release swallow the remedy. - Gate the new coverage repair and the cu126 advice on x86_64, matching the cap. - Add tests/studio/test_pre_turing_cap.ps1: the parity test only greps for the call spelling, so neither PowerShell copy had behavioural coverage. * Tighten comments for PR #7814 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-04 06:55:29 -07:00
benj	6a58ea0f0e	Add Intel Arc GPU detection and XPU PyTorch install to Windows installer (#7706) ... * Add Intel Arc GPU detection and XPU PyTorch install to Windows installer The installer's GPU detection chain (NVIDIA -> AMD ROCm -> else) has no Intel Arc/SYCL/XPU branch, so Intel Arc GPUs fall into the "none (chat-only / GGUF)" branch and get CPU PyTorch despite PyTorch publishing XPU wheels at download.pytorch.org/whl/xpu. This adds: - WMI-based Intel GPU detection (Arc, Iris, UHD, HD Graphics) - Torch XPU availability check for migrated/upgraded environments - An XPU PyTorch install path with the whl/xpu index - CPU fallback with a pointer to the Intel oneAPI docs when XPU isn't available - Updated messaging from "NVIDIA or AMD ROCm" to include Intel Arc The XPU wheels ship their own oneAPI runtime (intel-sycl-rt et al.) so no Intel oneAPI Base Toolkit is required for GPU training. Tested on: Windows 11, Intel Arc 140V GPU (8GB), PyTorch 2.9.0+xpu Co-authored-by: CommandCodeBot <noreply@commandcode.ai> * Fix Intel XPU detection and install path for PR #7706 The XPU index selected during GPU detection was overwritten by Get-TorchIndexUrl before the install branch read it, so Intel hosts still got CPU PyTorch while being told XPU wheels were being installed. - Move the XPU reroute after Get-TorchIndexUrl, and let an explicit pin win - Detect via Get-CimInstance (Get-WmiObject is absent in PowerShell 7) - Match only Arc / Data Center GPU, so UHD / HD / Iris Xe are not promised XPU - Split Intel GPU present from XPU-capable so the CPU fallback hint works - Bound the XPU torch trio like every other index (bare names resolved torch 2.13.0 + torchaudio 2.11.0 and pulled unsloth back to an old release) - Clear the XPU state after a CPU fallback, mirroring the ROCm path - Teach the index family, GPU branch and torch flavor helpers about xpu * Keep install.sh diagnostics in parity with the install.ps1 xpu family install.ps1 now classifies an /xpu index leaf as family xpu / branch xpu, so mirror the same two cases in _tauri_torch_index_family and _tauri_gpu_branch. These feed the [TAURI:DIAG] line only, and a Linux user can already reach the xpu index via UNSLOTH_TORCH_INDEX_FAMILY, where it previously reported auto/unknown. Linux Intel auto-detection is not added here. * Tighten the Intel XPU comments in install.ps1 Comment and whitespace only, no code change. * Address Codex review on the Intel XPU path - Run the Intel scan before the GPU report chain instead of inside its final else. A WMI-named-only AMD adapter set ROCmGpuLabel and took that chain, so a discrete Arc card next to an AMD CPU's integrated Radeon was never detected. The scan is gated on no usable NVIDIA or AMD, and the Intel branch ranks above the two AMD-present-but-unusable branches, so a usable AMD host is unaffected. - Let a migrated env's torch veto the hardware match only when it is itself an XPU build. A CPU build reports torch.xpu.is_available() False for lacking XPU support, not for unsuitable hardware, and was blocking the CPU to XPU upgrade. - Detect Intel in studio/setup.ps1 too. It only knew NVIDIA and AMD, so every successful Intel install printed none (chat-only / GGUF) right after install.ps1 reported a usable Arc GPU. Self-contained so studio update works. * Address the second Codex round on the Intel XPU path - Reset $script:IsIntelXpu at the start of each invocation. Under the documented irm | iex path $script: is the caller's session scope, so a second run in the same session inherited a stale true, skipped the scan on a now-NVIDIA host and still rerouted to the xpu index. Reproduced in pwsh before fixing. - Gate the Intel scan on whether AMD actually gets a wheel, not on whether an AMD arch was seen. An arch missing from the family map has no ROCm wheels and lands on CPU torch, so it must not outrank a usable Arc card. The map is hoisted above the scan and consumed unchanged by the AMD reroute. - Select the XPU index in studio/setup.ps1, not just report it. Previously setup printed Intel GPU detected and then installed CPU torch, so studio update never migrated an Arc box off CPU. Adds a bounded XPU install with a CPU fallback, teaches the stale-venv check about +xpu, and mirrors the wheel-aware AMD gate so the two files agree instead of wiping the venv on every update. * Address the third Codex round on the Intel XPU path - Force the dependency pass on an Arc host whose torch is not XPU-capable, the Intel counterpart of the existing AMD escape. Without it the fast up-to-date path skipped the install block, so the xpu index selection was never reached and a CPU venv never migrated. - Confirm a working XPU runtime before treating an xpu venv as stale. If CIM is unavailable or returns an Intel name outside the Arc match, the expected tag fell through to cpu and a valid XPU environment was rebuilt and lost. - Force-reinstall the XPU trio only when the installed wheel is not already +xpu, or the pin changed. It was unconditional, so a fresh install re-fetched multiple GB immediately and again on every update. - Warn when torch.xpu.is_available() is false after installing XPU torch, naming the Intel driver floor. Otherwise the installer promised GPU training while unsloth raised NotImplementedError at import on a stale driver. - Stop the detection probe vetoing the hardware match. Its cpu fallback could not displace the installed +xpu wheel, so it only mislabelled a capable GPU as unusable; the driver warning covers that case honestly, and setup.ps1 agrees. * Bound the XPU probes, repair xpu pins in install.sh, and floor bitsandbytes on the Intel path install.sh: teach _torch_flavor_tag, _expected_torch_flavor_tag and _torch_index_repairable about the xpu leaf. The diagnostic already reported gpu_branch=xpu, but an xpu pin fell to the custom arm so a migrated env kept its CPU wheel. The +xpu flavor arm is required alongside, otherwise a correct 2.10.0+xpu wheel reads as cpu and gets force-reinstalled every run. install.ps1 / studio/setup.ps1: route every torch probe through a new bounded Invoke-BoundedPythonProbe (ProcessStartInfo, both streams drained async, WaitForExit, kill on timeout). A hanging Intel driver init is exactly what these probes detect, and an unbounded one would hang the installer instead of reaching the warning. Timeouts read as not-available. Get-InstalledTorchTag now shares the helper rather than carrying a second copy of the pattern. install.ps1: install bitsandbytes>=0.50.0 on the XPU path. unsloth's floor is >=0.45.5, so a migrated venv keeps a pre-0.49 wheel with no XPU library and 4-bit QLoRA silently turns off. Same floor the AMD paths use, since <=0.49.2 NaNs at 4-bit decode and an Arc card can sit next to a Radeon. * Floor bitsandbytes on the Studio XPU migration and on an explicit xpu pin studio/setup.ps1: `unsloth studio update` migrating a CPU venv to XPU replaced only the torch trio. install_python_stack.py then upgrades unsloth and unsloth-zoo alone, so an installed bitsandbytes 0.45.x kept satisfying the base floor while carrying no Windows XPU kernels, and 4-bit QLoRA silently turned off. Adds the same bitsandbytes>=0.50.0 --no-deps pass install.ps1 got, placed after the stack so it is the last word, gated on $XpuIndexUrl (the CPU fallback clears it, no-torch never sets it) and still inside the -not $SkipPythonDeps block so the up-to-date escape does not reach it. install.ps1: key the bitsandbytes pass off the index leaf instead of $script:IsIntelXpu. An explicit UNSLOTH_TORCH_INDEX_FAMILY=xpu pin on a non-Intel host skips the XPU branch but still installs the trio from the xpu index, so torch is +xpu and needs the same floor. The CPU fallback rewrites $TorchIndexUrl, so a failed XPU install reads as cpu and stays quiet. * Tighten the Intel XPU comments across the three installers Comment-only pass now that the review has settled: several blocks grew over successive rounds and were restating the code or narrating the review. Net 36 lines removed, with the load-bearing facts kept -- why ProcessStartInfo rather than the call operator, why both probe streams drain async, why the helper is defined above the Intel scan, the 0.50.0 bitsandbytes floor and why not the curated extra, and why PEP 440 means a migrated env can confirm but never veto the Intel match. Also records why the Studio bitsandbytes pass must stay above the ErrorActionPreference restore: Fast-Install needs EAP=Continue or PS 5.1 turns pip stderr into a terminating error. No code tokens changed; verified with a PowerShell token-stream diff of install.ps1 and setup.ps1, and by hand for install.sh. * Bound the Intel WMI scan, bound the stale flavor probe, and stop CUDA Triton shadowing XPU studio/setup.ps1: the stale-venv flavor probe read StandardOutput.ReadToEnd() before WaitForExit, so the timeout was unreachable and a wedged import torch hung studio setup forever; stderr was never drained either. Routed through Invoke-BoundedPythonProbe, which already drains both streams and kills on timeout. A timeout now reads as unreadable flavor, so the venv rebuilds. install.ps1 / studio/setup.ps1: bound the Win32_VideoController query and add a registry fallback. -ErrorAction suppresses errors but bounds nothing, and -OperationTimeoutSec is not enforced for the local COM session this uses, so a degraded WMI repository blocks forever. install_llama_prebuilt.py already runs this query out of process for the same reason and documents an Arc A770 being misrouted by it. The registry class key answers in-process; it is the fallback rather than the fast path because a stale driver config can outlive the hardware, and here a false positive would install XPU torch on a host with no Arc. studio/setup.ps1: replace triton-windows with torch's own XPU triton after the stack. Both distributions own the top-level triton package, sharing 151 paths including __init__.py and _C/libtriton.pyd, so an in-place cu-to-xpu repair leaves the CUDA build shadowing the XPU one. Removing it alone would delete the shared files the XPU wheel overwrote, and unsloth declares triton-windows as a win32 dependency so an earlier removal is reinstalled by the stack: uninstall and reinstall, after the stack, only while triton-windows is present. The spec is read from the installed torch, since the name changed from pytorch-triton-xpu to triton-xpu in torch 2.10. * Tighten the comments added with the bounded scan and Triton replacement Comment-only pass over the previous commit's additions, which had not been through one: 15 lines removed across the two bounded-scan headers, the two registry-fallback headers and the Triton block. Kept the facts that cost measurement: -OperationTimeoutSec not being enforced for a local COM session, Ok being false on an empty answer because a Windows host always has an adapter, the registry class key being fallback rather than fast path here, the 151 shared Triton paths, and why the uninstall has to be paired with a reinstall after the stack. No code tokens changed; verified with a PowerShell token-stream diff of both files, which also confirms the two helper copies stay identical. * Stage the Triton replacement behind a download so the uninstall cannot strand the venv The replacement uninstalled triton-windows and then installed the XPU triton from the index. A failure between the two left the venv with a partially deleted triton, since the uninstall drops the paths shared with the XPU distribution, and the warning made that look like a skipped optional repair. The uninstall cannot go last, because it removes the paths in triton-windows' own record and those are the shared ones. So fetch first: pip download the wheel, confirm one is actually on disk (exit 0 alone is not enough, an sdist-only mirror satisfies that), and only then uninstall and install the local file. A local wheel installs with the network refused, so nothing after the destructive step depends on the index. A failed fetch leaves triton-windows in place, which is the pre-existing shadowing rather than a broken venv, and says so. Past that point only disk or permissions can fail, so restore triton-windows if the local install does, leaving a triton that imports. If both fail the message is loud and carries the repair command, with the index URL redacted since a mirror pin can carry a token. pip only: uv has no pip download (astral-sh/uv#3163). * Windows: harden the Intel registry fallback and declare the XPU install state up front Get-IntelRegistryAdapterNames wrapped the whole enumeration in a single try, so one unreadable subkey discarded every adapter found before it. windows_intel_gpu_in_registry(), the in-process Python probe over the same class key, skips per subkey and continues; the PowerShell copy now does too. It also matched on the PCI vendor id but returned DriverDesc, which the callers re-filter on "Intel", so a localized or OEM-branded Arc was found here and dropped there. Both installers carry the same copy and a test asserts they stay identical. setup.ps1 read $installedTorchTag and $XpuIndexUrl from outside the blocks that assign them. Unset and $null are both falsy so behaviour is unchanged, but a caller running with Set-StrictMode -Version Latest turned those reads into terminating errors, and install.ps1 is documented as irm | iex into the caller's own session. Two comment corrections: 0.48.2, not 0.49.0, is the first win_amd64 bitsandbytes wheel carrying libbitsandbytes_xpu.dll, and the triton package overlap is version-dependent rather than a fixed 151 paths. The new test drives the shipped helper with the registry cmdlets mocked rather than reading a hive, so it runs on Linux and macOS as well as Windows. * Studio: show the Intel XPU runtime row in the About tab hardware.py has always emitted versions["xpu"], but HardwareInfo only ever declared cuda and rocm. On an Arc host both of those are null, so the runtime row disappeared entirely while the GPU name and VRAM rows still rendered, leaving a host that looks half detected. That was unreachable on Windows until the installer learned to select XPU wheels, which is what makes it worth fixing here. The three-way choice is lifted into a helper at module scope: inlining it pushes AboutTab past the cognitive-complexity ceiling. The label is a proper noun, so every locale carries the same literal. * Windows: reach Intel XPU through a localized name, a stale fast path and an old wheel Four holes in the XPU paths, all found by driving the shipped code rather than reading it. The registry fallback only ran when the CIM scan failed. When it succeeds and returns a localized adapter name, which on non-English Windows carries no ASCII "Intel", the filter dropped the adapter and the host went to CPU torch. The registry now re-labels an adapter WMI already reported, matched by name so an entry naming nothing WMI listed stays ignored: a driver record outliving its card still cannot promote a host WMI answered for. The XPU trio accepted torch 2.4 and 2.5, which unsloth/models/_utils.py rejects at import for an XPU device. An xpu mirror carrying only an older wheel produced an install that reported success and then failed on the first import, and an existing 2.5+xpu venv was kept because it satisfied the range. The floor is 2.6 on the XPU paths only; the CPU fallback keeps 2.4. The "package is up to date" fast path escaped for an Arc host on CPU torch, but not for one already on XPU torch whose bitsandbytes predates the XPU kernels or whose triton-windows still shadows the XPU Triton. Those two live in the dependency pass, so a venv that reached +xpu without them, an explicit pin or an update whose first pass ran the pre-XPU setup.ps1, never got them on any later update either. An unreadable version reads as stale. install_python_stack.py writes its completion manifest immediately before returning, so an interrupt between the triton-windows uninstall and the XPU wheel install left a venv with no triton that the next update read as complete. The manifest is now held aside across the swap and restored only once a triton is importable again. * Windows: move the install manifest across the Triton swap instead of rewriting it Two problems with the hold added in 2603fc809, both on the restore side. Reading and rewriting the file cannot survive a manifest carrying a non-ASCII path. Windows PowerShell 5.1 writes Set-Content in the ANSI code page by default, and its -Encoding utf8 emits a BOM that install_manifest.read_manifest's json.load rejects outright ("Unexpected UTF-8 BOM"); Get-Content is ANSI on a BOM-less file too, so the read lost bytes before the write got a chance to. The manifest is now MOVED into the wheel's temp directory and moved back, so no encoding is involved at either end. That directory is already removed in the finally, which is what keeps an unrestored manifest gone. A manifest that would not move left the old valid one in place for the whole destructive window, since the failure only cleared the saved copy and carried on into the uninstall. That is the case the hold exists for, so it now skips the swap entirely and says so: triton-windows keeps shadowing the XPU Triton, which costs torch.compile on the GPU and is repairable on the next run, rather than risking a venv with no Triton that reads as complete. * Windows: confirm the install manifest actually moved before the Triton swap Move-Item across volumes is a copy followed by a delete, and it reports success when only the delete fails, leaving the original exactly where it was. So the guard added in af928dd88 could believe it had set the manifest aside while a valid one sat there for the whole destructive window, which is the case that guard exists to prevent. Found by modelling the manifest in the setup.ps1 scenario matrix, which this had no coverage for: with the parent directory read-only the swap still ran, and the locked scenario passed for the wrong reason. The move is now confirmed by testing the source path afterwards, and a manifest still standing aborts the swap like any other failure to move it. Four new scenarios cover it: the swap keeping a byte-identical manifest, a swap where neither Triton reinstalls correctly leaving it gone, a failed fetch never touching it, and a manifest that cannot move aborting the swap. * Windows: key the XPU fast-path remediation off the installed wheel, not just the GPU scan $HasNvidiaSmi suppresses the Intel scan, so on a mixed NVIDIA + Intel box under an explicit xpu pin $script:IsIntelXpu stays false while the pin still lands the venv on a +xpu wheel. The staleness check added in 2603fc809 was gated on that flag alone, so those hosts kept taking the fast path and never reached the bitsandbytes floor or the Triton replacement. This is the same gating mistake the bitsandbytes pass had in round 4, where the fix was to key off the index leaf rather than the scan. The leaf is not resolved yet at the fast path, but the installed flavor tag is, and whatever put the venv on a +xpu wheel the two remediations still apply. The runtime probe above stays on the scan: reinstalling XPU torch is only right where an Intel GPU was actually found. A pure NVIDIA host on a cu wheel never runs the probe, which the matrix asserts alongside the two new mixed-host rows. * Windows: reconcile Intel names for hybrid GPUs, and stop the XPU escapes firing where XPU is unreachable Five fixes from a review of the XPU work so far. The registry reconciliation was gated on "no ASCII Intel name present", so a hybrid laptop reporting its Intel UHD alongside a localized Arc stopped at the UHD and left the Arc unrecognised. It is now gated on the absence of an XPU match, and the regex behind both that gate and the classification is defined once so they cannot drift. The two fast-path escapes cleared $SkipPythonDeps for any Intel host, but the XPU install and its two remediations are all gated on $XpuIndexUrl, which an explicit cpu / rocm / custom-leaf pin never sets, and no-torch mode has no torch pass at all. Those hosts ran the whole dependency pass, installed nothing new, and re-fired the identical condition on every later update. Both escapes now require XPU to be reachable. The manifest path was learned by a subprocess whose output parsing could not work: `& python` returns one array element per line, interpolating that joins on $OFS, a SPACE, so splitting on newlines yields a single element and a banner ahead of the answer arrives glued to the path. Any such failure then skipped the hold silently and swapped anyway, which is the window the hold exists to close. manifest_path() is venv_root()/MANIFEST_NAME and venv_root() is sys.prefix, which is $VenvDir here, so it is assembled like Get-PersistedNoTorch already does. A test asserts the literal still matches MANIFEST_NAME. The uninstall's exit code was discarded. A triton-windows that will not uninstall, which on Windows means Studio is running and holding libtriton.pyd open, still shadows the XPU Triton, so installing over it achieved nothing and restored the manifest onto a venv this pass was supposed to have changed. The restore had no verification and an empty catch, while the finally deletes the held copy either way, so a failed restore lost the manifest with nothing on screen. * Windows: keep the WMI adapter list an array so the Intel re-label appends instead of concatenating `$_gpuNames = if (...) { @(...) } else { @(...) }` wraps each branch, and a one-element array unrolls on its way out of the if, so on any single-adapter host $_gpuNames was a String. The `+=` that re-labels a localized adapter then concatenated two strings rather than appending a name, and the GPU reported to the user came out doubled: Intel(R) UHD Graphics 620Intel Intel(R) UHD Graphics 620 No install decision changes. The re-label only appends a registry name that already contains the WMI name, so the concatenation matches the Arc / Data Center regex exactly when the registry name alone would, and every scenario in the matrix records the same verdict either way. It is the displayed adapter name that was wrong. Widened by the previous commit: gating on the absence of an XPU match rather than of any Intel name brought ordinary single Intel iGPU hosts into the re-label for the first time. @() now wraps the whole if in both installers, with a test asserting it stays that way. * Windows: give pin-only XPU installs the 2.6 floor, and treat an unreadable dependency probe as stale The XPU install branch required $script:IsIntelXpu as well as an xpu index leaf, so an explicit FAMILY=xpu or URL pin on a host whose Intel scan never ran -- a mixed NVIDIA box, where $HasNvidiaSmi suppresses it -- fell through to the generic branch and its torch>=2.4. Against a mirror carrying an older +xpu wheel that installs a torch unsloth rejects at import. Keyed off the leaf alone now, which is what the bitsandbytes gate below it already does and says in its own comment. install.sh had the same gap from the other direction: its xpu leaf is reachable only by an explicit pin and kept the generic floor, so it gets the same 2.6 trio. The fast-path dependency probe treated "did not answer" as "nothing to do". A timeout, or a malformed .dist-info making distributions() raise, then left the fast path intact and an XPU migration never reached the bitsandbytes floor or the Triton replacement on any later update either. It now clears the fast path, the same direction an unparseable version already took. Two install.ps1 rows move, both FAMILY=xpu pins on non-Intel hosts, both onto the XPU branch. The CPU fallback after a failed XPU install keeps its 2.4 floor. * Windows XPU: probe the preserved venv, drop torchaudio on ARM64, and give POSIX XPU the bitsandbytes floor Three fixes to the Intel XPU paths. install.ps1, migrated-runtime probe: a rerun over an existing install moves the old venv to $script:StudioVenvRollbackDir and creates an empty one in its place, both before this probe runs, so it always asked an interpreter with no torch and answered "no XPU". Ask the preserved environment when there is one, which is the migrated runtime the fallback exists for. install.ps1, Windows on ARM: no win_arm64 torchaudio wheel exists on any index. Keying the XPU branch off the index leaf alone routes an arm64 interpreter into a branch that hardcoded the trio, so the install aborted. Ask the interpreter for its platform tag, as the generic path already does, and drop that one pin on arm64. The CPU fallback below it gets the same treatment. install.sh, XPU pins: bitsandbytes ships XPU kernels (libbitsandbytes_xpu2025.so and _xpu2026.so) from 0.50.0 on manylinux, and nothing on the POSIX side raised the floor for them, so a migrated environment kept a pre-XPU build and lost 4-bit QLoRA on a torch that otherwise works. Matches what the Windows XPU pass already installs. * Studio: stop the xpu label test from forbidding a partial locale check-parity.ts states the contract plainly: "Locale files may be partial; missing keys must fall back to English." The new test required every overlay to carry the xpu label, which contradicts that and breaks on the next locale anyone adds. It already did: it.ts landed on main after this branch, so the merged tree fails on all three runners even though nothing about the label is wrong there. The label is a proper noun, so the English fallback is byte-identical to a translation and the requirement bought nothing. Assert what actually renders wrong instead: en.ts must carry the key, because it is the fallback every locale resolves to, and no overlay may define a value that disagrees with it. Both halves were checked against a merged working tree, and both still fail when the condition they guard is broken. * Linux XPU: hoist the bitsandbytes pass out of the fresh-install arm It sat inside `elif [ -n "$TORCH_INDEX_URL" ]`, which a migrated environment never enters because the `_MIGRATED` arm above it wins, so the one environment the pass existed for was the one that skipped it. The AMD passes handle this by existing twice, once per arm; this gate needs nothing branch-specific, so it moves past the chain instead and both arms reach a single copy. tests/sh/test_xpu_bitsandbytes_reachable.sh guards both halves: the block must be placed where every arm reaches it, and it must still fire only on the xpu leaf. 25 checks over [migrated, fresh] x [xpu, mirrored xpu, cuda, rocm, cpu, none] x [torch, no-torch], run against the block and the leaf parser extracted from install.sh. Moving the block back inside an arm fails it. * Report the XPU runtime before the hardware summary, and show every runtime in About setup.ps1: the hardware report runs ~1300 lines before the torch.xpu.is_available() check that keeps an XPU environment, so a host the WMI scan and the registry fallback both miss (wedged CIM service, an Intel part outside the Arc|Data Center regex) was told "none (chat-only / GGUF)" and then watched setup keep the XPU venv. Ask the same question before printing, so the report and the decision cannot disagree. A free disk read gates the interpreter launch: torch/version.py carries the local label, so a CPU-only host never pays for an `import torch` on every `studio update` just to be told it has no Intel GPU. The dist-info name cannot be used for this -- pip normalises the local label out of it (torch-2.9.1.dist-info for a +cu128 wheel). The promotion carries its own try: it must still run when the scan threw, which is the case it exists for, and a junk UNSLOTH_STUDIO_HOME would otherwise abort setup from Join-Path. about-tab.tsx: hardware.py reads versions["cuda"] off torch.version.cuda and sets versions["xpu"] from an independent torch.xpu.is_available() probe, and UNSLOTH_FORCE_XPU=1 is a supported configuration where CUDA is present but XPU is selected. Both are non-null there, so returning the first match hid the XPU row on exactly the host it was added for. Collect every reported runtime instead. tests/studio/test_setup_xpu_runtime_prereport.ps1 covers the two new helpers with the filesystem mocked, so it runs on all three runners: override precedence, ~ expansion, the four wheel flavours, a missing or unreadable version.py, and wiring assertions that the promotion precedes the report and that the cheap read gates the probe. The About-tab test gains a case that fails if the picker returns early again. * POSIX: recognise a working XPU runtime, and raise the bitsandbytes floor on the update path The hardware summary tested NVIDIA, AMD and Apple Silicon and then fell through to "none (chat-only / GGUF)", so a Linux host running the +xpu wheel install.sh had just installed was told training needs an NVIDIA or AMD GPU. Added an arm ranked below both, matching setup.ps1. The bitsandbytes floor was also unreachable on the route an existing XPU user actually takes. `unsloth studio update` runs this file, never install.sh (see the note at the top of setup.sh), and neither this file nor install_python_stack.py had an XPU floor, while unsloth's own dep floor is 0.45.5 -- which a pre-XPU wheel satisfies indefinitely. So 4-bit QLoRA stayed unavailable on a torch that otherwise works. One detection serves both, but they read different signals on purpose. The floor keys on the WHEEL (+xpu, read off torch/version.py) and the summary keys on the RUNTIME (torch.xpu.is_available()): a +xpu wheel installs fine on a host whose driver never initialises, and that host should still get the kernels while no GPU is claimed for it. The disk read gates the interpreter launch, so a CPU-only host pays nothing per update. tests/sh/test_setup_xpu_posix_summary.sh builds real venv trees, version.py files and stub interpreters rather than mocking, so the disk read and the runtime probe genuinely execute: 13 checks over the four wheel flavours, working/dead/missing runtime, no venv, and the arm's rank. Removing the arm fails four of them. * POSIX XPU: make the bitsandbytes step nonfatal, bound the probe, and act on an XPU pin Three defects in the POSIX XPU code from the previous commit. run_quiet routes failure to setup_fail and exits, so the best-effort bitsandbytes upgrade could abort an otherwise fine `studio update` over a transient download, and the warning after it was unreachable. run_quiet_no_exit is the nonfatal wrapper. The runtime probe had no timeout. A stalled Intel driver wedges inside `import torch`, which is exactly the host this probe classifies, so it could hang every update forever. Bounded at 60s rather than the 10s the smi probes use: a cold `import torch` takes seconds by itself and a short bound would read a healthy host as having no GPU. Systems without coreutils timeout keep the previous behaviour rather than losing detection. An explicit XPU pin was protected but never acted on. An xpu leaf names no family the cuda/rocm repair helpers know, so _explicit_unknown_family_torch_index_url makes both skip it, and `unsloth studio update` never runs install.sh -- so switching a CPU install to UNSLOTH_TORCH_INDEX_FAMILY=xpu left the CPU wheel in place indefinitely. The fix goes in install_python_stack.py, which already parses the pin, rather than setup.sh, which has no pin awareness at all: _ensure_xpu_torch mirrors the existing _ensure_cpu_torch, the xpu leaf is classified so the backend is no longer unknown, and the ROCm helper skips an xpu backend so it cannot treat the pin as an AMD host. Windows is excluded because setup.ps1 owns torch there and installs the trio itself. That put the XPU trio in a third file, so tests/sh/test_xpu_torch_spec_parity.sh asserts the floors match across install.sh, install_python_stack.py and install.ps1 plus the wiring. Each of its four structural guards was mutation-tested: a drifted floor, a lost classification, wiring at one call site instead of two, and the ROCm skip removed all fail it. The POSIX summary suite gains checks for the nonfatal wrapper and the bound. * Linux XPU: swap generic Triton, gate the pin repair on the version, and escape the fast path Three defects in the XPU code from the previous commit. _ensure_xpu_torch returned on the +xpu tag alone, so a migrated 2.5+xpu venv was left in place even though unsloth/models/_utils.py raises at import for an XPU device below 2.6. It now returns only when the flavour and the supported range both match. That repair was also unreachable on the route it was written for. setup.sh skips install_python_stack entirely when the package version is current, and that pass is the only thing that acts on an XPU pin, so a CPU install switched to the xpu family stayed CPU. Added a third fast-path escape beside the anyio and incomplete-manifest ones. Generic triton and torch's pytorch-triton-xpu / triton-xpu both own the top-level triton package, and resolving unsloth against a pinned +xpu torch pulls both -- uv reports pytorch-triton-xpu 3.5.0 alongside triton 3.7.1 -- so the CUDA-oriented build lands last and torch.compile loads the wrong library on an Intel GPU. This is the POSIX half of the Windows swap: the spec is read from torch's own metadata, so the pytorch-triton-xpu to triton-xpu rename at torch 2.10 needs no hardcoding, and the fetch happens before the uninstall because the uninstall drops the shared paths from generic triton's own record. test_torch_installs_do_not_use_deprecated_index_url forbade --index-url on "$TORCH_INDEX_URL" anywhere in install.sh. That rule is about uv, which deprecated the flag in favour of --default-index; pip never had --default-index, so the pre-fetch legitimately uses it. The assertion is now per occurrence and exempts pip download only, and it joins backslash continuations first, since the flag and its command are routinely on different physical lines. Both a same-line and a continuation-line uv offender were mutation-tested and are still caught. tests/sh/test_xpu_triton_swap_posix.sh asserts the swap by execution -- ordering, the rename, no generic triton, torch wanting CUDA triton, non-xpu index, no-torch, empty index, and a dead mirror that must warn without removing anything. * XPU: move the Triton swap where both routes reach it, and bootstrap pip for it Five defects in the XPU code from the previous commits. The Triton pre-fetch could never have run. `uv venv` is created without --seed, so a fresh venv has no pip and `python -m pip download` fails with "No module named pip" every time, leaving the swap a no-op that only ever warns. My shell test missed it because its stub interpreter answered pip commands. install.sh already bootstraps pip this way before its pre-release bitsandbytes wheel. The swap also never ran on `unsloth studio update`, which runs setup.sh and never install.sh. Both fixes fall out of moving it: install.sh runs setup.sh, which runs install_python_stack.py, so that module is the one place both routes pass through. The install.sh copy is deleted rather than duplicated, and the shell test is replaced by tests/studio/test_xpu_triton_swap.py, which covers the no-pip case and asserts install.sh carries no second copy. The fast-path pin match missed authenticated and fragmented mirrors (https://mirror/whl/xpu?token=...), which read as "no XPU pin" and skipped the repair; query and fragment are now stripped before the leaf test. That escape also launched an interpreter, which a wedged Intel driver hangs inside. It now reads the local label out of torch/version.py instead: nothing to bound, and a CPU-only host pays nothing per update. setup.ps1's fast path asked only whether XPU was available. A 2.5+xpu build answers yes and is still rejected by unsloth/models/_utils.py at import, so it now checks the supported range too, via Test-TorchXpuVersionSupported. The POSIX suite is up to 22 checks; the three new guards were mutation-tested by removing the query strip, the fragment strip, and by making the escape launch an interpreter. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * XPU: make a failed Triton swap unsurvivable, and widen the fast-path escapes Five defects in the XPU code from the last two commits. The Triton uninstall ignored its return code. A read-only or locked venv leaves generic triton registered, so installing over it lets a later upgrade of that distribution delete the shared files again, and every dependency pass repeats the swap. A failed uninstall now changes nothing at all. Past the uninstall the venv has no triton, because the uninstall takes the shared top-level files with it, so a warning there let the caller write a completion manifest over a venv whose torch.compile is broken -- and the next update fast-paths straight past it, since no generic distribution is left to trigger on. That install is now fatal. _ensure_xpu_torch returned when the probe timed out. On this path a wedged `import torch` is evidence rather than noise: the usual cause is a stalled Intel driver under an unsupported +xpu wheel, which the resolver keeps because it satisfies the base range. An authoritative pin now repairs on an inconclusive probe. This deliberately differs from the CPU counterpart, where a wedge has no such likely cause. The fast-path pin match stripped one trailing slash, so a ".../whl/xpu//" pin still read as no pin. It now strips them all, like the shared leaf parsers. Moving the Triton swap into the Python stack left the fast path with no reason to run it: a migrated environment with supported +xpu torch and a leftover generic triton kept the CUDA-oriented build forever. A stale generic triton now forces the dependency pass too, detected from the dist-info name so no interpreter is launched. The POSIX suite is up to 26 checks and the Triton tests to 16. Two of the guards were rebuilt after their own negative controls found them vacuous: the stale-triton check matched the detection loop rather than the branch that acts on it, and a fixed line window had drifted off the code it was meant to cover, so it is now anchored on the block. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not require the XPU pin again after install Three Intel paths still assumed the pin was still in the environment, or that every XPU host looks like x64 Linux. install_python_stack.py: the generic-Triton swap returned unless UNSLOTH_TORCH_INDEX_URL / _FAMILY was set. That pin is one-shot -- a user who ran UNSLOTH_TORCH_INDEX_FAMILY=xpu ./install.sh has nothing left in the environment by the next plain `unsloth studio update`, yet that update's dependency pass can pull generic triton back in and shadow torch's XPU build again. The installed +xpu wheel is the durable signal (setup.sh already raises the bitsandbytes floor off it), so fall back to it and to the default xpu index. The label is read off disk: importlib.metadata drops the local version label, and `import torch` loads the SYCL runtime, which can wedge. install.ps1: the flavor repair built its own XPU trio including torchaudio, which has no win_arm64 wheel on any index. A migrated ARM64 venv skips the fresh XPU branch and takes this path, so the repair failed outright before setup.ps1 could reach its ARM-aware fallback. One builder now serves both sites, since the two copies drifted the moment only one learned about ARM. install.sh: adding the xpu tag made the final flavor guard reachable on an Intel pin, and it probes with an unbounded `import torch`. On a host whose driver initialization wedges that hangs the installer, with no timeout anywhere before setup.sh's bounded probes. The xpu path reads torch/version.py off disk instead; every other family keeps the interpreter read unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Key the setup XPU paths on the installed wheel, not the pin Follow-up to d07c179f: install_python_stack now treats the installed +xpu wheel as the durable signal, but the two callers upstream of it did not. setup.sh fast path: the escape only ran under `case $_setup_pin in *xpu`, so after a one-shot UNSLOTH_TORCH_INDEX_FAMILY=xpu install every later `studio update` saw no pin, kept _SKIP_PYTHON_DEPS=true and never reached the Triton swap at all -- generic triton kept shadowing the XPU build forever. The disk read now happens unconditionally and the swap escape keys on the wheel. The pin leaf is also compared exactly, like the shared index parsers: a custom mirror ending in -xpu was classified as the curated family, which cleared the skip flag on every up-to-date run while _ensure_xpu_torch declined to act. setup.ps1: bounding the flavour probe turned a timeout into "rebuild", and the host most likely to time out inside `import torch` is an Arc box whose compute driver stalled -- where torch/version.py still names a good +xpu wheel. With no currently exported pin the stale path then deleted the venv. It now falls back to the same disk check and warns about the driver. Other families still rebuild on an unreadable flavour. setup.sh summary: a +xpu wheel whose runtime will not initialise fell through to "none (chat-only / GGUF)", telling an Arc owner their hardware is unsupported and hiding the driver update that fixes it. It gets its own arm. * Stop the XPU paths from stranding or wiping a venv Four ways the Intel paths could still leave a user worse off than before they ran anything. setup.ps1 stale check: on a hybrid NVIDIA + Arc host the XPU promotion is gated on -not $HasNvidiaSmi, so a pinless `unsloth studio update` expects a cu* tag, calls the working Arc venv stale and DELETES it -- then exits, because only install.ps1 creates venvs. A direct update now keeps any +xpu venv and says to re-run install.ps1, which rebuilds with a rollback copy. setup.ps1 Triton swap: when the staged XPU wheel failed to install after triton-windows was removed AND the generic restore also failed, the branch only printed. $stackExit stayed 0, so setup reported success and install.ps1 committed a venv with no importable triton over its rollback. It now carries the real failure code into the existing handler. install_python_stack: the `pip download` that stages the XPU Triton wheel inherited the user's pip index environment. PIP_NO_INDEX makes pip ignore --index-url outright, and PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS are consulted in addition to it, so the fetch could fail (leaving generic Triton shadowing the XPU build) or serve the wheel from an index the pin never named. It now takes the same _install_env_for_cmd scrub every other pinned install gets. setup.sh runtime probe: the arm taken when coreutils `timeout` is absent ran the probe with no deadline, on exactly the stalled-driver host the bounding exists for. The deadline now lives inside the probe as signal.alarm, which terminates the process even while the driver blocks in C. * Keep a preserved XPU venv on the XPU index Follow-up to 10ba6e31c, which stopped a direct update wiping a +xpu venv on a hybrid NVIDIA + Arc host but left the rest of the pass believing the host was CUDA. The index chain prefers NVIDIA over Intel, and the CUDA arm does not --reinstall-package torch, so uv left the +xpu wheel in place as satisfied while installing triton-windows over torch's XPU triton -- and with $XpuIndexUrl null nothing swapped it back. A half-converted venv is worse than either end state, so the preserved case now selects the xpu leaf, ahead of the NVIDIA arm and behind an explicit pin. The hardware report is untouched: there really is an NVIDIA GPU in the machine. install_python_stack: an inconclusive XPU probe was always read as a flavour mismatch, but on a stalled Intel driver under a SUPPORTED wheel that is two 90-second hangs and two force-reinstalls of the whole trio on every update, repairing nothing. The disk answers what the probe cannot, so a supported wheel now yields the driver warning and an unsupported or missing one still repairs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Lowercase the setup.sh pin leaf like every other index parser install.sh's _torch_index_url_leaf, setup.ps1's Get-TorchIndexLeaf and install_python_stack's _torch_index_leaf all lowercase before classifying. This copy did not, so UNSLOTH_TORCH_INDEX_FAMILY=XPU (or a URL ending in /XPU) left the leaf uppercase, the equality test against "xpu" failed, and the fast path stayed on. Those same classifiers call that pin XPU once they are reached, so the wheel was never migrated and the update silently repaired nothing. The comment above the line already claimed to match the shared parsers; now it does. Three cases added to tests/sh/test_setup_xpu_fastpath_escape.sh (FAMILY=XPU, FAMILY=Xpu, a URL ending /XPU), plus one that lowercasing must not widen the match: a custom leaf like PRIVATE-XPU stays an unknown family. All three fail against the previous line and pass now. * Do not promise CPU training when the XPU runtime will not start The unavailable-runtime arm said training and GPU inference run on CPU until the driver is fixed. They do not: with neither CUDA nor XPU available, get_device_type() in unsloth/device_type.py raises NotImplementedError, so importing unsloth fails outright rather than falling back. llama.cpp is unaffected, which is what chat and GGUF actually run on, so say that instead. The drift guard added with it needed two passes to be worth anything. Anchoring the arm on the flag name alone matched the bitsandbytes block instead, whose own "4-bit QLoRA may be unavailable" warning made both assertions pass on any wording; and the arm's explanatory comment quotes the phrase it must not use, so comment lines have to go before the grep. Restoring the old message now fails both checks. * Let an explicit non-XPU pin migrate off an XPU wheel Two halves of the same gap: asking for CUDA/ROCm/CPU on a host already running +xpu did nothing. setup.sh: the fast-path escape fired only when the pin itself was xpu, or when a stale generic triton shadowed the build. With an up-to-date install, a +xpu wheel and the pin switched to another family, neither arm matched, install_python_stack never ran, and the authoritative pin was ignored. Added an arm for that case, digit-gated like the shared classifiers so a custom verbatim leaf (rocm-current, cu-private) stays UNKNOWN and does not force a pass that repairs nothing. install_python_stack: _ensure_cpu_torch classifies the installed build and returns early on "already a CPU build". Its probe tested hip, rocm, cuda and +cu<digits>; an XPU wheel sets neither torch.version.cuda nor .hip, so it read as CPU and an explicit CPU pin over it did nothing at all. Keyed on the +xpu local label, since torch.version.xpu is None on some builds. Additive: +cu128 and +rocm still read gpu, +cpu and untagged still read cpu. The escape suite's extractor stopped after the second _SKIP_PYTHON_DEPS assignment, so adding a third arm truncated the block and the new cases failed while the old ones passed. It now stops at the next outer arm and asserts exactly three arms extract, so a future arm fails loudly instead of disappearing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop a wedged Intel driver from blocking the paths that repair it Both of these are fallout from making the CPU repair XPU-aware: it now has to classify an XPU wheel, and every route to that classification went through `import torch`, which loads the SYCL runtime and blocks on the exact host these paths exist to rescue. install_python_stack: the classifier probe times out after 90s and the except branch returned, so an explicit CPU pin over a wedged +xpu venv stayed a no-op. Classify off disk on timeout via _installed_torch_label_on_disk (find_spec, no interpreter) and fall through to the repair. Gated on a GPU label so a slow but healthy CPU-only host does not force-reinstall torch every update. install.sh: the rollback preservation probe read torch.__version__ through the interpreter at venv-replacement time, ahead of every bounded probe in setup.sh, so a hang there took the whole installer with it. It now reads torch/version.py, the same source _installed_torch_version_for_tag already uses for this reason. The interpreter stays as the fallback for a layout without one, where torch is absent and the import fails fast. The install.sh test executes the block against a fake venv whose stub interpreter records being called, so "read off disk" is proven by the interpreter never running rather than by reading the source. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match torch index families exactly, and judge the XPU fast path on the wheel studio/setup.sh classified a pin as a known non-XPU family with prefix globs (cu[0-9]*, rocm[0-9]*), so cu128-private, cu128rc1, cu128.1, rocm7.2-private, rocm7. and rocm7.2.1 all read as known while install_python_stack calls every one of them UNKNOWN and runs no repair: the fast path was cleared and the dependency pass that followed applied nothing, every update. It now matches exact families like install.sh _is_pip_rocm_family_leaf and install_python_stack _is_cuda_family_leaf: cpu, cu<digits>, rocm<digits>[.<digits>], gfx<digit>... (gfx stays a prefix on all three sides, since gfx120x-all is a real Radeon index leaf). studio/setup.ps1 keyed the same escape on torch.xpu.is_available(), which is also false for a supported +xpu wheel on an old or wedged compute driver. No dependency pass can repair a driver, and the pass force-reinstalls nothing when the flavour already matches, so each studio update repeated the bounded probes and a full resolution just to reach the warning Assert-XpuRuntimeReady already prints. The escape now asks Test-VenvTorchIsXpuSupported, which reads torch/version.py off disk and applies the same 2.6 <= v < 2.11 window, matching what setup.sh does on POSIX and removing the last import torch from a path an Arc host with a stalled driver is most likely to hit. Its only caller gone, Test-TorchXpuVersionSupported is removed. Tests: the escape test now also asks install_python_stack itself about a 28-leaf corpus and asserts the shell predicate agrees leaf for leaf, so the two cannot drift again (75 checks; 11 fail against the previous globs). The pre-report test covers the new helper and asserts the fast-path escape names no readiness probe and launches no interpreter. * Trim comments across the Intel XPU detection changes * Normalise setup.ps1 line endings before the wiring regexes A Windows checkout returns CRLF, so the fast-path escape pattern, which is anchored on a literal \n, matched nothing on windows-latest: the region came back empty, "the escape was found" failed, and the two -not checks inside it reported PASS with nothing to look at. Cross-platform parity caught it on windows-latest with 3 failures. $setupText is now normalised to LF once at the read, which covers both literal newline patterns in the file, and a new check asserts the raw CRLF form does NOT match the same pattern, so it is the normalisation rather than luck that makes this work. Verified against a CRLF copy of setup.ps1: the previous test fails there with exactly those 3 checks and the new one passes. * Run the Triton swap after every torch migration, not between two of them _ensure_xpu_triton keys off the installed +xpu label when no explicit XPU pin is set, and it ran ahead of _ensure_cpu_torch. So an existing +xpu venv updated with an explicit CPU pin had generic triton removed and XPU triton installed, and only then did _ensure_cpu_torch replace torch with the CPU build: a CPU environment whose top-level triton package is the XPU implementation, with the generic triton its own dependency set declares now gone. The CUDA and ROCm repairs already ran ahead of the swap, so their pins left the label correct by the time it read it; CPU was the one migration that did not. Moving the swap to the end of both repair blocks fixes it for every family at once and removes the ordering assumption entirely. The new test asserts the order on the AST at both call sites, so a reflow cannot fake it; against the previous order it fails on the first assertion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bound the wedged-driver probe without GNU timeout macOS ships no GNU timeout (Homebrew coreutils installs it as gtimeout), so on the macOS parity leg the `timeout 30 python3 ...` line exited 127 the instant it was called. The test reads only the exit code, and 127 is non-zero with an elapsed time of 0, so both assertions passed without python ever starting: the alarm behaviour they exist to prove was never exercised on macOS. Replaced with the script's own background watchdog, which behaves the same on every platform, and added a lower bound on the elapsed time. The alarm is 2s, so a run that returns instantly did not execute the probe, which is precisely how the missing-timeout case looked. Verified by shimming `timeout` to exit 127: the previous test still reports 37 passed, and by shimming python3 to return instantly: the previous test still reports 37 passed while this one fails on the deadline check. * Trim comments in the Intel XPU detection changes --------- Co-authored-by: CommandCodeBot <noreply@commandcode.ai> Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-04 03:03:10 -07:00
Daniel Han	fb8a992449	Anchor the host-defaults assertions and take the file off the skip list (#7774) ... * Anchor the host-defaults assertions and take the file off the skip list test_install_host_defaults.sh has been quarantined since #7431 and failing for longer, with a recorded reason that covers one of three drifts. None of them is a regression: install.sh, install.ps1 and the README all still behave as the test intends. Every failure comes from how the test carves out the text it greps. Both installers were sliced with a fixed tail. install.sh grew past its 50-line window in #6258 and install.ps1 past its 25-line window in #7be1085, the day after the test landed, so the prompt each assertion looks for sat just above the window. Both now start at the block's own 'In interactive terminals' comment. The README extractor stopped at a '#### Update' heading that was later deleted, so its Launch section silently became the rest of the file and picked up the -H 0.0.0.0 from the remote-access section 137 lines below. It now stops at the next heading of any name. Two assertions could not have failed. setup.sh's launch hint sits at line 2332 while tail -30 covered 2339 onward, so the negative check no longer looked at the hint at all, and 'read' as a needle also matches 'readable' and '_can_read_tty', so the prompt check passed with the prompt deleted. Both are pinned to what they mean to protect; each of the six assertions now fails when its target is removed. With the extraction fixed the file passes 10/10, so it comes off the skip list in run_all.sh, studio-backend-ci.yml and test_ci_shell_suite_coverage.py, which have to change together. * Give the setup.sh window a canary and stop at any heading level The setup.sh launch-hint extraction was the only one of the four with no positive assertion over it. The other three each pin something that must be present, so a broken extraction fails loudly; this one had a lone assert_not_contains, and a negative check over an empty window passes. Renaming the printf label empties it, and the test then stays green with 0.0.0.0 hard-coded in the hint, which is the failure this PR exists to remove. The README extractor's comment promised the next heading of any name while the pattern only matched an h4, so demoting the following heading would widen the window again. * Tighten the comments on the anchored host-defaults assertions	2026-08-03 02:42:26 -07:00
Daniel Han	ef08ee999e	Linux/macOS: tell an unreadable llama.cpp install apart from someone else's (#7771) ... * Linux/macOS: tell an unreadable llama.cpp install apart from someone else's setup.ps1 learned this in #7735 and #7757; setup.sh never did. A tree that is ours, carrying our own marker, reads as somebody else's the moment it stops being searchable, because every probe inside it reports the marker absent. The ownership guard then said the folder is not an Unsloth-owned install and told the user to move it aside or pick an empty UNSLOTH_STUDIO_HOME, when the actual fix is a permission change. Search (+x) is what those probes need, not read (+r): a directory can be readable and unsearchable (mode 444) or searchable and unreadable (mode 111), so the new probe tests search. The two destructive replaces also ran blind. Under errexit a failing rm -rf aborts on a raw permission error with no [TAURI:ERROR], so the desktop app shows a bare exit code and the freshly built llama.cpp is left in the temp directory with nothing saying where. Both sites now tolerate the failed remove, check that the directory is actually gone, and report either the permission problem or where the new build was left, exiting 3 as the Windows side does for a build that succeeded but could not be installed. Neither message tells the user to delete a folder Unsloth cannot prove is its own, matching the rule the Windows guard follows. * Keep the rm error when a replace fails The redirect this PR added swallowed the one line that names which subtree refused to go. rm reports the deepest failing path, llama.cpp/build say, while the message below it can only name the install root, so a user whose build directory alone is locked lost the part that told them where to look. || true is what keeps errexit from aborting; the redirect was never doing that job, so stderr stays. * Tighten the comments on the denied-install-tree guards	2026-08-03 00:58:23 -07:00
Michael Han	ebfefcf84e	Windows: do not abort setup on an unreadable llama.cpp install (#7735) ... * Windows: do not abort setup on an unreadable llama.cpp install Test-Path raises UnauthorizedAccessException instead of returning $false when an ACL denies the probe. setup.ps1 runs under $ErrorActionPreference = "Stop", so the bare probe of UNSLOTH_PREBUILT_INFO.json in the llama.cpp prebuilt phase killed setup with a raw "Test-Path : Access is denied" and exit code 1. The desktop app had nothing but [TAURI:ERROR_DEFAULT] to fall back on, so it showed "unsloth studio setup failed (exit code 1)". ~/.unsloth/llama.cpp sits beside the app, not inside it, so reinstalling reused the unreadable folder and hit the same line again, including a reinstall to a different drive. Add Get-PathState (Present / Absent / Denied) plus Test-PathQuiet, route the probes that read inside install trees we do not own through them, and report a denied llama.cpp install through Exit-SetupFailure so the reason and the recovery steps reach the desktop UI. Reported in unsloth-test/unsloth-test#9 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop on every denied path, and split the recovery commands Review follow-ups: - Assert-StudioOwnedOrAbsent treated a denied root as absent and returned, so the caller could go on to replace a tree it cannot read. Probe the root three-state and stop on Denied, still gated on $StudioHomeIsCustom so default-home behaviour is unchanged. - The source-build .git probe treated a denied checkout as "no checkout" and cloned a replacement. The swap that follows recursively removes the original and moves the temp tree over it under "Continue" and unchecked, so a denied child could leave a half-deleted install. Stop instead. This path already treated denied as absent before the previous commit (that probe runs under "Continue", so it printed an error and took the false branch), so the hazard is older than this branch, but it is in scope for the same reason. - Probe $LlamaCppDir itself three-state, so an unreadable parent is reported rather than dying on the bare probe under "Stop". - takeown and icacls were printed joined by "then", which is not a PowerShell separator: takeown would swallow the rest as arguments and icacls would never run. Print them on separate lines. Fold the repeated guidance into Exit-PathAccessDenied so all five denial routes report the same thing. * Harden the denial reporting path, found by simulation Ran the real decision blocks against simulated filesystems (denied file, denied parent, traverse-only and list-only dirs, symlinks, dangling links, wildcard and unicode paths, 3000 random paths) plus PSScriptAnalyzer's 5.1/6.2/7.0 syntax check. Two things came out of it: - Get-PathDenialDetail threw a parameter-binding exception on an empty path. It runs while a failure is being reported, so it would have replaced the actionable message with a raw binding error at exactly the wrong moment. Null and empty are now accepted and return no detail. - The link-target lookup used an empty catch, which PSScriptAnalyzer flags and which hid the intent. It assigns $null explicitly now. Both are covered by new checks. Also promoted the strongest invariant from the simulation into the suite: Get-PathState must agree with a bare Test-Path on every probe that did not throw, and Denied may only appear where the old probe threw, so no path that worked before can take a different branch now. Verified: PSUseCompatibleSyntax reports nothing for 5.1, 6.2 and 7.0; the Python contract tests pass on 3.10 through 3.13 in separate uv venvs; the tauri install:: unit tests pass (17), which is the code that prefers the [TAURI:ERROR] line over the generic exit-code message. * Trigger the Windows PowerShell tests when they change studio-windows-inference-smoke.yml runs six PowerShell unit tests out of tests/studio, but its pull_request paths filter matched none of them, and no other workflow runs them. A PR touching only one of those tests never ran it. Five predate this branch; the sixth is the ACL test added here. Scope the filter to tests/studio/*.ps1 rather than tests/studio/**, so a python-only change under that directory does not pull in the GGUF smoke jobs. This matches what the other two workflows already do: parity-ci lists its .ps1 test outright and update-smoke uses a scoped glob. Guard it in test_ci_shell_suite_coverage.py, which exists for this exact failure (tests/sh had the same hole): every tests/*.ps1 a workflow invokes must be matched by that workflow's paths filter, and must exist. The GitHub glob matcher it needs has its own table-driven test, since a wrong matcher would make the guard pass on everything. Verified by reverting the one-line filter change: the guard then names all six unrun tests. * Make the Windows PowerShell test step fail when a test fails Verifying the path-filter fix turned up a second hole in the same step. A `shell: pwsh` step inherits only the LAST command's exit code, and this step ran five tests as five bare commands, so only the last one could fail the build. test_resolve_cuda_toolkit.ps1 has been printing FAIL exits non-zero (scenario 2, forced source build) FAIL exits non-zero (scenario 6, no toolkit, forced) 2 check(s) FAILED on every Windows run, exiting 1, and the job reported success. Confirmed on main (run 30723608191, sha c67410a7), so it predates this branch, and it reproduces locally. The cause is in the test, not the installer. Resolve-CudaToolkit -RequireOrExit leaves through Exit-SetupFailure, which the child harness never stubbed, so under ErrorActionPreference=Continue the call was an ignored command-not-found, the child fell through and exited 0. The harness already injects the real Resolve-CudaToolkit and Write-CudaDriverToolkitMismatch by AST, so inject the real Exit-SetupFailure the same way. That test now passes 25/25. With it green, add the exit-code checks after each invocation, matching what studio-windows-update-smoke.yml already does, and guard the pattern in test_ci_shell_suite_coverage.py: any step running more than one PowerShell test must check $LASTEXITCODE after each. Verified by reverting each piece: dropping one check makes the guard name that test, and dropping the Exit-SetupFailure injection brings both scenario failures straight back. * Stop on a denied --with-llama-cpp-dir instead of reinstalling over it When UNSLOTH_LOCAL_LLAMA_CPP_DIR points at the canonical $LlamaCppDir and the llama-server.exe there is ACL-denied, Test-PathQuiet collapsed Denied to $false, $LocalLlamaServerFound stayed false, and the canonical branch reported "nothing built there yet; running the normal install". The prebuilt installer then moves that tree aside and replaces it, which is exactly what the branch's own comment says it exists to prevent. The old bare probe stopped first, by throwing under "Stop". Probe the candidates three-state and stop on Denied. Same for the directory probe itself, which reported an unreadable dir as "does not exist" and sent the user after the wrong problem. The generic message could not be reused as-is here: it tells the user to delete the folder because Unsloth reinstalls it, which is true of the managed cache and wrong for a build they pointed us at. Exit-PathAccessDenied takes -UserSupplied, which swaps that advice for restoring access or repointing UNSLOTH_LOCAL_LLAMA_CPP_DIR, and keeps the takeown/icacls lines. Verified by driving the real block through every state: a readable build is still reused, a genuinely empty canonical dir still falls through to the normal install, a missing dir still reports "does not exist", and a denied build now stops with exit 1 instead of being replaced. Reverting the probe puts the fall-through back, and the user-supplied path never prints "delete or rename" or "managed cache". * Carry the denial through three more probes Three review points, all reproduced before fixing: - The canonical --with-llama-cpp-dir override still got the managed advice ("delete it, Unsloth reinstalls it"). The override means the user asked to reuse whatever is in that tree, so deleting it is wrong wherever it sits. Both candidate denials now pass -UserSupplied, which collapses the branch to one call. - Phase 1b's git prerequisite scan probes the same candidate binaries with a bare Test-Path under "Stop", thousands of lines before the Phase 4 guards, so a denied override terminated the run with the raw error this change exists to replace. Reproduced, then guarded. - Test-StudioOwnedAdoptable collapsed a denied prebuilt marker to $false, so Assert-StudioOwnedOrAbsent called an Unsloth tree an unrelated directory and told the user to move it aside. Get-StudioAdoptableState returns Yes/No/Denied and the guard reports the denial first; Test-StudioOwnedAdoptable stays as the boolean view for the cosmetic cleanup gate. A denied file under a readable directory is a Windows-ACL-only state: POSIX keeps a mode-000 file stat-able, and a symlink into a denied directory still answers Test-Path. The local run injects that one state at the lowest seam and lets the real functions run; the Windows leg of test_path_probe_access_denied.ps1 builds it for real with icacls and skips elsewhere with the reason. test_setup_ps1_adopts_existing_whisper_prebuilt_marker sliced between two function names and the marker scan moved, so its anchor now points at Get-StudioAdoptableState. Its assertion is unchanged. Reverting each fix individually puts the original behaviour back: the ownership misdiagnosis, the raw "Access to the path ... is denied" from Phase 1b, and the delete-your-own-build advice. * Close the remaining denial gaps and the stale probe anchors for PR #7735 - tests/sh/test_with_llama_cpp_dir_flag.sh anchored the literal 'if ($ResolvedLocal -eq $LlamaCppDir) {', which 2a61343 hoisted into $LocalIsCanonical. Re-pin it to the comparison, not the branch. - The junction path deleted and replaced $LlamaCppDir behind a bare Test-Path, so a denied destination still threw raw under a default home. Probe it three-state, and treat Denied as surviving removal. - Get-Content on the prebuilt metadata still globbed while the probes gating it went literal, so a path holding [ or ] passed the probe and threw into the catch. Make both reads literal, with a test. - Get-PathDenialDetail could throw on a non-filesystem provider item whose .Attributes has no -band overload, replacing the failure being reported. - Win32Exception keeps E_FAIL in HResult and the code in NativeErrorCode, so the HRESULT check never matched it. Fix the comment and the check. - test_windows_git_gate.py ran the layout scan in a child that never defined Get-PathState, so de486ff had it silently report nothing built. Inject the real helpers, as test_resolve_cuda_toolkit.ps1 does. - Relax the exact Exit-PathAccessDenied count to a floor and key the -UserSupplied rule on the path reported rather than on position. * Stop advising deletion of a tree whose ownership cannot be read The ownership guard stops precisely because it could not read the marker, so it cannot claim the folder is ours either. It was still emitting the managed-cache text, telling the user to delete it. Eleven lines below, the readable-but-unowned branch says 'move it aside' instead, so we were being gentler when we could prove the tree was not ours than when we could not read it at all. New -OwnershipUnverified wording for those three stops. Also from re-reading the previous commit: - The reparse-point unlink above the junction path ran before the new three-state probe, and a link reports Present, so the probe could not cover it. A denied unlink still terminated on the raw .Delete() throw. - That junction destination probe had no test at all; reverting it left the suite green, since the count floor cannot see a swap. Pinned by name like every other route. - The re-pinned shell anchor accepted an assignment that nothing consumed. Pin the branch that uses it too. - Get-PathDenialDetail now checks the item type rather than the attribute type, which also covers a provider item with no Attributes at all. * Check both destructive steps of the temp-dir swap The guard above the clone path probes only .git, but its own comment names any unreadable child as the risk. A forced source build over a non-git install with a denied child elsewhere reads Absent, clones into a temp dir, and reaches the swap. Both steps there are non-terminating under Continue and neither was checked. Reproduced: Remove-Item partially fails, the original survives, and Move-Item then moves the temp dir INSIDE it, so the new binary lands at llama.cpp/llama.cpp.build.<pid>/llama-server.exe while $LlamaServerBin points at llama.cpp/build/bin/Release. Setup carries on reporting success with no usable server and a half-deleted install. Check the removal before moving, so the stop happens while the temp build is still whole, and check the move afterwards. Denied routes through Exit-PathAccessDenied; anything else surviving exits 3 like the other blocked-replacement paths. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-02 07:25:39 -07:00
Daniel Han	2d61c3aa04	Studio setup: report a missing llama.cpp in Tauri mode instead of aborting the install (#7720) ... * Studio setup: report a missing llama.cpp in Tauri mode instead of aborting the install setup.sh ends by exiting non-zero when the llama.cpp prebuilt did not produce a usable server and it was called from install.sh, so the installer can report the GGUF failure after finishing PATH and shortcut setup. In Tauri mode that non-zero exit is not "report", it is "abort": install.rs turns it into "Installation failed". So one transient prebuilt download failure, and a single HTTP 403 rate limit is enough, fails the entire first-launch install of the desktop app, moments after setup.sh's own footer said Installed. Everything except GGUF inference is in fact working, and whisper.cpp in this same script already degrades rather than failing for exactly this case. Tauri mode now emits a [TAURI:STEP] line naming what is missing and how to get it back. Every other caller keeps the non-zero exit unchanged. * Use the progress channel for the notice, and degrade on Windows too Two problems with the first version, both found in review. The notice used [TAURI:STEP], which is the wrong channel twice over. install.rs maps it to install-step, and use-tauri-backend.ts counts those without ever storing the payload, so the text was thrown away. Worse, install.sh already emits exactly seven unique STEP markers against the seven-entry INSTALL_STEPS list, so an eighth pushed the counter past the end and rendered 'Step 8 of 7'. [TAURI:PROGRESS] becomes install-progress-detail, which the installing screen renders verbatim. Windows had the identical defect and was not covered. install.ps1 sets both SKIP_STUDIO_BASE=1 and UNSLOTH_TAURI_MODE, then turns any non-zero setup.ps1 status into Exit-InstallFailure, which install.rs reports as 'Installation failed'. setup.ps1 now gates on UNSLOTH_TAURI_MODE the same way Exit-SetupFailure already does, and still fails for every other caller. Tests: 8 cases, adding one that the notice never emits [TAURI:STEP] and a setup.ps1 block check for the Windows parity.	2026-08-01 04:12:25 -07:00
Daniel Han	425c9081a3	Uninstall the whisper.cpp prebuilt and the node/whisper install locks (#7686) ... * Uninstall the whisper.cpp prebuilt and the node/whisper install locks scripts/uninstall.sh removed the llama.cpp prebuilt and its install lock but not the whisper.cpp prebuilt, and not the node or whisper install locks. Every prebuilt serializes on <parent>/.<name>.install.lock (prebuilt_core.py install_lock_path), so a normal install leaves ~/.unsloth/.node.install.lock behind. The final rmdir refuses a non-empty directory, so one stray zero-byte lock keeps the whole ~/.unsloth tree on disk. Confirmed on a real ubuntu-latest install -> update x3 -> uninstall run: ~/.unsloth and ~/.unsloth/.node.install.lock both survived. whisper.cpp is only installed when a prebuilt matching the pinned llama.cpp build exists, so runs that skip it never exercised that path. Also removes the .stale.<pid> locks that a lock takeover renames rather than deletes, documents the sibling prebuilts in --help, and adds a ~/.unsloth catch-all to the smoke workflow's leak list so a future artifact cannot be missed the same way. tests/sh/test_uninstall_prebuilt_artifacts.sh runs the uninstaller for real against a fixture HOME: 19 checks pass here and 8 of them fail against the previous uninstall.sh. It also asserts user content under ~/.unsloth is kept. * Mirror the prebuilt cleanup on Windows, and skip the artifact suite on WSL Two review findings. uninstall.ps1 removed llama.cpp, node, .staging and the llama lock, but not the whisper.cpp tree or the node and whisper locks, while setup.ps1 installs whisper.cpp at %USERPROFILE%\.unsloth\whisper.cpp. A native Windows uninstall therefore left a prebuilt tree behind and could never prune ~/.unsloth. Adds the whisper dir, all three locks and the .stale.<pid> sweep, puts the whisper dir in the handle-releasing list so a running whisper-server does not block the delete, updates -Help, and extends the Windows smoke assertion with the same catch-all the POSIX one got. tests/sh/test_uninstall_prebuilt_artifacts.sh runs the real uninstaller, and overriding HOME does not contain that on WSL: the body detects WSL from /proc/version and then deletes Windows-side shortcuts under /mnt/*/Users and /etc/profile.d/unsloth-rocm-wsl.sh. The sibling arg-guard suite already skips there for exactly this reason; this one now does too. tests/studio/test_uninstall_prebuilt_parity.ps1 asserts the two uninstallers cover the same artifacts, since drift between them is what caused this. It cannot run the body (that kills processes and writes the registry) so it parses uninstall.ps1 instead. Against the previous uninstall.ps1 it fails on exactly the four missing behaviours. Also corrects the stale-lock comment: install_node_prebuilt.py renames then unlinks, so a stranded .stale file needs a crash between the two steps.	2026-08-01 03:03:30 -07:00
Daniel Han	31a651a407	Close two false passes in the uninstaller argument tests, and document the piped help form (#7690) ... * Close two false passes in the uninstaller argument tests, and document the piped help form The help loop minted a fresh fixture home per iteration but asserted the install survived only after the loop, so only the -h iteration was ever inspected. A --help that removed files would have passed. Verified: a mutant whose --help prints usage, exits 0, and deletes the CLI shim passes the existing suite and fails the updated one in three places. The two multi-argument cases asserted an exit code and nothing else, so a guard that rejected the argument only after removal had started would also have passed. Separately, -h is a shell option (hashall), so `... | sh -h` and `... | bash -h` are consumed by the shell and the script runs with no arguments, which is a full uninstall. Someone reaching for help through the documented pipe destroys their install instead. Nothing in the script can detect that, since the argument never arrives, so the usage text now spells out `sh -s -- --help` and names the hazard. A check asserts the usage keeps saying so, and a new portable case proves that form reaches the guard on every platform rather than only in the Linux-only pipe-buffer test. 32 checks pass, up from 25. * Correct the -h claim: dash and busybox reject it rather than uninstalling The help text said piping to `sh -h` uninstalls. That holds only where -h is accepted. Measured against the instrumented copy in a sandboxed HOME: bash -h rc=0 reached the uninstall body zsh -h rc=1 reached the uninstall body bash --posix -h rc=0 reached the uninstall body dash -h rc=2 Illegal option -h, install untouched busybox sh -h rc=2 illegal option -h, install untouched Debian and Ubuntu ship dash as /bin/sh and Alpine ships busybox, so the text was wrong about a destructive operation for most of the Linux audience it is addressed to. Both outcomes are now stated, and the advice is unchanged: `sh -s -- --help` is the only piped form that prints help. The usage assertion also gets its own run instead of reusing $OUT from the last loop iteration, so a failure is attributed to the right check. * Spell out the URL in the piped-help example The abbreviated form was not copy-pasteable: curl takes `.../uninstall.sh` as the URL, fails to resolve it, and the pipeline still exits 0 because the status is sh's and sh reads an empty stdin. So a user who pasted it saw a resolve error and no help, which is exactly the confusion the paragraph exists to prevent.	2026-08-01 03:02:44 -07:00
Daniel Han	5344ec3e49	Tighten the comments around the uninstaller argument guard (#7647) ... Shorten the header blocks in scripts/uninstall.sh and scripts/uninstall.ps1 and the two test file headers, and drop five comments that restate the line below them. The comments that carry a non-obvious reason are kept: why the trailing compound block makes a truncated curl pipe inert, why PowerShell throws rather than exits, why the real uninstall body is skipped on WSL, why the pipe test guards against going vacuous, and why the fixture path is a global. Help text is program output, not commentary, so the _usage heredoc and the _Usage here-string are untouched. No executable code is changed.	2026-07-30 05:36:27 -07:00
Daniel Han	779d8fac77	Make the uninstaller test suites run on macOS and Windows PowerShell 5.1 (#7643) ... The suites added in #7631 pass on the ubuntu runner but abort elsewhere. tests/sh/test_uninstall_arg_guard.sh built its fixture with TMPDIR="$_TMP_ROOT" mktemp -d. BSD mktemp with no template implies -t and resolves the directory itself, so on macOS the result is a sibling of _TMP_ROOT rather than a child, the validation below it fires, and the whole suite exits at the first --help case. Pass an explicit template instead. tests/sh/test_uninstall_shared_icon.sh used mktemp -p, which is GNU only. tests/studio/test_uninstall_arg_guard.ps1 set $ErrorActionPreference to Stop and then merged a native command's stderr with 2>&1. Windows PowerShell 5.1 turns that into a terminating NativeCommandError, so the suite died on the first rejected-argument case; PowerShell 7.1 and later do not. Scope the preference down around the two native invocations. The suite is self-hosting through (Get-Process -Id $PID).Path, so the Windows workflow now also runs it under powershell.exe. Nothing in CI covered 5.1 before, which is why this went unnoticed.	2026-07-30 04:09:02 -07:00
oobabooga	a1dcd94846	Uninstaller: reject unrecognized arguments instead of uninstalling (#7631)	2026-07-30 03:47:22 -07:00
Daniel Han	132438282c	Installer: accept wget in studio/setup.sh, as install.sh already does (#7600) ... * Accept wget in studio/setup.sh, as install.sh already does install.sh takes either transport everywhere: download() and _http_get both try curl then wget, and the transport gate only fires when both are missing. A wget-only box therefore installs fine, reaches studio/setup.sh, and finds curl as the only way to fetch anything there. Two sites: the uv bootstrap, where the fallback is silent (USE_UV stays false and fast_install degrades to python -m pip), and the PyPI version check. Both now go through a helper with install.sh's preference order. Verified in a PATH containing wget but no curl: both helpers return 0 and the uv installer is fetched, while the bare `curl -LsSf` this replaces exits 127. * Cap the wget version check at one attempt wget's --timeout is per operation and it retries 20 times by default, so a server that accepts the request and then stalls stretched the bounded PyPI version check to 245s against a local stalling server; --tries=1 brings it back to the 5s curl's --max-time gives. Adds tests/sh/test_setup_http_get.sh covering both helpers: curl preferred, wget accepted, neither is a non-zero return, and the wget flags. * Bound the wget version check by wall clock, not per operation wget's --timeout is per network operation, so a response that dribbles a byte inside every interval never ends: against a local drip server the check was still running after 45s, where curl's --max-time 5 stops at 5s. Wrapping wget in coreutils timeout gives the same ceiling (5s, rc 124 in the same repro). timeout is not in a base macOS install, so its absence falls back to the per-operation bound instead of dropping the check; that path only matters without curl, which macOS ships. * Tighten the setup.sh fetch helper comments	2026-07-29 03:05:42 -07:00
Daniel Han	bc07d3a2df	Installer: wrap install.sh in a function so a piped install cannot report curl (56) (#7548) ... * Installer: wrap install.sh in a function so a piped install cannot report curl (56) `curl -fsSL https://unsloth.ai/install.sh | sh` makes sh the READER of a pipe. The file is ~150KB, far more than a pipe buffer holds, so a top-level `exit` left sh dead with thousands of lines unread. The write end then failed and curl appended curl: (56) Failure writing output to destination, passed 16357 returned 0 after the installer's own message, which reads as a download failure rather than the real diagnosis. 29 of the 35 exits are in the first half of the file, so every early failure on every platform looked like a bad download. Measured, piping this file into sh and forcing an early exit: before: writer rc=141 (SIGPIPE) reader rc=1 after: writer rc=0 reader rc=1 Through a real curl against a local server, curl rc went 23 -> 0 while the installer's own exit code kept propagating. Defining a function forces sh to parse to the closing brace before running anything, so the pipe is always drained. install.ps1 has always had this shape (Install-UnslothStudio invoked at the end of the file); this brings install.sh into line. Deliberately not reindented. Shell ignores leading whitespace, so the diff stays two hunks instead of 4400 reflowed lines, and `exit` still exits the shell from inside a function, so no control flow changes. tests/sh/test_install_pipe_safety.sh pins both halves of the contract: the writer must survive, and the installer's real exit code must still reach the caller. It fails against the unwrapped file (writer rc=141). * Tighten the pipe-safety comments Compress the install.sh wrapper rationale and the test header down to the parts that are not obvious from the code. Comments only, the parsed command tree of both files is byte identical. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-28 21:16:31 -07:00
Daniel Han	df63522369	Installer: stop requiring a developer toolchain on the consumer path (#7547) ... * Installer: stop requiring a developer toolchain on the consumer path A brand new Mac cannot install Studio at all. install.sh gates on `xcode-select -p` and exits 1 with 'Xcode Command Line Tools are required', and Linux exits 1 on any non-apt distro over cmake/gcc/git/libcurl headers. Nothing under either gate needs a toolchain. uv is a prebuilt binary, CPython comes from uv's managed python-build-standalone, llama.cpp and whisper.cpp are prebuilt downloads, Node is a pinned nodejs.org archive, and triton is skipped on macOS. unslothai/llama.cpp b10107-mix-1911198 publishes macos-arm64, macos-x64, linux-x64 and linux-arm64 builds covering cpu, cuda12, cuda13, rocm and vulkan. PR #6617 already dropped the Homebrew/cmake stop on macOS for this reason and just left the CLT stop behind. macOS: warn and continue when the CLT are absent. Linux: only a download transport (curl or wget) is fatal; build tooling warns. Both keep a hard git requirement for --local, which installs unsloth-zoo from a git+https URL. Both gates move into functions so tests/sh can extract them. The old inline form could not be reached by the tests/sh convention, which is why this shipped broken and stayed broken. test_macos_clt_gate.sh (19 assertions) and test_linux_deps_gate.sh (25) cover the clean machine, the CLT-stub shape where /usr/bin/git exists but fails, the non-apt distro, and the --local paths. Writing the Linux test caught a latent bug: the gate trimmed its list with $(echo ... | sed ...), so on a minimal image without sed the substitution yields empty and it reports 'all system dependencies found' on a machine with none of them. Replaced with parameter expansion. Also caps av<16 in the single-env constraints. av 16+ ships no cp313 macOS arm64 wheel, and it is a C extension over FFmpeg, so uv would silently fall back to a source build needing both a compiler and FFmpeg headers. Verified on GitHub-hosted macOS runners with /var/db/xcode_select_link, /Library/Developer/CommandLineTools, /Applications/Xcode*.app and Homebrew moved aside. macos-14, macos-15 and macos-26 fail on main and install cleanly with this; the recorded tool-invocation trace for the whole install is a single `xcode-select -p`, so nothing compiled and nothing installed a toolchain. * Linux: auto-install git rather than dropping it, and skip triton kernels without it Making git optional on Linux was too broad. studio/backend/requirements/ triton-kernels.txt line 2 is a git+https URL, so step 6/14 died with 'Cannot find command git' and failed the whole setup on ubuntu2404-root, ubuntu2404-arm-root and fedora41, all of which had been passing. The claim that nothing on the consumer path needs git holds on macOS, where triton is skipped, but not here. install.sh now auto-installs git through apt with the other optional tooling, so Debian and Ubuntu are unchanged. The triton kernels step skips with a message when git is absent instead of failing: they are a training speedup, not a boot requirement, and a GGUF chat install has no use for them. Six more assertions pin both halves. * macOS Intel: skip the one package with no x86_64 wheel The Intel clean-machine leg installed with the toolchain masked, then died in studio setup: subprocess.CalledProcessError: Command '['cmake', ...]' returned non-zero ERROR: Failed building wheel for pytorch_tokenizers pytorch_tokenizers publishes wheels for macOS arm64, linux x86_64, linux aarch64 and windows, but none for macOS x86_64 at any Python version, so uv falls back to an sdist that shells out to cmake. Nothing passes --only-binary, so the compiler-free property was an assumption rather than a contract, and Intel is where it broke. Marked so it installs everywhere except Intel macOS. Apple Silicon is unaffected. * Stop the optional dep gate from aborting the install _smart_apt_install exits rather than returns, and `|| true` does not catch an exit, so a box missing cmake or git aborted at the gate added to let it continue. Verified in sh, dash and bash. Run it in a subshell and re-raise only code 2, the NEED_SUDO handshake install.rs answers with an elevation prompt. install.sh treats a present-but-broken git as missing, but the Python side tested only shutil.which, so it promised to skip the git+https triton requirement and then fetched it anyway. Same check on both sides now. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Never elevate for optional build tools Re-raising code 2 turned the optional set into a NEED_SUDO handshake, so a box missing cmake or gcc got the desktop's mandatory permission dialog, whose Cancel drops back to not-installed. That re-imposes through a prompt the build-tool requirement this gate removes, and none of those tools are needed to run. Suppress the handshake for optional callers; a required package still elevates. Verified in sh, dash and bash. Also advance the progress bar on the no-git triton skip, which otherwise ends at 14/15. * Tighten the comments on the dependency gate * Correct why the PyAV cap is needed 16.0.0 does ship cp313-cp313-macosx_14_0_arm64; the comment claimed no cp313 wheel exists. The actual reason is the deployment target: 15.1.0 is macosx_13_0 and 16+ is macosx_14_0, so the cap is what keeps macOS 13 off a source build. * Tighten the installer gate comments * Cap cryptography on x86_64 macOS so the consumer install needs no Rust cryptography 49.0.0 (2026-06-12) dropped the macosx_10_9_universal2 wheel and now ships macosx_11_0_arm64 only, so x86_64 macOS has no wheel and uv falls back to the sdist. That build calls maturin, which pulls Rust and then fails at 'linking with cc failed' on a clean Mac without the Xcode Command Line Tools. It surfaced in the clean-machine leg mac macos-15-intel / mask / file, several minutes into the studio dependency step, which is exactly the up-front toolchain requirement this branch removes. 48.0.1 is the newest release carrying a universal2 wheel, and its cp39-abi3 / cp311-abi3 tags cover the 3.12 and 3.13 interpreters the installer creates. The cap is marker-scoped to darwin + x86_64, so arm64 macOS and every other platform still resolve to the latest. Lift it when cryptography ships an x86_64-capable macOS wheel again. Resolution of studio/backend/requirements/studio.txt under this constraints file gives 48.0.1 on x86_64-apple-darwin and 49.0.0 on aarch64-apple-darwin and x86_64-unknown-linux-gnu, on both 3.12 and 3.13. * Correct the av note now that cryptography also compiles on macOS * Never escalate for optional apt packages outside Tauri mode The optional bypass sat inside the TAURI_MODE branch, so a plain curl | sh install on a non-root Debian or Ubuntu box still fell through to the escalation branch and showed the default-yes permission prompt for cmake, GCC and the libcurl headers. That is exactly the toolchain this change set declared unnecessary on the consumer path, so the prompt asked for a password to install packages nothing here uses, and a headless run failed the same way instead of falling through to prebuilt llama.cpp. Move the check above the mode split so optional callers return 2 in both modes. Required packages such as curl still escalate unchanged. --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-28 18:50:38 -07:00
oobabooga	01c856c6c5	Surface actionable installer failures in Studio desktop (#7529) ... * Studio: surface actionable installer failures * Correct installer failure attribution * Preserve desktop installer failure context * Use explicit setup failure attribution * Preserve package manager failure details	2026-07-28 10:19:44 +02:00
Daniel Han	4b3809a2f4	tests: stop the installer constraint test counting occurrences (#7503) ... test_torch_constraint.sh asserted how many times each pin appears in install.sh. Every hardware branch assigns its own torch/torchvision/torchaudio triple, so #7354 adding gfx906 pushed three of those counts up by one and Backend CI has been red on main since: FAIL: default TORCH_CONSTRAINT assignment exists (expected '1', got '2') FAIL: hardcoded torch>=2.4 appears exactly once (expected '1', got '2') FAIL: torchvision bounded (<0.26) at default + custom-leaf (expected '2', got '3') FAIL: torchaudio bounded (<2.11) at default + custom-leaf (expected '2', got '3') install.sh is correct; the numbers were the stale part. Assert the invariants instead, so the next hardware branch is not a test edit: - the default assignment is the top-level one, so anchor the grep at column 0 rather than counting every occurrence. An indented branch pin no longer satisfies it, which the old count did not distinguish either. - what "appears exactly once" really guarded is that no pip install line spells a pin out instead of using "$TORCH_CONSTRAINT", so check that directly. - companions must be bounded everywhere, so compare bounded assignments against total assignments rather than pinning a count of 2. That is strictly stronger: it now covers all 7, not the 2 the old numbers happened to name. 45 pass, 0 fail. Each new assertion fails when its property is broken: a bare or unbounded companion, a hardcoded pin on an install line, or a missing top-level default.	2026-07-27 06:42:20 -07:00
Daniel Han	dc24bba43e	install.sh, setup.sh: apply the no-tty consent fix to the remaining sites (#7470) ... Follow-up to #7435, which fixed _smart_apt_install. Three sites were left. studio/setup.sh: the WSL GGUF build-deps block is the pre-#7435 install.sh pattern verbatim. It probes with 'test -r /dev/tty', assumes REPLY=y when that fails, and then runs the elevated apt-get with stdin open. Its own guard comment says a password is needed on WSL, so this is exactly the scenario from issue #7307, and install.sh runs setup.sh in the same install. Give it the same treatment: a real open probe, -n -k with stdin closed on the headless path, and the manual command plus the existing _SKIP_GGUF_BUILD degradation on failure. The helper is defined locally because setup.sh runs as its own process. install.sh autostart prompt: still used 'test -r /dev/tty' and printed the question before checking, leaving a dangling prompt in container logs. Reuse _can_read_tty and move the printf inside the branch. install.sh interactive escalation: a sudoers denial, a wrong password or an apt error aborted on the bare message while the headless branch printed what to run by hand. Make both symmetric. Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-26 05:22:28 -07:00
Leo Borcherding	c3d3680e7c	install.sh: do not assume sudo consent when there is no terminal (#7435) ... * install.sh: do not assume sudo consent when there is no terminal (#7307 P7) _smart_apt_install printed an "Accept? [Y/n]" prompt, and when /dev/tty was unreadable it set REPLY=y and escalated anyway. Every sudo call in that branch redirects stdin from /dev/null, so on any host where sudo needs a password the install died on sudo's own error rather than the actionable message the no-sudo path already prints. Containers, CI and locked-down corporate machines hit this. Probe with `sudo -n true` first. If there is no terminal to prompt on and sudo would need a password, exit with the missing packages and the exact command to run, matching the no-sudo path. Passwordless sudo still escalates unattended, which is the one case where that is legitimate, and says so in the log. With a readable /dev/tty the behaviour is unchanged, and the prompt now only prints when something can actually answer it. Extend tests/sh/test_apt_distro_prompt.sh to drive the real function across all four TTY/sudo combinations, rewriting /dev/tty to a fixture path the same way the existing cases rewrite /etc/os-release. Against the old install.sh five of these assertions fail. Register the file in studio-backend-ci.yml's shell suite, which did not run it before. * install.sh: probe the real tty and the real sudo commands (#7307) Codex review follow-ups on the no-TTY sudo escalation guard. `test -r /dev/tty` only reads the device node's permission bits. Inside containers and systemd units those bits look fine while open() fails with ENXIO, so the guard still fell through to a prompt nobody could answer. _can_read_tty() does a real open. The subshell is load-bearing: in dash a failed redirection on the special builtin `:` exits the script. `sudo -n true` proves only that `true` is allowed. Under a command-specific rule like `NOPASSWD: /usr/bin/apt-get` it is the wrong question in both directions. _sudo_runs_unattended() asks the sudoers policy about the exact argument vectors we are about to elevate, via `sudo -n -l --`, which checks without running and fails instead of prompting. Tests cover both: a NOPASSWD-on-trivia-but-not-apt-get sudoers stub, and a readable-but-unopenable /dev/tty faked with a unix socket (skipped where the platform cannot produce that shape). * install.sh: test sudo by running it with -n, not by asking sudo -l Codex follow-up. `sudo -n -l -- apt-get ...` answers authorization, not authentication: on a host where apt-get is permitted but still carries the PASSWD tag, list mode exits 0 while the actual run needs a password, so the guard reported unattended and the escalation died exactly as #7307 described. Inferring the answer from list output means parsing for `!authenticate`, which is human-readable text that varies by sudo version. Drop the inference. In the no-terminal branch, run the real commands with `sudo -n`: -n never prompts, so it cannot block on a closed stdin, and its exit status is the question we were trying to answer. If it is refused, print the actionable manual command as before. The terminal branch is unchanged: prompt, then plain sudo, which may ask for a password because someone is there to type it. The test stub now models sudo properly (-n refuses and runs nothing when a password is needed) instead of special-casing the probe's argv. * install.sh: require a real NOPASSWD rule, and stop blaming the password for apt failures Two review findings on the headless escalation branch. A cached authentication timestamp from an earlier, unrelated elevation made `-n` succeed for a PASSWD-tagged apt-get, so packages installed with nobody having answered the prompt. Add `-k` so the probe ignores the timestamp and only a real NOPASSWD rule counts as passwordless. Per sudo(8), `-k` alongside a command ignores the cached credentials for that invocation and "will not update the user's cached credentials", so an interactive session elsewhere does not have to re-authenticate afterwards. A nonzero status from the elevated apt-get was reported as "likely needs a password" even when sudo had authenticated fine and apt itself failed on a bad repository, a dpkg lock or a network outage. sudo returns the command's own exit status when the command runs, so the two cases are not distinguishable from the status alone. Report both possibilities and point at the real error. tests/sh/test_apt_distro_prompt.sh: teach the sudo stub about -k, add a cached mode, and assert both behaviours. The three new assertions fail against the previous commit. * install.sh: an unreadable answer at the consent prompt declines _can_read_tty proves the device opens, not that anyone is there to answer. A read that hits EOF still fell back to REPLY=y and escalated, so the branch that does have a terminal kept the behaviour this change removes from the branch that does not. A drained or half-closed terminal reached it. Default to n instead, which is what the post-install autostart prompt at the bottom of this file already does on the same condition. Enter still means yes: that is a successful read of an empty line, not a failed read. tests/sh/test_apt_distro_prompt.sh: add an eof tty fixture, which opens normally and returns EOF immediately. Both new assertions fail against the previous commit. * install.sh: tighten the escalation comments, and correct the exit-status claim Comment-only. The earlier note said a nonzero status from the elevated apt-get was not distinguishable from the status alone; sudo(8) is more specific than that. sudo exits 1 on an authentication or configuration failure and passes the command's own status through when the command runs, while apt-get(8) returns 100 on error, so the two usually are distinguishable. sudo also exits 1 when the command cannot be executed, which is why the message still states both causes rather than naming one. * install.sh, tests: tighten the comments added by this branch Comment-only pass over the branch's own comments in both files. Same intent, fewer lines: drop restatement, keep the parts a reader cannot derive from the code (why test -r is the wrong probe, why the subshell around the redirection is load-bearing under dash, what -k buys over -n, and why a nonzero status does not by itself name the cause). Verified to touch nothing but comments and blank lines. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-26 04:27:15 -07:00
Daniel Han	418ae14388	Fix ROCm wheel-index unit test: extract the gfx-arch probe helpers get_torch_index_url now calls (#7399) ... * Fix ROCm wheel-index test: extract the gfx-arch probe helpers get_torch_index_url now calls get_torch_index_url gained a gfx-arch probe on the ROCm path (Strix reroute work) and now calls _ensure_rocm_probe_env, _probe_amd_gfx_arch, _infer_linux_amd_gfx_arch and friends. The unit test in tests/sh/test_get_torch_index_url.sh sources a curated subset of install.sh functions, and that list was never updated, so those helpers were undefined in the harness. On the ROCm path the gfx probe hit an undefined function, the branch silently fell through to the CPU wheel index, and every ROCm assertion failed (9 failures: all ROCm versions resolved to /whl/cpu). Extract the six missing helpers so the ROCm branch runs end to end. All 49 assertions pass. Adds a comment noting these must stay in sync with install.sh. * Keep the ROCm wheel-index test hermetic: redirect the /opt/rocm prefix Extracting _ensure_rocm_probe_env pulled its absolute-path host probe into the harness: it appends /opt/rocm/bin to PATH and runs the real host rocminfo, and version detection reads /opt/rocm/.info/version. On a host with ROCm installed that leaks the host GPU into the minimal-PATH test, so the no-GPU and CUDA-visible-device assertions could select a host ROCm wheel index instead of their expected CPU result, making the test host-dependent. Redirect the whole /opt/rocm prefix to an empty temp dir in the same sed pass that stubs /usr/bin/nvidia-smi, so the probes stay hermetic. All 49 assertions pass and the generated harness contains no real /opt/rocm path. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-24 02:12:16 -07:00
Souravrajvi0	0807d03ed0	fix(install): show detected distro in sudo apt Accept prompt (#7324) ... * fix(install): show detected distro in sudo apt Accept prompt Make the package-install elevation prompt name the detected distro and state that packages come from official apt repos, so users know we are not installing a tarball outside their package manager (#6207). * fix(install): avoid case/;; inside $() for bash 3.2 macOS CI uses bash 3.2, which misparses case arms inside command substitution and fails install.sh at the apt distro helper. Use a plain subshell so the Accept? prompt still works everywhere.	2026-07-23 19:16:00 -07:00
Souravrajvi0	09b6bf6c39	fix(studio): opt-in source-build GPU smoke validation (#7322) ... * fix(studio): opt-in source-build GPU smoke validation (#5854) Gap 1 (empty CUDA arch -> CPU) already landed in #6481. Wire gap 2: after a GPU source build, optionally run the same staged llama-server smoke test as the prebuilt path, then CPU-fallback on failure. Gated by UNSLOTH_LLAMA_STAGED_VALIDATION (default off) to avoid Blackwell JIT stalls. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(install): normalize staged validation env in setup.sh (#7322) Strip and lowercase UNSLOTH_LLAMA_STAGED_VALIDATION before the shell gate so values like True and surrounding whitespace match the Python staged_validation_enabled() helper. * Rebuild visual server after staged-validation CPU fallback (#5854) Mirror the primary source-build path by best-effort building llama-diffusion-gemma-visual-server after smoke-failure CPU fallback. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-23 19:13:54 -07:00
oobabooga	88583dd2ec	Installer: restore interrupted updates and clean stale rollback environments (#7342) ... * Installer: restore interrupted updates and clean stale rollback environments * CI: run POSIX rollback lifecycle tests on Linux	2026-07-23 01:29:53 -07:00
Daniel Han	4e1cb4affa	install.sh: route Strix to the AMD arch index on rocm7.2 (#7264) + PCI detection hint (#7293) ... * install.sh: route Strix (gfx1151/gfx1150) to the AMD arch index on rocm7.2, add PCI hint Two Linux install fixes for AMD Strix Halo / Strix Point: 1. #7264: Strix reverts to rocm7.2. Modern ROCm (7.3+) caps to the generic rocm7.2 index and the Radeon repo can be unavailable, so gfx1151/gfx1150 landed on a non-arch-specific build (torch 2.11+rocm7.2) instead of repo.amd.com/rocm/whl/gfx<arch> (torch 2.11+rocm7.13, AMD's real Strix fixes). The reroute to that arch index only fired on rocm7.1; broaden it to rocm7.2 too. Only acts when a gfx1151/gfx1150 is actually detected, so other arches on rocm7.2 pass through unchanged. 2. Rows about Strix not detected -> CPU-only: when no GPU is detected but an AMD display GPU is on the PCI bus, print a targeted hint (ROCm kernel stack / /dev/kfd missing) instead of only the generic docs pointer. Purely additive diagnostic; does not change the torch index decision. * Address review on PR #7293: gate PCI hint on ROCm-detection failure, fix test marker, use read builtin - Only show the 'ROCm cannot see the GPU' hint when _has_amd_rocm_gpu fails; a detected-but-too-old ROCm (rocminfo works, wheels need 6.0+) has its own path. - Update test_previous_torch_pin.sh to the stable 'Strix Halo / Strix Point:' marker after the heading reworded (the old grep broke the ordering assert). - _amd_gpu_present_via_pci: read builtin instead of spawning cat twice per device, and guard /sys/bus/pci/devices existence. * install.sh: reroute Strix on any generic index older than the arch build Generalize the Strix reroute from the hardcoded rocm7.1/rocm7.2 match to a version compare against the arch index's own build (rocm7.13): - backwards: rocm6.0-6.4 and rocm7.0 now reroute (were silently missed) - forwards: any future intermediate rocm7.x below 7.13 reroutes; rocm7.13+ is left alone so a generic index that already carries the fix is not downgraded to the arch build _rocm_index_below does an integer major.minor compare (so rocm7.2 < rocm7.13); non-rocm, arch (gfx), and unparseable URLs return false, so NVIDIA/CPU and the arch index itself are untouched. Reroute still fires only for gfx1150/gfx1151. * install.sh: tighten _amd_gpu_present_via_pci comment (no code change) * install.sh: match the index leaf in the Strix version reroute (#7293 review) Address two review points on the rocm-version reroute: - Parse the final path segment (_torch_index_leaf) instead of grepping the whole URL. A custom mirror whose base path holds its own rocm token (e.g. .../rocm7.13/cache/rocm7.2) previously matched the base and skipped the reroute; now it compares the leaf (rocm7.2) like the nearby index-family logic. Renamed the helper to _rocm_leaf_below and switched the case selector to $_torch_index_leaf. - Replace the stale test_strix_override_only_fires_on_rocm71 (which passed by matching the new rocm7.13 comment) with an executed test that runs _rocm_leaf_below and asserts rocm6.0-7.12 reroute while rocm7.13+/gfx/cu leaves do not. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: keep gfx probes non-fatal under set -e (#7293 review) The Strix reroute now matches every rocm* index, not just rocm7.1, so its gfx detection runs on all AMD installs. Each `_gfx_all=$(rocminfo|amd-smi | grep -oE gfx...)` returns 1 when grep finds no match, which under set -euo pipefail aborts the installer before the next fallback runs (e.g. rocminfo present but emitting no gfx token). Append `|| true` to the three probes, matching the display block that already guards this. Add an executed regression test (shimmed rocminfo/amd-smi) that fails if any probe becomes fatal again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-21 19:40:59 -07:00
Daniel Han	3ab8dce97a	install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection (#6692) ... * install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection get_torch_index_url (and the studio-update mirror _detect_cuda_torch_index_url) chose the torch wheel family solely by probing the host GPU, with no override. In a headless / container / CI build the host driver is visible via the /proc/driver/nvidia/gpus fallback but nvidia-smi cannot report a CUDA version, so the function fell back to its cu126 default and installed the wrong wheels (e.g. a cu128 image got cu126 torch). Add an explicit override checked before any probing, in both the shell installer and the Python studio-update path: - UNSLOTH_TORCH_INDEX_URL full index URL, used verbatim (wins) - UNSLOTH_TORCH_INDEX_FAMILY family (cpu, cu128, rocm6.4, ...) appended to the mirror base (UNSLOTH_PYTORCH_MIRROR still honoured) This matches how the published GPU images select CUDA -- vLLM and SGLang take the CUDA version from an explicit build ARG rather than detecting it, and the Unsloth Docker base image already pins the cu128 index directly. Desktop installs are unchanged: with no override set, detection runs exactly as before. Adds test_get_torch_index_url.sh cases for the override (family, full URL, precedence, mirror base, trailing-slash strip, empty-ignored). * install: make the torch-index override authoritative across ROCm paths Address review feedback on the override added in this PR so a pinned index is honoured everywhere, not just in get_torch_index_url: - Skip the WSL ROCm bootstrap (root privilege + large downloads, probes /dev/dxg) when UNSLOTH_TORCH_INDEX_URL / _FAMILY is set; it previously ran before the override was consulted. - Skip the Radeon/Strix rerouting (which re-probes the GPU and overwrites the resolved URL with repo.radeon.com / repo.amd.com) when the index is pinned, so an explicit ROCm override (e.g. UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4) is kept. - install_python_stack.py: derive _TORCH_BACKEND from the override when UNSLOTH_TORCH_BACKEND is unset (standalone studio update), so _ensure_rocm_torch / _ensure_cuda_torch repair to the requested family instead of re-detecting. - Strip ALL leading/trailing slashes in the shell override to match the Python side (avoids 404s on strict pip proxies). Adds test cases for double-slash and leading/trailing-slash overrides. * install: honor pinned torch index in CUDA/ROCm repair paths Follow-up to the override work in this PR: the get_torch_index_url / install.sh reroute already respect a pinned UNSLOTH_TORCH_INDEX_URL / _FAMILY, but the Python repair helpers in install_python_stack.py still re-probed the GPU and could overwrite the pinned family. Make the pin authoritative there too: - _ensure_cuda_torch: an explicit cu* pin commits to CUDA wheels, so repair a ROCm-poisoned venv even when no NVIDIA GPU is visible here (headless / container / CI cross-install), instead of bailing on the GPU-presence gate. - _ensure_rocm_torch: skip the AMD per-gfx (Strix) reroute when a ROCm index is pinned, and in the generic reinstall path install from the pinned URL verbatim rather than re-detecting the host ROCm version. gfx*/rocm7.2 indexes serve torch 2.11+, so select the 2.11 package specs for a gfx leaf. - install.sh: raise the torch constraint to 2.11 for */gfx* indexes too, matching rocm7.2, so a pinned full-URL/family override that returns early keeps a valid constraint. Add _explicit_torch_index_url / _explicit_rocm_torch_index_url helpers and tests covering the no-GPU CUDA pin repair and the explicit gfx index honored verbatim. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: honor torch-index override on the Windows installers too The pinned-index work landed for install.sh and install_python_stack.py, but the Windows installers still picked the wheel index from GPU probing. Extend the same UNSLOTH_TORCH_INDEX_URL / _FAMILY contract so a pinned index wins on every platform: - install.ps1: Get-TorchIndexUrl returns the pinned URL/family before nvidia-smi probing; the AMD ROCm reroute is skipped when the index is pinned, so an explicit cpu/cu* pin on an AMD host is not overwritten. - studio/setup.ps1: add shared Get-PinnedTorchIndexUrl / Get-TorchIndexLeaf helpers; the stale-venv check, the install selection and the AMD reroute all honor the pin, and the CPU/CUDA install pulls from the resolved index URL. - tests: parity test that all four installers read both override vars and the two Windows installers gate the AMD reroute on the pinned flag. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: complete pinned-index handling for ROCm/Windows edge cases Follow-ups to the override work flagged in review: - install.ps1: a pinned gfx*/rocm>=7.2 index previously skipped the AMD reroute that sets the torch>=2.11 floor, so the generic install used torch>=2.4,<2.11 and could resolve the known-bad _grouped_mm wheel. Route a pinned ROCm index through the ROCm install path with the 2.11 floor + companions, and guard the companion-spec lookup so a skipped reroute block cannot null-deref. - studio/setup.ps1: the stale-venv check compared the installed flavor (cuXXX/cpu, with +rocm misread as cpu) against the raw pinned leaf (gfx1151 / rocm6.4), so a correct pinned ROCm venv was always marked stale. Classify +rocm wheels as the generic 'rocm' flavor and normalize a pinned rocm*/gfx* leaf to 'rocm' before comparing (cu* stays specific so cu126-vs-cu128 still rebuilds). - install_python_stack.py: _ensure_cuda_torch now also reinstalls from a pinned CUDA index when the venv carries a CPU wheel (headless CPU-venv-to-CUDA cross-install via 'studio update'), not only when it finds a ROCm build. - tests: parity assertions already cover all four installers honoring the override. * install: finish pinned ROCm/CUDA edge cases on Windows + repair path Follow-ups to the previous round: - studio/setup.ps1: a pinned gfx*/rocm>=7.2 index now routes through the ROCm install path with the 2.11 floor + companions (it previously fell through to the CUDA branch with bare torch/torchvision/torchaudio against the ROCm index). The CPU/CUDA fallback index is forced to the CPU wheel index when a ROCm index is active, so a failed pinned-ROCm install does not retry the ROCm mirror. - studio/setup.ps1: the stale-venv check no longer treats an unrecognized pinned URL leaf (e.g. a PEP 503 mirror ending in /simple) as a torch flavor tag, which was marking a correct venv stale; cu*/cpu/rocm/gfx leaves are still compared. - install.ps1: the post-failure CPU fallback uses an explicit CPU index instead of , which for a pinned ROCm index was the ROCm mirror itself (so the 'fallback' just retried the failing index and aborted the installer). - install_python_stack.py: _ensure_cuda_torch now also reinstalls when the venv's CUDA family differs from a pinned one (installed cu126 vs pinned cu128), not only CPU->CUDA; the probe reports the installed cuXXX tag for the comparison. * install: keep the ROCm to CPU fallback install inside the retry-helper window The pinned-ROCm CPU fallback computes an explicit CPU index, but the comment explaining why it cannot reuse $TorchIndexUrl pushed the actual Invoke-InstallCommandRetry / --force-reinstall call more than 600 chars past the "ROCm PyTorch install failed" message, so test_pr5940_followups's window check no longer saw the retry helper. Move the CPU-index computation and its comment above the failure substep so the retrying force-reinstall stays adjacent to the message. No behavior change: same explicit CPU index, same retry, same --force-reinstall. * install: address #6692 review round 5 (ROCm/CPU pin edge cases) setup.ps1: - Stale-venv check: treat an AMD/ROCm host (HasROCm or a resolved gfx arch) with no explicit pin as expecting "rocm", not "cpu", so a healthy +rocm venv is not flagged stale (which made installer-managed setup exit and direct update rebuild). - Pinned-ROCm install failure now routes into the force-reinstall CPU branch: CuTag stays the rocm/gfx leaf on failure, so the condition also checks ROCmCpuFallback; otherwise the CUDA branch installed from the CPU index without --force-reinstall and kept the partial ROCm torch. - Explicit ROCm pin compare no longer collapses gfx*/rocm* to a generic "rocm": it compares the +rocmX.Y version (and the torch 2.11 line for gfx pins) so changing the pinned family (e.g. rocm6.4 -> gfx1151) rebuilds and applies it. install_python_stack.py: - _ensure_rocm_torch: an explicit ROCm wheel-index pin now bypasses the NVIDIA-present / no-AMD-GPU / unreadable-ROCm gates (headless/container/CI cross-install), mirroring the explicit-CUDA-pin bypass in _ensure_cuda_torch. - Add _ensure_cpu_torch: an explicit CPU pin (FAMILY=cpu or /cpu URL) now has a repair path that reinstalls CPU torch over an existing CUDA/ROCm build on a standalone update (which skips install.sh's flavor enforcement). install.sh: - Pin torchvision/torchaudio companions alongside torch for the rocm7.2 / per-gfx index and the Strix reroute (those AMD indexes publish companions independently and a bare name can resolve a torch-2.12-built wheel, an ABI mismatch). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * torch-index override: classify CUDA pin by leaf; trim blank shell overrides _ensure_cuda_torch only overrode the NVIDIA-presence gate for *any* pinned index, so a non-CUDA mirror URL (or a ROCm/CPU pin) on a non-NVIDIA host with ROCm torch could force a CUDA reinstall over a working ROCm venv. Add _explicit_cuda_torch_index_url() (leaf cu*), matching the ROCm/CPU helpers, and gate on it instead. install.sh::get_torch_index_url treated a whitespace-only UNSLOTH_TORCH_INDEX_URL / _FAMILY as authoritative (yielding an invalid index), unlike the Python .strip() and PowerShell IsNullOrWhiteSpace paths; trim leading/trailing whitespace first. * install: honor pinned torch index over CVD/GPU gates and fix leaf-based ROCm classification - install_python_stack.py: an explicit cu* pin now clears the CUDA_VISIBLE_DEVICES empty/-1 hide gate as well as the NVIDIA-presence gate, so CVD=-1 UNSLOTH_TORCH_INDEX_FAMILY=cu128 studio update repairs to CUDA wheels (parity with install.sh's get_torch_index_url override, which skips all GPU probing). Unpinned CVD=-1 still skips. - install_python_stack.py: _ensure_cpu_torch installs the bounded _CPU_TORCH_PKG_SPEC instead of a bare torch/torchvision/torchaudio trio; the /cpu index now also serves torch 2.11+, which is outside the supported <2.11 range. - install.sh: the torch>=2.11 constraint case matches the index leaf (rocm7.2|gfx*) instead of the whole URL, so a mirror base path containing a gfx/rocm7.2 segment with a cu*/cpu family is not false-matched onto the 2.11 line. - setup.ps1: the stale-venv check expects rocm torch only for arches the install path maps to a repo.amd.com wheel index; an unmapped/unreadable arch installs CPU, so a correct CPU venv is no longer marked stale. - Tests for each of the above. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten pinned torch-index override edge cases - install.sh: trim whitespace-only UNSLOTH_TORCH_INDEX_URL/_FAMILY before the _torch_index_pinned guard, matching get_torch_index_url, so a blank override no longer skips the WSL bootstrap and Radeon/Strix reroutes while detection still picks the normal index. - install.sh / install.ps1 / setup.ps1 / install_python_stack.py: force the torch 2.11 floor only for the gfx families with the <2.11 _grouped_mm bug (gfx120X-all, gfx1151, gfx1150). A pinned override to gfx110X-all/gfx90a/gfx908 stays on the default range, matching the automatic AMD path. - install_python_stack.py _ensure_cuda_torch: treat an untagged CUDA build under a CUDA pin as a family mismatch (reinstall), and match cuXXX pins narrowly (cu + digits) so a custom/current mirror leaf no longer forces CUDA over a CPU/ROCm venv. - install_python_stack.py _ensure_rocm_torch: reinstall when an explicit ROCm pin names a different ROCm family than the already-installed ROCm torch (the ROCm analogue of the CUDA cuXXX mismatch repair). Adds tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: fix second-order edge cases in pinned torch-index ROCm/CUDA handling Parse the ROCm torch probe positionally so an empty HIP marker is kept: CPU/CUDA torch no longer reads as HIP, so the ROCm reinstall is not skipped. Emit one "<marker>|<version>" line (like the CUDA probe) for a robust parse. Limit the gfx torch 2.11 expectation to the install allowlist (gfx120X-all/gfx1151/gfx1150). A pinned gfx110X-all/gfx90a/gfx908 index stays on the default <2.11 specs, so a correct 2.10+rocm wheel is no longer judged a mismatch and force-reinstalled every update. Distinguish an AMD per-arch wheel (three-part +rocmA.B.C) from a generic pytorch.org wheel (two-part +rocmA.B): a gfx per-arch pin over a generic 2.11 wheel now reinstalls the per-arch wheel, while an already-installed per-arch wheel is not re-flagged (no reinstall loop). Mirror all of the above in setup.ps1 via new Test-RocmGfx211Leaf / Test-CudaFamilyLeaf / Get-RocmPinStaleTags helpers, reused by both the install-spec path and the stale-venv check so they cannot diverge again. Require a digit after "cu" (^cu[0-9]) in setup.ps1, install.ps1 and install.sh so a mirror leaf like /custom or /current is not branded CUDA and does not rebuild the venv every run. Add tests: CPU/CUDA probe -> has_hip_torch False; gfx110X-all pin + 2.10 wheel not stale; gfx1151 pin + generic 2.11 wheel stale; gfx1151 pin + per-arch wheel not stale; /custom and /current not CUDA; plus cross-language allowlist and cu-digit parity guards, and a PowerShell unit test for the new setup.ps1 helpers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix ROCm/gfx pin case normalization, ROCm-tag requirement, and CUDA-leaf classification Normalize torch-index leaves to lowercase before the gfx*/rocm*/cu* allowlist matches so the canonical gfx120X-all (capital X) gets the torch 2.11 floor in install.sh (leaf, flavor and repairable helpers). Require an installed +rocm local tag before a rocmX.Y or non-2.11 gfx pin is judged satisfied in setup.ps1 Get-RocmPinStaleTags and the Python _rocm_pin_family_mismatch, so an untagged CPU/CUDA wheel never leaves the pin unapplied. Classify a leaf as CUDA only via ^cu[0-9]: the Python _TORCH_BACKEND derivation now uses _is_cuda_family_leaf, and install.sh brands cuda only on cu[0-9]* (unset on an unknown /current /custom mirror leaf) so the stack probes the GPU instead of skipping ROCm repair. Add bash, Python and PowerShell tests for capital gfx120X-all floor, current/custom not-cuda, and untagged-wheel ROCm pins. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: converge torch-index pin detection via a per-venv marker Introduce a torch-index MARKER that records the exact wheel --index-url used after each successful torch install, so `unsloth studio update` / repair makes the "did the pinned index change?" decision by an EXACT string compare rather than inferring it from the wheel +rocm/+cu version tag. The tag cannot encode the AMD per-arch gfx family (two 2.11 gfx indexes both install +rocm7.13.0), so the tag heuristic missed a gfx1151 -> gfx120X-all switch and a custom-URL swap. Marker path is per-venv (.unsloth-torch-index), one line = the resolved index URL, written atomically (temp + rename). Path, format and normalization are shared across all four installers (install.sh, install_python_stack.py, setup.ps1, install.ps1). - Reapply gfx pins on a per-arch target change: the marker's exact compare reinstalls when the pinned index differs, even when both wheels share a tag. - Honor custom ROCm URL pins during repair: an explicit index whose leaf is not rocm/gfx/cu/cpu (e.g. simple, current) now reinstalls torch VERBATIM from the pin when it differs from the marker ("URL wins verbatim"). - Align the KNOWN-2.11 rocm/gfx set to exactly rocm7.2 plus the gfx allowlist gfx120x-all/gfx1151/gfx1150 in every language; stop treating an unknown newer rocm (rocm7.3, which does not exist) as the 2.11 line speculatively. Backward compatible: with no marker (old venvs, torch installed out-of-band) the existing +rocm/version-tag heuristics still decide, and a matching marker never reinstall-loops. A cu128 CUDA pin stays a CUDA pin; custom and current leaves are not CUDA. Adds marker tests (py/sh/ps) plus cross-installer parity checks. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: keep the torch-index marker additive to flavor validation Three narrow fixes in the marker-based stale-venv detection: - setup.ps1: a matching marker no longer overwrites the detected installed flavor. The marker compare is now an additional rebuild trigger, so a stale wheel (torch swapped to a +cpu build while the marker still records a cuXXX pin) is still caught by the flavor check instead of being masked as up to date. - setup.ps1: a supported AMD arch carrying CPU torch is no longer marked stale and wiped. The downstream AMD Windows ROCm override upgrades CPU torch to ROCm in place, so wiping first would delete the venv and abort with "Virtual environment not found". Only a genuinely wrong CUDA wheel still rebuilds. - install.sh: the Radeon --find-links path records its repo.radeon.com base in the marker instead of the generic pytorch.org ROCm fallback index, so a later pin to that generic family correctly reinstalls rather than comparing equal. Mirrors install.ps1/setup.ps1, which already record the real AMD index. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: honor custom pins and repair pinned venvs in place Four follow-ups to the torch-index marker work: - install_python_stack.py: _ensure_cuda_torch/_ensure_rocm_torch now bail when an explicit custom-index pin names no known torch family, so a verbatim URL override (a private/simple mirror) is not clobbered by auto-detected CUDA/ROCm wheels before _ensure_verbatim_torch_index applies it. - install_python_stack.py: the ROCm marker is additive, not a substitute -- a matching marker still runs the family/version check so a wheel swapped after the marker was written is caught. Mirrors setup.ps1. - setup.ps1: a stale venv under an explicit pin, whose torch still imports, is repaired in place (force-reinstall torch from the pin in the dependency pass) instead of wiped. The wipe path only delegates to install.ps1, so on a direct update it stranded the user at "Virtual environment not found" instead of applying the new pin. A broken venv or unpinned drift still wipes/delegates. - install.ps1: when a pinned ROCm install fails over to a CPU base, the marker now records the CPU index actually used instead of the ROCm pin, so the next managed setup does not see CPU torch under a ROCm pin and abort as stale. * setup.ps1: keep the ROCm CPU-fallback force line the pr5940 test guards 5c93ffd4 folded the pin-change force-reinstall into the ROCm CPU-fallback condition on one line, so the exact literal that test_pr5940_followups.py checks (if ($ROCmCpuFallback) { $cpuForce = @("--force-reinstall") }) no longer appeared and the test failed. Split the two conditions into separate if lines: the ROCm fallback line is restored verbatim and the pin-change force is its own line. Both still set $cpuForce to the array, so @splat passes one arg. * install: honor exact CUDA/custom index URL pins in the torch-index marker Address three Codex review findings on the torch-index marker mechanism: - install.sh: after the ROCm CPU repair reinstalls torch from the generic $TORCH_INDEX_URL, record that as the marker source. A Radeon --find-links install set _TORCH_MARKER_INDEX_URL to its repo.radeon.com base earlier, so leaving it made the marker misreport Radeon wheels and a later Radeon pin would compare equal and skip a needed reinstall. - install_python_stack.py: _ensure_cuda_torch now consults the exact-URL marker (_marker_pin_mismatch) when the installed +cuXXX tag matches the pinned leaf, so a same-leaf CUDA mirror change (official cu128 to an internal cu128 mirror) is reinstalled and re-recorded instead of skipped. - _normalize_index_url / _normalize_family_leaf (install.sh, setup.ps1, install_python_stack.py): lowercase only KNOWN wheel-family leaves (rocm/gfx/ cpu/cuXXX) so gfx120X-all still matches gfx120x-all, while a custom (unknown-family) leaf keeps its case so a verbatim URL pin like /Current does not compare equal to /current. Tests updated to assert the refined behavior. * install: fix 3 torch-index marker edge cases (CPU mirror pin, Radeon leaf, migrated venv) Addresses three review findings on the torch-index override path: 1. CPU index URL change on an already-CPU venv. _ensure_cpu_torch returned early whenever torch was already a CPU build, so a standalone update that moved the pin (official /cpu -> a private UNSLOTH_PYTORCH_MIRROR /cpu, same +cpu tag) never reinstalled. It now consults the exact-URL marker and reinstalls only when _marker_pin_mismatch reports a different index, mirroring the CUDA/ROCm same-family handling. A matching marker (or none) still leaves CPU torch untouched, so there is no reinstall loop. 2. Radeon find-links directory misclassified as a pip ROCm family. A repo.radeon.com/.../rocm-rel-7.2.1 leaf starts with "rocm" but is a find-links listing, not a pip --index-url. The old startswith(("rocm", "gfx")) test routed it into a --index-url reinstall that fails against find-links. New _is_pip_rocm_family_leaf gates on ^rocm\d / gfx (matching install.sh's rocm[0-9]* and setup.ps1's ^(rocm[0-9]|gfx)), so a Radeon URL routes to the verbatim/marker path instead. 3. Migrated venv rewriting its marker to a pin it did not install. install.sh and install.ps1 write the marker unconditionally, so a migration that preserves existing torch recorded the newly requested pin and a later update then found a matching marker and skipped the reinstall the pin needs (e.g. a per-arch gfx1151 -> gfx120X-all switch, identical +rocm tag). Both now track _TORCH_INSTALLED_THIS_RUN and write the marker only when torch was actually installed or repaired this run. Also add Get-NormalizedFamilyLeaf to the setup.ps1 helper-extraction list in test_torch_index_marker.ps1 (it was added to setup.ps1 and the shell test in an earlier round but missed here) and add two unit tests covering findings 1 and 2. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: keep pinned torch repairs on the pinned index Two fixes for explicit index pins (UNSLOTH_TORCH_INDEX_FAMILY / _URL): 1. install_python_stack.py's repair paths ran uv without clearing the inherited uv index env vars. uv resolves the default index (--index-url or --default-index) at the LOWEST priority, so a UV_INDEX or UV_EXTRA_INDEX_URL mirror in the environment won for any package it served: a cu128-pinned repair could install torch from the mirror and then record the cu128 marker it never used. Verified empirically: with UV_EXTRA_INDEX_URL=.../cu126 exported, uv pip install torch --index-url .../cu128 resolves torch 2.13.0+cu126. Strip the four uv index env vars for pinned-index commands only, mirroring the gate install.sh, install.ps1 and setup.ps1 already have; non-pinned installs keep the user's mirror. 2. install.ps1 routed any pinned leaf matching rocm* through the ROCm --default-index path, so a custom find-links leaf like rocm-rel-7.2.1 was treated as a PEP 503 ROCm index and could silently fall back to CPU torch on resolution failure. Require a digit after rocm, matching install.sh's rocm[0-9]* and install_python_stack.py's ^rocm\d. Adds parity + unit tests for both (11 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: keep pinned repairs off UV_TORCH_BACKEND and narrow setup.ps1's rocm pin match Round 2 of the pinned-index hardening: 1. _build_uv_cmd converted UV_TORCH_BACKEND into --torch-backend before the new env isolation could act, and uv's torch backend redirects torch resolution to its own per-backend index even when --index-url is given (verified: a cu128-pinned dry run with UV_TORCH_BACKEND=cpu resolves torch 2.13.0+cpu). Pinned-index commands now never receive the flag and UV_TORCH_BACKEND joins the stripped env vars, so uv cannot re-read it. 2. setup.ps1's pinned reroute had the same bare rocm* glob install.ps1 had: a custom find-links leaf like rocm-rel-7.2.1 was routed through the ROCm --index-url path instead of the verbatim unknown-pin path. Now requires a digit after rocm, matching install.ps1, install.sh and _is_pip_rocm_family_leaf. 3. The marker test's case-normalization checks used -eq, which is case-insensitive in PowerShell, making them vacuous, and the unknown-leaf expectation was written lowercased while the implementation deliberately preserves custom-leaf case. Tightened to -ceq with the case-preserving expected value. Adds unit + parity tests for 1 and 2 (5 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: extend the pinned-index guards to every remaining surface Round 3 of the pinned-index hardening, closing the same holes on the surfaces the earlier rounds missed: 1. install.sh's pinned-install env scrub now clears UV_TORCH_BACKEND (uv's torch backend redirects torch resolution to its own per-backend index even against --default-index), and both PowerShell wrappers clear it in their pinned-install scrubs, matching install_python_stack.py. 2. setup.ps1's marker stale check still classified any rocm* leaf as a PyTorch ROCm family while the install selection is digit-gated, so a custom rocm-current / rocm-rel-7.2.1 pin stale-compared as not-rocm vs rocm and force-reinstalled on every studio update. The stale check now uses the same ^rocm\d gate. 3. install_python_stack.py's pinned-command scrub also strips PIP_EXTRA_INDEX_URL for the pip fallback: pip adds the env extra index in addition to --index-url, so an inherited mirror could satisfy torch off the pin while the marker recorded the pinned URL. PIP_INDEX_URL needs no strip since the explicit --index-url flag overrides it. Parity + unit tests extended (4 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: scrub find-links and carry the pinned scrub through pip fallbacks Round 4 of the pinned-index hardening: 1. UV_FIND_LINKS joins every pinned-install scrub (install.sh, install.ps1, setup.ps1, install_python_stack.py): uv's --find-links locations can satisfy torch off the pinned index the same way an extra index does. 2. setup.ps1's Fast-Install restored the scrubbed vars in its finally BEFORE the pip fallback ran, and never touched the pip env vars at all, so a failed uv attempt fell back to python -m pip with an inherited PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS able to win over the pinned --index-url. The scrub now wraps the whole function (uv attempt + pip fallback) and includes the pip vars; restore happens after both. 3. install_python_stack.py's scrub also strips PIP_FIND_LINKS for its own pip fallback, completing the PIP_EXTRA_INDEX_URL fix from round 3. Parity tests extended (2 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: digit-gate rocm leaves in marker normalization and ROCm side effects Round 5 of the pinned-index hardening (three custom-rocm-leaf edge cases): 1. _normalize_family_leaf lowercased every leaf starting with rocm, so a custom mirror leaf like rocm-Current compared equal to its lowercase form and a case-only pin change was skipped. URL paths can be case-sensitive. The rocm prefix is now digit-gated (rocm[0-9]*, matching _is_pip_rocm_family_leaf) in install.sh, setup.ps1 and install_python_stack.py, so only true family leaves (rocm7.2) are lowercased; a custom rocm-* leaf keeps its case. 2. setup.ps1 Test-MarkerPinMismatch compared normalized URLs with -ne, which is case-insensitive in PowerShell, so a case-only marker change (Simple vs simple) was treated as matching and the reinstall skipped. Now -cne. 3. install.sh gated the AMD bitsandbytes install and the "repair ROCm torch" --default-index reinstall on a bare whole-URL rocm glob, so a custom CPU/CUDA/private index whose leaf merely starts with rocm (rocm-current) was force-repaired from the wrong ROCm-only path whenever torch.version.hip was empty. Both now gate on _torch_index_is_rocm_family, computed once from the digit-gated leaf (rocm[0-9]*/gfx*). Tests: 4 new parity assertions plus 2 case-sensitivity marker checks. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: apply an explicit custom torch-index pin on the first update Round 6: an explicitly-set custom (unknown-family) UNSLOTH_TORCH_INDEX_URL was silently ignored on the first `studio update` of a venv that predates the marker feature, on both platforms, because the no-marker case was treated as "do nothing" and the version-tag heuristics cannot judge an unknown leaf. 1. install_python_stack.py _ensure_verbatim_torch_index now reinstalls verbatim when the marker is ABSENT (None), not only when it differs, and short-circuits only when the marker already records this exact pin. It then writes the marker, so every later update is a no-op. A user who did not set the override gets pin=None and is untouched, so an out-of-band torch install is never clobbered. 2. setup.ps1: for an unknown-family pin on a marker-less venv the stale-venv check now sets PinChangedForceReinstall so the torch block reinstalls in place from the pin. It deliberately does NOT set shouldRebuild, which would wipe the venv and strand a direct `studio update`. 3. setup.sh (the Linux `studio update` entry point) skipped install_python_stack.py entirely when unsloth was already current, so the marker-driven reinstall (both the verbatim custom pin and the cu/rocm flavor and family-change repair, e.g. gfx1151 to gfx120X-all) never ran. It now forces the dependency pass when a torch-index pin env var is set; the pass is idempotent and no-ops when the marker already matches. This mirrors setup.ps1's stale-venv pre-check. Tests: 3 new parity assertions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: expect first-update reinstall for a no-marker custom index pin Follow-up to d671d8fb2: _ensure_verbatim_torch_index now applies an explicit unknown-family URL pin verbatim on the first update when the marker is absent (instead of no-op), so the old test_verbatim_custom_url_no_marker_is_noop assertion was stale. Rewritten as test_verbatim_custom_url_no_marker_reinstalls_once: asserts the one verbatim reinstall from the pinned URL, that the marker is written, and that a second call with the pin still set is idempotent (no reinstall loop). * install: gate the pinned update pass on the marker and record a pin baseline Round 8, two follow-ups to the round-6 first-update pin fix: 1. setup.sh forced the full dependency pass on EVERY `studio update` while a torch-index pin stayed exported, even after the marker already recorded the same pin, turning quick updates into the expensive pass every time. It now probes install_python_stack.py --torch-pin-needs-apply (which reuses the exact marker normalization) and forces the pass only when the pin is not yet applied (marker absent or different); an already-applied persistent pin keeps the fast path. A probe error fails safe toward running the pass. setup.ps1 gets the same probe in its fast path for parity. 2. A known-family full-URL pin on a venv predating the marker (e.g. an installed cu128 build and UNSLOTH_TORCH_INDEX_URL pointing at a same-family mirror) left the marker absent forever: the _ensure_* helpers deliberately do not force a multi-GB reinstall of identical-family wheels on an old venv, so nothing recorded the pin and every update re-entered the pass. _record_torch_index_pin_baseline now records the resolved pin as a baseline after the ensure sequence when the family already matches and no marker exists, so the pin is tracked (a later genuine change is detected and applied) and the update loop is broken, without the redundant reinstall. Tests: 3 new baseline unit tests, 4 new parity assertions, and the CLI probe. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.sh: keep the pin probe's exit 1 from killing the update under set -e The --torch-pin-needs-apply probe deliberately exits 1 for the common steady-state answer (pin already recorded, keep the fast path), but it ran as a bare command under set -euo pipefail, so the whole studio update aborted before the exit code was even captured. Absorb the status with || _PIN_NEEDS_APPLY=$? and pre-seed 0 so all three outcomes route as documented: 0 runs the pass, 1 keeps the fast path, anything else fails safe into the pass. Parity test asserts the guard. * install: strip pin credentials, disable uv config discovery, bound verbatim installs Four verified fix groups from a 12-reviewer audit of the torch-index override feature, each reproduced before fixing: 1. Credential persistence: all four marker writers stored the raw pin URL, so an authenticated pin (https://user:token@mirror/simple) persisted its credentials in .unsloth-torch-index (mode 0644 under a default POSIX umask) and install_python_stack.py printed pin URLs verbatim in repair messages. Userinfo is now stripped before persisting and in every log/substep that interpolates a pin, via lockstep helpers (_strip_index_url_credentials in install.sh / install_python_stack.py, Remove-IndexUrlCredentials in install.ps1 / setup.ps1). The three normalizers strip too, so an OLD marker that already carries credentials still compares equal to the same pin: no reinstall loop on upgrade. Query strings deliberately stay in the marker; two indexes distinguished only by query must not compare equal. 2. uv configuration discovery beat the explicit pin: with a discovered uv.toml declaring torch-backend = "cpu" or a [[index]] entry, uv 0.10.12 resolves torch 2.13.0+cpu against an explicit --index-url/.../cu126 pin; UV_NO_CONFIG=1 restores +cu126 (reproduced both ways). The pinned-install scrub in all four installers now sets UV_NO_CONFIG=1 and drops UV_CONFIG_FILE. 3. The verbatim custom-index update path installed a bare, unconstrained torch trio while fresh installs from the same unknown-leaf pin apply the supported range; _ensure_verbatim_torch_index now installs the bounded trio spec, closing the fresh-vs-update asymmetry. 4. Query-bearing pins (.../cu128?token=x) classified by raw leaf split and force-reinstalled on every update (the installed cu128 never equals cu128?token=x). Query/fragment are now stripped before leaf classification in all four implementations; the marker comparison keeps the query per (1). Rejected after verification (no change): the pin-baseline record cannot produce a wrong later decision (every pin change still mismatches and reinstalls from the new pin); the venv temp-file symlink scenarios require an attacker who already owns the environment; pathological inputs like " / cu128 / " have no realistic caller and fail loudly. Parity, stack, rocm-support, marker (sh + ps1), pin-stale, index-url and flavor suites all pass (455 python + full shell/ps1 batteries). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: harden custom-pin repair against clobber, broken torch, and pip config Four follow-ups to the pinned-index audit fixes: 1. setup.ps1 routed an unknown-leaf custom pin through the CUDA branch with a bare torch trio while install.ps1 (fresh) and the Python verbatim path bound the supported range; the pinned unknown-leaf route now applies the same torch>=2.4,<2.11.0 bound. Known cu* leaves and unpinned runs are unchanged. 2. The final torch safety pass could not repair a clobbered unknown-family pin: intermediate dependency steps can pull torch from PyPI (the pass exists for exactly that reason), but the verbatim helper short-circuited on marker==pin and no flavor tag exists to probe. The helper now keeps a per-run snapshot of the installed trio (taken after a verbatim reinstall or on the first matching-marker pass) and reinstalls from the pin when the final pass sees the trio drifted. Probe failure skips the comparison; a reinstall refreshes the snapshot, so no loop. 3. _record_torch_index_pin_baseline could freeze a known-family pin as applied on a venv whose torch is missing or broken (every family helper returns without reinstalling when its probe fails), making --torch-pin-needs-apply report done forever. The baseline now probes the installed flavor and records only on a match: a cuXXX pin requires the matching +cuXXX tag, cpu requires a cpu build, rocm/gfx requires hip; probe failure records nothing. 4. The pinned pip fallback stripped PIP_* env vars but user/site pip config files still applied (a configured global.extra-index-url can satisfy torch off the pin). PIP_CONFIG_FILE is now pointed at the null device for pinned commands (pip loads no config files then), in _install_env_for_cmd and setup.ps1's Fast-Install pinned scrub. install.sh / install.ps1 have no pip fallback (uv-only), verified. Tests: 7 new rocm_support tests (snapshot reset fixture), 1 stack test, 2 parity tests. Full battery green (464 python, sh and ps1 suites). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: complete the pin-repair coverage across the fast path and platforms Three cross-platform follow-ups to the round-2 pin-repair fixes: 1. The --torch-pin-needs-apply probe only compared marker==pin, so a torch trio clobbered to the wrong family (a cpu wheel replacing cu128 via a later pip install) with a still-matching marker reported "already applied" and the _ensure_{cuda,rocm,cpu} repair never ran on the Linux fast path. The probe is now a testable _torch_pin_needs_apply() that also checks the installed flavor against a known-family pin (via a shared _torch_flavor_matches_pin() helper, so the baseline and the probe cannot drift). An unknown-family pin has no flavor to validate and a failed probe cannot prove drift, so both keep the fast path. 2. macOS ARM (real CPU/MPS torch, not NO_TORCH) never applied an unknown- family custom pin on update: both the verbatim path and the baseline returned on IS_MACOS while fresh install.sh honors the pin, so the marker was never written and setup.sh forced the dependency pass on every update forever. The guards are now IS_MAC_INTEL (Intel mac is already NO_TORCH), and the final pass applies the pin on macOS ARM. 3. The round-2 final verbatim repair sat in the step-13 sequence guarded not IS_WINDOWS, so on Windows a dependency step that clobbered torch after the pin was applied was masked by the matching marker (setup.ps1 does not re-validate the main venv's torch after calling this script -- verified). Step 13 now runs the verbatim snapshot-drift repair on Windows and macOS ARM too; the Linux-oriented cuda/rocm/cpu family helpers stay Linux-only. Tests: 13 new rocm_support cases (flavor drift, macOS ARM, Windows repair), parity updates. Full battery green (475 python, sh and ps1 suites). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: strip query tokens from the marker and tighten the pin-drift probe Four follow-ups to the round-3 pin-repair fixes: 1. The credential stripper feeding the torch-index marker and the logged repair messages dropped only user:pass@ userinfo, so a private feed that carries its auth token in the query string (.../simple?token=SECRET) persisted the token in the world-readable marker (mode 0644 under a default umask) and printed it in substep output. All four strippers (install.sh, install.ps1, studio/setup.ps1, install_python_stack.py) now drop the query and fragment before building the sanitized URL. A query is not part of a PEP 503 index's identity, so this also stops a rotated token from spuriously mismatching the marker and forcing a needless reinstall. 2. The --torch-pin-needs-apply fast-path probe accepted an untagged CUDA build (no +cuXXX local tag) under a specific cuXXX pin, but _ensure_cuda_torch reinstalls exactly that build to enforce the pin. The probe was more lenient than the repair, so the repair pass was skipped on the fast path. _torch_flavor_matches_pin now reports a mismatch for an untagged build under a cuXXX pin, forcing the pass. 3. The probe's ROCm branch accepted any HIP build for a rocm/gfx pin, while _ensure_rocm_torch decides a reinstall with the per-arch _rocm_pin_family_mismatch predicate (a generic +rocm7.2 wheel under a per-arch gfx pin, or a wrong ROCm version, is a mismatch). The probe now reuses that predicate, so it is as strict as the repair. This needs the installed torch version, so _probe_torch_flavor now returns (marker, cutag, version) and _torch_flavor_matches_pin takes the pin URL (extracting the leaf internally). 4. On Windows a known-family cu*/cpu pin is applied to the main venv by setup.ps1 before install_python_stack.py runs; a later dependency step can clobber it, and the GPU-aware _ensure_{cuda,cpu}_torch self-skip on Windows while the verbatim helper handles only unknown-family pins, so nothing repaired the clobber (setup.ps1 does not re-validate the main venv's torch afterward, verified). New _ensure_pinned_known_family_torch reinstalls a drifted cu*/cpu pin in the step-13 Windows/macOS-ARM branch; rocm/gfx per-arch specs stay owned by setup.ps1, unknown-family by the verbatim helper. A speculative ROCm 2.11 floor was also raised but is unreachable: the rocm7.2 index publishes no 2.x wheel below 2.11.0, and an unknown newer rocm is not floored speculatively. Tests: query/fragment strip cases in the sh + ps1 marker suites and the Python strip/marker tests; the tri-state helper and the probe/baseline harnesses moved to the (marker, cutag, version) flavor with matching versions; new probe cases (untagged CUDA, generic-rocm-under-gfx) and 8 _ensure_pinned_known_family_torch tests; a four-way query-strip parity assertion. Full battery green (1150 python, sh 26/26 marker, ps1 marker/flavor/pin-stale). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: reinstall markerless gfx pins and cap custom-index updates at torch 2.11 Two follow-ups from the pin-marker audit: 1. A markerless venv with a gfx per-arch 2.11 pin trusted the wheel version tag, which is byte-identical (+rocm7.13.0) across gfx120X-all / gfx1151 / gfx1150. A pre-marker install holding one gfx arch's wheel that is now pinned to a DIFFERENT gfx index was therefore never switched: _rocm_pin_family_mismatch returns no-mismatch for any three-part +rocm 2.11 wheel, and _ensure_rocm_torch's absent-marker branch fell through to that heuristic. _ensure_rocm_torch now forces a one-time reinstall when the marker is absent AND the pin leaf is a 2.11 gfx per-arch index; the reinstall writes the marker, so the next update compares exactly and does not loop (the correctly-pinned no-reinstall guarantee then comes from the exact marker compare, not the ambiguous tag). Non-gfx-2.11 pins (rocmX.Y, non-2.11 gfx) stay on the tag heuristic -- their tags are distinguishable. 2. The verbatim custom-index update path used _CUDA_TORCH_PKG_SPEC (torch <2.12.0) while a FRESH install of the same unknown leaf caps torch at <2.11.0 (install.sh's default TORCH_CONSTRAINT, and setup.ps1's custom-pin branch), so a private /simple mirror publishing torch 2.11 could upgrade a `studio update` to a state the fresh installer never produces. Added _CUSTOM_INDEX_TORCH_PKG_SPEC (torch>=2.4,<2.11.0), used only by the verbatim path; companions stay pinned for the same exclusive --index-url ABI reason as _CUDA_TORCH_PKG_SPEC (a bare name could pull a torch-2.12-built torchvision). _CUDA_TORCH_PKG_SPEC is unchanged (known-family cu/cpu repair correctly tracks install.sh's widened cu ceiling). Tests: 2 new markerless-gfx cases (one-time reinstall + marker write + no-loop second run, and the rocmX.Y absent-marker no-op), the pre-existing markerless gfx no-reinstall test flipped to assert the one-time reinstall (it had encoded the old tag-trusting behavior), and the custom-index bound assertions. 488 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: a matching marker must not mask a broken, clobbered, or misclassified torch Four round-6 follow-ups, all closing cases where a matching torch-index marker wrongly vouched for a torch that is not actually the pinned one: 1. _is_cuda_family_leaf matched cu+digits by PREFIX (^cu[0-9]), so a custom mirror leaf like cu128-private classified as CUDA family; the flavor check then compared the installed cu128 tag to the whole leaf cu128-private and forced a reinstall on EVERY update (never converging). The cu family is now matched EXACTLY (re.fullmatch cu[0-9]+), so a cu-suffixed custom leaf routes through the verbatim/unknown path with a stable marker. Mirrored in install.sh (_normalize_family_leaf: strip cu, require an all-digit remainder) and setup.ps1 / install.ps1 (^cu[0-9]+$). 2. _torch_pin_needs_apply returned False on a failed torch probe (missing or unimportable) under a matching marker, so setup.sh kept the fast path and a broken torch was never repaired. A failed probe now forces the pass: the marker cannot vouch for a torch that does not import, forcing is idempotent, and once torch imports again the probe succeeds and the forcing stops (self-resolving). Reverses the round-4 conservative choice for this case. 3. _ensure_verbatim_torch_index snapshotted the installed trio on the first pass with a matching marker and treated an unimportable torch (snapshot None) as "no drift, skip", so a torch clobbered to a broken state before the run was masked. A None snapshot now reapplies the pin. A torch clobbered to a WORKING-but-wrong build under an unknown-family pin remains undetectable from metadata (no flavor tag; reinstalling every update would be the loop this avoids) and is documented as a known limitation. 4. The step-13 Windows final repair reran only the verbatim (unknown-family) and known-family cu*/cpu paths, so a clobbered explicit rocm/gfx pin (the wheel setup.ps1 installed from AMD's per-arch index) was left in place. The branch now also runs _ensure_rocm_torch on Windows for an explicit rocm/gfx pin; it has a Windows path and no-ops when torch already links HIP, so it only reinstalls a genuinely clobbered ROCm venv (loop-safe). Tests: the round-4 failed-probe-trusts-marker test flipped to force the pass; new cases for the cu-suffix no-loop, the broken-torch verbatim reinstall, and the Windows rocm final-repair structure; item-2 exact-cu parity assertions. 490 passed. sh/ps1 marker + flavor + pin-stale suites all green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: repair Windows ROCm pins from the pinned URL and honor NO_TORCH Four round-7 review items, two of them regressions in the round-6 work: 1. _torch_pin_needs_apply ignored UNSLOTH_NO_TORCH. With a torch-index env var set and no marker, the failed-probe branch forced the dependency pass on every `studio update`, and the pass (which also honors NO_TORCH) never installs torch or writes a marker, so nothing could ever stop the forcing. It now returns False immediately under NO_TORCH: the pin only matters once torch is actually installed. 2. The step-13 Windows final repair (round-6) restored a clobbered explicit rocm/gfx pin by calling _ensure_rocm_torch, whose Windows path reinstalls from the arch AUTO-DETECTED via hipinfo, not from the pin. A user pinning a different gfx family or a private mirror was restored from the wrong source (and the wrong marker written), and a headless box was skipped entirely (the arch probe returns nothing). The repair now goes through _ensure_pinned_known_family_torch, which reinstalls from the PINNED url with the same per-arch floor setup.ps1 uses (2.11-line gfx leaves) or a bare trio (older arches, rocmN mirrors). It is gated on IS_WINDOWS since macOS ARM has no ROCm, and the existing flavor check keeps it loop-safe (a matching HIP wheel is left alone). 3. _ensure_verbatim_torch_index's broken-torch check (round-6) used "_installed_trio_snapshot() is None", but that helper reports a REMOVED torch as "torch==absent" (a non-None tuple) and a broken import as the stale on-disk version, so a missing or unimportable torch under a matching marker was read as "no drift" and skipped. The matching-marker path now confirms torch health with an import probe (_probe_torch_flavor): a torch that does not import reapplies the pin, while a healthy torch keeps the snapshot-based intra-run drift detection. 4. A unit test for _ensure_cpu_torch did not pin NO_TORCH False like its siblings, so a suite run with UNSLOTH_NO_TORCH=1 in the environment made the guard return early and the reinstall assertions fail spuriously. Tests: the round-6 broken-torch verbatim test re-encodes the non-None "torch==absent" snapshot case (the exact state the old "is None" check missed); new Windows-ROCm pinned-repair cases (reinstall from the pin, per-arch floor vs bare spec, matching-wheel no-op, off-Windows no-op); a NO_TORCH fast-path probe case; the parity test now asserts the Windows final branch does not auto-detect the ROCm index and that the helper reinstalls from the explicit pin. 494 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: floor the rocm7.2 index in the Windows pin repair; isolate marker tests Three round-8 review items, two of them downstream of the round-7 changes: 1. _ensure_pinned_known_family_torch gave a rocm<d> index leaf a bare torch/torchvision/torchaudio trio while flooring only gfx* leaves, so a Windows venv clobbered under an explicit rocm7.2 pin could reinstall an unbounded or ABI-mismatched trio from that exclusive --index-url. It now mirrors the spec the initial ROCm paths pin: the rocm7.2 floor for 2.11-line gfx leaves and rocm<d> leaves that serve torch 2.11, the <2.11 default for older rocm versions, and a bare trio only for older gfx per-arch leaves (which publish no floor), matching _ROCM_TORCH_PKG_SPECS / _ensure_rocm_torch. 2. test_verbatim_custom_url_no_marker_reinstalls_once called _ensure_verbatim_torch_index twice; the second call now hits the matching-marker health probe, and with pip_install mocked torch never becomes importable, so in a no-torch environment _probe_torch_flavor returned None and forced another reinstall, failing the idempotence assertion. The test now pins a healthy flavor so the idempotence check is about the marker, not ambient torch. 3. The TestEnsureRocmTorchMarker fixture patched os.environ per test but not _TORCH_BACKEND, which install_python_stack.py computes once at import from UNSLOTH_TORCH_BACKEND. A runner starting with a cuda/cpu backend made _ensure_rocm_torch early-return and skip the mocked repair these tests exercise. The fixture now neutralizes _TORCH_BACKEND so the marker tests are independent of the caller's installer-pin environment. Tests: the Windows floor-spec test now asserts a rocm7.2 mirror pin uses the rocm7.2 floor (not bare), plus a new rocm7.1 case that must fall back to the <2.11 default; the marker suite passes under a hostile UNSLOTH_TORCH_BACKEND=cuda / UNSLOTH_TORCH_INDEX_URL env. 495 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: apply same-flavor pin repoints, keep ROCm fallback nonfatal, bound custom companions Four round-9 review items, two of them regressions in the round-7 pin helper: 1. _ensure_pinned_known_family_torch returned as satisfied whenever the installed flavor matched the pin, so a same-flavor SOURCE change (one /cpu or /cu128 mirror to another, or a gfx1151 -> gfx120x-all per-arch switch, both carrying the same wheel tag) was never applied, while _torch_pin_needs_apply kept forcing the pass on the marker mismatch forever. It now also reinstalls when the marker records a DIFFERENT index of the same flavor, rewriting the marker so the next update matches (no loop), exactly as the Linux _ensure_{cuda,cpu}_torch helpers do. An absent marker on an already-matching venv is still left to the baseline recorder (no forced reinstall of a correct pre-marker venv). 2. That helper reinstalled a Windows ROCm pin with the FATAL pip_install, so when setup.ps1 had taken its CPU fallback (the pinned AMD index unavailable), the final repair re-hit the same missing index and aborted the whole install. The ROCm reinstall is now nonfatal (pip_install_try): on failure it leaves the CPU base in place and writes no ROCm marker, so the install completes -- matching _ensure_rocm_torch's Windows path. cu*/cpu pins stay fatal (authoritative source). 3. install.sh left torchvision/torchaudio bare for a pinned custom/unknown-leaf index (a private /simple mirror), unlike the Python update path's _CUSTOM_INDEX_TORCH_PKG_SPEC, so a mirror also exposing newer companion wheels could resolve a torch-2.12-built torchvision against the capped <2.11 torch. It now bounds the companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) for a custom leaf, gated on an empty _expected_torch_flavor_tag so known families keep their curated bare/floored companions. 4. install.sh's _expected_torch_flavor_tag matched cu[0-9]* by prefix, so a custom leaf like cu128-private classified as the cu128 family and force-reinstalled a correct +cu128 wheel on every run. It now requires exact cu+digits (routing the suffixed leaf to the custom path), matching the Python re.fullmatch(cu[0-9]+) and PowerShell, and feeding item 3's custom-leaf detection. Tests: new cases for the same-flavor marker-change reinstall, the nonfatal ROCm fallback (no marker on failure), the rocm7.2/older-rocm floor selection now split across the nonfatal path, cu-suffixed custom leaves in test_torch_flavor.sh, and the custom-leaf companion bounds in test_torch_constraint.sh. 497 python + 143 shell assertions pass; the marker suite still passes under a hostile UNSLOTH_TORCH_BACKEND=cuda env. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: bound custom-pin companions on the Windows setup path; isolate pin-probe tests Two round-10 review items: 1. setup.ps1's custom/unknown-leaf pin branch capped only torch ($cudaTorchSpec) and still asked the exclusive index for bare torchvision/torchaudio, so a private mirror that also serves newer companion wheels could install a torch<2.11 wheel alongside a torchvision>=0.26 / torchaudio>=2.11 built for a newer torch ABI, after which the marker records the pin as applied. It now bounds the whole trio (torch>=2.4,<2.11.0 / torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) for a pinned non-cu-family leaf, matching install.sh, install.ps1's fresh pinned install, and install_python_stack.py's _CUSTOM_INDEX_TORCH_PKG_SPEC. This completes the companion-bounds fix across all three installers; known cu* leaves keep bare specs (the family index bounds them). 2. The _torch_pin_needs_apply probe tests did not pin NO_TORCH False, so a test process launched with UNSLOTH_NO_TORCH=1 short-circuited the probe (the round-7 guard) and returned False for cases that expect the pass to run. The _needs_apply helper now patches NO_TORCH (default False) around the call, and the dedicated no-torch case passes no_torch=True explicitly. Tests: the cross-platform parity test now asserts setup.ps1 bounds the full trio (not just torch) for a custom leaf; the pin-probe suite passes under a hostile UNSLOTH_NO_TORCH=1 environment. setup.ps1 parses clean; 497 python + shell suites green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: bound custom rocm-* pins, redact diag tokens, snapshot custom pins before base update Three round-11 review items, all reproduced before fixing: 1. install.sh's custom-index companion bounds gated on _expected_torch_flavor_tag returning empty, but that helper returned "rocm" for ANY rocm* leaf, so a custom mirror whose leaf starts with rocm but is not a pip family (a private rocm-current mirror, a Radeon find-links rocm-rel-7.2.1) escaped the bounds and installed bare torchvision/torchaudio. It now digit-gates rocm to rocm[0-9]* (matching the Python _is_pip_rocm_family_leaf ^rocm\d), so those custom leaves return "" and the <2.11 companion caps apply; real rocm7.2 / gfx per-arch indexes still classify as rocm. 2. _tauri_torch_index_family classified by the raw last path segment, so a pinned URL carrying auth in the query (.../rocm7.2?token=SECRET) had the token echoed verbatim into the emitted [TAURI:DIAG] line. It now strips query/fragment before classifying (mirroring the marker/log credential stripping), so no token reaches the diagnostic output; as a side effect .../cu128?token=x now classifies as cu128 instead of auto. 3. On studio update, the core package step (a newer unsloth can require a torch the custom pin does not satisfy, pulling a default PyPI trio) runs BEFORE the step-2b verbatim check, which then recorded the already-clobbered trio as the baseline for a matching marker and left the pin unapplied. A new _capture_verbatim_baseline() records the pre-clobber trio before the core step, so the verbatim pass detects the drift and reapplies the pin. Captures only for a matching custom pin with importable torch; a mismatched/absent marker or broken torch is left to _ensure_verbatim_torch_index. Tests: _expected_torch_flavor_tag rocm-current / rocm-rel cases; _tauri_torch_index_family token/fragment redaction with a no-leak regression guard; _capture_verbatim_baseline record/skip cases plus an end-to-end clobber-detection scenario; a structural guard that the capture runs before the core step. 501 python + shell suites pass; install.sh bash -n clean, shellcheck unchanged from base. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: match rocm family leaves exactly, enforce the rocm7.2 torch line, repair a broken pinned torch A pinned index is a pip ROCm --index-url family only when its leaf is an exact rocm<digits> / rocm<digits>.<digits> (rocm7.2) or a gfx* per-arch leaf. The prior ^rocm[0-9] prefix match also caught suffixed private-mirror leaves (rocm7.2-private, rocm7-current), routing them through the ROCm/companion-family path instead of the verbatim pin: the companion bounds were skipped and, on a pre-marker venv with a compatible +rocm wheel, the pin was never applied. Match the family exactly through one shared helper at every site: - install_python_stack.py: _is_pip_rocm_family_leaf (re.fullmatch), plus the two other loose gates it feeds (_normalize_family_leaf, _torch_flavor_matches_pin). - install.sh: a new _is_pip_rocm_family_leaf routes _expected_torch_flavor_tag, _torch_index_repairable, _normalize_family_leaf and the ROCm side-effect gate. - setup.ps1: a new Test-PipRocmFamilyLeaf routes Get-NormalizedFamilyLeaf and both pinned reroutes; install.ps1 anchors its reroute regex. _rocm_pin_family_mismatch (and its setup.ps1 mirror Get-RocmPinStaleTags) compared only the ROCm version, so a +rocm7.2 wheel whose torch release drifted off the 2.11 line (2.12/2.13 from an out-of-band upgrade or a custom rocm7.2 mirror) satisfied the family check while violating _ROCM_TORCH_PKG_SPECS['rocm7.2'] (torch>=2.11,<2.12). Flag it stale so the repair reinstalls to floor; >=2.11 alone is not enough, so the release is compared exactly against the 2.11 line for a KNOWN-2.11 rocm pin. _ensure_pinned_known_family_torch returned on a failed import probe, but _torch_pin_needs_apply forces the dependency pass on that same failed probe: a broken torch under a known-family pin was left in place and the pass was forced on every update. Treat an unimportable torch as drift and reinstall the pinned trio (the spec and marker derive from the pinned leaf, not the absent flavor); once it lands the probe succeeds and the fast path returns. Tests: exact-match cases across test_torch_flavor.sh, test_rocm_support.py, test_cross_platform_parity.py and the two .ps1 helper suites; the rocm7.2 release-line and broken-probe-reinstall cases; extraction lists updated for the new helpers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: anchor the PS pinned-ROCm floor gate and bound install.ps1 custom-pin companions Round 12 made every family CLASSIFIER exact, but the Windows install-flow floor gate reads $_pinRocm211 directly from the raw pinned leaf with an unanchored -match '^rocm(\d+)\.(\d+)' BEFORE any exact classification runs. A suffixed custom leaf (rocm7.2-private) matches that rocm7.2 prefix, so it takes the 2.11-floor branch and is force-routed through the ROCm install path before the exact-match elseif can send it to the verbatim install. Anchor the match ($) in both install.ps1 and setup.ps1 so only an exact rocmX.Y leaf is floored; a suffixed or newer-suffix leaf falls through to the verbatim path. The Python floor selection is already exact (dict lookups gated on _is_pip_rocm_family_leaf), so only the two PS scripts needed this. install.ps1's custom (non-cu-family) pinned-torch install bounded torch>=2.4,<2.11.0 but left torchvision/torchaudio bare, so a private mirror serving newer companions could pull a wheel built for a newer torch ABI while the marker records the pin as applied. Bound both companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) when the leaf is not a cu<digits> family index (a cu index bounds its own resolution), matching setup.ps1's Test-CudaFamilyLeaf gate and _CUSTOM_INDEX_TORCH_PKG_SPEC. Tests: parity guards for the anchored floor gate in both PS scripts and for install.ps1's bounded custom-pin companions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten comments in the torch-index-override paths Collapse the verbose comment and docstring blocks added across the installer scripts and their tests to fewer, clearer lines without changing behaviour. Remove a duplicated CUDA-spec comment block. Comments/docstrings only; no code changes (AST-verified). * install: repair a broken pinned torch on Linux, strip trailing slash in tauri family, count the final step _ensure_cuda_torch / _ensure_cpu_torch returned on a failed import probe (torch present but unimportable). With an explicit CUDA/CPU pin, _torch_pin_needs_apply forces the dependency pass on that same failed probe, and the base package update does not force-reinstall an already-installed torch distribution, so the broken torch was left in place and the pass reran every update without repairing it. Treat a failed probe under a pin as drift and reinstall from the pinned index (the reinstall rewrites the marker and the next probe imports, so no loop). This is the Linux counterpart of the known-family repair fix. _tauri_torch_index_family stripped the query/fragment before classifying but not a trailing slash, so a token-authenticated pin like .../cu128/?token=x collapsed to .../cu128/ and fell through the exact-suffix */cu128 and */cpu arms to "auto". Strip a trailing slash too, mirroring _torch_index_url_leaf. The Windows / macOS-ARM final torch-repair step (_ensure_pinned_known_family_torch) runs a progress step that base_total never counted (the final-step increment was gated to Linux), so _STEP ran one past _TOTAL on those platforms. Add the missing increment. Tests: broken-probe reinstall for the CUDA (family and URL pins) and CPU paths; trailing slash / slash+token cases for _tauri_torch_index_family; a full-flow progress-count guard asserting _STEP == _TOTAL on Windows and Linux. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten comments in the torch-index-override paths * install: harden the torch-index pin across all four installers Redact index-URL credentials from captured install logs before they print on failure. uv/pip failure text embeds the failing --index-url verbatim, so a user:token@ or ?token= secret could leak into the console. Add a shared redaction pass (_redact_install_output / Redact-InstallOutput) wired into the error-output dump in install.sh, install.ps1, setup.ps1 and install_python_stack.py. Verbose mode still streams live uncaptured output, so it is intentionally left unredacted (developer opt-in). Trim trailing slashes on the PATH only for a verbatim UNSLOTH_TORCH_INDEX_URL override, preserving a ?query/#fragment token. A whole-URL rstrip corrupted a base64 token ending in "/", and a single-slash strip left .../cu128// classifying as an empty leaf. Add _trim_index_path_slashes / Trim-IndexPathSlashes and route the override through it; strip ALL trailing slashes in the backend-branding leaf classifier so a double slash still yields the real leaf. Reject a trailing-dot ROCm leaf (rocm7.) in the bash family validator so it matches Python re.fullmatch(rocm\d+(?:\.\d+)?) and the PowerShell regex: both the major and the minor must be non-empty digits, so rocm7. is a custom verbatim pin, not a pip ROCm family. Scrub PIP_NO_INDEX and PIP_INDEX_URL for a pinned install in the two installers that have a plain-pip fallback (install_python_stack.py, setup.ps1): PIP_NO_INDEX=1 makes the fallback ignore every index including the pinned --index-url, and PIP_INDEX_URL replaces it. install.sh and install.ps1 install via uv --default-index (which ignores pip config/env), so they are unaffected. Add unit tests (bash, Python, PowerShell) and cross-platform parity tests covering credential redaction, path-only slash trimming, the rocm7. validator, the double-slash leaf, and the PIP_NO_INDEX/PIP_INDEX_URL scrub. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: redact captured torch-install output and warn on a failed pinned ROCm repair Close a redaction gap the earlier pass missed: setup.ps1's direct `Fast-Install ... | Out-String` branches (ROCm from $ROCmIndexUrl, CPU/CUDA from $TorchInstallIndexUrl, plus the Triton and T5 sub-venv installs) printed the captured $output verbatim on failure, bypassing Redact-InstallOutput. A private index carrying userinfo or a ?token= in the pin could leak into Windows Studio setup logs. Route every `Write-Host $output` through Redact-InstallOutput. Warn on a failed pinned Windows ROCm reinstall in _ensure_pinned_known_family_torch: the branch printed "reinstalling from it" then called pip_install_try, but had no else, so a failure continued silently and left the user believing the pin was applied while the old CPU/wrong torch survived. Mirror the auto-ROCm Windows path and warn, telling the user to retry. * install: redact captured output on the pip fallback and optional-install failure paths The uv install path already redacted its captured output, but pip_install's pip fallback runs through run(), which printed result.stdout verbatim on failure, and _print_optional_install_failure did the same. A pinned --index-url carrying userinfo or a ?token= could still leak there when uv is unavailable or the pip fallback also fails. Route both through _redact_install_output. The verbose pip_install_try path stays raw (developer opt-in), matching the other installers. * install: split the survive-updates marker subsystem into a follow-up The torch-index override PR grew a persisted per-venv marker plus repair machinery (stale-pin detection, verbatim re-apply, update-time reinstall triggers) that roughly doubled it. That subsystem is orthogonal to the core feature and is being reworked in a follow-up (versioned/hashed marker, full-URL pin baseline), so it moves there wholesale instead of shipping twice. What this PR still does: UNSLOTH_TORCH_INDEX_URL / UNSLOTH_TORCH_INDEX_FAMILY pick the torch wheel index at install time in all four installers, with the exact rocm/gfx/cpu/cu leaf classification, the torch 2.11 floor for the per-arch AMD indexes, bounded companions for custom leaves, credential redaction of captured installer output, path-only slash trimming, and the uv/pip index env scrubs. Flavor-based repair keeps honoring the pin: a wrong family under an explicit pin still reinstalls from the pinned URL, and setup.ps1 repairs a pinned stale venv in place instead of wiping it. What moves to the follow-up: the .unsloth-torch-index marker file and its writers/readers/normalizers, exact-URL pin-change detection on update (same-tag gfx switches, custom-mirror repoints), the verbatim trio snapshot and clobber re-apply, the pin-baseline recorder, and the --torch-pin-needs-apply fast-path probe in setup.sh / setup.ps1. Their tests (the marker sh/ps1 suites, the stale-pin suite, and the marker classes in the rocm/cuda/parity suites) move with them; the removed code is preserved on a local archive branch to seed that PR. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: re-apply a ROCm pin over an existing HIP wheel via the version tag The subsystem split left an explicit ROCm/gfx pin unenforced on `studio update` whenever the venv already imported ANY ROCm torch: the pinned reinstall lived inside the `elif not has_hip_torch` branch, so a rocm6.4 to rocm7.2 switch, a gfx1151 pin over a generic +rocm7.2 wheel, or a broken 2.12+rocm7.2 drift never re-applied the pin. Restore the markerless half of that detection: _rocm_pin_family_mismatch compares the pinned leaf against the installed wheel tag (exact rocmX.Y compare, the 2.11 gfx per-arch allowlist, the untagged-wheel rule), the HIP probe emits "<hip_marker>|<version>" again so the installed tag is available, and _ensure_rocm_torch reinstalls from the pinned URL when the tag mismatches even though HIP torch is present. setup.ps1 mirrors it: the stale-venv check routes a pinned rocm/gfx leaf through Get-RocmPinStaleTags instead of collapsing it to a generic "rocm" flavor, and the existing pinned in-place repair (no wipe) applies the change. What still waits for the follow-up marker PR, by design: pin changes the wheel tag cannot see -- a per-arch switch between two 2.11 gfx indexes (identical +rocm7.13.0 tag), a custom-mirror URL repoint under the same family leaf, and unknown-family verbatim pins. Those need the persisted index record. Tests restored with the code: the _rocm_pin_family_mismatch table, the five update-path cases (older-rocm reinstall, gfx-over-pre-2.11 reinstall, matching-pin no-reinstall, non-2.11 gfx no-reinstall, gfx-over-generic-2.11 reinstall), the "|" probe-format guards, and the AST-extracted Get-RocmPinStaleTags suite for setup.ps1. * install: compare major-only rocm pins, redact URL fragments, bound pinned CPU trio Three review fixes on the restored pin-repair path. The family classifier accepts a major-only rocm<d> leaf (rocm7), but the mismatch comparators only parsed rocmX.Y, so a rocm7 pin fell through to the 2.11-line fallback and INVERTED both verdicts: an installed +rocm6.4 wheel compared as satisfied (pin never re-applied) while a matching +rocm7.2 wheel compared as stale (reinstall loop). Major-only pins now compare on the major alone in _rocm_pin_family_mismatch and Get-RocmPinStaleTags: rocm6.x under a rocm7 pin is a mismatch, any rocm7.x satisfies it, an untagged wheel never does, and a bare +rocm tag with an unreadable version is accepted (matching the existing lenient unreadable fallback). The output redactors scrubbed userinfo and ?query= values but not #fragments, so a pin like https://mirror/whl/cu128#token=secret leaked the secret in captured uv/pip failure text -- inconsistent with the URL handling itself, which already treats fragments as sensitive. All four redactors gain a URL-anchored fragment rule (anchored so a bare "# comment" line in tool output is never touched). setup.ps1's CPU branch installed a bare torch/torchvision/torchaudio trio; fine for the unpinned host default, but a PINNED cpu index routes through the same branch and the /cpu index serves newer torch, so a fresh pinned CPU install could land an unsupported trio that _ensure_cpu_torch then keeps (it accepts any CPU build). Under a pin the branch now installs the bounded trio mirroring _CPU_TORCH_PKG_SPEC (torch>=2.4,<2.12.0 and matching companions); the unpinned path is unchanged. Tests: major-only rows in the Python mismatch table and the AST-extracted setup.ps1 suite; fragment + query-plus-fragment + bare-hash-comment cases in all four redactor suites; a parity check that the pinned CPU trio bounds exist, are gated on the pin, and mirror the Python repair spec. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten comments in the torch index override paths * tests: track the moved pass-through inheritance in the gguf order check Main moved the llama_extra_args pass-through inheritance out of the GGUF branch into _resolve_inherited_extra_args, which runs before it, so the source-order assertion's "if request.llama_extra_args is None" anchor no longer exists inside the branch and the check failed after the main merge. The test now asserts the same property in the current shape: inheritance before the GGUF branch (a carried --no-mmproj still shapes the hub guard's companion requirement), and marker, hub guard, unload in order within the branch. Full file passes (32 tests). * tests: anchor the inheritance order check on the call, not the definition source.index("_resolve_inherited_extra_args(") matched the function definition, which always precedes the endpoint, so the ordering assertion was vacuously true. Anchoring on "= _resolve_inherited_ extra_args(" pins the first call site inside the load endpoint (line 4505), which is the statement whose position relative to the GGUF branch the test is meant to guard. 32 tests pass. * tests: align the gguf order test with main Main fixed the stale ordering assertion in PR 7252; adopting its version verbatim removes this file from the branch diff entirely and avoids a conflict on the next main merge. 32 tests pass. * install: bound the companion constraints to torch's window everywhere A full platform x vendor validation matrix over this branch surfaced a real trio mismatch on the cpu/mac paths: torch is capped <2.11 (installs 2.10.0+cpu) but the bare torchaudio companion resolves 2.11.0+cpu, because torchaudio 2.11 dropped its exact torch pin. Reproduced in a sandboxed end to end cpu install. torchvision still exact-pins torch and self-corrected. The default companion constraints are now bounded to torch's window (<0.26 / <2.11) and widen together with the cu* torch window (<0.27 / <2.12), so every leaf resolves a paired trio. Verified with uv dry-runs on the cpu, cu130, and rocm6.4 leaves (2.10.0/0.25.0/2.10.0, 2.11.0/0.26.0/2.11.0, 2.9.1/0.24.1/2.9.1) and a rerun of the sandboxed cpu install, which now lands torch 2.10.0+cpu with torchaudio 2.10.0+cpu. The Strix WSL reroute now also forwards UNSLOTH_TORCH_INDEX_URL and UNSLOTH_TORCH_INDEX_FAMILY into the rerouted 24.04 distro; dropping them silently reverted the child install to auto-detection, defeating the pin this branch introduces. test_torch_constraint.sh updated: the bounded companions must appear at the defaults and the custom-leaf block, no bare companion may remain, and the cu* widen must carry the companions with it. * install: harden the override path against reroute drift and credential leaks Review sweep focused on default-path idempotency found no defects on the unset path; these fixes cover the override path and failure reporting. install.sh: - The early WSL Strix Halo distro reroute now honors an explicit index pin (UNSLOTH_TORCH_INDEX_URL / _FAMILY): the pin is used in the current distro instead of probing the GPU and re-entering another distribution, matching the contract of the later Radeon and Strix guards. Whitespace only values do not gate, in parity with get_torch_index_url. - Verbose mode now streams installer output through the credential redactor; it previously bypassed the redaction the quiet path applies. The exit code survives the pipe via an rc file since the script runs under plain sh with no pipefail. - The kept-release fallback warning now strips credentials from the index URL before printing it. install.ps1: - Bounded torchvision and torchaudio next to every capped torch install (custom pin, ROCm CPU fallback, CUDA flavor repair). torchaudio 2.11 dropped its exact torch pin from the wheel metadata, so a bare companion beside torch<2.11 can resolve a mismatched 2.11.0 build, cu family indexes included. Mirrors the install.sh companion bounds. studio/install_python_stack.py: - The verbose failure path now redacts index URLs in pip and uv output before printing, matching every other output site in the file. All sh, ps1 and python installer test suites pass (the host-defaults suite has a known pre-existing failure unrelated to this change). * install: redact verbose Windows installer output and repair the parity tests Follow-ups to the override-hardening commit, from review: - install.ps1 Invoke-InstallCommand and setup.ps1 Invoke-SetupCommand now pipe verbose output through Redact-InstallOutput per record, and the three verbose Fast-Install torch call sites (ROCm, CPU, CUDA) do the same: uv and pip echo the pinned index URL, credentials included, in their errors, and verbose mode previously bypassed the redaction the quiet paths apply. ForEach-Object and Out-Host leave $LASTEXITCODE untouched, verified with a native command exiting 7 behind the pipe. - test_cross_platform_parity.py: the install.ps1 companion-bounds assertion now matches the implemented behavior (bounds on every index, no cu-family exemption, since torchaudio 2.11 dropped its exact torch pin) instead of requiring the removed $_pinCuLeaf gate. - test_rocm_support.py: the WSL reroute guard test slices the whole function body to its closing brace instead of a fixed 1200-character window, which the new pin-gate preamble had outgrown. 428 tests pass across the parity, install stack and rocm support suites; the sh and ps1 installer suites pass unchanged. * install: tighten comments in the torch-index and ROCm/CUDA repair paths * install: digit-gate the gfx family leaf and honor ROCm pins in the Windows repair Two review follow-ups on the override path: - The pip ROCm family predicate accepted ANY gfx-prefixed leaf, so a custom verbatim pin like /gfx-private classified as a ROCm family and enabled the ROCm-only side effects (AMD bitsandbytes, ROCm torch repair) on a mirror that may serve CPU/CUDA wheels. gfx now requires a following digit (gfx90a, gfx1151, gfx120X-all), consistently in install.sh, install_python_stack.py, install.ps1 (family gate and expected-flavor classifier) and setup.ps1, matching the strictness the rocm side already had (rocm7.2-private stays verbatim). The broader backend BRANDING globs are unchanged on purpose: radeon repo leaves (rocm-rel-X.Y) must still brand the rocm backend without being force-repaired as a family. - The Windows branch of the ROCm torch repair always installed from the public per-arch index, ignoring an explicit ROCm-family pin: after a pinned setup.ps1 install failed to a CPU base, the repair retried repo.amd.com instead of the pinned index. The branch now resolves _explicit_rocm_torch_index_url() first, uses it as the install index when set, and mirrors the Linux pin contract by skipping the NVIDIA and gfx-detection gates a pin is documented to override. Source-assertion tests updated to the tightened predicate and the new repair label. 1165 tests pass across the parity, install stack and studio install suites; the sh and ps1 suites pass; both PowerShell installers parse clean. * Remove scratch archives accidentally committed with the comment pass The temp/ archive copies of installer and test files were working scratch, not PR content, and inflated the diff by about nine thousand lines. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-20 00:58:52 -07:00
Daniel Han	b3c0259cff	Installer: preserve the previous torch release across every flavor and vendor on re-runs (#7250) ... * install: preserve the previous torch release across every flavor and vendor A re-run of curl | sh over an existing install was supposed to keep the user's validated torch release, but the pin required the old build's local flavor tag to match the freshly chosen index leaf. That gate was wrong in practice: a PyPI-sourced torch reports a BARE version (on Linux the PyPI wheel IS a CUDA build), which classified as cpu and never matched a cu leaf, so a healthy 2.10 on a cu130 host was silently moved to 2.11 (reproduced end to end); the same happened for any flavor drift such as cu128 to cu130 after a driver upgrade, and AMD ROCm leaves were excluded from preservation entirely. The rule is now release-based and flavor-agnostic: the probed previous release is pinned whenever it sits inside the final constraint window, and the pin installs from the freshly chosen index, so the flavor always follows the machine (NVIDIA cu*, AMD rocm/gfx, Intel/CPU, mac) while the release follows the user. The pin is evaluated AFTER every index and constraint decision including the Strix reroute, so raised floors (rocm7.2 / Strix gfx need torch 2.11 for the _grouped_mm fix) correctly reject an older release and win. UNSLOTH_TORCH_UPGRADE=1 still opts out, out-of-window releases are never kept, and probe noise never becomes a pin. The kept-release install with its range fallback (for indexes that do not carry the exact release) is factored into _install_torch_default_index and used by every --default-index torch path: the default NVIDIA/CPU/mac path and all three ROCm-index fallbacks, which previously bypassed the fallback. The Radeon-repo direct-wheel path keeps its curated per-arch wheel set (those wheels are already exact-pinned per rocm release). Platform coverage: install.sh serves Linux, WSL (including the WoA fallback), and macOS for all vendors; native Windows install.ps1 still caps at <2.11.0 everywhere, so the silent 2.10-to-2.11 move cannot occur there (2.11 alignment is a separate follow-up). Verified: 35-check unit suite rewritten to the new spec (any-flavor keep, floor rejection, noise, window edges, opt-out, wiring including pin-after-reroute and helper coverage); end-to-end matrix against sandboxed UNSLOTH_STUDIO_HOME installs on a cu130 host covering PyPI bare, cu128 drift, cu130 same-flavor, out-of-window 2.3, the upgrade opt-out, the hidden-GPU cpu leaf, and a fresh-install control. * install: honor the kept torch release on the Radeon direct-wheel path The Radeon repo path installs an explicit wheel trio selected by _pick_radeon_wheel, bypassing --default-index, so the kept-release pin only took effect when the listing failed and the install fell back to the ROCm index. On a re-run over an in-window Radeon install the trio search started at the newest common minor and silently moved the user forward (2.9 to 2.10 whenever the repo offered both). The trio search now starts at the kept release's minor when _PREV_TORCH_PIN is set and the listing still offers a torch wheel for that minor. Radeon wheels are patch-curated per rocm release, so the minor is the unit of preservation there; the raised rocm7.2 / Strix floors still win because the pin is window-checked against the final constraint before this point, and gaps keep the existing downward search / ROCm-index fallback. Verified with a simulated listing carrying both a 2.9 and a 2.10 trio: no pin selects the 2.10 trio, a kept 2.9 release selects the matched 2.9 / 0.24 / 2.9 trio, and an unavailable minor degrades to the newest trio. Added a structural wiring check to test_previous_torch_pin.sh (now 36 checks). * install: tighten comments in the torch preservation paths * install: exact kept release on the Radeon path, pin fallback in ROCm repairs The minor-level clamp on the Radeon direct-wheel path still allowed patch drift (a kept 2.10.0 could become 2.10.1 when the listing carried both) and the downward gap search could settle below the kept minor, both breaking the exact preservation guarantee the other vendor paths honor. The kept release now gets an exact-first trio attempt before the newest-trio search: pick the kept patch (else the newest patch of the kept minor, for listings that pruned the exact patch) together with the paired torchvision/torchaudio wheels for that minor. Any gap warns and falls back to the unchanged newest-trio search, mirroring _install_torch_default_index, so a rerun installs either the kept release or the same set a fresh install would choose, never something in between. The two ROCm torch repair sites (torch overwritten by dependency resolution, on the migrated and fresh paths) installed TORCH_CONSTRAINT directly, so a pinned release missing from the generic ROCm index would abort the rerun instead of falling back. Both now route through _install_torch_default_index, which passes extra uv args through (--force-reinstall) and clears the pin once the fallback fires so later paths stay consistent. Verified against synthetic listings: both patches listed keeps exactly 2.10.0; a kept minor missing vision/audio warns and yields the newest complete trio rather than a silent undercut; a pruned patch stays on the kept minor; no pin keeps the existing newest-trio behavior. Unit suite now 39 checks, all passing. * install: never pin nightly/dev/source torch builds on a rerun A survey of published torch version strings (PyPI bare, +cpu, +cu116 through +cu132, +rocmX.Y and +rocmX.Y.Z, +xpu, nightly .devYYYYMMDD, source a0+git, rc tags) showed one gap: nightly, dev, rc, and source builds passed the loose release-shape check, producing a pin such as torch==2.11.0.dev20250704 that no stable index carries. The range fallback rescued the install, but it printed "keeping it" and then burned a doomed resolve first. The base must now be a plain numeric X.Y[.Z] release, so those builds skip the pin and go straight to the newest supported release. Added unit checks for +xpu and three-component +rocm7.2.1 tags (both already preserved correctly) and for nightly, a0 source, and rc builds (never pinned). Suite now 44 checks, all passing. * install: pair kept-release companions, protect the flavor repair, note substitutions Three fixes from a 12-way review pass over the preservation work: The kept-release install left torchvision and torchaudio unconstrained next to the exact torch pin. torchvision exact-pins its torch in wheel metadata so it always paired correctly, but torchaudio no longer does: a kept torch 2.9.0 on cu130 resolved torchaudio 2.11.0 (verified with uv dry-runs). The helper now pairs both companions to the kept minor (torchvision 0.minor+15, torchaudio 2.minor); if the index lacks the paired set the existing range fallback fires. Verified resolving correctly on cu130, cu126, and rocm6.4. The wrong-flavor repair at the end of the install was the one remaining default-index torch install outside the helper. It runs under set -e, so a retained pin absent from the repair index (reachable when the Radeon direct-wheel path installed the kept release and dependency resolution later overwrote it) aborted the installer at the last step instead of falling back. It now routes through the helper with its reinstall flags passed through. The Radeon kept-release path installed a same-series build silently when the listing had pruned the exact patch; it now prints what it is substituting. Unit suite extended with wiring checks for all three (46 checks, all passing).	2026-07-19 07:55:06 -07:00
Daniel Han	a9be36830e	Installer: allow torch 2.11.x on the CUDA install path (fresh install + studio) (#6959) ... * Studio: allow torch 2.11.x on the CUDA install path The CUDA torch repair path (_ensure_cuda_torch) installs torch/torchvision/ torchaudio from an exclusive --index-url, so _CUDA_TORCH_PKG_SPEC decides exactly which torch the Studio venv gets. It was capped at torch<2.11.0, so on a cu128/cu130 host the venv resolved torch 2.10.x even though the CUDA indexes now publish torch 2.11.0. That left the Studio venv a torch minor behind the torch 2.11.0 Docker base image, so the CUDA dedup step would relink base libs under a mismatched torch. Raise the upper bound to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so the CUDA install path lands on torch 2.11.x, matching the rocm7.2 spec and the base image. The torchao selector already maps torch 2.11 -> torchao 0.17.0, and _ensure_flash_attn degrades gracefully when no prebuilt wheel matches (Blackwell skips it outright; non-Blackwell prints a warning and continues), so no other pin needs to move. Add test_cuda_torch_spec.py to lock the bound (torch 2.11.x in, 2.12.x out) and assert the CUDA and rocm7.2 upper bounds stay in lockstep. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: use zip(strict=True) so a spec length mismatch fails loudly * install.sh: widen the CUDA torch ceiling to <2.12.0 so a fresh install matches the base Raising _CUDA_TORCH_PKG_SPEC alone was not enough: that spec only feeds _ensure_cuda_torch(), the ROCm-poisoning repair path that early-returns on a normal NVIDIA host. A fresh CUDA install (including the studio Docker build, which runs `bash install.sh --local`) takes its torch from install.sh's TORCH_CONSTRAINT, which was still capped at torch>=2.4,<2.11.0, so cu12x/cu13x resolved torch 2.10.x and the venv landed a minor behind the torch 2.11.0 base image. Extend the existing `case "$TORCH_INDEX_URL"` block (which already relaxes rocm7.2) with a `*/cu[0-9]*` branch that widens the ceiling to <2.12.0, keeping the >=2.4 floor so an older CUDA index (e.g. cu118) that tops out below 2.11 still resolves. The CPU wheel and older ROCm tags stay on <2.11.0 (the glob does not match /cpu). torchvision/torchaudio are bare on this install line and resolve their compatible companions via wheel metadata, matching the rocm7.2 pattern. Add behavioral tests (Python + shell) exercising the case block: cu118/124/126/ 128/130 widen to <2.12.0, rocm7.2 stays 2.11.x, and /cpu plus older ROCm keep the default <2.11.0. * install.sh: key the CUDA torch widening off the index leaf, not the full URL The `*/cu[0-9]*` glob matched a `cu<digit>` segment anywhere in TORCH_INDEX_URL, so a custom UNSLOTH_PYTORCH_MIRROR whose base path contains e.g. cu128 but whose final leaf is cpu or an older ROCm tag would still widen TORCH_CONSTRAINT to <2.12.0, contradicting the block's own comment and letting a CPU / older-ROCm mirror resolve torch 2.11.x. Match on _torch_index_leaf (the final path segment the backend classification just above already computes) so only a real cu*/ rocm7.2 leaf is affected; cpu and older ROCm keep the default <2.11.0. Update the Python + shell tests to mirror the leaf-anchored case and add regression cases for a mirror base that contains cu128 but resolves to a cpu / rocm7.1 leaf. * install: freeze the torch trio during the with-deps unsloth installs Released unsloth wheels can pin an older torch than Step 1 installed (unsloth 2026.7.2 declares torch<2.11.0), so the with-deps resolve from PyPI silently downgrades the pinned +cuXXX torch trio to PyPI's default wheel. The flavor guard cannot catch every such swap: PyPI's torch 2.10 default is itself cu128-flavored, so the cuXXX tag comparison still matches while the version silently drops. Freeze the just-installed trio with uv --overrides (overrides replace dependency requirements during resolution), keeping torch 2.11.0+cuXXX in place while unsloth's other dependencies resolve normally. Verified on the cu128 path: without the override torch drops 2.11.0+cu128 -> 2.10.0; with it the trio survives and unsloth 2026.7.2 + unsloth-zoo install cleanly. * install: fold UV_OVERRIDE env files into the torch-trio overrides file The CLI --overrides flag is the command-line form of UV_OVERRIDE, so passing it replaced any overrides file already exported for the process; macOS arm64 exports UV_OVERRIDE=overrides-darwin-arm64.txt for the same generic install path and would have lost those pins. Concatenate any UV_OVERRIDE files into the temp trio file so both keep applying. * install: extend the torch-trio overrides guard to migrated installs Four follow-ups to the Step-2 --overrides guard, all empirically verified: 1. The migrated-environment with-deps unsloth install resolved unsloth>=2026.7.2 (which pins torch<2.11.0) without the overrides file, so a migrated CUDA venv on torch 2.11 was silently downgraded -- the exact bug this branch fixes on the fresh path. The overrides build is now a function (_build_unsloth_torch_overrides, reading the trio installed at call time) invoked by both with-deps paths; the migrated no-torch path installs --no-deps and stays unguarded. 2. The overrides temp file is now cleaned by the EXIT trap (same pattern as _UV_OVERRIDE_TMPDIR, pre-initialized empty so an inherited value can never reach the trap's rm); previously any Step-2 failure leaked it. 3. Folding UV_OVERRIDE files used cat, which joins the last requirement of a file lacking a trailing newline onto the next file's first requirement (reproduced: idna==3.10certifi==2025.1.31 makes uv fail parsing). 4. Inherited torch/torchvision/torchaudio override lines are now filtered out when folding: uv intersects duplicate overrides rather than last-wins (verified on uv 0.10.12: direct conflict is unsatisfiable, transitive conflict silently backtracks), so a conflicting inherited trio pin would break the resolve the generated exact pins protect. Both 3 and 4 are handled by a single newline-terminating awk filter that preserves non-trio overrides (torchmetrics, torchao, ...). test_unsloth_torch_override.sh extended: migrated-path coverage, trap assertion, and a functional fold test (14 checks). * installer: tighten comments * install: keep the existing torch release when re-running the installer Re-running `curl -fsSL https://unsloth.ai/install.sh | sh` over an existing install rebuilds the venv for clean state, which silently moved users to the newest torch in range (2.10 -> 2.11 once the constraint widened). A torch the user already validated must survive an unsloth update. Before the old venv is moved aside for rollback, its torch version is probed (last stdout line only, so sitecustomize noise cannot corrupt it). After the index leaf is chosen, _previous_torch_pin turns that version into a torch==X.Y.Z pin, but only when it cannot do harm: - cu*/cpu leaves only; rocm leaves keep their floors (rocm7.2 must land 2.11 for the Strix _grouped_mm fix) and the Radeon wheel-matching path is untouched. - The wheel's flavor tag must match the freshly chosen leaf, so a flavor change (cpu -> cuda, cu126 -> cu130) still installs the correct new build. - The base must look like a release, so probe noise never becomes a pin. - UNSLOTH_TORCH_UPGRADE=1 opts out and restores the old always-newest behavior; the substep line advertises it. The supported range is kept in _PREV_FALLBACK_CONSTRAINT: if the exact release is not resolvable from the chosen index (custom mirrors prune old wheels), the install warns and falls back to the newest supported release instead of failing the whole run. The later flavor-mismatch repair reuses TORCH_CONSTRAINT, so a mid-install clobber is repaired back to the kept release rather than the newest one. Verified end to end: a venv seeded with torch 2.10.0+cu130 re-run through the full installer finishes with torch 2.10.0+cu130 (previously 2.11.0+cu130). Tests: tests/sh/test_previous_torch_pin.sh covers keep/flavor-change/rocm/ noise/opt-out plus wiring (probe ordering before venv replacement, fallback present, SKIP_TORCH gate). * install: constrain kept torch pins to the supported window Review caught that _previous_torch_pin pinned the previous venv's torch on flavor match alone, so a release outside the installer's active range (a 2.3.x manual install below the >=2.4 floor, or a 2.12.x manual upgrade above the ceiling) replaced the bounds computed just above it and a rerun kept a torch the installer otherwise deliberately excludes. New _torch_release_in_window checks the probed base against the active TORCH_CONSTRAINT ("torch>=A.B[,<C.D.F]") at major.minor granularity, which is exact for the windows this script uses (ceilings are always X.Y.0; a non-.0 ceiling would only make it conservative). Anything unparseable answers no, so probe noise or a malformed window fails toward the supported range instead of becoming a pin. _previous_torch_pin takes the active constraint as a third argument and refuses out-of-window releases; the in-window keep behavior is unchanged. Tests: out-of-window rows (2.3.x floor, 2.12.x ceiling, boundary keeps, cpu and macOS windows, malformed/empty windows) plus direct _torch_release_in_window coverage. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-19 06:19:29 -07:00
Daniel Han	ca979e9643	Studio: add UNSLOTH_SKIP_AUTOSTART installer flag (#7093) ... * Studio: add installer autostart opt-out * CI: run installer autostart tests cross-platform * Tests: combine Studio installer skip flags	2026-07-12 21:23:14 -07:00