80 commits

Author	SHA1	Message	Date
Etherl	e0a2bd8317	Studio: preserve the Windows launcher during updates (#8092) ... * Fix Windows Studio launcher updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the remaining damaged-file reports and launcher recovery for PR #8092 The report this PR targets listed three damaged files. Preserving the launcher fixes one; the other two still fail the update on their own, since _fail_if_install_damaged exits 1 on any finding. Both are produced by our own installer, so the update they fail is the update meant to repair them. einx and torchao both ship a top-level test/conftest.py, and install_python_stack.py force-reinstalls torchao every update, so pip deletes the file and the pinned torchao does not ship it. package-lock.json is rewritten in place by setup.ps1 and setup.sh, which run npm install inside the installed tree; under legacy-peer-deps npm dedupes hoisted entries and the file shrinks below its recorded size, reproduced exactly as 28473 to 27225. Drop both classes while reading RECORD rather than when reporting, so a filtered row also stays out of the ownership tally and the limit budget and cannot crowd out a real finding. Mirrored into the sidecar scanner, whose docstring asks for the two predicates to be kept in sync. Also three fixes to the transaction itself: - Recover from the hardlinked bin/unsloth.exe shim, which survives the old updater's .deleteme unlink. - Warn instead of exiting when the launcher is missing or invalid. An install already broken by the old updater has neither launcher nor .deleteme, and exiting before setup stopped exactly those users from updating. validate_launcher still judges the result. - Gate recovery on validity rather than existence, and treat a failed backup as a missing safety net rather than a fatal error. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve the launcher from the managed venv and keep a good backup Three follow-ups from review. The transaction resolved Scripts from sys.executable, but setup.ps1 installs into STUDIO_HOME/unsloth_studio (setup.ps1:3411). When a pip-installed or checkout CLI drives the update, those differ, so it backed up and --version validated the caller's launcher while the one actually being replaced went unprotected. Resolve the managed venv the same way _studio_deps._managed_root does for the damage scan. __enter__ overwrote the transaction backup unconditionally, guarded only by the two-byte MZ check. A backup outlives __enter__ only when a previous run died before validating, so it holds the last launcher known to run; overwriting it with a PE-shaped but unvalidated canonical file destroyed the only recovery copy. Write a backup only when there is no usable one already. package-lock.json was skipped outright, which dropped its existence check too. npm rewrites it in place but never deletes it, so keep the row and drop only its recorded size. Also add scripts/ to the shared non-runtime roots: unsloth_zoo ships a top-level scripts/, the same squatted-namespace shape as einx's test/, and it has no __init__.py so nothing imports it. * Move the launcher aside for setup, and restore it if nothing replaces it I rejected this on the strength of setup.ps1:4386-4391, which says renaming the running launcher "only ever failed (WinError 32)". That is not right. A probe on windows-latest builds a real console-script package, runs it, and renames the live launcher: the rename succeeds and a replacement can then be written at the freed path. RESULT idle-launcher: RENAME SUCCEEDED RESULT running-launcher: RENAME SUCCEEDED RESULT publish-replacement: WROTE a new launcher at the canonical path main moved the launcher aside before setup (studio.py:3182) and this branch had removed it, so uv could no longer replace Scripts\unsloth.exe. uv only self-replaces its own executable and deletes a third-party console script outright, and the pip fallback then no-ops on the already-satisfied bare unsloth, so the upgrade was silently skipped. Move it aside again, but keep what this branch was written for: when setup publishes no launcher, validate_launcher restores it rather than leaving the venv with none. That was the original bug, where the old updater renamed the launcher away and then deleted its own .deleteme. Restore prefers the backup over the moved-aside copy: the backup is the last launcher known to run, the moved-aside one is only this run's unvalidated canonical file. The mocked harness cannot reproduce a sharing violation, so the tests pin the invariant (the canonical path is free during setup, a recoverable copy always exists) while the CI probe covers the Windows semantics. * Tell a missing launcher from a broken one, and retry restores Two follow-ups, both from the restore path added in 724b274d4. Restoring before the health check could not tell setup publishing nothing from setup publishing something unusable. A zero-byte or non-PE replacement was quietly swapped for the previous launcher, which then passed --version, so the update reported success and deleted its own recovery copies. Sample whether setup published anything before any restore: nothing published and a good restore is the no-op update this transaction exists for, while a launcher setup did write and that cannot run stays a failure even though the previous one goes back. _restore_backup also picked the first candidate passing the two-byte header check and stopped there. Backups are taken after only that check, so an interrupted run can leave a PE-shaped but non-runnable one, and preferring it stranded the working launcher this run had moved aside. Split restoration: _restore_from puts one candidate back, and _restore_runnable walks the candidates until one actually runs. * Restore a runnable launcher on exceptional exit, narrow the exemption __exit__ restored the first PE-shaped candidate, so an interrupted run's non-runnable backup was installed over the working launcher this run had moved aside, and it could undo a restore validate_launcher had just made. It now uses _restore_runnable, which leaves an already-working launcher alone, walks the candidates until one passes --version, and falls back to the best candidate rather than whichever was tried last. The shared-namespace exemption was also too broad. I justified it on the grounds that tests/ and scripts/ ship no __init__.py, which is wrong: PEP 420 makes them importable, and this repo does 'from scripts import ...' itself. Restrict it to distributions Unsloth does not ship, so einx and torchao squatting on a top-level test/ is exempt while our own top-level trees stay checked. * Keep all four recovery copies as runtime candidates _recovery_candidates only offered the backup and the moved-aside copy, so when an interrupted run left a PE-shaped but non-runnable backup and the legacy .deleteme or the PATH shim was still good, the bad backup was accepted on its header alone and the good copy was never reached. The update then failed every time with the broken bytes canonical. All four are candidates now, deduplicated by normalised path, and _restore_runnable walks them until one passes --version. * Move the update lock out of the replaceable venv Resolving Scripts from the managed venv put the lock inside $VenvDir, and setup.ps1:3748 removes that whole directory to rebuild a stale torch. Windows refuses a recursive delete while a handle inside it is open, so an external CLI holding the lock for the whole setup run failed the repair with "Could not remove stale venv". Keep it under the Studio home instead, which is stable and is the right grain anyway: it is what names the managed venv. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-07 05:41:18 -07:00
Etherl	d495a09bf0	Guard Windows Studio installs against active runtimes (#7764) ... * Fix Studio installer runtime race * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Close remaining Studio installer races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep Windows installer tests Windows-only * Stabilize Windows process guard test * Coordinate terminal Studio launches * Guard all managed Studio launches * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Coordinate custom Studio roots * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Handle direct GGUF settings rows * Apply repository formatter * Close remaining Studio update races * Handle Windows runtime gate CI edge cases * Verify the updater parent shim by image * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Allow verified updater shim chains * Stabilize Windows process guard test * Handle Windows console-script redirectors * Close remaining Windows updater guard gaps * Handle spaced and repeated Windows update shells * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve Tauri root aliases before validation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix custom-root locks and updater ancestry * Align Windows runtime identity checks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope desktop fallback to the current user * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Preserve drive-root mutex identity * Use ordinal semantics in Studio idle scans * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard standalone Studio setup mutations * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the setup gate handoff independently * Version the Windows installer native helper * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use ordinal path comparison in Studio runtime scan * Tighten Studio runtime gate test comments * Exempt the venv Python redirector in the Studio runtime gate Windows venv Scripts\python.exe is a redirector that runs base Python as a child, so `unsloth studio setup` runs as unsloth.exe -> python.exe -> us. The ancestor walk only exempted the unsloth.exe shims and stopped at the redirector, so the installer flagged its own launcher and every Windows install failed with "The managed Studio environment is in use by unsloth.exe". Carry one redirector as pending and exempt it only when a shim sits directly above it, so a managed backend that spawns an update still blocks. Also stop gating the protected shim paths on exists(), so a shim renamed out of the way mid-update is still recognised. Drop "Studio" from the two runtime-lock messages so process.rs satisfies the desktop branding contract. * Close the redirector exemption when the updater is the managed image Only a base interpreter runs under a venv redirector. If our own executable is inside the managed root there is no redirector above us, so a managed parent is a real consumer and must keep blocking. Adds the regression to the redirector test. * Key the redirector exemption on sys.executable, not on a shim above it The Tauri updater runs `<venv>\Scripts\python.exe -I -c ... studio update` directly, so its chain is tauri.exe -> redirector -> base Python with no unsloth.exe in it. Requiring a shim above the redirector made every desktop update block on its own launcher. A venv redirector starts base Python as a child and waits, so when we are the base image and sys.executable still names the managed interpreter, our direct parent is that launcher. Exempt exactly that hop; ancestors above it must still be shims, and a managed image at depth two or more keeps blocking. * Stop the x86 guard test racing its own probe The 32-bit leg fired one scan against a probe that lives about five seconds, while a WOW64 shell start plus the Add-Type compile regularly costs more than that, so it read an empty list on a Windows runner. Give the probe a long life and retry like the 64-bit sibling already does. The assertion is unchanged. * Tighten comments in the Studio runtime gate changes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com>	2026-08-07 05:24:40 -07:00
Daniel Han	ee64eec51a	release-desktop: add a VirusTotal pre-flight scan of the release bundles (#8089) ... * release-desktop: add a VirusTotal pre-flight scan of the release bundles * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * virustotal_scan: register the signed upload URL with add-mask * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * virustotal_scan: check out the script, stop replaying single-use upload URLs, bound every request by the deadline * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * virustotal_scan: fail closed on malformed hash lookups and cap pacing by the deadline * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bound VirusTotal socket calls to the deadline and scan only validated releases - Pass a per-call socket timeout through the transport, clamped to the remaining scan deadline, so a request starting just before the deadline cannot consume the full 300s cushion ahead of the step timeout. - Retry a malformed upload acknowledgement instead of aborting, since the disclosure cost of the upload has already been paid at that point. - Move the scan after 'Create or validate versioned release' so a run that is rejected has not already uploaded all four bundles. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Defer non-draft release creation past the scan and cap retry backoff - Split 'Create or validate versioned release' into a validation step that runs before the scan and a creation step that runs after it. A dispatch with draft=false and a new tag previously published an empty release that stayed assetless for the length of the scan, and permanently so if the run was cancelled part way through. - Clamp the exponential retry backoff to the remaining deadline, so a 429 or 5xx arriving late cannot sleep past --timeout-seconds before the loop notices and writes its summary. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep release notes unconditional, fail closed on lookup errors, fix the permission test - Write desktop-release-notes.md in the validation step, which always runs. The updater metadata step reads it on every run, so leaving the write in the conditional create step broke reruns against an existing release. - Only treat a lookup as a missing release when gh reports 'release not found'. Any other failure now fails the step, rather than proceeding to disclose the bundles for a run that cannot publish. - Point test_release_desktop_permissions at the renamed validation step and assert the deferred create step and its gate. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Escape third-party text in the VirusTotal warning annotations Engine names, detection labels and API error strings are third-party data written straight into a workflow command. Actions truncates an annotation at the first newline and mis-parses a bare %, so a crafted or merely awkward detection string could drop the engine list exactly when the scan is trying to alert a maintainer. Mirrors _gha_escape in lockfile_supply_chain_audit.py, including the replace-% first ordering. * Never report an unanalysed bundle as clean, and escape the summary - A hash known to VirusTotal can have no completed analysis, in which case last_analysis_stats is absent and parse_stats yields all zeros. That row read as 'known to VirusTotal' with zero detections, which looks like 70 engines cleared a bundle that none of them scanned. Such a row now reports 'no completed analysis' with stats left unset, so it renders as dashes and cannot trip the threshold. The upload path polls until status is completed, so it only requires a stats object. - Escape third-party engine names, detection labels and error strings in the job summary. It is appended to GITHUB_STEP_SUMMARY and rendered as Markdown, so a newline ended the row and | opened a new cell. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Avoid a CodeQL clear-text-logging false positive on the skip message Interpolating API_KEY_ENV into the skip log trips CodeQL's py/clear-text-logging-sensitive-data rule at high severity, because the constant's name ends in _KEY. It only ever holds the env var name, never the value, but the repo uses CodeQL default setup so there is no config to filter the query on. Write the name out literally and pin it against the constant in test_missing_key_skips_without_failing so the two cannot drift. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-07 03:27:08 -07:00
oobabooga	5659a9c47e	Installer: keep status messages off the progress bar line (#8052) ... * Installer: keep status messages off the progress bar line * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix blank lines and stale progress state in PR #8052 Follow-ups on the centralised progress-line close: - Four ROCm messages still opened with a literal \n, which used to be the only line terminator. _safe_print() emits one too now, so inferred-gfx, Strix and gfx906 installs got a blank line. - _end_progress_line() caught only OSError, so a closed or detached stdout took down messages bound for stderr, including the manifest error paths. - install_python_stack() reset _STEP but not _PROGRESS_LINE_ACTIVE. Only _step() read it before; every _safe_print() does now, so an aborted run left a stray newline on the next run's first message. - _note() aligned to the value column in verbose mode, where there is no bar and no step line to align to. - Dropped the _end_progress_line() call that _safe_print() now makes itself. Tests: patch _HAS_COLOR in _render() so the layout assertions hold under FORCE_COLOR=1, plus AST guards for leading-newline messages and direct sys.stdout writes, and coverage for wrapping, verbose, colour, closed stdout and the entry-point reset. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in the progress-line changes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-07 01:20:47 -07:00
Daniel Han	d8effae0d5	Studio: fix the Windows desktop setup log mojibake and double-printed steps (#8083) ... * Studio: fix the Windows desktop setup log mojibake and double-printed steps The desktop 'Getting things ready...' log rendered as: ?? Unsloth Studio Setup <52 replacement chars> gpu none (chat-only / GGUF) gpu none (chat-only / GGUF) Encoding. studio/setup.ps1 never set [Console]::OutputEncoding, so Windows PowerShell 5.1 encoded redirected output with the OEM code page while the desktop app decodes the pipe as UTF-8 (String::from_utf8_lossy in src-tauri/src/install.rs). That corrupts two different ways: the sloth U+1F9A5 has no OEM representation so PowerShell substitutes one '?' per UTF-16 surrogate, and the rule U+2500 does have one, so it becomes a bare 0xC4 byte that is invalid UTF-8 and surfaces as U+FFFD. Both entry scripts now set the console encoding, $OutputEncoding, PYTHONUTF8 and PYTHONIOENCODING before the first write, and Refresh-Environment can no longer reload the two Python vars back over ours mid-run. The patch is ASCII-only: these files are UTF-8 without a BOM and 5.1 parses those as ANSI. Duplication. step/substep wrote through Write-Host AND a console-handle mirror. The mirror's comment assumed Write-Host does not survive the process chain; it does, because the CLI spawns setup.ps1 as -Command "& '...' *>&1" (unsloth_cli/commands/studio.py), which merges the Information stream into stdout deliberately. The sink is now resolved once and exactly one is used: redirected writes to the console handle, interactive writes to Write-Host. Splitting. step composed one logical line from two Write-Host calls using -NoNewline, and a redirected consumer turns each Information record boundary into a line break. Both scripts now emit one composed record; install.ps1 needs this most, having no mirror to fall back on. Rust children on Windows get PYTHONUTF8/PYTHONIOENCODING too, since install.rs, update.rs and process.rs all decode their output as UTF-8. The readers stay lossy on purpose -- strict decoding would turn display corruption into an installation failure. Tests: a Pester suite auto-discovered by the existing pester job, and a pytest byte-level probe that runs real PowerShell in both the -File and -Command launch shapes and asserts on raw bytes. Verified to fail against the unfixed tree (9 Pester and 13 pytest failures) rather than merely passing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bind a UTF-8 writer with no console, and pass -X utf8 to the isolated child Two holes in the previous commit, both on the exact path the desktop app takes. [Console]::OutputEncoding P/Invokes SetConsoleOutputCP, which needs a console handle. Under CREATE_NO_WINDOW there is none, so it throws, and it drops the cached writer BEFORE throwing while assigning OutputEncoding only after. Console .Out therefore rebuilt on the old code page. Swallowing the exception was not enough once redirected step/substep use Console.Out as their only sink, so the catch path now binds an explicit UTF-8 StreamWriter over OpenStandardOutput. build_update_command launches Python with -I, which implies -E, so that process ignores every PYTHON* variable and PYTHONUTF8/PYTHONIOENCODING never reached it. Pass -X utf8 as a switch instead. The env vars stay for its descendants. https://docs.python.org/3/using/cmdline.html#cmdoption-I * Bind the UTF-8 writer to stderr as well when there is no console The no-console fallback repaired Console.Out only. Tauri pipes stderr through the same lossy UTF-8 decode (install.rs) and emits it to the same UI log, and InstallFailureContext builds the user-facing failure message from those lines, so a PowerShell error carrying a non-ASCII path still arrived as U+FFFD. install.ps1 also writes its Clear-TauriInstallError markers there. * Tighten the comments added by this PR Comments only, no code change. Verified with the PowerShell AST tokenizer for both .ps1 files and the Pester suite (token streams identical with Comment and NewLine excluded), comment_tools.py for the Python test, and a code-only diff for update.rs. * Update the Windows command assertion for the added UTF-8 flags windows_update_command_uses_python_not_replaceable_console_stub asserts the exact argument vector, so adding -X utf8 broke it. The Windows cargo test job in studio-tauri-smoke.yml runs it; the Linux job skips it under cfg(windows), and cargo check type-checks tests without running them, so neither the org Linux run nor the staging check caught it. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-07 01:18:35 -07:00
Long Yixing	d6b1b7dcf2	Studio: drop the mlx-lm 0.31.3 exclusion so current mlx-vlm resolves (#7061)	2026-08-07 02:41:04 -03:00
Daniel Han	5e6b4aa1ee	Reuse the torch2.10 prebuilt accelerator wheels on torch 2.12 (#7495)	2026-08-07 01:19:14 -03:00
Tai An	6ea2b91d04	fix(studio/install): keep the rollback reference when a venv move stops partway (#7810) (#7823) ... * fix(studio/install): keep the rollback reference when a venv move stops partway (#7810) Start-StudioVenvRollback moves the existing environment aside with a single Move-Item and treats any failure as "the original is still in place". On Windows that is not the shape the failure takes: an open handle inside the tree -- a running Studio backend holds its own unsloth.exe there -- fails the rename *after* it has already walked part of it, so the entries handled before the locked one sit at the rollback path while the rest stay behind. Both paths then exist, and the catch tests only the source, so it scores the split tree as an untouched original, clears StudioVenvRollbackDir and drops the sole record of where the other half went. The reporter was left with a unsloth_studio\ holding 7 Scripts entries and no python.exe, an intact venv under unsloth_studio.rollback.<stamp>.<pid>\, and no output naming either. Retries cannot recover: the create branch keys off python.exe, and uv refuses to build a venv over the directory the stranded files still occupy. Clear the rollback state only when the destination is genuinely absent. When both paths exist the move is partial, so keep it active -- the existing finally-block Restore-StudioVenvRollback then reverses it -- and print both locations plus the "close Unsloth Studio" hint the launcher-shim path at install.ps1:3078 already gives for the same underlying cause. install.sh is unaffected: POSIX rename ignores open descriptors and both paths are siblings under , so that move really is atomic. Regression test extracts the function from install.ps1 and runs it under real PowerShell with Move-Item stubbed to fail after creating the destination. On current main the partial case reports active=False with an empty rollback dir -- the stranding itself; the clean-failure case is asserted alongside it so the untouched-original path keeps clearing state as before. Signed-off-by: Tai An <antai12232931@outlook.com> * fix(studio/install): merge a split venv back instead of clearing the target Keeping the rollback active after a partway move sent the failure path into Restore-StudioVenvRollback, which removes $target before moving $backup back. In the split case $target is not an incomplete *new* environment -- it holds the half of the previous one the move never reached -- so that removal deleted files present nowhere else and restored a corrupted venv. Flag the split and give restoration a merge path: move each entry of the backup into the target without overwriting, recursing where the move stopped inside a subtree, and only drop the backup once it is empty. Anything ambiguous is left in place and both locations are named. Regression test pins the file that never moved to surviving restoration. * Installer: fix split-move merge nesting siblings and walking through junctions Two problems in Merge-StudioVenvRollbackTree, both only reachable once a partway move leaves the venv split. Sibling nesting. The per-entry variable was named $destination, and PowerShell variable names are case-insensitive, so it reassigned the $Destination parameter. Only the first entry at a level landed correctly; every later sibling was joined onto the previous one's path, so a restored venv came back with pyvenv.cfg inside Lib. Renamed to $entryTarget. Junction traversal. Recursion keyed on "directory on both sides", which a junction or directory symlink satisfies. If the half left behind holds the link, venv files move through it and land outside $StudioHome. If the moved half holds it, the recursion enumerates the link target and pulls those files into the venv, emptying a directory that was never part of the environment. Either way the link is replaced by a real directory, which the whole-tree Move-Item this path replaced never did. Now checks both sides for a reparse point and falls through to keep-both-copies. Attributes are read via Get-Item on both sides, since Get-ChildItem has reported them inconsistently. Tests cover sibling placement and both link directions. The link test uses a junction on Windows so it does not need SeCreateSymbolicLinkPrivilege. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Tai An <antai12232931@outlook.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-06 02:09:52 -07:00
Daniel Han	cd3aef8a70	Bound dataset_num_proc by memory, and stop treating 1 as "no multiprocessing" (#7831) ... * Bound dataset_num_proc by memory, and stop treating 1 as "no multiprocessing" Training intermittently died with "One of the subprocesses has abruptly died during map operation", then succeeded on the next run (#2693, and a fresh Studio report). Measuring the tokenization map on an 8000-row dataset with a fast tokenizer found the mechanism: - Each pool task dill-pickles the tokenizer closure over a pipe, 5,369,755 bytes, once per worker per map. This happens under fork too: datasets does `from multiprocess import Pool`, and multiprocess/queues.py pickles every task regardless of start method. - Each worker peaks around 680 MB RSS. The old auto count was min(max(cpu_count + 4, 2), 64), so a large host forked up to 64 workers for roughly 43 GB resident. On a smaller box the OOM killer takes one and the parent reports only the generic message above, because datasets/utils/py_utils.py compares pool PIDs and never reads the child's exit status. Killing a worker with SIGKILL or SIGSEGV reproduces the error character for character, at any num_proc including 1. Two further defects made it worse: - The guard asked stdlib multiprocessing for the start method while datasets uses multiprocess, which keeps an independent default context. It was reading the wrong module. - num_proc=1 was used as the "no multiprocessing" sentinel. On datasets 4.3.0 (the Studio pin) map() takes the pool branch for any num_proc >= 1, so 1 still builds a Pool(1). Measured, num_proc=1 is 51% slower than None while buying no parallelism. Only None is in-process on every supported release. Changes: - New unsloth/utils/dataset_num_proc.py, one policy instead of four drifted copies. It asks multiprocess about the start method, caps the auto count at 8, and bounds any count, explicit ones included, by available memory at roughly 1 GB per worker over half of free RAM. Studio's explicit cpu_count // 4 previously bypassed every bound, which is how a 192-core host reached 48 workers. UNSLOTH_DATASET_NUM_PROC remains an uncapped escape hatch. - The config layer records intent and the map() call site makes it safe. These cannot be collapsed: unsloth_zoo reads a config None as "auto-size me", so writing None for a user who asked for 1 would inflate it. - worker.py no longer forces stdlib multiprocessing onto fork. It never reached Dataset.map, and Linux already defaults to fork. - A dead worker now raises with the start method, the worker count, the approximate memory cost and the escape hatch, chained from the original. No CUDA guard: 300 forced-fork map() runs on an initialized CUDA context produced no failures, and the child only runs the tokenizer. Since detect_hardware() always initializes CUDA, such a guard would cost every CUDA run its tokenization parallelism for no measured benefit. Known gap: the 1 -> None normalisation reaches SFT only, since that is the path sft_prepare_dataset owns. DPO, KTO, CPO, ORPO, Reward, PRM, PPO and BCO read args.dataset_num_proc in their own _prepare_dataset, so an explicit 1 there still builds a Pool(1). The memory bound does apply to all of them, so the OOM mechanism is covered everywhere. Reported by Eyera, who traced it to the commit and the call chain. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop UNSLOTH_DATASET_NUM_PROC=0 from inflating the worker count The env override returned before the serial encoding, so at the config layer it wrote None. unsloth_zoo.sft_prepare_dataset reads a config None as "auto-size me" and re-derives with its own uncapped min(max(cpu+4,2),64). So a user hitting a dead worker, following the escape hatch the new diagnostics message tells them to use, could get 64 workers. The hatch did the opposite of what it advertises, in exactly the OOM scenario it exists for. Affected 0, none, null, false, "" and 1: 14 of 42 config-layer cells. Measured end to end with the num_proc anchor skipped: before: requested auto / explicit 64 / Studio cpu//4 -> config None -> up to 64 after: all three -> config 1 -> bounded Also make the unsloth_zoo num_proc anchor non-required. It is the one anchor whose absence is harmless: the Zoo then reads args.dataset_num_proc, which the config layer has already bounded, so the memory ceiling still holds. unsloth_zoo is a floor dependency rather than a pin, and this is the block whose policy this branch changes, so it is likelier than the others to drift upstream. Hard-failing every install to recover an optimisation is the wrong trade. This is also what made the bug above reachable, so the two belong together. Verified across 630 cells: Python 3.10-3.14 x datasets 3.4.1-4.3.0 x fork/spawn/forkserver x four memory levels. Policy identical throughout. Zero config-layer None cells remain. 52 tests pass; reverting either fix fails them. Worth recording from that matrix: on Python 3.14 stdlib multiprocessing defaults to forkserver while multiprocess still defaults to fork, so the two disagree. Reading multiprocess, as this branch now does, is what matches what datasets will actually do; the old stdlib read would have silently disabled multiprocessing that was available. * Bound train_on_responses_only, the most-travelled map() in the library unsloth_zoo.dataset_utils.train_on_responses_only is a third copy of the same heuristic, and nothing in this branch touched it. unsloth re-exports it verbatim from chat_templates, it appears in essentially every Unsloth SFT notebook, and it is how Studio's apply_completion_masking reaches a map(). So the up-to-64-workers exposure sat on the most-travelled path while the branch fixed the quieter ones. Measured on a 192-core host with datasets 4.3.0, auto over a large split: 64 workers before, 8 after. Two constraints shaped the wrapper, both discovered before writing it: None cannot mean "in-process" at this boundary. The zoo's first act is `_num_proc_was_auto = num_proc is None or ...`, so a None arriving from outside reads as "size it for me" and triggers the very heuristic being bounded. Passing None for a caller who asked for 1 would inflate to 64, the same class of bug as the UNSLOTH_DATASET_NUM_PROC=0 one fixed earlier in this branch. So 1, not None, is the serial value here. The consequence is that an explicit 1 still builds a Pool(1) on datasets >= 4.0, unchanged from the raw zoo, so this is not a regression but the branch's "1 -> in-process" claim does not extend here. An explicit count also disables the zoo's 5000-row guard, which it applies to train_dataset and eval_dataset independently and only when it chose the count itself. Substituting unconditionally would hand workers to a small eval split that never had them. So substitute only when some split is actually at or above the threshold, which keeps the guard wherever it was doing work. A drift canary reads the zoo's constant off disk so the duplicated 5000 cannot diverge silently. 17 tests. Pool spy on a real masking run confirms a small auto dataset still builds no pool. Mutations: threshold drift 1 fail, explicit-serial returning None 2 fails, auto ignoring split size 6 fails. Known hole: when any split has no length (IterableDataset), this returns None and lets the zoo decide, so a huge train split beside a streaming eval split stays unbounded. Correct per the zoo's own rule, but it is a hole. * Keep the config layer serial on spawn, and stop an unsized split hiding a sized one Two genuine bugs from review, both of them regressions this branch introduced. The config sentinel leaked onto spawn platforms. On a non-fork start method the config layer wrote 1, and only SFT's map site rewrites that back to None. DPO, KTO, CPO, ORPO, Reward and PRM hand args.dataset_num_proc straight to Dataset.map, where datasets >= 4.1 builds a Pool(1) whose spawned child re-executes the user's __main__ (the Windows spawn loop, #3211/#3397). origin/main carried None there for the auto path, so this was a regression. The sentinel exists only to stop a downstream auto-sizer re-inflating serial, and no auto-sizer can do that when forking is unavailable, so it is now conditioned on fork. An unsized split masked a sized sibling. _largest_split_rows returned None the moment any split had no length, so a trainer with a large sized split next to a streaming one took the shortcut and returned a bare None past the env check. The Zoo reads that None as "auto" and picked 64 workers on this host: the exact inflation this branch exists to remove, and it happened with no env var set at all. Unsized splits are now skipped rather than allowed to veto, and an explicit UNSLOTH_DATASET_NUM_PROC wins on the shortcut too. Codex suggested resolving the override before the shortcut; that would return 1 for a small split and build a Pool(1) on datasets >= 4.1, so the fix is split in two instead. Also corrects the datasets boundary throughout: it is 4.1.0, not 4.0. huggingface/datasets#7702 flipped `num_proc > 1` to `>= 1`; 4.0.0 still ran num_proc=1 in-process. Verified against the 3.6.0, 4.0.0, 4.1.0 and 4.3.0 tags. One studio test asserted on 4.0.0 and would have failed on exactly that release. New file headers switched from LGPL to Apache 2.0, byte-identical to unsloth/dataprep/raw_text.py and tests/utils/data_utils.py, matching the repo LICENSE. 18 new tests. 87 pass; reverting the helper alone fails 16. * Keep macOS in-process by policy, not by a wrong start-method probe The probe read multiprocess.get_all_start_methods()[0]. multiprocess copies that function from the stdlib verbatim, darwin branch included, but not the darwin default that goes with it: its _default_context is still fork, carrying a literal '#FIXME: spawn'. So on macOS the probe said spawn while Dataset.map actually forks, and the dead-worker diagnostics printed the wrong method. Read the default context's own name instead, which fixes the report, and add _workers_unusable_reason() so the macOS refusal survives the corrected probe. Forking on macOS is what CPython itself declared unsafe when it moved the default to spawn in 3.8 (bpo-33725), and this parent has already loaded Torch and a threaded BLAS, so macOS stays in-process -- now as a stated policy rather than as a side effect of a misreport. Also run both num_proc suites in CI. They were never on the consolidated workflow's tests/utils allowlist, so all 87 guards were dead weight. * Bound the worker count Studio computes for itself A simulation across the platform x start-method x cpu x memory x request x env product found the one path that still reached Dataset.map unbounded. Studio's numbers are backend heuristics: trainer.py asks for cpu_count // 4 and safe_num_proc's own auto path is cpu_count // 3. By the time this module sees them they are explicit ints, which it reads as deliberate user intent and clamps by free memory only -- so a large host with RAM to spare kept every one of them. Measured end to end: 64 cores gave 16 workers, 96 gave 24, 192 gave 48 at ~1GB each, against a cap of 8 that the auto path has obeyed all along. The benchmark in dataset_num_proc.py has 32 workers at 14.2s versus 6.3s in-process, so those counts were slower as well as heavier, and Studio on a big machine is the configuration issue #2693 was reported from. Cap in safe_num_proc, which every Studio map() site routes through, and before the multi-GPU cap so the tighter of the two still wins. The constant is duplicated rather than imported, because importing it would pull unsloth's whole __init__ into hardware detection; a canary asserts the two stay equal, the same arrangement the Zoo's row threshold already uses in the other direction. UNSLOTH_DATASET_NUM_PROC is unaffected: it is read downstream and bypasses this. The simulation is scripts/matrix_numproc_policy.py. After the fix all 17280 cells hold every invariant: no workers on a start method that cannot support them, never 1 at a map() call site, never None at the config layer while forking works, never more workers than memory covers, never over the cap on the auto path, the env var obeyed verbatim, and deterministic throughout. * Say what UNSLOTH_DATASET_NUM_PROC=0 actually does The dead-worker message told the reader to tokenize in-process with UNSLOTH_DATASET_NUM_PROC=0. That is true almost everywhere and false in the one case the message is most likely to be read: train_on_responses_only on fork, with a split at or over the Zoo's 5000-row threshold, resolves to 1 rather than None, and datasets >= 4.1 turns 1 into a Pool(1). So the recovery advice offered for a large-dataset worker death did not remove the workers. The value is still right. A bare None there is read by the Zoo as 'size it for me' and would inflate to its uncapped count, and unsloth_zoo's _effective_num_proc returns num_proc unchanged when it is None or 1, so no value expresses in-process on fork for a large split without changing the Zoo. What was wrong was the sentence, so the sentence is now specific: fewest workers this path can use, in-process everywhere except that case, one worker there. Two tests. One reads the rendered message and requires it to name the exception, the path and the row threshold. The other drives resolve_responses_only_num_proc on both sides of the threshold and asserts 1 and None, so the message cannot claim a behaviour the resolver does not have. * Make the studio num_proc tests runnable off Linux and without torch The cross-platform staging legs failed all three, and the file's own docstring claimed it ran on any host, so both halves of that were wrong. dataset_map_num_proc returns None outright on win32 and darwin, so every assertion expecting a worker count was really an assertion about Linux and failed on the macOS and Windows runners. An autouse fixture pins the platform; the parametrised spawn-platform test sets its own value afterwards and still wins. _patch_runtime imported torch directly, which is a hard failure on a runner that has none. Worse than the error: dataset_map_num_proc treats an ImportError as "runtime not touched yet", so a torch-less host turns the XPU guard into a no-op and the test asserting None would have been passing for the wrong reason wherever it did not outright fail. It now falls back to a stub module in sys.modules, which a real "import torch" finds. Verified by reproducing both runner conditions locally rather than waiting on CI: 9 passed with torch and 9 passed with it removed. That harness needed correcting too -- it first blocked __import__ unconditionally, which is stricter than any real runner, since real Python consults sys.modules first and that is exactly what the stub relies on. * Do not trust a start method the host does not offer The cross-platform legs found a real bug in the probe, not just in its tests. On a Windows runner the private default-context chain answered "fork" while get_all_start_methods() was ["spawn"]. Those attributes are private and not consistent across builds, and a start method the platform does not offer cannot be the one in use. Believing it read Windows as forkable, so _workers_unusable_reason() returned None and workers were allowed through - the spawn re-import loop of #3211 / #3397 that this module exists to prevent. The probe now cross-checks its answer against the available methods and falls back to the documented list, with a regression test that reproduces the exact shape: spawn-only host, private chain saying fork, result None at both layers. The test failures around it were mine too, and they share a cause: the macOS policy added earlier made sys.platform load-bearing in get_dataset_num_proc, so a batch of tests that assert a worker count became platform-dependent. They passed on the Linux runner and failed on macOS. The module fixture pins the platform; the tests that are about the platform set their own value afterwards. The two studio tests that build real worker processes now skip when the host cannot fork. Under spawn inside pytest the pool fails for reasons that have nothing to do with the claim being made (WinError 10038 closing a handle, os.WNOHANG missing), and the version split they check is also asserted without processes in tests/utils. * Import multiprocess before the tests spoof the platform The Windows leg of staging CI failed inside a real worker pool with AttributeError: module 'os' has no attribute 'WNOHANG'. multiprocess picks its concrete contexts at import time from sys.platform, and both test files spoof that to linux, so the first import under the spoof handed a Windows runner the POSIX fork contexts. get_all_start_methods() then reported fork, the skip guard did not fire, and the pool tried to reap a child the way only POSIX can. Import multiprocess at module scope, before any fixture runs, and read the real platform there too so the two real-pool tests skip on Windows even if something later lies about it. * Tighten the comments added by this PR * Import the num_proc policy from the zoo, not back into unsloth The trainer source rl.py generates ran `from unsloth.utils.dataset_num_proc import ...`. unsloth/__init__.py is what generates that source, so the import reaches back into the package mid-flight, and it also drags unsloth/utils/__init__.py -> packing -> attention_dispatch -> models._utils, which means a module whose only imports are contextlib, os, sys and typing arrives through torch and the whole model stack. Nothing circular in practice: the injected imports are function-body imports that run at config construction and dataset prep, and tripping them mid-import at rl, attention_dispatch, packing, llama and chat_templates all resolved. But the coupling is real. Cold, in a process that imports only the compiled trainer cache, it costs a 9.7s `import unsloth`, and it inherits any unrelated failure in that import: on a box with a torchao/torch mismatch the stdlib-only helper failed to import along with everything else. The policy now lives in unsloth_zoo.dataset_num_proc (unslothai/unsloth-zoo#984), which unsloth already depends on and which never imports unsloth. Every call site tries the zoo first and falls back to the copy here, so upgrading unsloth alone still fixes the bug on an older zoo, and a new zoo takes over with no further change. test_the_two_copies_have_not_drifted compares the two, with docstrings stripped, whenever both are importable, so they cannot silently disagree about a worker count. Verified both directions end to end: with the zoo module present the generated config imports unsloth_zoo.dataset_num_proc and never touches the unsloth copy, and with it absent the fallback runs and the config still comes out bounded. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments this PR adds * Mirror the zoo's container and start-method-split fixes unslothai/unsloth-zoo#984 review found three holes in this policy, and the copy here is a code twin of that module, so it takes the same changes. psutil reports the HOST inside a cgroup, so a 2GB container on a large box read as having room for the full worker set, and a one-core pinned job auto-sized workers that contended for that core. Memory is now the smaller of the host reading and the cgroup limit less its current usage, and the CPU count the smallest of the host, the affinity mask and any cgroup quota. resolve_responses_only_num_proc handed the zoo a bare None to mean serial, but the zoo's own veto reads stdlib multiprocessing. Where multiprocess is on spawn while stdlib is on fork, that None is read as "size it for me". It re-encodes as 1 when the two disagree. The CPU-count test patches move to the resolved count: patching psutil alone would let a 4-vCPU runner override a test that asks for 128. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mirror the cgroup usage-path correction from the zoo /sys/fs/cgroup/memory.current is the whole machine's usage at the root, so subtracting it from a systemd unit's own MemoryMax left every run with nothing free. Usage now comes from the directories the limit was resolved from. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Re-run CI Every check on the previous head was cancelled at 16:33 by an Actions-level event that hit both repos and many unrelated branches, main included. * Bound the paths the review found still unbounded Five findings, all about a value reaching Dataset.map without the policy. The config sentinel was written as 1 for every patched trainer. Only SFT has a downstream auto-sizer to defend it against: DPO, KTO, CPO, ORPO, Reward and PRM hand args.dataset_num_proc straight to Dataset.map, where nothing can inflate a None but a 1 is a Pool(1) on datasets >= 4.1 -- one worker holding its own tokenizer copy, on the low-memory host that had just refused workers. The codegen now picks the encoding per trainer. train_on_responses_only with UNSLOTH_DATASET_NUM_PROC=0 and an explicit count returned 1, which bypasses the small-split guard and builds that Pool(1) even on a 100-row split. Under the threshold the guard is in-process, so None is what expresses the request exactly, and that is what it now returns. Studio's dataset_map_num_proc handed its own count straight to callers in format_conversion.py and chat_templates.py with no memory ceiling and no environment override, though the cap's log line advertised one. It now runs the count through the shared policy when unsloth_zoo has it, so those paths get the memory and cgroup clamp and the escape hatch, and the log line no longer names a variable that path never read. The fallback copy moves to unsloth/dataset_num_proc.py. Under unsloth/utils it sat behind an __init__ that imports .packing (torch) and .attention_dispatch (unsloth.models._utils), so a torch-free MLX host with an older zoo raised before train_on_responses_only could delegate. Also found while running the wider suite: the tokenizing map() anchor was required, so a Zoo release moving that line would hard-fail every SFT run over a diagnostic wrapper. It is optional now, like the selection anchor above it, and the drift canary is what reports it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mirror the zoo cgroup fix into the fallback copy unsloth_zoo.dataset_num_proc is the source of truth; this copy exists only so upgrading unsloth alone still fixes the bug, and test_the_two_copies_have_not_drifted holds the two together. Also moves the three new files onto the AGPL-3.0 header the repo now uses for new sources. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep serial requests serial across the config boundary and the drifting anchor Three review points. Studio computes dataset_num_proc for a config, not for a map() call, and the shared policy was applied with the map-site sentinel: the audio and CUDA-audio paths ask for 1, that became None, and SFTConfig read None as "auto-size me" and returned 8. Measured on the generated class, not simulated. The XPU leg was worse: unlike win32 and darwin, forking still works there, so a config None was auto-sized back up and forked the Level-Zero context the guard protects. dataset_map_num_proc now takes serial_as_none, default True so the seven map-site callers are untouched, and the trainer passes False. The spawn platforms keep None at both layers, where nothing can inflate it and a 1 would reach Dataset.map from DPO and friends as a Pool(1). The sft_prepare_dataset num_proc anchor was optional on the grounds that its absence was harmless. It was not: the config layer encodes serial as 1 for that rewrite to turn back into None, and an un-rewritten zoo hands the 1 to Dataset.map, which pools for any count from datasets 4.1. It now falls back to the assignment the block ends with, unchanged in the zoo since Aug 2025 while the block around it was rewritten three times in 2026, and warns only when both anchors miss. Hard-failing instead would break every install on a newer zoo. Two cgroup tests read the host tree once the fallback reader stopped needing unsloth_zoo, so they passed on a laptop and failed in a limited container. They are isolated now, and the six unaided-reader tests from the zoo copy came with them. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip the map-site half of the new tests when the policy is absent Two of the new config-boundary tests asserted dataset_map_num_proc(1) is None without guarding on the policy import, so they failed on all three cross-platform runners: unsloth_zoo there has no dataset_num_proc module yet (it ships in unslothai/unsloth-zoo#984), and _bounded_by_the_shared_policy returns the count unchanged in that case by design. Same pytest.importorskip guard the three memory and env tests beside them already use. With an unsloth_zoo that lacks the module the file is 15 passed, 6 skipped instead of 2 failed. * Let the shared policy see the request Studio was actually given Three points on the hardware.py side. safe_num_proc materialized an auto request before the policy could see it, so the policy never ran its own auto path: it reads this process's CPU affinity and cgroup quota, while safe_num_proc reads the host os.cpu_count(). A 2-core container on a 64-core box asked for cpu_count // 3 workers and was bounded only by memory. The request now passes through as written, and Studio's caps are applied to whatever the policy chose, since the multi-GPU fork-deadlock cap is knowledge the policy does not have. For an explicit count the two orders are equivalent, both being min(studio cap, request, affordable). The escape hatch is unvetoed by contract, but the win32/darwin return fired before the policy could read it, so UNSLOTH_DATASET_NUM_PROC was silently ignored on the platforms whose dead-worker message recommends it. It is now checked before that veto, and Studio's caps never apply to it. The older-zoo path returned the Studio count unchanged rather than trying unsloth.dataset_num_proc, the byte-identical fallback every other call site uses. It is used now, but only when unsloth is already imported: importing it from here would make hardware detection patch torch and pull in the model stack. The torch-less XPU branch routes through the policy too, having been the one path that ignored both the ceiling and the hatch. Seven new tests, 28 total, 17 passed and 11 skipped against an unsloth_zoo without the module. Reverting each of the three fails its own test. * Mirror the zoo test isolation and prose The fallback copy tracks unslothai/unsloth-zoo#984: the dnp fixture pins the memory ceiling at its sources, so a memory-limited runner cannot turn a start-method test into a clamp test, and the dead-worker advice now says that the single-worker exception applies to a Zoo older than the one that reads 1 as in-process. * Honour the hatch on XPU, leave the ordinary case to the policy, ignore typos Three follow-ups to the previous round, all of the same shape as fixes already made one line away. The XPU-initialized return bypassed the policy the way the spawn platforms did before this, so UNSLOTH_DATASET_NUM_PROC was ignored there too. It takes the same route now: the guard exists because fork corrupts the Level-Zero context, but a user who set the variable has accepted that, and unset the veto stands at both layers. The trainer's non-audio branch passed max(1, os.cpu_count() // 4), which the policy reads as an explicit request and so skips its own auto path, the only one that consults this process's affinity mask and cgroup quota. It passes None now, and Studio's caps still apply to whatever the policy chooses. The override probe treated any non-empty value as active, but the policy warns about and ignores an unparseable or negative one, so a typo skipped the multi-GPU cap while contributing nothing. It reads the parsed result through the zoo's new environment_override(), falling back to presence on a copy that predates it. Four new tests, 32 total. Reverting the XPU check or the override probe fails three of them; the trainer's None is pinned by an AST guard, since dropping it changes only the worker count on a container. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Name the encoding on the cgroup reads tests/test_runtime_text_encoding.py caught both call sites the unaided cgroup reader added: a locale-dependent text read crashes or produces mojibake on a Windows console codepage, and the gate is absolute even for ASCII kernel files. Mirrors unslothai/unsloth-zoo#984. * Neutralise the zoo cgroup readers by name in the num_proc fixture Pinning hf_xet_tuning.CGROUP_ROOT only works against a zoo that has that global. An older one still exposes the private dir helpers the policy prefers, so monkeypatch finds nothing to pin and the readers walk the runner's real cgroup: under a 2GB memory.max that turns a test about the start method into a test of the clamp. * Patch the zoo cgroup readers through the cache the policy actually uses unsloth_zoo/__init__ imports hf_xet_tuning near the top and only raises "Please install Unsloth" at the end, so a failed package import drops unsloth_zoo from sys.modules and leaves unsloth_zoo.hf_xet_tuning behind. The policy reaches the submodule through that surviving cache entry, so treating the failure as absence left the real readers live on the runner's own /sys/fs/cgroup and every sizing assertion silently became a test of the container's memory limit. * Make the Studio half of this PR actually run, and three tests mean what they say studio-backend-ci is the only job that executes studio/backend/tests, and it installs studio.txt, which carries no unsloth_zoo: 14 of the 32 cases importorskip away there, and with no policy installed the survivors fall back to the pre-PR safe_num_proc, so they would pass with the whole wiring deleted. Run the file in the hard-gate step instead, which has an editable unsloth_zoo, and pin the memory ceiling so the counts are not really assertions about the runner's free RAM. test_env_override_is_uncapped never exercised the exemption it is named for: the fixture leaves room for 512 workers, so asking for 100 was never near the clamp. test_unrelated_errors_pass_through_untouched held under 'except Exception' too, since the guard re-raises the same object; it now also passes a non-RuntimeError carrying the dead-worker text. And the codegen tests supplied their own copy of rl.py's serial_as_none rule, which made them self-fulfilling -- they now read it out of rl.py's AST, so flipping SFT to True fails the behavioural test and not only the literal match. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reach the policy without importing unsloth, so the gate is not inert The CI step I added runs the Studio file in a job with an editable unsloth_zoo, but the policy module is not on unsloth_zoo main -- it is in the companion PR -- and the job clones main, so all 14 cases that reach the policy still skipped and the survivors still exercised the pre-PR path. The claim in that step's comment was wrong. The same gap is live in production: _shared_policy only fell back to the in-repo copy when unsloth was already imported, and no Studio backend module imports it, so the API process reaching format conversion got no policy at all on every install whose zoo predates the module -- the 2GB container with eight cores this PR exists to fix. It now loads the file off disk when the package is not imported, which is safe because the module is stdlib-only by design, and memoises through sys.modules so the warn-once state and the cgroup reads are not redone per map() call. The tests ask _shared_policy for the same object, so they patch what production uses: 32 pass with the zoo copy blocked, where 18 passed and 14 skipped before. Also parenthesise the source segment _rl_serial_as_none evals, matching its sibling: a formatter reflowing that ternary in rl.py turned all eight codegen tests into an IndentationError. And mirror the two cgroup and escape-hatch tests just added on the zoo side. * Count pools at the class, not at a module attribute datasets moved The hard gate I added surfaced this the first time the file ran against HF=latest: datasets 3.x and 4.x do 'from multiprocess import Pool', so datasets.arrow_dataset.Pool exists, but 5.x calls mp.Pool() and a spawn context instead and the attribute is simply gone, so the spy raised AttributeError on four Python versions. Patching multiprocess.pool.Pool's __init__ catches every route. Verified against a real datasets 5.0.1: num_proc=None builds no pool, num_proc=1 builds one, so the claim the test makes about 4.1+ still holds there. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-05 05:31:27 -07:00
Daniel Han	976a1152ad	Ask uv for a Python that can import torch, and skip the one that cannot (#7830) ... * Ask uv for a Python that can import torch, and skip the one that cannot Fixes #7803. CPython 3.13.8 carries python/cpython#139783: inspect.getsourcelines() drops a function body when a decorator is followed by a comment. That is the shape of the @_overload_method blocks in torch 2.11's nn/modules/rnn.py, which are parsed at import time, so `import torch` dies with IndentationError. 3.13.9 was an expedited release carrying only that fix. install.sh asked uv for a bare "3.13" and let it choose the patch. Measured with uv 0.9.2 and only 3.13.8 present: --python 3.13 -> 3.13.8 then import torch: IndentationError --python >=3.13.9,<3.14 -> 3.13.12 then import torch: 2.11.0+cpu OK So the request is the fix. PYTHON_SKIP names the releases that cannot run the stack and _python_request turns a bare 3.13 into the range; a venv left on a skipped interpreter by an earlier run is recreated, on any platform, which the previous check could not do because it was gated on macOS arm64. UV_MIN_VERSION also moves to 0.9.3, the first uv whose bundled manifest carries 3.13.9. That is belt-and-braces rather than the fix, since the range resolves on 0.9.2 too. Raising it pulls every 0.8.16-0.9.2 host into the refresh block, so an existing uv in that range is no longer fatal when the network is unreachable. Windows reaches such an interpreter differently: uv is handed a resolved path, never a version, so it cannot pick the patch, but Find-CompatiblePython matches on the minor version and would return an already-installed 3.13.8. Remove-SkippedPython turns that into "not found" so the caller installs $PythonFallbackFullVersion (3.13.13). The uv floor is left alone there, since the uv-managed Python path is not taken on Windows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Screen the skipped patch inside the resolver, and never delete the only venv Windows: Find-CompatiblePython matched on the minor, so the screen sat on its result and every other caller of the resolver -- the two install helpers among them -- could still be handed the interpreter the first call had just rejected. Nulling the result also ended the search, so a host with 3.13.8 and a healthy 3.12 failed instead of using the 3.12. The patch is already in the string the minor comes from, so screening during enumeration costs no extra subprocess and keeps the fallback ladder. install.sh: the legacy-layout migration moves the old environment into $VENV_DIR without arming the rollback, so a plain rm -rf before a recreate that then fails leaves the machine with nothing. Move it aside through the existing rollback machinery instead. Ask uv for the series minus the skipped patches rather than for a floor above them: an offline host, or a uv whose manifest predates 3.13.9, can still have a good cached 3.13.7 that a floor would refuse. Measured with uv 0.10.7, only 3.13.7 and 3.13.8 installed, --offline: "3.13" gives 3.13.8, ">=3.13.9,<3.14" errors, ">=3.13,<3.14,!=3.13.8" gives 3.13.7. Hoist the install.ps1 extractions out of the f-strings: a backslash in an f-string expression is a syntax error before 3.12 and the repo is 3.9+. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Run the resolver driver only where its fake launcher can execute The launcher is a /bin/sh script, and Windows has neither a shebang nor a PATHEXT entry for an extensionless file, so Get-Command never finds it and Find-CompatiblePython reports "none" whatever versions the tree holds. Two cases failed on a real windows-latest runner and the third passed for that reason rather than on merit. Skip the three on Windows, where the rest of the file still covers the screen, and pair the negative case with a positive control so a harness that cannot run the launcher fails instead of quietly agreeing. * Do not screen the interpreter for an install that never imports torch Every entry in the skip list is there for one reason: it cannot import torch. A --no-torch/-NoTorch install never does, so refusing the machine's only 3.13 would send a locked-down GGUF-only host into a download it may not be able to complete, over a package it will not install. Also turn away anything that is not a plain X.Y before the arithmetic: a relative --python path like 3.13/bin/python survived the globs, and dash aborts the whole install with "Illegal number" rather than reaching uv. And keep the uv version probe alive on an image with no awk, which is precisely the host the offline exception around it exists to protect -- the pipeline exits 127 there and set -e was killing the install before the code could treat the version as unreadable. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-05 03:03:39 -07:00
oobabooga	f8730f4339	Installer: select CUDA wheels that cover the host's GPUs (#7814) ... * Installer: select CUDA wheels that cover the host's GPUs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Windows venv wipe and warning dedupe for PR #7814 - Windows pins torch<2.11, whose cu128 still ships sm_70, so capping a Volta to cu126 there rewrote a working family. The stale-venv check then read that as drift and deleted the venv on a direct "unsloth studio update", which cannot recreate it. Make the pre-Turing floor per-family (70 for cu128). - Repair an unpinned cu* -> cu* move in place instead of rebuilding the venv. - Decide the cu126 advice before deduping the uncovered-host warning: the host facts are release invariant but the artifact list is not, so the release walk-back let an unhelpful release swallow the remedy. - Gate the new coverage repair and the cu126 advice on x86_64, matching the cap. - Add tests/studio/test_pre_turing_cap.ps1: the parity test only greps for the call spelling, so neither PowerShell copy had behavioural coverage. * Tighten comments for PR #7814 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-04 06:55:29 -07:00
benj	6a58ea0f0e	Add Intel Arc GPU detection and XPU PyTorch install to Windows installer (#7706) ... * Add Intel Arc GPU detection and XPU PyTorch install to Windows installer The installer's GPU detection chain (NVIDIA -> AMD ROCm -> else) has no Intel Arc/SYCL/XPU branch, so Intel Arc GPUs fall into the "none (chat-only / GGUF)" branch and get CPU PyTorch despite PyTorch publishing XPU wheels at download.pytorch.org/whl/xpu. This adds: - WMI-based Intel GPU detection (Arc, Iris, UHD, HD Graphics) - Torch XPU availability check for migrated/upgraded environments - An XPU PyTorch install path with the whl/xpu index - CPU fallback with a pointer to the Intel oneAPI docs when XPU isn't available - Updated messaging from "NVIDIA or AMD ROCm" to include Intel Arc The XPU wheels ship their own oneAPI runtime (intel-sycl-rt et al.) so no Intel oneAPI Base Toolkit is required for GPU training. Tested on: Windows 11, Intel Arc 140V GPU (8GB), PyTorch 2.9.0+xpu Co-authored-by: CommandCodeBot <noreply@commandcode.ai> * Fix Intel XPU detection and install path for PR #7706 The XPU index selected during GPU detection was overwritten by Get-TorchIndexUrl before the install branch read it, so Intel hosts still got CPU PyTorch while being told XPU wheels were being installed. - Move the XPU reroute after Get-TorchIndexUrl, and let an explicit pin win - Detect via Get-CimInstance (Get-WmiObject is absent in PowerShell 7) - Match only Arc / Data Center GPU, so UHD / HD / Iris Xe are not promised XPU - Split Intel GPU present from XPU-capable so the CPU fallback hint works - Bound the XPU torch trio like every other index (bare names resolved torch 2.13.0 + torchaudio 2.11.0 and pulled unsloth back to an old release) - Clear the XPU state after a CPU fallback, mirroring the ROCm path - Teach the index family, GPU branch and torch flavor helpers about xpu * Keep install.sh diagnostics in parity with the install.ps1 xpu family install.ps1 now classifies an /xpu index leaf as family xpu / branch xpu, so mirror the same two cases in _tauri_torch_index_family and _tauri_gpu_branch. These feed the [TAURI:DIAG] line only, and a Linux user can already reach the xpu index via UNSLOTH_TORCH_INDEX_FAMILY, where it previously reported auto/unknown. Linux Intel auto-detection is not added here. * Tighten the Intel XPU comments in install.ps1 Comment and whitespace only, no code change. * Address Codex review on the Intel XPU path - Run the Intel scan before the GPU report chain instead of inside its final else. A WMI-named-only AMD adapter set ROCmGpuLabel and took that chain, so a discrete Arc card next to an AMD CPU's integrated Radeon was never detected. The scan is gated on no usable NVIDIA or AMD, and the Intel branch ranks above the two AMD-present-but-unusable branches, so a usable AMD host is unaffected. - Let a migrated env's torch veto the hardware match only when it is itself an XPU build. A CPU build reports torch.xpu.is_available() False for lacking XPU support, not for unsuitable hardware, and was blocking the CPU to XPU upgrade. - Detect Intel in studio/setup.ps1 too. It only knew NVIDIA and AMD, so every successful Intel install printed none (chat-only / GGUF) right after install.ps1 reported a usable Arc GPU. Self-contained so studio update works. * Address the second Codex round on the Intel XPU path - Reset $script:IsIntelXpu at the start of each invocation. Under the documented irm | iex path $script: is the caller's session scope, so a second run in the same session inherited a stale true, skipped the scan on a now-NVIDIA host and still rerouted to the xpu index. Reproduced in pwsh before fixing. - Gate the Intel scan on whether AMD actually gets a wheel, not on whether an AMD arch was seen. An arch missing from the family map has no ROCm wheels and lands on CPU torch, so it must not outrank a usable Arc card. The map is hoisted above the scan and consumed unchanged by the AMD reroute. - Select the XPU index in studio/setup.ps1, not just report it. Previously setup printed Intel GPU detected and then installed CPU torch, so studio update never migrated an Arc box off CPU. Adds a bounded XPU install with a CPU fallback, teaches the stale-venv check about +xpu, and mirrors the wheel-aware AMD gate so the two files agree instead of wiping the venv on every update. * Address the third Codex round on the Intel XPU path - Force the dependency pass on an Arc host whose torch is not XPU-capable, the Intel counterpart of the existing AMD escape. Without it the fast up-to-date path skipped the install block, so the xpu index selection was never reached and a CPU venv never migrated. - Confirm a working XPU runtime before treating an xpu venv as stale. If CIM is unavailable or returns an Intel name outside the Arc match, the expected tag fell through to cpu and a valid XPU environment was rebuilt and lost. - Force-reinstall the XPU trio only when the installed wheel is not already +xpu, or the pin changed. It was unconditional, so a fresh install re-fetched multiple GB immediately and again on every update. - Warn when torch.xpu.is_available() is false after installing XPU torch, naming the Intel driver floor. Otherwise the installer promised GPU training while unsloth raised NotImplementedError at import on a stale driver. - Stop the detection probe vetoing the hardware match. Its cpu fallback could not displace the installed +xpu wheel, so it only mislabelled a capable GPU as unusable; the driver warning covers that case honestly, and setup.ps1 agrees. * Bound the XPU probes, repair xpu pins in install.sh, and floor bitsandbytes on the Intel path install.sh: teach _torch_flavor_tag, _expected_torch_flavor_tag and _torch_index_repairable about the xpu leaf. The diagnostic already reported gpu_branch=xpu, but an xpu pin fell to the custom arm so a migrated env kept its CPU wheel. The +xpu flavor arm is required alongside, otherwise a correct 2.10.0+xpu wheel reads as cpu and gets force-reinstalled every run. install.ps1 / studio/setup.ps1: route every torch probe through a new bounded Invoke-BoundedPythonProbe (ProcessStartInfo, both streams drained async, WaitForExit, kill on timeout). A hanging Intel driver init is exactly what these probes detect, and an unbounded one would hang the installer instead of reaching the warning. Timeouts read as not-available. Get-InstalledTorchTag now shares the helper rather than carrying a second copy of the pattern. install.ps1: install bitsandbytes>=0.50.0 on the XPU path. unsloth's floor is >=0.45.5, so a migrated venv keeps a pre-0.49 wheel with no XPU library and 4-bit QLoRA silently turns off. Same floor the AMD paths use, since <=0.49.2 NaNs at 4-bit decode and an Arc card can sit next to a Radeon. * Floor bitsandbytes on the Studio XPU migration and on an explicit xpu pin studio/setup.ps1: `unsloth studio update` migrating a CPU venv to XPU replaced only the torch trio. install_python_stack.py then upgrades unsloth and unsloth-zoo alone, so an installed bitsandbytes 0.45.x kept satisfying the base floor while carrying no Windows XPU kernels, and 4-bit QLoRA silently turned off. Adds the same bitsandbytes>=0.50.0 --no-deps pass install.ps1 got, placed after the stack so it is the last word, gated on $XpuIndexUrl (the CPU fallback clears it, no-torch never sets it) and still inside the -not $SkipPythonDeps block so the up-to-date escape does not reach it. install.ps1: key the bitsandbytes pass off the index leaf instead of $script:IsIntelXpu. An explicit UNSLOTH_TORCH_INDEX_FAMILY=xpu pin on a non-Intel host skips the XPU branch but still installs the trio from the xpu index, so torch is +xpu and needs the same floor. The CPU fallback rewrites $TorchIndexUrl, so a failed XPU install reads as cpu and stays quiet. * Tighten the Intel XPU comments across the three installers Comment-only pass now that the review has settled: several blocks grew over successive rounds and were restating the code or narrating the review. Net 36 lines removed, with the load-bearing facts kept -- why ProcessStartInfo rather than the call operator, why both probe streams drain async, why the helper is defined above the Intel scan, the 0.50.0 bitsandbytes floor and why not the curated extra, and why PEP 440 means a migrated env can confirm but never veto the Intel match. Also records why the Studio bitsandbytes pass must stay above the ErrorActionPreference restore: Fast-Install needs EAP=Continue or PS 5.1 turns pip stderr into a terminating error. No code tokens changed; verified with a PowerShell token-stream diff of install.ps1 and setup.ps1, and by hand for install.sh. * Bound the Intel WMI scan, bound the stale flavor probe, and stop CUDA Triton shadowing XPU studio/setup.ps1: the stale-venv flavor probe read StandardOutput.ReadToEnd() before WaitForExit, so the timeout was unreachable and a wedged import torch hung studio setup forever; stderr was never drained either. Routed through Invoke-BoundedPythonProbe, which already drains both streams and kills on timeout. A timeout now reads as unreadable flavor, so the venv rebuilds. install.ps1 / studio/setup.ps1: bound the Win32_VideoController query and add a registry fallback. -ErrorAction suppresses errors but bounds nothing, and -OperationTimeoutSec is not enforced for the local COM session this uses, so a degraded WMI repository blocks forever. install_llama_prebuilt.py already runs this query out of process for the same reason and documents an Arc A770 being misrouted by it. The registry class key answers in-process; it is the fallback rather than the fast path because a stale driver config can outlive the hardware, and here a false positive would install XPU torch on a host with no Arc. studio/setup.ps1: replace triton-windows with torch's own XPU triton after the stack. Both distributions own the top-level triton package, sharing 151 paths including __init__.py and _C/libtriton.pyd, so an in-place cu-to-xpu repair leaves the CUDA build shadowing the XPU one. Removing it alone would delete the shared files the XPU wheel overwrote, and unsloth declares triton-windows as a win32 dependency so an earlier removal is reinstalled by the stack: uninstall and reinstall, after the stack, only while triton-windows is present. The spec is read from the installed torch, since the name changed from pytorch-triton-xpu to triton-xpu in torch 2.10. * Tighten the comments added with the bounded scan and Triton replacement Comment-only pass over the previous commit's additions, which had not been through one: 15 lines removed across the two bounded-scan headers, the two registry-fallback headers and the Triton block. Kept the facts that cost measurement: -OperationTimeoutSec not being enforced for a local COM session, Ok being false on an empty answer because a Windows host always has an adapter, the registry class key being fallback rather than fast path here, the 151 shared Triton paths, and why the uninstall has to be paired with a reinstall after the stack. No code tokens changed; verified with a PowerShell token-stream diff of both files, which also confirms the two helper copies stay identical. * Stage the Triton replacement behind a download so the uninstall cannot strand the venv The replacement uninstalled triton-windows and then installed the XPU triton from the index. A failure between the two left the venv with a partially deleted triton, since the uninstall drops the paths shared with the XPU distribution, and the warning made that look like a skipped optional repair. The uninstall cannot go last, because it removes the paths in triton-windows' own record and those are the shared ones. So fetch first: pip download the wheel, confirm one is actually on disk (exit 0 alone is not enough, an sdist-only mirror satisfies that), and only then uninstall and install the local file. A local wheel installs with the network refused, so nothing after the destructive step depends on the index. A failed fetch leaves triton-windows in place, which is the pre-existing shadowing rather than a broken venv, and says so. Past that point only disk or permissions can fail, so restore triton-windows if the local install does, leaving a triton that imports. If both fail the message is loud and carries the repair command, with the index URL redacted since a mirror pin can carry a token. pip only: uv has no pip download (astral-sh/uv#3163). * Windows: harden the Intel registry fallback and declare the XPU install state up front Get-IntelRegistryAdapterNames wrapped the whole enumeration in a single try, so one unreadable subkey discarded every adapter found before it. windows_intel_gpu_in_registry(), the in-process Python probe over the same class key, skips per subkey and continues; the PowerShell copy now does too. It also matched on the PCI vendor id but returned DriverDesc, which the callers re-filter on "Intel", so a localized or OEM-branded Arc was found here and dropped there. Both installers carry the same copy and a test asserts they stay identical. setup.ps1 read $installedTorchTag and $XpuIndexUrl from outside the blocks that assign them. Unset and $null are both falsy so behaviour is unchanged, but a caller running with Set-StrictMode -Version Latest turned those reads into terminating errors, and install.ps1 is documented as irm | iex into the caller's own session. Two comment corrections: 0.48.2, not 0.49.0, is the first win_amd64 bitsandbytes wheel carrying libbitsandbytes_xpu.dll, and the triton package overlap is version-dependent rather than a fixed 151 paths. The new test drives the shipped helper with the registry cmdlets mocked rather than reading a hive, so it runs on Linux and macOS as well as Windows. * Studio: show the Intel XPU runtime row in the About tab hardware.py has always emitted versions["xpu"], but HardwareInfo only ever declared cuda and rocm. On an Arc host both of those are null, so the runtime row disappeared entirely while the GPU name and VRAM rows still rendered, leaving a host that looks half detected. That was unreachable on Windows until the installer learned to select XPU wheels, which is what makes it worth fixing here. The three-way choice is lifted into a helper at module scope: inlining it pushes AboutTab past the cognitive-complexity ceiling. The label is a proper noun, so every locale carries the same literal. * Windows: reach Intel XPU through a localized name, a stale fast path and an old wheel Four holes in the XPU paths, all found by driving the shipped code rather than reading it. The registry fallback only ran when the CIM scan failed. When it succeeds and returns a localized adapter name, which on non-English Windows carries no ASCII "Intel", the filter dropped the adapter and the host went to CPU torch. The registry now re-labels an adapter WMI already reported, matched by name so an entry naming nothing WMI listed stays ignored: a driver record outliving its card still cannot promote a host WMI answered for. The XPU trio accepted torch 2.4 and 2.5, which unsloth/models/_utils.py rejects at import for an XPU device. An xpu mirror carrying only an older wheel produced an install that reported success and then failed on the first import, and an existing 2.5+xpu venv was kept because it satisfied the range. The floor is 2.6 on the XPU paths only; the CPU fallback keeps 2.4. The "package is up to date" fast path escaped for an Arc host on CPU torch, but not for one already on XPU torch whose bitsandbytes predates the XPU kernels or whose triton-windows still shadows the XPU Triton. Those two live in the dependency pass, so a venv that reached +xpu without them, an explicit pin or an update whose first pass ran the pre-XPU setup.ps1, never got them on any later update either. An unreadable version reads as stale. install_python_stack.py writes its completion manifest immediately before returning, so an interrupt between the triton-windows uninstall and the XPU wheel install left a venv with no triton that the next update read as complete. The manifest is now held aside across the swap and restored only once a triton is importable again. * Windows: move the install manifest across the Triton swap instead of rewriting it Two problems with the hold added in 2603fc809, both on the restore side. Reading and rewriting the file cannot survive a manifest carrying a non-ASCII path. Windows PowerShell 5.1 writes Set-Content in the ANSI code page by default, and its -Encoding utf8 emits a BOM that install_manifest.read_manifest's json.load rejects outright ("Unexpected UTF-8 BOM"); Get-Content is ANSI on a BOM-less file too, so the read lost bytes before the write got a chance to. The manifest is now MOVED into the wheel's temp directory and moved back, so no encoding is involved at either end. That directory is already removed in the finally, which is what keeps an unrestored manifest gone. A manifest that would not move left the old valid one in place for the whole destructive window, since the failure only cleared the saved copy and carried on into the uninstall. That is the case the hold exists for, so it now skips the swap entirely and says so: triton-windows keeps shadowing the XPU Triton, which costs torch.compile on the GPU and is repairable on the next run, rather than risking a venv with no Triton that reads as complete. * Windows: confirm the install manifest actually moved before the Triton swap Move-Item across volumes is a copy followed by a delete, and it reports success when only the delete fails, leaving the original exactly where it was. So the guard added in af928dd88 could believe it had set the manifest aside while a valid one sat there for the whole destructive window, which is the case that guard exists to prevent. Found by modelling the manifest in the setup.ps1 scenario matrix, which this had no coverage for: with the parent directory read-only the swap still ran, and the locked scenario passed for the wrong reason. The move is now confirmed by testing the source path afterwards, and a manifest still standing aborts the swap like any other failure to move it. Four new scenarios cover it: the swap keeping a byte-identical manifest, a swap where neither Triton reinstalls correctly leaving it gone, a failed fetch never touching it, and a manifest that cannot move aborting the swap. * Windows: key the XPU fast-path remediation off the installed wheel, not just the GPU scan $HasNvidiaSmi suppresses the Intel scan, so on a mixed NVIDIA + Intel box under an explicit xpu pin $script:IsIntelXpu stays false while the pin still lands the venv on a +xpu wheel. The staleness check added in 2603fc809 was gated on that flag alone, so those hosts kept taking the fast path and never reached the bitsandbytes floor or the Triton replacement. This is the same gating mistake the bitsandbytes pass had in round 4, where the fix was to key off the index leaf rather than the scan. The leaf is not resolved yet at the fast path, but the installed flavor tag is, and whatever put the venv on a +xpu wheel the two remediations still apply. The runtime probe above stays on the scan: reinstalling XPU torch is only right where an Intel GPU was actually found. A pure NVIDIA host on a cu wheel never runs the probe, which the matrix asserts alongside the two new mixed-host rows. * Windows: reconcile Intel names for hybrid GPUs, and stop the XPU escapes firing where XPU is unreachable Five fixes from a review of the XPU work so far. The registry reconciliation was gated on "no ASCII Intel name present", so a hybrid laptop reporting its Intel UHD alongside a localized Arc stopped at the UHD and left the Arc unrecognised. It is now gated on the absence of an XPU match, and the regex behind both that gate and the classification is defined once so they cannot drift. The two fast-path escapes cleared $SkipPythonDeps for any Intel host, but the XPU install and its two remediations are all gated on $XpuIndexUrl, which an explicit cpu / rocm / custom-leaf pin never sets, and no-torch mode has no torch pass at all. Those hosts ran the whole dependency pass, installed nothing new, and re-fired the identical condition on every later update. Both escapes now require XPU to be reachable. The manifest path was learned by a subprocess whose output parsing could not work: `& python` returns one array element per line, interpolating that joins on $OFS, a SPACE, so splitting on newlines yields a single element and a banner ahead of the answer arrives glued to the path. Any such failure then skipped the hold silently and swapped anyway, which is the window the hold exists to close. manifest_path() is venv_root()/MANIFEST_NAME and venv_root() is sys.prefix, which is $VenvDir here, so it is assembled like Get-PersistedNoTorch already does. A test asserts the literal still matches MANIFEST_NAME. The uninstall's exit code was discarded. A triton-windows that will not uninstall, which on Windows means Studio is running and holding libtriton.pyd open, still shadows the XPU Triton, so installing over it achieved nothing and restored the manifest onto a venv this pass was supposed to have changed. The restore had no verification and an empty catch, while the finally deletes the held copy either way, so a failed restore lost the manifest with nothing on screen. * Windows: keep the WMI adapter list an array so the Intel re-label appends instead of concatenating `$_gpuNames = if (...) { @(...) } else { @(...) }` wraps each branch, and a one-element array unrolls on its way out of the if, so on any single-adapter host $_gpuNames was a String. The `+=` that re-labels a localized adapter then concatenated two strings rather than appending a name, and the GPU reported to the user came out doubled: Intel(R) UHD Graphics 620Intel Intel(R) UHD Graphics 620 No install decision changes. The re-label only appends a registry name that already contains the WMI name, so the concatenation matches the Arc / Data Center regex exactly when the registry name alone would, and every scenario in the matrix records the same verdict either way. It is the displayed adapter name that was wrong. Widened by the previous commit: gating on the absence of an XPU match rather than of any Intel name brought ordinary single Intel iGPU hosts into the re-label for the first time. @() now wraps the whole if in both installers, with a test asserting it stays that way. * Windows: give pin-only XPU installs the 2.6 floor, and treat an unreadable dependency probe as stale The XPU install branch required $script:IsIntelXpu as well as an xpu index leaf, so an explicit FAMILY=xpu or URL pin on a host whose Intel scan never ran -- a mixed NVIDIA box, where $HasNvidiaSmi suppresses it -- fell through to the generic branch and its torch>=2.4. Against a mirror carrying an older +xpu wheel that installs a torch unsloth rejects at import. Keyed off the leaf alone now, which is what the bitsandbytes gate below it already does and says in its own comment. install.sh had the same gap from the other direction: its xpu leaf is reachable only by an explicit pin and kept the generic floor, so it gets the same 2.6 trio. The fast-path dependency probe treated "did not answer" as "nothing to do". A timeout, or a malformed .dist-info making distributions() raise, then left the fast path intact and an XPU migration never reached the bitsandbytes floor or the Triton replacement on any later update either. It now clears the fast path, the same direction an unparseable version already took. Two install.ps1 rows move, both FAMILY=xpu pins on non-Intel hosts, both onto the XPU branch. The CPU fallback after a failed XPU install keeps its 2.4 floor. * Windows XPU: probe the preserved venv, drop torchaudio on ARM64, and give POSIX XPU the bitsandbytes floor Three fixes to the Intel XPU paths. install.ps1, migrated-runtime probe: a rerun over an existing install moves the old venv to $script:StudioVenvRollbackDir and creates an empty one in its place, both before this probe runs, so it always asked an interpreter with no torch and answered "no XPU". Ask the preserved environment when there is one, which is the migrated runtime the fallback exists for. install.ps1, Windows on ARM: no win_arm64 torchaudio wheel exists on any index. Keying the XPU branch off the index leaf alone routes an arm64 interpreter into a branch that hardcoded the trio, so the install aborted. Ask the interpreter for its platform tag, as the generic path already does, and drop that one pin on arm64. The CPU fallback below it gets the same treatment. install.sh, XPU pins: bitsandbytes ships XPU kernels (libbitsandbytes_xpu2025.so and _xpu2026.so) from 0.50.0 on manylinux, and nothing on the POSIX side raised the floor for them, so a migrated environment kept a pre-XPU build and lost 4-bit QLoRA on a torch that otherwise works. Matches what the Windows XPU pass already installs. * Studio: stop the xpu label test from forbidding a partial locale check-parity.ts states the contract plainly: "Locale files may be partial; missing keys must fall back to English." The new test required every overlay to carry the xpu label, which contradicts that and breaks on the next locale anyone adds. It already did: it.ts landed on main after this branch, so the merged tree fails on all three runners even though nothing about the label is wrong there. The label is a proper noun, so the English fallback is byte-identical to a translation and the requirement bought nothing. Assert what actually renders wrong instead: en.ts must carry the key, because it is the fallback every locale resolves to, and no overlay may define a value that disagrees with it. Both halves were checked against a merged working tree, and both still fail when the condition they guard is broken. * Linux XPU: hoist the bitsandbytes pass out of the fresh-install arm It sat inside `elif [ -n "$TORCH_INDEX_URL" ]`, which a migrated environment never enters because the `_MIGRATED` arm above it wins, so the one environment the pass existed for was the one that skipped it. The AMD passes handle this by existing twice, once per arm; this gate needs nothing branch-specific, so it moves past the chain instead and both arms reach a single copy. tests/sh/test_xpu_bitsandbytes_reachable.sh guards both halves: the block must be placed where every arm reaches it, and it must still fire only on the xpu leaf. 25 checks over [migrated, fresh] x [xpu, mirrored xpu, cuda, rocm, cpu, none] x [torch, no-torch], run against the block and the leaf parser extracted from install.sh. Moving the block back inside an arm fails it. * Report the XPU runtime before the hardware summary, and show every runtime in About setup.ps1: the hardware report runs ~1300 lines before the torch.xpu.is_available() check that keeps an XPU environment, so a host the WMI scan and the registry fallback both miss (wedged CIM service, an Intel part outside the Arc|Data Center regex) was told "none (chat-only / GGUF)" and then watched setup keep the XPU venv. Ask the same question before printing, so the report and the decision cannot disagree. A free disk read gates the interpreter launch: torch/version.py carries the local label, so a CPU-only host never pays for an `import torch` on every `studio update` just to be told it has no Intel GPU. The dist-info name cannot be used for this -- pip normalises the local label out of it (torch-2.9.1.dist-info for a +cu128 wheel). The promotion carries its own try: it must still run when the scan threw, which is the case it exists for, and a junk UNSLOTH_STUDIO_HOME would otherwise abort setup from Join-Path. about-tab.tsx: hardware.py reads versions["cuda"] off torch.version.cuda and sets versions["xpu"] from an independent torch.xpu.is_available() probe, and UNSLOTH_FORCE_XPU=1 is a supported configuration where CUDA is present but XPU is selected. Both are non-null there, so returning the first match hid the XPU row on exactly the host it was added for. Collect every reported runtime instead. tests/studio/test_setup_xpu_runtime_prereport.ps1 covers the two new helpers with the filesystem mocked, so it runs on all three runners: override precedence, ~ expansion, the four wheel flavours, a missing or unreadable version.py, and wiring assertions that the promotion precedes the report and that the cheap read gates the probe. The About-tab test gains a case that fails if the picker returns early again. * POSIX: recognise a working XPU runtime, and raise the bitsandbytes floor on the update path The hardware summary tested NVIDIA, AMD and Apple Silicon and then fell through to "none (chat-only / GGUF)", so a Linux host running the +xpu wheel install.sh had just installed was told training needs an NVIDIA or AMD GPU. Added an arm ranked below both, matching setup.ps1. The bitsandbytes floor was also unreachable on the route an existing XPU user actually takes. `unsloth studio update` runs this file, never install.sh (see the note at the top of setup.sh), and neither this file nor install_python_stack.py had an XPU floor, while unsloth's own dep floor is 0.45.5 -- which a pre-XPU wheel satisfies indefinitely. So 4-bit QLoRA stayed unavailable on a torch that otherwise works. One detection serves both, but they read different signals on purpose. The floor keys on the WHEEL (+xpu, read off torch/version.py) and the summary keys on the RUNTIME (torch.xpu.is_available()): a +xpu wheel installs fine on a host whose driver never initialises, and that host should still get the kernels while no GPU is claimed for it. The disk read gates the interpreter launch, so a CPU-only host pays nothing per update. tests/sh/test_setup_xpu_posix_summary.sh builds real venv trees, version.py files and stub interpreters rather than mocking, so the disk read and the runtime probe genuinely execute: 13 checks over the four wheel flavours, working/dead/missing runtime, no venv, and the arm's rank. Removing the arm fails four of them. * POSIX XPU: make the bitsandbytes step nonfatal, bound the probe, and act on an XPU pin Three defects in the POSIX XPU code from the previous commit. run_quiet routes failure to setup_fail and exits, so the best-effort bitsandbytes upgrade could abort an otherwise fine `studio update` over a transient download, and the warning after it was unreachable. run_quiet_no_exit is the nonfatal wrapper. The runtime probe had no timeout. A stalled Intel driver wedges inside `import torch`, which is exactly the host this probe classifies, so it could hang every update forever. Bounded at 60s rather than the 10s the smi probes use: a cold `import torch` takes seconds by itself and a short bound would read a healthy host as having no GPU. Systems without coreutils timeout keep the previous behaviour rather than losing detection. An explicit XPU pin was protected but never acted on. An xpu leaf names no family the cuda/rocm repair helpers know, so _explicit_unknown_family_torch_index_url makes both skip it, and `unsloth studio update` never runs install.sh -- so switching a CPU install to UNSLOTH_TORCH_INDEX_FAMILY=xpu left the CPU wheel in place indefinitely. The fix goes in install_python_stack.py, which already parses the pin, rather than setup.sh, which has no pin awareness at all: _ensure_xpu_torch mirrors the existing _ensure_cpu_torch, the xpu leaf is classified so the backend is no longer unknown, and the ROCm helper skips an xpu backend so it cannot treat the pin as an AMD host. Windows is excluded because setup.ps1 owns torch there and installs the trio itself. That put the XPU trio in a third file, so tests/sh/test_xpu_torch_spec_parity.sh asserts the floors match across install.sh, install_python_stack.py and install.ps1 plus the wiring. Each of its four structural guards was mutation-tested: a drifted floor, a lost classification, wiring at one call site instead of two, and the ROCm skip removed all fail it. The POSIX summary suite gains checks for the nonfatal wrapper and the bound. * Linux XPU: swap generic Triton, gate the pin repair on the version, and escape the fast path Three defects in the XPU code from the previous commit. _ensure_xpu_torch returned on the +xpu tag alone, so a migrated 2.5+xpu venv was left in place even though unsloth/models/_utils.py raises at import for an XPU device below 2.6. It now returns only when the flavour and the supported range both match. That repair was also unreachable on the route it was written for. setup.sh skips install_python_stack entirely when the package version is current, and that pass is the only thing that acts on an XPU pin, so a CPU install switched to the xpu family stayed CPU. Added a third fast-path escape beside the anyio and incomplete-manifest ones. Generic triton and torch's pytorch-triton-xpu / triton-xpu both own the top-level triton package, and resolving unsloth against a pinned +xpu torch pulls both -- uv reports pytorch-triton-xpu 3.5.0 alongside triton 3.7.1 -- so the CUDA-oriented build lands last and torch.compile loads the wrong library on an Intel GPU. This is the POSIX half of the Windows swap: the spec is read from torch's own metadata, so the pytorch-triton-xpu to triton-xpu rename at torch 2.10 needs no hardcoding, and the fetch happens before the uninstall because the uninstall drops the shared paths from generic triton's own record. test_torch_installs_do_not_use_deprecated_index_url forbade --index-url on "$TORCH_INDEX_URL" anywhere in install.sh. That rule is about uv, which deprecated the flag in favour of --default-index; pip never had --default-index, so the pre-fetch legitimately uses it. The assertion is now per occurrence and exempts pip download only, and it joins backslash continuations first, since the flag and its command are routinely on different physical lines. Both a same-line and a continuation-line uv offender were mutation-tested and are still caught. tests/sh/test_xpu_triton_swap_posix.sh asserts the swap by execution -- ordering, the rename, no generic triton, torch wanting CUDA triton, non-xpu index, no-torch, empty index, and a dead mirror that must warn without removing anything. * XPU: move the Triton swap where both routes reach it, and bootstrap pip for it Five defects in the XPU code from the previous commits. The Triton pre-fetch could never have run. `uv venv` is created without --seed, so a fresh venv has no pip and `python -m pip download` fails with "No module named pip" every time, leaving the swap a no-op that only ever warns. My shell test missed it because its stub interpreter answered pip commands. install.sh already bootstraps pip this way before its pre-release bitsandbytes wheel. The swap also never ran on `unsloth studio update`, which runs setup.sh and never install.sh. Both fixes fall out of moving it: install.sh runs setup.sh, which runs install_python_stack.py, so that module is the one place both routes pass through. The install.sh copy is deleted rather than duplicated, and the shell test is replaced by tests/studio/test_xpu_triton_swap.py, which covers the no-pip case and asserts install.sh carries no second copy. The fast-path pin match missed authenticated and fragmented mirrors (https://mirror/whl/xpu?token=...), which read as "no XPU pin" and skipped the repair; query and fragment are now stripped before the leaf test. That escape also launched an interpreter, which a wedged Intel driver hangs inside. It now reads the local label out of torch/version.py instead: nothing to bound, and a CPU-only host pays nothing per update. setup.ps1's fast path asked only whether XPU was available. A 2.5+xpu build answers yes and is still rejected by unsloth/models/_utils.py at import, so it now checks the supported range too, via Test-TorchXpuVersionSupported. The POSIX suite is up to 22 checks; the three new guards were mutation-tested by removing the query strip, the fragment strip, and by making the escape launch an interpreter. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * XPU: make a failed Triton swap unsurvivable, and widen the fast-path escapes Five defects in the XPU code from the last two commits. The Triton uninstall ignored its return code. A read-only or locked venv leaves generic triton registered, so installing over it lets a later upgrade of that distribution delete the shared files again, and every dependency pass repeats the swap. A failed uninstall now changes nothing at all. Past the uninstall the venv has no triton, because the uninstall takes the shared top-level files with it, so a warning there let the caller write a completion manifest over a venv whose torch.compile is broken -- and the next update fast-paths straight past it, since no generic distribution is left to trigger on. That install is now fatal. _ensure_xpu_torch returned when the probe timed out. On this path a wedged `import torch` is evidence rather than noise: the usual cause is a stalled Intel driver under an unsupported +xpu wheel, which the resolver keeps because it satisfies the base range. An authoritative pin now repairs on an inconclusive probe. This deliberately differs from the CPU counterpart, where a wedge has no such likely cause. The fast-path pin match stripped one trailing slash, so a ".../whl/xpu//" pin still read as no pin. It now strips them all, like the shared leaf parsers. Moving the Triton swap into the Python stack left the fast path with no reason to run it: a migrated environment with supported +xpu torch and a leftover generic triton kept the CUDA-oriented build forever. A stale generic triton now forces the dependency pass too, detected from the dist-info name so no interpreter is launched. The POSIX suite is up to 26 checks and the Triton tests to 16. Two of the guards were rebuilt after their own negative controls found them vacuous: the stale-triton check matched the detection loop rather than the branch that acts on it, and a fixed line window had drifted off the code it was meant to cover, so it is now anchored on the block. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not require the XPU pin again after install Three Intel paths still assumed the pin was still in the environment, or that every XPU host looks like x64 Linux. install_python_stack.py: the generic-Triton swap returned unless UNSLOTH_TORCH_INDEX_URL / _FAMILY was set. That pin is one-shot -- a user who ran UNSLOTH_TORCH_INDEX_FAMILY=xpu ./install.sh has nothing left in the environment by the next plain `unsloth studio update`, yet that update's dependency pass can pull generic triton back in and shadow torch's XPU build again. The installed +xpu wheel is the durable signal (setup.sh already raises the bitsandbytes floor off it), so fall back to it and to the default xpu index. The label is read off disk: importlib.metadata drops the local version label, and `import torch` loads the SYCL runtime, which can wedge. install.ps1: the flavor repair built its own XPU trio including torchaudio, which has no win_arm64 wheel on any index. A migrated ARM64 venv skips the fresh XPU branch and takes this path, so the repair failed outright before setup.ps1 could reach its ARM-aware fallback. One builder now serves both sites, since the two copies drifted the moment only one learned about ARM. install.sh: adding the xpu tag made the final flavor guard reachable on an Intel pin, and it probes with an unbounded `import torch`. On a host whose driver initialization wedges that hangs the installer, with no timeout anywhere before setup.sh's bounded probes. The xpu path reads torch/version.py off disk instead; every other family keeps the interpreter read unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Key the setup XPU paths on the installed wheel, not the pin Follow-up to d07c179f: install_python_stack now treats the installed +xpu wheel as the durable signal, but the two callers upstream of it did not. setup.sh fast path: the escape only ran under `case $_setup_pin in *xpu`, so after a one-shot UNSLOTH_TORCH_INDEX_FAMILY=xpu install every later `studio update` saw no pin, kept _SKIP_PYTHON_DEPS=true and never reached the Triton swap at all -- generic triton kept shadowing the XPU build forever. The disk read now happens unconditionally and the swap escape keys on the wheel. The pin leaf is also compared exactly, like the shared index parsers: a custom mirror ending in -xpu was classified as the curated family, which cleared the skip flag on every up-to-date run while _ensure_xpu_torch declined to act. setup.ps1: bounding the flavour probe turned a timeout into "rebuild", and the host most likely to time out inside `import torch` is an Arc box whose compute driver stalled -- where torch/version.py still names a good +xpu wheel. With no currently exported pin the stale path then deleted the venv. It now falls back to the same disk check and warns about the driver. Other families still rebuild on an unreadable flavour. setup.sh summary: a +xpu wheel whose runtime will not initialise fell through to "none (chat-only / GGUF)", telling an Arc owner their hardware is unsupported and hiding the driver update that fixes it. It gets its own arm. * Stop the XPU paths from stranding or wiping a venv Four ways the Intel paths could still leave a user worse off than before they ran anything. setup.ps1 stale check: on a hybrid NVIDIA + Arc host the XPU promotion is gated on -not $HasNvidiaSmi, so a pinless `unsloth studio update` expects a cu* tag, calls the working Arc venv stale and DELETES it -- then exits, because only install.ps1 creates venvs. A direct update now keeps any +xpu venv and says to re-run install.ps1, which rebuilds with a rollback copy. setup.ps1 Triton swap: when the staged XPU wheel failed to install after triton-windows was removed AND the generic restore also failed, the branch only printed. $stackExit stayed 0, so setup reported success and install.ps1 committed a venv with no importable triton over its rollback. It now carries the real failure code into the existing handler. install_python_stack: the `pip download` that stages the XPU Triton wheel inherited the user's pip index environment. PIP_NO_INDEX makes pip ignore --index-url outright, and PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS are consulted in addition to it, so the fetch could fail (leaving generic Triton shadowing the XPU build) or serve the wheel from an index the pin never named. It now takes the same _install_env_for_cmd scrub every other pinned install gets. setup.sh runtime probe: the arm taken when coreutils `timeout` is absent ran the probe with no deadline, on exactly the stalled-driver host the bounding exists for. The deadline now lives inside the probe as signal.alarm, which terminates the process even while the driver blocks in C. * Keep a preserved XPU venv on the XPU index Follow-up to 10ba6e31c, which stopped a direct update wiping a +xpu venv on a hybrid NVIDIA + Arc host but left the rest of the pass believing the host was CUDA. The index chain prefers NVIDIA over Intel, and the CUDA arm does not --reinstall-package torch, so uv left the +xpu wheel in place as satisfied while installing triton-windows over torch's XPU triton -- and with $XpuIndexUrl null nothing swapped it back. A half-converted venv is worse than either end state, so the preserved case now selects the xpu leaf, ahead of the NVIDIA arm and behind an explicit pin. The hardware report is untouched: there really is an NVIDIA GPU in the machine. install_python_stack: an inconclusive XPU probe was always read as a flavour mismatch, but on a stalled Intel driver under a SUPPORTED wheel that is two 90-second hangs and two force-reinstalls of the whole trio on every update, repairing nothing. The disk answers what the probe cannot, so a supported wheel now yields the driver warning and an unsupported or missing one still repairs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Lowercase the setup.sh pin leaf like every other index parser install.sh's _torch_index_url_leaf, setup.ps1's Get-TorchIndexLeaf and install_python_stack's _torch_index_leaf all lowercase before classifying. This copy did not, so UNSLOTH_TORCH_INDEX_FAMILY=XPU (or a URL ending in /XPU) left the leaf uppercase, the equality test against "xpu" failed, and the fast path stayed on. Those same classifiers call that pin XPU once they are reached, so the wheel was never migrated and the update silently repaired nothing. The comment above the line already claimed to match the shared parsers; now it does. Three cases added to tests/sh/test_setup_xpu_fastpath_escape.sh (FAMILY=XPU, FAMILY=Xpu, a URL ending /XPU), plus one that lowercasing must not widen the match: a custom leaf like PRIVATE-XPU stays an unknown family. All three fail against the previous line and pass now. * Do not promise CPU training when the XPU runtime will not start The unavailable-runtime arm said training and GPU inference run on CPU until the driver is fixed. They do not: with neither CUDA nor XPU available, get_device_type() in unsloth/device_type.py raises NotImplementedError, so importing unsloth fails outright rather than falling back. llama.cpp is unaffected, which is what chat and GGUF actually run on, so say that instead. The drift guard added with it needed two passes to be worth anything. Anchoring the arm on the flag name alone matched the bitsandbytes block instead, whose own "4-bit QLoRA may be unavailable" warning made both assertions pass on any wording; and the arm's explanatory comment quotes the phrase it must not use, so comment lines have to go before the grep. Restoring the old message now fails both checks. * Let an explicit non-XPU pin migrate off an XPU wheel Two halves of the same gap: asking for CUDA/ROCm/CPU on a host already running +xpu did nothing. setup.sh: the fast-path escape fired only when the pin itself was xpu, or when a stale generic triton shadowed the build. With an up-to-date install, a +xpu wheel and the pin switched to another family, neither arm matched, install_python_stack never ran, and the authoritative pin was ignored. Added an arm for that case, digit-gated like the shared classifiers so a custom verbatim leaf (rocm-current, cu-private) stays UNKNOWN and does not force a pass that repairs nothing. install_python_stack: _ensure_cpu_torch classifies the installed build and returns early on "already a CPU build". Its probe tested hip, rocm, cuda and +cu<digits>; an XPU wheel sets neither torch.version.cuda nor .hip, so it read as CPU and an explicit CPU pin over it did nothing at all. Keyed on the +xpu local label, since torch.version.xpu is None on some builds. Additive: +cu128 and +rocm still read gpu, +cpu and untagged still read cpu. The escape suite's extractor stopped after the second _SKIP_PYTHON_DEPS assignment, so adding a third arm truncated the block and the new cases failed while the old ones passed. It now stops at the next outer arm and asserts exactly three arms extract, so a future arm fails loudly instead of disappearing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop a wedged Intel driver from blocking the paths that repair it Both of these are fallout from making the CPU repair XPU-aware: it now has to classify an XPU wheel, and every route to that classification went through `import torch`, which loads the SYCL runtime and blocks on the exact host these paths exist to rescue. install_python_stack: the classifier probe times out after 90s and the except branch returned, so an explicit CPU pin over a wedged +xpu venv stayed a no-op. Classify off disk on timeout via _installed_torch_label_on_disk (find_spec, no interpreter) and fall through to the repair. Gated on a GPU label so a slow but healthy CPU-only host does not force-reinstall torch every update. install.sh: the rollback preservation probe read torch.__version__ through the interpreter at venv-replacement time, ahead of every bounded probe in setup.sh, so a hang there took the whole installer with it. It now reads torch/version.py, the same source _installed_torch_version_for_tag already uses for this reason. The interpreter stays as the fallback for a layout without one, where torch is absent and the import fails fast. The install.sh test executes the block against a fake venv whose stub interpreter records being called, so "read off disk" is proven by the interpreter never running rather than by reading the source. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match torch index families exactly, and judge the XPU fast path on the wheel studio/setup.sh classified a pin as a known non-XPU family with prefix globs (cu[0-9]*, rocm[0-9]*), so cu128-private, cu128rc1, cu128.1, rocm7.2-private, rocm7. and rocm7.2.1 all read as known while install_python_stack calls every one of them UNKNOWN and runs no repair: the fast path was cleared and the dependency pass that followed applied nothing, every update. It now matches exact families like install.sh _is_pip_rocm_family_leaf and install_python_stack _is_cuda_family_leaf: cpu, cu<digits>, rocm<digits>[.<digits>], gfx<digit>... (gfx stays a prefix on all three sides, since gfx120x-all is a real Radeon index leaf). studio/setup.ps1 keyed the same escape on torch.xpu.is_available(), which is also false for a supported +xpu wheel on an old or wedged compute driver. No dependency pass can repair a driver, and the pass force-reinstalls nothing when the flavour already matches, so each studio update repeated the bounded probes and a full resolution just to reach the warning Assert-XpuRuntimeReady already prints. The escape now asks Test-VenvTorchIsXpuSupported, which reads torch/version.py off disk and applies the same 2.6 <= v < 2.11 window, matching what setup.sh does on POSIX and removing the last import torch from a path an Arc host with a stalled driver is most likely to hit. Its only caller gone, Test-TorchXpuVersionSupported is removed. Tests: the escape test now also asks install_python_stack itself about a 28-leaf corpus and asserts the shell predicate agrees leaf for leaf, so the two cannot drift again (75 checks; 11 fail against the previous globs). The pre-report test covers the new helper and asserts the fast-path escape names no readiness probe and launches no interpreter. * Trim comments across the Intel XPU detection changes * Normalise setup.ps1 line endings before the wiring regexes A Windows checkout returns CRLF, so the fast-path escape pattern, which is anchored on a literal \n, matched nothing on windows-latest: the region came back empty, "the escape was found" failed, and the two -not checks inside it reported PASS with nothing to look at. Cross-platform parity caught it on windows-latest with 3 failures. $setupText is now normalised to LF once at the read, which covers both literal newline patterns in the file, and a new check asserts the raw CRLF form does NOT match the same pattern, so it is the normalisation rather than luck that makes this work. Verified against a CRLF copy of setup.ps1: the previous test fails there with exactly those 3 checks and the new one passes. * Run the Triton swap after every torch migration, not between two of them _ensure_xpu_triton keys off the installed +xpu label when no explicit XPU pin is set, and it ran ahead of _ensure_cpu_torch. So an existing +xpu venv updated with an explicit CPU pin had generic triton removed and XPU triton installed, and only then did _ensure_cpu_torch replace torch with the CPU build: a CPU environment whose top-level triton package is the XPU implementation, with the generic triton its own dependency set declares now gone. The CUDA and ROCm repairs already ran ahead of the swap, so their pins left the label correct by the time it read it; CPU was the one migration that did not. Moving the swap to the end of both repair blocks fixes it for every family at once and removes the ordering assumption entirely. The new test asserts the order on the AST at both call sites, so a reflow cannot fake it; against the previous order it fails on the first assertion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bound the wedged-driver probe without GNU timeout macOS ships no GNU timeout (Homebrew coreutils installs it as gtimeout), so on the macOS parity leg the `timeout 30 python3 ...` line exited 127 the instant it was called. The test reads only the exit code, and 127 is non-zero with an elapsed time of 0, so both assertions passed without python ever starting: the alarm behaviour they exist to prove was never exercised on macOS. Replaced with the script's own background watchdog, which behaves the same on every platform, and added a lower bound on the elapsed time. The alarm is 2s, so a run that returns instantly did not execute the probe, which is precisely how the missing-timeout case looked. Verified by shimming `timeout` to exit 127: the previous test still reports 37 passed, and by shimming python3 to return instantly: the previous test still reports 37 passed while this one fails on the deadline check. * Trim comments in the Intel XPU detection changes --------- Co-authored-by: CommandCodeBot <noreply@commandcode.ai> Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-04 03:03:10 -07:00
Etherl	1770182b5a	Windows: validate managed Python before package installation (#7763) ... * Harden Windows pyenv interpreter handling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Windows managed Python recovery * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten-installer-comments * Finish the -S probe hardening and run the new tests on Windows for PR #7763 Get-PythonPlatformTag still probed without -S. Its result is compared with -eq "win-amd64", so a sitecustomize banner reads as "unknown", the x64-over-ARM64 preference is lost and Windows on ARM settles for a native ARM64 interpreter. Test-IsCondaPython gets -S for the same reason. Neither query needs site, and base_prefix and get_platform() are unchanged by -S on 3.11, 3.12 and 3.13. The new test module parametrizes over pwsh and powershell, but cross-platform-parity-ci.yml is the only three-OS job and its paths filter and pytest list are hardcoded, so the 5.1 leg never ran. Added the file to both. Also match the py launcher branch to the PATH branch with -LiteralPath -PathType Leaf, and fix the failure message: the empty base home leaked into the Exit-InstallFailure text, and the ownership marker is written before the gate, so a plain re-run already replaces the environment. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com>	2026-08-03 05:08:47 -07:00
Daniel Han	ea2238cc00	Forward revision to the config, weight and tokenizer loads (#4222) ... * fix: add revision parameter support and escape quotes in chat templates - Fix #3544: Add revision parameter to AutoConfig, AutoModelForCausalLM, AutoModelForSequenceClassification, and load_correct_tokenizer calls in FastLlamaModel.from_pretrained. This enables loading specific model revisions/branches from HuggingFace Hub. - Fix #3667: Escape single quotes in system messages before substituting into Jinja2 templates. This prevents TemplateSyntaxError when system messages contain apostrophes (e.g., "user's" in Vicuna templates). Signed-off-by: majiayu000 <1835304752@qq.com> (cherry picked from commit b0a6e4154b) * fix: propagate revision parameter to vLLM and PEFT loaders - Add revision to load_vllm_kwargs in llama.py to fix config/weights mismatch - Add revision to PEFT AutoConfig calls in loader.py (FastLanguageModel & FastModel) Addresses reviewer feedback from @chatgpt-codex-connector and @Datta0 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> (cherry picked from commit 14f89e4531) * fix: add revision parameter to FastBaseModel in vision.py Propagate revision parameter to all from_pretrained calls in vision.py to ensure consistent version pinning for vision models. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> (cherry picked from commit c5aa4ec927) * Forward revision to the config, weight and tokenizer loads FastLlamaModel.from_pretrained took a `revision` argument and never read it, so the config, the weights and the tokenizer all came from the repo's default branch while the caller believed they had pinned a ref. Reported in #3544 by someone versioning their fine-tunes with branches, which makes it a silently wrong base checkpoint rather than an error. Forward it in llama.py (both AutoConfig loads, the three model loads, the tokenizer, the prefetch warm and the fp8 scale restore), plumb it through load_correct_tokenizer, and read it from kwargs in vision.py for the four AutoConfig, two processor and two tokenizer loads plus the VLM processor fallback. vision.py must not bind it as a named parameter: the weight load there forwards **kwargs, so binding it would drop it from that load. model_name is not always the repo the caller named. get_model_name can swap in a pre-quantized mirror, _offline_quantize_to_fp8 an fp8 temp dir, ModelScope a local snapshot, and fast_inference_setup a -bnb-4bit variant, and use_exact_model_name only gates the first of those. A ref from the original repo does not exist on the substitute, so _revision_for_resolved_repo drops it with a warning naming both repos when the resolution changed the name. The adapter load keeps the caller's revision, since that one really is for old_model_name. Supersedes the earlier attempt on this branch, whose chat-template hunk is handled by #7731 and #7746, whose vision.py signature change caused the drop described above, and whose load_vllm(revision = ...) raised TypeError because load_vllm has no such parameter. Fixes #3544 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the revision comments * Gate the revision before the config probes, and never mix refs Four fixes from review: - The gate ran after the AutoConfig and PeftConfig probes, which already used the raw revision against the resolved name, so a pinned load_in_4bit load failed against the mirror instead of warning. Gate right after the resolution block and point both probes at the gated value, then re-gate before dispatch for the later fast_inference_setup remap. Feeding the second call the first result keeps the warning to one. - On a PEFT load model_name is necessarily the base model, so the late gate warned "Ignoring revision" for every versioned adapter and told the caller to pass use_exact_model_name, which cannot stop an adapter resolving its base. Skip the late gate for PEFT; PeftModel.from_pretrained already loads the adapter with the caller's revision. - load_vllm takes no revision, so vLLM fetches the default branch. Pinning only the config and the tokenizer put two refs in one model, which is worse than the old behaviour of ignoring the revision outright. Drop the pin with a warning before the config load whenever vLLM owns the weights. - _hub_repo_or_local_path resolved a cached snapshot without the revision, so an offline or local_files_only tokenizer load silently got the default ref: a revision handed to from_pretrained cannot re-point a local directory. Thread it into _resolve_hub_repo_local_dir and both call sites. Five new tests, one per fix, all failing before it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the revision when vLLM was requested but is unavailable The vLLM guard sat at the end of the same block that turns fast_inference off when vLLM is missing or the GPU is older than sm70. In that case the load falls through in-process and can honour the revision, but the guard dropped it anyway. Re-check fast_inference in the condition. * Keep the pin where the load can honour it, and tailor the warning Three more from review: - A num_labels load goes through AutoModelForSequenceClassification in-process no matter what fast_inference says, so the vLLM guard was discarding a revision the load could have used. Condition it on the same `fast_inference and num_labels is None` predicate the prefetch warm already uses. - use_exact_model_name only gates the mapper substitution. The ModelScope download, the ALLOW_PREQUANTIZED_MODELS strip and fast_inference_setup ignore it, so the warning was sending callers round the same loop. Record whether the mapper is what moved the name and only offer the remedy then. - The tokenizer does not always come from the base model's repo. Loading a PEFT repo with an explicit tokenizer_name pointing at the adapter dropped the pin for the tokenizer while PeftModel loaded the adapter from the requested ref, mixing two refs. _revision_for_tokenizer_repo now resolves it where the repos are known and both dispatches carry it, replacing the tokenizer_name == model_name guess in llama.py and vision.py. vision.py pops it from kwargs, since the weight load forwards **kwargs and transformers has no such argument. Seven new tests, all failing before this. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the adapter ref off the base tokenizer, and pin both or neither Three more from review, all fallout from splitting tokenizer_revision out: - Skipping the late gate for PEFT leaves base_revision naming the adapter, and a remote PEFT load without an explicit tokenizer_name reads its tokenizer from the base repo, so that ref was handed to the wrong repository. Both dispatches now derive one model_revision and pass it to the base load and to the tokenizer resolution alike, so the base tokenizer can only ever get the base model's ref. - FastLlamaModel is exported, and the architecture wrappers forward `revision` through **kwargs without the new internal tokenizer_revision, so a direct call pinned the config and weights while the tokenizer read the default branch. Fall back to `revision` when the tokenizer repo is the model repo, before the warm so it does not fetch the wrong ref either. - The vLLM guard cleared only the model pin, leaving vLLM on the default branch with the tokenizer still on the requested ref. Clear both, in llama.py and in the parallel FastBaseModel block. Seven new tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep one ref per repo on the fp8, vLLM config and tokenizer paths Four ways a pin could still land on the wrong ref: - A plain load that names its own repo as tokenizer_name kept the caller's revision even after a remap had already dropped it off the config and weights, so mirror weights paired with a pinned tokenizer. Only a PEFT adapter is a genuinely separate repo, so only it keeps that ref now. - FastModel probes the config before dispatching and FastBaseModel skips its own load while that config is set, so the vLLM path received a config read at the pinned ref alongside the default-branch weights vLLM fetches. The probed config is now withheld there; a caller's own config still goes down. - The get_auto_processor fallback under AutoProcessor ran unpinned. - _offline_quantize_to_fp8 read the default branch and cached under a name that ignored the revision, so load_in_fp8 with a revision quantized the wrong ref and could reuse another ref's artifact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop the vLLM pin before the probe, and key fp8 artifacts on the raw ref FastModel withheld the probed config from FastBaseModel on the vLLM path, but model_types, auto_model and the text-only decision had already been derived from it, so default-branch weights could load with pinned-ref dispatch. The drop now happens before the probe instead, using the same predicate FastBaseModel does, which makes that guard a no-op on this path and lets the config go down untouched again. FastLanguageModel keeps its drop inside llama.py: that one also turns fast_inference off on pre-Volta GPUs and for a num_labels load, and the loader cannot see either without duplicating the device checks, so gating early there would discard a pin llama.py would have honoured. The fp8 cache name sanitized the ref by replacing every unsafe character with the same one, so release/v1 and release.v1 shared a directory and the second load reused the first ref's artifact. A digest of the raw ref now rides along with the readable form. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate the language probe on vLLM too, spare the adapter probe, stamp the saved ref FastLanguageModel still probed the config at the pinned ref while llama.py dropped that same ref for its vLLM load, so model_types could pick the architecture class off one ref and load weights from another. It now drops the pin before the probe like FastModel does, through _vllm_will_load_weights in llama.py, which llama.py itself now calls: the language path also falls back in-process on pre-Volta GPUs and for a num_labels load, so the predicate has to live where those checks are rather than be guessed at by the loader. That drop runs before is_peft is known, and it was zeroing the ref the PeftConfig probe reads. An adapter is loaded in-process by peft, so it keeps the ref: adapter_revision holds the value from before the vLLM drop. Pinning the tokenizer also desynced the save path, which restores tokenizer.model from tokenizer.name_or_path and so had no idea which branch to read. The loaded ref is now stamped on the tokenizer the way local_files_only and cache_dir already are, and the sentencepiece probe, its memo key and the restore all use it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stamp the loaded ref on the vision processor as well FastBaseModel builds its processor without going through load_correct_tokenizer, so the stamp save.py reads was only being applied on the text path and a pinned FastVisionModel load still restored tokenizer.model from the default branch. Stamped at the return rather than at each of the processor branches, so the AutoTokenizer fallback that runs when patch_tokenizer raises cannot lose it either. * Tighten the revision forwarding comments * Keep the note on why a PEFT load pins nothing --------- Signed-off-by: majiayu000 <1835304752@qq.com> Co-authored-by: majiayu000 <1835304752@qq.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-02 22:35:19 -07:00
Michael Han	ebfefcf84e	Windows: do not abort setup on an unreadable llama.cpp install (#7735) ... * Windows: do not abort setup on an unreadable llama.cpp install Test-Path raises UnauthorizedAccessException instead of returning $false when an ACL denies the probe. setup.ps1 runs under $ErrorActionPreference = "Stop", so the bare probe of UNSLOTH_PREBUILT_INFO.json in the llama.cpp prebuilt phase killed setup with a raw "Test-Path : Access is denied" and exit code 1. The desktop app had nothing but [TAURI:ERROR_DEFAULT] to fall back on, so it showed "unsloth studio setup failed (exit code 1)". ~/.unsloth/llama.cpp sits beside the app, not inside it, so reinstalling reused the unreadable folder and hit the same line again, including a reinstall to a different drive. Add Get-PathState (Present / Absent / Denied) plus Test-PathQuiet, route the probes that read inside install trees we do not own through them, and report a denied llama.cpp install through Exit-SetupFailure so the reason and the recovery steps reach the desktop UI. Reported in unsloth-test/unsloth-test#9 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop on every denied path, and split the recovery commands Review follow-ups: - Assert-StudioOwnedOrAbsent treated a denied root as absent and returned, so the caller could go on to replace a tree it cannot read. Probe the root three-state and stop on Denied, still gated on $StudioHomeIsCustom so default-home behaviour is unchanged. - The source-build .git probe treated a denied checkout as "no checkout" and cloned a replacement. The swap that follows recursively removes the original and moves the temp tree over it under "Continue" and unchecked, so a denied child could leave a half-deleted install. Stop instead. This path already treated denied as absent before the previous commit (that probe runs under "Continue", so it printed an error and took the false branch), so the hazard is older than this branch, but it is in scope for the same reason. - Probe $LlamaCppDir itself three-state, so an unreadable parent is reported rather than dying on the bare probe under "Stop". - takeown and icacls were printed joined by "then", which is not a PowerShell separator: takeown would swallow the rest as arguments and icacls would never run. Print them on separate lines. Fold the repeated guidance into Exit-PathAccessDenied so all five denial routes report the same thing. * Harden the denial reporting path, found by simulation Ran the real decision blocks against simulated filesystems (denied file, denied parent, traverse-only and list-only dirs, symlinks, dangling links, wildcard and unicode paths, 3000 random paths) plus PSScriptAnalyzer's 5.1/6.2/7.0 syntax check. Two things came out of it: - Get-PathDenialDetail threw a parameter-binding exception on an empty path. It runs while a failure is being reported, so it would have replaced the actionable message with a raw binding error at exactly the wrong moment. Null and empty are now accepted and return no detail. - The link-target lookup used an empty catch, which PSScriptAnalyzer flags and which hid the intent. It assigns $null explicitly now. Both are covered by new checks. Also promoted the strongest invariant from the simulation into the suite: Get-PathState must agree with a bare Test-Path on every probe that did not throw, and Denied may only appear where the old probe threw, so no path that worked before can take a different branch now. Verified: PSUseCompatibleSyntax reports nothing for 5.1, 6.2 and 7.0; the Python contract tests pass on 3.10 through 3.13 in separate uv venvs; the tauri install:: unit tests pass (17), which is the code that prefers the [TAURI:ERROR] line over the generic exit-code message. * Trigger the Windows PowerShell tests when they change studio-windows-inference-smoke.yml runs six PowerShell unit tests out of tests/studio, but its pull_request paths filter matched none of them, and no other workflow runs them. A PR touching only one of those tests never ran it. Five predate this branch; the sixth is the ACL test added here. Scope the filter to tests/studio/*.ps1 rather than tests/studio/**, so a python-only change under that directory does not pull in the GGUF smoke jobs. This matches what the other two workflows already do: parity-ci lists its .ps1 test outright and update-smoke uses a scoped glob. Guard it in test_ci_shell_suite_coverage.py, which exists for this exact failure (tests/sh had the same hole): every tests/*.ps1 a workflow invokes must be matched by that workflow's paths filter, and must exist. The GitHub glob matcher it needs has its own table-driven test, since a wrong matcher would make the guard pass on everything. Verified by reverting the one-line filter change: the guard then names all six unrun tests. * Make the Windows PowerShell test step fail when a test fails Verifying the path-filter fix turned up a second hole in the same step. A `shell: pwsh` step inherits only the LAST command's exit code, and this step ran five tests as five bare commands, so only the last one could fail the build. test_resolve_cuda_toolkit.ps1 has been printing FAIL exits non-zero (scenario 2, forced source build) FAIL exits non-zero (scenario 6, no toolkit, forced) 2 check(s) FAILED on every Windows run, exiting 1, and the job reported success. Confirmed on main (run 30723608191, sha c67410a7), so it predates this branch, and it reproduces locally. The cause is in the test, not the installer. Resolve-CudaToolkit -RequireOrExit leaves through Exit-SetupFailure, which the child harness never stubbed, so under ErrorActionPreference=Continue the call was an ignored command-not-found, the child fell through and exited 0. The harness already injects the real Resolve-CudaToolkit and Write-CudaDriverToolkitMismatch by AST, so inject the real Exit-SetupFailure the same way. That test now passes 25/25. With it green, add the exit-code checks after each invocation, matching what studio-windows-update-smoke.yml already does, and guard the pattern in test_ci_shell_suite_coverage.py: any step running more than one PowerShell test must check $LASTEXITCODE after each. Verified by reverting each piece: dropping one check makes the guard name that test, and dropping the Exit-SetupFailure injection brings both scenario failures straight back. * Stop on a denied --with-llama-cpp-dir instead of reinstalling over it When UNSLOTH_LOCAL_LLAMA_CPP_DIR points at the canonical $LlamaCppDir and the llama-server.exe there is ACL-denied, Test-PathQuiet collapsed Denied to $false, $LocalLlamaServerFound stayed false, and the canonical branch reported "nothing built there yet; running the normal install". The prebuilt installer then moves that tree aside and replaces it, which is exactly what the branch's own comment says it exists to prevent. The old bare probe stopped first, by throwing under "Stop". Probe the candidates three-state and stop on Denied. Same for the directory probe itself, which reported an unreadable dir as "does not exist" and sent the user after the wrong problem. The generic message could not be reused as-is here: it tells the user to delete the folder because Unsloth reinstalls it, which is true of the managed cache and wrong for a build they pointed us at. Exit-PathAccessDenied takes -UserSupplied, which swaps that advice for restoring access or repointing UNSLOTH_LOCAL_LLAMA_CPP_DIR, and keeps the takeown/icacls lines. Verified by driving the real block through every state: a readable build is still reused, a genuinely empty canonical dir still falls through to the normal install, a missing dir still reports "does not exist", and a denied build now stops with exit 1 instead of being replaced. Reverting the probe puts the fall-through back, and the user-supplied path never prints "delete or rename" or "managed cache". * Carry the denial through three more probes Three review points, all reproduced before fixing: - The canonical --with-llama-cpp-dir override still got the managed advice ("delete it, Unsloth reinstalls it"). The override means the user asked to reuse whatever is in that tree, so deleting it is wrong wherever it sits. Both candidate denials now pass -UserSupplied, which collapses the branch to one call. - Phase 1b's git prerequisite scan probes the same candidate binaries with a bare Test-Path under "Stop", thousands of lines before the Phase 4 guards, so a denied override terminated the run with the raw error this change exists to replace. Reproduced, then guarded. - Test-StudioOwnedAdoptable collapsed a denied prebuilt marker to $false, so Assert-StudioOwnedOrAbsent called an Unsloth tree an unrelated directory and told the user to move it aside. Get-StudioAdoptableState returns Yes/No/Denied and the guard reports the denial first; Test-StudioOwnedAdoptable stays as the boolean view for the cosmetic cleanup gate. A denied file under a readable directory is a Windows-ACL-only state: POSIX keeps a mode-000 file stat-able, and a symlink into a denied directory still answers Test-Path. The local run injects that one state at the lowest seam and lets the real functions run; the Windows leg of test_path_probe_access_denied.ps1 builds it for real with icacls and skips elsewhere with the reason. test_setup_ps1_adopts_existing_whisper_prebuilt_marker sliced between two function names and the marker scan moved, so its anchor now points at Get-StudioAdoptableState. Its assertion is unchanged. Reverting each fix individually puts the original behaviour back: the ownership misdiagnosis, the raw "Access to the path ... is denied" from Phase 1b, and the delete-your-own-build advice. * Close the remaining denial gaps and the stale probe anchors for PR #7735 - tests/sh/test_with_llama_cpp_dir_flag.sh anchored the literal 'if ($ResolvedLocal -eq $LlamaCppDir) {', which 2a61343 hoisted into $LocalIsCanonical. Re-pin it to the comparison, not the branch. - The junction path deleted and replaced $LlamaCppDir behind a bare Test-Path, so a denied destination still threw raw under a default home. Probe it three-state, and treat Denied as surviving removal. - Get-Content on the prebuilt metadata still globbed while the probes gating it went literal, so a path holding [ or ] passed the probe and threw into the catch. Make both reads literal, with a test. - Get-PathDenialDetail could throw on a non-filesystem provider item whose .Attributes has no -band overload, replacing the failure being reported. - Win32Exception keeps E_FAIL in HResult and the code in NativeErrorCode, so the HRESULT check never matched it. Fix the comment and the check. - test_windows_git_gate.py ran the layout scan in a child that never defined Get-PathState, so de486ff had it silently report nothing built. Inject the real helpers, as test_resolve_cuda_toolkit.ps1 does. - Relax the exact Exit-PathAccessDenied count to a floor and key the -UserSupplied rule on the path reported rather than on position. * Stop advising deletion of a tree whose ownership cannot be read The ownership guard stops precisely because it could not read the marker, so it cannot claim the folder is ours either. It was still emitting the managed-cache text, telling the user to delete it. Eleven lines below, the readable-but-unowned branch says 'move it aside' instead, so we were being gentler when we could prove the tree was not ours than when we could not read it at all. New -OwnershipUnverified wording for those three stops. Also from re-reading the previous commit: - The reparse-point unlink above the junction path ran before the new three-state probe, and a link reports Present, so the probe could not cover it. A denied unlink still terminated on the raw .Delete() throw. - That junction destination probe had no test at all; reverting it left the suite green, since the count floor cannot see a swap. Pinned by name like every other route. - The re-pinned shell anchor accepted an assignment that nothing consumed. Pin the branch that uses it too. - Get-PathDenialDetail now checks the item type rather than the attribute type, which also covers a provider item with no Attributes at all. * Check both destructive steps of the temp-dir swap The guard above the clone path probes only .git, but its own comment names any unreadable child as the risk. A forced source build over a non-git install with a denied child elsewhere reads Absent, clones into a temp dir, and reaches the swap. Both steps there are non-terminating under Continue and neither was checked. Reproduced: Remove-Item partially fails, the original survives, and Move-Item then moves the temp dir INSIDE it, so the new binary lands at llama.cpp/llama.cpp.build.<pid>/llama-server.exe while $LlamaServerBin points at llama.cpp/build/bin/Release. Setup carries on reporting success with no usable server and a half-deleted install. Check the removal before moving, so the stop happens while the temp build is still whole, and check the move afterwards. Denied routes through Exit-PathAccessDenied; anything else surviving exits 3 like the other blocked-replacement paths. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-02 07:25:39 -07:00
Daniel Han	44f113cf0d	Escape the system message spliced into predefined chat templates (#7746) ... * Escape the system message spliced into predefined chat templates get_chat_template(..., system_message = ...) substitutes the message into a {system_message} placeholder that sits inside a Jinja string literal in all 15 predefined templates that carry one, so a quote closes the literal and a backslash is read as an escape: vicuna "Answer the user's question." -> TemplateSyntaxError vicuna r"Put it in \boxed{}." -> renders '\x08oxed{}' vicuna r"C:\Users\me" -> TemplateSyntaxError Reuse the escaper PR #7731 added for construct_chat_template, promoted to a module-level _escape_jinja_literal and extended to escape double quotes so the one helper covers llama-3.1's "..." literal as well as the '...' the rest use. Apply it to the predefined branch of _change_system_message and to the ShareGPT mapping values, and drop the hand-escaping from the two vicuna defaults, which would otherwise be escaped twice. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the escaping comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-02 07:18:09 -07:00
Vineeth Sai Varikuntla	90f5170d0a	Escape caller template text spliced into Jinja string literals (#7731) ... * Escape caller template text spliced into Jinja string literals construct_chat_template builds the HF Jinja template by concatenating the caller's template text straight into '...' literals, in three places: the process() helper, the add_generation_prompt literal, and full_system. None of them escape, so Jinja reads the text as template source. A single quote closes the literal early: default_system_message = "Answer the user's question." -> TemplateSyntaxError: expected token 'end of print statement', got 's' and a backslash is decoded as a Jinja escape, which is silent: default_system_message = r"Put the answer in \boxed{}." -> 'Put the answer in \x08oxed{}.\n### User: Hi\n' default_system_message = r"Files live in C:\Users\me" -> TemplateSyntaxError: truncated \UXXXXXXXX escape The backslash case is the worse one: no error, no warning, and every formatted training sample is built with a backspace character where \boxed was meant to be. It is not limited to the system message. process() handles the instruction and response sections too, so an apostrophe anywhere in the template breaks it, for example "### User's turn: {INPUT}". Escape backslashes then single quotes in each literal chunk. In process() the text is split on the {INPUT}/{OUTPUT}/{SYSTEM} sentinel first, so the ' + message['content'] + ' concatenation markers it inserts are not escaped along with it. The Ollama modelfile splices default_system_message into a double-quoted SYSTEM line with the same lack of escaping; that is a different format with different rules and is left alone here. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Escape \r and strip the BOS before escaping in construct_chat_template Two follow ups to the Jinja literal escaping in this PR. Jinja rewrites a raw carriage return to \n inside a string literal before it unescapes it, so a template authored on Windows loses its CRLF. Escaping \r alongside \ and ' makes the round trip exact, and makes the generated template render identically under jinja2, minja and llama.cpp's Jinja engine. The BOS was stripped from the system section after process() had already escaped it, so a bos_token holding a quote or a backslash no longer matched and was left in the literal, then emitted a second time alongside {{ bos_token }}. Strip it while the text is still raw. * Tighten the comments around the Jinja literal escaping * Trim the escaping comments further --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-02 05:32:25 -07:00
Vineeth Sai Varikuntla	2eae08cc28	Make auto-appended EOS deterministic in chat templates (#7702) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-08-01 00:53:07 -03:00
Irving Ernesto	a335c00605	Export ACCELERATE_MIXED_PRECISION when bf16/fp16 is set explicitly (fixes #4891) (#7534) ... * Export ACCELERATE_MIXED_PRECISION when bf16/fp16 is set explicitly In the default path the only branch that exports ACCELERATE_MIXED_PRECISION is gated on `(not use_bf16 and not use_fp16)`, i.e. on the user having set neither flag. Downstream readers resolve the autocast dtype from that variable with a hardcoded 'fp16' default: trainer._autocast_dtype = torch.float16 \ if os.environ.get('ACCELERATE_MIXED_PRECISION', 'fp16') == 'fp16' \ else torch.bfloat16 (unsloth_zoo/rl_replacements.py, and unsloth/models/rl_replacements.py) So passing bf16=True left an initially unset variable unset, the reader fell back to 'fp16', and a float16 autocast was wrapped around a bfloat16 model. GRPO then crashed on the first step inside matmul_lora: RuntimeError: self and mat2 must have the same dtype, but got Half and BFloat16 Paradoxically the default configuration worked (the automatic branch runs and exports 'bf16'), while explicitly selecting bf16 -- the configuration recommended in #4891 -- crashed. The same failure also reproduces with 16-bit loading, so 4-bit quantization is not required. Add an `elif use_bf16 or use_fp16` branch that exports the resolved precision, restoring the invariant the readers assume. Placed last so the force_float32 and UNSLOTH_MIXED_PRECISION branches keep precedence. Fixes #4891 * Add regression tests for explicit bf16/fp16 precision export Covers the #4891 gap and the precedence that must not change: - explicit bf16=True exports ACCELERATE_MIXED_PRECISION='bf16' - explicit fp16=True exports 'fp16' - force_float32 still wins over the explicit-flag branch - UNSLOTH_MIXED_PRECISION='bfloat16' still wins over it The two export tests fail against the pre-fix template and pass with it; the two precedence tests pass either way. Uses the existing _decide() harness, which executes the real template block extracted from rl.py. * Merge main and tighten the mixed-precision comment for PR #7534 Names the actual root cause: transformers exported ACCELERATE_MIXED_PRECISION itself until 5.x dropped the write. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-07-29 06:39:18 -07:00
Daniel Han	9bfa18cdb0	Windows: unblock the consumer install on clean and no-winget machines (#7549) ... * Windows: unblock the consumer install on clean and no-winget machines Four independent things stop a clean Windows box today. git was a hard Exit-SetupFailure in setup.ps1, justified as required by pip for git+https:// deps and by npm. Neither holds on the consumer path: the unsloth-zoo git+https URL is only used under STUDIO_LOCAL_INSTALL, node is a pinned nodejs.org prebuilt that never touches system npm, and the frontend lockfile has no VCS dependencies. It stays fatal for --local, where it really is needed. Ensure-VCRedist was winget-only, so on hosts without winget (LTSC, Server, managed corporate images) it silently did nothing while the install reported success, and torch then failed to import on a missing VCRUNTIME140.dll. Adds a direct aka.ms/vs/17/release/vc_redist.<arch>.exe download with /quiet /norestart, accepting exit codes 0 and 3010. The redistributable stays required: it is the runtime the prebuilt llama-server and torch link against, not the MSVC compiler, which is already detection-only. Windows on ARM has no PyTorch at all. Measured with uv against download.pytorch.org/whl/cpu and PyPI for aarch64-pc-windows-msvc / cp313: torch, torchvision and torchaudio all resolve to nothing, wheels exist only for win_amd64 and the manylinux targets. The installer burned three uv retries on an unsatisfiable resolution and reported a bare 'Failed to install PyTorch (exit code 1)'. Now it says what is actually wrong and points at --no-torch, which works because llama.cpp does publish windows-arm64-cpu. install_node_prebuilt.py hit '[WinError 5] Access is denied' on os.replace of the freshly extracted directory during a FRESH install, which is a scanner or indexer holding handles for a moment. Retries only winerror 5, 32 and 145 with capped exponential backoff; any other OSError still raises immediately. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give the ARM64 dead end a recovery that works for web installs The only remedy printed was .\install.ps1 --no-torch, but the documented path is irm | iex, where no file exists and flags cannot be forwarded. Name the env var the script already honours at line 145. * Windows on ARM: drop torchaudio, do not abort the install The fail-fast was based on a wrong premise. Counted against download.pytorch.org/whl/cpu: torch has 42 win_arm64 wheels and torchvision 60; only torchaudio has none. PyTorch has shipped Arm-native Windows builds since April 2025, so aborting blocked a platform that mostly works. Drop the one unsatisfiable pin instead. Decide from the interpreter uv will resolve for, not the PowerShell host: an x64 CPython under emulation gets working win_amd64 wheels on an ARM64 box, and powershell.exe inherits PROCESSOR_ARCHITECTURE from its parent. * Carry the ARM64 torchaudio omission into studio setup Dropping it from the first PyTorch command was not enough: install.ps1 then runs studio setup with SKIP_STUDIO_BASE=1 and setup.ps1 reinstalls the bare trio from the CPU index, so the ARM64 path still aborted. Apply the same interpreter-based test there. An unreadable platform keeps the full trio. * Build the torch spec list outside the verbose branch The ARM64 guard landed inside `if ($script:UnslothVerbose)`, so on the default path $_torchTrio was never assigned and the splat expanded to nothing: uv ran as `uv pip install --index-url ...` with no package, exit 2, straight to Exit-SetupFailure. That broke the ordinary Windows install. Hoist it above the branch and use substep, which prints on both paths. Realign the two parity guards to the splat form; they asserted the pre-refactor literal command and were the actual cause of the red parity legs. Both halves are still checked: the bounded list is built, and it reaches the install. * Tighten the comments on the Windows install path * Windows install: honour the ARM64 torchaudio skip everywhere and keep git for source builds Hoist the venv-interpreter platform probe above every torch branch in studio/setup.ps1 so the win_arm64 torchaudio omission applies to the ROCm, CPU and CUDA/custom paths. A pinned index whose leaf is not cpu routed an ARM64 host into the CUDA/custom branch, which still asked for torchaudio. Require git again when a llama.cpp source build is opted into up front (UNSLOTH_LLAMA_FORCE_COMPILE, UNSLOTH_LLAMA_PR / PR_FORCE, a non-upstream source). Those paths git clone in phase 4, so setup used to report git as not required, install the build toolchain, then fail at the clone. A local llama.cpp dir overrides them, and the automatic source fallback after a failed prebuilt download stays non-fatal. Also tighten the comments across the changed install paths. * Install the x64 VC++ runtime unconditionally in the direct-download fallback The winget branch always installs Microsoft.VCRedist.2015+.x64, but the direct-download fallback picked the package from PROCESSOR_ARCHITECTURE, which reports the architecture of the running PowerShell process rather than the interpreter that will load the DLLs. Find-CompatiblePython in install.ps1 selects an interpreter on version and non-Conda status alone, with no architecture predicate, so a native ARM64 shell can settle on an emulated x64 Python whose win_amd64 torch and prebuilt llama-server need the x64 runtime, while the fallback had just installed the ARM64-only package. Ensure-VCRedist also runs well before the venv exists, so the interpreter cannot be probed at that point. Microsoft ships the x64 redistributable as an Arm64X superset that carries both ARM64 and x64 binaries, so it is correct on both machines and the manual instruction printed on failure already pointed at it. * Windows on ARM: prefer an x64 Python interpreter An ARM64 host cannot complete the install with a native ARM64 interpreter. pyarrow, pulled in by unsloth -> datasets, has never published a win_arm64 wheel on any version, and neither has hf-transfer, a direct dependency. Both therefore fall back to a source build: pyarrow dies in scikit-build-core CMake configuration and hf-transfer dies in openssl-sys for want of perl, several minutes into a run that looked healthy. torch and torchvision are not the problem, they have win_arm64 wheels and install fine. Windows 11 on ARM runs x64 binaries under emulation and both packages ship win_amd64 wheels, so an x64 interpreter installs cleanly. Find-CompatiblePython accepted an interpreter on version and non-Conda status alone. It now ranks candidates by architecture on ARM64 hosts and returns an x64 one when present, asking each interpreter for its own sysconfig.get_platform() rather than guessing from its path. Host architecture comes from PROCESSOR_ARCHITEW6432 and OSArchitecture as well as PROCESSOR_ARCHITECTURE, which describes only the current process and reads AMD64 in an emulated shell. This is a preference, not a requirement. If only ARM64 is found, x64 is bootstrapped through winget --architecture x64 or the python.org fallback, and if neither works the installer names pyarrow and hf-transfer up front instead of failing later on a CMake or Rust error. The ARM64 torchaudio skip stays live for that path. Non-ARM hosts return on the first match exactly as before, with no extra interpreter probing. * Windows install: three correctness fixes on the ARM64 and git-less paths Ensure-VCRedist never reached its x64 download on an ARM64 machine that already had the arm64 redistributable: Test-VCRedistInstalled accepted System32\vcruntime140_1.dll regardless of architecture, and there that file can be the pure-ARM64 package. An ARM64 PE cannot load into an emulated x64 process, so the x64 Python this branch now prefers would have been left without a usable runtime. The x64 registry entry is the only x64-specific proof, and Microsoft registers Runtimes\{x86|x64|arm64} per architecture, so vc_redist.x64.exe still writes Runtimes\x64 on an ARM64 host and the check cannot loop. The DLL probe stays for x64 hosts. Phase 1 demanded git for any non-blank UNSLOTH_LLAMA_PR_FORCE, but the promotion that actually turns it into a source build requires a positive integer, so PR_FORCE=0 or a non-numeric value aborted a git-less consumer install for a build that never runs. Both sites now use the same predicate. The automatic fallback after a failed prebuilt llama.cpp download reached git clone with no git check anywhere in between, and Invoke-SetupCommand returns 0 for a command-not-found, so a git-less host did not stop there: it continued into an empty directory and reported a cmake configure failure instead. Git is now resolved where the source build is decided, with a last winget attempt, and a missing git degrades exactly like a missing cmake rather than aborting, since the opt-in source triggers already required git in Phase 1. Also tightened the comments across the changed Windows install code, keeping the reasons on the guards that prevent a specific failure. * Rank ARM64 Python candidates by minor version before architecture The x64 preference filtered the whole candidate list on architecture, which outranks the version preference the candidates were collected in. With UNSLOTH_PYTHON=3.12 on a Windows ARM64 box holding an ARM64 3.12 and an x64 3.13, it returned the x64 3.13: the explicit pin was silently broken, and because a x64 interpreter was found the caller never ran Install-X64Python to fetch an x64 3.12. With no pin it was worse still, since an x64 3.11 outranked a newer ARM64 3.13 and defeated the newest-first fallback. Walk $minors in order and take the x64 build of the best minor available, falling back to that minor's ARM64 build so the caller bootstraps x64 for the version actually requested. x64 still wins within a minor, and non-ARM hosts are untouched. * Windows install: see every registered Python, order git before the toolchain Find-CompatiblePython only ever probed `py -3.X`, which runs the launcher's preferred build for that minor. On an ARM64 box that is the native ARM64 interpreter, so a same-minor x64 install that is registered with the launcher but neither preferred nor on PATH never became a candidate. The x64 preference then lost to ARM64, and Install-X64Python re-downloaded an x64 CPython that was already on the machine; when that download is unavailable the install continues on ARM64 and source-builds pyarrow and hf-transfer, which publish no win_arm64 wheels. Enumerate `py -0p` on ARM64 hosts and probe each listed path. The `-3.12-64` suffix cannot be used for this: it has meant "not 32-bit" since 3.11 and does not distinguish arm64 from amd64. studio/setup.ps1 ran Ensure-BuildToolsForLlamaSourceBuild before checking git in Phase 4. That helper calls Exit-SetupFailure when Visual Studio Build Tools cannot be installed, so on a clean no-winget box the git degraded path added by this PR was unreachable and a standalone update aborted instead of finishing in limited mode; where winget does exist it spent a multi-GB Build Tools download on a clone that could never run. Check and install git first, skip the toolchain helper when git is still missing, and report the git branch before the cmake branch so the message names the real cause. _swap_into_place retried the forward rename for about 16 seconds but rolled back with a bare os.replace. A scanner holding the backup for the same WinError 5/32 then left no install_dir at all and stranded the working runtime in .old-*, and its exception replaced the original failure. The rollback now uses the same backoff and logs instead of masking the error it is recovering from. * Installer: use an already installed x64 Python on ARM64 when none can be downloaded Find-CompatiblePython ranks x64 within one minor and returns the native build when that minor is ARM64-only, leaving Install-X64Python to bootstrap x64. On an offline or winget-less box that bootstrap fails, and the retry went through the same resolver, so an x64 build of a lower-priority supported minor already on the machine was never picked up and setup continued on ARM64 Python, where pyarrow and hf-transfer have no wheels. Add an -X64Only mode that returns the best installed x64 interpreter or nothing, and call it as the last resort in Install-X64Python. The version-first preference is unchanged: x64 of the requested minor is still bootstrapped first. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in the Windows ARM64 installer changes * Setup: require Git for a source build behind an unbuilt local llama.cpp dir UNSLOTH_LOCAL_LLAMA_CPP_DIR only overrides the source-build opt-ins once the directory holds a reusable llama-server.exe. Pointing it at the canonical install location with nothing built there falls through to the normal install, so the Phase 1 gate now probes the same layout candidates as the Phase 4 reuse check before dropping the requirement. * Setup: require Git when UNSLOTH_LLAMA_TAG=master forces a source build * Tighten comments in the Windows installer changes * Setup: negotiate TLS 1.2 for the direct VC++ runtime download --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-28 22:24:40 -07:00
Daniel Han	f4f36a0d2d	Anchor the bnb bind assertion on the symbol, not the module alias (#7590) ... #7578 and #7580 landed within a minute of each other and compose correctly in kernels/utils.py, but the source-text assertion #7578 added does not: it looked for the literal "bnb.functional.lib" under the guard, and #7580 renamed that binding to "bnb_functional.lib" to survive a half-imported bitsandbytes. Git merged both cleanly because they touch different lines, so the break only shows at test time. Match "lib.cdequantize_blockwise_fp32" instead. That still pins the binds to the guard, which is what the test is for, and no longer breaks when the module alias changes. Co-authored-by: unslothai <unslothai@gmail.com>	2026-07-28 21:35:04 -07:00
Daniel Han	f44379d9e8	Clear ALLOW_BITSANDBYTES when the bitsandbytes native kernels are not real (#7578) ... * Clear ALLOW_BITSANDBYTES when the bitsandbytes native kernels are not real From bitsandbytes 0.46 a wheel whose native library never loaded still imports and resolves every ctypes handle: BNBNativeLibrary.__getattr__ returns a throw_on_call closure, and a dead library is replaced wholesale by ErrorHandlerMockBNBNativeLibrary, which does the same for every name. Nothing raises while kernels/utils.py binds them at module scope, so device_type.py's guarded import sees a healthy wheel, ALLOW_BITSANDBYTES stays true, loader.py forwards the default load_in_4bit=True and the run dies inside a kernel instead of degrading to 16bit. Probe the handles the kernels actually bind and clear the flags when they are not native. A real handle is a ctypes function pointer and carries restype; a deferred failure is a Python function and does not. Scoped to the capability flags on purpose. The module stays bound and get_ptr keeps pointing at bitsandbytes, because these shapes import perfectly well and treating them as absent would disable a wheel whose Python side works - a CPU-only install is exactly that shape. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only clear the flags when the native library is dead, not partially exporting ALLOW_BITSANDBYTES gates 8bit as well as 4bit - loader.py:505-510 clears both - so failing the check on one missing 4bit symbol would silently downgrade an otherwise valid LLM.int8 request to 16bit. A library that exports some of these handles is alive; only one where none of them is a ctypes function pointer is dead, which is the CPU-only and ErrorHandlerMockBNBNativeLibrary case this exists for. A genuinely missing symbol raises where kernels/utils.py binds it, so it is a crash no capability flag can rescue and not something to trade 8bit for. * Gate the bitsandbytes ctypes binds on the same verdict as the flags Clearing ALLOW_BITSANDBYTES is not enough on its own. kernels/utils.py guarded the bnb.functional.lib.* binds on `bnb is None` alone, so an importable but dead wheel still reached them at module scope: bitsandbytes 0.45.5, the floor in pyproject.toml, sets functional.lib = None when the native library fails to load, and None.cdequantize_blockwise_fp32 raises right there. That kills import unsloth outright instead of degrading to 16bit, which is the fallback the cleared flag exists to reach. Reuse native_kernels_ready so the bind path and the flag path agree, and take the _bnb_required branch when they say the library is dead. Touches only the guard expression, not the binds themselves. * Tighten the comments on the bitsandbytes kernel readiness probe * Require every probed handle, and license the module Apache like the rest of unsloth The readiness verdict now gates the module-scope ctypes binds as well as the flags, so "at least one handle is native" is no longer the right question. A library that resolves one symbol and not another passed the probe and then raised AttributeError at the bind the probe exists to prevent. Require all of them. That costs 8bit in the partial case, since ALLOW_BITSANDBYTES gates both, but a wheel missing a symbol is a shape no flag can make safe and refusing it beats crashing on it. Flipped the test that encoded the old behaviour and added the more realistic shape: the library loaded, one symbol is still a deferred-failure closure. LICENSE:190 assigns files under unsloth/* to Apache 2.0, and 87 of the 90 modules there carry that header, so use it here rather than AGPL. * State the all-handles rule once instead of three times --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-28 21:17:33 -07:00
Daniel Han	fa95054399	Gate the torchcodec audio extras to platforms that have a wheel (#7587)	2026-07-28 20:56:11 -07:00
Daniel Han	00646632bc	Tests: import bitsandbytes before the GPU-free harness spoofs CUDA (#7582) ... * Tests: import bitsandbytes before the GPU-free harness spoofs CUDA The CPU test harness patches torch.cuda.is_available to return True so device_type.py's cache captures "cuda" on a GPU-less runner. bitsandbytes reads the same flag at import time to decide whether to load its CUDA backend, and that backend reads torch._C._cuda_getCurrentRawStream, which a CPU-only torch build does not expose. An import landing inside the spoof window therefore raises, Python drops bitsandbytes from sys.modules while leaving its submodules cached, and every later import returns a module with no .functional, so unsloth/kernels/utils.py dies at module scope. Import bitsandbytes before the window so it stays on its CPU backend and remains fully usable, rather than being degraded to unavailable. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-28 18:52:25 -07:00
Leo Borcherding	411cb86d62	amd: require bitsandbytes>=0.50.0 in the amd extra (fixes ROCm 4-bit NaNs) (#7535) ... * amd: require bitsandbytes>=0.50.0 in the amd extra bnb <= 0.49.2 NaNs at decode shape on every AMD GPU. The ROCm 4-bit GEMV fix (bnb PR #1887) first ships in 0.50.0, on PyPI since 2026-07-24, so the old >=0.49.1 floor could still resolve the broken range. Mirrors the same change made on the pip release branch in #7278. * amd: cite the 0.50.0 ROCm work accurately in the bnb floor comment The comment credited bnb PR #1887 as "the ROCm 4-bit GEMV fix" for every AMD GPU. #1887 decouples blocksize from warp size and fixes a hardcoded warp size of 32 in kgemm_4bit_inference_naive, which is a CDNA problem by construction. The RDNA-side work is #1979 (fused 4-bit SIMT GEMM) and #2012 (RDNA3/4 workgroup resonance). All three first ship in 0.50.0, so the >=0.50.0 floor is unchanged; only the justification was wrong. * amd: raise the installer bitsandbytes fallback floors to 0.50.0 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * amd: stop reporting the bitsandbytes PyPI fallback as broken * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten AMD bnb floor comments * Keep the amd extra citation and the AMD install guide reference * amd: do not promise aarch64 a ROCm 4-bit backend it never gets * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * amd: fall back to the PyPI bitsandbytes floor on Windows ROCm too * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-28 18:12:26 -07:00
Daniel Han	52a9601032	Keep import unsloth working when bitsandbytes is absent (#7502) ... * Keep `import unsloth` working when bitsandbytes is absent device_type.py already prints "bitsandbytes is not installed - 4bit QLoRA unallowed, but 16bit and full finetuning works" and clears ALLOW_BITSANDBYTES / ALLOW_PREQUANTIZED_MODELS, but the import chain then hard-required the module anyway, so `import unsloth` raised instead. #7354 made this reachable: the gfx906 install path uninstalls the generic bitsandbytes wheel (no gfx906 kernels in it), which leaves an MI50 / Radeon VII host unable to import unsloth at all, not on the 16bit path the message promises. - kernels/utils.py: guard the bnb import; bind get_ptr and the five 4bit ctypes handles to a stub that raises a clear message if a 4bit path is entered. HAS_CUDA_STREAM stays False, which is the correct route. - save.py, models/granite.py: guard Bnb_Linear4bit and peft's Linear4bit (peft exports it only when bnb imported cleanly) with placeholder classes. Both names only feed isinstance checks, so nothing matching is exact. - _gpu_init.py: same degradation on the xpu branch as the cuda branch above. Verified on a Strix Halo (gfx1151, DEVICE_TYPE=hip, torch 2.11.0+rocm7.13.0) by blocking bitsandbytes with sys.modules["bitsandbytes"] = None, so find_spec returns None and the import raises exactly as when the package is absent. Before: ModuleNotFoundError at kernels/utils.py:136. After: import succeeds, FastLanguageModel/FastModel import, ALLOW_BITSANDBYTES=False, ALLOW_PREQUANTIZED=False, and the 4bit stub raises with the real cause. With bitsandbytes present, every binding is unchanged. New test walks the `import unsloth` module graph with ast and fails on any unguarded bitsandbytes (or peft Linear4bit) import; verified it catches the old code. Targeted suites: 702 passed, 18 skipped. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address the review on #7502: zoo coupling, non-hip flags, py3.9 collection Three findings, each reproduced first and negative-controlled after. 1. The fix still needed an unreleased unsloth_zoo (P1). save.py imported unsloth_zoo.saving_utils at module scope, and any zoo without the companion #953 fix imports bitsandbytes there, so `import unsloth` kept failing for a dependency set pyproject.toml allows. Raising the floor was not an option: PyPI's newest zoo is 2026.7.6 and #953 is merged but unreleased, so a bump would break every install today. Both names it pulled in are used only inside functions, so the import is now lazy at those two call sites, matching what determine_base_model_source in the same file already does. Verified against a real pre-#953 zoo checkout with bitsandbytes blocked: import succeeds, and restoring the eager import reproduces the failure at saving_utils.py:70. This PR no longer depends on a zoo release. 2. Capability flags were only cleared on hip (P2). device_type.py probed bitsandbytes inside its DEVICE_TYPE == "hip" branch, so a cuda or xpu host without bnb imported fine but still reported ALLOW_BITSANDBYTES=True, and the default load_in_4bit=True path in models/loader.py would select a 4bit checkpoint before failing. Clear both flags whenever the module is absent, on every backend, via find_spec so a working install pays nothing. A cuda host with bnb blocked now reports False/False; with bnb present nothing changes. 3. The new test could not be collected on Python 3.9 (P2). `Path | None` is a PEP 604 union and requires-python still allows 3.9, so pytest raised TypeError at import. Added `from __future__ import annotations`. Checked in real uv venvs on 3.9, 3.10 and 3.13: 2 passed each; removing the future import reproduces "unsupported operand type(s) for |" on 3.9 only. The xpu branch in _gpu_init.py needs no separate flag handling now that the probe is backend-independent. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address the second review on #7502: guarded probe, and 8bit in the same guard 1. The capability probe used find_spec while the fallbacks in kernels/utils.py and _gpu_init.py treat any import failure as unavailable, so an installed but unusable wheel would leave ALLOW_BITSANDBYTES true while the kernels had already bound the stub. Probe with the same guarded import instead, so all three agree by construction. No new cost on any path: _gpu_init.py already imports bnb before device_type is reached on cuda, and device_type's own hip block imports it a few lines later. Worth recording that the state this prevents is currently unreachable for an unrelated reason: a broken wheel takes `import unsloth` down earlier, in transformers/integrations/bitsandbytes.py:20 via unsloth_zoo/patching_utils.py:680, whichever exception it raises (OSError also escapes the zoo moe_utils `except ImportError`). So this is correctness for when those imports get guarded, not an observable fix today. 2. Both loader guards printed for load_in_4bit or load_in_8bit but only cleared load_in_4bit, so an explicit load_in_8bit=True survived and reached Transformers, which builds the bnb quantizer and fails there. Clear both. The message no longer says AMD either: the flag now goes false whenever bnb is unusable on any backend. Tests: the probe must not use find_spec, and an ast walk requires every ALLOW_BITSANDBYTES guard in loader.py to clear both flags, so a third guard cannot be added with the same omission. Dropping either fix reddens them (1 and 2 failures respectively). 4 passed on 3.9, 3.13 and the ROCm venv; absent and healthy bnb both stay consistent across hip and cuda. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop the importlib import left over from the find_spec probe on #7502 * Address the third review on #7502: exact-name bypass and a forwarded bnb config Both findings hold up, so both are fixed. 1. use_exact_model_name=True skipped the guard entirely. load_in_4bit defaults to True, so on a host without bitsandbytes FastLanguageModel.from_pretrained(name, use_exact_model_name=True) kept 4bit set and failed downstream. That option suppresses repo-name remapping and cannot make bitsandbytes available, so it has no business gating a capability check. Ungated at both sites. 2. A user-supplied quantization_config survived the fallback. It sets load_in_4bit/8bit at the top of from_pretrained and stays in kwargs, so clearing the local flags still let Transformers rebuild the bnb quantizer. Now dropped as part of the fallback. One correction to the second suggestion: it cannot be dropped whenever the fallback runs. quantization_config also carries GPTQ, AWQ, fp8 and torchao configs, which have nothing to do with bitsandbytes and must reach the loader untouched. The pop is gated on the config actually requesting load_in_4bit or load_in_8bit, reusing the same dict/attr probe from the top of the function. Behaviour, exercising the real guard block against synthetic inputs with use_exact_model_name=True and bnb unusable: default 4bit, no cfg 4bit=False 8bit=False explicit 8bit, no cfg 4bit=False 8bit=False BitsAndBytesConfig(4bit/8bit) 4bit=False 8bit=False config dropped dict bnb config 4bit=False 8bit=False config dropped GPTQ config 4bit=False 8bit=False config SURVIVES fp8 dict 4bit=False 8bit=False config SURVIVES Nothing changes when bitsandbytes works: the whole block is inside `if not ALLOW_BITSANDBYTES`. Tests: an ast walk requires neither guard to reference use_exact_model_name in its test, and requires each to pop quantization_config behind a _wants_bnb check, so an unconditional pop fails too. Re-gating one guard or removing one pop reddens a test each. 6 passed on 3.9, 3.13 and the ROCm venv. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address the fourth review on #7502: FastModel never reached the 16bit path Both findings are real, and the second one meant this PR did not actually deliver what it advertises for FastModel or vision loads. Reproduced first. 1. patch_compiling_bitsandbytes() ran unguarded at the top of FastModel.from_pretrained, and unsloth_zoo's copy imports bitsandbytes unconditionally (patching_utils.py:40). So every FastModel call on a bnb-less host died there, whatever the arguments: FastModel(load_in_16bit=True) -> ModuleNotFoundError at patching_utils.py:40 FastModel(full_finetuning=True) -> ModuleNotFoundError at patching_utils.py:40 The FastLanguageModel path already wraps this call in try/except with a warning, and its comment even says "Mirror FastModel" - FastModel was the unwrapped one. Wrapped it the same way, so behaviour is unchanged wherever bitsandbytes imports. 2. The mode-exclusivity check ran before the capability fallback. load_in_4bit defaults to True, so load_in_16bit=True made int(load_in_4bit) + int(load_in_16bit) == 2 and raised "Can only load in 4bit or 8bit or 16bit" before the fallback could clear the unavailable 4bit request. Moved the fallback ahead of that check. After both, the same three calls get past every bitsandbytes gate and reach model resolution, failing only on the deliberately fake repo name used by the probe. Nothing changes when bitsandbytes works: the fallback is still inside `if not ALLOW_BITSANDBYTES`, and the wrapper only swallows an import that previously crashed the load. Tests: the mode check must be preceded by an ALLOW_BITSANDBYTES fallback in the same function, and no call to patch_compiling_bitsandbytes may sit outside a try. The ordering assertion is scoped to the enclosing function on purpose - my first version compared line numbers file-wide, so the other loader's guard satisfied it and the negative control passed when it should have failed. With the scoping fixed, moving the fallback back after the mode check reddens it, as does unwrapping the patch call. 8 passed on 3.9, 3.13 and the ROCm venv. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-28 08:03:54 -07:00
Lee Jackson	d7594ec10f	Fix Windows no-torch setup (#7511) ... * Fix Windows no-torch setup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix no-torch env normalization on Windows * Accept on for Windows no-torch mode * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep no-torch mode across studio update on Windows Guarding the direct torch/Triton install made `install.ps1 --no-torch` actually produce a torch-free venv, which then broke the next `unsloth studio update`. That path exports no UNSLOTH_NO_TORCH, so $NoTorchMode was false, the stale-venv check read the missing torch as a broken venv, and setup tried to delete the venv it was running out of: [ERROR] Could not remove stale venv: Access to the path 'python.exe' is denied. That teardown can never succeed there, because setup.ps1 runs via unsloth.exe out of that same venv. The same gap also let the shared dependency pass reinstall torch from PyPI, unpinned, into a GGUF-only environment. install_python_stack.py now records the mode in the install manifest and setup.ps1 reads it back when no env var is exported, then re-exports a canonical value for the dependency pass (setup.ps1 drops the manifest before invoking it, so the child cannot repeat the lookup). The key is additive and MANIFEST_SCHEMA is unchanged, so existing manifests stay valid and a missing key keeps today's behaviour. Also: - read_manifest() caught only OSError, but UnicodeDecodeError is a ValueError. That is now on the installer's import path, so a manifest re-saved as ANSI or truncated mid-write would abort every install. - The env predicate now trims surrounding whitespace, matching the Python side. - The Windows update smoke workflow asserts the update leaves the venv GGUF-only, which is what would have caught this. Known follow-up, pre-existing: an install killed between the manifest drop and the dependency pass leaves no recorded mode, so a later update still walks the stale-venv path. Closing that needs a marker the installer never drops. * Persist no-torch mode in a marker the dependency pass cannot drop The install manifest alone was not enough. Both setup.ps1 and install_python_stack.py remove it before every dependency pass, and it is only rewritten on success, so a no-torch install interrupted in between left nothing recording the mode. The next update then resolved no-torch as false, read the expected missing torch as a stale venv, and tried to delete the environment whose python.exe was running it, which leaves the install unrepairable from the CLI. Add .unsloth-no-torch next to the existing .unsloth-studio-owned marker, written before the pass and cleared when torch is wanted. setup.ps1 writes it as soon as the mode resolves, so the window between the manifest drop and its own torch install is covered too. Read order stays manifest key first, then marker, so migrating out of no-torch is never blocked by a marker an earlier run left behind. Neither present still reads as "install torch", so nothing changes for installs made before either existed. Also adds the AGPL-3.0 header the new test file was missing. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-28 05:54:25 -07:00
Leo Borcherding	f03e669442	AMD: enable ROCm torch on gfx906 (MI50 / Radeon VII) on Linux (#7354) ... * Add community-maintained legacy support path for gfx906 (MI50 / Radeon VII) rocm6.4+/7.x torch wheels bundle ROCm libraries whose Tensile kernels dropped gfx906 (rocBLAS 'TensileLibrary.dat ... not read for gfx906', ROCm/TheRock#1844), so on MI50/Vega 20 hosts with newer ROCm the installer picked wheels that fail at the first BLAS call. The rocm6.3 index is the last one whose wheels run on gfx906 (torch 2.7.0 verified on MI50 32GB, up to 2.9 in community use). Dynamo/Inductor codegen is also broken on this arch, crashing compiled graphs that train fine in eager mode. - install.sh: when the runtime GPU is gfx906 and the picked index is newer than rocm6.3, reroute torch to the rocm6.3 index and reset the constraint trio to the default <2.11 window (a rocm7.2 pick raises the floor to 2.11, which rocm6.3 cannot satisfy), with a legacy-path warning. - install_python_stack.py: mirror the reroute in _ensure_rocm_torch using the _default pkg specs, including repairing an existing +rocm7.x torch and leaving a working rocm6.3 install alone. - device_type.py: default TORCHDYNAMO_DISABLE / TORCH_COMPILE_DISABLE / UNSLOTH_COMPILE_DISABLE on gfx906 (setdefault, user override wins). Windows allowlists are untouched: repo.amd.com publishes no gfx906 wheel family (verified in the RDNA2 enablement PR). 16-bit LoRA and full finetuning work out of the box; 4-bit QLoRA needs a source-built bitsandbytes for gfx906. Based on the verified MI50 32GB setup in namnguyen0503/mi50-gfx906-unsloth-bnb4bit-lab. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * gfx906: second Codex pass (bnb skip under pin, override beats Strix) - Compute the gfx906 runtime-target flag independently of any torch-index pin or Strix override, so the bitsandbytes skip still applies when a user pins the ROCm index and sets UNSLOTH_ROCM_GFX_ARCH=gfx906 (the pin suppresses the torch reroute, not the bnb skip). Probe only when no pin is set (an explicit pin means don't second-guess it, matching the Strix path's asserted no-probe invariant); an explicit gfx906 override needs no probe. - Let UNSLOTH_ROCM_GFX_ARCH=gfx906 suppress the Strix reroute (both install.sh and install_python_stack.py) so a mixed Strix + MI50 host routes to rocm6.3 instead of the gfx1151 wheels probe order would pick. - Fix test_hardcoded_torch_constraint: the default <2.11 window literal now legitimately appears on two TORCH_CONSTRAINT= assignments (default + the gfx906 reroute reset after the rocm7.2 floor bump); assert it only ever appears on assignment lines, never on a pip install line (its real intent). New tests: bnb skipped under an explicit pin, gfx906 override wins over Strix, install.sh suppresses Strix on the override. rocm_support + selection + cross-platform parity: 667 passed; structural constraint 9/9. * gfx906: collapse single-line asserts to match pre-commit formatting * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * gfx906: keep bnb skip + rocm6.3 routing correct under pins and suffixed overrides Address the four Codex P2 findings on #7354: - bnb skip under a pinned index (install.sh + install_python_stack.py): a real gfx906 host that pins UNSLOTH_TORCH_INDEX_URL to rocm6.3 without also setting UNSLOTH_ROCM_GFX_ARCH no longer reinstalls the generic bitsandbytes wheel over a source-built gfx906 bnb. A pin now suppresses only the torch reroute, not the gfx906 detection used for the bnb skip (Python drops the pin gate on _runtime_is_gfx906; bash _is_gfx906_bnb_skip probes via _probe_amd_gfx_arch when the index is pinned). - clear the Radeon marketing-name flag for every gfx906 target, not only when the >=6.4 reroute fires, so a Radeon VII already on rocm6.3 does not divert to the repo.radeon.com branch (whose wheels lack gfx906 kernels). - normalize a copied HIP gcnArchName (gfx906:sramecc-:xnack- -> gfx906) before the exact comparisons in install.sh and install_python_stack.py, mirroring device_type.py. Tests: relax the three Strix-pin tests (the gfx probe may now run for the bnb flag but must not reroute the pinned index) and add coverage for the pinned bnb skip, the suffixed override, and the bash Radeon-clear / pinned-probe paths. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * gfx906: log skipped vLLM aimv2 fix + robust source-scan test bounds Follow-up review polish: - import_fixes: log at info level when the vLLM aimv2 fix is skipped because the dist metadata is unreadable, so the skip is diagnosable instead of silent. - test_rocm_support: bound the gfx906 install.sh source-scan on the ';;' that closes its case arm via a shared _gfx906_reroute_block helper, replacing the brittle fixed-length (3200/3800) slices that shift when the block grows. * gfx906: trim whitespace on UNSLOTH_ROCM_GFX_ARCH in install.sh (py parity) The bash gfx906 comparisons lowercased and stripped the gfx906:… feature suffix but not surrounding whitespace, while the Python paths do .strip(). A stray newline (e.g. export UNSLOTH_ROCM_GFX_ARCH=$(cmd)) would make bash miss gfx906 while Python catches it. Trim with `tr -d '[:space:]'` at both comparison sites so the reroute target and bnb-skip agree across bash/Python. * gfx906: remove generic bitsandbytes pulled in transitively after the skip --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-27 05:22:19 -07:00
Leo Borcherding	1dd2fc4583	tests: read checked-in files as UTF-8 instead of the platform default (#7438) ... * tests: read checked-in files as UTF-8 instead of the platform default Path.read_text() with no encoding uses locale.getpreferredencoding(), which is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine module-level reads of checked-in source files were relying on that default. studio/backend/routes/inference.py carries the DeepSeek tool-call token regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py out at collection, not as failures. Green on CI, permanently broken for a Windows contributor running the suite locally. Adds a guard: at module scope there is no tmp_path fixture, so a bare read_text()/write_text()/open() there is always touching a checked-in file. That makes the rule mechanical enough to enforce with no allowlist, while staying quiet about temp-dir I/O inside test bodies where the platform default is harmless. The repo already spells this correctly in 464 other places; this only stops the stragglers coming back. * tests: cover import-time helper reads and keep the guard py3.9-safe Follows up on the Codex review: - add `from __future__ import annotations`, since `str | None` in `_offender` is evaluated at import on Python 3.9 and pyproject declares requires-python ">=3.9,<3.15". - widen the guard from module scope to import time. Class bodies and the bodies of module-level helpers called from an executing statement run during collection too, so `CODE = _extract_mixed_precision_code()` was the same hazard as an inline read. `if __name__ == "__main__":` blocks are skipped: pytest never executes them. - scan studio/backend/tests/ as well as tests/. Both trees are collected on Windows by separate CI jobs, and the offender that started this, test_tool_xml_strip.py reading routes/inference.py, lives there. Widening it surfaced seven more import-time reads of checked-in sources; all now name utf-8. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden the import-time encoding guard for PR #7438 Close the detector gaps raised in review, all of which I reproduced against the actual AST before changing anything. False negatives (the guard let a real hazard through): - _is_main_guard ignored the comparison operator, so if __name__ != "__main__" counted as script-only even though its body runs at import. - The else arm of a main guard was discarded with the rest of the If node. - Decorators and argument defaults on a module-level def were skipped with the body, though both are evaluated when the def executes. - Path.open() in text mode was invisible; only builtin open() was matched. - encoding = None and encoding = "locale" both re-select the platform default, but the keyword merely being present counted as pinned. False positives (the guard would have blocked a compliant contributor): - A non-literal mode fell through to the "r" default, so open(p, mode) was flagged even when mode is "rb", where adding encoding= is a ValueError and there is no edit that satisfies the rule. - Same for open(*args) and a **kwargs splat, which hide the mode and can hide an encoding. - Lambda bodies and comprehension elements were walked even though neither runs at definition. Verified: still reports the same 22 offenders on unpatched main, green on this branch and on the tree merged with latest main (557 files), and an adversarial corpus of 33 cases now scores zero false positives and zero false negatives. Also corrected two docstring claims: neither collecting job runs on Windows, and the read is governed by locale.getencoding(). * Walk eager comprehensions and treat io.open as the builtin Two regressions from the previous commit, both reproduced against the AST before changing anything. Lumping list, set and dict comprehensions in with generator expressions was wrong. Only a genexp is lazy; the other three run their element expression, their filters and their nested iterators immediately, so CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time read the guard was silently missing. Comprehensions are now walked in full and only the genexp keeps the outermost-iterable-only treatment. io was also in the not-a-path-opener list, but io.open is the builtin, with the same mode position and the same platform default. io.open(CHECKED_IN_FILE) is exactly the hazard this guard exists for, so it is matched now, with binary modes and a pinned encoding still exempt. tarfile.open and fitz.open stay exempt since neither has an encoding to name. Verified: 13 targeted cases covering all five eager comprehension forms and io.open in text, binary and pinned shapes all classify correctly; still 22 offenders on unpatched main; green on this branch and on the tree merged with latest main. * Close three more walker gaps in the import-time guard All three reproduced against the AST first. A generator expression handed straight to a call is consumed there, so DATA = "".join(p.read_text() for p in paths) runs its element at import. Only an unconsumed genexp bound to a name stays lazy, so the walker now follows the consumed ones in full and keeps the outermost-iterable-only treatment for the rest. if "__main__" == __name__ is an equivalent and accepted spelling of the main guard, but requiring __name__ on the left meant its body was treated as import-time code. That is a false positive on a block pytest never runs, so both operand orders are recognised now. The helper table was built from module-level defs only, so a def in a class body invoked while the class is constructed was never followed, contradicting the walker's stated coverage of class bodies. Helpers are now collected from the module body and from class bodies at any nesting. Verified: 15 targeted cases including all three fixes and the earlier ones still classify correctly; still 22 offenders on unpatched main; green on this branch and on the tree merged with latest main. * Handle positional read_text encodings, lazy generators and nested helpers * Guard reads reached from test bodies, unbound Path calls and __file__ paths * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow derived paths, skip lazy generator helpers, cover compressed openers * Guard the CLI tests, helper parameters and unbound Path arguments * Discover test roots and follow literal, in-place and tuple-derived paths * Identify module openers by import, unwrap starred paths, pin subprocess snippets * Resolve import origins, seed helper locals, follow named generators and parametrize * Scope imports lexically, list tracked test files, bind unpacked names * Resolve aliased openers, keyword-only params, destructured targets, next() * Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438 * Harden the CLI encoding guard against detached streams for PR #7438 * Tighten the encoding guard's path and scope analysis for PR #7438 * Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438 * Resolve qualified path classes and scope conditional imports for PR #7438 * Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-07-26 23:31:56 -07:00
Leo Borcherding	3ea6d14c39	AMD: CI coverage for recent fixes, plus three wrong gfx ids (#7431) ... * ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite Three merged ROCm fixes shipped without tests, and the CI wiring that would have run them was gated on files the fixes do not touch. Tests added (113): tests/studio/install/test_rocm_arch_table_parity.py (27) diffs the four duplicated gfx -> AMD pip-index tables across install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py, plus the GPU-name -> arch tables and the torch 2.11 pin allowlist. tests/studio/install/test_rocm_native_linux_lib_dirs.py (26) covers #7233: system-ROCm lib dirs prepended ahead of bundled libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env var, root resolution order, and source parity between the two copies. studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46) covers #7292: registration on the CUDA dispatch key, grouped and ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 + RDNA4 name gate, executed from the shipped source rather than a copy. tests/studio/test_ci_shell_suite_coverage.py (14) fails if either shell runner goes back to a hardcoded list or skips a file without a recorded reason. CI wiring: studio-backend-ci.yml: add install.sh / install.ps1 to the path filter (the suites it runs assert against those two files, so install-only changes -- the shape most AMD/ROCm routing fixes take -- skipped it), and replace the 13-file hardcoded shell list with directory discovery. That list had fallen seven files behind, including test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm WSL reroute, which had never run on a PR. tests/run_all.sh: same discovery loop so local and CI agree. * Test review fixes: assert on outcomes, not on the code under test Self-review of the previous commit found four tests that passed for the wrong reason. 1. The arch-table parity test pinned expected gfx ids copied out of the shipped tables, which enshrined three upstream inaccuracies as correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's ROCm compatibility matrix. The expectation is now the AMD pip index leaf -- the thing the tables exist to produce, and what a wrong answer costs the user. The three known drifts are listed explicitly with a test asserting they stay cosmetic, i.e. that the wrong and right ids still map to the same wheel index. That test turns red the day one of them starts routing users to the wrong wheel. 2. The RDNA4 device-name test extracted the regex from worker.py and then matched with it, so it could not fail. Widening the pattern -- the dangerous edit, since it forces the slow Python mm fallback onto RDNA3 users -- would have been silently accepted. It now reads the live pattern and checks it against fixed cases, plus asserts the name match stays guarded by `not _lin_arch` and that the name is lowercased before matching. 3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml, so reindenting the step would fail the build while a real regression to a hardcoded list could slip past a reformat. It now parses the YAML, finds the step by name, and asserts on the glob plus the absence of individual filenames. The path-filter test likewise reads the parsed trigger instead of scanning raw text. 4. A set comprehension in the parity helper had a ternary whose branches were identical. Mutation-tested: widening the RDNA4 regex, desyncing one copy of the name table, dropping install.sh from the path filter, and re-skipping the ROCm WSL shell suite each fail at least two tests. Verified on Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass. * Fix three wrong gfx ids in the GPU-name arch tables The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on three entries. Corrected against the "Radeon GPU" list at rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html: RX 9070, RX 9070 GRE gfx1200 -> gfx1201 (Navi 48, same die as the XT) RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101 (Navi 32, not Navi 31) PRO W7700 gfx1100 -> gfx1101 PRO V710 gfx1102 -> gfx1101 (Navi 32, not Navi 33) No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies of the index-family map. That collapse is why the errors survived being copied into six places -- the leaf-level tests could not see them. It was not purely cosmetic, though. install.sh's second copy feeds "Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the printed advice exported gfx1100 and made a wrong id authoritative for every later run. It would also have become a real misroute the moment AMD split a family across index leaves, as they already do for gfx1151/gfx1150. Fixed in all six places, which is two more than the table's own "kept in sync with" comments claim exist: install.sh _infer_amd_gfx_arch_from_gpu_name install.sh case "$_gpu_disp_mkt" (banner + env tip; undocumented) studio/setup.sh install.ps1 studio/setup.ps1 studio/install_python_stack.py Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the PowerShell copies keep the (?!S) lookahead. Test changes: - test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx ids transcribed from AMD rather than from the tables. Agreement between six copies proves nothing when all six were transcribed from the same mistake, so the ground truth has to come from outside. Verified it catches the bug: against the pre-fix tables it fails 6 tests. - The parity check now covers all six copies. It had four; the two install.sh copies were being treated as one, and _WIN_GPU_NAME_ARCH_TABLE was not checked at all. - test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong ids as expected values; updated, and extended with a 9060 XT and a 7900 XTX case so each RDNA3/4 die is represented. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard against unregistered copies of the GPU-name arch table Counting the copies by hand is what let them drift: the in-code "kept in sync with" comments claimed four, the arch-id fix found six, and scanning the tree turns up a seventh. TestNoUnregisteredArchTable rediscovers the copies from the source tree instead of trusting a hand-maintained list. A table line is one that names a card and gives its arch; real tables score 9-17 such lines and the only other hits in the repo are two single-line prose comments, so the three-line threshold is not load-bearing. A companion test asserts the scan still finds the known copies, so the heuristic cannot go blind and pass by finding nothing. The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests build their fake AMD host from. It states the mapping backwards (gfx -> the name torch should report), which makes it an independent witness: it had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six installer copies were wrong, and nothing compared the two. Now they are round-tripped against each other. RX 6700 XT is pinned as a known divergence rather than normalised. AMD's compatibility matrix documents no consumer RX 6000 card and no gfx1031 at all, the installer arm is commented "gfx103X family", and gfx1031 appears only as an index-family key, never as a value a name table emits. With no external source to correct against, changing shipped behaviour would be guesswork. A test fails if the divergence ever disappears, so the exemption cannot go stale. Also adds the reverse of the AMD-matrix check: a documented card that matches no arm anywhere is a silent CPU fallback rather than a wrong id. This cannot detect hardware nobody transcribed, which would need a live fetch of AMD's matrix and a non-hermetic suite; the docstring says so rather than implying coverage that is not there. Verified on Linux: 478 passed, plus all five new guards mutation-tested to confirm each fails when its invariant is broken. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Docstring said six copies; the list under it now has seven * tests: run discovered shell tests with bash, not sh tests/run_all.sh discovered tests/sh/ instead of listing files, but still invoked each one with sh. Every file there declares a bash shebang, and on Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh, test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh fail on bashisms under dash and pass under bash. The old hand-written list happened to name only dash-clean files, so switching to discovery is what surfaced it. Backend CI already used bash, so this was a local-only break. Guarded by a new test asserting both runners invoke tests/sh/ with bash. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table (src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152. Unlike the three ids already fixed here, this one is not wheel-neutral: repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with separately built torch wheels, so these laptops were installing wheels built for a different LLVM target. gfx1152 was absent from the codebase entirely, so it needed the index-family maps, the torch 2.11 floor lists (same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the Windows arch allowlist as well as the seven name tables. The parity test added in this PR did not catch it because its AMD-matrix expectations stopped at 890M/880M. Added the APU rows, so the case that actually changes a wheel is now covered: reverting the tables fails 9 tests naming 860M, 840M and Krackan. gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153 wheel family, so there is nothing to route it to. Verified: bash -n on both shell installers, PowerShell AST parse on both .ps1 files, python ast.parse on all touched modules, install suite 1334 passed with no new failures against main, shell suite 20 files. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add gfx1152 to unified-memory classifiers, make parity allowlist set-based Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and Strix Halo (gfx1151), but only the installers knew about it. The two runtime classifiers still had two-element arch sets, so a Krackan laptop got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp without GGML_CUDA_ENABLE_UNIFIED_MEMORY. - worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set, and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M cannot collide there: the function is only reached under _hw.IS_ROCM. - llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set. - Tests for both, including the :sramecc-:xnack- suffix form. TestGfx211AllowlistParity compared four hardcoded allowlist strings, so adding gfx1152 to all four installers correctly turned three assertions red without any installer actually disagreeing with another. Each test now extracts the set its installer holds and compares it to one EXPECTED constant. Order and spacing are free, membership is not, and the next leaf is a one-line edit instead of four. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-25 18:58:02 -05:00
Souravrajvi0	8c975fcbaf	fix: pin torchcodec for torch 2.10 and warn on ABI mismatch (#7299) ... * fix: pin torchcodec for torch 2.10 and warn on ABI mismatch Add unsloth[audio] extra with torchcodec>=0.10.0,<0.11.0 and emit a clear warning when installed torchcodec minors disagree with torch (unslothai/unsloth#7225). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(packaging): address Codex review on torchcodec/torch 2.10 compat (#7299) - Postpone annotations so import_fixes loads on Python 3.9 - Align TORCH_TORCHCODEC matrix with upstream (2.9: 0.8/0.9, 2.8: 0.6/0.7) - Fix mismatch hint upper bound (<0.11.0) and gate audio-torch210 suggestion - Split audio extra per torch minor; gate torch210 pin behind python>=3.10 - Bundle audio-torch210 only in *-torch2100 install extras * fix(security): refresh openai CRITICAL scan baseline hashes (#7299) openai package code drift reopened five CRITICAL findings in the extras pip-scan-packages shard (C2 loop body hashes + IMDS/network evidence). Update the reviewed allowlist evidence/hashes so CI gates on new findings only, not benign SDK churn. * chore: retrigger CI after baseline refresh (#7299) * chore: touch scan baseline comment to retrigger security audit (#7299) * Guard torchcodec version parsing so bad version strings cannot break import * Bundle audio pin into intel-gpu-torch210 and guard the mismatch warning * Tighten comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-23 19:12:52 -07:00
Daniel Han	3ab8dce97a	install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection (#6692) ... * install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection get_torch_index_url (and the studio-update mirror _detect_cuda_torch_index_url) chose the torch wheel family solely by probing the host GPU, with no override. In a headless / container / CI build the host driver is visible via the /proc/driver/nvidia/gpus fallback but nvidia-smi cannot report a CUDA version, so the function fell back to its cu126 default and installed the wrong wheels (e.g. a cu128 image got cu126 torch). Add an explicit override checked before any probing, in both the shell installer and the Python studio-update path: - UNSLOTH_TORCH_INDEX_URL full index URL, used verbatim (wins) - UNSLOTH_TORCH_INDEX_FAMILY family (cpu, cu128, rocm6.4, ...) appended to the mirror base (UNSLOTH_PYTORCH_MIRROR still honoured) This matches how the published GPU images select CUDA -- vLLM and SGLang take the CUDA version from an explicit build ARG rather than detecting it, and the Unsloth Docker base image already pins the cu128 index directly. Desktop installs are unchanged: with no override set, detection runs exactly as before. Adds test_get_torch_index_url.sh cases for the override (family, full URL, precedence, mirror base, trailing-slash strip, empty-ignored). * install: make the torch-index override authoritative across ROCm paths Address review feedback on the override added in this PR so a pinned index is honoured everywhere, not just in get_torch_index_url: - Skip the WSL ROCm bootstrap (root privilege + large downloads, probes /dev/dxg) when UNSLOTH_TORCH_INDEX_URL / _FAMILY is set; it previously ran before the override was consulted. - Skip the Radeon/Strix rerouting (which re-probes the GPU and overwrites the resolved URL with repo.radeon.com / repo.amd.com) when the index is pinned, so an explicit ROCm override (e.g. UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4) is kept. - install_python_stack.py: derive _TORCH_BACKEND from the override when UNSLOTH_TORCH_BACKEND is unset (standalone studio update), so _ensure_rocm_torch / _ensure_cuda_torch repair to the requested family instead of re-detecting. - Strip ALL leading/trailing slashes in the shell override to match the Python side (avoids 404s on strict pip proxies). Adds test cases for double-slash and leading/trailing-slash overrides. * install: honor pinned torch index in CUDA/ROCm repair paths Follow-up to the override work in this PR: the get_torch_index_url / install.sh reroute already respect a pinned UNSLOTH_TORCH_INDEX_URL / _FAMILY, but the Python repair helpers in install_python_stack.py still re-probed the GPU and could overwrite the pinned family. Make the pin authoritative there too: - _ensure_cuda_torch: an explicit cu* pin commits to CUDA wheels, so repair a ROCm-poisoned venv even when no NVIDIA GPU is visible here (headless / container / CI cross-install), instead of bailing on the GPU-presence gate. - _ensure_rocm_torch: skip the AMD per-gfx (Strix) reroute when a ROCm index is pinned, and in the generic reinstall path install from the pinned URL verbatim rather than re-detecting the host ROCm version. gfx*/rocm7.2 indexes serve torch 2.11+, so select the 2.11 package specs for a gfx leaf. - install.sh: raise the torch constraint to 2.11 for */gfx* indexes too, matching rocm7.2, so a pinned full-URL/family override that returns early keeps a valid constraint. Add _explicit_torch_index_url / _explicit_rocm_torch_index_url helpers and tests covering the no-GPU CUDA pin repair and the explicit gfx index honored verbatim. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: honor torch-index override on the Windows installers too The pinned-index work landed for install.sh and install_python_stack.py, but the Windows installers still picked the wheel index from GPU probing. Extend the same UNSLOTH_TORCH_INDEX_URL / _FAMILY contract so a pinned index wins on every platform: - install.ps1: Get-TorchIndexUrl returns the pinned URL/family before nvidia-smi probing; the AMD ROCm reroute is skipped when the index is pinned, so an explicit cpu/cu* pin on an AMD host is not overwritten. - studio/setup.ps1: add shared Get-PinnedTorchIndexUrl / Get-TorchIndexLeaf helpers; the stale-venv check, the install selection and the AMD reroute all honor the pin, and the CPU/CUDA install pulls from the resolved index URL. - tests: parity test that all four installers read both override vars and the two Windows installers gate the AMD reroute on the pinned flag. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: complete pinned-index handling for ROCm/Windows edge cases Follow-ups to the override work flagged in review: - install.ps1: a pinned gfx*/rocm>=7.2 index previously skipped the AMD reroute that sets the torch>=2.11 floor, so the generic install used torch>=2.4,<2.11 and could resolve the known-bad _grouped_mm wheel. Route a pinned ROCm index through the ROCm install path with the 2.11 floor + companions, and guard the companion-spec lookup so a skipped reroute block cannot null-deref. - studio/setup.ps1: the stale-venv check compared the installed flavor (cuXXX/cpu, with +rocm misread as cpu) against the raw pinned leaf (gfx1151 / rocm6.4), so a correct pinned ROCm venv was always marked stale. Classify +rocm wheels as the generic 'rocm' flavor and normalize a pinned rocm*/gfx* leaf to 'rocm' before comparing (cu* stays specific so cu126-vs-cu128 still rebuilds). - install_python_stack.py: _ensure_cuda_torch now also reinstalls from a pinned CUDA index when the venv carries a CPU wheel (headless CPU-venv-to-CUDA cross-install via 'studio update'), not only when it finds a ROCm build. - tests: parity assertions already cover all four installers honoring the override. * install: finish pinned ROCm/CUDA edge cases on Windows + repair path Follow-ups to the previous round: - studio/setup.ps1: a pinned gfx*/rocm>=7.2 index now routes through the ROCm install path with the 2.11 floor + companions (it previously fell through to the CUDA branch with bare torch/torchvision/torchaudio against the ROCm index). The CPU/CUDA fallback index is forced to the CPU wheel index when a ROCm index is active, so a failed pinned-ROCm install does not retry the ROCm mirror. - studio/setup.ps1: the stale-venv check no longer treats an unrecognized pinned URL leaf (e.g. a PEP 503 mirror ending in /simple) as a torch flavor tag, which was marking a correct venv stale; cu*/cpu/rocm/gfx leaves are still compared. - install.ps1: the post-failure CPU fallback uses an explicit CPU index instead of , which for a pinned ROCm index was the ROCm mirror itself (so the 'fallback' just retried the failing index and aborted the installer). - install_python_stack.py: _ensure_cuda_torch now also reinstalls when the venv's CUDA family differs from a pinned one (installed cu126 vs pinned cu128), not only CPU->CUDA; the probe reports the installed cuXXX tag for the comparison. * install: keep the ROCm to CPU fallback install inside the retry-helper window The pinned-ROCm CPU fallback computes an explicit CPU index, but the comment explaining why it cannot reuse $TorchIndexUrl pushed the actual Invoke-InstallCommandRetry / --force-reinstall call more than 600 chars past the "ROCm PyTorch install failed" message, so test_pr5940_followups's window check no longer saw the retry helper. Move the CPU-index computation and its comment above the failure substep so the retrying force-reinstall stays adjacent to the message. No behavior change: same explicit CPU index, same retry, same --force-reinstall. * install: address #6692 review round 5 (ROCm/CPU pin edge cases) setup.ps1: - Stale-venv check: treat an AMD/ROCm host (HasROCm or a resolved gfx arch) with no explicit pin as expecting "rocm", not "cpu", so a healthy +rocm venv is not flagged stale (which made installer-managed setup exit and direct update rebuild). - Pinned-ROCm install failure now routes into the force-reinstall CPU branch: CuTag stays the rocm/gfx leaf on failure, so the condition also checks ROCmCpuFallback; otherwise the CUDA branch installed from the CPU index without --force-reinstall and kept the partial ROCm torch. - Explicit ROCm pin compare no longer collapses gfx*/rocm* to a generic "rocm": it compares the +rocmX.Y version (and the torch 2.11 line for gfx pins) so changing the pinned family (e.g. rocm6.4 -> gfx1151) rebuilds and applies it. install_python_stack.py: - _ensure_rocm_torch: an explicit ROCm wheel-index pin now bypasses the NVIDIA-present / no-AMD-GPU / unreadable-ROCm gates (headless/container/CI cross-install), mirroring the explicit-CUDA-pin bypass in _ensure_cuda_torch. - Add _ensure_cpu_torch: an explicit CPU pin (FAMILY=cpu or /cpu URL) now has a repair path that reinstalls CPU torch over an existing CUDA/ROCm build on a standalone update (which skips install.sh's flavor enforcement). install.sh: - Pin torchvision/torchaudio companions alongside torch for the rocm7.2 / per-gfx index and the Strix reroute (those AMD indexes publish companions independently and a bare name can resolve a torch-2.12-built wheel, an ABI mismatch). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * torch-index override: classify CUDA pin by leaf; trim blank shell overrides _ensure_cuda_torch only overrode the NVIDIA-presence gate for *any* pinned index, so a non-CUDA mirror URL (or a ROCm/CPU pin) on a non-NVIDIA host with ROCm torch could force a CUDA reinstall over a working ROCm venv. Add _explicit_cuda_torch_index_url() (leaf cu*), matching the ROCm/CPU helpers, and gate on it instead. install.sh::get_torch_index_url treated a whitespace-only UNSLOTH_TORCH_INDEX_URL / _FAMILY as authoritative (yielding an invalid index), unlike the Python .strip() and PowerShell IsNullOrWhiteSpace paths; trim leading/trailing whitespace first. * install: honor pinned torch index over CVD/GPU gates and fix leaf-based ROCm classification - install_python_stack.py: an explicit cu* pin now clears the CUDA_VISIBLE_DEVICES empty/-1 hide gate as well as the NVIDIA-presence gate, so CVD=-1 UNSLOTH_TORCH_INDEX_FAMILY=cu128 studio update repairs to CUDA wheels (parity with install.sh's get_torch_index_url override, which skips all GPU probing). Unpinned CVD=-1 still skips. - install_python_stack.py: _ensure_cpu_torch installs the bounded _CPU_TORCH_PKG_SPEC instead of a bare torch/torchvision/torchaudio trio; the /cpu index now also serves torch 2.11+, which is outside the supported <2.11 range. - install.sh: the torch>=2.11 constraint case matches the index leaf (rocm7.2|gfx*) instead of the whole URL, so a mirror base path containing a gfx/rocm7.2 segment with a cu*/cpu family is not false-matched onto the 2.11 line. - setup.ps1: the stale-venv check expects rocm torch only for arches the install path maps to a repo.amd.com wheel index; an unmapped/unreadable arch installs CPU, so a correct CPU venv is no longer marked stale. - Tests for each of the above. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten pinned torch-index override edge cases - install.sh: trim whitespace-only UNSLOTH_TORCH_INDEX_URL/_FAMILY before the _torch_index_pinned guard, matching get_torch_index_url, so a blank override no longer skips the WSL bootstrap and Radeon/Strix reroutes while detection still picks the normal index. - install.sh / install.ps1 / setup.ps1 / install_python_stack.py: force the torch 2.11 floor only for the gfx families with the <2.11 _grouped_mm bug (gfx120X-all, gfx1151, gfx1150). A pinned override to gfx110X-all/gfx90a/gfx908 stays on the default range, matching the automatic AMD path. - install_python_stack.py _ensure_cuda_torch: treat an untagged CUDA build under a CUDA pin as a family mismatch (reinstall), and match cuXXX pins narrowly (cu + digits) so a custom/current mirror leaf no longer forces CUDA over a CPU/ROCm venv. - install_python_stack.py _ensure_rocm_torch: reinstall when an explicit ROCm pin names a different ROCm family than the already-installed ROCm torch (the ROCm analogue of the CUDA cuXXX mismatch repair). Adds tests for each case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: fix second-order edge cases in pinned torch-index ROCm/CUDA handling Parse the ROCm torch probe positionally so an empty HIP marker is kept: CPU/CUDA torch no longer reads as HIP, so the ROCm reinstall is not skipped. Emit one "<marker>|<version>" line (like the CUDA probe) for a robust parse. Limit the gfx torch 2.11 expectation to the install allowlist (gfx120X-all/gfx1151/gfx1150). A pinned gfx110X-all/gfx90a/gfx908 index stays on the default <2.11 specs, so a correct 2.10+rocm wheel is no longer judged a mismatch and force-reinstalled every update. Distinguish an AMD per-arch wheel (three-part +rocmA.B.C) from a generic pytorch.org wheel (two-part +rocmA.B): a gfx per-arch pin over a generic 2.11 wheel now reinstalls the per-arch wheel, while an already-installed per-arch wheel is not re-flagged (no reinstall loop). Mirror all of the above in setup.ps1 via new Test-RocmGfx211Leaf / Test-CudaFamilyLeaf / Get-RocmPinStaleTags helpers, reused by both the install-spec path and the stale-venv check so they cannot diverge again. Require a digit after "cu" (^cu[0-9]) in setup.ps1, install.ps1 and install.sh so a mirror leaf like /custom or /current is not branded CUDA and does not rebuild the venv every run. Add tests: CPU/CUDA probe -> has_hip_torch False; gfx110X-all pin + 2.10 wheel not stale; gfx1151 pin + generic 2.11 wheel stale; gfx1151 pin + per-arch wheel not stale; /custom and /current not CUDA; plus cross-language allowlist and cu-digit parity guards, and a PowerShell unit test for the new setup.ps1 helpers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix ROCm/gfx pin case normalization, ROCm-tag requirement, and CUDA-leaf classification Normalize torch-index leaves to lowercase before the gfx*/rocm*/cu* allowlist matches so the canonical gfx120X-all (capital X) gets the torch 2.11 floor in install.sh (leaf, flavor and repairable helpers). Require an installed +rocm local tag before a rocmX.Y or non-2.11 gfx pin is judged satisfied in setup.ps1 Get-RocmPinStaleTags and the Python _rocm_pin_family_mismatch, so an untagged CPU/CUDA wheel never leaves the pin unapplied. Classify a leaf as CUDA only via ^cu[0-9]: the Python _TORCH_BACKEND derivation now uses _is_cuda_family_leaf, and install.sh brands cuda only on cu[0-9]* (unset on an unknown /current /custom mirror leaf) so the stack probes the GPU instead of skipping ROCm repair. Add bash, Python and PowerShell tests for capital gfx120X-all floor, current/custom not-cuda, and untagged-wheel ROCm pins. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: converge torch-index pin detection via a per-venv marker Introduce a torch-index MARKER that records the exact wheel --index-url used after each successful torch install, so `unsloth studio update` / repair makes the "did the pinned index change?" decision by an EXACT string compare rather than inferring it from the wheel +rocm/+cu version tag. The tag cannot encode the AMD per-arch gfx family (two 2.11 gfx indexes both install +rocm7.13.0), so the tag heuristic missed a gfx1151 -> gfx120X-all switch and a custom-URL swap. Marker path is per-venv (.unsloth-torch-index), one line = the resolved index URL, written atomically (temp + rename). Path, format and normalization are shared across all four installers (install.sh, install_python_stack.py, setup.ps1, install.ps1). - Reapply gfx pins on a per-arch target change: the marker's exact compare reinstalls when the pinned index differs, even when both wheels share a tag. - Honor custom ROCm URL pins during repair: an explicit index whose leaf is not rocm/gfx/cu/cpu (e.g. simple, current) now reinstalls torch VERBATIM from the pin when it differs from the marker ("URL wins verbatim"). - Align the KNOWN-2.11 rocm/gfx set to exactly rocm7.2 plus the gfx allowlist gfx120x-all/gfx1151/gfx1150 in every language; stop treating an unknown newer rocm (rocm7.3, which does not exist) as the 2.11 line speculatively. Backward compatible: with no marker (old venvs, torch installed out-of-band) the existing +rocm/version-tag heuristics still decide, and a matching marker never reinstall-loops. A cu128 CUDA pin stays a CUDA pin; custom and current leaves are not CUDA. Adds marker tests (py/sh/ps) plus cross-installer parity checks. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: keep the torch-index marker additive to flavor validation Three narrow fixes in the marker-based stale-venv detection: - setup.ps1: a matching marker no longer overwrites the detected installed flavor. The marker compare is now an additional rebuild trigger, so a stale wheel (torch swapped to a +cpu build while the marker still records a cuXXX pin) is still caught by the flavor check instead of being masked as up to date. - setup.ps1: a supported AMD arch carrying CPU torch is no longer marked stale and wiped. The downstream AMD Windows ROCm override upgrades CPU torch to ROCm in place, so wiping first would delete the venv and abort with "Virtual environment not found". Only a genuinely wrong CUDA wheel still rebuilds. - install.sh: the Radeon --find-links path records its repo.radeon.com base in the marker instead of the generic pytorch.org ROCm fallback index, so a later pin to that generic family correctly reinstalls rather than comparing equal. Mirrors install.ps1/setup.ps1, which already record the real AMD index. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: honor custom pins and repair pinned venvs in place Four follow-ups to the torch-index marker work: - install_python_stack.py: _ensure_cuda_torch/_ensure_rocm_torch now bail when an explicit custom-index pin names no known torch family, so a verbatim URL override (a private/simple mirror) is not clobbered by auto-detected CUDA/ROCm wheels before _ensure_verbatim_torch_index applies it. - install_python_stack.py: the ROCm marker is additive, not a substitute -- a matching marker still runs the family/version check so a wheel swapped after the marker was written is caught. Mirrors setup.ps1. - setup.ps1: a stale venv under an explicit pin, whose torch still imports, is repaired in place (force-reinstall torch from the pin in the dependency pass) instead of wiped. The wipe path only delegates to install.ps1, so on a direct update it stranded the user at "Virtual environment not found" instead of applying the new pin. A broken venv or unpinned drift still wipes/delegates. - install.ps1: when a pinned ROCm install fails over to a CPU base, the marker now records the CPU index actually used instead of the ROCm pin, so the next managed setup does not see CPU torch under a ROCm pin and abort as stale. * setup.ps1: keep the ROCm CPU-fallback force line the pr5940 test guards 5c93ffd4 folded the pin-change force-reinstall into the ROCm CPU-fallback condition on one line, so the exact literal that test_pr5940_followups.py checks (if ($ROCmCpuFallback) { $cpuForce = @("--force-reinstall") }) no longer appeared and the test failed. Split the two conditions into separate if lines: the ROCm fallback line is restored verbatim and the pin-change force is its own line. Both still set $cpuForce to the array, so @splat passes one arg. * install: honor exact CUDA/custom index URL pins in the torch-index marker Address three Codex review findings on the torch-index marker mechanism: - install.sh: after the ROCm CPU repair reinstalls torch from the generic $TORCH_INDEX_URL, record that as the marker source. A Radeon --find-links install set _TORCH_MARKER_INDEX_URL to its repo.radeon.com base earlier, so leaving it made the marker misreport Radeon wheels and a later Radeon pin would compare equal and skip a needed reinstall. - install_python_stack.py: _ensure_cuda_torch now consults the exact-URL marker (_marker_pin_mismatch) when the installed +cuXXX tag matches the pinned leaf, so a same-leaf CUDA mirror change (official cu128 to an internal cu128 mirror) is reinstalled and re-recorded instead of skipped. - _normalize_index_url / _normalize_family_leaf (install.sh, setup.ps1, install_python_stack.py): lowercase only KNOWN wheel-family leaves (rocm/gfx/ cpu/cuXXX) so gfx120X-all still matches gfx120x-all, while a custom (unknown-family) leaf keeps its case so a verbatim URL pin like /Current does not compare equal to /current. Tests updated to assert the refined behavior. * install: fix 3 torch-index marker edge cases (CPU mirror pin, Radeon leaf, migrated venv) Addresses three review findings on the torch-index override path: 1. CPU index URL change on an already-CPU venv. _ensure_cpu_torch returned early whenever torch was already a CPU build, so a standalone update that moved the pin (official /cpu -> a private UNSLOTH_PYTORCH_MIRROR /cpu, same +cpu tag) never reinstalled. It now consults the exact-URL marker and reinstalls only when _marker_pin_mismatch reports a different index, mirroring the CUDA/ROCm same-family handling. A matching marker (or none) still leaves CPU torch untouched, so there is no reinstall loop. 2. Radeon find-links directory misclassified as a pip ROCm family. A repo.radeon.com/.../rocm-rel-7.2.1 leaf starts with "rocm" but is a find-links listing, not a pip --index-url. The old startswith(("rocm", "gfx")) test routed it into a --index-url reinstall that fails against find-links. New _is_pip_rocm_family_leaf gates on ^rocm\d / gfx (matching install.sh's rocm[0-9]* and setup.ps1's ^(rocm[0-9]|gfx)), so a Radeon URL routes to the verbatim/marker path instead. 3. Migrated venv rewriting its marker to a pin it did not install. install.sh and install.ps1 write the marker unconditionally, so a migration that preserves existing torch recorded the newly requested pin and a later update then found a matching marker and skipped the reinstall the pin needs (e.g. a per-arch gfx1151 -> gfx120X-all switch, identical +rocm tag). Both now track _TORCH_INSTALLED_THIS_RUN and write the marker only when torch was actually installed or repaired this run. Also add Get-NormalizedFamilyLeaf to the setup.ps1 helper-extraction list in test_torch_index_marker.ps1 (it was added to setup.ps1 and the shell test in an earlier round but missed here) and add two unit tests covering findings 1 and 2. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: keep pinned torch repairs on the pinned index Two fixes for explicit index pins (UNSLOTH_TORCH_INDEX_FAMILY / _URL): 1. install_python_stack.py's repair paths ran uv without clearing the inherited uv index env vars. uv resolves the default index (--index-url or --default-index) at the LOWEST priority, so a UV_INDEX or UV_EXTRA_INDEX_URL mirror in the environment won for any package it served: a cu128-pinned repair could install torch from the mirror and then record the cu128 marker it never used. Verified empirically: with UV_EXTRA_INDEX_URL=.../cu126 exported, uv pip install torch --index-url .../cu128 resolves torch 2.13.0+cu126. Strip the four uv index env vars for pinned-index commands only, mirroring the gate install.sh, install.ps1 and setup.ps1 already have; non-pinned installs keep the user's mirror. 2. install.ps1 routed any pinned leaf matching rocm* through the ROCm --default-index path, so a custom find-links leaf like rocm-rel-7.2.1 was treated as a PEP 503 ROCm index and could silently fall back to CPU torch on resolution failure. Require a digit after rocm, matching install.sh's rocm[0-9]* and install_python_stack.py's ^rocm\d. Adds parity + unit tests for both (11 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: keep pinned repairs off UV_TORCH_BACKEND and narrow setup.ps1's rocm pin match Round 2 of the pinned-index hardening: 1. _build_uv_cmd converted UV_TORCH_BACKEND into --torch-backend before the new env isolation could act, and uv's torch backend redirects torch resolution to its own per-backend index even when --index-url is given (verified: a cu128-pinned dry run with UV_TORCH_BACKEND=cpu resolves torch 2.13.0+cpu). Pinned-index commands now never receive the flag and UV_TORCH_BACKEND joins the stripped env vars, so uv cannot re-read it. 2. setup.ps1's pinned reroute had the same bare rocm* glob install.ps1 had: a custom find-links leaf like rocm-rel-7.2.1 was routed through the ROCm --index-url path instead of the verbatim unknown-pin path. Now requires a digit after rocm, matching install.ps1, install.sh and _is_pip_rocm_family_leaf. 3. The marker test's case-normalization checks used -eq, which is case-insensitive in PowerShell, making them vacuous, and the unknown-leaf expectation was written lowercased while the implementation deliberately preserves custom-leaf case. Tightened to -ceq with the case-preserving expected value. Adds unit + parity tests for 1 and 2 (5 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: extend the pinned-index guards to every remaining surface Round 3 of the pinned-index hardening, closing the same holes on the surfaces the earlier rounds missed: 1. install.sh's pinned-install env scrub now clears UV_TORCH_BACKEND (uv's torch backend redirects torch resolution to its own per-backend index even against --default-index), and both PowerShell wrappers clear it in their pinned-install scrubs, matching install_python_stack.py. 2. setup.ps1's marker stale check still classified any rocm* leaf as a PyTorch ROCm family while the install selection is digit-gated, so a custom rocm-current / rocm-rel-7.2.1 pin stale-compared as not-rocm vs rocm and force-reinstalled on every studio update. The stale check now uses the same ^rocm\d gate. 3. install_python_stack.py's pinned-command scrub also strips PIP_EXTRA_INDEX_URL for the pip fallback: pip adds the env extra index in addition to --index-url, so an inherited mirror could satisfy torch off the pin while the marker recorded the pinned URL. PIP_INDEX_URL needs no strip since the explicit --index-url flag overrides it. Parity + unit tests extended (4 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: scrub find-links and carry the pinned scrub through pip fallbacks Round 4 of the pinned-index hardening: 1. UV_FIND_LINKS joins every pinned-install scrub (install.sh, install.ps1, setup.ps1, install_python_stack.py): uv's --find-links locations can satisfy torch off the pinned index the same way an extra index does. 2. setup.ps1's Fast-Install restored the scrubbed vars in its finally BEFORE the pip fallback ran, and never touched the pip env vars at all, so a failed uv attempt fell back to python -m pip with an inherited PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS able to win over the pinned --index-url. The scrub now wraps the whole function (uv attempt + pip fallback) and includes the pip vars; restore happens after both. 3. install_python_stack.py's scrub also strips PIP_FIND_LINKS for its own pip fallback, completing the PIP_EXTRA_INDEX_URL fix from round 3. Parity tests extended (2 new tests). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: digit-gate rocm leaves in marker normalization and ROCm side effects Round 5 of the pinned-index hardening (three custom-rocm-leaf edge cases): 1. _normalize_family_leaf lowercased every leaf starting with rocm, so a custom mirror leaf like rocm-Current compared equal to its lowercase form and a case-only pin change was skipped. URL paths can be case-sensitive. The rocm prefix is now digit-gated (rocm[0-9]*, matching _is_pip_rocm_family_leaf) in install.sh, setup.ps1 and install_python_stack.py, so only true family leaves (rocm7.2) are lowercased; a custom rocm-* leaf keeps its case. 2. setup.ps1 Test-MarkerPinMismatch compared normalized URLs with -ne, which is case-insensitive in PowerShell, so a case-only marker change (Simple vs simple) was treated as matching and the reinstall skipped. Now -cne. 3. install.sh gated the AMD bitsandbytes install and the "repair ROCm torch" --default-index reinstall on a bare whole-URL rocm glob, so a custom CPU/CUDA/private index whose leaf merely starts with rocm (rocm-current) was force-repaired from the wrong ROCm-only path whenever torch.version.hip was empty. Both now gate on _torch_index_is_rocm_family, computed once from the digit-gated leaf (rocm[0-9]*/gfx*). Tests: 4 new parity assertions plus 2 case-sensitivity marker checks. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: apply an explicit custom torch-index pin on the first update Round 6: an explicitly-set custom (unknown-family) UNSLOTH_TORCH_INDEX_URL was silently ignored on the first `studio update` of a venv that predates the marker feature, on both platforms, because the no-marker case was treated as "do nothing" and the version-tag heuristics cannot judge an unknown leaf. 1. install_python_stack.py _ensure_verbatim_torch_index now reinstalls verbatim when the marker is ABSENT (None), not only when it differs, and short-circuits only when the marker already records this exact pin. It then writes the marker, so every later update is a no-op. A user who did not set the override gets pin=None and is untouched, so an out-of-band torch install is never clobbered. 2. setup.ps1: for an unknown-family pin on a marker-less venv the stale-venv check now sets PinChangedForceReinstall so the torch block reinstalls in place from the pin. It deliberately does NOT set shouldRebuild, which would wipe the venv and strand a direct `studio update`. 3. setup.sh (the Linux `studio update` entry point) skipped install_python_stack.py entirely when unsloth was already current, so the marker-driven reinstall (both the verbatim custom pin and the cu/rocm flavor and family-change repair, e.g. gfx1151 to gfx120X-all) never ran. It now forces the dependency pass when a torch-index pin env var is set; the pass is idempotent and no-ops when the marker already matches. This mirrors setup.ps1's stale-venv pre-check. Tests: 3 new parity assertions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: expect first-update reinstall for a no-marker custom index pin Follow-up to d671d8fb2: _ensure_verbatim_torch_index now applies an explicit unknown-family URL pin verbatim on the first update when the marker is absent (instead of no-op), so the old test_verbatim_custom_url_no_marker_is_noop assertion was stale. Rewritten as test_verbatim_custom_url_no_marker_reinstalls_once: asserts the one verbatim reinstall from the pinned URL, that the marker is written, and that a second call with the pin still set is idempotent (no reinstall loop). * install: gate the pinned update pass on the marker and record a pin baseline Round 8, two follow-ups to the round-6 first-update pin fix: 1. setup.sh forced the full dependency pass on EVERY `studio update` while a torch-index pin stayed exported, even after the marker already recorded the same pin, turning quick updates into the expensive pass every time. It now probes install_python_stack.py --torch-pin-needs-apply (which reuses the exact marker normalization) and forces the pass only when the pin is not yet applied (marker absent or different); an already-applied persistent pin keeps the fast path. A probe error fails safe toward running the pass. setup.ps1 gets the same probe in its fast path for parity. 2. A known-family full-URL pin on a venv predating the marker (e.g. an installed cu128 build and UNSLOTH_TORCH_INDEX_URL pointing at a same-family mirror) left the marker absent forever: the _ensure_* helpers deliberately do not force a multi-GB reinstall of identical-family wheels on an old venv, so nothing recorded the pin and every update re-entered the pass. _record_torch_index_pin_baseline now records the resolved pin as a baseline after the ensure sequence when the family already matches and no marker exists, so the pin is tracked (a later genuine change is detected and applied) and the update loop is broken, without the redundant reinstall. Tests: 3 new baseline unit tests, 4 new parity assertions, and the CLI probe. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.sh: keep the pin probe's exit 1 from killing the update under set -e The --torch-pin-needs-apply probe deliberately exits 1 for the common steady-state answer (pin already recorded, keep the fast path), but it ran as a bare command under set -euo pipefail, so the whole studio update aborted before the exit code was even captured. Absorb the status with || _PIN_NEEDS_APPLY=$? and pre-seed 0 so all three outcomes route as documented: 0 runs the pass, 1 keeps the fast path, anything else fails safe into the pass. Parity test asserts the guard. * install: strip pin credentials, disable uv config discovery, bound verbatim installs Four verified fix groups from a 12-reviewer audit of the torch-index override feature, each reproduced before fixing: 1. Credential persistence: all four marker writers stored the raw pin URL, so an authenticated pin (https://user:token@mirror/simple) persisted its credentials in .unsloth-torch-index (mode 0644 under a default POSIX umask) and install_python_stack.py printed pin URLs verbatim in repair messages. Userinfo is now stripped before persisting and in every log/substep that interpolates a pin, via lockstep helpers (_strip_index_url_credentials in install.sh / install_python_stack.py, Remove-IndexUrlCredentials in install.ps1 / setup.ps1). The three normalizers strip too, so an OLD marker that already carries credentials still compares equal to the same pin: no reinstall loop on upgrade. Query strings deliberately stay in the marker; two indexes distinguished only by query must not compare equal. 2. uv configuration discovery beat the explicit pin: with a discovered uv.toml declaring torch-backend = "cpu" or a [[index]] entry, uv 0.10.12 resolves torch 2.13.0+cpu against an explicit --index-url/.../cu126 pin; UV_NO_CONFIG=1 restores +cu126 (reproduced both ways). The pinned-install scrub in all four installers now sets UV_NO_CONFIG=1 and drops UV_CONFIG_FILE. 3. The verbatim custom-index update path installed a bare, unconstrained torch trio while fresh installs from the same unknown-leaf pin apply the supported range; _ensure_verbatim_torch_index now installs the bounded trio spec, closing the fresh-vs-update asymmetry. 4. Query-bearing pins (.../cu128?token=x) classified by raw leaf split and force-reinstalled on every update (the installed cu128 never equals cu128?token=x). Query/fragment are now stripped before leaf classification in all four implementations; the marker comparison keeps the query per (1). Rejected after verification (no change): the pin-baseline record cannot produce a wrong later decision (every pin change still mismatches and reinstalls from the new pin); the venv temp-file symlink scenarios require an attacker who already owns the environment; pathological inputs like " / cu128 / " have no realistic caller and fail loudly. Parity, stack, rocm-support, marker (sh + ps1), pin-stale, index-url and flavor suites all pass (455 python + full shell/ps1 batteries). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: harden custom-pin repair against clobber, broken torch, and pip config Four follow-ups to the pinned-index audit fixes: 1. setup.ps1 routed an unknown-leaf custom pin through the CUDA branch with a bare torch trio while install.ps1 (fresh) and the Python verbatim path bound the supported range; the pinned unknown-leaf route now applies the same torch>=2.4,<2.11.0 bound. Known cu* leaves and unpinned runs are unchanged. 2. The final torch safety pass could not repair a clobbered unknown-family pin: intermediate dependency steps can pull torch from PyPI (the pass exists for exactly that reason), but the verbatim helper short-circuited on marker==pin and no flavor tag exists to probe. The helper now keeps a per-run snapshot of the installed trio (taken after a verbatim reinstall or on the first matching-marker pass) and reinstalls from the pin when the final pass sees the trio drifted. Probe failure skips the comparison; a reinstall refreshes the snapshot, so no loop. 3. _record_torch_index_pin_baseline could freeze a known-family pin as applied on a venv whose torch is missing or broken (every family helper returns without reinstalling when its probe fails), making --torch-pin-needs-apply report done forever. The baseline now probes the installed flavor and records only on a match: a cuXXX pin requires the matching +cuXXX tag, cpu requires a cpu build, rocm/gfx requires hip; probe failure records nothing. 4. The pinned pip fallback stripped PIP_* env vars but user/site pip config files still applied (a configured global.extra-index-url can satisfy torch off the pin). PIP_CONFIG_FILE is now pointed at the null device for pinned commands (pip loads no config files then), in _install_env_for_cmd and setup.ps1's Fast-Install pinned scrub. install.sh / install.ps1 have no pip fallback (uv-only), verified. Tests: 7 new rocm_support tests (snapshot reset fixture), 1 stack test, 2 parity tests. Full battery green (464 python, sh and ps1 suites). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: complete the pin-repair coverage across the fast path and platforms Three cross-platform follow-ups to the round-2 pin-repair fixes: 1. The --torch-pin-needs-apply probe only compared marker==pin, so a torch trio clobbered to the wrong family (a cpu wheel replacing cu128 via a later pip install) with a still-matching marker reported "already applied" and the _ensure_{cuda,rocm,cpu} repair never ran on the Linux fast path. The probe is now a testable _torch_pin_needs_apply() that also checks the installed flavor against a known-family pin (via a shared _torch_flavor_matches_pin() helper, so the baseline and the probe cannot drift). An unknown-family pin has no flavor to validate and a failed probe cannot prove drift, so both keep the fast path. 2. macOS ARM (real CPU/MPS torch, not NO_TORCH) never applied an unknown- family custom pin on update: both the verbatim path and the baseline returned on IS_MACOS while fresh install.sh honors the pin, so the marker was never written and setup.sh forced the dependency pass on every update forever. The guards are now IS_MAC_INTEL (Intel mac is already NO_TORCH), and the final pass applies the pin on macOS ARM. 3. The round-2 final verbatim repair sat in the step-13 sequence guarded not IS_WINDOWS, so on Windows a dependency step that clobbered torch after the pin was applied was masked by the matching marker (setup.ps1 does not re-validate the main venv's torch after calling this script -- verified). Step 13 now runs the verbatim snapshot-drift repair on Windows and macOS ARM too; the Linux-oriented cuda/rocm/cpu family helpers stay Linux-only. Tests: 13 new rocm_support cases (flavor drift, macOS ARM, Windows repair), parity updates. Full battery green (475 python, sh and ps1 suites). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: strip query tokens from the marker and tighten the pin-drift probe Four follow-ups to the round-3 pin-repair fixes: 1. The credential stripper feeding the torch-index marker and the logged repair messages dropped only user:pass@ userinfo, so a private feed that carries its auth token in the query string (.../simple?token=SECRET) persisted the token in the world-readable marker (mode 0644 under a default umask) and printed it in substep output. All four strippers (install.sh, install.ps1, studio/setup.ps1, install_python_stack.py) now drop the query and fragment before building the sanitized URL. A query is not part of a PEP 503 index's identity, so this also stops a rotated token from spuriously mismatching the marker and forcing a needless reinstall. 2. The --torch-pin-needs-apply fast-path probe accepted an untagged CUDA build (no +cuXXX local tag) under a specific cuXXX pin, but _ensure_cuda_torch reinstalls exactly that build to enforce the pin. The probe was more lenient than the repair, so the repair pass was skipped on the fast path. _torch_flavor_matches_pin now reports a mismatch for an untagged build under a cuXXX pin, forcing the pass. 3. The probe's ROCm branch accepted any HIP build for a rocm/gfx pin, while _ensure_rocm_torch decides a reinstall with the per-arch _rocm_pin_family_mismatch predicate (a generic +rocm7.2 wheel under a per-arch gfx pin, or a wrong ROCm version, is a mismatch). The probe now reuses that predicate, so it is as strict as the repair. This needs the installed torch version, so _probe_torch_flavor now returns (marker, cutag, version) and _torch_flavor_matches_pin takes the pin URL (extracting the leaf internally). 4. On Windows a known-family cu*/cpu pin is applied to the main venv by setup.ps1 before install_python_stack.py runs; a later dependency step can clobber it, and the GPU-aware _ensure_{cuda,cpu}_torch self-skip on Windows while the verbatim helper handles only unknown-family pins, so nothing repaired the clobber (setup.ps1 does not re-validate the main venv's torch afterward, verified). New _ensure_pinned_known_family_torch reinstalls a drifted cu*/cpu pin in the step-13 Windows/macOS-ARM branch; rocm/gfx per-arch specs stay owned by setup.ps1, unknown-family by the verbatim helper. A speculative ROCm 2.11 floor was also raised but is unreachable: the rocm7.2 index publishes no 2.x wheel below 2.11.0, and an unknown newer rocm is not floored speculatively. Tests: query/fragment strip cases in the sh + ps1 marker suites and the Python strip/marker tests; the tri-state helper and the probe/baseline harnesses moved to the (marker, cutag, version) flavor with matching versions; new probe cases (untagged CUDA, generic-rocm-under-gfx) and 8 _ensure_pinned_known_family_torch tests; a four-way query-strip parity assertion. Full battery green (1150 python, sh 26/26 marker, ps1 marker/flavor/pin-stale). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: reinstall markerless gfx pins and cap custom-index updates at torch 2.11 Two follow-ups from the pin-marker audit: 1. A markerless venv with a gfx per-arch 2.11 pin trusted the wheel version tag, which is byte-identical (+rocm7.13.0) across gfx120X-all / gfx1151 / gfx1150. A pre-marker install holding one gfx arch's wheel that is now pinned to a DIFFERENT gfx index was therefore never switched: _rocm_pin_family_mismatch returns no-mismatch for any three-part +rocm 2.11 wheel, and _ensure_rocm_torch's absent-marker branch fell through to that heuristic. _ensure_rocm_torch now forces a one-time reinstall when the marker is absent AND the pin leaf is a 2.11 gfx per-arch index; the reinstall writes the marker, so the next update compares exactly and does not loop (the correctly-pinned no-reinstall guarantee then comes from the exact marker compare, not the ambiguous tag). Non-gfx-2.11 pins (rocmX.Y, non-2.11 gfx) stay on the tag heuristic -- their tags are distinguishable. 2. The verbatim custom-index update path used _CUDA_TORCH_PKG_SPEC (torch <2.12.0) while a FRESH install of the same unknown leaf caps torch at <2.11.0 (install.sh's default TORCH_CONSTRAINT, and setup.ps1's custom-pin branch), so a private /simple mirror publishing torch 2.11 could upgrade a `studio update` to a state the fresh installer never produces. Added _CUSTOM_INDEX_TORCH_PKG_SPEC (torch>=2.4,<2.11.0), used only by the verbatim path; companions stay pinned for the same exclusive --index-url ABI reason as _CUDA_TORCH_PKG_SPEC (a bare name could pull a torch-2.12-built torchvision). _CUDA_TORCH_PKG_SPEC is unchanged (known-family cu/cpu repair correctly tracks install.sh's widened cu ceiling). Tests: 2 new markerless-gfx cases (one-time reinstall + marker write + no-loop second run, and the rocmX.Y absent-marker no-op), the pre-existing markerless gfx no-reinstall test flipped to assert the one-time reinstall (it had encoded the old tag-trusting behavior), and the custom-index bound assertions. 488 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: a matching marker must not mask a broken, clobbered, or misclassified torch Four round-6 follow-ups, all closing cases where a matching torch-index marker wrongly vouched for a torch that is not actually the pinned one: 1. _is_cuda_family_leaf matched cu+digits by PREFIX (^cu[0-9]), so a custom mirror leaf like cu128-private classified as CUDA family; the flavor check then compared the installed cu128 tag to the whole leaf cu128-private and forced a reinstall on EVERY update (never converging). The cu family is now matched EXACTLY (re.fullmatch cu[0-9]+), so a cu-suffixed custom leaf routes through the verbatim/unknown path with a stable marker. Mirrored in install.sh (_normalize_family_leaf: strip cu, require an all-digit remainder) and setup.ps1 / install.ps1 (^cu[0-9]+$). 2. _torch_pin_needs_apply returned False on a failed torch probe (missing or unimportable) under a matching marker, so setup.sh kept the fast path and a broken torch was never repaired. A failed probe now forces the pass: the marker cannot vouch for a torch that does not import, forcing is idempotent, and once torch imports again the probe succeeds and the forcing stops (self-resolving). Reverses the round-4 conservative choice for this case. 3. _ensure_verbatim_torch_index snapshotted the installed trio on the first pass with a matching marker and treated an unimportable torch (snapshot None) as "no drift, skip", so a torch clobbered to a broken state before the run was masked. A None snapshot now reapplies the pin. A torch clobbered to a WORKING-but-wrong build under an unknown-family pin remains undetectable from metadata (no flavor tag; reinstalling every update would be the loop this avoids) and is documented as a known limitation. 4. The step-13 Windows final repair reran only the verbatim (unknown-family) and known-family cu*/cpu paths, so a clobbered explicit rocm/gfx pin (the wheel setup.ps1 installed from AMD's per-arch index) was left in place. The branch now also runs _ensure_rocm_torch on Windows for an explicit rocm/gfx pin; it has a Windows path and no-ops when torch already links HIP, so it only reinstalls a genuinely clobbered ROCm venv (loop-safe). Tests: the round-4 failed-probe-trusts-marker test flipped to force the pass; new cases for the cu-suffix no-loop, the broken-torch verbatim reinstall, and the Windows rocm final-repair structure; item-2 exact-cu parity assertions. 490 passed. sh/ps1 marker + flavor + pin-stale suites all green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: repair Windows ROCm pins from the pinned URL and honor NO_TORCH Four round-7 review items, two of them regressions in the round-6 work: 1. _torch_pin_needs_apply ignored UNSLOTH_NO_TORCH. With a torch-index env var set and no marker, the failed-probe branch forced the dependency pass on every `studio update`, and the pass (which also honors NO_TORCH) never installs torch or writes a marker, so nothing could ever stop the forcing. It now returns False immediately under NO_TORCH: the pin only matters once torch is actually installed. 2. The step-13 Windows final repair (round-6) restored a clobbered explicit rocm/gfx pin by calling _ensure_rocm_torch, whose Windows path reinstalls from the arch AUTO-DETECTED via hipinfo, not from the pin. A user pinning a different gfx family or a private mirror was restored from the wrong source (and the wrong marker written), and a headless box was skipped entirely (the arch probe returns nothing). The repair now goes through _ensure_pinned_known_family_torch, which reinstalls from the PINNED url with the same per-arch floor setup.ps1 uses (2.11-line gfx leaves) or a bare trio (older arches, rocmN mirrors). It is gated on IS_WINDOWS since macOS ARM has no ROCm, and the existing flavor check keeps it loop-safe (a matching HIP wheel is left alone). 3. _ensure_verbatim_torch_index's broken-torch check (round-6) used "_installed_trio_snapshot() is None", but that helper reports a REMOVED torch as "torch==absent" (a non-None tuple) and a broken import as the stale on-disk version, so a missing or unimportable torch under a matching marker was read as "no drift" and skipped. The matching-marker path now confirms torch health with an import probe (_probe_torch_flavor): a torch that does not import reapplies the pin, while a healthy torch keeps the snapshot-based intra-run drift detection. 4. A unit test for _ensure_cpu_torch did not pin NO_TORCH False like its siblings, so a suite run with UNSLOTH_NO_TORCH=1 in the environment made the guard return early and the reinstall assertions fail spuriously. Tests: the round-6 broken-torch verbatim test re-encodes the non-None "torch==absent" snapshot case (the exact state the old "is None" check missed); new Windows-ROCm pinned-repair cases (reinstall from the pin, per-arch floor vs bare spec, matching-wheel no-op, off-Windows no-op); a NO_TORCH fast-path probe case; the parity test now asserts the Windows final branch does not auto-detect the ROCm index and that the helper reinstalls from the explicit pin. 494 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: floor the rocm7.2 index in the Windows pin repair; isolate marker tests Three round-8 review items, two of them downstream of the round-7 changes: 1. _ensure_pinned_known_family_torch gave a rocm<d> index leaf a bare torch/torchvision/torchaudio trio while flooring only gfx* leaves, so a Windows venv clobbered under an explicit rocm7.2 pin could reinstall an unbounded or ABI-mismatched trio from that exclusive --index-url. It now mirrors the spec the initial ROCm paths pin: the rocm7.2 floor for 2.11-line gfx leaves and rocm<d> leaves that serve torch 2.11, the <2.11 default for older rocm versions, and a bare trio only for older gfx per-arch leaves (which publish no floor), matching _ROCM_TORCH_PKG_SPECS / _ensure_rocm_torch. 2. test_verbatim_custom_url_no_marker_reinstalls_once called _ensure_verbatim_torch_index twice; the second call now hits the matching-marker health probe, and with pip_install mocked torch never becomes importable, so in a no-torch environment _probe_torch_flavor returned None and forced another reinstall, failing the idempotence assertion. The test now pins a healthy flavor so the idempotence check is about the marker, not ambient torch. 3. The TestEnsureRocmTorchMarker fixture patched os.environ per test but not _TORCH_BACKEND, which install_python_stack.py computes once at import from UNSLOTH_TORCH_BACKEND. A runner starting with a cuda/cpu backend made _ensure_rocm_torch early-return and skip the mocked repair these tests exercise. The fixture now neutralizes _TORCH_BACKEND so the marker tests are independent of the caller's installer-pin environment. Tests: the Windows floor-spec test now asserts a rocm7.2 mirror pin uses the rocm7.2 floor (not bare), plus a new rocm7.1 case that must fall back to the <2.11 default; the marker suite passes under a hostile UNSLOTH_TORCH_BACKEND=cuda / UNSLOTH_TORCH_INDEX_URL env. 495 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: apply same-flavor pin repoints, keep ROCm fallback nonfatal, bound custom companions Four round-9 review items, two of them regressions in the round-7 pin helper: 1. _ensure_pinned_known_family_torch returned as satisfied whenever the installed flavor matched the pin, so a same-flavor SOURCE change (one /cpu or /cu128 mirror to another, or a gfx1151 -> gfx120x-all per-arch switch, both carrying the same wheel tag) was never applied, while _torch_pin_needs_apply kept forcing the pass on the marker mismatch forever. It now also reinstalls when the marker records a DIFFERENT index of the same flavor, rewriting the marker so the next update matches (no loop), exactly as the Linux _ensure_{cuda,cpu}_torch helpers do. An absent marker on an already-matching venv is still left to the baseline recorder (no forced reinstall of a correct pre-marker venv). 2. That helper reinstalled a Windows ROCm pin with the FATAL pip_install, so when setup.ps1 had taken its CPU fallback (the pinned AMD index unavailable), the final repair re-hit the same missing index and aborted the whole install. The ROCm reinstall is now nonfatal (pip_install_try): on failure it leaves the CPU base in place and writes no ROCm marker, so the install completes -- matching _ensure_rocm_torch's Windows path. cu*/cpu pins stay fatal (authoritative source). 3. install.sh left torchvision/torchaudio bare for a pinned custom/unknown-leaf index (a private /simple mirror), unlike the Python update path's _CUSTOM_INDEX_TORCH_PKG_SPEC, so a mirror also exposing newer companion wheels could resolve a torch-2.12-built torchvision against the capped <2.11 torch. It now bounds the companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) for a custom leaf, gated on an empty _expected_torch_flavor_tag so known families keep their curated bare/floored companions. 4. install.sh's _expected_torch_flavor_tag matched cu[0-9]* by prefix, so a custom leaf like cu128-private classified as the cu128 family and force-reinstalled a correct +cu128 wheel on every run. It now requires exact cu+digits (routing the suffixed leaf to the custom path), matching the Python re.fullmatch(cu[0-9]+) and PowerShell, and feeding item 3's custom-leaf detection. Tests: new cases for the same-flavor marker-change reinstall, the nonfatal ROCm fallback (no marker on failure), the rocm7.2/older-rocm floor selection now split across the nonfatal path, cu-suffixed custom leaves in test_torch_flavor.sh, and the custom-leaf companion bounds in test_torch_constraint.sh. 497 python + 143 shell assertions pass; the marker suite still passes under a hostile UNSLOTH_TORCH_BACKEND=cuda env. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: bound custom-pin companions on the Windows setup path; isolate pin-probe tests Two round-10 review items: 1. setup.ps1's custom/unknown-leaf pin branch capped only torch ($cudaTorchSpec) and still asked the exclusive index for bare torchvision/torchaudio, so a private mirror that also serves newer companion wheels could install a torch<2.11 wheel alongside a torchvision>=0.26 / torchaudio>=2.11 built for a newer torch ABI, after which the marker records the pin as applied. It now bounds the whole trio (torch>=2.4,<2.11.0 / torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) for a pinned non-cu-family leaf, matching install.sh, install.ps1's fresh pinned install, and install_python_stack.py's _CUSTOM_INDEX_TORCH_PKG_SPEC. This completes the companion-bounds fix across all three installers; known cu* leaves keep bare specs (the family index bounds them). 2. The _torch_pin_needs_apply probe tests did not pin NO_TORCH False, so a test process launched with UNSLOTH_NO_TORCH=1 short-circuited the probe (the round-7 guard) and returned False for cases that expect the pass to run. The _needs_apply helper now patches NO_TORCH (default False) around the call, and the dedicated no-torch case passes no_torch=True explicitly. Tests: the cross-platform parity test now asserts setup.ps1 bounds the full trio (not just torch) for a custom leaf; the pin-probe suite passes under a hostile UNSLOTH_NO_TORCH=1 environment. setup.ps1 parses clean; 497 python + shell suites green. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: bound custom rocm-* pins, redact diag tokens, snapshot custom pins before base update Three round-11 review items, all reproduced before fixing: 1. install.sh's custom-index companion bounds gated on _expected_torch_flavor_tag returning empty, but that helper returned "rocm" for ANY rocm* leaf, so a custom mirror whose leaf starts with rocm but is not a pip family (a private rocm-current mirror, a Radeon find-links rocm-rel-7.2.1) escaped the bounds and installed bare torchvision/torchaudio. It now digit-gates rocm to rocm[0-9]* (matching the Python _is_pip_rocm_family_leaf ^rocm\d), so those custom leaves return "" and the <2.11 companion caps apply; real rocm7.2 / gfx per-arch indexes still classify as rocm. 2. _tauri_torch_index_family classified by the raw last path segment, so a pinned URL carrying auth in the query (.../rocm7.2?token=SECRET) had the token echoed verbatim into the emitted [TAURI:DIAG] line. It now strips query/fragment before classifying (mirroring the marker/log credential stripping), so no token reaches the diagnostic output; as a side effect .../cu128?token=x now classifies as cu128 instead of auto. 3. On studio update, the core package step (a newer unsloth can require a torch the custom pin does not satisfy, pulling a default PyPI trio) runs BEFORE the step-2b verbatim check, which then recorded the already-clobbered trio as the baseline for a matching marker and left the pin unapplied. A new _capture_verbatim_baseline() records the pre-clobber trio before the core step, so the verbatim pass detects the drift and reapplies the pin. Captures only for a matching custom pin with importable torch; a mismatched/absent marker or broken torch is left to _ensure_verbatim_torch_index. Tests: _expected_torch_flavor_tag rocm-current / rocm-rel cases; _tauri_torch_index_family token/fragment redaction with a no-leak regression guard; _capture_verbatim_baseline record/skip cases plus an end-to-end clobber-detection scenario; a structural guard that the capture runs before the core step. 501 python + shell suites pass; install.sh bash -n clean, shellcheck unchanged from base. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: match rocm family leaves exactly, enforce the rocm7.2 torch line, repair a broken pinned torch A pinned index is a pip ROCm --index-url family only when its leaf is an exact rocm<digits> / rocm<digits>.<digits> (rocm7.2) or a gfx* per-arch leaf. The prior ^rocm[0-9] prefix match also caught suffixed private-mirror leaves (rocm7.2-private, rocm7-current), routing them through the ROCm/companion-family path instead of the verbatim pin: the companion bounds were skipped and, on a pre-marker venv with a compatible +rocm wheel, the pin was never applied. Match the family exactly through one shared helper at every site: - install_python_stack.py: _is_pip_rocm_family_leaf (re.fullmatch), plus the two other loose gates it feeds (_normalize_family_leaf, _torch_flavor_matches_pin). - install.sh: a new _is_pip_rocm_family_leaf routes _expected_torch_flavor_tag, _torch_index_repairable, _normalize_family_leaf and the ROCm side-effect gate. - setup.ps1: a new Test-PipRocmFamilyLeaf routes Get-NormalizedFamilyLeaf and both pinned reroutes; install.ps1 anchors its reroute regex. _rocm_pin_family_mismatch (and its setup.ps1 mirror Get-RocmPinStaleTags) compared only the ROCm version, so a +rocm7.2 wheel whose torch release drifted off the 2.11 line (2.12/2.13 from an out-of-band upgrade or a custom rocm7.2 mirror) satisfied the family check while violating _ROCM_TORCH_PKG_SPECS['rocm7.2'] (torch>=2.11,<2.12). Flag it stale so the repair reinstalls to floor; >=2.11 alone is not enough, so the release is compared exactly against the 2.11 line for a KNOWN-2.11 rocm pin. _ensure_pinned_known_family_torch returned on a failed import probe, but _torch_pin_needs_apply forces the dependency pass on that same failed probe: a broken torch under a known-family pin was left in place and the pass was forced on every update. Treat an unimportable torch as drift and reinstall the pinned trio (the spec and marker derive from the pinned leaf, not the absent flavor); once it lands the probe succeeds and the fast path returns. Tests: exact-match cases across test_torch_flavor.sh, test_rocm_support.py, test_cross_platform_parity.py and the two .ps1 helper suites; the rocm7.2 release-line and broken-probe-reinstall cases; extraction lists updated for the new helpers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: anchor the PS pinned-ROCm floor gate and bound install.ps1 custom-pin companions Round 12 made every family CLASSIFIER exact, but the Windows install-flow floor gate reads $_pinRocm211 directly from the raw pinned leaf with an unanchored -match '^rocm(\d+)\.(\d+)' BEFORE any exact classification runs. A suffixed custom leaf (rocm7.2-private) matches that rocm7.2 prefix, so it takes the 2.11-floor branch and is force-routed through the ROCm install path before the exact-match elseif can send it to the verbatim install. Anchor the match ($) in both install.ps1 and setup.ps1 so only an exact rocmX.Y leaf is floored; a suffixed or newer-suffix leaf falls through to the verbatim path. The Python floor selection is already exact (dict lookups gated on _is_pip_rocm_family_leaf), so only the two PS scripts needed this. install.ps1's custom (non-cu-family) pinned-torch install bounded torch>=2.4,<2.11.0 but left torchvision/torchaudio bare, so a private mirror serving newer companions could pull a wheel built for a newer torch ABI while the marker records the pin as applied. Bound both companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) when the leaf is not a cu<digits> family index (a cu index bounds its own resolution), matching setup.ps1's Test-CudaFamilyLeaf gate and _CUSTOM_INDEX_TORCH_PKG_SPEC. Tests: parity guards for the anchored floor gate in both PS scripts and for install.ps1's bounded custom-pin companions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten comments in the torch-index-override paths Collapse the verbose comment and docstring blocks added across the installer scripts and their tests to fewer, clearer lines without changing behaviour. Remove a duplicated CUDA-spec comment block. Comments/docstrings only; no code changes (AST-verified). * install: repair a broken pinned torch on Linux, strip trailing slash in tauri family, count the final step _ensure_cuda_torch / _ensure_cpu_torch returned on a failed import probe (torch present but unimportable). With an explicit CUDA/CPU pin, _torch_pin_needs_apply forces the dependency pass on that same failed probe, and the base package update does not force-reinstall an already-installed torch distribution, so the broken torch was left in place and the pass reran every update without repairing it. Treat a failed probe under a pin as drift and reinstall from the pinned index (the reinstall rewrites the marker and the next probe imports, so no loop). This is the Linux counterpart of the known-family repair fix. _tauri_torch_index_family stripped the query/fragment before classifying but not a trailing slash, so a token-authenticated pin like .../cu128/?token=x collapsed to .../cu128/ and fell through the exact-suffix */cu128 and */cpu arms to "auto". Strip a trailing slash too, mirroring _torch_index_url_leaf. The Windows / macOS-ARM final torch-repair step (_ensure_pinned_known_family_torch) runs a progress step that base_total never counted (the final-step increment was gated to Linux), so _STEP ran one past _TOTAL on those platforms. Add the missing increment. Tests: broken-probe reinstall for the CUDA (family and URL pins) and CPU paths; trailing slash / slash+token cases for _tauri_torch_index_family; a full-flow progress-count guard asserting _STEP == _TOTAL on Windows and Linux. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten comments in the torch-index-override paths * install: harden the torch-index pin across all four installers Redact index-URL credentials from captured install logs before they print on failure. uv/pip failure text embeds the failing --index-url verbatim, so a user:token@ or ?token= secret could leak into the console. Add a shared redaction pass (_redact_install_output / Redact-InstallOutput) wired into the error-output dump in install.sh, install.ps1, setup.ps1 and install_python_stack.py. Verbose mode still streams live uncaptured output, so it is intentionally left unredacted (developer opt-in). Trim trailing slashes on the PATH only for a verbatim UNSLOTH_TORCH_INDEX_URL override, preserving a ?query/#fragment token. A whole-URL rstrip corrupted a base64 token ending in "/", and a single-slash strip left .../cu128// classifying as an empty leaf. Add _trim_index_path_slashes / Trim-IndexPathSlashes and route the override through it; strip ALL trailing slashes in the backend-branding leaf classifier so a double slash still yields the real leaf. Reject a trailing-dot ROCm leaf (rocm7.) in the bash family validator so it matches Python re.fullmatch(rocm\d+(?:\.\d+)?) and the PowerShell regex: both the major and the minor must be non-empty digits, so rocm7. is a custom verbatim pin, not a pip ROCm family. Scrub PIP_NO_INDEX and PIP_INDEX_URL for a pinned install in the two installers that have a plain-pip fallback (install_python_stack.py, setup.ps1): PIP_NO_INDEX=1 makes the fallback ignore every index including the pinned --index-url, and PIP_INDEX_URL replaces it. install.sh and install.ps1 install via uv --default-index (which ignores pip config/env), so they are unaffected. Add unit tests (bash, Python, PowerShell) and cross-platform parity tests covering credential redaction, path-only slash trimming, the rocm7. validator, the double-slash leaf, and the PIP_NO_INDEX/PIP_INDEX_URL scrub. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: redact captured torch-install output and warn on a failed pinned ROCm repair Close a redaction gap the earlier pass missed: setup.ps1's direct `Fast-Install ... | Out-String` branches (ROCm from $ROCmIndexUrl, CPU/CUDA from $TorchInstallIndexUrl, plus the Triton and T5 sub-venv installs) printed the captured $output verbatim on failure, bypassing Redact-InstallOutput. A private index carrying userinfo or a ?token= in the pin could leak into Windows Studio setup logs. Route every `Write-Host $output` through Redact-InstallOutput. Warn on a failed pinned Windows ROCm reinstall in _ensure_pinned_known_family_torch: the branch printed "reinstalling from it" then called pip_install_try, but had no else, so a failure continued silently and left the user believing the pin was applied while the old CPU/wrong torch survived. Mirror the auto-ROCm Windows path and warn, telling the user to retry. * install: redact captured output on the pip fallback and optional-install failure paths The uv install path already redacted its captured output, but pip_install's pip fallback runs through run(), which printed result.stdout verbatim on failure, and _print_optional_install_failure did the same. A pinned --index-url carrying userinfo or a ?token= could still leak there when uv is unavailable or the pip fallback also fails. Route both through _redact_install_output. The verbose pip_install_try path stays raw (developer opt-in), matching the other installers. * install: split the survive-updates marker subsystem into a follow-up The torch-index override PR grew a persisted per-venv marker plus repair machinery (stale-pin detection, verbatim re-apply, update-time reinstall triggers) that roughly doubled it. That subsystem is orthogonal to the core feature and is being reworked in a follow-up (versioned/hashed marker, full-URL pin baseline), so it moves there wholesale instead of shipping twice. What this PR still does: UNSLOTH_TORCH_INDEX_URL / UNSLOTH_TORCH_INDEX_FAMILY pick the torch wheel index at install time in all four installers, with the exact rocm/gfx/cpu/cu leaf classification, the torch 2.11 floor for the per-arch AMD indexes, bounded companions for custom leaves, credential redaction of captured installer output, path-only slash trimming, and the uv/pip index env scrubs. Flavor-based repair keeps honoring the pin: a wrong family under an explicit pin still reinstalls from the pinned URL, and setup.ps1 repairs a pinned stale venv in place instead of wiping it. What moves to the follow-up: the .unsloth-torch-index marker file and its writers/readers/normalizers, exact-URL pin-change detection on update (same-tag gfx switches, custom-mirror repoints), the verbatim trio snapshot and clobber re-apply, the pin-baseline recorder, and the --torch-pin-needs-apply fast-path probe in setup.sh / setup.ps1. Their tests (the marker sh/ps1 suites, the stale-pin suite, and the marker classes in the rocm/cuda/parity suites) move with them; the removed code is preserved on a local archive branch to seed that PR. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: re-apply a ROCm pin over an existing HIP wheel via the version tag The subsystem split left an explicit ROCm/gfx pin unenforced on `studio update` whenever the venv already imported ANY ROCm torch: the pinned reinstall lived inside the `elif not has_hip_torch` branch, so a rocm6.4 to rocm7.2 switch, a gfx1151 pin over a generic +rocm7.2 wheel, or a broken 2.12+rocm7.2 drift never re-applied the pin. Restore the markerless half of that detection: _rocm_pin_family_mismatch compares the pinned leaf against the installed wheel tag (exact rocmX.Y compare, the 2.11 gfx per-arch allowlist, the untagged-wheel rule), the HIP probe emits "<hip_marker>|<version>" again so the installed tag is available, and _ensure_rocm_torch reinstalls from the pinned URL when the tag mismatches even though HIP torch is present. setup.ps1 mirrors it: the stale-venv check routes a pinned rocm/gfx leaf through Get-RocmPinStaleTags instead of collapsing it to a generic "rocm" flavor, and the existing pinned in-place repair (no wipe) applies the change. What still waits for the follow-up marker PR, by design: pin changes the wheel tag cannot see -- a per-arch switch between two 2.11 gfx indexes (identical +rocm7.13.0 tag), a custom-mirror URL repoint under the same family leaf, and unknown-family verbatim pins. Those need the persisted index record. Tests restored with the code: the _rocm_pin_family_mismatch table, the five update-path cases (older-rocm reinstall, gfx-over-pre-2.11 reinstall, matching-pin no-reinstall, non-2.11 gfx no-reinstall, gfx-over-generic-2.11 reinstall), the "|" probe-format guards, and the AST-extracted Get-RocmPinStaleTags suite for setup.ps1. * install: compare major-only rocm pins, redact URL fragments, bound pinned CPU trio Three review fixes on the restored pin-repair path. The family classifier accepts a major-only rocm<d> leaf (rocm7), but the mismatch comparators only parsed rocmX.Y, so a rocm7 pin fell through to the 2.11-line fallback and INVERTED both verdicts: an installed +rocm6.4 wheel compared as satisfied (pin never re-applied) while a matching +rocm7.2 wheel compared as stale (reinstall loop). Major-only pins now compare on the major alone in _rocm_pin_family_mismatch and Get-RocmPinStaleTags: rocm6.x under a rocm7 pin is a mismatch, any rocm7.x satisfies it, an untagged wheel never does, and a bare +rocm tag with an unreadable version is accepted (matching the existing lenient unreadable fallback). The output redactors scrubbed userinfo and ?query= values but not #fragments, so a pin like https://mirror/whl/cu128#token=secret leaked the secret in captured uv/pip failure text -- inconsistent with the URL handling itself, which already treats fragments as sensitive. All four redactors gain a URL-anchored fragment rule (anchored so a bare "# comment" line in tool output is never touched). setup.ps1's CPU branch installed a bare torch/torchvision/torchaudio trio; fine for the unpinned host default, but a PINNED cpu index routes through the same branch and the /cpu index serves newer torch, so a fresh pinned CPU install could land an unsupported trio that _ensure_cpu_torch then keeps (it accepts any CPU build). Under a pin the branch now installs the bounded trio mirroring _CPU_TORCH_PKG_SPEC (torch>=2.4,<2.12.0 and matching companions); the unpinned path is unchanged. Tests: major-only rows in the Python mismatch table and the AST-extracted setup.ps1 suite; fragment + query-plus-fragment + bare-hash-comment cases in all four redactor suites; a parity check that the pinned CPU trio bounds exist, are gated on the pin, and mirror the Python repair spec. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install: tighten comments in the torch index override paths * tests: track the moved pass-through inheritance in the gguf order check Main moved the llama_extra_args pass-through inheritance out of the GGUF branch into _resolve_inherited_extra_args, which runs before it, so the source-order assertion's "if request.llama_extra_args is None" anchor no longer exists inside the branch and the check failed after the main merge. The test now asserts the same property in the current shape: inheritance before the GGUF branch (a carried --no-mmproj still shapes the hub guard's companion requirement), and marker, hub guard, unload in order within the branch. Full file passes (32 tests). * tests: anchor the inheritance order check on the call, not the definition source.index("_resolve_inherited_extra_args(") matched the function definition, which always precedes the endpoint, so the ordering assertion was vacuously true. Anchoring on "= _resolve_inherited_ extra_args(" pins the first call site inside the load endpoint (line 4505), which is the statement whose position relative to the GGUF branch the test is meant to guard. 32 tests pass. * tests: align the gguf order test with main Main fixed the stale ordering assertion in PR 7252; adopting its version verbatim removes this file from the branch diff entirely and avoids a conflict on the next main merge. 32 tests pass. * install: bound the companion constraints to torch's window everywhere A full platform x vendor validation matrix over this branch surfaced a real trio mismatch on the cpu/mac paths: torch is capped <2.11 (installs 2.10.0+cpu) but the bare torchaudio companion resolves 2.11.0+cpu, because torchaudio 2.11 dropped its exact torch pin. Reproduced in a sandboxed end to end cpu install. torchvision still exact-pins torch and self-corrected. The default companion constraints are now bounded to torch's window (<0.26 / <2.11) and widen together with the cu* torch window (<0.27 / <2.12), so every leaf resolves a paired trio. Verified with uv dry-runs on the cpu, cu130, and rocm6.4 leaves (2.10.0/0.25.0/2.10.0, 2.11.0/0.26.0/2.11.0, 2.9.1/0.24.1/2.9.1) and a rerun of the sandboxed cpu install, which now lands torch 2.10.0+cpu with torchaudio 2.10.0+cpu. The Strix WSL reroute now also forwards UNSLOTH_TORCH_INDEX_URL and UNSLOTH_TORCH_INDEX_FAMILY into the rerouted 24.04 distro; dropping them silently reverted the child install to auto-detection, defeating the pin this branch introduces. test_torch_constraint.sh updated: the bounded companions must appear at the defaults and the custom-leaf block, no bare companion may remain, and the cu* widen must carry the companions with it. * install: harden the override path against reroute drift and credential leaks Review sweep focused on default-path idempotency found no defects on the unset path; these fixes cover the override path and failure reporting. install.sh: - The early WSL Strix Halo distro reroute now honors an explicit index pin (UNSLOTH_TORCH_INDEX_URL / _FAMILY): the pin is used in the current distro instead of probing the GPU and re-entering another distribution, matching the contract of the later Radeon and Strix guards. Whitespace only values do not gate, in parity with get_torch_index_url. - Verbose mode now streams installer output through the credential redactor; it previously bypassed the redaction the quiet path applies. The exit code survives the pipe via an rc file since the script runs under plain sh with no pipefail. - The kept-release fallback warning now strips credentials from the index URL before printing it. install.ps1: - Bounded torchvision and torchaudio next to every capped torch install (custom pin, ROCm CPU fallback, CUDA flavor repair). torchaudio 2.11 dropped its exact torch pin from the wheel metadata, so a bare companion beside torch<2.11 can resolve a mismatched 2.11.0 build, cu family indexes included. Mirrors the install.sh companion bounds. studio/install_python_stack.py: - The verbose failure path now redacts index URLs in pip and uv output before printing, matching every other output site in the file. All sh, ps1 and python installer test suites pass (the host-defaults suite has a known pre-existing failure unrelated to this change). * install: redact verbose Windows installer output and repair the parity tests Follow-ups to the override-hardening commit, from review: - install.ps1 Invoke-InstallCommand and setup.ps1 Invoke-SetupCommand now pipe verbose output through Redact-InstallOutput per record, and the three verbose Fast-Install torch call sites (ROCm, CPU, CUDA) do the same: uv and pip echo the pinned index URL, credentials included, in their errors, and verbose mode previously bypassed the redaction the quiet paths apply. ForEach-Object and Out-Host leave $LASTEXITCODE untouched, verified with a native command exiting 7 behind the pipe. - test_cross_platform_parity.py: the install.ps1 companion-bounds assertion now matches the implemented behavior (bounds on every index, no cu-family exemption, since torchaudio 2.11 dropped its exact torch pin) instead of requiring the removed $_pinCuLeaf gate. - test_rocm_support.py: the WSL reroute guard test slices the whole function body to its closing brace instead of a fixed 1200-character window, which the new pin-gate preamble had outgrown. 428 tests pass across the parity, install stack and rocm support suites; the sh and ps1 installer suites pass unchanged. * install: tighten comments in the torch-index and ROCm/CUDA repair paths * install: digit-gate the gfx family leaf and honor ROCm pins in the Windows repair Two review follow-ups on the override path: - The pip ROCm family predicate accepted ANY gfx-prefixed leaf, so a custom verbatim pin like /gfx-private classified as a ROCm family and enabled the ROCm-only side effects (AMD bitsandbytes, ROCm torch repair) on a mirror that may serve CPU/CUDA wheels. gfx now requires a following digit (gfx90a, gfx1151, gfx120X-all), consistently in install.sh, install_python_stack.py, install.ps1 (family gate and expected-flavor classifier) and setup.ps1, matching the strictness the rocm side already had (rocm7.2-private stays verbatim). The broader backend BRANDING globs are unchanged on purpose: radeon repo leaves (rocm-rel-X.Y) must still brand the rocm backend without being force-repaired as a family. - The Windows branch of the ROCm torch repair always installed from the public per-arch index, ignoring an explicit ROCm-family pin: after a pinned setup.ps1 install failed to a CPU base, the repair retried repo.amd.com instead of the pinned index. The branch now resolves _explicit_rocm_torch_index_url() first, uses it as the install index when set, and mirrors the Linux pin contract by skipping the NVIDIA and gfx-detection gates a pin is documented to override. Source-assertion tests updated to the tightened predicate and the new repair label. 1165 tests pass across the parity, install stack and studio install suites; the sh and ps1 suites pass; both PowerShell installers parse clean. * Remove scratch archives accidentally committed with the comment pass The temp/ archive copies of installer and test files were working scratch, not PR content, and inflated the diff by about nine thousand lines. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-20 00:58:52 -07:00
Andrew Chen	b307823b1d	fix(chat_templates): bind loop_messages when default_system_message is None (#7199) ... * fix(chat_templates): bind loop_messages when default_system_message is None construct_chat_template(default_system_message=None) built a system part that binds loop_messages only inside the `{% if messages[0]['role'] == 'system' %}` arm. The `Fix missing loop_messages` step right below then found no unconditional `{% set loop_messages = messages %}`, concluded loop_messages was missing, and rewrote `{% for message in loop_messages %}` back to `{% for message in messages %}` -- undoing the `messages[1:]` skip. A caller-supplied system message therefore reached the loop and tripped raise_exception: Only user and assistant roles are supported! Add the `{% else %}` arm so loop_messages is always bound, mirroring the default_system_message is not None branch minus the default text. That also stops the rewrite from firing, since the unconditional binding is now present. Renders before / after, same template, same inputs: default_system_message input before after None system msg raise_exception 'Be terse.\n### User: Hi\n' None no system '### User: Hi\n' unchanged 'You are helpful.' system msg 'Be terse.\n### User: Hi\n' unchanged 'You are helpful.' no system 'You are helpful.\n...' unchanged The rewrite still fires for templates with no {SYSTEM} part, which is what it was there for -- verified unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Scope loop_messages binding to {SYSTEM} templates for PR #7199 The None branch now only adds the else arm when system_part contains {SYSTEM}, so a static prefix with no {SYSTEM} placeholder keeps raising on a caller system message instead of silently dropping it. Strengthen the tests: assert the default does not leak when a caller system message is present, and add a regression test for the static prefix case. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-07-19 06:33:48 -07:00
Daniel Han	a9be36830e	Installer: allow torch 2.11.x on the CUDA install path (fresh install + studio) (#6959) ... * Studio: allow torch 2.11.x on the CUDA install path The CUDA torch repair path (_ensure_cuda_torch) installs torch/torchvision/ torchaudio from an exclusive --index-url, so _CUDA_TORCH_PKG_SPEC decides exactly which torch the Studio venv gets. It was capped at torch<2.11.0, so on a cu128/cu130 host the venv resolved torch 2.10.x even though the CUDA indexes now publish torch 2.11.0. That left the Studio venv a torch minor behind the torch 2.11.0 Docker base image, so the CUDA dedup step would relink base libs under a mismatched torch. Raise the upper bound to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so the CUDA install path lands on torch 2.11.x, matching the rocm7.2 spec and the base image. The torchao selector already maps torch 2.11 -> torchao 0.17.0, and _ensure_flash_attn degrades gracefully when no prebuilt wheel matches (Blackwell skips it outright; non-Blackwell prints a warning and continues), so no other pin needs to move. Add test_cuda_torch_spec.py to lock the bound (torch 2.11.x in, 2.12.x out) and assert the CUDA and rocm7.2 upper bounds stay in lockstep. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: use zip(strict=True) so a spec length mismatch fails loudly * install.sh: widen the CUDA torch ceiling to <2.12.0 so a fresh install matches the base Raising _CUDA_TORCH_PKG_SPEC alone was not enough: that spec only feeds _ensure_cuda_torch(), the ROCm-poisoning repair path that early-returns on a normal NVIDIA host. A fresh CUDA install (including the studio Docker build, which runs `bash install.sh --local`) takes its torch from install.sh's TORCH_CONSTRAINT, which was still capped at torch>=2.4,<2.11.0, so cu12x/cu13x resolved torch 2.10.x and the venv landed a minor behind the torch 2.11.0 base image. Extend the existing `case "$TORCH_INDEX_URL"` block (which already relaxes rocm7.2) with a `*/cu[0-9]*` branch that widens the ceiling to <2.12.0, keeping the >=2.4 floor so an older CUDA index (e.g. cu118) that tops out below 2.11 still resolves. The CPU wheel and older ROCm tags stay on <2.11.0 (the glob does not match /cpu). torchvision/torchaudio are bare on this install line and resolve their compatible companions via wheel metadata, matching the rocm7.2 pattern. Add behavioral tests (Python + shell) exercising the case block: cu118/124/126/ 128/130 widen to <2.12.0, rocm7.2 stays 2.11.x, and /cpu plus older ROCm keep the default <2.11.0. * install.sh: key the CUDA torch widening off the index leaf, not the full URL The `*/cu[0-9]*` glob matched a `cu<digit>` segment anywhere in TORCH_INDEX_URL, so a custom UNSLOTH_PYTORCH_MIRROR whose base path contains e.g. cu128 but whose final leaf is cpu or an older ROCm tag would still widen TORCH_CONSTRAINT to <2.12.0, contradicting the block's own comment and letting a CPU / older-ROCm mirror resolve torch 2.11.x. Match on _torch_index_leaf (the final path segment the backend classification just above already computes) so only a real cu*/ rocm7.2 leaf is affected; cpu and older ROCm keep the default <2.11.0. Update the Python + shell tests to mirror the leaf-anchored case and add regression cases for a mirror base that contains cu128 but resolves to a cpu / rocm7.1 leaf. * install: freeze the torch trio during the with-deps unsloth installs Released unsloth wheels can pin an older torch than Step 1 installed (unsloth 2026.7.2 declares torch<2.11.0), so the with-deps resolve from PyPI silently downgrades the pinned +cuXXX torch trio to PyPI's default wheel. The flavor guard cannot catch every such swap: PyPI's torch 2.10 default is itself cu128-flavored, so the cuXXX tag comparison still matches while the version silently drops. Freeze the just-installed trio with uv --overrides (overrides replace dependency requirements during resolution), keeping torch 2.11.0+cuXXX in place while unsloth's other dependencies resolve normally. Verified on the cu128 path: without the override torch drops 2.11.0+cu128 -> 2.10.0; with it the trio survives and unsloth 2026.7.2 + unsloth-zoo install cleanly. * install: fold UV_OVERRIDE env files into the torch-trio overrides file The CLI --overrides flag is the command-line form of UV_OVERRIDE, so passing it replaced any overrides file already exported for the process; macOS arm64 exports UV_OVERRIDE=overrides-darwin-arm64.txt for the same generic install path and would have lost those pins. Concatenate any UV_OVERRIDE files into the temp trio file so both keep applying. * install: extend the torch-trio overrides guard to migrated installs Four follow-ups to the Step-2 --overrides guard, all empirically verified: 1. The migrated-environment with-deps unsloth install resolved unsloth>=2026.7.2 (which pins torch<2.11.0) without the overrides file, so a migrated CUDA venv on torch 2.11 was silently downgraded -- the exact bug this branch fixes on the fresh path. The overrides build is now a function (_build_unsloth_torch_overrides, reading the trio installed at call time) invoked by both with-deps paths; the migrated no-torch path installs --no-deps and stays unguarded. 2. The overrides temp file is now cleaned by the EXIT trap (same pattern as _UV_OVERRIDE_TMPDIR, pre-initialized empty so an inherited value can never reach the trap's rm); previously any Step-2 failure leaked it. 3. Folding UV_OVERRIDE files used cat, which joins the last requirement of a file lacking a trailing newline onto the next file's first requirement (reproduced: idna==3.10certifi==2025.1.31 makes uv fail parsing). 4. Inherited torch/torchvision/torchaudio override lines are now filtered out when folding: uv intersects duplicate overrides rather than last-wins (verified on uv 0.10.12: direct conflict is unsatisfiable, transitive conflict silently backtracks), so a conflicting inherited trio pin would break the resolve the generated exact pins protect. Both 3 and 4 are handled by a single newline-terminating awk filter that preserves non-trio overrides (torchmetrics, torchao, ...). test_unsloth_torch_override.sh extended: migrated-path coverage, trap assertion, and a functional fold test (14 checks). * installer: tighten comments * install: keep the existing torch release when re-running the installer Re-running `curl -fsSL https://unsloth.ai/install.sh | sh` over an existing install rebuilds the venv for clean state, which silently moved users to the newest torch in range (2.10 -> 2.11 once the constraint widened). A torch the user already validated must survive an unsloth update. Before the old venv is moved aside for rollback, its torch version is probed (last stdout line only, so sitecustomize noise cannot corrupt it). After the index leaf is chosen, _previous_torch_pin turns that version into a torch==X.Y.Z pin, but only when it cannot do harm: - cu*/cpu leaves only; rocm leaves keep their floors (rocm7.2 must land 2.11 for the Strix _grouped_mm fix) and the Radeon wheel-matching path is untouched. - The wheel's flavor tag must match the freshly chosen leaf, so a flavor change (cpu -> cuda, cu126 -> cu130) still installs the correct new build. - The base must look like a release, so probe noise never becomes a pin. - UNSLOTH_TORCH_UPGRADE=1 opts out and restores the old always-newest behavior; the substep line advertises it. The supported range is kept in _PREV_FALLBACK_CONSTRAINT: if the exact release is not resolvable from the chosen index (custom mirrors prune old wheels), the install warns and falls back to the newest supported release instead of failing the whole run. The later flavor-mismatch repair reuses TORCH_CONSTRAINT, so a mid-install clobber is repaired back to the kept release rather than the newest one. Verified end to end: a venv seeded with torch 2.10.0+cu130 re-run through the full installer finishes with torch 2.10.0+cu130 (previously 2.11.0+cu130). Tests: tests/sh/test_previous_torch_pin.sh covers keep/flavor-change/rocm/ noise/opt-out plus wiring (probe ordering before venv replacement, fallback present, SKIP_TORCH gate). * install: constrain kept torch pins to the supported window Review caught that _previous_torch_pin pinned the previous venv's torch on flavor match alone, so a release outside the installer's active range (a 2.3.x manual install below the >=2.4 floor, or a 2.12.x manual upgrade above the ceiling) replaced the bounds computed just above it and a rerun kept a torch the installer otherwise deliberately excludes. New _torch_release_in_window checks the probed base against the active TORCH_CONSTRAINT ("torch>=A.B[,<C.D.F]") at major.minor granularity, which is exact for the windows this script uses (ceilings are always X.Y.0; a non-.0 ceiling would only make it conservative). Anything unparseable answers no, so probe noise or a malformed window fails toward the supported range instead of becoming a pin. _previous_torch_pin takes the active constraint as a third argument and refuses out-of-window releases; the in-window keep behavior is unchanged. Tests: out-of-window rows (2.3.x floor, 2.12.x ceiling, boundary keeps, cpu and macOS windows, malformed/empty windows) plus direct _torch_release_in_window coverage. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-19 06:19:29 -07:00
Michael Han	6d8c18cd1a	Replace standalone Studio wording with Unsloth (#7221) ... * Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error.	2026-07-19 00:47:04 -07:00
Vineeth Sai	a14b032d79	Propagate fp8 block_size before the early return in get_lora_parameters_bias (#7189) ... * Propagate fp8 block_size before the early return in get_lora_parameters_bias get_lora_parameters_bias set the fp8 block_size on W/W_quant only after the disable_adapters/merged early return, so on the merged or disabled path (merged inference, DPO reference model) a block-fp8 weight lost its real block_size and downstream fp8 kernels fell back to [128, 128]. The non-bias sibling get_lora_parameters already sets block_size before its early return; move the block so both behave the same. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard the fp8 block_size against a missing quant state A decompressed compressed-tensors layer keeps quant_method == "fp8" while its weight is back to bf16, so it has no quant state and get_lora_parameters_bias must still return W_quant None for fast_linear_forward to fall back to a plain matmul. Only attach block_size when a quant state was actually found. * Guard the sibling get_lora_parameters fp8 block_size against a missing quant state Mirror the get_lora_parameters_bias guard so a decompressed compressed-tensors layer (quant_method fp8, bf16 weight, no quant state) does not raise AttributeError on the fused-LoRA path. Add a CPU-only regression test. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-17 16:30:45 -07:00
Vineeth Sai	33119c9bf7	fix: guard remove_special_tokens against tokenizers without a BOS token (#7048)	2026-07-10 14:55:11 -03:00
alkinun	216a1fad33	Fix Windows installer torch index override (#6972) ... * Fix Windows installer torch index override * Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898) * Harden setup.ps1 index-var clearing to truly remove vars (#6898) * Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898) * Neutralize all uv index env vars for pinned torch installs (#6898) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-09 03:46:47 -07:00
Etherl	5e43c623b9	Fix FastSentenceTransformer Qwen embedding preprocessing (#6939) ... * Fix FastSentenceTransformer Qwen embedding preprocessing * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Document Transformer.load embedding modality fix for #6881 * Harden #6881 fix and add forwards/backwards-compatible regression tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fall back to Transformer constructor on legacy sentence-transformers without Hub-capable load * Mirror legacy sentence-transformers fallback in embedding-parity tripwire test * Tighten #6881 comments and docstrings * Skip embedding-parity test on CPU-only runners since FastSentenceTransformer requires CUDA * Honor the transformer module's saved subfolder when loading modules.json records a path for the Transformer module (root for decoder embedders like Qwen3-Embedding, 0_Transformer for the classic layout). Pooling/Normalize already load from their saved path; thread the same path into Transformer.load as subfolder so config and tokenizer resolve like stock ST. stays a no-op, so single-module models are unchanged. * Make embedding-parity test bf16-aware fp16 overflows to NaN on bf16-native embedders such as EmbeddingGemma (Gemma3), producing a false parity failure. Prefer bf16 when the GPU supports it so the tripwire can guard the full documented embedding matrix (Qwen3-Embedding, EmbeddingGemma, BGE-M3, all-MiniLM, GTE-ModernBERT), not just fp16-safe models. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-07-09 01:46:22 -07:00
Vineeth Sai	85a068cfe1	Fix to_sharegpt optional block rendering "None" for missing extra columns (#6827) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-07-08 17:57:41 -03:00
Thomas Eric 🇧🇷	03cbe211a3	Studio: fix flash-attn and torchao install on Blackwell (sm_100+) GPUs (Closes #6961) (#6970) ... * fix: Remove moot has_blackwell_gpu() function Fixes unslothai/unsloth#6961. This function skipped flash-attn on Blackwell GPUs because no prebuilt wheel existed; Dao-AILab now ships one and url_exists() already gates resolution. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: use torchao 0.17.0 for Blackwell Fixes #6961. Torchao 0.16.0's cpp extensions are built against CUDA 12, so on a CUDA-13 torch (cu130 / Blackwell) they fail to load with "libcudart.so.12: cannot open shared object file". Select 0.17.0 there instead: its cpp targets torch 2.11, so it is skipped cleanly rather than crashing. CUDA-12 / ROCm / CPU torch 2.10 keeps 0.16.0 and its working kernels. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Condense torchao version-selection comments (no behavior change) * Support torch 2.11 in the Studio installer via the torch2.10 prebuilt wheels Map torch 2.11 to the torch2.10 prebuilt wheels for flash-attn, causal-conv1d, and mamba through wheel_utils.prebuilt_wheel_torch_mm, applied in direct_wheel_url (filename) and flash_attn_wheel_url (version). Those torch2.10 CUDA wheels load and pass each project's own test suite on torch 2.11 (verified on B200), so a torch 2.11 environment gets the prebuilt accelerators instead of skipping or building from source. Raise _CUDA_TORCH_PKG_SPEC to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so the CUDA torch repair path can install torch 2.11, where torchao 0.17's cpp kernels load cleanly. Add tests for the mapping. * Keep has_blackwell_gpu as a False stub for future arch gating * Restore has_blackwell_gpu as a return-False probe kept for future arch gating Keep the nvidia-smi compute_cap detection and its two call sites, but short-circuit with return False at the top so flash-attn is no longer skipped on Blackwell (sm_100+ now has prebuilt wheels and url_exists gates resolution). Drop the early return to re-enable arch-based detection later. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-08 06:38:10 -07:00
Long Yixing	934f879043	feat(mlx): route trainer callbacks (#6929)	2026-07-08 03:25:50 -07:00
Long Yixing	d918245834	Add MLX-aware public Unsloth trainer API (#6462) ... * feat: add mlx public trainer api * test: cover mlx public trainer api * fix: preserve mlx epoch trainer configs * fix: pass mlx warmup ratio through config * fix: align mlx trainer dataset order * fix: keep mlx chat templates import-light * fix: infer mlx trainer context length * fix: mirror cuda mlx context defaults * fix: align mlx notebook trainer defaults * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: keep mlx public helpers import-light * refactor: reuse mlx optimizer normalization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: address mlx review feedback * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: tighten mlx training argument parity * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: align mlx trainer eos default * Fix MLX trainer to accept DataCollatorForSeq2Seq and handle TokenizerWrapper in get_chat_template * Trim redundant docstrings on internal MLX helpers * MLX review fixes: Studio optimizer import-safe on non-MLX hosts, preserve explicit max_length, skip MLX tests before import * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MLX review round 2: defer max_length to model context, optimizer alias fallback for older zoo, skip non-MLX test on missing GPU deps * MLX review round 3: keep chat_templates importable without torch on MLX * fix: preserve MLX trainer notebook shims * fix: ignore CUDA tokenizer moves on MLX * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: harden MLX trainer shims * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: unwrap MLX scheduler enum args * fix: coerce integral MLX epoch counts * fix: spoof CUDA compatibility APIs on MLX * fix: harden MLX notebook compatibility shims * MLX: add torch.cuda.mem_get_info to the compatibility shim Notebook memory cells call torch.cuda.mem_get_info()[0] directly (not gated by is_available), so on MLX it raises without a shim. Return (free, total) bytes from the MLX device stats, consistent with the other torch.cuda compat helpers, and add a matching assertion to the compat-API test. * MLX: use active memory for mem_get_info; fix BatchEncoding.to keyword device Address review on the MLX compatibility shim: - torch.cuda.mem_get_info() now derives free bytes from current active MLX memory instead of the peak high-water mark, so a capacity check stays accurate after a transient spike or a prior run. - BatchEncoding.to(device=...) passed by keyword no longer forwards a positional None alongside the keyword (which raised "multiple values for 'device'"), so non-CUDA keyword moves like .to(device="cpu") delegate correctly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MLX: accept preserve_dataset_order; stub RL trainers with a clear error Two fixes so unmigrated notebooks behave predictably on MLX (torch present): - preserve_dataset_order is a real MLXTrainingConfig field but was missing from the extra-argument allowlist, so passing it (as a config or trainer kwarg) could be rejected as unknown on a zoo without the field. Add it to _MLX_IMPLEMENTED_EXTRA_ARGUMENTS so the documented no-shuffle path is reachable. - GRPO/DPO/ORPO (and KTO/PPO/Reward) have no MLX trainer yet. Retarget the ones the installed trl exposes to a stub that raises a clear 'not supported on MLX' error instead of importing the real torch/CUDA trainer and crashing deep inside it. Only existing trainers are retargeted (no invented attributes), idempotent across re-imports. * MLX: make RL-trainer stubbing import-safe; back current-memory APIs with active memory Address review on the MLX shims: - The RL-trainer stub loop probed trl with getattr(_trl, name), which triggers trl's lazy trainer import and pulls torch -- that can crash import unsloth on a torch-free MLX install just to check existence. Decide what to stub from trl.__all__ + already-materialized attrs (vars) instead; never resolve the real trainer. All trl trainer names are in __all__, so they are still stubbed (even torch-free), and the probe no longer imports torch. - torch.cuda.memory_reserved / memory_allocated (the current, non-max APIs) were aliased to peak max_memory_reserved. Back them with current active MLX memory so cleanup / capacity checks see live usage; max_* keep the peak high-water mark. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MLX: keep TRL's SFTConfig epoch default under the trl.SFTConfig alias Unmigrated notebooks import SFTConfig from trl, which the MLX build aliases to the public training-args class. TRL/HF SFTConfig defaults to num_train_epochs=3 (max_steps=-1); the native MLX config defaults to max_steps=60. So an SFTConfig built without an explicit length silently ran 60 MLX steps instead of TRL's 3 epochs under the alias. Alias trl.SFTConfig to a thin subclass that seeds the TRL epoch default only when neither max_steps nor num_train_epochs is given; explicit lengths pass through untouched, and the native public args class keeps its MLX default. Epoch mode is supported by the MLX trainer. * MLX CI: keep the GGUF reload smoke under the job timeout The RELOAD-GGUF-via-llama-cli step timed out at 300s. BF16 GGUF decode is CPU-bound on the macOS runner (~10s+/token), so generating 24 tokens landed right on the 300s cliff and killed the process. This step is a save/reload integrity smoke (it only needs a few chars of output), so the token count is incidental: generate 8 tokens with explicit threads and a small headroom on the subprocess timeout, all env-tunable (UNSLOTH_GGUF_RELOAD_N / _THREADS / _TIMEOUT). Cuts the reload well under the 25 minute job budget. * MLX: broaden trainer stubs, real peak-memory reset, fix shim tests Address review on the MLX public API: - The SFTConfig identity tests asserted trl.SFTConfig is UnslothTrainingArguments, but the alias now points at the _MLXSFTConfig subclass that preserves TRL's epoch default, so the MLX suite failed before testing the shim. Assert issubclass instead. - torch.cuda.reset_peak_memory_stats was a no-op, so max_memory_reserved kept earlier model-load peaks across a scoped run. Wire it to mx.reset_peak_memory with the same core/metal fallback used for the reads. - The unsupported-trainer stubs were a fixed list, so trainers outside it (a newer RLOOTrainer) still routed to the real torch trainer. Derive the set from trl.__all__ (every non-SFT *Trainer) so all non-SFT surfaces fail with a clear MLX message; names come from __all__ so trl is never resolved. - The non-MLX export smoke skipped only on missing bitsandbytes/triton; other absent GPU deps (numpy/torch/unsloth-zoo, or _gpu_init re-raising ImportError) made it fail on CPU hosts. Skip on any ImportError. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: keep MLX notebook compatibility minimal * MLX CI: force CPU + small context for the GGUF reload smoke The RELOAD-GGUF-via-llama-cli step timed out even at 8 tokens (>420s), so it is a fixed hang, not per-token cost: on the paravirtual macOS runner GPU llama.cpp's Metal backend stalls, and the gemma3 GGUF advertises a 32768 context that llama-cli would otherwise fully allocate. Run llama-cli CPU-only (-ngl 0) with a small context (-c 256); keep generation short. All env-tunable (UNSLOTH_GGUF_RELOAD_NGL / _CTX / _N / _THREADS / _TIMEOUT). Also print llama.cpp's partial stdout/stderr on timeout so a future hang is diagnosable instead of an opaque TimeoutExpired. * MLX CI: export the reload-smoke GGUF as q8_0, not bf16 The GGUF reload via llama-cli timed out on the runner even CPU-only with a tiny context and 8 tokens. Root cause is the format, not the flags: the smoke exported quantization_method='not_quantized', which maps to a bf16 GGUF, and llama.cpp's bf16 CPU decode is unusably slow on the paravirtual macOS runner. Export q8_0 (fast_quantized, the exporter default and what users deploy) instead -- llama.cpp has optimized q8_0 CPU kernels, so the fresh-process reload loads and generates in seconds. The reload stays CPU-only (-ngl 0) with a small context. * test: clear TRL shim before availability check --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com>	2026-07-02 23:02:26 +01:00
Filip Trajkovic	c5adb69a10	Fix GRPO logit scaling when model is wrapped by DDP (#5955) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-07-02 00:01:15 -03:00
Daniel Han	8cc05ac89c	Reduce comments across recent fixes (#6776) ... Condense the verbose comments and docstrings added by the recent chat template, GPT-OSS detection, PEFT tensor-parallel, and Studio inference proxy fixes. Comments and whitespace only; no code changes.	2026-06-30 23:13:36 -07:00
Daniel Han	de3c745fab	Fix full finetuning precision on V100 / no-bf16 GPUs (#5880) ... --------- Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-29 18:35:23 -03:00
Vineeth Sai	20266a59eb	Fix custom chat templates with a {system_message} placeholder (dead code in _change_system_message) (#6735) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-29 00:36:39 -03:00
Daniel Han	4929c5f769	Keep pad-named pad_tokens (e.g. <|vision_pad|>); fix Qwen3-Base load crash (#6652) ... * Keep pad-named pad_tokens; defer pad repair to shared unsloth_zoo.pad_token A pad-named token (e.g. <|vision_pad|>) is a valid pad. The narrow fallback that stripped vision pad tokens on text-only models is now a no-op; the active path delegates to the shared fix_pad_token in unsloth_zoo, which keeps pad-named tokens and only heals missing / eos-collision / out-of-range pads. This fixes the Qwen3-4B-Base load crash (its config ships pad_token=<|vision_pad|>): the old swap could not find a safe text pad (eos is <|endoftext|>, no unk_token) and left the tokenizer broken. Removes the unused _VISION_PAD_TOKENS / _SAFE_TEXT_PAD_TOKENS sets. Tests updated. Pairs with unslothai/unsloth-zoo#831. * Remove _fix_vision_pad_token; inline the no-op fallback A pad-named token (e.g. <|vision_pad|>) is a valid pad, so the old vision-pad swap helper has no purpose. _fix_pad_token now returns the tokenizer unchanged when the shared unsloth_zoo.pad_token module is unavailable, instead of routing through a no-op helper. Test WANTED set updated.	2026-06-25 04:41:09 -07:00
Daniel Han	f436d204f6	Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503) (#6639) ... * Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503) On Apple Silicon, install.sh exports UV_OVERRIDE pointing at the bundled overrides-darwin-arm64.txt. uv splits UV_OVERRIDE on whitespace, so a repo cloned under a path containing a space (e.g. /Users/me/Open Source/unsloth) truncates the value and every later uv call aborts with 'error: File not found: <truncated>' (the PyTorch install step in #6503). Copy the overrides file into a space-free temp dir and point uv at the copy when the path contains a space, mirroring the macOS/Linux handling already merged for the Python installer in #6534. The temp dir is removed in the exit trap, and the code falls back to the original path when no space-free temp dir is available, so the no-space and non-macOS paths are unchanged. Adds tests/sh/test_install_uv_override_space.sh, which extracts and runs the install.sh hardening block and checks the spaced, no-space, and spaced-TMPDIR fallback cases. * Installer: match all whitespace (not just spaces) in UV_OVERRIDE handling uv splits UV_OVERRIDE on any whitespace, so use the POSIX class *[[:space:]]* rather than a literal space in install.sh (catches tabs and newlines in the path too) and the matching test assertions. Use the portable awk bracket expression [$] instead of \$ in the extraction so the test runs the same under BSD awk (macOS) and GNU awk (Linux). Adds a tab-in-path case. * Installer: clear _UV_OVERRIDE_TMPDIR before the exit trap The exit trap rm -rf's _UV_OVERRIDE_TMPDIR. Initialize it to empty before registering the trap so an inherited environment value can never be removed; only a temp dir this script creates (Apple Silicon, spaced path) is cleaned. Adds a structural test asserting the init precedes the trap. * Run the install.sh UV_OVERRIDE space test in CI via a pytest wrapper The Shell installer tests job uses a fixed script list (not tests/run_all.sh), so the new shell test would not run on PRs. Add a pytest wrapper under tests/python/ that invokes it; the auto-discovered repo CPU test job collects tests/python/ and so executes the Apple Silicon spaced-path regression. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-24 17:34:18 -07:00
Saicharan Ramineni	9d53656614	Make _uv_safe_path space-safe on macOS/Linux (#6503) (#6534) ... * Copy uv `-c`/`-r` paths to a space-free temp dir on macOS/Linux uv 0.11.x truncates a constraints/requirements path passed via `-c`/`-r` at the first space, so `unsloth studio` setup from a repo cloned under a path containing a space (e.g. `/Users/me/Open Source/unsloth`) fails with: error: File not found: `/Users/me/Open` _uv_safe_path() already worked around this on Windows via the 8.3 short path but returned the space-containing path unchanged on macOS/Linux, which have no 8.3 equivalent. Extend it to copy the (small, flat) constraints/requirements file into a space-free temp dir and hand uv the copy; the temp dirs are removed at process exit. Falls back to the original path on any error, so it is never worse than before. Refs unslothai/unsloth#6503 * Route UV_OVERRIDE through _uv_safe_path and fix temp-dir leak (#6503) The -c/-r fix did not cover UV_OVERRIDE, which uv also truncates at the first space. On Apple Silicon the overrides file is handed to uv via UV_OVERRIDE at install time (install_python_stack.py) and during the MLX self-heal (utils.mlx_repair), so a repo under a path containing a space still broke every uv call there. Move _uv_safe_path into backend.utils.uv_path_safety so both sites share it, and route UV_OVERRIDE through it. Also stop leaking the temp dir when shutil.copyfile fails after mkdtemp, and add tests for the UV_OVERRIDE channel, the TMPDIR-with-space fallback, the atexit cleanup, and the no-leak path. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-06-24 04:02:24 -07:00