14 commits

Author	SHA1	Message	Date
Daniel Han	b35bdcbea3	CI: prove an interrupted install can never masquerade as a healthy one (#7552) ... * CI: prove an interrupted install can never masquerade as a healthy one Nothing in CI had ever interrupted an install, which is how the reported failure shipped: quit the desktop app mid-install, the app SIGTERMs the installer process group, and if that lands during 'studio deps' the venv loses structlog. Preflight then probes 'unsloth -h' and 'studio desktop-capabilities', both of which succeed because the CLI's own deps are core, so the app reported ManagedReady with can_auto_repair=false while the backend died on import. A permanent dead end. This kills the installer at each interesting phase and asserts the result is either genuinely healthy or explicitly repairable, never silently ready. 13 legs across macos-14, ubuntu-latest and windows: each of the dependency-pass steps plus the coarse phases (venv, torch, unsloth, setup). The kill targets the process GROUP, matching install.rs. Killing only the leader leaves uv and python children to finish the dependency pass, and the test would quietly prove nothing. Windows has no process groups, so that leg walks the CIM parent links instead, which is the same reason the app carries windows_job.rs. One shared probe for all platforms. The Windows check used to be bespoke inline PowerShell that only ran -h and desktop-capabilities, so it could not observe studio_install_ok, verify-install or desktop-runtime-check: it would have reported FALSE_READY for the very PRs that add them, no matter how well they worked. The probe boots the backend as ground truth and owns the whole process tree, since terminating only the parent leaves children holding the port. install.sh runs with --local, which is load-bearing rather than a convenience: without it the installer resolves unsloth from PyPI and the venv gets the PUBLISHED CLI, so no branch-side change is present and every deeper probe reports 'absent' regardless of what the branch does. Verified: against a tree without the detection, windows kill@studio-deps reports FALSE_READY, reproducing the user report exactly. With #7492 merged the same leg reports REPAIRABLE, and all 13 legs pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the POSIX legs actually run the installer, and fail if they do not install.sh --tauri rejects a custom UNSLOTH_STUDIO_HOME outright (the desktop app still uses the legacy ~/.unsloth/studio root), and this workflow set one at workflow level for every job. So all 11 macOS and Linux legs exited about a second in with ERROR: UNSLOTH_STUDIO_HOME is not supported with --tauri. produced no CLI, took the probe's NO_CLI 'safe' branch and reported success. They were vacuously green. Only the two Windows legs were real, because install.ps1 has no equivalent guard. The override now applies to the Windows job only, and the POSIX legs read the legacy root, which is where --tauri installs. The runner is ephemeral so the real home is as disposable as the override. Also adds the check that makes this class of mistake loud: a leg asserts its kill actually landed on the marker it was aimed at, using the interrupt_reason the driver already records. A leg that never reached its kill point proves nothing, and NO_CLI made that indistinguishable from a pass. * Make the interrupted-install legs able to fail The probe treated a present .desktop-install-in-progress marker as proof of a repairable state, but the drivers seed it unconditionally and never clear it, so REPAIRABLE was unconditional and FALSE_READY unreachable. The Windows leg had no kill-landed guard, blanket continue-on-error, and no repair re-run; -SkipTorch was silently dropped, since install.ps1 parses only --no-torch. Judge the re-run by whether the backend boots, on both platforms. The log grep matched the frontend build printing "up to date" and failed a leg whose venv was fine. * Drop the interrupt cell that could never be interrupted install.sh --local sets skip_base, so install_python_stack returns before any "base packages" label is printed. The kill had nothing to land on and the installer ran to completion, reaching [TAURI:DONE] in 62s. * Fail the leg when the installer finished instead of being killed The driver set reason=marker-hit before the post-marker sleep and never rechecked, so a step whose work was already cached could run to completion inside that beat and still be recorded as an interruption. The landing assertion tests reason != marker-hit, so a fully completed install passed green having interrupted nothing. Reproduced with a stub that exits during the delay: reported marker-hit / killed=true / exit=0 next to "install finished fully". Set the reason after the sleep, on both drivers. Also trigger on _studio_deps.py and install_manifest.py, where the two decisions the probe asserts on are actually implemented. * Kill the group, and stop the probe blocking on a full pipe The escalation was gated on the leader still being alive, so a leader that exits promptly on SIGTERM while a uv or python descendant ignores it skipped the SIGKILL entirely, and wait reaped only the leader. Proven with a descendant that traps TERM: pre-fix its heartbeat keeps ticking while the probe would be running, post-fix it stops. Signal the group unconditionally and drain it after the reap, since an unreaped leader is still a member of its own group. The probe started the backend on stdout=PIPE and read nothing until after the poll loop, so a backend logging more than the pipe buffer during import blocked before binding. Measured 65536 bytes here; a child emitting 200 KB never reaches its bind line, which would make backend_ok false for a healthy install. Write straight to the artefact file. Also trigger on studio/backend/requirements/**, where structlog is declared. * Make the NO_CLI legs assert repair, and fix the Windows straggler sweep Two of the interrupted-install legs were passing without testing anything. The re-run assertion skipped verdict=NO_CLI, but a kill at "venv" or "torch" lands before install.sh ever prints "Installing Unsloth" (:2125, :3667, :3961), so those legs can only ever produce NO_CLI. Three non-gating-exempt cells (macos-14 kill@venv, macos-14 kill@torch, ubuntu-latest kill@torch) therefore asserted nothing beyond a marker appearing in a log. NO_CLI is now included: a re-run must produce a booting backend regardless of how little the first run managed to install. Each re-run step grows an existence check first, because the probe exits without writing verdict.json when the binary is absent and the json.load would crash rather than report. The Windows straggler sweep matched nothing at all. UNSLOTH_STUDIO_HOME arrives as D:\a\r\r/.studio-home, since the workflow joins ${{ github.workspace }} with a forward slash, while Process.Path is all backslashes, so the literal -like missed even the venv's own python.exe. uv is never under the studio home in any case: install.ps1 takes it from winget or astral.sh. Normalise the separators, match uv by name (the runner is ephemeral and runs no other uv), and skip the home comparison entirely when the variable is empty, which would otherwise turn the pattern into "**" and kill every python on the runner. * Run the Windows legs as the desktop does, and judge repair by what preflight reads The Windows matrix set a workspace-scoped UNSLOTH_STUDIO_HOME, which forces install.ps1 down the shell-install path: install.ps1:189-215 rejects a custom root under --tauri, so those legs ran with UNSLOTH_TAURI_MODE=0, the frontend build on and no bundled-file overlay, while the desktop always spawns the installer as --tauri with the variable scrubbed (install.rs:202 and :356). The torch leg could not even reach its marker: "Installing PyTorch" is printed only by Write-TauriLog (install.ps1:2440), so it was killed at the deadline. Both legs now run --tauri --local at the default root, and the probe and re-run resolve the CLI under %USERPROFILE%\.unsloth\studio. The probe counted `studio verify-install` and `studio desktop-runtime-check` failures as proof the app can repair, but preflight/managed.rs runs only `-h` and `studio desktop-capabilities --json` (:357) and reads studio_install_ok from that payload (:445); neither deeper command is invoked anywhere under studio/src-tauri. A leg where capabilities regressed to ready while only those standalone commands saw the damage would have passed green with the app stuck on ManagedReady, which is the exact false negative this workflow exists to catch. They are still run and recorded in verdict.json, just no longer repair evidence. An interrupted install can leave the console script in place while its venv interpreter is gone. The probes go through run(), which catches OSError, but the backend spawn did not, so the probe aborted before writing verdict.json and both workflows died on the json.load instead of reporting. That state is now recorded as backend_spawn_error and lands on REPAIRABLE, which is what `-h` failing already implies. On win32 the CLI re-spawns the server as a child and waits on it (unsloth_cli/commands/studio.py:1543), and CREATE_NEW_PROCESS_GROUP does not make terminate() reach descendants, so the reap left a server holding the venv open while the repair step reinstalled into files Windows had locked. Use taskkill /F /T for the tree. The straggler sweep now falls back to the default studio root, since under --tauri there is no UNSLOTH_STUDIO_HOME to match on. * Judge the install the way preflight does, and reap the whole probe group Read desktop-capabilities the way the desktop reads it. preflight/managed.rs pipes stdout and sends stderr to /dev/null (managed.rs:358), then hands the whole stdout buffer to serde_json (managed.rs:414). The probe concatenated both streams and scanned to the first brace, so a single diagnostic line on stderr made json.loads raise on the trailing text, studio_install_ok stayed "absent", and a broken backend was reported FALSE_READY over an install the real app parses, sees as incomplete, and offers to repair. That fails a valid recovery change for a reason that exists only in the probe. stdout and stderr are now captured separately and stdout is parsed strictly; a payload that does not parse counts as repair evidence, matching the Stale the desktop reports when the capability probe returns nothing (managed.rs:521). A booting backend alone is not a finished install. The manifest is written last (install_python_stack.py:3255), so a kill after "studio deps" but before it, the data-designer leg, leaves a venv whose backend boots while desktop-capabilities still reports studio_install_ok=false and preflight reports Stale (managed.rs:445). Calling that HEALTHY skipped the re-run step, so the leg asserted nothing beyond a marker appearing and never exercised the version fast path that is supposed to clear an incomplete install, which is the half of the bug that strands the user. HEALTHY now requires both. Escalate to the process group after reaping the probe's backend. reap() returned as soon as proc.wait() succeeded, and the leader exits promptly on SIGTERM while a uvicorn worker does not, so the SIGKILL iteration was skipped and that worker kept the port and the venv open while the repair step reinstalled underneath it. It also read os.getpgid(proc.pid) after the reap, which raises. The pgid is now captured up front and SIGKILL always goes to the group, the same escalation interrupt-install.sh:94 makes. A heartbeat experiment left the group alive with the old sequence and empty with the new one. Trigger the workflow on pyproject.toml. Every leg installs the checkout with --local, so that file decides the unsloth console script and the core dependencies the probe leans on: -h and desktop-capabilities only survive a torn install because typer/click/rich are declared there. No other install workflow interrupts the installer, so such a change would otherwise merge without a single leg running. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Judge an absent capability field and a dead -h the way preflight does The probe left studio_install_ok=absent undecided and judged those installs on whether the backend booted. preflight/managed.rs:445 tests studio_install_ok != Some(true), so an absent field is Stale exactly like a false one; a CLI too old to carry it is already rejected one check earlier on desktop_manageability_version. The gap mattered in both directions: a payload that stopped carrying the field reported HEALTHY on every booting leg and skipped the re-run assertion this workflow exists to make, and a torn venv with a working -h was failed as FALSE_READY even though the app would have offered repair. unsloth_cli/commands/studio.py is in this workflow's path filter precisely to catch that class of change, so it must not be the thing that silences it. The verdict also consulted cli_h_ok only in the repairable arm, so a CLI that cannot print help was called HEALTHY whenever the backend happened to boot. probe_managed_bin runs -h first and returns Stale cli_unusable before it ever reaches the capability probe (managed.rs:465-478), so that install goes to repair in the real app and the leg must assert it here. * Judge the probes on the desktop's deadline, and interrupt the host it uses Preflight gives each managed probe ten seconds and nothing more: managed.rs:337 wraps `unsloth -h` and managed.rs:390 wraps `studio desktop-capabilities --json` in a tokio timeout, kills the child on expiry, and returns Stale as "cli_unusable" or "desktop_capability_probe_failed". The probe allowed three minutes, so a venv torn badly enough that its CLI only answers after half a minute of retries was recorded HEALTHY here while the real app shows it as repairable. That skips the re-run assertion the leg exists to make, which is the same false-HEALTHY hole the studio_install_ok and -h gating already closed. Both calls now use the desktop's ten seconds, and the elapsed time is recorded so a leg that flips for timing reasons says so in the artefact. On Windows the installer child now runs where the desktop runs it. install.rs 325-339 spawns the bundled install.ps1 as powershell.exe with -NoLogo -NoProfile -NonInteractive -WindowStyle Hidden -ExecutionPolicy Bypass -File, so Windows PowerShell 5.1 is the only host a real desktop install ever uses. The interrupted run and the repair re-run both used pwsh 7, and every other Windows job in .github runs install.ps1 under pwsh too, so the installer's behaviour on 5.1 was covered by nothing: .NET Framework instead of .NET, OEM console encoding instead of UTF-8, and different native-command and OSArchitecture reporting are all real sources of divergence. A workflow whose point is to reproduce what the app does cannot run a different interpreter than the app does. The driver itself stays under pwsh; only the installer child and the repair invocation change. * Tighten the interrupted-install comments * Tighten the probe docstrings * Fail the leg when the installer completed inside the kill window * Land the kill in the marked step, and reject non-boolean capabilities Two holes found from the staging run's own logs. The venv leg never interrupted the venv step. Creating the venv takes ~0.1s, so by the time the 1s poll noticed its line the installer was already in "Installing PyTorch", and the flat 3s beat sent the signal there: staging run 30419729244 shows both step lines in the tail and a kill 4s in. That made the leg a duplicate of the torch leg while its label claimed otherwise. Both drivers now poll in half-second slices, cut the beat short the moment a later [TAURI:STEP] line appears, and print the step the signal actually landed in, warning when it is not the marked one. Sub-step markers such as "studio deps" print no step line, so they keep the whole beat and never warn. studio_install_ok is Option<bool> (managed.rs:43), so serde rejects a non-boolean and the whole payload fails to deserialize, which the desktop reports as Stale. bool() read a JSON string "false" as True, so the probe called a torn install ready. Only a literal JSON true counts now. * Tighten the interrupt driver comments * Fail the leg when the signal landed after the marked step The cut-short added last round only helps when the marked step is still the last [TAURI:STEP] line at the moment the poll notices it. Creating the venv takes ~0.1s (staging run 30419729244: 03:31:07.371 -> 07.478), less than the 0.5s poll, so the next step's line is usually already in the log when the marker matches, the step count never changes during the beat, and the full 3s elapses inside "Installing PyTorch". Reproduced with a stub installer against the driver at head: kill at 4.11s, step at kill "Installing PyTorch". The leg then duplicates the torch leg while its matrix label claims the venv step, and passed green on nothing but a :⚠️:. Both drivers now skip the beat entirely when the marked step is already over, so the kill goes out at once instead of deeper into the next step, and both record interrupt_step_mismatch in interrupt.env. The landing assertion fails on it: a warning that cannot fail the leg proves nothing. Sub-step markers ("studio deps", "pip bootstrap") print no step line of their own and stay exempt, as before. The venv leg becomes experimental. Its step is shorter than any log poll can resolve, so it must not block the PR on a race it cannot win, and it still probes the earliest torn state whenever it does land. * Tighten the interrupted-install workflow and driver comments * Land the kill in the phase each leg is named for Splitting the log on \r shows that 5 of the 12 legs of staging run 30419729244 interrupted a later phase than their label claims, and the run was fully green. The venv leg's install.log is byte-identical to the torch leg's. So is pip-bootstrap's to unsloth-extras'. Worse, both "studio deps" legs, the cells that reproduce the reported bug, were killed at "7/10 data designer deps" and "12/14 local plugin": their own probe artefacts report backend_ok=true, so structlog was installed and the flagship cell was passing on the manifest gate alone. Two causes. The dependency pass rewrites ONE physical line with \r (install_python_stack.py:2499), so its sub-steps are CR-separated segments and a line-based check could not see one end; the drivers exempted them and warned about nothing. And the flat 3s beat between the marker and the signal is longer than several phases, while every phase label prints BEFORE its work starts, so the beat pushed the signal past the phase instead of into it. Both drivers now split on \r, track the running phase at both levels, and judge a sub-step marker against the running sub-step and a step marker against the running step, so a step is not "over" because the sub-steps beneath it advanced. The beat defaults to 0 and is set per leg, 3s only for torch, unsloth and setup, which run for minutes. The mismatch is recorded in interrupt.env and the landing assertion fails on it, in both languages. Both detectors were replayed against the 12 real logs from 30419729244 and agree with the artefacts on every leg. venv and pip-bootstrap become experimental: their phases are shorter than any log poll can resolve. * Fail the Windows leg when the installer never exited The driver writes installer_exit=running when the installer outlived Stop-Tree and WaitForExit, and 'running' is not '0', so the landing assertion accepted it. A live installer writing into the venv while the probe reads it is not an interrupted install. Only a real integer exit code counts now, checked against 0, 143, 137, -1, running and the empty string. * Drop the legs that cannot land, and prove the kill was delivered Two cells never interrupted the phase they were named for. "Creating virtual environment" runs 0.107s (staging 30419729244, 03:31:07.371 -> 07.478) and "1/10 pip bootstrap" is over just as fast, both shorter than any poll that watches the log, so in 30423181897 and 30424366953 the signal landed in "Installing PyTorch" and "2/10 unsloth extras" every time. Each was another leg wearing a false label, so they are gone rather than allowed to fail, and continue-on-error goes with them: a leg permitted to fail asserts nothing. The only coverage lost is a venv caught half-written, which interruption cannot reach at this resolution; the torch leg's signal lands ~3s into a multi-minute download, so it already leaves a complete venv with nothing installed into it. The landing check also accepted an installer that failed on its own. A dependency error between the driver's last liveness check and the signal exits non-zero, which the exit != 0 guard let through as a kill. POSIX now requires 143 or 137, the only statuses a signal produces here and what all ten POSIX legs of 30424366953 reported. Recording whether kill(2) returned 0 would not separate them, since the unreaped leader keeps its group alive. Windows has no such status, so the driver records whether Stop-Process actually terminated the installer: it throws on a process already gone, so the flag is false exactly when there was nothing left to interrupt. * Signal at the marker, with no beat to overshoot the phase Staging run 30426111484 failed the macOS torch leg on the landing check: the 3s beat carried the signal from "Installing PyTorch" into "Installing Unsloth", because the PyTorch step, which this workflow called minutes long, finished in under three seconds. The beat only ever existed to land mid-work, and it cannot do that safely: every label prints before its work starts, so detection is already inside the phase, and any wait is a bet on how long that phase runs. It lost in 30419729244 and again here. So the beat is gone rather than retuned, and with it the matrix knob and the driver parameter on both platforms. The landing check stays and can still fail, since a phase shorter than one poll is seen only after it ends. The Windows driver also polled every 500ms while its own comment claimed a fifth of a second. That is 2.5 slices of overshoot the POSIX side does not carry, and it is now 200ms like the POSIX loop. * Kill the installer before its children, not after The depth-first walk killed the child install.ps1 was waiting on and only then the root, which races the leader's own reaction to that death. It is not a theoretical race: in staging run 30424366953 install.ps1 had already printed "unsloth studio setup failed (exit code -1)" by the time Stop-Process reached it. A leader that wins the race makes Stop-Process throw, and the new root-kill assertion would then fail a leg whose interruption the driver really did deliver. The tree is now snapshotted first, since a dead parent leaves nothing to walk, then the root goes down ahead of its descendants. A dead leader cannot react to a child and cannot respawn one either, which is what the depth-first order was for. * CI: give the probe the desktop's startup grace and fail a nonzero repair The probe allowed the backend 120s to answer /api/health while the desktop waits 5 minutes (BACKEND_STARTUP_GRACE_PERIOD, commands.rs:9), so a slow but healthy install could be reported FALSE_READY. A broken backend exits at once and the poll breaks on it, so the longer deadline only bounds a live backend. The re-run step also accepted a HEALTHY probe over an installer that exited nonzero. setup.sh does fallible sidecar and GPU setup after the manifest is written, and the desktop returns the repair error without starting the backend (commands.rs:615-630). The Windows leg ignored powershell.exe's status entirely. * Tighten the interrupted-install comments Comments only: shorter wording for the same rationale, no code touched. * CI: raise the interrupted-install job timeout above its own deadlines A leg configures up to 25 minutes to the marker plus two probe passes of up to 17 minutes each around a repair install, so the 60 minute limit could cancel a slow runner mid-assertion. Legs land in 6 to 10 minutes in practice. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-29 06:27:33 -07:00
Daniel Han	b6781a8bfe	CI: prove the installer works on a machine with no developer toolchain (#7551) ... * CI: prove the installer works on a machine with no developer toolchain No job has ever run the installer on a machine without one. studio-mac-install-matrix.yml is the only macOS installer job and it runs 'bash install.sh --local --no-torch' on runners that already have the Xcode CLT selected and setup-python preinstalled, so the CLT gate never fires there, and --local is precisely the mode that legitimately needs git. Repo-wide there was zero coverage of xcode-select or CommandLineTools outside install.sh itself. clean-machine-install-ci.yml runs the installer on a genuinely stripped machine. macOS legs move /var/db/xcode_select_link, /Library/Developer/CommandLineTools, /Applications/Xcode*.app and Homebrew aside, so xcode-select -p, git, cc and clang really do fail, and restore unconditionally afterwards. Removing the select-link alone is not enough: xcode-select falls through to a full Xcode.app and re-arms /usr/bin/git. Linux legs use containers, which are genuinely clean. Windows legs cover winget visible and masked, plus windows-11-arm. A WSL leg covers the 126 lines of WSL-specific install.sh logic that had no runtime test. Each macOS leg runs four deliveries: pipe (the advertised command, and the shape that turns an early exit into curl (56)), file (separates installer logic from pipe delivery), no-torch, and tauri (stdin closed, no tty, as the desktop app invokes it). One leg records every toolchain invocation and asserts the trace, which is the real deliverable: proof the installer never reached for a compiler rather than proof it happened to succeed. The asserts test that tools do NOT WORK rather than that they are absent from PATH. On a real virgin Mac /usr/bin/git and /usr/bin/cc exist as CLT stubs, so 'command -v git' succeeds and only running it tells the truth. desktop-app-clean-machine-ci.yml installs and launches the SHIPPED desktop app release on a stripped machine, covering Gatekeeper and quarantine on macOS, NSIS silent install on Windows, and Xvfb with WebKit2GTK on Linux. Known limit, stated plainly: hosted macOS runners are developer machines. Masking reproduces this bug and proves the installer does not invoke a toolchain, but it cannot prove no hidden dependency exists on a truly virgin Mac. An ephemeral-VM lane is the follow-up. * Point the llama assert at the right root, and name the Intel limitation The tauri leg installs to the legacy root because --tauri refuses a custom UNSLOTH_STUDIO_HOME. Its install succeeds end to end, but llama.cpp lives at <root>/llama.cpp while the venv is at <root>/studio, so the assert was pointed one level too deep. On macos-15-intel /usr/bin/git keeps working once the CLT are gone, so it is not CLT-provided there and no masking can remove it, while cc and clang do become stubs. Calling that 'masking failed' was wrong. That leg allowlists git explicitly and says why, so the assert stays strict everywhere else. * Make the clean-machine legs able to fail The toolchain strip never ran on the automatic triggers: inputs exists only for workflow_dispatch, and GitHub coerces '' and false alike to 0, so `inputs.strip_toolchain != false` was false. Confirmed on a pull_request run where the strip step reports skipped. Gate on the event instead. Also: scrub the Machine and User registry PATH, since install.ps1 rebuilds $env:Path from them mid-install and the toolchain came back; stop dropping WindowsApps unconditionally, which removed winget on the winget=visible leg too; fail rather than annotate when a bundle ships no installer or no CLI; run the bundled installer, which a headless launch never reaches; resolve the newest desktop-v* release instead of a pinned immutable tag; and give the two macOS matrix rows distinct artifact names. * Make the Windows and Linux clean-machine legs honest The Windows scrub only touched PATH, so the legs were green while not clean: run 30365014702 logged "python ABSENT" and then "Python 3.13 already installed" with uv resolving C:\hostedtoolcache\windows\Python\3.13.14\arm64\python.exe. py.exe lives in C:\Windows and uv discovers interpreters itself, so take the toolcache off disk and fail when tooling survives, instead of only printing it. The Linux desktop legs never stripped anything, and the tauri.log step was all || true so it could not fail. Run the bundled installer the way install.rs does, with --tauri alone, and assert torch: passing --no-torch skipped the slowest half of first launch and let the venv check pass over it. Pin the WSL rootfs to a dated build; current/ is a rolling alias and the digest next to it is fixed. * Give the Linux and WSL legs an assertion that can fail The Linux rows' only post-install gate was nobuild, a log grep, so an installer exiting 0 having produced nothing kept a required leg green. The WSL job and the Windows job both already check the install runs; the Linux job now does too. The WSL detection half only printed its Select-String, and the alternation also matches "platform linux", so a regression that skipped every WSL-specific branch would still pass as a plain-Linux install. Assert the exact marker, stripping ANSI first since step writes the label in reverse video. Probed against three fixtures: real wsl log passes, platform linux fails, missing log fails. * Tighten the clean-machine comments Compress the comment blocks across the clean-machine workflows and scripts. The explanations of why each check is written the way it is stay; the padding, restatement and duplication go. No code or workflow logic changes. * Point the nightly at the repo that publishes, and let its checks fail REL_REPO defaulted to unsloth-test/unsloth-test, which holds one release frozen at 2026-07-27, while release-desktop.yml publishes into github.repository. The schedule was re-testing the same fixture forever and could never see a broken production bundle. The windows job carried a blanket continue-on-error, so its NSIS assertions could not gate. lipo -archs prints and exits 0 for a thin binary and `|| true` swallowed even that, so the architecture was never checked; fall back to file, which survives the CLT mask. And require the preflight disposition line rather than the mere existence of tauri.log, which setup_logging creates at process start regardless. * Stop four clean-machine checks from passing over a real failure Re-run `absent` after the install on the masked macOS legs. It only ran before, so an installer that quietly selected the Xcode CLT or installed a compiler left the leg green while every later source build could succeed, which is the one thing clean-machine-assert.sh says `absent` guards the whole run against. Fail the Windows simulation when py.exe can still start an interpreter. The launcher binary itself may stay, but Find-CompatiblePython probes `py` first (install.ps1:1130-1153), so an interpreter registered outside the two renamed toolcache directories gets reused and Python bootstrap is never exercised. Exempting `py` without ever running it left that unchecked. Propagate the WSL installer exit code. It was printed and discarded, and the CLI check does not compensate: install.sh links the `unsloth` shim (4174-4182) before it reports a failing studio/setup.sh (4219-4230), so a late setup failure leaves a shim whose --version succeeds. Run the bundled installer in the Linux desktop jobs. The launch step only proves the process stayed alive, and on a fresh home preflight reports not_installed and the app waits on the install screen, so both required rows passed after 90 seconds without ever touching the shipped install.sh. Locate the resource in the deb payload or the extracted AppImage, run it the way install.rs does, and require a managed venv that can import torch. * Prove the trace wrapper records before trusting an empty trace The `notools` check reads an absence: it passes when the trace file contains no compiler, git or brew invocation. A shim directory that never reached PATH produces exactly the same empty file as an installer that touched nothing, so the single leg carrying that assertion would stay green no matter what the installer did. "Verify the simulation actually took effect" only ran for mask mode, which left the trace leg with nothing checking its own instrumentation. Call git explicitly after sourcing the environment and require it to appear in the trace, then truncate the file so the self-test entry does not count against the install. The call has to be explicit because macOS reaches _has_working_git only under STUDIO_LOCAL_INSTALL (install.sh:2026), so no consumer leg on that platform probes git on its own. * Stop the Windows clean-machine check failing on its own probe exit code All three Windows legs failed "Verify the simulation took effect" with no ::error:: printed at all. The check itself was right: the mask step logged "masked toolcache python: C:\hostedtoolcache\windows\Python", python/git/cmake/cl were ABSENT, no `py -3.x` probe started an interpreter, and the winget assertions were satisfied. The step still exited 1. The cause is $LASTEXITCODE leaking out of the step. The last external command is the `py -3.13` probe, which is SUPPOSED to fail; Get-Command and Write-Host are cmdlets and never reset $LASTEXITCODE, and the runner appends `if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }` to every pwsh step (actions/runner#351). So a clean machine reported failure, and because this step runs before Install, no Windows leg has ever reached the installer. Clear $LASTEXITCODE after the probe loop and end with an explicit exit 0. The leak detection is untouched: a surviving python/git/cmake/cl, or a `py -3.x` that actually starts, still exits 1. Also print each probe's exit code and output, so the next failure here explains itself instead of being silent, and label `py -0p` as what it is. The launcher reads the registry, which the on-disk toolcache rename cannot rewrite, so -0p keeps naming paths that no longer exist. Unlabelled it reads like a leak. Accept the Fedora leg's real outcome instead of a message that can be absent The fedora assertion only accepted the unsupported-package-manager hard exit. That is still what this ref's install.sh does, but the pending installer change replaces it with a warning that lets the install continue, at which point the old grep matches nothing and the step fails for the wrong reason. Handle both, strictly. If the log shows the newer "using prebuilt llama.cpp (missing:" warning, the Linux gate demonstrably did not hard-stop, and the only tolerated failure past that point is release lag: install.sh comes from this ref while unsloth comes from PyPI, and the released studio/install_python_stack.py has no "skip triton kernels when git is missing" guard, so it still fetches the git+https triton_kernels requirement on a machine with no git. Anything else after that warning fails the step. Otherwise the old hard-exit message is still required. A missing log, a bootstrap outage or any unrecognised failure all remain errors, and the step retires to a plain success assertion once a release ships the no-git skip. * Make the AppImage Linux row actually extract, and hold Linux to the macOS preflight bar The appimage row invoked the extractor by bare filename, and a command word with no slash is resolved through PATH rather than the working directory, so the extraction exited 127 and the bundled-installer assertion below it never ran. Prefix it with ./ so the row exercises what it claims to. The Linux log step also asserted nothing: it skipped a missing log with continue and discarded the grep with || true. The launch step only proves the process stayed alive for 90 seconds, and the bundled-installer checks do not exercise the Rust preflight path, so an app that hung before preflight completed passed both required Linux rows. Require the same desktop_preflight completed disposition= record the macOS rows already do. * Put the branch's own Python under test on the clean-machine legs install.sh and install.ps1 come from the ref under test, but they install unsloth from PyPI, which is the consumer path and has to stay that way. That left everything Python-side coming out of the released wheel: studio/setup.sh, studio/setup.ps1, studio/install_python_stack.py, and every requirements and constraints file those resolve through Path(__file__). A branch that changes constraints.txt or setup.ps1 therefore got a green run that proved nothing about the change, and some legs proved less than they looked. The Fedora assertion was already carrying a hand-written workaround for exactly this, tolerating a triton/git failure on the grounds that the released package lags the ref. Legs marked overlay: true now re-point the venv at the ref just before studio setup runs, through UNSLOTH_CI_SOURCE_OVERLAY: a --no-deps editable install of the checkout. That makes import studio resolve to the working tree, so the existing setup-script lookup finds the ref's setup.sh / setup.ps1 and install_python_stack reads the ref's constraints, with no other change to either installer. Not --local: --local additionally installs unsloth-zoo from a git+https URL, which genuinely needs git, and git absence is the whole point of the masked legs. The overlay resolves no dependencies and clones nothing, so it holds up with git, cmake and the compilers all gone. It is not a consumer knob either: no flag, no usage entry, ignored unless the variable names a directory with a pyproject.toml in it. Four legs stay on the released package deliberately, each for its own reason, recorded in the header: the mac pipe legs keep an end-to-end signal on what a user actually runs; the trace leg would otherwise answer its own question, since the editable build calls git through setuptools-scm's file finder; the non-root Linux leg dies before a venv exists; and WSL only ever receives install.sh, not a source tree. Two supporting fixes the overlay depends on or exposes: install_python_stack.py discarded uv's output whenever a step succeeded, so the nobuild assertion, which reads the install log, could not see a source build in the dependency phase at all. That is the phase that installs studio.txt, where an sdist-only dependency actually turns up, and it reported "built: none" regardless. It now echoes successful output under UNSLOTH_VERBOSE, matching what install.sh's run_install_cmd already does. nobuild now ignores "Building <name> @ file://" lines. A local-path build is something the caller pointed at, never a dependency resolution chose, and index dependencies always print <name>==<version>, so a real sdist from PyPI is still caught, including one named unsloth. Each overlaid leg also asserts it really was overlaid, so an unset variable cannot quietly put the whole matrix back on the released wheel. * Allowlist the triton-kernels pure-Python sdist, and record why Windows on ARM is red The two ubuntu2404 root legs went red at "Assert no source build" reporting triton-kernels. That is not a regression in what the installer does. Those builds have always happened; they only became visible now that pip_install stopped discarding uv's output on success, which is what finally let the nobuild check read the dependency phase at all. So the question was whether each build actually needs a compiler. Checked against the real artifacts rather than assumed: openai-whisper 20250625, randomname 0.2.1, argbind 0.3.9 -- no version of any of the three has ever published a wheel; antlr4-python3-runtime is pinned at 4.9.3, below the first release that ships one. All four sdists use setuptools.build_meta, declare no ext_modules, and contain no .c/.cpp/.pyx/.rs file. Already allowlisted, correctly. triton-kernels is the same category and was the only name failing. It is pinned to the triton repo's python/triton_kernels subdirectory; that tree is 75 files of Python, a four-line pyproject.toml, no setup.py and no native source at all. The kernels are Triton DSL compiled at runtime, not at install time. It is also a direct URL the installer names itself rather than something resolution picked, and only Linux reaches it. It belongs in the allowlist, so add it with that reasoning written down. The allowlist match is now lowercased and underscore-folded on both sides. The requirement spells the package triton_kernels while uv prints triton-kernels, and an allowlist that matched only one spelling would pass by luck rather than by intent. A plain pyarrow sdist is still caught. The two data-designer @ file:// plugin builds needed nothing: they are in-tree local paths, already dropped by the same rule that exempts the source overlay's own build. Separately, the windows-11-arm leg fails for a real reason and should keep failing. The ARM handling itself works, the log shows torchaudio being skipped and torch plus torchvision installing from wheels. What stops it is that pyarrow and hf-transfer publish no win_arm64 wheel at all, so uv falls back to their sdists and they fail on CMake configure and on openssl-sys wanting perl. That is a product gap on the platform, not a gap in the simulation, so the leg stays experimental and keeps reporting it. Record that above the matrix entry so the next reader does not re-diagnose it. * Exercise the bundled Windows installer, and stop mislabelling installer sources Four things that let a leg go green while proving nothing. The desktop Windows job installed the bundle and launched it, and that was all. On a fresh profile preflight reports not_installed and the app sits on the install screen waiting for a click, so the process happily stays alive for 90 seconds without the bundled install.ps1 ever running. A bundle that shipped no install.ps1 resource, or a broken one, passed this job -- which is the packaged app failure the workflow exists to catch. macOS and Linux already invoke their bundled script directly; Windows now does the same, via the resource NSIS laid down next to the exe, invoked the way install.rs invokes it, then asserts the managed venv exists and can import torch. Its timeout goes to 60 minutes because a full torch install on a Windows runner is the slowest of the three. A manual run that selects installer_source: published only redirected the macOS and Linux jobs. WSL kept copying the checked-out install.sh and Windows kept running the checked-out install.ps1, so a run asking whether the script on unsloth.ai works reported on this ref under the published label. Both now honor the selection; install.ps1 advertises its own unsloth.ai URL, so published has a meaning on Windows too. Both branches stay empty on pull_request and push, so automatic runs are unchanged. The push-to-main filter listed only install.sh, install.ps1 and this workflow, while the PR filter also covers setup.sh, setup.ps1, install_python_stack.py and the clean-machine helpers. A direct push touching those skipped the workflow entirely, so the post-merge backstop never ran for the files the source overlay was added to cover. The two lists now match. Neither filter covered studio/backend/requirements, even though the overlay exists precisely so a constraints change is resolved on a machine with no compiler and no cached wheels. The update-smoke workflows cannot stand in: they start from a preinstalled Python and full developer tooling. * Make the Linux and Windows desktop legs clean, and honour published on every macOS delivery The desktop workflow claims all three platforms are stripped, but only macOS and Windows had a strip step and the Windows one scrubbed the process PATH only. Both gaps let a bundle that needs a developer toolchain pass the one workflow whose premise is that it must not. Linux: the job ignored strip_toolchain entirely and ran the bundled install.sh with the runner's git, gcc, cmake and make in /usr/bin. clean-machine-env.sh now has a Linux --remove branch that moves the resolved tool binaries aside, recorded in restore.sh, and the job calls it plus `assert absent` after the apt step (the .deb install needs dpkg) and before the bundled installer, with a restore step to match macOS. The loop repeats per tool so a name present in both /usr/bin and /usr/local/bin is fully masked rather than half masked. Windows: rewriting $env:PATH does not survive the bundled install.ps1, which calls Refresh-SessionPath (318-337) and rebuilds $env:Path from the Machine and User registry values, and py.exe in C:\Windows reaches the toolcache whatever PATH says. Ported the on-disk toolcache rename, the Machine/User registry scrub and the py -3.11/-3.12/-3.13 start probe from clean-machine-install-ci.yml, so the strip is proven rather than assumed. Windows preflight: the log step was Test-Path, Get-Content and Select-String, none of which can fail, so an app that hangs before preflight passed on the 90 second liveness check alone. It now asserts a tauri.log exists and carries a `desktop_preflight completed disposition=` line, the same unconstrained check macOS and Linux already make. The disposition VALUE is deliberately not constrained: ManagedReady over an unbootable venv is the reported bug. installer_source on macOS: only the pipe delivery branched on it, so a `published` dispatch ran the checked-out script on six of the eight macOS rows while the run was labelled published. The script is now resolved once at the top of the Install step and used by the file and tauri deliveries; pipe still re-fetches through the live transport, because that is half of what it tests. Linux, WSL and Windows already honoured the input. Also shortened the comments across the changed files, keeping the reasoning that says why each check exists. * Run the Windows installer under PowerShell 5.1, the only shell a clean machine has The Windows Install step ran `& $script` inside a `shell: pwsh` step, so install.ps1 was executing under PowerShell 7. A genuinely clean Windows box does not have PowerShell 7: Windows ships powershell.exe (Windows PowerShell 5.1) and pwsh is a separate install that the hosted runner image happens to preinstall. So the one workflow whose premise is a machine that has never seen a developer toolchain was testing the installer under a shell that machine would not have, and no other Windows job anywhere in .github exercises install.ps1 under 5.1. Invoke it the way the desktop does (install.rs:325-339, and the bundled installer step in desktop-app-clean-machine-ci.yml): powershell.exe with -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass -File. The pwsh step wrapper stays, since it is only the installer that has to be under 5.1. Calling powershell.exe with `&` keeps the output in the pipeline, so Tee-Object still fills logs/install.log, and $LASTEXITCODE after the pipeline is the child's real exit code, so $rc and `exit $rc` are unchanged. install.ps1 and studio/setup.ps1 hold no PowerShell 7-only constructs: no `#Requires` above 5.1, no `&&`/`||` chain operators, no ternary, no null-coalescing, no ForEach-Object -Parallel, no $IsWindows/$PSStyle, and no 6+ cmdlets or parameters. setup.ps1 declares `#Requires -Version 5.1`, and its three $PSVersionTable branches gate a 7-only preference on the 7 side with a 5.1 fallback. Every Invoke-WebRequest already passes -UseBasicParsing, which 5.1 needs because it otherwise reaches for the IE engine. * Assert the Windows desktop strip actually took effect The desktop job's Windows masking renamed the toolcache Python, scrubbed the Machine and User registry PATH, and probed `py`, but nothing checked that `python`, `git`, `cmake` or `cl` were gone. The drop list is heuristic path fragment matching, so a runner image that moves any of those outside those fragments leaves the bundled install.ps1 reusing hosted developer tooling while the job still reports a clean machine. PATH written to $GITHUB_ENV only applies to later steps, so the check has to live in a step of its own; it carries the same event gate as the strip, exempts `py` (it lives in C:\Windows and stays, which is why the start probe is the real evidence), and resets $LASTEXITCODE before exiting 0 so an intentionally failing probe cannot fail a clean machine. Also correct the no-winget matrix note: that leg is not failing for an unfixed product reason. It stops at the unconditional git gate in setup.ps1 only on this ref, and with that gate relaxed it passes along with every other leg, so the row is a merge order dependency and stays required. * Resolve the desktop release including drafts, the convention this repo ships All three desktop legs died at the download step with an empty REL_TAG. The resolver passed --exclude-drafts while REL_REPO now defaults to github.repository, and every desktop-v* release in unslothai/unsloth is a draft: desktop-v0.1.50-beta and desktop-v0.1.471-beta are both drafts carrying the .dmg, .deb, .AppImage and setup.exe, while only the non-desktop tags like v0.1.501-beta are published. Excluding drafts therefore matched nothing and no leg could ever run against a production bundle. Drop --exclude-drafts so the newest desktop-v* release is found. A draft has no tag ref, so releases/tags/<tag> 404s for one, but gh resolves drafts over GraphQL and gh release download <tag> fetches their assets normally, so the download call is unchanged. Listing drafts requires push access, which for GITHUB_TOKEN means contents: write, so the workflow permission is raised from read and annotated. When nothing resolves the leg still fails hard rather than skipping: with no bundle to install there is nothing to prove, so a green run would be a lie. The error now names both causes, no release cut yet or a token that cannot see drafts. Also stop the restore step swallowing its own failure. `bash .clean-machine/restore.sh || true` printed "No such file or directory" whenever an earlier step failed before the toolchain was stripped, and hid a genuinely broken restore just the same. Skip explicitly when the file is absent and let a real restore failure surface. Same fix in clean-machine-install-ci.yml, which had the identical line. * Skip the desktop jobs on fork PRs instead of failing them Every desktop-v* release in this repo is a draft, and GitHub lists drafts only to a token with push access, which is why resolving one needs contents: write. A pull request from a fork receives a read-only token no matter what the workflow declares, so on those runs the resolver cannot see any release and the job died on "no desktop-v* release visible", accusing the repo of having no bundle when the real cause is the trigger. This workflow runs on pull_request for changes to itself and the stripping scripts, so an outside contributor editing either would have hit that. Guard the three jobs on the head repo not being a fork. A skipped job is honest here: it does not claim to have tested a bundle it was never able to download, and it is not reported as a pass. * Close the free headroom in the clean-machine simulation Assert arch and signature on every downloaded Mach-O. This is the one genuine gap the simulation had: Rosetta 2 is preinstalled on hosted runners and absent from a factory-fresh Mac, so an x86_64-only llama.cpp, whisper.cpp, Node or uv payload runs green here and dies with "bad CPU type in executable" for the user. llama-server launching under `assert-llama-loads.sh` does not rule that out, because Rosetta makes it launch. The new `macho` check reads `file -b` (`lipo` is an xcrun shim and is gone after masking, as the desktop lane already notes) and keys the expected arch off `uname -m`, so macos-15-intel expects x86_64. It also requires at least an ad-hoc signature on arm64, which closes the AMFI "Killed: 9" class that uv has already been bitten by; the check is skipped on x86_64, where unsigned code loads fine and so is not the same defect. It fails when the scan finds nothing, since an empty scan reads exactly like a clean one. Make absence real rather than PATH-hidden. uv probes well-known interpreter locations and the framework loader ignores PATH entirely, so hiding the toolcache only hid it from `command -v`. Empty /usr/local (it EXISTS on a factory-fresh Mac as a SIP-exempt firmlink, and is empty; it is /usr/local/bin that is absent, so the directory itself stays), move the hosted toolcache and /Library/Frameworks/Python.framework aside, and clear the developer dotdirs and caches. A populated uv or pip cache can also satisfy a resolution that would fail on a user's machine. Every removal goes through --remove and is recorded in the generated restore.sh, guarded so a path the install recreated is not buried inside its own restore. Unset CI, GITHUB_* and RUNNER_* for the installer process only. An installer branching on CI=true is a hidden dependency no consumer exercises. Scoped to the child so the step's own $GITHUB_OUTPUT still resolves. Record spctl --status and csrutil status. Neither is documented for these images and both change what a binary is allowed to do. * Pin the two failures no change here can fix, and add the virgin Windows container lane Three red checks, two of which test something this branch does not own. desktop linux deb / appimage run the SHIPPED bundle's own install.sh, and desktop-v0.1.50-beta was cut on 2026-07-21, before #7547 merged on 07-29. That bundle still carries the old optional-dependency gate, so on a stripped runner it exits 2 at [TAURI:NEED_SUDO] cmake git build-essential libcurl4-openssl-dev and never creates a venv. Current main's _check_linux_deps runs the same set through _SMART_APT_OPTIONAL, which suppresses every escalation path, so only a new release can change this. The step now pins that exact outcome: the exit code must be 2 and the log must carry exactly that package list, anything else still fails, and finding _SMART_APT_OPTIONAL in the extracted install.sh (the guard #7547 added) turns into a hard error saying to delete the pin. The venv and torch assertions stay and still run whenever the installer succeeds. win windows-11-arm gets a native ARM64 CPython, and torchaudio publishes no win_arm64 wheel at any version, so the PyTorch step cannot resolve. The fix is in install.ps1 on #7549, still open. Same treatment: the Install step is continue-on-error and a new step requires all three of the PyTorch step, the torchaudio resolution error and the missing win_arm64 platform tag, so any other failure is red. The row leaves experimental so the job is required, and the pin errors out as soon as the venv interpreter reports anything but win-arm64, which is what #7549 landing looks like. Adds the virgin Windows container lane as two jobs here rather than a sibling workflow: same premise as the win legs, same path filters, and masked-versus-real reads better side by side. The hosted Windows legs cannot test the VC++ 2015-2022 runtime (it ships in the runner image's System32) or a Windows with no Microsoft Store, and a servercore:ltsc2022 container on windows-2022 answers both. The probe asserts no python, py, git, cmake, cl, winget or uv on PATH, on disk or in the registry, and now also asserts vcruntime140.dll, vcruntime140_1.dll and msvcp140.dll are absent, which is the one thing the hosted runner cannot un-ship. Both container install rows stop at studio/setup.ps1's winget-only git gate on this branch, since #7549 is what relaxes it, so both are pinned the same way. The overlay row additionally requires the UNSLOTH_CI_SOURCE_OVERLAY hook to have fired, unconditionally: without that it would be indistinguishable from the released-wheel row, and the hook is this branch's own feature. Container notes carried over from the spike: never docker pull when the image is cached, since MCR has shipped an image ahead of the runner host before; wait for the Docker daemon, because one leg died in 21s on npipe:////./pipe/docker_engine and that flake misreads as "Windows containers unavailable"; drive docker from a run: step, because the job-level container: key is Linux-only. The root CA store is seeded after the virginity assertion, restoring what a real Windows already has, because studio/install_node_prebuilt.py downloads Node with bare urllib.request.urlopen and hits CERTIFICATE_VERIFY_FAILED against the empty container ROOT store. That product bug is left alone here. * Check signatures on Mach-O main executables only The macho check asserted a valid signature for every Mach-O under the studio home, and failed the macos-15 mask/pipe leg on 29 files: lxml, charset_normalizer, cygrpc, _upb, fontTools, caio, brotli and a bundled libportaudio.dylib. Those are MH_BUNDLE and MH_DYLIB images dlopen'd into a process without library validation, they ship unsigned in the wheels, and the same run had already installed and imported them with the installer exiting 0. Key the signature half off the Mach-O filetype and run it only on main executables. Report an absent seal separately from one that fails to verify, and capture codesign output instead of piping it into grep, which returned the unsigned exit status through pipefail and called every unsigned binary broken. The architecture half is unchanged and still a hard failure: it is what closes the Rosetta 2 gap. The zero-Mach-O guard is unchanged. The .venv_t5_* sidecars stay in scope; setup.sh creates them during a normal install and transformers_version.py puts them on sys.path, so they are payload. * Make the WSL job gate, assert Windows installed no toolchain, strip before the .deb * Assert the root Linux legs did not compile llama.cpp with the apt-installed toolchain * Pin the macOS desktop legs on the same pre-7547 release lag The Linux rows already pin the shipped bundle's own install.sh exiting 2 at the NEED_SUDO handshake. macos-15 and macos-26 fail the same way for the same reason: desktop-v0.1.50-beta predates #7547, so the bundled installer still hard-exits on the Xcode CLT gate that #7547 turned into a warning. Accept exit 1 plus that exact gate line, and nothing else. _check_macos_deps is the function #7547 added, so its presence in the bundle means the release caught up and the block errors out asking for the pin to be deleted. * Pin the WSL pipe truncation and the masked-winget git gate The WSL leg dies at install.sh:2082 with an unterminated quoted string. Nothing is wrong with that line: piping the script into sh is not atomic. dash reads it from the pipe in 8192-byte blocks and runs each command as it parses, and install.sh:2007 calls _maybe_reroute_strixhalo_to_2404, which on WSL alone shells out to Windows interop; interop relays the stdin it inherited and drains the pipe. dash has 11 blocks buffered at that point, ending at byte 90112, which falls inside "$STUDIO_LOCAL_INSTALL" on line 2082. Truncating install.sh at 90112 and parsing it reproduces the message verbatim, and running the whole file under a stdin-draining interop stub reproduces the exit code too. #7548 wraps the body in _unsloth_main so sh parses everything before running anything, and the same reproduction against its head is clean. The eight green staging runs cited when this job's continue-on-error came off were all on trees that already carried #7548, so that evidence never covered this branch. Pin the exact signature instead: exit 2 plus the shell's own unterminated-quoted-string error, with the _unsloth_main marker read back out of the distro as the flip condition. Pin winget=masked the same way. studio/setup.ps1:1655-1669 gates on git unconditionally and can only fetch it through winget, so masking winget leaves no way to satisfy it. #7549 relaxes the gate, and its wording appearing in the tree retires the pin. * Retire the WSL pipe pin now that #7548 is in main The pin flipped exactly as designed: it looks for _unsloth_main in the installer it actually ran, and #7548 put it there. Delete the pin and the CLI waiver, and assert the opposite instead. WSL is the only platform whose install shells out to Windows interop mid-script, and interop relays the stdin it inherited, so this job is the one that can catch the pipe being drained again. A truncation here is now a hard failure. * Gate the no-elevation Linux install and split off the no-transport case * Assert no source build on the hosted Windows legs and keep winget for the desktop lane * Retry the container root CA seeding instead of failing on one Windows Update timeout * Run the clean-machine workflow for the prebuilt installer helpers it overlays * Narrow the container pin to its own gates and scan uv and the venv interpreter for arch * Tighten the clean-machine comments * Re-assert toolchain absence after the desktop .deb pulls its dependencies * Retire the #7549 pins and add a wget-only Linux leg #7549 is in main, so the three known-outcome pins that were waiting on it are stale and would now hard-error by design. Each is replaced by the assertion it was standing in for rather than deleted: win windows-11-arm now gates. The x64-on-ARM64 resolver is asserted as an outcome: the venv interpreter reports win-amd64 from its own sysconfig, and torchaudio (no win_arm64 wheel at any version) is installed. Measured on the integration branch before #7549 merged: "only a native ARM64 Python 3.13 was found" -> "installing x64 Python" -> torchaudio 2.10.0+cpu, install green. win windows-latest / winget=masked now gates. The relaxed git gate is asserted from both sides: the old unconditional message must be absent, the no-git branch must have been reached (so the row cannot pass because git leaked back onto PATH), and setup.ps1 must report git as absent-but-not-required. Both Windows rows, and the visible one, gained the usability check the Linux legs have had and Windows never did: a managed interpreter, an unsloth CLI on disk, and that CLI actually running. nobuild and the toolchain check only read the log, so an installer that exited 0 having produced nothing satisfied them. The torch assert also loses its fallback to whatever `python` resolves to. The virgin container overlay row gates, and asserts what only that lane can: it is the one environment whose System32 does not already ship the VC++ 2015-2022 runtime, so it is the only place Ensure-VCRedist's direct aka.ms download can be proved to run rather than be short-circuited. The overlay=false row keeps a pin, with a new reason: it installs unsloth from PyPI on purpose, and setup.ps1 inside 2026.7.5 (uploaded the 23rd) predates #7549, so it still stops at the old gate. That is release lag, it flips on the next release, and the pinned signature is now the old wording rather than "#7549 has not landed". Also adds linux ubuntu2404-nonroot-wget. install.sh's download() takes curl or wget and _transport_missing is true only when both are gone, so a wget-only box is supported on paper, but the gating nonroot leg provisions ca-certificates AND curl, so curl won every probe and the wget branch had never run. Same image, same no-sudo user, same asserts, wget instead of curl, and curl proved absent on disk for root and for tester before AND after the install, so the claim is that every download went through wget rather than that curl happened to be unused. * Tighten the clean-machine CI comments Comments only, no assertion logic, pins or leg definitions touched. Reflowed every rationale block to denser wording and removed the duplication that had built up across repeated steps: the desktop workflow repeated the fork-PR skip, the desktop-v* tag resolution and the restore-runner note once per platform, and the installer workflow repeated its path-filter rationale in both the pull_request and push blocks. Those now point at the first copy. Every WHY is kept: why the masked legs avoid install.sh --local, what UNSLOTH_CI_SOURCE_OVERLAY is for, why `absent` tests "must not work" rather than command -v, why the .venv_t5_* sidecars are in the macho scan scope, why the signature check is main-executables-only, why each nobuild allowlist entry is a pure-Python sdist, why the WSL job gates and what the pipe truncation was, and why the virgin container's overlay=false row is still pinned. Proved comments-only three ways: both workflow revisions parsed with yaml.safe_load_all and every leaf walked (only `run:` scalars differ); every changed bash body and .sh compared byte-for-byte after `bash --pretty-print -n`; every changed pwsh body and .ps1 compared as a token stream with Comment and NewLine tokens dropped. A negative control injecting one non-comment line into each layer makes all of them fail. * Clean machine CI: strip Strawberry, make the Fedora pin gating, run the Linux CLI desktop windows failed the strip verification because windows-latest ships a MinGW toolchain under C:\Strawberry\c\bin, which matches none of the drop fragments; the installer workflow already scrubs it. Fedora sat behind job-level continue-on-error, so its outcome pin could not fail the run. Tolerate the install step instead, as the no-transport row does. The Linux usable-install check only tested the executable bit; Windows and WSL already execute the CLI. The macho scan now fails when no venv interpreter was scanned, rather than letting uv alone satisfy the outside-root guard. * Clean machine CI: tighten the comments Round 12 comment reduction: compress wording, keep every reason. Comments only, verified with a YAML leaf walk (differences only inside run: scalars, only on # lines), bash --pretty-print -n byte comparison, a PowerShell token-stream diff and a Python AST comparison. * Clean machine CI: dereference the venv interpreter, pin the deb deps and the Windows disposition file did not follow the <venv>/bin/python symlink find -L printed, so it answered 'symbolic link to ...' and the Mach-O test dropped the one interpreter the Rosetta scan exists to check. Read with file -Lb and count what was classified, not what was found. apt treats a toolchain package the strip only renamed as already installed, so a .deb that started declaring git or cmake would never restore it and the absent re-check would still pass. Assert the declared Depends instead. The Windows lane accepted any preflight disposition although the bundled installer was already required to build a working venv; NotInstalled or ManagedStale there means the app cannot boot what it just installed. * Clean machine CI: assert every masked tool, and re-select the developer dir last clean-machine-env.sh moves ten tools aside and only warns when a move fails, but absent checked four of them, so a surviving gcc -- which install.sh probes for build-essential -- went unnoticed. restore.sh ran xcode-select --switch before the line that moved CommandLineTools back, so it named a still-masked directory, failed into || true and left the selection link unrestored. Capture the original selection and re-apply it after both directory restores. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-29 06:21:40 -07:00
Nilay	52609fb890	Studio: reset-password rotates the credential in place instead of deleting auth.db (#7573) ... * reset-password: rotate the admin credential in place instead of deleting auth.db * reset-password: fix the CI callers and error handling for the in-place rotation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * reset-password: narrow the CI change to the jobs that read .bootstrap_password * reset-password: stop over-claiming what the reset revokes and when it takes effect * auth: bind token issuance to the credential version that was verified * auth: bind credential-creating writes to the version the request authenticated with * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * auth: bind the change-password and workflow-key writes to their own credential version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * auth: read the credential version inside the transaction that validated it * data-recipe: answer 401 when a reset revokes the credential mid job start * Fix lint blocker and Windows path assertion for PR #7573 Drop the now-unused validate_api_key import from studio/backend/auth/authentication.py. Every call site moved to validate_api_key_with_credential, so the Source lint job's import-hoist gate flagged it as a blocker. The wrapper itself stays in storage.py; test_api_key_expiry.py still exercises it. Make test_run_reexec_forwards_resolved_frontend_on_public_launch compare against str(Path(...)) instead of a POSIX literal. _find_frontend_dist returns a Path, so on Windows the forwarded value is \fake\studio\frontend\dist and the assertion could never pass there. Pre-existing, surfaced by running unsloth_cli/tests on Windows. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-07-29 01:40:12 -07:00
Michael Han	2916e84499	Studio: clarify tool permission controls (#7181)	2026-07-20 09:55:39 -03:00
Daniel Han	03590f696e	Give opencode real timeout headroom in Local Agent Guides CI (#7235) ... * Raise the opencode invoke timeout in Local Agent Guides CI The connection (opencode) cell flakes with a 600s timeout reported as guide drift, but it is not a hang: in a passing run the same opencode run finishes in ~482s (08:12:31 to 08:20:33), right against the shared AGENT_INVOKE_TIMEOUT of 600s, so about one run in six drifts past the cap. opencode is the slow outlier. The print-mode agents (claude -p, codex exec) run one turn against a minimal injected system prompt, while opencode run runs its own full turn with opencode's large system prompt plus a separate small_model call to name the session (start.py pins small_model to the same 4B the server hosts). On a CPU-served gemma-4-E4B that is about 8 minutes, leaving no margin under 600s. Double opencode's per-invoke timeout in agent-guides-drive.sh and keep the tight 600s cap for the fast agents, so a genuine headless-TTY hang still fails quickly. 1200s stays well under the 40-minute job budget. * Normalize the agent invoke timeout before doubling it for opencode Strip an optional trailing 's' from AGENT_INVOKE_TIMEOUT so the opencode arithmetic, and the "${TIMEOUT}s" timeout message, stay valid if a timeout(1)-style suffix is ever configured. * Only double the opencode timeout for a bare-integer seconds value Guard the arithmetic so a GNU timeout(1) duration suffix (s/m/h/d, including floats like 0.5s) is passed through unchanged instead of breaking the expansion; timeout(1) parses those directly. Bare seconds still double. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>	2026-07-19 06:08:54 -07:00
Michael Han	6d8c18cd1a	Replace standalone Studio wording with Unsloth (#7221) ... * Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error.	2026-07-19 00:47:04 -07:00
Daniel Han	c1e06e9ddf	unsloth start: add --persist to keep and reopen agent sessions (#7014) ... * unsloth start: add --resume to persist and reopen agent sessions `unsloth start <agent>` launches a coding agent whose home is a throwaway temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their whole home there) cannot resume a conversation after you quit. opencode and claude keep their session data in a fixed user dir, so they already resume. Add an opt-in --resume/--no-resume flag: it routes the launch to the stable Unsloth agents dir (the same one --no-launch already uses) so the session survives the exit, never touching the user's own ~/.<agent>. A bare --resume also reopens the last conversation via the agent's native flag (codex `resume --last`, opencode/claude/pi `--continue`). The default is unchanged: a plain launch still uses a temp dir and persists nothing. Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the real launch path and asserts the split: codex/pi are wiped without --resume and persist with it, while opencode/claude persist either way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * unsloth start: rename --resume to --persist The session flag collided with agents' own resume flags. `unsloth start claude --resume <id>` used to forward `--resume <id>` straight to Claude (which keeps its history in ~/.claude regardless), so a boolean --resume on unsloth start would have swallowed the session id and turned it into a stray prompt. Name the persistence flag --persist instead, so every agent's native resume flag (claude --resume <id>, codex resume, opencode --continue, ...) still passes through untouched. Behavior is otherwise identical: --persist keeps a launched agent's session under the Unsloth agents dir, and a bare --persist reopens the last conversation. Add a regression test that `--resume <id>` passes through verbatim, and in the CI resume experiment skip the redundant second pass for opencode/claude (they persist either way, and a second CPU turn only risks a timeout). * unsloth start: correct --persist help and drop the buggy auto-resume Reword the --persist help to be accurate: claude and opencode keep sessions in the user's own stores and resume regardless, so --persist only stabilizes the otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the bare-launch auto-append of native resume tokens: it errored on a first launch with no prior session, and was inconsistent between launch and no-launch. --persist now only keeps the session dir; resume via the agent's own command (e.g. `unsloth start codex --persist resume`), which now finds it. In the CI resume experiment, fail the pass when the launched turn exits non-zero, so a write-then-error is not misread as PERSISTED. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-09 11:47:59 +02:00
Lee Jackson	6ef0936180	Fix OpenClaw start default to local TUI (#6937) ... * fix: launch OpenClaw local TUI by default * Fix/adjust OpenClaw launch paths for PR #6937 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Default OpenClaw to the local TUI only on a bare invocation The first-arg startswith('-') branch rewrote passthrough globals into a broken command: OpenClaw's grammar is openclaw [--dev] [--profile <name>] <command>, so 'unsloth start openclaw --profile test' became 'openclaw tui --local --profile test', but tui does not accept --profile (or --dev), so the invocation failed. A leading '--flag value' is ambiguous between a global (--profile test) and a tui option (--message hi), so it cannot be reinterpreted safely. Default to the local TUI only when no passthrough args are given, and forward everything else verbatim so OpenClaw parses it under its own grammar. The bare-launch default (the point of this change) is preserved; explicit subcommands and global flags pass through. --------- Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>	2026-07-08 04:25:42 -07:00
Daniel Han	69f8e0b228	Clear stale yolo approval state on no-launch reruns (#6868) ... * Clear stale yolo approval state on no-launch reruns The no-launch session config dir is deliberately reused across runs, but the config writers only ever added the --yolo auto-approval settings and never removed them. After one --yolo --no-launch run, every later run without --yolo kept OpenClaw's tools.exec security=full/ask=off policy plus exec-approvals.json, and OpenCode's permission allow block, so tool execution stayed silently pre-approved. Non-yolo runs now reset that state: OpenClaw drops the exec policy keys and the yolo defaults in exec-approvals.json (approvals OpenClaw itself recorded are kept; the file is removed when only the yolo payload is left), and OpenCode drops the permission block. Launch mode is untouched since it already uses an ephemeral temp dir. * Strip only yolo-written values on non-yolo cleanup Match each field against the exact value the yolo path writes before removing it, so a stricter exec policy, approvals defaults set by the user or the OpenClaw UI, and deny/ask OpenCode permission entries all survive a plain no-launch rerun. An unparseable exec-approvals.json is left in place, matching how an unparseable config is handled. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Write a prompting policy on non-yolo instead of deleting to a permissive default OpenClaw and OpenCode both treat an omitted policy as permissive: OpenClaw's effective exec policy for an unset tools.exec is security=full/ask=off on the gateway host, and OpenCode defaults an unset permission to allow. So clearing the yolo values on a non-yolo run did not restore prompting, it fell back to those permissive defaults and left tool execution auto-approved. A non-yolo run now writes an explicit prompting policy: OpenClaw gets security=allowlist/ask=on-miss (verified to prompt even with the approvals file removed, since the stricter of config and approvals wins), and OpenCode gets edit/bash/webfetch=ask. Only a permissive/yolo value is tightened; a stricter deny (or an ask the user set) is preserved, and the yolo approvals defaults are still stripped. The file-edit CI path opts opencode/openclaw into --yolo, since those agents now prompt by default and the headless test needs auto-approval. * Respect existing exec mode, sandbox/node host, and global permission rules on non-yolo reset The non-yolo reset for openclaw/opencode assumed an omitted policy was the permissive yolo default and rewrote it, which corrupted or weakened stricter setups it should have preserved: - OpenClaw tools.exec.mode is the normalized policy knob and cannot be combined with explicit security/ask (OpenClaw rejects the whole config), so writing security+ask alongside a mode:deny/ask policy both broke the config and relaxed it. Leave a mode-based policy untouched. - host=sandbox defaults to security=deny and host=node routes to a paired node; neither is written by --yolo (which only writes host=gateway). Treating the missing security as full and popping host broadened those into gateway/auto exec. Only rewrite a gateway-routed permissive policy, and never pop a non-gateway host. - OpenCode permission can be a string ("deny") or a {"*": ...} catch-all. The old code dropped a string form and overrode a catch-all by writing per-tool ask, weakening a stricter user rule. Now a string is left in place, a catch-all governs absent tools, and only an effective allow is tightened. - The non-yolo ask policy only lived in OPENCODE_CONFIG, which loads below project opencode.json, so a project config allowing edit/bash/webfetch still auto-approved. Carry the ask policy in OPENCODE_CONFIG_CONTENT (above project config) too, symmetric to how yolo carries its allow. Also harden the openclaw path against a malformed non-dict tools value. Adds tests for mode/sandbox/node hosts, string and catch-all permissions, and the inline ask policy over a project config. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope non-yolo resets to the exact yolo fingerprint and preserve granular denies OpenClaw: reset only the exact host=gateway/security=full/ask=off policy --yolo writes, so an omitted or host=auto/sandbox/node policy (which can resolve to a sandbox security=deny default) is no longer broadened to allowlist/on-miss, and a deliberate tools.exec.mode is left alone (OpenClaw never migrates our security/ask write into a mode). OpenCode: carry a granular object or a deny inline verbatim so a per-tool user rule is not collapsed to a blanket ask, but floor any object that grants allow anywhere to the string ask (which fully replaces a project object) so no inline allow pattern can leak through into a silent auto-approve on a non-yolo session. * Stop overriding project config on non-yolo; require full approvals fingerprint The non-yolo OpenCode reset carried a session permission in OPENCODE_CONFIG_CONTENT, which outranks the project opencode.json we cannot read. That inline override could not correctly reflect the project: it weakened a project deny to a prompt, mishandled global string rules, leaked through a granular object's permissive default when no catch-all was present, collapsed an object with an allow (losing its deny), and missed per-agent permissions. All of these stem from forcing a value over an unknown project config. A non-yolo run now only undoes what --yolo wrote: it flips our own explicit per-tool allow back to ask in our config file and carries no permission inline, so the project's own permissions are honored as written. Clearing our persisted yolo state is the actual fix; --yolo still carries its allow inline so it works over a project config. OpenClaw approvals cleanup now strips the yolo defaults only when the full fingerprint (security=full, ask=off, askFallback=full) is present, so a mixed user policy that merely shares askFallback=full (whose omitted default is deny) is kept intact. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-07-07 00:06:48 -07:00
Nilay	b8400f40df	CLI: Rename unsloth connect to unsloth start (#6613) ... * replaced connect with start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix * Studio: build the coding-agent command from the selected server The API keys panel showed a hardcoded `unsloth start claude`. `unsloth start` defaults to 127.0.0.1:8888 and only mints a key for a loopback server, so a non-default port or a tunnel/remote base would target the wrong server or fail to mint. Build the command from the panel base/key (and emit a key for non-loopback), matching the other snippets in the panel. * CLI: keep `unsloth connect` as a hidden alias for `unsloth start` Avoids breaking existing scripts and docs that still call `unsloth connect`. * Tests: stub _unstarted_cleanup in same-task disconnect test The test builds _SameTaskStreamingResponse via __new__, so set the attribute that __call__ now reads. * Match coding-agent command loopback check to the CLI 127.0.0.0/8 rule (#6613) * Keep unsloth_cli.commands.connect importable as a deprecated shim (#6613) * Format the new coding-agents panel strings and import per biome (#6613) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop the unsloth connect alias and shim; unsloth start is the only command (#6613) * Route unsloth connect to unsloth start as a hidden backward-compatible alias (#6613) * Forward unsloth run model-load flags to unsloth start (gguf-variant, context-length, load-in-4bit, tensor-parallel) (#6613) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Session-scope coding agent config in unsloth start Configure each agent for the current session instead of writing the Studio endpoint, key, and default model into the user's own config. Codex, OpenCode, OpenClaw, and Hermes get a private config relocated through their config-path env vars (CODEX_HOME, OPENCODE_CONFIG overlay, OPENCLAW_CONFIG_PATH plus OPENCLAW_STATE_DIR, HERMES_HOME). Claude Code suppresses the attribution header for the session via the CLAUDE_CODE_ATTRIBUTION_HEADER env var plus a --settings overlay, with no ~/.claude write. --launch uses an ephemeral temp dir removed after the agent exits; --no-launch uses a stable Unsloth-owned dir and prints the matching export lines. * Read relocated agent session config in Local Agent Guides CI The contract crosscheck and the openclaw/hermes patch helpers now read each agent's config from the relocated path printed by unsloth start --no-launch (CODEX_HOME, OPENCODE_CONFIG, OPENCLAW_CONFIG_PATH, HERMES_HOME) instead of fixed home paths. The Claude attribution A/B toggles the header for the session only (shipped-config HIT vs vanilla MISS) instead of editing ~/.claude/settings.json. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip the POSIX-only --no-launch parser test on Windows test_no_launch_output_is_parseable mirrors the #6547 bash CI parser, which greps export/unset lines and only runs on Linux/macOS runners. On Windows --no-launch prints PowerShell ($env: / Remove-Item), so the export-line assertion does not apply there. Cross-OS staging CI surfaced this. * Size Claude Code's auto-compact window to the loaded model's context Claude Code auto-compacts against its native (~600k token) window, so against a smaller local model it overflows the server's context (silent truncation) long before it compacts. Set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the loaded model's real context length (the value codex/openclaw already get via model_context_window / contextWindow). Omitted when the model reports no context length. * Pin OpenCode/Hermes context window and set 90% compaction across agents Feed every agent the server-determined sequence length (the value /v1/models reports from runtime_context_length) and a ~90% compaction threshold. OpenCode: a custom-provider model with no limit defaults to context 0, which silently disables auto-compaction, so set limit.context/output and scale the compaction buffer to 10% of the window. Hermes: pin model.context_length (it otherwise falls back to a 256k default when the server's /v1/models omits the field) and set compression.threshold 0.9. Claude: add CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=90 alongside the window. Codex (model_context_window) and OpenClaw (contextWindow) already carried the window and auto-manage off it. * Add `unsloth start pi` recipe Pi was the only agent without a built-in recipe, so the agent-guides CI hand-wrote ~/.pi/agent/models.json. Add a first-class `pi` command mirroring the others: - write_pi_config writes the session-scoped OpenAI-compatible provider config (key in the config, like openclaw/opencode). - pi() launches `pi --provider unsloth --model <id>` (Pi defaults to the google provider, so the provider/model are pinned on the command line) with HOME relocated for the session. Pi has no config-dir env var and resolves ~/.pi off $HOME, so HOME-scoping keeps the user's ~/.pi untouched. Migrate the agent-guides CI off the hand-written config onto the `unsloth start pi --no-launch` path (connection + file-edit), with a crosscheck for the provider api, so the documented recipe is exercised. * Harden unsloth start for Windows and WSL agent launches Address the Codex review on PR 6613: - write_pi_config now pins the loaded contextWindow and a sane maxTokens so Pi compacts instead of overflowing a small Studio context (it otherwise assumes its 128000 default), matching the other agents. - pi() sets USERPROFILE (and HOMEDRIVE/HOMEPATH when present) alongside HOME on native Windows, where Node resolves ~/.pi via USERPROFILE rather than HOME, so the session no longer reads or writes the user's real ~/.pi. - The WSLENV bridge flags path-valued vars with /p so a Windows npm shim under /mnt receives translated paths, while scalar vars (the numeric context window) pass through untranslated. WSLENV is deduped on the bare name. - _print_env prints the launch command with PowerShell-safe quoting so the inline --settings JSON survives copy-paste on native Windows --no-launch. Add tests for the WSLENV path flagging, PowerShell quoting, the Pi context window, and the Pi USERPROFILE relocation. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Set CLAUDE_CODE_NO_FLICKER for the Claude session A local server streams in bursts, so Claude Code's full-screen TUI redraw flickers between tokens. Disable it for the session via CLAUDE_CODE_NO_FLICKER, alongside the other CLAUDE_CODE_* session env knobs. * Add a normalized --yolo flag routed to each agent's auto-approve mode It is easy to forget which agent spells "run tools without prompting" which way, so `unsloth start` now accepts all three spellings as one option (--yolo, --dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox) and routes to the agent's own mechanism: - claude: --dangerously-skip-permissions - codex: --dangerously-bypass-approvals-and-sandbox - hermes: --yolo - pi: --approve (Pi's only approval gate is project trust) - opencode: a permission allow block in opencode.json (no CLI flag exists) - openclaw: tools.exec security=full / ask=off / host=gateway (no CLI flag exists) Because the option is parsed by `unsloth start`, the "wrong" spelling for an agent still routes correctly instead of leaking through to the agent and erroring. IS_SANDBOX is deliberately left unset for Claude so its root/sandbox safety gate still applies. Adds routing, cross-routing, and per-config tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix review findings: IPv6 loopback command, pi USERPROFILE under WSL, yolo guard From a 10-reviewer pass over the PR: - studio/frontend agent-command.ts: normalize bracketed IPv6 hosts. URL.hostname returns "[::1]" for http://[::1]:8888, which never matched the "::1" loopback checks, so the copied command embedded the placeholder API key for a local IPv6 server instead of the bare auto-minting command. Now [::1] is treated as loopback like the CLI's is_loopback_url, so the command matches the CLI contract. - pi(): also relocate USERPROFILE (and HOMEDRIVE/HOMEPATH) when running under WSL against a /mnt Windows shim, not just on native Windows. Windows Node resolves ~/.pi via USERPROFILE, and the WSLENV bridge translates the path, so pi no longer falls back to the user's real ~/.pi in that case. - _yolo_command_flags: use .get so a config-based agent (or a typo) yields no flag instead of a latent KeyError. Adds tests for the WSL pi USERPROFILE relocation, the yolo unmapped-agent guard, and that opencode/openclaw --yolo stays config-only (no argv flag). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix round-2 review findings: WSLENV /p upgrade, agent help text - _merge_wslenv now upgrades a user's pre-existing unflagged WSLENV entry (e.g. a bare HOME or USERPROFILE) to the path-translated form (HOME/p) instead of leaving it as-is, so a Windows agent shim under WSL receives the translated session path rather than the raw Linux path. - Generalize the `unsloth start` registration help to list all six agents (was only "Claude Code, Codex"). Adds a test for the WSLENV unflagged-entry upgrade. * Fix round-3 review findings: complete openclaw --yolo, refresh stale copy - openclaw --yolo now also writes the host approvals file (exec-approvals.json with defaults security=full / ask=off / askFallback=full) alongside the tools.exec config. OpenClaw gates tool execution on both layers (the stricter wins), so the config alone could still leave it prompting or denying. Mirrors `openclaw exec-policy preset yolo`. ask=off means nothing is ever prompted, so the runtime socket block is unnecessary. - Studio API panel copy: clarify that a local server auto-mints the key while a remote one embeds it in the command, and add pi to the swap hint. - Local Agent Guides CI: drop the stale "pi has no start.py recipe" note now that all six agents are driven via `unsloth start <agent> --no-launch`. Adds the openclaw approvals-file assertions and a no-yolo openclaw test. * start: parse claude --version with a regex so a format change does not drop optimization flags * start: offer to install a missing agent (prompt then run its install command) * start: auto-start a Studio server for --model when none is running, and stop it on exit * inference: surface an actionable message when llama-server cannot compile a tool grammar * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix review findings: kill the auto-started server tree on Windows; apply the tool-grammar message to the OpenAI passthrough too * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * start: split --model org/repo:variant so a running session is not evicted `unsloth start <agent> --model org/repo:QUANT` failed against an already-running Studio server and, worse, killed whatever model another session had loaded. /v1/models lists a loaded GGUF under its bare repo id (e.g. unsloth/Qwen3-1.7B-GGUF), so _resolve_model never matched the `:QUANT`-suffixed request. It then POSTed /api/inference/load with model_path=org/repo:QUANT, which (a) Hugging Face rejects ("Repo id must use alphanumeric chars, '-', '_' or '.'") and (b) evicts the model the other session was using, so a second 'unsloth start' in a new tmux/terminal tore down the first. Re-running the command then attached to the now-empty server, which is why it 'worked the second time'. Mirror the org/repo:QUANT -> org/repo + --gguf-variant QUANT shorthand that 'unsloth run' and llama.cpp already accept, splitting it in _connect before we match or serve. Matching now resolves against the loaded bare repo id (no spurious reload, no eviction), and any real load uses a valid repo id plus gguf_variant. An explicit --gguf-variant still wins; local paths and Windows drive letters pass through untouched. The auto-serve path likewise spawns 'unsloth run --model org/repo --gguf-variant QUANT'. * start: harden auth-key handling, codex teardown, and CI transcript redaction Three review findings: 1. CI could leak a live key. agent-guides-drive.sh printed the raw 'unsloth start --no-launch' transcript (which carries export UNSLOTH_API_KEY / ANTHROPIC_AUTH_TOKEN lines) to the Actions log on both the failure path and the success path before redact() ran. Add cat_redacted() and use it for those two prints, so the key is scrubbed on the way to the log while the on-disk file stays intact for the env parsing that follows. 2. Outages masqueraded as bad keys. _key_accepted caught a broad Exception and returned False, so a 5xx or timeout while checking a cached key looked like a rejection: it discarded a good key and minted extra ones (local) or reported 'no saved key' (remote). Only treat HTTP 401/403 as a rejection; let other errors propagate so a real outage surfaces. 3. Codex preflight could leave the auto-started server up. _require_gguf_for_codex runs after _connect may have auto-started Studio but before _run installs its teardown finally, so a preflight rejection (e.g. a transformers-backend model) left the server holding the port/GPU until the atexit backstop. Tear it down explicitly at the point of failure. Tests: a 5xx on a saved key surfaces without minting; a non-GGUF codex preflight tears down the auto-served server. * start: fix IPv6/portless studio URLs, Pi config-dir isolation, and Pi install recipe Four review findings: 1. Pi ignored the session config when PI_CODING_AGENT_DIR was already set. Pi's getAgentDir() reads process.env.PI_CODING_AGENT_DIR before falling back to $HOME/.pi/agent, so a value inherited from the user's shell sent Pi to their real config and skipped our provider/key (the HOME relocation alone was not enough). Pin PI_CODING_AGENT_DIR at the session's .pi/agent dir; it is path-valued so the WSL bridge translates it automatically. 2. Pi install hint dropped Pi's documented --ignore-scripts. Pi's README installs with 'npm install -g --ignore-scripts @earendil-works/pi-coding-agent' and notes it needs no install scripts, so accepting the prompt now follows that safe recipe. 3. Auto-start ignored a portless UNSLOTH_STUDIO_URL. unsloth run binds to 'parsed.port or 8888', so http://127.0.0.1 launched the child on 8888 but the health poll (and the returned base) still used port 80, stalling until the startup timeout. Normalize the base to host:8888 (IPv6-safe) before starting and polling. 4. API-panel command mistook IPv6 loopback for the bare default. The bare 'unsloth start' only probes 127.0.0.1:8888 on the IPv4 stack, so http://[::1]:8888 must carry an explicit UNSLOTH_STUDIO_URL. Drop ::1 from the bare-default host set while keeping it a loopback host (URL emitted, no key needed). Tests: PI_CODING_AGENT_DIR is set to the session dir; _effective_base normalizes portless/IPv6 bases; a portless UNSLOTH_STUDIO_URL auto-serves on :8888. * start: apply fresh-review findings across CLI, CI, and the API-panel command From a fresh multi-reviewer pass over the merged head plus the latest Codex bot review: 1. Load knobs now always consult the server. _resolve_model matched on model id alone, so --gguf-variant / --context-length / --no-load-in-4bit / --tensor-parallel were silently ignored whenever the id was already loaded (asking for UD-Q4_K_XL kept a Q8_0 serving). With any explicit knob the CLI defers to /api/inference/load, whose already-loaded dedup answers without reloading when variant and settings match, so a second session running the same command still attaches without evicting the first. 2. OpenCode --yolo and the session model pin now ride in OPENCODE_CONFIG_CONTENT. A project's own opencode.json outranks OPENCODE_CONFIG, so a repo config could silently override the session model and the --yolo permission block; OPENCODE_CONFIG_CONTENT outranks project config. The API key stays in the private file, never in printed env. 3. The --no-launch recipe's last line is a self-contained one-liner (inline VAR=value assignments before the command, conflicting vars blanked). People copy just the last line, and a bare codex/claude there ran against the user's real ~/.codex or Anthropic credentials with zero isolation, e.g. inheriting a pre-existing damaged ~/.codex state DB and blaming the recipe. The CI drive script scrubs the key from the one 'invoking:' echo this adds. 4. The auto-serve log is 0600 and the parent handle is closed. It sat world-readable in the shared tempdir under a predictable name while carrying the minted sk-unsloth- key from the unsloth run banner. 5. _key_accepted fails with a clean message on outages. Non-auth errors (5xx, network, timeout) surfaced as a raw traceback; 401/403 still mean a rejected key. 6. _effective_base strips URL paths, and https loopback targets never auto-serve. http://127.0.0.1:8888/studio polled /studio/api/health (404) and https://127.0.0.1 polled the wrong scheme, both spinning until the 15-minute startup timeout. 7. API-panel command: only literal 127.0.0.1:8888 earns the bare command. localhost can resolve to ::1, which the bare CLI never probes, so it keeps UNSLOTH_STUDIO_URL. 8. CI artifact sweep covers redacted-configs/ and agent-workdir/, not just logs/. Tests: 125 CLI tests pass (new coverage for each fix), 156 backend tests pass, ruff clean. Adds an unsloth connect alias regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * start: hand Pi a clean screen at launch Pi paints inline from wherever the cursor sits: its first render assumes a clean screen instead of clearing or entering the alternate screen itself (current Pi never emits a clear at startup). Launched under unsloth start, that left the session starting mid-scroll beneath the connection output. Clear the screen (click.clear, cross-platform, no-op without a TTY) right before the Studio banner so Pi opens exactly one line down on a clean viewport. Launch path only: --no-launch recipes and piped output are never wiped, and alternate-screen agents are left alone. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * start: auto-override hermes' 64K context floor for small model windows Hermes refuses to initialize when the served model's context window is under 64,000 tokens, and a second copy of the same check rejects the compression model mid-session. write_hermes_config previously pinned the real window, so any small local model (e.g. 40,960) failed at startup with manual config.yaml instructions. For windows below the floor the recipe now claims 65,536 in model.context_length, scales compression.threshold so compaction still fires at 90% of the real window, and sets auxiliary.compression.context_length to cover the mid-session check. Windows at or above the floor keep the exact previous behavior. * ci: install pi with --ignore-scripts, matching the start.py hint The pi cell predates the pi recipe in start.py and still installed the package with lifecycle scripts enabled, so CI stopped exercising the exact command users are prompted to run. npm_retry now passes extra flags through, the pi branch mirrors the install hint verbatim, and the stale no-recipe comment is refreshed. * ci: fail loudly when a relocation var is missing from connect output The empty-string guards ran after appending /config.toml or /config.yaml, so they could never fire: crosscheck_contract silently skipped its contract checks and patch_hermes_tools died on the root path with a bare traceback. Check the raw variable first and guide_fail with the real cause. * staging: 6613 round 6 (https elision, no-launch home reuse, auto-start key fallback) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>	2026-07-03 08:17:27 -07:00
Daniel Han	264f1a04f8	Add Local Agent Guides CI (#6547) ... Boot `unsloth run --disable-tools` against a small GGUF and drive each supported coding agent (claude, codex, hermes, openclaw, opencode, pi) through its documented `unsloth connect <agent> --no-launch` recipe, so the connect flow in unsloth_cli/commands/connect.py stays exercised end to end and regressions surface as a failing check. Per-agent matrix, three jobs: - connection: assert a non-empty, error-free reply to a trivial prompt - file-edit: a two-turn create-and-run hello.py test (dispatch/schedule only, skipped on pull_request) - prompt-cache: verify llama.cpp prefix-cache reuse across requests The GitHub-hosted runners are CPU-only, so each request is trimmed to the smallest prompt that still drives the recipe: claude with --tools to drop unused tool schemas (--allowedTools only gates permission, it does not shrink the prompt), hermes with an empty platform_toolsets.cli, and openclaw with a minimal agent definition. hermes and openclaw run a multi-turn tool loop in file-edit that a CPU runner cannot finish in time, so those two cells are best-effort; their endpoint wiring is still hard-gated by the connection job. A preflight step HTTP-checks each agent's API dialect before install so a server-side contract regression is reported separately from agent or guide drift.	2026-06-22 04:21:48 -07:00
Daniel Han	e83d4ae072	Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) ... * Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH. shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed to report gcnArchName, which is why the arch came from the GPU-name table) and amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt). TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so the inductor/Triton cache lives under the user's Studio home, not the system drive root. Long paths are already enabled above so deep inductor paths still fit. unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32" warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because setup runs via unsloth.exe (the running launcher holds its own file). The 'studio update' flow still attempts it. CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after the AMD wheels land, so the log makes clear the final install is GPU-accelerated. Adds two regression tests covering the venv-internal vs external hipInfo gate. Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1 --local from this branch completed exit 0 with no DiskPart prompt, no rename warning, the cache under the Studio home, and "GPU ROCm PyTorch installed (gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon 8060S Graphics". * Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning setup.ps1 used to rename the running unsloth.exe out of the way before the base-package upgrade so pip could replace it. That rename never actually worked: setup runs *via* unsloth.exe, so renaming our own running uv-trampoline launcher failed with a sharing violation (WinError 32) and only printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32' warning on every Windows install and update. It also was not needed. pip tolerates a running/locked console-script .exe: it moves the old one aside and writes the new one. The base upgrade routes through pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left intact either way. Removing the rename block and its failed-install restore block removes the false warning for all Windows devices in both the install and update flows. * Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks Follow-up to PR #6296. - install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python installers, which already do this. - amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv containment check (Windows paths are case-insensitive) and run the HIP_PATH/ROCM_PATH candidate through it too. - setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are unavailable, and create the dir wildcard-safely. - tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and assert the PowerShell venv exclusion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Windows installer: install ROCm PyTorch directly for a known AMD arch When the GPU arch is known (name-inferred from the GPU-name table) but ROCm could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The repo.amd.com wheels bundle their own runtime (no HIP SDK required), which setup.ps1 already relies on, so the CPU base was a pure wasted download/install. - Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts still get CPU (unchanged). - Fall back to a CPU base if the ROCm-index install fails, so a transient repo.amd.com outage does not abort the install (setup.ps1 retries ROCm). - Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK. - Add a regression test for the arch-based gate. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Windows installer: correct the unsloth.exe rename-removal comment The comment claimed the base upgrade 'routes through pip on Windows' and that pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what the code does. install_python_stack tries uv first; on a locked launcher uv aborts and falls back to pip, but the pip fallback strips --upgrade-package and base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already satisfied and no-ops. The running unsloth.exe is left intact at its current version either way. Reword the comment to describe the real uv-first / pip-fallback-no-op behavior. No functional change. * Windows installer: close two gaps in the venv-internal hipinfo exclusion Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases: - setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the venv-internal hipInfo.exe was not recognized. Seed the venv root from UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers). - The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run Test-HipinfoIsVenvInternal on the candidate as well (both installers). Extend the PS gate test to assert both. Both .ps1 parse clean; install tests pass (the venv-internal / HIP probe coverage at 359 passed). * Windows installer: correct the CPU-base message for arches with no ROCm wheels After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl and installs ROCm directly, so it no longer reaches the "temporary CPU base" branch. That branch is now reached only by a name-inferred arch with no ROCm wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old text ("setup replaces it with GPU ROCm wheels ... the final install IS GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch stays on CPU for this GPU. * Windows installer: seed the venv-internal hipInfo check from a custom Studio home Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the setup python, and the default %USERPROFILE% path only. A standalone `unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias) and none of those set would not recognize the venv hipInfo on PATH, reopening the amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert it in the gate test. * Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter Two review points on the amd-smi/DiskPart UAC gate: 1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo path, which does not resolve symlinks, junctions, or 8.3 short names. A venv reached through an aliased path then fails the check, so its bundled hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py). 2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a leading ~, while the canonical resolver does. With a tilde form, [IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the same way as the resolver. tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink realpath case and a setup.ps1 tilde-expansion guard). * Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1 Follow-up review on the same install.ps1 paths: 1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal) seeded the venv root from a custom Studio home without expanding a leading ~, unlike the canonical resolver and setup.ps1. A tilde form left [IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart gate. Expand ~ in the probe, matching the setup.ps1 fix. 2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch to below 2.12. AMD's per-arch index publishes the companions independently and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a bare resolve can pick an ABI-incompatible set and fall back to CPU. Add torchvision/torchaudio floor maps and pass the pinned specs, mirroring setup.ps1 and install_python_stack.py. 3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no retry), the only torch step in the file without it. Switch to Invoke-InstallCommandRetry so the recovery path survives a transient index failure. tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde check over both installers, a torch/companion floor-map parity test, and a CPU-fallback retry guard). * Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo, which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe inside the venv and the bnb fix (plus the Studio backend) prepend the venv Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is unchanged. Applied to all three Python copies (install_llama_prebuilt.py, install_python_stack.py, backend/utils/hardware/amd.py) via a new _external_hipinfo_on_path helper, and both PowerShell callers (install.ps1, setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal. tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a shadow-regression test for the exact venv-first ordering, and a parity check that every Python copy uses the scanning helper). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock) Auditing a dual native+WSL uninstall on a real device surfaced three leftovers: 1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the shortcuts that reference that icon, so Explorer's icon cache briefly held it open. Remove-Item -Recurse reported success yet left the locked file, and the dir was never re-attempted, so it orphaned with a false "removed" log. _RemovePath now verifies the path is actually gone (retrying transient locks) and reports honestly, and the data dir is re-swept after the shortcuts go. 2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when empty, in both the powershell.exe and drvfs-fallback paths. 3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of ~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock. Verified by running both uninstallers on a real dual install: device fully clean (no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left). * install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04 ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu 24.04. When the installer runs in a newer default distro (e.g. 26.04) it cannot enable the GPU and silently falls back to CPU. If a 24.04 distro already exists, re-run the install there and stop in the current one so the GPU path is taken without the user having to know about the distro requirement. Runs before venv creation so the wrong distro is left untouched, guards against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm distro alone (librocdxg present), and skips the GGUF-only / opt-out / non-Strix cases. When no 24.04 distro exists we keep today's behaviour: continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never auto-downloading a distro. Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the function, rewrites its paths to fixtures, mocks wsl.exe) covering the full decision matrix, wired into tests/run_all.sh. * uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico: install.sh points the WSL shortcut's icon there while the native install owns the dir. The native uninstaller removed the whole dir unconditionally, so uninstalling native while keeping WSL left the WSL shortcut with a blank icon. The old code only avoided this when Explorer happened to hold the icon open, which is unreliable; on a real dual install the dir was deleted and the WSL shortcut went blank. _RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving "Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it removes the dir as before. uninstall.sh still drops the icon and the empty dir when WSL itself is uninstalled, so every uninstall order ends clean. Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and runs it against a temp dir with controlled shortcut dirs) covering the dual, native-only, empty, and missing-dir cases, wired into the windows-inference smoke workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico and the WSL shortcut's icon stays intact. * installer: condense AMD/ROCm code comments (no behavior change) Tighten the comments added for the Strix Halo native+WSL installer work so they are shorter and clearer without losing intent: the venv-internal hipInfo amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and the dual-install uninstall icon handling. Comment-only; code paths unchanged. 107 insertions, 166 deletions across 11 files. * install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration, so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin distro (and ran venv migration in the distro about to be abandoned). Move the reroute ahead of the venv section so the origin distro is left untouched, matching the function's own comment. Behavior is identical on every non-reroute path. * installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04 - install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the flavor-repair block does not retry the failed ROCm index and abort the install; pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too. - install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo. - amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding quotes from PATH entries before probing for hipinfo. - install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04. - uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or another WSL distro) still references it, in both the powershell and drvfs paths. - tests: regression coverage for all of the above. * installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate - install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on --local; run the reroute BEFORE dependency/uv install so the origin distro is left untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later ROCm-on-WSL bootstrap does not install into the unsupported origin distro. - install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real executables match (not an alias/function named hipinfo). - uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs. - tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the user the install continues CPU-only, but only the failed-reroute branch set UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off that flag, so the other three branches could still install ROCm into the unsupported origin distro (e.g. 26.04). Set the skip guard on all of them. Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap carries through to the rerouted 24.04 child instead of dropping to the prompt path. install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty $env:USERPROFILE so Join-Path does not throw on a profile-less service account. Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE guard assertion, and route shell-test fixtures through a single trap-cleaned root. * installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch() installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure could abort the whole install even after install.ps1/setup.ps1 fell back to CPU. Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor maps) and make the retry nonfatal: keep the existing build and let the user re-run update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe. install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred); both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with only 22.04 reaches the GPU instead of staying CPU-only. install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child (PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder. _path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath can't classify every path on the drive as venv-internal (defensive; venv never at root). uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA). Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text + behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: match WSL reroute target by exact distro name, not substring The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests. * install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only) The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as GPU-supported let the parent report a successful reroute while the child fell back to CPU. Drop 22.04 from the supported set and the reroute target list; 24.04 stays the sole target (keeping the exact whole-line distro match). An already-working ROCm on any other version is still left alone by the librocdxg check above. tests: reroute 22.04 cases updated to the 24.04-only behavior; make the "no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes; stop the tauri exit-order check from mis-flagging the reroute helper's [ "$TAURI_MODE" = true ] && ... --tauri one-liner. * installer: tighten comment wording across the Strix Halo install/uninstall paths Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer, clearer lines. Comments and a few docstrings only; no code, logic, or behavior change. Verified with bash -n, the PowerShell parser, and ast.parse, and the installer test suite still passes. * add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them Every shell and PowerShell script under the Studio/installer surface now carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright header (after the shebang where present): the installer (install.sh, install.ps1), build.sh, the .github and src-tauri scripts, the installer test suite, and the moe kernel test. Header-only, line endings preserved; bash -n, the PowerShell parser, and the installer tests all pass. * installer: drop the duplicate AGPL header from install.sh and install.ps1 Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below their usage comment block; the prior header pass added a second one at the top because it only scanned the first few lines. Remove the duplicate so each file keeps a single original header. * installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path () so the genuine CPU-only install stays fast. install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO]) to ask the desktop app to elevate for the target distro; capture the child's exit code and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line) instead of masking it. CLI mode still falls back to CPU on a generic failure. Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases; run_func now preserves the child exit code; force-reinstall assertions for both PowerShell installers. Note: codex's _rr_q apostrophe finding is a false positive -- the helper already emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") }) collapses the single-element array to a scalar string, so @cpuForce splatting enumerated it character-by-character into broken single-letter args (- - f o r c e ...), which made uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag). Test now asserts the array-build form and rejects the if-expression form. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * uninstall: remove the isolated Node.js runtime (~/.unsloth/node) The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting <studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block the delete. Env/custom mode nests node under the custom root, removed with that root. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-22 03:09:08 -07:00
Daniel Han	366937de44	studio: pick a macOS llama.cpp prebuilt that loads on the host OS (#5883) ... Make macOS llama.cpp prebuilt selection host-OS-version aware: skip a prebuilt whose minimum-OS exceeds the host and walk back to the newest release that loads (macOS 26 keeps latest; 14/15 land on a compatible older release). Source-build fallback pins CMAKE_OSX_DEPLOYMENT_TARGET=13.3. CI: binary-load assertion plus a macos-14/15/26 install matrix. No change to Linux/Windows or CUDA selection.	2026-05-31 00:59:25 -07:00
Daniel Han	54a86c3514	ci: route every hf download through xet-tuned stall-retry wrapper (#5476) ... Root cause of the Mac json-images 30 min timeout (run 25950714888 / PR #5430): huggingface_hub>=1.15 deprecated `hf_transfer` and routes every transfer through `hf-xet`. The CI step's unpinned `pip install --upgrade huggingface_hub hf_transfer` jumped to 1.15.0 + hf-xet 1.5.0, the 940 MB mmproj finished in ~21s, then the 3 GB gemma-4 GGUF made it to ~46% and went completely silent for the remaining 29 minutes -- no progress bytes, no error, no exit -- until the job timeout fired. This wraps every CI `hf download` in a new `.github/scripts/hf-download-with-retry.sh`: * Drops the no-op `HF_HUB_ENABLE_HF_TRANSFER=1` prefix and the `hf_transfer` install (both are deprecated on 1.15+ and only emit a FutureWarning now). * Exports the hf-xet high-performance knobs Daniel asked for: HF_XET_HIGH_PERFORMANCE=1 HF_XET_CHUNK_CACHE_SIZE_BYTES=0 HF_XET_NUM_CONCURRENT_RANGE_GETS=64 HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY=0 HF_XET_CLIENT_READ_TIMEOUT=500 * Watchdogs each attempt: if `hf download` has not exited after HF_DOWNLOAD_STALL_SECONDS (default 180s = 3 min), SIGTERM, sleep 2, SIGKILL, then loop. Retries are unbounded; the enclosing job's `timeout-minutes` is the real cap. * Optional 3rd positional `LOCAL_DIR` -- omitted lets `hf` use the default HF_HUB_CACHE, which is what the HF_HOME-priming jobs need. 19 call sites migrated across mlx-ci.yml + 9 studio-*-smoke.yml workflows. The inline `python -c "from huggingface_hub import hf_hub_download; ..."` block in mlx-ci.yml is also routed through the wrapper so every hf transfer in CI gets the same treatment. Also reverts the json-images timeout 45 -> 30 from #5475: the bump was masking this hang, not fixing it.	2026-05-15 21:11:56 -07:00