13 commits

Author	SHA1	Message	Date
Daniel Han	8080c93c9e	Studio: install sd.cpp under the Studio home, not beside it (#8226) ... * Studio: install sd.cpp under the Studio home, not beside it A custom UNSLOTH_STUDIO_HOME put the managed stable-diffusion.cpp tree at the home's parent, unlike llama.cpp, whisper.cpp and node, which all install under the home. For a relative home that parent collapses to the working directory, so a stable-diffusion.cpp checkout sitting there was picked up as the managed install and install_sd_cpp_prebuilt refused to run: the target was a pre-existing non-empty directory without the ownership marker. Derive the root as "studio home"/stable-diffusion.cpp from an absolutised home, in both the installer and the engine's finder, matching default_managed_llama_dir. The legacy default home ~/.unsloth/studio still maps to ~/.unsloth/stable-diffusion.cpp, and a tree an older build installed beside the home is still discovered and still repairable, gated on the ownership marker so an unrelated checkout is never adopted. * Stop the sd.cpp server under a custom root before uninstall deletes it Moving the managed tree under the Studio home left the uninstaller listing only the old sibling location as an owned root. The tree itself still goes, because the custom root is removed wholesale, but a resident sd-server survives unlinking its binary, so it kept running and holding its port while its install disappeared underneath it. List both locations, each still gated on the ownership marker so a checkout the user keeps at either path is never signalled. The Windows script needs no equivalent change: its handle scan already walks every known root by prefix, which now contains the tree. Comments in both scripts updated to describe where the install actually is. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Read the sd.cpp accelerator record from the root the binary is in The accelerator upgrade check read the record from the current managed root while the finder may hand back a binary from the tree an older build installed beside the Studio home. That root holds no record, unrecorded reads as a mismatch for a GPU target, and the matching bundle already on disk gets downloaded again on every load. owning_managed_root() returns the marker-owned root a binary actually lives under, and the check reads the record from there. is_managed_binary is now a thin wrapper over it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Find a legacy install beside a symlinked home, and never serve a stale legacy server after an install - The compatibility lookup resolved the home before taking its parent, while the old installer took the lexical parent. For a home that is itself a symlink the tree an older build created sits next to the LINK, so the lookup missed it: a needless re-download, and the old install left orphaned from the uninstaller too. Lexical first, resolved after, both still marker-gated. - After an install that ships no sd-server, the finder's legacy probe could hand back the old server built for another accelerator, and ensure_sd_server_binary returned it without rechecking. A forced CUDA load then ran the legacy CPU server. It now returns None there, so the router uses the sd-cli the install just landed. * Apply the kwarg-spacing formatter to two files it had not been run over * Clear the legacy sd.cpp beside a symlinked Studio home, and stop reinstalling a serverless bundle Uninstall derived the legacy sibling from the canonicalized custom root, so for a Studio home that is itself a symlink it looked beside the link's TARGET while an older build had installed beside the LINK: that tree was never stopped and never removed. Take the lexical parent as an extra marker-gated candidate, matching what the finder already does. And once the current managed root holds a completed install for the accelerator being asked for, a mismatched sd-server still sitting in that legacy tree is not a reason to install again: the bundle simply shipped no server. Without this, every model load re-downloaded it. * Make the serverless suppression evidence-based, and close two gaps in the uninstall stop pass Four follow-ups on the review of the previous commit. The serverless guard fired on the accelerator record alone, which reads a server that was deleted by hand (or by the runnability repair) as a bundle that never had one, and then suppresses the very reinstall that would put it back. install() now records ships_server, taken off the archive member list, and the guard needs that recorded false plus a genuine mismatch on the legacy side. An unrecorded install stays unknown and keeps its old behavior, and a legacy server that matches the accelerator being asked for is still preferred over the one-shot CLI. The guard also has to run before _accelerator_changed, which reports "unchanged" while the managed tree is in use, so a load starting during a generation was handed the mismatched legacy server. In uninstall.sh, the lexical sibling reached the string-based deny list without being resolved, so a home carrying ".." could aim a removal at a protected tree; canonicalize a copy for that check. And the nested <root>/stable-diffusion.cpp is now stopped even when unmarked, provided the root is a Studio root this run deletes: the current-root finder can select an unmarked binary there, and the marker gate belongs to the paths that survive when unowned, not to a tree that goes regardless. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Memoise the bundle's server capability with the accelerator, not separately An unwritable install record already kept the accelerator in process memory, so a serverless install whose record could not be written read back as server-capable and the load that finds a mismatched legacy server went on reinstalling. Remember both together, or neither is trustworthy. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give the Windows stop scan the physical path of a linked Studio home The backend resolves the home before deriving the nested sd.cpp tree, so on a junction or directory symlink the native binaries run out of the target. _CustomStudioRoots only normalizes the string (System.IO.Path.GetFullPath is lexical and never follows a reparse point), and the scan matches Win32_Process.ExecutablePath by prefix, so the running server was never matched and survived an uninstall that deleted its tree. Add the reparse target to the stop scan only; the deletes still refuse to chase a link out of the expected location. * Scope both uninstall stop paths to trees this run actually owns The lexical sd.cpp sweep skipped the ownership check the canonical loop makes first, so a stale or mistyped UNSLOTH_STUDIO_HOME pointing at a path that was never a Studio could take the marked legacy sibling of a different, valid install. Apply _is_studio_root there too. On Windows the stop scan was handed the whole physical target of a linked home. The delete leaves that target standing, so anything there that is not ours is neither locking nor being removed; pass the Studio-managed subtrees underneath it instead, and only for the homes, since a component dir can itself be a link onto a shared runtime. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-10 07:03:20 -07:00
Nilay	54462b80c7	Fix two macOS installer bugs (#8279) ... * Installer: stop planting a self-referential symlink inside the macOS launcher bundle * Uninstaller: keep the desktop app's data when that app is still installed * Cover the uninstaller's app-data gate in tests * Find the app that owns the bundle id instead of guessing its path for PR #8279 A packaged app kept in a renamed bundle or a subdirectory such as /Applications/AI & ML/Unsloth.app missed all three hard-coded candidates, so the uninstaller still wiped its settings, cookies and WebView storage. Match on CFBundleIdentifier via mdfind plus a scan of /Applications and ~/Applications, excluding install.sh's launcher, which shares the id but only opens a browser. * Drop the depth cap on the bundle scan for PR #8279 A bundle filed deeper than three levels, say /Applications/Development/AI/Local/ Unsloth.app, fell outside -maxdepth 3, so the uninstaller wiped the installed app's data. Prune at each .app instead: the walk never enters a bundle, which is what the cap was avoiding, so any nesting depth is reachable and free. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-09 22:40:14 -07:00
Michael Han	6712c7513d	Uninstall: remove WebView runtime data so reinstalls don't serve a stale frontend (#7360) ... * Uninstall: remove WebView runtime data (stale frontend after reinstall) The desktop app's WebView creates runtime data keyed by the Tauri bundle id (ai.unsloth.studio) at first launch, not at install time, so the uninstallers never removed it. A leftover WebKit/WebView2 cache then serves a stale frontend bundle to the next install, showing old styles after a supposedly clean reinstall. uninstall.sh: remove ~/Library Caches/WebKit/Application Support/ HTTPStorages/cookies/saved state/prefs on macOS and the XDG cache/data/ config/state dirs on Linux; stop the desktop app binary first. uninstall.ps1: remove LOCALAPPDATA (EBWebView profile) and APPDATA dirs for the bundle id; stop the desktop app and any msedgewebview2.exe helper holding handles on them first. Adds tests/sh/test_uninstall_webview_data.sh running the full script against a fixture HOME for both OS branches. * Test: never enter the real WSL cleanup from the webview-data test Stubbing uname alone is not enough on a WSL host: the script's 'grep -qi microsoft /proc/version' probe still fires and the Linux test cases would run the real WSL cleanup against the host's /mnt/* shortcuts and /etc profile. Add a PATH-stubbed grep that fails only the /proc/version probe (delegating everything else to the real grep via an absolute path so the stub can never self-exec), plus no-op powershell.exe and sudo stubs as defense in depth. * Test: sandbox XDG_RUNTIME_DIR and drop the GNU-only mktemp -p * Tighten the comments added for PR #7360 * Final comment pass for PR #7360 * Ignore relative XDG home overrides when removing WebView data * Final comment pass for PR #7360 * Scope the Studio process stop to the caller's session for PR #7360 * Scope the Studio app kill to the target user for PR #7360 * Scope by owning account, run defaults as the home owner, and de-vacuum the kill assertion for PR #7360 * Resolve a symlinked home and stop claiming removal succeeded for PR #7360 * Report failed removals in the PowerShell summary and across the custom-root subshell for PR #7360 * Count a deny-listed custom root as incomplete removal for PR #7360 * Remove the deep-link handler .desktop entry on uninstall * Only claim chat history is gone when a studio.db was actually removed * Refresh the desktop database in the directory the handler was removed from * Count a deny-listed custom root as incomplete cleanup in uninstall.ps1 * Harden the uninstall summary markers and the WebView2 helper sweep * Keep the uninstall summary conservative when it cannot account for a removal * Confirm the database is really gone before the summary says so * Stop claiming the uninstall removed provider API keys * Resolve a symlinked studio.db without relying on readlink -f * Scope the signed-out claim and anchor relative database links * Stop the legacy-named desktop process on Windows too * Anchor a relative install-root link before testing for the database * Tighten the comments in the WebView cleanup paths --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com>	2026-08-07 06:08:28 -07:00
oobabooga	df5f139bac	Studio: add image generation, editing workflows and LoRA training with Unsloth GGUFs (#6763) ... * Tighten comments in the image stack tests and scripts * Close video single-file, training reservation, and image mount-resume gaps Route on-device single-checkpoint video folders through the single_file loader: a bare local .safetensors directory (no model_index.json) is advertised as a pipeline with no filename, so validation rejected it before it could load. Reinterpret the pick as a single_file load of the sole checkpoint, mirroring the image load route. Treat a reserved-but-not-yet-spawned LLM training start as active in is_training_active() so /images/load, /video/load, and /diffusion/start cannot race the reserved run for VRAM during the pre-spawn free window. Mirrors the diffusion training service reservation. Resume an in-flight image generation on the Images page mount: probe generate-progress, re-enter the poll loop, and refresh the gallery on completion so a run started elsewhere is reflected and its saved image appears without a manual refresh. Seed resident image defaults from the resolved base_repo rather than a possibly path-shaped repo_id so the first resident generation uses the right recipe. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Publish image generation active state before pre-denoise setup generate() assigned self._gen only at the pipe() call, after deferred compile, LoRA resolution/application, and ControlNet download/build had run. Across that setup window generate_progress() reported inactive even though _generate_lock was held, so a reloaded page's mount probe showed idle and let a second generate queue behind the first. Publish an active step-0 _GenState the moment the generation lock is acquired, before the setup work, and clear it in the outer finally so a setup-time error cannot leave the UI stuck active. Mirrors the video backend's queued phase and the training start guard. * Studio: fix diffusion install ownership, dataset upload atomicity, gallery pagination, and teardown races install_sd_cpp_prebuilt: only write the .unsloth-studio-owned marker when the install created the target directory or it was empty. Adopting a pre-existing, unowned, non-empty directory (a user's own stable-diffusion.cpp checkout) made it eligible for the uninstaller's recursive delete. routes/training upload: make the multi-file promotion transactional. Back up each displaced original and roll every destination back on any failure, so a mid-loop rename error can no longer partially overwrite the live dataset. routes/training _resolve_dataset_folder: reject a symlinked dataset directory and prove the resolved folder stays under the datasets root, so image read/caption/delete cannot escape the root through a link. routes/training delete: escape glob metacharacters in the thumbnail filename so deleting an image named like [ab].png removes only its own thumbnails. image_gallery / video_gallery listing: filter records against the response schema inside the pager via a valid callback, so offset/limit/has_more all count over accepted records. A leading schema-invalid record no longer returns an empty page with has_more=true and stalls infinite scroll at offset 0. image_gallery / video_gallery save: publish via a temp file plus atomic rename (the sidecar is the video pair's commit marker) and clean up on failure, so a partial write never surfaces a truncated PNG or strands an orphan MP4. diffusion_train_common discovery: treat an empty caption sidecar as a metadata tombstone that still falls through to the dreambooth instance prompt, so clearing every metadata caption no longer fails with no captioned images found. diffusion backend unload: wait for an in-flight denoise to exit before tearing down process-wide patches and state, mirroring the load path. diffusion_engine_router: serialize the whole check/unload/publish transition so a concurrent selection cannot return the engine being unloaded. uninstall.ps1: gate the default sd.cpp process stop on the owner marker so a user's own sd-server is not terminated for a directory we then keep. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject native batch seeds outside the JSON-safe range * Refuse sd.cpp install into unowned non-empty target dir When the install target already exists, is non-empty and lacks the .unsloth-studio-owned marker (a user's own stable-diffusion.cpp checkout, or unrelated files beside a custom Studio root), install() previously still extracted the release into it. Skipping the ownership marker only stopped the uninstaller from deleting the directory; extraction still merged binaries into the user's working tree and could overwrite same-named files. Fail up front with a clear message pointing the user at a fresh/empty location before any download or extraction, leaving their directory untouched. Update the ownership test suite to assert the refusal. * Studio: gate dataset uploads on the symlink check and surface local video single-file checkpoints * Tighten comments and docstrings added by the image-generation fixes * Studio: close arbiter load-registration race and surface native progress + local pipeline folders Publish native sd.cpp generate progress (_gen) before LoRA resolution so a reload probe reads active during setup, matching the diffusers path. Register the diffusion/video GPU load under the arbiter lock (acquire_for now takes a register callback) so a competing acquire cannot evict an owner before its load is marked in-flight and let two loaders allocate VRAM at once. Admit local diffusers pipeline folders (root model_index.json, weights in component subdirs) in the local model scan so they reach task tagging and the On Device picker. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: include marker-owned custom sd.cpp roots in the uninstall stop scan; harden Pester install against the nuget.exe PSGallery bootstrap * Studio: surface local pipeline scan roots, tag single-file checkpoints by filename, mark companion-only pipelines partial - _scan_models_dir: admit a scan folder that is itself a diffusers pipeline (root model_index.json, weights in transformer/ vae/ subdirs). _is_model_directory rejects such a root, so the child scan would list the component subdirs as bogus models and hide the real pipeline; treat the root as one model via _local_pipeline_index. - _local_is_diffusers / _local_model_task: include the sole checkpoint filename in the family-detection needles (_local_family_needles, resolved via resolve_local_single_file). A generically named folder holding one loadable qwen-image-*.safetensors / ltx-*.safetensors identifies its family only from the filename; the load route already resolves that file, so tag it or the task-scoped picker (which rejects task=null) hides the on-device model. - list_cached_models: mark a companion-only base snapshot partial. A GGUF image load prefetches the base repo's VAE / text-encoder / model_index.json but skips the transformer (the GGUF supplies it); the snapshot has a pipeline manifest yet is not a loadable BF16 pipeline, and _cached_repo_partial misses it. _repo_pipeline_missing_denoiser flags a pipeline snapshot whose transformer/ or unet/ component carries no weight, so the picker drops it instead of advertising it as fully on-device. * Studio: preserve foreign gallery files, force safetensors on remote ControlNets, and close dataset/seed/GPU gaps Gallery clear/delete now scope to Studio-owned files: image_gallery and video_gallery skip PNGs / MP4s without a readable recipe (a hand-dropped or orphan file the listing already hides), so clear() and a guessed-id delete no longer destroy files the gallery never surfaced. Remote ControlNets now force use_safetensors: a bare owner/name reaches from_pretrained without the base trust gate, and the Hub scan fails open when unavailable, so requiring safetensors closes the pickle deserialization vector. POSIX uninstall now stops resident sd-server / sd-cli under an owned sd.cpp root before removing the tree (marker-gated), mirroring the Windows stop-before-delete scan; a live native server no longer survives unlinking its binary. Diffusion dataset containment: the training-start read path and the discovery picker route bare names through the protected resolver, so a symlinked dataset is rejected / not advertised like the caption/delete routes already do. Uploads gain the inference decode guard (oversized real images 400 before OOMing the trainer) and dataset upload/caption/delete/import are blocked with 409 while a diffusion run is active. JSONL readers (trainer + routes) tolerate non-object JSON and invalid UTF-8 instead of raising AttributeError / 500. LoRA family compatibility is enforced in the shared resolver, not only the picker, so a direct API client cannot apply a mismatched-family adapter. GPU arbiter gains release_if so the image/video unload idle-check and release are atomic against a concurrent same-owner load's registration. Native batch recipes persist the base batch_seed and restore replays from it, so a native batch_index>0 image no longer advances its seed twice. FLUX.2-klein selects its sd.cpp text encoder by variant (4B -> Qwen3-4B, 9B -> Qwen3-8B) instead of the single family default. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: track the loaded GGUF filename so native companion resolution reproduces the load identity * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate local-pipeline image tagging on a real family; validate video sidecars before delete/clear * Studio: tighten image-generation fix comments and docstrings * Studio: gate gallery serve/export on ownership; keep image progress active until persisted; reserve diffusion training before the dataset scan * Studio: restore Reapply target on async image/video load errors; recheck training state before dataset commit; reclaim partial sd.cpp installs on retry Images/Video: a background model load that fails AFTER starting (error/eviction during download) leaves the previous pipeline resident, but handleLoad had already overwritten lastLoad.current with the failed pick, so "Reapply to loaded model" reloaded the failed model. Carry the prior Reapply target into the poller and restore it on the async error/null paths, mirroring the quant rollback. Training: an in-flight diffusion dataset upload passed _require_diffusion_dataset_mutable() at entry but could still commit files after a concurrent /diffusion/start reserved the training slot, mutating the dataset underneath the trainer. Re-check the interlock immediately before the commit phase; a 409 there leaves the staged temps for the finally to clean. sd.cpp install: an interrupted extraction (disk full, killed process, a raising post-extract cudart fetch) left the target non-empty with no owner marker, so the next lazy install tripped the "not a Studio-managed directory" refusal and wedged native install. Write the ownership marker before the partial writes when the target is reclaimable, so a retry recognises the debris as ours and re-extracts. * fp8 DiT quant: floor the dynamic activation scale with activation_value_lb An all-zero activation token row makes the dynamic per-row fp8 scale 0, which turns the quantized data to NaN and the render to black frames on torchao's plain-torch kernel path. The fused fbgemm/mslk quantize kernels clamp zero rows internally, so the bug only reproduces on machines without them, which is most user environments. Zero rows are real inputs, not a corner case: Wan 2.2 zero-pads its text conditioning, and Hunyuan-1.5 and Qwen-Image regenerate zero rows inside their transformer blocks every step. Pass activation_value_lb=1e-12 to Float8DynamicActivationFloat8WeightConfig whenever the installed torchao supports the kwarg (Float8Tensor rework, 0.13+), checked via inspect.signature so older torchao keeps exactly the current behaviour; the existing Float8MMConfig fallback chain is unchanged. Verified on GPU: with the forced plain-torch kernel path a zero-row input NaNs without the floor and stays finite with it, and end to end on HunyuanVideo-1.5 fp8 goes from a solid black frame (LPIPS 1.00) to a normal render (LPIPS 0.225); on Wan the floor matches the condition_embedder exclusion (LPIPS 0.211 vs 0.206). Same-seed renders with fused kernels present are unaffected, and pre-quantized fp8 checkpoints stay valid since weight scales are untouched. * Wire hosted pre-quantized DiT checkpoints into the image families Point prequant_repos for flux.1, flux.2-klein, flux.2-dev, qwen-image (int8 only there; fp8 is family-denied), z-image and krea-2 at the unsloth/<Model>-FP8 Hub repos carrying gate-validated int8 and fp8 transformer checkpoints, so the fast quant path loads the small pre-quantized file instead of materialising the dense bf16 transformer and quantising on device. Measured on FLUX.2-dev int8: build peak drops from 60.7 GB (dense + quantize) to 30.7 GB (hosted prequant), identical 30.7 GB resident after either path since loading a checkpoint is bit-identical to on-the-fly quantisation. The hosted repos name files <Model>-<SCHEME>.pt, so resolve_prequant_source now derives that model-name filename from the repo id (scheme suffix stripped case-insensitively) and carries the legacy transformer_<scheme>.pt as a fallback the resolver tries when the primary 404s, keeping older repos loadable. Wiring a repo also exposed a fallback hazard: with a prequant source present, the dense-fit preflight used to be skipped entirely, so a failed prequant download would fall through to the dense bf16 load the memory plan never budgeted, OOMing after eviction. The preflight now always runs and gates an allow_dense_fallback flag through _load_dense_quant_pipeline: a dense misfit still skips the fast path when no prequant exists, but with one it proceeds and a prequant failure raises to the GGUF build instead of loading dense. The same flag is set when the auto-policy replans an offloaded GGUF against a prequant-sized transient. Tests updated to the new filename convention plus new coverage for the derivation and the legacy-name fallback; the prequant-skips-refit test now asserts the re-check runs and forbids the dense fallback. Verified end to end on GPU: z-image int8 resolves the hosted repo, downloads the model-name file and renders (6.8s load, 5.9 GB peak). * Route krea-2 through its per-component loader on the transformer-quant fast path _assemble_pipe used Pipeline.from_pretrained for every family, but the krea repo ships transformers-5.x configs and no top-level tokenizer files, so the tokenizer dies with vocab_file=None. The pre-quantized checkpoint loaded fine and then the assembly crashed, dropping the load to the GGUF build, which krea-2 cannot take (Krea2Transformer2DModel has no from_single_file). Assemble per-component via load_krea2_pipeline like the pipeline-kind and single-file paths already do. Verified live: Krea-2-Turbo int8 and fp8 hosted prequant loads now assemble and render through the Studio images tab. * Keep Qwen-Image's text-stream linears bf16 on int8 (short prompts break torch._int_mm) Qwen-Image's MMDiT runs every text-stream Linear at M = actual prompt tokens: the Qwen2.5-VL embeds are not padded to a fixed length like FLUX's 512-token T5. A short prompt (13 tokens) or the near-empty negative prompt drives torch._int_mm below its M > 16 floor and the first denoise step raises 'self.size(0) needs to be greater than 16, but got 13' (measured on B200 through the Studio images tab). Add per-family int8 exclusions (txt_in, add_q/k/v_proj, to_add_out, txt_mlp) for qwen-image and qwen-image-edit, threaded through exclude_tokens_for_scheme(scheme, family) and the prequant checkpoint validation, so a checkpoint baked under the old token list is rejected and re-quantised instead of loaded crashing. The text stream runs at M = tens vs the image stream's M ~ 4k, so the exclusion costs nothing; the rebuilt hosted checkpoint gates 28/28 PASS with LPIPS mean 0.057 (was 0.069). * Harden the diffusion memory plan against transient free-VRAM undercounts A cold FLUX.2-dev int8 load on an idle 183 GB B200 planned offload=model (companions exceed budget) and silently served the GGUF as-is; the identical retry went resident and engaged the hosted prequant. The plan arithmetic was byte-identical across both loads (required 90,228 MiB, resident needs free of about 124 GB); the only divergent input was torch.cuda.mem_get_info, which is device-wide and instantaneous: a transient foreign CUDA context briefly held about 100 GB at the first snapshot, and the planner trusted that single read. Three changes: - settled_snapshot_device_memory: on cuda, synchronize + empty_cache (best-effort) and take the MAX free over up to 3 spaced reads. A transient can only shrink free, so the max rejects transient undercounts while a persistent tenant still caps every read. _plan_memory now uses it. - plan_fits_total_capacity + one replan retry: when the dense/prequant candidate fits TOTAL device capacity under the standard reserve and the 0.85 resident margin, an offload verdict can only stem from the free reading, so the loader re-snapshots and replans once before declining the fast path. Explicit balanced/low_vram modes skip the retry (they offload by mode). - diffusion.transformer_quant_declined log line with required/budget/free and the plan reasons, so the next decline is diagnosable from the server log (previously silent). Verified: cold FLUX.2-dev int8 first load in a fresh server now engages the hosted prequant resident (offload=none). * Add FLUX.2 Klein and FLUX.2-dev DiT LoRA training Register flux.2-klein and flux.2-dev in the DiT trainer following the upstream DreamBooth references: latents train patchified and batch-norm normalized from the VAE posterior mode, the packed forward reuses step-invariant position ids, and the guidance vector (3.5) is gated on the variant's guidance_embeds config. Conditioning stacks load per variant (Mistral via Flux2Pipeline for dev, Qwen3 via Flux2KleinPipeline for Klein) and are encoded and freed before the transformer lands on the device. The fused single-stream to_qkv_mlp_proj joins the attention projections in the LoRA targets; the single-stream out projection stays dense because its to_out suffix would also match the double-stream ModuleList container. Wire both families through the training registry (family set, labels, VRAM notes, rank 16 / lr 1e-4 defaults, bf16-only preflight), mark them trainable with train base repos in the family registry, add FLUX.2-dev to the gated-repo token check, and trust both official bases for training downloads. Verified on B200: 30-step klein int8 (19.6s) and nf4 (20.9s) and dev int8 (52.0s) runs train with finite decreasing loss and the saved adapters apply on the bf16 base pipeline (weight 0 reproduces the base image exactly, weight 1 visibly restyles it). * Support LoRA adapters on torchao int8/fp8 quantized image pipelines Adapters are baked at load time: they attach to the dense transformer, then quantize_ converts only the frozen base linears (the lora_ side path is excluded by name), then the loader compiles. Post-quant PEFT injection is not possible on a manually quantized module, so the prequant shortcut is skipped for a baked load and the memory plan is sized for the dense build (force_dense on the quant candidate). At generation time the baked topology is frozen: weight tweaks and disabling (scale 0 reproduces the quantized base exactly) go through set_adapters, while adding or removing adapters returns a clean 400 telling the client to reload with the new selection. supports_lora now returns True for int8/fp8 diffusers loads (checked before the gguf-kind early return, since the quant fast path keeps the picker kind); nvfp4/mxfp8 and GGUF-via-diffusers stay blocked. The load request model takes an optional loras list, threaded through begin_load on both engines (native ignores it and keeps applying LoRA at generation). Verified end to end on GPU: Z-Image GGUF picker + int8 + trained adapter loads through the API, bake marker logged, weight 1.0 vs 0 renders differ visibly, weight 0.5 accepted live, unknown adapter rejected as 400. Affected suites: 296 passed. * Add FLUX.1 Krea dev to the image model catalog Krea's guidance-distilled FLUX.1-dev finetune keeps the exact dev layout, so it runs under the existing flux.1 family unchanged. Wire it up end to end: - Catalog group with the gated official bf16 pipeline and the open QuantStack GGUF quants; the gated artifact is skipped on auto-routing when undownloaded. - Trust the official repo for non-GGUF from_pretrained loads, next to the other black-forest-labs bases. - Generation defaults: 28 steps at guidance 4.5 per the model card. The generic "krea" defaults key (Krea-2-Turbo's 8-step no-CFG recipe) used to swallow the id, which would have produced garbage output; the new flux.1-krea key precedes it on both the backend table and the images page table. - The flux.1 prequant checkpoints are schnell-based; the loader's baked base_model_id validation refuses them for the Krea-dev base, so int8/fp8 requests dense-quantize instead (covered by existing prequant tests). * Resolve pre-quantized checkpoints per base variant One family entry covers several published variants whose weights differ (flux.1: schnell, dev, Krea-dev), but prequant resolution was keyed on (family, scheme) alone, so only the default base could ever be served: the loader's baked base_model_id validation correctly refused the schnell checkpoint for dev and Krea-dev bases and every such load paid the dense download plus on-the-fly quantise. Add an optional prequant_variant_repos table on DiffusionFamily as (base_repo, scheme, repo_id) triples and thread the resolved base repo through resolve_prequant_source / usable_prequant_source and their three call sites (load fast path, memory-plan probe, auto-policy candidate). A base without its own entry keeps returning the family default, preserving the existing refuse-then-dense behavior exactly. Wire the flux.1 variants: the gate-validated unsloth/FLUX.1-dev-FP8 checkpoints (built in the earlier campaign but never reachable) and the new unsloth/FLUX.1-Krea-dev-FP8. * Add the Lumina Image 2.0 family to the image catalog Alpha-VLLM/Lumina-Image-2.0 is a 2.6B single-stream DiT with a Gemma2-2B encoder and a standard 16-channel VAE, all transformers-4.x-compatible, so the generic from_pretrained pipeline path loads it as a new lumina-2 family: - Family entry (Lumina2Pipeline / Lumina2Transformer2DModel), aliased to lumina-image-2.0 / lumina-image-2 / lumina2. No bare lumina alias: Lumina-Next checkpoints are a different arch and must stay unknown rather than crash mid-load. bf16-only upstream, so the fp16 fallback stays off like z-image. - Trust the official repo for non-GGUF loads; bf16 component table entry (ships fp32, ~5.2 GB transformer + 5.2 GB encoder bf16-resident). - Generation defaults 50 steps / guidance 4.0 per the model card, and the generate call passes the card's cfg_trunc_ratio=0.25 itself (family-gated, signature-gated): the pipeline default (1.0) runs the CFG double-forward on every step and oversaturates output. - Catalog group with the single ungated bf16 pipeline artifact (11 GB resident) plus routing assertions; images page defaults row. - No GGUF artifact: none exists upstream (only finetune/LLM quants), so the dense transformer_quant fast path (GGUF-kind-only) stays unreachable for now. Offline probes of the future prequant campaign: int8 and fp8 both engage and render cleanly (fp8 LPIPS 0.11 vs bf16, int8 0.33 from 50-step trajectory drift with intact quality), so neither scheme is family-denied. * Wire the hosted Lumina Image 2.0 int8/fp8 checkpoints Gate-validated against same-seed bf16 renders (28/28 pairs per scheme, zero failures): int8 LPIPS mean 0.146 / SSIM 0.937, fp8 LPIPS mean 0.116 / SSIM 0.946. Uploaded to unsloth/Lumina-Image-2.0-FP8 following the existing checkpoint repo conventions. * Add the HunyuanImage 2.1 family to the image backend The hunyuanvideo-community diffusers mirror carries the full stack in standard layout: a 17B dual-stream DiT (32.5 GB bf16), a Qwen2.5-VL text encoder, a ByT5 glyph encoder, the 32x HunyuanImage VAE, and guider/ocr_guider components (AdaptiveProjectedMixGuidance) that diffusers 0.39 loads natively, so the generic from_pretrained pipeline path covers everything with no per-component assembly. Family notes: - The call's guidance knob is distilled_guidance_scale (there is no guidance_scale kwarg), so cfg_kwarg routes the UI value there; real CFG runs inside the repo's guider at its baked scale. Defaults follow the card recipe: 50 steps, 3.25. - 2K-native: verified live at both 1024 and 2048. - Coexists with the HunyuanImage-3.0 structured exclusion (3.0 has no diffusers pipeline and stays excluded with its stated reason). - int8/fp8 dense quantization verified live (LPIPS 0.186 both vs same-seed bf16); a short prompt does not trip the int8 torch._int_mm minimum on this arch, so no family exclude entry is needed. - bf16 component table for the memory planner: (32.5, 16.3, 0.8) GB. * Surface HunyuanImage 2.1 in the image model catalog Catalog group with the open bf16 mirror pipeline (~50 GB resident, so a bare click on a consumer card routes to the QuantStack GGUF quants, which load and render through the generic GGUF path, verified live) plus the images page defaults (50 steps, guidance 3.25 feeding distilled_guidance_scale). * Add the HiDream-I1 family to the image backend A 17B MoE DiT (16 double + 32 single layers, 4 routed experts) with four text encoders, on HiDreamImagePipeline (diffusers 0.39). One family covers the open Full / Dev / Fast repos (same arch); per-variant generation defaults follow the upstream inference recipes (Full 50 steps at guidance 5, the distilled Dev 28 and Fast 16 guidance-free). The repos name a Llama-3.1-8B text_encoder_4 in their model_index but do not ship its weights; the official example passes the gated meta-llama repo in by hand. The loader instead assembles the component from the open unsloth mirror (byte-identical weights, already inside the non-GGUF trust gate), injected at the three pipeline from_pretrained sites, with output_hidden_states matching the official example. Memory planning counts the assembled TE4: 34.2 GB DiT + 28.8 GB encoders, ~63 GB bf16-resident. * Surface HiDream I1 in the image model catalog One catalog group with the three official bf16 pipelines (Full, plus the Dev and Fast distillations as labeled artifacts) at their ~63 GB resident size, so auto-routing keeps this a datacenter-GPU pick. city96's GGUF is deliberately not wired: the GGUF path would need the same Llama TE4 assembly for very small demand. Images-page defaults mirror the backend table with the variant keys ahead of the generic hidream key. * Pin the measured HiDream quant verdict in tests int8 and fp8 both engage and render cleanly on this family, including short prompts on int8: the routed MoE expert Linears only ever see the concatenated image+text stream (M far above the torch._int_mm minimum), so no deny entry and no family exclude tokens are warranted. Assert that so a future table edit cannot silently regress the measured behavior. * Wire the hosted HunyuanImage 2.1 int8/fp8 checkpoints Verified bit-identical to on-the-fly quantize: all 1264 state dict tensors (456 quantized) dequantize equal between the loaded checkpoint and a fresh quantize_ pass, so quality matches the runtime Dtype path exactly. Same-seed LPIPS suite means (0.35 int8 / 0.28 fp8) blend trajectory divergence with this family's own run-to-run nondeterminism (identical weights and seed reproduce a 17/255 mean pixel delta through the 50-step guider pipeline); per-case hard checks pass and the drift is compositional, reviewed visually. Uploaded to unsloth/HunyuanImage-2.1-FP8. * Fix silent LoRA drop and wasted transformer prefetch on GGUF quant loads Two live-test findings on the images load path: - transformer_quant with baked LoRAs, when the dense quantized build is declined for memory or fails: the load completed as a plain GGUF with the adapters silently dropped (HTTP success, supports_lora=false after the fact) -- wrong output with no signal. The load now fails with the recovery options (drop the adapters, free VRAM, or pick a smaller model). Weight-0 adapters still count as no bake request, and the plain no-LoRA decline keeps its silent GGUF fallback. - A fresh GGUF load on a small GPU prefetched the base repo's full bf16 transformer shards (~47 GB on Qwen-Image) because the dense-quant prefetch widening only checked scheme viability, not whether the device could ever hold the candidate resident. Gate the widening on total device capacity (reserve + 0.85 margin, the plan_fits_total_capacity bar) so a card that is certain to decline the dense build never pays the download; capable devices keep the prefetch. * Fix video progress under-reporting during load and generate Two live-test findings on the video progress endpoints: - load-progress downloaded_bytes froze mid-download: the counter used scan_cache_dir, which skips in-flight *.incomplete blobs, so it sat at the last completed blob for the whole multi-GB shard pull while the disk kept filling. Count the repo's cache directory directly (completed plus incomplete blobs, snapshot symlinks skipped so nothing is double-counted). - generate-progress reported total_steps=null / fraction=0 while step advanced: the video API only carried the native total field while the image API exposes total_steps and fraction, so one poller could not work against both. Derive the image-compatible aliases in generate_progress and declare them on the response model; the native total stays for back-compat. * Wire the hosted HiDream I1 int8/fp8 checkpoints Gate-validated: all 28 per-case pairs pass per scheme (LPIPS suite means 0.291 int8 / 0.278 fp8, in the 50-step trajectory-divergence band; CLIP delta means 0.007-0.008), and the int8 checkpoint is verified bit-identical to on-the-fly quantize across all 1615 state dict tensors (1073 quantized, max abs diff 0.0). Uploaded to unsloth/HiDream-I1-Full-FP8. * Add a pre-cast text-encoder loader for the layerwise fp8 scheme The runtime text_encoder_quant=fp8 path downloads the full bf16 text encoder and layerwise-casts it in place on every fresh load. For the heavyweight encoders (LTX's Gemma3-27B ~50 GB, FLUX.2-dev's Mistral-24B ~48 GB, Qwen-Image's Qwen2.5-VL ~16.6 GB) that download dominates load time on a fresh machine. diffusion_te_prequant.py loads a pre-cast fp8-storage state dict instead: meta-init the encoder skeleton from the checkpoint's te_class, load_state_dict(assign=True), rebuild on CPU if non-persistent buffers stay on meta, then re-apply the same layerwise cast to install the upcast hooks. The cast is a deterministic storage transform, so the loaded encoder is bit-identical to dense-load-then-cast by construction. v1 hosts the layerwise fp8 storage scheme only: its state dict is plain tensors (torch.load(weights_only=True), no pickle execution). The dynamic-compute schemes (fp8_dynamic, int8, nvfp4) build torchao subclass wrappers at runtime and are deliberately not hosted. Checkpoints validate format, scheme, component and base_model_id before use and any problem falls back to the dense download and cast. Local path overrides reuse the DiT prequant allowlist env var. Families opt in via a new te_prequant_repos (scheme, component, repo_id) field on both DiffusionFamily and VideoFamily; the field defaults empty so nothing changes until a gate-validated artifact is wired. * Inject hosted pre-cast text encoders during pipeline assembly Wire te_prequant_pipe_kwargs into the three pipeline assembly sites: the diffusion full-pipeline branch, the diffusion transformer-only and GGUF branch (where the companion TE is the big remaining download), and the shared video assembly path before the pipeline/component split. Injection is gated exactly like the runtime cast (mode normalized to fp8, device supported, family not denied), so it can never engage where quantize_text_encoders would not; the later quantize_text_encoders call re-applies the cast idempotently and keeps status reporting truthful. With no hosted checkpoint configured the call returns {} and assembly loads the dense encoder as before. * Add the pre-cast text-encoder checkpoint builder Applies the runtime layerwise fp8 storage cast to a model's dense text encoder once and saves the cast state dict with baked metadata (format tag, base_model_id, family, scheme, component, te_class, versions) in the layout diffusion_te_prequant.py validates. Resolves the encoder class from the checkpoint's config.architectures so the recorded te_class matches what the pipeline instantiates. CPU-runnable: the cast touches storage dtypes only. * Test the pre-cast text-encoder load path Hermetic CPU coverage for diffusion_te_prequant: the checkpoint filename convention, family-table resolution by scheme and component with malformed entries skipped, resolution priority (path override, hosted repo, none) and the fp8-only scheme gate, the checkpoint validation matrix (wrong format, missing state_dict, wrong scheme, wrong component, wrong or missing base_model_id) with base case folding, the local-path allowlist refusal and missing-file fallback, and the assembly injection gating (mode, hosted entry, device support, family deny, load failure, successful injection). Also pins the te_prequant_repos field on both family dataclasses and that no family ships a hosted TE checkpoint until the campaign wires one. * Fix pre-cast TE checkpoint loading and engagement reporting Two bugs found while building the hosted checkpoints: - The builder recorded torch.__version__ (a TorchVersion object) in the checkpoint metadata, so torch.load(weights_only=True) rejected every artifact and the loader silently fell back to the dense download. Record plain strings. - Re-applying the layerwise fp8 cast to an injected pre-cast encoder raised on the duplicate hook registration, making quantize_text_encoders report the engaged cast as failed (status showed no TE quant while the encoder ran fp8). _cast_fp8 now returns early when the hooks are already installed. Also corrects the LTX TE size note: Gemma3-12B stored fp32 (~49 GB), not 27B. * Wire the hosted pre-cast fp8 text encoders qwen-image and flux.2-dev (diffusion) and ltx-2 (video) now resolve a hosted pre-cast fp8 text encoder from their unsloth -FP8 repos: - unsloth/Qwen-Image-FP8: Qwen2.5-VL-7B, 16.6 GB dense -> 8.8 GB - unsloth/FLUX.2-dev-FP8: Mistral-Small-24B, 48.0 GB dense -> 24.7 GB - unsloth/LTX-2-FP8: Gemma3-12B, 48.7 GB fp32 store -> 13.2 GB Every checkpoint verified bit-identical to dense-load-then-cast (729 / 585 / 1066 tensors, zero mismatches) and smoke-tested through the real backends with the repo engagement marker. Tests cover the wired entries, the resolver filenames, builder metadata weights_only survival, and the idempotent re-cast. * Report the compute dtype on fp8-cast encoders and inject the pre-cast TE on the dense fast path Two more findings from the hosted-TE GPU smokes: - Module.dtype reports the first floating parameter, which after the layerwise fp8 cast is the fp8 STORAGE dtype. Flux2 derives its prompt embed and latent dtypes from encoder.dtype and feeds them to randn_tensor, which has no fp8 kernel, so ANY flux.2 load with text_encoder_quant=fp8 crashed at generation (pre-existing, runtime cast included). The cast now swaps in a subclass whose dtype property reports the compute dtype; forward behaviour is unchanged. - The dense transformer_quant fast path assembles companions through _assemble_pipe, which never received the pre-cast TE injection, so the hosted encoder engaged on full-pipeline and GGUF builds but not on the fast path. Threaded through like the other two branches. Verified live on B200: qwen-image (full pipeline), flux.2-dev (GGUF picker with int8 DiT prequant), ltx-2 (video backend) all engage the hosted TE, render non-black, and report text_encoder_quant=fp8 truthfully. * Key the fp8 cast idempotency on an explicit completion marker Hook presence alone cannot distinguish a legitimately pre-cast text encoder from leftover hooks after a cast that failed mid-pass, so the early return now requires the completion marker _cast_fp8 sets once the hooks are fully installed. Leftover partial state keeps failing closed. Also tolerates non-Module encoder doubles in the hook probe and the dtype override. * Extend the fp8 TE quant to HiDream's Llama text_encoder_4 The generic quantize_text_encoders pass only covers text_encoder.._3, so HiDream's HEAVIEST encoder (Llama-3.1-8B TE4, 16.1 GB bf16) always stayed dense. TE4 is assembled separately (hidream_te4_kwargs), so the fp8 path now lives there: when the requested TE quant is layerwise fp8 and the device/family qualify, TE4 prefers the hosted pre-cast checkpoint (unsloth/HiDream-I1-Full-FP8, 8.6 GB) and falls back to dense-load-then- cast; a mid-pass cast failure reloads a fresh dense encoder instead of shipping partial state. The pre-cast loader and builder gain config_subfolder/config_overrides for standalone encoder repos whose config sits at the root and whose pipeline needs forward flags (output_hidden_states/attentions). Verified on B200: bit-identity 291 tensors (225 fp8, 0 mismatches), hosted checkpoint engages through the real backend (marker + status fp8), load 24.3 s vs 48.0 s dense, LPIPS 0.133 mean over 3 same-seed pairs vs the dense-TE render (gate 0.25), non-black frames. * Correct the ltx-2 resident TE estimate to the bf16 cast size The memory plan's bf16_components_gb held 50.4 GB for the LTX text encoder, which is the fp32 hub store of Gemma3-12B (~49 GB download), not what sits on device: the pipeline loads it torch_dtype=bf16, ~24.4 GB resident. The 26 GB over-estimate pushed the auto plan toward offload on cards that fit the real footprint. Comments and the size-table test now pin the resident semantics. * Host pre-cast fp8 text encoders for four more families Round 2 of the hosted TE set, each bit-identical to dense-load-then-cast and gated through the real backend (marker + status fp8 + same-seed LPIPS vs dense TEs): - FLUX.1 T5-XXL (text_encoder_2): 9.52 -> 5.90 GB, one artifact for schnell/dev/Krea-dev (T5 shards byte-identical across all three, verified sha256). 220 tensors, 144 fp8, LPIPS 0.109. - Lumina Gemma2-2B: fp32 hub store 10.46 -> 3.20 GB (3.3x download cut). 288 tensors, 182 fp8, LPIPS 0.041. - Z-Image Qwen3-4B: 8.04 -> 4.41 GB. 399 tensors, 252 fp8, LPIPS 0.112. NOT shared with flux.2-klein-4B: klein retrained layer 35's MLP (verified tensor diff, maxdiff 0.86), so klein hosts no entry. - Krea-2 Qwen3-VL-4B: 8.88 -> 4.83 GB. 713 tensors, 460 fp8, LPIPS 0.082. The constructor-assembled krea pipeline takes the encoder directly (load_krea2_pipeline text_encoder kwarg); the loader remaps 5.x rope_parameters and re-ties weights after assign so the rebuilt encoder matches the builder's structure. HunyuanImage 2.1 reuses the Qwen-Image artifact outright: its Qwen2.5-VL text encoder is byte-identical (every shard sha256, 16,584,414,544 bytes), recorded in the new component-level base-equivalence table the checkpoint validator consults. The injection loop now covers text_encoder.._3 so a family can host several components. Live check: LPIPS 0.123 vs dense. * Report the fp8-cast compute dtype without swapping the encoder class The dtype override swapped encoder.__class__ to a dynamic subclass, which breaks transformers' kwargs-based output recording: a fp8-cast Qwen3VLModel stopped returning hidden_states and every krea-2 generation with text_encoder_quant=fp8 crashed at encode_prompt (regression from the HiDream TE4 change; caught by the krea hosted-TE live smoke). The override is now a property shadowed on the ORIGINAL class that prefers a per-instance compute-dtype attribute, so class identity is preserved and uncast instances keep the stock behaviour. The idempotency test now pins exact class identity and the uncast-sibling fallback. * Pass the calibrated distilled sigma curve to LTX-2.3 8-step runs The 22B distilled DiT was trained against ltx_core's fixed DISTILLED_SIGMA_VALUES, but the diffusers scheduler derives 8-step spacing from resolution-shifted flow matching and lands far off at every reachable mu (second sigma 0.945-0.981 vs 0.99375, tail 0.37-0.61 -> 0.1 vs 0.725 -> 0.42 -> 0). At the distilled default step count the backend now passes the list verbatim, neutralising the scheduler's dynamic shift and terminal stretch for the call (they distort even explicit sigmas) and restoring them afterwards. Other step counts and the dev/base DiT keep the scheduler's own spacing. Live-verified on B200 through the video branch backend: the scheduler holds the exact curve after an 8-step distilled GGUF generation, config restored, healthy clip. Also reword the transformer_quant resolved reason to the measured reality: quant halves resident weights and hosted checkpoints cut load time, while per-step speed is roughly bf16 parity. * Pin the fp8 weight-quantize kernel against silent MSLK switching torchao's Float8Tensor KernelPreference defaults to AUTO, which switches the weight-quantize kernel to MSLK whenever an mslk package is importable on sm90+. Measured on B200: that changes fp8 scale rounding bitwise (8/8 FLUX matrices differ, scales ~55 percent of bytes), so a box that merely gains mslk would break the hosted-prequant bit-identity invariant; the mslk path is also slower under torch.compile (opaque extern call blocks inductor's quantize fusion, FLUX.1 fp8 e2e 1.149 to 1.624 s). Pin KernelPreference.TORCH explicitly, matching current no-mslk behaviour bit for bit; signature-gated for older torchao. GPU-smoked (finite, rel err 0.037) and pinned by test. * Shift Qwen-Image training sigmas to the inference distribution Qwen-Image's scheduler skips its static shift under use_dynamic_shifting, so the DiT trainer was drawing UNSHIFTED uniform-schedule sigmas for it (mean sigma 0.50) while inference always runs the exponential mu = log 3 shift plus the shift_terminal 0.02 stretch. Add a flow_shift config lever: "auto" (the new qwen-image default) rebuilds the training sigma table through the scheduler's own time_shift and stretch_shift_to_terminal so the draw matches the inference distribution exactly (mean sigma 0.72); a numeric value applies the standard linear shift s*u/(1+(s-1)*u); 1.0 keeps the historical identity behavior and stays the default for FLUX, Z-Image and Krea 2. The model timestep conditioning follows the shifted sigma, gathered in fp32 so bf16 rounding never skews it. Also wire two opt-in levers with off defaults: cfg_dropout (per-sample empty-prompt conditioning dropout, encoded alongside the captions before the text encoders are freed) and weighting_scheme="bell" (bsmntw-style mid-schedule Gaussian loss weighting normalized to mean 1). Verified with two 80-step rank-8 bf16 LoRA runs on Qwen/Qwen-Image (identity vs auto, same seed): both converge with finite decreasing loss and produce coherent same-seed previews. Unit tests cover the exact transform, the shifted sampling distribution, per-family defaults and config plumbing. * Add LoRA EMA, a persistent conditioning cache, and aspect bucketing helpers diffusion_train_extras hosts the opt-in training extras: LoRAEMA shadows only the trainable adapter params (warmup-ramped decay, default 0.99, exported as a second adapter under output_dir/ema), PersistentConditioningCache stores latent posterior stats and caption embeddings as safetensors keyed by content hash + family + resolution, and the aspect-ratio bucketing helpers group mixed-aspect datasets into same-area divisor-snapped shapes. The DiT trainer wires the first two behind config flags that default to the current behavior: ema_decay (0 disables) and cond_cache_dir (None disables). A fully warm cache skips loading the VAE and text encoders entirely; a cache hit is bit-identical to a fresh encode, including the per-channel qwen latent normalization. Also fixes the stale _gather_sigmas call in the perf test that still passed the scheduler instead of the sigma table. * Tighten torchao configs and note the FSDP2 design for the DiT trainer nf4 loads now enable double quantization (~0.4 bits/param off the frozen base scales at no fidelity cost), fp8 training uses the rowwise recipe when the torchao build ships it (per-row scaling confines the DiT activation outliers that a tensor-wide scale collapses), and the inference quant filter gains a per-scheme GEMM-tiling divisibility floor (16 for scaled_mm, 32 for MX blocks) so one ragged Linear cannot crash the first denoise after a clean quantize pass. plans/fsdp2_diffusion_design.md records the multi-GPU design: bf16/fp8 over FSDP2 with per-block units, LoRA attached before sharding, int8 out of scope (DTensor over the quantized subclass is undefined), per-family notes. * Batch diffusion inference with per-image seeds, an inference conditioning cache, and GGUF loader fixes Batched generation: /images/generate takes a prompts list (one image per prompt, txt2img only) or a seeds list (one prompt, one image per seed); the legacy batch_size path derives per-image seeds base..base+n-1 like the native engine. Every image gets its own torch.Generator so any batch member replays alone from its gallery recipe; the whole list runs as one forward by default with OOM backoff that halves a failed chunk, and an explicit batch_size caps images per forward. Validated 10-22x over serial engines on 32-image suites with LPIPS deltas within 0.002. Conditioning cache on the inference path: UNSLOTH_DIFFUSION_COND_CACHE_DIR (the inference sibling of the trainers' cond_cache_dir, same persistent store) wraps encode_prompt so repeated prompts skip the text-encoder forward entirely; verified bit-identical outputs. Bypassed while LoRA adapters are attached; tensor-argument calls pass through uncached. Compile cache: GGUF loads fingerprint their own bundles (quant=gguf, a different compiled graph than the dense family) and batched calls register every distinct (w, h, batch) chunk shape they ran, so the heavy GGUF batched warmups (~159 s at batch 32 on 12B-class, ~655 s on 20B CFG-batched) are paid once ever. GGUF loader: strip the sd.cpp model.diffusion_model. container prefix in the single-file converter; diffusers' FLUX.2 converter KeyErrors on it and the Qwen-Image identity mapping strands the model on meta. * Correct batched seed-replay docs to match measured behavior Same-seed images at the same batch shape are bit-identical; a solo regeneration with the recorded seed matches its batched rendition up to batch-size-dependent kernel numerics (mean abs pixel delta about 2.5/255, LPIPS delta under 0.002), not bit-exactly. The previous wording overclaimed bit-identity across batch shapes. * Note that batched bit-identity assumes a settled compiled graph The first generation issued while the deferred compile is still in flight can deviate transiently (observed once on a cold fp8 build: mean abs pixel delta 0.063/255); once the graph is settled, same-seed same-batch-shape images are bit-identical across runs. * Studio sidebar: Image03/FlimSlate icons, More flyout, Train row, New pills - Images uses Image03Icon and Video uses FlimSlateIcon. - New "More" row (MoreHorizontalIcon) opens a right-side flyout on click or hover holding Video, Recipes and Export; the close is delayed 180ms so the pointer can cross the gap. Its SidebarMenuButton deliberately takes `title` rather than `tooltip`: with `tooltip` the button returns a Tooltip root and DropdownMenuTrigger asChild would hand its ref to a non-DOM node. - Dropped the "Train" section heading; Train is now a top-level row between Images and More. data-tour="navbar" moves to the surviving nav group so the product tour keeps its anchor. - "New" pill beside Images and (inside the flyout) Video, via NavBadge. * Studio sidebar: match flyout rows and New pills to the existing scales - More flyout rows dropped their sidebar-row typography and size-icon override, which fought DropdownMenuItem's own scale (text-sm, gap-2.5, px-3 py-2 and size-4 icons) and rendered oversized glyphs and text next to the nav. - New pill reuses the brand "beta" badge recipe (nav-badge font, --ui-font-scale sizing, nav token colours) rather than hardcoded 9px values. - The More row's native title tooltip (an OS box on hover) is replaced by the app's Tooltip, wrapped around DropdownMenuTrigger so both triggers compose onto the same button, and shown only on the collapsed rail like other nav rows. * Settings: pin and reorder the sidebar navigation Adds a "Sidebar navigation" section to Settings -> Appearance, above the existing profile-menu customizer, with the same drag-to-reorder + switch UI. - New sidebarNav preference: one { id, pinned } entry per navigable row (projects, hub, images, train, video, recipes, export), array order = render order. Defaults match the shipped layout, so an untouched install is unchanged. - Unpinning moves a row into the More flyout rather than hiding it, so no page becomes unreachable. New chat and Search stay fixed as actions. - app-sidebar now renders from one navRows descriptor map, so a pinned row and its flyout counterpart cannot drift; the More row appears only when something is unpinned and highlights off whatever it actually holds. - Mirrored in the backend PersonalizationCustomization: without it the model's extra="ignore" would drop the field, and because sync replaces local state with the server's copy once customization is saved, the user's pin order would reset on the next sync. The validator dedupes and back-fills like sidebarMenu but preserves the client's order, since here order is meaningful. Frontend typecheck, i18n parity and catalog checks pass; 32 personalization tests pass, including a round-trip asserting a reordered list survives a save. * Sidebar customizer: drop the Search row, skip More for a lone item - Search is reached from the top bar, so it is no longer previewed as a fixed sidebar nav row; New chat stays. - More now appears only when it would hold two or more rows. A single unpinned row renders inline in its saved order position instead: a flyout wrapping one item costs a click and earns nothing. The customizer's More preview follows the same threshold. * Sidebar settings: hide a lone unpinned tab, match New chat icon, rename Profile menu - With exactly one tab unpinned, both More and that tab are dropped, so nothing is drawn for it (previously it rendered inline). The page stays reachable by URL. - The customizer's New chat preview uses PencilEdit02Icon, the icon the real row renders; Edit03Icon was a different glyph. - "Sidebar menu" is now "Profile menu", described as the shortcuts behind your name at the bottom of the sidebar, so it no longer reads as a second name for the navigation section above it. * Tighten comments in the new sidebar and delete-guard code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio sidebar: keep the More row highlighted while its panel is open Moving the pointer into the flyout left the row unhighlighted while the panel stayed open. The row now carries data-menu-open, added to the nav hover selector list. Not data-state: the tooltip and menu triggers both write that attribute, so whichever lands last wins. * Images: use the shared pill toggle for Create/Train and pad the panels - Create/Train was the only segmented control on its own Tabs styling. It now uses PillTabs, the same control as the model picker and Hub toggles, pinned to the header row's 34px. PillTabs takes an icon per tab, so the inline-span workaround for TabsTrigger goes away. - pt-3 on both the Create and Train panels, which sat flush against the model selector row. * Images: make the workflow picker a dropdown instead of a 7-up strip Seven workflows in a 340px rail left ~48px each, so the labels crowded and the hints were only reachable as title tooltips. The strip is now a dropdown: the trigger shows the current workflow and its hint, and each row carries its own description. A row the loaded model can't run is disabled and shows the reason in place of the hint, so the gating explains itself. Adding a workflow no longer shrinks the others. * Images: workflow icons, hint under the trigger, more top room, unclipped Train cards - Each workflow carries an icon, shown on the closed trigger and on every row. - The trigger is one line (icon plus name). The selected workflow's description moved below it, where it reads like the Field hints further down the rail. - pt-6 instead of pt-3 on both Create and Train, so the cards clear the model selector row. - The Train right column scrolls while its cards use ring-1, which draws outside the box and was clipped at the scroll edges. p-px gives the ring room. * Images: one-line workflow rows, stronger trigger fill, roomier mode tabs - Dropdown rows are icon plus name only. The selected row's description already shows under the trigger, and a disabled row keeps its reason as a title. - Trigger fill moves to the bg-foreground/[0.07] dark:bg-foreground/[0.12] pair the hub cards use, so it reads against the card in both modes. - Description under the trigger goes from text-ui-10 to text-ui-11p5. - More horizontal padding on the Create / Train tabs. * Images: drop card borders for the composer shadow, keep scrollbars inside, use app controls in Train - Cards lose ring-1 for .panel-soft-surface: the composer's shadow in light, flat in dark, matching .chat-composer-surface and the menus. - Both rails now clip (overflow-hidden) with the scroller inside, so the scrollbar can't ride over the rounded corner. Same shape video-page already uses. - Train's 9 native selects become the app Select, so they no longer open an OS-native menu, and the native file input is hidden behind a Choose images button that reports the count. - Image previews use explicit 8-10px radii: this theme sets --radius to 1.1rem, so rounded-md was 15.6px and the thumbnails read as circles. * Images: one card for controls and preview, chat sliders, wider softer shadow - Controls and preview were two floating cards; they now share one card split by a divider. The Advanced dock stays separate since it toggles. - SliderField wraps Chat's ParamSlider, so the sliders match Chat (label row with the value, full-width neutral track) instead of a green track with a spin box. All 14 call sites keep their props. - panel-soft-surface goes from 0 2px 8px -2px /0.16 to 0 4px 22px -6px /0.10: lighter, spread wider. * Images: flat Create and Train panes, hover-only scrollbars, tidier Train dataset step Both Images tabs now sit on the page background like the Hub: no card, no shadow, no bounding box. A single rule divides the controls rail from the preview canvas (Create) and from the run area (Train), and the settings and previous-runs sections read as panes rather than nested cards. Also: - Scrollbars in these panes use the existing hover-scrollbar recipe, so the thumb only shows while the pane is hovered. - Workflow rows explain themselves with a tooltip after a short hover, which also works on disabled rows, and the descriptions are much shorter. - Training images rows are name plus image count; the license stays on the example card. - The upload step loses its dashed box, the buttons match the sizes around them, and Upload only appears once files are picked. - The empty preview uses the same icon as the Images nav item. * Images: full-height panes, wider settings rail, Create/Train offset from the selector The rule between the panes now runs the whole page height (the row drops its bottom padding and each pane pads its own content), the settings rail is wider on both Create and Train, and the Create/Train switch sits further right of the model selector. * Images: put both tabs on the Hub's centered measure Top bar and content now share mx-auto max-w-1100 with px-5 / sm:px-8, so Create and Train sit at the same width and position as the Hub instead of running edge to edge. * Images: restore the top bar position, drop the panes lower under it * Images: center the mode switch, flip the arrow with the orientation, app tooltips everywhere The Create/Train switch is centered on the page instead of trailing the model selector, with wider buttons. The flip control's arrows now rotate with the orientation and its label says which way the flip goes. Every native title tooltip on the page is now the app's tooltip, so they all get the rounded surface instead of the OS box. * Images Train: plainer field text, no green buttons, columns that stop colliding - The dataset name, trigger prompt, adapter name and custom base fields now say what they are in plain words instead of leaning on example values. - Import, Upload, Back, Back to settings and Train another are outline buttons, not green ones. - Example thumbnails are landscape tiles, so photos are not cropped to chunky squares. - Settings cells get min-w-0 and the select value truncates, so a long option like the nf4 label no longer widens its column into the next one. - The number stepper sits a little further in from the field edge. - Create and Train are wider. * Images Train: roomier example cards with Import on the thumbnail row * Video: same treatment as the Images tabs - No cards: the rail and the canvas sit on the page background, divided by a rule that runs the full page height, on the Hub's centered measure. - Wider rail, chat's sliders, hover-only scrollbars. - Every native title tooltip is now the app's tooltip, including the clip cards. - Reapply and Cancel are outline buttons, the empty state uses the Video nav icon, and the clip tiles are less rounded. * Images and Video: narrower generation rail, matching Train headings Create and Video rails go from 392px to 368px. Train a LoRA and Training settings are now the same size and both in the heading font: the h2 already picks it up from the base rule, so the settings header opts in with font-heading and the weight that rule pins. * Images Train: shorter copy throughout Family notes, example descriptions, precision labels and every helper line are trimmed so they stop wrapping to three lines and colliding with the next column. The nf4 label now fits its select without truncating. * Images Train: a little more spacing between field groups * Images and Video: tighten code comments * Fix training start NameError, the load-order guard test and CPU-only diffusion tests - start_training forwards resume_source_run_id to _start_training_impl, which reads it. Without it every start raised NameError. - Restore main's anchor in the load-marker order test: the file now has an earlier `if config.is_gguf:`, so indexing the first one compared the wrong branch. - The two diffusion tests that reach diffusers now skip when it is absent, matching the CPU repo-test env. - The UI smoke finds nav rows that live in the sidebar's More flyout. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Treat a null metadata caption as no caption str(None) stored the literal "None" as the caption, so a null row counted as captioned and would have trained on that text. Also drops an unused import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix invalid-UTF-8 500s, the flat Canny map and the dropped DiT knobs read_text raises UnicodeDecodeError, which is not an OSError, so one bad caption sidecar or video sidecar 500d the info, upload and gallery routes. A flat image now yields the all-black edge map instead of its own luminance, and the four DiT loss knobs the trainer implements are declared so model_dump keeps them. * Use the ui font-size tokens instead of raw px text utilities text-[11px] and friends ignore the UI font size preference, which the repo's font-scale contract test enforces. Same rendered size at the default scale. * Fix diffusion dataset 500s, the dropout-1.0 no-op run and the reset base pick Four correctness fixes on the training side: - The labeling grid read caption sidecars under except OSError, but a non-UTF-8 sidecar raises UnicodeDecodeError (a ValueError), so one bad file 500d /diffusion/dataset/{name}/images and the grid could not be opened to repair it. Read it as no caption, matching the info summary. - An image past Pillow's own hard limit raises DecompressionBombError, which derives straight from Exception and so escaped the upload guard's (OSError, UnidentifiedImageError, ValueError) and returned 500 instead of the intended 400. - lora_dropout accepted 1.0, which makes PEFT build nn.Dropout(p=1.0): lora_A and lora_B receive no gradient and the run saves an untrained adapter while reporting normal progress. Bound it below 1.0, matching the LLM request schema. - The train panel re-seeded the base repo on every dataset refresh because the family object identity changes on each info fetch, so an upload or caption save silently replaced the user's chosen base and the run started on a different model. Track the pick and only re-seed on a real family change. * Show the retained failure when a video page mounts after a failed job Mount-time recovery handled only phase=completed, so reloading the page after a multi-minute generation failed left an idle view with no diagnosis: the backend keeps the terminal failed record only until the next job, and nothing else survives the reload. Surface it the same way the poll does, filtering the cancelled sentinel. * Fix batched generation crashes, cache keying and unreplayable recipes Four bugs in the batched inference path, all found by review: - A mixed-prompt batch sent a scalar negative prompt against a prompt list. Z-Image asserts on the length, and Qwen-Image, Krea 2 and FLUX true-CFG encode a batch-1 negative against batch-N latents and fail in the transformer's text/image concat. Broadcast it to match the batch. - The FBCache step-cache reset sat above the chunk loop. diffusers only resets that state at the end of a successful call, so a forward that raised (the OOM the backoff is meant to recover) left its own residual behind and the halved retry died on a shape mismatch. Reset before every forward instead. - The conditioning cache keyed on the checkpoint alone, but a GGUF or single-file load takes its text encoders from the companion base, so the same checkpoint against a different base reused the previous base's embeddings. Key the base too. - Gallery records stored the base seed and the requested batch size even when a prompts/seeds list drove the run, so restoring the second image of seeds=[5, 99] replayed seed 5. List-driven outputs now record as single-image recipes on their own seed. Also bound strength above 0: every img2img pipeline derives its step count from it, so 0 leaves zero denoising steps and either raises or, on SDXL, crashes on empty latents. * Fix quantized-load LoRA bake, prequant family exclusions and outpaint canvas Six review findings across the Images page and model scanning: - The quantized (int8/fp8) load path can only attach LoRA adapters before quantization, but the frontend load request had no loras field, so every generation after such a load was rejected and each reload repeated it. Send the selection with the load. - build_prequant_checkpoint passed no family to the scheme exclusions while recording the family in metadata, so a Qwen int8 artifact baked the short-M text-stream linears and was then rejected wholesale by the loader's family-keyed check. - Registering a bare single-file checkpoint directory produced no On Device row even though the images loader can load it; only its parent worked. Admit that shape when nothing else matched. - Unload left the Reapply target set, so the repair path was skipped and Reapply reloaded the ejected model. Clear it, as the video page does. - Both FLUX.2 bases were trusted for training but not inference, so Deploy to Create rejected every FLUX.2 adapter. - Outpaint allocated the grown canvas before downscaling, exceeding the browser canvas area cap on a large photo; an over-cap canvas is unusable, so Extend silently posted a fully transparent image and mask. Scale the source first. * Send the picked GGUF filename with the quant so diffusion loads fire The variant expander emitted only the quant label, and nothing else in the frontend set ggufFilename, so the Images and Video pages could never take their GGUF branch: both gate it on meta.ggufVariant and meta.ggufFilename, then fall through to the single-file path, which returns because the id is a repo id and not a .gguf name. Every quant pick was a silent dead click, with no load request reaching the backend. The filename was already on the variant row (the picker keys its list on it, and the variant validator requires a non-empty string), so thread it through the click handler. The chat path is unaffected: it reads ggufVariant and never needed the filename. * Version the conditioning cache key and reject non-finite flow_shift Two correctness fixes: - The cache keyed the checkpoint and its companion base by name only, so a Hub repo advancing to a new commit, or a local directory updated in place, kept returning embeddings from the previous text encoder. Pair both with a revision marker: the locally resolved commit sha for a Hub repo, config plus text-encoder file stats for a directory. Neither loads the encoders, so a warm run still keeps them off the GPU. - flow_shift only checked positivity, but JSON accepts 1e309, which floats to inf, and inf <= 0 is False while NaN fails every comparison. The sigma table then evaluates s * u / (1 + (s - 1) * u) as NaN, which poisons every sampled sigma and saves a corrupted adapter while progress looks normal. Require a finite value. * Keep curated models listed, guard the video companion repo, pin diffusers Three review findings: - The picker filtered every catalog member out of Recommended and Hub search on the way to canonical group rows, but nothing renders those rows yet (catalogGroupFitsDevice and groupMatchesQuery are imported and unused). A task-scoped picker's models list is catalogToModelOptions(), i.e. group members exclusively, so both lists came back empty and no curated model could be discovered or downloaded. Keep the artifacts listed until the grouped UI exists. - The video delete guard compared only repo_id, so deleting the companion base of a loaded GGUF video model was allowed even though it supplies the VAE and text encoders. Compare base_repo too, matching what the images guard already does for its companions. - diffusers was declared unversioned while the diffusion stack requires 0.39 (Krea2Pipeline, the cache_context child registries, the Flux2 and Z-Image pipelines), so an upgrade could keep an older release and selecting an advertised model failed until the user upgraded by hand. * Namespace the trainer conditioning cache per checkpoint, bound the learning rate - The trainer keyed its persistent conditioning cache on family and resolution only, while the keys themselves carry just the caption or image content and crop variant. One cache directory reused for two checkpoints, or for the same repo at a new revision, let a warm run skip loading its encoders and train on the other model's embeddings and latent statistics. Namespace on the base checkpoint and its resolved revision as well. The revision helper now lives beside the cache in diffusion_train_extras and the inference wrapper delegates to it, so the two cannot disagree about what counts as the same source. - The diffusion learning rate only checked positivity, but 1e309 floats to inf and satisfies gt, so the route evicted the resident models and started AdamW with an infinite rate: the first step destroys the adapter while progress looks normal and the result is saved. Bound it below 1.0, matching the LLM schema, which rejects inf for the same reason. * Fix GGUF image model picks doing nothing, and pick the train base in the top bar The quant rows never forwarded the .gguf filename, so every hub GGUF pick on Images/Video fell through to a silent return. On Train the top bar now picks the training base instead of a generation model, which is GGUF-only and untrainable. * Pin diffusion and video loads to the live HF cache root Both read huggingface_hub's import-time HF_HUB_CACHE, which changing the cache folder does not update: progress counted the old root while the download wrote to the new one, and from_pretrained could split one model across both. * Add the diffusion download plan endpoint Reports the repos and exact files a pick needs so the download manager can stage them with the loader's own file scope. A plain snapshot would add the packaged root single, transformer shards and fp16 twins the loader never opens. * Add a file-scoped flavour to the Hub download job Lets a consumer that reads a deliberate subset of a repo stage it through the normal download manager. Keyed as "@scope" so it never collides with a quant or with the repo's full snapshot, and the file list rides the registry so an XET to HTTP retry respawns the same scoped job. * Stage image and video downloads through the Hub download manager They downloaded inline inside the load, so they had none of the manager's disk preflight, manifest verification, resume or panel progress. Picks now stage as scoped jobs carrying the loader's own file list, then load from a warm cache. * Fetch staged GGUF checkpoints as scoped jobs, and stop calling diffusion models unsupported A GGUF entry went out as a full snapshot, whose ignore list drops *.gguf: the job finished at once having fetched only docs, and the repo landed on device unloadable. Every entry is scoped now. The Hub also no longer tags image/video models as unsupported (they run on their own pages), and those pickers name what they select. * Apply the picker task filter to local model sections LM Studio, ./models and custom-folder rows ignored it, so the Images picker listed chat GGUFs that 400 on a diffusion load. The backend already tags every local model with a task for this purpose. * Route a chat pick of a diffusion model to the Images or Video page Chat cannot load one, so it was either hidden or failed on load. The unfiltered picker now lists on-device diffusion models and navigates to the page that runs them, passing the repo and quant so that page loads it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Route the real GGUF filename, keep non-GGUF curated models, key scoped downloads by file set Five review findings, four of them ways a click did nothing or fetched the wrong thing: - A chat pick of a diffusion model routed ggufVariant (a label like Q4_K_M) in the search param the target page uses verbatim as the GGUF filename, so the load asked for a file that does not exist. Route ggufFilename; no filename means a curated non-GGUF pick, loaded as a pipeline. - The task-scoped pickers kept only GGUF repos, so the catalog's bf16, bnb-4bit and single-file fp8 artifacts could not be discovered or downloaded on the Images and Video pages even though loadSpecFor knows how to load them. Keep curated artifacts whatever their format, in Recommended and in Hub search. - Both pages deduplicated routed selections on the model alone, and they now stay mounted, so picking the same repo again -- another quant, or the same one after chat evicted it -- returned early without loading or clearing the query string. Key on model and quant. - Every scoped image download shared one @diffusion job key regardless of the requested files, so switching quant mid-download adopted the running job: the UI waited on the first file set, then loaded a file that was never fetched. Include a digest of the file set in the key. - A scoped plan silently dropped requested files missing from Hub metadata, and snapshot_download succeeds when an allow pattern matches nothing, so the job reported completion and triggered a load with required files absent. Fail the job instead. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the scoped download key derivable, and stop the hidden page hijacking a route Four review findings, the first a regression from my own last commit: - Keying scoped download jobs by a digest of the file set broke the download manager: it builds that key client-side (it polls and cancels before any response tells it a key), so it watched and cancelled a key no worker owned and never fired its ready callback. Keep the derivable "@scope" key and refuse the second request instead when a live job on the slot is fetching a different file set -- decided inside the registry claim, under the lock, so a concurrent claim cannot slip past it. The manager records the file set on the job as well, so a sibling quant's transfer is not adopted locally either. - Both diffusion pages read the route query through a loose useSearch and both stay mounted once visited, so the hidden one consumed the other's ?model=: it navigated back to its own route and tried to load, say, an image checkpoint as a video model. Only the visible page consumes it. - The staged download plan was built without the configured HF token or the Advanced values the load itself sends. The token matters most: the backend's Hub metadata lookup is best-effort, so a gated base silently planned no companion entry and the load pulled those multi-GB files inline, outside the manager. The memory/quant controls decide whether the base transformer/ shards are needed at all, and the route dropped memory_mode, cpu_offload, the prequant path and the LoRA selection before asking for the plan. - The video preview kept playing after leaving the page: the keep-alive layout only hides it, and display:none does not pause a media element, so a clip the user unmuted kept its audio going over the next page. Pause on the active transition and do not auto-replay while hidden. Also completes the hand-built request bodies in the hub download tests: the scoped-files field this branch added to the route read as an AttributeError against them, failing five tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Serialize the GPU handoffs, gate DiT training on a GPU, and keep 3.9 installable Six review findings, three of them evict-then-fail orderings: - The chat load reclaimed the GPU without telling the arbiter it existed. A chat load holds no llama-server process until its GGUF has downloaded, which is minutes, so a competing Images/Video acquire in that window found nothing to cancel, took the GPU, and the chat load then spawned onto the same device. It now registers an in-flight marker through acquire_for's register hook (under the arbiter lock, as the image and video loads do), the evictor cancels a marked load, and the route undoes itself if ownership moved while it loaded. - The Hub-download conflict check ran after that handoff, so a GGUF the download manager already owns destroyed the resident Images/Video pipeline and then 409'd, having loaded nothing. It moves above the handoff, together with the marker it handshakes with. - The image load released the engine router's transition lock before registering the load, so a second load choosing the other engine could unload the still-idle engine this one captured; the load then landed on a deactivated engine, where generate, status, unload and the arbiter's evictor can no longer reach it. Registration now happens under that lock and refuses if the engine changed. - Training a DiT family on a host with no GPU was accepted: nf4 is not a CPU fallback, its 4-bit load goes through bitsandbytes, which requires CUDA, XPU or MPS. The start unloaded the working Images pipeline, pulled the text encoders, and only then died in the child. Rejected before the teardown now, and /info stops advertising a precision that always 400s. SDXL keeps its documented fp32-on-CPU path. - Both diffusion pages kept the routed-pick marker forever, so re-picking the same checkpoint (after chat evicted it) neither loaded nor cleared the query string. The marker is released once the query is gone. The Images key also carried a stray NUL byte, which made the file read as binary to grep and other tooling. - diffusers dropped Python 3.9 in 0.38, so the unconditional >=0.39.0 pin left pip no candidate at all on 3.9 and made every install that composes the huggingface extras unresolvable there. The floor is conditional now. Also fixes tests that were already red on the branch: two hand-built request fakes had gone stale against fields this branch added, and the handoff-ordering test only failed on a host with fewer than two GPUs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the GGUF variant contract test against the merged handler signature The assertion pinned the exact single-line call handleVariantClick(v.quant, v.downloaded, expectedBytes, v.filename), but the handler takes (quant, filename, downloaded, sizeBytes) and prettier wraps the call across lines, so the mandatory repository test job failed on every push. Match the call structurally and assert the filename really is forwarded in the handler's argument order. * Stop a background page and a stale record taking the GPU or a download with them Five fixes from a review pass over the diffusion work. delete-finetuned rmtree'd a model the Images or Video engine was holding: every guard on that route is chat-only, and Images loads any local path, so deleting a local diffusion model under the storage root pulled the weights (and the companion VAE / text encoders sd.cpp re-reads each generation) out from under a live pipeline. The cached-model route already refuses this; the trained/exported one now does too, matching by path rather than repo id, and failing open on a chat-only install so it cannot block ordinary deletes. A staged download finishing while its page was hidden loaded the model and evicted whatever the user was actually using: both diffusion pages stay mounted behind the router and a load takes the GPU unconditionally. The pick is now held until its page is on screen again, which is also what chat does. A scoped download could report success having fetched nothing. With Hugging Face metadata unavailable no manifest is written, so verification is a no-op, and snapshot_download returns an existing snapshot folder without downloading when its own repo_info call fails. A repo already on disk from a full snapshot job (which ignores *.gguf) therefore completed with no weights and auto-loaded against them. The requested file list needs no network, so it is checked against the disk directly. The XET to HTTP retry reclaimed the job slot without the scoped file list, and that claim overwrites the stored record, so a later identical scoped start compared an empty list against the real one and 409'd instead of adopting the running download. The DiT accelerator gate probed torch.mps.is_available(), which only exists from torch 2.5 while the supported floor is 2.4. All three probes shared one try/except, so on torch 2.4 the AttributeError read as 'no block' and a CPU-only host still evicted the resident pipeline, downloaded the encoders and died in the child. Each accelerator is probed on its own now, through torch.backends.mps. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stage what an LTX-2.3 load reads, keep a routed file's load kind, drop an unbakeable LoRA Three from the latest review. The video download plan always asked for the wide base file list, so an LTX-2.3 pick staged the 2.0 base's VAEs, vocoder and connectors that the checkpoint supplies itself, while the companion files the 2.3 assembly does read were left out of the plan and pulled inline at load, outside the panel's progress, cancel and disk preflight. The plan now recognises a 2.3 pick by name (the load keeps the authoritative header probe, and under-guessing only falls back to the load-time pull), narrows the base list, and stages the extras in the same entry as the checkpoint so one repo stays one scoped job. A pick routed from the chat picker arrives as ?model= and ?quant= with no picker metadata, so a bare local .gguf or .safetensors was loaded as a pipeline: an explicit model_kind wins over the backend's filename sniffing, so it evicted the resident model and then failed on the missing model_index.json. Both pages now derive the load kind from the path, the same way their own picker handlers do. A torchao int8/fp8 build takes adapters only at load time. Switching artifact inside one family keeps the LoRA selection, since the family did not change, but the load did not bake it, so the next generation was rejected with 'reload the model with the adapter selection' while the picker still showed the adapter as active. The selection is now dropped once per resident build, with a message saying to pick and load again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cancel an evicted safetensors load, spare the arbiter for CPU-only chat, fetch clips lazily Four fixes from the latest review round: - The GPU arbiter's chat evictor only cancelled the llama.cpp side. The orchestrator publishes active_model_name once its worker reports success, so an in-flight safetensors load was visible only as an entry in loading_models and finished onto the GPU after ownership had transferred. Cancel every pending load, and give the safetensors branch the post-load ownership recheck the GGUF branch already had. - A manual gpu_layers=0 GGUF load runs on the CPU with the GPUs hidden from the child, yet it took the arbiter unconditionally: it cancelled a running image or video generation for a model needing no VRAM, then held CHAT ownership so the next GPU workload unloaded it for nothing. Gate the acquire on the same predicate the launch-time CPU-only mask uses, as the image and video loaders gate on their resolved device. - The staged-download hook subscribes per repo, not per job, so another job on the same repo advanced the staged queue (starting a load whose scoped files were still downloading) or wiped a queue that was still running. Compare the variant each callback carries, like the chat page's auto-load does. - The video gallery fetched every record of a page into an object URL that lives until the page closes: 50 clips at tens to hundreds of MB each, for cards the user may never scroll to. Fetch a clip as its card nears the strip's edge, plus the selected one the player needs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim the comments across the diffusion backend Comment-only pass over the Python this PR touches: drop what the code already says, collapse multi-line explanations that still read on one line, and keep the reasoning that is not recoverable from the code. No code, docstring semantics or behaviour changes; verified with an AST comparison against the previous revision, and the backend suite is unchanged (same 37 environment failures as before: the API integration tests that need a live keyed server, the flash-attn install hooks, and the GPU memory fields). * Invalidate latents on a VAE swap, keep a cut-off generation, surface the EMA adapter - source_revision() scanned the checkpoint root plus text_encoder/tokenizer but not vae, so swapping or fine-tuning the VAE in place left the conditioning cache namespace unchanged and a warm run trained against latents from the old checkpoint. Include the vae directory, like any other component the cached tensors come from. - /images/generate answers only when the images are saved, and secure mode's tunnel caps an origin response near 100 seconds, which a native CPU or a high-step run passes routinely. The page reported failure while the work kept running, and a retry would duplicate it. A lost response (fetch rejection or a gateway status the origin never answered) is now told apart from a refusal: the page waits out generate-progress and reloads the gallery, so the run it started still lands. - The trainer emits the EMA adapter's path with the terminal event, but the state update dropped it, so neither the run history nor either response schema carried it and an enabled EMA left nothing discoverable. Keep it, and show it next to the primary adapter. - weighting_scheme advertised a choice of timestep sampling; sampling is always logit-normal and the flag only selects the bell loss weights. Describe what it does. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Hide unloadable cached rows, hold the dataset interlock, bound a GIF export - The cached-model listing tagged any repo with a model_index.json as text-to-image, so a community pipeline the image loader's trust rule refuses still got a row in the Images picker, and a detected-but-untrusted video repo fell through to that same tag. Gate the image tag on the load path's rule and hide an untrusted video repo outright. - A routed diffusion pick only carries a GGUF filename, which is all the chat picker has, so a curated single-file artifact arrived with no quant and was loaded as a pipeline: from_pretrained on a repo with no model_index.json. Pass the page's own catalog spec into the route pick, so a routed pick resolves to exactly what a direct pick on that page resolves to. - The dataset mutation endpoints checked is_active() and only then handed their filesystem work to a thread, so a start reserving in that gap changed captions or removed images underneath the preflight or the running trainer. The interlock is now registered for the whole request under the lock reserve() uses, and a start refuses while a mutation is open rather than waiting on it. - GIF export held every kept frame as a paletted image before encoding; a clip may be 2048x2048 for 1024 frames, and at the 12 fps target the step is 1, so one export click could allocate over 4 GB and take the backend down. Downscale past 720 px and widen the step to keep at most 300 frames. - seed accepted any Python int, so an out-of-range one passed every preflight, evicted the resident models, spawned the trainer and only then died in torch.manual_seed. Bound it to torch's 64-bit range in the request and config. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the accelerator probes in the DiT family-metadata tests Six tests read family_train_infos() (or a start preflight) without pinning the host probes, so they only held on a machine with a bf16 accelerator: on a GPU-less runner the DiT gate empties precision_modes, turns supports_compile off, and replaces any other preflight message with the no-accelerator note, and all six failed there. A conftest fixture pins both probes for exactly those tests, so they assert the family metadata they are about on every host. The gate's own CPU-only behaviour keeps its dedicated tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Carry the pipeline task into the hub inventory the pickers read - The task-scoped pickers filter On Device rows on a task, and the chat picker routes a diffusion pick by the same field, but those rows come from the /api/hub inventory, which never carried one: the Images and Video pickers listed nothing on device and the routing never fired. Both cached scans and the local listing now tag rows with the classifiers the models API already uses, the schemas and the frontend adapter carry it through, and a row the backend classified as a generation task is exempt from the chat-only guard that was also dropping it. - The local routing map was keyed by model_id while the row click passes id (a filesystem load id for a models_dir or LM Studio entry), so the lookup missed and the pick fell through to the chat loader. Key both. - A staged download whose start answered "error" left its head in place, where the effect never re-runs and onReady never fires, so the pick was stranded until the user reselected. Clear the queue and say so. - Every scoped pick in a repo shares the @diffusion variant, so the variant alone cannot tell two file sets apart: restaging while the first job finished let its completion pass for the new pick and load a checkpoint that had not downloaded. Bind the callbacks to the repo + file set they started, and to the staging generation. - A rejected generate POST does not say whether it reached the backend, so an immediately idle progress read was ambiguous and a submission that never landed looked like a finished image. Require evidence: progress seen active, or a gallery record that was not there before the POST. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the OpenAI image URL fetchable, keep WebM audio, stream example imports Four review items on the diffusion Studio work: - response_format=url returned the bearer-gated gallery route, which a standard image client downloads with no Authorization header, so the default response format was unusable. Mint a short-lived HMAC link instead (the shape RAG already uses for pdf.js) served by a signed route, and leave the gallery route itself bearer-only. - A manual gpu_layers=0 load carrying speculative_type="off" -- a value the UI persists and sends -- read as GPU-bearing, so it took the GPU arbiter and evicted a resident image/video pipeline even though the launcher hides the GPUs for it. Canonicalize the mode and exempt "off". - The curated example import prepared the whole split before the loop stopped at the 10-100 image cap; m1guelpf/nouns is 49,859 rows / 328 MB. Stream instead, with the prepared load kept as a fallback for a repo that cannot stream. - WebM export dropped the audio track an LTX-2 clip carries, silently, on the format offered for web embeds. Mux it as Opus through a resampler + FIFO, and keep exporting the video alone on a build without libopus. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not stub out triton on a GPU host when the Xet backend fails to import The lazy loader retries `import unsloth_zoo.hf_xet_fallback` under UNSLOTH_ZOO_DISABLE_GPU_INIT=1 whenever the first attempt raises. That flag makes unsloth_zoo take its MLX/CPU path, which injects triton and bitsandbytes STUBS into sys.modules for the rest of the process. On a working GPU box whose first import failed for an unrelated reason (a bitsandbytes/CUDA mismatch, say) the retry succeeds, so Studio boots looking healthy and then dies at the first CUDA-only kernel: a GGUF or compiled diffusion generation hits the stub and returns NotImplementedError: Unsloth: 'triton.tools.experimental_descriptor.enable_in_pytorch' was called on Apple Silicon / MLX, where triton is stubbed out. so every image generation 500s with an Apple-Silicon message on a Linux CUDA host, while the load reports success. Found by loading Z-Image-Turbo GGUF through the API on a box where bitsandbytes could not initialise. Gate the retry on the host genuinely having no accelerator. The Xet stall watchdog is optional and already degrades with a warning; a process whose triton is stubbed out is not recoverable. The warning now says why it did not retry. * Fix the lost-generation proof set, the settle timeout and the hub inventory's diffusion gates Seven fixes from the latest review round on the Images page and the hub cache inventory. Images page: - The lost-POST settle path built its "already seen" gallery id set inside the catch, after the request failed. By then the earlier runs of the same batch had already prepended their records, so run 2 could accept run 1's image as proof that its own request reached the backend. The set is now captured once before the first POST and grows with every record the batch produces. - settleLostGeneration fell out of its SETTLE_MAX_MS loop and returned normally, so a wedged generation was counted as done and the next run started against a busy backend. It now throws on timeout. - Restoring a recipe cleared the ControlNet selection but left the workflow tab and the init / mask / reference images pointing at whatever was loaded, so the next Generate conditioned on an unrelated image. It now clears all of them and returns to Create. - The download plan omitted the adapter selection the load itself bakes in. A baked LoRA forces the dense build path, so the plan described a different file set than the load that followed and the rest was pulled inline, outside the download manager. Both now derive the list from one helper. Hub cache inventory: - A download for a repo an Images or Video load is staging was allowed to start: only the llama.cpp loader was consulted. Both diffusion backends already expose loading_repo_ids for the delete guard, and the download guard now reads them too. - A companion-only prefetch (pipeline manifest plus VAE and text encoder, no transformer) passed the snapshot-partial check, since every file its manifest expected did arrive, and was advertised as on-device although from_pretrained cannot load it. - The single-file flag never reached the picker through the hub inventory path, so a checkpoint-only diffusion repo read as a full pipeline and failed after the handoff. The two pipeline-shape helpers now live in hub/utils/inventory_scan.py so /api/models/cached and the hub inventory classify the same repos the same way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop adopting an unknown scoped download, leaking raced blobs and resurrecting deleted clips Three items from the latest review round. A scoped download job carries a deliberate file subset, and every file set of one repo rides the same "@scope" slot. A client that adopts a live job from the backend had no file list to compare against: the active-downloads response never carried one, so an adopted job's set was unknown and any later scoped request for the same repo read as "already started". Selecting a different checkpoint then waited on the wrong transfer and tried to load a file nobody fetched. The response now publishes the scoped file list, adoption records it, and an unknown set no longer satisfies a scoped request. A gallery record can be deleted while its blob is still downloading. The delete revokes the URL present at that moment, so the fetch that lands afterwards inserted a fresh object URL for a record no card renders and nothing can revoke: a full MP4, tens to hundreds of MB, pinned for the rest of the session, and once per raced fetch. Both galleries now discard a blob whose record went away, with an epoch covering the video page's Clear all. The video backend keeps the last completed job until the next one starts, and the Video page merges that record on mount to cover a job that finished after the gallery fetch. Deleting the clip left the record in place, so every reload prepended a ghost card whose file request 404s until another generation replaced it. Deleting the clip, or clearing the gallery, now clears the matching terminal record, and the page skips a record it deleted itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve revisions from the live cache, serialize dataset imports, and stop pinning every gallery blob Five more items from the review round. The conditioning-cache revision marker read huggingface_hub's import-time HF_HUB_CACHE constant. Studio can move its cache during a session and loading follows the live setting, so after a move the marker went unresolved (or pointed into the previous root) and pulling a new revision of the same checkpoint no longer invalidated the cache: a warm run could reuse the old encoder's embeddings and the old VAE's latents. It now looks in the active Studio cache first and keeps the environment and the library constant as fallbacks, which the trainer subprocess still needs. The dataset interlock counts mutations rather than excluding them, so two imports of different examples into the same empty name both got past the emptiness check. The winner promoted its staging directory atomically; the loser found the folder non-empty, fell back to a per-file move, and merged its images and captions into the winner's dataset. Imports now take a per-folder lock, a second one is refused with 409, and the emptiness check is repeated under the lock. On Windows the sd.cpp asset resolver filtered only by accelerator token, so a Windows arm64 host matched an x64 zip, downloaded and installed it, and failed later when the binary would not run. It now filters by architecture the way the Darwin and Linux branches do. Every gallery page fetched every PNG up front and kept the object URL for the session, so scrolling a large gallery grew memory without bound for tiles the user may never look at. The Images strip now fetches a tile as it nears view, like the Video strip, and keeps the eager path only where IntersectionObserver is unavailable. A 503 carrying a JSON body comes from the application, not a proxy, so it is surfaced as the error it is instead of entering lost-response settlement and being reported as a request that never reached the server. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Name the class of a failed generation instead of a bare "Image generation failed." Found on a macOS runner: the native renderer aborts inside its own text encoder there, and the page showed only "Image generation failed." with the sd-server backtrace left in the server log, so nothing about the failure reached the user. The failure is now classified into fixed text, out of memory and native-process death, so the message says what happened and what to try. None of the engine's own output is echoed, since a native tail carries local paths and argv; that stays in the log, and an unrecognised failure keeps the original literal. * Treat an undecodable caption sidecar as the tombstone the trainer sees Uploads store .txt and .caption sidecars as raw bytes, so one can hold invalid UTF-8. The trainer treats any existing sidecar, decodable or not, as an empty tombstone and never falls back to the metadata row for that image. The labeling grid and the dataset summary read an undecodable sidecar as absent instead, so both showed a metadata caption that the run would silently replace with the instance prompt, and counted the image as captioned. Both now track sidecar presence separately, so what the user reviews is what the run trains on. * Keep the reason a native server died, not just its backtrace A ggml abort prints its cause first and then a stack trace, so reporting the last twenty captured lines gave twenty addresses and nothing about the failure: on the macOS runner the native server died on an unimplemented Metal op and the message carried only frame pointers. The captured tail now leads with the lines that name a cause and keeps recent context after them, for both the startup failure and the mid-request death. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop staging the dense text encoder for an fp8 video load Two halves of the same gap, found while measuring the LTX-2.3 download plan: - The video download plan and the scoped pre-download never saw text_encoder_quant. An fp8 request loads a hosted pre-cast encoder, so asking for one still staged and downloaded the base repo's dense Gemma3 (48.79 GB of Lightricks/LTX-2 on the 2.3 distilled pick) that the pipeline then never opened. The plan now drops those shards and stages the pre-cast checkpoint instead; their configs stay, since the pre-cast loader still meta-inits the encoder from the base repo's component config. - The LTX-2.3 assembly builds every component itself, so pipe_kwargs (which carries the pre-cast encoder for from_pretrained) never reached it and an fp8 request silently loaded the dense encoder anyway. It is passed across explicitly now. The dense skip is earned, not assumed: only a pre-cast checkpoint that resolves on the Hub lets the plan drop the dense shards, and only one already fetched to disk lets the pull drop them, so an unpublished or gated artifact leaves both exactly as they were. If injection still fails after that, the load tops the dense weights back up rather than handing from_pretrained a snapshot with no encoder in it. Measured against the real Hub on the 2.3 distilled Q4_K_M pick: 67.24 GB before, 18.92 GB with a 0.43 GB stand-in for the pre-cast artifact (the base entry drops from 24 files / 48.79 GB to 13 files / 0.04 GB). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match the file's typing idiom for flow_shift models/training.py annotates with typing constructs throughout (105 Optional[...], no Union), and flow_shift was the one place using PEP 604. Union[] reads like the rest of the file, and it also drops the runtime evaluation that would raise on Python 3.9. * Bound the gallery blob cache, and three interlock fixes Four review findings, all reproduced first: - The gallery object-URL caches were unbounded. A clip runs from a few MB to a few hundred, both pages stay mounted after their first visit, and entries were only dropped on delete, so scrolling pinned everything for the session. Both pages now share a byte-budgeted LRU (512 MB video / 192 MB images) keyed off the visibility signal the near-viewport fetching already provides. On-screen media, the selected clip or image, and the item just fetched are never evicted, so eviction is invisible and a single item larger than the whole budget cannot evict itself into a refetch loop. - The image, video and chat load guards ran two independent training probes but returned early when the FIRST one raised, so an unreadable LLM backend disabled the diffusion interlock and a load could proceed straight into an active diffusion trainer on the same GPU. The probes are independent now. - An engine switch swallowed a failed teardown and published the new engine anyway, which is exactly the leak the unload exists to prevent: the arbiter's evictor, /images/unload and the next load all resolve through get_active_diffusion_engine(), so the still-resident pipeline (or a live sd-server) became unreachable and the next load allocated on top of it. The switch now fails and leaves the old engine published, so it stays reclaimable. - The native generation timeout was 30 minutes while the Images page waits up to 6 hours (SETTLE_MAX_MS), so slow-but-progressing CPU jobs died deterministically at the deadline. Measured on GPU-less runners, a 512x512 4-step Q2_K generation took 900 s on Linux and 1465 s on Windows, so larger images or step counts clear half an hour easily. The ceiling now matches the page's window and applies to the whole request: chunks of a split batch share one deadline instead of each getting a full budget. Cancellation is unchanged. Declined: gating the huggingfacenotorch extra off Python 3.9 over the conditional diffusers marker. The marker is deliberate and its comment says why: diffusers dropped 3.9 in 0.38, so pinning >=0.39 outright leaves pip no candidate and the whole extra unresolvable there. The pipelines it names live in studio/backend, which cannot install on 3.9 anyway (studio.txt pins matplotlib==3.10.9 and fastmcp>=3.0.2, both requires_python >=3.10), and the extra is the general core one, so the alternative drops 3.9 for library users who never touch Studio. * Close the load-versus-training-start race, and two picker fixes - The image and video load guards read is_active() and only then selected an engine, acquired the arbiter and registered the load. A /train/diffusion/start reserving inside that window freed residents the load had not registered yet, so the trainer came up beside a brand-new pipeline. The service already had exactly the right pattern for this in dataset_mutation, so gpu_load_admission mirrors it: reserve() refuses while an admission is open, an admission refuses once a start is reserved, both decided under the one lock. The span is only the registration, since begin_load returns as soon as the load is registered and _free_gpu_for_diffusion_training preempts an in-flight load from that point. Chat is deliberately not covered: its load spans an eviction plus a multi-minute GGUF load, and it admits models that fit beside training by design, which is a different contract from the diffusion pipeline's all-or-nothing one. - Hugging Face gives the LTX-2 family the image-to-video pipeline_tag (both Lightricks/LTX-2 and unsloth/LTX-2.3-GGUF report it), so a text-to-video-only filter dropped the flagship audio family out of Video Hub search while the rest of the app routed it to Video. - Task-scoped quant fit sized picks against the LARGEST visible device while resolve_diffusion_device_target returns a bare "cuda" and torch places on the current one. On a heterogeneous host that recommended a checkpoint sized for the bigger card and then loaded it onto the smaller one. Fit now uses the device the load actually lands on; identical on a homogeneous host. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Expose the persistent conditioning cache in the start schema DiffusionLoraConfig has carried cond_cache_dir for a while and the DiT trainer acts on it, but DiffusionTrainingStartRequest omitted the field, so Pydantic dropped it silently and every API-driven run fell back to the in-memory cache that is rebuilt from scratch each time. The warm path skips loading the VAE and the multi-GB text encoders on a rerun whose images, captions and resolution are unchanged, so this was a real capability that could not be reached. Contained like output_dir rather than left to the trainer subprocess's cwd, since it is another directory the trainer writes to. Blank or omitted still means the in-memory cache, so it must not resolve to the outputs root. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix diffusion policy and classification issues from review fp8 auto precision defaulted to precise accumulate on any non-consumer GPU, which made fp8 2.05x slower than int8 on RTX 6000 Ada and slower than not quantising at all. NVIDIA's professional whitepapers do publish equal FP8 rates for both accumulate modes there, so the hardware premise held, but the cost is in the cuBLAS path rather than the published rate. Default to fast accumulate: measured on B200 the flag is a no-op (4096^3 _scaled_mm at 3023.8 vs 3041.8 TFLOP/s, bitwise-identical output, 1.213 s vs 1.230 s end to end), so it is a large win where it bites and free where it does not. Precise accumulate stays available via transformer_quant_fast_accum. Z-Image's DiT is a Lumina2 derivative, so unsloth/Z-Image-GGUF and unsloth/Z-Image-Turbo-GGUF both declare general.architecture = "lumina2" and the whole line was tagged image-diffusion-unsupported and hidden from the Images "On Device" list, though validate_load_request loads them. Resolve shared archs from the repo/file name like bare "wan" already does, with a test asserting the picker and the loader agree for every family. The sage attention on-demand install ran an unpinned `pip install sageattention`, but PyPI's newest wheel is 1.0.6 and diffusers refuses anything below 2.1.1: the install always "succeeded", wrote an unusable version into the running venv, and was rejected on the next line. Carry the dispatcher's floor so pip resolves nothing instead. The dense-quant disk gate sized the download from the bf16-RESIDENT table. The fp32 families download twice that (Z-Image: 23,479 MiB against a 21,970 MiB gate), leaving a window where the check passed and the download filled the disk; Ideogram 4 ships fp8 and was overcharged the other way. Size the gate by published bytes, verified against HF sibling metadata for all 12 families. Patch installs went through unsloth_zoo, which refuses to import unless UNSLOTH_IS_PRESENT is set, and that is set by unsloth itself. The server imports unsloth at boot so it never showed there, but any other process ran silently unpatched with every install returning False, which is 13 test failures on a clean environment. Import unsloth and retry once, memoised per process. Also: the GGUF+LoRA refusal pointed at the native engine without saying a GPU host only selects it under UNSLOTH_DIFFUSION_ENGINE=sd_cpp, so the suggestion was unreachable; the gallery recipe recorded loras from the generate request alone, losing a load-time bake; load-progress claimed "40.07 GB downloaded" for a fully cached load; and pickers.tsx imported three catalog-group helpers it never used. Reported by oobabooga. * Keep the sd.cpp text encoder on CPU under Metal macos-14 loads FLUX.2-klein-4B Q2_K natively on mps and then dies on the first generation with exit code -6: ggml_metal_op_encode_impl: error: unsupported op 'RMS_NORM' -> ggml_abort LLMEmbedder::encode_prompt -> LLMRunner::compute -> GGMLRunner::compute ggml's Metal backend gates RMS_NORM on contiguous rows and aborts the process when that does not hold, with no per-op CPU fallback, so any LLM text encoder (Qwen3 for FLUX.2 and Z-Image, T5 for FLUX.1) takes sd-server down. The encoder runs once per prompt while the DiT runs every step, so pinning only the encoder keeps Metal for the part that matters. UNSLOTH_DIFFUSION_SD_CPP_METAL_TE_GPU=1 opts back in once ggml grows the kernel. * Gate the unsloth retry in the diffusion patch backend The retry added for the clean-environment patch failures is not free: importing unsloth pulls torch in behind it, which costs ~940 MB of RSS measured in a process that had neither, and on a host with no accelerator it fails anyway. A cross-platform CI job that had generated fine at ~900 s later died 19 s in with SIGTERM and every 'if: always()' step skipped, which is the runner being torn down rather than a step failing. Retry only when torch is already imported (true of the server and of anything patching a real module, and the condition that stops the retry from being what loads torch), unsloth is installed but not yet imported, and the first failure was the ImportError the sentinel guard raises. The clean-environment case it was added for still passes 29/29. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only retry the unsloth import where it can succeed The gate still let the retry run on hosts unsloth does not support, which is where it is most harmful: a 7 GB macOS runner lost the Studio server 26 s into a load, and the Linux runner was torn down mid-generation. Neither MPS nor plain CPU can complete the import, so the retry there pays the cost and fails anyway. Require an accelerator unsloth actually supports (CUDA/ROCm via torch.cuda, or XPU), with UNSLOTH_ALLOW_CPU as the documented override, and hoist the predicate to module level so it is tested directly rather than through the import system. On a CPU-only host the retry no longer fires at all; on CUDA the clean-environment case it was added for still passes 29/29. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard old diffusers, stream video exports, record conditioned recipes Three fixes from review. The 0.39-only pipeline classes (Flux2Klein, Z-Image, Krea 2, LTX-2, HunyuanImage) were resolved by getattr deep in the load, so on the older diffusers that packaging still allows on Python 3.9 -- diffusers dropped 3.9 in 0.38 and this project still supports it, so the 0.39 floor has to be conditional or the extra becomes unresolvable -- an advertised model failed with a bare AttributeError after its checkpoint had already been downloaded. Krea 2 already guarded itself this way; assert_pipeline_class_available now runs the same check for every image and video family from validation, before any fetch, and names the version and the fix. WebM export accumulated the whole VP9 output in a BytesIO and returned it as one bytes object that the response held again. The request caps allow 2048x2048 for 1024 frames, so an export runs to hundreds of MB and concurrent clicks could exhaust the process, while the MP4 route beside it already streamed from disk. transcode_to_file encodes to a temp file and the route returns a FileResponse with a background unlink, so nothing large is resident. A conditioned generation's recipe carried only the txt2img fields, so the gallery presented an inpaint or upscale result as a complete Create recipe and restoring it replayed an unrelated text-to-image request. The images themselves are still not persisted (user uploads with their own lifetime), but the workflow and its scalars are, restore reapplies them, and the toast now names the inputs that have to be supplied again instead of silently landing on Create. Reported by Codex. * Per-load video cancel event, family-gated image picker, cond cache refusal A cancelled video load could resume: begin_load cleared the shared cancel event, and unload() drops _loading without waiting for the worker, so the next load cleared the very object the cancelled worker was watching and its multi-gigabyte pull ran on alongside the replacement until the token check at the end. Each load now gets its own threading.Event, passed down through _fetch_te_prequant and _predownload_base, so a cancelled worker stays cancelled. A cached repo with a model_index.json was advertised as text-to-image on the trust rule alone, but validate_load_request also requires a detected image family, so a trusted pipeline of an unsupported class produced a picker row that deterministically 400s. The picker now applies both gates, mirroring the video branch. cond_cache_dir was accepted for sdxl and then ignored: only the DiT trainer reads it, while the SDXL trainer builds a per-run in-memory latent cache, so the promised cross-run reuse never happened. The route now refuses it with a 400 that names the families which do support it, checked against the resolved family so an omitted model_family with an SDXL base is caught too. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the frontend build broken by the gallery blob cache tsc -b failed on the branch head, so npm run build produced no dist and every platform job fell back to --api-only: blob-url-cache.ts(29,15): TS1294: This syntax is not allowed when 'erasableSyntaxOnly' is enabled dataset-labeling-grid.tsx / dataset-showcase.tsx: Argument of type '{ url: string; bytes: number; }' is not assignable to parameter of type 'string' The cache took its budget as a constructor parameter property, which the project's tsconfig forbids, and fetchGalleryObjectUrl now returns the blob size alongside the URL for that budget, which the two dataset thumbnail components still consumed as a bare string. Declare the field explicitly and destructure the URL at both call sites. tsc -b is clean and vite build emits dist again. * Recover from a ggml unsupported-op abort by restarting on the CPU backend ggml checks every node against the device's supports_op and calls GGML_ABORT when one is not implemented, because a single-backend graph has nowhere else to put it: there is no per-op CPU fallback. The whole sd-server dies with SIGABRT mid-generation and the user gets "the native image renderer stopped unexpectedly" with no way forward. Seen on macos-14 arm64 with FLUX.2-klein-4B Q2_K through the cross-platform CI: the text encoder is already pinned to CPU, and the abort moved into the denoise loop instead. ggml_metal_op_encode_impl: error: unsupported op 'MUL_MAT' -> ggml_abort StableDiffusionGGML::sample -> sample_k_diffusion A retry on the same backend would abort identically, so the load is restarted once with --backend cpu (the only flag that changes which backend executes the graph; --offload-to-cpu moves parameters, not compute) and the generation is re-submitted. The same checkpoint then renders slower rather than not at all. Strictly bounded: the signature must carry both the unsupported-op line and ggml_abort, the device must not already be CPU, and it happens once per load, so an OOM kill or a genuine crash still surfaces as itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix two tests that only fail in a full-suite run The 3.10 CI leg resolves PyAV 17, where av.container.OutputContainer is an immutable C type, so the no-libopus export test died on "cannot set 'add_stream' attribute of immutable type" before it asserted anything. Inject the refusal by wrapping the container av.open() returns instead; modules stay patchable on every build. Removing the injection makes the test fail again, so it still covers the branch it is named for. The Xet shim's degraded-path tests drop utils.hf_xet_fallback from sys.modules and import a throwaway copy. Restoring only the sys.modules entry left the utils package attribute bound to the throwaway, and the two disagreed for the rest of the process: a later monkeypatch of the dotted target patched one copy while the code under test imported the other, so the patch did nothing and test_fetch_te_prequant_only_reports_what_it_downloaded reached the real Hub and got a 401. Restore both bindings. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop an ignored-cancel sd-server, guard deletes during diffusion training, repair unusable managed binaries sd-server does not interrupt an in-flight job, so when it ignores a cancel the grace branch abandoned the poll and reported cancellation while the native job kept a core (or the GPU) busy to completion and held the server's job slot. The comment said the caller stops the server, but only unload does that immediately: a superseding load stops it after its multi-gigabyte download, and a load that then fails never gets there. Stop it here, as the deadline branch already does. DELETE /api/models/delete-finetuned checked only the LLM trainer, so it could rmtree the output directory a live diffusion LoRA run was about to write its adapter into. Consult the diffusion training service too, like the dataset mutation and model-load routes. find_sd_*_binary only checks is_file(), so an interrupted extraction (or a prebuilt for the wrong CPU) left a present-but-unrunnable binary the installer never retried: every load probed it, fell back to diffusers, and native inference stayed off until the directory was deleted by hand. Probe it and reinstall, but only for a copy under the installer-owned root -- SD_CLI_PATH, UNSLOTH_SD_CPP_PATH, an in-tree build and anything on PATH are the user's. * Plan the pre-cast text encoder, and make the cross-trainer GPU admission atomic An fp8 text-encoder request loads a hosted PRE-CAST checkpoint, but the image download plan never received text_encoder_quant, so the manager staged the base repo's dense encoder (FLUX.2-dev's Mistral-24B is ~48 GB, Qwen-Image's Qwen2.5-VL ~16.6 GB) and the load then pulled the pre-cast file inline, outside the manager's progress and disk preflight. The plan now takes the field, resolves the hosted artifact with the same resolver the injection uses, stages that file, and drops only those components' dense weight shards. The load's own prefetch takes the same treatment, since it paid the same cost. Only a checkpoint that really resolves on the Hub earns the drop, so a gated or renamed artifact still stages the dense encoder the load will fall back to. The two trainers admitted each other with independent check-then-act guards: the diffusion route checks the LLM backend several network-bound preflights before it reserves, and the LLM route checks the diffusion service well before it spawns, so two near-simultaneous starts could both pass and train on one GPU. reserve() now re-tests the LLM backend under its own lock, and the LLM route holds the diffusion service's gpu_load_admission across its spawn, so exactly one of the two wins. Both halves fail open, so a chat-only install still trains. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the Advanced options a staged download planned against Staging does not set busy, so while a multi-gigabyte download runs the user can still change precision, memory mode, speed or the baked LoRA selection. The pending record held only the repo and artifact, and the completed download fired a load that read the CURRENT state: the staged file set could then be missing files that load needs (fetched inline, with no progress and no disk preflight) or hold gigabytes it no longer uses. One snapshot of every Advanced control is now taken when the plan is built, and it travels with the pending record into the load, so the load that runs is the one the download was planned for. * Do not advertise a family the installed diffusers cannot build The newer families (Z-Image, Krea 2, FLUX.2, LTX-2, HunyuanImage) exist only from diffusers 0.39, and 0.39 cannot be installed on Python 3.9 at all -- diffusers dropped 3.9 in 0.38, so the requirement is conditional or the whole extra becomes unresolvable. On such an environment the picker still offered those rows, every pick failed deterministically, and the error's advice to run pip install -U diffusers could not fix it without also upgrading Python. The cached-repo picker now applies the same availability check validate_load_request does, which is keyed on the pipeline class actually present rather than on the Python version, so it is also right for an intentionally pinned older diffusers on 3.10+. Fails open when diffusers cannot be imported at all: that is a different problem and the load path reports it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the diffusion engine selection after each router test The active engine is module state, and several tests set it by plain assignment because what _activate does to it is the thing under test, so monkeypatch could not undo it. A leaked ENGINE_SD_CPP left get_active_diffusion_engine() handing back the sd.cpp backend for the rest of the process, and every later route that reads the active engine then saw an unloaded model: eight tests in test_openai_images_generations_route.py returned 503 in a full-suite run while passing on their own. The autouse fixture now snapshots and restores it. * Stream gallery clips, and close three races around them Four fixes from the latest review pass. The video gallery downloaded each clip into a blob before it could play, so playback waited on the whole file (tens to hundreds of MB), seeking was limited to what had arrived, and every viewed clip stayed pinned in the webview. The file route already streams and serves ranges; it just could not be a <video src> because it is bearer-gated. Mint a short-lived signed link instead (its own HMAC secret, 12 hour TTL, separate from the image links) and hand it to the element, which then fetches only the ranges it plays. That removes the blob budget, its LRU and every revoke on this page. The sd.cpp readiness probe accepted any process answering on the port, so a foreign server that grabbed the port between the bind check and the spawn was adopted as ours. Confirm the listener is our child before reporting ready, and stay best-effort (psutil missing, an unknown owner, or any probe error still passes) so the check can only reject a definitely foreign process. Dataset import held its lock for the extract but not for the upload path, so two concurrent uploads into the same folder interleaved; take the same lock and return 409. And reject Windows device names (CON, NUL, COM1..9, LPT1..9, with or without an extension) plus trailing periods in dataset names, which are unopenable on Windows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Build the image download plan for the engine that will load /images/download-plan always asked the diffusers backend, while /images/load picks the engine per host: a GGUF pick on a machine with no usable GPU routes to native sd.cpp, which reads a single-file VAE plus text encoders and never opens the base repo's sharded components. Measured on unsloth/FLUX.2-klein-4B-GGUF (Q2_K): the plan staged 7.66 GB of FLUX.2-klein-4B components the native load discards, and the 7.80 GB sd-cli actually needs was then fetched inline by the loader, outside the download manager's progress and its disk preflight. Z-Image-Turbo is the same shape. The plan now asks whichever engine the load will select. predict_engine() applies the selection policy without any side effect: it activates nothing (staging a download must not unload the resident model) and only locates the binary rather than installing it, but still counts an installable binary as available, since that is what the load does on a fresh host. The native backend gains a download_plan built from the same _asset_specs the loader fetches, returning the same envelope, so the manager stages exactly the files sd-cli opens. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not let a queued generation outlive the model, and three scan fixes Five items from the latest review; four were real. An unload or arbiter eviction only cancels the generation holding _generate_lock. A second request queued behind it holds no cancel event yet, and Python locks are not FIFO, so it could take the lock the instant the active denoise released it, still see a loaded pipeline, and run a whole new denoise after the model was told to go away: the eviction then waits minutes for it and an image lands after the eject. Unload and a superseding load now raise a fence under _lock before they queue, and a generation that wins the lock while one is pending refuses instead. The cached-model scan judged pipeline completeness across every revision, so a repo holding an older complete snapshot plus a newer companion-only one read as complete while the snapshot from_pretrained actually opens has no transformer. Both scans now look at the revision the loader will open. Deleting a dataset image deleted its caption sidecar unconditionally, which for cat.jpg alongside cat.png removed the caption the survivor still resolves to. The sidecar now goes only with the last image of that stem, matching what the thumbnail cleanup beside it already did. Importing an example into a folder that holds no images but does hold files fell back to promoting the staging dir one file at a time, so an interruption left a partial dataset that the image_count check accepts as complete on retry. Those files are folded into the staging dir instead and the promotion stays a single atomic rename. The MPS generator report does not apply: torch.Generator(device="mps") has worked since PyTorch 2.0 (pytorch/pytorch#91348) and the studio installer pins torch>=2.4. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten diffusion comments Collapse the multi-line comment blocks across the image, video, sd.cpp and diffusion-training code to one or two lines each, and drop comments that only restate the statement below them. Comments only, no code or behaviour changes. * Tighten diffusion comments (second pass) Collapse the remaining multi-line comment blocks in the video page, training routes and service, sd.cpp server and installer, memory and speed planners, and the shared request models. Comments only, no code or behaviour changes. * Tighten diffusion comments (third pass) Collapse the remaining multi-line comment blocks in the attention, cache, LoRA, prequant, precision and compile-cache modules, the sd.cpp arg builder and engine, the video routes, the Ideogram 4 assembly, the model picker, and the diffusion test suites. Comments only, no code or behaviour changes. * Restore the dataset when an example import cannot be promoted Promotion folds the folder's pre-existing entries into the staging dir so the swap is one atomic rename. Every failure after that fold left the user with nothing: the 409 path and an os.replace error both fell through to 'finally: shutil.rmtree(staging)', which deleted the entries that had just been moved in there, while the response said 'Nothing was written'. A same-named entry was also unlinked outright before the promotion was known to succeed. Park superseded same-name entries in a rescue dir instead of deleting them, record every move, and restore all of them if any step of the promotion fails. The fold loop itself is covered too: renaming a non-writable directory raises EACCES on POSIX, which previously escaped as a 500 after the earlier entries had already been moved out. A failed rename now maps to the same retryable 409 as the rmdir conflict. Verified with the reported trigger (a non-empty mode-500 directory whose name collides with an imported file): the folder listing is now identical before and after the failed import. * Drain the teardown fence on a failing unload, give each load its own cancel event Two independent leaks on the image path, both already solved elsewhere in the same file. unload() incremented _teardown_waiters, ran _unload_locked() and decremented, with no try/finally, while the superseding-load path used a finally for the same pair. _unload_locked ends in clear_gpu_cache(), whose CUDA branch calls synchronize/empty_cache/ipc_collect unguarded, and a sticky CUDA fault makes those raise. The count then never drained, so every later generation was refused as cancelled for the life of the process, a fresh load included, since begin_load's own increment and decrement are symmetric. unload() is reached from the chat/video GPU handoff, the engine router and two training routes, so one fault during an ordinary handoff wedged image generation until restart. Release it in a finally. The image and native backends each cleared one shared cancel Event on a new load. unload() sets that event to cancel an in-flight multi-GB download and drops _loading in the same breath, so a replacement load is admitted while the cancelled worker is still inside the fetch, and its clear() re-enabled the very object that worker was watching: the cancelled download resumed and ran alongside the replacement. Take a fresh Event per load and thread it to the worker, as the video backend already does, and set it under the lock since begin_load now rebinds the attribute. * Name utf-8 on the diffusion text I/O and the sd.cpp subprocess pipes tests/test_text_io_encoding.py failed on five files this branch adds. Text I/O without an explicit encoding falls back to the Windows ANSI codepage, so a non-ASCII path or manifest value round-trips corrupted, and the three sd.cpp pipes decode the child's UTF-8 output as ANSI on Windows despite already passing errors = 'replace'. Eleven read_text() / write_text() sites across diffusion_compile_cache, diffusion_ideogram4 and diffusion_krea2, plus text = True on the sd-cli version probe, the sd-cli run and the sd-server pipe. * Record the load-time build on a gallery image's recipe A gallery record documents itself as the image's full generation recipe and is embedded in the PNG, but the only load-related field it carried was the repo id. A GGUF repo holds many quants, so that does not say which one made the pixels, and it says nothing about an adapter baked in at load time. The fallback meant to cover the baked case could never fire: with no loras on the request _adjust_baked_loras zeroes every baked adapter and _active_lora_pairs drops zero-weight entries, so active_loras was always empty. A baked-and-disabled build is not the same pipeline as a never-baked one, so the recipe could not reconstruct the image once the model was rebuilt. Persist model_kind, gguf_filename, transformer_quant and the baked adapter names, read off the load state rather than the request, and show them in the recipe popover. The new fields are optional with defaults, which matters because list_gallery_images drops any record that fails validation, so a required field would have emptied every existing gallery; a regression test pins that. * Drop eleven duplicated comment tails, restore the mxfp8 denial note The comment passes collapsed several wrapped blocks onto one line without deleting the last physical line of the original wrap, leaving the tail of each sentence repeated as its own comment underneath. Two of the eleven were re-worded rather than byte-identical, so a strict suffix match missed them. 6e16ad16f also dropped the line justifying the qwen-image mxfp8 denial while that rule stayed live in _FAMILY_SCHEME_DENY, under a header that then documented only fp8 and nvfp4. Restored. Comments only; verified with the AST gate. * Only let the dense-quant fallback use shards the prefetch actually staged The prefetch skips the base repo's transformer/ shards whenever a prequant checkpoint is expected, since that checkpoint replaces them. But a prequant fetch can fail for reasons the planner cannot see: an unpublished, gated or renamed artifact, a hub 5xx, a proxy, a checkpoint the validator rejects. The loader then fell through to from_pretrained(subfolder = 'transformer') and pulled those shards inside the load lock during 'finalizing', after the previous pipeline was already evicted, where the cancel event has no reach, load_progress has already reported bytes_downloaded == bytes_total, and the cache-disk gate had only reserved the small prequant checkpoint. That is verbatim the situation _dense_quant_prefetch_needed's own docstring exists to prevent. Gate the in-loader dense fallback on the shards being staged, read off the returned file list rather than the request so a failed size estimate closes it too, and let the GGUF build take over otherwise, which is what the prequant-sized replan already does one branch over. It is also the invariant the text-encoder path already enforces: only a component that really resolves may have its dense weights dropped from a plan or a prefetch. Adds the missing coverage for the gate's prequant arm, which the existing disk-gate tests never reached. * Refuse a training output_dir that resolves to the outputs root resolve_output_dir strips a leading 'outputs' and drops '' / '.' segments, so '.', './', './.', 'outputs', 'outputs/outputs' and ' . ' all clean away to nothing and land on the outputs root itself rather than a run directory under it. The DiT trainer then writes pytorch_lora_weights.safetensors flat into the root, where scan_trained_models and scan_checkpoints cannot see it (both filter is_dir()), and a second such run overwrites the first. The UI only checks the field is non-empty, so 'outputs' is one plausible run name away. Refuse it with a 400 that says what to do instead. cond_cache_dir collapses the same way but has an honest 'off' to fall back to, so a root-resolving value now means the in-memory cache, which is what the comment above it already promised: otherwise a run drops one flat safetensors per cached latent and caption into the directory trained models live in. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Repair three defects the main merge left behind Git merged all three files without a conflict, but the result was wrong in each case. The tests only started failing once the merged tree was run. routes/inference.py duplicated the GGUF load block: this branch had moved the gguf_load_in_flight marker and the _hub_download_blocks_gguf_load guard under "if config.is_gguf and config.gguf_hf_repo", and the conflict resolution re-added main's copy at the old position, so both ran. Dropped main's copy; the earlier placement is the deliberate one, so a 409 from the hub guard cannot tear down a resident Images or Video pipeline. test_gpu_selection.py still called _hf_offline_if_dns_dead, which main renamed to _hf_offline_if_unreachable_for (#7591). Disjoint edits, so no conflict, but four route-error tests referenced a function that no longer exists. test_gguf_load_cache_reuse.py anchored its ordering assertion with rindex over "if config.is_gguf:", taking the last one before the load marker. That only held while _resolve_inherited_extra_args sat above every such line; main has since hoisted it above the gpu_ids preflight, so the anchor landed between the call and the marker and the assertion compared against an unrelated later call site. The ordering it checks is a property of _load_model_impl as a whole, so it now anchors on the function. The ordering itself is intact: _resolve_inherited_extra_args, then the gguf_load_in_flight marker, then the hub guard, then the chat handoff, then unload_model. * Stop a real sd.cpp install from breaking its own discovery tests The five "nothing is installed" assertions in test_sd_cpp_engine.py cleared SD_CLI_PATH and UNSLOTH_SD_CPP_PATH and patched Path.home, which covers hops 1, 2 and the fallback half of hop 3. It leaves two hops live. Hop 3 goes through managed_install_root(), which honors UNSLOTH_STUDIO_HOME and STUDIO_HOME and resolves to the stable-diffusion.cpp directory beside the studio home. That is the documented way to run side-by-side Studios, so anyone who has one set gets a real sd-cli back from a finder the test expects to return None. Hop 4 is the in-tree developer build, which does the same for anyone who built sd.cpp inside the checkout. Isolated with an autouse fixture rather than another helper call, because reaching the failure needs no fixture: SdCppEngine(binary = None) runs the finder from its constructor, which is why test_generate_raises_when_binary_missing failed too. The fixture points the studio home and the in-tree root at an empty tmp tree, so every hop is answered by the test rather than by the box. The in-tree root moves into a named in_tree_install_root() so it can be pointed somewhere empty; behaviour is unchanged, including the OSError and IndexError guard on an unexpected layout. * Make the sd.cpp uninstall test actually extract the code it tests The two sed ranges anchored the production fragments at column 0, but both blocks sit inside the main removal function and are indented, so each range matched nothing and LOOP_FILE / DEFAULT_FILE came out empty. Sourcing an empty file is a no-op, so the suite reported 4 passed / 7 failed: the seven removal assertions failed because nothing ran, and the four that passed were 'kept' assertions that pass trivially when nothing runs. The shell job auto-discovers tests/sh/test_*.sh and runs each under set -e, and this file is not in its skip list, so it was a deterministic red. Anchor on optional leading whitespace and fail loudly on an empty or _remove_path-less fragment, so a future reshuffle of uninstall.sh cannot make the suite vacuous again. Now 11 passed, 0 failed, against the real removal loop and the real default-mode block. * Agree on what an engine can build, and on what the installer owns Two gates, each half-applied. The unbuildable-family gate had one caller, the image branch of the cached repo picker. The GGUF classifier, the local-model classifier and the video branch had none, so on a diffusers too old for a family the picker still offered its GGUF and the load then failed. Meanwhile the loader asserted the diffusers pipeline class before engine selection, so a GGUF this host routes to native sd.cpp, which instantiates no pipeline class at all, was refused with an upgrade instruction that could not help. Both are wrong one-sidedly: hide a family only when NEITHER engine can build it, and demand the diffusers class only when diffusers is what will load it. One predicate, family_buildable_here, now answers both, so the picker and the loader cannot disagree. The population is real on Python 3.9, whose diffusers ceiling is 0.36: Flux2KleinPipeline, Krea2Pipeline and LTX2Pipeline are all absent there, and FLUX.2-klein GGUF is a repo in this PR's title. That assertion also raised RuntimeError, which /images/load maps to 409, the status that otherwise means a load is already in progress, and which escaped /images/download-plan (it catches ValueError and FileNotFoundError) as a bare 500 with the message lost. It is an unloadable pick like every other, so it raises ValueError and both routes answer 400 with the text intact. The managed-binary repair used a path test for ownership while the installer uses a marker. On a managed root without the marker, which is any install predating it, the repair deleted sd-server and the reinstall was then refused, permanently, because the surviving sd-cli keeps the directory non-empty so the marker can never be claimed: the user went from an unrunnable binary to no binary and no way back. Require the marker before discarding, which is the same definition of ours that uninstall.sh already uses to keep a user's own stable-diffusion.cpp checkout. A genuinely interrupted extraction still self-heals, since install() writes the marker before it extracts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add gguf and av to the studio extra so the wheel install matches studio.txt This branch added both to studio/backend/requirements/studio.txt, which is what install.sh uses, but never to the studio extra in pyproject.toml, which is what 'pip install unsloth[studio]' uses. Nothing else keeps the two in sync, so tests/studio/install/test_studio_extra_matches_requirements.py failed deterministically, and the CI job runs the whole tests/ tree. The drift is not only a red test: gguf backs diffusers' GGUFQuantizationConfig and av does the MP4 encode and audio mux, so a wheel install got a Studio that cannot read a GGUF or export a video, which is most of what this PR adds. * Fix two test-suite defects: the CPU-only patch gate and a popped module The patch backend reaches unsloth_zoo's helpers only through an ~940 MB 'import unsloth', which a CPU-only host cannot complete, so every patch install returned False and nine arch/eager/compile tests failed there -- exactly the runners the retry was narrowed to protect. unsloth_zoo only wants UNSLOTH_IS_PRESENT in the environment, which costs nothing and needs no accelerator, and the conftest already sets its sibling UNSLOTH_ALLOW_CPU. Set it there too, as run.py and main.py already do at module scope, so no real server ever took the expensive route. CPU-only goes from 9 failed to 24 passed, 1 skipped; the GPU run is unchanged at 29 passed. test_setup_cache_env_hf_home popped utils.hf_cache_settings to model a fresh process and never restored it, so a later import built a second module object and rebound it on the utils package. test_hf_cache_settings then wrote its setting into one object while core.inference.diffusion read the other, the same split-module failure the xet shim already had. Restore both bindings in teardown: 21 pass together, and each file still passes alone. * Restore the setup_fail assertions the main merge reverted A fourth instance of the class 9541cc535 fixed: the merge took main's studio/setup.sh, which #7644 changed to abort through the setup_fail helper so desktop mode still emits [TAURI:ERROR], but kept this branch's older copy of the test, which still asserted the literal 'exit 1'. setup.sh is byte-identical to main here and the only diff in this file was the reverted assertions, so take main's version. 60 passed. * Fence the video teardown, reject non-finite clipping knobs, stop a test installing sd.cpp The video backend never had the teardown fence the image backend documents. Its unload and its superseding-load path both signalled the active generation, then did 'with self._generate_lock: pass' and tore the state down with the lock free. A generation queued behind that barrier holds no cancel event yet, so the signal cannot reach it, and Python locks are not FIFO: it won the lock the instant the barrier released it, read a still-loaded state and denoised a whole clip against the pipeline being freed. Reproduced on both paths, where the queued generation returned a finished MP4. Mirror the image backend: count waiters, refuse a generation while one is pending, and tear down inside the barrier with the counter released in a finally. _teardown_state becomes _teardown_state_locked since the caller now holds both locks. max_grad_norm and snr_gamma were bounded on one side only, so 1e309 floated to inf and passed. clip_grad_norm_ then computes an infinite clip coefficient, clamps it to 1.0 and scales nothing, and min(snr, inf) / snr makes every min-SNR weight 1.0. The run starts, reports normal progress and trains with the requested knob silently disabled. Measured both. NaN already failed the bounds; allow_inf_nan makes that explicit. learning_rate and flow_shift already guard this exact vector. test_load_routes_to_sd_cpp_on_cpu stubbed ensure_sd_cpp_binary but not ensure_sd_server_binary, which select_and_activate_engine probes first with installs enabled, so the unit test downloaded and unpacked 108 MB into the developer's real ~/.unsloth when UNSLOTH_STUDIO_HOME was unset. Now 0 bytes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Key the video schedule reset on the resolved defaults, not the repo id A GGUF repo holds several variants, so another client swapping a distilled build for the base one from the same repo changes the steps and guidance the backend reports while repo_id stays put. modelChanged stayed false, the page kept the previous schedule, and later generations ran an 8-step distilled setting on a model that expects roughly 40 steps and CFG 4. Include the reported defaults in the key. They only move when the resident artifact does, so a manual steps change still survives a status poll. * Retire the release-lag pin now that 2026.7.6 carries the relaxed gates The virgin Windows container row that installs from PyPI on purpose was tolerated with continue-on-error, because a Server Core container has no Microsoft Store and so no winget, and the released studio/setup.ps1 hard-stopped on a winget-only git gate and reached for winget again for the VC++ runtime. #7549 relaxed both, but it landed on 2026-07-28 and the newest wheel was 2026.7.5 from the 23rd, so the row could not get past it. The pin carried its own tripwire for exactly this moment, and it has fired: unsloth 2026.7.6 shipped 2026-07-29 and the released wheel now installs end to end, so the step reported success and the assertion errored with "delete this pin and drop continue-on-error from the Install step so this row gates". Doing that. Both rows now gate unconditionally, and the assertion that enumerated the accepted failure signatures goes with the pin it protected, since there is no longer an accepted failure for it to describe. The other three release-lag mentions in this file are descriptive rather than tolerated-failure pins: they explain why the fedora and ubuntu-nonroot legs run with overlay: true, and both already refuse a triton failure as a regression rather than accepting it as lag. Left alone. * Pin the hosted pre-quant onto the plain-torch fp8 kernel A Z-Image GGUF pick at the DEFAULT speed mode, once the hosted fp8 pre-quant is actually reachable, dies at generate with an HTTP 500: torch._dynamo.exc.Unsupported: Operator does not support running with fake tensors. Developer debug context: unsupported operator: mslk.f8f8bf16_rowwise.default _fp8_config already pins KernelPreference.TORCH when it BUILDS a config, precisely because the default AUTO switches to the MSLK kernel wherever an mslk package is importable (sm90+). A hosted checkpoint escapes that pin completely: the preference is serialized per Float8Tensor, and all 239 weights in the published Z-Image-Turbo-FP8 checkpoint carry AUTO. Loading it re-arms the very kernel the pin exists to avoid, mslk.f8f8bf16_rowwise has no fake impl, and the first compiled generate therefore cannot be traced. Isolated away from the product to be sure of the mechanism: quantise one Linear three ways on this box and compile each. KernelPreference.TORCH eager ok, compiled ok KernelPreference.AUTO eager ok, compiled FAILS on mslk.f8f8bf16_rowwise library default eager ok, compiled FAILS the same way So the pin is correct and necessary, and the only gap is that the hosted path never got it. _validate_checkpoint checks scheme, granularity, base model, min_features, exclude tokens and fast_accum, but not this. Rewriting the preference on load is safe: it selects a matmul kernel, it is not weight data, so the tensors stay bit-identical and the checkpoint's own state_dict_sha256 still describes them. It is also the faster path compiled, since the opaque extern call blocks inductor quantize fusion. Why this went unnoticed: the pre-quant repos are private, so nothing that could not read them ever took this path. It becomes the default the moment they are readable. Verified end to end at 1024x1024, 8 steps, speed_mode default, on the published checkpoint: before HTTP 500 at generate, diffusion.generate_failed after pinned 239 weights to the plain-torch fp8 kernel, load 10.0 s, transformer_quant fp8 with compiled engaged, cold 7.9 s, warm 0.87 0.87 0.87 0.86 0.87 s That warm number also beats the 1.4 s recorded for this shape on 2026-07-26. 159 prequant and transformer-quant tests pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover the fp8 kernel pin so the 500 cannot come back quietly The hosted pre-quant re-arming MSLK was only visible as an HTTP 500 on a compiled generate, which needs a GPU, a readable private repo and an importable mslk to reproduce. None of that is available in CI, so the pin would rot unnoticed. Three hermetic cases on _pin_kernel_preference instead: AUTO weights are rewritten and already-TORCH ones are left alone (counting only what changed), a weight that refuses the assignment does not sink the whole load, and with no torchao enum available the checkpoint is left exactly as saved. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: tighten comments across the image generation, video and training code Rewrite the comments added by this branch to be shorter and clearer: collapse multi-line explanations into one or two lines, drop comments that only restate the code, and keep the rationale that explains why a choice was made. No code changes. * Studio: let the pipeline-class guard survive a host with no diffusers Backend CI installs the CPU-only dependency set, which has no diffusers, and runs without a GPU. 13 tests failed there for three separate reasons, one of which is a real product bug rather than a test-environment gap. assert_pipeline_class_available did a bare "import diffusers", so on any host without it the call raised ModuleNotFoundError instead of the ValueError its own contract promises. /images/download-plan catches only ValueError and FileNotFoundError, so that escaped as a bare 500 with the message lost, which is the exact failure the guard exists to prevent. It is reachable in production: the native sd.cpp engine serves GGUF picks on a CPU or Apple host where diffusers is never installed. Absent diffusers now returns, since the check answers "is the installed diffusers new enough for this family" and there is no version to judge; a pick that genuinely needs diffusers still fails in the loader with its own message. The rest were tests asserting through gates unrelated to what they cover: - Three cond_cache_dir route tests drive a DiT family, which the start route refuses without an accelerator. They now take the existing dit_train_host fixture, written for this case, so they keep testing the schema on every host. - test_download_plan_forwards_the_load_time_controls stubs the diffusers planner but sends a GGUF pick, which routes to the native planner on a GPU-less host. It now pins the engine, so it tests kwarg forwarding rather than the hardware. Engine selection keeps its own tests. - The pipeline-class guard test needed a real diffusers only for its sweep over every shipped family. That half is split out and skips on its own; the stub-driven refusal needs no diffusers and still runs. Added a test for the absent-diffusers contract above. 409 pass on a GPU host with diffusers; 301 pass and 1 skips with diffusers blocked and no GPU. * Studio: cut the image generation comments down further Second pass over the comments this branch adds. Collapse the multi-line blocks that still read as paragraphs, rewrite the longest one-liners so they say the same thing in fewer words, and drop a stale comment that had drifted away from the constant it described. No code changes: only comments and whitespace. * Studio: workflow rail, sidebar Video row, and Train settings polish Sidebar - Pin Video under Images by default; persist store bumped to v3 so an untouched install adopts it while a custom arrangement is left alone. - Add "Customize sidebar" to the end of the More flyout, opening Settings > Appearance scrolled to the sidebar nav section. Images - Replace the workflow dropdown with an icon rail down the left edge: all seven workflows visible, keyboard nav, tooltips carry the labels. - Workflows stay selectable with no model loaded, so one can be set up before picking a model. Generate is already gated on a loaded model. - Video moves out of the Create/Train strip into a link at the far right, with a matching Images link on the Video page. - Wider gutters around the settings column, headings matched to Train. Train - Field guidance moves into "i" tooltips; only state that limits a control stays on the page. Steps, LoRA rank and the rest gain hints. - /info reports params, qlora_vram_gb, gated and note as fields, so the family note renders as chips. vram_note is rebuilt from them unchanged for older clients. Shared - Number steppers appear on hover or focus. - Lighter control border (#e9e9e9) on Images and Video, light mode only. * Studio: move Advanced inline on Images and Video - Replace the top-bar toggle and right-docked Advanced panel with a disclosure under Seed, so load-time tuning sits with the settings it affects and opening it no longer shrinks the preview canvas. - Video uses the bordered variant, since offload and memory decide whether a model fits at all. - Open state persists per page in localStorage. - Tighten the Steps unit trigger: 14px chevron, less right padding. * Studio: full width media pages and layout polish - Drop the 1100px cap on Images Create, Images Train and Video, so the preview canvas grows with the window instead of sitting in a band. - Align the workflow rail to the model selector label, widen the gap before its divider, and restore the divider itself. - Advanced: more room above and below, icon and label sized to the slider rows, and the same quiet row on Video as on Images. - Give Video a pane heading and description, matching Images. - More space above Seed on both pages. - More no longer takes the active style when the current page is one of its own rows. * Studio: fix the Extend side toggles The resting outline used a ring, and index.css blanks the ring on the button holding mouse focus, so the side you just clicked lost its border until focus moved. Use a border for the outline and leave the ring to focus-visible. Also separates the two states properly (the off state had no surface of its own), drops the border in dark mode as the inputs do, and adds the aria-pressed and focus-visible styling the buttons were missing. * Studio: drop the border on the Extend side toggles Light mode should carry no border either, so the fill alone marks the state: a muted surface when off, a primary tint when on. Matches the borderless treatment dark mode already had. * Studio: move the Images workflows into the sidebar The workflow switcher was a vertical icon rail on the page. It now lives in the sidebar under Images, so the page keeps that width for the canvas. - Workflows list under the Images row, with a chevron to fold them away. Hovering the row peeks them in a flyout on the standard menu surface. - Clicking Images while already there toggles the list instead of navigating. - The listed workflow carries the highlight, so the Images row drops it. - Create takes its own icon: it was sharing the New chat pencil. - Model hub moves above Projects. Persisted layouts bump to v4 and only adopt the new order where the stored one is still a shipped default. - Images and Video content both start at 32px, clear of the sidebar. * Studio: simplify the Images workflow submenu Dropping the hover flyout: two ways to reach the same seven workflows, one of them overlapping the list right below it, read as clutter. - The flyout is gone. The workflows are rows under Images and nothing else. - On the Images page they are always open, since they are that page's switcher. Elsewhere they stay folded, and the row's chevron opens them. - Create takes the sparkles icon. - Create and Train both pad their settings column to 40px a side, so the two tabs line up with each other and with the model selector. * Studio: dock the generate action and tidy the Create controls The primary action sat at the foot of a long scroll, so it was off screen until you scrolled for it. - Generate, Video's Generate and Train's Start training float at the bottom of their settings column. No bar behind them: hover lightens the fill rather than thinning it, and the disabled state is opaque, so the controls underneath never show through. - Aspect ratios read as names: Square (1:1), Widescreen (16:9) and so on. - Width and height replace their sliders with two compact boxes. Type a size or pick one from the menu; the value still snaps into range and still drives the locked ratio. - Negative prompt gets a quiet disclosure under the prompt. It was there before but only above guidance 0, which is not the default, so it never showed. - The info "i" is smaller across the settings UI. * Studio: reveal Images and Video field hints on hover The "i" next to every field label sat there permanently, which made a column of settings read as busier than it is. Scoped to the diffusion pages by CSS rather than per component: the hints come from the page's own Field and SelectRow, the train panel's FieldLabel, and the chat ParamSlider, which is shared and should not change for chat. Reveals when the pointer is inside the field, on focus-within for keyboard, and stays inert while hidden. InfoHint grows a data-slot so the rule has a stable hook, rather than keying off its aria-label. * Studio: one-line sliders, edge fades, shared negative prompt - Steps, Guidance and the rest put label, track and value on one row. ParamSlider is shared with chat, so this is an opt-in prop and chat keeps the stacked layout. - The settings columns fade at whichever edge they run past, as the sidebar and model picker do, instead of cutting off. New useScrollFades hook drives all three. - Negative prompt moves to a shared component and Video picks it up, so both pages collapse it the same way. - Right gutter sits closer to the rule, Steps takes a bigger break above it, and the Advanced rule sits between its neighbours rather than up against the field above. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add the API monitor to the sidebar nav Sits under More, below Export. Points at the monitor page, not the API keys dialog the profile menu opens. No store version bump: an id missing from a stored layout is appended with its default pinned state, so existing sidebars gain the row at the end and keep their own order. * Studio: take main's copy of the sidebar spinner test The merge kept this branch's older version, which pins the Recents row to h-[33px]. Main already relaxed that to any height, since row density moves independently of the trailing column the test is about. * Studio: fix FLUX.2 klein size resolution, HunyuanVideo tier routing, and the personalization save Five fixes found while auditing the model registries against the live Hub. FLUX.2 klein 9B loaded against the 4B config. One family covers both klein sizes and defaults to 4B, relying on the base_model card tag for the real base, but the trust allowlist only had klein-4B, so the correct tag was discarded and a 9B GGUF (inner_dim 4096) was loaded against a 4B config (3072). That surfaced as a bare shape mismatch from inside the GGUF quantizer naming neither the file nor the repo. Adds the missing allowlist entries and a header-level size check that fails early with a legible message; the check is fail-open, so an unreadable file, a non-FLUX.2 family, or an unmapped base leaves the load exactly as it was. The sd.cpp text encoder rule matched the literal "klein-9b", so klein-base-9B was handed the 4B encoder. HunyuanVideo-1.5 720p checkpoints routed to the 480p family. Only the literal 720p_t2v path was aliased, so 720p_i2v and every GGUF repack fell through to the generic token and inherited the 480p base repo, which also supplies the VAE and text encoder. The tier is baked into the weights (transformer target_size 640 vs 960, scheduler shift 5.0 vs 9.0) and the two bucket lists are disjoint, so this ran the whole pipeline off-tier. Resolution presets that the checkpoints were never trained for. Wan2.2 TI2V-5B is 720P-only upstream (SUPPORTED_SIZES is exactly 704x1280 and 1280x704, asserted in generate.py), and the HunyuanVideo 480p square preset 624x624 is not a bucket of its tier; 640x640 is. Personalization could never be saved. The frontend ships eight sidebar nav ids and refills every missing one on each save, but the backend Literal had seven and no "api", so every PUT to /api/settings/personalization returned 422. Aligns the server defaults and the Literal with the shipped layout. A catalog row pointing at a file that does not exist. unsloth/Qwen-Image-2512-FP8 holds torch prequant .pt checkpoints, not qwen-image-2512-fp8.safetensors, and fp8 is denied for this family anyway because it renders black, yet the row was the auto-route target on any GPU above 40 GB. Also marks FLUX.1-schnell, Krea-2-Turbo and the two Ideogram repos gated: all four are gated on the Hub today, and schnell being Apache-2.0 does not make it ungated. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stop a case-only duplicate upload destroying an image, and size the load against the right GPU Two fixes from the review round. A dataset upload containing two names that differ only by case silently loses one on Windows and macOS. The in-batch duplicate check is an exact string compare, so "Cat.png" and "cat.png" both pass, but on a case-folding filesystem they are one destination: the commit step moves the first staged part aside, writes the second over it, then deletes the backup, while the response still reports both as uploaded. Captions collapse the same way and never even reach the image branch. No single folder can hold such a pair, but the Windows and macOS open dialogs flatten search and Recents results across folders into one multi-selectable list, which is a normal way a LoRA dataset gets assembled. Rejecting the pair everywhere would regress Linux, where the two really are different files with their own sidecars, so the check probes the filesystem once per process instead of keying off sys.platform: macOS also ships case-sensitive APFS volumes, and a Linux host can keep its Studio home on an exFAT or NTFS mount. A failed probe answers "case-sensitive", which leaves today's behaviour untouched. The quantization fit budget was sized against the wrong card under a reordering CUDA_VISIBLE_DEVICES. A bare "cuda" load lands on visible ordinal 0, but the reducer took the lowest PHYSICAL index, which the nvidia-smi path reports as index_kind "physical". Under "3,1" that sizes against GPU 1 while the pipeline loads onto GPU 3. The hook's own interface doc already promised the lowest visible ordinal. Ranks by visible_ordinal, falling back to index only for an older backend that omits it. * Studio: rework the diffusion dataset panel and align the pane gutters Upload: - one Upload button beside the dataset name, with an upload icon - picking files uploads them, no second confirm click - Add button beside the Training images dropdown, so images can go into a set that already exists - fall back to the upload form when a selected dataset no longer resolves Labeling grid: - two columns at any width; sm:grid-cols-3 crushed the tiles in a fixed width column - pin the caption size so the Textarea's md:text-sm does not outsize a tile - name the tile hover group; a bare one revealed every tile's Remove at once - Remove is an icon button on the image, not a word over the artwork - drop the amber tile fill, keep the No caption label - match the header, toggle and status text to the section's type scale Gutters: - the run area and the preview canvas now sit 40px off the rule, the gutter the settings column has off the page edge * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: give the dataset preview per-image hover and remove, and wrap it onto rows - each thumbnail is its own button, so it can carry an X that removes just that image; the strip was one button and could not nest a second control - hover lightens the image instead of drawing a border - wraps onto more rows rather than scrolling sideways, which also keeps the bottom corners rounded - samples 12 thumbnails, up from 8, now that a second row is available - a delete refreshes the panel's dataset counts * Studio: show the workflow icon beside the Images and Video headings Images reads it off WORKFLOW_TABS, so it is the same icon the sidebar submenu shows and it follows the active workflow. Video's heading is also Create, so it takes the Create icon rather than the nav row's film slate. * Studio: reject a duplicate LoRA id on the diffusion load path too DiffusionGenerateRequest already refuses a repeated adapter id, with a comment naming the hazard: _resolve_lora_set suffixes colliding adapter names, so the same id resolves the SAME adapter twice and set_adapters stacks both copies past the per-adapter weight bound. DiffusionLoadRequest bounds only the list length, and it matters more there. Generation-time stacking spoils one image; on the load path the adapters are baked into the quantized build before compilation, so the unintended combination rides every image until a reload. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: teach the GGUF reuse test double about holds_no_vram The resident-GGUF fast path consults llama_backend.holds_no_vram before asserting CHAT ownership, but two tests hand it a SimpleNamespace built before that attribute existed, so they raised AttributeError and returned a 500. A real LlamaCppBackend exposes it as a property; the doubles now carry it too. Product code is unchanged: these pass on main and were the branch breaking its own test, not a defect in the guard. --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: michaelhan <michaelhan2050@gmail.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>	2026-08-04 08:11:01 -07:00
Daniel Han	425c9081a3	Uninstall the whisper.cpp prebuilt and the node/whisper install locks (#7686) ... * Uninstall the whisper.cpp prebuilt and the node/whisper install locks scripts/uninstall.sh removed the llama.cpp prebuilt and its install lock but not the whisper.cpp prebuilt, and not the node or whisper install locks. Every prebuilt serializes on <parent>/.<name>.install.lock (prebuilt_core.py install_lock_path), so a normal install leaves ~/.unsloth/.node.install.lock behind. The final rmdir refuses a non-empty directory, so one stray zero-byte lock keeps the whole ~/.unsloth tree on disk. Confirmed on a real ubuntu-latest install -> update x3 -> uninstall run: ~/.unsloth and ~/.unsloth/.node.install.lock both survived. whisper.cpp is only installed when a prebuilt matching the pinned llama.cpp build exists, so runs that skip it never exercised that path. Also removes the .stale.<pid> locks that a lock takeover renames rather than deletes, documents the sibling prebuilts in --help, and adds a ~/.unsloth catch-all to the smoke workflow's leak list so a future artifact cannot be missed the same way. tests/sh/test_uninstall_prebuilt_artifacts.sh runs the uninstaller for real against a fixture HOME: 19 checks pass here and 8 of them fail against the previous uninstall.sh. It also asserts user content under ~/.unsloth is kept. * Mirror the prebuilt cleanup on Windows, and skip the artifact suite on WSL Two review findings. uninstall.ps1 removed llama.cpp, node, .staging and the llama lock, but not the whisper.cpp tree or the node and whisper locks, while setup.ps1 installs whisper.cpp at %USERPROFILE%\.unsloth\whisper.cpp. A native Windows uninstall therefore left a prebuilt tree behind and could never prune ~/.unsloth. Adds the whisper dir, all three locks and the .stale.<pid> sweep, puts the whisper dir in the handle-releasing list so a running whisper-server does not block the delete, updates -Help, and extends the Windows smoke assertion with the same catch-all the POSIX one got. tests/sh/test_uninstall_prebuilt_artifacts.sh runs the real uninstaller, and overriding HOME does not contain that on WSL: the body detects WSL from /proc/version and then deletes Windows-side shortcuts under /mnt/*/Users and /etc/profile.d/unsloth-rocm-wsl.sh. The sibling arg-guard suite already skips there for exactly this reason; this one now does too. tests/studio/test_uninstall_prebuilt_parity.ps1 asserts the two uninstallers cover the same artifacts, since drift between them is what caused this. It cannot run the body (that kills processes and writes the registry) so it parses uninstall.ps1 instead. Against the previous uninstall.ps1 it fails on exactly the four missing behaviours. Also corrects the stale-lock comment: install_node_prebuilt.py renames then unlinks, so a stranded .stale file needs a crash between the two steps.	2026-08-01 03:03:30 -07:00
Daniel Han	31a651a407	Close two false passes in the uninstaller argument tests, and document the piped help form (#7690) ... * Close two false passes in the uninstaller argument tests, and document the piped help form The help loop minted a fresh fixture home per iteration but asserted the install survived only after the loop, so only the -h iteration was ever inspected. A --help that removed files would have passed. Verified: a mutant whose --help prints usage, exits 0, and deletes the CLI shim passes the existing suite and fails the updated one in three places. The two multi-argument cases asserted an exit code and nothing else, so a guard that rejected the argument only after removal had started would also have passed. Separately, -h is a shell option (hashall), so `... | sh -h` and `... | bash -h` are consumed by the shell and the script runs with no arguments, which is a full uninstall. Someone reaching for help through the documented pipe destroys their install instead. Nothing in the script can detect that, since the argument never arrives, so the usage text now spells out `sh -s -- --help` and names the hazard. A check asserts the usage keeps saying so, and a new portable case proves that form reaches the guard on every platform rather than only in the Linux-only pipe-buffer test. 32 checks pass, up from 25. * Correct the -h claim: dash and busybox reject it rather than uninstalling The help text said piping to `sh -h` uninstalls. That holds only where -h is accepted. Measured against the instrumented copy in a sandboxed HOME: bash -h rc=0 reached the uninstall body zsh -h rc=1 reached the uninstall body bash --posix -h rc=0 reached the uninstall body dash -h rc=2 Illegal option -h, install untouched busybox sh -h rc=2 illegal option -h, install untouched Debian and Ubuntu ship dash as /bin/sh and Alpine ships busybox, so the text was wrong about a destructive operation for most of the Linux audience it is addressed to. Both outcomes are now stated, and the advice is unchanged: `sh -s -- --help` is the only piped form that prints help. The usage assertion also gets its own run instead of reusing $OUT from the last loop iteration, so a failure is attributed to the right check. * Spell out the URL in the piped-help example The abbreviated form was not copy-pasteable: curl takes `.../uninstall.sh` as the URL, fails to resolve it, and the pipeline still exits 0 because the status is sh's and sh reads an empty stdin. So a user who pasted it saw a resolve error and no help, which is exactly the confusion the paragraph exists to prevent.	2026-08-01 03:02:44 -07:00
Daniel Han	5344ec3e49	Tighten the comments around the uninstaller argument guard (#7647) ... Shorten the header blocks in scripts/uninstall.sh and scripts/uninstall.ps1 and the two test file headers, and drop five comments that restate the line below them. The comments that carry a non-obvious reason are kept: why the trailing compound block makes a truncated curl pipe inert, why PowerShell throws rather than exits, why the real uninstall body is skipped on WSL, why the pipe test guards against going vacuous, and why the fixture path is a global. Help text is program output, not commentary, so the _usage heredoc and the _Usage here-string are untouched. No executable code is changed.	2026-07-30 05:36:27 -07:00
oobabooga	a1dcd94846	Uninstaller: reject unrecognized arguments instead of uninstalling (#7631)	2026-07-30 03:47:22 -07:00
Michael Han	6d8c18cd1a	Replace standalone Studio wording with Unsloth (#7221) ... * Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error.	2026-07-19 00:47:04 -07:00
Daniel Han	e83d4ae072	Windows installer: fix DiskPart UAC mid-install, drive-root cache, and spurious unsloth.exe rename warning (#6296) ... * Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH. shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed to report gcnArchName, which is why the arch came from the GPU-name table) and amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt). TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so the inductor/Triton cache lives under the user's Studio home, not the system drive root. Long paths are already enabled above so deep inductor paths still fit. unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32" warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because setup runs via unsloth.exe (the running launcher holds its own file). The 'studio update' flow still attempts it. CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after the AMD wheels land, so the log makes clear the final install is GPU-accelerated. Adds two regression tests covering the venv-internal vs external hipInfo gate. Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1 --local from this branch completed exit 0 with no DiskPart prompt, no rename warning, the cache under the Studio home, and "GPU ROCm PyTorch installed (gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon 8060S Graphics". * Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning setup.ps1 used to rename the running unsloth.exe out of the way before the base-package upgrade so pip could replace it. That rename never actually worked: setup runs *via* unsloth.exe, so renaming our own running uv-trampoline launcher failed with a sharing violation (WinError 32) and only printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32' warning on every Windows install and update. It also was not needed. pip tolerates a running/locked console-script .exe: it moves the old one aside and writes the new one. The base upgrade routes through pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left intact either way. Removing the rename block and its failed-install restore block removes the false warning for all Windows devices in both the install and update flows. * Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks Follow-up to PR #6296. - install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python installers, which already do this. - amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv containment check (Windows paths are case-insensitive) and run the HIP_PATH/ROCM_PATH candidate through it too. - setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are unavailable, and create the dir wildcard-safely. - tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and assert the PowerShell venv exclusion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Windows installer: install ROCm PyTorch directly for a known AMD arch When the GPU arch is known (name-inferred from the GPU-name table) but ROCm could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The repo.amd.com wheels bundle their own runtime (no HIP SDK required), which setup.ps1 already relies on, so the CPU base was a pure wasted download/install. - Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts still get CPU (unchanged). - Fall back to a CPU base if the ROCm-index install fails, so a transient repo.amd.com outage does not abort the install (setup.ps1 retries ROCm). - Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK. - Add a regression test for the arch-based gate. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Windows installer: correct the unsloth.exe rename-removal comment The comment claimed the base upgrade 'routes through pip on Windows' and that pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what the code does. install_python_stack tries uv first; on a locked launcher uv aborts and falls back to pip, but the pip fallback strips --upgrade-package and base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already satisfied and no-ops. The running unsloth.exe is left intact at its current version either way. Reword the comment to describe the real uv-first / pip-fallback-no-op behavior. No functional change. * Windows installer: close two gaps in the venv-internal hipinfo exclusion Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases: - setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the venv-internal hipInfo.exe was not recognized. Seed the venv root from UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers). - The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run Test-HipinfoIsVenvInternal on the candidate as well (both installers). Extend the PS gate test to assert both. Both .ps1 parse clean; install tests pass (the venv-internal / HIP probe coverage at 359 passed). * Windows installer: correct the CPU-base message for arches with no ROCm wheels After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl and installs ROCm directly, so it no longer reaches the "temporary CPU base" branch. That branch is now reached only by a name-inferred arch with no ROCm wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old text ("setup replaces it with GPU ROCm wheels ... the final install IS GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch stays on CPU for this GPU. * Windows installer: seed the venv-internal hipInfo check from a custom Studio home Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the setup python, and the default %USERPROFILE% path only. A standalone `unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias) and none of those set would not recognize the venv hipInfo on PATH, reopening the amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert it in the gate test. * Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter Two review points on the amd-smi/DiskPart UAC gate: 1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo path, which does not resolve symlinks, junctions, or 8.3 short names. A venv reached through an aliased path then fails the check, so its bundled hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py). 2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a leading ~, while the canonical resolver does. With a tilde form, [IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the same way as the resolver. tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink realpath case and a setup.ps1 tilde-expansion guard). * Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1 Follow-up review on the same install.ps1 paths: 1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal) seeded the venv root from a custom Studio home without expanding a leading ~, unlike the canonical resolver and setup.ps1. A tilde form left [IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart gate. Expand ~ in the probe, matching the setup.ps1 fix. 2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch to below 2.12. AMD's per-arch index publishes the companions independently and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a bare resolve can pick an ABI-incompatible set and fall back to CPU. Add torchvision/torchaudio floor maps and pass the pinned specs, mirroring setup.ps1 and install_python_stack.py. 3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no retry), the only torch step in the file without it. Switch to Invoke-InstallCommandRetry so the recovery path survives a transient index failure. tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde check over both installers, a torch/companion floor-map parity test, and a CPU-fallback retry guard). * Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo, which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe inside the venv and the bnb fix (plus the Studio backend) prepend the venv Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is unchanged. Applied to all three Python copies (install_llama_prebuilt.py, install_python_stack.py, backend/utils/hardware/amd.py) via a new _external_hipinfo_on_path helper, and both PowerShell callers (install.ps1, setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal. tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a shadow-regression test for the exact venv-first ordering, and a parity check that every Python copy uses the scanning helper). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock) Auditing a dual native+WSL uninstall on a real device surfaced three leftovers: 1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the shortcuts that reference that icon, so Explorer's icon cache briefly held it open. Remove-Item -Recurse reported success yet left the locked file, and the dir was never re-attempted, so it orphaned with a false "removed" log. _RemovePath now verifies the path is actually gone (retrying transient locks) and reports honestly, and the data dir is re-swept after the shortcuts go. 2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when empty, in both the powershell.exe and drvfs-fallback paths. 3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of ~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock. Verified by running both uninstallers on a real dual install: device fully clean (no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left). * install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04 ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu 24.04. When the installer runs in a newer default distro (e.g. 26.04) it cannot enable the GPU and silently falls back to CPU. If a 24.04 distro already exists, re-run the install there and stop in the current one so the GPU path is taken without the user having to know about the distro requirement. Runs before venv creation so the wrong distro is left untouched, guards against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm distro alone (librocdxg present), and skips the GGUF-only / opt-out / non-Strix cases. When no 24.04 distro exists we keep today's behaviour: continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never auto-downloading a distro. Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the function, rewrites its paths to fixtures, mocks wsl.exe) covering the full decision matrix, wired into tests/run_all.sh. * uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico: install.sh points the WSL shortcut's icon there while the native install owns the dir. The native uninstaller removed the whole dir unconditionally, so uninstalling native while keeping WSL left the WSL shortcut with a blank icon. The old code only avoided this when Explorer happened to hold the icon open, which is unreliable; on a real dual install the dir was deleted and the WSL shortcut went blank. _RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving "Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it removes the dir as before. uninstall.sh still drops the icon and the empty dir when WSL itself is uninstalled, so every uninstall order ends clean. Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and runs it against a temp dir with controlled shortcut dirs) covering the dual, native-only, empty, and missing-dir cases, wired into the windows-inference smoke workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico and the WSL shortcut's icon stays intact. * installer: condense AMD/ROCm code comments (no behavior change) Tighten the comments added for the Strix Halo native+WSL installer work so they are shorter and clearer without losing intent: the venv-internal hipInfo amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and the dual-install uninstall icon handling. Comment-only; code paths unchanged. 107 insertions, 166 deletions across 11 files. * install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration, so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin distro (and ran venv migration in the distro about to be abandoned). Move the reroute ahead of the venv section so the origin distro is left untouched, matching the function's own comment. Behavior is identical on every non-reroute path. * installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04 - install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the flavor-repair block does not retry the failed ROCm index and abort the install; pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too. - install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo. - amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding quotes from PATH entries before probing for hipinfo. - install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04. - uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or another WSL distro) still references it, in both the powershell and drvfs paths. - tests: regression coverage for all of the above. * installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate - install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on --local; run the reroute BEFORE dependency/uv install so the origin distro is left untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later ROCm-on-WSL bootstrap does not install into the unsupported origin distro. - install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real executables match (not an alias/function named hipinfo). - uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs. - tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the user the install continues CPU-only, but only the failed-reroute branch set UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off that flag, so the other three branches could still install ROCm into the unsupported origin distro (e.g. 26.04). Set the skip guard on all of them. Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap carries through to the rerouted 24.04 child instead of dropping to the prompt path. install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty $env:USERPROFILE so Join-Path does not throw on a profile-less service account. Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE guard assertion, and route shell-test fixtures through a single trap-cleaned root. * installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch() installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure could abort the whole install even after install.ps1/setup.ps1 fell back to CPU. Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor maps) and make the retry nonfatal: keep the existing build and let the user re-run update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe. install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred); both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with only 22.04 reaches the GPU instead of staying CPU-only. install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child (PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder. _path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath can't classify every path on the drive as venv-internal (defensive; venv never at root). uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA). Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text + behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: match WSL reroute target by exact distro name, not substring The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests. * install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only) The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as GPU-supported let the parent report a successful reroute while the child fell back to CPU. Drop 22.04 from the supported set and the reroute target list; 24.04 stays the sole target (keeping the exact whole-line distro match). An already-working ROCm on any other version is still left alone by the librocdxg check above. tests: reroute 22.04 cases updated to the 24.04-only behavior; make the "no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes; stop the tauri exit-order check from mis-flagging the reroute helper's [ "$TAURI_MODE" = true ] && ... --tauri one-liner. * installer: tighten comment wording across the Strix Halo install/uninstall paths Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer, clearer lines. Comments and a few docstrings only; no code, logic, or behavior change. Verified with bash -n, the PowerShell parser, and ast.parse, and the installer test suite still passes. * add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them Every shell and PowerShell script under the Studio/installer surface now carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright header (after the shebang where present): the installer (install.sh, install.ps1), build.sh, the .github and src-tauri scripts, the installer test suite, and the moe kernel test. Header-only, line endings preserved; bash -n, the PowerShell parser, and the installer tests all pass. * installer: drop the duplicate AGPL header from install.sh and install.ps1 Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below their usage comment block; the prior header pass added a second one at the top because it only scanned the first few lines. Remove the duplicate so each file keeps a single original header. * installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path () so the genuine CPU-only install stays fast. install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO]) to ask the desktop app to elevate for the target distro; capture the child's exit code and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line) instead of masking it. CLI mode still falls back to CPU on a generic failure. Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases; run_func now preserves the child exit code; force-reinstall assertions for both PowerShell installers. Note: codex's _rr_q apostrophe finding is a false positive -- the helper already emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") }) collapses the single-element array to a scalar string, so @cpuForce splatting enumerated it character-by-character into broken single-letter args (- - f o r c e ...), which made uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag). Test now asserts the array-build form and rejects the if-expression form. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * uninstall: remove the isolated Node.js runtime (~/.unsloth/node) The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting <studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block the delete. Env/custom mode nests node under the custom root, removed with that root. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-22 03:09:08 -07:00
Daniel Han	62191c4765	Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940) ... * Fix Windows installer winget msstore certificate failure `winget install` was invoked without `--source winget`, so winget also queried the msstore source. When msstore fails certificate pinning (error 0x8a15005e, "The server certificate did not match any of the expected values") winget aborts and demands `--source`, so the Python (and uv) install fails even though the package exists in the winget source. - Pass `--source winget` to all winget install calls (Python x2, uv). Both packages live in the winget source, so this is strictly correct and skips the failing msstore round-trip entirely. - Add a python.org fallback (Install-PythonFromPythonOrg) that downloads the official installer and runs it silently per-user (no admin/UAC) when winget is unavailable or fails for any reason. Mirrors the existing uv -> astral.sh fallback so Python installs without manual steps. Resolves the latest 3.13.x from python.org with a pinned fallback, and selects the amd64/arm64/x86 installer per architecture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Pin remaining setup.ps1 winget calls to --source winget Two winget invocations in studio/setup.ps1 still queried all sources and could hit the same msstore certificate-pinning failure (0x8a15005e) that broke the Python install in install.ps1: - `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe) - `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server) Every other winget call in this file already passes `--source winget` (Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own Python 3.12 install), so these two were stragglers. Both packages live in the winget source; pinning it makes setup robust to an unhealthy msstore source, matching the rest of the file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Stop amd-smi GPU probe from popping a DiskPart UAC prompt On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs `amd-smi list` / `static --asic` / `version`. amd-smi (shipped in System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory details, surfacing a confusing DiskPart UAC prompt mid-install. The Studio backend already documents and circuit-breaks on this in studio/backend/utils/hardware/amd.py, but the installers did not. Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate (no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi cannot stall the install for minutes. On failure/timeout the existing WMI name -> gfx fallback still resolves the arch, so detection is unchanged on working hosts. Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone and the probe is bounded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151) install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but it does not install AMD's driver/ROCm stack -- a large, admin-gated prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151 GPU smoke test. A hard preflight refuses to run until the Adrenalin >=26.3.1 driver is actually present, so it cannot half-install. Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Detect AMD GPUs by name so native Windows gets a GPU llama.cpp The gfx-arch inference from the WMI GPU name was gated behind $HasROCm, which the hipinfo/amd-smi probe leaves false on the common Windows case (Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the arch without elevation). So an AMD GPU was detected by name but never mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup fell back to a CPU llama.cpp build. Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx, which makes install_llama_prebuilt.py download the matching lemonade-sdk ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs with just the Adrenalin driver. PyTorch's ROCm wheels still require a confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference and never pulls broken ROCm torch. Also broaden the name->arch table to every family lemonade ships Windows assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150 (RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths The earlier PowerShell guard covered install.ps1 / setup.ps1, but the Python installer (install_llama_prebuilt.py detect_host, install_python_stack.py ROCm probes) and the Studio backend monitor (amd.py) also shell out to amd-smi on Windows, where it auto-elevates and pops the same DiskPart UAC prompt mid-install / at runtime. Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on Windows so it runs un-elevated (no prompt). Callers already tolerate an empty/failed result and fall back to WMI / name detection (installer) or the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS amd-smi behaviour is unchanged. - install_llama_prebuilt.py: handled centrally in run_capture (covers detect_host's `amd-smi list` and the version probe). - install_python_stack.py: new _amd_smi_env() helper on its 3 raw subprocess.run amd-smi calls. - amd.py: merge RunAsInvoker into the existing child env. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Tighten AMD GPU name->arch patterns to avoid mismatches The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were speculative and over-broad: W9xxx would also match old GCN FirePro W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that fails validation), and RX 90[0-9]{2} was redundant with the explicit 9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched AMD names still fall back cleanly to CPU. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Fetch the llama.cpp validation model via huggingface_hub The prebuilt validation downloads a tiny GGUF test model from huggingface via bare urllib. On Windows / proxy setups where the server sends an incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so a perfectly good GPU prebuilt is rejected and the installer falls back to a CPU source build. Route the validation-model download through huggingface_hub (hf_hub_download) -- the same mechanism Studio uses for model downloads, which completes the chain where urllib cannot -- keeping the direct URL as a fallback. This lets the lemonade ROCm prebuilt validate and install on cert-restricted machines (verified: hf_hub_download succeeds where urllib returns CERTIFICATE_VERIFY_FAILED). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Guard the remaining raw amd-smi version probe via run_capture A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Forward --rocm-gfx even when the ROCm runtime is unconfirmed setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm stays false) the gfx arch was name-inferred but never forwarded, so install_llama_prebuilt.py saw has_rocm=False and installed the CPU build -- even though the lemonade gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime; verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB). Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as windows-hip in the existing-install mismatch check. --has-rocm stays gated on the confirmed-runtime signal. Verified on Radeon 8060S / gfx1151: the installer now selects, validates, and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present) instead of the CPU build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export) setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`. On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is false) the gfx arch was name-inferred but the ROCm-wheel branch never ran, so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export. Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels (repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without a HIP SDK; a failed install still falls back to CPU. Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics", GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Force amd-smi un-elevated process-wide in the Python installers Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match Start-Process -PassThru leaves the returned process object's .ExitCode $null after WaitForExit on Windows PowerShell 5.1, so the helper set $LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)` was always false -- the amd-smi GPU / gfx-token / ROCm-version detection branch was effectively dead (masked only because the un-gated WMI name->gfx inference still ran). Reproduced on PS 5.1.26100. Rewrite the helper to use [System.Diagnostics.Process]::Start with a ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable, with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER= RunAsInvoker (inherited via the process env) still suppresses the auto-elevation / DiskPart prompt. Also drops the temp files and the empty-ArgumentList edge case. Verified: exit code propagates (7 -> $LASTEXITCODE=7), output captured, env restored. Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the spaceless retail name "RX 7700S" is correctly excluded (it belongs to the gfx1102 row). Both found by PR review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Address PR review follow-ups (install.sh table, update path, tests, WSL) From the multi-agent PR review: - install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1 (the bash table had drifted to the old narrow patterns). Adds RDNA 2 (gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S" resolves correctly (bash case has no negative lookahead). AMD-ROCm-only: the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA / CPU / macOS are unaffected. - setup.ps1: the "dependencies up to date" fast path skipped the torch reinstall, so an existing user who had CPU torch (installed before ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch is known AND the installed torch is CPU-only, don't skip -- force the dependency pass so the ROCm wheels install. - scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg symlink (breaks if amdgpu-install lays ROCm under a patch-version dir); add a LIBROCDXG_REF pin knob and a "verified against" freshness header. - tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts, _fetch_validation_model_bytes (hf path + urllib fallback), run_capture's Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1 name-table parity (catches future drift). 14 tests, all passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK On Windows, amd-smi re-initialises the ROCm runtime on every invocation (even `amd-smi version`) and, on hosts without a working HIP runtime (consumer APUs/dGPUs with only the Adrenalin driver), elevates a child process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress that runtime elevation (verified: even `amd-smi version` hangs and times out with RunAsInvoker set). Replace the ineffective RunAsInvoker-only approach with a real gate: only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo present, so amd-smi runs un-elevated) or the user opts in with UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates and is untouched (no regression). RunAsInvoker is kept as harmless belt-and-suspenders for tools that DO use manifest elevation. Applied consistently across: - studio/backend/utils/hardware/amd.py (runtime GPU polling) - install.ps1, studio/setup.ps1 (install-time detection) - studio/install_llama_prebuilt.py (prebuilt arch probe + version) - studio/install_python_stack.py (ROCm version + arch probe) Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh install detects the GPU, installs ROCm torch (torch.cuda.is_available() True), launches Studio with no DiskPart prompt, and inference, tool calling, web search, LoRA finetuning, and GGUF export all run on the GPU. Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate assertions; update the three amd-smi monitoring tests to opt in (they mock amd-smi as available). Full suite: 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: helpful WSL message when the GPU isn't exposed to ROCm In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds nothing and we silently fall back to CPU. Add an actionable hint in the CPU-fallback path, shown only on WSL and only AFTER detection has already failed -- so it is forward-compatible: the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds Ubuntu 26.04 support), detection succeeds and the hint never fires. The message: - notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed, - lists the two prerequisites (Adrenalin driver + Ubuntu 24.04), - if the distro is not 24.04, says AMD may not support it yet, - tells the user to `wsl --install Ubuntu-24.04` and re-run, - links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper. Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted) and the CPU install completes; on Ubuntu-26.04 the extra "this distro may not be supported" line appears and points to 24.04. Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's repo.radeon.com/amdgpu-install/ is indexed by unified installer version (30.30, 31.30, ...), NOT ROCm version, so the hard-coded amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first for a noble .deb matching the target ROCm major.minor (ROCm 7.2 -> 30.30.x/amdgpu-install_7.2.x), falling back to the newest available. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151): 1. Shortcut collision (real bug). install.sh's WSL branch wrote "Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the native-Windows installer (install.ps1 New-StudioShortcuts). Running install.sh in WSL therefore silently retargeted the native shortcut at the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon stopped launching native GPU Studio. Now the WSL shortcut uses a DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the proper icon. Native and WSL shortcuts now coexist. 2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL compatibility matrix, gianni.rosagallina.com Feb-2026 guide, ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees only the CPU. install.sh's WSL hint and the experimental install_rocm_wsl_strixhalo.sh header/preflight now state the exact driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link AMD's radeon-ryzen docs, and document the known librocdxg caveat that usable VRAM is currently capped at the .wslconfig memory setting. bash -n clean; install test suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: hint when the AMD driver is too old for ROCm-on-WSL Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on native Windows can also be used inside WSL2, but only with AMD Adrenalin Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows GPU works with any recent driver, so this is purely about enabling the WSL path. We intentionally do NOT auto-install the driver: AMD referrer-gates driver downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish Adrenalin via winget, so no installer can reliably fetch it -- and silently swapping a live display driver is risky. Instead we point the user at AMD's official download page (one click), after which the existing WSL detection lights up automatically. - install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the installed driver predates the 26.2.2 release (DriverDate < 2026-02-01), print a concise tip with the AMD download URL. Handles DriverDate as either a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1. - install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note that AMD downloads are referrer-gated (open in a browser). Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01; install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.ps1: refresh shell icon cache after creating the shortcut After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh its icon (ie4uinit.exe -show). Without this, a stale icon cache can show a blank shortcut icon until the next explorer restart -- most visible when a shortcut of the same name was rewritten (e.g. a native install followed by a WSL install, which previously shared the name; now they use distinct names, but the cache nudge makes the icon appear immediately regardless). Best-effort and wrapped in try/catch so it never fails the install. The bundled unsloth.ico itself is valid (verified it renders). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid. Previously, if an AMD-GPU host ever fell through to the source build (e.g. the prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only, no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt is the AMD path and how to restore it (re-run / check network / set UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU. No behavior change on the happy path: AMD still gets the GPU prebuilt (verified on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA (CUDA source build) and CPU-only hosts are unchanged. setup.ps1 parses; install suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut Three gaps found by running a real uninstall on a native-Windows + WSL host; all fixes are scoped to Unsloth-owned paths and no-op on the other pathways (env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses. uninstall.ps1: - Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio, not under it -- so deleting <studio> left hundreds of MB behind. Now removed explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated content). No-op in env/custom mode (llama.cpp nests under the custom root, removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept. - New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp and (b) an orphaned multiprocessing python fork that ran from the SYSTEM python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle blocks the directory delete, leaving a half-removed install. The new helper kills any process whose image path OR loaded module is under a target root (module scan scoped to python/unsloth/llama-server names; vendor-agnostic). - _RemovePath now retries (transient post-kill handle release). uninstall.sh: - Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir ~/.unsloth only if empty. - WSL Windows-side shortcut cleanup now matches by TARGET (any "Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk" -- and never removes a native-Windows shortcut (which launches wscript.exe). uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut The Start Menu shortcut kept showing a blank/generic icon even after the Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost keeps its OWN pre-rendered tile-icon cache (%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\ TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch it, and they don't recycle the host -- so a rewritten same-name shortcut keeps showing the first-rendered (often the generic wscript ">") tile until the host restarts on its own. Fix: after creating the shortcut, drop only the Start Menu RENDER caches (TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost (Windows auto-relaunches it), so the tile re-resolves the real icon via the shell image factory. start2.bin (the user's pinned layout) is deliberately preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and wrapped in try/catch so it can never fail the install. Windows-only (install.ps1); no effect on Linux/macOS/Studio. Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched, after this invalidation -- previously it returned the generic script tile. install.ps1 parses; install suite 267 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo (Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026): rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool. Every change is a strict no-op for all other configs (NVIDIA/CUDA, discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix WSL) and can never abort the installer. - scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe. Fixes that would have broken a working box: drop the /usr/lib/wsl/lib preflight (a working ROCDXG host has only d3d12/dxcore there); remove the obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname (was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent. - install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then loads the env so detection routes to the gfx1151 wheels. Fast-path when already configured. Fix an inaccurate WSL hint line. - studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1 in-process before torch (gated on /dev/dxg AND librocdxg.so), so the worker uses the GPU even when launched outside a login shell. Mirrors the existing BNB_ROCM_VERSION injection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache - uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env (/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system ROCm userspace is a shared prereq like CUDA and is kept by default; UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux. - uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing the shortcut so its tile disappears promptly (mirrors install.ps1), preserving start2.bin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: accurate AMD ROCm messaging (HIP SDK optional, not required) The Windows installer printed "HIP SDK not found - GPU-accelerated training unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.), setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com) which ship their own ROCm runtime and do NOT need the HIP SDK -- verified end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a Radeon 8060S with no HIP SDK installed. Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK is optional; only when the arch is unknown fall back to the HIP-SDK hint. Behavior (torch routing) is unchanged; this is messaging only. No-op for NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust Two fixes from the 3-reviewer regression audit + live testing on a systemd-enabled WSL distro (interop disabled): F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair could force-delete a pre-existing REAL ROCm install. The guard only checked that /opt/rocm is a real directory, not that it is the stray librocdxg stub. Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo, bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak) instead of deleting it, so a wrong guess can never lose data. WSL interop robustness (install.sh + uninstall.sh): both relied on `command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE it (on systemd distros powershell.exe fails with "Exec format error"). Result: the WSL shortcut silently failed to create (install) and to remove (uninstall). - uninstall.sh: test that powershell.exe actually runs; if not, remove the "Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which works without interop. The name is WSL-install-specific, so a native install's "Unsloth Studio.lnk" is never touched. - install.sh: when the shortcut cannot be created, warn with the manual launch command + how to re-enable interop, instead of failing silently. No behavior change on the interop-on path. The regression audit otherwise found no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone Reinstall regression found by uninstall->reinstall testing: after a Studio uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared ROCm (the default), a non-login reinstall hit the bootstrap fast-path (librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION -- NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed, and the installer fell back to CPU-only PyTorch. Fix: when librocdxg is present but the env drop-in is missing, restore the FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH + LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio worker get it too. No change to the env-present fast-path or any other host. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: clear Explorer icon cache so shortcut icons aren't blank Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct (the shell renders them non-blank via IShellItemImageFactory; the .ico has real image data at 16/32/48/128 px), but the stale cache entry wins. The previous fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it never invalidated Explorer's icon cache, so the desktop icon stayed blank. Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in install.sh): - ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh) - SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh WITHOUT restarting explorer - keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL shortcut path too, preserving start2.bin) Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the previously-resolved (often generic "white page") icon for that item and won't re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct (every icon API renders the sloth) -- only Explorer's cached display is stale. The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED broadcast) does NOT recover a stale item -- confirmed by reproduction. The reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created shortcut, which forces Explorer to re-read that exact item's icon. Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the installer's exact new refresh code, and the sloth icon recovered with NO explorer restart (confirmed by capturing the live desktop via PrintWindow). Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11 Start Menu tile cache (preserving start2.bin). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned The llama.cpp atomic-install staging root (install_llama_prebuilt.py INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install dir (~/.unsloth/.staging in default mode). It is normally pruned after a successful activate, but an interrupted or retained build can leave a <name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and .cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall. Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in env/custom mode (staging nests under the custom root removed already) and when absent. Cross-platform fix (the staging logic is platform-agnostic). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer: WSL-absent hint + fix here-string lint false positive install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo user with no WSL yet gets an actionable next step (the hint previously assumed an Ubuntu-24.04 distro already existed). Best-effort, informational only. test_rocm_support.py: test_no_here_strings did a crude substring check that false-positived on the conda-style block marker printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written into the /etc/profile.d drop-in, also used as a sed delimiter pair by uninstall.sh, not a here-string. Strip quoted spans before the check so the lint still catches a real here-string operator but ignores quoted literals. install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make) Apply the valid bot review findings on #5940; reject the ones that don't hold. Fixed: - AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX 370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x / AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150; 8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm prebuilt/wheels. - amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI= 0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out exists to avoid. - amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and _detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass _amd_smi_env()), closing the last unguarded amd-smi spawn on Windows. - WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now requires rocminfo to enumerate the real gfx1151 agent instead of the generic _has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback "gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer skipped. The shared helper is untouched (no gfx90a regression). - install_rocm_wsl_strixhalo.sh: * Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files (x86)/...") word-split on the space and never matched; use find + read loop. * Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it and the librocdxg `make -j` build failed). * Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an unrelated RDNA GPU can't pass while the real GPU is absent. Reviewed but NOT changed: - "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct -- --rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`. - "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1 installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live the native install ends on torch 2.11.0+rocm7.13.0. - "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process already returns "" with LASTEXITCODE=1 (no uncaught exception). - "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed; only a comment about it remains. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps) librocdxg's build needs the Windows SDK 'shared' headers on the Windows host. Previously the helper just die()d with "install the Windows 11 SDK and re-run" if they were missing -- a manual prerequisite that broke the otherwise-seamless `curl ... install.sh | sh` one-liner on Strix Halo. Now, when the headers aren't found, the helper installs the Windows 11 SDK on the Windows host from inside WSL via winget (powershell.exe interop), then re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot). The user already consented to the ROCm-on-WSL setup, so no extra prompt is added beyond the OS UAC gate. - New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir) and _install_windows_sdk_via_winget helpers. - winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then .22621. The presence of the headers (re-check) is the source of truth, not winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin. - Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall through to the existing clear manual-install die(). Opt out with UNSLOTH_SKIP_WIN_SDK_INSTALL=1. Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD Adrenalin driver (AMD referrer-gates the download) remains manual. Verified _find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all winget flags validated against `winget install --help`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing" warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the _amd_smi_allowed() gate that every other amd-smi call site in the file uses. On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress (amd-smi's manifest is asInvoker). The probe also ran before the ROCm-torch-installed check, so it fired on every Windows AMD install. Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching _has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the best-effort "AMD GPU detected" note on HIP-SDK-less hosts. Adds a per-function AST regression test asserting every function in install_python_stack.py that names the amd-smi command and spawns a subprocess also references _amd_smi_allowed() (flags the pre-fix code; passes after). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * studio(cli): fix `unsloth studio stop` crashing on Windows `stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows CPython raises OSError (WinError 87, "The parameter is incorrect") for *every* pid -- alive or dead. `stop` only catches ProcessLookupError / PermissionError, so the OSError propagated and the command crashed with a traceback before ever reaching its (correct) `taskkill /F` path. Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows, signal-0 elsewhere) and use it for both the pre-check and the post-kill wait loop. The actual kill path is unchanged. Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for both a live and a dead pid; `_pid_alive` returns True/False correctly and the full stop() flow (alive -> taskkill -> dead -> "stopped") passes end-to-end against a throwaway process. Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for the win32 tasklist branch and the POSIX signal-0 branch). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151 The bash name->arch inference table in install.sh placed Strix Point identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1 PowerShell tables which correctly map them to gfx1150. It also carried a stray "HX 38" token absent from the PowerShell source-of-truth. Align install.sh with the PowerShell tables: gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|... Impact is low (the bash table only feeds the display label _gpu_disp_gfx and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven by the detected ROCm version, not this name string) but a Strix Point user would otherwise see/copy the wrong gfx arch. Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd) that parses install.sh's case table and asserts Strix Halo->gfx1151, Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match, cross-checking against install.ps1 (the previous parity test only compared install.ps1 <-> setup.ps1, missing install.sh). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * setup.ps1: keep prebuilt-llama ownership guard within the test's block window The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs windows-cpu existing-install kind validation) pushed the install_llama_prebuilt.py invocation to ~1999 chars after the "installing prebuilt llama.cpp bundle (preferred path)" anchor, right at the edge of the 2000-char window that test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the helper string was truncated and the test failed with "substring not found" (CI: Repo tests (CPU)). The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the install_llama_prebuilt.py call) was already satisfied; only the proximity to the anchor regressed. Move the "installing prebuilt..." substep to immediately before the install (after the existing-install pre-cleanup), which also reads better (validate/clean existing -> then "installing"), shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged (console message ordering only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * install.sh: auto-run Strix Halo ROCm-on-WSL setup by default `curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo (gfx1151) GPU usable inside WSL with no extra commands. Previously the ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty), so the piped one-liner never set the GPU up automatically. Flip it to auto-by-default for the single narrow case the existing guards allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU setup the user ran the installer for. Opt out with UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI (only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS / CPU paths are unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * PR comments: condense to be succinct (comments/docstrings only) Shorten the verbose explanatory comments and docstrings this PR added across the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines, while preserving every non-obvious "why" (os.kill WinError 87, amd-smi RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard chain, ownership guards, etc.). No executable code, string literals, messages, or behavior changed. Verified comments-only: docstring-normalized AST equality (Python, 9 files), non-comment token equality (PowerShell, 3 files), comment-stripped diff + sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url + gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups / cli_studio_stop tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root) Fixes valid findings from the Codex/Gemini PR review: - install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with $amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt. - install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK `winget install` to --source winget (matches the msstore-cert fix rationale). - install.ps1: python.org fallback installs the py launcher per-user (InstallLauncherAllUsers=0, avoids admin), and derives the fallback full version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12) isn't silently replaced with 3.13 when the listing is unreachable. - install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a non-root reinstall (a plain redirect failed silently, dropping the ROCm env). - uninstall.sh: scope WSL Windows-side shortcut removal to the current WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one distro no longer deletes other distros' launchers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Studio ROCm Windows: fix field-reported issues from Strix Halo testers Four fixes from PR #5940 field reports (Win11 native, gfx1151): 1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships hipInfo.exe in the venv Scripts dir, which is only on PATH for activated venvs. Every bnb import logged "Could not detect ROCm GPU architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK installed, whose bin dir is not on PATH either). Prepend the Scripts dir to PATH before bnb imports in main.py, worker.py, and install_python_stack.py, gated on the file existing (only AMD wheels ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151 with zero errors. 2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's total is the WDDM budget the driver grants HIP (BIOS carve + ~half of remaining RAM) -- the OS share is already outside it. The 0.80 unified cap on top denied loads that fit (field report: 48.49 GiB budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08 free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90. 3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with the fix (BIOS UMA frame buffer / AMD Software Variable Graphics Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a 96 GiB box reads as policy, not a Studio bug. 4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade gfx1151): --fit defaults to 'on' upstream, so the fit step runs even when Studio already placed the model via -ngl -1, and aborts in ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off when the server crashes during startup and Studio's own VRAM math had placed the model (never when use_fit or an explicit fit flag was passed). Also keep the TAIL of crash output in the error log (the diagnostic line prints last; head-truncation cut exactly that) and reference the full on-disk log. Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads and generates through the new spawn loop, stub-crash retry appends --fit off and recovers, fraction probes confirm WDDM overcommit and sub-1.0-only enforcement on current AMD wheels. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi amd-smi does not reliably exist on Windows: the HIP SDK never ships a CLI, inbox Windows Update drivers do not, and only some full Adrenalin packages drop amd-smi.exe into System32 (field report: fresh Win11 + Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer work without it: - install_python_stack._detect_windows_gfx_arch: two new probes after hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD torch wheels (drives `studio update` on driver-only hosts), and (4) a last-resort GPU marketing-name -> gfx table via WMI (Win32_VideoController), mirroring setup.ps1's $nameArchTable so a standalone repair resolves the arch with zero AMD tooling installed. - install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir so a standalone rerun finds hipInfo.exe without HIP_PATH. - hardware/amd.py _run_amd_smi: which() guard before spawning -- absence now disables the poller in one step instead of burning the 3-strike circuit breaker on FileNotFoundError; corrected the stale comment claiming Adrenalin ships amd-smi. Simulated against the real detection functions on gfx1151: amd-smi absent, present-but-crashing (exit 1), present-but-hanging (60s sleep vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no exceptions, bounded time. Full adversarial install (broken amd-smi stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first): exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt, torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots healthy and stops cleanly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: per-attempt llama-server log names + amd-smi test portability Found by cross-platform simulation of the --fit off retry (Windows + Linux sandboxes, real load_model with stub servers): - llama-server log filename now carries the spawn-attempt index. The retry can respawn within the same epoch second; reusing the name opened the same file with "w" and truncated the crash log the retry warning had just pointed the user at (proven with a frozen time.time: one file, crash evidence gone; with the suffix both attempts keep their logs). Regression-pinned in test_llama_cpp_wait_for_health.py. - test_amd_primary_gpu_with_mock now mocks shutil.which alongside subprocess.run: the amd-smi absence guard which()-checks before spawning, so on hosts without a real amd-smi (Linux CI, driver-only Windows) the subprocess mock was never reached and the test failed. Surfaced by running the suite in a clean Linux sandbox. Simulation coverage on both OSes: 67-case platform/edge matrix (real shipped code blocks under win32/linux/darwin spoofs: OOM-guard fractions + VGM-hint boundary, bnb PATH-prepend gates, retry eligibility incl. equals-forms and decoy tokens, GPU-name table adversarial set, WMI fallback without powershell, monitor absence semantics), 6-scenario live retry matrix (crash-once/crash-always/ exit-zero/explicit-fit/hang/log-collision) against real llama-server spawns on Windows and WSL (GPU success legs on the 8060S), and a 3-engine browser matrix (chromium/firefox/webkit) driving the live backend's health + authed /v1 chat completion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: classify unified-memory via props.is_integrated first Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded arch set. Strictly additive -- truthy upgrades to unified; 0/absent falls through to the existing gfx1150/gfx1151 + device-name logic, so wheels that omit or zero the field cannot downgrade the known APU set. Extends correct unified-cap treatment to APUs outside that set (e.g. gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers on one driver signal. Verified live on gfx1151 (is_integrated == 1 on the AMD Windows wheel -> ('gfx1151', True) via the new path). * AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table Fleet validation on a Strix Halo WSL2 box showed the system rocminfo (HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin off PATH outside login shells. Detection probes that miss either of these report no GPU on a working ROCDXG host and select the CPU build even though the lemonade bundle offloads fine (95.7 tok/s measured vs 64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and the PATH fallback in install.sh, studio/setup.sh, and the installer's Linux rocm probe, mirroring what main.py/worker.py already do for the runtime. Also sync studio/setup.sh's name->gfx table with install.sh: 890M and the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S must match gfx1102 before the gfx1100 row, and the RDNA2/workstation rows were missing. New parity test pins the two bash tables together so they cannot drift again. * Studio: persist server session logs + native-crash stacks to disk Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows): "the studio just terminates without a warning". A native crash in the GPU runtime kills the process with no Python traceback, and a desktop- shortcut console closes before anything can be read. The server only ever logged to the console, so there was nothing to send back. run_server now tees stdout/stderr to ~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior unchanged; file copy is best-effort), arms faulthandler at the same file so access violations / SIGSEGV leave a stack trace on disk, and exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps on their captured stderr. Armed before `from main import app` so even import-time failures leave evidence. Keeps the newest 20 session logs; opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>" at startup so users know what to attach. Verified on this box: a forced real segfault (faulthandler._sigsegv) leaves the full session output plus "Fatal Python error: Segmentation fault" and the thread stack in the file while the console shows nothing; a normal server boot captures the startup banner and serves health as before. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value Match the shell helpers, which use the parameter-default form: a user who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU from DXG detection should not have the probe override it. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com>	2026-06-10 04:24:49 -07:00
Daniel Han	c6e86d5e77	Update Install Scripts (#5968) ... * Update Install Scripts Add SPDX AGPL-3.0 headers to the installer scripts and let the piped web installs take their common options from the environment. - install.sh / install.ps1: read UNSLOTH_NO_TORCH (and UNSLOTH_PYTHON for install.sh) so a piped install needs no positional flags. Flags and the pipe forms still work; an explicit flag wins. - Fix the UNSLOTH_STUDIO_HOME example so the variable sits after the pipe and reaches sh instead of curl. - Add SPDX headers to install.sh, install.ps1, the uninstall scripts, and the MLX install scripts. - Drop the internal test package names from the studio install comments. * Mirror UNSLOTH_PYTHON env var to install.ps1 install.ps1 now reads UNSLOTH_PYTHON to pin the Python version, matching install.sh, and lists all three env vars (UNSLOTH_NO_TORCH, UNSLOTH_PYTHON, UNSLOTH_STUDIO_HOME) in the header examples. The requested version is preferred during detection and used as the winget install target; behavior is unchanged when the variable is unset.	2026-06-03 05:39:42 -07:00
Daniel Han	a74a1080e0	Move uninstall scripts into scripts/ and fix references (#5644) ... * Move uninstall scripts into scripts/ and fix all references Relocates `uninstall.sh` and `uninstall.ps1` from the repo root into the existing `scripts/` directory, alongside the other helper scripts. Reference fixes: * `README.md`: Studio uninstall instructions now point at the raw GitHub URLs under `scripts/`. The previous `unsloth.ai/uninstall.*` short URLs currently 404 (unlike `unsloth.ai/install.sh`, which 301s to the raw github URL), so the raw URL is the working entry point until that redirect is configured. * `scripts/uninstall.sh` header `Usage:` example updated to the new raw GitHub path. * `scripts/uninstall.ps1` header `Usage:` example updated to the new raw GitHub path. * `.github/workflows/studio-update-smoke.yml`: `paths:` trigger and round-trip exec/exists checks now use `scripts/uninstall.sh`. * `.github/workflows/studio-mac-update-smoke.yml`: same. * `.github/workflows/studio-windows-update-smoke.yml`: `paths:` trigger and round-trip exec/exists checks now use `scripts/uninstall.ps1`. The in-script help hints (e.g. `sh uninstall.sh`, `.\uninstall.ps1`) are left unchanged because they are user-facing examples shown after the user already has the file locally, and the basename form works regardless of which directory the user downloaded the script into. Follow-up note for unsloth.ai: once this lands, please add the `unsloth.ai/uninstall.sh` and `unsloth.ai/uninstall.ps1` short-URL redirects to `raw.githubusercontent.com/unslothai/unsloth/main/scripts/...` (matching the existing `unsloth.ai/install.sh` redirect pattern). * Update remaining uninstall script help hints for new scripts/ path Three user-facing strings inside the uninstall scripts still showed the old basename form, which became misleading after the move: * `scripts/uninstall.ps1` header `# Local:` example: now references `.\scripts\uninstall.ps1` (the actual path from the cloned repo root). * `scripts/uninstall.sh` env-var re-run hint: now shows the canonical curl-pipe form documented in README, since callers who came via `curl -fsSL ... | sh` never had a local `uninstall.sh` to invoke. * `scripts/uninstall.ps1` env-var re-run hint: same, switched to the `irm ... | iex` form documented in README. Pure string changes, no behavior change. --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-05-20 04:42:03 -07:00

Renamed from uninstall.sh (Browse further)