mirror of
https://github.com/unslothai/unsloth.git
synced 2026-08-17 12:53:59 +00:00
81 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Daniel Han
|
47dcad27c0
|
Scan package archives across cores instead of one at a time (#9024)
* Scan package archives across cores instead of one at a time
The three pip scan-packages shards cost 21.7 runner-minutes per push
(studio 10.17, extras 6.63, hf-stack 4.88), the bulk of Security audit.
Almost none of that is network. Timing the hf-stack shard locally:
Scanning 49 package(s) (with transitive deps)...
0.0s
Downloaded 110 archive(s).
9.7s
Summary
315.6s
pip download is 9.7s of 315.6s. The rest is the serial loop calling
scan_archive on each archive, which is pure CPU: regex over decompressed
archive members. Every archive is independent.
Pooled at 4 workers the same shard runs in 97.7s, and the report is
byte-identical to the serial one. imap with chunksize=1 yields in
submission order so findings are assembled exactly as before; chunksize=1
is also what makes next(timeout=) available at all, since above 1 CPython
returns a bare generator with no timeout support.
The archive-limit [WARN] lines are captured from the worker and replayed
in task order rather than landing wherever a worker reached them, so two
runs of the same input produce identical logs.
Default is min(4, cpu count), matching the runner, with --jobs to override
and --jobs 1 to force the old path. No workflow change needed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Exit 2, not 1, when the scan pool stalls
raise SystemExit(<string>) prints the string and exits 1, per the language
reference: "if it has another type (such as a string), the object's value is
printed and the exit status is one".
1 already means "non-baselined CRITICAL or HIGH findings detected" in this
scanner's documented contract, and 2 means an incomplete scan. So a dead worker
reported an infrastructure failure as a detected threat, and skipped the SCAN
INCOMPLETE block that tells the operator coverage was lost.
The stall is now recorded beside the pip-download failures and reported by that
same block, so it exits 2 and says why. The header counts both kinds, so it no
longer calls a scan stall a "pip download failure".
Regression test drives a pool whose first next() raises TimeoutError and asserts
exit 2 plus the report. Reverting to SystemExit fails it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Daniel Han
|
7b13a787cf
|
Studio: tokenize the dataset online for plain-text single-pass runs (#8960)
* Studio: tokenize the dataset online for plain-text single-pass runs
TRL's tokenizing map is the largest fixed cost of starting a text run:
71s of the 78s of preparation on 100k rows of OpenMathReasoning, with
dataset_num_proc already at 8. It is per-row string work, so it can run
in DataLoader workers while the GPU is busy instead of blocking the
start.
Four parts, all needed together:
* datasets.with_transform attaches a batched tokenizer that runs on
__getitem__. with_transform, not set_transform: the caller's split is
also held by the preview and the row-count checks.
* TRL gets dataset_kwargs = {"skip_prepare_dataset": True} so it does
not run its own map over the view. Feature-detected on SFTConfig's
fields plus SFTTrainer.__init__'s source, never assumed.
* dataloader_num_workers / prefetch_factor / persistent_workers, sized
from the same shared policy that sizes dataset_num_proc and capped at
four.
* a prewarm barrier inside _preflight_first_batch, which already built
a loader and pulled a batch. It now drains max(grad_accum,
workers * prefetch) microbatches, and memoizes the train loader --
transformers caches only the eval ones, so without that train() forks
a second worker set and drops everything the barrier filled.
The transform reproduces unsloth_zoo's sft_prepare_dataset tokenize step
exactly: same truncation and max_length, the same double-BOS rule, and
the tokenizer's whole output rather than input_ids alone, because the
collator and the attention dispatcher both branch on which keys are
present.
Default ON only for: Linux, plain text, plain tokenizer, map-style
datasets.Dataset, packing off, no custom collator, no completion masking,
not already tokenized, no token_type_ids, a raw eval split or none, at
least 10k rows, and at most one pass over the data. Everything else takes
today's path with config_args and the dataset wrapper untouched, and any
failure in the gate or the attach degrades the same way.
UNSLOTH_STUDIO_ONLINE_TOKENIZATION=0 forces it off; =1 lifts the two cost
gates but never a correctness gate.
The one-pass rule is what the measurements support: within a single pass
the workers stay ahead and there is no steady-state cost (225.45s eager
vs 225.33s online over 200 steps), while a lazy view re-tokenizes on
every further pass where Arrow would just be read.
rl.py: a split may now attest its own truncation width through
_unsloth_truncated_to, and the max_length enforcement believes it instead
of scanning. Scanning a lazily-tokenizing split reads every row, which is
the whole eager tokenize pass again, run inside __init__ where nothing
overlaps it -- and the fallback it would then take turns padding-free
off. Both copies of the scan honour it, the module-level one and the one
inlined into every generated trainer.
Measured on one B200, Qwen3-0.6B + LoRA, 100k rows, cold datasets cache:
preparation 71.2s -> 0.4s, time to first step 91.9s -> 17.7s. Losses
match: the largest per-step gap between the eager and online arms is
7e-4, smaller than the 9e-4 between two eager runs of the same seed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Shut the online path's workers down, and refuse the rows it would fail late on
Review of the online tokenization path found two things it got wrong once it
was running, rather than in the gate.
The persistent DataLoader workers were never torn down. Persistence is what
lets the prewarm barrier's workers survive into `train()`, and the memo holds
the loader that owns them, so after `train()` returned nothing dropped the
last reference: four `pt_data_worker` processes, 5.27 GB resident between
them, still alive through merging, quantizing and GGUF export, each a fork of
a process that had already initialised CUDA. Measured on one B200, Qwen3-0.6B
+ LoRA, 12k rows: 4 workers still there fifteen seconds after training ended,
and only the process exiting cleared them. `release_train_dataloader` shuts
them down and puts the real `get_train_dataloader` back, called from a
`finally` around `train()` so it runs before `_finalize_training` rather than
after it, and again from the outer `finally` for the paths that return before
training starts -- the preflight error has already forked the workers. Same
measurement after: 4 workers before the release, 0 after. An accelerate
wrapper and the loader inside it share one iterator, so the walk counts a
worker set once and clears the reference on both.
The second is the one asymmetry the gate did not cover. A null or non-string
row fails the eager map inside the trainer constructor, in seconds, before
anything else has happened; the lazy view reads a row only when the sampler
draws it, so the same dataset trained twenty steps and exited clean, and
would have died at whatever step drew row 137. That is the one way this
feature can make a failing run worse rather than slower. Both checks are
metadata -- the dtype off the schema, `null_count` off Arrow's per-chunk
statistics -- so neither reads a row, and a `select`ed split over-reports,
which vetoes a split that might have been fine and never the reverse. No
runtime fallback on top: switching a running job to the eager path would
tokenize the whole split mid-run and hide the bad data, where an error naming
the transform says what is actually wrong.
Also:
- The Linux gate tested `sys.platform`, but the hazard it names is `spawn`
re-importing the entry point against a `sys.path` Studio modified in
process. A Linux host whose start method is set to spawn or forkserver is
the identical hazard and a platform check cannot see it. Read the start
method instead, via `allow_none` and the method list, since resolving it
the other way pins the context and makes a later `set_start_method()`
raise.
- The transform truncated to the `max_seq_length` the user asked for, while
the generated `__init__` reduces that to the model's own cap before
deriving `max_length` from it. Read the same cap, or the two paths stop
producing the same rows and the attestation claims a width nothing applied.
- Delete `prewarm_dataloader`. It was called from nowhere, and its docstring
described tearing the loader down so the workers do not survive, which is
the opposite of what the shipped barrier does on purpose.
- `scripts/online_tokenization_ab.py` defaulted `--dataset` and `--model` to
paths under one workspace. `--dataset` is required now and the rest resolve
without them.
- Note in the module docstring that the pass gate counts train passes only:
an eval split is re-tokenized on every evaluation, where the eager map
tokenized it once.
The gate was well covered and the mechanism was not. Neutering `attach`,
`online_config_args` and the memo while leaving the gate saying yes left 64
of 72 tests passing. `test_online_tokenization_runtime.py` pins the three
claims that needed a real DataLoader with real forked workers to establish:
the prewarm re-iterates from the start instead of continuing (a sequential
sampler makes it exact -- continuing the prewarmed iterator loses exactly
`prewarm * batch` rows and starts at the wrong one), the loader the barrier
filled is the one handed back afterwards, and the workers are gone once
training is over.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe the eval split's own BOS convention instead of reusing the train split's
TRL calls _prepare_dataset once per split, so the eager path derives
add_special_tokens from each split's own first row. The online path reused the
train split's answer for the eval view, which tokenizes eval differently from
the map it stands in for whenever the two splits disagree about a leading BOS.
Also correct the prewarm barrier's docstring. torch answers a second iter() on
a persistent-workers loader with _iterator._reset(), which restarts the sampler
at row 0 and drops what is in flight, so the drained batches are tokenized
again rather than handed to step 1. No rows are lost; what the barrier buys is
workers that are already forked and past their first tokenizer touch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Release the memoized eval loader's workers alongside the train loader
dataloader_num_workers and dataloader_persistent_workers are
TrainingArguments settings, so an online run with evaluation on forks the
same workers for the eval loader. Transformers parks that prepared loader
in _eval_dataloaders (Trainer._get_dataloader, unchanged from 4.51.3
through 5.5.0) and torch never drops _iterator on a persistent-workers
loader once it has been iterated, so those workers outlived train() and sat
resident through the merge and export that the existing cleanup exists to
protect. Drain and drop the memo too.
* Stop the online tokenization tests depending on the runner's TRL and torch
Two CPU CI environments were red for reasons that had nothing to do with
what the tests cover. The gate tests read the installed TRL through
trl_supports_skip_prepare_dataset, and the CPU job installs no TRL, so
every refusal reported the missing hook instead of the gate under test.
Pin it in the autouse fixture, the way sys.platform is already pinned, and
cover the detector and its veto directly instead.
The wiring tests import UnslothTrainer, which imports torch, at module
scope, so a runner without torch failed collection and interrupted the
whole run rather than skipping the module. Guard it with importorskip, as
the runtime tests already do.
* Tighten online tokenization comments
* Route a Hugging Face dataset id through dataset_source in the A/B harness
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Daniel Han
|
135147814b
|
Fix CI on main: stale test doubles, a stale router stub, and two source defects (#8956)
* Fix CI on main: stale test doubles, a stale router stub, and two source defects
main has been red since Aug 14 and every open PR inherits it. Five clusters, none
of them caused by the PRs that were showing them.
context_length (13 tests, plus 4 more in disguise). #8700 added an unguarded
llama_backend.context_length read to the chat-completions path and updated five
test files, missing three. The real LlamaCppBackend has had the property for a
long time, so no user was ever affected; the doubles were simply
under-specified. The four gguf_stream_slot_release failures are the SAME bug:
those doubles reach the same line, but the AttributeError is swallowed into the
response task and surfaces as a 20 second timeout, which reads as a flake.
Nine hand-written doubles across five files each re-declared the same attribute
block with no shared base, so one new read broke whichever files happened not to
be updated. They now share FakeLlamaCppBackend, and a canary drives the real
route with a bare double so the next such read fails in one place, named, at the
point of the change. The stream waits no longer discard the driving task's
exception, so that class of failure cannot present as a bare timeout again.
youtube_router (2 tests). routes/__init__.py exports it and main.py imports it;
the app is fine. test_desktop_auth stubs sys.modules[routes] with a hardcoded
list of 17 routers, deliberately, to avoid importing the ML stack. #8648 added a
router and did not update it, the second time this has happened after
openai_codex_auth_router in #8511. The stub is now derived from main.py's own
import block, so it cannot go stale.
Repo tests (CPU), 6 failures, of which two are real source defects:
llama-extra-args.ts put the Studio brand into a user-visible validation message,
which the desktop branding contract forbids in runtime surfaces, and
test_playwright_server_lifecycle.py read checked-in files without an encoding,
which is a real Windows cp1252 crash the lint exists to catch. Both fixed in the
source. The other four are stale assertions chasing text that #8702 legitimately
moved or reflowed; they now assert the behaviour instead, via the real
override_lookup_candidates() and the element-scan pattern their own siblings
already use.
Not addressed here: pip scan-packages :: hf-stack reports 173 findings in
third-party deps under SCAN_ENFORCE=1 and is red on main too. Baselining a
supply-chain scanner to get green is the wrong reflex, so it wants its own look.
* Studio: break the settings/chat import cycle that stopped the UI rendering
#8932 added SIDEBAR_ORGANIZATION_STORAGE_KEY to the @/features/chat barrel and
had general-tab.tsx read it back out of that barrel. The key is used at MODULE
scope, in the storage-key list, and the barrel is part of an import cycle that
reaches this file, so the binding is still in its temporal dead zone when the
list is built:
Cannot access 'SIDEBAR_ORGANIZATION_STORAGE_KEY' before initialization
That kills the whole module graph, so the page renders nothing. It is why
Frontend CI has been failing on main with a Playwright locator that finds no
elements, which reads as a flaky browser test rather than a module-init error.
Importing the key straight from its module breaks the cycle. Verified by
bisection with the real browser smoke: it passes at
|
||
|
Daniel Han
|
1b48147d8e
|
Windows: stop depending on the generated unsloth.exe console script (#8592)
* Windows setup: install uv from a pinned release instead of running remote script text
studio/setup.ps1 piped astral's install.ps1 straight into Invoke-Expression. That
download-and-execute shape is the single construct AMSI providers and cloud ML
scanners score hardest, and install.ps1 already replaced it with a pinned-SHA-256
archive download. Port the same implementation across.
Progress goes to the pipeline rather than the console, so the quiet path swallows
it exactly as it swallowed astral's installer output and the printed lines around
the call site are unchanged.
* Windows: stop pairing a hidden window with a bypassed execution policy
The Studio shortcut launched launch-studio.ps1 with -WindowStyle Hidden and
-ExecutionPolicy Bypass on the same command line. That pair is what Microsoft's
own detections key on, and studio/src-tauri/src/install.rs already refuses it for
the app's own launch of install.ps1.
The installer writes launch-studio.ps1 itself, so the file carries no
mark-of-the-web and RemoteSigned loads it. The hidden window is unchanged, so the
shortcut behaves exactly as before. The generated launcher's own child launch
moves to RemoteSigned for the same reason: it runs an inline -Command against an
executable, where no script file is loaded and the two policies are equivalent.
Also refresh a stale comment in studio/setup.ps1 that attributed the PSModulePath
fix to astral's uv installer, which no longer runs in-process.
* Installers: keep download-and-run command lines out of the shipped script text
AMSI scans install.ps1 in full before a single line of it runs, and generic
script classifiers read install.sh the same way inside the Linux bundle. Both
headers rehearsed the piped web one-liner five times over, plus a scriptblock
form and an execution-policy bypass, none of which anything in the scripts reads
and all of which the README already documents.
Point at the README instead and reword the in-body comments that quoted the
one-liner as shorthand. Every printed line is untouched: the remediation text the
installers show users still spells out the command in full.
Same treatment for scripts/uninstall.ps1's header.
* Windows: resolve process image paths with one Win32_Process query
install.ps1's venv-holder probe opened a handle to every running PID through
inline C# compiled at runtime. Opening a handle per process is a shape AV
heuristics score hard, and it bought nothing: Win32_Process reports
ExecutablePath for exactly the processes those handles could be opened against,
and answers for all of them in a single query instead of once per PID.
The remaining file-canonicalisation imports stay -- handle-based resolution of
linked ancestors has no faithful Windows PowerShell 5.1 equivalent, and it runs
on security-relevant paths.
Falls back to the per-process .Path when the query is unavailable, so a degraded
WMI repository degrades exactly as the old code did on a process it could not
open.
* Desktop: say who blocked the install when AMSI stops the script
PowerShell hands the whole top-level script block to AMSI while compiling it, so
a security product's verdict arrives as a parse error over the entire file before
install.ps1 runs a statement: no [TAURI:ERROR] marker, no phase log, and a stderr
tail the user cannot act on. unsloth#8523 shows what that looks like in the UI --
"Installation failed: + FullyQualifiedErrorId : ScriptContainedMaliciousContent".
Recognise the two stable error ids on either stream and append what the user
actually needs: nothing was installed, nothing was changed, it is a false
positive, update definitions and retry, do not turn off endpoint protection. The
raw id stays in the message, because the diagnostics report and any vendor
submission both need it.
Matches the id, never the message text, which is localized, and tolerates the
cmdlet suffix the Invoke-Expression form carries.
* Desktop: ship each bundle only the installer it can run
resolve_install_script picks install.sh on unix and install.ps1 everywhere else,
but the shared Tauri config bundled both into every target. The Linux AppImage
therefore carried 280 KB of Windows PowerShell it can never execute -- and it is
the largest script body a generic classifier walking the squashfs reads, which is
where Microsoft's Trojan:Script/Wacatac.B!ml verdict on 0.1.701-beta landed.
Move the resource map into the per-platform configs. The clean-machine job
already fails when a Linux bundle ships no install.sh; it now also fails when one
ships install.ps1, so the split cannot silently regress in either direction.
The .deb scanned clean with the same payload, so this is surface reduction rather
than a proven fix for that verdict.
* POSIX installers: install uv from a pinned release before falling back
install.sh downloaded astral's install.sh to a temp file, ran it and deleted the
file; studio/setup.sh piped it straight into a shell. Both are, shape for shape,
what a dropper does, and generic ML script classifiers score them accordingly --
the 0.1.701-beta Linux AppImage came back Trojan:Script/Wacatac.B!ml while the
.deb carrying the same scripts came back clean.
Fetch the pinned release archive and verify a hardcoded SHA-256 instead, matching
what install.ps1 already does on Windows. Only the four mainstream targets are
pinned: musl, armv7 and any host without a digest tool keep the path they have
today, because guessing a target triple wrong would break the install outright
and that costs far more than the heuristic score of the fallback.
Destination, PATH handling and every printed line are unchanged, so a host that
takes either path ends up in the same state it did before.
* tests: pin the installer shapes antivirus heuristics score
One file collecting what was removed, so it cannot drift back: no remote script
run in-process, no encoded or base64 payload, no hidden window paired with a
bypassed execution policy, no handle opened against another process, and no new
runtime-compiled native import outside an allowlist that carries a reason for
each entry that stays.
The last test is the other half of the contract. Hardening must not change what a
user sees, so the remediation lines the installers print -- which still spell out
the web one-liner in full -- are asserted verbatim. Removing the one-liner from
comments is the point; removing it from what the user is told to run would be a
regression.
Runs on the existing discovery-based pytest step, no workflow list to update.
* release: emit a false-positive submission packet for whatever gets flagged
The build job assembles a Microsoft submission packet, but only for the Windows
-setup.exe. The detection that actually arrived on 0.1.701-beta was
Trojan:Script/Wacatac.B!ml on the Linux AppImage, so nothing was produced for the
one asset that needed it.
The VirusTotal job already knows which assets were flagged and by which engines,
so put the packet there: hash, size and both portals, for every flagged asset
whatever platform it came from, with a note that clearance is per hash and per
vendor. Engine names are not repeated -- they are third-party text and already
appear escaped under Flagging engines.
The gate stays advisory; this only makes acting on it take seconds.
* Revert "Windows: resolve process image paths with one Win32_Process query"
This reverts commit
|
||
|
Daniel Han
|
5a5bf64130
|
Reduce antivirus false positives in the desktop installers (#8586)
* Windows setup: install uv from a pinned release instead of running remote script text
studio/setup.ps1 piped astral's install.ps1 straight into Invoke-Expression. That
download-and-execute shape is the single construct AMSI providers and cloud ML
scanners score hardest, and install.ps1 already replaced it with a pinned-SHA-256
archive download. Port the same implementation across.
Progress goes to the pipeline rather than the console, so the quiet path swallows
it exactly as it swallowed astral's installer output and the printed lines around
the call site are unchanged.
* Windows: stop pairing a hidden window with a bypassed execution policy
The Studio shortcut launched launch-studio.ps1 with -WindowStyle Hidden and
-ExecutionPolicy Bypass on the same command line. That pair is what Microsoft's
own detections key on, and studio/src-tauri/src/install.rs already refuses it for
the app's own launch of install.ps1.
The installer writes launch-studio.ps1 itself, so the file carries no
mark-of-the-web and RemoteSigned loads it. The hidden window is unchanged, so the
shortcut behaves exactly as before. The generated launcher's own child launch
moves to RemoteSigned for the same reason: it runs an inline -Command against an
executable, where no script file is loaded and the two policies are equivalent.
Also refresh a stale comment in studio/setup.ps1 that attributed the PSModulePath
fix to astral's uv installer, which no longer runs in-process.
* Installers: keep download-and-run command lines out of the shipped script text
AMSI scans install.ps1 in full before a single line of it runs, and generic
script classifiers read install.sh the same way inside the Linux bundle. Both
headers rehearsed the piped web one-liner five times over, plus a scriptblock
form and an execution-policy bypass, none of which anything in the scripts reads
and all of which the README already documents.
Point at the README instead and reword the in-body comments that quoted the
one-liner as shorthand. Every printed line is untouched: the remediation text the
installers show users still spells out the command in full.
Same treatment for scripts/uninstall.ps1's header.
* Windows: resolve process image paths with one Win32_Process query
install.ps1's venv-holder probe opened a handle to every running PID through
inline C# compiled at runtime. Opening a handle per process is a shape AV
heuristics score hard, and it bought nothing: Win32_Process reports
ExecutablePath for exactly the processes those handles could be opened against,
and answers for all of them in a single query instead of once per PID.
The remaining file-canonicalisation imports stay -- handle-based resolution of
linked ancestors has no faithful Windows PowerShell 5.1 equivalent, and it runs
on security-relevant paths.
Falls back to the per-process .Path when the query is unavailable, so a degraded
WMI repository degrades exactly as the old code did on a process it could not
open.
* Desktop: say who blocked the install when AMSI stops the script
PowerShell hands the whole top-level script block to AMSI while compiling it, so
a security product's verdict arrives as a parse error over the entire file before
install.ps1 runs a statement: no [TAURI:ERROR] marker, no phase log, and a stderr
tail the user cannot act on. unsloth#8523 shows what that looks like in the UI --
"Installation failed: + FullyQualifiedErrorId : ScriptContainedMaliciousContent".
Recognise the two stable error ids on either stream and append what the user
actually needs: nothing was installed, nothing was changed, it is a false
positive, update definitions and retry, do not turn off endpoint protection. The
raw id stays in the message, because the diagnostics report and any vendor
submission both need it.
Matches the id, never the message text, which is localized, and tolerates the
cmdlet suffix the Invoke-Expression form carries.
* Desktop: ship each bundle only the installer it can run
resolve_install_script picks install.sh on unix and install.ps1 everywhere else,
but the shared Tauri config bundled both into every target. The Linux AppImage
therefore carried 280 KB of Windows PowerShell it can never execute -- and it is
the largest script body a generic classifier walking the squashfs reads, which is
where Microsoft's Trojan:Script/Wacatac.B!ml verdict on 0.1.701-beta landed.
Move the resource map into the per-platform configs. The clean-machine job
already fails when a Linux bundle ships no install.sh; it now also fails when one
ships install.ps1, so the split cannot silently regress in either direction.
The .deb scanned clean with the same payload, so this is surface reduction rather
than a proven fix for that verdict.
* POSIX installers: install uv from a pinned release before falling back
install.sh downloaded astral's install.sh to a temp file, ran it and deleted the
file; studio/setup.sh piped it straight into a shell. Both are, shape for shape,
what a dropper does, and generic ML script classifiers score them accordingly --
the 0.1.701-beta Linux AppImage came back Trojan:Script/Wacatac.B!ml while the
.deb carrying the same scripts came back clean.
Fetch the pinned release archive and verify a hardcoded SHA-256 instead, matching
what install.ps1 already does on Windows. Only the four mainstream targets are
pinned: musl, armv7 and any host without a digest tool keep the path they have
today, because guessing a target triple wrong would break the install outright
and that costs far more than the heuristic score of the fallback.
Destination, PATH handling and every printed line are unchanged, so a host that
takes either path ends up in the same state it did before.
* tests: pin the installer shapes antivirus heuristics score
One file collecting what was removed, so it cannot drift back: no remote script
run in-process, no encoded or base64 payload, no hidden window paired with a
bypassed execution policy, no handle opened against another process, and no new
runtime-compiled native import outside an allowlist that carries a reason for
each entry that stays.
The last test is the other half of the contract. Hardening must not change what a
user sees, so the remediation lines the installers print -- which still spell out
the web one-liner in full -- are asserted verbatim. Removing the one-liner from
comments is the point; removing it from what the user is told to run would be a
regression.
Runs on the existing discovery-based pytest step, no workflow list to update.
* release: emit a false-positive submission packet for whatever gets flagged
The build job assembles a Microsoft submission packet, but only for the Windows
-setup.exe. The detection that actually arrived on 0.1.701-beta was
Trojan:Script/Wacatac.B!ml on the Linux AppImage, so nothing was produced for the
one asset that needed it.
The VirusTotal job already knows which assets were flagged and by which engines,
so put the packet there: hash, size and both portals, for every flagged asset
whatever platform it came from, with a note that clearance is per hash and per
vendor. Engine names are not repeated -- they are third-party text and already
appear escaped under Flagging engines.
The gate stays advisory; this only makes acting on it take seconds.
* Revert "Windows: resolve process image paths with one Win32_Process query"
This reverts commit
|
||
|
Daniel Han
|
90a6a236b5
|
Harden the workflow-trigger lint: scan .yaml, and host it outside the workflow it audits (#8545)
* lint_workflow_triggers: scan .yaml workflows too, and pin security-audit to every PR GitHub Actions loads both .yml and .yaml out of .github/workflows/, but the trigger lint only globbed *.yml, so an evil.yaml carrying pull_request_target would run for real and still lint clean. Also adds a regression test asserting security-audit.yml's pull_request trigger has no paths filter, since the lint job lives inside that workflow. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Host the workflow-trigger lint in its own unfiltered workflow The lint job lived inside security-audit.yml and inherited its triggers, so the gate was only ever as broad as an unrelated heavy audit workflow's filter policy. Move it to workflow-trigger-lint.yml, which triggers on pull_request with no paths filter. pull_request resolves the workflow file from the PR merge ref, so a PR that adds a paths or paths-ignore filter to the host skips the host for its own PR and the gate never reviews the change. A pytest assertion cannot catch that, because the test runs inside the workflow being skipped. The lint now checks its own host instead: it rejects both filter keys on any workflow that runs the script, and fails when no unfiltered host exists at all. That still leaves the tampering PR itself, so CODEOWNERS now covers .github/workflows/ and CODEOWNERS itself, which is the merge-time control. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Detect the lint host from parsed steps, and check effective CODEOWNERS Host detection matched a regex against the raw workflow text, so a commented out '# - run: python3 scripts/lint_workflow_triggers.py' registered as a host. That defeats the fail-closed check: delete the real workflow, leave a comment behind, and --require-host still passes. Read the parsed jobs/steps instead, which also covers multi-line run blocks. The CODEOWNERS guard asserted a matching rule existed somewhere in the file. GitHub applies only the LAST matching pattern, so appending '* @someone-else' would take over while the guard stayed green. Resolve the effective owners for a workflow path and for CODEOWNERS itself, and add a test that the guard fails when a broader rule is appended. * Require the lint host to be unnarrowed and blocking, and check every workflow's owner The host check only rejected paths and paths-ignore. branches, branches-ignore and types skip PRs just as effectively, so a host restricted to another branch counted as unfiltered. Require a bare pull_request: with no configuration at all, which covers those keys and any future one. A host whose lint step or job carries continue-on-error runs but cannot fail, so --require-host passed while findings were advisory. Reject that too. The CODEOWNERS guard probed only the lint host and CODEOWNERS itself, so a narrower trailing rule could take a different workflow away from its owner while both probes still passed. Check every workflow file has some effective owner, keeping the danielhanchen requirement for the two that matter most. Delegating a workflow to another maintainer stays fine; leaving one unowned does not. The rule parser also skipped ownerless lines, which are valid CODEOWNERS and clear ownership, so a bare pattern was an invisible carve-out. * Reject an if-gated lint host, and glob CODEOWNERS directory patterns An if: condition on the lint step or its job skips the lint while the run still succeeds, which is the same defeat as continue-on-error by another key. Reject any if: on a host rather than trying to prove one always true. The CODEOWNERS matcher compared trailing-slash patterns as literal substrings, so a valid trailing rule like '**/workflows/ @someone-else' took the lint host away from its owner while the guard still computed danielhanchen. Match every directory prefix with fnmatch, allowing unanchored patterns to start at any depth, and cover globbed, unanchored and wildcard-segment rules in the regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Require a real lint invocation, reject needs:, and tighten owner matching Host detection accepted any run text containing the script name, so 'echo scripts/lint_workflow_triggers.py' counted as running the gate. Require the script to appear as the argument of a python interpreter at the start of a command. A lint job with needs: is skipped when its prerequisite is skipped, and the workflow still succeeds. Reject needs: on a host, same as if: and continue-on-error. CODEOWNERS wildmatch lets '**/' match zero directories, so '/.github/**/workflows/' overrides the host while fnmatch returned false. Expand '**/' into both forms before matching. Owner tokens are also validated: GitHub cannot request review from a bare word, so a trailing rule naming 'not-an-owner' leaves the path effectively unowned and no longer counts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject a defanged lint invocation, and match CODEOWNERS globs per segment A host running 'lint_workflow_triggers.py || true' passed every check: it is a real invocation, the trigger is unfiltered, and no metadata flags it, yet findings can never fail the run. Same for a command passing --workflows-dir at somewhere empty, or --no-require-host, which leaves the job green while it gates nothing. Reject failure-masking shell and both neutering flags. The CODEOWNERS matcher used fnmatch, whose '*' consumes '/'. That made '/.github/*' appear to claim nested workflow files, so a valid CODEOWNERS change touching only direct children of .github would have failed the guard. Translate patterns to a regex instead, where '*' stops at a separator, '**' crosses them and '**/' may match zero directories, and only expand directory prefixes for patterns without a wildcard. This replaces the '**/' expansion workaround, so the helper is now wrong in neither direction. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Require python to execute the lint file, and anchor slashed CODEOWNERS patterns 'python3 -c pass scripts/lint_workflow_triggers.py' names the script but runs the -c program and exits 0. Confirmed directly: rc=0, nothing scanned. The old regex allowed arbitrary arguments before the basename, so that counted as a host. Tokenize the command instead and require the script to be python's executed file, walking option flags but stopping at -c or -m. CODEOWNERS anchoring followed gitignore only for a leading slash. A pattern with an internal separator is root-relative too, so 'workflows/lint.yml' was being tried at every depth and appeared to override '.github/workflows/lint.yml'. That failed valid CODEOWNERS changes aimed at a top-level workflows/ directory. A bare 'workflows/' still floats to any depth. Also pins that ordinary invocations keep working: -u, -X with a value, and plain 'python'. * Require the host to run the repo lint as a plain command with no arguments Three separate holes had one shape, so this replaces the growing list of special cases with a single rule: a host runs scripts/lint_workflow_triggers.py as a standalone command, with no arguments. That covers a decoy /tmp/lint_workflow_triggers.py sharing the basename; a pipeline or background job, where the step's exit status need not be the lint's, since the default run: shell is bash -e with no pipefail; and every argument, including --help, which exits 0 before scanning, and the abbreviated --workflows-d that slipped past the old substring denylist. The _MASKED regex and the NEUTERING_FLAGS list are both gone. argparse also now runs with allow_abbrev=False, so --workflows-d is rejected by the script itself rather than only by the host check. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the lint path exactly, require a lone command, and check the step shell Three more ways a host could look wired while running nothing. The path was a suffix match, so a decoy at /tmp/scripts/lint_workflow_triggers.py passed. Require the exact repo-relative token, or the ./ form. Judging lines in isolation cannot tell a call from a definition, so an invocation parked in an uncalled function or a here-document read as enforcing. The step body must now be the lint command and nothing else, which sidesteps shell parsing entirely and makes the separate set +e check redundant. A custom shell template such as bash -c '"{0}" || true' wraps the command and drops its exit status while the run line stays plain. Only bash and sh count, resolved through step, job defaults.run and workflow defaults.run. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Classify publishers by stem, and tighten what counts as running the lint Scanning .yaml made PUBLISH_WORKFLOW_NAMES inconsistent: a rename to release-desktop.yaml would be loaded but no longer classified as a publisher, so a cache key shared with a PR workflow stopped being a finding. Match on the stem instead. This one was introduced by the .yaml change in this PR. Three ways the interpreter was accepted without running the file. The regex matched any command containing python, so /tmp/fakepython passed; require the BASENAME to be a python, keeping any directory prefix. -V, --version, -h and --help before the path make python print and exit 0, confirmed directly, so they now disqualify like -c and -m already did. And working-directory resolves the same plain command to a different file, so a host setting it on the step, the job defaults or the workflow defaults is rejected, alongside the existing shell check. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Allowlist interpreter flags, reject redirecting env, validate the pull_request value Interpreter flags are now an allowlist rather than a denylist. Each round of review found another flag that stops the file running or masks its status, and -i was the latest: it enters the REPL after the script, so on EOF the process exits 0 even though the lint called sys.exit(1). Confirmed directly, rc=0. Only flags that leave run-this-file-and-return-its-status intact are accepted, so an unrecognised flag fails closed instead of needing to be enumerated. BASH_ENV, ENV and PATH redirect the step without the command text changing at all: non-interactive bash sources BASH_ENV before the step script, so an exit 0 there ends the step before the lint runs. Rejected at workflow, job and step scope. pull_request: false was read as unrestricted, because only mappings were inspected. GitHub rejects a non-mapping event configuration and will not load the workflow, so the value must now be bare or a mapping. * Tighten comments on the workflow-trigger lint Comment and docstring wording only, no behaviour change. The blocks grew a clause per review round; this keeps the reason each rule exists and drops the retelling. * Stop trusting PR-controlled interpreters, option values and startup vars Three variants of one mistake: the host check trusted content the PR itself can supply. A basename-only interpreter check accepted ./python3, which a PR can add to the repository root. Require a bare command or an absolute system path. An allowlisted value-taking option had its value consumed unchecked, so a command substitution in it would run before python started. Option values are now rejected for shell syntax like trailing arguments already were. PYTHONPATH, PYTHONHOME and PYTHONSTARTUP join BASH_ENV, ENV and PATH: a sitecustomize.py on PYTHONPATH is imported before the script and can exit 0. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject expansions in the interpreter token and containerized host jobs The trusted-path check read the token before bash expands it, so "/usr/$(...)/python3" passed on basename and prefix while the substitution ran first. Reject shell syntax there, as option values and trailing arguments already were. A job with container: runs its steps in a PR-selected image that controls the shell and environment, which the workflow-level env merge cannot see. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
4a53e80118
|
security: the network check could not see httpx2 (#8565)
* security: the network check could not see httpx2 httpx2 is the pydantic-maintained successor to httpx and a separate import name, so the httpx-only alternative in RE_NETWORK did not match it. openai 3.0.0 requires httpx2 and routes every call through it, which made the SDK's own HTTP invisible to every combined check that needs a network half: secrets plus network, IMDS plus network, archive plus network. The visible symptom was openai/auth/_workload.py reporting as a standalone HIGH "accesses cloud metadata" rather than the combined CRITICAL, because the scanner could not correlate the IMDS URLs with the httpx2.Client() calls beside them. Widening it surfaces four findings, all in openai, all reviewed and benign, all baselined here. Two tests pin the behaviour so the widening cannot be reverted silently. * security: pin the openai baseline entries to the reviewed file digest RE_NETWORK matches httpx2.Client where it appears in a signature, but not a call through an instance. So a client.post(..., data=api_key) appended to one of these files contributes no evidence: the evidence hash is unchanged and the entry keeps suppressing it. Reproduced, the two hashes are byte-identical. All four entries now pin the reviewed digest, the mechanism _load_baseline already documents for files whose danger sits outside the matched lines. _workload.py is pinned too: it has real httpx2.Client() calls, but an added client.post would leave its evidence unchanged just the same. Five older entries for these paths are dropped. They were baselined when openai used bare httpx and carry the identical annotation-only evidence, inert against 3.0.0 but suppressing on any resolve back to an httpx-based version. They cannot be pinned to an artifact that was never reviewed, so they go and reopen if that evidence recurs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
e468965b43
|
Pin the ROCm-on-WSL bootstrap to immutable refs (#8540)
* Pin the ROCm-on-WSL bootstrap to immutable refs
The WSL AMD bootstrap runs unattended and installs with sudo, so both pieces
of code it pulls in should be addressed by commit, not by a moving branch:
- install.sh fetched the helper from raw.githubusercontent .../unsloth/main,
so anything landing on main became root code on affected WSL hosts. It now
fetches a pinned commit SHA.
- the helper cloned ROCm/librocdxg at develop, then cmake/make/sudo make
install. It now builds tag v1.2.2 and verifies the clone resolves to that
tag's commit SHA before anything is built or installed.
v1.2.2 is what librocdxg develop points at today and the pinned helper commit
is the current main copy, so the installed result is unchanged. An explicit
UNSLOTH_LIBROCDXG_REF with no UNSLOTH_LIBROCDXG_SHA still builds whatever the
operator asked for.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Point the helper pin at current main and forward the librocdxg pin
Two follow-ups so WSL users keep getting the current helper and the pinned
third-party source applies immediately:
- _ROCM_WSL_HELPER_REF now points at current main (
|
||
|
Daniel Han
|
f5c64f4a08
|
security: lockfile audit must block non-registry sources and missing integrity by default (#8541)
* security: lockfile audit must block non-registry sources and missing integrity by default The pre-install lockfile gate detects non-registry npm resolved URLs, missing npm integrity hashes, non-registry Cargo sources and missing Cargo checksums, but #5604 left all four out of BLOCKING_KINDS, so they only printed a :⚠️: and the script exited 0. Every workflow calls the auditor without --strict immediately before `npm ci`, which runs package lifecycle scripts, so a lockfile pointing at an attacker controlled tarball with a matching integrity value passed the gate and its postinstall ran on the runner. Move those four kinds into BLOCKING_KINDS. An integrity hash written by whoever also wrote the resolved URL proves nothing; the registry origin is what makes it meaningful, so both belong on the blocking side of a pre-install fetch gate. Kinds that describe an incomplete entry rather than a fetchable source (missing-resolved-url, unsupported-lockfile-version) stay advisory, and --strict still escalates everything. All four checked-in lockfiles pass in default mode and under --strict, so no existing job changes colour. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * lockfile audit: correct the --strict help text for the new default blockers The --strict help still told operators that default mode blocks only known-malicious versions, IOC strings and broken lockfiles and that everything else exits 0. Provenance and integrity findings now block by default, so --help contradicted BLOCKING_KINDS for anyone deciding whether they needed --strict. * lockfile audit: tighten the comments added by this change * lockfile audit tests: tighten docstrings --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
993e3e4529
|
Studio: drop the duplicated HubModelPicker import in model-selector (#8534)
* Studio: drop the duplicated HubModelPicker import in model-selector model-selector.tsx imports HubModelPicker and hasDownloadedModels twice, identically, at lines 46 and 53. #8470 added the second copy. A repeated named import binds the same identifier twice in module scope, which is a hard error rather than a lint nit: tsc reports TS2300 Duplicate identifier, and the equivalent ESM is a SyntaxError, SyntaxError: Identifier 'readFileSync' has already been declared so the bundler rejects it too. That fails the frontend build during studio setup, before any test runs, which is why unrelated jobs have been red on every studio branch. Removing the earlier line rather than the later one keeps the import grouped with the other ./model-selector/* imports it belongs with. * Security audit: baseline the five reopened scan-packages findings openai 3.0.0 landed on the extras shard and pulled in httpx2 (the Pydantic-maintained httpx fork), and botocore/httpsession.py picked up a timeout kwarg, so five reviewed heuristic matches came back with new evidence hashes and failed all three pip scan-packages shards. Each one was re-reviewed against the published archive, and every flagged file was diffed byte-for-byte against its upstream source repo before being suppressed: - httpx2 2.10.0 httpx2/_models.py, "Enumerates filesystem AND makes network calls": RE_FS_ENUM's `history.*read` alternative matching the `history: list[Response] | None = None` Response kwarg, plus the urllib.request cookie-compat shim. All 30 .py files in the wheel are identical to pydantic/httpx2 v2.10.0. - openai 3.0.0 openai/_base_client.py, "C2 polling/beaconing loop": `while True:` in SyncPage.iter_pages, the pagination generator. - openai 3.0.0 openai/auth/_workload.py, "Accesses cloud metadata / IMDS endpoints": the Azure managed-identity and GCP instance-identity token providers behind workload identity federation. This file was already baselined under the combined IMDS+network check; it now fires the standalone IMDS check because RE_NETWORK does not recognise `httpx2.` calls. - unsloth-zoo 2026.8.10 unsloth_zoo/device_map_planner.py, "Advanced obfuscation + exec/eval": `__import__` of two literal module paths in the compute_module_sizes lookup, and `model.eval()`, which is the PyTorch eval mode, not eval(). - botocore 1.43.69 botocore/httpsession.py, "Harvests environment variables/secrets AND makes network calls": the standard SSLKEYLOGFILE support plus urlopen in URLLib3Session.send. The evidence changed because the urlopen call now takes **extra_kwargs for the per-request timeout. - pandas 2.3.3 pandas/tests/io/test_pickle.py, "Creates archive with sensitive data AND makes network calls": tarfile.open in the compression test helper plus a monkeypatched urllib.request.urlopen. The six entries are appended by hand with evidence_hash computed by scan_packages._evidence_hash over the scanner's own evidence, so the existing 218 entries are untouched (218 -> 224). Verified locally with `scan_packages.py --with-deps` over the CI-identical audit-reqs files: extras and hf-stack now exit 0. |
||
|
Daniel Han
|
8080c93c9e
|
Studio: install sd.cpp under the Studio home, not beside it (#8226)
* Studio: install sd.cpp under the Studio home, not beside it A custom UNSLOTH_STUDIO_HOME put the managed stable-diffusion.cpp tree at the home's parent, unlike llama.cpp, whisper.cpp and node, which all install under the home. For a relative home that parent collapses to the working directory, so a stable-diffusion.cpp checkout sitting there was picked up as the managed install and install_sd_cpp_prebuilt refused to run: the target was a pre-existing non-empty directory without the ownership marker. Derive the root as "studio home"/stable-diffusion.cpp from an absolutised home, in both the installer and the engine's finder, matching default_managed_llama_dir. The legacy default home ~/.unsloth/studio still maps to ~/.unsloth/stable-diffusion.cpp, and a tree an older build installed beside the home is still discovered and still repairable, gated on the ownership marker so an unrelated checkout is never adopted. * Stop the sd.cpp server under a custom root before uninstall deletes it Moving the managed tree under the Studio home left the uninstaller listing only the old sibling location as an owned root. The tree itself still goes, because the custom root is removed wholesale, but a resident sd-server survives unlinking its binary, so it kept running and holding its port while its install disappeared underneath it. List both locations, each still gated on the ownership marker so a checkout the user keeps at either path is never signalled. The Windows script needs no equivalent change: its handle scan already walks every known root by prefix, which now contains the tree. Comments in both scripts updated to describe where the install actually is. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Read the sd.cpp accelerator record from the root the binary is in The accelerator upgrade check read the record from the current managed root while the finder may hand back a binary from the tree an older build installed beside the Studio home. That root holds no record, unrecorded reads as a mismatch for a GPU target, and the matching bundle already on disk gets downloaded again on every load. owning_managed_root() returns the marker-owned root a binary actually lives under, and the check reads the record from there. is_managed_binary is now a thin wrapper over it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Find a legacy install beside a symlinked home, and never serve a stale legacy server after an install - The compatibility lookup resolved the home before taking its parent, while the old installer took the lexical parent. For a home that is itself a symlink the tree an older build created sits next to the LINK, so the lookup missed it: a needless re-download, and the old install left orphaned from the uninstaller too. Lexical first, resolved after, both still marker-gated. - After an install that ships no sd-server, the finder's legacy probe could hand back the old server built for another accelerator, and ensure_sd_server_binary returned it without rechecking. A forced CUDA load then ran the legacy CPU server. It now returns None there, so the router uses the sd-cli the install just landed. * Apply the kwarg-spacing formatter to two files it had not been run over * Clear the legacy sd.cpp beside a symlinked Studio home, and stop reinstalling a serverless bundle Uninstall derived the legacy sibling from the canonicalized custom root, so for a Studio home that is itself a symlink it looked beside the link's TARGET while an older build had installed beside the LINK: that tree was never stopped and never removed. Take the lexical parent as an extra marker-gated candidate, matching what the finder already does. And once the current managed root holds a completed install for the accelerator being asked for, a mismatched sd-server still sitting in that legacy tree is not a reason to install again: the bundle simply shipped no server. Without this, every model load re-downloaded it. * Make the serverless suppression evidence-based, and close two gaps in the uninstall stop pass Four follow-ups on the review of the previous commit. The serverless guard fired on the accelerator record alone, which reads a server that was deleted by hand (or by the runnability repair) as a bundle that never had one, and then suppresses the very reinstall that would put it back. install() now records ships_server, taken off the archive member list, and the guard needs that recorded false plus a genuine mismatch on the legacy side. An unrecorded install stays unknown and keeps its old behavior, and a legacy server that matches the accelerator being asked for is still preferred over the one-shot CLI. The guard also has to run before _accelerator_changed, which reports "unchanged" while the managed tree is in use, so a load starting during a generation was handed the mismatched legacy server. In uninstall.sh, the lexical sibling reached the string-based deny list without being resolved, so a home carrying ".." could aim a removal at a protected tree; canonicalize a copy for that check. And the nested <root>/stable-diffusion.cpp is now stopped even when unmarked, provided the root is a Studio root this run deletes: the current-root finder can select an unmarked binary there, and the marker gate belongs to the paths that survive when unowned, not to a tree that goes regardless. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Memoise the bundle's server capability with the accelerator, not separately An unwritable install record already kept the accelerator in process memory, so a serverless install whose record could not be written read back as server-capable and the load that finds a mismatched legacy server went on reinstalling. Remember both together, or neither is trustworthy. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give the Windows stop scan the physical path of a linked Studio home The backend resolves the home before deriving the nested sd.cpp tree, so on a junction or directory symlink the native binaries run out of the target. _CustomStudioRoots only normalizes the string (System.IO.Path.GetFullPath is lexical and never follows a reparse point), and the scan matches Win32_Process.ExecutablePath by prefix, so the running server was never matched and survived an uninstall that deleted its tree. Add the reparse target to the stop scan only; the deletes still refuse to chase a link out of the expected location. * Scope both uninstall stop paths to trees this run actually owns The lexical sd.cpp sweep skipped the ownership check the canonical loop makes first, so a stale or mistyped UNSLOTH_STUDIO_HOME pointing at a path that was never a Studio could take the marked legacy sibling of a different, valid install. Apply _is_studio_root there too. On Windows the stop scan was handed the whole physical target of a linked home. The delete leaves that target standing, so anything there that is not ours is neither locking nor being removed; pass the Studio-managed subtrees underneath it instead, and only for the homes, since a component dir can itself be a link onto a shared runtime. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
f776ce30a5
|
Video: make MiniMax-H3's Diffusers path fast by default (#8320)
* Video: make MiniMax-H3's Diffusers path fast by default MiniMax-H3 on the Diffusers path was the slowest thing Studio ships. The same 960x544, 124-frame, 8-step job took 70-207 seconds a generation on a B200 with nothing else resident, and the spread is the tell: every component was in the ComponentsManager offload rotation, so the time was weight traffic, not maths. Three separate things caused it, and all three were decisions rather than limits. 1. The conditioner. H3 reads hidden_states[50] out of a 64-layer Qwen3-VL and never calls lm_head, but the released encoder is 66.7 GB of bfloat16 and transformers has no early exit, so the default paid 64 layers of compute and moved 66.7 GB across the offload boundary to read layer 50. The hosted quantized conditioner is 27.1 GB over 50 layers and computes the same state. It already existed and was already loadable, but only behind an explicit text_encoder_quant, which the video page never sends. An unset request now resolves to it. 2. The denoiser's placement. With the conditioner at 27.1 GB the released 66.3 GB denoiser fits alongside it, so it is now taken out of the offload rotation and pinned when the free VRAM measured at load time holds it plus the conditioner, the VAEs and a frames-aware activation headroom. A module that moves every forward also cannot be compiled, so this is what unlocks 3. 3. The speed layer. apply_speed_optims and select_attention_backend live below the modular dispatch in load_pipeline, which returns before reaching them, so every H3 load reported speed_optims [] and attention_backend null: no channels_last VAE, no cudnn.benchmark, no backend pinning, no compile. supports_torch_compile was also False for the family. Both are fixed. The compile is engaged only over a resident denoiser: inside a full offload rotation it measured slower than eager, so that case keeps the lossless tier. Measured on a B200, 960x544x124, 8 steps, guidance 1.0, seed 11, warm: free VRAM before after 191 GB 70.1 / 114.6 s 12.84 / 12.83 s 123 GB 83.6 / 176.6 s 45.9 / 42.1 s 80 GB 119.1 / 136.2 s 90.0 / 99.5 s Quality, same harness (scripts/video_quality.py), at H3's own 30-step default against the released dense bfloat16 components as reference: mean SSIM 0.9504, PSNR 33.0 dB, temporal deviation 0.042, no NaN, no black frames. The reference compared against itself scores 0.9935, so that is a small perturbation of the same sample, and the frames are visually indistinguishable. What is deliberately NOT defaulted: the hosted pre-quantized denoisers. They are much faster still (11.3 s warm, and they run in 58 GB) and they produce no NaN, no black frames and no visible degradation, but at 30 steps they re-roll the sample: mean SSIM 0.49 for int8 and 0.43 for fp8 against that same 0.9935 ceiling. Different is not the same as worse, but nothing available here can tell those apart, so they stay opt-in and the default gets its speed from placement and compilation, which change no weights at all. The new default is CUDA-only. Apple Silicon keeps the engine it has today: the modular loader cannot run there at all, because ComponentsManager's auto CPU offload needs mem_get_info on the execution device and torch.mps has none. CPU-only hosts keep the native sd.cpp engine, untouched. torch.compile now also asks whether inductor can actually run in this process before engaging: the three Studio workers already refuse it when Triton is missing on Windows, but the diffusion backends run in the server process, which those gates never reach. * Close two gaps where the new tests did not cover the guard they name Mutation-checked the additions and two survived, both in the same shape: the test exercises a helper the guard uses, never the guard itself. compile_eligible could lose its torch_compile_runtime_available() call entirely and every test still passed. The two assertions that looked like they covered it read compile_eligible(...) is False without stubbing torch, and without the stub the dtype check makes it return False for every input, so they held whatever the gate did. Stub torch and add the positive control, so the False lines have something to be false against. The pin decision had the same problem the other way round: the test named "pinned only when it actually fits" only drove the sizing helper, and the comparison that authorises the pin lived inline in the modular load, where nothing reached it. Changing it to `if True` left the suite green. Lifted it to _h3_dense_denoiser_fits and asserted it directly, including the boundary and the denoiser-alone case, since a pin that should not have happened is an OOM rather than a slow generation. Six mutations now fail: the conditioner default, the runtime gate, the TORCHDYNAMO_DISABLE branch, the win32 branch, and both halves of the fit test. 511 passed across the video, speed and H3 suites. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments this branch added * Fence the H3 placement, honour speed=off, and reach the dense conditioner Three things the new default got wrong on the way in. The dense denoiser pin is a speed optimisation by its own reasoning, so an explicit speed=off has to decline it. Taking the denoiser out of the offload rotation trades the ability to budget it against the requested frame count for throughput, and "off" is the one request that says do not make that trade. The speed profile is now resolved above the placement so the pin can read it. The pre-quantized pin is not gated the same way: there it is correctness, a torchao module does not survive the mid-block move. load_components spends minutes building ~145 GB, and everything after it either moves weights onto the card or mutates process-wide backend flags. The conventional placement path fences on the load token for exactly that reason; this one did not, so a cancelled or superseded worker resumed there and put a 66.3 GB denoiser next to a model a replacement load already owned. The next check was the state commit, which is after the placement it is meant to prevent. An omitted text_encoder_quant now selects the hosted INT8 conditioner, which makes none/off the only way to ask for the released bfloat16 one, and neither spelling could be said. VideoLoadRequest accepted only the four schemes, and normalize_te_quant raised on "off" and "auto" behind it, so the bfloat16 reference configuration was unreachable through the API and no comparison against it could be run. Both gates now pass the opt-out through to the tri-state, which already read it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
ed656020cd
|
Start the bottom taper under the icon so the disc still reads round (#8321)
Easing from the icon centre line took the halo down to 75 percent of its sideways value at 40pt and 47 percent at 60pt, which is inside the disc a viewer actually sees, so the glow read as flattened along the bottom. The taper now waits until 50pt, inside the icon's own lower half, and runs over 30pt: 99 percent of sideways at 40pt, 80 percent at 60pt, and the icon label sits on 9.3 percent mean tint against 10.0 before. Regenerated on macOS with tiffutil, so the asset keeps the same two-page structure and sRGB profile. |
||
|
Daniel Han
|
7ef075b9bd
|
Studio: run MiniMax-H3's INT8 denoiser from the ConvRot checkpoint (#8293)
* Studio: run MiniMax-H3's INT8 denoiser from the ConvRot checkpoint The hosted INT8 denoiser quantizes both sides of every GEMM to 8 bits, so its error is set by whichever side has the heaviest outliers, and a DiT's activations are outlier-heavy by construction. ConvRot spreads that magnitude across a block-Hadamard group: the weight is rotated offline, the activation is rotated online, and because the normalized Hadamard is symmetric and orthogonal the two cancel exactly in float. Nothing about the model changes; the quantizer just sees a flatter distribution on both sides. The conditioner already runs this rotation (#8283). This is the denoiser half. The recipe lives in the prequant checkpoint's own metadata, beside adaln_form / curve_dim / curve_grid: a kind, a group size, and the exact list of rotated FQNs. The list is recorded rather than recomputed on purpose. The two halves of the identity live in different places, and if they ever disagree about which Linears are rotated, the mismatched ones produce finite, plausible-looking, completely wrong pixels with nothing to raise and nothing to notice. A rule for "which Linears" can drift with a code change; a list cannot. Everything fails closed against that one failure mode: - rotated artifacts carry a v2 format tag, so a Studio predating this code refuses them outright instead of running them unrotated, and the tag and the declared rotation are checked as a biconditional in both directions; - the online half is installed in the prequant loader itself, not in a family hook, so no route can load a rotated checkpoint without it; - the loader raises on an FQN it cannot find, a target that is not a Linear, an in_features the group does not divide, or a kind it does not implement, and validates every target before swapping any, so a partial install is impossible. A raise becomes a refused checkpoint and a dense fallback. The artifact ships under its own filename with the plain one still behind it as the fallback, so an already-installed Studio keeps resolving the checkpoint it understands rather than refusing the v2 tag and falling back to the 66.3 GB dense download. build_convrot_hadamard and rotate_convrot_activation move out of video_minimax_h3_te into the shared module and are re-exported from their old home. The conditioner's rotation and the denoiser's have to agree with the same comfy-kitchen definition down to the normalizer, and two copies of a matrix nobody re-derives at review time is how they would stop agreeing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refuse a ConvRot build that rotates nothing, and publish the rotated artifact under the name the loader asks for Two ways the builder could spend GPU-hours and tens of gigabytes on a file nothing can ever use: - A group that divides no quantized input axis leaves the rotatable set empty, and the build still stamped the v2 tag with an empty fqn list. rotation_metadata_error refuses exactly that at load time, so the artifact was unloadable by construction. Refused up front instead. - --upload-repo published every build as the legacy transformer_<scheme>.pt. The loader asks for the family's declared prequant_filenames name first and the derived <Model>-<SCHEME>.pt second, so a rotated artifact at the legacy name is either never resolved or resolved as the fallback by a build too old to honour the rotation, which refuses the v2 tag and drops to the dense download. A rotated build now goes to the declared name, refuses when the family declares none, and --upload-filename is the escape hatch. Plain builds keep the legacy name. The destination is resolved before the dense load so an unpublishable build fails in a second. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
Maheswar Kumar
|
7ef653d1b7
|
soften the dmg background glow and clear it off the icon label (#8296)
* soften and even out the dmg background glow The halo behind the app icon in the macOS install window read as a hard saturated disc with a soft rim, and it fell off faster downward than in the other three directions. scripts/make_dmg_background.py: - GLOW_STRENGTH 1.85 to 1.48 and GLOW_SIGMA 47 to 50, so the fully saturated core shrinks from about 52pt to 40pt radius and stays hidden under the 128pt icon Finder draws at the default DMG icon size - GLOW_MIX_RADIUS 130 to 110, keeping the core-to-edge hue blend in step with the wider sigma - drop GLOW_BOTTOM_FLOOR and GLOW_BOTTOM_SPAN and the downward easing in render_glow, so the halo is now radially even: 21.4 / 21.4 / 21.7 / 21.2 percent at 90pt left, right, up and down Removing the downward easing puts more tint behind the "Unsloth" icon label, 29.0 percent mean against 18.3 percent before. Black label text on that background still measures 13:1 luminance contrast, so it stays legible. studio/src-tauri/dmg/background.tiff is the regenerated two-page output, 660x400 plus the 1320x800 hidpi page. Verified by building a real DMG with the tauri bundler's own bundle_dmg script and the same arguments tauri-bundler passes, then opening the mounted volume in Finder. tests/studio/test_tauri_branding_contract.py passes and ruff is clean. No GPU checks apply. * Pin the dmg background asset to its renderer Add a contract test that rebuilds the two TIFF pages from scripts/make_dmg_background.py and compares them to the checked-in studio/src-tauri/dmg/background.tiff, so a constant change without a regenerated asset fails instead of shipping. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ease the dmg glow off the icon label The even halo put 29.0 percent mean tint behind the "Unsloth" label under the app icon. GLOW_BOTTOM_FLOOR 0.70 over a GLOW_BOTTOM_SPAN of 120pt takes that to 22.8 percent and leaves left, right and top at the full falloff, so the halo still reads as round. * Tighten the comment on the background art tolerance * clear the dmg glow off the icon label GLOW_BOTTOM_FLOOR 0.70 to 0.30 over a GLOW_BOTTOM_SPAN of 90pt instead of 120pt. The "Unsloth" label under the app icon now sits on 10.0 percent mean tint, down from 22.8, while left, right and top stay at the full falloff. * Hold the dmg icon label to AAA contrast over the halo The halo tint under the "Unsloth" label is the accessibility-relevant part of this artwork, and the asset-matches-renderer check passes happily if both sides move together. Assert the darkest pixel in the label band keeps 7:1 luminance contrast against black text. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
Nilay
|
54462b80c7
|
Fix two macOS installer bugs (#8279)
* Installer: stop planting a self-referential symlink inside the macOS launcher bundle * Uninstaller: keep the desktop app's data when that app is still installed * Cover the uninstaller's app-data gate in tests * Find the app that owns the bundle id instead of guessing its path for PR #8279 A packaged app kept in a renamed bundle or a subdirectory such as /Applications/AI & ML/Unsloth.app missed all three hard-coded candidates, so the uninstaller still wiped its settings, cookies and WebView storage. Match on CFBundleIdentifier via mdfind plus a scan of /Applications and ~/Applications, excluding install.sh's launcher, which shares the id but only opens a browser. * Drop the depth cap on the bundle scan for PR #8279 A bundle filed deeper than three levels, say /Applications/Development/AI/Local/ Unsloth.app, fell outside -maxdepth 3, so the uninstaller wiped the installed app's data. Prune at each .app instead: the walk never enters a bundle, which is what the cap was avoiding, so any nesting depth is reachable and free. --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
0cd73cf3fb
|
Make the desktop release contract tests fail when the contract breaks (#8228)
* Realign the desktop release tests with the post-publish VirusTotal job * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tie the release contract tests to what they are meant to guard Checkpoint of in-progress work, mutation testing still outstanding. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope the release wait and scan assertions to the mechanism they guard The wait checks searched the whole step, so a one-shot jobs API read beside an unrelated loop passed; they now run against the poll loop body and also require a break. The scan job's condition was accepted on a success() substring, which let a disjunction through; it now has to require success() conjunctively. And the scan's directory was compared by leaf between two download steps, so moving both under a new parent, or repointing the script argument, scanned an empty directory and still reported clean; the argument the step passes is now compared against the download path. * Tighten the comments on the release contract tests * Reject any job-level condition on the scan and require a live poll loop Accepting a condition that merely opened with success() let success() && false through, which skips the sweep after a publication that succeeded, so no job-level if: is accepted at all now: reaching virustotal-scan is needs:'s decision alone. The wait helper likewise selected a loop by shape, so turning while :; do into while false; do kept every assertion green while the shell skipped the API reads and fell through to a download that races the matrix; the helper now only selects an unconditional poll. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
39147e4b68
|
Report the precision actually in use, and refuse an explicit one we cannot honor (#8165)
* Studio: report the precision that actually ran, and refuse one that cannot
The loader already knew the truth and threw it away at the API boundary. Status
reported the ENGAGED transformer / text-encoder precision, but nothing echoed
back what the caller ASKED for, so once a fallback happened the request was
gone: the Advanced panel kept its dropdown on FP8 while a Q4_K_M GGUF ran, the
"Auto: X" badge was suppressed for exactly the case that needed it
(source !== "auto" rendered nothing), and a successfully generated image or
saved clip carried no evidence of the precision behind it.
Backend
- DiffusionResolvedControl gains `requested` (the raw ask, null when left to the
backend) and `status` ("applied" | "fell_back" | "unsupported") beside the
existing value/source/reason. Both default, so an older payload still parses.
- build_resolved_record keeps the request beside the engaged value and derives a
mismatch for the controls that answer in the vocabulary they are asked in.
memory_mode and attention_backend do not, so they are classified by the call
site instead of compared blindly.
- Every transformer decline site now records WHY, in the caller's terms: an
uncached hosted prequant, a re-plan that still needs offload, a dense-fit
miss, a failed quant build, an unsupported scheme, and the wrong load kind.
- quantize_text_encoders returns a TEQuantOutcome (mode + reason + status). The
int8 -> fp8 downgrade, the offload skip and the unsupported-device path were
all bare `return None`; the last one had no log line at all.
- Explicit precision fails closed. Host-level impossibilities are refused in
begin_load, so /images/load and /video/load answer 409 before anything is
evicted; footprint-dependent declines raise inside the load and surface on
load-progress. `auto` still falls back silently, and
UNSLOTH_DIFFUSION_ALLOW_PRECISION_FALLBACK=1 restores the old behaviour.
- Saved output metadata: images add text_encoder_quant / memory_mode /
offload_policy; video clips gain the whole build block images already had.
All read from the engaged state, none added to the required-key sets, so
older PNGs and sidecars still list.
Frontend
- resolved-precision.ts holds the badge/select decisions as pure functions. A
declined request now renders "FP8 -> OFF" in a warning tone with the reason in
the tooltip, instead of nothing.
- The Advanced selects reseed from the loaded build, so a declined scheme stops
advertising itself, and a "Loaded build" summary reports the transformer and
text-encoder precision plus the memory mode and resolved offload behaviour.
- A 409 refusal is surfaced as a titled, actionable toast; transformer_quant is
no longer sent for load kinds that cannot use it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Raise the precision refusal before the GPU handoff, not inside it
The 409 exists to preserve two things: the chat model holding the GPU,
and the several GB the load would otherwise pull down before failing.
The check was made in begin_load, which is too late for both.
begin_load runs inside acquire_for, and acquire_for evicts the current
owner under the arbiter lock BEFORE it runs the register callback. On
the image path it also runs after select_and_activate_engine, which
unloads the resident model on an engine switch. So an impossible
explicit precision was refused having already destroyed exactly what
the refusal was meant to protect: the user got a 409 and an empty GPU.
Both checks are now made by the route, alongside the sibling refusals
that already run there (the unloadable pick, the gated companion), and
before the device is taken. The copy in begin_load stays, since it is
the load path's own invariant and other callers reach it directly.
Diffusers only, on the image path. The native sd.cpp engine accepts
transformer_quant / text_encoder_quant for interface parity and ignores
them, so gating that path on a torchao capability would refuse loads
that work today. A probe failure leaves pending_name None and skips the
route check, which is the pre-existing behaviour rather than a new one.
`auto` is never refused, so a caller that left the precision to the
backend cannot reach any of this.
* Stop the Advanced reseed firing on generation-time record rewrites
Two separate ways the resolved record was read too literally.
The reseed effect keyed on JSON.stringify(resolved). That record is not
load-time-only: the backend rewrites entries of it during GENERATION.
speed_mode and attention_backend change when the deferred compile
profile engages on the 3rd image, and transformer_cache changes
whenever the step-cache threshold flips. Each of those moved the key
with no load behind it, so the effect re-ran and overwrote a Precision
the user had picked but not yet loaded. An edit made after a load is
meant to survive until the next LOAD replaces it. resolvedSeedKey
covers only the three controls the effect actually writes, and for
attention only the request side, since that is the field the reseed
reads for an auto or honored request and the one a rewrite leaves
alone. A real reload still re-fires: it always moves a request or an
engaged value on one of the three.
isResolvedHonored treated every status that was not "applied" as a
decline. `status` is typed wider than the backend's union on purpose,
so a newer backend can add a value, but that reading threw the
tolerance away: an unknown status painted a red "FP8 -> FP8" over a
request that was honored, and on memory_mode (asked "low_vram",
answered "sequential") a "LOW_VRAM -> SEQUENTIAL" that never happened.
Only the two statuses that mean a decline are now read as one. Staying
quiet is the safe direction, since the build that adds a status ships
the frontend that understands it.
* Name the fault behind a refused precision, and stop caching an OOM as one
Two things the fail-closed 409 made load-bearing that were fine while a
declined explicit scheme fell back quietly.
select_transformer_quant_scheme answers None for three different faults
and the refusal reported all of them as "'fp8' is not usable for family
'X' on this GPU". Measured here on a B200: torchao could not import at
all (cannot import name 'ScalingType' from torch.nn.functional, a
torch/torchao version skew), the smoke probe swallowed that, and every
explicit scheme was refused with a message blaming Blackwell hardware
that runs all of them. A skew is fixed by a pip install; a GPU limit is
not; a family the accuracy gate rules out is neither. explain_unusable_
scheme separates the three, shared by the image and video resolvers so
they cannot drift.
The smoke probe also cached an out-of-memory as a verdict on the scheme.
That probe now runs on the ROUTE thread, which is the point of raising
the refusal before the GPU handoff, so it meets a full GPU by design:
the resident chat model has not been evicted yet. One transient OOM
therefore refused that scheme for the rest of the process, on a host
that runs it fine seconds later. Allocator failures are no longer
remembered; every other failure still is, because those really are
properties of the build. torch.OutOfMemoryError subclasses RuntimeError
rather than MemoryError and has moved between torch and torch.cuda, so
both names are tried with the message as the backstop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Correct a comment the reseed-key change left behind
The dependency is no longer the serialized record; it is the load-time
projection of it, which is the whole point of the previous commit.
* Close the three places an explicit precision was still reported wrong
The native engine was exempt from the precision gate, on the grounds that refusing there would
break loads that work today. But the loads it works for are exactly the silent mismatch this
change exists to remove: sd.cpp accepts transformer_quant and text_encoder_quant for interface
parity, ignores them, and reports null, so an explicit FP8 succeeded having quantised nothing.
The diffusers path already refuses on the same CPU-only host, so the exemption also left the two
engines disagreeing about one request. It now refuses, with a message naming the engine; auto,
none and an omitted value still pass through untouched, and the existing escape hatch waives it.
The Loaded build panel labelled any non-GGUF load BF16, but a single-file safetensors keeps
whatever precision it was saved in and FP8 checkpoints are explicitly supported, so the one
panel whose job is to say what actually loaded was asserting a wrong number. It reads "As in
checkpoint" for single_file now, on the video page as well.
And the video loader rewrites an omitted transformer_quant to "off" under speed_mode="off"
before building the resolved record, so the record claimed the user had pinned bf16: the Auto
badge disappeared and the Precision select reseeded to none, leaving quantisation pinned off
after a Speed change and reload. The raw request is captured before the rewrite and reported.
Four tests, each confirmed against a mutation.
* Report the truth on partial casts, native builds and unprovable probes
- text-encoder quant that cast one encoder and not its sibling reported
"applied" and both loaders let the load through, recording the requested
mode while conditioning ran off a mixture; it is now reported as a
fallback and refused like any other declined explicit precision.
- the refusal message pointed users at "Choose Auto" for text_encoder_quant,
which both request models reject, so following it returned a 422.
- the native sd.cpp engine reports dtype "gguf" and no model_kind, so the
Loaded build panel labelled every native checkpoint BF16; the label rule
is now one shared helper covering both pages.
- native generation results carried no offload state, so every native image
recipe persisted it as null.
- the video load route probed CUDA precision before the training guard, which
allocated next to a training subprocess for a load about to be refused.
- a smoke-probe OOM before the arbiter eviction is not a verdict on the
scheme, but it reached the new route gate as one and refused the load the
eviction was about to make room for.
- the torchao import error is interpolated into the 409 detail and names the
absolute file that raised it; paths are stripped there and logged in full.
* Do not read a native null text-encoder quant as BF16
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refuse an unhonourable precision before the download plan is staged
The load routes refuse an explicit precision this host cannot honour, but the UI
plans and stages first, so the refusal arrived after the GGUF and its companions
(tens of GB on the video side) had already been pulled. Both checks are
network-free, so /images/download-plan and /video/download-plan now make them
before building the plan, and map the refusal to the same 409 the load routes
give.
Also fixes three Loaded-build panel rows that were only correct for diffusers:
the dense dtype label no longer calls a float16/float32 load BF16, the attention
row names the native sd.cpp engine instead of Native SDPA, and the Memory row
renders when an offload is active but no memory mode is set.
* Keep the plan-time precision check off the GPU, and off the load's blind spot
The plan runs before the load's training guard, so an uncached scheme sent
assert_precision_available into its quantise-and-matmul smoke probe and
initialised CUDA in the Studio process beside a running trainer. Staging needs no
GPU, so the check is skipped while training is active on both the image and video
plan routes; the load still refuses the same pick afterwards.
The video page also asked for its plan without the selected precision while
sending it on the load, so the plan cleared a scheme the load would reject and
staged the pipeline first. It now sends it under the same pipeline-only rule.
A single_file transformer is no longer labelled 'As in checkpoint':
from_single_file is handed the resolved torch_dtype, so an fp8 checkpoint is
upcast on load and the panel was hiding the dtype it actually runs in.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep two video route tests off the host's precision support
Both assert that a text_encoder_quant reaches the backend, and both now run
through a precision gate whose answer depends on the machine: a GPU-less runner
refuses fp8 with a 409 and the forwarding under test never happens. They run
under the product's own fallback escape hatch instead.
* Gate diffusion precision on the engine and mode that actually run
Three places reported or refused a precision that was not the one the
runtime would use.
The load route asked predict_engine which gate to apply, and a probe
failure left pending_name None, skipping both arms. Selection could then
land on sd.cpp anyway, which accepts the knobs and ignores them, so an
explicit fp8 loaded, quantised nothing and reported null. The gate is now
re-asked of the engine that was actually activated, and only when the
prediction missed, so a correct one is never paid twice.
quantize_text_encoders rewrites an int8 request to layerwise fp8 on any
family with no keep-bf16 schedule, and that path needs no torchao. Both
precision asserts consulted te_quant_supported about the raw int8 and so
refused loads the runtime would run and report as fell_back. New
effective_te_quant() resolves the downgrade before support is consulted.
The Recipe popover's Memory row substituted "auto" for a null memory_mode,
which is what the native engine always records, claiming the memory planner
had picked a mode on the one path that never runs it. An absent mode now
reports the offload alone.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refuse an offload-bound precision up front, and read the video precision live
An explicit dense precision with Memory=balanced/low_vram (or the legacy
cpu_offload flag) is incompatible on its face: those requests name their
offload policy without measuring anything, offload hooks move modules with
Module.to(), and torchao tensors do not survive it, so the loader skips the
dense build. The strict refusal then landed after the resident image model
had been torn down. The pre-handoff gate now takes the memory request and
refuses the pair before the GPU is acquired. fast and auto are decided from
the measured footprint and are untouched.
The video page's loadOrStage is memoized on [stage, pickGuard], so its plain
capture of transformerQuant froze at whatever was selected when the callback
was built. The ordinary auto to FP8 change then asked the plan with no
precision, skipping the pre-download refusal, and staged tens of GB before
the load rejected the same pick. It reads through a ref now, the same way
handleLoad already does.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Extend the offload precision gate to the encoder and to video
The pre-handoff gate refused an offload-bound dense transformer quant but
not the two adjacent cases.
quantize_text_encoders reports the torchao encoder modes (int8,
fp8_dynamic, nvfp4) unsupported once offload is active, for the same reason:
the hooks move modules with Module.to() and those tensor subclasses do not
survive it. The image gate now refuses them alongside an offload-forcing
memory request. Layerwise fp8 is a dtype cast and is untouched.
assert_video_precision_available took no memory request at all, so a video
load with an explicit precision and balanced or low_vram passed the route
preflight and was refused inside load_pipeline, after acquire_for and the
teardown had evicted the resident model. It takes memory_mode now and
applies both rules.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refuse a torchao encoder mode a host cannot import, and plan with the memory request
te_quant_supported only asks the device: a CUDA bf16 host with a broken or
absent torchao passed every capability check, and the casters import torchao
only after the pipeline has been downloaded and built, so the refusal came
through load-progress instead of the pre-load 409. Both gates now ask
torchao_quantize_importable() for the torchao-backed encoder modes. Layerwise
fp8 is a plain dtype cast and does not need it.
The video staged plan sent the precision but not the memory mode, and the
route refuses the incompatible pair only when it can see both -- so the plan
succeeded and tens of GB were staged before /video/load rejected the same
pick. It reads the memory mode through a live ref, like the precision.
---------
Co-authored-by: Daniel Han <moonshotaisubstack@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Daniel Han
|
dc49ff6b56
|
Studio: stop MTP forcing llama-server to a single parallel slot (#8172)
* Studio: stop MTP forcing llama-server to a single parallel slot #7717 clamped --parallel to 1 whenever MTP resolved, six days after #7455 raised the default slot count from 1 to 4. Anyone serving an MTP GGUF went from four concurrent slots to one, with a logger.warning as the only signal. Measured on b10310 with Qwen3.5-9B-MTP-GGUF Q4_K_M, 8 concurrent requests: four slots with MTP is 1.97x the batch throughput of one and 1.29x four slots without MTP, so the acceptance collapse the clamp was avoiding no longer outweighs the concurrency it cost. Removes both clamp sites and the two restore paths that existed only to undo them. The --kv-unified and diffusion downgrades stay: those are real limits, and they now carry an allow-slot-clamp marker. A new stdlib-ast lint plus a launch-argv test matrix keep a silent downgrade from coming back. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten the comments in the parallel-slot changes * Catch the annotated spelling of a slot clamp `n_parallel: int = 1` in a function body is an ast.AnnAssign, which the lint never visited, so the exact regression it exists to block could be reintroduced past a green gate. Handle both nodes, with self-test and pytest cases for the annotated form, its allow-slot-clamp opt-out, and a non-literal annotated assignment. * Record llama.cpp#26031 next to the slot decision It reports concurrent MTP garbling on hybrid architectures. It did not reproduce on qwen35 or qwen35moe here, so the slots stay, but a reader weighing this code should find the open report from it. * Re-price the MTP reserve for each slot-fit candidate The reserve is not slot-independent after all: compact SWA scales its window allowance by the slot count under kv_unified, and an MLA target with recurrent KDA layers charges per slot. _base_footprint carried it at the originally requested count, so every reduced candidate was over-charged and one that really fits could be rejected, leaving the load on --fit with layers offloaded to host. _slots_that_fit_on_gpu now takes mtp_bytes_for_slots and re-prices the reserve alongside the KV and compute buffer; the base footprint drops it. Passing None keeps the non-MTP path byte-identical. Also catch a clamp spelled as an expression: min(n_parallel, 1) and a conditional with a literal 1 branch both pinned the count past a green gate. A real bound like min(n, cap) and a conditional between two live counts stay clean. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Price the MTP reserve at the candidate micro-batch, and close two lint gaps The slot re-pricing added last commit only overrode the slot count. A reduced candidate also lowers the batch floor, so its micro-batch shrinks, and compact SWA adds one micro-batch to its window allowance. The reserve kept the ubatch captured for the original request, so it was still over-priced. The callback now takes (slots, ubatch) and the fit passes the candidate pair. The lint missed two shapes it should have caught from the start. Tuple targets were only tested at the outer ast.Tuple, so `gpu_indices, use_fit, n_parallel = gi, False, 1` -- the very style load_model uses for the VRAM fit -- passed a green gate. Targets are flattened and paired positionally with a tuple value now; an unpairable right-hand side says nothing and is still skipped. And the route resolves the request into `_n_parallel` before the load paths see it, so that alias counts as a slot variable too. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Track the server-wide slot alias in the clamp lint A request that names no count resolves to llama_parallel_slots, so a clamp on that name reduces the same user-facing slots the rule exists to protect. The existing bindings are keyword arguments rather than assignments, so the tree stays clean; self-test and pytest cover both the clamp and the handoff. * Tighten the comments added after the opening pass --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
da72577146
|
Deduplicate the scan-packages baseline, and gate it with tests (#8110)
#8135 re-baselined the huggingface_hub 1.x backoff loop, so that half of this branch is done upstream. What is left is a defect it did not touch: the shipped baseline carries five openai entries twice. openai/_base_client.py C2 polling/beaconing loop openai/auth/_workload.py IMDS + network calls openai/resources/beta/responses/responses.py C2 polling/beaconing loop openai/resources/realtime/realtime.py C2 polling/beaconing loop openai/resources/responses/responses.py C2 polling/beaconing loop 223 entries, 218 distinct. A duplicated suppression is not harmless: whoever removes one copy to un-suppress a finding still gets no finding, because the second copy is still matching. The 111 tests here cover the baseline's shape and matching semantics, including the duplicate check that caught this. Every one passes against main once the five copies are gone. Co-authored-by: danielhanchen <unslothshared@gmail.com> |
||
|
Daniel Han
|
4e193139d1
|
fix(security): re-baseline the huggingface-hub HTTP backoff loop for 1.x (#8135)
All three pip scan-packages shards fail on the same single unsuppressed
CRITICAL:
[1] CRITICAL C2 polling/beaconing loop detected
Package: huggingface-hub
File: huggingface_hub/utils/_http.py
Evidence: L461: while True: sha256:0c9641548adea74be4a8b0e86e8b75cc937a2b0f833b57dfa3d30d3e24d2537a
Each shard reports 1 CRITICAL and 0 HIGH, so this one finding is what turns
SCAN_ENFORCE=1 into exit 1 on studio, hf-stack and extras alike.
The finding is the retry-with-exponential-backoff loop RE_C2_POLLING is meant
to catch in malware, matching here on a legitimate library. It was already
reviewed and baselined twice, at L462 and L298, against huggingface-hub 0.x.
huggingface-hub has since released 1.x, which moved from requests to httpx and
refactored http_backoff into a shared _http_backoff_impl. The greedy DOTALL
span the check records therefore covers different code, its digest changed, and
the baseline correctly reopened the finding for review rather than suppressing
changed code under a coarse key.
Reviewed the new code in 1.27.0. huggingface_hub/utils/_http.py:461 is
_http_backoff_impl, "Internal implementation of HTTP backoff logic shared
between http_backoff and http_stream_backoff": it retries an httpx request on
configured exception types and status codes, sleeps with exponential backoff
bounded by max_wait_time, and honours a 429 rate-limit reset. No beaconing, no
remote-controlled interval, no payload execution. Benign, same as the two
entries it succeeds.
Adds the 1.x entry rather than replacing the 0.x ones, matching how this file
already carries several entries per file across versions (hf_api.py has three),
so a pinned older huggingface-hub stays covered.
Verified: scan_packages.py huggingface-hub goes from
"Summary: 1 CRITICAL, 6 MEDIUM" (3 suppressed) to
"Summary: 6 MEDIUM" (4 suppressed), exit 0.
|
||
Michael Han
|
6712c7513d
|
Uninstall: remove WebView runtime data so reinstalls don't serve a stale frontend (#7360)
* Uninstall: remove WebView runtime data (stale frontend after reinstall) The desktop app's WebView creates runtime data keyed by the Tauri bundle id (ai.unsloth.studio) at first launch, not at install time, so the uninstallers never removed it. A leftover WebKit/WebView2 cache then serves a stale frontend bundle to the next install, showing old styles after a supposedly clean reinstall. uninstall.sh: remove ~/Library Caches/WebKit/Application Support/ HTTPStorages/cookies/saved state/prefs on macOS and the XDG cache/data/ config/state dirs on Linux; stop the desktop app binary first. uninstall.ps1: remove LOCALAPPDATA (EBWebView profile) and APPDATA dirs for the bundle id; stop the desktop app and any msedgewebview2.exe helper holding handles on them first. Adds tests/sh/test_uninstall_webview_data.sh running the full script against a fixture HOME for both OS branches. * Test: never enter the real WSL cleanup from the webview-data test Stubbing uname alone is not enough on a WSL host: the script's 'grep -qi microsoft /proc/version' probe still fires and the Linux test cases would run the real WSL cleanup against the host's /mnt/* shortcuts and /etc profile. Add a PATH-stubbed grep that fails only the /proc/version probe (delegating everything else to the real grep via an absolute path so the stub can never self-exec), plus no-op powershell.exe and sudo stubs as defense in depth. * Test: sandbox XDG_RUNTIME_DIR and drop the GNU-only mktemp -p * Tighten the comments added for PR #7360 * Final comment pass for PR #7360 * Ignore relative XDG home overrides when removing WebView data * Final comment pass for PR #7360 * Scope the Studio process stop to the caller's session for PR #7360 * Scope the Studio app kill to the target user for PR #7360 * Scope by owning account, run defaults as the home owner, and de-vacuum the kill assertion for PR #7360 * Resolve a symlinked home and stop claiming removal succeeded for PR #7360 * Report failed removals in the PowerShell summary and across the custom-root subshell for PR #7360 * Count a deny-listed custom root as incomplete removal for PR #7360 * Remove the deep-link handler .desktop entry on uninstall * Only claim chat history is gone when a studio.db was actually removed * Refresh the desktop database in the directory the handler was removed from * Count a deny-listed custom root as incomplete cleanup in uninstall.ps1 * Harden the uninstall summary markers and the WebView2 helper sweep * Keep the uninstall summary conservative when it cannot account for a removal * Confirm the database is really gone before the summary says so * Stop claiming the uninstall removed provider API keys * Resolve a symlinked studio.db without relying on readlink -f * Scope the signed-out claim and anchor relative database links * Stop the legacy-named desktop process on Windows too * Anchor a relative install-root link before testing for the database * Tighten the comments in the WebView cleanup paths --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
ee64eec51a
|
release-desktop: add a VirusTotal pre-flight scan of the release bundles (#8089)
* release-desktop: add a VirusTotal pre-flight scan of the release bundles * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * virustotal_scan: register the signed upload URL with add-mask * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * virustotal_scan: check out the script, stop replaying single-use upload URLs, bound every request by the deadline * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * virustotal_scan: fail closed on malformed hash lookups and cap pacing by the deadline * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bound VirusTotal socket calls to the deadline and scan only validated releases - Pass a per-call socket timeout through the transport, clamped to the remaining scan deadline, so a request starting just before the deadline cannot consume the full 300s cushion ahead of the step timeout. - Retry a malformed upload acknowledgement instead of aborting, since the disclosure cost of the upload has already been paid at that point. - Move the scan after 'Create or validate versioned release' so a run that is rejected has not already uploaded all four bundles. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Defer non-draft release creation past the scan and cap retry backoff - Split 'Create or validate versioned release' into a validation step that runs before the scan and a creation step that runs after it. A dispatch with draft=false and a new tag previously published an empty release that stayed assetless for the length of the scan, and permanently so if the run was cancelled part way through. - Clamp the exponential retry backoff to the remaining deadline, so a 429 or 5xx arriving late cannot sleep past --timeout-seconds before the loop notices and writes its summary. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep release notes unconditional, fail closed on lookup errors, fix the permission test - Write desktop-release-notes.md in the validation step, which always runs. The updater metadata step reads it on every run, so leaving the write in the conditional create step broke reruns against an existing release. - Only treat a lookup as a missing release when gh reports 'release not found'. Any other failure now fails the step, rather than proceeding to disclose the bundles for a run that cannot publish. - Point test_release_desktop_permissions at the renamed validation step and assert the deferred create step and its gate. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Escape third-party text in the VirusTotal warning annotations Engine names, detection labels and API error strings are third-party data written straight into a workflow command. Actions truncates an annotation at the first newline and mis-parses a bare %, so a crafted or merely awkward detection string could drop the engine list exactly when the scan is trying to alert a maintainer. Mirrors _gha_escape in lockfile_supply_chain_audit.py, including the replace-% first ordering. * Never report an unanalysed bundle as clean, and escape the summary - A hash known to VirusTotal can have no completed analysis, in which case last_analysis_stats is absent and parse_stats yields all zeros. That row read as 'known to VirusTotal' with zero detections, which looks like 70 engines cleared a bundle that none of them scanned. Such a row now reports 'no completed analysis' with stats left unset, so it renders as dashes and cannot trip the threshold. The upload path polls until status is completed, so it only requires a stats object. - Escape third-party engine names, detection labels and error strings in the job summary. It is appended to GITHUB_STEP_SUMMARY and rendered as Markdown, so a newline ended the row and | opened a new cell. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Avoid a CodeQL clear-text-logging false positive on the skip message Interpolating API_KEY_ENV into the skip log trips CodeQL's py/clear-text-logging-sensitive-data rule at high severity, because the constant's name ends in _KEY. It only ever holds the env var name, never the value, but the repo uses CodeQL default setup so there is no config to filter the query on. Write the name out literally and pin it against the constant in test_missing_key_skips_without_failing so the two cannot drift. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
d24f0f175a
|
Security: pin the credential-send allowlist entry to the reviewed file (#8104)
* Pin the credential-send allowlist entry to the file it was reviewed against The evidence for "Harvests environment variables/secrets AND makes network calls" records the Request/urlopen calls but not the destination, so the unsloth_zoo/hf_xet_health.py entry added in the previous commit kept the same evidence hash after repointing _endpoint() at another host. Verified both ways: swapping the default endpoint and swapping the url assignment each leave the hash at 674a558c, so the entry would have gone on suppressing a credential send to an attacker-controlled host. Widening the network evidence to capture destinations would rehash and reopen around 78 unrelated entries across every network-involving check, so bind the one entry instead. Finding now carries file_sha256, and a baseline entry may pin it. Absent means unpinned and the key alone suppresses, exactly as before, so the other 216 keys are untouched. Present means the entry covers only those file bytes, so any other edit reopens the CRITICAL and forces a fresh review. --write-baseline carries a pin over from the baseline in effect, not from its own output path, which would drop pins whenever the output goes somewhere new. Only hf_xet_health.py is pinned, being the one entry here that transmits a credential. Any future change to that file re-reds the gate until someone re-reviews it, which is the right trade for a file that sends HF_TOKEN. The digest is taken from the published wheel through the scanner's own archive reader, not from a source checkout: the two differ, and a checkout digest never matches what CI scans. Tests: a pinned entry reopens on changed file bytes, an unpinned entry stays content-agnostic, --write-baseline preserves a pin, and check_py_file stamps the digest. Two existing tests move to the dict return type. 103 pass, and all three scan shards exit 0. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
6afcf22889
|
Security: baseline seven reviewed scan_packages findings (#8096)
pip scan-packages :: studio and :: hf-stack have been red on main since 2026-08-06, so every PR branched from it inherits two red checks. Seven CRITICAL/HIGH findings are no longer suppressed: five were never baselined (caio and torchao are newly resolved, two unsloth-zoo test files are new), and two unsloth-zoo entries had their evidence change under the same key. Each was read against the package source before being added: - torchao/prototype/gptq/gptq_example.py: /tmp appears only inside an argparse help string. The path feeds save_pretrained; the file's one subprocess.run invokes the lm_eval CLI with fixed arguments. - unsloth-zoo tests/test_hf_xet_fallback.py: a monkeypatch lambda whose return value contains the literal "/tmp/warm". No write, no exec. - unsloth-zoo unsloth_zoo/hf_xet_health.py: the Xet health probe. _endpoint() is HF_ENDPOINT or https://huggingface.co, and the token is only ever sent there as a Bearer header. It is optional; the endpoint answers anonymously. - unsloth-zoo tests/test_mlx_vlm_label_masks.py: __import__("threading") with a literal argument, plus mx.eval, which is MLX array evaluation. - unsloth-zoo unsloth_zoo/compiler.py: __import__(model_location, ...) guarded by try/except to check TRL internal names still import, plus the banner's chr(92) backslashes. - unsloth-zoo unsloth_zoo/mlx/loader.py: __import__ over a hardcoded patch_targets tuple of mlx_lm.tuner module names. - caio tests/test_raw_low_level.py: os.dup2 re-points a closed fd number at a fresh temp file to prove a completed one-shot io_uring Operation cannot be resubmitted. No sockets, no stdio redirection. Appended rather than regenerated. --write-baseline rewrites the whole file from whichever shard invoked it, which would drop the other two shards' entries, so the three shards were regenerated separately and only new keys merged in. The 215 existing entries are byte-identical and in their original order, which keeps the diff reviewable. |
||
Long Yixing
|
d6b1b7dcf2
|
Studio: drop the mlx-lm 0.31.3 exclusion so current mlx-vlm resolves (#7061) | ||
|
Daniel Han
|
f0b74090d9
|
Diffusion: replace the attention perf claims with re-measured numbers (#8021)
* Diffusion: replace the attention perf claims with re-measured numbers The three performance claims in diffusion_attention.py came from the stale #7021 and were measured on an older torch/torchao stack. Re-measured all of them on torch 2.12.1+cu130 / B200. One does not reproduce, one was off by a third, and the third turns out to understate the win badly. 1. "~1.18x end-to-end on B200, LPIPS ~0.004" for the cuDNN swap does not reproduce, and cannot: at Qwen-Image's 1024px shape the default SDPA dispatch ALREADY selects cuDNN, so pinning it is bitwise-identical (LPIPS 0.000000, np.array_equal True). Measured 1.02x compiled (1.599s -> 1.572s) and 0.93x eager, the per-call sdpa_kernel wrapper cost that compile folds away. The original claim is self-consistent with an older torch where the default was NOT cuDNN, since a nonzero LPIPS means the output changed, which cannot happen when you select the kernel already in use. Pinning is still right, for a better reason than a speedup: torch's dispatch is a heuristic. FLASH and EFFICIENT at that same shape run 3.9x and 9.0x slower, so pinning is insurance against the heuristic picking one of them on another card, head_dim or torch build. 2. "421 ms with the dense mask vs 19 ms, a ~22x tax" is now 296 ms vs 15 ms, ~20x. The structural claims all reproduce: FLASH refuses a dense bool mask, cuDNN silently falls back, MATH OOMs on the 75.5 GiB score matrix. 3. The trim's END TO END effect was never stated. It is 10.4x: a 121-frame 832x480 10-step HunyuanVideo-1.5 render goes 353.8s to 33.9s, medians of 3, reproduced across two independent runs. Also qualify "exact", which was too strong. No information is discarded, but the render is not bit-reproducible: a masked-to-fused SDPA swap perturbs each step at bf16 rounding scale (one DiT forward on identical inputs differs by 6.6e-3 relative at cosine 0.99998) and 10 denoising steps amplify that, so the video is visibly a different sample. That is the kernel change, not the trim. Rendering the SAME dense-mask path under two different exact SDPA kernels diverges MORE than the trim does (LPIPS 0.303 vs 0.285, SSIM 0.744 vs 0.767 over 13 sampled frames), which is the control that establishes it. Whole-video LPIPS cannot judge a kernel change at this step count; the single-forward relative error is the metric that can, and the comment now says so. Comments and docstrings only; no code change. * Name the real dense-mask fallback: cuDNN, not math The paragraph said a dense bool mask 'forces the slow math path', but the probe this same commit cites shows MATH OOMing on the 75.5 GiB score matrix while the measured dense attention completes in 296 ms. Both cannot be true. Settled by output identity rather than timing, since dispatch overhead makes timings ambiguous on their own. At the production shape (B=1 H=16 N=50345 D=128 bf16) on torch 2.12 / B200: MATH OOM FLASH refuses a non-null mask EFFICIENT runs, 168.25 ms, differs from default by 2.44e-4 CUDNN runs, 296.00 ms, BITWISE-equal to the default's output Default dense measures 296.11 ms, so cuDNN is what the dispatcher picks, on a masked path 20x slower than its own unmasked one. Recorded the EFFICIENT number too: the dispatcher's masked choice is not the fastest available, which is worth knowing before anyone optimises this path further. * Make the cited probe actually reproduce the backend-identity claim The comment concludes cuDNN is what the dispatcher picks under a dense mask, says explicitly that timings cannot establish that, and then cites scripts/sdpa_mask_backend_probe.py as the reproducer. The probe only times each backend. It never compared outputs, so it could not reach the conclusion it was cited for. It now captures the default dispatch's dense output once and reports, per backend, whether the forced result is bitwise-identical to it. Exactly one backend answering yes names the kernel the dispatcher chose. A backend that cannot run the dense mask reports OOM or unsupported instead, so the column never reads as a mismatch when nothing ran. Output on torch 2.12 / B200: backend mask=dense(ms) mask=None(ms) ==default(dense) default(dispatch) 295.98 14.84 yes MATH OOM OOM OOM FLASH UNSUPPORTED (RuntimeError) 51.46 unsupported EFFICIENT 168.21 123.12 no (2.4e-04) CUDNN 295.96 14.77 yes This makes the PR no longer comment-only. The change is confined to a standalone developer probe that nothing imports, and it exists to make a claim in the shipped comment verifiable rather than asserted. |
||
Eyera
|
b52d3b56ed
|
feat(studio): rework train page setup flow (#7633)
* feat(studio): rework training setup and cache handling * Rework train page resource selection * Improve train page workflows and resource selection * Fix train page picker behavior * Fix stale training errors after token updates * Fix training eval steps regression * Fix shared token use when resuming training * Harden train page resource workflows Unify model and dataset picker behavior, training start guards, and token handling. Validate cache provenance for local models, downloaded resources, and processed datasets. Prevent stale preview and history state while keeping the train page modular. * Fix training method race and test isolation * Restore training upload limit exports * Fix training config formatting contract * fix(studio): harden training model selection and preflight * fix training resume, cache, and lifecycle reliability * Fix training stop watchdog path stub * Fix training start state and train page UI * Unify train page selection controls * Fix train page validation, localization, and paging * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Clean up train locales and restore recipes link * fix(studio): correct training model selection and preflight Preserve freeform local model paths, reject remote GGUF-only repositories, and correct the training start snapshot contract. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): harden training resource selection Preserve local dataset path intent, localize setup-change errors, and remove unused picker tour hooks. Refresh model picker and PDF recipe contracts for the refactored training flow. * fix(studio): remove train tab scrolling * fix(studio): harden training resource preflight Reject missing local models and binary adapter artifacts. Preserve selected model and dataset cache pins through preflight and QLoRA loading. Detect cache path changes before training starts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix train page validation and cache handling Keep transient model checks nonblocking and bind cached models and datasets to their selected snapshots. Improve Hub auth errors, retries, task search, streaming consistency, locale formatting, reset handling, payload mapping, partial inventory filtering, theme accents, and regression coverage. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): stabilize train page pickers Unify Hub validation, local inventory, picker styling, and dataset names. Localize training feedback and explain streaming modality changes. Add unit, contract, and cross-browser picker coverage. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): align cached scans and variant state purge Scan selected cached snapshots during training security preflight. Purge manifests and cancel markers using their stored GGUF variant. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): harden train cache, consent, and UI Fix cache cleanup and cross-platform model path handling. Align resume and adapter consent scans with pinned load targets. Localize dataset flows and refine navigation, state cleanup, and theming. * fix(studio): harden training resource selection Preserve cached model configuration HTTP errors so stale cache references return their intended 404 responses. Wait for device inventory settlement before locking inferred picker tabs, while keeping known device items visible during scans and retries. Apply modality name heuristics only to the final model path component across supported platforms. Expose full model and dataset identities on truncated picker triggers. Add regression coverage for picker settlement, retry behavior, and modality inference. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix train picker flow and resume safety Scan the exact cached model snapshot before resume consent and preserve the actual repository identity. Keep cold pickers on Device until inventory settles, cap automatic Hub pagination, and provide a stable Load more action. Improve train control and history card accessibility, remove duplicate token labeling and dead styling, and correct picker spacing. Normalize Windows relative model paths and add coverage for resume pins, picker policy, pagination, path identity, and accessibility contracts. Fix the reported formatting and import ordering issues. * Fix cross-platform train dataset path detection Use the shared local path detector so persisted Windows, Unix, UNC, and home-relative dataset references do not render Hugging Face-only controls. Recognize Arrow dataset files and add regression coverage for supported local path formats and Hub repository identifiers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix training start recovery and dataset selection Reconcile failed fresh and resumed training starts with backend status before reporting an error. Treat the selected dataset source as authoritative so filename-like Hub repositories retain subset and split controls. Align dataset picker sorting and remove unused advisor template state with a persisted-state migration. Add regression coverage for transport recovery and Hub dataset selection. * Fix train picker validation and start recovery Update the resume training contract test for the asynchronous failure handler. Filter invalid Hugging Face search results and block invalid saved model and dataset selections. Restrict local dataset selection to inventoried application paths. Load training YAML through a bounded native file picker while keeping the browser fallback. Only recover uncertain training starts after network or response parsing failures, and match the active job to its request identifier. Treat cached full precision models as ready for QLoRA without showing a false download warning. Add regression coverage for the picker, config import, and training recovery behavior. * Harden training starts and config imports Track training start request IDs from preflight through pending, accepted, and rejected states so retries remain idempotent. Disable automatic retries for start posts and reconcile ambiguous client responses with server request status. Clear terminal training status on reset so later runs do not inherit stale state. Enforce the 1 MiB YAML limit for browser imports and keep read errors specific to the selected file label. Remove unused dataset picker translation entries across all locales. Add focused tests for request idempotency, start recovery, config limits, and picker contracts. * Harden train picker feedback and start handling Show explicit empty states for device dataset searches and invalid Hub model or dataset queries while keeping pagination sentinels mounted. Run the train picker Playwright suite across Chromium, Firefox, and WebKit in Windows CI. Keep training start outcomes owned by backend spawn completion so cancelled handlers cannot mark active jobs as rejected. Correct the Train section heading hierarchy and format the branch-owned native file dialog test. Add picker contracts, browser assertions, and cancellation race coverage for the updated behavior. * Fix training setup races and configuration state Keep hardware-based method selection within model-default loading so starts cannot race changing defaults. Clear start errors only after deliberate configuration or token edits while background reconciliation stays silent. Generate training request IDs safely for LAN HTTP access before acquiring the runtime lease. Filter persisted state, sanitize invalid methods, and guard preview metadata. Localize config size failures, use the toast wrapper, and remove obsolete training files. Add regression coverage for the updated start, persistence, and import behavior. * Refine training configuration and picker workflows Split the training configuration store and parameter panel into focused persistence, policy, LoRA, hyperparameter, memory, and MLX modules. Run picker coverage on platform-appropriate browser engines and parameterize the unmanaged dataset path. Surface configuration changes when streaming is disabled and clear stale dataset modality state after failed probes. Consolidate dataset format and AI-assisted mapping on hub routes with header-based Hugging Face tokens. Hold local path actions until model inventory loading settles. * Fix train picker state and preflight regressions Open online pickers on the Hub before inventory settles and keep the inferred tab stable for the session. Preserve parameter and VRAM metadata for pinned selected models. Reject failed backend starts once and cancel automatic modality corrections without showing a destructive setup error. Update persistence contracts to follow the extracted training configuration module. Move shared parameter option styling out of the React component module to preserve Fast Refresh. Add regression coverage for picker tab behavior, pinned model metadata, persistence contracts, and start rejection handling. * fix(train): harden model selection and start feedback Preserve cache metadata and model capabilities when freeform local paths resolve to discovered models. Return stable training start error codes through direct and recovered requests, then localize Hugging Face preflight failures across every supported locale. Show whether submitted advanced settings are default or non-default in Simple mode, and migrate the parameter mode preference to the standard storage key. Restore absent-key migration semantics and update the embedding security test for the extracted gate helper. Add focused coverage for structured errors, advanced setting summaries, start recovery, and security gates. * fix(train): harden picker validation and focus Align Hub ID validation with Hugging Face rules while accepting valid underscore boundaries. Reject persisted path-shaped datasets from Hugging Face-specific controls. Reuse model identity normalization for cache path comparisons. Move picker search focus into the popover focus lifecycle. Update source contract tests for extracted model selection and structured resume errors. * fix(train): refine picker behavior and dataset structure Wait for device inventory before locking inferred picker tabs while preserving explicit user choices. Align Hub resource ID validation with backend rules for leading and trailing underscores. Support ArrowDown navigation from picker tabs into available options. Split dataset selection, uploads, streaming settings, and inventory refresh logic into focused modules. * fix(studio): stabilize train picker contracts and state Update dataset contracts to follow the extracted selection, upload, and inventory modules. Keep the inferred picker tab stable while device inventory settles. Use the shared TrainingMethod type for VRAM estimation and remove redundant casts. * fix(train): align Hugging Face repo validation Allow underscores at repository segment boundaries to match Hugging Face repo ID rules. Keep model and dataset search queries unvalidated while preserving selection safeguards. Add regression coverage for accepted IDs and unrestricted picker searches. * Fix training resource compatibility and cache safety Restore legacy dataset format and mapping routes while moving training uploads and checks to the canonical Hub endpoints. Enforce configured upload limits, clean partial files after failures or cancellation, and support both upload route families in middleware. Align Hugging Face repository validation with accepted underscore boundaries and skip PyTorch dependent tests when the dependency is unavailable. Keep model format and cache metadata correct when the same repository is selected from another source. Only promote complete runnable model weights as cached training resources, align download notices with picker capabilities, and reject weightless local snapshots during start preflight. Cancel stale dataset mapping assistant requests before they can overwrite a newer selection. Preserve touch selection for training methods by separating tooltip toggling from Select item activation. Add focused backend, frontend, and picker regression coverage for these paths. * fix(studio): finish train page and onboarding rework Ignore cache-only fields when comparing training start inputs so reconciliation does not cancel valid runs. Route native dataset drops through Tauri path validation and lease-backed managed uploads. Use the production model and dataset selectors in onboarding and replace placeholder uploads with real imports. Require positive learning rates while keeping learning rate, epoch, and step inputs editable. Normalize model identities, improve dataset summary order, and guard persisted selection state. Recognize Windows rooted and drive-relative paths consistently in picker logic. Add MLX optimizer help and learning rate validation messages to every supported locale. Add frontend, backend, and native policy regression coverage for the updated behavior. * chore(ci): remove train picker smoke workflow changes Remove the train picker Playwright step from the macOS Studio smoke checks. Remove the train picker Playwright step from the Linux Studio smoke checks. Remove the train picker Playwright step from the Windows Studio smoke checks. Drop the matching artifact upload paths so the workflow files match upstream main. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): harden train picker behavior and boundaries Expose dataset drop helpers through the training public API and move local path detection into shared code. Block freeform Hub choices while offline for pointer and keyboard selection. Make native dataset drop handling safe for committed React renders. Keep recipe navigation working when session storage is unavailable. Preserve Windows drive root identities and cover the distinction from drive relative paths. Update picker and training input contracts to match the extracted implementations. Remove orphaned translations and align the train style block with the existing indentation. * Fix train picker cache and upload behavior Keep selected model cache paths strict and preserve fallback discovery only when no path is supplied. Offload multipart writes and native dataset copies from the backend event loop. Prewarm train picker inventories so device-first tab inference can settle before the first open. Use shared picker tab constants across the model and dataset selectors. Update picker contracts and add regression coverage for cache resolution and dataset uploads. * Refine train picker controls and component structure Split picker retry, error, and pagination UI into focused components. Add a shared segmented control and use native radio semantics for dataset source selection. Centralize segmented indicator positioning with RTL-aware transforms. Separate dataset advanced settings state wiring from its presentation and source toggle. Make picker arrow navigation move symmetrically from tabs through search and options. * Fix train page reconciliation and picker contracts Preserve tuned training parameters when model cache references change, while refreshing defaults when no user edits occurred. Keep streaming modality corrections visible through the Start CTA after preflight stops. Localize the new training parameter and dataset mapping text across all supported locales. Refresh picker structural contracts after the component split and apply Biome formatting to the changed files. * Fix train upload tests and desktop drop handling Pass an explicit empty native path lease in legacy upload limit tests so direct route calls match FastAPI request behavior. Track Tauri scale factor changes during dataset drag and drop, clean up both listeners safely, and cover runtime updates. Replace the run preview card's important border utility with an explicit dark theme card modifier. * fix(studio): harden training setup and history state Restore Hugging Face token entry and validation in the onboarding model and dataset steps. Apply the configured upload size preflight before onboarding dataset uploads. Update moved model and dataset cache references without showing false missing cache warnings. Parse SSE frame delimiters by their matched length so mixed line endings preserve event data. Match sensitive pid paths by exact segment while retaining database state protections. Clear deleted Data Recipe selections after inventory settlement while preserving direct uploads. Show the files deleted status only when a run previously recorded an output directory. Handle Data Recipe navigation failures and keep the native drop listener stable during uploads. Replace formatting-specific contract checks and add focused behavioral regression coverage. * Harden training start and model picker flows Split the train model picker into its own entry point to keep it out of shared route bundles. Preserve pending idempotent start reservations and reconcile them before accepting a run. Keep unconfirmed starts in a polling state with accurate localized feedback. Continue starts after automatically disabling unsupported multimodal dataset streaming. Localize the updated model picker controls across supported languages. Lock dataset uploads before asynchronous preflight to prevent concurrent selections. Add regression coverage for reservation states and unconfirmed runtime starts. * fix(studio): harden training picker edge cases Preserve POSIX model path identity while retaining Windows relative path normalization, and clarify the override migration behavior. Localize the onboarding model and dataset controls across every supported locale. Truncate native dataset filenames by Unicode code point so generated labels cannot contain lone surrogates. Cancel training after preflight disables streaming for image or audio datasets so users can review the changed setting before restarting. Update runtime contract assertions for the setStartPending rename and add regression coverage for path identity and Unicode truncation. * fix(studio): align training picker controls Restore full-height Browse and Amazon S3 selection with an accessible radiogroup that avoids fieldset sizing behavior. Align model and dataset picker tabs with the shared 36 px segmented control geometry and typography. Remove active borders and focus rings from picker search inputs while preserving keyboard focus indicators on other controls. Add contract coverage for segmented control sizing and picker search focus styling. * fix(studio): harden onboarding picker flows Classify native dataset drops from the full path before shortening display names. Keep onboarding model choices within the selected training type across Hub, device, and freeform selections. Retry model default loading when onboarding restores an interrupted model selection. Localize the Hugging Face token field across supported languages. Reset picker result scrolling when switching between Device and Hugging Face tabs. Add regression coverage for native filenames, model constraints, hydration, and picker scrolling. * fix(studio): reconcile multimodal streaming before training Disable streaming when dataset checks detect image or audio data, then recheck cached selections in non-streaming mode. Keep start-time modality detection as a safe fallback that updates the configuration and continues without an error or a second click. Build onboarding training method options from the shared method order and metadata so onboarding stays aligned with the train page. Add contract coverage for the streaming reconciliation and shared training method list. * Fix train picker contracts and dataset recovery Allow local models without reliable modality metadata to pass onboarding constraints while retaining explicit mismatch checks. Reconcile GPU selection without effect-driven state updates and mark intentional deep Hub imports for lint. Return and consume stable local dataset cache miss codes, and send the canonical train_split field. Move train-specific picker code into its owning feature and update the related contract coverage. Use the canonical Hub dataset progress route from Chat. * Harden training resource selection and offline handling Return an actionable preflight error when offline mode is enabled and the selected model is not cached. Validate evaluation dataset uploads with the shared extension policy and reuse the centralized accept list. Disable streaming for explicit on-device dataset selections while preserving Hugging Face selection intent. Keep dataset identifiers safety checked without blocking benign values before Hugging Face handles repository validation. Remove the branch-added source assertions and cover the new selection policy with behavioral tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix training setup validation and async state races Validate Hugging Face dataset IDs and repository access before training starts while keeping local paths on their filesystem validation flow. Scope model default race protection to fields that defaults can overwrite so unrelated dataset edits no longer suppress model initialization. Stop Hugging Face token edits from invalidating token-independent device inventory scans on every keystroke. Add regression coverage for dataset source validation, cached and remote Hub preflight, model default application, and token inventory behavior. * Tidy up train page layout, tooltips and dark mode Layout - Move the Browse / Amazon S3 toggle into the Dataset section header - Put Subset, Train Split and Evaluation Split on one row - Put Target Format, Train Split Start and Train Split End on one row - Pair Project Name with Max Steps, and Context Length with Learning Rate - Centre the hyperparameter tabs and give them a fixed width so the sliding indicator lines up with its segment - Give parameter rows a min height so slider and select rows share one vertical rhythm - Slightly more spacing below section headings and at card bottoms Tooltips - Add hints for Model, Method, Dataset and Project Name - Move field descriptions into the tooltips and drop the inline copy - Upload field keeps the accepted file types inline, with size limit and Learning Recipes note in the tooltip HF token - Show a masked preview of a saved token instead of a generic label - Read Not set when no token is stored Dark mode - Drop borders on the LoRA option cards and target module chips, and separate states with background fill instead - Use a lighter, less saturated green for the selected state Other - Expand LoRA Settings and Training Hyperparameters when Advanced opens - Shorten the upload label to Drop file or click to upload - Localise the new wizard tooltips across all locales * Fix training defaults, MLX validation, and dataset preflight Prevent late hardware recommendations from replacing configuration values loaded after model selection. Block unsupported MLX methods and embedding runs in the UI and backend before training is queued. Require live Hub access for streaming dataset preflight instead of accepting unrelated cached data. Compare advanced settings against applied model defaults and cover the updated behavior with focused tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix training picker validation and platform safeguards Update the Tauri YAML contract test to verify save filter behavior without depending on the Rust vector implementation. Align frontend Hub resource validation and training provenance normalization with the shared repository ID rules, including valid underscore and maximum length identifiers. Reuse one frontend training method policy so Apple Silicon onboarding and the Train page consistently disable CPT and prevent unsupported onboarding completion. Add regression coverage for rejected Hub IDs and exact resume provenance. * Fix MLX training capability validation Detect hardware before platform validation so unsupported MLX configurations fail before a job is queued. Reject audio dataset training during server preflight while preserving the worker guard. Share Apple Silicon capability checks across model selection, onboarding, readiness, submission, and LoRA controls. Cover audio, CPT, embedding, LoftQ, DoRA, and warm-start detection with regression tests. * Fix training quit protection, translations, and preference reset Protect desktop quits while a training start is pending or a run is active, and keep the warning text accurate. Localize dataset upload details across every supported locale and require those keys in parity checks. Centralize the training picker and parameter mode storage keys so resetting local preferences clears current and legacy values. Add regression coverage for training activity transitions and preference reset ownership. * Preserve multimodal training support Keep upstream training eligibility authoritative while preserving separate vision, audio, and embedding capabilities. Classify dual audio and vision models for vision training without changing inference support or enabling pure audio training on MLX. Add coverage for model type constraints, backend type resolution, and MLX validation. Apply the pending Biome formatting fixes to the dataset selector and training wizard. * Fix training defaults retry and clean stale translations Allow model defaults retries after failed cache reconciliation without overwriting user-edited settings. Ignore stale fallback vision checks once a newer defaults request starts. Remove unused parameter description keys from every locale and cover the retry behavior in the training contract tests. * Document direct model identity imports Explain why the model identity helpers bypass the Hub barrel beside each lint suppression. * Preserve training settings across reloads Persist applied model defaults and their advanced settings baseline while refreshing transient model metadata without overwriting tuned hyperparameters. Keep remote format probe tests runnable without an installed huggingface_hub package by supplying a fake module. Normalize tilde-prefixed path separators while preserving case-sensitive identities. Remove unmatched model and dataset tour anchors. Add regression coverage for persistence migration and model identity normalization. * Harden train page validation and persistence Validate CPT embedding learning rates against backend bounds and add localized client feedback. Keep invalid learning rate drafts out of persisted state and YAML exports. Notify users when multimodal model selection restores a non-S3 dataset source. Remove W&B tokens from training config persistence and migrate stored secrets safely. Update the model defaults contract test to cover metadata refresh without requiring the removed early return. Move segmented control styles into a non-component module to preserve Fast Refresh behavior. Add validation and persistence coverage for the new contracts. * Fix training model IDs, cache state, and localization Preserve root-level Hugging Face model IDs instead of rewriting them into the Unsloth namespace. Debounce token-scoped inventory reconciliation so token and inventory updates use stable request keys. Keep the selected training history run when the Studio view remounts. Localize dataset subset and split selectors across every supported locale. Complete Italian picker and training translations and enforce Studio-wide locale parity. * Fix training history errors and stale stop state Return structured artifact deletion errors and show localized messages that distinguish active training output from filesystem failures while keeping history intact. Clear superseded stop requests only when they belong to the current runtime generation, and cover start invalidation and stale stop handling with regression tests. Clarify that shared picker styles apply outside the Hub while Hub-only rules remain scoped. * Fix training method persistence and settings summaries Persist training method provenance so manual learning rates, model adapter rates, and pre-CPT dataset formats survive reloads and method changes. Deduplicate imported target modules and compare advanced-setting arrays by value counts so duplicate entries cannot hide non-default settings. Add migration and regression coverage for legacy state, rehydrated method transitions, restored learning rates, and duplicate target modules. * Fix persisted completion defaults and inventory retries Persist trainOnCompletions across reloads so model defaults and advanced settings summaries remain accurate. Defer missing legacy values until model metadata loads, preserving tuned settings while respecting streaming, raw text, CPT, embedding models, and explicit user changes. Settle partial inventory failures when usable rows exist so manual local model paths and Enter submission remain available. Add retry actions for partially failed model and dataset scans, including when filtering leaves no visible results. Add regression coverage for persistence migration, constrained completion defaults, inventory settlement, and picker empty states. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix training validation and route state regressions Retry newly detected image datasets with vision-aware validation, clear stale dataset check failures, and cover the failed background classification case. Restore Train history cleanup and centralize the learning recipes navigation intent key. Repair backend route stubs for canonical model IDs and path normalization, and isolate consent tests from shared package import state. * Preserve training cache pins during preflight Keep model and dataset cache flags and local paths in the pending start comparison so reconciliation cannot submit a stale snapshot. Cover changed cached copies and uncached transitions with focused regression assertions. * Preserve Hub identity for cached training exports Keep exact cached snapshots for training while restoring the standard Hub repository identity before PEFT saves. Recover repository identity in memory for legacy adapters so imatrix exports work without rewriting adapter configs. Cover repository mismatches, local models, and Windows cache paths with focused regressions. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix cached training, evaluation, and MLX resume behavior Limit cached dataset slice loads so small training ranges do not materialize full duplicate caches. Restore Hub model identity for cached Torch and MLX runs so saved adapters remain portable. Attest MLX model provenance across runtime and prequantized 4-bit formats so valid checkpoints can resume. Recognize missing cached eval splits, reload train and eval together, treat explicit eval split errors as fatal, and use deterministic held-out data for automatic fallback. Retain non-fatal evaluation warnings in training status and render them in the train UI. Apply pin and model format validation to every MLX worker entry point and keep the worker test fixture aligned with its imports. * Stabilize training progress updates across job handoffs Keep active and pending training ownership separate so status polling cannot switch identities during a running job. Scope metrics, progress streams, stop, and reset operations to the expected job and recheck ownership during handoffs. Make polling, stream parsing, and start reconciliation abortable, monotonic, and safe against stale overlapping requests. Preserve same-run interface state so live updates do not remount progress controls or dismiss the stop confirmation. Add backend and frontend regression coverage for ownership changes, stale events, request races, and stream cleanup. * Fix cached training fallback and dataset reconciliation Recognize incomplete SentencePiece tokenizer caches and retry online model loads through the Hub while preserving strict offline and resume pins. Prevent rejected dataset snapshots from being promoted again until the selection or inventory changes. Forward cancellation signals to dataset format requests so superseded checks stop at the transport layer. Add focused backend and frontend regression coverage for these cache paths. * Harden offline training and cache fallback behavior Validate cached model snapshots for tokenizer and processor support, then fall back to the pinned Hub revision when the cache is incomplete. Remember rejected dataset cache entries and cancel superseded checks so metadata-only snapshots cannot trigger request loops. Reject deleted local datasets before any Hub access and clear only the matching stale selection in background, preview, and start flows. Reserve training starts before validation so overlapping requests and GPU consumers cannot race the spawn window. Keep GGUF fallback metadata tied to the cache that supplied variants when Hub access fails. Filter DSpark and DFlash drafter companions across picker, loading, inventory, and deletion paths while preserving MTP behavior. * Harden training setup and picker behavior Keep training starts blocked while a stop request is pending and preserve the stop latch through backend failures. Skip malformed training progress events while validating SSE payloads and releasing stream readers safely. Enforce non-streaming state for upload and S3 datasets across source changes, persistence, and rehydration. Route Markdown dataset drops to Data Recipes across browser and native path formats. Give model and dataset source tablists localized accessible names. Refresh training start contracts and add focused regression coverage for the corrected behavior. * Fix offline dataset cache selection and token status Prefer finalized processed dataset caches for offline training while keeping raw cache paths available for scoped management. Treat malformed Hugging Face tokens as unset in Train and surface local validation feedback without unnecessary requests. Add regression coverage for cache coexistence, interrupted cache builds, path propagation, and malformed token presentation. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Preserve sidecar Hub version during dataset cache checks Keep processed dataset cache discovery free of eager datasets imports so training does not preload the base Hugging Face Hub package. Preserve cache root discovery through HF_DATASETS_CACHE, HF_HOME, and XDG_CACHE_HOME before Transformers sidecar activation. * Add the AGPL-3.0 header to the two new studio training contract tests * Scan the fallback target and pin cached snapshots that hold weights Cache fallback scanned the pinned snapshot it was about to discard, then loaded the Hub repo unscanned. Scan after the pin is dropped instead, in all four training paths. Preflight probed the literal model name, so the registry bicodec alias Spark-TTS-0.5B/LLM 404'd and rejected a supported model. Probe the repo the trainer downloads and treat its load subdir as the weight root. An unpinned start still downloads its dataset, so a stray cached copy no longer skips Hub verification. Offline keeps accepting the cache. refs/main can point at a metadata-only revision, so the model pin now prefers a snapshot that carries weights before falling back. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Let the capability-cache guard see through the key type alias model_config.py names the cache key _CapabilityCacheKey, so matching the literal Dict[Tuple no longer finds it and the guard fails on a cache that is still correctly tuple-keyed. Resolve module-level aliases first. Checked by mutation: reverting a cache to Dict[str, ...] is still caught. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: point the extra-UI tour anchor check at studio-model-picker The train page rework renamed the model tour anchor from studio-model to studio-model-picker and updated tour/steps/base-model.tsx accordingly, but tests/studio/playwright_extra_ui.py still looked for the old anchor, so the Chat UI Tests job failed with "[data-tour='studio-model'] not found". Verified against two live Studio instances: the anchor is studio-model on main and studio-model-picker at this head; studio-dataset and studio-params are unchanged on both. * Studio: require metadata and weights together when pinning a model snapshot Pass 1 of _resolve_model_snapshot matched on weights alone, so two cases still selected a snapshot that /training/start then rejects with "does not contain trainable weights": - a newer weights-only fetch (interrupted download, or an allow_patterns pull that never took config.json) displaced an older complete sibling. That is a regression against the previous metadata-first ordering, which started the run; reproduced with a two-snapshot cache where the complete one is older. - consolidated.safetensors counted as weights for selection but is absent from _MODEL_WEIGHT_CANDIDATES in routes/training.py, and transformers 4.57.6 has no loader path for it (zero references in the package), so a config plus consolidated snapshot won over a loadable sibling. Pass 1 now demands metadata AND weights via a required_groups argument on latest_snapshot_from_cache_path, and consolidated.safetensors is dropped from the selection tuple so it matches what the route accepts. Pass 2 keeps the metadata-only fallback unchanged, so caches that never held weights resolve exactly as before. Both new tests in test_model_cache_snapshot.py fail without this change and pass with it; 401 tests across the cache, preflight, provenance and identity suites pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the new cache tests pass on Windows Four of the PR's new tests fail on windows-latest while passing on ubuntu and macos. All four are defects in the tests, not in the production code. The three blob-symlink tests hardcode a POSIX relative target ("../../blobs/x"). Windows stores the reparse-point substitute name verbatim and resolves it in the object namespace, where / is not a separator, so the link is created but dangles: Path.resolve(strict=True) raises and the provenance and dataset-cache helpers correctly return None. huggingface_hub builds these targets with os.path.relpath (file_download.py _create_symlink), which yields ..\..\blobs\x on Windows, so a real cache never has this shape and no Windows user is affected. Building the target with os.path.relpath / os.path.join matches what huggingface_hub writes. The cross-snapshot rejection tests had the same POSIX targets, so on Windows they were passing for the wrong reason (dangling link rather than the escape check). They now use native separators too, so the rejection logic is actually exercised there. test_runtime_4bit_resume_reaches_worker_with_source_resource_pins compared model_local_path against str(Path); the route posix-normalizes that field via normalize_path, so the assertion now uses Path.as_posix(), which is a no-op on POSIX. 324 tests across the three files pass on Linux. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: bound the cached dataset re-check and stop adopting unconfirmed starts Two defects introduced by this PR's new modules. 1. Unbounded dataset re-check (#7853). training-config-store.ts re-runs the cached format check whenever rejectValidation() reports the in-flight token as stale, with no counter and no backoff. DatasetCacheRejectionTracker advances its generation on any inventory-fingerprint change, and the fingerprint includes sizeBytes, so a dataset that is still downloading invalidates every check in flight and the pair never converges. Measured at 480 requests in 60s; a probe driving the real tracker ran 500 iterations without settling while a stable inventory settles in 1. The retry now draws from a per-selection budget (dataset-recheck-budget.ts) and falls through to the uncached check once spent, so the answer still refreshes when the inventory genuinely changes but a churning one cannot spin. The budget resets when the dataset or split changes. Fixing the tracker fingerprint instead was rejected: existing tests pin the behaviour that a sizeBytes change makes a rejected cache retryable. 2. Unconfirmed starts adopted as recovered. reconcileTrainingStartTransportFailure ended with adoptAcceptedTrainingStart(pending.jobId, ...) and returned "recovered". The backend reserves the request id and job id before the heavy preflight, so a start still pending when the 30s window closes may yet be rejected. Adopting there reported success and pinned a job id that never became current_job_id: acknowledgeTrainingStartRequest was never sent, and dismissTrainingRun's expectedJobId check then returned "superseded", leaving the rejected state unclearable until a new start re-reserved it. It now calls the module's existing settleUnconfirmedTrainingStart and returns "unknown", which both callers already handle by warning startUnconfirmed and settling unconfirmed, and which preserves start_request_id for acknowledgement. New tests fail without each change and pass with it; removing the bound makes them fail rather than hang. typecheck, 598 frontend tests, build, and the 154 studio contract tests all pass. * Studio: retry tokenizer-less pinned snapshots, and skip Hub preflight when offline Two more defects introduced by this rework. Neither mechanism exists on main: worker.py there has 0 occurrences of local_files_only / model_snapshot_path / cache_artifact, and routes/training.py has 0 of model_info / dataset_info. 1. Tokenizer-less pinned snapshot is terminal (#7845). The pin only requires config.json plus a weights file, so a snapshot with no tokenizer pins clean, local_files_only is set, and AutoTokenizer then fails. Recovery is gated on _is_model_cache_artifact_error, whose marker list matches only three of the message shapes transformers actually emits. Sweeping all 159 tokenizer classes against a tokenizer-less snapshot: 37 failures were classified not retryable, of which 26 are genuine cache problems that got zero Hub retry, including XLMRoberta (BGE-M3, multilingual-e5, LaBSE), MBart, NLLB, Bloom, GPTNeoX, Cohere, Marian and the generic PreTrainedTokenizerFast. SentencePiece and BPE families resolve a missing vocab path to None and then dereference it, so the failure arrives as a bare AttributeError with no cache-specific text. Adding those four shapes takes the sweep from 37 misses to 11, and all 11 remaining are correctly fatal (missing optional Python dependency, or an unsupported tokenizer class), which no Hub retry can fix. Widening the classifier rather than validating tokenizers before pinning: the vocab filename space is as open-ended as the exception space across 159 classes, requiring a tokenizer would break the deliberate adapter_config.json pin path, and a real loadability check means loading a tokenizer inside the start request. The recovery path is already gated on offline and require_exact, so a false positive costs one Hub attempt while the current false negative fails the run. 2. Blocking Hub preflight with no reachability guard. Both preflight legs retry metadata at 5s then 10s with no reachability check, and requests applies the timeout per resolved address. Measured 30.0s added to a single POST /training/start with the Hub black-holed, scaling with the number of addresses, surfacing as 503. utils.utils already provides a bounded, memoised hf_unreachable/hf_dns_dead that the training worker subprocess uses one layer down; the route that spawns it did not consult it. The model leg raises inside the existing try, so the except HTTPException handler still runs _resolve_model_snapshot and a cached snapshot pins exactly as before, just without first burning the remote budget. The guard fails open, so an online start is unchanged. New tests fail without each change and pass with it, including a wiring contract that fails if the guard stops being consulted. 443 tests across the preflight, cached-start, provenance, snapshot and streaming suites pass. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the import-hoist lint honour __all__ re-exports Source lint failed on studio/backend/utils/security/__init__.py: [BLOCKER] HOISTED-IMPORT-UNUSED 'load_scan_target' (['from:utils.security.file_security:load_scan_target']) added but unused That file is a barrel __init__: every name it imports carries "# noqa: F401" and is listed in __all__, and routes/training.py imports load_scan_target from the package rather than the submodule, so the re-export is load-bearing. HOISTED-IMPORT-UNUSED only excluded re-exports that already existed before the change, so adding any new name to an existing barrel was an automatic blocker. The script's own docstring already records this shape as a known false positive for NEW-UNUSED-IMPORT. Skip module-level imports whose bound name appears in __all__, since those are loaded by importers rather than by the module itself. The rule keeps its teeth: a newly added import whose name is not in __all__ still blocks, and a dangling alias still trips UNRESOLVED-NEW. Verified with a four-case mutation matrix plus --self-test, and the full 79-file changed-set lint now reports OVERALL: PASS. * Keep studio off evaluated PEP 604 unions on the 3.9 floor tests/test_python39_compatibility.py failed on Core (HF=4.57.6, default and latest) and Repo tests (CPU): test_no_pep604_unions_are_evaluated_on_the_declared_floor model_config.py:937: Tuple[...] | Tuple[...] (type alias) test_studio_evaluated_unions_do_not_grow 36 studio files now evaluate PEP 604 unions, up from 35 Both offenders came from this branch. model_config.py:937 is a module-level type alias, so it is evaluated at import time and "from __future__ import annotations" cannot defer it; it needs typing.Union, which is what the test message prescribes. hub/utils/dataset_cache.py is the one file that pushed the count to 36, and all five of its unions are function annotations, so the future import is enough there. No behaviour change: the alias is only a cache-key type, and the annotations are unevaluated either way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Key the dataset re-check budget on the whole selection The budget added for #7853 keyed on dataset + split only, but a dataset cache usability identity has four user-chosen dimensions: dataset, subset, split and streaming. Changing subset or toggling streaming therefore kept the same key, so a genuinely different selection inherited an exhausted budget, skipped the cache-preferring re-check and dropped straight to a remote resolution with datasetKnownCached cleared. Key on all four instead, JSON-encoded so no delimiter can collide with a name containing it and null stays distinguishable from the string "null". cachePath is deliberately still excluded even though the usability identity carries it. It is derived state that moves as a download populates the cache, so feeding it into the key would mint a fresh budget on every poll and re-arm the exact non-terminating loop this module exists to bound. Tests: two new cases cover subset and streaming, and an inverse case pins that nothing outside the selection can refresh the budget. Reverting the key to dataset::split fails exactly the two new cases and no others. * Stop the pinned snapshot path reaching PEFT as the base model name A completed run wrote a machine-local path as the adapter's base model: base_model_name_or_path = '/home/user/.cache/huggingface/hub/ models--unsloth--Llama-3.2-1B-Instruct/snapshots/0123…' where main writes the Hub id. It lands in adapter_config.json, every checkpoint-*/adapter_config.json, the run card, export_metadata.json for merged and GGUF exports, and the model card push_to_hub uploads, none of which resolve on another machine. restore_hf_cache_repo_identity runs in UnslothTrainer.load_model before get_peft_model, so its peft_config branch has nothing to repair yet, and PEFT then derives the name itself: # peft/mapping_func.py new_name = model.__dict__.get("name_or_path", None) peft_config.base_model_name_or_path = new_name PreTrainedModel.__init__ copies config.name_or_path onto the instance, so restoring only config._name_or_path leaves that slot holding the snapshot path. Restore the instance attribute too. It goes through the same guard as the other fields, so an ordinary local model, an unrelated Hub id and a repo mismatch are all still left alone. Tests are behavioural: the existing model identity suite asserts the call site via AST and stays green with this bug present, which is why it was missed. Removing the new line fails 3 of the 6 added tests and none of the other 11. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve cached snapshots that load from a subdirectory unsloth/Spark-TTS-0.5B keeps everything trainable under LLM/: its snapshot root carries only README.md and config.yaml, no config.json and no weights. _resolve_model_snapshot only looked at the root, so a cached copy resolved to None. _apply_model_cache_pin then warned "Cached copy not found on disk; downloading", and offline the start route turned the same None into a 409 hf_model_not_cached_offline for a model that was sitting in the cache. The remote preflight already handles this: it expands load roots through load_scan_target. Only the cached path disagreed. Reuse security_load_subdirs, which already reports ("LLM",) for BiCodec, so both paths share one source of truth, and apply it to the resume and preflight pin lookups too rather than just the fresh-start one. Detection can raise offline or for a gated repo, so a failure degrades to root-only rather than propagating. Tests build the real models--org--name/snapshots/<rev> layout. Making the helper a no-op fails 3 of the 6, and a snapshot with nothing loadable in either place still resolves to None, so the widening cannot mask an empty cache. * Use Optional in the deprecated dataset alias signatures The rewritten alias module annotates two parameters as UploadFile | None and str | None with no postponed annotations, so they evaluate at import on the declared 3.9 floor while the rest of the file already uses Optional[...]. The file was an evaluated-union offender before this change too, so this is not a new break, just keeping the new code consistent with its own convention and off the debt list. * Apply the repo kwarg-spacing formatter to the new code pre-commit.ci flagged the ruff-format-with-kwargs hook on this branch. Running scripts/run_ruff_format.py locally keeps the fix in the authoring commits instead of trailing a separate bot commit. * Say why a resume is refused instead of blaming the checkpoint can_resume_run gained a provenance clause in this rework, so it now returns False for runs whose checkpoint is entirely intact, most realistically once the pinned model snapshot is evicted from the HF cache. The start route answered every False with "Resume checkpoint must belong to a stopped or errored run with complete saved trainer state." pointing the user at trainer state that is fine, while the real cause was the resource gate. exact_resume_resource_requirements already raises with a precise explanation and resource_provenance_allows_resume was discarding it. Add resource_provenance_resume_blocker, which returns that explanation (or None when the run is resumable), define allows_resume in terms of it so the two cannot drift, and use it in the start route when it is the actual cause. The gate itself is unchanged: refusing an unattested resume is deliberate, only the diagnosis was wrong. Tests are behavioural, with one narrow wiring contract for the branch that regressed. Replacing the precise reason with a generic string fails the message test and nothing else. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Honour load subdirs everywhere a cached snapshot is probed Follow-up to 590ac9f22. Resolving cached Spark-TTS/BiCodec snapshots from their LLM/ load root got a start past _resolve_model_snapshot, but three later probes still asked for root-level config.json only, so the same snapshot was accepted in one place and rejected in the next: - provenance.py attested incomplete model provenance, after which exact_resume_resource_requirements refused resume for a snapshot still sitting on disk; - the start preflight reported a valid cached model as having no trainable weights; - the worker cleared model_snapshot_path before loading, falling back to the Hub instead of the selected snapshot. Promote the helper to hub.utils.hf_cache_state.with_load_subdirs so there is one definition rather than four, and use it at every site. _has_trainable_local_weights takes subdirectory roots instead, since it probes directories rather than a filename list. Unchanged for ordinary models: with no load subdirs the helper returns its input, and detection failures still degrade to root-only. * Let the model routes see the same cached snapshots training does Two ways /api/models disagreed with the training resolver, both reachable from resume: _model_config_inspection_target probed only the snapshot root, so a cached Spark-TTS/BiCodec copy answered "Selected cached model is no longer available" for a cache the training resolver accepts, and the exact-snapshot remote-code scan could fail with it. It now uses the shared with_load_subdirs helper. The model_snapshot_repo_id guard used the owner/repo-only regex, so resuming or scanning a namespace-less Hub model such as gpt2 or bert-base-uncased returned 400 before the snapshot could be inspected, even though hub.utils.paths .is_valid_repo_id and the picker both allow the one-segment form. That call site now uses the shared validator. The other five uses of the local regex predate this branch and are left alone. Reverting either fix fails one of the new tests and nothing else. * Get the resume refusal reason all the way to the user 300fe6321 fixed the diagnosis on POST /api/train/start, but the History UI never reaches it. can_resume: false hides the Resume button outright, and in the one window where the server could answer, resume-training-run.ts throws its own "Only stopped or errored runs with a saved checkpoint can be resumed" before sending any request, so no start call is ever made. For a run whose checkpoint is intact and whose pinned snapshot was evicted, that sentence is the wrong diagnosis, which is what 300fe6321 removed server-side. Carry the reason on TrainingRunSummary and prefer it in that guard. The field is optional and defaults to None, so old clients are unaffected, and it is only computed for rows already known unresumable. A checkpoint problem still reports None, leaving the client's existing wording for that case, and a failure inside the gate is swallowed so History still renders. Reverting either half fails one of the new tests: the summary stops carrying the reason, or the client stops preferring it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Honour load subdirs on the resume pin and keep the selected model cache copy Two follow-ups to the cached-snapshot work. The resume branch of _reject_untrainable_model_request still probed the pinned snapshot with a hardcoded (config.json, adapter_config.json) tuple, so a repo that loads from a subdirectory (unsloth/Spark-TTS-0.5B keeps everything trainable under LLM/) resolved to None. Offline that became a 409 hf_model_not_cached_offline for a snapshot present on disk; online it fell through to the remote metadata round trip the pin exists to avoid. It now uses with_load_subdirs like the other cached-snapshot probes. The cache reconciliation effect took the first usable inventory row for the selected repo. A repo can be present under more than one HF cache root, so that silently retargeted an explicit selection at a different copy on the next inventory tick. The selection logic moved to a pure module and now prefers a row whose path matches the current selection, falling back to the first usable row when there is no selection or the selected copy has gone. Tests: test_resume_pin_load_subdirs.py (5) and model-cache-reference-selection.test.ts (8); reverting either fix fails them. * Revert the model cache reference preference; it cannot fire The frontend half of 2c2a95309 assumed the model inventory can return more than one usable row for a repo. It cannot. hub/services/models/cache_inventory.py::_scan_cached_models collects into seen_lower keyed by repo_id.lower() and resolves collisions with _prefer_cache_row, and _dedupe_local_models keys hf_cache rows by (model_id, model_format, format_variant). Both return one row per repo, and a live check with the same repo under four HF cache roots got exactly one row from /api/hub/cached-models and one from /api/hub/local. With a single usable row usable.find(pred) ?? usable[0] is identically usable[0], so the change was a no-op and its tests asserted an array shape the API cannot emit. The resume pin fix in the same commit stands: that one was reproduced against the real unsloth/Spark-TTS-0.5B cache and reverting its hunk restores the 409. * Report the resume refusal that actually happened 300fe6321 and a98e721f4 set out to stop a provenance refusal being reported as a checkpoint problem. They overshot. Both sites asked resource_provenance_resume_blocker whenever can_resume_run said no, but that function refuses for several reasons and the blocker is computed independently of which one fired. initialize_resource_provenance writes {version: 1, status: pending} at the start of every run, so the blocker answers "The model revision used by this run was not attested." for any Hub-model run, including one whose checkpoint is simply missing. That is the more common way to be unresumable, so the change traded one misdiagnosis for another and the new one covered the larger population. Both sites now gate on has_resume_state, the same discriminator can_resume_run short-circuits on: no saved trainer state means the checkpoint is the cause and the client's own wording is correct; an intact checkpoint means a refusal really is provenance's doing and the specific reason is worth surfacing. test_resume_reason_matches_cause.py pins it; reverting either guard fails it. The route contract assertion is over the AST rather than the source text, because a substring search is satisfied by the explanatory comment beside the code and passes with the guard deleted. Two assertions in test_resume_blocked_reason_surfaces.py encoded the old behaviour and are corrected, including the docstring claiming History survives a raising gate: can_resume_run calls the same gate unguarded one line earlier, so that only holds when it short-circuits first. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope the import-hoist __all__ skip to package __init__.py 68bce3934 skipped any name listed in __all__. The comment said "the whole point of a package __init__", but compare() never consulted the path, so the skip applied to every module defining __all__ -- 27 non-package modules exempting 224 names, unsloth/models/_utils.py alone 74. The cost is not just breadth. lint-ci.yml names rename-clash as one of the two bugs this tool exists to catch because ruff and pyflakes miss it, and the skip disabled that detection for any name in __all__. Adding a name there became a one-line, reviewer-invisible way to switch the check off. The self-test I cited did not cover this: --self-test reported ALL PASS with the skip hunk reverted, because every case ran through the "<name>" path placeholder and none defined __all__. Cases may now carry their own path, and three new ones pin the behaviour from both sides -- a re-export in a package __init__ is allowed, the same shape in an ordinary module is still blocked, and a new import absent from __all__ is still blocked in an __init__ too. Deleting the skip fails the first; widening it back fails the second. Removes two genuinely unused imports the correctly-scoped check then found in test_resume_blocked_reason_surfaces.py. * Make deleting a run's artifacts reversible until the row is gone DELETE /runs/{run_id}?delete_artifacts=true is new in this PR; on |
||
|
Daniel Han
|
17fe73cd9e
|
Diffusion: four fixes salvaged from the stale #7021 (#8006)
* Keep NVFP4 out of the Blackwell auto quant ladder The comment above _AUTO_LADDER already states that nvfp4 "stays an explicit opt-in", but the Blackwell tier still listed TQ_NVFP4 between fp8 and mxfp8, so an auto request could silently select it. The measurement the comment cites is that nvfp4 is both slower and less accurate than fp8 at DiT shapes (0.81x end to end on Z-Image 1024px, LPIPS 0.166 vs fp8's 0.044), so auto picking it is never the right answer: it is a scheme a user should have to ask for. Reachable rather than theoretical. fp8 leads the tier and _prefer_consumer_scheme moves int8 first on consumer Blackwell, so nvfp4 only wins when the fp8 smoke probe fails on an sm_100+ part while nvfp4's passes, but in that case auto drops to a scheme that regresses both speed and quality instead of falling through to mxfp8. The Blackwell tier is now (fp8, mxfp8, int8) with the previous tuple kept commented directly beneath it, so re-enabling nvfp4 is a one line revert once the FP4 tensor-core GEMM wins at the DiT's real shapes and its accuracy clears the prequant gate. Explicit transformer_quant="nvfp4" is untouched. Salvaged from #7021. * Refuse the kernels auto-install on a pre-1.0 huggingface_hub _ensure_attention_backend_installed pip installs the package a requested attention backend needs. For flash_4_hub and _flash_3_hub that package is kernels, and current kernels releases require huggingface_hub >= 1.0: kernels 0.16.0 declares huggingface-hub>=1.10.0. The install runs with --no-deps, so on a stack pinned to an older hub it writes a kernels that cannot import. The damage is not contained to the requested backend. diffusers reaches for kernels whenever it is installed (attention_dispatch guards on is_kernels_available() and then does "from kernels import get_kernel"), so a single on-demand install from one explicit flash3/flash4 request leaves every later pipeline import on the box importing a broken package until it is removed by hand. The installer now checks the resident hub major version before installing kernels and refuses below 1.0, logging why and leaving the load on the native default, which is the fallback the installer already promises for a wheel it cannot install. Three details worth noting: - The refusal is a policy decision, not a failed attempt, so it is checked before the _INSTALL_ATTEMPTED memo and records nothing. A later request on a fixed environment can still install. - The gate is scoped to the kernels package. The sage, flash-attn and xformers wheels do not import huggingface_hub at module scope and are unaffected. - An undeterminable hub version keeps the previous permissive behaviour. Tests cover the refusal and that nothing is memoised, the hub >= 1.0 allow, the package scoping via sage, and the version-parse fallback. Salvaged from #7021. * Key the image-gen LoRA rows by index so typing a repo id keeps focus The LoRA rows on the Images page were keyed as key={sel.id || i}, but sel.id is the value of the editable repo-id Input inside the row. Typing the first character of a repo id changes the key from the index to "b", React treats that as a different element, and the row is unmounted and remounted with the input losing focus after every single keystroke. The list is index-addressed everywhere else (each mutation matches on j === i, removal filters on the index) and the sibling reference-image rows already use key={i}, so the index is the stable identity here. Salvaged from #7021. * Skip padded text tokens in HunyuanVideo-1.5 joint attention HunyuanVideo15AttnProcessor2_0 runs a joint [video ; text] self-attention and rebuilds a dense [B,1,N,N] boolean mask on every block and every step so the video never attends to padded text. A dense bool attn_mask disables every fused SDPA kernel: flash rejects it outright ("Flash Attention does not support non-null attn_mask"), and cuDNN and the memory-efficient kernel both report "kernel not used" and fall back to math. Measured at the production joint shape (B=1, H=16, N=50345, D=128, bf16) with scripts/sdpa_mask_backend_probe.py, added here: 638.1 ms per attention call with the dense mask against 29.4 ms with attn_mask=None, a 21.7x tax paid purely to mask padding. And the text is almost entirely padding: a t2v prompt fills roughly 9 of about 1985 slots (image 729 + byt5 256 + mllm 1000). Removing the padded tokens is exact for the video. RoPE is applied to the video stream before the text is concatenated, so text has no positional dependence on its length; the model already masks the padded text and discards its attention output, since only the video split feeds proj_out. The only numeric change is which SDPA kernel runs. install_hunyuan_attention_trim adds an eager forward pre-hook on the DiT that drops the all-zero image stream (upstream's own t2v sentinel, and torch.all of an empty tensor stays vacuously True so is_t2v is unchanged) and trims the mllm/byt5 streams to the columns that are valid for at least one batch element. It also installs a null-mask attention processor that passes attn_mask=None once no partially-padded column remains, which is the batch-1 per-guidance-branch case the pipeline actually produces, and otherwise delegates to the stock dense-mask processor, so a mixed-padding batch stays correct. Correctness scoping: the null-mask flag is only ever true inside the one forward whose pre-hook did the trimming. A post-hook registered with always_call clears it even when the forward raises, so an exception can never leave the authorisation latched. Any failure in the pre-hook restores the caller's untrimmed kwargs and forces the flag false. An empty prompt reverts entirely, because the TokenRefiner's pooling divides by the mask sum. Verified against a real HunyuanVideo15Transformer3DModel in fp32 on a B200, stock versus trimmed, over t2v, i2v, a uniform batch, a mixed-padding batch, an empty prompt and an unpadded prompt: max relative difference 1.9e-7 with cosine 1.0000000 and no NaNs, and the empty-prompt case is bit-identical. Re-installing is idempotent (hook counts stay 1 and 1). Scoped narrower than the original change: the trim is installed only on the speed tiers that compile with dynamic shapes. speed=off must stay bit-identical, and speed=max compiles the blocks with dynamic=False, where the prompt-dependent trimmed length would make every new prompt its own graph and eventually exhaust dynamo's recompile limit under fullgraph. No-op for every other family, and reversible: any failure leaves the stock dense-mask path, so correctness never depends on this. Salvaged from #7021. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Exclude HunyuanVideo-1.5's small-M text streams from int8 The attention trim in this PR is what makes these necessary, so they have to ship with it. quantize_transformer runs before the trim hook is installed, so the trimmed activations flow through already-int8 linears: M = 0 comes back unprojected (torchao passes the input through, and the 2048-wide cond-type add then crashes) and M <= 16 trips torch._int_mm's floor. Reachable by an explicit transformer_quant=int8, or by leaving it unset on any Ampere card, where the auto ladder's only matching tier is int8. Before the trim the streams stayed padded at ~1985 tokens and int8 was fine, so this PR without it is a regression rather than a pre-existing gap. Salvaged from #7021. * Refuse the kernels auto-install below huggingface_hub 1.10 The gate accepted every 1.x hub, but kernels 0.14.1+ declare huggingface-hub>=1.10.0 and the installer runs with --no-deps, so pip never reads that requirement. Measured with kernels 0.16.0: hub 1.0.0-1.2.4 raise StrictDataclassFieldValidationError on import kernels, which is exactly the every-later-diffusers-import breakage the gate exists to prevent. Compare (major, minor) against the declared floor instead of the major alone. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Maheswar Kumar
|
c8e657e717
|
add a background image to the macos dmg install window (#7827)
* add a background image to the macos dmg install window The disk image opened onto an empty Finder window. It now shows a green halo behind the app icon and a chevron pointing at the Applications alias. studio/src-tauri/dmg/background.tiff holds two pages, one at the 660x400 window size and one at double that for retina displays. Finder maps icon coordinates onto the background from the same origin, so the base page has to match dmg.windowSize or the artwork slides out from under the icons. The icon positions move from y=220 to y=170. Finder's title bar and path bar leave roughly 340pt of the 400pt window on screen, and y=170 centres the icon row in what remains. scripts/make_dmg_background.py renders the image and reports the halo's evenness, reach and tint across the icon label. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
oobabooga
|
df5f139bac
|
Studio: add image generation, editing workflows and LoRA training with Unsloth GGUFs (#6763)
* Tighten comments in the image stack tests and scripts * Close video single-file, training reservation, and image mount-resume gaps Route on-device single-checkpoint video folders through the single_file loader: a bare local .safetensors directory (no model_index.json) is advertised as a pipeline with no filename, so validation rejected it before it could load. Reinterpret the pick as a single_file load of the sole checkpoint, mirroring the image load route. Treat a reserved-but-not-yet-spawned LLM training start as active in is_training_active() so /images/load, /video/load, and /diffusion/start cannot race the reserved run for VRAM during the pre-spawn free window. Mirrors the diffusion training service reservation. Resume an in-flight image generation on the Images page mount: probe generate-progress, re-enter the poll loop, and refresh the gallery on completion so a run started elsewhere is reflected and its saved image appears without a manual refresh. Seed resident image defaults from the resolved base_repo rather than a possibly path-shaped repo_id so the first resident generation uses the right recipe. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Publish image generation active state before pre-denoise setup generate() assigned self._gen only at the pipe() call, after deferred compile, LoRA resolution/application, and ControlNet download/build had run. Across that setup window generate_progress() reported inactive even though _generate_lock was held, so a reloaded page's mount probe showed idle and let a second generate queue behind the first. Publish an active step-0 _GenState the moment the generation lock is acquired, before the setup work, and clear it in the outer finally so a setup-time error cannot leave the UI stuck active. Mirrors the video backend's queued phase and the training start guard. * Studio: fix diffusion install ownership, dataset upload atomicity, gallery pagination, and teardown races install_sd_cpp_prebuilt: only write the .unsloth-studio-owned marker when the install created the target directory or it was empty. Adopting a pre-existing, unowned, non-empty directory (a user's own stable-diffusion.cpp checkout) made it eligible for the uninstaller's recursive delete. routes/training upload: make the multi-file promotion transactional. Back up each displaced original and roll every destination back on any failure, so a mid-loop rename error can no longer partially overwrite the live dataset. routes/training _resolve_dataset_folder: reject a symlinked dataset directory and prove the resolved folder stays under the datasets root, so image read/caption/delete cannot escape the root through a link. routes/training delete: escape glob metacharacters in the thumbnail filename so deleting an image named like [ab].png removes only its own thumbnails. image_gallery / video_gallery listing: filter records against the response schema inside the pager via a valid callback, so offset/limit/has_more all count over accepted records. A leading schema-invalid record no longer returns an empty page with has_more=true and stalls infinite scroll at offset 0. image_gallery / video_gallery save: publish via a temp file plus atomic rename (the sidecar is the video pair's commit marker) and clean up on failure, so a partial write never surfaces a truncated PNG or strands an orphan MP4. diffusion_train_common discovery: treat an empty caption sidecar as a metadata tombstone that still falls through to the dreambooth instance prompt, so clearing every metadata caption no longer fails with no captioned images found. diffusion backend unload: wait for an in-flight denoise to exit before tearing down process-wide patches and state, mirroring the load path. diffusion_engine_router: serialize the whole check/unload/publish transition so a concurrent selection cannot return the engine being unloaded. uninstall.ps1: gate the default sd.cpp process stop on the owner marker so a user's own sd-server is not terminated for a directory we then keep. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject native batch seeds outside the JSON-safe range * Refuse sd.cpp install into unowned non-empty target dir When the install target already exists, is non-empty and lacks the .unsloth-studio-owned marker (a user's own stable-diffusion.cpp checkout, or unrelated files beside a custom Studio root), install() previously still extracted the release into it. Skipping the ownership marker only stopped the uninstaller from deleting the directory; extraction still merged binaries into the user's working tree and could overwrite same-named files. Fail up front with a clear message pointing the user at a fresh/empty location before any download or extraction, leaving their directory untouched. Update the ownership test suite to assert the refusal. * Studio: gate dataset uploads on the symlink check and surface local video single-file checkpoints * Tighten comments and docstrings added by the image-generation fixes * Studio: close arbiter load-registration race and surface native progress + local pipeline folders Publish native sd.cpp generate progress (_gen) before LoRA resolution so a reload probe reads active during setup, matching the diffusers path. Register the diffusion/video GPU load under the arbiter lock (acquire_for now takes a register callback) so a competing acquire cannot evict an owner before its load is marked in-flight and let two loaders allocate VRAM at once. Admit local diffusers pipeline folders (root model_index.json, weights in component subdirs) in the local model scan so they reach task tagging and the On Device picker. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: include marker-owned custom sd.cpp roots in the uninstall stop scan; harden Pester install against the nuget.exe PSGallery bootstrap * Studio: surface local pipeline scan roots, tag single-file checkpoints by filename, mark companion-only pipelines partial - _scan_models_dir: admit a scan folder that is itself a diffusers pipeline (root model_index.json, weights in transformer/ vae/ subdirs). _is_model_directory rejects such a root, so the child scan would list the component subdirs as bogus models and hide the real pipeline; treat the root as one model via _local_pipeline_index. - _local_is_diffusers / _local_model_task: include the sole checkpoint filename in the family-detection needles (_local_family_needles, resolved via resolve_local_single_file). A generically named folder holding one loadable qwen-image-*.safetensors / ltx-*.safetensors identifies its family only from the filename; the load route already resolves that file, so tag it or the task-scoped picker (which rejects task=null) hides the on-device model. - list_cached_models: mark a companion-only base snapshot partial. A GGUF image load prefetches the base repo's VAE / text-encoder / model_index.json but skips the transformer (the GGUF supplies it); the snapshot has a pipeline manifest yet is not a loadable BF16 pipeline, and _cached_repo_partial misses it. _repo_pipeline_missing_denoiser flags a pipeline snapshot whose transformer/ or unet/ component carries no weight, so the picker drops it instead of advertising it as fully on-device. * Studio: preserve foreign gallery files, force safetensors on remote ControlNets, and close dataset/seed/GPU gaps Gallery clear/delete now scope to Studio-owned files: image_gallery and video_gallery skip PNGs / MP4s without a readable recipe (a hand-dropped or orphan file the listing already hides), so clear() and a guessed-id delete no longer destroy files the gallery never surfaced. Remote ControlNets now force use_safetensors: a bare owner/name reaches from_pretrained without the base trust gate, and the Hub scan fails open when unavailable, so requiring safetensors closes the pickle deserialization vector. POSIX uninstall now stops resident sd-server / sd-cli under an owned sd.cpp root before removing the tree (marker-gated), mirroring the Windows stop-before-delete scan; a live native server no longer survives unlinking its binary. Diffusion dataset containment: the training-start read path and the discovery picker route bare names through the protected resolver, so a symlinked dataset is rejected / not advertised like the caption/delete routes already do. Uploads gain the inference decode guard (oversized real images 400 before OOMing the trainer) and dataset upload/caption/delete/import are blocked with 409 while a diffusion run is active. JSONL readers (trainer + routes) tolerate non-object JSON and invalid UTF-8 instead of raising AttributeError / 500. LoRA family compatibility is enforced in the shared resolver, not only the picker, so a direct API client cannot apply a mismatched-family adapter. GPU arbiter gains release_if so the image/video unload idle-check and release are atomic against a concurrent same-owner load's registration. Native batch recipes persist the base batch_seed and restore replays from it, so a native batch_index>0 image no longer advances its seed twice. FLUX.2-klein selects its sd.cpp text encoder by variant (4B -> Qwen3-4B, 9B -> Qwen3-8B) instead of the single family default. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: track the loaded GGUF filename so native companion resolution reproduces the load identity * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: gate local-pipeline image tagging on a real family; validate video sidecars before delete/clear * Studio: tighten image-generation fix comments and docstrings * Studio: gate gallery serve/export on ownership; keep image progress active until persisted; reserve diffusion training before the dataset scan * Studio: restore Reapply target on async image/video load errors; recheck training state before dataset commit; reclaim partial sd.cpp installs on retry Images/Video: a background model load that fails AFTER starting (error/eviction during download) leaves the previous pipeline resident, but handleLoad had already overwritten lastLoad.current with the failed pick, so "Reapply to loaded model" reloaded the failed model. Carry the prior Reapply target into the poller and restore it on the async error/null paths, mirroring the quant rollback. Training: an in-flight diffusion dataset upload passed _require_diffusion_dataset_mutable() at entry but could still commit files after a concurrent /diffusion/start reserved the training slot, mutating the dataset underneath the trainer. Re-check the interlock immediately before the commit phase; a 409 there leaves the staged temps for the finally to clean. sd.cpp install: an interrupted extraction (disk full, killed process, a raising post-extract cudart fetch) left the target non-empty with no owner marker, so the next lazy install tripped the "not a Studio-managed directory" refusal and wedged native install. Write the ownership marker before the partial writes when the target is reclaimable, so a retry recognises the debris as ours and re-extracts. * fp8 DiT quant: floor the dynamic activation scale with activation_value_lb An all-zero activation token row makes the dynamic per-row fp8 scale 0, which turns the quantized data to NaN and the render to black frames on torchao's plain-torch kernel path. The fused fbgemm/mslk quantize kernels clamp zero rows internally, so the bug only reproduces on machines without them, which is most user environments. Zero rows are real inputs, not a corner case: Wan 2.2 zero-pads its text conditioning, and Hunyuan-1.5 and Qwen-Image regenerate zero rows inside their transformer blocks every step. Pass activation_value_lb=1e-12 to Float8DynamicActivationFloat8WeightConfig whenever the installed torchao supports the kwarg (Float8Tensor rework, 0.13+), checked via inspect.signature so older torchao keeps exactly the current behaviour; the existing Float8MMConfig fallback chain is unchanged. Verified on GPU: with the forced plain-torch kernel path a zero-row input NaNs without the floor and stays finite with it, and end to end on HunyuanVideo-1.5 fp8 goes from a solid black frame (LPIPS 1.00) to a normal render (LPIPS 0.225); on Wan the floor matches the condition_embedder exclusion (LPIPS 0.211 vs 0.206). Same-seed renders with fused kernels present are unaffected, and pre-quantized fp8 checkpoints stay valid since weight scales are untouched. * Wire hosted pre-quantized DiT checkpoints into the image families Point prequant_repos for flux.1, flux.2-klein, flux.2-dev, qwen-image (int8 only there; fp8 is family-denied), z-image and krea-2 at the unsloth/<Model>-FP8 Hub repos carrying gate-validated int8 and fp8 transformer checkpoints, so the fast quant path loads the small pre-quantized file instead of materialising the dense bf16 transformer and quantising on device. Measured on FLUX.2-dev int8: build peak drops from 60.7 GB (dense + quantize) to 30.7 GB (hosted prequant), identical 30.7 GB resident after either path since loading a checkpoint is bit-identical to on-the-fly quantisation. The hosted repos name files <Model>-<SCHEME>.pt, so resolve_prequant_source now derives that model-name filename from the repo id (scheme suffix stripped case-insensitively) and carries the legacy transformer_<scheme>.pt as a fallback the resolver tries when the primary 404s, keeping older repos loadable. Wiring a repo also exposed a fallback hazard: with a prequant source present, the dense-fit preflight used to be skipped entirely, so a failed prequant download would fall through to the dense bf16 load the memory plan never budgeted, OOMing after eviction. The preflight now always runs and gates an allow_dense_fallback flag through _load_dense_quant_pipeline: a dense misfit still skips the fast path when no prequant exists, but with one it proceeds and a prequant failure raises to the GGUF build instead of loading dense. The same flag is set when the auto-policy replans an offloaded GGUF against a prequant-sized transient. Tests updated to the new filename convention plus new coverage for the derivation and the legacy-name fallback; the prequant-skips-refit test now asserts the re-check runs and forbids the dense fallback. Verified end to end on GPU: z-image int8 resolves the hosted repo, downloads the model-name file and renders (6.8s load, 5.9 GB peak). * Route krea-2 through its per-component loader on the transformer-quant fast path _assemble_pipe used Pipeline.from_pretrained for every family, but the krea repo ships transformers-5.x configs and no top-level tokenizer files, so the tokenizer dies with vocab_file=None. The pre-quantized checkpoint loaded fine and then the assembly crashed, dropping the load to the GGUF build, which krea-2 cannot take (Krea2Transformer2DModel has no from_single_file). Assemble per-component via load_krea2_pipeline like the pipeline-kind and single-file paths already do. Verified live: Krea-2-Turbo int8 and fp8 hosted prequant loads now assemble and render through the Studio images tab. * Keep Qwen-Image's text-stream linears bf16 on int8 (short prompts break torch._int_mm) Qwen-Image's MMDiT runs every text-stream Linear at M = actual prompt tokens: the Qwen2.5-VL embeds are not padded to a fixed length like FLUX's 512-token T5. A short prompt (13 tokens) or the near-empty negative prompt drives torch._int_mm below its M > 16 floor and the first denoise step raises 'self.size(0) needs to be greater than 16, but got 13' (measured on B200 through the Studio images tab). Add per-family int8 exclusions (txt_in, add_q/k/v_proj, to_add_out, txt_mlp) for qwen-image and qwen-image-edit, threaded through exclude_tokens_for_scheme(scheme, family) and the prequant checkpoint validation, so a checkpoint baked under the old token list is rejected and re-quantised instead of loaded crashing. The text stream runs at M = tens vs the image stream's M ~ 4k, so the exclusion costs nothing; the rebuilt hosted checkpoint gates 28/28 PASS with LPIPS mean 0.057 (was 0.069). * Harden the diffusion memory plan against transient free-VRAM undercounts A cold FLUX.2-dev int8 load on an idle 183 GB B200 planned offload=model (companions exceed budget) and silently served the GGUF as-is; the identical retry went resident and engaged the hosted prequant. The plan arithmetic was byte-identical across both loads (required 90,228 MiB, resident needs free of about 124 GB); the only divergent input was torch.cuda.mem_get_info, which is device-wide and instantaneous: a transient foreign CUDA context briefly held about 100 GB at the first snapshot, and the planner trusted that single read. Three changes: - settled_snapshot_device_memory: on cuda, synchronize + empty_cache (best-effort) and take the MAX free over up to 3 spaced reads. A transient can only shrink free, so the max rejects transient undercounts while a persistent tenant still caps every read. _plan_memory now uses it. - plan_fits_total_capacity + one replan retry: when the dense/prequant candidate fits TOTAL device capacity under the standard reserve and the 0.85 resident margin, an offload verdict can only stem from the free reading, so the loader re-snapshots and replans once before declining the fast path. Explicit balanced/low_vram modes skip the retry (they offload by mode). - diffusion.transformer_quant_declined log line with required/budget/free and the plan reasons, so the next decline is diagnosable from the server log (previously silent). Verified: cold FLUX.2-dev int8 first load in a fresh server now engages the hosted prequant resident (offload=none). * Add FLUX.2 Klein and FLUX.2-dev DiT LoRA training Register flux.2-klein and flux.2-dev in the DiT trainer following the upstream DreamBooth references: latents train patchified and batch-norm normalized from the VAE posterior mode, the packed forward reuses step-invariant position ids, and the guidance vector (3.5) is gated on the variant's guidance_embeds config. Conditioning stacks load per variant (Mistral via Flux2Pipeline for dev, Qwen3 via Flux2KleinPipeline for Klein) and are encoded and freed before the transformer lands on the device. The fused single-stream to_qkv_mlp_proj joins the attention projections in the LoRA targets; the single-stream out projection stays dense because its to_out suffix would also match the double-stream ModuleList container. Wire both families through the training registry (family set, labels, VRAM notes, rank 16 / lr 1e-4 defaults, bf16-only preflight), mark them trainable with train base repos in the family registry, add FLUX.2-dev to the gated-repo token check, and trust both official bases for training downloads. Verified on B200: 30-step klein int8 (19.6s) and nf4 (20.9s) and dev int8 (52.0s) runs train with finite decreasing loss and the saved adapters apply on the bf16 base pipeline (weight 0 reproduces the base image exactly, weight 1 visibly restyles it). * Support LoRA adapters on torchao int8/fp8 quantized image pipelines Adapters are baked at load time: they attach to the dense transformer, then quantize_ converts only the frozen base linears (the lora_ side path is excluded by name), then the loader compiles. Post-quant PEFT injection is not possible on a manually quantized module, so the prequant shortcut is skipped for a baked load and the memory plan is sized for the dense build (force_dense on the quant candidate). At generation time the baked topology is frozen: weight tweaks and disabling (scale 0 reproduces the quantized base exactly) go through set_adapters, while adding or removing adapters returns a clean 400 telling the client to reload with the new selection. supports_lora now returns True for int8/fp8 diffusers loads (checked before the gguf-kind early return, since the quant fast path keeps the picker kind); nvfp4/mxfp8 and GGUF-via-diffusers stay blocked. The load request model takes an optional loras list, threaded through begin_load on both engines (native ignores it and keeps applying LoRA at generation). Verified end to end on GPU: Z-Image GGUF picker + int8 + trained adapter loads through the API, bake marker logged, weight 1.0 vs 0 renders differ visibly, weight 0.5 accepted live, unknown adapter rejected as 400. Affected suites: 296 passed. * Add FLUX.1 Krea dev to the image model catalog Krea's guidance-distilled FLUX.1-dev finetune keeps the exact dev layout, so it runs under the existing flux.1 family unchanged. Wire it up end to end: - Catalog group with the gated official bf16 pipeline and the open QuantStack GGUF quants; the gated artifact is skipped on auto-routing when undownloaded. - Trust the official repo for non-GGUF from_pretrained loads, next to the other black-forest-labs bases. - Generation defaults: 28 steps at guidance 4.5 per the model card. The generic "krea" defaults key (Krea-2-Turbo's 8-step no-CFG recipe) used to swallow the id, which would have produced garbage output; the new flux.1-krea key precedes it on both the backend table and the images page table. - The flux.1 prequant checkpoints are schnell-based; the loader's baked base_model_id validation refuses them for the Krea-dev base, so int8/fp8 requests dense-quantize instead (covered by existing prequant tests). * Resolve pre-quantized checkpoints per base variant One family entry covers several published variants whose weights differ (flux.1: schnell, dev, Krea-dev), but prequant resolution was keyed on (family, scheme) alone, so only the default base could ever be served: the loader's baked base_model_id validation correctly refused the schnell checkpoint for dev and Krea-dev bases and every such load paid the dense download plus on-the-fly quantise. Add an optional prequant_variant_repos table on DiffusionFamily as (base_repo, scheme, repo_id) triples and thread the resolved base repo through resolve_prequant_source / usable_prequant_source and their three call sites (load fast path, memory-plan probe, auto-policy candidate). A base without its own entry keeps returning the family default, preserving the existing refuse-then-dense behavior exactly. Wire the flux.1 variants: the gate-validated unsloth/FLUX.1-dev-FP8 checkpoints (built in the earlier campaign but never reachable) and the new unsloth/FLUX.1-Krea-dev-FP8. * Add the Lumina Image 2.0 family to the image catalog Alpha-VLLM/Lumina-Image-2.0 is a 2.6B single-stream DiT with a Gemma2-2B encoder and a standard 16-channel VAE, all transformers-4.x-compatible, so the generic from_pretrained pipeline path loads it as a new lumina-2 family: - Family entry (Lumina2Pipeline / Lumina2Transformer2DModel), aliased to lumina-image-2.0 / lumina-image-2 / lumina2. No bare lumina alias: Lumina-Next checkpoints are a different arch and must stay unknown rather than crash mid-load. bf16-only upstream, so the fp16 fallback stays off like z-image. - Trust the official repo for non-GGUF loads; bf16 component table entry (ships fp32, ~5.2 GB transformer + 5.2 GB encoder bf16-resident). - Generation defaults 50 steps / guidance 4.0 per the model card, and the generate call passes the card's cfg_trunc_ratio=0.25 itself (family-gated, signature-gated): the pipeline default (1.0) runs the CFG double-forward on every step and oversaturates output. - Catalog group with the single ungated bf16 pipeline artifact (11 GB resident) plus routing assertions; images page defaults row. - No GGUF artifact: none exists upstream (only finetune/LLM quants), so the dense transformer_quant fast path (GGUF-kind-only) stays unreachable for now. Offline probes of the future prequant campaign: int8 and fp8 both engage and render cleanly (fp8 LPIPS 0.11 vs bf16, int8 0.33 from 50-step trajectory drift with intact quality), so neither scheme is family-denied. * Wire the hosted Lumina Image 2.0 int8/fp8 checkpoints Gate-validated against same-seed bf16 renders (28/28 pairs per scheme, zero failures): int8 LPIPS mean 0.146 / SSIM 0.937, fp8 LPIPS mean 0.116 / SSIM 0.946. Uploaded to unsloth/Lumina-Image-2.0-FP8 following the existing checkpoint repo conventions. * Add the HunyuanImage 2.1 family to the image backend The hunyuanvideo-community diffusers mirror carries the full stack in standard layout: a 17B dual-stream DiT (32.5 GB bf16), a Qwen2.5-VL text encoder, a ByT5 glyph encoder, the 32x HunyuanImage VAE, and guider/ocr_guider components (AdaptiveProjectedMixGuidance) that diffusers 0.39 loads natively, so the generic from_pretrained pipeline path covers everything with no per-component assembly. Family notes: - The call's guidance knob is distilled_guidance_scale (there is no guidance_scale kwarg), so cfg_kwarg routes the UI value there; real CFG runs inside the repo's guider at its baked scale. Defaults follow the card recipe: 50 steps, 3.25. - 2K-native: verified live at both 1024 and 2048. - Coexists with the HunyuanImage-3.0 structured exclusion (3.0 has no diffusers pipeline and stays excluded with its stated reason). - int8/fp8 dense quantization verified live (LPIPS 0.186 both vs same-seed bf16); a short prompt does not trip the int8 torch._int_mm minimum on this arch, so no family exclude entry is needed. - bf16 component table for the memory planner: (32.5, 16.3, 0.8) GB. * Surface HunyuanImage 2.1 in the image model catalog Catalog group with the open bf16 mirror pipeline (~50 GB resident, so a bare click on a consumer card routes to the QuantStack GGUF quants, which load and render through the generic GGUF path, verified live) plus the images page defaults (50 steps, guidance 3.25 feeding distilled_guidance_scale). * Add the HiDream-I1 family to the image backend A 17B MoE DiT (16 double + 32 single layers, 4 routed experts) with four text encoders, on HiDreamImagePipeline (diffusers 0.39). One family covers the open Full / Dev / Fast repos (same arch); per-variant generation defaults follow the upstream inference recipes (Full 50 steps at guidance 5, the distilled Dev 28 and Fast 16 guidance-free). The repos name a Llama-3.1-8B text_encoder_4 in their model_index but do not ship its weights; the official example passes the gated meta-llama repo in by hand. The loader instead assembles the component from the open unsloth mirror (byte-identical weights, already inside the non-GGUF trust gate), injected at the three pipeline from_pretrained sites, with output_hidden_states matching the official example. Memory planning counts the assembled TE4: 34.2 GB DiT + 28.8 GB encoders, ~63 GB bf16-resident. * Surface HiDream I1 in the image model catalog One catalog group with the three official bf16 pipelines (Full, plus the Dev and Fast distillations as labeled artifacts) at their ~63 GB resident size, so auto-routing keeps this a datacenter-GPU pick. city96's GGUF is deliberately not wired: the GGUF path would need the same Llama TE4 assembly for very small demand. Images-page defaults mirror the backend table with the variant keys ahead of the generic hidream key. * Pin the measured HiDream quant verdict in tests int8 and fp8 both engage and render cleanly on this family, including short prompts on int8: the routed MoE expert Linears only ever see the concatenated image+text stream (M far above the torch._int_mm minimum), so no deny entry and no family exclude tokens are warranted. Assert that so a future table edit cannot silently regress the measured behavior. * Wire the hosted HunyuanImage 2.1 int8/fp8 checkpoints Verified bit-identical to on-the-fly quantize: all 1264 state dict tensors (456 quantized) dequantize equal between the loaded checkpoint and a fresh quantize_ pass, so quality matches the runtime Dtype path exactly. Same-seed LPIPS suite means (0.35 int8 / 0.28 fp8) blend trajectory divergence with this family's own run-to-run nondeterminism (identical weights and seed reproduce a 17/255 mean pixel delta through the 50-step guider pipeline); per-case hard checks pass and the drift is compositional, reviewed visually. Uploaded to unsloth/HunyuanImage-2.1-FP8. * Fix silent LoRA drop and wasted transformer prefetch on GGUF quant loads Two live-test findings on the images load path: - transformer_quant with baked LoRAs, when the dense quantized build is declined for memory or fails: the load completed as a plain GGUF with the adapters silently dropped (HTTP success, supports_lora=false after the fact) -- wrong output with no signal. The load now fails with the recovery options (drop the adapters, free VRAM, or pick a smaller model). Weight-0 adapters still count as no bake request, and the plain no-LoRA decline keeps its silent GGUF fallback. - A fresh GGUF load on a small GPU prefetched the base repo's full bf16 transformer shards (~47 GB on Qwen-Image) because the dense-quant prefetch widening only checked scheme viability, not whether the device could ever hold the candidate resident. Gate the widening on total device capacity (reserve + 0.85 margin, the plan_fits_total_capacity bar) so a card that is certain to decline the dense build never pays the download; capable devices keep the prefetch. * Fix video progress under-reporting during load and generate Two live-test findings on the video progress endpoints: - load-progress downloaded_bytes froze mid-download: the counter used scan_cache_dir, which skips in-flight *.incomplete blobs, so it sat at the last completed blob for the whole multi-GB shard pull while the disk kept filling. Count the repo's cache directory directly (completed plus incomplete blobs, snapshot symlinks skipped so nothing is double-counted). - generate-progress reported total_steps=null / fraction=0 while step advanced: the video API only carried the native total field while the image API exposes total_steps and fraction, so one poller could not work against both. Derive the image-compatible aliases in generate_progress and declare them on the response model; the native total stays for back-compat. * Wire the hosted HiDream I1 int8/fp8 checkpoints Gate-validated: all 28 per-case pairs pass per scheme (LPIPS suite means 0.291 int8 / 0.278 fp8, in the 50-step trajectory-divergence band; CLIP delta means 0.007-0.008), and the int8 checkpoint is verified bit-identical to on-the-fly quantize across all 1615 state dict tensors (1073 quantized, max abs diff 0.0). Uploaded to unsloth/HiDream-I1-Full-FP8. * Add a pre-cast text-encoder loader for the layerwise fp8 scheme The runtime text_encoder_quant=fp8 path downloads the full bf16 text encoder and layerwise-casts it in place on every fresh load. For the heavyweight encoders (LTX's Gemma3-27B ~50 GB, FLUX.2-dev's Mistral-24B ~48 GB, Qwen-Image's Qwen2.5-VL ~16.6 GB) that download dominates load time on a fresh machine. diffusion_te_prequant.py loads a pre-cast fp8-storage state dict instead: meta-init the encoder skeleton from the checkpoint's te_class, load_state_dict(assign=True), rebuild on CPU if non-persistent buffers stay on meta, then re-apply the same layerwise cast to install the upcast hooks. The cast is a deterministic storage transform, so the loaded encoder is bit-identical to dense-load-then-cast by construction. v1 hosts the layerwise fp8 storage scheme only: its state dict is plain tensors (torch.load(weights_only=True), no pickle execution). The dynamic-compute schemes (fp8_dynamic, int8, nvfp4) build torchao subclass wrappers at runtime and are deliberately not hosted. Checkpoints validate format, scheme, component and base_model_id before use and any problem falls back to the dense download and cast. Local path overrides reuse the DiT prequant allowlist env var. Families opt in via a new te_prequant_repos (scheme, component, repo_id) field on both DiffusionFamily and VideoFamily; the field defaults empty so nothing changes until a gate-validated artifact is wired. * Inject hosted pre-cast text encoders during pipeline assembly Wire te_prequant_pipe_kwargs into the three pipeline assembly sites: the diffusion full-pipeline branch, the diffusion transformer-only and GGUF branch (where the companion TE is the big remaining download), and the shared video assembly path before the pipeline/component split. Injection is gated exactly like the runtime cast (mode normalized to fp8, device supported, family not denied), so it can never engage where quantize_text_encoders would not; the later quantize_text_encoders call re-applies the cast idempotently and keeps status reporting truthful. With no hosted checkpoint configured the call returns {} and assembly loads the dense encoder as before. * Add the pre-cast text-encoder checkpoint builder Applies the runtime layerwise fp8 storage cast to a model's dense text encoder once and saves the cast state dict with baked metadata (format tag, base_model_id, family, scheme, component, te_class, versions) in the layout diffusion_te_prequant.py validates. Resolves the encoder class from the checkpoint's config.architectures so the recorded te_class matches what the pipeline instantiates. CPU-runnable: the cast touches storage dtypes only. * Test the pre-cast text-encoder load path Hermetic CPU coverage for diffusion_te_prequant: the checkpoint filename convention, family-table resolution by scheme and component with malformed entries skipped, resolution priority (path override, hosted repo, none) and the fp8-only scheme gate, the checkpoint validation matrix (wrong format, missing state_dict, wrong scheme, wrong component, wrong or missing base_model_id) with base case folding, the local-path allowlist refusal and missing-file fallback, and the assembly injection gating (mode, hosted entry, device support, family deny, load failure, successful injection). Also pins the te_prequant_repos field on both family dataclasses and that no family ships a hosted TE checkpoint until the campaign wires one. * Fix pre-cast TE checkpoint loading and engagement reporting Two bugs found while building the hosted checkpoints: - The builder recorded torch.__version__ (a TorchVersion object) in the checkpoint metadata, so torch.load(weights_only=True) rejected every artifact and the loader silently fell back to the dense download. Record plain strings. - Re-applying the layerwise fp8 cast to an injected pre-cast encoder raised on the duplicate hook registration, making quantize_text_encoders report the engaged cast as failed (status showed no TE quant while the encoder ran fp8). _cast_fp8 now returns early when the hooks are already installed. Also corrects the LTX TE size note: Gemma3-12B stored fp32 (~49 GB), not 27B. * Wire the hosted pre-cast fp8 text encoders qwen-image and flux.2-dev (diffusion) and ltx-2 (video) now resolve a hosted pre-cast fp8 text encoder from their unsloth -FP8 repos: - unsloth/Qwen-Image-FP8: Qwen2.5-VL-7B, 16.6 GB dense -> 8.8 GB - unsloth/FLUX.2-dev-FP8: Mistral-Small-24B, 48.0 GB dense -> 24.7 GB - unsloth/LTX-2-FP8: Gemma3-12B, 48.7 GB fp32 store -> 13.2 GB Every checkpoint verified bit-identical to dense-load-then-cast (729 / 585 / 1066 tensors, zero mismatches) and smoke-tested through the real backends with the repo engagement marker. Tests cover the wired entries, the resolver filenames, builder metadata weights_only survival, and the idempotent re-cast. * Report the compute dtype on fp8-cast encoders and inject the pre-cast TE on the dense fast path Two more findings from the hosted-TE GPU smokes: - Module.dtype reports the first floating parameter, which after the layerwise fp8 cast is the fp8 STORAGE dtype. Flux2 derives its prompt embed and latent dtypes from encoder.dtype and feeds them to randn_tensor, which has no fp8 kernel, so ANY flux.2 load with text_encoder_quant=fp8 crashed at generation (pre-existing, runtime cast included). The cast now swaps in a subclass whose dtype property reports the compute dtype; forward behaviour is unchanged. - The dense transformer_quant fast path assembles companions through _assemble_pipe, which never received the pre-cast TE injection, so the hosted encoder engaged on full-pipeline and GGUF builds but not on the fast path. Threaded through like the other two branches. Verified live on B200: qwen-image (full pipeline), flux.2-dev (GGUF picker with int8 DiT prequant), ltx-2 (video backend) all engage the hosted TE, render non-black, and report text_encoder_quant=fp8 truthfully. * Key the fp8 cast idempotency on an explicit completion marker Hook presence alone cannot distinguish a legitimately pre-cast text encoder from leftover hooks after a cast that failed mid-pass, so the early return now requires the completion marker _cast_fp8 sets once the hooks are fully installed. Leftover partial state keeps failing closed. Also tolerates non-Module encoder doubles in the hook probe and the dtype override. * Extend the fp8 TE quant to HiDream's Llama text_encoder_4 The generic quantize_text_encoders pass only covers text_encoder.._3, so HiDream's HEAVIEST encoder (Llama-3.1-8B TE4, 16.1 GB bf16) always stayed dense. TE4 is assembled separately (hidream_te4_kwargs), so the fp8 path now lives there: when the requested TE quant is layerwise fp8 and the device/family qualify, TE4 prefers the hosted pre-cast checkpoint (unsloth/HiDream-I1-Full-FP8, 8.6 GB) and falls back to dense-load-then- cast; a mid-pass cast failure reloads a fresh dense encoder instead of shipping partial state. The pre-cast loader and builder gain config_subfolder/config_overrides for standalone encoder repos whose config sits at the root and whose pipeline needs forward flags (output_hidden_states/attentions). Verified on B200: bit-identity 291 tensors (225 fp8, 0 mismatches), hosted checkpoint engages through the real backend (marker + status fp8), load 24.3 s vs 48.0 s dense, LPIPS 0.133 mean over 3 same-seed pairs vs the dense-TE render (gate 0.25), non-black frames. * Correct the ltx-2 resident TE estimate to the bf16 cast size The memory plan's bf16_components_gb held 50.4 GB for the LTX text encoder, which is the fp32 hub store of Gemma3-12B (~49 GB download), not what sits on device: the pipeline loads it torch_dtype=bf16, ~24.4 GB resident. The 26 GB over-estimate pushed the auto plan toward offload on cards that fit the real footprint. Comments and the size-table test now pin the resident semantics. * Host pre-cast fp8 text encoders for four more families Round 2 of the hosted TE set, each bit-identical to dense-load-then-cast and gated through the real backend (marker + status fp8 + same-seed LPIPS vs dense TEs): - FLUX.1 T5-XXL (text_encoder_2): 9.52 -> 5.90 GB, one artifact for schnell/dev/Krea-dev (T5 shards byte-identical across all three, verified sha256). 220 tensors, 144 fp8, LPIPS 0.109. - Lumina Gemma2-2B: fp32 hub store 10.46 -> 3.20 GB (3.3x download cut). 288 tensors, 182 fp8, LPIPS 0.041. - Z-Image Qwen3-4B: 8.04 -> 4.41 GB. 399 tensors, 252 fp8, LPIPS 0.112. NOT shared with flux.2-klein-4B: klein retrained layer 35's MLP (verified tensor diff, maxdiff 0.86), so klein hosts no entry. - Krea-2 Qwen3-VL-4B: 8.88 -> 4.83 GB. 713 tensors, 460 fp8, LPIPS 0.082. The constructor-assembled krea pipeline takes the encoder directly (load_krea2_pipeline text_encoder kwarg); the loader remaps 5.x rope_parameters and re-ties weights after assign so the rebuilt encoder matches the builder's structure. HunyuanImage 2.1 reuses the Qwen-Image artifact outright: its Qwen2.5-VL text encoder is byte-identical (every shard sha256, 16,584,414,544 bytes), recorded in the new component-level base-equivalence table the checkpoint validator consults. The injection loop now covers text_encoder.._3 so a family can host several components. Live check: LPIPS 0.123 vs dense. * Report the fp8-cast compute dtype without swapping the encoder class The dtype override swapped encoder.__class__ to a dynamic subclass, which breaks transformers' kwargs-based output recording: a fp8-cast Qwen3VLModel stopped returning hidden_states and every krea-2 generation with text_encoder_quant=fp8 crashed at encode_prompt (regression from the HiDream TE4 change; caught by the krea hosted-TE live smoke). The override is now a property shadowed on the ORIGINAL class that prefers a per-instance compute-dtype attribute, so class identity is preserved and uncast instances keep the stock behaviour. The idempotency test now pins exact class identity and the uncast-sibling fallback. * Pass the calibrated distilled sigma curve to LTX-2.3 8-step runs The 22B distilled DiT was trained against ltx_core's fixed DISTILLED_SIGMA_VALUES, but the diffusers scheduler derives 8-step spacing from resolution-shifted flow matching and lands far off at every reachable mu (second sigma 0.945-0.981 vs 0.99375, tail 0.37-0.61 -> 0.1 vs 0.725 -> 0.42 -> 0). At the distilled default step count the backend now passes the list verbatim, neutralising the scheduler's dynamic shift and terminal stretch for the call (they distort even explicit sigmas) and restoring them afterwards. Other step counts and the dev/base DiT keep the scheduler's own spacing. Live-verified on B200 through the video branch backend: the scheduler holds the exact curve after an 8-step distilled GGUF generation, config restored, healthy clip. Also reword the transformer_quant resolved reason to the measured reality: quant halves resident weights and hosted checkpoints cut load time, while per-step speed is roughly bf16 parity. * Pin the fp8 weight-quantize kernel against silent MSLK switching torchao's Float8Tensor KernelPreference defaults to AUTO, which switches the weight-quantize kernel to MSLK whenever an mslk package is importable on sm90+. Measured on B200: that changes fp8 scale rounding bitwise (8/8 FLUX matrices differ, scales ~55 percent of bytes), so a box that merely gains mslk would break the hosted-prequant bit-identity invariant; the mslk path is also slower under torch.compile (opaque extern call blocks inductor's quantize fusion, FLUX.1 fp8 e2e 1.149 to 1.624 s). Pin KernelPreference.TORCH explicitly, matching current no-mslk behaviour bit for bit; signature-gated for older torchao. GPU-smoked (finite, rel err 0.037) and pinned by test. * Shift Qwen-Image training sigmas to the inference distribution Qwen-Image's scheduler skips its static shift under use_dynamic_shifting, so the DiT trainer was drawing UNSHIFTED uniform-schedule sigmas for it (mean sigma 0.50) while inference always runs the exponential mu = log 3 shift plus the shift_terminal 0.02 stretch. Add a flow_shift config lever: "auto" (the new qwen-image default) rebuilds the training sigma table through the scheduler's own time_shift and stretch_shift_to_terminal so the draw matches the inference distribution exactly (mean sigma 0.72); a numeric value applies the standard linear shift s*u/(1+(s-1)*u); 1.0 keeps the historical identity behavior and stays the default for FLUX, Z-Image and Krea 2. The model timestep conditioning follows the shifted sigma, gathered in fp32 so bf16 rounding never skews it. Also wire two opt-in levers with off defaults: cfg_dropout (per-sample empty-prompt conditioning dropout, encoded alongside the captions before the text encoders are freed) and weighting_scheme="bell" (bsmntw-style mid-schedule Gaussian loss weighting normalized to mean 1). Verified with two 80-step rank-8 bf16 LoRA runs on Qwen/Qwen-Image (identity vs auto, same seed): both converge with finite decreasing loss and produce coherent same-seed previews. Unit tests cover the exact transform, the shifted sampling distribution, per-family defaults and config plumbing. * Add LoRA EMA, a persistent conditioning cache, and aspect bucketing helpers diffusion_train_extras hosts the opt-in training extras: LoRAEMA shadows only the trainable adapter params (warmup-ramped decay, default 0.99, exported as a second adapter under output_dir/ema), PersistentConditioningCache stores latent posterior stats and caption embeddings as safetensors keyed by content hash + family + resolution, and the aspect-ratio bucketing helpers group mixed-aspect datasets into same-area divisor-snapped shapes. The DiT trainer wires the first two behind config flags that default to the current behavior: ema_decay (0 disables) and cond_cache_dir (None disables). A fully warm cache skips loading the VAE and text encoders entirely; a cache hit is bit-identical to a fresh encode, including the per-channel qwen latent normalization. Also fixes the stale _gather_sigmas call in the perf test that still passed the scheduler instead of the sigma table. * Tighten torchao configs and note the FSDP2 design for the DiT trainer nf4 loads now enable double quantization (~0.4 bits/param off the frozen base scales at no fidelity cost), fp8 training uses the rowwise recipe when the torchao build ships it (per-row scaling confines the DiT activation outliers that a tensor-wide scale collapses), and the inference quant filter gains a per-scheme GEMM-tiling divisibility floor (16 for scaled_mm, 32 for MX blocks) so one ragged Linear cannot crash the first denoise after a clean quantize pass. plans/fsdp2_diffusion_design.md records the multi-GPU design: bf16/fp8 over FSDP2 with per-block units, LoRA attached before sharding, int8 out of scope (DTensor over the quantized subclass is undefined), per-family notes. * Batch diffusion inference with per-image seeds, an inference conditioning cache, and GGUF loader fixes Batched generation: /images/generate takes a prompts list (one image per prompt, txt2img only) or a seeds list (one prompt, one image per seed); the legacy batch_size path derives per-image seeds base..base+n-1 like the native engine. Every image gets its own torch.Generator so any batch member replays alone from its gallery recipe; the whole list runs as one forward by default with OOM backoff that halves a failed chunk, and an explicit batch_size caps images per forward. Validated 10-22x over serial engines on 32-image suites with LPIPS deltas within 0.002. Conditioning cache on the inference path: UNSLOTH_DIFFUSION_COND_CACHE_DIR (the inference sibling of the trainers' cond_cache_dir, same persistent store) wraps encode_prompt so repeated prompts skip the text-encoder forward entirely; verified bit-identical outputs. Bypassed while LoRA adapters are attached; tensor-argument calls pass through uncached. Compile cache: GGUF loads fingerprint their own bundles (quant=gguf, a different compiled graph than the dense family) and batched calls register every distinct (w, h, batch) chunk shape they ran, so the heavy GGUF batched warmups (~159 s at batch 32 on 12B-class, ~655 s on 20B CFG-batched) are paid once ever. GGUF loader: strip the sd.cpp model.diffusion_model. container prefix in the single-file converter; diffusers' FLUX.2 converter KeyErrors on it and the Qwen-Image identity mapping strands the model on meta. * Correct batched seed-replay docs to match measured behavior Same-seed images at the same batch shape are bit-identical; a solo regeneration with the recorded seed matches its batched rendition up to batch-size-dependent kernel numerics (mean abs pixel delta about 2.5/255, LPIPS delta under 0.002), not bit-exactly. The previous wording overclaimed bit-identity across batch shapes. * Note that batched bit-identity assumes a settled compiled graph The first generation issued while the deferred compile is still in flight can deviate transiently (observed once on a cold fp8 build: mean abs pixel delta 0.063/255); once the graph is settled, same-seed same-batch-shape images are bit-identical across runs. * Studio sidebar: Image03/FlimSlate icons, More flyout, Train row, New pills - Images uses Image03Icon and Video uses FlimSlateIcon. - New "More" row (MoreHorizontalIcon) opens a right-side flyout on click or hover holding Video, Recipes and Export; the close is delayed 180ms so the pointer can cross the gap. Its SidebarMenuButton deliberately takes `title` rather than `tooltip`: with `tooltip` the button returns a Tooltip root and DropdownMenuTrigger asChild would hand its ref to a non-DOM node. - Dropped the "Train" section heading; Train is now a top-level row between Images and More. data-tour="navbar" moves to the surviving nav group so the product tour keeps its anchor. - "New" pill beside Images and (inside the flyout) Video, via NavBadge. * Studio sidebar: match flyout rows and New pills to the existing scales - More flyout rows dropped their sidebar-row typography and size-icon override, which fought DropdownMenuItem's own scale (text-sm, gap-2.5, px-3 py-2 and size-4 icons) and rendered oversized glyphs and text next to the nav. - New pill reuses the brand "beta" badge recipe (nav-badge font, --ui-font-scale sizing, nav token colours) rather than hardcoded 9px values. - The More row's native title tooltip (an OS box on hover) is replaced by the app's Tooltip, wrapped around DropdownMenuTrigger so both triggers compose onto the same button, and shown only on the collapsed rail like other nav rows. * Settings: pin and reorder the sidebar navigation Adds a "Sidebar navigation" section to Settings -> Appearance, above the existing profile-menu customizer, with the same drag-to-reorder + switch UI. - New sidebarNav preference: one { id, pinned } entry per navigable row (projects, hub, images, train, video, recipes, export), array order = render order. Defaults match the shipped layout, so an untouched install is unchanged. - Unpinning moves a row into the More flyout rather than hiding it, so no page becomes unreachable. New chat and Search stay fixed as actions. - app-sidebar now renders from one navRows descriptor map, so a pinned row and its flyout counterpart cannot drift; the More row appears only when something is unpinned and highlights off whatever it actually holds. - Mirrored in the backend PersonalizationCustomization: without it the model's extra="ignore" would drop the field, and because sync replaces local state with the server's copy once customization is saved, the user's pin order would reset on the next sync. The validator dedupes and back-fills like sidebarMenu but preserves the client's order, since here order is meaningful. Frontend typecheck, i18n parity and catalog checks pass; 32 personalization tests pass, including a round-trip asserting a reordered list survives a save. * Sidebar customizer: drop the Search row, skip More for a lone item - Search is reached from the top bar, so it is no longer previewed as a fixed sidebar nav row; New chat stays. - More now appears only when it would hold two or more rows. A single unpinned row renders inline in its saved order position instead: a flyout wrapping one item costs a click and earns nothing. The customizer's More preview follows the same threshold. * Sidebar settings: hide a lone unpinned tab, match New chat icon, rename Profile menu - With exactly one tab unpinned, both More and that tab are dropped, so nothing is drawn for it (previously it rendered inline). The page stays reachable by URL. - The customizer's New chat preview uses PencilEdit02Icon, the icon the real row renders; Edit03Icon was a different glyph. - "Sidebar menu" is now "Profile menu", described as the shortcuts behind your name at the bottom of the sidebar, so it no longer reads as a second name for the navigation section above it. * Tighten comments in the new sidebar and delete-guard code * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio sidebar: keep the More row highlighted while its panel is open Moving the pointer into the flyout left the row unhighlighted while the panel stayed open. The row now carries data-menu-open, added to the nav hover selector list. Not data-state: the tooltip and menu triggers both write that attribute, so whichever lands last wins. * Images: use the shared pill toggle for Create/Train and pad the panels - Create/Train was the only segmented control on its own Tabs styling. It now uses PillTabs, the same control as the model picker and Hub toggles, pinned to the header row's 34px. PillTabs takes an icon per tab, so the inline-span workaround for TabsTrigger goes away. - pt-3 on both the Create and Train panels, which sat flush against the model selector row. * Images: make the workflow picker a dropdown instead of a 7-up strip Seven workflows in a 340px rail left ~48px each, so the labels crowded and the hints were only reachable as title tooltips. The strip is now a dropdown: the trigger shows the current workflow and its hint, and each row carries its own description. A row the loaded model can't run is disabled and shows the reason in place of the hint, so the gating explains itself. Adding a workflow no longer shrinks the others. * Images: workflow icons, hint under the trigger, more top room, unclipped Train cards - Each workflow carries an icon, shown on the closed trigger and on every row. - The trigger is one line (icon plus name). The selected workflow's description moved below it, where it reads like the Field hints further down the rail. - pt-6 instead of pt-3 on both Create and Train, so the cards clear the model selector row. - The Train right column scrolls while its cards use ring-1, which draws outside the box and was clipped at the scroll edges. p-px gives the ring room. * Images: one-line workflow rows, stronger trigger fill, roomier mode tabs - Dropdown rows are icon plus name only. The selected row's description already shows under the trigger, and a disabled row keeps its reason as a title. - Trigger fill moves to the bg-foreground/[0.07] dark:bg-foreground/[0.12] pair the hub cards use, so it reads against the card in both modes. - Description under the trigger goes from text-ui-10 to text-ui-11p5. - More horizontal padding on the Create / Train tabs. * Images: drop card borders for the composer shadow, keep scrollbars inside, use app controls in Train - Cards lose ring-1 for .panel-soft-surface: the composer's shadow in light, flat in dark, matching .chat-composer-surface and the menus. - Both rails now clip (overflow-hidden) with the scroller inside, so the scrollbar can't ride over the rounded corner. Same shape video-page already uses. - Train's 9 native selects become the app Select, so they no longer open an OS-native menu, and the native file input is hidden behind a Choose images button that reports the count. - Image previews use explicit 8-10px radii: this theme sets --radius to 1.1rem, so rounded-md was 15.6px and the thumbnails read as circles. * Images: one card for controls and preview, chat sliders, wider softer shadow - Controls and preview were two floating cards; they now share one card split by a divider. The Advanced dock stays separate since it toggles. - SliderField wraps Chat's ParamSlider, so the sliders match Chat (label row with the value, full-width neutral track) instead of a green track with a spin box. All 14 call sites keep their props. - panel-soft-surface goes from 0 2px 8px -2px /0.16 to 0 4px 22px -6px /0.10: lighter, spread wider. * Images: flat Create and Train panes, hover-only scrollbars, tidier Train dataset step Both Images tabs now sit on the page background like the Hub: no card, no shadow, no bounding box. A single rule divides the controls rail from the preview canvas (Create) and from the run area (Train), and the settings and previous-runs sections read as panes rather than nested cards. Also: - Scrollbars in these panes use the existing hover-scrollbar recipe, so the thumb only shows while the pane is hovered. - Workflow rows explain themselves with a tooltip after a short hover, which also works on disabled rows, and the descriptions are much shorter. - Training images rows are name plus image count; the license stays on the example card. - The upload step loses its dashed box, the buttons match the sizes around them, and Upload only appears once files are picked. - The empty preview uses the same icon as the Images nav item. * Images: full-height panes, wider settings rail, Create/Train offset from the selector The rule between the panes now runs the whole page height (the row drops its bottom padding and each pane pads its own content), the settings rail is wider on both Create and Train, and the Create/Train switch sits further right of the model selector. * Images: put both tabs on the Hub's centered measure Top bar and content now share mx-auto max-w-1100 with px-5 / sm:px-8, so Create and Train sit at the same width and position as the Hub instead of running edge to edge. * Images: restore the top bar position, drop the panes lower under it * Images: center the mode switch, flip the arrow with the orientation, app tooltips everywhere The Create/Train switch is centered on the page instead of trailing the model selector, with wider buttons. The flip control's arrows now rotate with the orientation and its label says which way the flip goes. Every native title tooltip on the page is now the app's tooltip, so they all get the rounded surface instead of the OS box. * Images Train: plainer field text, no green buttons, columns that stop colliding - The dataset name, trigger prompt, adapter name and custom base fields now say what they are in plain words instead of leaning on example values. - Import, Upload, Back, Back to settings and Train another are outline buttons, not green ones. - Example thumbnails are landscape tiles, so photos are not cropped to chunky squares. - Settings cells get min-w-0 and the select value truncates, so a long option like the nf4 label no longer widens its column into the next one. - The number stepper sits a little further in from the field edge. - Create and Train are wider. * Images Train: roomier example cards with Import on the thumbnail row * Video: same treatment as the Images tabs - No cards: the rail and the canvas sit on the page background, divided by a rule that runs the full page height, on the Hub's centered measure. - Wider rail, chat's sliders, hover-only scrollbars. - Every native title tooltip is now the app's tooltip, including the clip cards. - Reapply and Cancel are outline buttons, the empty state uses the Video nav icon, and the clip tiles are less rounded. * Images and Video: narrower generation rail, matching Train headings Create and Video rails go from 392px to 368px. Train a LoRA and Training settings are now the same size and both in the heading font: the h2 already picks it up from the base rule, so the settings header opts in with font-heading and the weight that rule pins. * Images Train: shorter copy throughout Family notes, example descriptions, precision labels and every helper line are trimmed so they stop wrapping to three lines and colliding with the next column. The nf4 label now fits its select without truncating. * Images Train: a little more spacing between field groups * Images and Video: tighten code comments * Fix training start NameError, the load-order guard test and CPU-only diffusion tests - start_training forwards resume_source_run_id to _start_training_impl, which reads it. Without it every start raised NameError. - Restore main's anchor in the load-marker order test: the file now has an earlier `if config.is_gguf:`, so indexing the first one compared the wrong branch. - The two diffusion tests that reach diffusers now skip when it is absent, matching the CPU repo-test env. - The UI smoke finds nav rows that live in the sidebar's More flyout. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Treat a null metadata caption as no caption str(None) stored the literal "None" as the caption, so a null row counted as captioned and would have trained on that text. Also drops an unused import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix invalid-UTF-8 500s, the flat Canny map and the dropped DiT knobs read_text raises UnicodeDecodeError, which is not an OSError, so one bad caption sidecar or video sidecar 500d the info, upload and gallery routes. A flat image now yields the all-black edge map instead of its own luminance, and the four DiT loss knobs the trainer implements are declared so model_dump keeps them. * Use the ui font-size tokens instead of raw px text utilities text-[11px] and friends ignore the UI font size preference, which the repo's font-scale contract test enforces. Same rendered size at the default scale. * Fix diffusion dataset 500s, the dropout-1.0 no-op run and the reset base pick Four correctness fixes on the training side: - The labeling grid read caption sidecars under except OSError, but a non-UTF-8 sidecar raises UnicodeDecodeError (a ValueError), so one bad file 500d /diffusion/dataset/{name}/images and the grid could not be opened to repair it. Read it as no caption, matching the info summary. - An image past Pillow's own hard limit raises DecompressionBombError, which derives straight from Exception and so escaped the upload guard's (OSError, UnidentifiedImageError, ValueError) and returned 500 instead of the intended 400. - lora_dropout accepted 1.0, which makes PEFT build nn.Dropout(p=1.0): lora_A and lora_B receive no gradient and the run saves an untrained adapter while reporting normal progress. Bound it below 1.0, matching the LLM request schema. - The train panel re-seeded the base repo on every dataset refresh because the family object identity changes on each info fetch, so an upload or caption save silently replaced the user's chosen base and the run started on a different model. Track the pick and only re-seed on a real family change. * Show the retained failure when a video page mounts after a failed job Mount-time recovery handled only phase=completed, so reloading the page after a multi-minute generation failed left an idle view with no diagnosis: the backend keeps the terminal failed record only until the next job, and nothing else survives the reload. Surface it the same way the poll does, filtering the cancelled sentinel. * Fix batched generation crashes, cache keying and unreplayable recipes Four bugs in the batched inference path, all found by review: - A mixed-prompt batch sent a scalar negative prompt against a prompt list. Z-Image asserts on the length, and Qwen-Image, Krea 2 and FLUX true-CFG encode a batch-1 negative against batch-N latents and fail in the transformer's text/image concat. Broadcast it to match the batch. - The FBCache step-cache reset sat above the chunk loop. diffusers only resets that state at the end of a successful call, so a forward that raised (the OOM the backoff is meant to recover) left its own residual behind and the halved retry died on a shape mismatch. Reset before every forward instead. - The conditioning cache keyed on the checkpoint alone, but a GGUF or single-file load takes its text encoders from the companion base, so the same checkpoint against a different base reused the previous base's embeddings. Key the base too. - Gallery records stored the base seed and the requested batch size even when a prompts/seeds list drove the run, so restoring the second image of seeds=[5, 99] replayed seed 5. List-driven outputs now record as single-image recipes on their own seed. Also bound strength above 0: every img2img pipeline derives its step count from it, so 0 leaves zero denoising steps and either raises or, on SDXL, crashes on empty latents. * Fix quantized-load LoRA bake, prequant family exclusions and outpaint canvas Six review findings across the Images page and model scanning: - The quantized (int8/fp8) load path can only attach LoRA adapters before quantization, but the frontend load request had no loras field, so every generation after such a load was rejected and each reload repeated it. Send the selection with the load. - build_prequant_checkpoint passed no family to the scheme exclusions while recording the family in metadata, so a Qwen int8 artifact baked the short-M text-stream linears and was then rejected wholesale by the loader's family-keyed check. - Registering a bare single-file checkpoint directory produced no On Device row even though the images loader can load it; only its parent worked. Admit that shape when nothing else matched. - Unload left the Reapply target set, so the repair path was skipped and Reapply reloaded the ejected model. Clear it, as the video page does. - Both FLUX.2 bases were trusted for training but not inference, so Deploy to Create rejected every FLUX.2 adapter. - Outpaint allocated the grown canvas before downscaling, exceeding the browser canvas area cap on a large photo; an over-cap canvas is unusable, so Extend silently posted a fully transparent image and mask. Scale the source first. * Send the picked GGUF filename with the quant so diffusion loads fire The variant expander emitted only the quant label, and nothing else in the frontend set ggufFilename, so the Images and Video pages could never take their GGUF branch: both gate it on meta.ggufVariant and meta.ggufFilename, then fall through to the single-file path, which returns because the id is a repo id and not a .gguf name. Every quant pick was a silent dead click, with no load request reaching the backend. The filename was already on the variant row (the picker keys its list on it, and the variant validator requires a non-empty string), so thread it through the click handler. The chat path is unaffected: it reads ggufVariant and never needed the filename. * Version the conditioning cache key and reject non-finite flow_shift Two correctness fixes: - The cache keyed the checkpoint and its companion base by name only, so a Hub repo advancing to a new commit, or a local directory updated in place, kept returning embeddings from the previous text encoder. Pair both with a revision marker: the locally resolved commit sha for a Hub repo, config plus text-encoder file stats for a directory. Neither loads the encoders, so a warm run still keeps them off the GPU. - flow_shift only checked positivity, but JSON accepts 1e309, which floats to inf, and inf <= 0 is False while NaN fails every comparison. The sigma table then evaluates s * u / (1 + (s - 1) * u) as NaN, which poisons every sampled sigma and saves a corrupted adapter while progress looks normal. Require a finite value. * Keep curated models listed, guard the video companion repo, pin diffusers Three review findings: - The picker filtered every catalog member out of Recommended and Hub search on the way to canonical group rows, but nothing renders those rows yet (catalogGroupFitsDevice and groupMatchesQuery are imported and unused). A task-scoped picker's models list is catalogToModelOptions(), i.e. group members exclusively, so both lists came back empty and no curated model could be discovered or downloaded. Keep the artifacts listed until the grouped UI exists. - The video delete guard compared only repo_id, so deleting the companion base of a loaded GGUF video model was allowed even though it supplies the VAE and text encoders. Compare base_repo too, matching what the images guard already does for its companions. - diffusers was declared unversioned while the diffusion stack requires 0.39 (Krea2Pipeline, the cache_context child registries, the Flux2 and Z-Image pipelines), so an upgrade could keep an older release and selecting an advertised model failed until the user upgraded by hand. * Namespace the trainer conditioning cache per checkpoint, bound the learning rate - The trainer keyed its persistent conditioning cache on family and resolution only, while the keys themselves carry just the caption or image content and crop variant. One cache directory reused for two checkpoints, or for the same repo at a new revision, let a warm run skip loading its encoders and train on the other model's embeddings and latent statistics. Namespace on the base checkpoint and its resolved revision as well. The revision helper now lives beside the cache in diffusion_train_extras and the inference wrapper delegates to it, so the two cannot disagree about what counts as the same source. - The diffusion learning rate only checked positivity, but 1e309 floats to inf and satisfies gt, so the route evicted the resident models and started AdamW with an infinite rate: the first step destroys the adapter while progress looks normal and the result is saved. Bound it below 1.0, matching the LLM schema, which rejects inf for the same reason. * Fix GGUF image model picks doing nothing, and pick the train base in the top bar The quant rows never forwarded the .gguf filename, so every hub GGUF pick on Images/Video fell through to a silent return. On Train the top bar now picks the training base instead of a generation model, which is GGUF-only and untrainable. * Pin diffusion and video loads to the live HF cache root Both read huggingface_hub's import-time HF_HUB_CACHE, which changing the cache folder does not update: progress counted the old root while the download wrote to the new one, and from_pretrained could split one model across both. * Add the diffusion download plan endpoint Reports the repos and exact files a pick needs so the download manager can stage them with the loader's own file scope. A plain snapshot would add the packaged root single, transformer shards and fp16 twins the loader never opens. * Add a file-scoped flavour to the Hub download job Lets a consumer that reads a deliberate subset of a repo stage it through the normal download manager. Keyed as "@scope" so it never collides with a quant or with the repo's full snapshot, and the file list rides the registry so an XET to HTTP retry respawns the same scoped job. * Stage image and video downloads through the Hub download manager They downloaded inline inside the load, so they had none of the manager's disk preflight, manifest verification, resume or panel progress. Picks now stage as scoped jobs carrying the loader's own file list, then load from a warm cache. * Fetch staged GGUF checkpoints as scoped jobs, and stop calling diffusion models unsupported A GGUF entry went out as a full snapshot, whose ignore list drops *.gguf: the job finished at once having fetched only docs, and the repo landed on device unloadable. Every entry is scoped now. The Hub also no longer tags image/video models as unsupported (they run on their own pages), and those pickers name what they select. * Apply the picker task filter to local model sections LM Studio, ./models and custom-folder rows ignored it, so the Images picker listed chat GGUFs that 400 on a diffusion load. The backend already tags every local model with a task for this purpose. * Route a chat pick of a diffusion model to the Images or Video page Chat cannot load one, so it was either hidden or failed on load. The unfiltered picker now lists on-device diffusion models and navigates to the page that runs them, passing the repo and quant so that page loads it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Route the real GGUF filename, keep non-GGUF curated models, key scoped downloads by file set Five review findings, four of them ways a click did nothing or fetched the wrong thing: - A chat pick of a diffusion model routed ggufVariant (a label like Q4_K_M) in the search param the target page uses verbatim as the GGUF filename, so the load asked for a file that does not exist. Route ggufFilename; no filename means a curated non-GGUF pick, loaded as a pipeline. - The task-scoped pickers kept only GGUF repos, so the catalog's bf16, bnb-4bit and single-file fp8 artifacts could not be discovered or downloaded on the Images and Video pages even though loadSpecFor knows how to load them. Keep curated artifacts whatever their format, in Recommended and in Hub search. - Both pages deduplicated routed selections on the model alone, and they now stay mounted, so picking the same repo again -- another quant, or the same one after chat evicted it -- returned early without loading or clearing the query string. Key on model and quant. - Every scoped image download shared one @diffusion job key regardless of the requested files, so switching quant mid-download adopted the running job: the UI waited on the first file set, then loaded a file that was never fetched. Include a digest of the file set in the key. - A scoped plan silently dropped requested files missing from Hub metadata, and snapshot_download succeeds when an allow pattern matches nothing, so the job reported completion and triggered a load with required files absent. Fail the job instead. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the scoped download key derivable, and stop the hidden page hijacking a route Four review findings, the first a regression from my own last commit: - Keying scoped download jobs by a digest of the file set broke the download manager: it builds that key client-side (it polls and cancels before any response tells it a key), so it watched and cancelled a key no worker owned and never fired its ready callback. Keep the derivable "@scope" key and refuse the second request instead when a live job on the slot is fetching a different file set -- decided inside the registry claim, under the lock, so a concurrent claim cannot slip past it. The manager records the file set on the job as well, so a sibling quant's transfer is not adopted locally either. - Both diffusion pages read the route query through a loose useSearch and both stay mounted once visited, so the hidden one consumed the other's ?model=: it navigated back to its own route and tried to load, say, an image checkpoint as a video model. Only the visible page consumes it. - The staged download plan was built without the configured HF token or the Advanced values the load itself sends. The token matters most: the backend's Hub metadata lookup is best-effort, so a gated base silently planned no companion entry and the load pulled those multi-GB files inline, outside the manager. The memory/quant controls decide whether the base transformer/ shards are needed at all, and the route dropped memory_mode, cpu_offload, the prequant path and the LoRA selection before asking for the plan. - The video preview kept playing after leaving the page: the keep-alive layout only hides it, and display:none does not pause a media element, so a clip the user unmuted kept its audio going over the next page. Pause on the active transition and do not auto-replay while hidden. Also completes the hand-built request bodies in the hub download tests: the scoped-files field this branch added to the route read as an AttributeError against them, failing five tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Serialize the GPU handoffs, gate DiT training on a GPU, and keep 3.9 installable Six review findings, three of them evict-then-fail orderings: - The chat load reclaimed the GPU without telling the arbiter it existed. A chat load holds no llama-server process until its GGUF has downloaded, which is minutes, so a competing Images/Video acquire in that window found nothing to cancel, took the GPU, and the chat load then spawned onto the same device. It now registers an in-flight marker through acquire_for's register hook (under the arbiter lock, as the image and video loads do), the evictor cancels a marked load, and the route undoes itself if ownership moved while it loaded. - The Hub-download conflict check ran after that handoff, so a GGUF the download manager already owns destroyed the resident Images/Video pipeline and then 409'd, having loaded nothing. It moves above the handoff, together with the marker it handshakes with. - The image load released the engine router's transition lock before registering the load, so a second load choosing the other engine could unload the still-idle engine this one captured; the load then landed on a deactivated engine, where generate, status, unload and the arbiter's evictor can no longer reach it. Registration now happens under that lock and refuses if the engine changed. - Training a DiT family on a host with no GPU was accepted: nf4 is not a CPU fallback, its 4-bit load goes through bitsandbytes, which requires CUDA, XPU or MPS. The start unloaded the working Images pipeline, pulled the text encoders, and only then died in the child. Rejected before the teardown now, and /info stops advertising a precision that always 400s. SDXL keeps its documented fp32-on-CPU path. - Both diffusion pages kept the routed-pick marker forever, so re-picking the same checkpoint (after chat evicted it) neither loaded nor cleared the query string. The marker is released once the query is gone. The Images key also carried a stray NUL byte, which made the file read as binary to grep and other tooling. - diffusers dropped Python 3.9 in 0.38, so the unconditional >=0.39.0 pin left pip no candidate at all on 3.9 and made every install that composes the huggingface extras unresolvable there. The floor is conditional now. Also fixes tests that were already red on the branch: two hand-built request fakes had gone stale against fields this branch added, and the handoff-ordering test only failed on a host with fewer than two GPUs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the GGUF variant contract test against the merged handler signature The assertion pinned the exact single-line call handleVariantClick(v.quant, v.downloaded, expectedBytes, v.filename), but the handler takes (quant, filename, downloaded, sizeBytes) and prettier wraps the call across lines, so the mandatory repository test job failed on every push. Match the call structurally and assert the filename really is forwarded in the handler's argument order. * Stop a background page and a stale record taking the GPU or a download with them Five fixes from a review pass over the diffusion work. delete-finetuned rmtree'd a model the Images or Video engine was holding: every guard on that route is chat-only, and Images loads any local path, so deleting a local diffusion model under the storage root pulled the weights (and the companion VAE / text encoders sd.cpp re-reads each generation) out from under a live pipeline. The cached-model route already refuses this; the trained/exported one now does too, matching by path rather than repo id, and failing open on a chat-only install so it cannot block ordinary deletes. A staged download finishing while its page was hidden loaded the model and evicted whatever the user was actually using: both diffusion pages stay mounted behind the router and a load takes the GPU unconditionally. The pick is now held until its page is on screen again, which is also what chat does. A scoped download could report success having fetched nothing. With Hugging Face metadata unavailable no manifest is written, so verification is a no-op, and snapshot_download returns an existing snapshot folder without downloading when its own repo_info call fails. A repo already on disk from a full snapshot job (which ignores *.gguf) therefore completed with no weights and auto-loaded against them. The requested file list needs no network, so it is checked against the disk directly. The XET to HTTP retry reclaimed the job slot without the scoped file list, and that claim overwrites the stored record, so a later identical scoped start compared an empty list against the real one and 409'd instead of adopting the running download. The DiT accelerator gate probed torch.mps.is_available(), which only exists from torch 2.5 while the supported floor is 2.4. All three probes shared one try/except, so on torch 2.4 the AttributeError read as 'no block' and a CPU-only host still evicted the resident pipeline, downloaded the encoders and died in the child. Each accelerator is probed on its own now, through torch.backends.mps. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stage what an LTX-2.3 load reads, keep a routed file's load kind, drop an unbakeable LoRA Three from the latest review. The video download plan always asked for the wide base file list, so an LTX-2.3 pick staged the 2.0 base's VAEs, vocoder and connectors that the checkpoint supplies itself, while the companion files the 2.3 assembly does read were left out of the plan and pulled inline at load, outside the panel's progress, cancel and disk preflight. The plan now recognises a 2.3 pick by name (the load keeps the authoritative header probe, and under-guessing only falls back to the load-time pull), narrows the base list, and stages the extras in the same entry as the checkpoint so one repo stays one scoped job. A pick routed from the chat picker arrives as ?model= and ?quant= with no picker metadata, so a bare local .gguf or .safetensors was loaded as a pipeline: an explicit model_kind wins over the backend's filename sniffing, so it evicted the resident model and then failed on the missing model_index.json. Both pages now derive the load kind from the path, the same way their own picker handlers do. A torchao int8/fp8 build takes adapters only at load time. Switching artifact inside one family keeps the LoRA selection, since the family did not change, but the load did not bake it, so the next generation was rejected with 'reload the model with the adapter selection' while the picker still showed the adapter as active. The selection is now dropped once per resident build, with a message saying to pick and load again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cancel an evicted safetensors load, spare the arbiter for CPU-only chat, fetch clips lazily Four fixes from the latest review round: - The GPU arbiter's chat evictor only cancelled the llama.cpp side. The orchestrator publishes active_model_name once its worker reports success, so an in-flight safetensors load was visible only as an entry in loading_models and finished onto the GPU after ownership had transferred. Cancel every pending load, and give the safetensors branch the post-load ownership recheck the GGUF branch already had. - A manual gpu_layers=0 GGUF load runs on the CPU with the GPUs hidden from the child, yet it took the arbiter unconditionally: it cancelled a running image or video generation for a model needing no VRAM, then held CHAT ownership so the next GPU workload unloaded it for nothing. Gate the acquire on the same predicate the launch-time CPU-only mask uses, as the image and video loaders gate on their resolved device. - The staged-download hook subscribes per repo, not per job, so another job on the same repo advanced the staged queue (starting a load whose scoped files were still downloading) or wiped a queue that was still running. Compare the variant each callback carries, like the chat page's auto-load does. - The video gallery fetched every record of a page into an object URL that lives until the page closes: 50 clips at tens to hundreds of MB each, for cards the user may never scroll to. Fetch a clip as its card nears the strip's edge, plus the selected one the player needs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim the comments across the diffusion backend Comment-only pass over the Python this PR touches: drop what the code already says, collapse multi-line explanations that still read on one line, and keep the reasoning that is not recoverable from the code. No code, docstring semantics or behaviour changes; verified with an AST comparison against the previous revision, and the backend suite is unchanged (same 37 environment failures as before: the API integration tests that need a live keyed server, the flash-attn install hooks, and the GPU memory fields). * Invalidate latents on a VAE swap, keep a cut-off generation, surface the EMA adapter - source_revision() scanned the checkpoint root plus text_encoder/tokenizer but not vae, so swapping or fine-tuning the VAE in place left the conditioning cache namespace unchanged and a warm run trained against latents from the old checkpoint. Include the vae directory, like any other component the cached tensors come from. - /images/generate answers only when the images are saved, and secure mode's tunnel caps an origin response near 100 seconds, which a native CPU or a high-step run passes routinely. The page reported failure while the work kept running, and a retry would duplicate it. A lost response (fetch rejection or a gateway status the origin never answered) is now told apart from a refusal: the page waits out generate-progress and reloads the gallery, so the run it started still lands. - The trainer emits the EMA adapter's path with the terminal event, but the state update dropped it, so neither the run history nor either response schema carried it and an enabled EMA left nothing discoverable. Keep it, and show it next to the primary adapter. - weighting_scheme advertised a choice of timestep sampling; sampling is always logit-normal and the flag only selects the bell loss weights. Describe what it does. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Hide unloadable cached rows, hold the dataset interlock, bound a GIF export - The cached-model listing tagged any repo with a model_index.json as text-to-image, so a community pipeline the image loader's trust rule refuses still got a row in the Images picker, and a detected-but-untrusted video repo fell through to that same tag. Gate the image tag on the load path's rule and hide an untrusted video repo outright. - A routed diffusion pick only carries a GGUF filename, which is all the chat picker has, so a curated single-file artifact arrived with no quant and was loaded as a pipeline: from_pretrained on a repo with no model_index.json. Pass the page's own catalog spec into the route pick, so a routed pick resolves to exactly what a direct pick on that page resolves to. - The dataset mutation endpoints checked is_active() and only then handed their filesystem work to a thread, so a start reserving in that gap changed captions or removed images underneath the preflight or the running trainer. The interlock is now registered for the whole request under the lock reserve() uses, and a start refuses while a mutation is open rather than waiting on it. - GIF export held every kept frame as a paletted image before encoding; a clip may be 2048x2048 for 1024 frames, and at the 12 fps target the step is 1, so one export click could allocate over 4 GB and take the backend down. Downscale past 720 px and widen the step to keep at most 300 frames. - seed accepted any Python int, so an out-of-range one passed every preflight, evicted the resident models, spawned the trainer and only then died in torch.manual_seed. Bound it to torch's 64-bit range in the request and config. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the accelerator probes in the DiT family-metadata tests Six tests read family_train_infos() (or a start preflight) without pinning the host probes, so they only held on a machine with a bf16 accelerator: on a GPU-less runner the DiT gate empties precision_modes, turns supports_compile off, and replaces any other preflight message with the no-accelerator note, and all six failed there. A conftest fixture pins both probes for exactly those tests, so they assert the family metadata they are about on every host. The gate's own CPU-only behaviour keeps its dedicated tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Carry the pipeline task into the hub inventory the pickers read - The task-scoped pickers filter On Device rows on a task, and the chat picker routes a diffusion pick by the same field, but those rows come from the /api/hub inventory, which never carried one: the Images and Video pickers listed nothing on device and the routing never fired. Both cached scans and the local listing now tag rows with the classifiers the models API already uses, the schemas and the frontend adapter carry it through, and a row the backend classified as a generation task is exempt from the chat-only guard that was also dropping it. - The local routing map was keyed by model_id while the row click passes id (a filesystem load id for a models_dir or LM Studio entry), so the lookup missed and the pick fell through to the chat loader. Key both. - A staged download whose start answered "error" left its head in place, where the effect never re-runs and onReady never fires, so the pick was stranded until the user reselected. Clear the queue and say so. - Every scoped pick in a repo shares the @diffusion variant, so the variant alone cannot tell two file sets apart: restaging while the first job finished let its completion pass for the new pick and load a checkpoint that had not downloaded. Bind the callbacks to the repo + file set they started, and to the staging generation. - A rejected generate POST does not say whether it reached the backend, so an immediately idle progress read was ambiguous and a submission that never landed looked like a finished image. Require evidence: progress seen active, or a gallery record that was not there before the POST. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make the OpenAI image URL fetchable, keep WebM audio, stream example imports Four review items on the diffusion Studio work: - response_format=url returned the bearer-gated gallery route, which a standard image client downloads with no Authorization header, so the default response format was unusable. Mint a short-lived HMAC link instead (the shape RAG already uses for pdf.js) served by a signed route, and leave the gallery route itself bearer-only. - A manual gpu_layers=0 load carrying speculative_type="off" -- a value the UI persists and sends -- read as GPU-bearing, so it took the GPU arbiter and evicted a resident image/video pipeline even though the launcher hides the GPUs for it. Canonicalize the mode and exempt "off". - The curated example import prepared the whole split before the loop stopped at the 10-100 image cap; m1guelpf/nouns is 49,859 rows / 328 MB. Stream instead, with the prepared load kept as a fallback for a repo that cannot stream. - WebM export dropped the audio track an LTX-2 clip carries, silently, on the format offered for web embeds. Mux it as Opus through a resampler + FIFO, and keep exporting the video alone on a build without libopus. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not stub out triton on a GPU host when the Xet backend fails to import The lazy loader retries `import unsloth_zoo.hf_xet_fallback` under UNSLOTH_ZOO_DISABLE_GPU_INIT=1 whenever the first attempt raises. That flag makes unsloth_zoo take its MLX/CPU path, which injects triton and bitsandbytes STUBS into sys.modules for the rest of the process. On a working GPU box whose first import failed for an unrelated reason (a bitsandbytes/CUDA mismatch, say) the retry succeeds, so Studio boots looking healthy and then dies at the first CUDA-only kernel: a GGUF or compiled diffusion generation hits the stub and returns NotImplementedError: Unsloth: 'triton.tools.experimental_descriptor.enable_in_pytorch' was called on Apple Silicon / MLX, where triton is stubbed out. so every image generation 500s with an Apple-Silicon message on a Linux CUDA host, while the load reports success. Found by loading Z-Image-Turbo GGUF through the API on a box where bitsandbytes could not initialise. Gate the retry on the host genuinely having no accelerator. The Xet stall watchdog is optional and already degrades with a warning; a process whose triton is stubbed out is not recoverable. The warning now says why it did not retry. * Fix the lost-generation proof set, the settle timeout and the hub inventory's diffusion gates Seven fixes from the latest review round on the Images page and the hub cache inventory. Images page: - The lost-POST settle path built its "already seen" gallery id set inside the catch, after the request failed. By then the earlier runs of the same batch had already prepended their records, so run 2 could accept run 1's image as proof that its own request reached the backend. The set is now captured once before the first POST and grows with every record the batch produces. - settleLostGeneration fell out of its SETTLE_MAX_MS loop and returned normally, so a wedged generation was counted as done and the next run started against a busy backend. It now throws on timeout. - Restoring a recipe cleared the ControlNet selection but left the workflow tab and the init / mask / reference images pointing at whatever was loaded, so the next Generate conditioned on an unrelated image. It now clears all of them and returns to Create. - The download plan omitted the adapter selection the load itself bakes in. A baked LoRA forces the dense build path, so the plan described a different file set than the load that followed and the rest was pulled inline, outside the download manager. Both now derive the list from one helper. Hub cache inventory: - A download for a repo an Images or Video load is staging was allowed to start: only the llama.cpp loader was consulted. Both diffusion backends already expose loading_repo_ids for the delete guard, and the download guard now reads them too. - A companion-only prefetch (pipeline manifest plus VAE and text encoder, no transformer) passed the snapshot-partial check, since every file its manifest expected did arrive, and was advertised as on-device although from_pretrained cannot load it. - The single-file flag never reached the picker through the hub inventory path, so a checkpoint-only diffusion repo read as a full pipeline and failed after the handoff. The two pipeline-shape helpers now live in hub/utils/inventory_scan.py so /api/models/cached and the hub inventory classify the same repos the same way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop adopting an unknown scoped download, leaking raced blobs and resurrecting deleted clips Three items from the latest review round. A scoped download job carries a deliberate file subset, and every file set of one repo rides the same "@scope" slot. A client that adopts a live job from the backend had no file list to compare against: the active-downloads response never carried one, so an adopted job's set was unknown and any later scoped request for the same repo read as "already started". Selecting a different checkpoint then waited on the wrong transfer and tried to load a file nobody fetched. The response now publishes the scoped file list, adoption records it, and an unknown set no longer satisfies a scoped request. A gallery record can be deleted while its blob is still downloading. The delete revokes the URL present at that moment, so the fetch that lands afterwards inserted a fresh object URL for a record no card renders and nothing can revoke: a full MP4, tens to hundreds of MB, pinned for the rest of the session, and once per raced fetch. Both galleries now discard a blob whose record went away, with an epoch covering the video page's Clear all. The video backend keeps the last completed job until the next one starts, and the Video page merges that record on mount to cover a job that finished after the gallery fetch. Deleting the clip left the record in place, so every reload prepended a ghost card whose file request 404s until another generation replaced it. Deleting the clip, or clearing the gallery, now clears the matching terminal record, and the page skips a record it deleted itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve revisions from the live cache, serialize dataset imports, and stop pinning every gallery blob Five more items from the review round. The conditioning-cache revision marker read huggingface_hub's import-time HF_HUB_CACHE constant. Studio can move its cache during a session and loading follows the live setting, so after a move the marker went unresolved (or pointed into the previous root) and pulling a new revision of the same checkpoint no longer invalidated the cache: a warm run could reuse the old encoder's embeddings and the old VAE's latents. It now looks in the active Studio cache first and keeps the environment and the library constant as fallbacks, which the trainer subprocess still needs. The dataset interlock counts mutations rather than excluding them, so two imports of different examples into the same empty name both got past the emptiness check. The winner promoted its staging directory atomically; the loser found the folder non-empty, fell back to a per-file move, and merged its images and captions into the winner's dataset. Imports now take a per-folder lock, a second one is refused with 409, and the emptiness check is repeated under the lock. On Windows the sd.cpp asset resolver filtered only by accelerator token, so a Windows arm64 host matched an x64 zip, downloaded and installed it, and failed later when the binary would not run. It now filters by architecture the way the Darwin and Linux branches do. Every gallery page fetched every PNG up front and kept the object URL for the session, so scrolling a large gallery grew memory without bound for tiles the user may never look at. The Images strip now fetches a tile as it nears view, like the Video strip, and keeps the eager path only where IntersectionObserver is unavailable. A 503 carrying a JSON body comes from the application, not a proxy, so it is surfaced as the error it is instead of entering lost-response settlement and being reported as a request that never reached the server. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Name the class of a failed generation instead of a bare "Image generation failed." Found on a macOS runner: the native renderer aborts inside its own text encoder there, and the page showed only "Image generation failed." with the sd-server backtrace left in the server log, so nothing about the failure reached the user. The failure is now classified into fixed text, out of memory and native-process death, so the message says what happened and what to try. None of the engine's own output is echoed, since a native tail carries local paths and argv; that stays in the log, and an unrecognised failure keeps the original literal. * Treat an undecodable caption sidecar as the tombstone the trainer sees Uploads store .txt and .caption sidecars as raw bytes, so one can hold invalid UTF-8. The trainer treats any existing sidecar, decodable or not, as an empty tombstone and never falls back to the metadata row for that image. The labeling grid and the dataset summary read an undecodable sidecar as absent instead, so both showed a metadata caption that the run would silently replace with the instance prompt, and counted the image as captioned. Both now track sidecar presence separately, so what the user reviews is what the run trains on. * Keep the reason a native server died, not just its backtrace A ggml abort prints its cause first and then a stack trace, so reporting the last twenty captured lines gave twenty addresses and nothing about the failure: on the macOS runner the native server died on an unimplemented Metal op and the message carried only frame pointers. The captured tail now leads with the lines that name a cause and keeps recent context after them, for both the startup failure and the mid-request death. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop staging the dense text encoder for an fp8 video load Two halves of the same gap, found while measuring the LTX-2.3 download plan: - The video download plan and the scoped pre-download never saw text_encoder_quant. An fp8 request loads a hosted pre-cast encoder, so asking for one still staged and downloaded the base repo's dense Gemma3 (48.79 GB of Lightricks/LTX-2 on the 2.3 distilled pick) that the pipeline then never opened. The plan now drops those shards and stages the pre-cast checkpoint instead; their configs stay, since the pre-cast loader still meta-inits the encoder from the base repo's component config. - The LTX-2.3 assembly builds every component itself, so pipe_kwargs (which carries the pre-cast encoder for from_pretrained) never reached it and an fp8 request silently loaded the dense encoder anyway. It is passed across explicitly now. The dense skip is earned, not assumed: only a pre-cast checkpoint that resolves on the Hub lets the plan drop the dense shards, and only one already fetched to disk lets the pull drop them, so an unpublished or gated artifact leaves both exactly as they were. If injection still fails after that, the load tops the dense weights back up rather than handing from_pretrained a snapshot with no encoder in it. Measured against the real Hub on the 2.3 distilled Q4_K_M pick: 67.24 GB before, 18.92 GB with a 0.43 GB stand-in for the pre-cast artifact (the base entry drops from 24 files / 48.79 GB to 13 files / 0.04 GB). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match the file's typing idiom for flow_shift models/training.py annotates with typing constructs throughout (105 Optional[...], no Union), and flow_shift was the one place using PEP 604. Union[] reads like the rest of the file, and it also drops the runtime evaluation that would raise on Python 3.9. * Bound the gallery blob cache, and three interlock fixes Four review findings, all reproduced first: - The gallery object-URL caches were unbounded. A clip runs from a few MB to a few hundred, both pages stay mounted after their first visit, and entries were only dropped on delete, so scrolling pinned everything for the session. Both pages now share a byte-budgeted LRU (512 MB video / 192 MB images) keyed off the visibility signal the near-viewport fetching already provides. On-screen media, the selected clip or image, and the item just fetched are never evicted, so eviction is invisible and a single item larger than the whole budget cannot evict itself into a refetch loop. - The image, video and chat load guards ran two independent training probes but returned early when the FIRST one raised, so an unreadable LLM backend disabled the diffusion interlock and a load could proceed straight into an active diffusion trainer on the same GPU. The probes are independent now. - An engine switch swallowed a failed teardown and published the new engine anyway, which is exactly the leak the unload exists to prevent: the arbiter's evictor, /images/unload and the next load all resolve through get_active_diffusion_engine(), so the still-resident pipeline (or a live sd-server) became unreachable and the next load allocated on top of it. The switch now fails and leaves the old engine published, so it stays reclaimable. - The native generation timeout was 30 minutes while the Images page waits up to 6 hours (SETTLE_MAX_MS), so slow-but-progressing CPU jobs died deterministically at the deadline. Measured on GPU-less runners, a 512x512 4-step Q2_K generation took 900 s on Linux and 1465 s on Windows, so larger images or step counts clear half an hour easily. The ceiling now matches the page's window and applies to the whole request: chunks of a split batch share one deadline instead of each getting a full budget. Cancellation is unchanged. Declined: gating the huggingfacenotorch extra off Python 3.9 over the conditional diffusers marker. The marker is deliberate and its comment says why: diffusers dropped 3.9 in 0.38, so pinning >=0.39 outright leaves pip no candidate and the whole extra unresolvable there. The pipelines it names live in studio/backend, which cannot install on 3.9 anyway (studio.txt pins matplotlib==3.10.9 and fastmcp>=3.0.2, both requires_python >=3.10), and the extra is the general core one, so the alternative drops 3.9 for library users who never touch Studio. * Close the load-versus-training-start race, and two picker fixes - The image and video load guards read is_active() and only then selected an engine, acquired the arbiter and registered the load. A /train/diffusion/start reserving inside that window freed residents the load had not registered yet, so the trainer came up beside a brand-new pipeline. The service already had exactly the right pattern for this in dataset_mutation, so gpu_load_admission mirrors it: reserve() refuses while an admission is open, an admission refuses once a start is reserved, both decided under the one lock. The span is only the registration, since begin_load returns as soon as the load is registered and _free_gpu_for_diffusion_training preempts an in-flight load from that point. Chat is deliberately not covered: its load spans an eviction plus a multi-minute GGUF load, and it admits models that fit beside training by design, which is a different contract from the diffusion pipeline's all-or-nothing one. - Hugging Face gives the LTX-2 family the image-to-video pipeline_tag (both Lightricks/LTX-2 and unsloth/LTX-2.3-GGUF report it), so a text-to-video-only filter dropped the flagship audio family out of Video Hub search while the rest of the app routed it to Video. - Task-scoped quant fit sized picks against the LARGEST visible device while resolve_diffusion_device_target returns a bare "cuda" and torch places on the current one. On a heterogeneous host that recommended a checkpoint sized for the bigger card and then loaded it onto the smaller one. Fit now uses the device the load actually lands on; identical on a homogeneous host. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Expose the persistent conditioning cache in the start schema DiffusionLoraConfig has carried cond_cache_dir for a while and the DiT trainer acts on it, but DiffusionTrainingStartRequest omitted the field, so Pydantic dropped it silently and every API-driven run fell back to the in-memory cache that is rebuilt from scratch each time. The warm path skips loading the VAE and the multi-GB text encoders on a rerun whose images, captions and resolution are unchanged, so this was a real capability that could not be reached. Contained like output_dir rather than left to the trainer subprocess's cwd, since it is another directory the trainer writes to. Blank or omitted still means the in-memory cache, so it must not resolve to the outputs root. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix diffusion policy and classification issues from review fp8 auto precision defaulted to precise accumulate on any non-consumer GPU, which made fp8 2.05x slower than int8 on RTX 6000 Ada and slower than not quantising at all. NVIDIA's professional whitepapers do publish equal FP8 rates for both accumulate modes there, so the hardware premise held, but the cost is in the cuBLAS path rather than the published rate. Default to fast accumulate: measured on B200 the flag is a no-op (4096^3 _scaled_mm at 3023.8 vs 3041.8 TFLOP/s, bitwise-identical output, 1.213 s vs 1.230 s end to end), so it is a large win where it bites and free where it does not. Precise accumulate stays available via transformer_quant_fast_accum. Z-Image's DiT is a Lumina2 derivative, so unsloth/Z-Image-GGUF and unsloth/Z-Image-Turbo-GGUF both declare general.architecture = "lumina2" and the whole line was tagged image-diffusion-unsupported and hidden from the Images "On Device" list, though validate_load_request loads them. Resolve shared archs from the repo/file name like bare "wan" already does, with a test asserting the picker and the loader agree for every family. The sage attention on-demand install ran an unpinned `pip install sageattention`, but PyPI's newest wheel is 1.0.6 and diffusers refuses anything below 2.1.1: the install always "succeeded", wrote an unusable version into the running venv, and was rejected on the next line. Carry the dispatcher's floor so pip resolves nothing instead. The dense-quant disk gate sized the download from the bf16-RESIDENT table. The fp32 families download twice that (Z-Image: 23,479 MiB against a 21,970 MiB gate), leaving a window where the check passed and the download filled the disk; Ideogram 4 ships fp8 and was overcharged the other way. Size the gate by published bytes, verified against HF sibling metadata for all 12 families. Patch installs went through unsloth_zoo, which refuses to import unless UNSLOTH_IS_PRESENT is set, and that is set by unsloth itself. The server imports unsloth at boot so it never showed there, but any other process ran silently unpatched with every install returning False, which is 13 test failures on a clean environment. Import unsloth and retry once, memoised per process. Also: the GGUF+LoRA refusal pointed at the native engine without saying a GPU host only selects it under UNSLOTH_DIFFUSION_ENGINE=sd_cpp, so the suggestion was unreachable; the gallery recipe recorded loras from the generate request alone, losing a load-time bake; load-progress claimed "40.07 GB downloaded" for a fully cached load; and pickers.tsx imported three catalog-group helpers it never used. Reported by oobabooga. * Keep the sd.cpp text encoder on CPU under Metal macos-14 loads FLUX.2-klein-4B Q2_K natively on mps and then dies on the first generation with exit code -6: ggml_metal_op_encode_impl: error: unsupported op 'RMS_NORM' -> ggml_abort LLMEmbedder::encode_prompt -> LLMRunner::compute -> GGMLRunner::compute ggml's Metal backend gates RMS_NORM on contiguous rows and aborts the process when that does not hold, with no per-op CPU fallback, so any LLM text encoder (Qwen3 for FLUX.2 and Z-Image, T5 for FLUX.1) takes sd-server down. The encoder runs once per prompt while the DiT runs every step, so pinning only the encoder keeps Metal for the part that matters. UNSLOTH_DIFFUSION_SD_CPP_METAL_TE_GPU=1 opts back in once ggml grows the kernel. * Gate the unsloth retry in the diffusion patch backend The retry added for the clean-environment patch failures is not free: importing unsloth pulls torch in behind it, which costs ~940 MB of RSS measured in a process that had neither, and on a host with no accelerator it fails anyway. A cross-platform CI job that had generated fine at ~900 s later died 19 s in with SIGTERM and every 'if: always()' step skipped, which is the runner being torn down rather than a step failing. Retry only when torch is already imported (true of the server and of anything patching a real module, and the condition that stops the retry from being what loads torch), unsloth is installed but not yet imported, and the first failure was the ImportError the sentinel guard raises. The clean-environment case it was added for still passes 29/29. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only retry the unsloth import where it can succeed The gate still let the retry run on hosts unsloth does not support, which is where it is most harmful: a 7 GB macOS runner lost the Studio server 26 s into a load, and the Linux runner was torn down mid-generation. Neither MPS nor plain CPU can complete the import, so the retry there pays the cost and fails anyway. Require an accelerator unsloth actually supports (CUDA/ROCm via torch.cuda, or XPU), with UNSLOTH_ALLOW_CPU as the documented override, and hoist the predicate to module level so it is tested directly rather than through the import system. On a CPU-only host the retry no longer fires at all; on CUDA the clean-environment case it was added for still passes 29/29. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard old diffusers, stream video exports, record conditioned recipes Three fixes from review. The 0.39-only pipeline classes (Flux2Klein, Z-Image, Krea 2, LTX-2, HunyuanImage) were resolved by getattr deep in the load, so on the older diffusers that packaging still allows on Python 3.9 -- diffusers dropped 3.9 in 0.38 and this project still supports it, so the 0.39 floor has to be conditional or the extra becomes unresolvable -- an advertised model failed with a bare AttributeError after its checkpoint had already been downloaded. Krea 2 already guarded itself this way; assert_pipeline_class_available now runs the same check for every image and video family from validation, before any fetch, and names the version and the fix. WebM export accumulated the whole VP9 output in a BytesIO and returned it as one bytes object that the response held again. The request caps allow 2048x2048 for 1024 frames, so an export runs to hundreds of MB and concurrent clicks could exhaust the process, while the MP4 route beside it already streamed from disk. transcode_to_file encodes to a temp file and the route returns a FileResponse with a background unlink, so nothing large is resident. A conditioned generation's recipe carried only the txt2img fields, so the gallery presented an inpaint or upscale result as a complete Create recipe and restoring it replayed an unrelated text-to-image request. The images themselves are still not persisted (user uploads with their own lifetime), but the workflow and its scalars are, restore reapplies them, and the toast now names the inputs that have to be supplied again instead of silently landing on Create. Reported by Codex. * Per-load video cancel event, family-gated image picker, cond cache refusal A cancelled video load could resume: begin_load cleared the shared cancel event, and unload() drops _loading without waiting for the worker, so the next load cleared the very object the cancelled worker was watching and its multi-gigabyte pull ran on alongside the replacement until the token check at the end. Each load now gets its own threading.Event, passed down through _fetch_te_prequant and _predownload_base, so a cancelled worker stays cancelled. A cached repo with a model_index.json was advertised as text-to-image on the trust rule alone, but validate_load_request also requires a detected image family, so a trusted pipeline of an unsupported class produced a picker row that deterministically 400s. The picker now applies both gates, mirroring the video branch. cond_cache_dir was accepted for sdxl and then ignored: only the DiT trainer reads it, while the SDXL trainer builds a per-run in-memory latent cache, so the promised cross-run reuse never happened. The route now refuses it with a 400 that names the families which do support it, checked against the resolved family so an omitted model_family with an SDXL base is caught too. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the frontend build broken by the gallery blob cache tsc -b failed on the branch head, so npm run build produced no dist and every platform job fell back to --api-only: blob-url-cache.ts(29,15): TS1294: This syntax is not allowed when 'erasableSyntaxOnly' is enabled dataset-labeling-grid.tsx / dataset-showcase.tsx: Argument of type '{ url: string; bytes: number; }' is not assignable to parameter of type 'string' The cache took its budget as a constructor parameter property, which the project's tsconfig forbids, and fetchGalleryObjectUrl now returns the blob size alongside the URL for that budget, which the two dataset thumbnail components still consumed as a bare string. Declare the field explicitly and destructure the URL at both call sites. tsc -b is clean and vite build emits dist again. * Recover from a ggml unsupported-op abort by restarting on the CPU backend ggml checks every node against the device's supports_op and calls GGML_ABORT when one is not implemented, because a single-backend graph has nowhere else to put it: there is no per-op CPU fallback. The whole sd-server dies with SIGABRT mid-generation and the user gets "the native image renderer stopped unexpectedly" with no way forward. Seen on macos-14 arm64 with FLUX.2-klein-4B Q2_K through the cross-platform CI: the text encoder is already pinned to CPU, and the abort moved into the denoise loop instead. ggml_metal_op_encode_impl: error: unsupported op 'MUL_MAT' -> ggml_abort StableDiffusionGGML::sample -> sample_k_diffusion A retry on the same backend would abort identically, so the load is restarted once with --backend cpu (the only flag that changes which backend executes the graph; --offload-to-cpu moves parameters, not compute) and the generation is re-submitted. The same checkpoint then renders slower rather than not at all. Strictly bounded: the signature must carry both the unsupported-op line and ggml_abort, the device must not already be CPU, and it happens once per load, so an OOM kill or a genuine crash still surfaces as itself. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix two tests that only fail in a full-suite run The 3.10 CI leg resolves PyAV 17, where av.container.OutputContainer is an immutable C type, so the no-libopus export test died on "cannot set 'add_stream' attribute of immutable type" before it asserted anything. Inject the refusal by wrapping the container av.open() returns instead; modules stay patchable on every build. Removing the injection makes the test fail again, so it still covers the branch it is named for. The Xet shim's degraded-path tests drop utils.hf_xet_fallback from sys.modules and import a throwaway copy. Restoring only the sys.modules entry left the utils package attribute bound to the throwaway, and the two disagreed for the rest of the process: a later monkeypatch of the dotted target patched one copy while the code under test imported the other, so the patch did nothing and test_fetch_te_prequant_only_reports_what_it_downloaded reached the real Hub and got a 401. Restore both bindings. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Stop an ignored-cancel sd-server, guard deletes during diffusion training, repair unusable managed binaries sd-server does not interrupt an in-flight job, so when it ignores a cancel the grace branch abandoned the poll and reported cancellation while the native job kept a core (or the GPU) busy to completion and held the server's job slot. The comment said the caller stops the server, but only unload does that immediately: a superseding load stops it after its multi-gigabyte download, and a load that then fails never gets there. Stop it here, as the deadline branch already does. DELETE /api/models/delete-finetuned checked only the LLM trainer, so it could rmtree the output directory a live diffusion LoRA run was about to write its adapter into. Consult the diffusion training service too, like the dataset mutation and model-load routes. find_sd_*_binary only checks is_file(), so an interrupted extraction (or a prebuilt for the wrong CPU) left a present-but-unrunnable binary the installer never retried: every load probed it, fell back to diffusers, and native inference stayed off until the directory was deleted by hand. Probe it and reinstall, but only for a copy under the installer-owned root -- SD_CLI_PATH, UNSLOTH_SD_CPP_PATH, an in-tree build and anything on PATH are the user's. * Plan the pre-cast text encoder, and make the cross-trainer GPU admission atomic An fp8 text-encoder request loads a hosted PRE-CAST checkpoint, but the image download plan never received text_encoder_quant, so the manager staged the base repo's dense encoder (FLUX.2-dev's Mistral-24B is ~48 GB, Qwen-Image's Qwen2.5-VL ~16.6 GB) and the load then pulled the pre-cast file inline, outside the manager's progress and disk preflight. The plan now takes the field, resolves the hosted artifact with the same resolver the injection uses, stages that file, and drops only those components' dense weight shards. The load's own prefetch takes the same treatment, since it paid the same cost. Only a checkpoint that really resolves on the Hub earns the drop, so a gated or renamed artifact still stages the dense encoder the load will fall back to. The two trainers admitted each other with independent check-then-act guards: the diffusion route checks the LLM backend several network-bound preflights before it reserves, and the LLM route checks the diffusion service well before it spawns, so two near-simultaneous starts could both pass and train on one GPU. reserve() now re-tests the LLM backend under its own lock, and the LLM route holds the diffusion service's gpu_load_admission across its spawn, so exactly one of the two wins. Both halves fail open, so a chat-only install still trains. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pin the Advanced options a staged download planned against Staging does not set busy, so while a multi-gigabyte download runs the user can still change precision, memory mode, speed or the baked LoRA selection. The pending record held only the repo and artifact, and the completed download fired a load that read the CURRENT state: the staged file set could then be missing files that load needs (fetched inline, with no progress and no disk preflight) or hold gigabytes it no longer uses. One snapshot of every Advanced control is now taken when the plan is built, and it travels with the pending record into the load, so the load that runs is the one the download was planned for. * Do not advertise a family the installed diffusers cannot build The newer families (Z-Image, Krea 2, FLUX.2, LTX-2, HunyuanImage) exist only from diffusers 0.39, and 0.39 cannot be installed on Python 3.9 at all -- diffusers dropped 3.9 in 0.38, so the requirement is conditional or the whole extra becomes unresolvable. On such an environment the picker still offered those rows, every pick failed deterministically, and the error's advice to run pip install -U diffusers could not fix it without also upgrading Python. The cached-repo picker now applies the same availability check validate_load_request does, which is keyed on the pipeline class actually present rather than on the Python version, so it is also right for an intentionally pinned older diffusers on 3.10+. Fails open when diffusers cannot be imported at all: that is a different problem and the load path reports it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the diffusion engine selection after each router test The active engine is module state, and several tests set it by plain assignment because what _activate does to it is the thing under test, so monkeypatch could not undo it. A leaked ENGINE_SD_CPP left get_active_diffusion_engine() handing back the sd.cpp backend for the rest of the process, and every later route that reads the active engine then saw an unloaded model: eight tests in test_openai_images_generations_route.py returned 503 in a full-suite run while passing on their own. The autouse fixture now snapshots and restores it. * Stream gallery clips, and close three races around them Four fixes from the latest review pass. The video gallery downloaded each clip into a blob before it could play, so playback waited on the whole file (tens to hundreds of MB), seeking was limited to what had arrived, and every viewed clip stayed pinned in the webview. The file route already streams and serves ranges; it just could not be a <video src> because it is bearer-gated. Mint a short-lived signed link instead (its own HMAC secret, 12 hour TTL, separate from the image links) and hand it to the element, which then fetches only the ranges it plays. That removes the blob budget, its LRU and every revoke on this page. The sd.cpp readiness probe accepted any process answering on the port, so a foreign server that grabbed the port between the bind check and the spawn was adopted as ours. Confirm the listener is our child before reporting ready, and stay best-effort (psutil missing, an unknown owner, or any probe error still passes) so the check can only reject a definitely foreign process. Dataset import held its lock for the extract but not for the upload path, so two concurrent uploads into the same folder interleaved; take the same lock and return 409. And reject Windows device names (CON, NUL, COM1..9, LPT1..9, with or without an extension) plus trailing periods in dataset names, which are unopenable on Windows. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Build the image download plan for the engine that will load /images/download-plan always asked the diffusers backend, while /images/load picks the engine per host: a GGUF pick on a machine with no usable GPU routes to native sd.cpp, which reads a single-file VAE plus text encoders and never opens the base repo's sharded components. Measured on unsloth/FLUX.2-klein-4B-GGUF (Q2_K): the plan staged 7.66 GB of FLUX.2-klein-4B components the native load discards, and the 7.80 GB sd-cli actually needs was then fetched inline by the loader, outside the download manager's progress and its disk preflight. Z-Image-Turbo is the same shape. The plan now asks whichever engine the load will select. predict_engine() applies the selection policy without any side effect: it activates nothing (staging a download must not unload the resident model) and only locates the binary rather than installing it, but still counts an installable binary as available, since that is what the load does on a fresh host. The native backend gains a download_plan built from the same _asset_specs the loader fetches, returning the same envelope, so the manager stages exactly the files sd-cli opens. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not let a queued generation outlive the model, and three scan fixes Five items from the latest review; four were real. An unload or arbiter eviction only cancels the generation holding _generate_lock. A second request queued behind it holds no cancel event yet, and Python locks are not FIFO, so it could take the lock the instant the active denoise released it, still see a loaded pipeline, and run a whole new denoise after the model was told to go away: the eviction then waits minutes for it and an image lands after the eject. Unload and a superseding load now raise a fence under _lock before they queue, and a generation that wins the lock while one is pending refuses instead. The cached-model scan judged pipeline completeness across every revision, so a repo holding an older complete snapshot plus a newer companion-only one read as complete while the snapshot from_pretrained actually opens has no transformer. Both scans now look at the revision the loader will open. Deleting a dataset image deleted its caption sidecar unconditionally, which for cat.jpg alongside cat.png removed the caption the survivor still resolves to. The sidecar now goes only with the last image of that stem, matching what the thumbnail cleanup beside it already did. Importing an example into a folder that holds no images but does hold files fell back to promoting the staging dir one file at a time, so an interruption left a partial dataset that the image_count check accepts as complete on retry. Those files are folded into the staging dir instead and the promotion stays a single atomic rename. The MPS generator report does not apply: torch.Generator(device="mps") has worked since PyTorch 2.0 (pytorch/pytorch#91348) and the studio installer pins torch>=2.4. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten diffusion comments Collapse the multi-line comment blocks across the image, video, sd.cpp and diffusion-training code to one or two lines each, and drop comments that only restate the statement below them. Comments only, no code or behaviour changes. * Tighten diffusion comments (second pass) Collapse the remaining multi-line comment blocks in the video page, training routes and service, sd.cpp server and installer, memory and speed planners, and the shared request models. Comments only, no code or behaviour changes. * Tighten diffusion comments (third pass) Collapse the remaining multi-line comment blocks in the attention, cache, LoRA, prequant, precision and compile-cache modules, the sd.cpp arg builder and engine, the video routes, the Ideogram 4 assembly, the model picker, and the diffusion test suites. Comments only, no code or behaviour changes. * Restore the dataset when an example import cannot be promoted Promotion folds the folder's pre-existing entries into the staging dir so the swap is one atomic rename. Every failure after that fold left the user with nothing: the 409 path and an os.replace error both fell through to 'finally: shutil.rmtree(staging)', which deleted the entries that had just been moved in there, while the response said 'Nothing was written'. A same-named entry was also unlinked outright before the promotion was known to succeed. Park superseded same-name entries in a rescue dir instead of deleting them, record every move, and restore all of them if any step of the promotion fails. The fold loop itself is covered too: renaming a non-writable directory raises EACCES on POSIX, which previously escaped as a 500 after the earlier entries had already been moved out. A failed rename now maps to the same retryable 409 as the rmdir conflict. Verified with the reported trigger (a non-empty mode-500 directory whose name collides with an imported file): the folder listing is now identical before and after the failed import. * Drain the teardown fence on a failing unload, give each load its own cancel event Two independent leaks on the image path, both already solved elsewhere in the same file. unload() incremented _teardown_waiters, ran _unload_locked() and decremented, with no try/finally, while the superseding-load path used a finally for the same pair. _unload_locked ends in clear_gpu_cache(), whose CUDA branch calls synchronize/empty_cache/ipc_collect unguarded, and a sticky CUDA fault makes those raise. The count then never drained, so every later generation was refused as cancelled for the life of the process, a fresh load included, since begin_load's own increment and decrement are symmetric. unload() is reached from the chat/video GPU handoff, the engine router and two training routes, so one fault during an ordinary handoff wedged image generation until restart. Release it in a finally. The image and native backends each cleared one shared cancel Event on a new load. unload() sets that event to cancel an in-flight multi-GB download and drops _loading in the same breath, so a replacement load is admitted while the cancelled worker is still inside the fetch, and its clear() re-enabled the very object that worker was watching: the cancelled download resumed and ran alongside the replacement. Take a fresh Event per load and thread it to the worker, as the video backend already does, and set it under the lock since begin_load now rebinds the attribute. * Name utf-8 on the diffusion text I/O and the sd.cpp subprocess pipes tests/test_text_io_encoding.py failed on five files this branch adds. Text I/O without an explicit encoding falls back to the Windows ANSI codepage, so a non-ASCII path or manifest value round-trips corrupted, and the three sd.cpp pipes decode the child's UTF-8 output as ANSI on Windows despite already passing errors = 'replace'. Eleven read_text() / write_text() sites across diffusion_compile_cache, diffusion_ideogram4 and diffusion_krea2, plus text = True on the sd-cli version probe, the sd-cli run and the sd-server pipe. * Record the load-time build on a gallery image's recipe A gallery record documents itself as the image's full generation recipe and is embedded in the PNG, but the only load-related field it carried was the repo id. A GGUF repo holds many quants, so that does not say which one made the pixels, and it says nothing about an adapter baked in at load time. The fallback meant to cover the baked case could never fire: with no loras on the request _adjust_baked_loras zeroes every baked adapter and _active_lora_pairs drops zero-weight entries, so active_loras was always empty. A baked-and-disabled build is not the same pipeline as a never-baked one, so the recipe could not reconstruct the image once the model was rebuilt. Persist model_kind, gguf_filename, transformer_quant and the baked adapter names, read off the load state rather than the request, and show them in the recipe popover. The new fields are optional with defaults, which matters because list_gallery_images drops any record that fails validation, so a required field would have emptied every existing gallery; a regression test pins that. * Drop eleven duplicated comment tails, restore the mxfp8 denial note The comment passes collapsed several wrapped blocks onto one line without deleting the last physical line of the original wrap, leaving the tail of each sentence repeated as its own comment underneath. Two of the eleven were re-worded rather than byte-identical, so a strict suffix match missed them. |
||
|
Daniel Han
|
cf4acbcce5
|
Make the Colab oracle tripwire able to fire, and stop blaming start.py for a hung agent CLI (#7838)
* Make the Colab oracle tripwire able to fire, and stop blaming start.py for a hung agent CLI * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep a connection-mode cap fatal, and do not read 137 as a timeout * Note that the opencode non-exit is intermittent * Close three gaps the waiver opened: turn-2 side effect, attribution-ab, and the guard tests * Bound a TERM-resistant CLI again, and judge a capped turn 2 on an artifact * Gate both timeout statuses on the clock, keep the cron lint reachable, and make refresh --all atomic * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
aa4ea7585b
|
Security audit: re-baseline the benign findings that reopened on main (#7741)
All three pip scan-packages shards fail on main. Every unsuppressed finding is
a pattern false positive in a mainstream library or in our own published test
files, and each one already has an entry in the allowlist under an older
evidence hash.
The baseline key is (package, package-relative file, check, evidence_hash),
and the hash is over the matched code. Line shifts and version bumps are
absorbed, but when an upstream release edits a flagged line the hash moves and
the entry reopens for re-review. That is what happened here.
Reviewed and appended the 11 reopened findings:
- fastapi/routing.py, huggingface_hub/hf_api.py, openai/_base_client.py:
flagged as C2 beaconing. All three are ordinary loops, hf_api and
_base_client being paginators that advance until there is no next page.
- huggingface_hub/_sandbox.py: flagged as a staged dropper. This is the
Sandbox feature bootstrap, fetching sbx-server from the endpoint configured
on the HfApi client.
- IPython/core/interactiveshell.py and traitlets/config/loader.py: flagged as
downloading and executing remote code. Both are exec(compile(...)) over a
local file, which is what an interactive shell and a config loader do.
- unsloth_zoo/mlx/loader.py: __import__ over a hardcoded tuple of mlx_lm
module names, paired with mx.eval() calls that evaluate MLX arrays rather
than Python source.
- unsloth-zoo tests: a literal temporary_location="/tmp/ignored" kwarg,
compile()/exec() over codegen the test itself produced, and
__import__("math").exp inside a pytest.approx assert.
Existing entries are untouched: the diff is a single append hunk with no
deletions, and the first 203 entries are byte identical.
Verified by reproducing all three shards locally on Python 3.12, matching the
workflow pin. Before the change the shards reproduce the exact CI counts. After
it, all three exit 0 with unchanged MEDIUM counts and suppressed counts up by
exactly the number of findings each shard was failing on.
|
||
|
Daniel Han
|
425c9081a3
|
Uninstall the whisper.cpp prebuilt and the node/whisper install locks (#7686)
* Uninstall the whisper.cpp prebuilt and the node/whisper install locks scripts/uninstall.sh removed the llama.cpp prebuilt and its install lock but not the whisper.cpp prebuilt, and not the node or whisper install locks. Every prebuilt serializes on <parent>/.<name>.install.lock (prebuilt_core.py install_lock_path), so a normal install leaves ~/.unsloth/.node.install.lock behind. The final rmdir refuses a non-empty directory, so one stray zero-byte lock keeps the whole ~/.unsloth tree on disk. Confirmed on a real ubuntu-latest install -> update x3 -> uninstall run: ~/.unsloth and ~/.unsloth/.node.install.lock both survived. whisper.cpp is only installed when a prebuilt matching the pinned llama.cpp build exists, so runs that skip it never exercised that path. Also removes the .stale.<pid> locks that a lock takeover renames rather than deletes, documents the sibling prebuilts in --help, and adds a ~/.unsloth catch-all to the smoke workflow's leak list so a future artifact cannot be missed the same way. tests/sh/test_uninstall_prebuilt_artifacts.sh runs the uninstaller for real against a fixture HOME: 19 checks pass here and 8 of them fail against the previous uninstall.sh. It also asserts user content under ~/.unsloth is kept. * Mirror the prebuilt cleanup on Windows, and skip the artifact suite on WSL Two review findings. uninstall.ps1 removed llama.cpp, node, .staging and the llama lock, but not the whisper.cpp tree or the node and whisper locks, while setup.ps1 installs whisper.cpp at %USERPROFILE%\.unsloth\whisper.cpp. A native Windows uninstall therefore left a prebuilt tree behind and could never prune ~/.unsloth. Adds the whisper dir, all three locks and the .stale.<pid> sweep, puts the whisper dir in the handle-releasing list so a running whisper-server does not block the delete, updates -Help, and extends the Windows smoke assertion with the same catch-all the POSIX one got. tests/sh/test_uninstall_prebuilt_artifacts.sh runs the real uninstaller, and overriding HOME does not contain that on WSL: the body detects WSL from /proc/version and then deletes Windows-side shortcuts under /mnt/*/Users and /etc/profile.d/unsloth-rocm-wsl.sh. The sibling arg-guard suite already skips there for exactly this reason; this one now does too. tests/studio/test_uninstall_prebuilt_parity.ps1 asserts the two uninstallers cover the same artifacts, since drift between them is what caused this. It cannot run the body (that kills processes and writes the registry) so it parses uninstall.ps1 instead. Against the previous uninstall.ps1 it fails on exactly the four missing behaviours. Also corrects the stale-lock comment: install_node_prebuilt.py renames then unlinks, so a stranded .stale file needs a crash between the two steps. |
||
|
Daniel Han
|
31a651a407
|
Close two false passes in the uninstaller argument tests, and document the piped help form (#7690)
* Close two false passes in the uninstaller argument tests, and document the piped help form The help loop minted a fresh fixture home per iteration but asserted the install survived only after the loop, so only the -h iteration was ever inspected. A --help that removed files would have passed. Verified: a mutant whose --help prints usage, exits 0, and deletes the CLI shim passes the existing suite and fails the updated one in three places. The two multi-argument cases asserted an exit code and nothing else, so a guard that rejected the argument only after removal had started would also have passed. Separately, -h is a shell option (hashall), so `... | sh -h` and `... | bash -h` are consumed by the shell and the script runs with no arguments, which is a full uninstall. Someone reaching for help through the documented pipe destroys their install instead. Nothing in the script can detect that, since the argument never arrives, so the usage text now spells out `sh -s -- --help` and names the hazard. A check asserts the usage keeps saying so, and a new portable case proves that form reaches the guard on every platform rather than only in the Linux-only pipe-buffer test. 32 checks pass, up from 25. * Correct the -h claim: dash and busybox reject it rather than uninstalling The help text said piping to `sh -h` uninstalls. That holds only where -h is accepted. Measured against the instrumented copy in a sandboxed HOME: bash -h rc=0 reached the uninstall body zsh -h rc=1 reached the uninstall body bash --posix -h rc=0 reached the uninstall body dash -h rc=2 Illegal option -h, install untouched busybox sh -h rc=2 illegal option -h, install untouched Debian and Ubuntu ship dash as /bin/sh and Alpine ships busybox, so the text was wrong about a destructive operation for most of the Linux audience it is addressed to. Both outcomes are now stated, and the advice is unchanged: `sh -s -- --help` is the only piped form that prints help. The usage assertion also gets its own run instead of reusing $OUT from the last loop iteration, so a failure is attributed to the right check. * Spell out the URL in the piped-help example The abbreviated form was not copy-pasteable: curl takes `.../uninstall.sh` as the URL, fails to resolve it, and the pipeline still exits 0 because the status is sh's and sh reads an empty stdin. So a user who pasted it saw a resolve error and no help, which is exactly the confusion the paragraph exists to prevent. |
||
|
Daniel Han
|
5344ec3e49
|
Tighten the comments around the uninstaller argument guard (#7647)
Shorten the header blocks in scripts/uninstall.sh and scripts/uninstall.ps1 and the two test file headers, and drop five comments that restate the line below them. The comments that carry a non-obvious reason are kept: why the trailing compound block makes a truncated curl pipe inert, why PowerShell throws rather than exits, why the real uninstall body is skipped on WSL, why the pipe test guards against going vacuous, and why the fixture path is a global. Help text is program output, not commentary, so the _usage heredoc and the _Usage here-string are untouched. No executable code is changed. |
||
|
oobabooga
|
a1dcd94846
|
Uninstaller: reject unrecognized arguments instead of uninstalling (#7631) | ||
|
Daniel Han
|
bd3972804d
|
Measure where Studio's startup time actually goes (#7553)
* Measure where Studio's startup time actually goes Nothing measured this. studio/backend/main.py logs 'lifespan startup completed in X ms' but no test or CI job ever asserted a budget, a repo-wide grep for startup_ms or time_to_ready matches only that one file, and studio_test_kit polls /healthz in a loop that discards the elapsed time it already computes. Its default healthz_timeout_s of 180 was the only recorded expectation. scripts/profile_startup.py breaks a launch into phases: import cost via python -X importtime in a subprocess (top cumulative contributors), process spawn to first output, and spawn to /healthz 200, over N repeats with median and p90. First numbers on Linux: importing the backend module costs 5.7 to 6.6 seconds before the server can even bind, and it dominates everything else. That is eager module-level imports pulled in by the routes package, not the hardware detection I first suspected: utils.hardware is 23ms and does not pull torch. --max-healthz-seconds exists so a budget can be enforced once per-platform numbers are agreed. It is not wired into a gate yet, deliberately: a threshold picked before the data is in would either be meaningless or flaky. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Profile the code under test, and let the profile fail Both installer calls omitted --local, so every phase measured the published PyPI backend and could not move when a PR edits main.py, run.py or routes. t_first_byte was a dead local, advertised in the docstring but never returned, and the reader could deadlock once the child filled the pipe. A failed launch and an impossible budget both produced a warning and exit 0, and the importtime parse reported the largest cumulative row, which is site, not main, so a raising import published a number as success. Pin the controller to the profiled venv's interpreter. * Stop the startup summary hiding failed launches The aggregates cover only the runs that reached healthz, so two dead launches and one fast one rendered as a normal fast startup, and an all-failed phase printed nothing at all. With continue-on-error and no budget wired, that summary is the only thing anyone sees. Say how many launches the number is made of, and say so explicitly when none came up. * Reject --repeats below 1 range(0) launches nothing, so the empty runs list reached the budget check as "no healthz measurement", warned and exited 0: a gate that cannot fail. The value comes straight from a dispatch input, so reject it loudly instead. * Run the startup profile when the imported startup tree changes The path filter listed main.py, run.py and routes/**, but the graph the profiler measures is far wider: main.py imports auth, core, hub, loggers, models, picker and utils at module scope, and routes/models.py imports utils.utils and utils.hidden_models. A change to any of those moved `import main` without ever running this job, so the regressions the workflow exists to catch went unmeasured. Cover studio/backend/** (tests excluded) and unsloth_cli/**, since the launch phase spawns `unsloth studio --api-only` and the CLI is on the process-to-healthz path. * Read the labelled main row and kill the Windows launcher tree total_seconds took by_cum[0], the largest cumulative row in -X importtime output. That output also carries the interpreter's own startup graph (site, encodings, whatever a venv sitecustomize pulls in), which is not part of import main, and the two are not ordered by construction. With a trivial main the old code reported site's 0.027s as "import main" while main actually cost 0.000249s. Today's backend dwarfs site so the published figures are unchanged, but the headline number must not silently become another module's cost once the backend imports get optimized, so read the row named main. profile_launch spawned Scripts/unsloth.exe on Windows. A pip console-script .exe is a distlib launcher stub that CreateProcess's the venv python and waits, so terminate() reaped the stub and left the backend holding the inherited stdout handle: the reader thread never saw EOF and burned the full 10s join, and with --repeats each iteration stranded another server on the shared UNSLOTH_STUDIO_HOME. Walk the tree with taskkill /T, matching the cleanup in unsloth_cli/commands/start.py and unsloth/dataprep/synthetic.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fail the startup budget when nothing was measured and fall back when taskkill fails * Trigger on installer inputs and harden the startup gate tests * Tighten comments in the startup profiler and its workflow * Trigger the startup profile on the Studio setup scripts install.sh --local runs the checkout's studio/setup.sh, install.ps1 reaches studio/setup.ps1 through the editable install, and both call install_python_stack.py, which decides the dependency set that gets imported. Editing any of them could change startup time with no measurement taken. * Shorten the startup profiler comments Comments and docstrings only. * Reject non-finite startup budgets and profile when the desktop argv changes --max-healthz-seconds nan or inf parses as a float but compares False against any median, so the gate reported success without bounding anything. Require a finite value. The profiler hardcodes the argv that process.rs::backend_args builds, but that file was not in the trigger paths, so a change to the desktop launch command scheduled no measurement. Add it, and anchor the two argv lists with a test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com> |
||
Leo Borcherding
|
1dd2fc4583
|
tests: read checked-in files as UTF-8 instead of the platform default (#7438)
* tests: read checked-in files as UTF-8 instead of the platform default Path.read_text() with no encoding uses locale.getpreferredencoding(), which is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine module-level reads of checked-in source files were relying on that default. studio/backend/routes/inference.py carries the DeepSeek tool-call token regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py out at collection, not as failures. Green on CI, permanently broken for a Windows contributor running the suite locally. Adds a guard: at module scope there is no tmp_path fixture, so a bare read_text()/write_text()/open() there is always touching a checked-in file. That makes the rule mechanical enough to enforce with no allowlist, while staying quiet about temp-dir I/O inside test bodies where the platform default is harmless. The repo already spells this correctly in 464 other places; this only stops the stragglers coming back. * tests: cover import-time helper reads and keep the guard py3.9-safe Follows up on the Codex review: - add `from __future__ import annotations`, since `str | None` in `_offender` is evaluated at import on Python 3.9 and pyproject declares requires-python ">=3.9,<3.15". - widen the guard from module scope to import time. Class bodies and the bodies of module-level helpers called from an executing statement run during collection too, so `CODE = _extract_mixed_precision_code()` was the same hazard as an inline read. `if __name__ == "__main__":` blocks are skipped: pytest never executes them. - scan studio/backend/tests/ as well as tests/. Both trees are collected on Windows by separate CI jobs, and the offender that started this, test_tool_xml_strip.py reading routes/inference.py, lives there. Widening it surfaced seven more import-time reads of checked-in sources; all now name utf-8. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden the import-time encoding guard for PR #7438 Close the detector gaps raised in review, all of which I reproduced against the actual AST before changing anything. False negatives (the guard let a real hazard through): - _is_main_guard ignored the comparison operator, so if __name__ != "__main__" counted as script-only even though its body runs at import. - The else arm of a main guard was discarded with the rest of the If node. - Decorators and argument defaults on a module-level def were skipped with the body, though both are evaluated when the def executes. - Path.open() in text mode was invisible; only builtin open() was matched. - encoding = None and encoding = "locale" both re-select the platform default, but the keyword merely being present counted as pinned. False positives (the guard would have blocked a compliant contributor): - A non-literal mode fell through to the "r" default, so open(p, mode) was flagged even when mode is "rb", where adding encoding= is a ValueError and there is no edit that satisfies the rule. - Same for open(*args) and a **kwargs splat, which hide the mode and can hide an encoding. - Lambda bodies and comprehension elements were walked even though neither runs at definition. Verified: still reports the same 22 offenders on unpatched main, green on this branch and on the tree merged with latest main (557 files), and an adversarial corpus of 33 cases now scores zero false positives and zero false negatives. Also corrected two docstring claims: neither collecting job runs on Windows, and the read is governed by locale.getencoding(). * Walk eager comprehensions and treat io.open as the builtin Two regressions from the previous commit, both reproduced against the AST before changing anything. Lumping list, set and dict comprehensions in with generator expressions was wrong. Only a genexp is lazy; the other three run their element expression, their filters and their nested iterators immediately, so CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time read the guard was silently missing. Comprehensions are now walked in full and only the genexp keeps the outermost-iterable-only treatment. io was also in the not-a-path-opener list, but io.open is the builtin, with the same mode position and the same platform default. io.open(CHECKED_IN_FILE) is exactly the hazard this guard exists for, so it is matched now, with binary modes and a pinned encoding still exempt. tarfile.open and fitz.open stay exempt since neither has an encoding to name. Verified: 13 targeted cases covering all five eager comprehension forms and io.open in text, binary and pinned shapes all classify correctly; still 22 offenders on unpatched main; green on this branch and on the tree merged with latest main. * Close three more walker gaps in the import-time guard All three reproduced against the AST first. A generator expression handed straight to a call is consumed there, so DATA = "".join(p.read_text() for p in paths) runs its element at import. Only an unconsumed genexp bound to a name stays lazy, so the walker now follows the consumed ones in full and keeps the outermost-iterable-only treatment for the rest. if "__main__" == __name__ is an equivalent and accepted spelling of the main guard, but requiring __name__ on the left meant its body was treated as import-time code. That is a false positive on a block pytest never runs, so both operand orders are recognised now. The helper table was built from module-level defs only, so a def in a class body invoked while the class is constructed was never followed, contradicting the walker's stated coverage of class bodies. Helpers are now collected from the module body and from class bodies at any nesting. Verified: 15 targeted cases including all three fixes and the earlier ones still classify correctly; still 22 offenders on unpatched main; green on this branch and on the tree merged with latest main. * Handle positional read_text encodings, lazy generators and nested helpers * Guard reads reached from test bodies, unbound Path calls and __file__ paths * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow derived paths, skip lazy generator helpers, cover compressed openers * Guard the CLI tests, helper parameters and unbound Path arguments * Discover test roots and follow literal, in-place and tuple-derived paths * Identify module openers by import, unwrap starred paths, pin subprocess snippets * Resolve import origins, seed helper locals, follow named generators and parametrize * Scope imports lexically, list tracked test files, bind unpacked names * Resolve aliased openers, keyword-only params, destructured targets, next() * Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438 * Harden the CLI encoding guard against detached streams for PR #7438 * Tighten the encoding guard's path and scope analysis for PR #7438 * Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438 * Resolve qualified path classes and scope conditional imports for PR #7438 * Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com> |
||
|
Daniel Han
|
d72b58a35e
|
Baseline the fastapi SSE keepalive loop after its 0.140.0 rewrite (#7480)
Some checks are pending
Unsloth GGUF CI / JSON, images (push) Waiting to run
Unsloth load-orchestrator CI / test (push) Waiting to run
Mac Studio API CI / Unsloth API & Auth Tests (push) Waiting to run
Mac Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Mac Studio GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio GGUF CI / JSON, images (push) Waiting to run
Windows Unsloth GGUF CI / GPU prebuilt resolves without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 unit tests (VS 2026 / CMake guard) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2026) (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-2025-vs2026) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-14) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio UI CI / Chat UI Tests (push) Waiting to run
Mac Studio Update CI / Unsloth Updating Tests (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Windows Unsloth GGUF CI / Tool calling Tests (push) Waiting to run
Windows Unsloth GGUF CI / JSON, images (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / VC++ runtime detect + install round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Security audit fails on main with 1 unsuppressed CRITICAL:
CRITICAL C2 polling/beaconing loop detected
Package: fastapi
File: fastapi/routing.py
The same file and check are already baselined, but the entry is keyed on a
digest of the matched code, so fastapi 0.140.0 rewriting the block reopened it.
That is the baseline working as intended, not a stale pin, so the new code
needs its own review rather than a regenerated file.
The flagged block is the SSE keepalive inserter:
async def _keepalive_inserter() -> None:
async with send_keepalive, receive_stream:
try:
while True:
try:
with anyio.fail_after(_PING_INTERVAL):
data = await receive_stream.receive()
await send_keepalive.send(data)
except TimeoutError:
await send_keepalive.send(KEEPALIVE_COMMENT)
except anyio.EndOfStream:
pass
It forwards one in-memory anyio stream to another and emits a keepalive comment
when the read times out. No socket, no outbound host, no fetched command, and it
terminates on EndOfStream. The heuristic matches it on the shape alone, a loop
with a timeout and a send, so it is a false positive.
Adds that one entry. The existing fastapi entry stays, since the requirement is
unpinned and an older resolve still needs it.
Co-authored-by: danielhanchen <unslothai@gmail.com>
|
||
Souravrajvi0
|
8c975fcbaf
|
fix: pin torchcodec for torch 2.10 and warn on ABI mismatch (#7299)
* fix: pin torchcodec for torch 2.10 and warn on ABI mismatch Add unsloth[audio] extra with torchcodec>=0.10.0,<0.11.0 and emit a clear warning when installed torchcodec minors disagree with torch (unslothai/unsloth#7225). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(packaging): address Codex review on torchcodec/torch 2.10 compat (#7299) - Postpone annotations so import_fixes loads on Python 3.9 - Align TORCH_TORCHCODEC matrix with upstream (2.9: 0.8/0.9, 2.8: 0.6/0.7) - Fix mismatch hint upper bound (<0.11.0) and gate audio-torch210 suggestion - Split audio extra per torch minor; gate torch210 pin behind python>=3.10 - Bundle audio-torch210 only in *-torch2100 install extras * fix(security): refresh openai CRITICAL scan baseline hashes (#7299) openai package code drift reopened five CRITICAL findings in the extras pip-scan-packages shard (C2 loop body hashes + IMDS/network evidence). Update the reviewed allowlist evidence/hashes so CI gates on new findings only, not benign SDK churn. * chore: retrigger CI after baseline refresh (#7299) * chore: touch scan baseline comment to retrigger security audit (#7299) * Guard torchcodec version parsing so bad version strings cannot break import * Bundle audio pin into intel-gpu-torch210 and guard the mismatch warning * Tighten comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
b0d6131567
|
Security audit: refresh scan baselines for current dependency set (#7362)
The pip scan-packages extras shard has been red on main because openai
2.47.0 changed the code inside five previously baselined findings, so
their evidence hashes no longer matched the allowlist. The hf-stack
shard was about to go red the same way: unsloth-zoo 2026.7.5 changed
two baselined test files.
All seven reopened findings were re-verified against the exact resolved
archives before re-baselining:
- openai/_base_client.py: while True in SyncPage.iter_pages, the
pagination iterator.
- openai/auth/_workload.py: Azure IMDS and GCP metadata token
providers for the documented workload identity federation feature.
- openai/resources/{beta/responses,realtime,responses}: while True in
websocket __aiter__ event loops; the loop bodies gained reconnect
handling in 2.47.0, which is what shifted the hashes.
- unsloth-zoo tests/test_vision_collator_audio.py: asserts that an
inline /tmp/a.wav path is passed through by the audio collator.
- unsloth-zoo tests/test_gemma4_forced_float32_ple_dtype.py:
compile()/exec() of the project's own generated Gemma4 PLE cast
helper source in tests.
No existing entries were removed. All three shards now exit 0 locally
against the same requirement sets CI uses.
|
||
|
Michael Han
|
d5cf96d628
|
Studio: add local speech-to-text dictation engine (#7095)
* Studio: add Voice settings tab (dictation, dictionary, read aloud) New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR. * Studio: drop the single option STT engine select, rename TTS option The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer. * Studio: harden Voice settings against edge cases found in simulation Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery. * Studio: address Voice settings review feedback Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior. * Studio: use the chat mic icon in Voice settings for consistency The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere. * Studio: address second round of Voice settings review feedback Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race. * Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item * Studio: guard dictation mic lifecycle in Voice test and Compare composer Release a microphone opened after the component unmounts, and stop Compare dictation on a permission or security failure instead of silently recording from the default device, matching the main chat adapter. * Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings - Join final dictation chunks with a space so recorded transcripts do not merge words - Ignore a stale recognizer onend so a quick stop then restart is not torn down - Use previewingRef so a double click on TTS preview does not orphan the first request - Keep the read-aloud stop control visible when a new run starts while a message is spoken - Stop the dictionary remove button from deleting an adjacent entry on a blur then click race * Studio: trim redundant Voice settings comments * Studio: fix Voice preview and Compare dictation edge cases - Only cancel the shared speechSynthesis for a system-voice preview, so stopping a Studio preview no longer stops an unrelated chat read-aloud - Release the Studio preview audio and its WAV data URL on normal completion - Iterate every finalized result in Compare dictation so batched phrases are kept - Cap persisted recent dictations to the last 20 on hydration * Studio: use clipboard fallback for recents and release failed preview audio - Copy recent dictations via the copyToClipboard helper so the execCommand fallback works in Safari and insecure http LAN contexts - Release the Studio preview audio when play() rejects, not just on ended/error * Studio: add local speech-to-text dictation engine Add an offline dictation engine that transcribes with a local faster-whisper model, alongside the existing browser (Web Speech) engine. The browser engine streams audio to Apple or Google speech services and needs internet; the new engine runs on the server, works offline, and drives any chat model without evicting it (it loads in the backend process, separate from the model subprocess). It also gives Firefox dictation, which has no Web Speech support. Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper is torch-free, so this does not disturb the existing model stack. Frontend: a Dictation engine setting (browser or local model), a curated model picker with sizes, and MediaRecorder capture posted to the transcribe route. The model warms automatically when the engine is selected, with live status. * Studio: stream local STT transcription as you speak Local dictation showed nothing until you stopped, because the whole clip was transcribed once on stop. Now the growing recording is re-transcribed on a fast pass every second and emitted as live interim text, with an accurate final pass on stop. Partial recordings decode fine, and the model refines earlier words as more audio arrives. Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast preview pass; the final stop uses the accurate path. * Studio: make local dictation stop instant and reliable Stopping local dictation waited for a final network transcription before the session ended, so the stop button did not flip and a second click ended the session early and dropped the text. Now stop commits the live transcript immediately, releases the mic at once, and ignores a second stop while finalizing. Previews run more often so the committed text is current. * Studio: record local dictation in short clips for reliable streaming Re-transcribing a growing buffer every second got slower as it grew, flooded the backend, showed stale words, and could leave the stop button stuck waiting on a backlog. Record short independent clips instead and transcribe each once, appending the text as you speak. Work per clip is bounded, so stopping is prompt (with a hard timeout as a safety net) and long dictations stay smooth. * Studio: dictate then transcribe once on stop, ChatGPT style Local STT dictation streamed by re-transcribing the growing clip, which was quadratic and saturated the backend (multi-second lag), and stop only halted the recorder without releasing the mic, so it kept recording. Record the microphone continuously, release it the instant the user stops, and transcribe the whole clip once. Stopping is immediate and the transcript lands in about a second. Also add the tiny model for the fastest option. * Studio: surface dictation and read-aloud failures instead of failing silently - Compare dictation reports microphone and speech-recognition errors via toast, reusing the main chat adapter's describeMediaError and describeSpeechError - Read-aloud toasts genuine model or synthesis failures while ignoring cancellations * Studio: ChatGPT-style recording bar for dictation Clicking the mic now drops the composer into a dedicated recording bar with a live waveform, a discard (X) and a confirm (tick), instead of a plain stop button. The tick stops recording and transcribes the clip; the X throws the recording away and keeps whatever text was already in the composer. The model adapter taps the mic with an analyser to drive the waveform, and the router tracks the live session so the X can cancel it without transcribing. * Studio: transcribe dictation while speaking, ChatGPT layout Match ChatGPT's recording layout: the bar now renders in place of the input with the left plus button kept, the waveform in the middle, and the discard and confirm buttons together on the right. Cut the post-confirm delay by transcribing in the background as the user talks. The audio is split at natural pauses (voice-activity detection off the same analyser that drives the waveform) and each clip is transcribed as it is cut, so confirming only has to finish the short final tail. The model is also warmed when recording starts so the first run never pays a cold load. * Studio: ChatGPT waveform, hide tools while dictating, faster STT Make the recording UI read like ChatGPT: the waveform is now a dense row of round dots that rise into thin centered bars, and while dictating only the plus button shows, with the mode badge and tool toggles hidden so the bar is just the waveform and controls. Speed up transcription: decode greedily (beam_size=1), which is several times faster on CPU with negligible accuracy loss on short dictation clips, and cap background segments at 6s so the final tail after confirm stays short. * Studio: finish ChatGPT voice bar and low-latency STT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: full-width waveform with a timer that freezes on stop Use the full-width waveform for the recording bar: brighter, bigger bars that advance on a fixed cadence (keeping peaks between advances) so they glide instead of racing by, inset from the composer edges. Keep a visible timer and the green confirm button, matching the ChatGPT reference, and freeze the timer and waveform the moment the user confirms. * Studio: fix multilingual local dictation * Studio: speed up dictation and release local STT * Studio: harden dictation finalization and STT decoding * Studio: restore Firefox dictation fallback * Studio: add dictation history manager * Studio: manage speech model downloads * Studio: remove em dash from voice model label * Studio: move dictation history into Voice * Studio: source local STT from Unsloth Whisper models Point the dictation STT sidecar and its Model Hub download entries at Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3) and run them through Transformers, so Studio only ever downloads Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs repos; keep the Model Hub as the only download path via local_files_only, and keep PyAV for audio decoding. Device selection uses float16 on CUDA and float32 on MPS and CPU, since Whisper's decoder is unstable in float16 on MPS and repeats tokens. Shorten the model picker labels to name plus download size and update the STT tests for the new backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: smooth dictation waveform and keep pill height * Studio: align STT model dropdown width and tidy voice copy * Studio: guide to local engine when browser dictation is offline * Studio: clarify voice section and STT model copy * Studio: keep STT warm with training-aware eviction * Harden STT lifecycle and browser compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model discovery test lint * Harden cross-browser microphone errors * Harden cross-browser microphone errors * Surface voice test recognition errors and fall back to Studio TTS - Voice test now toasts non-abort speech-recognition failures instead of ending silently, matching the main and Compare dictation paths. - Read-aloud routes to the backend model when the runtime lacks Web Speech synthesis (audio-only WebView), so it no longer errors immediately. * Fix reviewed STT lifecycle races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix read-aloud fallback controls * Guard read-aloud stop when deleting a non-speaking message aui.message().stopSpeaking() throws unless this message is the one being read aloud, so calling it unconditionally rejected the delete handler before the message was removed. Only stop speech when this message is speaking. * Cap recent dictation transcript length before persisting Recent dictations only limited entry count, so a long transcript stored the full text in the persisted voice settings and a few could exceed the localStorage quota, throwing synchronously from the uncaught dictation cleanup path. Truncate each entry on save and on hydration, matching the dictionary cap. * Studio: keep dictation mic clickable and guide to local model Register the dictation adapter unconditionally so the mic stays enabled for any engine and starts working right after switching to the local model on an already-open thread. When the browser engine cannot run (Firefox, Brave, non-secure origins), clicking the mic shows a toast that points to the local speech-to-text model instead of leaving a disabled button. The toast stacks its action below the text with a fully rounded button. * Studio: add bottom padding below the dictation guidance toast button * Studio: increase bottom padding under the dictation toast button * Studio: add bottom padding inside the dictation toast button * Studio: add five Whisper defaults and custom model search Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end. Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary. Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use public Unsloth Whisper repositories Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests. * Studio: update Whisper download sizes Reflect the cleaned public Tiny and Base repositories in the curated model labels. * Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes - Show the download size on the right of each model row so long names like Whisper Large v3 Turbo no longer hide it - Update curated Whisper sizes to the safetensors weights actually downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB - Drive the model list scroll from a wheel handler so the mouse wheel scrolls it inside the Settings dialog, not just the scrollbar - Add a search icon and shorten the placeholder to Search model * Studio: do not search when a dictation model is picked, shrink repo label - Treat the filled-in model text as a selection, not a query, so choosing a model no longer kicks off a Hugging Face search - Make the repository line under each model name smaller * Studio: tighten dictation model and local engine descriptions * Studio: keep model display on pick instead of the query, shrink row text - Guard the combobox input so selecting a model shows its name and does not echo the typed query back or start a search - Map the item label to the friendly display so picks fill the field - Reduce the model name and size text in each row * Studio: show only the model name in the dictation field, shrink size label - Drop the download size from the search field; the name alone is shown once a model is selected, with sizes kept in the dropdown list - Reduce the size label text in each row * Studio: clarify the dictation model description * Studio: drop Hugging Face from the dictation model description * Studio: move the dictation dictionary to its own Manage subpage - Replace the inline entry list with a Manage row, matching Dictation history, so a long dictionary no longer crowds Voice settings - Add a DictationDictionaryView subpage that holds the entry editor * Studio: match STT field font, use best voice for System default - Bump the dictation model field text to text-sm so it matches the engine dropdown next to it - Resolve the System default read-aloud voice to the top curated voice instead of the browser default, which is a robotic legacy voice on macOS * Studio: rerank read-aloud voices and drop duplicate voice entries - Rank by vendor quality, then the user's locale, then a preferred list of natural voices, so the best voice leads instead of the first alphabetically - Collapse voices that macOS reports twice under one name and language * Studio: fold dictionary and recents into the dictation section - Drop the separate Dictation dictionary and Recent dictations headings; their Manage rows now sit under Dictation, split by the row divider - Shorten the custom spellings description * Studio: add search and sort to dictation history - Filter saved dictations by text with a search field - Sort by newest, oldest, or A to Z; show a no-matches message - Keep Clear all available regardless of the current filter * Studio: settle cancelled STT loads before training and fix dictation review items Wait for a cancelled STT load to exit and release its memory before reporting it freed for training, so the loader cannot still be inside from_pretrained()/.to(device) holding VRAM when the training subprocess starts. A load that finishes before observing the cancel now gets unloaded so the memory is actually reclaimed. Clear the accelerator cache before the CPU fallback in load() so a failed CUDA/MPS load does not strand reserved VRAM once the sidecar is marked CPU-resident. Send the saved Hugging Face token when polling STT download progress so a gated or private repo resolves and shows the correct Load/Downloaded state instead of reporting missing. Mark the composer Dictate button as type="button" so clicking it does not also submit the draft when the composer already has text or attachments. * Studio: pin dictation settings per session and close STT startup races Capture the STT model and language when a dictation session starts and pass them to every queued segment and the warm-up load, so changing the model or language mid-recording no longer transcribes the same clip with the wrong model or a model that is not downloaded. Check the local runtime at the top of transcribe(), before the model cache lookup and the bounded audio decode, so a server missing PyTorch or Transformers returns 501 up front instead of decoding a long clip first. Treat the training startup window as active for STT device selection. start_training frees VRAM in before_spawn but only assigns _proc later, so a concurrent STT load could take the GPU that was just cleared. A startup flag now reports training active from the free until the process is live, forcing those loads to CPU; a finally clears it on every exit. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stub the STT runtime check in transcribe orchestration tests transcribe() now verifies the local runtime up front, so the unit tests that exercise transcription orchestration must treat the runtime as present to keep passing where PyTorch, Transformers, and PyAV are not installed. Stub ensure_stt_available in the shared fixture and restore the real check in the availability and load-rejection tests. * Harden custom Whisper dictation models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add whisper.cpp dictation engine with per-engine downloads and history rework Engines - New GGML STT sidecar that runs a managed whisper-server subprocess with idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh) - Dictation engine picker now offers Browser, Local transcription (whisper.cpp), and Local transcription (Transformers) - Both local engines serve the same five curated Whisper models and download them directly with byte-level progress reported by /audio/stt/status - Models auto load on selection and when their download finishes - Unload and training admission account for both engines Benchmarks (Apple Silicon, greedy, warm, same checkpoints) - whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in about 0.45s vs 0.86s for Whisper Small - whisper.cpp GGUF path is unchanged by the Transformers addition (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s) Voice settings UI - Plain curated model select replaces the searchable combobox - Single download progress bar with transfer rate for both engines - Dictation history now stores every dictation with Show more pagination, a top Clear history action, and links back to the chat it was spoken into - Archived chats dialog gets the same pagination - Delete dialog offers deleting a dictation together with its chat Tests: 88 backend STT tests pass, including new snapshot download coverage. Frontend typecheck, lint, i18n parity, and production build pass. * Merge local engines into one option and source GGML models from unslothai Engine selection - The dictation engine dropdown is back to two choices: Browser and Local transcription. The selected model decides the backend: curated ids run GGML checkpoints through whisper.cpp, searched Hugging Face repositories run safetensors through Transformers - Model picker lists the curated models and searches Hugging Face for other Whisper repositories, validating them before selection. The trigger is a plain button so the selection never renders inside a text input - /audio/stt/status accepts a model query param so downloaded state works for custom repositories; the engine param on load, transcribe, and download routes is derived from the model everywhere Model source - Curated GGML checkpoints now download from the Unsloth-hosted unslothai/whisper-*-GGUF repositories (one repo per model) instead of ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob tracking are per-model Fixes - Voice settings and dictation history were not persisting: the quota-safe localStorage wrapper was declared after the store that uses it, so the persist storage factory failed silently. Every settings write also threw mid-click, which kept the model picker popover from closing on selection - is_model_downloaded now verifies config, preprocessor config, and real weight files instead of trusting an offline snapshot lookup, so a partial download left by an aborted fetch shows the Download button instead of failing to load - Removed whisper.cpp mentions from user-facing text: the ready status shows Loaded instead of the runtime name, picker rows show the source repository, and runtime error messages say local transcription runtime Verified with automated browser sessions and live API checks: selection closes the picker with no page errors, persisted settings hydrate on reload, a stale partial snapshot triggers download then loads on MPS and transcribes, and curated models download from the unslothai repos. 88 backend STT tests, typecheck, lint, i18n parity, and build pass. * Skip the duplicate source line for custom models in the STT picker A custom repository's display name is its id, so search results and the appended current selection rendered the same string twice. The source line now only renders when it differs from the name; curated rows keep their name, unslothai source repository, and download size. * Verify every shard of a sharded checkpoint in the downloaded check A snapshot holding one of N shards (or a corrupt shard index) passed the downloaded check and then failed at load. When model.safetensors.index.json exists, every shard in its weight map must now be present. Found by simulation; covered by a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename stale _starting references in the pump resilience tests The startup flag on TrainingBackend was renamed to _spawn_in_progress but two tests added alongside it still asserted on the old name, failing the Python 3.11 to 3.13 CI jobs. * Make the selected model row clearly highlighted in the STT picker The current selection was a faint background tint. It now uses the accent background with a medium weight name. Two line rows use a small corner radius; single line custom repo rows keep the pill shape. * Address review feedback on STT snapshot checks, VRAM release, and dictation UX Verify snapshot completeness in the load preflight so a partial download fails before the audio is decoded, for curated and custom repos alike. Drop the failed accelerator traceback before the CPU retry so the cache clear can actually release that memory. Keep unloading the GGUF sidecar after cancelling an in-flight Transformers load; both engines can hold memory at once. Allow Auto language with English-only .en checkpoints, matching the backend which sends no forced language. Keep the discard button usable while a transcription is pending so a slow or hung request cannot trap the composer in dictation mode. Stop linking Compare and settings test dictations to the unrelated active single chat thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move the CPU retry out of the exception handler On Python 3.10 the interpreter exception state keeps its own reference to the traceback, so dropping it from the caught exception was not enough to release the failed accelerator load during the retry. Leaving the handler before clearing the cache works on every supported version. * Address review feedback on session handoff, chat pinning, and server lifetime Starting a dictation from a second entry point now cancels the session it replaces, so the old recording cannot keep the microphone open or save a transcript with no discard button pointing at it. The linked chat is pinned when recording starts, so switching threads while a transcription finalizes cannot relink the transcript to the newly opened chat. whisper-server is now bound to Studio's lifetime like the other long-lived children: PDEATHSIG on Linux, the parent job object on Windows, and pid adoption so the shutdown sweep reaps it; before this it survived a Ctrl+C exit as an orphan still holding the model. * Remove the dictation mic test from Voice settings The composer dictate button covers the same check, so the test row, its transcript panel, the unsupported fallback row, and their strings and search entry are gone. * Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits GGUF (whisper.cpp) sidecar: - Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed. - Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind. - Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription. - Reject a missing model before decoding audio, matching the Transformers download preflight. Voice settings: - The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it. Dictation dictionary: - Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fix curated GGUF whisper filenames to match hosted repos The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin, not ggml-<id>.bin, so every curated dictation download and cached-path lookup 404'd and the whisper.cpp engine could never load a model. Point GGML_STT_MODELS at the real filenames and guard the naming with a test. * Studio STT: validate a custom dictation repo before downloading it The Transformers STT engine accepts an arbitrary owner/model repo, but the download route handed it straight to snapshot_download, pulling a possibly large non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper checkpoint first with the existing metadata-only validate_remote_model (no weights); curated ids short-circuit and the GGUF engine (curated-only) is unaffected. A non-Whisper repo now 422s before any download. * Studio STT: preempt a still-loading GGUF server for training admission A whisper-server still in its startup window binds accelerator memory but has no loaded_model yet, so training admission could miss it and launch into an OOM. Make the GGUF startup cancellable (cancel_pending_load signals an abort event and terminates the starting process without the load lock; _wait_for_server observes it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock until the killed server is reaped), and always fold the GGUF sidecar into the resident-STT summary so a resident Transformers model cannot mask a loading GGUF server. free_stt_model_for_training now cancels an in-flight load and waits for it to settle before training claims the memory. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fall back to Transformers when whisper-server is absent A curated dictation model (including the default small) hard-pinned the GGUF engine, but standard installs do not ship whisper-server, so every recording 501'd instead of using the Transformers engine that serves the same checkpoint -- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine: a GGUF request for a curated id (the only ids GGUF accepts, all Transformers- servable) downgrades to Transformers when whisper-server is unavailable, applied consistently to download, load and transcribe (not unload, which targets a specific engine). The Voice tab likewise falls back to the Transformers status so the model is not shown unavailable and download is not blocked. * Studio STT: hide custom Whisper caches from the legacy model pickers The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with only the owner/model id, which cannot reach the config-based Whisper check, so a downloaded custom (non-curated) Whisper checkpoint was still offered as a chat model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo config and hides it, matching the discovery route. * Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction - Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat model inventory and pickers, backend and frontend. Only their Transformers safetensors companions were hidden; the GGUF repos use a different org and a -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked into chat pickers. - Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the Transformers sidecar. transcribe() holds self._lock across the whole inference call, so /audio/stt status polls and training admission previously blocked behind an in-flight transcription. - stt_unload resolves through the serving resolver: a "gguf" pick on a host without whisper-server is served by the Transformers fallback, so unload must target that engine or the resident model is never freed. Unload also attempts every engine even if one raises, so a failure freeing one backend no longer skips the other. - free_stt_model_for_training frees the Transformers and GGUF sidecars under independent exception boundaries so a failure unloading one no longer skips the other before training claims the memory. Adds tests/test_stt_review_fixes.py covering all four. * Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness - The model dictation adapter sent the raw setting (the literal "auto") to the backend, while the browser engine resolves Auto via resolveDictationLanguage. A batch of non-English voice notes came back mostly English on Auto. Add resolveModelDictationLanguage: only the literal "auto" is resolved to a concrete locale, gated so it becomes a language the model AND Whisper can honor (mirroring the backend's known-whisper-languages set); an explicit language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire it into both adapter call sites. - GgmlSttSidecar._process_alive() read self._process twice; a concurrent unload() nulls it under the lock while loaded_model/device read lock-free, so a null between the two reads called None.poll(). Snapshot once. Adds a deterministic regression test. * studio: tighten comments and docstrings in the dictation modules * studio: harden dictation model downloads, GGML readiness, and recording paths Address review findings on the STT dictation feature: - build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom Studio home unless it carries the Studio ownership marker, matching the setup.sh policy, and marks trees it creates - _snapshot_is_complete validates every shard of a sharded PyTorch (pytorch_model.bin.index.json) checkpoint like the safetensors path, and requires tokenizer assets (tokenizer.json or vocab.json + merges.txt) - custom-repo downloads pin the revision resolved at validation time and restrict snapshot_download to the model/tokenizer/config/preprocessor file classes Studio loads - the GGML sidecar holds its port reservation until just before spawning whisper-server and only accepts readiness from a responder that both looks like whisper.cpp's server and belongs to the still-running managed child, probing twice, so mic audio cannot be posted to a foreign local process - the recording adapter transcribes every non-empty segment; the RMS meter only shapes segment boundaries and can no longer discard quiet speech - Compare-pane dictation can cancel a pending transcription on second click, with the button relabeled while finalizing - localStorage quota recovery halves the dictation history until the save fits, so small histories shrink too - the System default TTS voice resolves to the platform default voice - new dictation UI imports go through the chat and hub feature barrels Regression tests cover the build-script gate, sharded PyTorch and tokenizer completeness, revision pinning and allow patterns, and the whisper-server readiness probe. * Fix STT download and voice picker follow-ups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add dictation button regression coverage * Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294) * Studio STT: add prebuilt whisper.cpp (whisper-server) installer New install_whisper_prebuilt.py downloads a per-platform whisper-server bundle published by the unslothai/whisper.cpp prebuilt CI into the managed whisper.cpp dir (build/bin/whisper-server) so local dictation needs no compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py: host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the trust anchor, staging + install lock + atomic swap, traversal-safe extract, co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired into setup yet; the pins ship empty so every asset fails closed until the first fork release is published and its digests are reviewed in. * Studio STT: install prebuilt whisper.cpp during setup and update Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so `unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server into the managed whisper.cpp dir the sidecar discovers. It skips a user-set WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL, forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence. * Studio STT: harden whisper-server child env + WSL ROCm detection - Sidecar spawns whisper-server with a scrubbed child env that prepends the binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child. - find_whisper_server_binary now requires an executable, not just a file. - Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only; gfx parsing skips the gfx000 CPU agent and generic ISA lines. - Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the executable check, and the WSL rocm detection. * Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can detect and install a newer whisper-server release from inside the app: - backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json and compare the installed release against the newest unslothai/whisper.cpp release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a (major, minor, patch, serial) key with a strict downgrade guard; 24h cache; fail-open. - backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch and atomically swap the newest bundle, unloading the warm GGUF sidecar first. - backend/routes/whisper.py mounted at /api/whisper (update-status + update). - pyproject: add whisper_prebuilt_pins.json to studio package-data so the installer's trust anchor ships in the wheel (it is a data file, not a .py module, so package discovery alone does not include it; node_prebuilt_pins.json is listed for the same reason). Without this a pip-installed wheel had no pins and the prebuilt install aborted to Transformers STT. Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade guard, marker layouts, stale decision, fail-open). * Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust model: instead of a committed whisper_prebuilt_pins.json, verify every download against the release's own whisper-prebuilt-sha256.json checksum index, fetched from the same GitHub release. - parse_release_checksums / fetch_release_checksums / expected_sha256_for replace the pins layer. The index is validated for schema/component and that its release_tag matches the resolved release; an asset absent from it, a release that does not publish it, or a manifest sha256 that disagrees with it all fail closed to a source build. - resolve_release_tag now resolves the newest published release at runtime (or an explicit --published-release-tag), matching llama and the freshness check; removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in. - Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data entry (nothing to ship now, same as llama which has no committed pins). - Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on uncovered asset, tampered-manifest guard, newest-release resolution). This is a same-origin checksum (integrity, not authenticity), identical to the llama.cpp installer; pair releases with GitHub artifact attestations for provenance. * Resolve whisper prebuilt release via the download host (no GitHub API) Mirror install_llama_prebuilt.py's fast path: resolve the release tag from the releases/latest redirect and fetch the manifest + checksum index from constructed releases/download URLs, so the common install path makes zero api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour per IP; the download host is not). Fall back to the GitHub API only on a 404, malformed asset, or tag mismatch. * Studio STT: coverage-aware whisper prebuilt selection via a shared core whisper's select_artifact returned the first os/arch/backend manifest match and ignored the SM-coverage fields the release manifest already carries, so a Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks cuda13-newer. Extract the coverage-aware selection into a shared, component-agnostic core under studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and generalised over a normalised artifact. whisper's HostInfo now records the GPU compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and select_artifact routes CUDA/ROCm through the shared selector: every visible SM must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes, and "already matches" contract are unchanged. On the B200 the installer now resolves cuda13-newer, matching llama. * Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship libcudart/libcublas -- they load the same runtime the host already has. So the driver's advertised CUDA version is only an upper bound: a cuda13 bundle still needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the shared core and intersect it with the driver-compatible lines in select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g. torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13 one; a host with no CUDA runtime at all falls back to CPU. Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator truthiness, not a match) that made every major report present; add a real filesystem test that exercises the scan. * studio: harden shared prebuilt core to full llama parity Apply the review findings on the shared coverage-aware prebuilt-consumer core so whisper.cpp selection is exactly equivalent to the llama.cpp path. hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an index/UUID selector now reports has_usable_nvidia False instead of staying usable, via supports_explicit_visible_device_matching plus the physical / explicit-match branches, and _select_visible_rows now matches rows the way llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus fallback and has_physical_nvidia. Adds parse_macos_version. runtime_libs.py: the Linux on-disk scan now requires the exact libcudart / libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare versioned file without the SONAME symlink no longer counts as loadable. Hardens the ldconfig parse against an empty left-hand side. selection.py: fix the Blackwell/torch reordering so it keys on the covering runtime lines (falls through to the torch preference when the covering lines were filtered out), matching linux_cuda_choice_from_release. Corrects the compatible_runtime_lines_for_driver docstring: the bundles do not ship the CUDA runtime, so the driver version is only an upper bound and the caller must intersect with the on-disk scan. install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new HostInfo.macos_version) so a bundle that cannot load on the host OS version is dropped. Keep resolver stdout to only the JSON line by leaving logs on stderr in --resolve-prebuilt mode, and map an unexpected probe failure to prebuilt_available False instead of a traceback. Tests: new host-probe suite for the visible-device logic, exact-SONAME runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON, exit-code mapping, and the repo key. * studio: fix whisper prebuilt selection + launch parity gaps from review A parallel review surfaced integration defects where the whisper path could select or launch a bundle that cannot run on a concrete host. Each is fixed to match install_llama_prebuilt.py. macOS min_os: the manifest labels macOS requirements as macos-<version> (e.g. macos-14.0), which the version parser could not read, so the guard was a no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the platform prefix before parsing. ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact ROCm matching treats that token as the active GPU, a mixed APU + dGPU host (gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU sections and honors the visibility vars (empty / -1 -> no AMD GPU). --rocm-gfx override: recording the arch without setting has_rocm left the host on its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies has_rocm and clears NVIDIA state, like llama's _apply_host_overrides. CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so on a host whose CUDA runtime lives only in the PyTorch wheels the selection would gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch runtime dirs to the child loader path for CUDA bundles (bundle dir still first), mirroring binary_env. Also normalize a manifest artifact's supported_sms defensively (parity with llama's parser) and document that blackwell_min_toolkit_for_caps is retained for the Phase B llama Windows path. Not changed (verified parity, not defects): Linux/Windows min_os is enforced nowhere in llama (macOS only); the resolver is optimistic about the checksum index and the install path verifies. * studio: tighten prebuilt-core code comments * studio: lift shared prebuilt installer core out of the whisper installer * studio: reuse the llama.cpp prebuilt installer machinery for whisper * studio: unify llama and whisper prebuilt installers on a shared descriptor core * studio: consolidate prebuilt installer tests into the shared core suite Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every component-agnostic behavior runs against both descriptors: the full seven profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle stability, missing SM metadata, dotted SM normalization, no-driver fallback policy), the ROCm gfx family matrix, macOS min_os gating and its helper, backend resolution incl. cpu-fallback precedence and Intel-mac auto detect, checksum-index non-object and plain-lookup cases, the tar symlink/hardlink extraction guards moved from the llama suite, and the compute-cap, visible device, runtime-line and Blackwell helper value tables moved verbatim from the llama characterization suites. Delete only tests whose exact behavior the master now asserts for the same component: 40 pure-alias helper cases in test_selection_logic.py (replaced by value-identical master tables plus an alias-identity pin), 6 extraction moves and the master-absorbed zip-symlink case in the llama logic suite, 3 routing twins in test_rocm_support.py already pinned byte-for-byte in test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by the master whisper parameterization. Wrapper wiring pins, the llama release plan dialect, fingerprints and every llama-only behavior stay untouched. * studio: dedupe sidecar and update helpers into the backend prebuilt package * studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow * studio: consume paired slim whisper prebuilts via the llama ggml runtime * studio: serve every whisper backend from slim prebuilts * studio: drop the whisper fat per-accelerator selection chain unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan selection glue; keep slim selection + pairing, link_ggml_runtime, and one legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim release. Exit 2 now reads as prebuilt unavailable (whisper never source builds); setup already treats it that way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire libomp runtime DLL alongside ggml in slim whisper installs llama's clang-built windows-arm64 ggml-base.dll imports libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL. Without it next to whisper-server.exe the loader fails with STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64 was affected. The empty-runtime guard still requires a real ggml library; libomp alone is not a pairing. * studio: drop whisper-side fat-selection support structure Slim whisper bundles are selected per os/arch only; all accelerator capability comes from the installed llama.cpp prebuilt, whose installer already did the coverage-aware selection. Remove the machinery that only existed to pick among fat per-accelerator whisper bundles: - prebuilt_core: delete the generic CUDA/ROCm coverage selection (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters, detected_cuda_runtime_lines, the exact-SONAME linux probe) that no shipped component routes through; llama keeps its own selection chain and whisper shadows select_artifact with the slim-only version. select_artifact is now a plain os/arch/backend first-match. - install_whisper_prebuilt: drop the HostInfo CUDA fields (compute_caps, driver_cuda_version, torch_runtime_line) and the torch runtime probe that populated them; nothing reachable reads them, and the resolver payload sources runtime_line from the artifact. - whisper_cpp_update: delete the standalone start_update job worker; whisper applies only run as the chained phase of the combined llama+whisper update. The status payload keeps its job field (idle). - routes/whisper: drop the progress logger that could never fire. - tests: remove tests of the deleted paths and tests duplicating the descriptor-parameterized core suite or the llama freshness suite. Contracts unchanged: resolver JSON keys, exit codes, marker fields, pairing logs, and the pinned pre-slim fat CPU escape hatch. * Address review feedback on the whisper prebuilt update and install paths - Pin the chained whisper phase to the release the freshness check offered, so the download-host latest pointer cannot reinstall an older build in a loop - Wire the whisper prebuilt install into setup.ps1 (Windows setup previously skipped it entirely) - Treat a non-executable server or missing wired ggml libraries as a broken install instead of reporting already matches - Keep whisper sidecar reloads out of the job-level reload flag and resync chat state after a partial chained update that unloaded llama - Repoint home and profile vars for the whisper-server subprocess at a managed scratch dir and drop credential-store pointers - Clear the prebuilt marker before the opt-in source build overwrite - Write the prebuilt marker with explicit utf-8 encoding * Tighten comments in the whisper prebuilt consumer * Harden the Windows whisper setup phase and the chained update edges - setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH / UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard before the atomic install, and forward the release-tag pin and ROCm hints like setup.sh - sidecar: a cpu-selected install launches whisper-server with --no-gpu (slim wiring links every llama backend, so the flag is what keeps a deliberate CPU choice off the GPU) - chained update: leave whisper unpinned on macOS (the llama phase can walk back there, and a newest-tag pin could be an impossible pairing on every retry) and treat installer exit 2 as kept-existing-runtime instead of failing the combined job - job.to_tag now comes only from the llama phase, so a whisper-only round cannot report a llama update that never ran * Fix slim whisper runtime follow-ups * Address remaining whisper update reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address remaining prebuilt update reviews * Fix remaining chained update reviews * Fix remaining whisper runtime review edges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> |
||
|
Daniel Han
|
f2f41bf9b1
|
Baseline two benign unsloth-zoo test-file findings in scan_packages (#7325)
The enforcing pip scan-packages hf-stack shard fails on two CRITICAL staged-dropper findings in unsloth-zoo test files: tests/test_mlx_save_export_regressions.py and tests/test_vision_collator_audio.py. Both are false positives: the combination heuristic matches a /tmp path literal alongside unrelated subprocess/import references in the same file, but those are mocked test fixtures (monkeypatch.setattr on subprocess, asserted /tmp path strings), not droppers. Add both to the reviewed allowlist so the gate stops red-failing on legitimate test code. The scan then exits 0 on both the hf-stack shard and a direct unsloth-zoo scan. |
||
|
Michael Han
|
6d8c18cd1a
|
Replace standalone Studio wording with Unsloth (#7221)
* Replace standalone Studio wording with Unsloth Replace the single word Studio with Unsloth wherever it is used as shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n locales, workflow display names, comments and docstrings. Kept unchanged: the full name Unsloth Studio, third party product names (LM Studio, Visual Studio, Mac Studio), feature names (Recipe Studio, Fine-tuning Studio and its translations), and all identifiers such as env vars, commands, paths and filenames. * Address review feedback on the Studio wording rename Use "an" before Unsloth where the rename left the article as "a". Restore the split brand where Unsloth and Studio render as two halves of the full product name: the onboarding sidebar subtitle and the IPv6 localhost warning. Scope two messages to the full name Unsloth Studio where plain Unsloth was misleading: the AMD README bullet and the CLI studio setup error. |
||
|
Daniel Han
|
030524ae8e
|
security: refresh the fastapi C2-loop baseline entry for the current release (#7223)
The pip scan-packages studio shard is red on main and on every open PR: the baselined fastapi finding (the benign SSE keepalive `while True:` loop in fastapi/routing.py, reviewed and suppressed long ago) records its evidence at L586 with the span digest of the fastapi release current at baseline time. The latest fastapi shifts that loop to L587 and its span digest with it, so the evidence hash no longer matches and the scanner reports the finding as new, failing the shard with one unsuppressed CRITICAL. Re-reviewed the flagged code in the current release before refreshing: L587 is the same keepalive loop inside the streaming response machinery, not a beacon. Only the one entry's evidence and evidence_hash change. Verified with the scanner itself: `scan_packages.py fastapi --no-baseline` reproduces the exact CI evidence string, and with the updated baseline the same scan exits 0 with the finding suppressed as 1 CRITICAL baselined. |
||
|
Daniel Han
|
744b59f04a
|
scan_packages: baseline sentencepiece dup2 finding after upstream reindent (#7120)
The supply-chain scan gates on non-baselined CRITICAL/HIGH findings. A newer sentencepiece release reindented the stdout/stderr fd-redirect helper in sentencepiece/__init__.py (the os.dup2 pair the heuristic flags as a reverse/bind-shell pattern), moving it from L1221/L1226 to L772/L777 and changing its leading indentation. The baseline key is (package, package-relative file, check, evidence_hash), where evidence_hash is over the matched code with the L<NN>: markers stripped but the code's own indentation preserved. The reindent therefore changed the hash (bba233.. -> 65b5a11c..), so the existing entry no longer suppressed the finding and it resurfaced as a blocking CRITICAL in the hf-stack and studio scan legs. Add the new indentation variant to the allowlist. The calls are sentencepiece redirecting stdout/stderr file descriptors to capture its C++ logs, not a shell; no socket or networking is involved. The old L1221 entry is kept so both versions stay covered. |
||
|
Daniel Han
|
9fa6fd40e1
|
scripts: refresh scan_packages allowlist baseline (#7078)
Some checks are pending
Studio Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Studio UI CI / Chat UI Tests (push) Waiting to run
Studio Update CI / Studio Updating Tests (push) Waiting to run
Windows Studio API CI / Studio API & Auth Tests (push) Waiting to run
Windows Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Windows Studio GGUF CI / Tool calling Tests (push) Waiting to run
Windows Studio GGUF CI / JSON, images (push) Waiting to run
Windows Studio GGUF CI / Studio install + inference without Visual Studio (push) Waiting to run
Windows Studio GGUF CI / GPU prebuilt resolves without Visual Studio (push) Waiting to run
Windows Studio GGUF CI / setup.ps1 unit tests (VS 2026 / CMake guard) (push) Waiting to run
Windows Studio GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Studio GGUF CI / real-VS detection (VS 2026) (push) Waiting to run
Windows Studio GGUF CI / VC++ runtime detect + install round-trip (windows-2025-vs2026) (push) Waiting to run
Windows Studio GGUF CI / VC++ runtime detect + install round-trip (windows-latest) (push) Waiting to run
Windows Studio UI CI / Chat UI Tests (push) Waiting to run
Studio GGUF CI / Tool calling Tests (push) Waiting to run
Studio GGUF CI / JSON, images (push) Waiting to run
Mac Studio API CI / Studio API & Auth Tests (push) Waiting to run
Mac Studio GGUF CI / OpenAI, Anthropic API tests (push) Waiting to run
Mac Studio GGUF CI / Tool calling Tests (push) Waiting to run
Mac Studio GGUF CI / JSON, images (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-14) (push) Waiting to run
Mac Studio Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Studio UI CI / Chat UI Tests (push) Waiting to run
Mac Studio Update CI / Studio Updating Tests (push) Waiting to run
Windows Studio Update CI / Studio Updating Tests (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
New releases of huggingface-hub (1.23.0) and openai (2.45.0) shifted or added polling loops that the C2 polling/beaconing check flags, failing all three pip scan-packages shards (studio 1, hf-stack 1, extras 3 new CRITICAL findings) org-wide including on main. Regenerated with scan_packages.py --write-baseline per CI shard (same shard-to-requirements mapping and --with-deps as security-audit.yml) and merged. All entries were manually reviewed at the resolved versions: - huggingface-hub hf_api.py: create_repo 409-concurrency retry loop body changed in 1.23.0; refreshed evidence hash. The loop POSTs to the canonical Hub endpoint and retries only on a specific conflict error. Benign client retry. - openai beta/threads/runs/runs.py: create_and_poll run-status helper refactored in 2.45.0 (Assistants deprecation annotations); refreshed evidence hash. Documented polling helper against api.openai.com. - openai beta/responses/responses.py: new beta websocket client whose __aiter__ yields server events until the connection closes. New entry; standard event-stream iterator, not beaconing. - openai resources/responses/responses.py: evidence line number refreshed only, hash unchanged. The two dropped entries are the pre-refactor hashes of the same two loops above; they no longer occur at the resolved versions. Verified locally: all three shards exit 0 with 0 unsuppressed CRITICAL/HIGH (hf-stack 120, studio 151, extras 99 suppressed). |
||
|
Daniel Han
|
b5dca66cb1
|
scripts: refresh scan_packages allowlist baseline (#7032)
* scripts: refresh scan_packages allowlist baseline Regenerate scripts/scan_packages_baseline.json against the current resolved dependency set so the blocking pip scan-packages gate matches what the scanner now finds. Refreshes evidence hashes for benign findings whose code shifted lines (unsloth-zoo mlx loader, gguf/mlx test /tmp fixtures) and adds two mainstream-library entries that were newly surfaced (torch inductor codecache base64+subprocess compile cache, torch testing common_utils socket import). Stale entries whose matching code changed and no longer triggers are dropped. All entries remain CRITICAL/HIGH findings manually judged benign; matched on (package, file, check, evidence_hash). * ci(security-audit): re-run scan when the allowlist baseline changes The security-audit pull_request trigger listed the scanners but not their allowlist baselines, so a baseline-only edit never re-ran the scan that consumes it. A refreshed baseline could therefore merge without CI confirming its evidence hashes match what the scanner finds. Add scan_packages_baseline.json and scan_npm_packages_baseline.json to the paths filter so baseline changes are validated on their own PR. |
||
|
Leo Borcherding
|
296cacb5a1
|
ROCm-on-WSL: support discrete Radeon (RDNA 3/4) in WSL, not just Strix Halo (#6915)
* WSL ROCm: generalize ROCm-on-WSL bootstrap from Strix-only to any RDNA arch install_rocm_wsl_strixhalo.sh hardcoded gfx1151, so its verify step died on discrete Radeon cards even though the ROCm + librocdxg setup is arch-agnostic. Auto-detect the GPU arch from rocminfo (override via UNSLOTH_WSL_GFX), verify any GPU agent enumerates over DXG, and map the arch to AMD's per-arch wheel family for the optional smoke test (injecting librocdxg into torch/lib so torch's bundled ROCr finds the DXG bridge). Verified on gfx1200 (Radeon RX 9060 XT) in WSL2 + Ubuntu 24.04 -- torch.cuda now enumerates the GPU. * WSL ROCm: trigger the ROCm-on-WSL bootstrap for discrete Radeon GPUs too _maybe_bootstrap_rocm_wsl only fired for Strix APUs (matched via /proc/cpuinfo, which discrete cards don't appear in). Add _wsl_amd_gpu_name() -- queries the Windows host via WMI -- and broaden the trigger gate plus the 'already-usable ROCm' rocminfo check from gfx1151-only to any real GPU agent (gfxNNNN, excluding the gfx11-generic fallback ISA). The generalized bootstrap then auto-detects the arch. Enables 'curl install.sh | sh' to set up ROCm-on-WSL on discrete Radeon RX 7000/9000 in WSL2 + Ubuntu 24.04, not just Strix Halo/Point. * WSL ROCm: address review -- filter generic ISA in bootstrap, bound the host GPU query - install_rocm_wsl_strixhalo.sh: exclude the gfx11-generic fallback ISA in arch detection (grep -v generic), matching install.sh's rocminfo check, so a generic agent listed before the real one can't be picked as the arch. - install.sh: wrap the powershell.exe Win32_VideoController query in _run_bounded (10s timeout) so an unstable WSL-interop / busy host can't hang the installer. * WSL ROCm: harden arch-detect + librocdxg copy under set -eo pipefail (review) - _detected_gfx: append '|| true' so a no-GPU rocminfo (empty pipeline, non-zero under pipefail) doesn't abort the assignment before the '[ -z ]' branch prints the diagnostic + die message. - smoke-test librocdxg copy: gate on '[ -d "$_tlib" ]' instead of '[ -n ]' so a non-directory value can't make cp rename librocdxg to 'lib'. * WSL ROCm: address Codex review (gfx000, 24.04 reroute for discrete, test locator) - Exclude gfx000 (the CPU agent) from the WSL 'usable ROCm' check and the bootstrap arch-detect: match gfx[1-9] (nonzero arch), so a partial ROCm install that only reports the CPU ISA no longer short-circuits the librocdxg setup. (P2) - Reuse the Ubuntu-24.04 reroute for discrete Radeon: broaden _maybe_reroute_strixhalo_to_2404's gate with the same _wsl_amd_gpu_name (WMI) fallback, so a discrete card on 26.04 reroutes to a 24.04 distro like Strix does instead of falling to CPU. Moved _wsl_amd_gpu_name above the reroute and made it self-contained + 10s-bounded (it runs before _run_bounded is defined). (P2) - Update TestInstallShDropinPersistence to locate the gate by its unique '!/generic/' clause now that the gfx1151 literal is gone. (P1) * Condense ROCm-on-WSL comments in install.sh and bootstrap helper * Guard WSL reroute from NVIDIA hybrid hosts and fix GFX-override pipefail check * Honor CUDA_VISIBLE_DEVICES-hidden NVIDIA in the WSL reroute guard * Reuse _has_usable_nvidia_gpu in the WSL reroute guard --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> |