Speed=off contract: the companion (text encoder / VAE) suppression under
an explicit Speed=off only matched an UNSET request, but auto is
backend-owned like transformer_quant, so an explicit
text_encoder_quant/vae_quant=auto would still engage fp8/int8 and break
the bit-exact request. Match 'auto' as well in both the image and video
loaders (a concrete scheme still forces quant). Covered by new
explicit-auto suppression tests.
Benchmark accuracy:
- quant_speedmem_bench teacc: when quantize_text_encoders returns None
(scheme skipped) the encoder is still dense, so scoring it against the
dense reference falsely certified a scheme that never ran. Record it
NOT engaged instead of collecting accuracy metrics.
- quant_speedmem_bench e2e: report the actual engaged te/vae scheme,
falling back to dense (not the requested auto) when the caster stayed
bf16, so a no-op default is not mislabelled as an auto-quantised run.
- video_speedmem_bench: HunyuanVideo ignores callback_on_step_end, so
step_ts stayed empty and per_step_ms was published as 0.0 for every
row. Time the denoise via a scheduler.step wrapper for that path.
The explicit text_encoder_quant=nvfp4 path gated on the transformer
smoke probe, which builds the dynamic-activation NVFP4 config, while the
TE caster _cast_nvfp4 applies weight-only NVFP4WeightOnlyConfig. On a
Blackwell build that carries the weight-only FP4 path but not the
dynamic FP4 GEMM, the probe would fail and the encoder would silently
stay dense even though the caster would run. Add a dedicated weight-only
NVFP4 smoke probe (mirroring _cast_nvfp4's config) and route TE nvfp4
through it; int8 / fp8_dynamic keep the dynamic transformer probe since
their TE casters are also dynamic-activation.
The video route forwards vae_quant to validate_load_request, but the
_FakeBackend stub in test_video_routes.py had not been updated to accept
it, so all 11 route tests raised TypeError: unexpected keyword argument
'vae_quant'. Add the keyword to mirror the real backend signature.
The main merge left listStoredChatThreads imported twice -- once as a standalone import from
the deep utils path and once via the @/features/chat barrel (which re-exports it) -- tripping
TS2300 'Duplicate identifier' and failing the Tauri frontend build. Keep the barrel import,
grouped with the other chat imports.
The main merge left listStoredChatThreads imported twice -- once as a standalone import from
the deep utils path and once via the @/features/chat barrel (which re-exports it) -- tripping
TS2300 'Duplicate identifier' and failing the Tauri frontend build. Keep the barrel import,
grouped with the other chat imports.
Address the Codex review round on the video/quant work:
- Companion auto-quant now honors an explicit Speed=off. Both loaders already pin the DiT dense
under an explicit off (bit-exact reference), but the unset text-encoder / VAE quant still promoted
to auto and silently fp8/int8'd the companions, breaking the bit-exact request. An UNSET speed
still auto-quantises; an explicit companion scheme still forces it.
- The HunyuanVideo joint-attention trim is a speed lever (it swaps to the fused SDPA kernel), so gate
it on a non-off speed tier exactly like the adjacent attention-backend selection -- the off path
keeps the stock dense-mask attention.
- Explicit torchao text-encoder modes (int8 / fp8_dynamic / nvfp4) now run the same kernel smoke
test the auto ladder uses. They could clear the capability gate yet fail the real GEMM on a build
where quantize_ wraps the encoder but the kernel is broken; the caster's try/except only covers the
cast, not the first forward, so the load would report engaged then crash at generation. Now it
falls back to dense. Layerwise fp8 has no torchao GEMM, so the probe is a no-op for it.
- The trim pre-hook's fallback restores the caller's original kwargs (it may have emptied the image
stream / trimmed a text stream before failing), so the stock dense-mask path runs on exactly what
it expects, matching the empty-prompt guard.
- video_speedmem_bench mirrors the loader: installs the Hunyuan trim before the backend set (gated on
an active tier) and skips the auto int8 quant when it is the fp8-denied memory fallback and dense
fits resident, so the shipped/auto rows measure what the loader actually runs.
Tests: TE explicit-mode kernel probe (+ layerwise-fp8 bypass), trim mid-trim restore, and loader-level
speed=off companion suppression + trim skip for both backends. 262 backend tests pass; ruff clean.
The Recommended list's fit-on-device toggle filtered the live Hub rows and the
flat curated rows, but the canonical catalog GROUP rows (Images / Video pages)
were gated only by the format filter. A bare click on a filtered list could then
still start an OOM load the toggle was meant to hide (LTX-2 base at 90 GB, the
Wan2.2-A14B MoE at 114 GB, both bf16-only with no GGUF fallback).
Add catalogGroupFitsDevice: a group stays visible when at least one artifact can
actually run here (already downloaded, a GGUF whose quant ladder self-fits, or a
sized artifact within 0.7*GPU + 0.7*RAM), mirroring the Recommended fit predicate
across a group's formats. Gate the search (matchedCatalogGroups) and the
Recommended-section catalog rows (render + roving keys) on it. Node-native
catalog:check assertions cover the over-budget, GGUF-fallback, downloaded, unknown
-budget, and datacenter-budget cases.
- diffusion_cache: do not engage FBCache when the selected pipeline opens no cache_context.
A CacheMixin transformer is necessary but not sufficient -- Flux Kontext / img2img /
inpaint / controlnet reuse the CacheMixin FluxTransformer2DModel yet their __call__ never
opens a cache_context, so the First-Block-Cache hook raised 'No context is set' on the
first forward, crashing every default FLUX.1-Kontext edit (28 steps, above the FBCache
threshold). Detect it from the pipeline __call__ source, resolved off the instance so the
per-expert proxy view delegates to the real pipe.
- diffusion_attention: honor an explicit aiter backend on ROCm/AMD targets instead of
dropping it via the NVIDIA-only guard (aiter is the AMD ROCm kernel; it only works there).
- video: clear the CUDA cache on a failed load so a partially built pipeline's reserved VRAM
does not OOM the next load (mirrors the image backend), and re-check cancellation after the
export/mux so a clip cancelled during the blocking encode is discarded, not persisted.
- diffusion_auto_policy / diffusion_prequant: validate a request-supplied prequant path
override (present AND allowlisted) before budgeting the small prequant plan, so the loader
does not skip the dense shards and then rebuild dense after evicting the resident pipeline.
- diffusion_controlnet: family-gate a curated ControlNet addressed by its full repo id, not
only its short catalog id, so a cross-family repo id 400s up front instead of downloading
and loading through the wrong ControlNet class.
Two fidelity fixes to the video speed/mem bench so its numbers match production:
- _build_pipe loaded the pipeline with a scalar bf16 torch_dtype and then upcast the
VAE, which truncates the fp32-stored Wan VAE at load (a later .to(float32) only widens
the lossy values). Pin the VAE fp32 per-component like the production loader
({"vae": fp32, "default": bf16}) so the bench decodes the same weights production does.
- The persisted reference frames were written/read as a single unkeyed ref_frames.npz in
the fixed default --out dir, so a reference-less run of a different family/seed/steps/
frames/resolution scored LPIPS against a stale baseline. Key the cache by those
parameters so a run only reuses a reference computed for the same parameters.
The image load route calls engine.begin_load(..., vae_quant=request.vae_quant, ...)
uniformly for both engines, but the native SdCppDiffusionBackend.begin_load accepted
every other diffusers-only knob except vae_quant and had no **kwargs, so a native
(CPU-only / MPS / forced-native) image GGUF load raised TypeError on every request
(vae_quant is always passed, defaulting to None). Accept and ignore it like the other
diffusers-only knobs; sd.cpp has no torchao VAE quant.
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
test_diffusion_attention.py documents itself as hermetic with no torch/diffusers
needed, but the HunyuanVideo trim tests added a module-level 'import torch' that
aborted collection of the whole file when torch is absent. Move those 15 tests to
test_diffusion_attention_trim.py (which declares the torch dependency) so the
attention-backend policy tests stay collectable and runnable without torch.
Five studio diffusion source files added by earlier sub-PRs (Krea 2 Turbo,
Images LoRA, ControlNet) were missing the AGPL-3.0 SPDX header the rest of
studio/backend carries. Add it so the whole backend is consistently licensed.
torch.cuda.is_bf16_supported() reports True on pre-Ampere GPUs that only
emulate bf16, so the SDXL LoRA trainer would keep bf16 there and fail at
load/forward. Use native_bf16_supported() (the same compute-capability
probe the DiT trainer already uses) so T4 / V100 / RTX 20xx fall back to
fp16 instead.
HunyuanVideo-1.5's DiT runs a joint [video; text] self-attention and, on every
block and step, builds a dense [B,1,N,N] boolean mask so the video never attends
to the padded text. A dense bool attn_mask disables every fused SDPA kernel
(flash rejects it; cuDNN and memory-efficient fall back), so the attention runs
the slow math-style path: at the production shape (121 frames, 480p, N about 50k)
one attention call is ~421ms with the mask vs ~19ms with attn_mask=None. The text
is ~99.5% padding (a t2v prompt fills ~9 of ~1985 slots), so nearly all of that
cost is spent masking padding.
install_hunyuan_attention_trim installs an eager forward pre-hook that drops the
all-zero image stream (t2v) and trims the mllm/byt5 text streams to their
globally-valid columns, plus a null-mask attention processor that runs
attn_mask=None once no partially-padded column remains (the batch-1 /
per-guidance-branch case) and otherwise delegates to the stock dense-mask
processor. The model already zeroes and masks the padded text and discards its
attention output (only the video split feeds proj_out), so removing it is exact
for the video; the only numeric change is the SDPA kernel (masked fallback to
fused). Measured on a B200: 23.3s to 1.3s per DiT forward at 121 frames (~18x with
regional compile, 0 graph breaks); per-forward cosine 0.99998 vs stock; equal
distance to an fp32 reference (LPIPS fp32-vs-stock 0.292, fp32-vs-trim 0.307), so
it is not less accurate than the current bf16 default.
Wired auto-on for HunyuanVideo-1.5 in the video loader, before the attention
backend set so the requested kernel pins onto the new processors; a no-op for
every other family and reversible (stock dense-mask path on any anomaly). Adds
hermetic tests and the diagnostic/validation scripts.
* unstructured block removal
* Enhance unstructured block handling
* Restrict block cleanup to upload UIDs
* cleanup for seed block uploads
* upload cleanup queue for unstructured blocks in recipe studio
* Fix unstructured upload cleanup edge cases
* Fix unstructured upload import ownership
* Fix-unstructured-import-path-ownership
* Guard failed-delete restore against stale block in unstructured drop zone
* Drain queued upload cleanups when autosave is skipped
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Root-caused "HunyuanVideo-1.5 int8 is slower than dense" with a per-forward profiler
(scripts/hunyuan_int8_profile.py, dynamo-reset, back-to-back on a clean B200): int8 compiles
cleanly (0 recompiles, 0 graph breaks, steady 268.3 ms/forward) and is only ~7% slower than dense
+ regional compile (250.5 ms/forward), not the 38% a contended-GPU bench run suggested. int8 is
also less accurate (LPIPS 0.085 vs dense+compile 0.037). So for a family where fp8 is denied
(Hunyuan black-frames on per-row fp8), int8 is a MEMORY lever, not a speed win, yet the auto-quant
default quantised it even when the dense DiT already fit resident.
Fix: is_int8_memory_fallback(target, family) is True only when AUTO quant lands on int8 as a
denied/black-frame fallback on a data-center, fp8-capable GPU (fp8 would be the arch pick but is
denied for the family). The video loader now skips the auto-quant and runs dense+compile when that
holds AND the bf16 memory plan already fits resident (offload_policy == none), so there is no new
OOM risk. Scoped tightly: only an AUTO request (explicit int8/fp8 honored), only int8-fallback
families (Wan / LTX resolve to fp8 -> keep quantising), only data-center fp8-capable parts (consumer
GPUs and pre-Ada, where int8 is a genuine accelerator, keep int8), and only when dense provably
fits; a memory-constrained plan still quantises. Result: Hunyuan on a resident-fit B200 now runs
faster AND more accurate, quantising only when memory is the constraint.
Also resets dynamo per config in the video bench (so compiled graphs cannot leak across configs in
one process) and adds the per-forward profiler used for the diagnosis.
Adds wan2.2-t2v-a14b and hunyuanvideo-1.5-720p to the bench family table and makes _apply_levers
quantize / compile BOTH experts of a dual-expert MoE (Wan2.2-A14B) via a _SecondExpertView proxy
that mirrors the loader's _SecondDiTView, so A14B latency and accuracy are measured on the real
two-DiT path instead of only the first expert.
Used to validate that every video family is on the fastest DiT quant scheme that is not less
accurate than its alternative (B200, 512x320, 25 frames, 30 steps, no cache, LPIPS vs dense bf16):
- Wan2.2-TI2V-5B fp8 49.9 ms/step vs int8 64.6 vs dense 59.8; LPIPS 0.129 vs 0.180
- Wan2.2-T2V-A14B fp8 195.8 ms/step vs int8 193.5 vs dense 369.1; LPIPS 0.288 vs 0.349 (both experts)
- LTX-2 / LTX-2.3 fp8 133.5 ms/step vs int8 138.4 vs dense 204.7; LPIPS 0.026 vs 0.027
- HunyuanVideo-1.5 fp8 is black (denied, not localizable); int8 21.2 s vs dense+compile 15.4 s -- a
memory saving at a speed cost, so int8 stays only because fp8 is impossible there.
fp8 is faster than dense AND more accurate than int8 on all three Wan/LTX families (the two Wan ones
via the condition_embedder exclude; LTX-2 needs none -- its conditioning has no zero-amax padding
rows). The auto-ladder + deny + exclude already select exactly these schemes, so no scheme-selection
change was needed; this commit is the bench faithfulness improvement that let the campaign confirm it.
The Wan fp8 black frame was root-caused (scripts/fp8_layer_ablation.py,
measured on B200 with the production torch._scaled_mm path): per-row fp8
scales each activation row by row_amax/448, and the text prompt is padded to
512 tokens (~all padding for a short prompt), so condition_embedder's text
embedder divides a zero padding row by a zero scale, which infs and renders
every frame black. That embedder's bias makes every downstream row non-zero,
so the whole 30-block attn1/attn2/ffn stack is fp8-clean (fp8-except-
condition_embedder measured cosine 0.9998 vs bf16, 0 non-finite; fp8-
everywhere is 100% non-finite).
So the blanket fp8 deny was heavier than needed for Wan. Remove fp8 from the
Wan deny and keep only condition_embedder in bf16 via a new
_FP8_FAMILY_EXCLUDE_NAME_TOKENS; auto now restores fp8 (the Blackwell ladder
head) for Wan2.2-TI2V-5B and -T2V-A14B (shared DiT class and padded-text
conditioning). Full-generation check (512x320, 25 frames, 30 steps, cache on
and off): mixed-fp8 is non-black (mean luma 182.6 vs dense 181.2), more
accurate than int8 (LPIPS 0.129 vs 0.180 no-cache, 0.224 vs 0.251 with
FBCache), faster (49.9 vs 64.6 ms/step; int8 was a per-step regression vs the
59.8 ms/step dense), at the same memory (19.34 GB, both -20% vs dense).
HunyuanVideo-1.5 keeps the fp8 deny: its MMDiT masks the padding text tokens
to zero inside every block, so the per-block context stream (add_*_proj /
to_add_out / ff_context) regenerates zero rows layer after layer (fp8 on only
the main blocks is 100% non-finite) so no small exclude set exists and int8
stays. mxfp8 / nvfp4 remain denied for Wan (same per-row scaled_mm family, not
separately validated).
exclude_tokens_for_scheme now takes an optional family, threaded through the
runtime quantiser and the offline prequant builder + validator so offline ==
runtime (a stale Wan fp8 checkpoint baked without the exclude is rejected and
re-quantised rather than loaded). Adds scripts/fp8_layer_ablation.py (the
per-layer ablation probe) and a mean-luma black-frame metric plus mixed-fp8
vs int8 configs to the video bench.
* Studio: render thinking blocks for safetensors inference with prefilled <think> templates
Reasoning templates like Qwen3.6 end the generation prompt with an open
<think> tag. skip_prompt streaming drops it, so the frontend never sees
the opening tag and shows reasoning as plain text. Detect the prefill
and re-emit it at the start of the stream on the transformers and MLX
paths. Also stop stripping think tags in _clean_generated_text when a
tokenizer marks them special.
* Studio: guard think re-emit for special close tags, yield prefill early
Address review feedback:
- Guard: skip re-emitting the open <think> when the tokenizer marks </think>
as a special token, since skip_special_tokens would strip the model's close
tag and leave an unclosed block that swallows the answer. Falls back to
plain text (pre-fix behaviour) for those tokenizers.
- Yield the prefilled <think> before the first token so the thinking block
renders during prompt prefill instead of after the first generated token.
- Drop the now-unnecessary _clean_generated_text think-tag exemption; the
guard handles the special-token case at the source.
No mainstream reasoning model (Qwen3.6, Qwen3, DeepSeek-R1, QwQ, GLM-4.6)
marks think tags special, so behaviour is unchanged for them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
Measured the fp8 DiT auto-quant path across the remaining dense-pipeline video families on
B200 (production torch._scaled_mm per-row fp8, no MSLK):
- HunyuanVideo-1.5 (480p + 720p repacks): every frame black (mean luma 0.0, LPIPS 0.82);
int8 is clean (mean 102.7 vs dense 99.9). Same failure as Wan / qwen-image.
- LTX-2: fp8 renders clean (mean 153.7, matches int8's 157.7) -- NOT a black-frame family.
So deny fp8/mxfp8/nvfp4 for hunyuanvideo-1.5 and hunyuanvideo-1.5-720p (fall to int8), and
deliberately leave LTX-2 on fp8. The deny stays measured per family, not a blanket video rule:
a blanket deny would have wrongly forced LTX-2 off fp8. Adds a Hunyuan deny test that also
asserts LTX-2 keeps fp8; 49/49 transformer-quant tests pass.
video_speedmem_bench.py gains guidance_via_guider support (HunyuanVideo-1.5 sets CFG on a
guider component and its __call__ takes no guidance_scale / callback_on_step_end), so the
harness can drive Hunyuan the same way the loader does.
PyPI release unsloth 2026.7.2 is now live. Bumps the pinned floor in
install.sh and install.ps1 from 2026.7.1 to 2026.7.2 for both unsloth and
unsloth-zoo across all 5 install commands (no-torch / reinstall / upgrade /
local / auto torch backend paths) so fresh installs resolve to the new wheel.
Follows the same pattern as #5716.
video_speedmem_bench.py drives the real video-loader lever functions
(quantize_transformer / quantize_text_encoders / quantize_vae / apply_speed_optims
/ apply_attention_backend / apply_step_cache) with the loader's own defaults, so each
measured config reflects a real load. It decomposes the video speed/memory stack
(compile, cuDNN attention, First-Block-Cache, DiT/TE/VAE quant) with per-step latency,
peak resident GB, and per-frame LPIPS vs a bit-exact reference. This is the harness that
surfaced and validated the Wan fp8 black-frame fix.
quant_speedmem_bench.py gains the DiT-quant mode (dense vs fp8/int8/mxfp8 speed, peak
memory, and LPIPS vs the dense render) plus the shared LPIPS(AlexNet) helper.
The dense video default engages transformer auto-quant, and on Blackwell the
auto ladder leads with fp8. On the Wan DiT the production per-row fp8 path
(torch._scaled_mm) renders every frame black (mean luma 0.0 at 512x320 and
704x480, LPIPS ~0.80 vs bf16): Wan's activation outliers exceed per-row fp8's
range, the same failure already denied for qwen-image. First-Block-Cache then
over-caches the degenerate activations (per-step collapses to ~10ms),
compounding it.
Add the Wan families (wan2.2-ti2v-5b, wan2.2-t2v-a14b, same WanTransformer3DModel)
to _FAMILY_SCHEME_DENY for fp8/mxfp8/nvfp4 so auto falls through to int8, which is
clean on Wan (per-token, outlier-robust), saves the same weight memory on the DiT,
and lets First-Block-Cache engage normally instead of over-caching. mxfp8/nvfp4 are
denied alongside fp8 conservatively so auto lands on the battle-tested int8; they
can be re-enabled per family once validated in-bar, like the nvfp4 auto-ladder TODO.
Validated on B200: the shipped video default now selects int8 for the Wan DiT and
renders clean frames (mean 172.6) at 15.6 GB resident (down from 24.2 GB dense),
with First-Block-Cache engaged. Adds two deny tests; 48/48 transformer-quant tests pass.
* Studio: allow CPU-only DiffusionGemma by granting the diffusion runner the CPU device
* Studio: mark CPU-only DiffusionGemma as non-GPU-resident for training VRAM preflight
* Studio: keep the CPU DiffusionGemma change minimal (revert VRAM-flag tweak; Metal hosts still hold unified memory)
* Studio: keep CPU DiffusionGemma fallback fully CPU-masked so a masked GPU host does not re-expose GPU 0
worker.py imports has_blackwell_gpu from utils.wheel_utils, but _load_worker_module
stubs utils.wheel_utils with a fixed name tuple that omitted it, so loading the worker
raised ImportError (cannot import name 'has_blackwell_gpu') and Backend CI could not
collect test_mlx_training_worker_config.py. Add the name to the stub so it matches
worker.py's imports.
* fix: Remove moot has_blackwell_gpu() function
Fixes unslothai/unsloth#6961. This function skipped flash-attn on Blackwell GPUs because no prebuilt wheel existed;
Dao-AILab now ships one and url_exists() already gates resolution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: use torchao 0.17.0 for Blackwell
Fixes#6961. Torchao 0.16.0's cpp extensions are built against CUDA 12, so on a CUDA-13
torch (cu130 / Blackwell) they fail to load with "libcudart.so.12: cannot
open shared object file". Select 0.17.0 there instead: its cpp targets torch
2.11, so it is skipped cleanly rather than crashing. CUDA-12 / ROCm / CPU
torch 2.10 keeps 0.16.0 and its working kernels.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Condense torchao version-selection comments (no behavior change)
* Support torch 2.11 in the Studio installer via the torch2.10 prebuilt wheels
Map torch 2.11 to the torch2.10 prebuilt wheels for flash-attn, causal-conv1d,
and mamba through wheel_utils.prebuilt_wheel_torch_mm, applied in direct_wheel_url
(filename) and flash_attn_wheel_url (version). Those torch2.10 CUDA wheels load and
pass each project's own test suite on torch 2.11 (verified on B200), so a torch 2.11
environment gets the prebuilt accelerators instead of skipping or building from source.
Raise _CUDA_TORCH_PKG_SPEC to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA torch repair path can install torch 2.11, where torchao 0.17's cpp kernels
load cleanly. Add tests for the mapping.
* Keep has_blackwell_gpu as a False stub for future arch gating
* Restore has_blackwell_gpu as a return-False probe kept for future arch gating
Keep the nvidia-smi compute_cap detection and its two call sites, but short-circuit
with return False at the top so flash-attn is no longer skipped on Blackwell (sm_100+
now has prebuilt wheels and url_exists gates resolution). Drop the early return to
re-enable arch-based detection later.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* models: auto-target per-expert Linear MoE experts for LoRA (gpt-oss 4bit)
MoE checkpoints whose experts are stored as per-expert nn.Linear ModuleLists
could not receive expert LoRA. gpt-oss bnb-4bit is the canonical case: its
experts live at mlp.experts.gate_up_projs.<i> and mlp.experts.down_projs.<i> as
per-expert Linear4bit modules, not a fused nn.Parameter. The target_parameters
path only handles the fused nn.Parameter layout, and the plain
gate_proj/up_proj/down_proj leaf names do not match the per-expert indices, so
get_peft_model attached LoRA to attention only and left every expert frozen
(0 of 1536 on gpt-oss-20b) even though the grouped bnb-4bit training forward
exists.
Add get_moe_target_modules, the module-LoRA counterpart of
get_moe_target_parameters: it detects per-expert Linear ModuleLists under an
experts container and returns their suffix target_modules names
(gate_up_projs.<i> / down_projs.<i>). get_peft_model in both llama.py and
vision.py extends target_modules with these, handling the explicit leaf-list
form and the regex form (auto / all-linear / scoped). It is gated on the same
MLP-in-scope condition as the parameter path, so an attention-only request still
skips the experts.
Also gate get_moe_target_parameters on the fused parameter actually existing, so
a per-expert-Linear layout no longer produces a dead target_parameters path or a
misleading "Enabling LoRA on MoE parameters" line; those experts are handled
through target_modules instead.
Validated on gpt-oss-20b-unsloth-bnb-4bit (transformers 5.5.0): experts attach
(1536 modules, trainable 0.036 percent to 1.65 percent) across the default, None
and all-linear paths; training memorizes and the LoRA adapter reproduces exactly
after a cold reload in a fresh process. No regression: fused-parameter MoEs
(Qwen3-30B-A3B-4bit), non-MoE models, and attention-only requests are unaffected
(get_moe_target_modules returns an empty list).
Merging these per-expert adapters into a merged_16bit checkpoint is handled by a
companion unsloth-zoo change (saving_utils folds each per-expert delta into the
fused gate_up_proj / down_proj tensor). With both, the LoRA adapter and the
merged_16bit checkpoint reload the trained behavior identically.
* models: scope per-expert MoE targets, keep repeat get_peft_model idempotent, warn on old zoo
Address review of the per-expert Linear MoE targeting:
- Scope get_moe_target_modules to the requested projection leaves (gate/up map to
the gate_up ModuleList, down maps to the down ModuleList), so a narrowed request
such as target_modules=["down_proj"] no longer also trains gate_up_projs, matching
get_moe_target_parameters.
- Detect experts through a PEFT-wrapped base_layer as well, and recompute the
auto-added expert targets in the llama.py existing-adapter check, so a repeat
get_peft_model call with the same arguments stays idempotent instead of raising on
the saved expert targets.
- Warn when the installed unsloth_zoo cannot fold these per-expert experts into a
merged_16bit checkpoint (older releases keep the fused gate_up_proj / down_proj
tensors and drop the per-expert deltas), so the expert LoRA is not silently lost on
save_pretrained_merged; the fold lands in unsloth-zoo #885. The LoRA adapter itself
is unaffected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio/hub): apply repo_id length limit per segment, not whole string
is_valid_repo_id() applied the 96-char limit to the full "namespace/repo_name"
string, so a repo with a valid (<=96 char) name but a long combined id was
falsely rejected. Match huggingface_hub.validate_repo_id by checking the length
per segment instead. Fixes#6946.
* Fix long repo id state filenames
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: source CPU llama.cpp prebuilts from the unslothai fork
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt
* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt
* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments
* Studio: correct stale fork-routing comments and --resolve-prebuilt help
* Refresh stale ggml-org routing comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: keep transformers off sys.modules until the training worker activates the sidecar
The training worker (core/training/worker.py:run_training_process) decides the per-worker
Xet env flip during preflight by importing utils/hf_xet_fallback.py, which eagerly imported
unsloth_zoo at module load. unsloth_zoo's __init__ imports transformers, so the default
transformers 4.57.x was cached in sys.modules before activate_transformers_for_subprocess
prepended the 5.x sidecar to sys.path. Since activation only edits sys.path, the already
cached module won, and 5.x models failed to load their tokenizer or config:
- Qwen3.5 / GLM-4.7 (tokenizer_class TokenizersBackend): "Tokenizer class TokenizersBackend
does not exist or is not currently imported."
- gemma-4: "... is not supported yet in transformers==4.57.6."
Fix: load the shared unsloth_zoo backend lazily (only when a heavy download helper is first
used, which is after activation). child_should_disable_xet and the DEFAULT_* constants are
defined locally so importing the shim stays light. The download wrappers, the DownloadStallError
class, start_watchdog and get_hf_download_state resolve the shared backend on first use, and the
degraded no-unsloth_zoo fallback is preserved.
Tests:
- test_hf_xet_fallback.py: existing suite kept green via the restored _shared_* seam; the
GPU-init retry test now triggers the lazy load explicitly; new guard asserts importing
child_should_disable_xet does not import transformers/unsloth_zoo.
- test_training_worker_import_discipline.py: new invariant test that the worker preflight
imports leave transformers unimported, so this class of regression cannot return silently.
Runs in studio-backend-ci (CPU only, no network/GPU/weights).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: CPU-only guard that activation switches transformers to the model's sidecar version
Adds test_worker_activates_correct_transformers.py: runs the real worker preflight
(from utils.hf_xet_fallback import child_should_disable_xet) plus the real tier
detection and activate_transformers_for_subprocess for a transformers-5.x model
(Qwen3.5, tier 530), then asserts the in-process transformers actually switched to
the 5.x sidecar. A stale pre-activation import leaves 4.57.x pinned and fails the
assertion, which is exactly the TokenizersBackend regression (#6951).
Self-contained CUDA spoof (mirrors tests/_zoo_aggressive_cuda_spoof.py) forces
unsloth_zoo down its full, transformers-importing init path on a GPU-less runner;
without it unsloth_zoo degrades and never preloads transformers, masking the bug.
A one-line stub sidecar stands in for the 5.x venv, so no GPU, network, weights, or
real sidecar are needed. Passes on this fix, fails on buggy main.
* Studio: load the repo's canonical CUDA spoof in the correct-version guard
Load tests/_zoo_aggressive_cuda_spoof.py (the committed spoof the consolidated CI
already relies on) as the single source of truth so the guard matches CI and stays
robust on a CPU-only torch wheel, where a partial hand-rolled spoof could miss a
torch.cuda call and let the unsloth_zoo import raise (masking the bug). Falls back to
a minimal inline spoof for a standalone studio checkout. Verified: passes on this fix,
fails on buggy main, and the fallback path passes when the spoof file is absent.
* Studio: declare the lazily-resolved xet names so ruff F822 stays green
DownloadStallError, start_watchdog and get_hf_download_state are provided via the
module __getattr__ (PEP 562), so ruff F822 flagged them as undefined names in __all__
and the Source-lint / pre-commit checks went red. Add annotation-only declarations
(no value bound, so __getattr__ still resolves them lazily to the shared unsloth_zoo
backend) to mark them defined for the linter while keeping F822 active for the rest
of __all__.
* Studio: tighten comments on the sidecar-activation fix and its tests
* Studio: mirror the new MLX-dispatch preflight import in the import-discipline guard
The worker preflight now also runs 'from core.training.training import
is_apple_silicon_training_platform, should_use_mlx_training_backend' before it
activates the transformers sidecar. Add that import (guarded) to the guard's
preflight snippet so the invariant test stays a faithful mirror: a future change
that makes core.training.training pull transformers/unsloth_zoo eagerly would then
be caught too. Verified clean on the current tree (no leak).
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* feat: detect installed coding agent CLIs in Studio settings
The API-keys panel only ever showed the "claude" flavor of the
`unsloth start` command, so anyone using Codex, OpenCode, OpenClaw,
Hermes, or Pi had to manually rewrite the copied command by hand.
Add a backend check that looks for each agent's CLI binary on PATH
(shutil.which, mirroring the pattern already used elsewhere in
studio/backend/utils) and expose it as GET /api/settings/coding-agents.
The API-keys panel now renders a picker for all six supported agents,
marks the ones it finds installed, and defaults to one of those instead
of always falling back to claude.
Includes unit tests for the detection helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review feedback on coding-agent detection
Three fixes from PR review:
- detect_installed_coding_agents now treats a PATH lookup failure as
"not installed" instead of letting it bubble up and break the
settings endpoint; added a regression test for it.
- CodingAgentsResponse.agents is now typed as an immutable tuple
instead of a list built from one, matching CODING_AGENTS itself.
- Fixed a race in the API-keys panel: picking an agent while the
installed-CLI check is still in flight could get silently overwritten
once that check resolved. A ref now tracks whether the user has made
a manual choice, so the auto-detected default only applies before
that happens.
* Address Codex feedback: GGUF gating and remote-detection scope
- codex refuses to launch against a non-GGUF (transformers-backed) model
(unsloth_cli's _require_gguf_for_codex), so auto-defaulting to it produced
a copy-pasteable command that fails immediately whenever the loaded model
isn't GGUF. Add useActiveModelIsGguf() (looks up the active checkpoint in
the chat runtime store) and a correction effect that steers the auto-pick
away from codex unless the loaded model qualifies, without ever touching a
choice the user made by hand.
- Detection runs via shutil.which on the Studio backend host, which isn't
the same machine as the browser in a tunnel/remote session. Reword the
'installed'/'detected' copy to say so explicitly when the tunnel URL is
in use, instead of implying the check ran on the viewer's own device.
* Rework auto-default per review: loopback gating + inline GGUF check
Replaces the previous approach with the exact shape discussed on the PR:
- Export isLoopbackHost/normalizeHost from agent-command.ts. The detection
endpoint runs shutil.which on the Studio backend, which only describes the
browser's own machine when the base this panel targets resolves to
loopback. For a LAN or tunnel/remote base, gate the whole thing off --
don't mark anything as "detected" and don't let it drive the default --
instead of just relabeling the copy.
- Drop the separate GGUF-correction effect and useActiveModelIsGguf hook.
Read useChatRuntimeStore.getState().activeGgufVariant inline inside the
existing detection effect's .then() (so it doesn't need to sit in the
effect's deps), and pick the first detected agent that isn't codex unless
the loaded model is GGUF, leaving the existing default untouched when no
compatible agent is detected.
Verified both branches (loopback vs LAN/tunnel base, gguf vs non-gguf,
manual pick preserved, no-compatible-agent fallback) with a standalone
port of the .then() logic.
* Address latest Codex findings: stale detection, model swap, cache
- Clear detectedAgents (and skip the network call entirely) when the panel
leaves a loopback base, instead of leaving a previous loopback detection
result marked 'installed' for a command that now targets a LAN/tunnel/
remote host.
- Add a separate, network-free correction effect keyed on the live
activeGgufVariant: if codex was auto-picked while a GGUF model was loaded
and the user then switches to a transformers-backed model while this panel
stays mounted, steer away from codex instead of leaving a command that
unsloth_cli's _require_gguf_for_codex will now reject. Never touches a
manual pick.
- Drop coding-agents.ts's module-lifetime cache. Installed-CLI detection is
environment state, not a persisted setting, so a stale positive/negative
from before the user installed something (or reopened the tab) is worse
than one extra cheap local API call per mount; keep only the in-flight
de-dupe for concurrent callers.
Verified the correction-effect logic (gguf->non-gguf swap with/without a
fallback, still-gguf no-op, manual pick never overridden) with a standalone
port of the effect.
* Make the codex/GGUF auto-pick symmetric in both directions
The correction effect only steered away from codex when the model stopped
being GGUF; it never steered back toward codex if the model became GGUF
*after* a non-GGUF-gated fallback had already picked something else (e.g.
codex is the only detected CLI, a transformers model is loaded so the
selection correctly falls back to the claude default, then the user loads a
GGUF model while the panel stays mounted -- codex never gets reconsidered).
Consolidate into one effect that re-derives the preferred detected agent
from scratch whenever detectedAgents or activeGgufVariant changes, in either
direction, instead of only reacting to the codex-specific downgrade case.
The fetch effect now only populates detectedAgents/availableAgents; this
effect is the single source of truth for what gets auto-picked from that
list. Never overrides a manual choice.
Verified both transition directions plus the manual-pick-survives and
initial-detection cases with a standalone port of the derivation logic.
* Reset the auto-pick to the default when it stops being trustworthy
Two more real gaps from the latest Codex pass on d988f52:
- The unified derivation effect only handled the case where a *different*
detected agent could take over. If codex was the only detected agent and
auto-picked while a GGUF model was loaded, then the model stopped being
GGUF, 'preferred' came back undefined and the effect silently left the
selection on codex -- exactly the command unsloth_cli's
_require_gguf_for_codex now rejects. Fall back to DEFAULT_AGENT in that
case instead of leaving it untouched.
- Leaving a loopback base cleared detectedAgents (so the 'installed' badges
correctly disappear) but left whatever agent had been auto-picked from
that now-stale, server-side-only detection still selected. Reset to
DEFAULT_AGENT there too, unless the user picked by hand.
Introduces a shared DEFAULT_AGENT constant instead of repeating the "claude"
literal at each reset site. Verified all five cases (both new resets, both
manual-pick-survives variants, and the existing multi-detected-agent
fallback still preferring another compatible agent over resetting) with a
standalone port of the effects.
* Derive GGUF-ness from the actual loaded state, not just the variant string
activeGgufVariant only covers an HF-repo GGUF pick (a specific quant
variant string). A direct local .gguf file -- custom folder, LM
Studio, or drag-drop -- is just as much a GGUF the codex preflight
(unsloth_cli's _require_gguf_for_codex) would accept, but it never has
a "variant" to report, so it read as non-GGUF here even though
/api/inference/status correctly reports is_gguf: true for it. That
mismatch could leave a Codex-only install not auto-selected, or reset
an auto-picked Codex, for a model that actually supports it.
Combined activeGgufVariant with activeNativePathToken (covers the
drag-drop/picked-file case) and ggufContextLength (only ever populated
when the backend last reported is_gguf: true for the active model, see
applyActiveModelStatusToStore) so all three paths a model can be GGUF
through are covered, matching the same is_gguf-or-equivalent check
hasGgufSource already applies to a staged pick elsewhere in this
codebase.
* Clear stale native-path token on a non-GGUF status refresh
When a native (drag-dropped or picked) GGUF was loaded and the backend later
switches to a transformers model outside the UI load path, refresh() adopts the
new /api/inference/status via setCheckpoint and applyActiveModelStatusToStore.
Those reset activeGgufVariant and ggufContextLength but never clear
activeNativePathToken, so the isGguf OR stays true after the switch and a
Codex-only detection auto-selects unsloth start codex for a non-GGUF model its
preflight rejects.
Drop activeNativePathToken in applyActiveModelStatusToStore whenever the status
is non-GGUF. A real GGUF load reports is_gguf: true, so its token is preserved
(the load path owns it); only a non-GGUF status clears it.
* Add the AGPL-3.0 header to the new studio contract test
* Fix/adjust agent detection for PR #6909
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
A B200 speed/memory sweep (new scripts/quant_speedmem_bench.py) shows the VAE quant
win is a video story. Image AutoencoderKLs are ~0.15-0.26 GB, so fp8 saves ~0.1 GB
and only slows their tiny decode (+6-16%); the video Conv3d VAEs are ~2.5 GB and
halve to ~1.2 GB at ~2% decode cost. So VAE auto now only engages above a ~1 GB size
floor: small image VAEs stay dense (faster decode, no quant quality risk), video VAEs
still quantize. An explicit fp8 / fp8_dynamic request skips the gate (opted in).
The same sweep confirmed the text-encoder default is already right: fp8_dynamic is
E2E-neutral (denoise per-step unchanged; +2% one-time encode) and, by hidden-state
cosine vs bf16, marginally more accurate than layerwise fp8 -- so that default is left
as is. Tests cover the gate (small skipped, large quantized, explicit bypasses).
* fix: launch OpenClaw local TUI by default
* Fix/adjust OpenClaw launch paths for PR #6937
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default OpenClaw to the local TUI only on a bare invocation
The first-arg startswith('-') branch rewrote passthrough globals into a broken
command: OpenClaw's grammar is openclaw [--dev] [--profile <name>] <command>, so
'unsloth start openclaw --profile test' became 'openclaw tui --local --profile
test', but tui does not accept --profile (or --dev), so the invocation failed.
A leading '--flag value' is ambiguous between a global (--profile test) and a tui
option (--message hi), so it cannot be reinterpreted safely. Default to the local
TUI only when no passthrough args are given, and forward everything else verbatim
so OpenClaw parses it under its own grammar. The bare-launch default (the point
of this change) is preserved; explicit subcommands and global flags pass through.
---------
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* version-compat CI: fake CPU training runs for SFT/GRPO/DPO
Adds a runtime layer on top of the patch-run canary: actually runs
trainer.train() for a couple of steps on a CPU-only runner under the CUDA
spoof, wrapping a plain tiny HF model in the Unsloth-patched trainer. Exercises
the real train() loop (collation, generation, the injected
_get_per_token_logps_and_entropies, loss, backward, optimizer) so a TRL or
transformers change that breaks the loop at runtime -- not just the source
structure -- surfaces here. No GPU, no meaningful numerics.
Needs a chain of small CPU shims (eager torch.compile, dynamo suppress, cuda
tensor-alloc redirect to CPU, model.for_training/for_inference equivalents)
documented inline. Does not exercise Unsloth's Triton/GPU kernels (CPU can't).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cpu fake-train: force adamw_torch + disable dynamo for CPU runner
On a real CPU-build torch runner (GitHub CI) two things bit that a CUDA-build
torch with GPUs hidden masked locally:
- The default optimizer is adamw_8bit (bitsandbytes), whose is_on_gpu() check
dies on CPU tensors. Force optim=adamw_torch in all three configs.
- import unsloth reinstalls the real torch.compile over the eager passthrough,
so the GRPO hot path (chunked_selective_log_softmax) actually compiles and
inductor picks the spoofed CUDA device, crashing on device props
(gcnArchName). Re-apply the eager passthrough after import and flip
torch._dynamo.config.disable so every @torch.compile runs eager at call time.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cpu fake-train: write checkpoints under pytest tmp_path
Use pytest's tmp_path for each trainer's output_dir instead of a hardcoded
relative temp/ci_* path, so a local pytest run does not leave untracked dirs in
the repo tree and the tests are CWD-independent.
* version-compat CI: disable dynamo at process level for the fake-run job
Set TORCHDYNAMO_DISABLE / TORCH_COMPILE_DISABLE in the fake-run step env so
dynamo/inductor is off before conftest.py's early import unsloth, not only via
the per-test runtime shim. Defense in depth on the GPU-less runner: the GRPO
hot path never compiles regardless of when its functions were decorated.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix PEFT replacement for TRL >= 1.7.0, add missing compute_aux_loss for TRL 1.7.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GRPO for TRL >= 1.7.0: PEFT ref-adapter removal and return arity
rl.py: for trl >= 1.7.0, scope the PEFT removal regex to the ref-adapter
block only by anchoring the end on ref_param.data.copy_(param.data), so it
no longer also deletes the following gradient-checkpointing
enable_input_require_grads() block. Neutralize TRL 1.7.0's
`if _is_quantized_model:` bf16 cast the same way the existing
is_loaded_in_4bit cast is handled.
rl_replacements.py: initialize _extra_moe_kwargs before use (it was
referenced before assignment whenever compute_aux_loss was passed) and only
request output_router_logits when the aux loss is actually wanted.
rl_replacements.py: _get_per_token_logps_and_entropies now returns a 3-tuple
(logps, entropies, aux_loss) for trl >= 1.7.0 and a 2-tuple for older TRL,
matching how every TRL call site unpacks the result. Without this, TRL 1.7.x
_generate_and_score_completions unpacks 3 values from a 2-tuple and raises
"not enough values to unpack (expected 3, got 2)".
* Return zero aux_loss placeholder and drop inference-mode aux collection
* GRPO TRL >= 1.7.0: reject router aux-loss opt-in at init; drop zero aux placeholder
Unsloth's optimized GRPO forward cannot compute the MoE router auxiliary loss.
Previously an explicit opt-in (router_aux_loss_coef > 0) returned a fabricated
zero, silently training without the requested load-balancing penalty. Now reject
it at trainer init with a clear NotImplementedError, and return None (not zero)
for the aux slot of TRL's 3-tuple. Default stays off (coef 0), so the common
path is unaffected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO hidden-states fallback: free ModelOutput before chunked log-softmax
The old/ref logprob fallback binds the full ModelOutput (which holds every
layer's hidden_states when output_hidden_states=True) and kept it alive across
chunked_hidden_states_selective_log_softmax, an avoidable OOM on large models.
Extract logits then del outputs in both the text and VLM branches.
* Version-compat CI: proactively catch TRL GRPO breakage
The existing TRL canary is a static symbol/source grep: it verifies symbols
exist but is blind to structural changes (TRL 1.7.0's 2->3-tuple per-token-logps
return arity and restructured PEFT ref-adapter block, which the fix in this PR
addresses, both slipped past it because the methods still existed).
Two additions:
- test_trl_grpo_pinned_symbols.py: extend TRL_TAGS to 1.5/1.6/1.7 and pin the
exact source-string contracts the rl.py / rl_replacements.py transforms depend
on for TRL >= 1.7.0 (PEFT elif ref-adapter block + enable_input_require_grads
survival, if _is_quantized_model, aux_loss_enabled anchor, compute_aux_loss
arity). A future TRL change fails on main a few days before the PyPI release.
- test_trl_grpo_fake_run.py + a version-compat-ci job: fake-CUDA run that drives
the real GRPO/SFT/DPO source-transform patchers against latest + main TRL on a
CPU-only runner (no training) and asserts the generated Unsloth trainer still
satisfies the transform contracts. Catches behavioral regressions the grep
cannot see.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fake-run test: use a normal Version import for the aux gate
* version-compat CI: fix fake-run job gate + torch-absent collection
- Drop the invalid job-level matrix if (matrix is not available in
jobs.<id>.if -> 'Unrecognized named-value: matrix' fails the whole
workflow). Use a single job that runs vs TRL latest always and re-runs
vs TRL main only on schedule/dispatch via a step-level github.event_name
guard. Validated with actionlint.
- Module-level skip the fake-run test when torch is absent so
daily-fresh-fetch (pytest-only, collects tests/version_compat/) does not
crash on the top-level spoof import.
* fake-run test: do not skip on import failure
unsloth/trl are installed in the grpo-fake-run job, so a failing import is the
import-time drift this canary must catch. Keep only the not-installed find_spec
skips; let a real import error fail the test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO arity gate: regex downgrade + fail loud + CI coverage
The TRL < 1.7.0 per-token-logps return downgrade was an exact-string replace
anchored on the full return line incl. its comment, so a reformat (e.g.
pre-commit) could silently no-op it and ship a 3-tuple to older TRL. Switch to
a regex tolerant of comment/whitespace drift, and raise if the anchor stops
matching (re.subn count != 1) instead of failing silently. Add a monkeypatched
trl_version unit test asserting both arities, since CI only installs TRL >= 1.7.0
and never exercised the downgrade otherwise.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fake-run: give SFT/DPO a real contract, not just ast-parse
The SFT/DPO fake patch runs only checked the generated trainer parses. Also
assert the shared QLoRA _is_quantized_model bf16 cast is neutralized (TRL 1.7's
spelling, present in both sft_trainer and dpo_trainer), so a structural TRL
change to that block is caught for SFT/DPO too, not just GRPO.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO PEFT ref-adapter removal: lower gate to the TRL 1.4.0 floor
The elif is_peft_model(model) and args.beta != 0.0: ref-adapter block was
introduced in TRL 1.4.0 and is unchanged through 1.7.x, but the removal was
gated at >= 1.7.0, so for 1.4 <= TRL < 1.7 the transform fell through to the
0.27 branch (which matches the older if is_peft_available()... form) and
silently no-oped: a PEFT + beta != 0 GRPO run then computed the KL reference
from the copied ref adapter instead of the base model. Lower the gate to
1.4.0 and keep the 1.7.0-only router aux-loss fail-fast nested. Widen the
pinned-symbol contract test to run from 1.4.0 so the covered versions are
actually exercised.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Five diffusion source and test files landed without the standard two-line SPDX
header the rest of studio/backend carries. Prepend it (matching the sibling
convention) so the whole backend is uniformly licensed. Header-only, no code
change.
