6131 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Lee Jackson	07578eab60	Fix on-device locations dialog layout (#6743) ... * Fix on-device location path overflow * Remove redundant native path tooltip	2026-06-29 15:50:55 +02:00
Wasim Yousef Said	755da2f155	Speed up Studio desktop startup (#6742) ... * Speed up Studio desktop startup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address Studio startup review findings * Keep orphaned run cleanup before readiness * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-29 15:27:39 +02:00
Muhammad Ikhwan Fathulloh	02540371c2	perf(dataprep): cache regex and field lists, fix typos (#6714) ... * Improve code quality & performance: fix typos, compile regex & cache fields - Fix typos across core files (repeatted → repeated, splitted → split, etc.) - Compile regex patterns once as class attributes in TextPreprocessor - Cache text fields/columns in RawTextDataLoader - Improve comments (re-use → reuse) * Use immutable raw text field constants --------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-29 13:07:09 +01:00
Lee Jackson	2f8521ed54	Fix compare adapter selection (#6411)	2026-06-29 12:57:48 +02:00
Matt Van Horn	54b95fbcc8	fix(studio): show local file path tooltip for Hub-tab local models (#6715) ... Local models in the Studio Hub tab (Custom folders, LM Studio, and Local models sections) did not reveal their on-disk path on hover, unlike the Fine-tuned rows which already do. Each of these rows maps over a LocalModelInfo with a required path, so pass tooltipText built from the model name and path via a small shared localPathTooltip helper, matching the existing FT-row tooltip format. Refs #6382 Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-29 11:12:47 +01:00
Vineeth Sai	677ec0cc20	Fix gpt-oss detection in save: config.architectures is a list, not a string (#6711)	2026-06-29 02:44:00 -03:00
Vineeth Sai	20266a59eb	Fix custom chat templates with a {system_message} placeholder (dead code in _change_system_message) (#6735) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-29 00:36:39 -03:00
Nilay	b56d24ea3e	Studio: cascade user message deletion to include assistant reply (#6720) ... * cascade user message deletion to include assistant reply * Fix comment typo in delete-thread-message --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-28 04:16:03 -07:00
Daniel Han	693ab8069d	Remove unused FalconH1RMSNormGated import (#6728) ... FalconH1RMSNormGated is imported from transformers but never referenced in unsloth/models/falcon_h1.py. The unused hoist trips the import-hoist lint gate on the merge commit of every open PR (the gate lints PR-head merged into main), so clearing it here unblocks those PRs.	2026-06-28 02:43:34 -07:00
Daniel Han	0ad814a452	Revert "feat: add GPU-aware model filtering and For You section- Add fit filt…" (#6722) ... This reverts commit a636693019.	2026-06-27 17:48:16 -07:00
Daniel Han	4c72e09480	Studio: stop handing CI/user secrets to downloaded llama.cpp binaries (#6696) ... * Studio: stop handing CI/user secrets to downloaded llama.cpp binaries The macOS prebuilt path installs llama.cpp from the unslothai/llama.cpp fork's latest (unpinned, mutable) release and then executes the downloaded llama-server / llama-quantize binaries during install-time validation. binary_env() built that child environment from a full os.environ.copy(), so a compromised or tampered prebuilt would inherit every secret in the process: HF_TOKEN and the workflow GitHub tokens in CI, and HF / cloud credentials for end users running install.sh / setup.sh. We publish prebuilts daily, so pinning a release tag is not workable. Instead, neutralise the impact: these binaries have no reason to read any token, so strip secret-bearing variables (exact names plus TOKEN/SECRET/PASSWORD/CREDENTIAL/PRIVATE_KEY/API_KEY markers) before handing the env to a downloaded binary. The installer's own GitHub and Hugging Face API calls read os.environ directly, so authentication and release-API rate limiting are unaffected; PATH, LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and CUDA/ROCm vars are preserved. One change covers the install-time validation path for all six macOS workflows and end users. Follow-up (separate, sequenced): publish build-provenance attestations from the fork's prebuilt workflows and verify them in CI, so a forged release is rejected rather than merely starved of secrets. * Strip KUBECONFIG, SSH_AUTH_SOCK, and PASSPHRASE-marked vars from binary env Extend the deny-list per PR review: KUBECONFIG and SSH_AUTH_SOCK are credential pointers/capabilities a downloaded binary never needs, and a PASSPHRASE marker catches SSH_PASSPHRASE / GPG_PASSPHRASE. Tests updated. * Studio: also scrub proxy/index env vars and URL-embedded credentials before running prebuilt binaries * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope mlx-ci secrets to the install + download commands for PR #6696 Drop the ambient step-level env block and pass GH/GITHUB/HF tokens only on the installer and GGUF-download commands, so the directly invoked llama-quantize / llama-server smoke runs see no secrets. The installer still reads tokens from os.environ for the releases API and probe fetch. * Trim verbose comments around the secret-env scrubber for PR #6696 Comment-only: condense the block comments added across this PR. Logic unchanged (comment_tools.py check confirms code-only signature equal). * Redirect HOME / cache pointers to an empty dir for prebuilt binaries (PR #6696) Address Codex P2: stripping token env vars still let a tampered binary read on-disk token stores (~/.cache/huggingface/token, ~/.aws/credentials, ~/.config/gh) through $HOME and the cache/config pointers. Point HOME plus the HF / XDG / Windows home pointers at a single empty throwaway dir for the downloaded-binary env. Defense in depth: a binary resolving the real home via getpwuid is out of scope and needs OS sandboxing. * Close residual credential-probe gaps for PR #6696 Address the latest Codex review: - Strip token-only URL userinfo too (scheme://ghp_token@host), not just the user:pass form. - Redirect HOMEDRIVE/HOMEPATH alongside USERPROFILE so a Windows binary cannot reconstruct the real profile from %HOMEDRIVE%%HOMEPATH%. - Drop explicit credential-file pointers (NETRC, PIP_CONFIG_FILE, DOCKER_CONFIG, GIT_CONFIG_GLOBAL) that live outside HOME. - Probe ldd with a secret-free env: linux_runtime_dirs ran ldd on the untrusted prebuilt with the inherited os.environ, and ldd may execute the binary, so it could observe HF_TOKEN/GITHUB_TOKEN during the probe. Factored the shared scrub into secret_free_environ(). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Separate token-bearing install from binary smoke; drop CI command files (PR #6696) Address the two P1s in the latest review: - mlx-ci: GitHub bakes secrets into the run-script text, so inline token assignments in a step that later runs the prebuilt let a tampered binary read them from the script. Split into a token-bearing install + download step that never launches a binary, and a secret-free smoke step that runs llama-quantize / llama-server. - secret_free_environ now drops the GitHub Actions command files (GITHUB_ENV, GITHUB_PATH, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY, BASH_ENV) and the smoke step unsets them, so a tampered prebuilt cannot inject PATH/env into the later token-bearing MLX steps. * Run the prebuilt smoke last, after all token-bearing steps (PR #6696) Address the P1 workspace-poisoning vector: even with no secrets in its env, a tampered prebuilt could edit the checkout or installed modules, and the later HF_TOKEN MLX steps would then execute that poisoned code on push builds. Move the prebuilt install + smoke to the end of the job so the untrusted binary runs after every token-bearing step, leaving nothing for it to corrupt. The MLX GGUF reload uses a source-built llama-cli, not this prebuilt, so nothing depends on the earlier position. * Trim comments around the secret-env scrubber and prebuilt CI steps (PR #6696) Comment-only: condense the security-rationale block comments and merge the duplicated prebuilt-step description in mlx-ci. Logic unchanged (comment_tools.py check confirms the code-only signature is equal; install suite still passes). * Authenticate the GGUF export release-API lookup with the read-only GITHUB_TOKEN (PR #6696) * Rename env scrubber off the secret-named identifier CodeQL flags as a clear-text sink (PR #6696) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-27 05:21:05 -07:00
Daniel Han	98a01e70cd	Studio: restore tensor parallelism for vision/mmproj GGUFs (#6659) ... * Studio: restore tensor parallelism for vision/mmproj GGUFs #6416 disabled --split-mode tensor for any GGUF that ships an mmproj projector to dodge a GGML_ASSERT crash (#6415) seen on an older llama.cpp build with consumer Blackwell (sm_120). The blanket skip silently dropped tensor_parallel=true for every multimodal/MTP GGUF (e.g. Qwen3.6-35B-A3B-MTP); on hardware where the model fits on one GPU the load then collapsed to a single GPU. mmproj + --split-mode tensor works on current builds (verified end to end on B200/sm_100), so the skip was disabling a working configuration. Make the vision skip self-healing per binary: - attempt tensor for vision models by default - skip upfront only on a binary already seen to abort on tensor + mmproj this session (_vision_tensor_split_aborts), recorded when such a launch crashes at startup (_record_vision_tensor_split_abort). Process scoped, so a studio update re-probes the new build. The route-level layer-split fallback stays the net. - add _select_gpus(min_gpus=...) so a downgraded tensor request can keep multiple GPUs instead of collapsing to one (default 1, no behavior change). Add tests/test_tp_vision_regression.py: an AST allowlist guard over the tensor_parallel drop sites (which would have flagged #6416), plus cache and _select_gpus coverage. No GPU required. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: address review on vision tensor-parallel self-healing Three fixes from the PR review: - Record a vision-tensor abort only after every startup retry fails. The first version cached the binary on the first spawn crash, which on every build (including capable ones) is the benign --fit step abort that the existing --fit off retry resolves. That poisoned the cache so the next vision load in the same process skipped tensor. Recording now happens at the post-retry failure block (after fit-off, flash-attn-off and MTP-drop), so a binary that actually works is never cached. - Gate the record on the tensor/mmproj crash signature: a hard signal fault (_is_signal_crash) with no non-tensor cause (_output_has_nonprojector_diagnostic excludes OOM and unknown-arch), so an OOM, bad extra args, or MTP/flash-attn crash no longer marks an otherwise capable binary incompatible. - Preserve the multi-GPU request on the cached downgrade. The vision gate now raises _layer_min_gpus to the visible GPU count and threads it through the layer-split GPU selection (_select_gpus min_gpus and the subset loops), so a downgraded tensor request still spreads across GPUs instead of collapsing to a single card the model happens to fit. Verified two vision+tensor loads in one backend process both tensor-split across 4 GPUs (the benign fit abort no longer poisons the cache). Tests updated. * Studio: harden vision tensor-parallel self-healing (review round 2) Address the second review round on the vision/mmproj tensor-parallel fix: - Preserve vision on the first load: a --split-mode tensor + --mmproj GGML_ASSERT now raises so the route-level tensor->layer fallback retries layer split with the projector intact, instead of stripping --mmproj and silently loading text-only (which returned success and skipped the fallback, losing vision on the first load until the next cached load). - Symmetric multi-GPU preservation: the pooled-VRAM tensor downgrade now raises _layer_min_gpus from the usable tensor GPUs like the vision downgrade, so it no longer collapses a multi-GPU request to a single card. - Base the layer fallback minimum on usable GPUs: _select_gpus caps min_gpus to the count of cards with usable VRAM, so a downgrade never forces a nearly-full card in (or trips --fit) just to hit the count. - Re-probe after in-app updates: key the per-binary abort cache on (path, mtime) like _capability_cache, so POST /api/llama/update swapping the binary in place (no backend restart) re-probes the new build instead of inheriting the old build's abort. - Bump _layer_min_gpus for a known-bad vision binary independent of the tensor drop, so the route fallback's layer retry (tensor already off) still spreads across GPUs. Adds deterministic non-GPU regression tests for each. * Studio: gate cached-vision layer minimum on the current tensor request The cached-vision _layer_min_gpus bump fired for every later vision load on a binary recorded as tensor+mmproj-incompatible, including loads that did not request tensor parallelism. A plain non-tensor vision load that fits on one card would then grab every GPU just because an earlier TP attempt aborted in the same backend process. Re-tie the bump to the current tensor request (back inside the tensor-drop guard), so only a downgraded tensor request preserves the multi-GPU spread; a non-tensor vision load minimizes device count as before. * Studio: preserve GPU count + confirm assert on vision tensor fallback Third review round on the vision/mmproj tensor-parallel fix: - Preserve multi-GPU on the first tensor->layer fallback. The route-level retry runs tensor-off, so the in-function downgrades can't see the original tensor request and a fits-on-one-card model loaded the first successful fallback on a single GPU. The GGUF load closure now passes preserve_multi_gpu_on_layer (the toggle asked for tensor, this attempt is layer) and load_model raises _layer_min_gpus for it, so the downgrade still spreads across GPUs. - Cap the auto-context layer loops to usable GPUs. They bypass _select_gpus, so a raised _layer_min_gpus could force a nearly-full card into the subset (or trip --fit). They now start from _auto_min_gpus, capped to the GPUs with usable VRAM. - Confirm the tensor/mmproj assert before caching. Recording (and the layer-retry raise) now require the ggml assert marker via _is_tensor_split_assert, not the bare-signal predicate shared with the projector-incompat branch, so a corrupt or too-new projector that SIGSEGVs independent of split mode is no longer cached as tensor/mmproj-incompatible. Adds deterministic non-GPU regression tests for each. * Studio: extend multi-GPU fallback to extra/env tensor + overhead-aware cap Fourth review round on the vision/mmproj tensor-parallel fix: - Preserve multi-GPU fallback for all tensor requests, not just the UI toggle. Tensor can also be requested via --split-mode tensor in extra args or an inherited LLAMA_ARG_SPLIT_MODE=tensor env; the fallback retries those too, so the preserve_multi_gpu_on_layer hint now keys off _effective_tensor_parallel (the same check the fallback uses), comparing the overall request against the current attempt instead of only request.tensor_parallel. - Cap the auto-context layer fallback to GPUs that can pay the per-device layer overhead. The cap counted any card with positive usable VRAM, so a nearly-full GPU with a few MiB free stayed eligible and could be exposed to llama.cpp and OOM. It now mirrors _select_gpus: a card counts only if usable VRAM exceeds the per-device pipeline overhead. Adds deterministic non-GPU regression tests for both. * Studio: match the #6415 split-axis assert + replay layer-preserve hint Fifth review round on the vision/mmproj tensor-parallel fix: - Narrow the tensor/mmproj crash signature. _is_tensor_split_assert matched any GGML_ASSERT/GGML_ABORT, so an unrelated invariant a corrupt GGUF or projector trips with --mmproj present could be cached as tensor/mmproj-incompatible. It now matches the specific #6415 warmup assertion (GGML_ASSERT(src_ss[0].axis != GGML_BACKEND_SPLIT_AXIS_0) in ggml-backend-meta), whose split-axis signature is inherent to tensor splitting. A reworded future assert just re-crashes-then-falls-back (vision preserved via layer split) instead of poisoning the cache for other models. - Persist the layer-preserve hint for respawns. A successful tensor->layer fallback committed _last_load_kwargs without preserve_multi_gpu_on_layer, so _respawn_if_dead replayed only --split-mode layer + tensor_parallel=False and a mid-session respawn of a fits-on-one-card model came back single-GPU. The hint is now in the replay snapshot, so recovery keeps the multi-GPU placement. Adds deterministic non-GPU regression tests for both. * Studio: tighten comments on the vision tensor-parallel fix Make the comments and docstrings added by this PR succinct: collapse the multi-line block comments in llama_cpp.py / inference.py to one or two lines, trim the verbose test docstrings (the names and assert messages already carry the intent), and shorten the module docstring. No code changes; verified comment-only with scripts/comment_tools.py check --strip-docstrings. * Studio: cache vision tensor abort only on the split-axis token _is_tensor_split_assert also accepted any GGML_ASSERT/GGML_ABORT from ggml-backend-meta, but that file holds many asserts, so an unrelated scheduler/projector/model invariant on an --mmproj launch could cache the binary as tensor/mmproj-incompatible and make later compatible vision models skip tensor parallelism. Match the GGML_BACKEND_SPLIT_AXIS_* token itself (unique to the #6415 warmup assert), not the source file name. * Studio: don't leak the httpx test stub into later tests The regression module stubbed httpx via sys.modules.setdefault, which installs the lightweight stub even when real httpx is present but not yet imported. The stub then persists for the whole pytest process, so provider/HF tests collected later (importing httpx or huggingface_hub.errors) got a module missing HTTPError/Response. Mirror the neighboring llama_cpp helper tests: import real httpx first and only fall back to a stub on ImportError. * Studio: latch the #6415 tensor-split abort on the first spawn, key it per model The self-heal recorded the --split-mode tensor abort only in the post-retry failure block, after the flash-attn-off retry. But SPLIT_MODE_TENSOR requires flash_attn, so the flash-off retry can't run tensor and its output no longer carries the warmup split-axis assert (ggml-backend-meta :541). The record therefore never fired on the real reproducer and the crash loop repeated on every load (reported by oobabooga on #6659). Latch instead on the first spawn that shows the signal crash + split-axis marker: record it, kill the process, and raise straight to the route's layer fallback, skipping the futile flash-attn/MTP retry ladder for this crash. The crash is a tensor-split geometry limit (e.g. MQA n_head_kv=1 splitting to GGML_BACKEND_SPLIT_AXIS_0), not a vision/mmproj property: it reproduces without --mmproj and even single-GPU tensor. So drop the vision/mmproj scoping, rename _vision_tensor_* -> _tensor_split_*, and key the session cache on (binary, mtime, model) rather than (binary, mtime) so one model's abort no longer skips tensor for every other model on the same build. Regression tests updated to pin the early-spawn record, the per-model cache, and that an unrelated ggml-backend-meta assert is not treated as the marker. * Studio: reload on explicit tensor-off after a multi-GPU layer fallback When a tensor load is downgraded to layer but kept multi-GPU to honor the tensor request (preserve_multi_gpu_on_layer, the geometry-cache gate, or the budget downgrade), the server reports tensor_parallel=False with --split-mode layer stored. A later Apply that explicitly turns the tensor toggle off then matched the loaded state and deduped to already_loaded, so Studio kept the fallback's all-GPU CUDA_VISIBLE_DEVICES placement instead of re-selecting normal placement (a single GPU for a model that fits on one card). Latch a _layer_preserves_tensor_intent flag in load_model whenever a tensor request is downgraded to layer with the multi-GPU floor raised (_layer_min_gpus > 1), clear it when tensor stays on or on unload, and force a reload in _request_matches_loaded_settings when the user explicitly turns the tensor toggle off while that flag is set. An Apply that does not touch the toggle still dedupes, so a working multi-GPU layer server is not churned. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: address reviewer.py findings on the tensor-split self-heal P1 (dedup): tensor intent can be dropped via extras, not only the toggle. An explicit llama_extra_args=["--split-mode", "layer"] matches the stored fallback extras, so _request_matches_loaded_settings deduped to the preserved all-GPU placement instead of reloading. Now reload when layer_preserves_tensor_intent and the user explicitly drops tensor via the toggle OR via extras (_effective_tensor_parallel of the explicit extras is false). P1 (downgrade symmetry): the len(tp_gpus) < 2 compute-buffer downgrade cleared tensor_parallel without raising _layer_min_gpus, unlike the budget and geometry downgrades. GPUs below tensor's replicated compute-buffer reserve can still take layer split's lower overhead, so keep the multi-GPU request (len(gpus) >= 2) and let _select_gpus cap unusable cards. P2 (cache key): key the tensor-split abort cache on st_mtime_ns, so a binary replaced in place within the same second after an abort is re-probed instead of inheriting the stale entry. P2 (test hygiene): load routes/inference.py via importlib in the regression tests instead of importing the routes package, which runs routes/__init__.py and pulls in every router (e.g. python-multipart). Added regression coverage for the extras-off reload, the compute-buffer multi-GPU preservation, and the same-second nanosecond cache invalidation. * Studio: record the tensor-split abort on the Windows CRT abort exit too The first-spawn split-axis latch only recorded when _is_signal_crash matched (POSIX signal or 0xC0000000+ NTSTATUS). On MSVC builds GGML_ASSERT terminates through the CRT abort() path with exit code 3, which is neither, so the cache never filled on Windows and every later load of the same bad binary/model repeated the tensor crash before falling back to layer. The split-axis marker is definitive, so accept either a signal crash or the Windows abort() exit (3) when the marker is present. Add _is_abort_exit and a unit test, and assert the early latch honors it. * Studio: fix UnboundLocalError on --fit-on fallback, reload backend fast path Two follow-ups from review on the tensor-split self-heal: UnboundLocalError: _layer_min_gpus was initialized inside the GPU-selection try. If NVML probing or GGUF/mmproj sizing raised, the except path logged "using --fit on" and fell through to the command builder, where the new self._layer_preserves_tensor_intent = _layer_min_gpus > 1 then raised, turning a safe --fit-on layer fallback into a hard load failure. Bind _layer_min_gpus before the try so the except path always has it. Backend fast path: _request_matches_loaded_settings forces a reload when a preserved tensor->layer fallback gets an explicit tensor-off request, but load_model's own _already_in_target_state still matched the tensor-off/layer settings and short-circuited, so the placement re-selection never ran. Mirror the guard there: reload when layer_preserves_tensor_intent and the request drops tensor intent. The flag clears on that reload, so there's no loop. Added regression coverage for both. * Studio: testable tensor-split record decision; skip futile fit-off retry Follow-ups from a deeper review of the tensor-split self-heal: Extract the record decision into _should_record_tensor_split_abort(rc, output) (marker AND (signal crash OR Windows abort)) and call it from the early latch. The combined boolean was only covered by source-inspection substring checks, so an or->and typo would silently stop recording on Windows (CRT abort exit 3 is not a signal) with every test still green. Add a behavioral test over the POSIX / Windows / NTSTATUS / clean-exit / SIGKILL / no-marker matrix. Skip the --fit off retry inside _spawn_and_wait when the crash already shows the split-axis marker: that abort is fit-independent, so the retry just warms up and crashes a second time before the latch records it. Skipping it lets the caller latch immediately and corrects the latch comment. Also clarify the dedup-guard comments (toggle read from model_fields_set vs extras via _effective_tensor_parallel without env; the backend fast path is intentionally broader and only ever forces a reload). * Studio: don't reload-loop tensor-off requests under env tensor The preserved-fallback reload guard fired on the raw tensor toggle, ignoring LLAMA_ARG_SPLIT_MODE=tensor. For an env-driven tensor user, an explicit tensor_parallel=false request then forced a reload that re-engaged tensor via the env and re-created the same preserved layer fallback, so every /load reloaded -- bypassing the env-downgrade matching that exists to avoid exactly this loop. Gate the guard on the env-aware effective tensor state: reload only when an explicit toggle/extras change leaves _effective_tensor_parallel (which consults the env) off. If the env still forces tensor, fall through to the existing env-downgrade match, which dedupes instead of looping. Added a regression test with LLAMA_ARG_SPLIT_MODE=tensor set. * Studio: tighten comments and test docstrings on the TP self-heal Condense the verbose comments and test docstrings added across the review rounds into fewer, succinct lines without changing their intent: the early-latch and downgrade-site rationale, the cache/key and helper docstrings, the dedup-guard comments, and the per-test docstrings. No code changes (AST-verified comments and docstrings only); tests and lint unchanged. * Studio: clear preserved tensor flag on diffusion; carry it across non-drop reloads Two follow-ups on the preserved-fallback machinery: Diffusion: the DiffusionGemma path early-returns from load_model before the command builder that sets/clears _layer_preserves_tensor_intent, so the flag from a prior tensor->layer fallback leaked onto a later diffusion load and forced needless reloads of the diffusion server on tensor-off/extra Applies. Clear it when starting diffusion. Settings reload: the preserve hint was recomputed only from the new request, so a reload for an unrelated setting (e.g. max_seq_length) with the tensor toggle omitted dropped a preserved multi-GPU layer placement back to one GPU. Carry llama_backend.layer_preserves_tensor_intent into the hint when the request is not an explicit tensor-off/extras-off drop, so a fitting model stays multi-GPU. Added regression tests for the diffusion clear, the carry-forward, and the updated tensor-intent computation. * Studio: gate the preserve carry-forward on the same model being loaded The tensor-intent carry-forward read llama_backend.layer_preserves_tensor_intent without checking it belonged to the model being loaded. On a direct model switch (load B without an explicit /unload of A), the flag is still set from A's downgrade (it isn't reset until B's load_model reaches the command builder, after the route reads it), so a plain load of B got preserve_multi_gpu_on_layer=True and was spread across all GPUs even though it fits on one and the user never requested tensor for it. The backend dedup doesn't have this leak (it checks model_identifier first); the leak was only in the route hint. Extract the decision into _carry_preserved_tensor_intent(preserved, same_model, explicit_drop) and gate it on the backend still holding the same model. Add a behavioral truth-table test (catches a `not` inversion and a missing same-model guard) and tighten the compute-buffer downgrade test to bound its source window. * Studio: match the HF quant too when carrying preserved tensor intent The same-model guard on the preserve carry-forward compared only model_identifier, which is variant-agnostic for HF repos. A later load of the same repo with a different gguf_variant (which already bypassed dedupe on the variant mismatch) was treated as the same model, so a request that omits tensor settings inherited the prior variant's preserved intent and forced multi-GPU layer placement for a quant that never requested tensor. Also require the loaded hf_variant to match for HF repos (local direct-file loads already differ by model_identifier path). Added a regression test for the variant guard. * Studio: match the loaded GGUF by path too when carrying preserved tensor intent A local directory holding multiple GGUF variants keeps one variant-agnostic model_identifier (the directory) while config.gguf_file selects the file, so the same-model guard let variant B inherit variant A's preserved tensor->layer fallback and forced B onto multi-GPU. Mirror _already_in_target_state's identity logic: match by resolved path when both sides have a local file, else by HF variant. #6659 * Studio: let implicit same-settings reloads dedupe after a preserved fallback The backend _already_in_target_state mirror forced a reload on ANY effective tensor-off request once a tensor->layer fallback was preserved. In the HF auto-pick / local-directory flows the route-level dedup is skipped, so an identical /load with tensor omitted reached this guard and reloaded every time even without an explicit drop. Thread the route's preserve_multi_gpu_on_layer decision in so only an explicit drop reloads; implicit carry-forward dedupes. #6659 * Studio: only an explicit tensor/split-mode change drops preserved intent The explicit-drop test treated request.llama_extra_args is not None as a drop, so a same-model reload that merely added an unrelated pass-through arg (e.g. --top-k 20) without touching the tensor field or --split-mode disabled the carry-forward and collapsed a fitting model back to one GPU. A drop now requires an explicit tensor_parallel field change or a non-tensor --split-mode override, via a shared _is_explicit_tensor_drop helper used by both the already-loaded dedup and the load carry-forward so the two readers agree. #6659 * Studio: treat an explicit clear of extras as a tensor drop When tensor intent was extras-driven (--split-mode tensor) and fell back to a preserved layer split, a later request that explicitly clears extras (llama_extra_args=[]) but omits tensor_parallel left the empty list with no split-mode override, so the carry-forward kept the model pinned multi-GPU instead of returning to normal layer selection. _is_explicit_tensor_drop now also counts an explicit empty-list clear as a drop, while an unrelated extra (--top-k) or inherit (None) still carries the preserved intent. #6659 * Studio: don't treat the UI's tensor_parallel echo as a tensor drop The Studio frontend always sends tensor_parallel and copies the /load response's resolved value back into its state, so after a tensor->layer fallback every ctx/settings reload carries tensor_parallel=false even though the user never changed it. Keying the drop on the field (or on an empty extras clear) collapsed the preserved multi-GPU placement on the next reload. A fallback also always stores --split-mode layer, never a tensor split mode, so a clear never wipes tensor intent. _is_explicit_tensor_drop now drops only on an explicit non-tensor --split-mode override; the bare field echo, an empty clear, an unrelated extra, and inherit all keep the preserved placement, and --split-mode tensor / tensor_parallel=true re-engage tensor. #6659 * Studio: match the resolved config.identifier when carrying tensor intent The same-model guard for the carry-forward compared the raw request id, but ModelConfig.from_identifier normalizes it (adds the unsloth/ prefix for a shorthand, fixes repo-id case) before load_model stores config.identifier. So a ctx/settings reload using the shorthand id missed the match, dropped _carry_preserved_tensor_intent, and could collapse a preserved multi-GPU layer placement to one GPU. Compare against config.identifier (what the backend stores), keeping it symmetric with _already_in_target_state. #6659 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-27 01:52:18 -07:00
oobabooga	c8bcacc3fe	Fix fast_inference crash on ABI-broken vLLM: probe compiled extensions, not just import vllm (#6621) ... * Fix fast_inference crash on ABI-broken vLLM: force-load compiled extensions in the broken-vLLM probe * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Broaden broken-vLLM probe: catch non-libcudart .so failures and _moe_C_stable_libtorch * Revert stray reformat of the PDL fix log line * Trim verbose comments in the broken-vLLM probe * Drop non-existent vllm._moe_C_stable_libtorch from the broken-vLLM probe * Shorten comments in broken vLLM extension detection Condense the docstrings and inline comments for the lazy-loaded vLLM probe and the new regression test while keeping the rationale. Comments only, no code changes (verified with an AST signature check and the existing tests). --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-26 22:43:36 -07:00
Daniel Han	1fcd69e662	Harden flaky Studio CI: retry VS-hide rename and tolerate same-URL nav interrupt (#6713) ... Two intermittent Studio CI failures, both runner-environment flakes unrelated to test logic: Windows 'Studio install + inference without Visual Studio': the 'Hide Visual Studio + CMake' step renames C:\Program Files\Microsoft Visual Studio to simulate a host with no build tools. A background handle on a Program Files directory (Defender scan or an MSBuild node) makes Rename-Item intermittently fail with 'Access is denied', and $ErrorActionPreference = Stop turns that into a hard job failure. Wrap the VS and cmake renames in both Hide steps in a short Rename-WithRetry (6 tries, 3s apart) to ride out the transient lock. macOS 'Chat UI Tests': the re-login goto to /login can be interrupted by the SPA auth guard redirecting to the same /login URL, which Playwright reports as 'Navigation to .../login is interrupted by another navigation to .../login'. The goto already tolerated ERR_ABORTED; broaden it to also tolerate the same-URL interrupt (the password-field wait right after confirms we landed on /login), and add the same signature to the two Playwright flake-retry harnesses as a safety net for any other navigation. Validated: playwright_chat_ui.py parses + byte-compiles, both workflow YAMLs parse, bash -n on the retry harnesses, PowerShell AST parse on all pwsh steps, and a functional check of Rename-WithRetry (succeeds, and rethrows after exhausting retries).	2026-06-26 19:45:46 -07:00
Daniel Han	101de1927a	Silence torchao _C*.so load-failure WARNING on torch >= 2.11 (#6712) ... On torch >= 2.11 torchao tries to dlopen each prebuilt _C*.so and logs a per-file "Failed to load .../_C*.so" WARNING via the torchao logger when one cannot load. This happens on an ABI tag mismatch in the prebuilt wheel (for example a cp310 .so under a cp312 runtime, as on Colab) or when the kernel targets an arch the GPU does not have (mxfp8 needs FP8 hardware, _C_cutlass_90a is Hopper/SM90 only). torchao falls back to its non-cpp paths and Unsloth's bnb-4bit / Triton kernels do not use these, so the warning is cosmetic. Add a HideLoggingMessage filter on the same torchao logger that already filters the torch < 2.11 "Skipping import of cpp extensions" message, so only these records are dropped rather than raising the whole logger to ERROR.	2026-06-26 19:45:32 -07:00
Daniel Han	b11966b2db	studio: list the full local model catalog from /v1/models (#6519) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-26 20:42:06 -03:00
Long Yixing	e594e5d201	fix: stop faking 8bit load flag (#6708) ... Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-26 23:54:28 +01:00
Luca Cesarano	4a5d41eb3d	fix(install): enable UV_NATIVE_TLS on macOS for corporate TLS-inspection proxies (#6671) ... --------- Co-authored-by: Luca Cesarano <luca.cesarano@sygnum.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-26 16:48:30 -03:00
Daniel Han	9451aef51e	studio: return a clean model id from the OpenAI API instead of the local .gguf path (#6518) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-26 16:07:53 -03:00
Avaya Aggarwal	cb274484a6	Add GGUF --tensor-parallel CLI option (#6561) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-06-26 15:30:10 -03:00
James Dawdy	b693ed7c91	fix: wrap unprotected evaluate() calls with robust_evaluate() to handle navigation context loss (#6677) ... * fix: wrap unprotected evaluate() calls with robust_evaluate() to handle navigation context loss Fixes PR #5911 - Playwright UI test error: 'Execution context was destroyed' The test had several direct page.evaluate() and locator.evaluate() calls that weren't wrapped with robust_evaluate(), which retries when navigation destroys the execution context mid-operation. Changes: - Wrap picker_visible_text() evaluate in robust_evaluate() - Wrap _bubble_count() evaluate in robust_evaluate() - Wrap assistant text query in robust_evaluate() - Wrap theme_item click evaluation in robust_evaluate() - Wrap background color/theme query in robust_evaluate() This ensures all execution context losses from concurrent navigation are properly caught and retried with exponential backoff, preventing transient failures in the UI test suite. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: revert robust_evaluate on theme_item.evaluate per Codex review The theme_item.evaluate('el => el.click()') is side-effecting — retrying after a context loss could double-toggle the theme. It's already inside a 3-attempt try/except loop that handles click failures gracefully. The other 4 changes (all read-only queries) remain wrapped in robust_evaluate() since retrying them is safe. * fix: wrap remaining chat UI evaluate --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com>	2026-06-26 16:56:52 +01:00
Daniel Han	2ef394137a	Studio: harden background consumer loops and streaming paths against silent UI freezes (#6653) ... * Studio: harden the data-recipe and inference consumer loops against pump death Follow-up to #6643. The same single-unsupervised-consumer pattern the training pump had lives in two sibling loops, with the same failure mode: one bad event kills the only thread that updates the in-memory state every UI surface reads, while the worker subprocess keeps running. - data_recipe JobManager._pump_loop: a malformed worker log line that makes parse_log_message raise no longer kills the pump. Guard _handle_event, the queue read, and the worker-exit finalize, and broaden _drain_queue so a drain error still finalizes the job instead of leaving it wedged "active" (which also leaked the workflow-scoped API key until its 24h expiry). - inference InferenceOrchestrator._dispatcher_loop: guard the routing body so a malformed response or a mailbox put error can't kill the dispatcher and hang every in-flight generation (callers key liveness on the subprocess, not on this thread). Adds regression tests for both. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: extend consumer-loop hardening to RAG, hub, auth, and stream-reader paths Continuation of the data-recipe and inference pump hardening: the same "background producer updates in-memory state that a single unsupervised consumer surfaces to the UI" pattern shows up in several more Studio paths, each able to silently freeze a UI surface while the worker keeps running. RAG ingestion SSE (core/rag/ingestion.py): - job_events polled the queue with a blocking get and never noticed client disconnect or a dead worker, so a closed tab or a producer that died without emitting a terminal event left the stream hanging. It now polls with a timeout, emits heartbeats, ends on terminal job status, caps idle time, and always pops the job registry in finally. - Added _reap_finished_jobs() and call it from start_ingestion so finished job state does not accumulate. Startup reconcile (storage/rag_db.py, main.py): - reconcile_orphaned_ingestion_jobs() marks ingestion jobs (and their documents) that were left non-terminal by a previous crash as failed, so the UI does not show jobs stuck "running" forever after a restart. Wired in at startup next to cleanup_orphaned_runs(). Hub download watcher (hub/services/download_lifecycle.py): - _watch() could leave a job pinned "running" if finalize raised. Body is now guarded: on failure it logs and sets the job to error, and always invalidates the hf cache scan in finally. External provider stream (core/inference/external_provider.py): - read timeout was None (no stall ceiling); set to 300s so a wedged upstream surfaces as an error instead of an indefinitely hung stream. Auth store (auth/storage.py): - Enable WAL + busy_timeout on the auth DB so token validation (read on every request) and login writes stop serialising on the rollback journal. Matches studio_db / rag_db / providers_db. Login rate limiter (routes/auth.py): - _LOGIN_IP_BUCKETS could grow unbounded under spoofed-IP traffic; cap it and prune stale buckets, mirroring the per-account bucket handling. Training progress SSE (routes/training.py): - Break promptly on client disconnect instead of waiting for the next yield to fail on a closed socket, matching the export / data-recipe SSE routes. llama-server stdout drain (core/inference/llama_cpp.py): - Broaden the drain guard so an unexpected decode/read error logs at debug and stops the drainer cleanly instead of escaping the thread. Frontend stream readers (chat-api.ts, rag-api.ts): - Wrap the SSE read loops in try/finally + reader.cancel() so early return ([DONE]), thrown errors, and consumer aborts release the reader lock instead of holding it until GC. Tests: - test_training_progress_stream_nan: fake request now implements the async is_disconnected() the route polls, matching the other SSE route fakes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: address Codex review feedback on the consumer-loop hardening Four follow-ups from the automated review, all on code this PR introduced: - Data-recipe pump (manager.py): a queue read that keeps raising an error outside the read's narrow catch set (e.g. a broken queue pipe after the child died) hit the `continue` guard and skipped the dead-worker finalize below, spinning forever and leaving the job wedged "active" with its workflow key unretired. On a read failure, fall through to finalize when the worker is no longer alive. Added a regression test. - RAG ingestion SSE (ingestion.py): the 5-minute idle cap could end the stream while the job was still pending/running (a large document spends minutes in embedding/storing with no per-batch progress event). The route then sends [DONE], and the client treats a no-terminal-frame end as completion, marking the document indexed mid-ingestion. Drop the idle cap: while the worker is alive and non-terminal we keep heartbeating; the stream ends only on terminal DB status, the None sentinel, or client disconnect. - Login rate limiter (auth.py): the per-IP path pruned but then added the new IP unconditionally, so a spoofed-source-IP spray kept _LOGIN_IP_BUCKETS unbounded and made every new IP pay a full-dict prune scan. Gate the add on the cap, mirroring the account path. - Hub download watcher (download_lifecycle.py): if finalize raised before it reaped (proc.wait) and dropped the worker (e.g. an I/O error draining stderr), the crash path published a terminal state while the live Popen stayed registered and kept writing the cache, and the terminal set_job let claim() admit a retry on the same repo. Terminate + drop the worker before setting the terminal state. * Studio: keep login throttling working when the per-IP bucket dict saturates Review follow-up. The previous cap fix skipped creating a bucket for a new IP once _LOGIN_IP_BUCKETS was full, returning ip_fails=0. Under a sustained spray that also fills the account dict, every failure from such an IP then looked first-seen and _login_blocked had no bucket to enforce, so the cap effectively disabled throttling once saturated. Bound the dict with a FIFO eviction instead: if the IP is new and the dict is full, reclaim expired buckets (rate-limited so a burst of distinct IPs can't make each failure an O(n) sweep) and, if still full, evict the oldest-inserted IP. The new IP always gets a real bucket, so a saturating (e.g. spoofed X-Forwarded-For) spray stays throttled while memory stays bounded. Added a regression test that saturates the dict and asserts a later IP is still blocked. * Studio: address Codex review (RAG queue lifecycle, stream error, orphan chunks) Three follow-ups on the Phase 6 changes: - RAG ingestion SSE (ingestion.py): job_events removed the per-job queue in its finally on ANY exit, including an early client disconnect while the worker is still running. That dropped the worker's later events (the queue is the only one _emit writes to) and made a reconnect find no queue and receive only [DONE], which the client treats as completion. Only drop the queue on a terminal exit (None sentinel / terminal DB status); leftover terminal queues are still swept by _reap_finished_jobs. Added queue-lifecycle tests. - External provider stream (routes/inference.py): once the 300s read timeout can fire, the stream's except path failed the monitor but ended without an error frame or [DONE], so the chat client saw a bare EOF and saved the timed-out answer as a successful partial with no error. Emit an SSE error frame (and [DONE]) on stream failure so the client surfaces it. - RAG startup reconcile (storage/rag_db.py): marking a half-ingested document failed left its chunks/fts/vec rows intact, and retrieval filters by scope not status, so a failed document could still be retrieved and cited. Purge the document's chunks when reconciling it to failed (the doc row stays for re-ingest). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: release the remaining SSE stream readers (training, data-recipe, export) reviewer.py follow-up. The chat and RAG SSE readers were wrapped in try/finally + reader.cancel(), but the other three readers built on the same response.body.getReader() pattern were left without it: streamTrainingProgress, streamRecipeJobEvents, and streamExportLogs leak the ReadableStreamDefaultReader lock (held until GC) when the consumer aborts, returns early, or a parse/callback throws. Wrap each in try/finally + reader.cancel() (export already had a try/catch, so it only needed the finally). All five frontend SSE readers now release the reader symmetrically. * Tighten resilience comments and docstrings Condense the verbose explanatory comments and internal-helper docstrings added in this branch to shorter, clearer forms. Comment/whitespace only; verified no code changed via AST diff. No behaviour change. * Studio: keep chunks for completed docs during ingestion reconcile Startup reconciliation flips orphaned (non-terminal) ingestion jobs to failed and purges the document's chunks so a failed source can't be retrieved. But it dropped the chunks unconditionally, so a document the worker had already committed as 'completed' before the crash (only its job row left non-terminal) lost every chunk while still reporting 'completed'. That leaves an empty source that retrieval can't return and dedup (status != 'failed') blocks from re-ingest. Only purge chunks when the document UPDATE actually transitions it to failed; an already-completed document keeps its chunks. Adds reconcile regression tests for both the completed-doc and genuine in-flight-orphan cases. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: drop a finished RAG job's queue when the client disconnects job_events kept the per-job queue until it consumed the None sentinel, so a UI that stops on the terminal event (its reader.cancel aborts the stream before [DONE]) left the queue registered until the next _reap_finished_jobs sweep; a batch of uploads followed by idling retained them all. _run writes the terminal DB status before emitting the terminal event, so on generator exit, drop the queue when the job's DB row is already terminal (worker done, nothing to resume) and keep it only while the worker is still running. Adds a disconnect-after-terminal-event regression test. * Remove stray async task output files committed by mistake * Studio: harden login IP throttle and end progress stream on disconnect Two Codex review items: Login per-IP throttle: when the per-IP bucket dict saturated, FIFO eviction could drop a still-hot (blocked) bucket, so an IP could flood the dict with distinct (or spoofed) source IPs to push out its own bucket and retry as first-seen. Stop evicting hot buckets; a new IP that can't fit now shares a bounded overflow counter that still trips the per-IP threshold, so a saturating spray stays throttled and no live counter is reset. Progress SSE: on client disconnect the polling loop only broke and fell through to the unconditional final 'complete' frame, so a buffered or proxying consumer could read a still-active run as completed. Return from the generator instead. Adds regression tests for both (spray cannot reset a hot bucket; disconnect while active emits no complete frame). * Studio: shard the login overflow counter and stop cancelling chat stream after [DONE] Two Codex review items: Login throttle overflow: the single shared overflow counter meant that once a saturating spray pushed it past the per-IP threshold, _login_blocked returned 429 for every new unbucketed source IP, before credentials were checked -- a global login denial. Shard the overflow into a fixed array of counters keyed by hash(ip), so a hot shard only throttles the IPs that map to it while a single source's repeated failures still concentrate in one shard and stay throttled. Memory stays bounded and no live bucket is evicted. Adds a regression test that a hot overflow shard does not block an unrelated IP. Chat stream: the reader.cancel() in the SSE finally fired even after a natural [DONE]/EOF. The backend finalizes its api-monitor entry right after yielding the sentinel (the local pass-through finishes after the last yield), so a client cancel there can be observed as a disconnect and mark a completed request as cancelled. Track natural completion and only cancel on an early/abnormal exit. (No frontend unit test: the Studio frontend has no test harness.) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: give prep-timeout test fakes an is_disconnected method The progress stream now ends on client disconnect (await request.is_disconnected() before falling through to the terminal frame). After merging that into the prep-timeout tests added later on main, their _FakeRequest/_ReconnectRequest must provide is_disconnected or the generator raises AttributeError under CI. * Studio: keep the login overflow throttle when bucket capacity frees up _login_blocked only consulted the per-IP overflow shard while the bucket dict was still at capacity. If a slot freed before the 60s window expired (e.g. another IP's successful login calls _clear_login_bucket), a source counted in a hot shard stopped being blocked and its next failure got a fresh per-IP bucket, resetting the throttle the overflow path exists to preserve. Always max in the IP's shard (shards are empty outside saturation, so it is a no-op in the common case). Adds a regression test that a hot source stays throttled after a bucket frees. * Studio: clear a login IP's overflow throttle on successful login _clear_login_bucket reset the per-IP and per-account buckets on a successful login but not the overflow shard, so after the dict saturated and an IP was counted in overflow, a later successful login left those entries behind and the next failed attempt could immediately return 429. Store overflow entries as (timestamp, ip) so a source is throttled by its own count within the shard (also removing cross-IP collateral within a shard), and drop just that IP's entries in _clear_login_bucket. Adds a regression test that a successful login clears the overflow throttle. * Studio: bound the login overflow shard memory under high-cardinality spray The per-IP overflow tracked failures in a time-pruned deque of (timestamp, ip) tuples, so a spoofed-X-Forwarded-For spray of distinct one-off IPs grew memory and the per-check scan with request cardinality for the whole window -- undermining the bucket cap that exists to bound memory. Replace each shard with a fixed- capacity dict (ip -> [count, window_start]): O(1) lookups, and when a shard is full a one-off IP evicts the lowest-count entry (Space-Saving) so memory is hard- bounded while a persistent attacker keeps a high count and is never evicted. Adds a regression test that shards stay within the per-shard cap under a 5000-IP spray. * Studio: purge chunks for already-failed docs during ingestion reconcile The reconcile chunk-purge was gated on the documents UPDATE actually flipping a non-terminal doc to failed. A doc the worker had already marked 'failed' before the crash (job row left non-terminal) was not re-flipped, so its committed chunks were kept and stayed retrievable/citable, since retrieval filters by scope not status. Purge chunks whenever the document is not 'completed' (failed, in-flight, or gone), preserving the completed-doc carve-out. Adds a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: don't inherit an evicted IP's count onto a new overflow source When a full overflow shard evicted the lowest-count entry, the new source inherited that count (Space-Saving base + 1). If a shard was saturated with hot entries, an unrelated new IP could land at/over the threshold and be 429'd after a single attempt -- cross-IP collateral despite the per-source-isolation intent. New entries now start clean at count 1; the only cost is that a heavy hitter that is the lowest-count entry in a fully saturated shard can briefly reset, which is preferable to blocking a bystander. Adds a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: carry overflow failures into a new IP bucket on transition _login_blocked took max(per-IP bucket, overflow shard) rather than combining them, so a source could log (threshold-1) failures in overflow during saturation and, once a bucket slot freed, another (threshold-1) in a fresh bucket within the same window -- roughly doubling the per-IP limit. When a saturated-era IP first gets a real bucket, migrate its windowed overflow count into that bucket (and drop the overflow entry) so the combined failures throttle at the intended limit. Adds a regression test. * Studio: reconcile a completed doc's orphaned job to completed, not failed When a crash left an ingestion job non-terminal after its document was already committed as completed, reconcile marked the job failed. After restart the upload UI has no in-memory SSE queue and falls back to getJob(), which treats a failed job as an indexing failure and removes/toasts a document that is actually searchable. Mark the job completed (keeping its chunks) when its document is completed. Extends the completed-doc reconcile test to assert the job status. * Studio: clamp the overflow failure count migrated into a login bucket A saturated source could accrue an unbounded overflow count, then materialize one deque entry per recorded failure when a bucket slot freed, allocating an arbitrarily large deque under the login lock. Only at-or-above the per-IP threshold matters for blocking, so cap the count there at the record and take sites; the migration is now bounded without weakening the limit. * Studio: keep the RAG job stream alive on a transient status read The heartbeat poll read the job row unguarded; a momentarily-locked DB would raise out of job_events, which the SSE route turns into a terminal error frame, and the UI drops a document whose worker is still running. Treat a failed status read as non-terminal: heartbeat and retry, and keep the queue so a reconnect can resume. * Studio: set busy_timeout before journal_mode on the auth DB Switching journal_mode needs a lock, so if a refresh-token write already holds one, journal_mode=WAL raises SQLITE_BUSY and the shared try leaves the connection on SQLite's default zero lock wait. Set busy_timeout first so the switch waits instead of failing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-26 03:31:33 -07:00
Matt Van Horn	e9c6364e1e	feat: improve Unsloth Studio chat title generation quality (#6697) ... * feat: improve Unsloth Studio chat title generation quality * fix: address self-review (guard echoed role labels before punctuation stripping) * Address title generation review feedback Consolidate the echo guard into a single leading-label check (now also covering base and lora) and drop the post-punctuation duplicate that could never match a colon once punctuation is stripped. Swap the slice-based first-assistant lookup for an indexed find to avoid copying the messages array, and note the brace counter's assumptions in the test helper. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Daniel Han <unslothai@gmail.com>	2026-06-26 01:55:59 -07:00
Anmol Mishra	3e43ed7b4a	Patch FalconH1RMSNorm to fix float64 compilation crash on Intel Arc DG2 (#6691) ... * Patch FalconH1RMSNorm to fix float64 compilation on Intel Arc DG2 Fixes unslothai/unsloth#6555 Root cause: FalconH1RMSNorm.forward() does hidden_states.pow(2).mean().rsqrt() with self.variance_epsilon being a Python float64. When torch.compile fuses this pattern into the auto-generated Triton kernel 'triton_per_fused__to_copy_mean_mul_pow_rsqrt_*', the float64 epsilon causes type promotion to double. Intel Arc DG2 GPUs do not support double precision (Double type is not supported on this platform). The existing patch_rms_layernorm() only patches LlamaRMSNorm, not the separate FalconH1RMSNorm class in transformers.models.falcon_h1. Fix: add Unsloth_FalconH1RMSNorm that delegates to fast_rms_layernorm (@torch.compiler.disable, handles epsilon as tl.float32), and call the patch in FastFalconH1Model.pre_patch() before model creation. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Condense FalconH1RMSNorm patch comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-26 01:45:36 -07:00
Abdul Moiz	a9c8bcf0e1	Fix DDP crash from CPU-resident rotary inv_freq buffer (#6662) ... * Fix DDP crash from CPU-resident rotary inv_freq buffer DistributedDataParallel broadcasts all named buffers regardless of persistence or device, but Unsloth's RoPE inv_freq buffer is kept on CPU on purpose (per-GPU cos/sin caches are precomputed instead). That mismatch crashed multi-GPU DDP training with "No backend type associated with device type cpu" during _sync_module_states. Mark inv_freq/short_inv_freq/long_inv_freq buffers as DDP-ignored instead of moving them to GPU, so they're skipped during the buffer broadcast without disabling broadcast_buffers for the rest of the model. Fixes #6656 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: harden DDP-ignore against private API drift, re-apply after PEFT wrap - Wrap the private DistributedDataParallel._set_params_and_buffers_to_ignore_for_model call in try/except, falling back to setting _ddp_params_and_buffers_to_ignore directly so a future PyTorch API change can't block model loading. - Move _exclude_rope_inv_freq_from_ddp to loader_utils.py (shared by loader.py, llama.py, vision.py without circular imports) and call it again after get_peft_model wraps the model in a PeftModel, since the rotary buffers' fully qualified names change once nested under "base_model.model...". * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com>	2026-06-26 09:44:41 +01:00
Daniel Han	7f45635280	Studio: auto-shut-down an exposed first-run instance if the admin password is never changed (#6651) ... * Studio: set the admin password before exposing it on the network On first run Studio seeds the default `unsloth` admin with a random bootstrap password and embeds it into index.html (window.__UNSLOTH_BOOTSTRAP__) so the local user can change it without typing it. A request with no Origin header counts as same-origin, which is what a normal top-level GET sends, so the page hands out the password to whoever loads it. That is harmless on the default 127.0.0.1 bind, but `--secure` (public Cloudflare tunnel) and `--host 0.0.0.0` (raw port reachable on the network) would serve the plaintext admin password to remote visitors during the bootstrap window. Fix this at the source: when launching a network-exposed web UI, prompt the operator in the terminal for a real admin password (with confirmation) before the socket binds or the tunnel opens, and persist it via update_password (which clears must_change_password and deletes the .bootstrap_password file). After that there is no bootstrap secret to leak. Non-interactive launches can supply it via UNSLOTH_STUDIO_ADMIN_PASSWORD. The masked reader echoes '*' per character and works on Linux, macOS, and Windows (PowerShell/cmd). Loopback binds, --api-only (no web UI), and Colab are unaffected. As defense in depth, the index handler now embeds the bootstrap object only for a direct local navigation: same-origin AND a loopback TCP peer with no proxy/tunnel forwarding headers (cf-ray, cf-connecting-ip, x-forwarded-for, x-forwarded-host, x-real-ip, forwarded). Colab stays exempt. This keeps the password off the wire even when the prompt is skipped (no TTY and no env var). Adds unit coverage for the prompt/confirm/decision logic, an integration test that provisioning clears the bootstrap state, and regression tests for the local-direct gate (loopback/IPv6/mapped/localhost peers, LAN/public peers, missing client, each forwarding header, spoofed XFF, and the Colab exemption). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fail fast on an explicitly empty admin-password env var resolve_admin_password_source treated UNSLOTH_STUDIO_ADMIN_PASSWORD="" like the var was unset and fell back to the bootstrap backstop. Treat any set value (including empty) as the env source so it reaches the minimum-length guard and refuses to expose the server instead of silently keeping the seeded password. * Studio: apply repo kwarg-spacing format to the secure-admin-password files * Studio: drop the pre-exposure password prompt; keep the local-direct gate Per review, the blocking prompt added friction for --secure / 0.0.0.0 first-run launches without extra security: the local-direct injection gate in main.py already keeps the bootstrap password off the network for any remote request. Remove the prompt module and its tests; the gate plus the existing must_change_password first-login flow are the fix. * Studio: shut down an exposed first-run instance if the admin password is never changed The local-direct gate keeps the seeded bootstrap password off the network, but it stays a valid credential until first login changes it. For an exposed web UI (--secure / 0.0.0.0, not --api-only, not Colab), arm a daemon timer: if the password is still the seeded one after the deadline (UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT, default 3600s, 0 disables), print a message and shut Studio down via the existing graceful-shutdown path; if it was changed, leave Studio running. * Studio: revert the local-direct injection gate; keep the 1-hour auto-shutdown Per maintainer decision, keep the first-run auto-fill behavior unchanged (the bootstrap password still seeds the login form for convenience) and rely on the exposed-instance auto-shutdown to bound the window: an exposed web UI that never changes the seeded admin password is torn down after UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT (default 1h). Restores studio/backend/main.py and its origin test to upstream. * Studio: render the bootstrap-timeout shutdown message with a human duration The message hardcoded 'minute(s)' via timeout//60, so a sub-minute timeout (e.g. a 30s test value) printed 'within 1 minute(s)'. Add _format_duration so it reads '30 seconds' / '1 minute 30 seconds' / '60 minutes' as appropriate. The default 3600s still renders '60 minutes'. * Studio: drop stale local-direct gate reference from bootstrap_timeout docstring The gate was reverted (timer-only), so the module docstring should not describe a main.py gate that no longer exists. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-26 01:27:27 -07:00
Leo Borcherding	c86165e735	Studio Colab: opt-in shareable Cloudflare tunnel link (#6684) ... * Studio Colab: add opt-in shareable Cloudflare tunnel link colab.start(cloudflare=True) opts in to a free Cloudflare quick tunnel and shows a trycloudflare.com link above the proxy iframe, reachable from any device. Default OFF: bare start() keeps the in-tab Colab-proxy behavior. run_server suppresses the tunnel on Colab by design, so colab.py starts it directly via cloudflare_tunnel.start_studio_tunnel(); failures degrade to the Colab proxy only. * Studio Colab notebook: surface opt-in cloudflare=True in start cell * Studio Colab: reskin shareable Cloudflare link to match the proxy banner Retrofit _shareable_link_html to reuse the original Colab proxy banner skin from show_link (white card, black border, Unsloth gem, black Open button) instead of the plain dark box, so the shareable Cloudflare link gets the same prominent 'Ready!' treatment. * Studio Colab: address review feedback on Cloudflare tunnel - try/finally around tunnel start + embed + keepalive so a KeyboardInterrupt while the tunnel is starting or the iframe is rendering tears it down instead of orphaning the cloudflared process (Gemini review). - Publish the directly-started tunnel URL onto app.state.cloudflare_url via a new _publish_cloudflare_url helper so /api/health advertises it; otherwise the frontend's API examples fall back to the unreachable raw server_url (Codex P2). _stop_cloudflare_tunnel now also clears it so health stops showing a dead tunnel. - Notebook: make cloudflare=True a replacement for start(), not an addition, since start() blocks and the second call would never run if both are left in (Codex P2). * Studio Colab: gate Cloudflare tunnel on auth + honor opt-out in run_server - Refuse to open the Cloudflare tunnel while the admin still holds its seeded bootstrap password. While requires_password_change is true the server injects that password into same-origin index GETs, and a public tunnel request counts as same-origin, so sharing the link would leak admin access. New _bootstrap_password_pending() gate (fails safe) blocks the tunnel and tells the user to change the password first, then re-run start(cloudflare=True) (P1). - Pass cloudflare=False into run_server so the opt-out holds even when Colab detection fails; this helper is now the sole owner of the tunnel decision, preventing run_server from opening a tunnel on the 0.0.0.0 bind by default (P2). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio Colab: drop duplicate tunnel link log and simplify start cell guidance * Studio Colab: validate /api/health identity before reusing or tunneling a port * Studio Colab: condense verbose docstrings and comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-26 00:56:23 -07:00
Daniel Han	1396c01253	Fix offline checkpoint load/export: "tokenizer is weirdly not loaded" (#6554) ... * Fix offline checkpoint load/export failing with "tokenizer is weirdly not loaded" Loading a fine-tuned checkpoint with no internet (e.g. a Studio export) crashed with "Unsloth: The tokenizer is weirdly not loaded? Please check if there is one." For a LoRA adapter the loader reassigns model_name to the base model repo id and only keeps the local checkpoint dir as tokenizer_name when it contains tokenizer_config.json, tokenizer.json AND special_tokens_map.json. Modern tokenizers (e.g. Gemma) store special tokens inside tokenizer_config.json and omit special_tokens_map.json, so tokenizer_name fell back to the base repo id. The tokenizer/processor loads in vision.py then hit the Hub with no local_files_only, so with no network they failed (AutoProcessor) or hung for minutes (AutoTokenizer) even though every file was already cached. loader.py: keep the local checkpoint dir as tokenizer_name when it has a tokenizer config plus the actual tokenizer files (tokenizer.json / tokenizer.model / vocab files); special_tokens_map.json is no longer required. vision.py: compute an effective local_files_only (explicit kwarg plus the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars, mirroring loader.py and diffusion.py) and thread it through every AutoConfig, AutoProcessor, AutoTokenizer and the manual VLM processor fallback, including the hf_hub_download in that fallback (which now prefers a local file). When a load fails and no offline env var is set, retry against the local cache. The retry forces HF offline mode because local_files_only alone does not stop AutoProcessor / AutoTokenizer from issuing a /api/models request during class resolution. The final error now explains the offline/cache cause instead of the misleading "weirdly not loaded" message. studio export: probe Hub reachability once per checkpoint load and pass local_files_only when offline so exports use the local checkpoint dir / cache instead of hanging or crashing with no internet. Online behavior is unchanged: the new flags default to off and the retry only runs after a network related failure. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: safer offline forcing, cached fallback config, proxy-aware probe Follow-up to the offline checkpoint load fix, addressing review feedback: - vision.py: only flip the process-wide HF offline flag when offline is actually requested (local_files_only / env) or after a real network failure, never pre-emptively while we might be online. The flip is now guarded by a lock + depth counter so nested or concurrent windows restore the flag correctly (no stale value). - vision.py: guard the get_auto_processor fallback so a network error there returns None and the local-cache retry still runs instead of escaping. - vision.py: in the manual VLM processor fallback, read tokenizer_config.json via hf_hub_download(..., local_files_only=...) so a cached repo-id config is still resolved offline and the model-specific image/video tokens are restored. - studio export: make the reachability probe proxy aware (probe the configured HTTP(S) proxy egress, honour NO_PROXY, use the endpoint port) so a proxy-only setup is not wrongly marked offline; allow UNSLOTH_OFFLINE_PROBE=0 to disable. - studio export: run the audio/vision type-detection probes inside the forced-offline window when offline, so their config/tokenizer reads hit the local cache instead of waiting out connection timeouts. Online behavior remains unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: gate offline retry, safer tokenizer_name pop, skip audio net probe offline - vision.py: only force the process-wide HF offline flag on the tokenizer retry when offline was requested or the captured primary error is actually network related, so a permanent tokenizer error no longer toggles global offline mode for other concurrent loads. - loader.py: always pop tokenizer_name out of kwargs and let a caller-supplied value win, avoiding a "multiple values for keyword argument 'tokenizer_name'" TypeError when it is also passed explicitly downstream. - model_config.py / export.py: add local_files_only to detect_audio_type so the raw requests.get tokenizer_config fetch is skipped offline (it ignores the HF offline flag), and pass it from the export probe. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: classify LocalEntryNotFoundError as offline-related huggingface_hub's LocalEntryNotFoundError subclasses FileNotFoundError, so the "not isinstance(cur, FileNotFoundError)" guard in _is_offline_related_error was swallowing it and it could never be recognised as offline, despite being listed in the network error types. It means "not in cache and the Hub is unreachable", which is genuinely offline. Capture the class into an isinstance-checkable tuple (empty, hence a no-op, if the import is unavailable) and exclude it from the FileNotFoundError guard, so a real offline failure now triggers the local-cache retry while a plain missing-file error still propagates. * Address review: require merges.txt for BPE, status-gate HTTP errors, isolate local-only audio cache - loader.py: a local dir with vocab.json but no merges.txt (and no tokenizer.json) is not a loadable BPE tokenizer, so do not treat it as self-sufficient; require merges.txt alongside vocab.json in both gate blocks, otherwise fall back to the base model tokenizer as before. - vision.py: _is_offline_related_error no longer buckets every HfHubHTTPError / requests HTTPError as offline. HTTP errors are judged by status code: only a transient 5xx triggers the forced local-cache retry, while 401/403 (auth/gated) and 404 (missing) propagate as the real error instead of being masked. Hard signals (connection/timeout/OfflineModeIsEnabled/LocalEntryNotFoundError) still classify as offline. - model_config.py: include local_files_only in the audio-detection cache key so a local-only (offline) negative result cannot be reused by a later online probe, which would otherwise route an audio model through the text loader until restart. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address re-review: fix studio test stubs, force offline env in probe window, drop redundant retry - studio/backend/tests/test_vision_cache.py: the three _detect_audio_from_tokenizer stubs were called with the new local_files_only kwarg and raised TypeError, failing Backend CI. Add local_files_only to the stub signatures and add a test that a local-only negative does not poison a later online audio probe. - export.py: the type-detection probe window now also sets HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars (saved/restored), not just the in-process flag. transformers_version._load_config_json / _check_tokenizer_config_needs_v5 gate their urllib fetches on the env vars, and is_vision_model may spawn a subprocess that inherits os.environ but not the in-process flag; without the env vars a probe-detected offline export could still block on a network timeout. - vision.py: only retry the processor load when the first attempt was online and failed with a network error. When local_files_only was already requested the first attempt was forced offline, so the previous retry just repeated identical failing work before the last-resort path. - model_config.py: correct the _audio_detection_cache type annotation to the 3-tuple key (name, token_fingerprint, local_files_only). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: thread-safe probe-offline env window, clear error for local dir without config - export.py: guard the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE mutation in _force_offline_probe_window with a lock + depth counter (mirrors _force_hf_offline), so concurrent / nested export probes only flip on first entry and restore on last exit. This prevents overlapping export requests from permanently poisoning those env vars or restoring a stale value. - vision.py: in the VLM processor fallback, when tokenizer_name is a local directory, read its tokenizer_config.json directly and raise a clear FileNotFoundError if it is absent, instead of handing the local path to hf_hub_download (which would treat it as a repo id and raise a confusing HFValidationError / RepositoryNotFoundError). hf_hub_download is now only used for actual repo ids. * Address review: classify raw socket.gaierror DNS failures as offline Add the platform-specific getaddrinfo / DNS-resolution wording to the offline detection list in _is_offline_related_error so a bare socket.gaierror (an OSError subclass) is recovered from the local cache: "Name or service not known" and "Temporary failure in name resolution" (Linux) and "nodename nor servname provided" (macOS). Genuine non-network OSErrors (disk full, permission denied) and plain FileNotFoundError still propagate. * Address review: retry degraded VLM offline, force offline for text export + patch-tokenizer fallback - vision.py: a degraded VLM processor (text-only, no image_processor) whose manual fallback fails offline used to be kept, so image inputs broke even with cached files. _construct_vlm_processor_fallback now returns its failure error; _acquire_processor surfaces it, and the caller retries forced-offline when the result is None OR a degraded VLM and the failure was network related, keeping the original result if the retry is not strictly better (never regress). The retry is still gated on an online first attempt + offline-related error so a permanent error never flips the global offline flag. - vision.py: wrap the patch_tokenizer except-branch AutoTokenizer.from_pretrained in the same forced-offline-on-network-error pattern as the primary / last-resort loads, so an offline export where patch_tokenizer raises does not hang or fail. - export.py: force HF offline around the two FastLanguageModel loads (text and SNAC) when the probe detected offline. Their text tokenizer path (load_correct_tokenizer -> AutoTokenizer) does not forward local_files_only, so without this a text export could still contact the Hub. Added a small _offline_window_if helper reused by the probe and load windows. * Consolidate offline loading into one entry-point decision Decide offline once per entry point instead of at every HF call site. The prior approach threaded local_files_only into ~15 scattered config / tokenizer / processor / weight loads, each wrapped in its own try-online, classify-error, retry-forced-offline dance, which is what kept surfacing "another call site you missed", "another error shape misclassified", and global-flag thread-safety in review. FastLanguageModel / FastModel / FastBaseModel.from_pretrained now share an @_offline_aware_load decorator: when offline (explicit local_files_only kwarg or HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env) it sets local_files_only and runs the whole load inside one _force_hf_offline() window so every nested HF call inherits it; when online it runs normally and, only if the load fails with a genuinely network-related error, retries once forced-offline. The online path is unchanged (no probe added) and 401 / 403 / 404 / permanent errors still propagate. Centralise the offline helpers in loader_utils.py as the single source of truth (shared by loader.py, re-exported from vision.py, and reused by the Studio exporter): - _force_hf_offline now sets the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars AND the in-process huggingface_hub / transformers flags, refcounted under one lock so nested / concurrent windows restore correctly. Setting the env vars covers env-gated urllib probes and spawned subprocesses too. - _get_effective_local_files_only, _is_offline_related_error (unchanged classifier, retains the 5xx-vs-4xx, LocalEntryNotFound and gaierror handling), _offline_aware_load, and _resolve_checkpoint_tokenizer_name. loader.py: wrap both entry points; drop the two duplicated env-var fallback blocks and the two byte-identical local-tokenizer-gate blocks (now _resolve_checkpoint_tokenizer_name). vision.py: drop the per-site force_offline params and the three retry gates (processor, patch_tokenizer fallback, last-resort). They now just surface the underlying error so the single entry-point safety net retries forced-offline. A network fallback error now takes precedence over a permanent primary error so the offline retry still fires when the manual VLM fallback needs cached repo files. studio/backend export.py: reuse the unified core _force_hf_offline (env + flags) and drop the duplicate probe-window primitive; the snac / text branches no longer need their own window. model_config.py: also gate the raw requests.get audio fallback on the HF offline env vars so it is covered even without the kwarg. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address 10-reviewer P1 findings: vision cache split, PEFT offline, retry OOM Split the Studio vision-detection cache by local_files_only, mirroring the audio cache fix. is_vision_model / _is_vision_model_uncached / _raw_config_has_vision_config / load_model_config now thread local_files_only, the cache key includes it, and the exporter passes it. Offline detection also skips the transformers-5 network subprocess and stays on the local cache, so an offline negative can no longer be keyed under the online entry and poison a later online probe. Adds a regression test mirroring the audio poison test. Forward local_files_only to both PeftModel.from_pretrained adapter-attach sites in loader.py so a cached remote LoRA adapter resolves from the local cache under explicit local-only / offline loads (defence-in-depth alongside the forced-offline window). _offline_aware_load: run the forced-offline retry OUTSIDE the except block and collect + empty the device cache first. An except-scoped exception keeps its __traceback__, which pins the failed attempt's frame locals (a partially loaded model) until the block exits; loading the model again while that copy is still alive could OOM a large VLM. Letting the except block close drops the traceback so the partial load is freed before the retry reallocates. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address Codex review: env-offline cache key + rebuild HF sessions in offline window Key the Studio audio and vision detection caches on the EFFECTIVE offline state (local_files_only OR the HF offline env vars), not just the kwarg. detect_audio_type and is_vision_model both skip the remote fetch / network subprocess when HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE is set even with the default local_files_only=False, so the result reflects offline; storing it under the online (False) key let an env-offline negative poison a later online lookup once the env var was cleared. Both now compute effective_offline once and use it for the cache key and the downstream call. Adds a regression test for the env-offline dimension. _force_hf_offline now rebuilds huggingface_hub's cached sessions on enter and exit (best-effort _reset_hf_sessions). On hub 0.x the offline adapter is baked into the per-thread requests.Session at creation, so flipping the constant alone leaves an already-cached online session able to hit the network inside the window (and an offline one stuck offline after restore); resetting forces the next get_session() to match the current flag. On hub 1.x offline is checked dynamically per request, so reset_sessions does not exist and the helper is a safe no-op. The third review point (release the failed load before retrying) was already fixed in af0f58a: the forced-offline retry now runs outside the except block and frees the device cache first, so the failed attempt's traceback-pinned partial model is released before the retry reallocates. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Align Studio _env_offline parsing with the canonical offline helper model_config._env_offline gates the raw requests.get tokenizer-config fallback in detect_audio_type and the audio/vision detection cache keys, but it only accepted unstripped "1"/"true"/"yes". unsloth's offline helpers (loader_utils._env_says_offline and the from_pretrained env fallback) accept the canonical set {1,true,yes,on} after strip + lowercase, so HF_HUB_OFFLINE=on or HF_HUB_OFFLINE=" 1 " was treated as offline by the loaders but online here, leaving the raw network fetch reachable while "offline". Use the same strip + lowercase {1,true,yes,on} set. Adds parsing tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix lint: drop dead offline-helper re-exports from vision.py The import-hoist verifier (scripts/verify_import_hoist.py) flagged vision.py's re-export block as HOISTED-IMPORT-UNUSED blockers: it imported eight offline helpers from loader_utils but only used three internally (_get_effective_local_files_only, _is_offline_related_error, _offline_aware_load). The other five were imported purely to preserve `from unsloth.models.vision import X`, but nothing imports four of them from vision, and loader.py already imports _resolve_checkpoint_tokenizer_name straight from loader_utils. Import only the three names vision.py actually uses, and point the Studio exporter at the canonical source (from unsloth.models.loader_utils import _force_hf_offline) instead of re-exporting it through vision. loader_utils stays the single source of truth; no behaviour change. * Address Opus review: chain probe errors, unify env-offline, status-less HTTP Chain the original AutoConfig/PeftConfig probe exception into the combined RuntimeError in both FastLanguageModel.from_pretrained and FastModel.from_pretrained (`raise RuntimeError(combined_error) from (autoconfig_exc or peft_exc)`). The probes caught every Exception and stringified it, so the re-raised RuntimeError had no __cause__/__context__ and _is_offline_related_error could not classify it -- the network-down-but-cached auto-retry never fired for these entry points. With the cause chained, the decorator sees a ConnectionError/LocalEntryNotFoundError/5xx and retries forced-offline from cache; a permanent cause (404 / bad config) is still not offline-classified and propagates without a wasted retry. Unify the third offline-env parser: studio/backend/utils/transformers_version._env_offline now uses the canonical {1,true,yes,on} + strip + lowercase set (matching loader_utils._env_says_offline and model_config._env_offline), so HF_HUB_OFFLINE=on or " 1 " no longer leaks the direct urllib metadata fetches to the network. _is_offline_related_error: a status-less HTTP error (no response / unparseable code) now falls back to the network-wording check instead of being dropped, so a transient HTTP failure with clear "couldn't connect" wording is treated as offline. HTTP errors with a real status code still decide by code (4xx propagates, 5xx is offline). * Condense offline-loading code comments, drop dead helper, dedupe import for PR #6554 * Add unit tests for offline-loading helpers for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard load cleanup with try/finally and add retry-contract tests for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add gc.collect retry-step test for PR #6554 * Tighten offline-loading comments and docstrings for PR #6554 * Raise the both-config-failed error before model-type lookup so offline retry fires for PR #6554 * Prefer offline cause for retry and bound export reachability probe for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip remote mapper while offline, harden text-load cleanup, and stop stacked offline retries for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Surface VLM fallback offline errors, probe offline before export version activation, and restore progress bars across retries for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore offline env after export version activation so the persistent worker re-decides per load for PR #6554 * Classify socket.gaierror and urllib URLError as offline by type for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Probe offline around export load preflights and never offline-retry TLS failures for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Force in-process offline for export preflights, verify proxy egress in probe, and skip caching offline version negatives for PR #6554 * Snapshot offline constants before forcing env and require local processor files for VLM checkpoints for PR #6554 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-25 23:16:53 -07:00
Daniel Han	80d3434d61	Studio: require signed capability tokens for /p preview links (#6666) ... * Studio: require signed capability tokens for /p preview links The public /p preview routes added in #6486 run model load and chat generation as the admin user with no authentication. The only gate is the preview ref, a deterministic outputs-root path (run or run/checkpoint) that is guessable rather than secret. On a network-reachable Studio (--secure tunnel or -H 0.0.0.0), an unauthenticated caller who guesses a ref can consume GPU and probe a private fine-tuned checkpoint. Make the share link an unguessable, revocable capability: - Sign the canonical ref with a dedicated server-side secret (HMAC-SHA256, stored in app_secrets, independent of the JWT/login secret). - Require a valid token on every /p chat, models, and page request before resolving a checkpoint or loading a model; missing or invalid tokens get a generic 404 so the surface never confirms a ref exists. - Accept the token via ?k= (browser link and preview page) or Authorization: Bearer (OpenAI-compatible clients). - Rotate the secret to revoke every outstanding link (POST /api/settings/preview-links/rotate). - Clamp preview generation (max_tokens/max_completion_tokens <= 1024, n = 1) and set Referrer-Policy: no-referrer on the page so the token is not leaked via Referer. Training history hands the authenticated owner the signed token, and the copy-link button builds /p/{ref}?k={sig}. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: honor a lower caller token limit in the preview clamp Codex review: when only the legacy max_tokens was sent, the clamp left max_completion_tokens at the 1024 default, and _effective_max_tokens prefers max_completion_tokens, so a request like max_tokens=16 could still generate up to 1024 tokens. Derive one effective limit (max_completion_tokens wins, else the legacy max_tokens) and pin both fields to it so a caller's lower limit is kept. * Studio: add preview kill switch, rate limit, and revoke-links UI Follow-ups to the /p preview capability work: - Public-sharing kill switch: a persisted setting (default on) gates the public /p surface. When off, every preview request 404s even with a valid token, and the owner UI stops offering share links. GET/PUT /api/settings/preview-sharing; enforced in _verify_or_404. - Per-IP rate limit on the preview chat route: a coarse in-process sliding-window limiter (20 req/min/IP) returns 429 + Retry-After before the GPU lock is taken. Client IP honors X-Forwarded-For only when UNSLOTH_STUDIO_TRUST_FORWARDED is set, matching the login limiter's trust model. - Settings UI: a "Preview sharing" section with the public-sharing toggle and a "Revoke all preview links" button (confirm dialog) that rotates the secret. Tests cover the kill switch (404 when off), the 429 path, the sliding window, client-IP trust behavior, and the setting default. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fix preview-fields sharing arg and refresh sigs after revoke Codex review: - P1: get_training_run_detail and update_training_run called _preview_fields with only output_dir after it gained a required sharing_on parameter, raising a 500 TypeError once get_run succeeded. Pass get_preview_sharing_enabled() at both sites; add a detail-endpoint regression test. - P2: after rotating the preview secret from settings, the history grid still held stale preview_sig values, so a freshly copied link would 404. Emit emitTrainingRunsChanged() after a successful revoke so the grid refetches freshly signed refs. * Studio: harden preview sharing controls (Codex review) - Fail closed: a read failure on the preview-sharing kill switch now returns False instead of defaulting to enabled, so an unavailable settings DB can't reopen the public surface. A missing key still defaults to enabled. - Per-IP rate limit behind the managed Cloudflare tunnel: client_ip now honors CF-Connecting-IP when the socket peer is loopback, so tunneled visitors are keyed by their real IP instead of collapsing onto the local cloudflared peer. - GET /p no longer mints key/share_url when sharing is disabled; it returns sharing_enabled=false so clients don't distribute links that 404. - Settings UI: toggling public sharing emits the training-runs-changed event so the history grid shows/hides Copy preview link without a manual refresh. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden preview rate limiter and IP keying (Opus review) From a two-agent review of the PR: - Rate limiter no longer evicts an active bucket when the table is full: a flood of distinct keys could otherwise cycle out a throttled bucket and reset its counter. Evict only aged-out buckets; if the table is full of live clients, fail closed (deny the new key) instead. - client_ip keys on the rightmost (proxy-appended) X-Forwarded-For hop when the trust env is set; the leftmost is client-spoofable. Documented the append/overwrite-proxy assumption. - _verify_or_404 checks the capability token before the kill-switch DB read, so unauthenticated /p spam can't be used as an unbounded settings-DB sink and the response is identical regardless of the sharing on/off state. Tests: nested run/checkpoint happy path + wrong-ref rejection, the eviction fail-closed behavior, and route-level coverage for the rotate / preview-sharing settings endpoints. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-25 21:40:48 -07:00
Daniel Han	ed5e2a1590	Verify linuxdeploy AppImage digest before use in desktop release (#6673) ... * Verify linuxdeploy AppImage digest before use in desktop release The desktop release workflow downloaded linuxdeploy-x86_64.AppImage from a GitHub release and ran chmod +x with no integrity check. Pinning the versioned release path is reproducibility, not integrity: a release asset can be replaced (or its delivery path compromised) after upload. The next step builds the AppImage with the Tauri signing private key and a contents:write GITHUB_TOKEN in scope, so a substituted linuxdeploy that ran during packaging could exfiltrate signing material or tamper with published release artifacts. Pin the immutable SHA-256 of the asset and verify it with sha256sum -c before chmod +x, so a mismatch fails the job closed before the binary is ever executable. Extend the existing in-workflow guard to require both the pinned digest and the verification step, so a future edit cannot silently drop the check. * Scope linuxdeploy guard to real step content, not its own text The self-check searched every workflow line, so the digest assertion was satisfied by the guard's own expectedLinuxdeployDigest line and the verification assertion by a comment. Deleting the LINUXDEPLOY_SHA256 env pin or the actual sha256sum -c command would still have passed. Match the digest against the LINUXDEPLOY_SHA256 env line specifically and require sha256sum -c on a non-comment line, so dropping either the pin or the verification now fails the guard. * Scope linuxdeploy guard to the Pin step block and check ordering The previous predicate still scanned the whole workflow, so the literal sha256sum -c in the guard's own code satisfied the verification check; a deleted or post-chmod verification command would still pass. Extract the 'Pin linuxdeploy for AppImage' step block and assert within it: the LINUXDEPLOY_SHA256 env pins the expected digest, a non-comment line runs sha256sum -c, and that verification precedes chmod +x.	2026-06-25 20:45:24 -07:00
Lee Jackson	c873ef052d	Studio: prompt variables into prompt editor (#6434) ... * add custom and system variable feature in system prompt * missing function use * feat: prompt variables editor ux Co-authored-by: CodeMan62 <175127021+CodeMan62@users.noreply.github.com> * fix: guard prompt variable defaults * refine prompt variables editor layout * fix: harden prompt variable substitution * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refine prompt variables editor copy and built-in token labels --------- Co-authored-by: CodeMan62 <sharmahimanshu15082007@gmail.com> Co-authored-by: CodeMan62 <175127021+CodeMan62@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>	2026-06-25 18:09:16 +01:00
Time	a636693019	feat: add GPU-aware model filtering and For You section- Add fit filt… (#6645) ... * feat: add GPU-aware model filtering and For You section- Add fit filter toggle (All / Fits GPU / Comfortable) to Hub discover tab- Add For You section showing only hardware-compatible models- Fix MoE active parameter extraction (Qwen3.5-35B-A3B now correctly reads as 3B active, not 35B)- Add gpu-fit-filter.ts with instant VRAM estimation from HF metadata without fetching model configs- Add fit badges to model cards and table rows- No backend changes- Closes #6556 * fix: handle unified memory systems in GPU fit classification * fix: tighten GPU model fit filtering --------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-25 14:52:22 +01:00
Daniel Han	8ca09b86dc	Studio: stop leaking the auth token through HTML canvas preview frames (#6634) ... * Studio: stop leaking the auth token through HTML canvas preview frames The artifact preview frame placed the Studio bearer token in the iframe URL (?token=) whenever canvas network access was enabled. Untrusted canvas HTML runs in that frame and can read its own window.location.href, and the network-mode CSP allows outbound http/https, so the token could be exfiltrated and replayed against authenticated Studio APIs. The auto-render HTML cards widened the reach: ordinary or prompt-injected assistant html fences become a Preview card that opens this same frame, and the render_html tool path auto-opens it without a click. Root cause: never put the token in the frame URL. The preview shell is a static document that only renders HTML posted to it by its embedder, and frame-ancestors plus the no-same-origin sandbox already constrain it, so the endpoint no longer accepts or validates the token and selects the network CSP from allow_network alone. No credential ever reaches the frame. Defense in depth: only tool-rendered canvases may opt into network mode; fences auto-extracted from assistant text never do. * Studio: stop strict canvas frames from self-upgrading to network mode Network mode is selected from the allow_network query param alone, so untrusted canvas code in a strict frame could navigate its own iframe to ?allow_network=1; the frame's onLoad handler then reposted the same untrusted HTML into the now network-enabled frame, giving a no-network or fenced canvas unauthorized network egress. Only inject the artifact for loads we initiated (mount or a src change), tracked by a pending flag set when src changes. A self-navigation also fires onLoad but is no longer fed, so the upgraded frame stays the inert shell. The strict CSP default-src 'none' already blocks the child-iframe variant. * Studio: trim comments in the canvas artifact security fix Condense the added explanatory comments and the artifact-preview-frame docstring to one line each while keeping the security rationale. No code change (verified comment-only).	2026-06-25 05:25:06 -07:00
Daniel Han	1cb04be328	Studio: keep the training event pump alive so progress can't silently freeze (#6643) ... * Studio: keep the training event pump alive so progress can't silently freeze The parent-side event pump is the only writer of the in-memory progress state that SSE /progress, /status, /metrics and the DB history all read. It ran in a single unsupervised daemon thread with no guard around event handling, so one malformed event or a transient queue/DB error would terminate it permanently. The worker subprocess keeps training regardless (mp.Queue puts never block on an unbounded queue), so a run kept burning GPU for hours while every progress surface froze on the last step the pump saw. - Guard each pump iteration: a bad event or queue-read error is logged and skipped instead of ending the loop. _read_queue now reads any error as "no event", not just Empty/EOFError/OSError/ValueError. - Add a _pump_running flag and an _ensure_pump_alive watchdog wired into is_training_active, so a pump that dies while the worker is alive is restarted on the next status poll and the UI catches up from the still-open queue. - Start respawned and restarted pumps under the lock so the watchdog can never spawn a duplicate during the brief start window. Adds tests/test_training_pump_resilience.py covering both guarantees. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio training pump: address review (drain guard, start race, read backoff, respawn flag) Follow-up to the event-pump resilience change, closing four edge cases a review surfaced in the same pump/queue surface: - _drain_queue now tolerates any error during the worker-exit drain and finalizes with whatever it drained, instead of skipping finalization and leaving the run wedged "active" with a dead worker. - start_training clears a stale _pump_running flag during reset and assigns the subprocess handles plus starts the pump under the lock, so a concurrent status/SSE poll can't spawn a duplicate pump during setup. - _read_queue goes back to the narrow EOFError/OSError/ValueError catch; truly unexpected errors are left to _pump_loop's guarded read, which logs and backs off so a persistently raising queue can't spin a hot loop. - The xet respawn-failure path clears _pump_running so a later run can't inherit a stale flag. Adds regression tests for all four. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: revive a crashed pump after worker exit + stop test module pollution Two review follow-ups on the training event pump: - _ensure_pump_alive refused to restart once the worker had exited (not self._proc.is_alive()), so a pump that crashed just before the worker finished never drained the terminal complete/error events still sitting in the queue. progress.is_training stayed True and is_training_active() returned True forever, leaving the run stuck "running" behind a dead pump. A True _pump_running flag with a dead thread is an unambiguous crash regardless of worker state, so restart there too: the fresh pump drains the backlog and finalizes. Updated the watchdog test to assert the revive-and-finalize. - The resilience test imports core.training.training while heavy module-level deps are stubbed, then restores the stubs -- but the cached training module kept the stubs bound in its globals, so a later test in the same session could exercise the fakes (e.g. prepare_gpu_selection) instead of the real code. Evict the training module (and its package) after import when this file created it, so subsequent tests re-import it cleanly. * Studio: finalize training run when queue reads keep failing on a dead worker reviewer.py follow-up. _read_queue only swallows EOFError/OSError/ValueError; an unexpected error escapes to the pump's outer guard, which logged, slept and `continue`d. If those reads keep raising after the worker has already exited (e.g. a broken queue pipe), the loop never reaches the dead-worker finalize block, so the pump spins on with _pump_running True and progress.is_training stuck True -- the run looks like it is still training forever. On a read failure now fall through to finalize when the worker is gone, only backing off and retrying while it is still alive. Mirrors the data-recipe pump fix; added a regression test. * Tighten training pump resilience comments and docstrings Condense the verbose explanatory comments and docstrings on the training event pump and its tests to shorter, clearer forms. Comment/whitespace only; verified no code changed via AST diff. No behaviour change. * Studio: create the training DB run before starting the event pump start_training started the event pump before the eager _ensure_db_run_created() call, so for a worker that completes or fails immediately the pump could race the main thread into creating and finalizing the same run row (duplicate INSERT, or a finalize skipped while _db_run_created was still false). Create the run first; the pump then only ever finalizes. Adds a regression test asserting the pump observes an already-created run. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-25 05:19:32 -07:00
Daniel Han	4929c5f769	Keep pad-named pad_tokens (e.g. <|vision_pad|>); fix Qwen3-Base load crash (#6652) ... * Keep pad-named pad_tokens; defer pad repair to shared unsloth_zoo.pad_token A pad-named token (e.g. <|vision_pad|>) is a valid pad. The narrow fallback that stripped vision pad tokens on text-only models is now a no-op; the active path delegates to the shared fix_pad_token in unsloth_zoo, which keeps pad-named tokens and only heals missing / eos-collision / out-of-range pads. This fixes the Qwen3-4B-Base load crash (its config ships pad_token=<|vision_pad|>): the old swap could not find a safe text pad (eos is <|endoftext|>, no unk_token) and left the tokenizer broken. Removes the unused _VISION_PAD_TOKENS / _SAFE_TEXT_PAD_TOKENS sets. Tests updated. Pairs with unslothai/unsloth-zoo#831. * Remove _fix_vision_pad_token; inline the no-op fallback A pad-named token (e.g. <|vision_pad|>) is a valid pad, so the old vision-pad swap helper has no purpose. _fix_pad_token now returns the tokenizer unchanged when the shared unsloth_zoo.pad_token module is unavailable, instead of routing through a no-op helper. Test WANTED set updated.	2026-06-25 04:41:09 -07:00
Daniel Han	09852ba18b	Studio: keep the live progress stream alive during pre-first-step preparation (#6665) ... * Studio: don't time out the live progress stream during pre-first-step prep The live progress SSE counts every 1s poll without a step update toward a 30-minute stall timeout, after which it emits an error event and ends the stream. But that counter also runs during the pre-first-step phase (model load + tokenizing the dataset), which is never reset because no step has happened yet. On a large dataset that prep can take well over 30 minutes, so the live view is torn down with an error while the run is perfectly healthy and still preparing -- the run then trains on in the background with the UI showing nothing, exactly the "no progress for hours" decoupling. Apply the stall timeout only once the stream has actually seen a live step. Before the first step the run is preparing and may legitimately emit no step for a long time; heartbeats still flow so the client stays connected and the worker's liveness still ends the loop when training finishes. A genuine post-step stall still times out. Extracted the threshold to a module constant so it can be tuned/tested. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: seed seen_live_step from the resume point on reconnect Review follow-up: seen_live_step reset to False on every SSE request, so a client reconnecting past the first step (Last-Event-ID set, or the run already has step history) only receives heartbeats and never flips it true. A worker that hangs after step N would then never trip the stall timeout for that reconnected client. Initialize it from resume_from_step / existing step history so reconnects keep the post-step timeout behavior, while a genuine pre-first-step run still stays exempt. Added a reconnect regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten prep-phase progress timeout comments Condense the verbose explanatory comments and docstring on the prep-phase stall timeout exemption to shorter, clearer forms. Comment/whitespace only; verified no code changed via AST diff. No behaviour change. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-25 04:39:44 -07:00
Daniel Han	54f25bf17e	Studio: UNSLOTH_NPM_REGISTRY opt-in for corporate npm mirrors (#6491) (#6663) ... * Studio: UNSLOTH_NPM_REGISTRY opt-in for corporate npm mirrors (#6491) studio/frontend/.npmrc pins registry=https://registry.npmjs.org/ as a supply-chain lock. A project-level pin takes precedence over a user's ~/.npmrc, so behind a corporate firewall that blocks npmjs.org the frontend bun/npm install hit npmjs.org directly and failed with 403. Add an opt-in UNSLOTH_NPM_REGISTRY env var (off by default). When set it is threaded as --registry into every registry-touching install in setup.sh, setup.ps1 and build.sh (bun bootstrap, bun install + retry, npm fallback, OXC validator runtime). --registry is the highest-precedence override for both bun and npm and leaves min-release-age and save-exact in force, so the default lock is unchanged for everyone else. On an install failure that looks like a blocked registry, print guidance pointing at UNSLOTH_NPM_REGISTRY and auto-suggest the mirror already set in the user's npm config. Registries are never switched automatically. Also correct the .npmrc comment: the pin does not block an ambient NPM_CONFIG_REGISTRY env var (npm and bun honor that at higher precedence); it only guards against a lower-precedence stale ~/.npmrc. * Studio: make the registry hint reachable under set -e; clean temp log (#6491) run_quiet_no_exit returns non-zero on failure, which under `set -euo pipefail` exits the script at the call site before the exit code is captured, so the new UNSLOTH_NPM_REGISTRY hint never printed on the npm fallback and OXC validator paths. Guard both with `|| _rc=$?` (the same idiom every other run_quiet_no_exit caller already uses) so the failure branch runs, and remove the _FRONTEND_INSTALL_LOG temp file on the early-exit path. * Studio: detect the user's mirror outside the pinned frontend dir (#6491) _suggest_npm_registry / Show-NpmRegistryHint run while the cwd is still studio/frontend, whose .npmrc pins registry=https://registry.npmjs.org/. So `npm config get registry` returned that pin instead of the user's ~/.npmrc mirror, and the "Detected a registry" branch was skipped for the main corporate case (mirror set in ~/.npmrc). Run the lookup from a directory with no project .npmrc (/ in bash, the temp dir in PowerShell) so the user/global mirror is surfaced. The NPM_CONFIG_REGISTRY env check is unchanged and still takes precedence.	2026-06-25 04:01:43 -07:00
Daniel Han	e1698e05c7	Studio: fix misleading "increase max_seq_length" message for train-on-completions (#6664) ... The post-filter safety net for 'Train on completions' fires when train_on_responses_only() masks every token in too many rows. Its trigger is a row-drop ratio, not a token-length check, but the message hardcoded "max_seq_length is too short, try increasing (e.g. 8192)" -- advice that fires identically at any max_seq_length and can recommend a value below the user's current setting (telling someone already at 16384 to use 8192). The dominant real cause is that the model's response template is not found in the formatted samples: the dataset is already formatted, or its structure doesn't match the model's chat template, so every token gets masked and the rows are dropped. Reword the error (and the comment above it) to lead with that cause and the actionable fix (turn off 'Train on completions'), and mention max_seq_length only as a secondary possibility without a hardcoded recommendation.	2026-06-25 03:30:12 -07:00
Daniel Han	c72da05741	Studio: clean up empty leftover quant folders so they can be deleted (#6616) ... * Studio: clean up empty leftover quant folders so they can be deleted An interrupted or cancelled split GGUF download leaves snapshots/<rev>/<quant>/ behind with no shards. Such a folder is neither a completed download nor a tracked partial (no .incomplete blobs, no manifest), so it was invisible in the variant list and a per-variant delete returned 404, leaving it on disk forever. - list_empty_gguf_variant_dirs: detect quant folders that are empty in every snapshot, excluding any quant that has shards in another snapshot. - get_gguf_variants_response: surface those quants as partial (cleanable) so the UI shows a delete affordance. - _delete_gguf_variant_from_repos: remove the empty (or just-emptied) quant subfolder and count it toward the result so the delete succeeds instead of 404. Adds hub/tests/test_empty_variant_folder.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: simplify empty-dir check to any(iterdir()) * Studio: tighten comments on empty-quant-folder cleanup * Studio: surface empty-folder removal failures and cleanables on local/offline paths Address review feedback on the empty leftover quant folder cleanup: - _remove_empty_variant_dirs now returns removal failures (read-only cache or a locked dir), and the variant delete raises 409 instead of a misleading 404; a concurrent download refilling the dir (ENOTEMPTY) is still treated as a skip. - Empty leftover folders are surfaced as cleanable on every variant-listing path (prefer_local_cache / offline / HF-fallback), not just a remote listing, via a single post-process that flips a listed quant to partial or appends an unlisted one. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: surface empty-folder cleanables even when metadata fetch fails When the cache holds only an empty leftover snapshots/<rev>/<quant>/ folder from an interrupted split download and the client is offline or the HF metadata request fails, _compute() re-raised before cleanables were marked, leaving the folder undeletable. Now fall back to marking cleanables against an empty response and return them if any; otherwise re-raise the original error. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-25 01:14:21 -07:00
Wasim Yousef Said	2aef1a23cb	Fix Linux AppImage packaging (#6657) ... * Fix Linux AppImage packaging stack * Fix desktop release workflow guard	2026-06-24 19:40:00 -07:00
Wasim Yousef Said	e25e7895a5	Polish Studio desktop chrome (#6332) ... * Polish Studio desktop chrome * Fix desktop chrome chat header overlap * Blend desktop titlebar with sidebar * Refine desktop chrome alignment * Fix desktop chrome review items * Reserve mac sidebar chrome space * Fix mac chrome review items * Polish macOS desktop chrome * Align macOS desktop chrome controls * Lower macOS traffic lights * Remove mac sidebar logo from chrome row * Match Tauri update banner styling * Update Tauri updater public key * Fix Tauri startup screen spacing * Work around AppImage WebKitGTK blank screen * Mark Linux AppImage as experimental * Address true desktop chrome review issues * Fix remaining desktop chrome review issues * Fix desktop titlebar inset review issues * Refresh desktop platform after backend auth	2026-06-24 17:56:25 -07:00
Daniel Han	f436d204f6	Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503) (#6639) ... * Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503) On Apple Silicon, install.sh exports UV_OVERRIDE pointing at the bundled overrides-darwin-arm64.txt. uv splits UV_OVERRIDE on whitespace, so a repo cloned under a path containing a space (e.g. /Users/me/Open Source/unsloth) truncates the value and every later uv call aborts with 'error: File not found: <truncated>' (the PyTorch install step in #6503). Copy the overrides file into a space-free temp dir and point uv at the copy when the path contains a space, mirroring the macOS/Linux handling already merged for the Python installer in #6534. The temp dir is removed in the exit trap, and the code falls back to the original path when no space-free temp dir is available, so the no-space and non-macOS paths are unchanged. Adds tests/sh/test_install_uv_override_space.sh, which extracts and runs the install.sh hardening block and checks the spaced, no-space, and spaced-TMPDIR fallback cases. * Installer: match all whitespace (not just spaces) in UV_OVERRIDE handling uv splits UV_OVERRIDE on any whitespace, so use the POSIX class *[[:space:]]* rather than a literal space in install.sh (catches tabs and newlines in the path too) and the matching test assertions. Use the portable awk bracket expression [$] instead of \$ in the extraction so the test runs the same under BSD awk (macOS) and GNU awk (Linux). Adds a tab-in-path case. * Installer: clear _UV_OVERRIDE_TMPDIR before the exit trap The exit trap rm -rf's _UV_OVERRIDE_TMPDIR. Initialize it to empty before registering the trap so an inherited environment value can never be removed; only a temp dir this script creates (Apple Silicon, spaced path) is cleaned. Adds a structural test asserting the init precedes the trap. * Run the install.sh UV_OVERRIDE space test in CI via a pytest wrapper The Shell installer tests job uses a fixed script list (not tests/run_all.sh), so the new shell test would not run on PRs. Add a pytest wrapper under tests/python/ that invokes it; the auto-discovered repo CPU test job collects tests/python/ and so executes the Apple Silicon spaced-path regression. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-24 17:34:18 -07:00
Long Yixing	a3954edd15	Fix Studio GGUF variant expansion crash (#6636) ... * fix: handle empty GGUF variants * fix: gate local GGUF expansion * fix: normalize GGUF variant payload --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-24 15:39:20 +01:00
oobabooga	ab6c9ecfee	Studio: honor stream=false on the GGUF agentic tool path (#6570) (#6618) ... * Studio: honor stream=false on the GGUF agentic tool path (#6570) * Studio: dedup the #6570 non-streaming tool tests and cover cached_tokens * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: cover the cached_tokens metadata fix and clarify the drain comment (#6570) * Studio: align the GGUF tool drain naming and tighten its comment (#6570) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>	2026-06-24 15:37:08 +01:00
Daniel Han	bd2438ea65	Verify DiffusionGemma visual-server binary against approved checksums (#6635) ... ensure_diffusion_visual_server() downloaded the visual-server release asset with the unverified download_file() and marked it executable, bypassing the approved-checksum manifest that gates every other prebuilt llama.cpp artifact. The backend later auto-discovers that binary and launches it through DG_VISUAL_BIN, so a compromised or substituted release asset could place attacker-controlled native code in the install tree and have it executed under the Studio user. Require the matched asset to be present in the approved checksum manifest and download it through download_file_verified() with the published sha256. A name-matching asset that is absent from the manifest is refused rather than executed. Add regression tests covering the verified-download path and the refusal of an unapproved asset.	2026-06-24 06:37:41 -07:00
Nilay	e5cf956601	Studio: shareable per-checkpoint preview links (#6486) ... * checkpoint preview endpoint * harden new preview endpoints * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * address review * Studio preview: pin adapter, guard streaming submit, robust copy-link Harden the public per-checkpoint preview surface: - Pin use_adapter=True in the preview payload sanitizer. Otherwise an unauthenticated /p caller can POST use_adapter=false, which calls disable_adapter_layers() on the shared in-memory model without restoring it; since load_model skips reloads for the same checkpoint, every later visitor (the page never sends the field) keeps getting base-model output instead of the fine-tuned checkpoint. Forcing it on also re-enables a previously disabled adapter and no-ops on merged checkpoints. - Ignore preview-page submits while a response is streaming. The send button was disabled but the Enter handler still called requestSubmit(), so a second request could start before the first reply landed in msgs and reorder the chat history. Both the keydown and submit handlers now honor the disabled button. - Keep the cloudflare-URL polling loop alive across transient startup fetch errors instead of letting one rejection halt it. - Build the copy-link from a backend preview_ref (output dir relative to outputs_root, gated on previewability and the two-segment /p route limit) so a nested output dir no longer copies a basename-only link that 404s. Expose preview_ref on training run summaries. Add route-level security tests (path traversal, payload sanitization, asset containment, CSP header, HTML title escaping, streaming lock held until drained) and preview_ref unit tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio preview: Safari-safe submit and adapter pin only for LoRA Follow-ups from cross-browser and route simulations: - Preview page: send the message from a shared send() helper called by both the form submit and the Enter key, instead of form.requestSubmit(). The latter throws on Safari < 16 and older iOS, which broke Enter-to-send there. Verified across Chromium, Firefox and WebKit with Playwright. - Only pin use_adapter=True when the resolved checkpoint is a LoRA adapter (adapter_config.json present); for a merged checkpoint strip it to None. A merged model has no adapter to toggle, so forcing it on only produced a per-request "not a PeftModel" warning. The cross-request base-model contamination fix still holds for LoRA previews. Add a merged-checkpoint test asserting use_adapter is stripped to None. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio preview: trim verbose comments Tighten comments across the preview routes, page, checkpoint helpers, and tests to short single-line notes; drop ones that just restate the code. No behavior change (verified comment/docstring-only with comment_tools.py check). * Harden preview routes for PR #6486 - Return a generic 400 detail on a rejected preview path so the public /p route never echoes the absolute install path (the real reason is logged server-side instead). - Strip confirm_tool_calls, session_id and rag_scope in the preview payload sanitizer so the public surface stays inert regardless of the tool gate. - Use Path.is_relative_to for the asset containment check, matching the rest of the codebase. - Add img-src 'self' and font-src 'self' to the preview page CSP. - Preview page: on a mid-stream error keep the streamed text, flag the break, and restore the prompt so the user can retry; drop the unused --font-sans var. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2026-06-24 06:31:53 -07:00
Daniel Han	c7c353d740	Pin isolated Node.js installer to committed sha256 digests (#6625) ... * Pin isolated Node.js installer to committed sha256 digests The isolated Node installer verified each downloaded archive only against SHASUMS256.txt fetched from the same nodejs.org origin as the archive, so a compromised CDN or TLS path could serve a malicious archive plus a matching checksum and gain code execution when the extracted node is run during the npm floor check and version probe. Anchor trust in studio/node_prebuilt_pins.json, a committed manifest of per-arch sha256 digests, and verify archives against it. The default channel installs the pinned version and never fetches the remote SHASUMS. Unpinned lts, latest, or explicit versions fail closed via UnpinnedNodeRefused unless UNSLOTH_NODE_ALLOW_UNVERIFIED=1, and the refusal is not swallowed by the keep-existing-on-transient-failure path. Ship the manifest in package-data. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review nits on the pinned Node installer - Drop the unused npm_min_major field from node_prebuilt_pins.json; the floor is the NPM_MIN_MAJOR module constant and the dead field could silently drift. - Reword the unpinned-refusal message so it does not tell a user already on the default to install it, and point the "add a pin" hint at the exact asset. - Decode the opt-in SHASUMS body with errors="replace" so a non-UTF8 response yields a clean PrebuiltFallback instead of an uncaught UnicodeDecodeError. - Tests: assert the refusal message (guards the main() catch order, not just the exit code), cover malformed-manifest parsing, and drive the opt-in remote-SHASUMS path end to end through install_prebuilt. * Tighten comments in the pinned Node installer Collapse multi-line rationale comments to single lines, drop docstrings on the obvious internal helpers (load_pins, pinned_sha256), and shorten the manifest note. Comments/docstrings only; verified code-unchanged via AST comparison. * Address Codex review: verify pins on existing installs; tomllib fallback - existing_install_matches now takes an expected_sha and the short-circuit passes the committed pin, so a version-matching but non-pinned or tampered install (e.g. from the old remote-SHASUMS path) is re-verified instead of kept. An unpinned target without opt-in no longer short-circuits on an existing install; it falls through to the UnpinnedNodeRefused fail-closed path. - The package-data test uses pytest.importorskip(tomllib/tomli) so it does not ModuleNotFoundError on the supported 3.9/3.10 interpreters. * Make the transient-failure keep-existing path pin-aware The previous commit added the pinned-digest check to the existing-install short-circuit but not to the post-download-failure fallback, which still kept any runnable same-version install via existing_install_usable(). A same-version install whose recorded sha256 is not the pin could therefore be kept on a transient download failure, the exact artifact the short-circuit rejects. Refuse to keep a same-version pin-mismatched install there too; a different usable version is still kept for offline resilience. * Bump pinned default Node to the current 24 LTS (24.18.0) Node 24 LTS moved to 24.18.0; since the default channel now resolves straight to the manifest, a frozen 24.17.0 would downgrade fresh installs and make UNSLOTH_NODE_VERSION=lts refuse the current LTS as unpinned. Update default_version and all six per-arch digests (verified against the official SHASUMS256.txt), and point the test INDEX/short-circuit fixtures at the new LTS. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-06-24 05:47:58 -07:00
Michael Han	8750d86cf7	Studio: keep the sidebar bottom fade in sync when groups collapse (#6640) ... The fade is recomputed on a group's open/close state flip, but the groups animate their height, so it measured scrollHeight mid-animation and the fade could vanish at random. Re-measure on the collapsible animationend, and add the missing pinnedOpen dep to the recompute effect.	2026-06-24 05:46:49 -07:00
Michael Han	61eb9eabd0	Studio: tighten the sidebar footer spacing and update-card fade (#6641) ... Footer padding moves from pt-3 pb-4 to pb-3 with a conditional top: pt-1.5 when the update card is shown so the fade hugs it, pt-2.5 for the profile on its own. When the update card is shown, shorten the fade above it (h-10 -> h-3) so the list reads closer to the card.	2026-06-24 05:46:39 -07:00
Michael Han	0e2f9ce0b6	Studio: group the project export menu by Combined and Per chat (#6637) ... * Studio: group the project export menu by Combined and Per chat Replace the repeated (combined)/(per chat) suffix on every export row with a section subheading, so the rows read Raw JSONL / CSV / ShareGPT JSONL under Combined and Per chat headings. * Studio: group export sections with DropdownMenuGroup Wrap each Combined and Per chat section in DropdownMenuGroup for screen-reader semantics, and drop the redundant px-3 already set by DropdownMenuLabel.	2026-06-24 05:08:15 -07:00