DiffusionGemma serves via the visual runner, which streams per-step
canvas frames so the answer resolves live in the bubble. The agentic
tool loop (generate_chat_completion_with_tools) does not forward those
frames, so whenever a tool pill (Search/Code) was on the live canvas
silently vanished while text still streamed. DiffusionGemma is not a
tool-calling target anyway, so report supports_tools=False for it: the
chat always takes the frame-forwarding path, and the Search/Code pills
disable themselves (a local model has no builtin web search either).
Also turn the artifacts canvas on by default for DiffusionGemma so a
full-HTML answer (e.g. a playable game) renders as an interactive
sandboxed card without the user flipping the global artifacts toggle.
* Studio: use a clearer check mark for the app-wide tick
Route every check mark through a shared tick-icon module with a plain
tick geometry, sized slightly larger than the stock icon so it reads
clearly in menus. No Pro icon assets are vendored.
* Studio: even out plus-menu hover padding on all sides
Trim the plus-menu side gutter to 9px so it matches the 0.5rem top/bottom
padding, and drop the container radius to 21px to keep corners concentric.
* Studio: bake the tick fallback stroke at 1.5 to match the icon set
---------
Co-authored-by: shimmyshimmer <info@unsloth.ai>
* Studio: make project sources work with RAG and polish project UI
Projects had a disabled Sources tab with an Add sources placeholder.
This wires it up end to end on top of the RAG engine:
- Add a project scope to the RAG store, ingestion and retrieval
- New endpoints: POST/GET /api/rag/projects/{id}/documents
- search_knowledge_base resolves kb, project and thread scopes; an
explicit KB stays exclusive, project and thread scopes combine
- Multi-scope search: FTS uses scope IN (...), vec0 KNN runs per
scope and merges by cosine score
- Lazy ALTER TABLE adds documents.project_id on existing databases
- Deleting a project also removes its indexed sources
- Sources tab now uploads with progress chips and drag and drop
- Chats inside a project auto-enable retrieval over project sources
when the project has indexed documents (cached probe, no Docs pill
needed); external providers still never receive rag_scope
UI polish:
- Rounder project cards with folder icon chip and softer shadow
- Project header icon in a rounded chip
- Chats/Sources pills and Add sources button without borders
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: match Add sources button shadow to the chat composer in light mode
* Studio: round project switcher hover pill and pad the folder icon
* Studio: remove border from project sources box
* Studio: grey hover on project cards and menu, move search into header, widen page spacing
* Studio: shorten sources copy, white header pills with composer shadow, fixed-width search, hub-size page headings
* Studio: align project landing blocks to the composer width
* Studio: restore muted background and flat look on projects header controls
* Studio: darker grey hover on project cards in light mode
* Studio: soften project card hover grey
* Studio: keep project card menu button visible while its menu is open
* Studio: drop focus outlines and rings on buttons and clickable icons, keep input focus styles
* Studio: address review feedback on project sources
- Remove uploaded files from disk when a project is deleted, confined
to the uploads root
- 404 project uploads when the project does not exist, matching the KB
endpoint
- Guard lexical search against an empty scope list
- Re-invalidate the project sources probe after uploads and removals
settle so a chat sent mid-upload cannot cache a stale negative
- Keep keyboard focus rings: only mouse focus drops the Tailwind ring,
the browser default outline stays removed
* Studio: add a green New badge to the project Sources tab
* Studio: unify New pills, fully round with soft emerald fill and no border
* Studio: a touch more vertical padding on New pills
* Fix project RAG source edge cases for PR #6205
* Fix duplicate RAG upload cleanup for PR #6205
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
The prebuilt bundles ship llama-diffusion-gemma-visual-server, but
runtime_patterns_for_choice pruned it, so a fresh install never placed
it next to llama-server. ensure_diffusion_visual_server then found no
standalone release asset and skipped it, leaving Studio unable to serve
DiffusionGemma GGUFs natively (it required DG_VISUAL_BIN or a source
build). Keep the binary in the runtime allowlist on Linux, macOS and
Windows so it lands in build/bin and is activated automatically.
* Studio: cache MCP tool discovery instead of re-probing every chat send
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Stop re-probing offline/down MCP servers every time
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add tests for mid-probe delete and OAuth cool-off paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Don't cool-off a server edited or deleted mid-probe
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard MCP refresh cache writes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* feat(studio): add S3 dataset configuration foundation (#4539)
Add foundational types and configuration for S3 bucket dataset loading:
- Add S3Config type to frontend training types
- Add S3Config Pydantic model to backend training models
- Add "s3" as a DatasetSource option
- Add s3Config state and setS3Config action to training config store
- Add i18n translations for S3 configuration (English and Chinese)
This provides the type definitions and UI text for S3 integration.
Full implementation requires boto3 dependency and data loading logic.
Refs: #4539
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire S3 config into training pipeline and prevent secrets persistence
- Pass s3_config from request into training_kwargs so it flows to training subprocess
- Add s3Config to NON_PERSISTED_STATE_KEYS to prevent AWS secrets from being
saved to localStorage
Addresses code review feedback on PR #5951.
* Exclude S3 config from database persistence to protect secrets
Filter out s3_config (which contains secret_access_key) from the
config_json stored in training_runs table, preventing AWS credentials
from being persisted to disk.
Addresses P1 security feedback on PR #5951.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Re-raise HTTPException in start_training and defer s3 DatasetSource widening for PR #5951
* Redact s3_config from W&B run config and accept camelCase S3 credential aliases for PR #5951
* feat(studio): implement S3 dataset loading end-to-end
Builds the actual S3 loader on top of the hardened #5951 foundation,
turning the 501-gated scaffold into a working dataset source.
Backend:
- Add core/training/s3_dataset.py: lists and downloads supported dataset
files (parquet/json/jsonl/csv) from an S3 bucket to a temp dir, using
IAM-role or access-key credentials. boto3 is imported lazily (optional dep).
- Wire s3_config into UnslothTrainer.load_and_format_dataset (downloads then
reuses the existing local-file path) and thread it through worker.py.
- Replace the 501 "not implemented" gate with a boto3-availability guard so
S3 works when boto3 is present and fails clearly when it is not.
- Add boto3 to studio.txt requirements.
- Add tests/test_s3_dataset.py (8 tests) covering download/filtering,
collisions, missing-boto3, and S3Config camelCase/IAM validation.
Frontend:
- Widen DatasetSource to include "s3"; add s3_config to the training payload
type and mapper; add an S3 validation branch and selectS3Source store action.
- Add s3-config-form.tsx (bucket/region/prefix/keys/IAM toggle) reusing the
existing studio.dataset.s3.* i18n strings.
- Add a Hugging Face / Local / Amazon S3 source toggle in dataset-section;
the S3 config card replaces the dataset combobox when S3 is selected.
- Fix DatasetPreviewDialog to accept the widened DatasetSource type.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix S3 dataset loader for PR #6222
* Fix S3 dataset edge cases for PR #6222
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix S3 IAM payload handling for PR #6222
* Block multimodal S3 datasets for PR #6222
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Ash <ash@MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: override chat template for unsloth/gemma-4-*-GGUF with bundled gemma-4.jinja
The chat templates baked into the shipped unsloth/gemma-4-*-GGUF quants predate
Google's gemma-4 chat-template PR #118 and lack the preserve_thinking flag, so
Studio cannot surface the "Preserve thinking" toggle for Gemma 4. Bundle the updated
template and override the embedded one at llama-server launch via --chat-template-file,
scoped to the gemma-4 GGUF family, so users do not need to re-download any quant.
- Add studio/backend/assets/chat_templates/gemma-4.jinja (PR #118 based;
preserve_thinking defaults false, the one deliberate divergence from upstream).
- Add core/inference/chat_templates.py: gemma-4 GGUF matcher plus an
effective-override resolver (explicit user template still wins).
- Wire the resolver into routes/inference.py ahead of the reload-dedup check and
both load_model calls so the live backend and the incoming request compare against
the same template text (no spurious reloads).
- Default preserve_thinking off in the launch-time chat_template_kwargs so direct
API callers match the UI default.
- Ship the asset via package-data and add unit tests.
* Studio: ship E2B/E4B edge variant of the bundled Gemma 4 template
Google ships two distinct gemma-4 chat templates: E2B and E4B omit the empty
"<|channel>thought<channel|>" block on enable_thinking=false, while the
12b/26B-A4B/31B family emits it (confirmed against google/gemma-4-E2B-it,
-E4B-it, -12b-it, -26B-A4B-it, -31B-it; the two families differ only in that
one block). The single PR #118 based template followed the larger-model
behavior, which is wrong for the E2B/E4B GGUFs this feature most targets.
- Add studio/backend/assets/chat_templates/gemma-4-edge.jinja: identical to
gemma-4.jinja minus the empty-thought-block, matching E2B/E4B behavior.
- Route unsloth/gemma-4-E2B-it-GGUF and -E4B-it-GGUF to the edge template;
12b/26B-A4B/31B keep gemma-4.jinja.
- Extend tests for the edge matcher, per-family routing, and the empty-thought
block difference (off for edge, on for standard).
* Studio: address review feedback on the gemma-4 template override
- Normalize owner-less shorthand model ids in the template matcher: a bare
"gemma-4-E2B-it-GGUF" is canonicalized to "unsloth/" the same way
ModelConfig.from_identifier does, so shorthand loads still get the override
(and the preserve_thinking capability) instead of falling back to the
embedded template.
- Scope the test's module stubs with unittest.mock.patch.dict instead of
sys.modules.setdefault, and only stub deps that are missing, so the global
module registry is not polluted for tests that run afterwards.
- Guard the Jinja render tests with pytest.importorskip("jinja2") so the suite
stays runnable in minimal Studio environments where jinja2 is not present.
- Add tests for shorthand resolution.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address 10-reviewer P1 findings on the gemma-4 template override
- /status no longer surfaces Studio's auto-applied bundled template as a
user-authored chat_template_override. The frontend adopts that field as
editable state and would otherwise re-send the gemma-4 template as an explicit
override for a later, unrelated model. /status now reports None when the live
override equals the model's auto-resolved bundled template.
- When a bundled family template is in effect, strip an inherited
--chat-template-file from llama_extra_args too (not only when the raw request
set chat_template_override). Otherwise a stale inherited template, appended
last, shadows the bundled one while Studio reports the bundled template's
capabilities.
- Write the temp chat-template file as UTF-8 explicitly, and keep the bundled
templates ASCII (replaced em dashes), so non-UTF-8 Windows locales cannot raise
UnicodeEncodeError or emit a mis-encoded template. Added an ASCII guard test.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: serve DiffusionGemma GGUFs with the on-device visual decoder
* Studio: render the DiffusionGemma denoising canvas live in chat with honest stats
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden DiffusionGemma runner resolution (Windows .exe, build/bin lookup, clear stale audio flag, safe PYTHONPATH, Linux-only pdeathsig)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: refine menu chevron, tick icon, and one-line plus-menu shape
- Shrink submenu > chevron to size-3.5 in the dropdown, context, and
menubar sub-triggers so it stays centered but reads lighter.
- Use the tick-02 duotone-standard geometry for the app-wide check mark
via a shared tick-icon module.
- Drop the fully rounded pill on one-line plus-menus so they match the
standard menu shape.
* Studio: nudge submenu chevron down to 13px
* Studio: nudge submenu chevron down to 12px
* Studio: drop vendored Pro tick geometry, keep free check icon
The tick-02 duotone-standard path is a Hugeicons Pro asset and is not part
of the MIT free pack, so vendoring it into the source is a licensing risk.
Revert the check mark to the bundled free Tick02Icon and remove the shared
tick-icon module. Keeps the chevron sizing and one-line plus-menu changes.
---------
Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* Studio: persist speculative decoding preference across restart and model switch
* Studio: persist speculative preference on apply, not on edit
* Fix/adjust speculative decoding persistence for PR #6169
* Fix speculative ngram alias for PR #6169
* Fix compare speculative preference for PR #6169
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* new-chat shortcut, composer draft autosave, archive threads
* fix
* Studio: harden chat UX additions for legacy threads and unavailable storage
Two robustness fixes on top of the new chat UX features:
- groupThreads: coerce archived to a boolean before comparing
(Boolean(t.archived) !== archived). Threads from the older browser-only
Studio, or any record predating the archived field, can carry
archived === undefined/null. The raw `!== archived` comparison dropped
those from BOTH the Recents and Archived sidebar groups, hiding existing
chats. Treat missing as not-archived so legacy chats still appear in
Recents.
- composer draft autosave: wrap the localStorage read and write in
try/catch. When storage is unavailable (private mode, disabled cookies,
blocked storage) or full (quota exceeded), getItem/setItem throw; the
throw in the restore effect would surface to React and break the chat
page. Draft persistence is best-effort, so degrade quietly.
Verified with bun unit tests on the real groupThreads (legacy undefined no
longer vanishes) and Playwright across chromium, firefox and webkit (draft
save/restore/isolation/clear, new-chat shortcut + crypto.randomUUID, and
localStorage blocked/quota throw handling). tsc clean; no new eslint findings.
* Fix composer draft bleed and orphan cleanup for PR #5771
Centralize composer draft storage in a small util and tighten two edge cases:
- New chat draft bleed: every new chat shared the chat-draft:__new__ slot, so
starting a fresh chat could restore the previous one's half-typed text. Clear
that slot at every new chat entry point (sidebar buttons and Cmd/Ctrl+Shift+O).
- Orphan drafts: deleting a thread left its chat-draft:<id> key behind. Clear
the draft for every deleted thread id.
New util utils/composer-draft.ts owns the key format and wraps localStorage in
try/catch (private mode, blocked storage, quota), replacing the inline copy in
thread.tsx so reads and writes stay best effort everywhere.
* address review
* fix: remove unused chat sidebar binding
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: fix Downloaded model list disappearing and order it by last download
The chat model picker scan for cached GGUF and safetensors models aborted
whenever an auxiliary Hugging Face cache dir (such as ~/.cache/huggingface/hub)
was unreadable, returning an empty list. That hid the Downloaded section and
let already downloaded models appear under Recommended. Isolate each cache
probe so an inaccessible directory is skipped instead of failing the scan.
Also order Downloaded newest-first using cached blob mtimes (multi-quant repos
group by their most recent quant), keep the section visible while searching,
and make the per-quant downloaded check per-snapshot and mmproj aware so a
Recommended quant is never falsely marked downloaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden gguf-variants scan and dedupe by newest timestamp
Guard f.stat() per file so a broken symlink or unreadable file in a
snapshot no longer aborts the downloaded check early, and match quant
labels case-insensitively. When the same repo is present in multiple
caches with equal size, keep the newest last_modified so Downloaded
ordering reflects the most recent copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: apply cache-scan guards to sibling endpoints found in review
Extend the inaccessible-cache guard and mmproj/stat hardening to the
parallel HF cache code paths flagged in review:
- list_local_models and the Hub inventory scan now skip an unreadable
auxiliary cache instead of returning 500.
- The GGUF download-progress endpoint excludes mmproj adapters and
guards f.stat() so one bad file does not zero a repo's progress.
- The offline snapshot scanner guards its is_dir() probes.
- The chat-only picker no longer renders a blank list when a search
matches only cached non-GGUF models.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Swaps the arbitrary px-[10px] on the Hub option menu surface for the
standard px-2.5 utility. Both resolve to 10px, so this is a no-op
visually and just keeps to the standard spacing scale.
Rounds out the Hub chrome and aligns a handful of icons with the rest
of the app. The header stat pills, HTTP/Xet toggle, HF token button,
info chips, status badges and the single-row download card are now full
pills, and the All formats style dropdown matches the composer + menu
geometry. Also widens the chat options menu so "Move to project" fits
on one line, and swaps a few icons to their HugeIcons equivalents.
Changes:
- hub.css: hub-tag-soft/hub-tag-meta pills go fully rounded; a single-
row hub-download-card reads as a full pill via :has(), while the
expanded card keeps its 22px radius, border and background.
- hub-option-menu.tsx: dropdown surface and items match the composer
+ menu (22px container, 10px side padding, 12px item radius).
- hf-token-indicator.tsx, transport-toggle.tsx: fully rounded with a
fixed 26px height so they line up with the neighbouring pills.
- model-inspector.tsx, dot-tag.tsx, gguf-download-card.tsx: status and
info chips go fully rounded.
- models-header.tsx: Eject uses RemoveCircleIcon; active model pill
fully rounded.
- on-device-folders-dialog.tsx: folder rows use Folder02Icon.
- pickers.tsx: Recommended star uses the HugeIcons star.
- command.tsx, chat-search-dialog.tsx, prompt-storage-dialog.tsx: search
fields use the HugeIcons Search01Icon.
- app-sidebar.tsx: chat options menu widened to w-56.
Pure styling and icon swaps, no behavior changes.
* install.sh: persist ROCm-on-WSL drop-in even when rocminfo already works
_maybe_bootstrap_rocm_wsl calls _ensure_rocm_probe_env (which exports a
transient HSA_ENABLE_DXG_DETECTION + adds /opt/rocm/bin to PATH on the
installer process) right before the "rocminfo enumerates gfx1151 -> already
set up, return early" gate. On any reinstall over an existing /opt/rocm --
the common case, since the uninstaller keeps shared ROCm userspace but
removes /etc/profile.d/unsloth-rocm-wsl.sh -- that probe env makes rocminfo
succeed, so the gate returns 0 WITHOUT ever persisting the drop-in. The
transient env dies with the installer, so the next login shell (Studio,
llama-server) sees no GPU: torch cuda_avail=False, rocminfo finds nothing,
the llama.cpp ROCm prebuilt segfaults on a GPU it can't reach.
Factor the drop-in writer into _persist_rocm_wsl_dropin() and call it before
the early return so the persistent env is restored whenever librocdxg is
present. Idempotent (only writes when the drop-in is missing), gated on
librocdxg so it never fires on non-WSL/non-ROCDXG hosts, root-writes or
sudo-tees like before. The fast-path branch now reuses the same helper.
Reproduced on gfx1151 (Radeon 8060S) under dash (the curl|sh shell):
before the fix a reinstall left the drop-in absent and torch cuda_avail
False; after, the drop-in is persisted and a fresh login shell reports
cuda_avail True. Verified under both dash and bash, and idempotent on
re-run.
* Studio WSL: load system HIP before a prebuilt's bundled runtime (gfx1151)
The lemonade / published llama.cpp ROCm prebuilts bundle their own HIP
runtime (libamdhip64) built for bare-metal Linux. In WSL the GPU is reached
through the system ROCm's librocdxg bridge over /dev/dxg, which the bundled
runtime cannot drive -- it segfaults on the first GPU call. So:
- install_llama_prebuilt.py: the prebuilt's llama-quantize/llama-server
validation runs with the bundle dir first on LD_LIBRARY_PATH, segfaults
(empty stderr), and the install silently falls back to a CPU source build
(which on this host can't even build for GPU -- hipcc absent). The Strix
Halo WSL user ends up on CPU despite a working GPU.
- llama_cpp.py: even if a GPU prebuilt were kept, the serve-time launcher
put the bundle dir first too, so it would crash at load.
Fix: on a ROCDXG WSL host (gated on /dev/dxg + "microsoft" /proc/version +
a librocdxg-providing /opt/rocm), prepend the system ROCm lib dir to
LD_LIBRARY_PATH so the WSL-capable libamdhip64 + librocdxg load first, while
the bundle still supplies libggml-hip / librocblas with the gfx1151 kernels.
Set HSA_ENABLE_DXG_DETECTION=1 alongside. Added _wsl_system_rocm_lib_dirs()
to both modules (kept identical so a prebuilt that passed install validation
runs the same way at serve time). Strict no-op on bare-metal Linux, NVIDIA,
macOS, and Windows.
Verified on gfx1151 (Radeon 8060S) in WSL (ROCm 7.2.1 + librocdxg, Adrenalin
ROCDXG): before, the lemonade gfx1151 prebuilt segfaulted and the install
fell back to a broken CPU build; after, install_llama_prebuilt validates and
keeps the GPU prebuilt (source=published, prebuilt_fallback_used=False), and
Studio serves Qwen3-1.7B-GGUF at 53 tok/s with the model resident in GPU
memory (llama-server device_info: ROCm0 = AMD Radeon 8060S).
* tests: cover the WSL ROCDXG drop-in + system-HIP-ordering fixes
- _wsl_system_rocm_lib_dirs: no-op without /dev/dxg, on bare-metal Linux,
and on WSL without librocdxg; returns the system lib dir on a ROCDXG WSL
host.
- binary_env: prepends the system ROCm lib dir ahead of the bundle and sets
HSA_ENABLE_DXG_DETECTION on WSL; unchanged on bare-metal Linux.
- install.sh: _persist_rocm_wsl_dropin exists, is gated on librocdxg, and the
rocminfo-already-works early return calls it before returning.
- llama_cpp.py: the serve-time launcher prepends the WSL rocm dirs before the
bundle dir (mirrors binary_env).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten WSL ROCDXG fix comments (no logic change)
Condense the drop-in / system-HIP-ordering comments and docstrings added in
this PR. Verified comment-only via AST parse + py_compile + sh/bash -n, the
308-test rocm_support suite, and a dash functional re-run of the bootstrap
(drop-in still persisted, env still set).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio/responses): forward chat_template_kwargs enable_thinking to chat request
The /v1/responses translation in _build_chat_request dropped
chat_template_kwargs (e.g. {"enable_thinking": true}) sent via the
Responses extra-body, so reasoning control was silently ignored.
Lift enable_thinking onto the typed ChatCompletionRequest field,
mirroring openai_chat_completions, so both the non-streaming and
streaming Responses pass-through paths honor it.
Fixes#6198
Signed-off-by: Tai An <antai12232931@outlook.com>
* Fix/adjust Responses reasoning for PR #6202
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust reasoning none for PR #6202
* Fix/adjust structured reasoning for PR #6202
* Fix/adjust responses reasoning review findings for PR #6202
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust responses reasoning follow-ups for PR #6202
* Fix/adjust think parsing gate for PR #6202
---------
Signed-off-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Web "New Unsloth version" banner now matches the llama.cpp banner design
(borderless rounded card, drop shadow, heading title) and moves to the
bottom-right, stacked with the llama.cpp banner in a shared container. Keeps
the Release notes link, and the copy command is platform aware (curl for
macOS/Linux/WSL, irm for Windows).
The web banner is checked once per launch only, with no polling; a re-check
just happens the next time the user opens the app. Copying the install
command no longer dismisses it for good: it hides and returns on the next
launch while the install is still behind. The X button still dismisses the
version permanently.
llama.cpp progress bar: poll the job every 500ms and smooth the displayed
value so it animates with the real download instead of snapping to the ~90%
post-download hold and finishing.
Adjustable items (Chat with Files, MCP, Saved prompts, Compare chat,
Export chat, Canvas, Projects) can be pinned to the top level of the
composer plus menu from Settings -> Chat. Unpinned items move into the
More submenu, which hides itself when empty. Saved prompts can be
pinned individually so they surface in the Saved prompts submenu.
Rounder corners and more padding on the login card, a submit button
that sizes to its label, and the sidebar chat and run rows nudged 2px
left with the section headers brought flush to the row text so the
labels line up like Gemini's sidebar.
Removes the empty band under Eject loaded model, centers the trigger
chevrons, moves the model reload Apply and Reset buttons above Chat
Template so they sit under the settings they apply, matches the
settings nav divider to the sidebar line, and bumps the settings tab
headings slightly.
* Studio: Add Tensor-Parallel llama.cpp support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Tensor-Parallel fallback and GPU selection
* Studio: reconcile split-mode extras and harden tensor-split planning
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reconcile split-mode extras in backend duplicate-load guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: preserve inherited non-tensor split modes on reload
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor cancellation in tensor fallback, preserve tensor mode on rollback, and don't raise an explicit small context
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reconcile split-mode in reload check and strip it on tensor downgrade
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip --tensor-split alongside --split-mode so inherited ratios don't override the tensor planner
An inherited or stale --tensor-split in llama_extra_args was appended after
Studio's computed --tensor-split and won last in llama.cpp, re-introducing the
asymmetric-GPU OOM tensor mode is meant to prevent. Group -ts/--tensor-split
into the split-mode shadow set so it is stripped on inherit and on the layer
fallback; parse_split_mode_override still keys on the mode value only.
* Drop quantized KV for the tensor attempt and report native max context
Tensor mode aborts on a quantized KV cache, so a user with q8_0/q4_1 etc. who
enabled Tensor Parallelism silently fell back to layer split. Clear the cache
type (and strip inherited/explicit --cache-type) for the tensor attempt only;
the layer fallback re-runs with tensor off and keeps the user's choice.
Also report max_available_ctx from the native context, not an explicit small
-c, so the context slider no longer warns too early in tensor mode.
* Reconcile inherited split-mode extras in the already-loaded check
When a same-model load omitted llama_extra_args, the tensor comparison resolved
the raw (None) request and treated an inherited --split-mode tensor server as a
mismatch, forcing a needless reload. Compare using the stored extras stripped
the same way the reload strips them.
* Pass tensor_parallel through compare-mode loads
The generalized compare path loaded each GGUF without tensor_parallel, so
compare ran layer split even with the toggle on and left the settings sheet
stale. Send the toggle and hydrate the loaded state from the response, matching
the main chat and recipe load paths.
* Add --tensor-parallel flag to unsloth studio run
The headless one-liner could only reach tensor mode by passing --split-mode
tensor as a raw llama.cpp extra. Add a first-class --tensor-parallel/
--no-tensor-parallel option that sets the tensor_parallel field on the
/api/inference/load payload, forwarded through the studio-venv re-exec like the
other polarity flags. Matches the web UI toggle and the API field.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* chore(studio/frontend): normalize line endings to LF
45 source files under studio/frontend/ were committed with CRLF or mixed
line endings while the rest of the repo and the JS/TS tooling assume LF.
Add a scoped `studio/frontend/** text=auto eol=lf` rule to .gitattributes
and run `git add --renormalize studio/frontend` so these files are stored
with LF in the index. The rule is scoped to the frontend tree (not a
repo-wide *.ts/*.tsx/... policy) so it cannot force LF on files elsewhere;
text=auto leaves binary assets (logos, fonts) untouched.
This commit is whitespace-only (CRLF -> LF) — no source content changed
(verified with `git diff --ignore-cr-at-eol`). It is intentionally
isolated so it can be listed in .git-blame-ignore-revs and skipped by
reviewers and `git blame`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: ignore the frontend LF-normalization commit in git blame
Add .git-blame-ignore-revs listing the whitespace-only line-ending
normalization commit so it doesn't pollute `git blame` output. GitHub
applies this file automatically; locally run
`git config blame.ignoreRevsFile .git-blame-ignore-revs`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Studio: use rounded rectangles for menu item hover states instead of pills
Dropdown, select, and model picker items previously used fully rounded
pill highlights. Switch them to an 11px rounded rectangle so hover and
selected states match across the plus menu, profile menu, run settings,
selects, and the model picker. Also add a small side gutter to the plus
menu so item highlights sit slightly inset from the menu edge.
* Studio: concentric menu corners, wider gutters, single-item pill menus
Container radius now equals the item hover radius plus the side gutter
(12px + 10px = 22px) so the curves run parallel. Menus with a single
item render as fully rounded pills. The profile menu gets the same
gutter and hover radius. Model picker rows go back to their original
fully rounded hover.
The --local / GGUF-only install resolves its Python deps from
no-torch-runtime.txt, installed with --no-deps. That file was missing the
RAG group that studio.txt declares (sqlite-vec, pymupdf, python-docx), so a
fresh `unsloth studio` came up with RAG disabled: rag_db.py cannot import
sqlite_vec and logs "RAG unavailable: sqlite-vec extension could not be
loaded", and the knowledge-base routes return 503. python-docx was also
absent, so DOCX ingestion failed.
Add the three RAG store and document-parsing deps with the same pins as
studio.txt so knowledge bases work out of the box on the no-torch path.
sentence-transformers (dense embeddings) was already present.
The post-install path cleared only the in-memory freshness caches and then
re-primed the 24h disk cache with a forced GitHub refresh. When that refresh
cannot reach GitHub, latest_published_release falls back to the last-good disk
value, so a still-fresh same-base mix tag cached before the swap (b9596-mix-aaa
vs the just-installed b9596-mix-bbb) is replayed and the prebuilt reads as
behind, surfacing a false update banner that points back at the build that was
just replaced.
Give reset_caches a drop_disk option and use it on the update path: with the
disk cache gone, an offline post-install refresh leaves latest as None and the
banner fails open (off) instead of lingering on the stale same-base value. The
no-arg form stays in-memory only. Adds regression coverage for the drop, the
default no-op, and the fail-open vs stale-replay contrast.
* Studio: tune llama.cpp env for data-center GPUs
Detect datacenter/professional NVIDIA GPUs at llama-server launch and set
the llama.cpp env flags that help them, gated so consumer GeForce, AMD/ROCm,
CPU and macOS are never touched.
- GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F=1 for any DC GPU (FP32 cuBLAS
accumulation). On a B200 this is ~0% throughput cost with identical
perplexity (7.3230 wikitext-2-raw, baseline and on), where on GeForce the
same flag costs real throughput, hence the gate.
- GGML_CUDA_P2P=1 and CUDA_SCALE_LAUNCH_QUEUES=4x for multi-GPU DC boxes.
Benchmarked on 6x B200: +33-51% prompt processing on tensor (row) split and
+8-16% on the default pipeline (layer) split, with no regression on the
other split or on token generation.
Detection uses torch device names (A100/A30/H100/H200/H800/GH200/B200/GB200/
GB300/L40/L4/RTX PRO 6000/RTX 6000 Ada). A mixed box with one consumer GPU in
the selection is treated as non-DC. All writes are setdefault so a user value
always wins, and UNSLOTH_DISABLE_DC_TUNING=1 turns the whole thing off.
37 unit tests cover detection, multi-GPU gating, user-override precedence, the
disable flag and fail-open on error.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix data-center GPU detection false positives and physical-id mapping
Two issues in the data-center llama.cpp env tuning gate:
- _is_datacenter_gpu matched the marker allowlist as unbounded substrings, so
workstation/laptop parts "NVIDIA RTX A1000" and "NVIDIA RTX A3000" matched
"a100"/"a30" and were wrongly tuned as data-center GPUs (forcing FP32 cuBLAS
accumulation and the multi-GPU env, which carry a real cost on those cards).
Switch to a word-boundary regex.
- gpu_indices carries physical GPU ids (translated from torch ordinals by
_get_gpu_free_memory via CUDA_VISIBLE_DEVICES), but they were passed straight
into torch.cuda.get_device_properties, which expects mask-relative ordinals.
On a masked host (e.g. CUDA_VISIBLE_DEVICES=4,5,6,7) a selection like [4,5]
fell out of range and silently dropped the tuning, and on a mixed mask it could
probe the wrong GPU class. Build a physical-id to device-name map mirroring
_get_gpu_free_memory, then look up the selection by physical id.
Add regression tests for the A1000/A3000 false positives and for masked-host
physical-id selection (reordered and mixed-class masks included).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten data-center GPU tuning comments
Comment-only pass over the DC tuning block and its tests: shorten verbose
docstrings/comments, drop ones that restate the code, collapse multi-line
blocks. Keep the load-bearing rationale (physical-id vs ordinal mapping, the
word-boundary reason, the B200 benchmark numbers). No code change: verified
with comment_tools.py check --strip-docstrings (code unchanged, comments only).
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix/uv-bytecode-timeout
* make sure that win installer upgrades uv for bytecode timeout
* Clarify uv bytecode timeout comment in install.sh and install.ps1
* Read installer scripts as UTF-8 in parity test so it runs on Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer freshly installed uv when an older one shadows it on PATH
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix step count mismatch when sequence packing is enabled
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Emit a single step-0 progress event and guard applyStatus totalSteps
Merge the two consecutive _update_progress calls before train() so the
step-0 gate in _on_progress fires once instead of twice, avoiding a
duplicate startup event and a null-metric step-0 row in training_metrics.
Apply the same positive-number guard to applyStatus that applyProgress
uses, so a stale or startup status poll can no longer overwrite the
packed step count with 0 or replace it with a stale total.
* Log debug message when train_dataset length is unavailable
The TypeError fallback for length-less datasets (e.g. streaming
IterableDataset) was silent, leaving no trace that the step estimate
came from the raw dataset rather than the packed one.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
IOReport energy counters can reset (sleep/wake, power gating), making a poll
delta negative. Return None for a negative total so the monitor shows -- for
that poll instead of a bogus negative wattage; it self-corrects next poll.
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix race in tool-call confirmation gate
* Studio: gate built-in tool calls and harden the confirmation handshake
The Allow / Always allow / Deny controls only lived in the fallback tool
card, but the built-in tools (web search, python, terminal, code
execution, image generation) render with their own components and so
never showed the buttons. Those calls paused after tool_start with no way
to approve them, hanging until the 1 hour timeout. Only MCP tools, which
use the fallback renderer, actually worked.
Render the controls for every tool card by wrapping each registered tool
component (and the fallback) in thread.tsx with a shared
ToolConfirmationControls, so the gate applies uniformly.
Also make the handshake robust:
- The gate keys on a per-call approval_id minted by the backend and
echoed in tool_start, instead of session_id alone, so a stale or
concurrent confirmation can no longer resolve the wrong call.
- The approval slot is registered before tool_start is yielded, closing
the race where a fast click or an auto "Always allow" could reach the
backend before the waiter existed.
- The frontend resolves with the same session id the request was sent
with (plus the approval_id), fixing the new-thread mismatch where the
confirmation targeted a different session than the blocked stream.
- The confirm endpoint returns {resolved}; the UI keeps the buttons and
shows a retry hint until the backend confirms a match, instead of
hiding them on a failed or mistargeted post.
- The gate runs after the disabled-tool and duplicate-call checks, so a
call that will not execute is not put up for approval. A denied call is
still excluded from duplicate detection, so re-issuing and approving it
works.
- "Always allow" is scoped per session to match the backend gate.
Add backend tests for the approval registry, the SSE no-deadlock
handshake, and the loop integration (allow, deny, disabled, duplicate,
re-issue after deny).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move "Confirm tool calls" to the Tools section
* Studio: Keep tool group open while a tool call awaits confirmation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool confirmation session scope for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix confirmation follow-ups for PR #5869
* Apply pre-commit formatting for PR #5869
* Fix confirmation cleanup for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden confirmation lookups for PR #5869
* Studio: make the tool-call confirmation decision immutable
resolve_tool_decision accepted a second confirmation for the same approval_id
and overwrote slot["decision"] in the window before the waiter reads it and
pops the slot, so a duplicate or out-of-order POST could flip an Allow to Deny
(and returned a misleading resolved:true). Reject once the slot's event is
already set so the first decision wins. Adds a regression test.
* Fix/adjust tool confirmations for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: show Apple GPU temperature and power in the GPU monitor (macOS)
The GPU monitor on Apple Silicon always showed -- for Temperature and
Power: the MLX branch of get_gpu_utilization() hardcoded None because
ioreg's AGXAccelerator PerformanceStatistics carries neither metric.
Add utils/hardware/apple.py, mirroring macmon's no-sudo approach:
- Temperature: average of the AppleSMC "Tg*" float keys via the
AppleSMCKeysEndpoint user client (ctypes/IOKit, macOS 14+).
- Power: IOReport "Energy Model" group, "GPU Energy" channels; each
poll diffs the energy counter against the previous poll's sample, so
the value is the average wattage over the polling window. The first
poll only sets the baseline and returns None.
Both readers latch to None on first failure and never raise, so
non-Mac platforms and locked-down hosts keep the previous behavior.
* Sample IOReport with the subscribed channels descriptor for PR #6187
IOReportCreateSubscription writes the channel descriptor that later samples
must use; sampling with the original requested group can return no Energy
Model entries on hosts that normalize the channel set, leaving power_draw_w
null after the baseline. Use the subscribed descriptor (matching macmon) and
fall back to the requested channels if the OS leaves it unset.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
* Fix EmptyLogits gathering in nested structure and patch recursively_apply on accelerator module
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire EmptyLogits Accelerate patch into startup and fix find_device, pickling, tests for PR #6092
- Call patch_accelerate_recursively_apply() in _gpu_init.py so real imports
install it; previously it was only invoked by the tests
- Make both wrappers idempotent so repeated calls do not stack
- Rework find_device: skip EmptyLogits while still finding real tensors in any
order, keep returning None for tensor-free payloads (AlignDevicesHook relies
on None), fall back to PartialState().device only for sentinel-only payloads
- Give EmptyLogits stateless __reduce__ and drop the stomped pickle stubs on
EMPTY_LOGITS so debug mode gather_object works in real distributed runs
- Put test tensors on PartialState().device so the debug mode test also passes
on GPU machines, and add drift tests for startup wiring, idempotency and
find_device ordering
Verified on 2x B200: ACCELERATE_DEBUG_MODE=1 torchrun gather/broadcast/pad of
sentinel and mixed payloads all pass, training losses unchanged, full drift
suite 25/25.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Define EmptyLogits equality on the class for PR #6092
Gathered sentinel copies must compare equal in accelerate debug mode
regardless of whether the patched recursively_apply saw the sentinel first
in that process. Class body __eq__ requires restoring __hash__ explicitly.
Verified: 123 case simulation battery on accelerate 0.34.2 through latest,
2 process gloo CPU and NCCL GPU debug mode runs, drift suite 25/25.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix UnicodeDecodeError on Windows reading subprocess output in save path
On Windows the default text encoding is the locale code page (cp1252), not
UTF-8. The text-mode subprocess calls in save.py (text=True /
universal_newlines=True) set no explicit encoding, so they decode
llama.cpp / Ollama output with cp1252. When a child process emits a byte
undefined in cp1252 -- e.g. 0x9d, which appears inside the UTF-8 encoding
of common punctuation / box-drawing glyphs and in non-ASCII file paths --
the read raises UnicodeDecodeError and aborts GGUF export.
Add encoding="utf-8", errors="replace" to all 8 text-mode subprocess calls.
errors="replace" also avoids silent mojibake for inputs whose bytes happen
to be valid-but-wrong in cp1252.
Add tests/saving/test_save_subprocess_utf8_encoding.py:
- an AST drift detector asserting every text-mode subprocess call in
save.py pins encoding="utf-8" (runs without importing torch/unsloth_zoo)
- a behavioural test reproducing the cp1252 failure and the utf-8 fix
Relates-to: #2660
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* add FastModel config passthrough
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix fastmodel config passthrough for task configs
* fix config-driven FastModel task model selection
* fix text only fastmodel task config selection
* fix fastmodel task config inference from user configs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix fastmodel problem_type config passthrough
* fix fastlanguagemodel config passthrough: FastLlamaModel owns user config
* fix fastlanguagemodel config passthrough: forward user config to causal loads and keep checkpoint quantization_config
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
The post-download llama-quantize / llama-server smoke test JIT-compiles CUDA kernels on the first GPU forward pass and stalls every install and update by minutes on Blackwell (sm_100). Gate it behind _RUN_STAGED_PREBUILT_VALIDATION, disabled for now, keeping the smoke test and the source-build fallback it triggers fully intact so it can be restored by flipping the flag to True.
Hashless external prebuilts (e.g. lemonade) are not in the approved-sha256 manifest and rely on the functional smoke test as their only integrity gate, so they are always validated regardless of the flag; only approved bundles, already proven by the sha256 manifest, skip it.
The sha256 archive verification and the static Linux/macOS preflights are unchanged and still run for every install.
test_sidebar_account_block_uses_leading_tight hardcoded gap-0.5 in its selector, but the sidebar account-block div moved to gap-px during UI polish (#6196), so the regex stopped matching and the test failed across every studio PR's Repo tests (CPU). Match the gap utility loosely (gap-\S+) since this guard is about the leading-* class for descender clipping, not the spacing.
* fix(studio): adopt server-loaded model before chat auto-load
When the user starts Studio via `studio run -m`, the web UI could still
auto-load a different cached GGUF on the first message because the chat
checkpoint was empty. Sync from /api/inference/status before falling back
to autoLoadSmallestModel so CLI-loaded models are not replaced.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(studio): hydrate adopted CLI model and harden auto-load errors
Extract shared inference-status hydration for refresh() and CLI adopt
paths so the first chat turn gets reasoning/tools flags. Wrap auto-load
(including adopt) in try/catch for image-edit cleanup, and drop the
redundant adopt call in run().
Co-authored-by: Cursor <cursoragent@cursor.com>
* Guard model adoption against status failures and mid-flight selection for PR #5900
* ci: trigger pre-commit.ci after main merge
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): inherit llama_extra_args and honor --no-mmproj
Reloading the same GGUF from the UI without gguf_variant no longer drops
CLI pass-through args like --no-mmproj. Skip mmproj download and launch
when --no-mmproj is present in llama_extra_args.
Co-authored-by: Cursor <cursoragent@cursor.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): tighten GGUF llama_extra_args variant inheritance guard
Reject inherited CLI args when the request changes gguf_variant or when
omitted variant resolves differently from the stored extra_args source.
Co-authored-by: Cursor <cursoragent@cursor.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Treat --no-mmproj-auto and --mmproj-auto with last-wins parsing for PR #5902
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): load run.py by path for editable installs
`studio update` can leave a partial site-packages/studio/backend/ tree
(plugin build artefacts only). That shadowed tree wins over an editable
install and breaks `from studio.backend.run import ...`. Loading run.py
by file path via importlib sidesteps the conflict.
The module is cached in _RUN_MODULE so repeated calls are cheap.
If exec_module fails, the module is removed from sys.modules before
re-raising so a subsequent retry starts clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle None __file__ when checking cached run module for PR #5909
* Harden _load_backend_auth_storage against None __file__ and resolve cache-key path (PR #5909)
* Adapt studio run/cloudflare in-venv tests to _load_run_module loader (PR #5909)
---------
Co-authored-by: Jim Dawdy <jimdawdy@Jims-MacBook-Pro.local>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Handle rope_type 'default' on transformers 5 to stop false RoPE warning
transformers 5 reports rope_type="default" for every plain (unscaled) config
and dropped "default" from ROPE_INIT_FUNCTIONS. _compute_config_rope_inv_freq
then did ROPE_INIT_FUNCTIONS["default"], hit KeyError, returned None and logged
"Could not apply RoPE scaling 'default'; long-context generation may degrade"
on every model load. The inv_freq was still correct (the constructor recomputes
vanilla on None), but the warning is a false alarm for unscaled models.
Compute the unscaled inv_freq directly for rope_type "default"/None instead of
going through ROPE_INIT_FUNCTIONS, so plain configs return the right value with
no warning. Scaled types (llama3/linear/yarn/...) are unchanged.
Also skip test_object_style_rope_scaling_on_config_delegates_correctly when
transformers strict-validates rope_scaling (5.x): it rejects a non-dict object
on config.rope_scaling, so the object-style delegation path cannot be set up
there. The test still runs and asserts on transformers <5.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: deduplicate lemonade ROCm prebuilt selection log
resolve_lemonade_rocm_choice() is called twice per install (direct
planner + resolve_upstream_asset_choice). The API fetch is already
memoised via _fetch_lemonade_release_cached but the selection log
lines were still emitted on both calls, printing the 'trying
lemonade-sdk ROCm prebuilt' banner and hash-manifest NOTE twice.
Add _lemonade_selection_logged set keyed on (gfx_target, asset_name)
and guard the two log() calls behind a membership check so they print
exactly once per process regardless of call count.
Also extend the _clear_lemonade_release_cache test fixture to clear
the new set between tests to prevent cross-test state bleed.
Fixes#6020
* fix: write log() output to stdout to avoid PowerShell NativeCommandError
On Windows, PowerShell treats any stderr output from a native process as
an error record and prefixes it with 'python.exe :' and sets the
ErrorId to NativeCommandError. Since log() wrote to sys.stderr, every
[llama-prebuilt] status line triggered this, making normal progress
output look like errors in the installer console.
Switch log() to sys.stdout. The download progress bar (DownloadProgress)
retains its stderr/tty logic unchanged -- that path is for interactive
terminal rendering, not status logging.
* fix: remove redundant 'or ""' in lemonade log_key
host.rocm_gfx_target is already guaranteed truthy by the early
return at the top of resolve_lemonade_rocm_choice. The fallback
was dead code.
* Keep resolver stdout machine-readable, route install logs to stdout
log() sending everything to stdout breaks the resolver modes: setup.sh
json.load()s the whole stdout, so one helper log line (network retry,
release-tag scan) corrupts the parse and silently drops back to building
"latest". Default log() to stderr and flip to stdout only on the install
path, where PowerShell otherwise renders stderr as NativeCommandError
noise. Also tighten the lemonade dedup comments.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
On a 0.0.0.0 bind whose public ip:port is not reachable (cloud firewall),
the banner still printed "Secure link access via Cloudflare: <url>" right
after "is NOT reachable from the public internet", which reads as if the
tunnel might also be blocked. The Cloudflare quick-tunnel works regardless.
Thread the reachability probe result through a module-level _public_reachable
tri-state and, when the public probe definitively failed but the tunnel is
up, print "Also, the secure link access via Cloudflare works: <url>".
Reachable or undecided cases keep the existing wording.
