4 commits

Author	SHA1	Message	Date
Maheswar Kumar	d20db3f1f0	studio: add audio page with tts/stt create tab, train tab, and openai audio endpoints (#7984) ... * add studio audio page: tts and stt create tab, audio train panel, openai audio endpoints new /audio page mirroring images: create tab with speak (tts via the main inference slot) and transcribe (stt via the dictation sidecars) modes, an always-visible capability line so the loaded model's task is never ambiguous, and a train tab driving the generic /api/train/* audio branches. backend: audio_gallery.py persists tts clips as wav + json sidecar pairs; /v1/audio/speech (openai createSpeech shape, raw wav out) and /v1/audio/transcriptions (multipart, json/text) on the dual-mounted router; gallery list/file/delete/clear on the studio router; the tts core of /audio/generate extracted into _generate_tts_wav so both routes share it and persist clips; keep-warm suffix and transcriptions body cap wired. frontend: AUDIO_CATALOG (orpheus, csm, spark, oute, llasa as tts; whisper and qwen3-asr as stt) painted into the model selector with a per-group task tag, chat-picker speech picks rerouting to /audio, persistent mount in __root, sidebar row under more below video, nav registry + personalization defaults, and the audio nav label in all 12 locales. tests: audio gallery unit tests, speech/transcriptions route round-trips with a faked tts/stt core, middleware body-cap and /v1 surface additions, and the sidebar parity fixtures extended for the audio id. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/audio: fix TTS detection, surface backend capability errors, searchable train pickers Detection: _AUDIO_TOKEN_PATTERNS is first-match-wins, and audio_vlm's generic <|audio|> was tested before the codec fingerprints. Orpheus carries both that token and 28k <custom_token_N> SNAC codes, so it typed as audio_vlm, leaving is_audio False and the Audio page refusing a model that had loaded fine. Codec patterns now go first; Orpheus reports snac. Errors: safe_error_detail flattened "Text-to-speech is not supported on the MLX backend yet" into "An internal error occurred", so a safetensors TTS load on Apple Silicon failed with no reason. Capability answers are now a typed AudioBackendUnsupportedError tagged by the worker and returned as 501 with the message and the GGUF workaround. Tests: /audio/generate persists every clip, so suites driving it with a fake TTS core wrote silent wavs into the real gallery, where the page listed them. An autouse conftest fixture redirects studio_root. Picker: Recommended seeds curated rows in the order given, so a fixed order left every STT row below the fold on Transcribe. The active mode's task now leads. Adds Whisper Tiny/Base, which both sidecars already carry. Audio opts into community models via includeCommunity: non-unsloth TTS/ASR appear in search and trending ones below the unsloth rows, and Search Hub is restored for that case. Train: base model and dataset are search-as-you-type over the Hub with curated entries pinned; default dataset is Etherll/kaira (audio + text columns, no overrides). Panel is sectioned model/data/parameters with a run preview and its left edge tracks the header selector. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/audio: drop the audio train panel, send Train to the Train page Audio fine-tuning was a second, thinner copy of a flow the Train page already owns. Removes the panel and its charts and Hub comboboxes; the Train pill now toasts that Unsloth trains TTS and STT there, given the right base and a dataset with audio plus transcript columns, and navigates to /studio. Also stops TTS picks dead-ending on Mac. Safetensors loads through MLX, which has no TTS branch, so the model loaded and every generation 501'd. A TTS pick with a GGUF build published now loads that instead and says why, since llama.cpp is the only backend carrying the snac/bicodec/dac decoders. Picks with no GGUF still get the 501, which now explains itself. * studio/audio: trim duplicated comments on the GGUF fallback * studio/audio: reword the Train page redirect toast * studio/audio: match the media pane heading treatment, drop the cross-page links Generate audio and Transcribe now use the same heading block as the Images and Video Create panes from #7986: text-xl with leading-none, an 18px icon on the heading line, and a text-xs line under it. Also drops the Images and Video links from the header. They belong between the two visual pages; audio is a different kind of output, so the row was noise here. * studio/audio: per-clip actions menu in History, drop the Audio cross-page links History rows were a single button with no per-row actions, so deleting one clip meant selecting it first and using the player's buttons. Each row now carries a dots menu (use text again, copy text, download WAV, delete), revealed on hover, focus or while open. The row becomes a shell div since the trigger is a button and cannot nest inside one. Downloading from a row fetches the clip bytes on demand: only the selected clip has them cached. Removes the Audio link from the Images and Video headers, matching the Audio page dropping its links to them. * studio/audio: address the Codex review findings Community models in the picker introduced most of these. - Route uncurated ASR picks to the STT sidecar. audioTaskFor returns null for a repo outside AUDIO_CATALOG, so community Whisper repos loaded into the TTS slot. Picks now carry their Hub pipeline tag and fall back to it. - Let community safetensors into Recommended. The curated-artifact clause in keep() can never pass for a community row, so third-party TTS and ASR checkpoints were browse-invisible. Community rows use the rest of the gate. - Feed the community listings into resultGgufIds, so a tag-only GGUF repo opens the variant expander instead of loading as safetensors. - Stop recording when the page goes inactive. The page stays mounted, so the unmount cleanup never ran and the mic stayed hot after navigating away. - Keep generated audio when the gallery write fails. Persistence is best-effort server-side and still returns the WAV; the page now plays it. - Guard gallery pagination with an in-flight flag. Repeated scrolls reused one offset and appended the same page, duplicating clips and React keys. - Select the mtmd engine for Qwen3-ASR in /v1/audio/transcriptions. Whisper ids are shared with the Transformers sidecar, so those keep the default. - Persist TTS clips via asyncio.to_thread, matching the image gallery routes. - Reword the MLX hint: only Orpheus publishes a GGUF build, so it now names the host as the general fix and GGUF as the conditional one. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/audio: fix lifecycle and device inventory * Fix community audio routing and pagination * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix fresh audio review findings * Fix audio lifecycle review findings * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix remaining audio review findings * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Whisper cache inventory contract * Fix final audio convergence findings * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cancel hidden audio model loads * Cancel hidden transcriptions and bound audio work * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix audio model discovery and gallery paging * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix concurrent audio cancellation and streaming * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope STT startup cancellation to request owner * Stabilize health auth test across hardware states * Scope audio STT lifecycle ownership * Close STT load cancellation race * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Hide unsupported filesystem ASR rows * Fix audio runtime residency edge cases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bound stalled audio cancellation * Scope audio load cancellation by request * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix audio model handoff routing * Trim redundant audio comments * Update the source and fixture contracts this branch changed The audio page renamed diffusionPageForTask to mediaPageForTask, added isAudioRoute to isChatLike, moved the recommendable-format gate into keep, folded the community listing into the recommended pager, forwards a scoped load_cancel_event through the GGUF loader, and emits cached Whisper repos as ASR rows for the Audio page. Point the exact-source and fixture assertions at the new shape; behaviour is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/audio: fix dictation residency, audio dataset decoding and the cancel paths Leo's run on Windows 11 with an RX 9060 XT reported two blockers on PR 7984, and a high-effort review of the branch found ten more defects. Both reports are addressed here. Dictation kept one model resident per engine, so a Transformers Whisper and a llama.cpp Qwen3-ASR sat in VRAM together for the whole 5-minute keep-alive, and a speech model loaded beside a dictation model the Audio page no longer needed. `stt_registry.load` now releases every other engine before loading, under a lock so two loads on different engines cannot interleave, and leaving Transcribe releases the sidecar that tab loaded. Eject and the mode transition share one release path. Audio datasets were unreadable whenever torchcodec cannot dlopen its FFmpeg libraries, which is the Windows default: `disable_torchcodec_if_broken` clears `datasets.config.TORCHCODEC_AVAILABLE`, and datasets 4.x then raises "To support decoding audio data, please install 'torchcodec'" for the format check and all six audio trainer paths. `utils/datasets/audio_decode` installs a soundfile decoder in that case, restoring the pre-4.0 `{"path", "array", "sampling_rate"}` contract those callers already read. `audio_array_and_rate` reads a cell from either backend, which also fixes the `.get("array")` reads that raised AttributeError against the torchcodec AudioDecoder on a working host. Cancelling a GGUF dictation request used to SIGTERM the shared whisper-server, so the next dictation paid a relaunch plus a model load. The sidecar now speaks http.client and shuts the socket instead, sharing `_close_connection_on_cancel` with the mtmd sidecar. `_transcribe_audio_result`'s CancelledError branch no longer calls the lock-taking `cancel_transcription` inline on the event loop. `refreshGallery` replaced the whole clip list with the newest page, collapsing a paginated History and moving the player to a different clip on every delete and generate. It merges the page into the list now and only reselects when the selected clip is gone. A superseded refresh returns the clips its own fetch saw, so a generation whose clip did persist is no longer told it was not saved. The chat picker routed cached repos tagged text-to-speech to /audio with no runtime-support gate, though the Audio page filters exactly those out, and forwarded `meta.pipelineTag` rather than the task that chose the route. Both now go through `audioPickIsRoutable` and `pickedTask`. Also: a drain-cancelled generation raises `AudioGenerationCancelledError` so an idle auto-unload reports 499 rather than a flattened 500; the load path's cancel handshake is bounded, since only the cancelling unload sets it and that unload needs a pool thread of its own; the cache inventory reuses the metadata it already probed; and the Audio page adopts the container-query layout Images and Video use, so the 408px rail stacks below 50rem instead of squeezing the preview. The review also flagged the TTS cancel drain tearing down the worker. Left alone: the stopping criteria is checked per token, not per decode step, and the pre-audio_started teardown is a deliberate invariant that `test_audio_tts_cancellation.py` asserts. Verified on CPU: 794 backend tests over the stt, audio, whisper, inventory and model sweeps, 1591 frontend node tests, typecheck. No GPU here, so the dictation residency and decode fixes still need a run on Leo's ROCm host. * studio/audio: drop the duplicate seed spread the merge left in recommendedMeta * Bound the scoped load cancel handshake _run_tracked_load_model_impl waited on cancel_complete with no timeout. Only /unload's finally sets it for a running attempt, so a disconnect or a shutdown between the cancel and that finally left nobody to set it: /load then parked forever while holding inference_lifecycle_gate, and since asyncio.to_thread runs on non-daemon executor threads the process could not exit either. Reproduced by dropping the handshake and watching both the request and the interpreter hang. Wait 15s, log, and release. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/audio: correct the decode probe, dictation release order and gallery merge Follow-up to the previous commit, from a multi-agent review of its own diff. Seven defects it introduced, each with a reproduction. The decode fix did not fire in the API process. `datasets.config.TORCHCODEC_AVAILABLE` is `find_spec("torchcodec") is not None`, which is true for an installed torchcodec whose native libraries cannot dlopen, and only `unsloth.import_fixes` corrects that. The API process never imports unsloth, so the dataset format check still reached the broken decoder. `ensure_audio_decoding` now probes the import itself and clears the flag, so `datasets`' own gates agree with it. `np.mean(array, axis = tuple(range(array.ndim - 1)))` was copied from datasets' torchcodec shim, which yields (channels, frames); soundfile yields (frames, channels). Every stereo clip collapsed to one sample per channel and trained as near-silence with no warning. Now `axis = -1`, with a test asserting the frame count survives. `Audio.encode_example` needs torchcodec too, and the audio VLM path maps without `remove_columns`, so reading the decoded array writes it back through `cast_storage`. The gate passed and the run then died on the error it was meant to prevent, so encode is patched alongside decode. `MtmdSttSidecar.unload(wait = False)` called `RLock.locked()`, added in Python 3.14, on a 3.10 to 3.13 matrix. It raised, `stt_registry.unload` logged and swallowed it, and the llama-server kept its model: the exact doubling the release exists to prevent. The lock was also the wrong probe, since `transcribe` runs `_post_transcribe` outside it and counts `_active_requests` instead. That is what it checks now. The registry released other engines before the target's preflight, so a 409 for a model that is not downloaded cost the user the engine they were using. `_load_locked` orders preflight ahead of release for that reason; the registry now does too. On the frontend, `owned` tested residency rather than ownership. The activation resync adopts whatever a sidecar holds, including a model chat dictation loaded, so leaving Transcribe could unload it. An explicit `sttLoadedByThisPage` ref now gates the release, and the selection is forgotten only once the unload lands, so a failed unload still has an Eject to retry with. `mergeGalleryPage` stitched unconditionally. "Clear all" merged an empty page into the cache and left every deleted row on screen, and a cache with no ids in common with the page rendered a gap as contiguous with a cursor that could never reach it. It reports whether it stitched, and the cursor is only preserved when it did. Also reverted: `audio_array_and_rate` and its five trainer call sites. `unsloth_zoo.patch_torchcodec_audio_decoder` already gives the torchcodec AudioDecoder a `.get`, and the soundfile decoder returns a plain dict, so the pre-existing reads worked on both backends and the helper was an unrelated refactor. The chat picker now refuses an unrunnable speech pick with a message instead of falling through to a chat load that evicts the resident model. The Audio header pill takes the Images page's `px-3` below 68rem so it stops covering the model name. Tests that passed on deleted code are anchored, `audioPickIsRoutable` gets behavioural cases, and the backend CI job installs soundfile and librosa so the decode module no longer skips. Verified on CPU: 1149 backend tests over the stt, audio, whisper, inventory, monitor, load and admission sweeps, 1600 frontend node tests, typecheck, catalog:check, i18n strict. Two backend failures and 23 in tests/studio/install/test_rocm_support.py reproduce on a clean checkout. No GPU here, so the dictation residency and audio decode paths still need a run on Leo's ROCm host. * studio/audio: drop the duplicate fallthrough in the encode shim * Gate the Audio recorder on the browser capability check Safari and other WebKit builds ship no MediaRecorder, and Studio reached over plain http on a LAN address (-H 0.0.0.0) is not a secure context, so navigator.mediaDevices is undefined in every engine there. The composer already gates its microphone on StudioModelDictationAdapter.isSupported(); the Audio page did not, so Record was enabled and could only ever fail with 'Could not access the microphone', which blames the wrong thing. Reuse the same check, say why in the field hint, and leave file upload available so transcription still works on those hosts. * Fix eight review findings for PR #7984 Backend: - Propagate disconnect cancellation to the base64 JSON transcribe route, so a client that goes away no longer leaves the sidecar transcribing under its lock. - Serialize installation of the soundfile audio decoder. Two first-time callers could both pass the _installed check, and the loser captured the shim as _ORIGINAL_ENCODE, recursing into itself until RecursionError. - Match the GGUF audio read timeout to the exposed token limit instead of a fixed 300s, keeping 300s as the floor. Frontend: - Keep TTS generation running across route changes. Only unmount aborts, matching Images and Video and the note in routes/audio.tsx. - Distinguish a failed gallery refresh from a failed save, so a clip the server did persist is no longer reported as unsaved. - Drop server-deleted clips when merging gallery pages, and do not stitch a page that no longer overlaps the cache. - Preserve Hub evidence (base model, tags, library) when routing community audio picks, so a checkpoint whose family is only in its metadata still routes. - Refresh Audio residency after a global model eject. Tests: regression coverage for the decoder install race and the JSON transcribe request forwarding; restore Audio.encode_example in the decode fixture; skip the decode module without librosa; stub the torchcodec probe so the left-alone case holds on hosts without it; read trainer.py rather than importing the whole torch stack for a source-contract assertion. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in the audio code for PR #7984 Condense 51 multi-line docstrings and JSDoc blocks added by this PR across 24 files, dropping 43 lines while keeping what each one is there to say. Comments, docstrings and whitespace only: no code, signature or behaviour changes. * studio/audio: fetch Spark-TTS into the HF cache, and stop reporting an unreadable repo as non-audio snapshot_download(repo, local_dir = repo.split('/')[-1]) resolves against the process CWD. Under the desktop shell that is studio/src-tauri, so the model landed inside the Tauri crate, the dev watcher rebuilt on every file and killed the backend mid-load. It also bypasses hf_cache_settings, so the copy was invisible to the inventory and re-downloaded per CWD, while the trainer's local_files_only branch was already reading the cache. Dropped local_dir at all four Spark-TTS sites. detect_audio_type folded 'not an audio model' and 'could not read the repo' into a bare None, so a gated repo (401 on tokenizer_config.json) reached the Train page as a definitively non-audio model and the run was refused with 'This model does not support audio'. _detect_audio_from_tokenizer already tracked this correctly; the value was just discarded. Exposed it as detect_audio_type_checked, carried it to /api/models/config as audio_type_known, and the modality gate now blocks only on a known negative. A gated audio repo behaves like a gated text one: the run starts and fails on the real download error. The frontend flag is negative (audioCapabilityUnknown) so an absent value keeps the old blocking behaviour. Also: datasets 4.3 imports torchcodec.encoders at the top of Audio.encode_example before it inspects the value, so the soundfile shim delegating str/Path/bytes to the original re-raised the ImportError it exists to avoid. Those forms need no encoder and are handled directly now. (cherry picked from commit 1b6b6750eb1fad4e804f40091286432c7c0888a4) * studio/audio: summarise decoded audio cells in the dataset preview _serialize_preview_value compressed the undecoded {bytes, path} shape only. When torchcodec cannot load its FFmpeg libraries the soundfile fallback decodes instead and the dataset formatter returns {path, array, sampling_rate} with the waveform as a plain list, so the preview serialised one float per sample. Ten rows of a few seconds each is tens of MB of JSON; the client died with 'Maximum call stack size exceeded' before it could POST /api/train/start, which is why Spark-TTS training never reached the backend on a no-FFmpeg host. A decoded cell now collapses to '<audio, N samples @ R Hz, Ds>' the way a binary cell collapses. Also short-circuit detect_audio_type_checked on a falsy model name. Callers already passed None on every poll, which interpolated into the Hub URL and fetched /None/resolve/main/tokenizer_config.json every few seconds. Previously silent; the new not-definitive log made it visible. (cherry picked from commit 2ec97e54c917e946674fec3f89a3b038ce93b4da) * studio/audio: tag trained checkpoints with their codec and offer them on the Audio page A scan row carried no modality: scan_trained_models returns only (display_name, path, lora|merged) and LoRAInfo had no audio field. So a TTS checkpoint fine-tuned in Studio read as a text model everywhere -- the Audio page's task gate filtered it out, and chat sent it to the GGUF auto-switch, which cannot resolve a local adapter directory and answered 'is not downloaded on this server' for a model sitting in outputs/. /models/loras now reports audio_type, detected from the checkpoint's own tokenizer first (a merged export has one) and falling back to the base repo an adapter names. The Audio page feeds the TTS ones to the picker through the same additionalOnDeviceModels path Transcribe already uses for downloaded STT artifacts, so a checkpoint trained here is selectable where it was trained. Verified against the real run output: the trained Orpheus adapter detects as snac, a text model still reports None. (cherry picked from commit 6259cded9e05abc9c8ed792a0ae114ef6c283976) * studio/audio: label a trained checkpoint by name, not its directory renderAdditionalOnDeviceModelRow always labelled with model.id and linked it to the Hub. That reads fine for a repo id, but a checkpoint trained here is identified by its output directory, so the Audio picker showed two rows of truncated 'C:\Users\...' with a Hub link that goes nowhere. A local path now shows the model name with its base model as the meta line. (cherry picked from commit ba76272b409d5b6b351f182165712bd0c0941850) * studio/audio: stop the TTS watchdog killing a Transformers generation, and name checkpoints Three things from a field run of a trained Orpheus LoRA. Spark-TTS datasets could not be trained at all. The audio text-column allowlist is text/sentence/transcript/transcription/label; every svjack/SparkTTS_* set names the line to speak 'prompt', so detection found no text column, requires_manual_mapping came back True, and the mapping dialog left Continue disabled with no way to satisfy it. Orpheus's dataset uses 'text' and sailed through, which is the whole difference between the two. Added prompt and normalized_text (LJSpeech derivatives). Generation timed out at 120s. That bound only governs the Transformers subprocess path -- llama.cpp TTS never reaches it -- and was tuned against GGUF speeds, where the same clip returns in seconds. A safetensors LoRA needs minutes for it. The worker emits audio_started once and nothing until audio_done, so there is no progress signal to build a stall timeout on; raised the bound instead. A dead worker is already caught every second by _ensure_subprocess_alive, so this only has to bound a live wedged one. The load toast read 'Loading C:\Users\...\outputs\unsloth_orpheus-3b-0.1-ft_1786351654'. A trained checkpoint is identified by its directory, so it now shows the leaf with the training epoch stripped. (cherry picked from commit 00211b548c4a83087e0b889e4cbdd92d2b8ca1bc) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the chat runtime LoRA type for PR #7984 toLoraSummary reads lora.audio_type, which was added to BackendLoraInfo but not to this function's own parameter type, so the frontend build failed with TS2339 and the Tauri Linux job stopped before the Rust steps. * Fix six review findings for PR #7984 /v1/audio/speech asked for the chat default of 2048 new tokens, which the OpenAI CreateSpeech shape gives a client no way to raise, so any input past roughly half a minute of speech came back as a truncated WAV with HTTP 200. It now asks for AUDIO_GENERATION_MAX_TOKENS, the same ceiling the Audio page's slider uses. The gallery had no retention limit, so an automated client on that route could grow the Studio data directory until the disk filled. Oldest owned pairs beyond UNSLOTH_AUDIO_GALLERY_MAX_CLIPS (default 2000) are now pruned after a save. A TTS cancel arriving before audio_started armed the 5s drain deadline even though _cancel_generation is gated on the worker having started, so the window expired with no cancel ever sent and the teardown unloaded the model the user had just loaded. The pre-start wait now has its own 30s teardown budget, and the 5s drain is armed where the cancel is actually delivered. The worker's audio_error carried no cancelled flag, so a cancellation that sets the worker's shared event without the route's own event (an unload, a training admission, the GPU arbiter) surfaced as HTTP 500 rather than a cancellation. Every repo whose config sniffs as Whisper was un-hidden, but the can_chat guard was keyed on the seven curated ids, so a third-party or fine-tuned Whisper checkpoint stayed eligible for chat auto-load. The Audio page sent temperature on every request, which the backend reads as an explicit client override, so per-model recommendations (Spark-TTS 0.8, OuteTTS 0.4) never applied. It is now sent only once the slider has been moved. The page also loaded at exactly the max-token ceiling, leaving no context for the prompt itself; it now reserves room for both. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix two more review findings for PR #7984 The mtmd sidecar's wait=False unload read _active_requests without the lock and then acquired it, so a transcription claiming the slot in between had llama-server killed underneath it and lost the recording. Rechecked under the lock before releasing. Audio detection interpolated a local filesystem path into a Hub URL once the local read found no tokenizer_config.json. The /loras scan hits that for every adapter directory without its own tokenizer, and a transient failure is never cached, so each pass paid two 15s timeouts per checkpoint while blocking the event loop that called it. A local path now stops after the local read. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio/audio: load the Spark-TTS tokenizer from LLM/, not the repo root With the dataset gate fixed, a Spark run reaches pre_detect_and_load_tokenizer and dies there: unsloth/Spark-TTS-0.5B keeps only BiCodec/, config.yaml, src/ and wav2vec2-* at its repo root, so AutoTokenizer finds no vocab and raises 'Couldn't instantiate the backend tokenizer ... You need to have sentencepiece or tiktoken installed'. Both are installed; the message sends you after the wrong thing. _load_model already reads weights from LLM/. The tokenizer pre-detect now agrees, via subfolder, and only for a bicodec repo root -- a local checkpoint or an alias that already names LLM/ is left alone. Verified against the real repo: root raises, subfolder='LLM' returns Qwen2Tokenizer with 165158 tokens. * studio/audio: three Poseidon findings -- speech token budget, GGUF read timeout, checkpoint scan P1, /v1/audio/speech was pinned to the 2048 chat default. The route builds a ChatCompletionRequest with no max_tokens, so _tts_max_new_tokens fell through to 'or 2048' and CreateSpeech has no field a client could use to raise it. Anything past roughly half a minute of speech came back as a truncated WAV with HTTP 200 and no signal. Ask for AUDIO_GENERATION_MAX_TOKENS; the orchestrator clamps and scales its watchdog off the same value. My own regression: raising _AUDIO_GENERATION_TIMEOUT to 900s for the Transformers path also moved llama.cpp, which imports _audio_generation_timeout for its GGUF read timeout -- I had claimed llama.cpp never reaches it and was wrong. max(300.0, ...) went dead and every GGUF read got 900s minimum, up to 3600s. Since /audio/speech is in _INFERENCE_SUFFIXES a wedged server holds other_inference_request_count() up for that whole window, blocking idle auto-unload and 409-ing a training start. _audio_generation_timeout takes a base now: 900s subprocess, 300s GGUF. Also mine: _audio_type_of_checkpoint called detect_audio_type with no local_files_only, turning a filesystem scan into N Hub reads per poll, and a non-definitive miss is deliberately uncached so a gated or offline base re-fetched every time. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve the audio timeout base at call time for PR #7984 base defaulted to _AUDIO_GENERATION_TIMEOUT in the signature, so it was bound once at import and reassigning the module constant afterwards had no effect. Resolved inside the function instead. Both backends keep their intended budgets: 900s scaling to 3600s for the Transformers subprocess, 300s scaling to 1200s for GGUF. * Fix four more review findings for PR #7984 /v1/audio/speech is reachable after any /api/inference/load, including the default max_seq_length=0 that becomes 2048, so asking for the full 8192 ceiling overflowed or truncated. Capped to what is left of the loaded context once the prompt is accounted for. A generation whose gallery refresh missed it selected an id that is not in clips, so the player fell through to the empty state and the audio could not be played or downloaded. My earlier fix for the mislabelling caused that. The response WAV is now kept as a fallback until the real record is observed, labelled as saved rather than unsaved. download_status() clears model once the worker thread stops, so a cancellation the user made while the Audio page was hidden matched nothing on return and the deferred load restarted the whole multi-GB download. All three sidecars now report cancelled_model alongside cancelled, which the page matches on. Kept separate from model so the Downloads panel does not start tracking a cancelled download. A locally trained Whisper checkpoint was tagged automatic-speech-recognition and routed to the Audio page, which hands the filesystem path to /audio/stt/load, where resolve_model_id takes only a curated key or an owner/model Hub id and 422s. Local checkpoints no longer get the ASR tag; TTS still routes, since that loads through the main slot, which accepts a local path. * Fix four more review findings for PR #7984 The mtmd sidecar's active-request guard only covered wait=False, but the training VRAM path unloads with wait=True, so llama-server was killed under a live transcription. A blocking unload now drains active requests for up to 30s first, then proceeds so training is not stalled by a long recording. /v1/audio/speech floored an over-context prompt at one output token and forwarded it anyway, failing deep in generation. It now returns 400 while the caller can still shorten the input. Clearing the gallery reset the module cache but not the React clips state, so a failed follow-up refresh left every cleared row rendered against a revoked object URL. Cleared synchronously on the DELETE. The Spark-TTS tokenizer helper treated an LLM/ child as proof the path was already the tokenizer directory, but a cache-pinned or offline snapshot root has one, which is exactly the case needing the subfolder. Only a path ending in LLM, or one carrying its own tokenizer_config.json, skips it now. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix three more review findings for PR #7984 selectClip nulls the fallback clip, which undid the setFallbackClip immediately before it, so a clip the server persisted but the refresh missed still rendered the empty state. My earlier fix for that case was a no-op. selectClip now takes keepFallback for the one caller that needs it. Deleting a clip left the row on screen against an already-revoked object URL when the follow-up refresh failed, since refreshGallery returns the cache without calling setClips. The row is dropped on the DELETE now, as clear-all does. A merged Spark-TTS export reached the non-LoRA BiCodec branch, which called snapshot_download on an absolute path and then looked for an LLM/ child that a merged export does not have. It now loads the LLM from the export directory and resolves BiCodec assets from the base model recorded in export_metadata.json, mirroring the processor fallback already in this file. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix five more review findings for PR #7984 Switching STT engines loaded the new sidecar while the old one was still resident and released it only afterwards, so a switch could OOM on a device with room for either model alone. The other engines are now released before the allocation, but only once the checkpoint is known to be on disk, so a 409 for a model that was never downloaded still cannot cost the user the engine they were using. _tts_max_new_tokens ignored the prompt, so a Max tokens slider near the ceiling plus a long prompt overflowed the context the page loads with. It now subtracts the prompt from the loaded context, covering both the Studio and OpenAI routes. UNSLOTH_AUDIO_GALLERY_MAX_CLIPS documented that a non-numeric value disables pruning, but the parser restored the 2000-clip default, which would then delete the oldest recordings an operator had asked to keep. A client disconnecting mid-decode was only noticed after PyAV reached EOF or the 30-minute cap. The cancel event is polled in the frame loop now. A microphone recording had no duration or size bound and no timeslice, so an over-long take was buffered whole and uploaded only to be refused. It stops at the sidecar's own limits and reports why. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix five more review findings for PR #7984 The GGUF and MTMD sidecars still called the now-cancellable decoder without the event, so only the Transformers path actually stopped on a disconnect. The recorder's byte cap was 96 MiB against the raw route's 25 MiB STT_AUDIO_RAW_MAX_BYTES, so a dense codec could still build a recording that was refused with 413. It mirrors the raw limit now and stops before appending the chunk that would cross it. The TTS prompt reserve used len(prompt) // 3, which under-counts CJK and emoji badly, which is exactly the input that then overflows the context. It asks the loaded tokenizer where one is reachable, and otherwise estimates by character class rather than a flat ratio. Trained TTS checkpoints were offered on macOS even though MLX has no TTS decoder, so selecting one always returned "not supported on the MLX backend yet". Only GGUF exports are listed there now, matching how the catalog rows are filtered. The transcript download revoked its blob URL immediately after the synthetic click, which races browsers that resolve that navigation asynchronously. Deferred, as the gallery download already does. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix the CI backend failure and five review findings for PR #7984 The (Python 3.11) Backend tests job was failing on two counts this branch owns. scan_loras probed detect_audio_type without an hf_token, which test_security_gate_consistency forbids because a token-less probe misclassifies a gated model and poisons a token-keyed cache; the route takes the token as a dependency now and threads it through. The health-gate test stubbed _hardware_snapshot as a two-tuple, and main has since added chat_only_detail, so health_check raised IndexError reading snapshot[2] after the merge. Review findings: - /v1/audio/speech preflighted with len(input) // 3 while the budget helper used the tokenizer-aware estimate, so dense text passed the check and was then floored to one output token. Both use the same estimate now, and no budget left is a 400 rather than a one-token clip. - The audio routing evidence map held only remote search results, so a cached community Whisper row picked from the chat picker was judged on its id alone and refused routing to the page that does list it. Cached rows are included now. - Leaving Transcribe fired the sidecar release and returned, so a following TTS load allocated while the sidecar still held its model. The load waits for that teardown. - An audio dataset carrying both an instruction-like prompt column and a real transcript was mapped by schema order, which silently trains ASR against the instructions. Transcript names are matched first, prompt and normalized_text only as a fallback. Also merged current main. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix three more review findings for PR #7984 The merged Spark export's BiCodec fetch ran snapshot_download without the request token, so a private or gated base 401'd while the load that followed it would have authenticated fine. Only /v1/audio/speech rejected an over-context prompt; /api/inference/audio/generate floored the budget at one token and generated a clip too short to hold codec tokens. The guard moved into _generate_tts_wav, the core both routes share, so they cannot diverge again. The two route tests that covered it were retargeted at the helper and the shared core, since the route tests fake that core. The response fallback was kept when a refresh missed a persisted clip, as intended, but never cleared once the record arrived. Deleting the now-visible clip then made the fallback reappear from a stale data URL, labelled as saved. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip the Transformers TTS cancellation test without the training stack core.inference.inference imports peft transitively, which the backend-test CI job does not install, so this test failed the whole job with ModuleNotFoundError instead of reporting a skip for something that cannot run there. It is the only backend failure in CI that is not also present on main. * Fix /loras audio probe re-walking the cache on every poll for PR #7984 Measured before/after from two isolated installs at the merge base and the head. With 50 trained checkpoints, GET /api/models/loras went 6.0ms -> 26.5ms steady state, +340%, and it runs on the event loop, so it delayed unrelated requests too. The per-checkpoint audio probe answers non-definitively for an adapter directory without its own tokenizer and for a base repo that is not downloaded, and a non-definitive answer is deliberately never cached, so both repeated on every poll. - Remember an offline miss for 60s instead of re-probing. Bounded rather than permanent because both cases can become answerable without a restart: the base gets downloaded, or a training run finishes writing its tokenizer. - Key that on the raw name, before the casing resolution, since resolving a repo id that is not cached walks every HF cache directory, which is the cost itself. - Drop the per-row "could not determine" log to debug when offline. It was one line per checkpoint per poll, and offline it is the ordinary answer. - Run the scan in a worker thread. It was already blocking before the probe was added; the probe made the block long enough to matter. Steady state is now 6.0ms -> 7.3ms, +0.027ms per checkpoint. The latency tail is unchanged: over 400 samples p99 is 103ms before and 121ms after, max 264ms and 260ms, which is this box, not the PR. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reserve prompt overhead, bound the gallery by bytes, own STT by identity for PR #7984 Three review findings, all confirmed: - The TTS budget was context minus the RAW text, but no backend generates from that: llama_cpp's _TTS_PROMPTS wraps it in codec delimiters and the Transformers path builds its own prompt, so zero headroom meant those tokens pushed prompt plus max_new_tokens back over the context. Reserve 32 tokens for the wrapper. - The gallery cap counted clips, so 2000 maximum-length WAVs was still tens of gigabytes on a route an API client drives. Added a byte quota alongside it, whichever binds first, keeping the newest clip so a single oversized request does not read as a silent failure. - STT ownership was a boolean, so when another surface replaced the sidecar's model while Audio was inactive the activation resync adopted it and Eject unloaded a model this page never loaded. Store the model and engine and require both to match current residency before releasing. Backend 877 passed for the audio, gallery and STT suites; frontend 1716 passed; typecheck clean. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix STT engine mismatch, routed pick loss, cached load id and Spark alias for PR #7984 Four review findings, all confirmed: - Keying STT ownership on model AND engine, which I added last commit, broke the fallback case: a "gguf" pick on a host without whisper-server is served by the Transformers fallback and comes back resident under that engine, so the compare never matched and the sidecar was never freed. Key on the model alone. The registry keeps one model resident, and the unload resolves the serving engine server-side already (_resolve_serving_stt_engine). - A pick arriving while a cancelled TTS load was still settling hit the in-flight guard and was dropped, and the route effect had already cleared ?model=, so nothing retried it. Queue the loser and replay it when the load settles. - meta.loadId was discarded, so a row cached in a non-active HF cache was sent as its display repo id: it failed to load offline, or downloaded again into the active cache. Thread it through as the load target, as chat-page.tsx does. - A merged BiCodec export records its base as the registry alias "Spark-TTS-0.5B/LLM", which names a load subdirectory rather than a repo, so snapshot_download rejected it. Resolve it the way the trainer does. Extracted to core/inference/spark_tts_paths.py, a dependency-light leaf, so the mapping is testable without the Unsloth stack. Backend 1303 passed across the audio, gallery, STT, LoRA, capability and Spark suites; frontend 1734 passed; typecheck clean. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Settle a text tokenizer without parsing it, for PR #7984 The cold /api/models/loras path: with 50 checkpoints the first call after a restart was 172ms against 9ms before the PR, and profiling put 84 percent of the scan in the per-checkpoint audio probe. Most of that was json.loads on tokenizer_config files that were never going to match anything. A pattern can only match if its marker text appears in the file at all, so the raw text is scanned for the markers first and an ordinary text checkpoint is settled without a parse. Two details that matter: - The markers cannot be derived from _AUDIO_TOKEN_PATTERNS, which is lambdas, so a codec added there without a marker here would silently stop being detected. A test pins the pattern set and drives every pattern through the marker scan, and it fails when a codec is added. - A marker miss counts as "read" only when the text ends in a closing brace. Without that, a training run part-way through writing its tokenizer would become a definitive "not audio" and be cached for the life of the process, where before it stayed unknown because json.loads raised on the truncated text. Also stops the snac count summing all 28k of Orpheus's codes to answer a question settled by the first 10,001. First call 172ms -> 74ms, of which 27ms is now the scan itself, measured in process on the same fixtures. Steady state is unchanged at +1.4ms. Backend 1152 passed for the detection suites. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Spark-TTS capability detection and a broken whisper runtime for PR #7984 From the Windows/ROCm report on this PR. Spark-TTS training was blocked by the modality gate. Root cause is capability detection, not the gate: the Train page passes the registry alias "Spark-TTS-0.5B/LLM", which names a load subdirectory rather than a repo, so the probe fetched a repo that does not exist, got a 404 on every candidate path, and read that as a DEFINITIVE "not an audio model" rather than "not a repo id". unsloth/Spark-TTS-0.5B -> ('bicodec', True) correct Spark-TTS-0.5B/LLM -> (None, True) wrong, and definitive Resolved through load_scan_target first, the same way routes/training.py already resolves it for the trainer's own preflight. A whisper.cpp build that starts, answers GET /, and then dies on the first inference kept reporting as available: the binary and every linked library are present, so slim_runtime_intact() is satisfied and _resolve_serving_stt_engine never fell back, which left every recording 501-ing next to a loaded chip. Only inference can prove this case, so a failure there now marks the engine unavailable and the existing Transformers fallback takes over. A cancel closes that socket deliberately and is excluded, and a later success clears the flag. Also report the dictation device as rocm rather than cuda on ROCm. Torch keeps the "cuda" device name for HIP, which is right for the API and reads as a bug on an AMD card. Backend: 1324 passed across the STT, audio, capability and model route suites. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Drop Llasa from the audio picker: Studio cannot decode XCodec2, for PR #7984 Swept all 13 curated audio models against a running Studio rather than reading the catalog. Eleven are fine. Two are not: unsloth/Llasa-1B is_audio=False audio_type=None known=True unsloth/Llasa-3B is_audio=False audio_type=None known=True Llasa speaks XCodec2: 65,536 <|s_N|> tokens, confirmed from its tokenizer_config. That is in neither _AUDIO_TOKEN_PATTERNS nor AudioCodecManager, which decodes snac, csm, bicodec and dac only. So the row loaded and then failed at generation with "loaded but is not a supported TTS model", which is the same shape as the Orpheus defect this PR was opened to fix. The policy module already said so and contradicted itself: "the main-slot TTS backend decodes only the four codec families below", above a list of five. Removed Llasa from both the curated catalog and that community family list, so a searched Llasa repo is not admitted either. Studio can still TRAIN Llasa (unsloth_Llasa-3B.yaml is untouched); this catalog only feeds the Generate picker. Re-add both together with an xcodec2 decoder. An existing test asserted a community Llasa row WAS runnable, encoding the same wrong assumption. Corrected it, with the live reading recorded next to it. The other two entries that do not report as audio are the Qwen3-ASR GGUF pair, and that one is expected: a GGUF repo has no tokenizer_config.json at its root, and those models are served by the mtmd sidecar, which routes by catalog engine rather than by this flag. Left alone. Frontend 1736 passed, typecheck and catalog check clean. * Consolidate: one alias resolver, and name the STT tests after their subject Looked for duplication rather than assuming it. There is less than expected: the three STT sidecars share only ~116 near-identical lines between ggml and mtmd and ~26 across all three, and their big methods (_run, load, transcribe) are 25 to 40 percent similar, so a shared base would churn delicate concurrency code to save about 100 lines. Not worth it. Across 309 STT and audio tests exactly one pair is a near-duplicate, and that pair is the same check for two different engines, so it should stay. This PR is large because it does a lot, not because it repeats itself. Two things were worth consolidating. I had added core/inference/spark_tts_paths.py with its own copy of the "Spark-TTS-0.5B/LLM" to "unsloth/Spark-TTS-0.5B" mapping, while the capability probe and routes/training.py both resolve it with load_scan_target. Three copies of one mapping is how they drift. The export path now uses load_scan_target too, and the module and its test file are gone. The test that replaces them reads inference.py as text rather than importing it, because that import pulls the whole Unsloth stack, which was what made a second copy tempting. Four test files were named after the review rounds that produced them, which told a reader when they were written and nothing about what they cover. 105 tests, none duplicated: test_stt_review_fixes_3 + _4 -> test_stt_mtmd_sidecar (43 tests; the mtmd sidecar had no file of its own) test_stt_review_fixes -> merged into test_stt_ggml_sidecar (its subject) test_stt_review_fixes_2 -> test_stt_install_and_snapshot_validation 105 tests before, 105 after, five files down to three, and 1009 passed across the STT, audio, Spark and capability suites. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix nine review findings across the audio page and its backend Backend: - audio_decode: pick the token belonging to the repository a URL points at, the way datasets.Audio.decode_example does. Concatenated or interleaved streaming splits carry one token per source repo, so taking an arbitrary value sent one repo's credential to another repo's host. - stt_ggml_sidecar fallback: a GGUF pick downloads one .bin, so when the runtime turns out to be broken at inference time the Transformers engine it is redirected to has no snapshot and every retry raised SttModelNotDownloadedError. Fetch the equivalent snapshot in the background so the promised fallback is real; /audio/stt/status reports its progress. - TTS budget: recheck the prompt against the context after an idle-evicted model is restored. With nothing loaded there is no context to measure, so the first request after an eviction reached generation over-context and came back as a one-token clip. - mtmd unload: the under-lock recheck of _active_requests only guarded wait=False, so a blocking unload could reap llama-server underneath a transcription that started during the acquire. Drain outside the lock and retry, bounded by the same window (draining under the lock would block the request being waited on). - TTS prompt estimate: count every non-ASCII character as a token. The cut at U+2E7F billed Arabic, Cyrillic, Hebrew and the Indic scripts at the Latin third-of-a-token rate, so a long prompt in any of them passed the guard and overflowed during generation. Frontend: - Cancel a deactivating load with the target it was actually started with, not the repo id, so a load keyed on a path or a gguf file is really cancelled. - Replay a queued TTS pick only once the page is active again, and drain the queue on activation. - Abort the TTS handoff when releasing the STT sidecar fails, instead of loading TTS on top of a resident sidecar. - Gallery merge: a complete first page (has_more=false) is everything the server holds, so drop the cached tail rather than rendering clips another client deleted or the size cap pruned. * Move the imports left mid-file by the test merge to the top of the module * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Finish the merge: drop the expander prop main replaced, and type loadTarget #8383 moved GGUF partition identity to the backend and removed filenamePrefix from GgufVariantExpander, so the four call sites this branch touches keep pipelineTag and drop the prop that no longer exists. loadTarget was added to the pendingTtsLoad value but not to the ref's type, which only the project build (tsc -b) catches. * Hold speech generation until the transcribe sidecar release settles Switching straight from Transcribe to Speak with a speech model already resident needs no load, so the gate in the load path never ran and Generate could allocate beside the dictation model. Generate now waits on the same release, and a release that failed puts the page back in Transcribe instead of showing Speak while the sidecar is still resident. * Skip the two torch-dependent STT tests on a runner without torch Both drive a path that reaches `import torch` before the behaviour under test, so on a bare cross-platform runner they failed for a missing wheel rather than for anything they assert. * Pin the language list the GGUF language check reads in its test Without Transformers the helper returns None and the check is skipped, so the request fell through to the download guard and the test asserted nothing on any runner that lacks the dependency. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Give the whisper.cpp discovery tests the filenames Windows actually uses The launcher looks for whisper-server.exe there, and the slim guard checks the libraries the marker names verbatim, so fixtures that hardcoded whisper-server and libggml.so.0 were invisible to both: six tests failed on a Windows runner for the spelling rather than for anything they assert. The product code already handled both platforms. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep dictation to one resident engine, and scope releases to the claimed model - Implicit loads on the transcribe routes went straight to the sidecar, and only the registry releases the other engines, so an API client alternating between Qwen3-ASR and Whisper through /v1/audio/transcriptions held both until their independent idle timers fired. Routed through the shared lifecycle, which is a no-op once the model is resident. - Unload now takes the model the caller claims and compares it under the sidecar's own lock. Ownership is decided by the caller, so another surface can switch the same engine before a queued Eject or mode transition lands, and an unscoped release tore down a model the caller never owned. Threaded through the registry, the orchestrator, the route and the Audio page. - A whisper.cpp pick on a host without whisper-server is served and loaded through Transformers, but the residency resolver read only the gguf block, so the refresh that completes the load found nothing and the Transcribe controls stayed disabled until the page was revisited. Same fallback sttEngineStatusFor already applies. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Claim the generation slot before awaiting the transcribe release The Generate button only disables on busy, so awaiting the release first let every click during a slow unload through: each resumed into its own generateAudio while generateAbort tracked only the last, and either finally cleared the busy state out from under the other. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: shimmyshimmer <danielhanchen@gmail.com> Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com> Co-authored-by: danielhanchen <elliegouldingstuff@gmail.com> Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>	2026-08-11 04:47:26 -07:00
Michael Han	f61103d411	Studio: add Qwen3-ASR dictation, cancellable downloads, and a download prompt (#7835) ... * Studio: add Qwen3-ASR dictation, cancellable downloads, and a download prompt Rebuilt on current main. The dictation default work this branch started with is dropped, since #7799 landed the same behaviour. Adds a third STT engine so local dictation is not limited to Whisper. whisper.cpp is architecture-locked to Whisper, so Qwen3-ASR 0.6B and 1.7B run through llama-server with an audio mmproj instead, and the two models join the Voice settings dropdown. Model transfers move out of the request thread into a subprocess, so a download can actually be stopped. Partial files stay in the HF cache and a restarted download resumes from where it left off. Selecting local dictation with nothing downloaded now opens a confirmation naming the model and its size, rather than only failing with a toast. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: put dictation downloads in the shared panel and load on arrival A dictation model download reported progress only as text on the Voice tab, so it was the one download in Studio with no row in the download panel. The panel's poll loop is wired to the hub API, which does not own these transfers, so add a small external-job hook instead: another subsystem registers a job, reports progress, and supplies its own cancel. The STT status poll drives it, giving these downloads the same row, progress bar, rate, ETA and cancel as every other download. External jobs are never persisted or probed against the hub API, since there is no hub job behind them. An in-flight transfer is adopted on load, so a refresh keeps its row. A confirmed download now loads the model as soon as it lands and says it is ready, so the next mic press starts straight away. The old "open Voice settings for progress" toast goes with it. Also: the confirmation's mic glyph was sized as if it were a padded hugeicon, and toast buttons were the only square buttons left. * Studio: offer the local model when a browser has no speech service Firefox and other browsers without a speech service could only be told to go and pick local dictation themselves. They now get the same confirmation the mic raises, and accepting it selects local transcription as well as starting the download. Cancelling leaves the engine alone. An already downloaded model just switches, with no dialog. Pressing Download in Voice settings is itself the confirmation, so it starts the download directly instead of asking a second time. The prompt stays for the paths that never asked, like the mic finding nothing on disk. The confirmation's media circle is sized for a padded hugeicon, which left the smaller mic glyph adrift in it, so the circle comes down to match. * Studio: serve Qwen3-ASR from unslothai and recommend it over Whisper The Qwen3-ASR GGUFs now live under unslothai on Hugging Face, so point both sidecar and picker there. File names are unchanged, so this is only a repo swap. The 1.7B size label was 0.1 GB short of the model plus its mmproj. Qwen3-ASR is more accurate than Whisper at a comparable size and covers more languages, so it leads the picker, carries a Recommended badge, and is the default for a fresh install. A saved choice is untouched. * Studio: fix process lifetime, training VRAM and cancel bugs from review The mtmd sidecar diverged from how Studio spawns every other child process, and the review caught it. Matching stt_ggml_sidecar and the chat backend: - stderr was a pipe nobody read, so llama-server blocked mid-startup once its logs filled the buffer and the load waited out the full timeout - the ambient environment could not resolve bundled libs or pip CUDA runtimes, so it now uses the same env the chat backend builds for this binary - no child_popen_kwargs()/adopt_pid(), so the server outlived Studio holding a port and GPU memory. The download worker had the same gap, plus start_new_session, which kept it pulling gigabytes after the app closed Training freed the Transformers and GGUF sidecars but not mtmd, so a resident Qwen3-ASR held VRAM at -ngl 99 through a run, and admission could not see it either. Both now cover it, each behind its own exception boundary so one engine failing cannot discard the others. Also from review: two clients could each spawn a server and strand the first; the cancel endpoint reported a no-op as a cancellation because the status spread overwrote its own result; the mic kept recording after reporting the model was missing, so speech was lost; a relocated Hugging Face cache was written by the worker and looked for in the old place; and scrubbing the ambient token broke gated repos that worked before. The mmproj is now pinned to the commit the model landed on, rather than both resolving main separately. Only the second file is pinned: hf_hub_download writes refs/main only for a named revision, and the cached-file lookup resolves through that ref. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: move an untouched Whisper Small setting to the recommended model Changing DEFAULT_STT_MODEL only reached fresh installs. Anyone who had opened Voice settings before had "small" saved, so the store kept it and the download prompt still offered Whisper Small on a machine with no model at all. A v1 migration moves a saved "small", the old default, to the recommended model. Any other saved model was picked on purpose and is left alone, and choosing Small again persists at v1 so it sticks. The model list and the migration move to stt-model-catalog, which imports nothing from the app, so the node test runner can exercise them directly. The store re-exports it, so no consumer changes. * Studio: fix mtmd GPU and download-follow-through bugs from review The mtmd sidecar only checked for an active training run when transcribing, not when loading, so a download finishing mid-run auto-loaded llama-server at -ngl 99 into an experiment's VRAM. It now launches at -ngl 0 while training is active, as whisper.cpp already does with --no-gpu. It also held its lock across the inference request, so an unload (including the one training performs) blocked for up to the full request timeout. Only the port read is locked now. Three more from the same review: - a finished download loaded its model even if the user had since selected a different one or left local dictation, undoing that switch's unload - a failed cancel left the panel row stuck on "cancelling" for the rest of the transfer, because progress updates never reset state - a browser with no speech service offered to download gigabytes even when the local engine had no runtime installed to load them with * Studio: say the download failed, not that its status could not be read The Voice tab reaches this state when download.error is set, which means the transfer failed. It reported "Could not check download status", which points at the wrong thing entirely: the status came back fine and said the download had failed. Uses the existing failure message instead. * Studio: give dictation one owner and keep its models out of chat Voice settings and Model Hub could disagree about the same model. Unloading the transcription model in settings left Model Hub still reporting it loaded, because the two were looking at different things. Two causes, both fixed here. The Qwen3-ASR GGUFs were never registered as dictation checkpoints, so Model Hub treated them as ordinary chat models. They are small, so downloading one made it the smallest cached GGUF and it got picked up as the chat model. That is the "still loaded" the report was about: it really was loaded, as chat. They now sit in the hidden-STT lists next to the Whisper GGUFs, backend and frontend, which is where every other dictation checkpoint already lived. Dictation lifecycle also had no single owner. Each route resolved its own sidecar and looped its own unload. core/inference/stt_registry.py is now that owner, and InferenceOrchestrator exposes it through load_stt_model, unload_stt_model and resident_stt_model, so the object Model Hub loads a chat model with also knows what dictation holds. The sidecars are untouched: stt_ggml_sidecar.py still owns its whisper-server, stt_mtmd_sidecar.py its llama-server, stt_sidecar.py its in-process Transformers loader. Only the lifecycle above them moved. The routes take the orchestrator when one already exists and the registry it forwards to otherwise, so loading a GGUF dictation model on a cold process no longer builds the chat orchestrator (which waits on hardware detection) for a model that never reaches the chat worker. Same functions either way. Also passes encoding as a keyword in the mtmd revision read, which the text I/O check requires. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the mtmd sidecar's process lifetime hold up under review Third pass. Five findings, all reproduced before fixing. llama-server is only assigned to `_process` once it answers /health, so for the whole startup (up to 180s) `unload()` had nothing to act on. Training called exactly that, so a run could start while an `-ngl 99` child was still allocating. The starting process is now published as soon as Popen returns, and `cancel_pending_load()` / `wait_for_load_to_settle()` mirror what the whisper.cpp sidecar has always done; `free_stt_model_for_training` uses them. A startup that times out sent SIGTERM and dropped the handle. A child that ignores it kept its port and its GPU allocation until Studio exited, with `_process` never assigned so nothing could reach it. One `_reap` helper now terminates, waits, kills, waits and drops the PID, and both the release and timeout paths go through it. The idle timer stayed armed across a transcription. Keep-alive is five minutes, the request allows ten, and posting happens outside the lock, so long audio could have llama-server killed mid-request and the dictation thrown away. In-flight requests now hold the timer off. Download workers were adopted for the shutdown sweep but never dropped after being reaped. On macOS and Windows a reused PID could then be signalled by `terminate_all`. `reap_download()` pairs every spawn with a `forget_pid`. The llama.cpp updater coordinated only with the chat backend, but this sidecar runs the same binary out of the same tree: on Windows a live one locks the exe and the swap fails. It now takes the same maintenance guard whisper.cpp uses, and unlike whisper.cpp it fails open, since llama.cpp updates predate dictation and must work where it cannot even import. Voice settings also kept showing Download for the whole transfer: the status effect only re-polls while it can see a download, its last read predated this one, and the on-demand branch schedules nothing. Starting a download nudges it now, as cancelling already did. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stop mtmd loads and dictation sessions from costing the user work Fourth pass. Six findings, all confirmed. A load released the running server before checking the requested model was on disk, so switching to something not downloaded raised a 409 after the working server had already been reaped. The cache check moves ahead of the release, as whisper.cpp already does. A switch also ignored an in-flight transcription and tore the server down under it. That request is the user's dictation, so a switch during one is now refused with a busy 409 rather than silently discarded. Only when a live server is there to protect: a request against one that already died must not block recovery. The same model stays a no-op, so concurrent transcriptions on one model are unaffected. `max_tokens` was a flat 2048 while the decoder accepts up to thirty minutes of audio, so long clips returned success with the tail missing. The cap scales with the clip now, bounded well inside the context that also holds the audio. The server keeps llama.cpp's default context, which is already loaded from the model. `is_available()` checked only for llama-server, so a host without PyAV was offered gigabytes of Qwen download and every transcription then failed on decode. It checks PyAV too, which is what the whisper.cpp sidecar does and for the same reason. The dictation session only ended for a model that was not downloaded. Any other preload failure, a missing runtime or a load refused for training, left the recorder and level meter running behind a toast, so the user kept speaking into a session where no segment could ever be transcribed. The preload is cache-only, so nothing it reports is transient: it ends the session now. Segment failures still keep what was already transcribed. A completed dictation download published a chat-inventory hint. Curated repos are filtered by the hidden lists, but a custom Whisper repo is only hidden after the backend has read its config, so it could surface as a chat model for the hint's TTL. External jobs no longer produce hints. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: pin the mtmd projector, read the configured cache, and unload a startup Fifth pass. Three findings, all confirmed. The training branch set `-ngl 0` and stopped there, but the command still carries `--mmproj`, and clip.cpp gates the projector on its own flag. This repo's own `LlamaCppBackend._cmd_has_gpu_companion()` says so outright: any `--mmproj` without a last-wins `--no-mmproj-offload` is a GPU companion whatever `--gpu-layers` is. A dictation load during training could therefore still take VRAM. The flag is appended in the training path, and `_training_active()` is read once so the two decisions cannot disagree. The cached-file lookup called `hf_hub_download` with no cache_dir while the download worker spawns with `get_hf_cache_paths()`. A cache relocated in Studio settings was written in the new place and looked for in the old one, so a finished multi-gigabyte download read as missing and prompted again. Both the lookup and the progress readout go through the active cache now, which is what the Transformers path already did. Unload during a startup only reaped a `_process` that had not been assigned yet, so it returned success while llama-server kept booting and then published itself. Pressing Unload, or switching engines, left the model resident moments later. Unload cancels a pending load and waits for it to settle first, outside `_lock` so it cannot invert with the startup's `_start_lock`. 17 tests in this file now, including the exact argv for a training load and a relocated cache. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep dictation on the configured cache, the GPU, and its own server Sixth pass. Three findings, all confirmed. The whisper.cpp sidecar had the same relocated-cache gap the mtmd one did: `_cached_model_path` called `hf_hub_download` with no cache_dir and `_incomplete_bytes` read the import-time `HF_HUB_CACHE`, while the worker spawns with `get_hf_cache_paths()`. A Whisper GGUF could finish downloading in the configured cache and still be reported missing. Both go through the active cache now, as the other two sidecars already do. A server started during training kept `-ngl 0 --no-mmproj-offload` forever: the reuse check matched on model id alone, so dictation after the run stayed on CPU until the keep-alive expired. The offload mode is part of that check now. Deliberately not at any cost: a restart purely to pick the GPU back up is an optimisation, so an in-flight transcription keeps the server it has and the next idle load does the upgrade. `transcribe` also reserved its request slot after `load()` returned, so another client could switch the singleton in between and the port read under the lock was that other server's. The model is confirmed while the slot is taken, and a mismatch is refused rather than transcribed on the wrong model. 20 tests in this file now, including the CPU-to-GPU upgrade, the in-flight request that outranks it, and the swapped-server case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: answer a busy dictation switch with a retry, not a 500 Correct. `SttModelBusyError` was mapped on `/audio/stt/load` but not on the transcribe routes, and both places that raise it are reachable from there: the model-switch race in `transcribe()` and the busy guard inside the load it performs. A second client switching the Qwen model therefore surfaced as a server failure rather than something the caller should just try again. Test drives the route and asserts the 409, and fails without the catch. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: let dictation run on CPU during training, and type its errors properly Seventh pass. Three findings, all confirmed. The mtmd sidecar contradicted itself: `load()` starts llama-server at `-ngl 0 --no-mmproj-offload` while a run is active, and `transcribe()` then refused every request anyway. So the preload succeeded, recording started, and each segment came back 501, which the frontend reports as an installation problem. The guard is gone. Neither of the other two sidecars has one: whisper.cpp adds `--no-gpu` and Transformers picks CPU, and both transcribe normally. A startup cancelled by `free_stt_model_for_training` raised `SttUnavailableError`, which the route maps to 501, so ordinary preemption read as a missing or broken runtime. It raises `SttLoadCancelledError` now, the 409 the other sidecars already use for this. On the frontend, only `loadSttModel` turned a "not downloaded" 409 into `SttModelNotDownloadedError`; `transcribeAudioBlob` threw a plain Error. A very short first-run recording can finish before the fire-and-forget preload rejects, and that segment failure then latched the error reporter, so the download prompt never opened and the preload's own rejection was swallowed. Both calls share one mapping now, in `stt-errors.ts` so they cannot drift. 23 backend tests in this file, 3 frontend over the mapping. Every STT and VRAM suite passes (870) apart from `test_whisper_plan_eligible_when_behind`, which fails on main too. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: decide mtmd offload after the reap, and track STT downloads per model _load_locked() read _training_active() before releasing the previous server. _release_locked() reaps that server, which can take seconds, and training admission that already ran its own check cannot come back to cancel the load, so the flags could still spawn llama-server at -ngl 99 onto VRAM a training run had just claimed. Read training last instead, after _loading is published: one order is covered by the cancel, the other by the re-read. The download mirror also tracked a single model, but each STT engine owns its own download state, so a Qwen transfer and a Whisper one really do run at once. Starting the second marked the first "cancelled" in the panel without cancelling it, leaving a multi-GB transfer running with no row and no stop control. Trackers are keyed by model now. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-04 23:29:50 -07:00
Michael Han	d5cf96d628	Studio: add local speech-to-text dictation engine (#7095) ... * Studio: add Voice settings tab (dictation, dictionary, read aloud) New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR. * Studio: drop the single option STT engine select, rename TTS option The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer. * Studio: harden Voice settings against edge cases found in simulation Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery. * Studio: address Voice settings review feedback Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior. * Studio: use the chat mic icon in Voice settings for consistency The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere. * Studio: address second round of Voice settings review feedback Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race. * Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item * Studio: guard dictation mic lifecycle in Voice test and Compare composer Release a microphone opened after the component unmounts, and stop Compare dictation on a permission or security failure instead of silently recording from the default device, matching the main chat adapter. * Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings - Join final dictation chunks with a space so recorded transcripts do not merge words - Ignore a stale recognizer onend so a quick stop then restart is not torn down - Use previewingRef so a double click on TTS preview does not orphan the first request - Keep the read-aloud stop control visible when a new run starts while a message is spoken - Stop the dictionary remove button from deleting an adjacent entry on a blur then click race * Studio: trim redundant Voice settings comments * Studio: fix Voice preview and Compare dictation edge cases - Only cancel the shared speechSynthesis for a system-voice preview, so stopping a Studio preview no longer stops an unrelated chat read-aloud - Release the Studio preview audio and its WAV data URL on normal completion - Iterate every finalized result in Compare dictation so batched phrases are kept - Cap persisted recent dictations to the last 20 on hydration * Studio: use clipboard fallback for recents and release failed preview audio - Copy recent dictations via the copyToClipboard helper so the execCommand fallback works in Safari and insecure http LAN contexts - Release the Studio preview audio when play() rejects, not just on ended/error * Studio: add local speech-to-text dictation engine Add an offline dictation engine that transcribes with a local faster-whisper model, alongside the existing browser (Web Speech) engine. The browser engine streams audio to Apple or Google speech services and needs internet; the new engine runs on the server, works offline, and drives any chat model without evicting it (it loads in the backend process, separate from the model subprocess). It also gives Firefox dictation, which has no Web Speech support. Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper is torch-free, so this does not disturb the existing model stack. Frontend: a Dictation engine setting (browser or local model), a curated model picker with sizes, and MediaRecorder capture posted to the transcribe route. The model warms automatically when the engine is selected, with live status. * Studio: stream local STT transcription as you speak Local dictation showed nothing until you stopped, because the whole clip was transcribed once on stop. Now the growing recording is re-transcribed on a fast pass every second and emitted as live interim text, with an accurate final pass on stop. Partial recordings decode fine, and the model refines earlier words as more audio arrives. Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast preview pass; the final stop uses the accurate path. * Studio: make local dictation stop instant and reliable Stopping local dictation waited for a final network transcription before the session ended, so the stop button did not flip and a second click ended the session early and dropped the text. Now stop commits the live transcript immediately, releases the mic at once, and ignores a second stop while finalizing. Previews run more often so the committed text is current. * Studio: record local dictation in short clips for reliable streaming Re-transcribing a growing buffer every second got slower as it grew, flooded the backend, showed stale words, and could leave the stop button stuck waiting on a backlog. Record short independent clips instead and transcribe each once, appending the text as you speak. Work per clip is bounded, so stopping is prompt (with a hard timeout as a safety net) and long dictations stay smooth. * Studio: dictate then transcribe once on stop, ChatGPT style Local STT dictation streamed by re-transcribing the growing clip, which was quadratic and saturated the backend (multi-second lag), and stop only halted the recorder without releasing the mic, so it kept recording. Record the microphone continuously, release it the instant the user stops, and transcribe the whole clip once. Stopping is immediate and the transcript lands in about a second. Also add the tiny model for the fastest option. * Studio: surface dictation and read-aloud failures instead of failing silently - Compare dictation reports microphone and speech-recognition errors via toast, reusing the main chat adapter's describeMediaError and describeSpeechError - Read-aloud toasts genuine model or synthesis failures while ignoring cancellations * Studio: ChatGPT-style recording bar for dictation Clicking the mic now drops the composer into a dedicated recording bar with a live waveform, a discard (X) and a confirm (tick), instead of a plain stop button. The tick stops recording and transcribes the clip; the X throws the recording away and keeps whatever text was already in the composer. The model adapter taps the mic with an analyser to drive the waveform, and the router tracks the live session so the X can cancel it without transcribing. * Studio: transcribe dictation while speaking, ChatGPT layout Match ChatGPT's recording layout: the bar now renders in place of the input with the left plus button kept, the waveform in the middle, and the discard and confirm buttons together on the right. Cut the post-confirm delay by transcribing in the background as the user talks. The audio is split at natural pauses (voice-activity detection off the same analyser that drives the waveform) and each clip is transcribed as it is cut, so confirming only has to finish the short final tail. The model is also warmed when recording starts so the first run never pays a cold load. * Studio: ChatGPT waveform, hide tools while dictating, faster STT Make the recording UI read like ChatGPT: the waveform is now a dense row of round dots that rise into thin centered bars, and while dictating only the plus button shows, with the mode badge and tool toggles hidden so the bar is just the waveform and controls. Speed up transcription: decode greedily (beam_size=1), which is several times faster on CPU with negligible accuracy loss on short dictation clips, and cap background segments at 6s so the final tail after confirm stays short. * Studio: finish ChatGPT voice bar and low-latency STT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: full-width waveform with a timer that freezes on stop Use the full-width waveform for the recording bar: brighter, bigger bars that advance on a fixed cadence (keeping peaks between advances) so they glide instead of racing by, inset from the composer edges. Keep a visible timer and the green confirm button, matching the ChatGPT reference, and freeze the timer and waveform the moment the user confirms. * Studio: fix multilingual local dictation * Studio: speed up dictation and release local STT * Studio: harden dictation finalization and STT decoding * Studio: restore Firefox dictation fallback * Studio: add dictation history manager * Studio: manage speech model downloads * Studio: remove em dash from voice model label * Studio: move dictation history into Voice * Studio: source local STT from Unsloth Whisper models Point the dictation STT sidecar and its Model Hub download entries at Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3) and run them through Transformers, so Studio only ever downloads Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs repos; keep the Model Hub as the only download path via local_files_only, and keep PyAV for audio decoding. Device selection uses float16 on CUDA and float32 on MPS and CPU, since Whisper's decoder is unstable in float16 on MPS and repeats tokens. Shorten the model picker labels to name plus download size and update the STT tests for the new backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: smooth dictation waveform and keep pill height * Studio: align STT model dropdown width and tidy voice copy * Studio: guide to local engine when browser dictation is offline * Studio: clarify voice section and STT model copy * Studio: keep STT warm with training-aware eviction * Harden STT lifecycle and browser compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model discovery test lint * Harden cross-browser microphone errors * Harden cross-browser microphone errors * Surface voice test recognition errors and fall back to Studio TTS - Voice test now toasts non-abort speech-recognition failures instead of ending silently, matching the main and Compare dictation paths. - Read-aloud routes to the backend model when the runtime lacks Web Speech synthesis (audio-only WebView), so it no longer errors immediately. * Fix reviewed STT lifecycle races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix read-aloud fallback controls * Guard read-aloud stop when deleting a non-speaking message aui.message().stopSpeaking() throws unless this message is the one being read aloud, so calling it unconditionally rejected the delete handler before the message was removed. Only stop speech when this message is speaking. * Cap recent dictation transcript length before persisting Recent dictations only limited entry count, so a long transcript stored the full text in the persisted voice settings and a few could exceed the localStorage quota, throwing synchronously from the uncaught dictation cleanup path. Truncate each entry on save and on hydration, matching the dictionary cap. * Studio: keep dictation mic clickable and guide to local model Register the dictation adapter unconditionally so the mic stays enabled for any engine and starts working right after switching to the local model on an already-open thread. When the browser engine cannot run (Firefox, Brave, non-secure origins), clicking the mic shows a toast that points to the local speech-to-text model instead of leaving a disabled button. The toast stacks its action below the text with a fully rounded button. * Studio: add bottom padding below the dictation guidance toast button * Studio: increase bottom padding under the dictation toast button * Studio: add bottom padding inside the dictation toast button * Studio: add five Whisper defaults and custom model search Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end. Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary. Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use public Unsloth Whisper repositories Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests. * Studio: update Whisper download sizes Reflect the cleaned public Tiny and Base repositories in the curated model labels. * Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes - Show the download size on the right of each model row so long names like Whisper Large v3 Turbo no longer hide it - Update curated Whisper sizes to the safetensors weights actually downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB - Drive the model list scroll from a wheel handler so the mouse wheel scrolls it inside the Settings dialog, not just the scrollbar - Add a search icon and shorten the placeholder to Search model * Studio: do not search when a dictation model is picked, shrink repo label - Treat the filled-in model text as a selection, not a query, so choosing a model no longer kicks off a Hugging Face search - Make the repository line under each model name smaller * Studio: tighten dictation model and local engine descriptions * Studio: keep model display on pick instead of the query, shrink row text - Guard the combobox input so selecting a model shows its name and does not echo the typed query back or start a search - Map the item label to the friendly display so picks fill the field - Reduce the model name and size text in each row * Studio: show only the model name in the dictation field, shrink size label - Drop the download size from the search field; the name alone is shown once a model is selected, with sizes kept in the dropdown list - Reduce the size label text in each row * Studio: clarify the dictation model description * Studio: drop Hugging Face from the dictation model description * Studio: move the dictation dictionary to its own Manage subpage - Replace the inline entry list with a Manage row, matching Dictation history, so a long dictionary no longer crowds Voice settings - Add a DictationDictionaryView subpage that holds the entry editor * Studio: match STT field font, use best voice for System default - Bump the dictation model field text to text-sm so it matches the engine dropdown next to it - Resolve the System default read-aloud voice to the top curated voice instead of the browser default, which is a robotic legacy voice on macOS * Studio: rerank read-aloud voices and drop duplicate voice entries - Rank by vendor quality, then the user's locale, then a preferred list of natural voices, so the best voice leads instead of the first alphabetically - Collapse voices that macOS reports twice under one name and language * Studio: fold dictionary and recents into the dictation section - Drop the separate Dictation dictionary and Recent dictations headings; their Manage rows now sit under Dictation, split by the row divider - Shorten the custom spellings description * Studio: add search and sort to dictation history - Filter saved dictations by text with a search field - Sort by newest, oldest, or A to Z; show a no-matches message - Keep Clear all available regardless of the current filter * Studio: settle cancelled STT loads before training and fix dictation review items Wait for a cancelled STT load to exit and release its memory before reporting it freed for training, so the loader cannot still be inside from_pretrained()/.to(device) holding VRAM when the training subprocess starts. A load that finishes before observing the cancel now gets unloaded so the memory is actually reclaimed. Clear the accelerator cache before the CPU fallback in load() so a failed CUDA/MPS load does not strand reserved VRAM once the sidecar is marked CPU-resident. Send the saved Hugging Face token when polling STT download progress so a gated or private repo resolves and shows the correct Load/Downloaded state instead of reporting missing. Mark the composer Dictate button as type="button" so clicking it does not also submit the draft when the composer already has text or attachments. * Studio: pin dictation settings per session and close STT startup races Capture the STT model and language when a dictation session starts and pass them to every queued segment and the warm-up load, so changing the model or language mid-recording no longer transcribes the same clip with the wrong model or a model that is not downloaded. Check the local runtime at the top of transcribe(), before the model cache lookup and the bounded audio decode, so a server missing PyTorch or Transformers returns 501 up front instead of decoding a long clip first. Treat the training startup window as active for STT device selection. start_training frees VRAM in before_spawn but only assigns _proc later, so a concurrent STT load could take the GPU that was just cleared. A startup flag now reports training active from the free until the process is live, forcing those loads to CPU; a finally clears it on every exit. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stub the STT runtime check in transcribe orchestration tests transcribe() now verifies the local runtime up front, so the unit tests that exercise transcription orchestration must treat the runtime as present to keep passing where PyTorch, Transformers, and PyAV are not installed. Stub ensure_stt_available in the shared fixture and restore the real check in the availability and load-rejection tests. * Harden custom Whisper dictation models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add whisper.cpp dictation engine with per-engine downloads and history rework Engines - New GGML STT sidecar that runs a managed whisper-server subprocess with idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh) - Dictation engine picker now offers Browser, Local transcription (whisper.cpp), and Local transcription (Transformers) - Both local engines serve the same five curated Whisper models and download them directly with byte-level progress reported by /audio/stt/status - Models auto load on selection and when their download finishes - Unload and training admission account for both engines Benchmarks (Apple Silicon, greedy, warm, same checkpoints) - whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in about 0.45s vs 0.86s for Whisper Small - whisper.cpp GGUF path is unchanged by the Transformers addition (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s) Voice settings UI - Plain curated model select replaces the searchable combobox - Single download progress bar with transfer rate for both engines - Dictation history now stores every dictation with Show more pagination, a top Clear history action, and links back to the chat it was spoken into - Archived chats dialog gets the same pagination - Delete dialog offers deleting a dictation together with its chat Tests: 88 backend STT tests pass, including new snapshot download coverage. Frontend typecheck, lint, i18n parity, and production build pass. * Merge local engines into one option and source GGML models from unslothai Engine selection - The dictation engine dropdown is back to two choices: Browser and Local transcription. The selected model decides the backend: curated ids run GGML checkpoints through whisper.cpp, searched Hugging Face repositories run safetensors through Transformers - Model picker lists the curated models and searches Hugging Face for other Whisper repositories, validating them before selection. The trigger is a plain button so the selection never renders inside a text input - /audio/stt/status accepts a model query param so downloaded state works for custom repositories; the engine param on load, transcribe, and download routes is derived from the model everywhere Model source - Curated GGML checkpoints now download from the Unsloth-hosted unslothai/whisper-*-GGUF repositories (one repo per model) instead of ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob tracking are per-model Fixes - Voice settings and dictation history were not persisting: the quota-safe localStorage wrapper was declared after the store that uses it, so the persist storage factory failed silently. Every settings write also threw mid-click, which kept the model picker popover from closing on selection - is_model_downloaded now verifies config, preprocessor config, and real weight files instead of trusting an offline snapshot lookup, so a partial download left by an aborted fetch shows the Download button instead of failing to load - Removed whisper.cpp mentions from user-facing text: the ready status shows Loaded instead of the runtime name, picker rows show the source repository, and runtime error messages say local transcription runtime Verified with automated browser sessions and live API checks: selection closes the picker with no page errors, persisted settings hydrate on reload, a stale partial snapshot triggers download then loads on MPS and transcribes, and curated models download from the unslothai repos. 88 backend STT tests, typecheck, lint, i18n parity, and build pass. * Skip the duplicate source line for custom models in the STT picker A custom repository's display name is its id, so search results and the appended current selection rendered the same string twice. The source line now only renders when it differs from the name; curated rows keep their name, unslothai source repository, and download size. * Verify every shard of a sharded checkpoint in the downloaded check A snapshot holding one of N shards (or a corrupt shard index) passed the downloaded check and then failed at load. When model.safetensors.index.json exists, every shard in its weight map must now be present. Found by simulation; covered by a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename stale _starting references in the pump resilience tests The startup flag on TrainingBackend was renamed to _spawn_in_progress but two tests added alongside it still asserted on the old name, failing the Python 3.11 to 3.13 CI jobs. * Make the selected model row clearly highlighted in the STT picker The current selection was a faint background tint. It now uses the accent background with a medium weight name. Two line rows use a small corner radius; single line custom repo rows keep the pill shape. * Address review feedback on STT snapshot checks, VRAM release, and dictation UX Verify snapshot completeness in the load preflight so a partial download fails before the audio is decoded, for curated and custom repos alike. Drop the failed accelerator traceback before the CPU retry so the cache clear can actually release that memory. Keep unloading the GGUF sidecar after cancelling an in-flight Transformers load; both engines can hold memory at once. Allow Auto language with English-only .en checkpoints, matching the backend which sends no forced language. Keep the discard button usable while a transcription is pending so a slow or hung request cannot trap the composer in dictation mode. Stop linking Compare and settings test dictations to the unrelated active single chat thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move the CPU retry out of the exception handler On Python 3.10 the interpreter exception state keeps its own reference to the traceback, so dropping it from the caught exception was not enough to release the failed accelerator load during the retry. Leaving the handler before clearing the cache works on every supported version. * Address review feedback on session handoff, chat pinning, and server lifetime Starting a dictation from a second entry point now cancels the session it replaces, so the old recording cannot keep the microphone open or save a transcript with no discard button pointing at it. The linked chat is pinned when recording starts, so switching threads while a transcription finalizes cannot relink the transcript to the newly opened chat. whisper-server is now bound to Studio's lifetime like the other long-lived children: PDEATHSIG on Linux, the parent job object on Windows, and pid adoption so the shutdown sweep reaps it; before this it survived a Ctrl+C exit as an orphan still holding the model. * Remove the dictation mic test from Voice settings The composer dictate button covers the same check, so the test row, its transcript panel, the unsupported fallback row, and their strings and search entry are gone. * Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits GGUF (whisper.cpp) sidecar: - Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed. - Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind. - Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription. - Reject a missing model before decoding audio, matching the Transformers download preflight. Voice settings: - The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it. Dictation dictionary: - Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fix curated GGUF whisper filenames to match hosted repos The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin, not ggml-<id>.bin, so every curated dictation download and cached-path lookup 404'd and the whisper.cpp engine could never load a model. Point GGML_STT_MODELS at the real filenames and guard the naming with a test. * Studio STT: validate a custom dictation repo before downloading it The Transformers STT engine accepts an arbitrary owner/model repo, but the download route handed it straight to snapshot_download, pulling a possibly large non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper checkpoint first with the existing metadata-only validate_remote_model (no weights); curated ids short-circuit and the GGUF engine (curated-only) is unaffected. A non-Whisper repo now 422s before any download. * Studio STT: preempt a still-loading GGUF server for training admission A whisper-server still in its startup window binds accelerator memory but has no loaded_model yet, so training admission could miss it and launch into an OOM. Make the GGUF startup cancellable (cancel_pending_load signals an abort event and terminates the starting process without the load lock; _wait_for_server observes it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock until the killed server is reaped), and always fold the GGUF sidecar into the resident-STT summary so a resident Transformers model cannot mask a loading GGUF server. free_stt_model_for_training now cancels an in-flight load and waits for it to settle before training claims the memory. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fall back to Transformers when whisper-server is absent A curated dictation model (including the default small) hard-pinned the GGUF engine, but standard installs do not ship whisper-server, so every recording 501'd instead of using the Transformers engine that serves the same checkpoint -- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine: a GGUF request for a curated id (the only ids GGUF accepts, all Transformers- servable) downgrades to Transformers when whisper-server is unavailable, applied consistently to download, load and transcribe (not unload, which targets a specific engine). The Voice tab likewise falls back to the Transformers status so the model is not shown unavailable and download is not blocked. * Studio STT: hide custom Whisper caches from the legacy model pickers The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with only the owner/model id, which cannot reach the config-based Whisper check, so a downloaded custom (non-curated) Whisper checkpoint was still offered as a chat model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo config and hides it, matching the discovery route. * Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction - Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat model inventory and pickers, backend and frontend. Only their Transformers safetensors companions were hidden; the GGUF repos use a different org and a -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked into chat pickers. - Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the Transformers sidecar. transcribe() holds self._lock across the whole inference call, so /audio/stt status polls and training admission previously blocked behind an in-flight transcription. - stt_unload resolves through the serving resolver: a "gguf" pick on a host without whisper-server is served by the Transformers fallback, so unload must target that engine or the resident model is never freed. Unload also attempts every engine even if one raises, so a failure freeing one backend no longer skips the other. - free_stt_model_for_training frees the Transformers and GGUF sidecars under independent exception boundaries so a failure unloading one no longer skips the other before training claims the memory. Adds tests/test_stt_review_fixes.py covering all four. * Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness - The model dictation adapter sent the raw setting (the literal "auto") to the backend, while the browser engine resolves Auto via resolveDictationLanguage. A batch of non-English voice notes came back mostly English on Auto. Add resolveModelDictationLanguage: only the literal "auto" is resolved to a concrete locale, gated so it becomes a language the model AND Whisper can honor (mirroring the backend's known-whisper-languages set); an explicit language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire it into both adapter call sites. - GgmlSttSidecar._process_alive() read self._process twice; a concurrent unload() nulls it under the lock while loaded_model/device read lock-free, so a null between the two reads called None.poll(). Snapshot once. Adds a deterministic regression test. * studio: tighten comments and docstrings in the dictation modules * studio: harden dictation model downloads, GGML readiness, and recording paths Address review findings on the STT dictation feature: - build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom Studio home unless it carries the Studio ownership marker, matching the setup.sh policy, and marks trees it creates - _snapshot_is_complete validates every shard of a sharded PyTorch (pytorch_model.bin.index.json) checkpoint like the safetensors path, and requires tokenizer assets (tokenizer.json or vocab.json + merges.txt) - custom-repo downloads pin the revision resolved at validation time and restrict snapshot_download to the model/tokenizer/config/preprocessor file classes Studio loads - the GGML sidecar holds its port reservation until just before spawning whisper-server and only accepts readiness from a responder that both looks like whisper.cpp's server and belongs to the still-running managed child, probing twice, so mic audio cannot be posted to a foreign local process - the recording adapter transcribes every non-empty segment; the RMS meter only shapes segment boundaries and can no longer discard quiet speech - Compare-pane dictation can cancel a pending transcription on second click, with the button relabeled while finalizing - localStorage quota recovery halves the dictation history until the save fits, so small histories shrink too - the System default TTS voice resolves to the platform default voice - new dictation UI imports go through the chat and hub feature barrels Regression tests cover the build-script gate, sharded PyTorch and tokenizer completeness, revision pinning and allow patterns, and the whisper-server readiness probe. * Fix STT download and voice picker follow-ups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add dictation button regression coverage * Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294) * Studio STT: add prebuilt whisper.cpp (whisper-server) installer New install_whisper_prebuilt.py downloads a per-platform whisper-server bundle published by the unslothai/whisper.cpp prebuilt CI into the managed whisper.cpp dir (build/bin/whisper-server) so local dictation needs no compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py: host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the trust anchor, staging + install lock + atomic swap, traversal-safe extract, co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired into setup yet; the pins ship empty so every asset fails closed until the first fork release is published and its digests are reviewed in. * Studio STT: install prebuilt whisper.cpp during setup and update Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so `unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server into the managed whisper.cpp dir the sidecar discovers. It skips a user-set WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL, forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence. * Studio STT: harden whisper-server child env + WSL ROCm detection - Sidecar spawns whisper-server with a scrubbed child env that prepends the binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child. - find_whisper_server_binary now requires an executable, not just a file. - Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only; gfx parsing skips the gfx000 CPU agent and generic ISA lines. - Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the executable check, and the WSL rocm detection. * Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can detect and install a newer whisper-server release from inside the app: - backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json and compare the installed release against the newest unslothai/whisper.cpp release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a (major, minor, patch, serial) key with a strict downgrade guard; 24h cache; fail-open. - backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch and atomically swap the newest bundle, unloading the warm GGUF sidecar first. - backend/routes/whisper.py mounted at /api/whisper (update-status + update). - pyproject: add whisper_prebuilt_pins.json to studio package-data so the installer's trust anchor ships in the wheel (it is a data file, not a .py module, so package discovery alone does not include it; node_prebuilt_pins.json is listed for the same reason). Without this a pip-installed wheel had no pins and the prebuilt install aborted to Transformers STT. Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade guard, marker layouts, stale decision, fail-open). * Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust model: instead of a committed whisper_prebuilt_pins.json, verify every download against the release's own whisper-prebuilt-sha256.json checksum index, fetched from the same GitHub release. - parse_release_checksums / fetch_release_checksums / expected_sha256_for replace the pins layer. The index is validated for schema/component and that its release_tag matches the resolved release; an asset absent from it, a release that does not publish it, or a manifest sha256 that disagrees with it all fail closed to a source build. - resolve_release_tag now resolves the newest published release at runtime (or an explicit --published-release-tag), matching llama and the freshness check; removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in. - Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data entry (nothing to ship now, same as llama which has no committed pins). - Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on uncovered asset, tampered-manifest guard, newest-release resolution). This is a same-origin checksum (integrity, not authenticity), identical to the llama.cpp installer; pair releases with GitHub artifact attestations for provenance. * Resolve whisper prebuilt release via the download host (no GitHub API) Mirror install_llama_prebuilt.py's fast path: resolve the release tag from the releases/latest redirect and fetch the manifest + checksum index from constructed releases/download URLs, so the common install path makes zero api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour per IP; the download host is not). Fall back to the GitHub API only on a 404, malformed asset, or tag mismatch. * Studio STT: coverage-aware whisper prebuilt selection via a shared core whisper's select_artifact returned the first os/arch/backend manifest match and ignored the SM-coverage fields the release manifest already carries, so a Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks cuda13-newer. Extract the coverage-aware selection into a shared, component-agnostic core under studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and generalised over a normalised artifact. whisper's HostInfo now records the GPU compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and select_artifact routes CUDA/ROCm through the shared selector: every visible SM must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes, and "already matches" contract are unchanged. On the B200 the installer now resolves cuda13-newer, matching llama. * Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship libcudart/libcublas -- they load the same runtime the host already has. So the driver's advertised CUDA version is only an upper bound: a cuda13 bundle still needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the shared core and intersect it with the driver-compatible lines in select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g. torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13 one; a host with no CUDA runtime at all falls back to CPU. Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator truthiness, not a match) that made every major report present; add a real filesystem test that exercises the scan. * studio: harden shared prebuilt core to full llama parity Apply the review findings on the shared coverage-aware prebuilt-consumer core so whisper.cpp selection is exactly equivalent to the llama.cpp path. hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an index/UUID selector now reports has_usable_nvidia False instead of staying usable, via supports_explicit_visible_device_matching plus the physical / explicit-match branches, and _select_visible_rows now matches rows the way llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus fallback and has_physical_nvidia. Adds parse_macos_version. runtime_libs.py: the Linux on-disk scan now requires the exact libcudart / libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare versioned file without the SONAME symlink no longer counts as loadable. Hardens the ldconfig parse against an empty left-hand side. selection.py: fix the Blackwell/torch reordering so it keys on the covering runtime lines (falls through to the torch preference when the covering lines were filtered out), matching linux_cuda_choice_from_release. Corrects the compatible_runtime_lines_for_driver docstring: the bundles do not ship the CUDA runtime, so the driver version is only an upper bound and the caller must intersect with the on-disk scan. install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new HostInfo.macos_version) so a bundle that cannot load on the host OS version is dropped. Keep resolver stdout to only the JSON line by leaving logs on stderr in --resolve-prebuilt mode, and map an unexpected probe failure to prebuilt_available False instead of a traceback. Tests: new host-probe suite for the visible-device logic, exact-SONAME runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON, exit-code mapping, and the repo key. * studio: fix whisper prebuilt selection + launch parity gaps from review A parallel review surfaced integration defects where the whisper path could select or launch a bundle that cannot run on a concrete host. Each is fixed to match install_llama_prebuilt.py. macOS min_os: the manifest labels macOS requirements as macos-<version> (e.g. macos-14.0), which the version parser could not read, so the guard was a no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the platform prefix before parsing. ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact ROCm matching treats that token as the active GPU, a mixed APU + dGPU host (gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU sections and honors the visibility vars (empty / -1 -> no AMD GPU). --rocm-gfx override: recording the arch without setting has_rocm left the host on its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies has_rocm and clears NVIDIA state, like llama's _apply_host_overrides. CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so on a host whose CUDA runtime lives only in the PyTorch wheels the selection would gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch runtime dirs to the child loader path for CUDA bundles (bundle dir still first), mirroring binary_env. Also normalize a manifest artifact's supported_sms defensively (parity with llama's parser) and document that blackwell_min_toolkit_for_caps is retained for the Phase B llama Windows path. Not changed (verified parity, not defects): Linux/Windows min_os is enforced nowhere in llama (macOS only); the resolver is optimistic about the checksum index and the install path verifies. * studio: tighten prebuilt-core code comments * studio: lift shared prebuilt installer core out of the whisper installer * studio: reuse the llama.cpp prebuilt installer machinery for whisper * studio: unify llama and whisper prebuilt installers on a shared descriptor core * studio: consolidate prebuilt installer tests into the shared core suite Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every component-agnostic behavior runs against both descriptors: the full seven profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle stability, missing SM metadata, dotted SM normalization, no-driver fallback policy), the ROCm gfx family matrix, macOS min_os gating and its helper, backend resolution incl. cpu-fallback precedence and Intel-mac auto detect, checksum-index non-object and plain-lookup cases, the tar symlink/hardlink extraction guards moved from the llama suite, and the compute-cap, visible device, runtime-line and Blackwell helper value tables moved verbatim from the llama characterization suites. Delete only tests whose exact behavior the master now asserts for the same component: 40 pure-alias helper cases in test_selection_logic.py (replaced by value-identical master tables plus an alias-identity pin), 6 extraction moves and the master-absorbed zip-symlink case in the llama logic suite, 3 routing twins in test_rocm_support.py already pinned byte-for-byte in test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by the master whisper parameterization. Wrapper wiring pins, the llama release plan dialect, fingerprints and every llama-only behavior stay untouched. * studio: dedupe sidecar and update helpers into the backend prebuilt package * studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow * studio: consume paired slim whisper prebuilts via the llama ggml runtime * studio: serve every whisper backend from slim prebuilts * studio: drop the whisper fat per-accelerator selection chain unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan selection glue; keep slim selection + pairing, link_ggml_runtime, and one legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim release. Exit 2 now reads as prebuilt unavailable (whisper never source builds); setup already treats it that way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire libomp runtime DLL alongside ggml in slim whisper installs llama's clang-built windows-arm64 ggml-base.dll imports libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL. Without it next to whisper-server.exe the loader fails with STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64 was affected. The empty-runtime guard still requires a real ggml library; libomp alone is not a pairing. * studio: drop whisper-side fat-selection support structure Slim whisper bundles are selected per os/arch only; all accelerator capability comes from the installed llama.cpp prebuilt, whose installer already did the coverage-aware selection. Remove the machinery that only existed to pick among fat per-accelerator whisper bundles: - prebuilt_core: delete the generic CUDA/ROCm coverage selection (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters, detected_cuda_runtime_lines, the exact-SONAME linux probe) that no shipped component routes through; llama keeps its own selection chain and whisper shadows select_artifact with the slim-only version. select_artifact is now a plain os/arch/backend first-match. - install_whisper_prebuilt: drop the HostInfo CUDA fields (compute_caps, driver_cuda_version, torch_runtime_line) and the torch runtime probe that populated them; nothing reachable reads them, and the resolver payload sources runtime_line from the artifact. - whisper_cpp_update: delete the standalone start_update job worker; whisper applies only run as the chained phase of the combined llama+whisper update. The status payload keeps its job field (idle). - routes/whisper: drop the progress logger that could never fire. - tests: remove tests of the deleted paths and tests duplicating the descriptor-parameterized core suite or the llama freshness suite. Contracts unchanged: resolver JSON keys, exit codes, marker fields, pairing logs, and the pinned pre-slim fat CPU escape hatch. * Address review feedback on the whisper prebuilt update and install paths - Pin the chained whisper phase to the release the freshness check offered, so the download-host latest pointer cannot reinstall an older build in a loop - Wire the whisper prebuilt install into setup.ps1 (Windows setup previously skipped it entirely) - Treat a non-executable server or missing wired ggml libraries as a broken install instead of reporting already matches - Keep whisper sidecar reloads out of the job-level reload flag and resync chat state after a partial chained update that unloaded llama - Repoint home and profile vars for the whisper-server subprocess at a managed scratch dir and drop credential-store pointers - Clear the prebuilt marker before the opt-in source build overwrite - Write the prebuilt marker with explicit utf-8 encoding * Tighten comments in the whisper prebuilt consumer * Harden the Windows whisper setup phase and the chained update edges - setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH / UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard before the atomic install, and forward the release-tag pin and ROCm hints like setup.sh - sidecar: a cpu-selected install launches whisper-server with --no-gpu (slim wiring links every llama backend, so the flag is what keeps a deliberate CPU choice off the GPU) - chained update: leave whisper unpinned on macOS (the llama phase can walk back there, and a newest-tag pin could be an impossible pairing on every retry) and treat installer exit 2 as kept-existing-runtime instead of failing the combined job - job.to_tag now comes only from the llama phase, so a whisper-only round cannot report a llama update that never ran * Fix slim whisper runtime follow-ups * Address remaining whisper update reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address remaining prebuilt update reviews * Fix remaining chained update reviews * Fix remaining whisper runtime review edges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2026-07-23 01:39:03 -07:00
Michael Han	74d1a284eb	Studio: hide the RAG embedder and llama.cpp probe from the hub cached inventory (#7018) ... * Studio: hide infra models from the hub cached inventory The hub inventory scans behind /api/hub/cached-gguf and /api/hub/cached-models returned the llama.cpp install validation probe (ggml-org/models) and the RAG embedder (unsloth/bge-small-en-v1.5[-GGUF]) as on-device models. Share the hidden-model check from routes/models.py via utils/models/hidden_models.py and apply it in both scans. A GGUF infra repo stays visible when the user explicitly downloaded a variant through the Hub, since variant manifests only exist for user-initiated downloads. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make On Device trust the hub inventory, match repo ids exactly, lighten the hidden-model import Follow-up on the hub cached-inventory hidden-model change, addressing the review. On Device now trusts the Hub inventory API for cached rows. The backend already hides the RAG embedder and the llama.cpp probe and re-includes a GGUF infra repo once the user downloads a variant through the Hub, but the frontend was re-hiding it by repo id, so the user-downloaded variant never appeared in the On Device list or the count. isVisibleInventoryRow now short-circuits cached rows (kind === "cache") to visible and keeps client-side needle hiding only for local filesystem rows and Discover. is_hidden_model matches Hub repo ids exactly (case-insensitive) against the probe plus the effective embedder and its GGUF companion, instead of substring matching the configured-embedder basename. A custom embedder with a generic basename like org/model no longer hides unrelated cached repos such as user/model-chat or org/model-instruct. The probe filename and local-path embedders keep exact matching. The helper moves to utils/hidden_models.py and is imported at module scope in the hub cache scanner, so it no longer pulls in utils/models/__init__ (the eager model-config/checkpoint stack) and a broken import fails at startup instead of being swallowed per-repo and silently emptying the inventory. routes.models keeps the _is_hidden_model and _safe_resolve aliases and drops the unused _HF_REPO_ID_RE re-export that was failing source lint. Tests: exact repo-id matching with a custom embedder, the cached-models scan keeping an unrelated repo, and a clean-interpreter check that the helper imports without the model-config stack. * Studio: match the llama.cpp probe filename on both path separators The hidden-model check compared the probe's on-disk filename with Path(value).name, which on a POSIX interpreter does not split a Windows-style path ("...\stories260K.gguf") and would let the probe through. Split on both separators so the probe is matched regardless of which OS produced the path, matching the tolerance of the previous substring check. Adds a Windows-path assertion to the probe test. * Studio: harden hidden infra model handling * Fix hidden cache row confirmation * Fix hidden local rows and confirmed hint merges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Handle snapshot-configured hidden models * Hide basename-only default embedders * Fix dynamic embedder inventory filtering * Studio: hide the configured RAG embedder from Discover and feed rows --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothshared@gmail.com> Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>	2026-07-19 03:20:56 -07:00