fix(local): keep a ChatGPT plan out of speech settings, and docs that match v1

After dev's one-click ChatGPT sign-in, the local-only speech section must
still offer no remote model; a test now proves it with a plan that claims
the slot, and fails without the localOnly gate. The docs drop two stale
Known gaps (no speech screen; no transcription test), name the settings nav
as it reads, count migrations to 0030, mark steps 2 to 7 done in the
proposal, and note the curated recogniser's type in the catalog proposal.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-10-14 15:04:38 -07:00
parent d9c0a86d98
commit 531c1f6810
7 changed files with 35 additions and 12 deletions

View file

@ -687,7 +687,7 @@ on search and repo reads, and the job's generic error during an install.
## On the screen
Settings has one section per model type, **Chat**, **Image**, **Image editing**, **Video**, **Audio** and **Speech recognition** in the nav, each headed **Text generation models**, **Image generation models**, **Image editing models**, **Video generation models**, **Audio generation models** or **Speech recognition models** on its own page.
Settings has one section per model type, **Text gen**, **Image gen**, **Image edit**, **Video**, **Audio** and **Speech recognition** in the nav, each headed **Text generation models**, **Image generation models**, **Image editing models**, **Video generation models**, **Audio generation models** or **Speech recognition models** on its own page.
Each names the model in use at the top, then groups the slot's models by source:
**This computer**, every build on disk, curated or not, with Use (when its type
can fill the slot) or In use and Delete after confirmation; then one group per
@ -813,7 +813,6 @@ and the screen in `download-chat-models.test.tsx`, `install-view.test.tsx`, `ins
## Known gaps
- Only the three voice defaults and Nemotron's `Q8_0` are validated; `validated` is empty on every other build.
- No screen offers the speech recogniser yet: the Audio section drops rows without `voicing`, and Settings has no speech recognition section ([proposal](../../proposals/local-speech-to-text.md#catalog-selection-and-settings)).
- Qwen3.8 27B is a hybrid model (`qwen35`): the fit estimate charges a KV cache on all 65 layers where 16 keep one, about four times too much, and does not size its recurrent state ([runtime](runtime.md)).
- Qwen3.8 27B commits no `sampling`: its repo has no `params` file, so it runs on llama-server's defaults rather than the model card's.
- `template.system_role` and llama.cpp's `run.args` are committed but unread: chat asks the loaded template for its system role, and the router ignores per-model load arguments. sd.cpp's `image` defaults and `run.args` reach sd-server as launch flags. `template.tools` and `template.reasoning` reach a row's support, which the screen does not show.

View file

@ -114,7 +114,7 @@ The rules that keep the processes out of each other's way:
- [`backend/modules/`](../../surfsense_local/backend/modules/): one folder per feature, holding whichever of its models, schemas, router and service it needs: `workspaces`, `documents`, `chunks`, `chat`, `artifacts` (Studio), `events`, `llm` (runtimes, catalog, connections, selection), `egress`, `agent`, `plugins`, `license`, `migration` (import) and `health`. `shared.db.import_models()` imports every model at startup, because relationships name their targets as strings.
- [`backend/worker/`](../../surfsense_local/backend/worker/): `consumer.py` drains one queue per process, `jobs.py` holds status transitions and cancellation, `notify.py` the change notice, and `ingestion/` and `studio/` the two pipelines.
- [`backend/shared/`](../../surfsense_local/backend/shared/): configuration, the engine and its pragmas, `upgrade_to_head()`, the two Huey queues, `retrieve()`, and the secret that encrypts stored keys.
- [`backend/alembic/`](../../surfsense_local/backend/alembic/): revisions `0001` to `0028`, all hand-written. `env.py` has no `target_metadata`, so autogenerate cannot run by accident.
- [`backend/alembic/`](../../surfsense_local/backend/alembic/): revisions `0001` to `0030`, all hand-written. `env.py` has no `target_metadata`, so autogenerate cannot run by accident.
- [`frontend/src/features/`](../../surfsense_local/frontend/src/features/): `chat`, `agent`, `sources`, `studio`, `workspaces`, `dashboard`, `models`, `onboarding`, `settings`, `egress`, `plugins`, `license`, `migration`, `updates` and `feedback`.
- [`electron/src/main/`](../../surfsense_local/electron/src/main/): `index.ts` (the window, IPC and the paths it hands to boot), `sidecars/` (boot, the supervisor, one spec per sidecar, and the image-model, preset, audio and agent watchers), `session-log/` (this run's output, for issue reports), `permissions/` (who may use the microphone), `microphone/` (the operating system's switch), `secret.ts`, `updater.ts` and `document-files.ts`.

View file

@ -164,7 +164,7 @@ Seven run in CI: `test_license_key.py` inside the release workflow, `test_frozen
## Known gaps
- No issue on audio.cpp asks for archives that meet the app's floors yet, so `build-audiocpp.yml` has no end date.
- `nemotron_asr` is checked only for a loader: no packaging test transcribes with it yet, so a staged server that cannot transcribe would still ship.
- On macOS `nemotron_asr` is checked only for a loader: the `macos-archive` job stages upstream's archive and transcribes nothing, so a macOS server that cannot transcribe would still ship.
- No packaging test checks that the frozen API carries the agent's `SKILL.md`; `api.spec` collects it, and it was checked by hand.
- [`desktop-tests.yml`](../../.github/workflows/desktop-tests.yml) runs on pull requests and by hand (`workflow_dispatch`), never on a push to `dev`, and only on Linux, so a break that lands on `dev`, or shows only on Windows or macOS, shows first in a release build ([#2226](https://github.com/MODSetter/SurfSense/issues/2226)).
- The packaged `llama-server`, `audiocpp_server` and `sd-server` are started only on the Linux runner ([#1986](https://github.com/MODSetter/SurfSense/issues/1986)).

View file

@ -382,12 +382,12 @@ The runtime licence policy keeps GPL and LGPL code out of the app's own binaries
Each step ships alone and leaves the app working. Steps 1 to 7 are v1.
1. **The measurement, no app change. Done** on 14 Oct 2026 ([Measurements](#measurements)): Nemotron 3.5 and Qwen3-ASR-0.6B on audio.cpp v0.8.2 built with the app's recipe flags. Nemotron took half the memory and a third of the time, so it is the model.
2. **The runtime, staged and invisible.** `FAMILIES`, the Linux `objdump` and Windows per-CPU gates, the macOS archive check, the sidecar spec, its watcher with crash restart, and the API's URL. Nothing writes `transcription.json` yet, so nothing starts. Adds ADR 0056, or the next number free then (0040 to 0046 are proposed by the file-agent work): "Speech recognition runs on a second audio.cpp server that only the API calls". Updates [`packaging.md`](../architecture/packaging.md).
3. **Catalog, selection and Settings.** The type, the revision, the slot, the manifest block and entry, the classifier, `transcription.json`, the rows and the Settings section. Users can install and choose a speech model; nothing reads it yet, which [selection](../architecture/local-models/selection.md) allows. Updates [`data-model.md`](../architecture/data-model.md), [`selection.md`](../architecture/local-models/selection.md) and [`catalog.md`](../architecture/local-models/catalog.md), and adds glossary rows for "speech recognition", "dictation" and "transcript", apart from the "speech" row.
4. **The API slice,** `modules/llm/transcription/` (router, internal router, gate, sessions, admission, selected model, languages, audio checks, cleanup, Silero) and the provider client behind the `Transcriber` protocol, with the hold rule and `fetch_silero_vad.py`. Tested against a fake server, and against the staged server by `test_audiocpp_transcription.py`; no screen changes. Extends the ADR with the gate, and updates [`admission.md`](../architecture/local-models/admission.md) and [`studio.md`](../architecture/studio.md).
5. **Audio files as sources,** with `miniaudio`, yielding, progress, cancel, awaiting rows and the set-up popover they open, and the search caveat in the same release, because without yielding one recording blocks every upload and note. Updates [`documents.md`](../architecture/documents.md) and `data-model.md`.
6. **The microphone in Electron,** and a notarized macOS build that shows the prompt. Updates `packaging.md`.
7. **Dictation,** phrase by phrase. Updates [`chat.md`](../architecture/chat.md). Folds v1 into `architecture/`, moves steps 8 to 11 into a new `proposals/speech-recognition-next.md`, and deletes this file.
2. **The runtime, staged and invisible. Done.** `FAMILIES`, the Linux `objdump` and Windows per-CPU gates, the macOS archive check, the sidecar spec, its watcher with crash restart, and the API's URL. Nothing writes `transcription.json` yet, so nothing starts. Adds [ADR 0056](../adr/0056-speech-recognition-runs-on-a-second-audiocpp-server.md), "Speech recognition runs on a second audio.cpp server that only the API calls". Updates [`packaging.md`](../architecture/packaging.md).
3. **Catalog, selection and Settings. Done.** The type, the revision, the slot, the manifest block and entry, the classifier, `transcription.json`, the rows and the Settings section. Users can install and choose a speech model; nothing reads it yet, which [selection](../architecture/local-models/selection.md) allows. Updates [`data-model.md`](../architecture/data-model.md), [`selection.md`](../architecture/local-models/selection.md) and [`catalog.md`](../architecture/local-models/catalog.md), and adds glossary rows for "speech recognition", "dictation" and "transcript", apart from the "speech" row.
4. **The API slice, done:** `modules/llm/transcription/` (router, internal router, gate, sessions, admission, selected model, languages, audio checks, cleanup, Silero) and the provider client behind the `Transcriber` protocol, with the hold rule and `fetch_silero_vad.py`. Tested against a fake server, and against the staged server by `test_audiocpp_transcription.py`; no screen changes. Extends the ADR with the gate, and updates [`admission.md`](../architecture/local-models/admission.md) and [`studio.md`](../architecture/studio.md).
5. **Audio files as sources, done:** with `miniaudio`, yielding, progress, cancel, awaiting rows and the set-up popover they open, and the search caveat in the same release, because without yielding one recording blocks every upload and note. Updates [`documents.md`](../architecture/documents.md) and `data-model.md`.
6. **The microphone in Electron, done.** A notarized macOS build that shows the prompt is still owed ([packaging](../architecture/packaging.md), Known gaps). Updates `packaging.md`.
7. **Dictation, done:** phrase by phrase. Updates [`chat.md`](../architecture/chat.md). Folding v1 into `architecture/`, moving steps 8 to 11 into `proposals/speech-recognition-next.md` and deleting this file wait for the measurements under [Open questions](#open-questions).
8. **Next: light and broad languages.** The pin moved to v0.9.1 or later on its own, with the TTS peaks and the voicing gate re-run, then OWSM v4 base and small after a FLEURS eval of ar, hi, bn, ur, ta, te, sw, uk, he and fa, listing only the languages that pass.
9. **Next: more formats.** `scripts/ffmpeg-decode/` builds a pinned FFmpeg in CI on `ubuntu-22.04`, Windows and macOS arm64 with `--disable-everything --disable-network --disable-autodetect`, only the needed demuxers, decoders and parsers, `aresample` and a `pcm_s16le` WAV muxer, gated on `-buildconf` showing no `--enable-gpl`, `--enable-nonfree` or `--enable-lib` option, on fixtures in every allowed format and on the `objdump` floor. The worker runs it as the decode child, with a timeout. `.opus` joins; `.m4a` and `.aac` join once the AAC question is settled; `.webm` joins only for a file whose only stream is audio, until the maintainer answers the video question. Evaluated beside it: decoding in the renderer through Electron's own FFmpeg for files added from the window, streamed in chunks to 16 kHz mono WAV.
10. **Next: listen and seek.** `GET /documents/{id}/original` commits before its `FileResponse`, since every transaction is `BEGIN IMMEDIATE` and a player would hold the write lock, as Studio's file route does. An audio viewer joins the [source preview](source-preview.md)'s registry, with the transcript beside it, and a citation of a recording opens it at the time from `segments.json`.

View file

@ -294,7 +294,7 @@ The snapshot this replaced dropped the provider and keyed by model id alone. Tha
- Diffusion architectures (`sd1`, `sdxl`, `sd3`, `flux`, `flux2`, `qwen_image`, `z_image`, `lumina2` and the rest) are `IMAGE_GEN`. They were refused because the old catalog was text only; the app ships sd.cpp.
- Video architectures (`wan`, `ltxv`, `hyvid`, `cosmos`) are `VIDEO_GEN`.
- Text-to-speech models are `AUDIO_GEN`.
- Embedders, rerankers, speech recognisers, labellers, OCR, draft heads and projectors are known-none: no type describes what they produce.
- Embedders, rerankers, speech recognisers, labellers, OCR, draft heads and projectors are known-none: no type describes what they produce. The exception is a curated speech recogniser's family, which is `SPEECH_TO_TEXT` (`nemotron_asr`, [catalog](../architecture/local-models/catalog.md)).
- Any other architecture is `TEXT_GEN`. A denylist ages the right way: an unknown architecture is usually a chat model released last week.
Evidence rules:

View file

@ -21,7 +21,7 @@ export function AddModelOptions({
}: {
/** The slot's catalog for this computer. */
download: ReactNode
/** The slot being filled, so the connect dialog offers only what fills it. */
/** The slot being filled, so a ChatGPT plan is offered only where one fills it. */
modelType: ModelType
/** Absent where the slot offers no server. */
onConnected?: (connection: Connection) => void

View file

@ -109,6 +109,30 @@ describe("speech recognition settings", () => {
expect(screen.queryByText(/Kokoro/)).toBeNull()
})
it("offers no ChatGPT plan either, even one that says it serves the slot", async () => {
vi.stubGlobal(
"fetch",
serving([], (path) =>
path === "/llm/connections/chatgpt"
? Response.json({ serves: ["text_gen", "speech_to_text"], hosts: [] })
: null
)
)
const user = userEvent.setup()
renderSettings()
await user.click(await screen.findByRole("button", { name: "Add model" }))
expect(
await screen.findByText(
"Speech recognition models cannot run on this computer"
)
).toBeTruthy()
expect(
screen.queryByRole("region", { name: "Use your ChatGPT plan" })
).toBeNull()
expect(screen.queryByRole("region", { name: "Use a server" })).toBeNull()
})
it("says the models could not load, not that this computer cannot run one", async () => {
vi.stubGlobal(
"fetch",