From 32a0c0db7ac87987567b288780c2106634a798ee Mon Sep 17 00:00:00 2001 From: Cedric Karungu Date: Tue, 6 Oct 2026 17:56:36 +0200 Subject: [PATCH 1/5] feat(local): encode a podcast episode as MP3 audio.cpp's joined WAV is encoded in-process by LAME through lameenc, at 64 kbit/s per channel, the model's channels and rate. Without lameenc the episode stays WAV. The frozen worker ships lameenc's dist-info for LAME's LGPL text. Fixes #2001 --- docs/architecture/packaging.md | 1 + docs/architecture/studio.md | 7 +-- surfsense_local/backend/bundling/worker.spec | 5 ++ .../llm/providers/audiocpp/encoded_episode.py | 44 +++++++++++++++ .../llm/providers/audiocpp/joined_wav.py | 4 +- .../modules/llm/providers/audiocpp/speech.py | 3 +- surfsense_local/backend/pyproject.toml | 1 + .../llm/providers/audiocpp/test_speech.py | 41 ++++++++++++-- .../tests/unit/worker/test_studio_podcast.py | 29 ++++++++-- surfsense_local/backend/uv.lock | 54 +++++++++++++++++++ 10 files changed, 175 insertions(+), 14 deletions(-) create mode 100644 surfsense_local/backend/modules/llm/providers/audiocpp/encoded_episode.py diff --git a/docs/architecture/packaging.md b/docs/architecture/packaging.md index 7d4b400b1..1bf6d1685 100644 --- a/docs/architecture/packaging.md +++ b/docs/architecture/packaging.md @@ -47,6 +47,7 @@ What else each spec names, and why the analyser cannot find it on its own: | `worker.spec` | python-docx, python-pptx, xlsxwriter, reportlab, openpyxl | the Office formats and document scripts run model-written code that imports them, so no static import exists; openpyxl, named in `pyproject.toml` though Docling brings it too, also reads every document script's workbook for its summary | | `worker.spec` | `matplotlib.pyplot`, `matplotlib.backends.backend_pdf`, `matplotlib.backends.backend_svg`, with the matplotlib hook set to the `Agg` backend | document scripts draw charts, and nothing imports matplotlib statically. Naming pyplot runs PyInstaller's hook, which adds `mpl-data` and the backend chosen; a chart saved as PDF or SVG, or `PdfPages`, imports its canvas by name, which the hook does not follow. `Agg` alone keeps Tk's GUI backend out, since scripts run with `MPLBACKEND=Agg` | | `worker.spec` | `modules.documents.tasks`, `modules.artifacts.tasks` | Huey resolves a task by its name | +| `worker.spec` | `copy_metadata("lameenc")` | a podcast is encoded as MP3 by LAME, statically linked into `lameenc`'s extension, which the analyser finds by its import ([studio](studio.md#voicing-a-podcast)); LAME's LGPL licence text is only in the wheel's dist-info, which ships only when named | ## Model packs diff --git a/docs/architecture/studio.md b/docs/architecture/studio.md index 8ae35932b..a14feab5a 100644 --- a/docs/architecture/studio.md +++ b/docs/architecture/studio.md @@ -34,7 +34,7 @@ The catalog is a tuple of twelve `Format` rows in [`formats.py`](../../surfsense | `mindmap` | Mind map | text_gen | `content/mindmap/` | none; the outline is the body | | `flashcards` | Flashcards | text_gen | `content/flashcards/` | deck JSON | | `quiz` | Quiz | text_gen | `content/quiz/` | quiz JSON | -| `podcast` | Podcast | text_gen, audio_gen | `media/audio/podcast/` | WAV audio | +| `podcast` | Podcast | text_gen, audio_gen | `media/audio/podcast/` | MP3 audio, or WAV from a server that sends WAV | | `image` | Image | image_gen, text_gen | `media/visual/image/` | the image | | `infographic` | Infographic | image_gen, text_gen | `media/visual/infographic/` | the image | @@ -82,7 +82,7 @@ Studio's Word and PDF buttons take one of two paths, chosen by [`office/document - The job sends the whole spec, fenced, and the instruction in one call, gets the whole revised spec back, and renders it as a draft of that kind would. Nothing is patched or diffed. The rewritten spec is kept only when it renders: a failed refine keeps none, is not retried by Huey, and leaves every earlier version as it was. - The agent's `surfsense_read_document` and `surfsense_render_document` refuse any version of a document Studio made, with "This document was made in Studio: refine it there, or make a new document." ([07](../proposals/file-agent/07-create-and-edit-mvp.md), decision 8). -Studio uses no Electron `printToPDF`, which only draws the agent's Word and PowerPoint previews and its sources' pages ([agent](agent.md#previews)), and there is no ffmpeg in `surfsense_local`. A PDF is ReportLab through the builder, a Studio script or a script document, and a podcast is one `audio/wav` joined from each turn's WAV, or one `audio/mpeg` from a server that sends only MP3. +Studio uses no Electron `printToPDF`, which only draws the agent's Word and PowerPoint previews and its sources' pages ([agent](agent.md#previews)), and there is no ffmpeg in `surfsense_local`. A PDF is ReportLab through the builder, a Studio script or a script document. A local podcast is one `audio/mpeg` encoded by LAME, through the `lameenc` package, from each turn's WAV joined; a server's is one `audio/wav` joined from its WAV turns, or one `audio/mpeg` from a server that sends only MP3. ## Script documents @@ -124,6 +124,7 @@ The libraries a script may use are the worker's: python-docx, ReportLab, python- - **The voices are the model's roster**, from its manifest entry: each voice with the languages it speaks, one for a Kokoro or Kitten voice and all 31 for a Supertonic voice. The brief's language list, A to Z by the name shown in the interface language, and each speaker's voice picker come from it, the picker grouped into female and male voices. The brief opens in American English where a voice speaks it, else in any English, else in the roster's first language, and a remembered brief the chosen model cannot voice falls back to that. - **Memory is checked before anything loads, twice.** The worker compares the operating system's available memory (`MemAvailable` on Linux, free plus inactive pages on macOS, `GlobalMemoryStatusEx` on Windows) with the model's peak measured while voicing with every chunk of text full, from its entry, plus 1 GiB, and refuses short of it: "Voicing needs about 3.5 GB free; this computer has 1.8 GB. Supertonic 3 needs about 1.6 GB." The second sentence names the first other curated audio model, in the manifest's order, that would fit, and is left out when none would. It checks first, before the chat model drafts anything, so a machine that cannot voice the episode does not spend minutes writing it. Short there, it gives up the text runtime through the API ([the image job's hold](local-models/admission.md#the-image-jobs-hold)), which lets running replies finish and unloads the chat model, checks again, and lets it go for drafting, because voicing frees that memory anyway: on 16 GB, Qwen3 1.7B resident left 3.3 GB free and refused Kokoro's 3.5 before a word was drafted. At voicing it gives the runtime up again and holds it until the last turn is voiced, so a chat sent meanwhile waits instead of loading the chat model back into memory voicing needs. The router reloads the chat model on the next request after either. Each check waits up to 10 seconds for memory the app is giving back before it refuses, because a starting podcast ends sd-server's idle minutes but Electron stops it only on its next 5-second poll ([the image path](#the-image-path)). audio.cpp's own guard counts only the file it reads. - **One speech request per turn**, with the turn's voice. The brief's language goes in the request only for a voice that speaks several; a Kokoro voice's name already fixes its language. The turns are joined with 0.35 s between them. +- **The episode is MP3**: the joined 16-bit WAV is encoded in-process by LAME through `lameenc`, at 64 kbit/s per channel and the model's own channels and rate, about a sixth of the WAV ([`encoded_episode.py`](../../surfsense_local/backend/modules/llm/providers/audiocpp/encoded_episode.py)). No ffmpeg and nothing on the user's `PATH` is used. Where `lameenc` cannot be imported, as in a checkout that skipped it, the episode stays WAV, which the viewer's `