vrr/talkis
2
0
Fork 0
mirror of https://github.com/SerTimBerrners-Lee/talkis.git synced 2026-07-09 17:29:15 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Release v0.3.7
Some checks failed
CI / cargo check (linux) (push) Has been cancelled
Details
CI / cargo check (macos) (push) Has been cancelled
Details
CI / cargo check (windows) (push) Has been cancelled
Details
CI / tsc + hotkey smoke (push) Has been cancelled
Details

Browse source
This commit is contained in:
David Perov 2026-07-03 17:36:34 +03:00
parent 2f0bc345c5
commit 7f7d85f3f3
51 changed files with 2173 additions and 3650 deletions
Download patch file Download diff file Expand all files Collapse all files

24
AGENTS.md
View file

@ -128,6 +128,30 @@ The sidebar CTA is a **light card** (not inverted black):
## Code Style
## Modularity and AI Context Discipline
Treat module boundaries as part of maintainability. The goal is not just fewer
lines per file, but fewer reasons to read an entire file before making a safe
change.
- Prefer small modules with a clear responsibility, public API, and contract.
- Keep unrelated workflows separated. A diarization change should not require
reading the whole STT pipeline, settings UI, and media conversion layer.
- When a file grows because it owns multiple responsibilities, extract along
stable boundaries: parsing/probing, conversion, chunking, runtime lifecycle,
request routing, error mapping, and UI state/view composition.
- Do not create abstractions only to reduce line count. Extract when it reduces
coupling, makes behavior easier to test, or limits the context needed for
future changes.
- For frontend code, split large tabs/views into container state, command/data
hooks, pure UI sections, and small reusable controls.
- For Rust code, keep Tauri commands thin where practical and move pipeline
logic into focused modules with explicit input/output structs.
- Preserve behavior during extraction. Prefer small, test-backed refactors over
broad rewrites.
- When touching a large file, consider whether the current change can leave
behind a clearer boundary without expanding the task scope.
### TypeScript/React
**Imports:** Group by external → internal, use explicit file extensions for clarity when needed.

15
README.md
View file

@ -35,11 +35,18 @@ It is designed for practical daily work: IDEs, chats, notes, CRM fields, email,
- Transcribe audio and video files from the Files tab or by dropping files onto the widget.
- Record macOS calls with separate microphone and system-audio tracks.
- Keep local history for voice recordings and file transcriptions.
- Run managed local runtimes for Whisper, Qwen ASR, NVIDIA Parakeet, and speaker diarization.
- Run managed local STT through one native `transcribe.cpp` runtime for Whisper, Qwen ASR, NVIDIA Parakeet, plus speaker diarization.
- Build native bundles for macOS, Windows, and Linux.
## Latest Changes
### v0.3.7
- Migrated managed local file transcription to a unified native `transcribe.cpp` runtime for Whisper, Qwen ASR, and NVIDIA Parakeet.
- Removed the old Qwen/Parakeet MLX/Python sidecars from the desktop bundle and release build.
- Fixed local STT settings migration so old Qwen/Parakeet endpoints move to `http://127.0.0.1:8000` and do not require an API key.
- Fixed selecting a local transcription model so it no longer clears the selected text model used for summarization.
### v0.3.6
- Fixed permissions onboarding recovery for existing installs so updates do not reopen the full permissions screen when required macOS access is already granted.
@ -124,13 +131,11 @@ Supported STT model names include:
### Local Models
Install and run Talkis-managed local runtimes from Settings. Local mode is transcription-only unless you also configure a separate LLM endpoint.
Install and run Talkis-managed local models from Settings. Local mode is transcription-only unless you also configure a separate LLM endpoint.
Managed local runtimes:
- Whisper, default endpoint `http://127.0.0.1:8000`
- NVIDIA Parakeet MLX, default endpoint `http://127.0.0.1:8001`
- Qwen ASR, default endpoint `http://127.0.0.1:8002`
- Whisper, NVIDIA Parakeet GGUF, and Qwen ASR GGUF through the unified `transcribe.cpp` runtime, default endpoint `http://127.0.0.1:8000`
- Speaker diarization, default endpoint `http://127.0.0.1:8003`
If a default port is busy, Talkis starts the managed runtime on a fallback port and saves the actual endpoint in settings.

8
README.ru.md
View file

@ -35,7 +35,7 @@ Talkis сделан для повседневной работы: IDE, чаты,
- Транскрибация аудио и видео из вкладки `Файлы` или через перетаскивание файла на виджет.
- Запись созвонов на macOS с отдельными дорожками микрофона и системного аудио.
- Локальная история голосовых записей и файловых транскрибаций.
- Управляемые локальные рантаймы для Whisper, Qwen ASR, NVIDIA Parakeet и diarization.
- Единый управляемый локальный STT runtime `transcribe.cpp` для Whisper, Qwen ASR, NVIDIA Parakeet и отдельный runtime для diarization.
- Нативные сборки для macOS, Windows и Linux.
## Демонстрация
@ -104,13 +104,11 @@ Talkis сделан для повседневной работы: IDE, чаты,
### Локальные модели
Установите и запустите управляемые Talkis локальные рантаймы из настроек. Локальный режим отвечает только за транскрибацию, если отдельно не настроен LLM endpoint.
Установите и запустите управляемые Talkis локальные модели из настроек. Локальный режим отвечает только за транскрибацию, если отдельно не настроен LLM endpoint.
Управляемые локальные рантаймы:
- Whisper, endpoint по умолчанию `http://127.0.0.1:8000`
- NVIDIA Parakeet MLX, endpoint по умолчанию `http://127.0.0.1:8001`
- Qwen ASR, endpoint по умолчанию `http://127.0.0.1:8002`
- Whisper, NVIDIA Parakeet GGUF и Qwen ASR GGUF через единый `transcribe.cpp` runtime, endpoint по умолчанию `http://127.0.0.1:8000`
- Speaker diarization, endpoint по умолчанию `http://127.0.0.1:8003`
Если порт по умолчанию занят, Talkis запускает управляемый рантайм на свободном fallback-порту и сохраняет фактический endpoint в настройках.

3
bun.lock
View file

@ -10,7 +10,6 @@
"@tauri-apps/plugin-clipboard-manager": "^2.3.2",
"@tauri-apps/plugin-deep-link": "^2.4.8",
"@tauri-apps/plugin-dialog": "^2.7.1",
"@tauri-apps/plugin-global-shortcut": "^2.3.1",
"@tauri-apps/plugin-opener": "^2",
"@tauri-apps/plugin-process": "^2.3.1",
"@tauri-apps/plugin-store": "^2.4.2",
@ -252,8 +251,6 @@
"@tauri-apps/plugin-dialog": ["@tauri-apps/plugin-dialog@2.7.1", "", { "dependencies": { "@tauri-apps/api": "^2.11.0" } }, "sha512-OK1UBXYt+ojcmxMktzzuyonYIFta8CmAASpX+CA+DTGK24KlHjhYI6x2iOJ/TjZF4N7/ACK1oFmEOjIY9IhzOQ=="],
"@tauri-apps/plugin-global-shortcut": ["@tauri-apps/plugin-global-shortcut@2.3.1", "", { "dependencies": { "@tauri-apps/api": "^2.8.0" } }, "sha512-vr40W2N6G63dmBPaha1TsBQLLURXG538RQbH5vAm0G/ovVZyXJrmZR1HF1W+WneNloQvwn4dm8xzwpEXRW560g=="],
"@tauri-apps/plugin-opener": ["@tauri-apps/plugin-opener@2.5.3", "", { "dependencies": { "@tauri-apps/api": "^2.8.0" } }, "sha512-CCcUltXMOfUEArbf3db3kCE7Ggy1ExBEBl51Ko2ODJ6GDYHRp1nSNlQm5uNCFY5k7/ufaK5Ib3Du/Zir19IYQQ=="],
"@tauri-apps/plugin-process": ["@tauri-apps/plugin-process@2.3.1", "", { "dependencies": { "@tauri-apps/api": "^2.8.0" } }, "sha512-nCa4fGVaDL/B9ai03VyPOjfAHRHSBz5v6F/ObsB73r/dA3MHHhZtldaDMIc0V/pnUw9ehzr2iEG+XkSEyC0JJA=="],

67
docs/release/review-v0.3.7.md Normal file
View file

@ -0,0 +1,67 @@
# Release Review v0.3.7
## Release
- Version: 0.3.7
- Release branch: release/v0.3.7
- Target tag: v0.3.7
- Reviewer: Codex
- Date: 2026-07-03
## Scope
- Key changes included in this release:
- Migrated managed local STT to one native `transcribe.cpp` runtime for Whisper, Qwen ASR, and NVIDIA Parakeet.
- Removed old Qwen/Parakeet MLX/Python sidecars, manifests, and runtime engine files from the app bundle path.
- Migrated legacy local STT endpoints `8001` and `8002` to the unified `8000` runtime.
- Fixed local STT file-transcription errors so local runtime failures do not show API-key authorization copy.
- Fixed local STT model selection so it no longer clears the selected text/LLM model used for summarization.
- Kept `handy-keys` based hotkey registration and removed the old Tauri global-shortcut dependency.
- User-facing changes:
- Local file transcription on installed local models is much faster through `transcribe.cpp`.
- Qwen ASR and NVIDIA Parakeet now install as GGUF models under the unified local STT runtime.
- Settings no longer expose separate Qwen/Parakeet runtime ports.
- Summary warning copy no longer uses the long dash.
- Risky areas:
- Local STT model download/install flow for fresh Qwen/Parakeet GGUF installs.
- macOS Accessibility/hotkey behavior after switching to `handy-keys`.
- Release updater signing depends on GitHub Actions secrets because the local updater key is encrypted.
## Checks run
- `git diff --check`: passed.
- `bun run check:release`: passed.
- `cargo check --manifest-path src-tauri/Cargo.toml`: passed without warnings after cleanup.
- `cargo test --manifest-path src-tauri/Cargo.toml --bin talkis-stt`: passed, 4 tests.
- `bun test src/lib/summarize.test.ts src/lib/fileTranscription.test.ts`: passed, 13 tests.
- `TAURI_SIGNING_PRIVATE_KEY_PATH=~/.tauri/talkis-updater.key bun run build:release:macos`: built release sidecars, frontend, `Talkis.app`, and `Talkis.app.tar.gz`; failed only at updater signature step because the local encrypted key password was not available in this shell.
- Native/GitHub Windows build: delegated to GitHub Actions release workflow.
- Native/GitHub Linux build: delegated to GitHub Actions release workflow.
- Additional manual checks:
- Temporary `talkis-stt` runtime smoke test on port `18081`: `/health`, `/v1/models`, and silent local transcription passed.
- User manually tested the faster local transcription path before release request.
## Manual review
- Hotkey flow: smoke tests passed for start, stop, lock, suppressed release, and reset states.
- Onboarding permissions: previous permission-onboarding changes are included; no new permission prompt changes in this release slice.
- Widget position and notice behavior: no release-blocking changes found in this review slice.
- Transcription quality and short-utterance handling: `talkis-stt` low-signal guard test passed; local runtime smoke test returned empty text for silence.
- README refreshed: yes, README and README.ru now document unified `transcribe.cpp` local STT runtime and v0.3.7 changes.
## Findings
- Blockers:
- No code blocker found for main/tag.
- Local macOS updater signing cannot complete without `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` for the encrypted local key. GitHub Actions must have `TAURI_SIGNING_PRIVATE_KEY` and `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` configured for the release to publish updater metadata.
- Non-blocking issues:
- Vite reports the existing large chunk warning for the main JS bundle.
- Local `gh` CLI auth is invalid, so the previous failed release run could not be inspected from this machine.
- Follow-ups after release:
- Verify GitHub Actions release run for macOS, Windows, Linux, and `latest.json` publication.
- If Actions fail at signing, refresh GitHub updater signing secrets before rerunning the workflow.
## Decision
- Ready for `main` merge: yes.
- Ready for tag publish: yes, with the signing-secret caveat above.

3
package.json
View file

@ -1,7 +1,7 @@
{
"name": "talkis",
"private": true,
"version": "0.3.6",
"version": "0.3.7",
"license": "AGPL-3.0-or-later",
"type": "module",
"packageManager": "bun@1.2.13",
@ -33,7 +33,6 @@
"@tauri-apps/plugin-clipboard-manager": "^2.3.2",
"@tauri-apps/plugin-deep-link": "^2.4.8",
"@tauri-apps/plugin-dialog": "^2.7.1",
"@tauri-apps/plugin-global-shortcut": "^2.3.1",
"@tauri-apps/plugin-opener": "^2",
"@tauri-apps/plugin-process": "^2.3.1",
"@tauri-apps/plugin-store": "^2.4.2",

4
scripts/prepare-stt-sidecar.mjs
View file

@ -119,7 +119,7 @@ function ensureLinuxBuildDependencies(targetTriple) {
console.error("");
console.error("Missing Linux build dependencies for local STT sidecars.");
console.error("whisper-rs-sys needs libclang for bindgen, cmake for whisper.cpp, and xdo for Tauri/global hotkey linking.");
console.error("transcribe-cpp needs cmake for native ggml/transcribe.cpp builds, and xdo is needed for Tauri/global hotkey linking.");
console.error("Install it on Ubuntu/Debian with:");
console.error("");
console.error(` sudo apt update && sudo apt install -y ${missingPackages.join(" ")}`);
@ -153,7 +153,7 @@ ensureLinuxBuildDependencies(targetTriple);
const extension = targetTriple.includes("windows") ? ".exe" : "";
const profile = process.env.TALKIS_STT_RELEASE === "1" ? "release" : "debug";
const sidecars = ["talkis-stt", "talkis-stt-nvidia", "talkis-stt-qwen", "talkis-diarize", "talkis-llm"];
const sidecars = ["talkis-stt", "talkis-diarize", "talkis-llm"];
const cargoArgs = ["build", "--manifest-path", join(tauriDir, "Cargo.toml")];
for (const sidecar of sidecars) {

362
src-tauri/Cargo.lock generated
View file

@ -355,6 +355,12 @@ dependencies = [
"serde_core",
]
[[package]]
name = "block"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d8c1fef690941d3e7788d328517591fecc684c084084702d6ff1641e993699a"
[[package]]
name = "block-buffer"
version = "0.10.4"
@ -633,6 +639,21 @@ dependencies = [
"cc",
]
[[package]]
name = "cocoa"
version = "0.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "667fdc068627a2816b9ff831201dd9864249d6ee8d190b9532357f1fc0f61ea7"
dependencies = [
"bitflags 1.3.2",
"block",
"core-foundation 0.9.4",
"core-graphics 0.21.0",
"foreign-types 0.3.2",
"libc",
"objc",
]
[[package]]
name = "combine"
version = "4.6.7"
@ -694,13 +715,23 @@ version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9885fa71e26b8ab7855e2ec7cae6e9b380edff76cd052e07c683a0319d51b3a2"
[[package]]
name = "core-foundation"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57d24c7a13c43e870e37c1556b74555437870a04514f7685f5b354e090567171"
dependencies = [
"core-foundation-sys 0.7.0",
"libc",
]
[[package]]
name = "core-foundation"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f"
dependencies = [
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"libc",
]
@ -710,16 +741,46 @@ version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6"
dependencies = [
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"libc",
]
[[package]]
name = "core-foundation-sys"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3a71ab494c0b5b860bdc8407ae08978052417070c2ced38573a9157ad75b8ac"
[[package]]
name = "core-foundation-sys"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "core-graphics"
version = "0.19.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3889374e6ea6ab25dba90bb5d96202f61108058361f6dc72e8b03e6f8bbe923"
dependencies = [
"bitflags 1.3.2",
"core-foundation 0.7.0",
"foreign-types 0.3.2",
"libc",
]
[[package]]
name = "core-graphics"
version = "0.21.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52a67c4378cf203eace8fb6567847eb641fd6ff933c1145a115c6ee820ebb978"
dependencies = [
"bitflags 1.3.2",
"core-foundation 0.9.4",
"foreign-types 0.3.2",
"libc",
]
[[package]]
name = "core-graphics"
version = "0.23.2"
@ -775,7 +836,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "321077172d79c662f64f5071a03120748d5bb652f5231570141be24cfcd2bace"
dependencies = [
"bitflags 1.3.2",
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"coreaudio-sys",
]
@ -795,7 +856,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "873dab07c8f743075e57f524c583985fbaf745602acbe916a01539364369a779"
dependencies = [
"alsa",
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"coreaudio-rs",
"dasp_sample",
"jni",
@ -1269,6 +1330,16 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "epoll"
version = "4.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e74d68fe2927dbf47aa976d14d93db9b23dced457c7bb2bdc6925a16d31b736e"
dependencies = [
"bitflags 2.11.0",
"libc",
]
[[package]]
name = "equivalent"
version = "1.0.2"
@ -1302,6 +1373,29 @@ version = "3.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dea2df4cf52843e0452895c455a1a2cfbb842a1e7329671acf418fdc53ed4c59"
[[package]]
name = "evdev-rs"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b92abc30d5fd1e4f6440dee4d626abc68f4a9b5014dc1de575901e23c2e02321"
dependencies = [
"bitflags 1.3.2",
"evdev-sys",
"libc",
"log",
]
[[package]]
name = "evdev-sys"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cdcf0d489f4d9a80ac2b3b35b92fdd8fcf68d33bb67f947afe5cd36e482de576"
dependencies = [
"cc",
"libc",
"pkg-config",
]
[[package]]
name = "event-listener"
version = "5.4.1"
@ -1479,12 +1573,6 @@ dependencies = [
"percent-encoding",
]
[[package]]
name = "fs_extra"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
[[package]]
name = "futf"
version = "0.1.5"
@ -1840,24 +1928,6 @@ version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
[[package]]
name = "global-hotkey"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9247516746aa8e53411a0db9b62b0e24efbcf6a76e0ba73e5a91b512ddabed7"
dependencies = [
"crossbeam-channel",
"keyboard-types",
"objc2 0.6.4",
"objc2-app-kit",
"once_cell",
"serde",
"thiserror 2.0.18",
"windows-sys 0.59.0",
"x11rb",
"xkeysym",
]
[[package]]
name = "gobject-sys"
version = "0.18.0"
@ -1951,6 +2021,25 @@ dependencies = [
"zerocopy",
]
[[package]]
name = "handy-keys"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1c1bb519fbead4e7344bf0fe32718c4cea30fc05e03fc65c242df064f515b03"
dependencies = [
"bitflags 2.11.0",
"block2 0.6.2",
"objc2 0.6.4",
"objc2-app-kit",
"objc2-core-foundation",
"objc2-core-graphics",
"objc2-foundation",
"rdev",
"serde",
"thiserror 2.0.18",
"windows 0.58.0",
]
[[package]]
name = "hashbrown"
version = "0.12.3"
@ -2155,7 +2244,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
dependencies = [
"android_system_properties",
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"iana-time-zone-haiku",
"js-sys",
"log",
@ -2371,6 +2460,26 @@ dependencies = [
"cfb",
]
[[package]]
name = "inotify"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46dd0a94b393c730779ccfd2a872b67b1eb67be3fc33082e733bdb38b5fde4d4"
dependencies = [
"bitflags 1.3.2",
"inotify-sys",
"libc",
]
[[package]]
name = "inotify-sys"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ea94e891b3606826e9c998be69ddca42247dad8ad50b1649a5cb7e1c9ae06fd"
dependencies = [
"libc",
]
[[package]]
name = "ipnet"
version = "2.12.0"
@ -2531,6 +2640,12 @@ dependencies = [
"selectors 0.24.0",
]
[[package]]
name = "lazy_static"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "leb128fmt"
version = "0.1.0"
@ -2696,6 +2811,15 @@ dependencies = [
"libc",
]
[[package]]
name = "malloc_buf"
version = "0.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62bb907fe88d54d8d9ce32a3cceab4218ed2f6b7d35617cafe9adf84e43919cb"
dependencies = [
"libc",
]
[[package]]
name = "markup5ever"
version = "0.14.1"
@ -2991,6 +3115,15 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "objc"
version = "0.2.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "915b1b472bc21c53464d6c8461c9d3af805ba1ef837e1cac254428f4a77177b1"
dependencies = [
"malloc_buf",
]
[[package]]
name = "objc-sys"
version = "0.3.5"
@ -3092,7 +3225,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
dependencies = [
"bitflags 2.11.0",
"block2 0.6.2",
"dispatch2",
"libc",
"objc2 0.6.4",
]
@ -3103,10 +3238,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e022c9d066895efa1345f8e33e584b9f958da2fd4cd116792e15e07e4720a807"
dependencies = [
"bitflags 2.11.0",
"block2 0.6.2",
"dispatch2",
"libc",
"objc2 0.6.4",
"objc2-core-foundation",
"objc2-io-surface",
"objc2-metal",
]
[[package]]
@ -3183,6 +3321,17 @@ dependencies = [
"objc2-core-foundation",
]
[[package]]
name = "objc2-metal"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a0125f776a10d00af4152d74616409f0d4a2053a6f57fa5b7d6aa2854ac04794"
dependencies = [
"bitflags 2.11.0",
"objc2 0.6.4",
"objc2-foundation",
]
[[package]]
name = "objc2-osa-kit"
version = "0.3.2"
@ -3991,6 +4140,25 @@ version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20675572f6f24e9e76ef639bc5552774ed45f1c30e2951e1e99c59888861c539"
[[package]]
name = "rdev"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00552ca2dc2f93b84cd7b5581de49549411e4e41d89e1c691bcb93dc4be360c3"
dependencies = [
"cocoa",
"core-foundation 0.7.0",
"core-foundation-sys 0.7.0",
"core-graphics 0.19.2",
"epoll",
"evdev-rs",
"inotify",
"lazy_static",
"libc",
"winapi",
"x11",
]
[[package]]
name = "redox_syscall"
version = "0.5.18"
@ -4280,7 +4448,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d99feebc72bae7ab76ba994bb5e121b8d83d910ca40b36e0921f53becc41784"
dependencies = [
"core-foundation 0.10.1",
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"jni",
"log",
"once_cell",
@ -4406,7 +4574,7 @@ checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d"
dependencies = [
"bitflags 2.11.0",
"core-foundation 0.10.1",
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"libc",
"security-framework-sys",
]
@ -4417,7 +4585,7 @@ version = "2.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2691df843ecc5d231c0b14ece2acc3efb62c0a398c7e1d875f3983ce020e3"
dependencies = [
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"libc",
]
@ -4937,7 +5105,7 @@ version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4"
dependencies = [
"core-foundation-sys",
"core-foundation-sys 0.8.7",
"libc",
]
@ -4969,7 +5137,7 @@ dependencies = [
[[package]]
name = "talkis"
version = "0.3.6"
version = "0.3.7"
dependencies = [
"base64 0.22.1",
"block2 0.6.2",
@ -4977,6 +5145,7 @@ dependencies = [
"cpal",
"dirs 5.0.1",
"enigo",
"handy-keys",
"hex",
"hound",
"include_dir",
@ -4998,7 +5167,6 @@ dependencies = [
"tauri-plugin-clipboard-manager",
"tauri-plugin-deep-link",
"tauri-plugin-dialog",
"tauri-plugin-global-shortcut",
"tauri-plugin-opener",
"tauri-plugin-process",
"tauri-plugin-shell",
@ -5006,9 +5174,9 @@ dependencies = [
"tauri-plugin-store",
"tauri-plugin-updater",
"tokio",
"transcribe-cpp",
"url",
"webkit2gtk",
"whisper-rs",
"window-vibrancy 0.7.1",
"x11rb",
]
@ -5309,21 +5477,6 @@ dependencies = [
"url",
]
[[package]]
name = "tauri-plugin-global-shortcut"
version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "424af23c7e88d05e4a1a6fc2c7be077912f8c76bd7900fd50aa2b7cbf5a2c405"
dependencies = [
"global-hotkey",
"log",
"serde",
"serde_json",
"tauri",
"tauri-plugin",
"thiserror 2.0.18",
]
[[package]]
name = "tauri-plugin-opener"
version = "2.5.3"
@ -5928,6 +6081,27 @@ dependencies = [
"valuable",
]
[[package]]
name = "transcribe-cpp"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b32749d8a4c16bc1b4e7ea07a463e454a7bddde459a29fdf998025a4ce874d52"
dependencies = [
"log",
"thiserror 2.0.18",
"transcribe-cpp-sys",
]
[[package]]
name = "transcribe-cpp-sys"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28e526bd52b0b16ea54a296c40273a0573a5886c5c6a69aa1790c3b9bde578d0"
dependencies = [
"cmake",
"serde_json",
]
[[package]]
name = "tray-icon"
version = "0.21.3"
@ -6502,28 +6676,6 @@ version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a28ac98ddc8b9274cb41bb4d9d4d5c425b6020c50c46f25559911905610b4a88"
[[package]]
name = "whisper-rs"
version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2088172d00f936c348d6a72f488dc2660ab3f507263a195df308a3c2383229f6"
dependencies = [
"whisper-rs-sys",
]
[[package]]
name = "whisper-rs-sys"
version = "0.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6986c0fe081241d391f09b9a071fbcbb59720c3563628c3c829057cf69f2a56f"
dependencies = [
"bindgen",
"cfg-if",
"cmake",
"fs_extra",
"semver",
]
[[package]]
name = "winapi"
version = "0.3.9"
@ -6605,6 +6757,16 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6"
dependencies = [
"windows-core 0.58.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows"
version = "0.61.3"
@ -6649,6 +6811,19 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99"
dependencies = [
"windows-implement 0.58.0",
"windows-interface 0.58.0",
"windows-result 0.2.0",
"windows-strings 0.1.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.61.2"
@ -6697,6 +6872,17 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "windows-implement"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bbd5b46c938e506ecbce286b6628a02171d56153ba733b6c741fc627ec9579b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
@ -6719,6 +6905,17 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "windows-interface"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053c4c462dc91d3b1504c6fe5a726dd15e216ba718e84a0e46a88fbe5ded3515"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "windows-interface"
version = "0.59.3"
@ -6783,6 +6980,15 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-result"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d1043d8214f791817bab27572aaa8af63732e11bf84aa21a45a78d6c317ae0e"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-result"
version = "0.3.4"
@ -6801,6 +7007,16 @@ dependencies = [
"windows-link 0.2.1",
]
[[package]]
name = "windows-strings"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10"
dependencies = [
"windows-result 0.2.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-strings"
version = "0.4.2"

9
src-tauri/Cargo.toml
View file

@ -1,6 +1,6 @@
[package]
name = "talkis"
version = "0.3.6"
version = "0.3.7"
description = "Talkis - Voice to Text Desktop Widget"
authors = ["trixter"]
license = "AGPL-3.0-or-later"
@ -17,7 +17,6 @@ tauri-build = { version = "2", features = [] }
[dependencies]
tauri = { version = "2", features = ["macos-private-api", "tray-icon", "image-png"] }
tauri-plugin-opener = "2"
tauri-plugin-global-shortcut = "2"
tauri-plugin-store = "2"
tauri-plugin-clipboard-manager = "2"
serde = { version = "1", features = ["derive"] }
@ -38,16 +37,18 @@ url = "2.5.8"
tauri-plugin-updater = "2.10.1"
tauri-plugin-process = "2.3.1"
tauri-plugin-shell = "2.3.2"
whisper-rs = "0.16.0"
transcribe-cpp = { version = "0.1.1", default-features = false }
hound = "3.5.1"
tauri-plugin-dialog = "2"
handy-keys = "0.2.4"
# Local LLM runtime engine (talkis-llm sidecar) — llama.cpp bindings, compiled
# and bundled with the app like whisper-rs. Metal acceleration added on macOS.
# and bundled with the app like transcribe-cpp. Metal acceleration added on macOS.
llama-cpp-2 = { version = "0.1.150", default-features = false, features = ["openmp", "sampler"] }
[target.'cfg(target_os = "macos")'.dependencies]
# Adds Metal GPU acceleration to the bundled local LLM runtime on macOS.
llama-cpp-2 = { version = "0.1.150", default-features = false, features = ["metal"] }
transcribe-cpp = { version = "0.1.1", default-features = false, features = ["metal"] }
block2 = "0.6"
objc2 = "0.6"
objc2-app-kit = { version = "0.3", features = ["NSView", "NSWindow", "NSEvent", "block2"] }

BIN
src-tauri/binaries/talkis-llm-aarch64-apple-darwin
View file

Binary file not shown.

BIN
src-tauri/binaries/talkis-stt-nvidia-x86_64-unknown-linux-gnu
View file

Binary file not shown.

BIN
src-tauri/binaries/talkis-stt-qwen-x86_64-unknown-linux-gnu
View file

Binary file not shown.

2
src-tauri/capabilities/default.json
View file

@ -15,8 +15,6 @@
"core:window:allow-show",
"opener:default",
"dialog:default",
"global-shortcut:allow-register",
"global-shortcut:allow-unregister",
"store:allow-load",
"store:allow-get",
"store:allow-set",

529
src-tauri/src/ai.rs
View file

@ -1,3 +1,11 @@
mod errors;
mod routing;
use self::routing::{
is_likely_local_url, local_runtime_kind_from_endpoint, port_from_url,
resolve_chat_completions_url, resolve_managed_models_url, resolve_managed_transcription_url,
resolve_whisper_model_download_url, resolve_whisper_models_url, resolve_whisper_url,
};
use crate::local_stt;
use crate::logger;
use crate::media;
@ -583,16 +591,15 @@ async fn transcribe_audio_bytes_internal(
}
}
let whisper_key = req
.whisper_api_key
.as_ref()
.map(|s| s.trim())
.filter(|s| !s.is_empty())
.or_else(|| {
(!is_local_endpoint)
.then_some(req.api_key.trim())
.filter(|s| !s.is_empty())
});
let whisper_key = if is_local_endpoint {
None
} else {
req.whisper_api_key
.as_ref()
.map(|s| s.trim())
.filter(|s| !s.is_empty())
.or_else(|| Some(req.api_key.trim()).filter(|s| !s.is_empty()))
};
let stt_client = if is_local_endpoint {
reqwest::Client::builder()
@ -610,20 +617,23 @@ async fn transcribe_audio_bytes_internal(
whisper_request = whisper_request.bearer_auth(whisper_key);
}
let whisper_res = whisper_request
.send()
.await
.map_err(|e| {
let err = format!("Whisper request failed: {}", e);
logger::log_error("WHISPER", &err);
err
})?;
let whisper_res = whisper_request.send().await.map_err(|e| {
let err = format!("Whisper request failed: {}", e);
logger::log_error("WHISPER", &err);
err
})?;
let status = whisper_res.status();
logger::log_info("WHISPER", &format!("Response status: {}", status));
if !status.is_success() {
let body = whisper_res.text().await.unwrap_or_default();
if is_local_endpoint && matches!(status.as_u16(), 401 | 403) {
let err = errors::local_stt_runtime_rejected_message(status.as_u16(), &body);
logger::log_error("WHISPER", &err);
return Err(err);
}
let err = format!("Whisper API error ({}): {}", status, body);
logger::log_error("WHISPER", &err);
return Err(err);
@ -916,18 +926,6 @@ struct DiarizationSegment {
speaker_id: String,
}
fn local_runtime_kind_from_endpoint(endpoint: Option<&str>) -> Option<local_stt::LocalRuntimeKind> {
let whisper_url = resolve_whisper_url(endpoint);
let models_url = resolve_whisper_models_url(&whisper_url);
local_stt::managed_runtime_kind(&models_url)
}
fn port_from_url(value: &str) -> Option<u16> {
reqwest::Url::parse(value)
.ok()
.and_then(|url| url.port_or_known_default())
}
fn is_repairable_diarization_runtime_error(message: &str) -> bool {
let normalized = message.to_lowercase();
normalized.contains("sherpa-onnx установлен")
@ -936,14 +934,19 @@ fn is_repairable_diarization_runtime_error(message: &str) -> bool {
|| normalized.contains("python runtime для разметки говорящих"))
}
fn file_transcription_uses_own_key(req: &FilePathTranscriptionRequest) -> bool {
let whisper_url = resolve_whisper_url(req.whisper_endpoint.as_deref());
req.use_own_key || is_likely_local_url(&whisper_url)
}
fn ensure_diarized_file_preconditions(req: &FilePathTranscriptionRequest) -> Result<(), String> {
if !req.use_own_key {
let kind = local_runtime_kind_from_endpoint(req.whisper_endpoint.as_deref());
if !req.use_own_key && kind.is_none() {
return Err(
"Для разделения по говорящим нужна локальная Whisper-модель с таймкодами.".to_string(),
);
}
let kind = local_runtime_kind_from_endpoint(req.whisper_endpoint.as_deref());
if kind != Some(local_stt::LocalRuntimeKind::Whisper) {
return Err(
"Для разделения по говорящим нужна локальная Whisper-модель с таймкодами.".to_string(),
@ -986,6 +989,10 @@ async fn diarize_audio_file(
let form = multipart::Form::new()
.part("file", file_part)
.text("model", local_stt::LOCAL_DIARIZATION_MODEL_ID);
logger::log_info(
"DIARIZATION",
&format!("Sending request to {}", diarization_url),
);
let response = client
.post(&diarization_url)
.multipart(form)
@ -993,12 +1000,19 @@ async fn diarize_audio_file(
.await
.map_err(|err| format!("Diarization request failed: {}", err))?;
let status = response.status();
logger::log_info("DIARIZATION", &format!("Response status: {}", status));
let body = response
.text()
.await
.map_err(|err| format!("Diarization response read failed: {}", err))?;
if !status.is_success() {
if matches!(status.as_u16(), 401 | 403) {
let message = errors::local_stt_runtime_rejected_message(status.as_u16(), &body);
logger::log_error("DIARIZATION", &message);
return Err(message);
}
return Err(format!("Diarization error ({}): {}", status, body));
}
@ -1172,6 +1186,21 @@ pub async fn transcribe_file_path(
app: AppHandle,
req: FilePathTranscriptionRequest,
) -> Result<TranscribeResponse, String> {
let whisper_url = resolve_whisper_url(req.whisper_endpoint.as_deref());
let is_local_stt_endpoint = is_likely_local_url(&whisper_url);
let effective_use_own_key = file_transcription_uses_own_key(&req);
logger::log_info(
"FILE_TRANSCRIPTION",
&format!(
"Resolved file transcription mode: local_endpoint={}, use_own_key={}, effective_use_own_key={}, speaker_diarization={}, endpoint={}",
is_local_stt_endpoint,
req.use_own_key,
effective_use_own_key,
req.speaker_diarization,
req.whisper_endpoint.as_deref().unwrap_or("")
),
);
let input_path = PathBuf::from(&req.file_path);
let metadata =
fs::metadata(&input_path).map_err(|err| format!("Не удалось прочитать файл: {}", err))?;
@ -1202,12 +1231,12 @@ pub async fn transcribe_file_path(
"Файл слишком большой. Максимальный размер для транскрибации: 8 ГБ.".to_string(),
);
}
if req.speaker_diarization && req.use_own_key {
if req.speaker_diarization && effective_use_own_key {
ensure_diarized_file_preconditions(&req)?;
}
emit_file_progress(&app, &req.request_id, "preparing", 0, 0, "Готовим файл");
if req.speaker_diarization && !req.use_own_key {
if req.speaker_diarization && !effective_use_own_key {
let prepared = media::prepare_media_file_for_proxy_transcription(&app, &input_path).await?;
emit_file_progress(
&app,
@ -1247,7 +1276,12 @@ pub async fn transcribe_file_path(
None
};
let prepared = media::prepare_media_file_chunks_for_transcription(&app, &input_path).await?;
let prepared = media::prepare_media_file_chunks_for_transcription(
&app,
&input_path,
is_local_stt_endpoint,
)
.await?;
let total_chunks = prepared.chunks.len();
emit_file_progress(
&app,
@ -1296,7 +1330,7 @@ pub async fn transcribe_file_path(
),
);
let text = if !req.use_own_key {
let text = if !effective_use_own_key {
let result = transcribe_file_chunk_via_proxy(&req, chunk).await;
let result = match result {
Ok(result) => result,
@ -1429,71 +1463,6 @@ struct ModelListItem {
id: String,
}
fn is_likely_local_url(value: &str) -> bool {
let normalized = value.trim().to_lowercase();
normalized.contains("127.0.0.1") || normalized.contains("localhost")
}
fn resolve_whisper_url(endpoint: Option<&str>) -> String {
endpoint
.filter(|s| !s.is_empty())
.map(|s| {
let base = s.trim_end_matches('/');
if base.ends_with("/transcriptions") {
base.to_string()
} else if base.ends_with("/audio") {
format!("{}/transcriptions", base)
} else {
format!("{}/v1/audio/transcriptions", base)
}
})
.unwrap_or_else(|| "https://api.openai.com/v1/audio/transcriptions".to_string())
}
fn resolve_managed_transcription_url(base_url: &str) -> String {
format!("{}/v1/audio/transcriptions", base_url.trim_end_matches('/'))
}
fn resolve_managed_models_url(base_url: &str) -> String {
format!("{}/v1/models", base_url.trim_end_matches('/'))
}
fn resolve_whisper_models_url(whisper_url: &str) -> String {
if let Some(base) = whisper_url.strip_suffix("/v1/audio/transcriptions") {
return format!("{}/v1/models", base);
}
if let Some(base) = whisper_url.strip_suffix("/audio/transcriptions") {
return format!("{}/models", base);
}
if let Some(base) = whisper_url.strip_suffix("/transcriptions") {
return format!("{}/models", base);
}
format!("{}/v1/models", whisper_url.trim_end_matches('/'))
}
/// Resolve a user/runtime endpoint to an OpenAI-compatible chat-completions URL.
/// Empty endpoint falls back to OpenAI. Accepts a base, a `/v1` base, or a full
/// `/chat/completions` URL.
fn resolve_chat_completions_url(endpoint: Option<&str>) -> String {
endpoint
.map(str::trim)
.filter(|value| !value.is_empty())
.map(|value| {
let base = value.trim_end_matches('/');
if base.ends_with("/chat/completions") {
base.to_string()
} else if base.ends_with("/v1") {
format!("{}/chat/completions", base)
} else {
format!("{}/v1/chat/completions", base)
}
})
.unwrap_or_else(|| "https://api.openai.com/v1/chat/completions".to_string())
}
#[derive(Serialize, Deserialize, Clone)]
pub struct ProcessTextRequest {
pub text: String,
@ -1630,28 +1599,6 @@ pub async fn process_text(req: ProcessTextRequest) -> Result<ProcessTextResponse
Ok(ProcessTextResponse { result })
}
fn percent_encode_path_segment(value: &str) -> String {
let mut encoded = String::with_capacity(value.len());
for byte in value.bytes() {
let is_unreserved =
byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'.' | b'_' | b'~');
if is_unreserved {
encoded.push(char::from(byte));
} else {
encoded.push('%');
encoded.push_str(&format!("{:02X}", byte));
}
}
encoded
}
fn resolve_whisper_model_download_url(models_url: &str, model: &str) -> String {
let encoded_model = percent_encode_path_segment(model);
format!("{}/{}", models_url.trim_end_matches('/'), encoded_model)
}
async fn test_stt_connection(
app: &AppHandle,
client: &reqwest::Client,
@ -1660,11 +1607,14 @@ async fn test_stt_connection(
let whisper_url = resolve_whisper_url(req.whisper_endpoint.as_deref());
let mut models_url = resolve_whisper_models_url(&whisper_url);
let managed_runtime_kind = local_stt::managed_runtime_kind(&models_url);
let whisper_key = req
.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.unwrap_or(req.api_key.as_str());
let whisper_key = if is_likely_local_url(&models_url) {
None
} else {
req.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.or_else(|| Some(req.api_key.as_str()).filter(|s| !s.trim().is_empty()))
};
if managed_runtime_kind.is_some() {
let runtime_base_url =
@ -1674,7 +1624,7 @@ async fn test_stt_connection(
}
let mut request = client.get(&models_url);
if !whisper_key.trim().is_empty() {
if let Some(whisper_key) = whisper_key {
request = request.bearer_auth(whisper_key);
}
@ -1695,6 +1645,13 @@ async fn test_stt_connection(
let status = response.status();
if !status.is_success() {
let error_text = response.text().await.unwrap_or_default();
if is_likely_local_url(&models_url) && matches!(status.as_u16(), 401 | 403) {
return Err(errors::local_stt_runtime_rejected_message(
status.as_u16(),
&error_text,
));
}
let message = match status.as_u16() {
401 => "STT endpoint отклонил API-ключ.".to_string(),
403 => "STT endpoint запретил доступ. Проверьте ключ и endpoint.".to_string(),
@ -1813,11 +1770,14 @@ pub async fn list_stt_models(
let whisper_url = resolve_whisper_url(req.whisper_endpoint.as_deref());
let mut models_url = resolve_whisper_models_url(&whisper_url);
let managed_runtime_kind = local_stt::managed_runtime_kind(&models_url);
let whisper_key = req
.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.unwrap_or(req.api_key.as_str());
let whisper_key = if is_likely_local_url(&models_url) {
None
} else {
req.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.or_else(|| Some(req.api_key.as_str()).filter(|s| !s.trim().is_empty()))
};
let client = http_client();
if managed_runtime_kind.is_some() {
@ -1830,7 +1790,7 @@ pub async fn list_stt_models(
}
let mut request = client.get(&models_url);
if !whisper_key.trim().is_empty() {
if let Some(whisper_key) = whisper_key {
request = request.bearer_auth(whisper_key);
}
@ -1954,11 +1914,14 @@ pub async fn install_stt_model(
let mut models_url = resolve_whisper_models_url(&whisper_url);
let mut download_url = resolve_whisper_model_download_url(&models_url, requested_model);
let mut effective_whisper_endpoint: Option<String> = None;
let whisper_key = req
.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.unwrap_or(req.api_key.as_str());
let whisper_key = if is_likely_local_url(&models_url) {
None
} else {
req.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.or_else(|| Some(req.api_key.as_str()).filter(|s| !s.trim().is_empty()))
};
let client = http_client();
if local_stt::is_managed_whisper_runtime_url(&models_url) {
@ -2000,10 +1963,8 @@ pub async fn install_stt_model(
if let Some(kind) = managed_runtime_kind {
if kind != local_stt::LocalRuntimeKind::Whisper {
let runtime_label = match kind {
local_stt::LocalRuntimeKind::Nvidia => "Parakeet",
local_stt::LocalRuntimeKind::Qwen => "Qwen",
local_stt::LocalRuntimeKind::Diarization => "Diarization",
local_stt::LocalRuntimeKind::Whisper => "Whisper",
local_stt::LocalRuntimeKind::Whisper => "Transcribe.cpp",
};
local_stt::emit_model_download_progress_message(
&app,
@ -2027,157 +1988,16 @@ pub async fn install_stt_model(
download_url = resolve_whisper_model_download_url(&models_url, requested_model);
}
let already_installed_snapshot = match managed_runtime_kind {
Some(local_stt::LocalRuntimeKind::Qwen)
if local_stt::qwen_model_is_installed(&app, req.local_models_dir.as_deref())
.unwrap_or(false) =>
{
Some((
local_stt::qwen_model_progress_snapshot(&app, req.local_models_dir.as_deref())
.unwrap_or((1, Some(1))),
"Модель Qwen уже скачана.",
))
}
Some(local_stt::LocalRuntimeKind::Nvidia)
if local_stt::nvidia_model_is_installed(
&app,
req.local_models_dir.as_deref(),
requested_model,
)
.unwrap_or(false) =>
{
Some((
local_stt::nvidia_model_progress_snapshot(
&app,
req.local_models_dir.as_deref(),
requested_model,
)
.unwrap_or((1, Some(1))),
"Модель Parakeet уже скачана.",
))
}
_ => None,
};
if let Some(((downloaded, total), message)) = already_installed_snapshot {
local_stt::emit_model_download_progress_message(
&app,
requested_model,
"downloaded",
downloaded,
total,
message,
);
return Ok(InstallSttModelResult {
success: true,
message: format!(
"Модель «{}» скачана и готова к локальному распознаванию.",
requested_model
),
whisper_endpoint: effective_whisper_endpoint.clone(),
});
}
let install_client =
if managed_runtime_kind.is_some_and(|kind| kind != local_stt::LocalRuntimeKind::Whisper) {
reqwest::Client::builder()
.pool_max_idle_per_host(0)
.connect_timeout(Duration::from_secs(15))
.timeout(Duration::from_secs(7200))
.build()
.unwrap_or_else(|_| (*client).clone())
} else {
(*client).clone()
};
let local_progress_stop = if matches!(
managed_runtime_kind,
Some(local_stt::LocalRuntimeKind::Qwen | local_stt::LocalRuntimeKind::Nvidia)
) {
let stop = Arc::new(AtomicBool::new(false));
let task_stop = Arc::clone(&stop);
let task_app = app.clone();
let task_model = requested_model.to_string();
let task_models_dir = req.local_models_dir.clone();
let task_kind = managed_runtime_kind.unwrap();
tokio::spawn(async move {
let mut last_percent: Option<u8> = None;
let mut last_downloaded = 0u64;
while !task_stop.load(Ordering::Relaxed) {
let snapshot = match task_kind {
local_stt::LocalRuntimeKind::Qwen => local_stt::qwen_model_progress_snapshot(
&task_app,
task_models_dir.as_deref(),
),
local_stt::LocalRuntimeKind::Nvidia => {
local_stt::nvidia_model_progress_snapshot(
&task_app,
task_models_dir.as_deref(),
&task_model,
)
}
local_stt::LocalRuntimeKind::Whisper
| local_stt::LocalRuntimeKind::Diarization => Ok((0, None)),
};
if let Ok((downloaded, total)) = snapshot {
let percent = total
.filter(|value| *value > 0)
.map(|value| ((downloaded.saturating_mul(100) / value).min(99)) as u8);
if percent != last_percent
|| downloaded.saturating_sub(last_downloaded) >= 4 * 1024 * 1024
{
local_stt::emit_model_download_progress_message(
&task_app,
&task_model,
"downloading",
downloaded,
total,
if downloaded == 0 {
match task_kind {
local_stt::LocalRuntimeKind::Nvidia => {
"Устанавливаем Parakeet зависимости."
}
local_stt::LocalRuntimeKind::Diarization => {
"Скачиваем sherpa-onnx модели diarization."
}
_ => "Устанавливаем Qwen зависимости.",
}
} else {
match task_kind {
local_stt::LocalRuntimeKind::Nvidia => {
"Скачиваем файлы Parakeet модели."
}
local_stt::LocalRuntimeKind::Diarization => {
"Скачиваем sherpa-onnx модели diarization."
}
_ => "Скачиваем файлы Qwen модели.",
}
},
);
last_percent = percent;
last_downloaded = downloaded;
}
}
tokio::time::sleep(Duration::from_secs(1)).await;
}
});
Some(stop)
} else {
None
};
let install_client = (*client).clone();
let local_progress_stop: Option<Arc<AtomicBool>> = None;
let mut request = install_client.post(&download_url);
if !whisper_key.trim().is_empty() {
if let Some(whisper_key) = whisper_key {
request = request.bearer_auth(whisper_key);
}
// The Qwen/Parakeet model download runs inside the managed Python runtime via
// this one blocking POST, so the streaming cancel flag (which Whisper/LLM poll
// per chunk) has nothing to watch here — that's why "Отмена" did nothing for
// Qwen. Race the request against the cancel flag; on abort, kill the managed
// runtime (stops the in-flight huggingface download) and reset to "not downloaded".
let cancellable_engine_download =
managed_runtime_kind.is_some_and(|kind| kind != local_stt::LocalRuntimeKind::Whisper);
managed_runtime_kind == Some(local_stt::LocalRuntimeKind::Diarization);
if cancellable_engine_download {
crate::download_cancel::clear(requested_model);
}
@ -2318,21 +2138,27 @@ pub async fn install_stt_model(
.trim()
.to_string()
});
let message = match status_code {
401 => "STT endpoint отклонил API-ключ при установке модели.".to_string(),
403 => "STT endpoint запретил установку модели. Проверьте права доступа.".to_string(),
404 => format!(
"Модель «{}» не найдена в реестре локального STT runtime.",
requested_model
),
409 => format!(
"Модель «{}» уже устанавливается или уже доступна.",
requested_model
),
_ => format!(
"STT endpoint вернул ошибку {} при установке модели: {}",
status_code, error_detail
),
let message = if is_likely_local_url(&download_url) && matches!(status_code, 401 | 403) {
errors::local_stt_runtime_rejected_message(status_code, &error_text)
} else {
match status_code {
401 => "STT endpoint отклонил API-ключ при установке модели.".to_string(),
403 => {
"STT endpoint запретил установку модели. Проверьте права доступа.".to_string()
}
404 => format!(
"Модель «{}» не найдена в реестре локального STT runtime.",
requested_model
),
409 => format!(
"Модель «{}» уже устанавливается или уже доступна.",
requested_model
),
_ => format!(
"STT endpoint вернул ошибку {} при установке модели: {}",
status_code, error_detail
),
}
};
logger::log_error("STT_INSTALL", &message);
return Ok(InstallSttModelResult {
@ -2346,24 +2172,13 @@ pub async fn install_stt_model(
"STT_INSTALL",
&format!("STT model install request accepted: {}", requested_model),
);
if managed_runtime_kind.is_some_and(|kind| kind != local_stt::LocalRuntimeKind::Whisper) {
let (downloaded, total) = match managed_runtime_kind {
Some(local_stt::LocalRuntimeKind::Nvidia) => local_stt::nvidia_model_progress_snapshot(
&app,
req.local_models_dir.as_deref(),
requested_model,
)
.unwrap_or((1, Some(1))),
Some(local_stt::LocalRuntimeKind::Diarization) => (1, Some(1)),
_ => local_stt::qwen_model_progress_snapshot(&app, req.local_models_dir.as_deref())
.unwrap_or((1, Some(1))),
};
if managed_runtime_kind == Some(local_stt::LocalRuntimeKind::Diarization) {
local_stt::emit_model_download_progress_message(
&app,
requested_model,
"downloaded",
downloaded,
total,
1,
Some(1),
"Модель скачана.",
);
}
@ -2405,11 +2220,14 @@ pub async fn delete_stt_model(
let whisper_url = resolve_whisper_url(req.whisper_endpoint.as_deref());
let mut models_url = resolve_whisper_models_url(&whisper_url);
let mut delete_url = resolve_whisper_model_download_url(&models_url, requested_model);
let whisper_key = req
.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.unwrap_or(req.api_key.as_str());
let whisper_key = if is_likely_local_url(&models_url) {
None
} else {
req.whisper_api_key
.as_deref()
.filter(|s| !s.trim().is_empty())
.or_else(|| Some(req.api_key.as_str()).filter(|s| !s.trim().is_empty()))
};
let client = http_client();
if local_stt::is_managed_whisper_runtime_url(&models_url) {
@ -2444,7 +2262,7 @@ pub async fn delete_stt_model(
}
let mut request = client.delete(&delete_url);
if !whisper_key.trim().is_empty() {
if let Some(whisper_key) = whisper_key {
request = request.bearer_auth(whisper_key);
}
@ -2468,18 +2286,24 @@ pub async fn delete_stt_model(
let status = response.status();
if !status.is_success() {
let error_text = response.text().await.unwrap_or_default();
let message = match status.as_u16() {
401 => "STT endpoint отклонил API-ключ при удалении модели.".to_string(),
403 => "STT endpoint запретил удаление модели. Проверьте права доступа.".to_string(),
404 => format!(
"Модель «{}» не найдена в реестре локального STT runtime.",
requested_model
),
_ => format!(
"STT endpoint вернул ошибку {} при удалении модели: {}",
status.as_u16(),
error_text.chars().take(200).collect::<String>()
),
let message = if is_likely_local_url(&delete_url) && matches!(status.as_u16(), 401 | 403) {
errors::local_stt_runtime_rejected_message(status.as_u16(), &error_text)
} else {
match status.as_u16() {
401 => "STT endpoint отклонил API-ключ при удалении модели.".to_string(),
403 => {
"STT endpoint запретил удаление модели. Проверьте права доступа.".to_string()
}
404 => format!(
"Модель «{}» не найдена в реестре локального STT runtime.",
requested_model
),
_ => format!(
"STT endpoint вернул ошибку {} при удалении модели: {}",
status.as_u16(),
error_text.chars().take(200).collect::<String>()
),
}
};
logger::log_error("STT_DELETE", &message);
return Ok(DeleteSttModelResult {
@ -2514,6 +2338,41 @@ mod tests {
}
}
fn file_req(use_own_key: bool, endpoint: &str) -> FilePathTranscriptionRequest {
FilePathTranscriptionRequest {
request_id: "test".to_string(),
file_path: "test.wav".to_string(),
file_name: Some("test.wav".to_string()),
file_size: None,
language: "ru".to_string(),
api_key: "stale-key".to_string(),
whisper_api_key: None,
style: "classic".to_string(),
whisper_endpoint: Some(endpoint.to_string()),
local_models_dir: None,
whisper_model: Some("whisper-large-v3-turbo".to_string()),
use_own_key,
device_token: Some("stale-cloud-token".to_string()),
speaker_diarization: true,
}
}
#[test]
fn local_file_diarization_preconditions_follow_endpoint_not_stale_mode_flag() {
let request = file_req(false, "http://127.0.0.1:8000");
assert!(ensure_diarized_file_preconditions(&request).is_ok());
}
#[test]
fn file_transcription_routing_uses_local_endpoint_over_stale_mode_flag() {
let local_request = file_req(false, "http://127.0.0.1:8000");
let remote_request = file_req(false, "https://api.openai.com/v1");
assert!(file_transcription_uses_own_key(&local_request));
assert!(!file_transcription_uses_own_key(&remote_request));
}
#[test]
fn assigns_speaker_by_maximum_overlap() {
let diarization = vec![dia(0.0, 3.0, "SPEAKER_00"), dia(3.0, 8.0, "SPEAKER_01")];

19
src-tauri/src/ai/errors.rs Normal file
View file

@ -0,0 +1,19 @@
pub(super) fn local_stt_runtime_rejected_message(status_code: u16, body: &str) -> String {
let detail = body
.chars()
.take(200)
.collect::<String>()
.trim()
.to_string();
if detail.is_empty() {
format!(
"Локальный STT runtime отклонил запрос ({}). Это не ошибка API-ключа.",
status_code
)
} else {
format!(
"Локальный STT runtime отклонил запрос ({}). Это не ошибка API-ключа: {}",
status_code, detail
)
}
}

102
src-tauri/src/ai/routing.rs Normal file
View file

@ -0,0 +1,102 @@
use crate::local_stt;
pub(super) fn is_likely_local_url(value: &str) -> bool {
let normalized = value.trim().to_lowercase();
normalized.contains("127.0.0.1") || normalized.contains("localhost")
}
pub(super) fn local_runtime_kind_from_endpoint(
endpoint: Option<&str>,
) -> Option<local_stt::LocalRuntimeKind> {
let whisper_url = resolve_whisper_url(endpoint);
let models_url = resolve_whisper_models_url(&whisper_url);
local_stt::managed_runtime_kind(&models_url)
}
pub(super) fn port_from_url(value: &str) -> Option<u16> {
reqwest::Url::parse(value)
.ok()
.and_then(|url| url.port_or_known_default())
}
pub(super) fn resolve_whisper_url(endpoint: Option<&str>) -> String {
endpoint
.filter(|s| !s.is_empty())
.map(|s| {
let base = s.trim_end_matches('/');
if base.ends_with("/transcriptions") {
base.to_string()
} else if base.ends_with("/audio") {
format!("{}/transcriptions", base)
} else {
format!("{}/v1/audio/transcriptions", base)
}
})
.unwrap_or_else(|| "https://api.openai.com/v1/audio/transcriptions".to_string())
}
pub(super) fn resolve_managed_transcription_url(base_url: &str) -> String {
format!("{}/v1/audio/transcriptions", base_url.trim_end_matches('/'))
}
pub(super) fn resolve_managed_models_url(base_url: &str) -> String {
format!("{}/v1/models", base_url.trim_end_matches('/'))
}
pub(super) fn resolve_whisper_models_url(whisper_url: &str) -> String {
if let Some(base) = whisper_url.strip_suffix("/v1/audio/transcriptions") {
return format!("{}/v1/models", base);
}
if let Some(base) = whisper_url.strip_suffix("/audio/transcriptions") {
return format!("{}/models", base);
}
if let Some(base) = whisper_url.strip_suffix("/transcriptions") {
return format!("{}/models", base);
}
format!("{}/v1/models", whisper_url.trim_end_matches('/'))
}
/// Resolve a user/runtime endpoint to an OpenAI-compatible chat-completions URL.
/// Empty endpoint falls back to OpenAI. Accepts a base, a `/v1` base, or a full
/// `/chat/completions` URL.
pub(super) fn resolve_chat_completions_url(endpoint: Option<&str>) -> String {
endpoint
.map(str::trim)
.filter(|value| !value.is_empty())
.map(|value| {
let base = value.trim_end_matches('/');
if base.ends_with("/chat/completions") {
base.to_string()
} else if base.ends_with("/v1") {
format!("{}/chat/completions", base)
} else {
format!("{}/v1/chat/completions", base)
}
})
.unwrap_or_else(|| "https://api.openai.com/v1/chat/completions".to_string())
}
pub(super) fn resolve_whisper_model_download_url(models_url: &str, model: &str) -> String {
let encoded_model = percent_encode_path_segment(model);
format!("{}/{}", models_url.trim_end_matches('/'), encoded_model)
}
fn percent_encode_path_segment(value: &str) -> String {
let mut encoded = String::with_capacity(value.len());
for byte in value.bytes() {
let is_unreserved =
byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'.' | b'_' | b'~');
if is_unreserved {
encoded.push(char::from(byte));
} else {
encoded.push('%');
encoded.push_str(&format!("{:02X}", byte));
}
}
encoded
}

87
src-tauri/src/bin/talkis-diarize.rs
View file

@ -713,6 +713,7 @@ fn validate_sherpa_python_runtime(config: &RuntimeConfig, python: &Path) -> bool
fn prepare_sherpa_venv(venv_dir: &Path) -> Result<(), (u16, String)> {
let venv_python = venv_python_path(venv_dir);
let mut force_portable_python = false;
if venv_dir.is_dir() && !venv_is_usable(&venv_python) {
fs::remove_dir_all(venv_dir).map_err(|err| {
@ -724,57 +725,72 @@ fn prepare_sherpa_venv(venv_dir: &Path) -> Result<(), (u16, String)> {
),
)
})?;
force_portable_python = true;
}
if !venv_python.is_file() {
let python = find_system_python()
.map(Ok)
.unwrap_or_else(|| install_portable_python(venv_dir))?;
let output = Command::new(&python)
.args(["-m", "venv"])
.arg(venv_dir)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.map_err(|err| {
(
if let Err(first_error) = create_sherpa_venv(venv_dir, force_portable_python) {
if force_portable_python {
return Err(first_error);
}
let _ = fs::remove_dir_all(venv_dir);
if let Err(second_error) = create_sherpa_venv(venv_dir, true) {
return Err((
500,
format!("Не удалось создать Python venv для sherpa-onnx: {}", err),
)
})?;
if !output.status.success() {
format!(
"{}\nПовтор через встроенный Python runtime тоже не удался: {}",
first_error.1, second_error.1
),
));
}
}
}
if let Err(first_error) = install_sherpa_dependencies(&venv_python) {
let _ = fs::remove_dir_all(venv_dir);
create_sherpa_venv(venv_dir, true)?;
if let Err(second_error) = install_sherpa_dependencies(&venv_python_path(venv_dir)) {
return Err((
500,
format!(
"Не удалось создать Python venv для sherpa-onnx: {}",
command_output_message(
&output,
"python -m venv завершился без диагностического текста. На Debian/Ubuntu установите пакет python3.12-venv."
)
"Не удалось установить sherpa-onnx dependencies: {}\nПовтор через встроенный Python runtime тоже не удался: {}",
first_error, second_error
),
));
}
}
let output = Command::new(&venv_python)
.args(["-m", "pip", "install", "--upgrade", "pip", "sherpa-onnx"])
Ok(())
}
fn create_sherpa_venv(venv_dir: &Path, force_portable_python: bool) -> Result<(), (u16, String)> {
let python = if force_portable_python {
install_portable_python(venv_dir)?
} else {
find_system_python()
.map(Ok)
.unwrap_or_else(|| install_portable_python(venv_dir))?
};
let output = Command::new(&python)
.args(["-m", "venv"])
.arg(venv_dir)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.map_err(|err| {
(
500,
format!("Не удалось установить sherpa-onnx dependencies: {}", err),
format!("Не удалось создать Python venv для sherpa-onnx: {}", err),
)
})?;
if !output.status.success() {
return Err((
500,
format!(
"Не удалось установить sherpa-onnx dependencies: {}",
"Не удалось создать Python venv для sherpa-onnx: {}",
command_output_message(
&output,
"python -m pip завершился без диагностического текста. Проверьте, что в Python venv доступен pip."
"python -m venv завершился без диагностического текста. На Debian/Ubuntu установите пакет python3.12-venv."
)
),
));
@ -783,6 +799,23 @@ fn prepare_sherpa_venv(venv_dir: &Path) -> Result<(), (u16, String)> {
Ok(())
}
fn install_sherpa_dependencies(venv_python: &Path) -> Result<(), String> {
let output = Command::new(venv_python)
.args(["-m", "pip", "install", "--upgrade", "pip", "sherpa-onnx"])
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.map_err(|err| format!("Не удалось установить sherpa-onnx dependencies: {}", err))?;
if output.status.success() {
Ok(())
} else {
Err(command_output_message(
&output,
"python -m pip завершился без диагностического текста. Проверьте, что в Python venv доступен pip.",
))
}
}
fn command_output_message(output: &std::process::Output, fallback: &str) -> String {
let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string();
let stdout = String::from_utf8_lossy(&output.stdout).trim().to_string();
@ -813,7 +846,7 @@ fn venv_python_path(venv_dir: &Path) -> PathBuf {
fn venv_is_usable(python: &Path) -> bool {
python_is_supported_path(python)
&& Command::new(python)
.args(["-m", "pip", "--version"])
.args(["-c", "import pip, pyexpat, xml.parsers.expat"])
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
@ -1034,7 +1067,7 @@ fn python_is_supported(name: &str) -> bool {
fn python_supports_venv(name: &str) -> bool {
Command::new(name)
.args(["-c", "import venv, ensurepip"])
.args(["-c", "import venv, ensurepip, pyexpat, xml.parsers.expat"])
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()

1085
src-tauri/src/bin/talkis-stt-nvidia.rs
View file

File diff suppressed because it is too large Load diff

1050
src-tauri/src/bin/talkis-stt-qwen.rs
View file

File diff suppressed because it is too large Load diff

395
src-tauri/src/bin/talkis-stt.rs
View file

@ -6,13 +6,13 @@ use std::fs;
use std::io::{Cursor, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::path::PathBuf;
use whisper_rs::{
convert_integer_to_float_audio, convert_stereo_to_mono_audio, FullParams, SamplingStrategy,
WhisperContext, WhisperContextParameters,
};
use std::sync::Once;
use transcribe_cpp::{ExtSlot, Model, RunExtension, RunOptions, TimestampKind, WhisperRunOptions};
const SERVER_NAME: &str = "talkis-stt";
const MAX_REQUEST_BYTES: usize = 128 * 1024 * 1024;
const WHISPER_RUN_EXT_KIND: u32 = 0x4E524857;
static INIT_TRANSCRIBE_CPP: Once = Once::new();
struct RuntimeConfig {
host: String,
@ -32,6 +32,7 @@ struct WhisperModel {
id: &'static str,
aliases: &'static [&'static str],
file_name: &'static str,
gguf_file_name: &'static str,
url: &'static str,
}
@ -43,38 +44,80 @@ struct MultipartData {
const WHISPER_MODELS: &[WhisperModel] = &[
WhisperModel {
id: "whisper-tiny",
aliases: &["tiny", "Systran/faster-whisper-tiny"],
aliases: &[
"tiny",
"Systran/faster-whisper-tiny",
"openai/whisper-tiny",
"ggml-tiny.bin",
"whisper-tiny.gguf",
],
file_name: "ggml-tiny.bin",
gguf_file_name: "whisper-tiny.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny.bin",
},
WhisperModel {
id: "whisper-base",
aliases: &["base", "Systran/faster-whisper-base"],
aliases: &[
"base",
"Systran/faster-whisper-base",
"openai/whisper-base",
"ggml-base.bin",
"whisper-base.gguf",
],
file_name: "ggml-base.bin",
gguf_file_name: "whisper-base.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin",
},
WhisperModel {
id: "whisper-small",
aliases: &["small", "Systran/faster-whisper-small"],
aliases: &[
"small",
"Systran/faster-whisper-small",
"openai/whisper-small",
"ggml-small.bin",
"whisper-small.gguf",
],
file_name: "ggml-small.bin",
gguf_file_name: "whisper-small.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin",
},
WhisperModel {
id: "whisper-medium",
aliases: &["medium", "Systran/faster-whisper-medium"],
aliases: &[
"medium",
"Systran/faster-whisper-medium",
"openai/whisper-medium",
"ggml-medium.bin",
"whisper-medium.gguf",
],
file_name: "ggml-medium.bin",
gguf_file_name: "whisper-medium.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-medium.bin",
},
WhisperModel {
id: "whisper-large-v2",
aliases: &["large-v2", "Systran/faster-whisper-large-v2"],
aliases: &[
"large-v2",
"Systran/faster-whisper-large-v2",
"openai/whisper-large-v2",
"ggml-large-v2.bin",
"whisper-large-v2.gguf",
],
file_name: "ggml-large-v2.bin",
gguf_file_name: "whisper-large-v2.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v2.bin",
},
WhisperModel {
id: "whisper-large-v3",
aliases: &["large-v3", "Systran/faster-whisper-large-v3"],
aliases: &[
"large-v3",
"Systran/faster-whisper-large-v3",
"openai/whisper-large-v3",
"ggml-large-v3.bin",
"whisper-large-v3.gguf",
],
file_name: "ggml-large-v3.bin",
gguf_file_name: "whisper-large-v3.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3.bin",
},
WhisperModel {
@ -83,13 +126,55 @@ const WHISPER_MODELS: &[WhisperModel] = &[
"large-v3-turbo",
"Systran/faster-whisper-large-v3-turbo",
"mlx-community/whisper-large-v3-turbo-4bit",
"openai/whisper-large-v3-turbo",
"ggml-large-v3-turbo.bin",
"whisper-large-v3-turbo.gguf",
],
file_name: "ggml-large-v3-turbo.bin",
gguf_file_name: "whisper-large-v3-turbo.gguf",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin",
},
WhisperModel {
id: "nvidia/parakeet-tdt-0.6b-v3",
aliases: &[
"mlx-community/parakeet-tdt-0.6b-v3",
"parakeet-tdt-06b-v3",
"parakeet-tdt-0.6b-v3",
"parakeet-tdt-0.6b-v3-Q8_0.gguf",
],
file_name: "parakeet-tdt-0.6b-v3-Q8_0.gguf",
gguf_file_name: "parakeet-tdt-0.6b-v3-Q8_0.gguf",
url: "https://huggingface.co/handy-computer/parakeet-tdt-0.6b-v3-gguf/resolve/main/parakeet-tdt-0.6b-v3-Q8_0.gguf",
},
WhisperModel {
id: "nvidia/parakeet-tdt-0.6b-v2",
aliases: &[
"mlx-community/parakeet-tdt-0.6b-v2",
"parakeet-tdt-06b-v2",
"parakeet-tdt-0.6b-v2",
"parakeet-tdt-0.6b-v2-Q8_0.gguf",
],
file_name: "parakeet-tdt-0.6b-v2-Q8_0.gguf",
gguf_file_name: "parakeet-tdt-0.6b-v2-Q8_0.gguf",
url: "https://huggingface.co/handy-computer/parakeet-tdt-0.6b-v2-gguf/resolve/main/parakeet-tdt-0.6b-v2-Q8_0.gguf",
},
WhisperModel {
id: "Qwen/Qwen3-ASR-0.6B",
aliases: &[
"qwen3-asr-06b",
"Qwen3-ASR-0.6B",
"qwen3-asr-0.6b",
"Qwen3-ASR-0.6B-Q8_0.gguf",
],
file_name: "Qwen3-ASR-0.6B-Q8_0.gguf",
gguf_file_name: "Qwen3-ASR-0.6B-Q8_0.gguf",
url: "https://huggingface.co/handy-computer/Qwen3-ASR-0.6B-gguf/resolve/main/Qwen3-ASR-0.6B-Q8_0.gguf",
},
];
fn main() {
init_transcribe_cpp();
let config = parse_args();
if let Err(err) = fs::create_dir_all(&config.data_dir) {
eprintln!("failed to prepare data dir: {}", err);
@ -260,7 +345,7 @@ fn route_request(request: &HttpRequest, config: &RuntimeConfig) -> (u16, String)
json!({
"status": "ok",
"runtime": SERVER_NAME,
"engine": "whisper.cpp"
"engine": "transcribe.cpp"
})
.to_string(),
),
@ -341,19 +426,28 @@ fn find_model(value: &str) -> Option<&'static WhisperModel> {
}
fn model_path(config: &RuntimeConfig, model: &WhisperModel) -> PathBuf {
config.models_dir.join(model.file_name)
let gguf_path = config.models_dir.join(model.gguf_file_name);
if gguf_path.is_file() {
gguf_path
} else {
config.models_dir.join(model.file_name)
}
}
fn marker_path(config: &RuntimeConfig, model: &WhisperModel) -> PathBuf {
config.models_dir.join(format!("{}.json", model.id))
}
fn download_model_path(config: &RuntimeConfig, model: &WhisperModel) -> PathBuf {
config.models_dir.join(model.file_name)
}
fn install_model(config: &RuntimeConfig, requested: &str) -> Result<String, (u16, String)> {
let model = find_model(requested).ok_or_else(|| {
(
404,
format!(
"Модель «{}» не поддерживается встроенным Whisper runtime.",
"Модель «{}» не поддерживается встроенным transcribe.cpp runtime.",
requested
),
)
@ -366,7 +460,7 @@ fn install_model(config: &RuntimeConfig, requested: &str) -> Result<String, (u16
)
})?;
let path = model_path(config, model);
let path = download_model_path(config, model);
if !path.is_file() {
let temp_path = path.with_extension("download");
let mut response = reqwest::blocking::get(model.url)
@ -416,13 +510,13 @@ fn delete_model(config: &RuntimeConfig, requested: &str) -> Result<String, (u16,
(
404,
format!(
"Модель «{}» не поддерживается встроенным Whisper runtime.",
"Модель «{}» не поддерживается встроенным transcribe.cpp runtime.",
requested
),
)
})?;
let path = model_path(config, model);
let path = download_model_path(config, model);
if path.is_file() {
fs::remove_file(&path).map_err(|err| {
(
@ -457,7 +551,7 @@ fn write_model_marker(config: &RuntimeConfig, model: &WhisperModel) -> Result<()
let marker = json!({
"id": model.id,
"file": model.file_name,
"engine": "whisper.cpp"
"engine": "transcribe.cpp"
});
fs::write(marker_path(config, model), marker.to_string()).map_err(|err| {
(
@ -473,7 +567,10 @@ fn write_model_marker(config: &RuntimeConfig, model: &WhisperModel) -> Result<()
fn installed_models(config: &RuntimeConfig) -> Vec<String> {
let mut models = WHISPER_MODELS
.iter()
.filter(|model| model_path(config, model).is_file())
.filter(|model| {
config.models_dir.join(model.file_name).is_file()
|| config.models_dir.join(model.gguf_file_name).is_file()
})
.map(|model| model.id.to_string())
.collect::<Vec<_>>();
models.sort();
@ -491,7 +588,7 @@ fn transcribe(request: &HttpRequest, config: &RuntimeConfig) -> Result<String, (
(
404,
format!(
"Модель «{}» не поддерживается встроенным Whisper runtime.",
"Модель «{}» не поддерживается встроенным transcribe.cpp runtime.",
requested_model
),
)
@ -503,66 +600,115 @@ fn transcribe(request: &HttpRequest, config: &RuntimeConfig) -> Result<String, (
}
let audio = read_wav_mono_16k(&multipart.file)?;
let context = WhisperContext::new_with_params(&path, WhisperContextParameters::default())
.map_err(|err| {
(
500,
format!("Не удалось загрузить модель «{}»: {}", model.id, err),
)
})?;
let mut state = context.create_state().map_err(|err| {
(
500,
format!("Не удалось создать состояние Whisper: {}", err),
)
})?;
let mut params = FullParams::new(SamplingStrategy::Greedy { best_of: 0 });
params.set_n_threads(
std::thread::available_parallelism()
.map(|value| value.get().min(8) as i32)
.unwrap_or(4),
);
params.set_print_special(false);
params.set_print_progress(false);
params.set_print_realtime(false);
params.set_print_timestamps(false);
params.set_no_context(true);
params.set_suppress_nst(true);
params.set_temperature(0.0);
params.set_temperature_inc(0.0);
params.set_entropy_thold(2.2);
if let Some(language) = multipart.fields.get("language") {
let language = language.trim();
if !language.is_empty() && language != "auto" {
params.set_language(Some(language));
}
if is_low_signal_audio(&audio) {
return transcription_response(String::new(), Vec::new(), &multipart.fields);
}
state
.full(params, &audio)
.map_err(|err| (500, format!("Whisper не смог распознать аудио: {}", err)))?;
let loaded_model = Model::load(&path).map_err(|err| {
(
500,
format!(
"Не удалось загрузить модель «{}» через transcribe.cpp: {}",
model.id, err
),
)
})?;
let capabilities = loaded_model.capabilities();
let supports_whisper_options = loaded_model.accepts_ext(ExtSlot::Run, WHISPER_RUN_EXT_KIND);
let mut session = loaded_model.session().map_err(|err| {
(
500,
format!("Не удалось создать transcribe.cpp session: {}", err),
)
})?;
let options = run_options_from_fields(
&multipart.fields,
capabilities.max_timestamp_kind,
supports_whisper_options,
);
let transcript = session.run(&audio, &options).map_err(|err| {
(
500,
format!("transcribe.cpp не смог распознать аудио: {}", err),
)
})?;
let segments = state
.as_iter()
let segments = transcript
.segments
.iter()
.map(|segment| {
let text = segment.to_string();
json!({
"start": segment.start_timestamp() as f64 / 100.0,
"end": segment.end_timestamp() as f64 / 100.0,
"text": text.trim()
"start": segment.t0_ms as f64 / 1000.0,
"end": segment.t1_ms as f64 / 1000.0,
"text": segment.text.trim()
})
})
.collect::<Vec<_>>();
let text = state
.as_iter()
.map(|segment| segment.to_string())
.collect::<Vec<_>>()
.join("")
.trim()
.to_string();
let response_format = multipart
.fields
let text = transcript.text.trim().to_string();
transcription_response(text, segments, &multipart.fields)
}
fn init_transcribe_cpp() {
INIT_TRANSCRIBE_CPP.call_once(|| {
transcribe_cpp::init_logging();
if let Err(err) = transcribe_cpp::init_backends_default() {
eprintln!("failed to initialize transcribe.cpp backends: {}", err);
return;
}
let devices = transcribe_cpp::devices()
.into_iter()
.map(|device| format!("{} ({})", device.name, device.kind))
.collect::<Vec<_>>()
.join(", ");
eprintln!(
"{} using transcribe.cpp {} devices=[{}]",
SERVER_NAME,
transcribe_cpp::version(),
devices
);
});
}
fn run_options_from_fields(
fields: &HashMap<String, String>,
max_timestamp_kind: TimestampKind,
supports_whisper_options: bool,
) -> RunOptions {
let language = fields
.get("language")
.map(|language| language.trim())
.filter(|language| !language.is_empty() && *language != "auto")
.map(ToString::to_string);
RunOptions {
timestamps: match max_timestamp_kind {
TimestampKind::None => TimestampKind::None,
_ => TimestampKind::Segment,
},
language,
family: supports_whisper_options.then(|| {
RunExtension::Whisper(WhisperRunOptions {
condition_on_prev_tokens: Some(false),
temperature: Some(0.0),
temperature_inc: Some(0.0),
compression_ratio_thold: Some(2.2),
logprob_thold: Some(-1.0),
no_speech_thold: Some(0.6),
max_prev_context_tokens: Some(0),
..Default::default()
})
}),
..Default::default()
}
}
fn transcription_response(
text: String,
segments: Vec<serde_json::Value>,
fields: &HashMap<String, String>,
) -> Result<String, (u16, String)> {
let response_format = fields
.get("response_format")
.map(|value| value.trim())
.unwrap_or("json");
@ -574,6 +720,23 @@ fn transcribe(request: &HttpRequest, config: &RuntimeConfig) -> Result<String, (
}
}
fn is_low_signal_audio(audio: &[f32]) -> bool {
if audio.is_empty() {
return true;
}
let mut peak = 0.0f32;
let mut sum_squares = 0.0f64;
for sample in audio {
let abs = sample.abs();
peak = peak.max(abs);
sum_squares += (*sample as f64) * (*sample as f64);
}
let rms = (sum_squares / audio.len() as f64).sqrt() as f32;
peak < 0.001 && rms < 0.0001
}
fn parse_multipart(request: &HttpRequest) -> Result<MultipartData, (u16, String)> {
let content_type = request.headers.get("content-type").ok_or_else(|| {
(
@ -642,21 +805,20 @@ fn read_wav_mono_16k(bytes: &[u8]) -> Result<Vec<f32>, (u16, String)> {
.into_samples::<i16>()
.collect::<Result<Vec<_>, _>>()
.map_err(|err| (400, format!("Не удалось прочитать WAV samples: {}", err)))?;
let mut audio = vec![0.0f32; samples.len()];
convert_integer_to_float_audio(&samples, &mut audio).map_err(|err| {
(
400,
format!("Не удалось конвертировать WAV samples: {}", err),
)
})?;
if channels == 1 {
Ok(audio)
Ok(samples
.into_iter()
.map(|sample| sample as f32 / i16::MAX as f32)
.collect())
} else {
let mut mono = vec![0.0f32; audio.len() / 2];
convert_stereo_to_mono_audio(&audio, &mut mono)
.map_err(|err| (400, format!("Не удалось сделать mono WAV: {}", err)))?;
Ok(mono)
Ok(samples
.chunks_exact(2)
.map(|frame| {
let left = frame[0] as f32 / i16::MAX as f32;
let right = frame[1] as f32 / i16::MAX as f32;
(left + right) * 0.5
})
.collect())
}
}
@ -737,3 +899,68 @@ fn percent_decode(value: &str) -> String {
String::from_utf8_lossy(&decoded).to_string()
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::{SystemTime, UNIX_EPOCH};
#[test]
fn low_signal_audio_is_detected_before_model_run() {
assert!(is_low_signal_audio(&vec![0.0; 16_000]));
assert!(is_low_signal_audio(&vec![0.00005; 16_000]));
assert!(!is_low_signal_audio(&vec![0.01; 16_000]));
}
#[test]
fn run_options_keep_auto_language_empty() {
let mut fields = HashMap::new();
fields.insert("language".to_string(), "auto".to_string());
let options = run_options_from_fields(&fields, TimestampKind::Segment, true);
assert_eq!(options.language, None);
assert_eq!(options.timestamps, TimestampKind::Segment);
}
#[test]
fn run_options_pass_explicit_language() {
let mut fields = HashMap::new();
fields.insert("language".to_string(), "ru".to_string());
let options = run_options_from_fields(&fields, TimestampKind::Segment, true);
assert_eq!(options.language.as_deref(), Some("ru"));
}
#[test]
fn model_path_prefers_existing_gguf_over_legacy_bin() {
let root = env::temp_dir().join(format!(
"talkis-stt-test-{}",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("time")
.as_nanos()
));
let models_dir = root.join("models");
fs::create_dir_all(&models_dir).expect("models dir");
let model = find_model("whisper-base").expect("model");
let legacy_path = models_dir.join(model.file_name);
let gguf_path = models_dir.join(model.gguf_file_name);
fs::write(&legacy_path, b"legacy").expect("legacy");
fs::write(&gguf_path, b"gguf").expect("gguf");
let config = RuntimeConfig {
host: "127.0.0.1".to_string(),
port: 0,
data_dir: root.clone(),
models_dir,
};
assert_eq!(model_path(&config, model), gguf_path);
assert_eq!(download_model_path(&config, model), legacy_path);
let _ = fs::remove_dir_all(root);
}
}

290
src-tauri/src/hotkey_manager.rs Normal file
View file

@ -0,0 +1,290 @@
use std::sync::mpsc::{self, Receiver, Sender};
use std::sync::Mutex;
use std::thread::{self, JoinHandle};
use handy_keys::{Hotkey, HotkeyId, HotkeyManager, HotkeyState};
use serde::Serialize;
use tauri::{AppHandle, Emitter, Manager};
use crate::logger;
pub const HANDY_HOTKEY_EVENT: &str = "handy-hotkey-event";
#[derive(Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct HandyHotkeyEventPayload {
hotkey: String,
state: String,
}
enum HotkeyCommand {
Register {
hotkey: String,
response: Sender<Result<(), String>>,
},
Unregister {
hotkey: Option<String>,
response: Sender<Result<(), String>>,
},
Shutdown,
}
pub struct HandyHotkeyState {
command_sender: Mutex<Sender<HotkeyCommand>>,
thread_handle: Mutex<Option<JoinHandle<()>>>,
}
struct ActiveHotkey {
id: HotkeyId,
value: String,
}
impl HandyHotkeyState {
pub fn new(app: AppHandle) -> Self {
let (command_sender, command_receiver) = mpsc::channel();
let thread_handle = thread::spawn(move || run_hotkey_thread(app, command_receiver));
Self {
command_sender: Mutex::new(command_sender),
thread_handle: Mutex::new(Some(thread_handle)),
}
}
fn send_command(&self, command: HotkeyCommand) -> Result<(), String> {
self.command_sender
.lock()
.map_err(|_| "Не удалось заблокировать hotkey sender".to_string())?
.send(command)
.map_err(|_| "Hotkey manager не запущен".to_string())
}
}
impl Drop for HandyHotkeyState {
fn drop(&mut self) {
if let Ok(sender) = self.command_sender.lock() {
let _ = sender.send(HotkeyCommand::Shutdown);
}
if let Ok(mut handle) = self.thread_handle.lock() {
if let Some(handle) = handle.take() {
let _ = handle.join();
}
}
}
}
pub fn init(app: &tauri::App) {
app.manage(HandyHotkeyState::new(app.handle().clone()));
}
#[tauri::command]
pub fn register_handy_hotkey(app: AppHandle, hotkey: String) -> Result<(), String> {
let state = app
.try_state::<HandyHotkeyState>()
.ok_or_else(|| "Handy hotkey manager не инициализирован".to_string())?;
let (response_sender, response_receiver) = mpsc::channel();
state.send_command(HotkeyCommand::Register {
hotkey,
response: response_sender,
})?;
response_receiver
.recv()
.map_err(|_| "Hotkey manager не вернул ответ".to_string())?
}
#[tauri::command]
pub fn unregister_handy_hotkey(app: AppHandle, hotkey: Option<String>) -> Result<(), String> {
let state = app
.try_state::<HandyHotkeyState>()
.ok_or_else(|| "Handy hotkey manager не инициализирован".to_string())?;
let (response_sender, response_receiver) = mpsc::channel();
state.send_command(HotkeyCommand::Unregister {
hotkey,
response: response_sender,
})?;
response_receiver
.recv()
.map_err(|_| "Hotkey manager не вернул ответ".to_string())?
}
fn run_hotkey_thread(app: AppHandle, command_receiver: Receiver<HotkeyCommand>) {
logger::log_info("HOTKEY", "Starting handy-keys manager thread");
let manager_result = HotkeyManager::new_with_blocking()
.map_err(|err| format!("Не удалось запустить handy-keys: {}", err));
let mut active_hotkey: Option<ActiveHotkey> = None;
loop {
if let Ok(manager) = manager_result.as_ref() {
while let Some(event) = manager.try_recv() {
let Some(active) = active_hotkey.as_ref() else {
continue;
};
if event.id != active.id {
continue;
}
let state = match event.state {
HotkeyState::Pressed => "Pressed",
HotkeyState::Released => "Released",
};
let payload = HandyHotkeyEventPayload {
hotkey: active.value.clone(),
state: state.to_string(),
};
let _ = app.emit(HANDY_HOTKEY_EVENT, payload);
}
}
match command_receiver.recv_timeout(std::time::Duration::from_millis(10)) {
Ok(command) => {
if matches!(command, HotkeyCommand::Shutdown) {
break;
}
let result = match manager_result.as_ref() {
Ok(manager) => handle_command(manager, command, &mut active_hotkey),
Err(err) => reply_manager_error(command, err),
};
if let Err(err) = result {
logger::log_error("HOTKEY", &err);
}
}
Err(mpsc::RecvTimeoutError::Timeout) => {}
Err(mpsc::RecvTimeoutError::Disconnected) => break,
}
}
logger::log_info("HOTKEY", "Stopped handy-keys manager thread");
}
fn handle_command(
manager: &HotkeyManager,
command: HotkeyCommand,
active_hotkey: &mut Option<ActiveHotkey>,
) -> Result<(), String> {
match command {
HotkeyCommand::Register { hotkey, response } => {
let result = register_hotkey(manager, active_hotkey, &hotkey);
let _ = response.send(result.clone());
result
}
HotkeyCommand::Unregister { hotkey, response } => {
let result = unregister_hotkey(manager, active_hotkey, hotkey.as_deref());
let _ = response.send(result.clone());
result
}
HotkeyCommand::Shutdown => Ok(()),
}
}
fn reply_manager_error(command: HotkeyCommand, err: &str) -> Result<(), String> {
let result = Err(err.to_string());
match command {
HotkeyCommand::Register { response, .. } | HotkeyCommand::Unregister { response, .. } => {
let _ = response.send(result.clone());
}
HotkeyCommand::Shutdown => {}
}
result
}
fn register_hotkey(
manager: &HotkeyManager,
active_hotkey: &mut Option<ActiveHotkey>,
raw_hotkey: &str,
) -> Result<(), String> {
let hotkey_value = normalize_for_handy_keys(raw_hotkey);
if active_hotkey
.as_ref()
.is_some_and(|active| active.value == hotkey_value)
{
return Ok(());
}
let hotkey = hotkey_value.parse::<Hotkey>().map_err(|err| {
format!(
"Не удалось разобрать горячую клавишу «{}»: {}",
raw_hotkey, err
)
})?;
let next_id = manager
.register(hotkey)
.map_err(|err| format!("Не удалось зарегистрировать «{}»: {}", raw_hotkey, err))?;
if let Some(previous) = active_hotkey.replace(ActiveHotkey {
id: next_id,
value: hotkey_value.clone(),
}) {
let _ = manager.unregister(previous.id);
}
logger::log_info(
"HOTKEY",
&format!("Registered handy-keys hotkey: {}", hotkey_value),
);
Ok(())
}
fn unregister_hotkey(
manager: &HotkeyManager,
active_hotkey: &mut Option<ActiveHotkey>,
raw_hotkey: Option<&str>,
) -> Result<(), String> {
let should_unregister = match (active_hotkey.as_ref(), raw_hotkey) {
(None, _) => false,
(Some(_), None) => true,
(Some(active), Some(value)) => active.value == normalize_for_handy_keys(value),
};
if !should_unregister {
return Ok(());
}
if let Some(previous) = active_hotkey.take() {
manager
.unregister(previous.id)
.map_err(|err| format!("Не удалось снять hotkey «{}»: {}", previous.value, err))?;
logger::log_info(
"HOTKEY",
&format!("Unregistered handy-keys hotkey: {}", previous.value),
);
}
Ok(())
}
fn normalize_for_handy_keys(raw_hotkey: &str) -> String {
raw_hotkey
.split('+')
.map(str::trim)
.filter(|part| !part.is_empty())
.map(|part| match part.to_ascii_lowercase().as_str() {
"control" => "Ctrl".to_string(),
"command" | "meta" => "Cmd".to_string(),
"option" => "Alt".to_string(),
other if other.len() == 1 => other.to_ascii_uppercase(),
_ => part.to_string(),
})
.collect::<Vec<_>>()
.join("+")
}
#[cfg(test)]
mod tests {
use super::normalize_for_handy_keys;
#[test]
fn normalizes_talkis_hotkey_labels_for_handy_keys() {
assert_eq!(
normalize_for_handy_keys("Control+Alt+Shift+Space"),
"Ctrl+Alt+Shift+Space"
);
assert_eq!(normalize_for_handy_keys("Command+K"), "Cmd+K");
}
}

11
src-tauri/src/lib.rs
View file

@ -4,6 +4,7 @@ mod commands;
mod download_cancel;
mod history_storage;
mod hotkey_capture;
mod hotkey_manager;
mod llm_runtime;
mod local_stt;
mod logger;
@ -52,9 +53,9 @@ pub fn run() {
// would otherwise start another process with its own floating widget.
#[cfg(any(windows, target_os = "linux"))]
{
builder = builder.plugin(tauri_plugin_single_instance::init(
|app, _argv, _cwd| focus_existing_instance(app),
));
builder = builder.plugin(tauri_plugin_single_instance::init(|app, _argv, _cwd| {
focus_existing_instance(app)
}));
}
builder
@ -63,12 +64,12 @@ pub fn run() {
.plugin(tauri_plugin_opener::init())
.plugin(tauri_plugin_process::init())
.plugin(tauri_plugin_shell::init())
.plugin(tauri_plugin_global_shortcut::Builder::new().build())
.plugin(tauri_plugin_store::Builder::default().build())
.plugin(tauri_plugin_clipboard_manager::init())
.plugin(tauri_plugin_updater::Builder::new().build())
.setup(|app| {
logger::log_info("INIT", "Application starting...");
hotkey_manager::init(app);
#[cfg(desktop)]
app.handle().plugin(tauri_plugin_autostart::init(
@ -188,6 +189,8 @@ pub fn run() {
get_cleanup_prompt_preview,
start_native_hotkey_capture,
stop_native_hotkey_capture,
hotkey_manager::register_handy_hotkey,
hotkey_manager::unregister_handy_hotkey,
])
.run(tauri::generate_context!())
.expect("error while running Talkis");

334
src-tauri/src/local_stt.rs
View file

@ -12,8 +12,6 @@ use tokio::io::AsyncWriteExt;
use tokio::process::Command as TokioCommand;
const WHISPER_RUNTIME_NAME: &str = "talkis-stt";
const NVIDIA_RUNTIME_NAME: &str = "talkis-stt-nvidia";
const QWEN_RUNTIME_NAME: &str = "talkis-stt-qwen";
const DIARIZATION_RUNTIME_NAME: &str = "talkis-diarize";
const DEFAULT_RUNTIME_MANIFEST_URL: &str = "https://talkis.ru/downloads/talkis-stt/manifest.json";
pub const MODEL_DOWNLOAD_PROGRESS_EVENT: &str = "local-stt-model-download-progress";
@ -22,8 +20,6 @@ pub const LOCAL_DIARIZATION_MODEL_ID: &str = "sherpa-diarization-pyannote-titane
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum LocalRuntimeKind {
Whisper,
Nvidia,
Qwen,
Diarization,
}
@ -31,17 +27,13 @@ impl LocalRuntimeKind {
fn runtime_name(self) -> &'static str {
match self {
LocalRuntimeKind::Whisper => WHISPER_RUNTIME_NAME,
LocalRuntimeKind::Nvidia => NVIDIA_RUNTIME_NAME,
LocalRuntimeKind::Qwen => QWEN_RUNTIME_NAME,
LocalRuntimeKind::Diarization => DIARIZATION_RUNTIME_NAME,
}
}
fn engine_name(self) -> &'static str {
match self {
LocalRuntimeKind::Whisper => "whisper.cpp",
LocalRuntimeKind::Nvidia => "parakeet-mlx",
LocalRuntimeKind::Qwen => "qwen-asr",
LocalRuntimeKind::Whisper => "transcribe.cpp",
LocalRuntimeKind::Diarization => "sherpa-onnx",
}
}
@ -49,8 +41,6 @@ impl LocalRuntimeKind {
fn default_port(self) -> u16 {
match self {
LocalRuntimeKind::Whisper => 8000,
LocalRuntimeKind::Nvidia => 8001,
LocalRuntimeKind::Qwen => 8002,
LocalRuntimeKind::Diarization => 8003,
}
}
@ -58,8 +48,6 @@ impl LocalRuntimeKind {
fn label(self) -> &'static str {
match self {
LocalRuntimeKind::Whisper => "Whisper",
LocalRuntimeKind::Nvidia => "NVIDIA",
LocalRuntimeKind::Qwen => "Qwen",
LocalRuntimeKind::Diarization => "Diarization",
}
}
@ -121,17 +109,6 @@ struct LocalModelInfo {
url: &'static str,
}
struct QwenModelFile {
file_name: &'static str,
size: u64,
}
struct NvidiaModelInfo {
id: &'static str,
dir_name: &'static str,
files: &'static [QwenModelFile],
}
const LOCAL_WHISPER_MODELS: &[LocalModelInfo] = &[
LocalModelInfo {
id: "whisper-tiny",
@ -168,132 +145,24 @@ const LOCAL_WHISPER_MODELS: &[LocalModelInfo] = &[
file_name: "ggml-large-v3-turbo.bin",
url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin",
},
LocalModelInfo {
id: "nvidia/parakeet-tdt-0.6b-v3",
file_name: "parakeet-tdt-0.6b-v3-Q8_0.gguf",
url: "https://huggingface.co/handy-computer/parakeet-tdt-0.6b-v3-gguf/resolve/main/parakeet-tdt-0.6b-v3-Q8_0.gguf",
},
LocalModelInfo {
id: "nvidia/parakeet-tdt-0.6b-v2",
file_name: "parakeet-tdt-0.6b-v2-Q8_0.gguf",
url: "https://huggingface.co/handy-computer/parakeet-tdt-0.6b-v2-gguf/resolve/main/parakeet-tdt-0.6b-v2-Q8_0.gguf",
},
LocalModelInfo {
id: "Qwen/Qwen3-ASR-0.6B",
file_name: "Qwen3-ASR-0.6B-Q8_0.gguf",
url: "https://huggingface.co/handy-computer/Qwen3-ASR-0.6B-gguf/resolve/main/Qwen3-ASR-0.6B-Q8_0.gguf",
},
];
const LOCAL_QWEN_MODEL_ID: &str = "Qwen/Qwen3-ASR-0.6B";
const LOCAL_QWEN_MODEL_DIR: &str = "qwen3-asr-06b";
const LOCAL_QWEN_MODEL_FILES: &[QwenModelFile] = &[
QwenModelFile {
file_name: ".gitattributes",
size: 1_519,
},
QwenModelFile {
file_name: "README.md",
size: 57_456,
},
QwenModelFile {
file_name: "chat_template.json",
size: 1_161,
},
QwenModelFile {
file_name: "config.json",
size: 6_193,
},
QwenModelFile {
file_name: "generation_config.json",
size: 142,
},
QwenModelFile {
file_name: "merges.txt",
size: 1_671_853,
},
QwenModelFile {
file_name: "model.safetensors",
size: 1_876_091_704,
},
QwenModelFile {
file_name: "preprocessor_config.json",
size: 330,
},
QwenModelFile {
file_name: "tokenizer_config.json",
size: 12_487,
},
QwenModelFile {
file_name: "vocab.json",
size: 2_776_833,
},
];
const LOCAL_NVIDIA_V3_MODEL_ID: &str = "mlx-community/parakeet-tdt-0.6b-v3";
const LOCAL_NVIDIA_V3_MODEL_DIR: &str = "parakeet-tdt-06b-v3";
const LOCAL_NVIDIA_V3_MODEL_FILES: &[QwenModelFile] = &[
QwenModelFile {
file_name: ".gitattributes",
size: 1_519,
},
QwenModelFile {
file_name: "README.md",
size: 1_081,
},
QwenModelFile {
file_name: "config.json",
size: 244_093,
},
QwenModelFile {
file_name: "model.safetensors",
size: 2_508_288_736,
},
QwenModelFile {
file_name: "tokenizer.model",
size: 360_916,
},
QwenModelFile {
file_name: "tokenizer.vocab",
size: 101_024,
},
QwenModelFile {
file_name: "vocab.txt",
size: 46_772,
},
];
const LOCAL_NVIDIA_V2_MODEL_ID: &str = "mlx-community/parakeet-tdt-0.6b-v2";
const LOCAL_NVIDIA_V2_MODEL_DIR: &str = "parakeet-tdt-06b-v2";
const LOCAL_NVIDIA_V2_MODEL_FILES: &[QwenModelFile] = &[
QwenModelFile {
file_name: ".gitattributes",
size: 1_519,
},
QwenModelFile {
file_name: "README.md",
size: 945,
},
QwenModelFile {
file_name: "config.json",
size: 36_200,
},
QwenModelFile {
file_name: "model.safetensors",
size: 2_469_999_999,
},
QwenModelFile {
file_name: "tokenizer.model",
size: 251_000,
},
QwenModelFile {
file_name: "tokenizer.vocab",
size: 10_400,
},
QwenModelFile {
file_name: "vocab.txt",
size: 5_071,
},
];
const LOCAL_NVIDIA_MODELS: &[NvidiaModelInfo] = &[
NvidiaModelInfo {
id: LOCAL_NVIDIA_V3_MODEL_ID,
dir_name: LOCAL_NVIDIA_V3_MODEL_DIR,
files: LOCAL_NVIDIA_V3_MODEL_FILES,
},
NvidiaModelInfo {
id: LOCAL_NVIDIA_V2_MODEL_ID,
dir_name: LOCAL_NVIDIA_V2_MODEL_DIR,
files: LOCAL_NVIDIA_V2_MODEL_FILES,
},
];
fn resolve_stt_base_url_from_models_url(models_url: &str) -> String {
models_url
.trim_end_matches('/')
@ -322,12 +191,8 @@ fn runtime_kind_for_port(port: u16) -> Option<LocalRuntimeKind> {
return Some(LocalRuntimeKind::Whisper);
}
if port == LocalRuntimeKind::Nvidia.default_port() || (18050..=18099).contains(&port) {
return Some(LocalRuntimeKind::Nvidia);
}
if port == LocalRuntimeKind::Qwen.default_port() || (18100..=18149).contains(&port) {
return Some(LocalRuntimeKind::Qwen);
if port == 8001 || port == 8002 || (18050..=18149).contains(&port) {
return Some(LocalRuntimeKind::Whisper);
}
if port == LocalRuntimeKind::Diarization.default_port() || (18150..=18199).contains(&port) {
@ -340,8 +205,6 @@ fn runtime_kind_for_port(port: u16) -> Option<LocalRuntimeKind> {
fn dynamic_port_range(kind: LocalRuntimeKind) -> std::ops::RangeInclusive<u16> {
match kind {
LocalRuntimeKind::Whisper => 18000..=18049,
LocalRuntimeKind::Nvidia => 18050..=18099,
LocalRuntimeKind::Qwen => 18100..=18149,
LocalRuntimeKind::Diarization => 18150..=18199,
}
}
@ -350,7 +213,11 @@ fn requested_port(base_url: &str, kind: LocalRuntimeKind) -> u16 {
url::Url::parse(base_url)
.ok()
.and_then(|url| url.port())
.filter(|port| runtime_kind_for_port(*port) == Some(kind))
.filter(|port| {
runtime_kind_for_port(*port) == Some(kind)
&& !(kind == LocalRuntimeKind::Whisper
&& (*port == 8001 || *port == 8002 || (18050..=18149).contains(port)))
})
.unwrap_or_else(|| kind.default_port())
}
@ -454,6 +321,25 @@ fn local_model_info(value: &str) -> Option<&'static LocalModelInfo> {
| "ggml-large-v3-turbo.bin" => LOCAL_WHISPER_MODELS
.iter()
.find(|model| model.id == "whisper-large-v3-turbo"),
"nvidia/parakeet-tdt-0.6b-v3"
| "mlx-community/parakeet-tdt-0.6b-v3"
| "parakeet-tdt-06b-v3"
| "parakeet-tdt-0.6b-v3"
| "parakeet-tdt-0.6b-v3-q8_0.gguf" => LOCAL_WHISPER_MODELS
.iter()
.find(|model| model.id == "nvidia/parakeet-tdt-0.6b-v3"),
"nvidia/parakeet-tdt-0.6b-v2"
| "mlx-community/parakeet-tdt-0.6b-v2"
| "parakeet-tdt-06b-v2"
| "parakeet-tdt-0.6b-v2"
| "parakeet-tdt-0.6b-v2-q8_0.gguf" => LOCAL_WHISPER_MODELS
.iter()
.find(|model| model.id == "nvidia/parakeet-tdt-0.6b-v2"),
"qwen/qwen3-asr-0.6b" | "qwen3-asr-06b" | "qwen3-asr-0.6b" | "qwen3-asr-0.6b-q8_0.gguf" => {
LOCAL_WHISPER_MODELS
.iter()
.find(|model| model.id == LOCAL_QWEN_MODEL_ID)
}
_ => None,
}
}
@ -605,7 +491,7 @@ fn write_model_marker(path: &Path, model: &LocalModelInfo) -> Result<(), String>
let marker = serde_json::json!({
"id": model.id,
"file": model.file_name,
"engine": "whisper.cpp"
"engine": "transcribe.cpp"
});
fs::write(path, marker.to_string()).map_err(|err| {
format!(
@ -652,14 +538,6 @@ pub fn installed_model_ids(
.map(|model| model.id.to_string())
.collect::<Vec<_>>();
if qwen_model_is_installed_in_dir(&models_dir) {
models.push(LOCAL_QWEN_MODEL_ID.to_string());
}
for model in LOCAL_NVIDIA_MODELS {
if nvidia_model_is_installed_in_dir(&models_dir, model) {
models.push(model.id.to_string());
}
}
if diarization_model_is_installed_in_dir(&models_dir) {
models.push(LOCAL_DIARIZATION_MODEL_ID.to_string());
}
@ -676,18 +554,6 @@ fn installed_model_ids_for_runtime(kind: LocalRuntimeKind, models_dir: &Path) ->
.filter(|model| models_dir.join(model.file_name).is_file())
.map(|model| model.id.to_string())
.collect::<Vec<_>>(),
LocalRuntimeKind::Qwen => {
if qwen_model_is_installed_in_dir(models_dir) {
vec![LOCAL_QWEN_MODEL_ID.to_string()]
} else {
Vec::new()
}
}
LocalRuntimeKind::Nvidia => LOCAL_NVIDIA_MODELS
.iter()
.filter(|model| nvidia_model_is_installed_in_dir(models_dir, model))
.map(|model| model.id.to_string())
.collect::<Vec<_>>(),
LocalRuntimeKind::Diarization => {
if diarization_model_is_installed_in_dir(models_dir) {
vec![LOCAL_DIARIZATION_MODEL_ID.to_string()]
@ -702,32 +568,6 @@ fn installed_model_ids_for_runtime(kind: LocalRuntimeKind, models_dir: &Path) ->
models
}
fn qwen_model_is_installed_in_dir(models_dir: &Path) -> bool {
let model_dir = models_dir.join(LOCAL_QWEN_MODEL_DIR);
LOCAL_QWEN_MODEL_FILES
.iter()
.all(|file| model_dir.join(file.file_name).is_file())
}
fn nvidia_model_info(requested: &str) -> Option<&'static NvidiaModelInfo> {
LOCAL_NVIDIA_MODELS.iter().find(|model| {
model.id.eq_ignore_ascii_case(requested)
|| model.dir_name.eq_ignore_ascii_case(requested)
|| (requested.eq_ignore_ascii_case("nvidia/parakeet-tdt-0.6b-v3")
&& model.id == LOCAL_NVIDIA_V3_MODEL_ID)
|| (requested.eq_ignore_ascii_case("nvidia/parakeet-tdt-0.6b-v2")
&& model.id == LOCAL_NVIDIA_V2_MODEL_ID)
})
}
fn nvidia_model_is_installed_in_dir(models_dir: &Path, model: &NvidiaModelInfo) -> bool {
let model_dir = models_dir.join(model.dir_name);
model
.files
.iter()
.all(|file| model_dir.join(file.file_name).is_file())
}
fn diarization_model_is_installed_in_dir(models_dir: &Path) -> bool {
let model_dir = models_dir.join(LOCAL_DIARIZATION_MODEL_ID);
model_dir
@ -736,51 +576,6 @@ fn diarization_model_is_installed_in_dir(models_dir: &Path) -> bool {
&& model_dir.join("nemo_en_titanet_small.onnx").is_file()
}
pub fn qwen_model_is_installed(app: &AppHandle, custom_dir: Option<&str>) -> Result<bool, String> {
let models_dir = resolve_models_dir(app, custom_dir)?;
Ok(qwen_model_is_installed_in_dir(&models_dir))
}
pub fn nvidia_model_is_installed(
app: &AppHandle,
custom_dir: Option<&str>,
requested: &str,
) -> Result<bool, String> {
let models_dir = resolve_models_dir(app, custom_dir)?;
Ok(nvidia_model_info(requested)
.map(|model| nvidia_model_is_installed_in_dir(&models_dir, model))
.unwrap_or(false))
}
pub fn qwen_model_progress_snapshot(
app: &AppHandle,
custom_dir: Option<&str>,
) -> Result<(u64, Option<u64>), String> {
let models_dir = resolve_models_dir(app, custom_dir)?;
let model_dir = models_dir.join(LOCAL_QWEN_MODEL_DIR);
let total = LOCAL_QWEN_MODEL_FILES
.iter()
.map(|file| file.size)
.sum::<u64>();
let downloaded = directory_size(&model_dir).unwrap_or(0).min(total);
Ok((downloaded, Some(total)))
}
pub fn nvidia_model_progress_snapshot(
app: &AppHandle,
custom_dir: Option<&str>,
requested: &str,
) -> Result<(u64, Option<u64>), String> {
let models_dir = resolve_models_dir(app, custom_dir)?;
let model = nvidia_model_info(requested).unwrap_or(&LOCAL_NVIDIA_MODELS[0]);
let model_dir = models_dir.join(model.dir_name);
let total = model.files.iter().map(|file| file.size).sum::<u64>();
let downloaded = directory_size(&model_dir).unwrap_or(0).min(total);
Ok((downloaded, Some(total)))
}
pub fn resolve_installed_model_for_runtime(
app: &AppHandle,
kind: LocalRuntimeKind,
@ -815,28 +610,6 @@ pub fn resolve_installed_model_for_runtime(
Ok(None)
}
LocalRuntimeKind::Qwen => {
if qwen_model_is_installed_in_dir(&models_dir) {
Ok(Some(LOCAL_QWEN_MODEL_ID.to_string()))
} else {
Ok(None)
}
}
LocalRuntimeKind::Nvidia => {
if let Some(model) = nvidia_model_info(requested) {
if nvidia_model_is_installed_in_dir(&models_dir, model) {
return Ok(Some(model.id.to_string()));
}
}
for model in LOCAL_NVIDIA_MODELS {
if nvidia_model_is_installed_in_dir(&models_dir, model) {
return Ok(Some(model.id.to_string()));
}
}
Ok(None)
}
LocalRuntimeKind::Diarization => {
if requested.eq_ignore_ascii_case(LOCAL_DIARIZATION_MODEL_ID)
&& diarization_model_is_installed_in_dir(&models_dir)
@ -849,25 +622,6 @@ pub fn resolve_installed_model_for_runtime(
}
}
fn directory_size(path: &Path) -> Result<u64, std::io::Error> {
if !path.exists() {
return Ok(0);
}
let mut total = 0u64;
for entry in fs::read_dir(path)? {
let entry = entry?;
let metadata = entry.metadata()?;
if metadata.is_file() {
total = total.saturating_add(metadata.len());
} else if metadata.is_dir() {
total = total.saturating_add(directory_size(&entry.path())?);
}
}
Ok(total)
}
fn runtime_executable_path(app: &AppHandle) -> Result<PathBuf, String> {
Ok(runtime_dir(app)?.join(WHISPER_RUNTIME_NAME))
}
@ -1279,7 +1033,7 @@ pub async fn ensure_runtime(
custom_models_dir: Option<&str>,
) -> Result<String, String> {
let kind = managed_runtime_kind(models_url).ok_or_else(|| {
"Автоматический запуск локального runtime поддержан только для портов Talkis 8000/8001/8002/8003."
"Автоматический запуск локального runtime поддержан только для портов Talkis 8000/8003."
.to_string()
})?;

114
src-tauri/src/media/ffmpeg.rs Normal file
View file

@ -0,0 +1,114 @@
use crate::logger;
#[cfg(debug_assertions)]
use std::env;
#[cfg(debug_assertions)]
use std::path::PathBuf;
#[cfg(debug_assertions)]
use std::process::Command;
use std::time::Instant;
use tauri_plugin_shell::ShellExt;
#[cfg(debug_assertions)]
fn ffmpeg_candidates() -> Vec<PathBuf> {
let mut candidates = Vec::new();
if let Ok(value) = env::var("FFMPEG_PATH") {
if !value.trim().is_empty() {
candidates.push(PathBuf::from(value));
}
}
candidates.push(PathBuf::from("ffmpeg"));
candidates.push(PathBuf::from("/opt/homebrew/bin/ffmpeg"));
candidates.push(PathBuf::from("/usr/local/bin/ffmpeg"));
candidates.push(PathBuf::from("/usr/bin/ffmpeg"));
candidates
}
#[cfg(debug_assertions)]
fn resolve_ffmpeg() -> Result<PathBuf, String> {
for candidate in ffmpeg_candidates() {
if Command::new(&candidate)
.arg("-version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
{
return Ok(candidate);
}
}
Err("Системный ffmpeg не найден.".to_string())
}
pub(super) async fn run_ffmpeg(
app: &tauri::AppHandle,
args: Vec<String>,
) -> Result<Vec<u8>, String> {
match app.shell().sidecar("talkis-ffmpeg") {
Ok(command) => {
logger::log_info("MEDIA", "Running bundled ffmpeg sidecar");
let started_at = Instant::now();
let output = command
.args(args.clone())
.output()
.await
.map_err(|err| format!("Не удалось запустить встроенный ffmpeg: {}", err))?;
let elapsed_ms = started_at.elapsed().as_millis();
if output.status.success() {
logger::log_info(
"MEDIA",
&format!("Bundled ffmpeg sidecar finished in {}ms", elapsed_ms),
);
return Ok(output.stderr);
}
logger::log_error(
"MEDIA",
&format!("Bundled ffmpeg sidecar failed in {}ms", elapsed_ms),
);
return Err(String::from_utf8_lossy(&output.stderr).trim().to_string());
}
Err(err) => {
logger::log_error(
"MEDIA",
&format!("Bundled ffmpeg sidecar unavailable: {}", err),
);
}
}
#[cfg(debug_assertions)]
{
let ffmpeg = resolve_ffmpeg()?;
logger::log_info(
"MEDIA",
&format!("Running system ffmpeg fallback: {:?}", ffmpeg),
);
let started_at = Instant::now();
let output = Command::new(&ffmpeg)
.args(&args)
.output()
.map_err(|err| format!("Не удалось запустить системный ffmpeg: {}", err))?;
let elapsed_ms = started_at.elapsed().as_millis();
if output.status.success() {
logger::log_info(
"MEDIA",
&format!("System ffmpeg fallback finished in {}ms", elapsed_ms),
);
return Ok(output.stderr);
}
logger::log_error(
"MEDIA",
&format!("System ffmpeg fallback failed in {}ms", elapsed_ms),
);
return Err(String::from_utf8_lossy(&output.stderr).trim().to_string());
}
#[cfg(not(debug_assertions))]
{
Err("Встроенный медиаконвертер недоступен. Переустановите приложение или обратитесь в поддержку Talkis.".to_string())
}
}

329
src-tauri/src/media.rs → src-tauri/src/media/file_prep.rs
View file

@ -1,263 +1,24 @@
use crate::logger;
use base64::Engine;
use hound::{SampleFormat, WavReader};
use serde::{Deserialize, Serialize};
use std::env;
use std::fs;
use std::io::Cursor;
use std::path::{Path, PathBuf};
#[cfg(debug_assertions)]
use std::process::Command;
use std::time::{Instant, SystemTime, UNIX_EPOCH};
use tauri_plugin_shell::ShellExt;
use std::path::Path;
const MAX_TRANSCRIPTION_BYTES: u64 = 25 * 1024 * 1024;
pub const MAX_FILE_TRANSCRIPTION_INPUT_BYTES: u64 = 8 * 1024 * 1024 * 1024;
const FILE_TRANSCRIPTION_SEGMENT_SECONDS: u32 = 600;
#[derive(Deserialize)]
pub struct PrepareMediaRequest {
pub file_base64: String,
pub file_name: String,
}
#[derive(Serialize)]
pub struct PrepareMediaResponse {
pub audio_base64: String,
pub file_name: String,
pub mime_type: String,
pub size_bytes: u64,
}
pub struct PreparedMediaChunk {
pub path: PathBuf,
pub file_name: String,
pub mime_type: String,
pub size_bytes: u64,
pub start_offset_seconds: f64,
}
pub struct PreparedMediaChunks {
pub temp_dir: PathBuf,
pub chunks: Vec<PreparedMediaChunk>,
}
pub struct PreparedProxyMedia {
pub temp_dir: PathBuf,
pub path: PathBuf,
pub file_name: String,
pub mime_type: String,
pub size_bytes: u64,
}
pub struct PreparedDiarizationAudio {
pub temp_dir: PathBuf,
pub path: PathBuf,
}
fn file_extension(file_name: &str) -> &str {
Path::new(file_name)
.extension()
.and_then(|value| value.to_str())
.unwrap_or("bin")
}
fn unique_temp_path(prefix: &str, extension: &str) -> PathBuf {
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|duration| duration.as_nanos())
.unwrap_or_default();
let pid = std::process::id();
env::temp_dir().join(format!(
"talkis-{}-{}-{}.{}",
prefix,
pid,
now,
extension.trim_start_matches('.')
))
}
fn is_local_stt_ready_wav(input_bytes: &[u8]) -> bool {
let Ok(reader) = WavReader::new(Cursor::new(input_bytes)) else {
return false;
};
let spec = reader.spec();
spec.sample_rate == 16000
&& spec.channels == 1
&& spec.bits_per_sample == 16
&& spec.sample_format == SampleFormat::Int
}
#[cfg(debug_assertions)]
fn ffmpeg_candidates() -> Vec<PathBuf> {
let mut candidates = Vec::new();
if let Ok(value) = env::var("FFMPEG_PATH") {
if !value.trim().is_empty() {
candidates.push(PathBuf::from(value));
}
}
candidates.push(PathBuf::from("ffmpeg"));
candidates.push(PathBuf::from("/opt/homebrew/bin/ffmpeg"));
candidates.push(PathBuf::from("/usr/local/bin/ffmpeg"));
candidates.push(PathBuf::from("/usr/bin/ffmpeg"));
candidates
}
#[cfg(debug_assertions)]
fn resolve_ffmpeg() -> Result<PathBuf, String> {
for candidate in ffmpeg_candidates() {
if Command::new(&candidate)
.arg("-version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
{
return Ok(candidate);
}
}
Err("Системный ffmpeg не найден.".to_string())
}
async fn run_ffmpeg(app: &tauri::AppHandle, args: Vec<String>) -> Result<Vec<u8>, String> {
match app.shell().sidecar("talkis-ffmpeg") {
Ok(command) => {
logger::log_info("MEDIA", "Running bundled ffmpeg sidecar");
let started_at = Instant::now();
let output = command
.args(args.clone())
.output()
.await
.map_err(|err| format!("Не удалось запустить встроенный ffmpeg: {}", err))?;
let elapsed_ms = started_at.elapsed().as_millis();
if output.status.success() {
logger::log_info(
"MEDIA",
&format!("Bundled ffmpeg sidecar finished in {}ms", elapsed_ms),
);
return Ok(output.stderr);
}
logger::log_error(
"MEDIA",
&format!("Bundled ffmpeg sidecar failed in {}ms", elapsed_ms),
);
return Err(String::from_utf8_lossy(&output.stderr).trim().to_string());
}
Err(err) => {
logger::log_error(
"MEDIA",
&format!("Bundled ffmpeg sidecar unavailable: {}", err),
);
}
}
#[cfg(debug_assertions)]
{
let ffmpeg = resolve_ffmpeg()?;
logger::log_info(
"MEDIA",
&format!("Running system ffmpeg fallback: {:?}", ffmpeg),
);
let started_at = Instant::now();
let output = Command::new(&ffmpeg)
.args(&args)
.output()
.map_err(|err| format!("Не удалось запустить системный ffmpeg: {}", err))?;
let elapsed_ms = started_at.elapsed().as_millis();
if output.status.success() {
logger::log_info(
"MEDIA",
&format!("System ffmpeg fallback finished in {}ms", elapsed_ms),
);
return Ok(output.stderr);
}
logger::log_error(
"MEDIA",
&format!("System ffmpeg fallback failed in {}ms", elapsed_ms),
);
return Err(String::from_utf8_lossy(&output.stderr).trim().to_string());
}
#[cfg(not(debug_assertions))]
{
Err("Встроенный медиаконвертер недоступен. Переустановите приложение или обратитесь в поддержку Talkis.".to_string())
}
}
pub async fn convert_audio_to_local_stt_wav(
app: &tauri::AppHandle,
input_bytes: &[u8],
file_name: &str,
) -> Result<Vec<u8>, String> {
if is_local_stt_ready_wav(input_bytes) {
logger::log_info(
"MEDIA",
&format!(
"Skipping ffmpeg for local STT: input is already 16 kHz mono PCM WAV, size={} bytes",
input_bytes.len()
),
);
return Ok(input_bytes.to_vec());
}
let input_ext = file_extension(file_name);
let input_path = unique_temp_path("local-stt-input", input_ext);
let output_path = unique_temp_path("local-stt-output", "wav");
fs::write(&input_path, input_bytes)
.map_err(|err| format!("Не удалось подготовить аудио для локального STT: {}", err))?;
let ffmpeg_args = vec![
"-hide_banner".to_string(),
"-loglevel".to_string(),
"error".to_string(),
"-y".to_string(),
"-i".to_string(),
input_path.to_string_lossy().to_string(),
"-vn".to_string(),
"-ac".to_string(),
"1".to_string(),
"-ar".to_string(),
"16000".to_string(),
"-acodec".to_string(),
"pcm_s16le".to_string(),
output_path.to_string_lossy().to_string(),
];
let ffmpeg_result = run_ffmpeg(app, ffmpeg_args).await;
let _ = fs::remove_file(&input_path);
if let Err(message) = ffmpeg_result {
let _ = fs::remove_file(&output_path);
return Err(if message.is_empty() {
"Не удалось подготовить аудио для локального STT.".to_string()
} else {
format!(
"Не удалось подготовить аудио для локального STT: {}",
message
)
});
}
let output_bytes = fs::read(&output_path).map_err(|err| {
let _ = fs::remove_file(&output_path);
format!("Не удалось прочитать WAV для локального STT: {}", err)
})?;
let _ = fs::remove_file(&output_path);
Ok(output_bytes)
}
use super::ffmpeg::run_ffmpeg;
use super::paths::{file_extension, unique_temp_path};
use super::probe::is_local_stt_ready_wav;
use super::types::{
PrepareMediaRequest, PrepareMediaResponse, PreparedDiarizationAudio, PreparedMediaChunk,
PreparedMediaChunks, PreparedProxyMedia,
};
use super::{
FILE_TRANSCRIPTION_SEGMENT_SECONDS, LOCAL_STT_FILE_TRANSCRIPTION_SEGMENT_SECONDS,
MAX_FILE_TRANSCRIPTION_INPUT_BYTES, MAX_TRANSCRIPTION_BYTES,
};
pub async fn prepare_media_file_chunks_for_transcription(
app: &tauri::AppHandle,
input_path: &Path,
local_stt_target: bool,
) -> Result<PreparedMediaChunks, String> {
let metadata =
fs::metadata(input_path).map_err(|err| format!("Не удалось прочитать файл: {}", err))?;
@ -308,6 +69,19 @@ pub async fn prepare_media_file_chunks_for_transcription(
fs::create_dir_all(&chunks_dir)
.map_err(|err| format!("Не удалось подготовить временную папку: {}", err))?;
let segment_seconds = if local_stt_target {
LOCAL_STT_FILE_TRANSCRIPTION_SEGMENT_SECONDS
} else {
FILE_TRANSCRIPTION_SEGMENT_SECONDS
};
logger::log_info(
"MEDIA",
&format!(
"Preparing file transcription chunks: local_stt_target={}, segment_seconds={}",
local_stt_target, segment_seconds
),
);
let output_pattern = chunks_dir.join("chunk-%05d.mp3");
let ffmpeg_args = vec![
"-hide_banner".to_string(),
@ -326,7 +100,7 @@ pub async fn prepare_media_file_chunks_for_transcription(
"-f".to_string(),
"segment".to_string(),
"-segment_time".to_string(),
FILE_TRANSCRIPTION_SEGMENT_SECONDS.to_string(),
segment_seconds.to_string(),
"-reset_timestamps".to_string(),
"1".to_string(),
output_pattern.to_string_lossy().to_string(),
@ -381,7 +155,7 @@ pub async fn prepare_media_file_chunks_for_transcription(
file_name,
mime_type: "audio/mpeg".to_string(),
size_bytes: metadata.len(),
start_offset_seconds: chunks.len() as f64 * FILE_TRANSCRIPTION_SEGMENT_SECONDS as f64,
start_offset_seconds: chunks.len() as f64 * segment_seconds as f64,
});
}
@ -398,19 +172,7 @@ pub async fn prepare_media_file_for_proxy_transcription(
let metadata =
fs::metadata(input_path).map_err(|err| format!("Не удалось прочитать файл: {}", err))?;
if !metadata.is_file() {
return Err("Выбранный путь не является файлом.".to_string());
}
if metadata.len() == 0 {
return Err("Пустой файл нельзя транскрибировать.".to_string());
}
if metadata.len() > MAX_FILE_TRANSCRIPTION_INPUT_BYTES {
return Err(
"Файл слишком большой. Максимальный размер для локальной подготовки: 8 ГБ.".to_string(),
);
}
validate_input_file(&metadata)?;
let temp_dir = unique_temp_path("file-proxy-transcription", "dir");
fs::create_dir_all(&temp_dir)
@ -468,19 +230,7 @@ pub async fn prepare_media_file_for_diarization(
let metadata =
fs::metadata(input_path).map_err(|err| format!("Не удалось прочитать файл: {}", err))?;
if !metadata.is_file() {
return Err("Выбранный путь не является файлом.".to_string());
}
if metadata.len() == 0 {
return Err("Пустой файл нельзя транскрибировать.".to_string());
}
if metadata.len() > MAX_FILE_TRANSCRIPTION_INPUT_BYTES {
return Err(
"Файл слишком большой. Максимальный размер для локальной подготовки: 8 ГБ.".to_string(),
);
}
validate_input_file(&metadata)?;
let temp_dir = unique_temp_path("file-diarization", "dir");
fs::create_dir_all(&temp_dir)
@ -521,7 +271,6 @@ pub async fn prepare_media_file_for_diarization(
})
}
#[tauri::command]
pub async fn prepare_media_for_transcription(
app: tauri::AppHandle,
req: PrepareMediaRequest,
@ -596,3 +345,21 @@ pub async fn prepare_media_for_transcription(
size_bytes: metadata.len(),
})
}
fn validate_input_file(metadata: &fs::Metadata) -> Result<(), String> {
if !metadata.is_file() {
return Err("Выбранный путь не является файлом.".to_string());
}
if metadata.len() == 0 {
return Err("Пустой файл нельзя транскрибировать.".to_string());
}
if metadata.len() > MAX_FILE_TRANSCRIPTION_INPUT_BYTES {
return Err(
"Файл слишком большой. Максимальный размер для локальной подготовки: 8 ГБ.".to_string(),
);
}
Ok(())
}

70
src-tauri/src/media/local_stt.rs Normal file
View file

@ -0,0 +1,70 @@
use crate::logger;
use std::fs;
use super::ffmpeg::run_ffmpeg;
use super::paths::{file_extension, unique_temp_path};
use super::probe::is_local_stt_ready_wav;
pub async fn convert_audio_to_local_stt_wav(
app: &tauri::AppHandle,
input_bytes: &[u8],
file_name: &str,
) -> Result<Vec<u8>, String> {
if is_local_stt_ready_wav(input_bytes) {
logger::log_info(
"MEDIA",
&format!(
"Skipping ffmpeg for local STT: input is already 16 kHz mono PCM WAV, size={} bytes",
input_bytes.len()
),
);
return Ok(input_bytes.to_vec());
}
let input_ext = file_extension(file_name);
let input_path = unique_temp_path("local-stt-input", input_ext);
let output_path = unique_temp_path("local-stt-output", "wav");
fs::write(&input_path, input_bytes)
.map_err(|err| format!("Не удалось подготовить аудио для локального STT: {}", err))?;
let ffmpeg_args = vec![
"-hide_banner".to_string(),
"-loglevel".to_string(),
"error".to_string(),
"-y".to_string(),
"-i".to_string(),
input_path.to_string_lossy().to_string(),
"-vn".to_string(),
"-ac".to_string(),
"1".to_string(),
"-ar".to_string(),
"16000".to_string(),
"-acodec".to_string(),
"pcm_s16le".to_string(),
output_path.to_string_lossy().to_string(),
];
let ffmpeg_result = run_ffmpeg(app, ffmpeg_args).await;
let _ = fs::remove_file(&input_path);
if let Err(message) = ffmpeg_result {
let _ = fs::remove_file(&output_path);
return Err(if message.is_empty() {
"Не удалось подготовить аудио для локального STT.".to_string()
} else {
format!(
"Не удалось подготовить аудио для локального STT: {}",
message
)
});
}
let output_bytes = fs::read(&output_path).map_err(|err| {
let _ = fs::remove_file(&output_path);
format!("Не удалось прочитать WAV для локального STT: {}", err)
})?;
let _ = fs::remove_file(&output_path);
Ok(output_bytes)
}

28
src-tauri/src/media/mod.rs Normal file
View file

@ -0,0 +1,28 @@
mod ffmpeg;
mod file_prep;
mod local_stt;
mod paths;
mod probe;
mod types;
pub use file_prep::{
prepare_media_file_chunks_for_transcription, prepare_media_file_for_diarization,
prepare_media_file_for_proxy_transcription,
};
pub use local_stt::convert_audio_to_local_stt_wav;
pub use types::{
PrepareMediaRequest, PrepareMediaResponse, PreparedMediaChunk, PreparedProxyMedia,
};
const MAX_TRANSCRIPTION_BYTES: u64 = 25 * 1024 * 1024;
pub const MAX_FILE_TRANSCRIPTION_INPUT_BYTES: u64 = 8 * 1024 * 1024 * 1024;
const FILE_TRANSCRIPTION_SEGMENT_SECONDS: u32 = 600;
const LOCAL_STT_FILE_TRANSCRIPTION_SEGMENT_SECONDS: u32 = 240;
#[tauri::command]
pub async fn prepare_media_for_transcription(
app: tauri::AppHandle,
req: PrepareMediaRequest,
) -> Result<PrepareMediaResponse, String> {
file_prep::prepare_media_for_transcription(app, req).await
}

25
src-tauri/src/media/paths.rs Normal file
View file

@ -0,0 +1,25 @@
use std::env;
use std::path::{Path, PathBuf};
use std::time::{SystemTime, UNIX_EPOCH};
pub(super) fn file_extension(file_name: &str) -> &str {
Path::new(file_name)
.extension()
.and_then(|value| value.to_str())
.unwrap_or("bin")
}
pub(super) fn unique_temp_path(prefix: &str, extension: &str) -> PathBuf {
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|duration| duration.as_nanos())
.unwrap_or_default();
let pid = std::process::id();
env::temp_dir().join(format!(
"talkis-{}-{}-{}.{}",
prefix,
pid,
now,
extension.trim_start_matches('.')
))
}

14
src-tauri/src/media/probe.rs Normal file
View file

@ -0,0 +1,14 @@
use hound::{SampleFormat, WavReader};
use std::io::Cursor;
pub(super) fn is_local_stt_ready_wav(input_bytes: &[u8]) -> bool {
let Ok(reader) = WavReader::new(Cursor::new(input_bytes)) else {
return false;
};
let spec = reader.spec();
spec.sample_rate == 16000
&& spec.channels == 1
&& spec.bits_per_sample == 16
&& spec.sample_format == SampleFormat::Int
}

42
src-tauri/src/media/types.rs Normal file
View file

@ -0,0 +1,42 @@
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
#[derive(Deserialize)]
pub struct PrepareMediaRequest {
pub file_base64: String,
pub file_name: String,
}
#[derive(Serialize)]
pub struct PrepareMediaResponse {
pub audio_base64: String,
pub file_name: String,
pub mime_type: String,
pub size_bytes: u64,
}
pub struct PreparedMediaChunk {
pub path: PathBuf,
pub file_name: String,
pub mime_type: String,
pub size_bytes: u64,
pub start_offset_seconds: f64,
}
pub struct PreparedMediaChunks {
pub temp_dir: PathBuf,
pub chunks: Vec<PreparedMediaChunk>,
}
pub struct PreparedProxyMedia {
pub temp_dir: PathBuf,
pub path: PathBuf,
pub file_name: String,
pub mime_type: String,
pub size_bytes: u64,
}
pub struct PreparedDiarizationAudio {
pub temp_dir: PathBuf,
pub path: PathBuf,
}

59
src-tauri/src/runtime_engines/nvidia_engine.py
View file

@ -1,59 +0,0 @@
#!/usr/bin/env python3
import argparse
import json
import sys
def print_json(payload):
print(json.dumps(payload, ensure_ascii=False), flush=True)
def download_model(args):
from huggingface_hub import snapshot_download
snapshot_download(
repo_id=args.model_id,
local_dir=args.model_dir,
local_dir_use_symlinks=False,
resume_download=True,
)
print_json({"ok": True, "model_dir": args.model_dir})
def transcribe(args):
from parakeet_mlx import from_pretrained
model = from_pretrained(args.model_dir)
result = model.transcribe(args.audio)
text = getattr(result, "text", "") if result is not None else ""
print_json({"text": (text or "").strip()})
def main():
parser = argparse.ArgumentParser(prog="nvidia_engine.py")
subparsers = parser.add_subparsers(dest="command", required=True)
download_parser = subparsers.add_parser("download")
download_parser.add_argument("--model-id", required=True)
download_parser.add_argument("--model-dir", required=True)
transcribe_parser = subparsers.add_parser("transcribe")
transcribe_parser.add_argument("--model-dir", required=True)
transcribe_parser.add_argument("--audio", required=True)
transcribe_parser.add_argument("--language", default="auto")
args = parser.parse_args()
try:
if args.command == "download":
download_model(args)
elif args.command == "transcribe":
transcribe(args)
else:
parser.error(f"Unknown command: {args.command}")
except Exception as exc:
print(f"Parakeet engine error: {exc}", file=sys.stderr)
raise
if __name__ == "__main__":
main()

157
src-tauri/src/runtime_engines/qwen_engine.py
View file

@ -1,157 +0,0 @@
#!/usr/bin/env python3
import argparse
import json
import os
import sys
import warnings
LANGUAGE_NAMES = {
"ar": "Arabic",
"cs": "Czech",
"da": "Danish",
"de": "German",
"el": "Greek",
"en": "English",
"es": "Spanish",
"fa": "Persian",
"fi": "Finnish",
"fil": "Filipino",
"fr": "French",
"hi": "Hindi",
"hu": "Hungarian",
"id": "Indonesian",
"it": "Italian",
"ja": "Japanese",
"ko": "Korean",
"mk": "Macedonian",
"ms": "Malay",
"nl": "Dutch",
"pl": "Polish",
"pt": "Portuguese",
"ro": "Romanian",
"ru": "Russian",
"sv": "Swedish",
"th": "Thai",
"tr": "Turkish",
"vi": "Vietnamese",
"yue": "Cantonese",
"zh": "Chinese",
}
def print_json(payload):
print(json.dumps(payload, ensure_ascii=False), flush=True)
def normalize_language(value):
if not value:
return None
value = value.strip()
if not value or value.lower() == "auto":
return None
return LANGUAGE_NAMES.get(value.lower(), value)
def choose_runtime():
import torch
forced = os.environ.get("TALKIS_QWEN_DEVICE_MAP", "").strip()
if forced:
dtype_name = os.environ.get("TALKIS_QWEN_DTYPE", "").strip().lower()
dtype = {
"bfloat16": torch.bfloat16,
"float16": torch.float16,
"float32": torch.float32,
}.get(dtype_name, torch.float32)
return forced, dtype
if torch.cuda.is_available():
return "cuda:0", torch.bfloat16
if getattr(torch.backends, "mps", None) and torch.backends.mps.is_available():
return "mps", torch.float16
return "cpu", torch.float32
def build_model(model_dir):
import torch
from qwen_asr import Qwen3ASRModel
device_map, dtype = choose_runtime()
attempts = [
{"device_map": device_map, "dtype": dtype},
]
if device_map != "cpu":
attempts.append({"device_map": "cpu", "dtype": torch.float32})
last_error = None
for kwargs in attempts:
try:
return Qwen3ASRModel.from_pretrained(
model_dir,
max_inference_batch_size=1,
max_new_tokens=512,
**kwargs,
)
except Exception as exc:
last_error = exc
warnings.warn(f"Qwen load failed with {kwargs}: {exc}")
raise last_error
def download_model(args):
from huggingface_hub import snapshot_download
snapshot_download(
repo_id=args.model_id,
local_dir=args.model_dir,
local_dir_use_symlinks=False,
resume_download=True,
)
print_json({"ok": True, "model_dir": args.model_dir})
def transcribe(args):
model = build_model(args.model_dir)
result = model.transcribe(
audio=args.audio,
language=normalize_language(args.language),
)
first = result[0] if isinstance(result, list) and result else result
text = getattr(first, "text", "") if first is not None else ""
language = getattr(first, "language", None) if first is not None else None
print_json({"text": (text or "").strip(), "language": language})
def main():
parser = argparse.ArgumentParser(prog="qwen_engine.py")
subparsers = parser.add_subparsers(dest="command", required=True)
download_parser = subparsers.add_parser("download")
download_parser.add_argument("--model-id", required=True)
download_parser.add_argument("--model-dir", required=True)
transcribe_parser = subparsers.add_parser("transcribe")
transcribe_parser.add_argument("--model-dir", required=True)
transcribe_parser.add_argument("--audio", required=True)
transcribe_parser.add_argument("--language", default="auto")
args = parser.parse_args()
try:
if args.command == "download":
download_model(args)
elif args.command == "transcribe":
transcribe(args)
else:
parser.error(f"Unknown command: {args.command}")
except Exception as exc:
print(f"Qwen engine error: {exc}", file=sys.stderr)
raise
if __name__ == "__main__":
main()

4
src-tauri/tauri.conf.json
View file

@ -2,7 +2,7 @@
"$schema": "https://schema.tauri.app/config/2",
"productName": "Talkis",
"mainBinaryName": "Talkis",
"version": "0.3.6",
"version": "0.3.7",
"identifier": "com.trixter.talkis",
"build": {
"beforeDevCommand": "bun run prepare:sidecars && bun run dev",
@ -43,8 +43,6 @@
"externalBin": [
"binaries/talkis-ffmpeg",
"binaries/talkis-stt",
"binaries/talkis-stt-nvidia",
"binaries/talkis-stt-qwen",
"binaries/talkis-diarize",
"binaries/talkis-llm"
],

15
src/lib/callCapture.ts
View file

@ -14,6 +14,7 @@ import {
const callInterruptedMessage = (): string => tn("callCapture.interrupted");
import {
type FileTranscriptionResult,
toFileTranscriptionErrorMessage,
transcribeFilePathOnly,
transcribeFileOnly,
type FileTranscriptionProgress,
@ -312,11 +313,11 @@ async function buildCallCaptureHistoryEntry({
});
micPlainPart = `${tn("callCapture.speakerYou")}:\n${micResult.text.trim()}`;
} catch (error) {
const message = errorMessage(error);
const message = toFileTranscriptionErrorMessage(error, { settings });
failedTracks.push(`${tn("callCapture.trackMic")}: ${message}`);
void logError(
"CALL_CAPTURE",
`Mic track transcription failed: ${message}`,
`Mic track transcription failed: ${errorMessage(error)}`,
);
}
}
@ -362,13 +363,13 @@ async function buildCallCaptureHistoryEntry({
parts.push(formatTrackTranscript(track, result.text));
}
} catch (error) {
const message = errorMessage(error);
const message = toFileTranscriptionErrorMessage(error, { settings });
failedTracks.push(`${callTrackTitle(track).toLowerCase()}: ${message}`);
requiredSystemDiarizationFailed =
requiredSystemDiarizationFailed || shouldDiarizeSystemTrack;
void logError(
"CALL_CAPTURE",
`${track.kind} track transcription failed: ${message}`,
`${track.kind} track transcription failed: ${errorMessage(error)}`,
);
}
}
@ -401,11 +402,11 @@ async function buildCallCaptureHistoryEntry({
);
}
} catch (error) {
const message = errorMessage(error);
const message = toFileTranscriptionErrorMessage(error, { settings });
failedTracks.push(`${tn("callCapture.trackMic")}: ${message}`);
void logError(
"CALL_CAPTURE",
`Mic track diarization fallback failed: ${message}`,
`Mic track diarization fallback failed: ${errorMessage(error)}`,
);
}
}
@ -611,7 +612,7 @@ export async function retryCallCaptureHistoryEntry(
return interrupted;
}
const userFacingMessage = tn("callCapture.errProcessRetry");
const userFacingMessage = toFileTranscriptionErrorMessage(error, { settings });
const failedEntry: HistoryEntry = {
...entry,
status: "failed",

68
src/lib/fileTranscription.test.ts Normal file
View file

@ -0,0 +1,68 @@
import { describe, expect, it } from "bun:test";
import type { AppSettings } from "./store";
import { toFileTranscriptionErrorMessage } from "./fileTranscription";
function settings(overrides: Partial<AppSettings>): AppSettings {
return {
useOwnKey: true,
provider: "openai",
whisperEndpoint: "http://127.0.0.1:8000",
...overrides,
} as AppSettings;
}
describe("file transcription error messages", () => {
it("does not show API key auth text for local STT 401 errors", () => {
const message = toFileTranscriptionErrorMessage(
new Error("Whisper API error (401 Unauthorized): invalid api key"),
{ settings: settings({}) },
);
expect(message).toBe(
"Локальный runtime отклонил файл. Проверьте, что выбранная локальная модель скачана и запущена, затем попробуйте ещё раз.",
);
});
it("treats a local endpoint as local even when mode flags are stale", () => {
const message = toFileTranscriptionErrorMessage(
new Error("Whisper API error (401 Unauthorized): invalid api key"),
{
settings: settings({
useOwnKey: false,
provider: "openai",
whisperEndpoint: "http://localhost:8000",
}),
},
);
expect(message).toBe(
"Локальный runtime отклонил файл. Проверьте, что выбранная локальная модель скачана и запущена, затем попробуйте ещё раз.",
);
});
it("does not leak already-localized API key text in local STT mode", () => {
const message = toFileTranscriptionErrorMessage(
new Error("Не удалось авторизоваться в API. Проверьте ключ доступа."),
{ settings: settings({}) },
);
expect(message).toBe(
"Локальный runtime отклонил файл. Проверьте, что выбранная локальная модель скачана и запущена, затем попробуйте ещё раз.",
);
});
it("keeps API key auth text for remote STT 401 errors", () => {
const message = toFileTranscriptionErrorMessage(
new Error("Whisper API error (401 Unauthorized): invalid api key"),
{
settings: settings({
provider: "custom",
whisperEndpoint: "https://api.openai.com/v1",
}),
},
);
expect(message).toBe("Не удалось авторизоваться в API. Проверьте ключ доступа.");
});
});

75
src/lib/fileTranscription.ts
View file

@ -119,6 +119,11 @@ interface CloudTranscriptionCapabilities {
speakerDiarizationMaxSpeakers?: number;
}
interface FileTranscriptionErrorContext {
settings?: Pick<AppSettings, "useOwnKey" | "provider" | "whisperEndpoint"> | null;
localStt?: boolean;
}
let cloudCapabilitiesCache: { value: CloudTranscriptionCapabilities; expiresAt: number } | null = null;
function fileExtension(fileName: string): string {
@ -172,18 +177,37 @@ function shouldConvert(file: File): boolean {
return isVideo || !isDirectFormat || file.size > TRANSCRIPTION_MAX_BYTES;
}
function isLocalSttSettings(
settings: Pick<AppSettings, "useOwnKey" | "provider" | "whisperEndpoint">,
): boolean {
return /127\.0\.0\.1|localhost/i.test(settings.whisperEndpoint || "");
}
function isAuthFailureLike(normalized: string): boolean {
return (
normalized.includes("401") ||
normalized.includes("unauthorized") ||
normalized.includes("invalid api key") ||
normalized.includes("api-ключ") ||
normalized.includes("ключ доступа") ||
(normalized.includes("авторизоваться") && normalized.includes("api"))
);
}
function buildFilePathRequestSettings(
settings: AppSettings,
speakerDiarization: boolean,
useCloudSpeakerDiarization: boolean,
): FilePathRequestSettings {
const isLocalStt = isLocalSttSettings(settings);
if (!speakerDiarization || useCloudSpeakerDiarization) {
return {
whisperApiKey: settings.whisperApiKey || null,
whisperApiKey: isLocalStt ? null : settings.whisperApiKey || null,
whisperEndpoint: settings.whisperEndpoint || null,
whisperModel: settings.whisperModel || null,
useOwnKey: settings.useOwnKey,
deviceToken: settings.deviceToken || null,
useOwnKey: isLocalStt ? true : settings.useOwnKey,
deviceToken: isLocalStt ? null : settings.deviceToken || null,
};
}
@ -315,9 +339,19 @@ export function getFileTranscriptionPercent(
return 0;
}
export function toFileTranscriptionErrorMessage(error: unknown): string {
export function toFileTranscriptionErrorMessage(
error: unknown,
context: FileTranscriptionErrorContext = {},
): string {
const raw = formatErrorMessage(error);
const normalized = raw.toLowerCase();
const isLocalRuntimeError =
context.localStt === true ||
(context.settings ? isLocalSttSettings(context.settings) : false) ||
normalized.includes("локальный stt runtime") ||
normalized.includes("local stt runtime") ||
normalized.includes("127.0.0.1") ||
normalized.includes("localhost");
if (normalized.includes("ffmpeg") || normalized.includes("медиаконвертер")) {
return tn("fileTranscription.errMediaConverterUnavailable");
@ -352,6 +386,18 @@ export function toFileTranscriptionErrorMessage(error: unknown): string {
return tn("fileTranscription.errCloudDiarizationUnavailable");
}
if (
isLocalRuntimeError &&
(
normalized.includes("403") ||
normalized.includes("forbidden") ||
isAuthFailureLike(normalized) ||
normalized.includes("отклонил запрос")
)
) {
return tn("fileTranscription.errLocalRuntimeRejected");
}
if (normalized.includes("subscription inactive") || normalized.includes("403")) {
return tn("fileTranscription.errSubscriptionInactive");
}
@ -390,7 +436,7 @@ export function toFileTranscriptionErrorMessage(error: unknown): string {
return tn("fileTranscription.errLocalModelNotInstalled");
}
if (normalized.includes("401") || normalized.includes("unauthorized") || normalized.includes("invalid api key")) {
if (isAuthFailureLike(normalized)) {
return tn("fileTranscription.errAuthFailed");
}
@ -498,12 +544,14 @@ async function transcribeViaBackend(
prepared: PreparedTranscriptionFile,
settings: AppSettings,
): Promise<NativeTranscriptionResult> {
const isLocalStt = isLocalSttSettings(settings);
return invoke<NativeTranscriptionResult>("transcribe_only", {
req: {
audio_base64: prepared.audioBase64,
language: settings.language,
api_key: settings.apiKey,
whisper_api_key: settings.whisperApiKey || null,
whisper_api_key: isLocalStt ? null : settings.whisperApiKey || null,
llm_api_key: null,
style: settings.style || "classic",
whisper_endpoint: settings.whisperEndpoint || null,
@ -522,6 +570,10 @@ async function transcribePreparedFile(
prepared: PreparedTranscriptionFile,
settings: AppSettings,
): Promise<NativeTranscriptionResult> {
if (isLocalSttSettings(settings)) {
return transcribeViaBackend(prepared, settings);
}
if (!settings.useOwnKey && settings.deviceToken?.trim()) {
return transcribeViaProxy(prepared, settings);
}
@ -609,8 +661,10 @@ export async function transcribeFilePathOnly({
});
logInfo("FILE_TRANSCRIPTION", `Sending file path ${fileName} through native pipeline`);
const useCloudSpeakerDiarization = speakerDiarization && await canUseCloudSpeakerDiarization(settings);
if (speakerDiarization && !settings.useOwnKey && !useCloudSpeakerDiarization) {
const isLocalStt = isLocalSttSettings(settings);
const effectiveUseOwnKey = isLocalStt || settings.useOwnKey;
const useCloudSpeakerDiarization = speakerDiarization && !isLocalStt && await canUseCloudSpeakerDiarization(settings);
if (speakerDiarization && !effectiveUseOwnKey && !useCloudSpeakerDiarization) {
throw new Error("Cloud speaker diarization unavailable");
}
const requestSettings = buildFilePathRequestSettings(settings, speakerDiarization, useCloudSpeakerDiarization);
@ -712,12 +766,13 @@ export async function retryFileHistoryEntry(
const message = formatErrorMessage(error);
void logError("WIDGET_FILE", `File retry failed: ${message}`);
const userFacingMessage = toFileTranscriptionErrorMessage(error, { settings });
const failed: HistoryEntry = {
...entry,
status: "failed",
errorMessage: tn("fileTranscription.errProcessFileRetry"),
errorMessage: userFacingMessage,
};
await finishProcessing(failed);
throw new Error(failed.errorMessage);
throw new Error(userFacingMessage);
}
}

6
src/lib/hotkeyEvents.ts
View file

@ -6,6 +6,7 @@ export const HOTKEY_CHANGE_REQUEST_EVENT = "hotkey-change-request";
export const HOTKEY_REGISTRATION_RESULT_EVENT = "hotkey-registration-result";
export const NATIVE_HOTKEY_CAPTURE_EVENT = "native-hotkey-capture";
export const HOTKEY_CAPTURE_STATE_EVENT = "hotkey-capture-state";
export const HANDY_HOTKEY_EVENT = "handy-hotkey-event";
export const SETTINGS_NAVIGATE_EVENT = "settings-navigate";
export const WIDGET_RETRY_PROCESSING_EVENT = "widget-retry-processing";
export const PROCESSING_CANCEL_REQUEST_EVENT = "processing-cancel-request";
@ -31,6 +32,11 @@ export interface HotkeyCaptureStatePayload {
active: boolean;
}
export interface HandyHotkeyEventPayload {
hotkey: string;
state: "Pressed" | "Released";
}
export interface SettingsNavigatePayload {
tab: "main" | "file" | "interpreter" | "settings" | "model" | "style";
resultId?: string | null;

4
src/lib/i18n/dict/libMessages.ts
View file

@ -100,6 +100,10 @@ export const libMessages = {
ru: "Локальная модель распознавания не установлена. Откройте Настройки -> Модели -> Локально и нажмите «Скачать» для выбранной модели.",
en: "The local recognition model is not installed. Open Settings -> Models -> Local and click “Download” for the selected model.",
},
"fileTranscription.errLocalRuntimeRejected": {
ru: "Локальный runtime отклонил файл. Проверьте, что выбранная локальная модель скачана и запущена, затем попробуйте ещё раз.",
en: "The local runtime rejected the file. Make sure the selected local model is downloaded and running, then try again.",
},
"fileTranscription.errAuthFailed": {
ru: "Не удалось авторизоваться в API. Проверьте ключ доступа.",
en: "Could not authenticate with the API. Check your access key.",

12
src/lib/i18n/dict/settingsModels.ts
View file

@ -240,16 +240,16 @@ export const settingsModels = {
en: "Minimal size and maximum speed, with lower quality than the other Whisper models.",
},
"models.local.parakeet-tdt-06b-v3.description": {
ru: "Быстрая локальная ASR-модель Parakeet через MLX runtime для Apple Silicon.",
en: "A fast local Parakeet ASR model via the MLX runtime for Apple Silicon.",
ru: "Быстрая локальная ASR-модель Parakeet через transcribe.cpp GGUF runtime.",
en: "A fast local Parakeet ASR model via the transcribe.cpp GGUF runtime.",
},
"models.local.parakeet-tdt-06b-v2.description": {
ru: "Стабильная английская Parakeet TDT-модель через MLX runtime для Apple Silicon.",
en: "A stable English Parakeet TDT model via the MLX runtime for Apple Silicon.",
ru: "Стабильная английская Parakeet TDT-модель через transcribe.cpp GGUF runtime.",
en: "A stable English Parakeet TDT model via the transcribe.cpp GGUF runtime.",
},
"models.local.qwen3-asr-06b.description": {
ru: "Компактная ASR-модель Qwen для локального распознавания через совместимый runtime.",
en: "A compact Qwen ASR model for local recognition via a compatible runtime.",
ru: "Компактная ASR-модель Qwen для локального распознавания через transcribe.cpp GGUF runtime.",
en: "A compact Qwen ASR model for local recognition via the transcribe.cpp GGUF runtime.",
},
// ── Speed / accuracy value labels ──

4
src/lib/i18n/dict/settingsRest.ts
View file

@ -122,8 +122,8 @@ export const settingsRest = {
"localLlm.cancel": { ru: "Отмена", en: "Cancel" },
"localLlm.download": { ru: "Скачать", en: "Download" },
"localLlm.required": {
ru: "Выберите текстовую модель — без неё саммаризация недоступна (распознавание работает и так).",
en: "Select a text model — summarization is unavailable without one (transcription still works).",
ru: "Выберите текстовую модель. Без неё саммаризация недоступна (распознавание работает и так).",
en: "Select a text model. Summarization is unavailable without one (transcription still works).",
},
"localLlm.select": { ru: "Выбрать", en: "Select" },
"localLlm.delete": { ru: "Удалить", en: "Delete" },

4
src/lib/i18n/dict/settingsTabsMisc.ts
View file

@ -44,6 +44,10 @@ export const settingsTabsMisc = {
ru: "Не удалось подготовить локальные компоненты: STT runtime отклонил API-ключ.",
en: "Could not prepare local components: the STT runtime rejected the API key.",
},
"fileTab.setupError.localRuntimeRejected": {
ru: "Не удалось подготовить локальные компоненты: локальный STT runtime отклонил запрос. Это не ошибка API-ключа.",
en: "Could not prepare local components: the local STT runtime rejected the request. This is not an API key error.",
},
"fileTab.setupError.forbidden": {
ru: "Не удалось подготовить локальные компоненты: STT runtime запретил установку модели.",
en: "Could not prepare local components: the STT runtime forbade installing the model.",

69
src/lib/store.ts
View file

@ -491,6 +491,50 @@ export const BUILTIN_PROMPTS: PromptPreset[] = [
const BUILTIN_PROMPT_ID_SET = new Set(BUILTIN_PROMPTS.map((preset) => preset.id));
function isLocalSttEndpoint(endpoint?: string): boolean {
return /127\.0\.0\.1|localhost/i.test(endpoint || "");
}
function normalizeLocalSttEndpoint(endpoint?: string): string | undefined {
if (!endpoint) return endpoint;
try {
const parsed = new URL(endpoint);
if (parsed.hostname !== "127.0.0.1" && parsed.hostname !== "localhost") {
return endpoint;
}
if (parsed.port === "8001" || parsed.port === "8002") {
parsed.port = "8000";
return parsed.toString().replace(/\/$/, "");
}
} catch {
return endpoint;
}
return endpoint;
}
function needsLocalSttSettingsMigration(saved: unknown): boolean {
if (!saved || typeof saved !== "object") {
return false;
}
const raw = saved as Record<string, unknown>;
const endpoint = typeof raw.whisperEndpoint === "string" ? raw.whisperEndpoint : "";
if (!isLocalSttEndpoint(endpoint)) {
return false;
}
const normalizedEndpoint = normalizeLocalSttEndpoint(endpoint);
return (
raw.useOwnKey !== true ||
raw.provider !== "custom" ||
raw.whisperApiKey !== "" ||
normalizedEndpoint !== endpoint
);
}
const DEFAULT_SETTINGS: AppSettings = {
apiKey: "",
apiAdapters: {},
@ -735,7 +779,7 @@ export function normalizeSavedSettings(saved: unknown): Partial<AppSettings> {
? DEFAULT_DESKTOP_HOTKEY
: normalizedHotkey;
return {
const normalized: Partial<AppSettings> = {
apiKey: typeof raw.apiKey === "string" ? raw.apiKey : undefined,
apiAdapters: rawApiAdapters,
selectedApiAdapter:
@ -807,6 +851,15 @@ export function normalizeSavedSettings(saved: unknown): Partial<AppSettings> {
raw.realtimeInterpreter,
),
};
if (isLocalSttEndpoint(normalized.whisperEndpoint)) {
normalized.whisperEndpoint = normalizeLocalSttEndpoint(normalized.whisperEndpoint);
normalized.useOwnKey = true;
normalized.provider = "custom";
normalized.whisperApiKey = "";
}
return normalized;
}
let _store: Awaited<ReturnType<typeof load>> | null = null;
@ -905,6 +958,20 @@ export async function getSettings(
Object.entries(normalized).filter(([, v]) => v !== undefined),
);
const result = { ...DEFAULT_SETTINGS, ...defined } as AppSettings;
if (isLocalSttEndpoint(result.whisperEndpoint)) {
result.whisperEndpoint = normalizeLocalSttEndpoint(result.whisperEndpoint) || result.whisperEndpoint;
result.useOwnKey = true;
result.provider = "custom";
result.whisperApiKey = "";
}
if (needsLocalSttSettingsMigration(saved)) {
try {
await store.set("settings", result);
await store.save();
} catch (error) {
console.warn("Failed to persist local STT settings migration", error);
}
}
return result;
}

26
src/lib/summarize.test.ts
View file

@ -82,3 +82,29 @@ describe("stored settings migration for bundled local LLM", () => {
expect(normalized.llmLocalModelId).toBeUndefined();
});
});
describe("stored settings migration for local STT", () => {
it("repairs stale mode flags when Whisper endpoint is local", () => {
const normalized = normalizeSavedSettings({
useOwnKey: false,
provider: "openai",
whisperApiKey: "stale-key",
whisperEndpoint: "http://127.0.0.1:8000",
});
expect(normalized.useOwnKey).toBe(true);
expect(normalized.provider).toBe("custom");
expect(normalized.whisperApiKey).toBe("");
});
it("migrates removed Qwen and Parakeet endpoints to unified transcribe.cpp runtime", () => {
const normalized = normalizeSavedSettings({
useOwnKey: true,
provider: "custom",
whisperApiKey: "",
whisperEndpoint: "http://127.0.0.1:8002",
});
expect(normalized.whisperEndpoint).toBe("http://127.0.0.1:8000");
});
});

101
src/windows/settings/tabs/FileTranscriptionTab.tsx
View file

@ -87,6 +87,10 @@ function isModelDownloaded(settings: AppSettings, modelId: string): boolean {
return settings.localModels?.[modelId]?.status === "downloaded";
}
function isLocalSttSettings(settings: AppSettings): boolean {
return /127\.0\.0\.1|localhost/i.test(settings.whisperEndpoint || "");
}
function getDiarizationWhisperOption(
settings: AppSettings,
): (typeof DIARIZATION_WHISPER_OPTIONS)[number] | null {
@ -186,11 +190,43 @@ function isSpeakerSetupRepairError(error: unknown): boolean {
);
}
function toSpeakerSetupErrorMessage(error: unknown, t: TFunc): string {
function isAuthFailureLike(normalized: string): boolean {
return (
normalized.includes("401") ||
normalized.includes("unauthorized") ||
normalized.includes("invalid api key") ||
normalized.includes("api-ключ") ||
normalized.includes("ключ доступа") ||
(normalized.includes("авторизоваться") && normalized.includes("api"))
);
}
function toSpeakerSetupErrorMessage(
error: unknown,
t: TFunc,
settings: AppSettings | null = null,
): string {
const raw = formatErrorMessage(error);
const normalized = raw.toLowerCase();
const isLocalStt = settings ? isLocalSttSettings(settings) : false;
if (normalized.includes("401") || normalized.includes("api-ключ")) {
if (
isLocalStt ||
normalized.includes("локальный stt runtime") ||
normalized.includes("local stt runtime") ||
normalized.includes("127.0.0.1") ||
normalized.includes("localhost")
) {
if (isAuthFailureLike(normalized) || normalized.includes("403") || normalized.includes("forbidden")) {
return t("fileTab.setupError.localRuntimeRejected");
}
return raw.trim()
? t("fileTab.setupError.genericWithDetail", { detail: raw })
: t("fileTab.setupError.generic");
}
if (isAuthFailureLike(normalized)) {
return t("fileTab.setupError.rejectedKey");
}
@ -519,12 +555,14 @@ export function FileTranscriptionTab({
setSelectedFile({ name: file.name, size: file.size });
resetResult();
setStatus("reading");
let activeSettings: AppSettings | null = null;
try {
if (speakerDiarization) {
throw new Error(t("fileTab.error.speakerNeedsDialog"));
}
const settings = await getSettings();
activeSettings = settings;
const startedAt = Date.now();
const transcription = await transcribeFileOnly({
file,
@ -558,7 +596,11 @@ export function FileTranscriptionTab({
);
setStatus("done");
} catch (caughtError) {
setError(toFileTranscriptionErrorMessage(caughtError));
setError(
toFileTranscriptionErrorMessage(caughtError, {
settings: activeSettings,
}),
);
setStatus("error");
}
};
@ -576,19 +618,20 @@ export function FileTranscriptionTab({
const settings = await refreshSpeakerInstalledModels(
await getSettings({ reload: true }),
);
const cloudSpeakerReady = await canUseCloudSpeakerDiarization(
settings,
true,
);
const isLocalStt = isLocalSttSettings(settings);
const effectiveUseOwnKey = isLocalStt || settings.useOwnKey;
const cloudSpeakerReady = isLocalStt
? false
: await canUseCloudSpeakerDiarization(settings, true);
settingsRef.current = settings;
syncSpeakerSetupState(settings);
if (speakerMode && !settings.useOwnKey && !cloudSpeakerReady) {
if (speakerMode && !effectiveUseOwnKey && !cloudSpeakerReady) {
setSpeakerDiarization(false);
await saveSettings({ fileSpeakerDiarization: false });
throw new Error("IconCloud speaker diarization unavailable");
}
if (speakerMode && settings.useOwnKey && !isSpeakerSetupReady(settings)) {
if (speakerMode && effectiveUseOwnKey && !isSpeakerSetupReady(settings)) {
setPendingSpeakerFilePath(filePath);
setSpeakerSetupIntent("process");
setSpeakerSetupMessage("");
@ -634,13 +677,21 @@ export function FileTranscriptionTab({
setSpeakerSetupIntent("process");
setSpeakerSetupForceRepair(true);
setSpeakerSetupMessage("");
setSpeakerSetupError(toFileTranscriptionErrorMessage(caughtError));
setSpeakerSetupError(
toFileTranscriptionErrorMessage(caughtError, {
settings: settingsRef.current,
}),
);
setSpeakerSetupModalOpen(true);
setStatus("idle");
return;
}
setError(toFileTranscriptionErrorMessage(caughtError));
setError(
toFileTranscriptionErrorMessage(caughtError, {
settings: settingsRef.current,
}),
);
setStatus("error");
}
};
@ -855,7 +906,9 @@ export function FileTranscriptionTab({
}
setSpeakerSetupIntent(null);
} catch (caughtError) {
setSpeakerSetupError(toSpeakerSetupErrorMessage(caughtError, t));
setSpeakerSetupError(
toSpeakerSetupErrorMessage(caughtError, t, settingsRef.current),
);
} finally {
setSpeakerSetupInstalling(false);
}
@ -907,9 +960,15 @@ export function FileTranscriptionTab({
void saveSettings({ fileSpeakerDiarization: true });
const cachedSettings = settingsRef.current;
if (
!cachedSettings ||
(cachedSettings.useOwnKey && !isSpeakerSetupReady(cachedSettings))
if (!cachedSettings) {
setPendingSpeakerFilePath(null);
setSpeakerSetupIntent("toggle");
setSpeakerSetupMessage("");
setSpeakerSetupError("");
setSpeakerSetupModalOpen(true);
} else if (
(isLocalSttSettings(cachedSettings) || cachedSettings.useOwnKey) &&
!isSpeakerSetupReady(cachedSettings)
) {
setPendingSpeakerFilePath(null);
setSpeakerSetupIntent("toggle");
@ -923,7 +982,9 @@ export function FileTranscriptionTab({
const settings = await refreshSpeakerInstalledModels(
await getSettings({ reload: true }),
);
const cloudSpeakerReady = settings.useOwnKey
const isLocalStt = isLocalSttSettings(settings);
const effectiveUseOwnKey = isLocalStt || settings.useOwnKey;
const cloudSpeakerReady = effectiveUseOwnKey
? false
: await canUseCloudSpeakerDiarization(settings, true);
@ -934,7 +995,7 @@ export function FileTranscriptionTab({
settingsRef.current = settings;
syncSpeakerSetupState(settings);
if (!settings.useOwnKey) {
if (!effectiveUseOwnKey) {
if (cloudSpeakerReady) {
setSpeakerSetupModalOpen(false);
setSpeakerSetupIntent(null);
@ -971,7 +1032,11 @@ export function FileTranscriptionTab({
setSpeakerSetupModalOpen(false);
setSpeakerSetupIntent(null);
setError(toFileTranscriptionErrorMessage(caughtError));
setError(
toFileTranscriptionErrorMessage(caughtError, {
settings: settingsRef.current,
}),
);
setSpeakerDiarization(false);
await saveSettings({ fileSpeakerDiarization: false });
return;

75
src/windows/settings/tabs/SettingsTabs.tsx
View file

@ -75,7 +75,7 @@ import volcengineAvatar from "../../../assets/adapters/volcengine.webp";
import xAiAvatar from "../../../assets/adapters/xai.png";
const IS_DEV = import.meta.env.DEV;
type LocalRuntimeKind = "whisper" | "nvidia" | "qwen" | "diarization";
type LocalRuntimeKind = "whisper" | "diarization";
type DesktopPlatform = "macos" | "windows" | "linux" | "unknown";
function detectDesktopPlatform(): DesktopPlatform {
@ -94,14 +94,16 @@ function detectDesktopPlatform(): DesktopPlatform {
const LOCAL_RUNTIME_ENDPOINTS: Record<LocalRuntimeKind, string> = {
whisper: "http://127.0.0.1:8000",
nvidia: "http://127.0.0.1:8001",
qwen: "http://127.0.0.1:8002",
diarization: "http://127.0.0.1:8003",
};
const LOCAL_STT_PRESET_ENDPOINT = LOCAL_RUNTIME_ENDPOINTS.whisper;
const LOCAL_STT_PRESET_MODEL = "whisper-large-v3-turbo";
const LOCAL_STT_MODEL_DOWNLOAD_PROGRESS_EVENT = "local-stt-model-download-progress";
function isLocalSttEndpoint(endpoint?: string | null): boolean {
return /127\.0\.0\.1|localhost/i.test(endpoint || "");
}
interface SettingsTabsProps { type: "model" | "style"; }
interface PromptPreview {
@ -296,7 +298,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Рекомендуемый Whisper-вариант: быстрый, качественный и хорошо подходит для диктовки.",
model: "whisper-large-v3-turbo",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "large",
speed: "быстро",
@ -313,7 +315,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Более легкий вариант Turbo для локальной работы с меньшим расходом памяти.",
model: "whisper-large-v3-turbo",
engineLabel: "Whisper",
runtime: "OpenAI-compatible / MLX runtime",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "4-bit",
speed: "быстро",
@ -328,7 +330,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Баланс скорости и качества для слабых машин и быстрых коротких диктовок.",
model: "whisper-small",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "small",
speed: "быстро",
@ -344,7 +346,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Максимальное качество Whisper, но выше требования к памяти и времени обработки.",
model: "whisper-large-v3",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "large",
speed: "средне",
@ -360,7 +362,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Предыдущая large-версия Whisper для совместимости с существующими локальными установками.",
model: "whisper-large-v2",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "large",
speed: "средне",
@ -376,7 +378,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Промежуточный вариант между Small и Large: заметно качественнее Small, но тяжелее.",
model: "whisper-medium",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "medium",
speed: "средне",
@ -392,7 +394,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Быстрая и легкая модель для простых сценариев и слабых машин.",
model: "whisper-base",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "base",
speed: "очень быстро",
@ -408,7 +410,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
description: "Минимальный размер и максимальная скорость, качество ниже остальных Whisper-моделей.",
model: "whisper-tiny",
engineLabel: "Whisper",
runtime: "Talkis Local / whisper.cpp",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "tiny",
speed: "очень быстро",
@ -421,11 +423,11 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
{
id: "parakeet-tdt-06b-v3",
name: "NVIDIA Parakeet TDT 0.6B v3",
description: "Быстрая локальная ASR-модель Parakeet через MLX runtime для Apple Silicon.",
model: "mlx-community/parakeet-tdt-0.6b-v3",
description: "Быстрая локальная ASR-модель Parakeet через transcribe.cpp GGUF runtime.",
model: "nvidia/parakeet-tdt-0.6b-v3",
engineLabel: "Parakeet",
runtime: "OpenAI-compatible / Parakeet MLX runtime",
runtimeKind: "nvidia",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "0.6B",
speed: "быстро",
accuracy: "высокая",
@ -433,16 +435,16 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
accent: "#76b900",
avatar: nvidiaAvatar,
runtimeReady: true,
downloadBytes: 2_509_044_141,
downloadBytes: 740_000_000,
},
{
id: "parakeet-tdt-06b-v2",
name: "NVIDIA Parakeet TDT 0.6B v2",
description: "Стабильная английская Parakeet TDT-модель через MLX runtime для Apple Silicon.",
model: "mlx-community/parakeet-tdt-0.6b-v2",
description: "Стабильная английская Parakeet TDT-модель через transcribe.cpp GGUF runtime.",
model: "nvidia/parakeet-tdt-0.6b-v2",
engineLabel: "Parakeet",
runtime: "OpenAI-compatible / Parakeet MLX runtime",
runtimeKind: "nvidia",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "0.6B",
speed: "быстро",
accuracy: "высокая",
@ -450,16 +452,16 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
accent: "#5f9f00",
avatar: nvidiaAvatar,
runtimeReady: true,
downloadBytes: 2_470_305_134,
downloadBytes: 730_000_000,
},
{
id: "qwen3-asr-06b",
name: "Qwen3-ASR 0.6B",
description: "Компактная ASR-модель Qwen для локального распознавания через совместимый runtime.",
description: "Компактная ASR-модель Qwen для локального распознавания через transcribe.cpp GGUF runtime.",
model: "Qwen/Qwen3-ASR-0.6B",
engineLabel: "Qwen",
runtime: "OpenAI-compatible / Qwen runtime",
runtimeKind: "qwen",
runtime: "Talkis Local / transcribe.cpp",
runtimeKind: "whisper",
size: "0.6B",
speed: "средне",
accuracy: "высокая",
@ -467,7 +469,7 @@ const LOCAL_MODEL_OPTIONS: LocalModelOption[] = [
accent: "#2563eb",
avatar: qwenAvatar,
runtimeReady: true,
downloadBytes: 1_880_619_678,
downloadBytes: 811_000_000,
},
];
@ -1283,7 +1285,7 @@ export function SettingsTabs({ type }: SettingsTabsProps) {
}, [clearLocalAuthPolling, loadCloudProfile, syncSettings]);
const refreshLocalInstalledModels = useCallback(async () => {
if (!settings || type !== "model" || !settings.useOwnKey || settings.provider !== "custom") {
if (!settings || type !== "model" || !settings.useOwnKey || !isLocalSttEndpoint(settings.whisperEndpoint)) {
return;
}
@ -1527,12 +1529,12 @@ export function SettingsTabs({ type }: SettingsTabsProps) {
const currentMode: ModelMode = !settings.useOwnKey
? "cloud"
: settings.provider === "custom"
: isLocalSttEndpoint(settings.whisperEndpoint)
? "local"
: "api";
setModelModeView((current) => current ?? currentMode);
}, [settings?.provider, settings?.useOwnKey, type]);
}, [settings?.useOwnKey, settings?.whisperEndpoint, type]);
useEffect(() => {
if (!settings || type !== "model" || cloudProfile === undefined) return;
@ -1635,10 +1637,10 @@ export function SettingsTabs({ type }: SettingsTabsProps) {
if (type === "model") {
const hasActiveSubscription = cloudProfile?.subscription.active === true;
const isCloudMode = !settings.useOwnKey;
const isCustom = settings.provider === "custom";
const isLocalSttMode = settings.useOwnKey && isLocalSttEndpoint(settings.whisperEndpoint);
const isCloudSelected = isCloudMode && hasActiveSubscription;
const desktopPlatform = detectDesktopPlatform();
const activeModelMode: ModelMode = isCloudSelected ? "cloud" : isCustom ? "local" : "api";
const activeModelMode: ModelMode = isCloudSelected ? "cloud" : isLocalSttMode ? "local" : "api";
const visibleModelMode = modelModeView ?? activeModelMode;
const isApiMode = visibleModelMode === "api";
const isLocalMode = visibleModelMode === "local";
@ -1694,9 +1696,7 @@ export function SettingsTabs({ type }: SettingsTabsProps) {
return null;
}
if (port === 8000 || (port >= 18000 && port <= 18049)) return "whisper";
if (port === 8001 || (port >= 18050 && port <= 18099)) return "nvidia";
if (port === 8002 || (port >= 18100 && port <= 18149)) return "qwen";
if (port === 8000 || port === 8001 || port === 8002 || (port >= 18000 && port <= 18149)) return "whisper";
if (port === 8003 || (port >= 18150 && port <= 18199)) return "diarization";
} catch {
return null;
@ -2088,12 +2088,12 @@ export function SettingsTabs({ type }: SettingsTabsProps) {
const cachedState = settings.localModels?.[model.id];
const isPlatformSupported = model.runtimeKind === "whisper" || model.runtimeKind === "diarization" || desktopPlatform === "macos";
const isRuntimeReady = model.runtimeReady === true && isPlatformSupported;
const isInstalled = isRuntimeReady && (localInstalledModelSet.has(model.model) || cachedState?.status === "downloaded");
const isInstalled = isRuntimeReady && localInstalledModelSet.has(model.model);
const isSelected = activeModelMode === "local" && localSttTargetModel === model.model && isInstalled;
if (!isRuntimeReady) {
const runtimeName = model.runtimeKind === "nvidia" ? "NVIDIA" : model.runtimeKind === "qwen" ? "Qwen" : model.runtimeKind === "diarization" ? "Diarization" : "MLX";
const isRuntimeSlotReady = model.runtimeKind === "qwen" || model.runtimeKind === "nvidia" || model.runtimeKind === "diarization";
const runtimeName = model.runtimeKind === "diarization" ? "Diarization" : "transcribe.cpp";
const isRuntimeSlotReady = model.runtimeKind === "diarization";
return {
label: isRuntimeSlotReady ? t("models.local.modelNotConnected") : t("models.local.engineNotConnected"),
connectionLabel: isRuntimeSlotReady
@ -2297,9 +2297,6 @@ export function SettingsTabs({ type }: SettingsTabsProps) {
whisperApiKey: "",
whisperEndpoint: endpointOverride || getLocalModelEndpoint(model),
whisperModel: model.model,
llmApiKey: "",
llmEndpoint: "",
llmModel: "none",
});
setModelModeView("local");
resetTestState();

4
src/windows/widget/Widget.tsx
View file

@ -873,12 +873,12 @@ export function Widget() {
await finishProcessing({
...baseEntry,
status: "failed",
errorMessage: toFileTranscriptionErrorMessage(error),
errorMessage: toFileTranscriptionErrorMessage(error, { settings }),
});
setFileDropState("error");
setFileStatus(null);
setFileProgress(null);
setFileDropName(toFileTranscriptionErrorMessage(error));
setFileDropName(toFileTranscriptionErrorMessage(error, { settings }));
scheduleFileDropReset();
} finally {
handle.finish();

26
src/windows/widget/hooks/useWidgetHotkey.ts
View file

@ -1,8 +1,6 @@
import { useCallback, useEffect, useRef } from "react";
import type { Dispatch, MutableRefObject, SetStateAction } from "react";
import { invoke } from "@tauri-apps/api/core";
import { register, unregister } from "@tauri-apps/plugin-global-shortcut";
import type { ShortcutEvent } from "@tauri-apps/plugin-global-shortcut";
import { emit, listen } from "@tauri-apps/api/event";
import { AppSettings, DEFAULT_HOTKEY, getSettings, normalizeHotkey, saveSettings } from "../../../lib/store";
@ -10,6 +8,8 @@ import {
HOTKEY_CAPTURE_STATE_EVENT,
HOTKEY_CHANGE_REQUEST_EVENT,
HOTKEY_REGISTRATION_RESULT_EVENT,
HANDY_HOTKEY_EVENT,
HandyHotkeyEventPayload,
HotkeyCaptureStatePayload,
HotkeyChangeRequestPayload,
HotkeyRegistrationResultPayload,
@ -53,7 +53,7 @@ export function useWidgetHotkey({
if (!currentHotkey) return;
logInfo("HOTKEY", `Unregistering: ${currentHotkey}`);
await unregister(currentHotkey).catch(() => {});
await invoke("unregister_handy_hotkey", { hotkey: currentHotkey }).catch(() => {});
registeredHotkeyRef.current = null;
}, [registeredHotkeyRef]);
@ -64,7 +64,7 @@ export function useWidgetHotkey({
}, []);
const handleHotkeyPress = useCallback(
(event: ShortcutEvent) => {
(event: HandyHotkeyEventPayload) => {
if (isHotkeyCaptureActiveRef.current) {
dispatch({ type: "RESET_HOTKEY_STATE" });
return;
@ -111,17 +111,10 @@ export function useWidgetHotkey({
logInfo("HOTKEY", `Attempting to register: ${nextHotkey}`);
try {
await register(nextHotkey, handleHotkeyPress);
if (currentHotkey && currentHotkey !== nextHotkey) {
logInfo("HOTKEY", `Unregistering previous hotkey: ${currentHotkey}`);
await unregister(currentHotkey).catch((error) => {
logError("HOTKEY", `Failed to unregister previous hotkey: ${error}`);
});
}
await invoke("register_handy_hotkey", { hotkey: nextHotkey });
registeredHotkeyRef.current = nextHotkey;
logInfo("HOTKEY", `Registered successfully: ${nextHotkey}`);
logInfo("HOTKEY", `Registered successfully via handy-keys: ${nextHotkey}`);
return {
success: true,
@ -179,6 +172,10 @@ export function useWidgetHotkey({
dispatch({ type: "RESET_HOTKEY_STATE" });
});
const unlistenHandyHotkey = listen<HandyHotkeyEventPayload>(HANDY_HOTKEY_EVENT, ({ payload }) => {
handleHotkeyPress(payload);
});
const unlistenHotkeyRequests = listen<HotkeyChangeRequestPayload>(HOTKEY_CHANGE_REQUEST_EVENT, async ({ payload }) => {
const result = await attemptHotkeyRegistrationRef.current(payload.hotkey);
@ -205,10 +202,11 @@ export function useWidgetHotkey({
return () => {
unlistenSettings.then((unlisten) => unlisten());
unlistenCaptureState.then((unlisten) => unlisten());
unlistenHandyHotkey.then((unlisten) => unlisten());
unlistenHotkeyRequests.then((unlisten) => unlisten());
void unregisterCurrentHotkey();
};
}, [clearReleaseStopTimer, dispatch, setSettings, settingsRef, unregisterCurrentHotkey]);
}, [clearReleaseStopTimer, dispatch, handleHotkeyPress, setSettings, settingsRef, unregisterCurrentHotkey]);
}
async function attemptHotkeyRegistrationPlaceholder(): Promise<HotkeyRegistrationResultPayload> {

3
src/windows/widget/hooks/useWidgetRecording.ts
View file

@ -225,10 +225,9 @@ export function useWidgetRecording({
const hasKey = activeSettings.apiKey.trim().length > 0 || activeSettings.whisperApiKey.trim().length > 0 || (activeSettings.llmApiKey || "").trim().length > 0;
// In local STT mode the whisper server runs on localhost and requires no API key.
// Detect this case: custom provider + local-looking endpoint + no whisperApiKey.
// Detect this case by the local-looking endpoint; provider can be stale after upgrades.
const isLocalSttMode =
activeSettings.useOwnKey &&
activeSettings.provider === "custom" &&
(activeSettings.whisperEndpoint || "").match(/127\.0\.0\.1|localhost/i) !== null &&
(activeSettings.whisperApiKey || "").trim().length === 0;

14
src/windows/widget/services/transcriptionPipeline.ts
View file

@ -40,11 +40,21 @@ function arrayBufferToBase64(buffer: ArrayBuffer): string {
function isLocalSttSettings(settings: AppSettings): boolean {
return (
settings.useOwnKey &&
settings.provider === "custom" &&
/127\.0\.0\.1|localhost/i.test(settings.whisperEndpoint || "")
);
}
function isAuthFailureLike(normalized: string): boolean {
return (
normalized.includes("401") ||
normalized.includes("unauthorized") ||
normalized.includes("invalid api key") ||
normalized.includes("api-ключ") ||
normalized.includes("ключ доступа") ||
(normalized.includes("авторизоваться") && normalized.includes("api"))
);
}
function toUserFacingErrorMessage(error: unknown, settings: AppSettings): string {
const raw = formatErrorMessage(error);
const normalized = raw.toLowerCase();
@ -99,7 +109,7 @@ function toUserFacingErrorMessage(error: unknown, settings: AppSettings): string
return tn("widget.error.cloudUnavailable");
}
if (normalized.includes("401") || normalized.includes("unauthorized") || normalized.includes("invalid api key")) {
if (isAuthFailureLike(normalized)) {
if (isLocalStt) {
return tn("widget.error.localRuntimeRejected");
}

43
talkis-stt-nvidia/manifest.json
View file

@ -1,43 +0,0 @@
{
"version": "0.1.0",
"engine": "parakeet-mlx",
"models": [
{
"id": "mlx-community/parakeet-tdt-0.6b-v3",
"aliases": ["nvidia/parakeet-tdt-0.6b-v3"],
"directory": "parakeet-tdt-06b-v3",
"source": "huggingface",
"url": "https://huggingface.co/mlx-community/parakeet-tdt-0.6b-v3",
"upstream": "https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3"
},
{
"id": "mlx-community/parakeet-tdt-0.6b-v2",
"aliases": ["nvidia/parakeet-tdt-0.6b-v2"],
"directory": "parakeet-tdt-06b-v2",
"source": "huggingface",
"url": "https://huggingface.co/mlx-community/parakeet-tdt-0.6b-v2",
"upstream": "https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2"
}
],
"python": {
"runtime": {
"provider": "astral-sh/python-build-standalone",
"version": "3.12.13",
"release": "20260510",
"assets": {
"aarch64-apple-darwin": {
"url": "https://github.com/astral-sh/python-build-standalone/releases/download/20260510/cpython-3.12.13%2B20260510-aarch64-apple-darwin-install_only.tar.gz",
"sha256": "5a30271f8d345a5b02b0c9e4e31e0f1e1455a8e4a04fba95cd9762472abc3b17"
},
"x86_64-apple-darwin": {
"url": "https://github.com/astral-sh/python-build-standalone/releases/download/20260510/cpython-3.12.13%2B20260510-x86_64-apple-darwin-install_only.tar.gz",
"sha256": "cd369e76973c3179bc578230d8615ab621968ed758c5e32f636eecef4ad79894"
}
}
},
"packages": [
"parakeet-mlx",
"huggingface_hub"
]
}
}

33
talkis-stt-qwen/manifest.json
View file

@ -1,33 +0,0 @@
{
"version": "0.1.0",
"engine": "qwen-asr",
"models": [
{
"id": "Qwen/Qwen3-ASR-0.6B",
"directory": "qwen3-asr-06b",
"source": "huggingface",
"url": "https://huggingface.co/Qwen/Qwen3-ASR-0.6B"
}
],
"python": {
"runtime": {
"provider": "astral-sh/python-build-standalone",
"version": "3.12.13",
"release": "20260510",
"assets": {
"aarch64-apple-darwin": {
"url": "https://github.com/astral-sh/python-build-standalone/releases/download/20260510/cpython-3.12.13%2B20260510-aarch64-apple-darwin-install_only.tar.gz",
"sha256": "5a30271f8d345a5b02b0c9e4e31e0f1e1455a8e4a04fba95cd9762472abc3b17"
},
"x86_64-apple-darwin": {
"url": "https://github.com/astral-sh/python-build-standalone/releases/download/20260510/cpython-3.12.13%2B20260510-x86_64-apple-darwin-install_only.tar.gz",
"sha256": "cd369e76973c3179bc578230d8615ab621968ed758c5e32f636eecef4ad79894"
}
}
},
"packages": [
"qwen-asr",
"huggingface_hub"
]
}
}