mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-07-09 17:19:02 +00:00
d360dd276e
* feat(serve): voice dictation over the daemon for the Web Shell Bring voice input to the `qwen serve` Web Shell. The browser captures the microphone, streams raw 16 kHz mono PCM to a new `/voice/stream` WebSocket, and the daemon transcribes server-side by reusing the existing CLI voice pipeline (realtime streaming + on-stop batch) — provider credentials never reach the browser, and it works whether the daemon is local or remote. Daemon: - `/voice/stream` WebSocket handler (serve/voice) reusing openVoiceStream / openQwenAsrRealtimeStream / transcribeVoiceAudio; resolves the workspace voiceModel from a ModelsConfig built off settings. - Routed through the existing ACP upgrade listener so it shares the loopback / host-allowlist / CSRF / bearer checks; concurrency-capped. - Advertises the `voice_transcribe` capability (unconditional, like auth_device_flow; the WS errors when no voice model is configured). - Relax `Permissions-Policy` to `microphone=(self)` so the same-origin shell can request the mic — an empty `microphone=()` allowlist blocked the prompt entirely. - Allowlist `voiceModel` on `/workspace/settings` so the picker can read it. Web Shell: - Mic button in the composer: click to record, click to stop -> the transcript is inserted into the input for review before sending. - `/model --voice` picker (sourced from `/workspace/providers`, since voice models are hidden from the session model list) and `/model --voice <id>`, persisted via the prompt channel like `/model --fast`. The voiceModel resolver now accepts a structural model lookup so the daemon can resolve it without constructing a full CLI Config. Co-authored-by: Qwen-Coder <noreply@qwen.ai> * fix(serve): harden web voice streaming * fix(serve): clean stale voice capture resources * fix(serve): harden voice websocket errors * fix(serve): hide voice capability when token auth is configured * fix(serve): address voice review blockers * fix(web-shell): keep voice cancel available when disabled * fix(web-shell): inline the voice mic in the composer toolbar The mic was an absolute-positioned overlay anchored to the composer wrapper, so after the #5775 chat-UI restructure it floated outside the input box (clipped at the bottom-right). Render it inside ChatEditor's `toolbarRight`, just before the send button, wiring the transcript insert to the composer core — it sits inline with the `/`, `@`, and send controls like the rest of the toolbar. Restyle to a Codex-style recording experience: an idle mic that matches the other toolbar icon buttons, and a recording pill with a live waveform (driven by the RMS meter), an elapsed `M:SS` timer, and a stop control. Also forward `/voice/stream` through the vite dev proxy (`ws: true`, scoped to the exact path so it doesn't shadow the client's own `client/voice/*` source modules) so the WebSocket reaches the daemon in dev instead of silently failing against vite. Co-authored-by: Qwen-Coder <noreply@qwen.ai> * feat(serve): authenticate the voice WebSocket via a bearer subprotocol Voice was suppressed on any token-configured daemon because browsers cannot set an `Authorization` header on a WebSocket, so the bearer check on the shared ACP upgrade listener always failed for the browser mic. Let the browser carry the token in `Sec-WebSocket-Protocol` as `qwen-bearer.<base64url(token)>` (the only header a browser can set on a WS). The upgrade listener now accepts the token from either the `Authorization` header (non-browser clients, unchanged) or the subprotocol, hash-compared in constant time. `handleProtocols` selects a non-secret subprotocol (or none) so the token is never echoed back in the handshake response. This aligns the voice WS with how the ACP WS will authenticate browsers. With the WS now authenticated, drop the `!tokenConfigured` / `requireAuth` suppression of the `voice_transcribe` capability — it is advertised whenever the `/voice/stream` endpoint exists. Tests updated to assert voice is advertised under a token / `--require-auth`. Co-authored-by: Qwen-Coder <noreply@qwen.ai> * fix(serve): make the voice WS subprotocol handshake complete + test it Review follow-up to the bearer-subprotocol auth. Fixes a real handshake bug and closes the test gap on the security-critical path. - Handshake bug: when the browser offered only the `qwen-bearer.*` subprotocol, the daemon's `handleProtocols` selected none, and a strict WS client (the `ws` library) aborts with "Server sent no subprotocol". The web-shell now offers a non-secret marker (`qwen-ws`) alongside the bearer one, so the daemon always selects the marker — never echoing the secret — and the handshake completes for strict and lenient clients alike. - Tests: add daemon-side coverage for the subprotocol auth path (accept valid token, reject wrong/malformed, no-token loopback) and assert the secret is never echoed back; add client coverage that the bearer token is offered as `[qwen-ws, qwen-bearer.<b64url>]` and round-trips. - Drop the unreachable try/catch around `Buffer.from(_, 'base64url')` (it never throws) and correct the comment. - a11y: mark the transcribing/interim regions as `role="status"` / `aria-live`, and add a `:focus-visible` ring to the idle mic. - Refresh the stale `useVoiceCapture` doc comment (token deployments are now supported in the browser) and fix the mock typing in VoiceButton.test.tsx. Co-authored-by: Qwen-Coder <noreply@qwen.ai> * fix(web-shell): ignore stale voice socket callbacks * test(serve): sync voice capability integration expectation * fix(serve): harden voice websocket review gaps * fix(serve): sanitize voice websocket failures * fix(serve): address voice review suggestions * fix(serve): harden voice websocket lifecycle --------- Co-authored-by: Qwen-Coder <noreply@qwen.ai> |
||
|---|---|---|
| .. | ||
| baselines | ||
| cli | ||
| concurrent-runner | ||
| fixtures | ||
| hook-integration | ||
| interactive | ||
| sdk-typescript | ||
| terminal-bench | ||
| terminal-capture | ||
| channel-plugin.test.ts | ||
| fake-openai-server.test.ts | ||
| fake-openai-server.ts | ||
| globalSetup.ts | ||
| test-helper.ts | ||
| test-mcp-server.ts | ||
| tsconfig.json | ||
| vitest.config.ts | ||
| vitest.loadtest.config.ts | ||
| vitest.terminal-bench.config.ts | ||