qwen-code/scripts/build.js
qqqys b9c5e3566b
feat(voice): voice dictation with native capture, streaming, and biasing (#5502)
* feat(voice): voice dictation with native capture, streaming, and biasing

Add voice dictation for the prompt input:
- /voice [hold|tap|off|status] command + general.voice.{enabled,mode,language,protocol} settings; push-to-talk via Space, /model --voice to pick the model
- Native microphone capture (@qwen-code/audio-capture, miniaudio N-API) with arecord/SoX fallback, silence auto-stop, cold-start warm-up, and macOS permission query
- Batch transcription via DashScope Qwen-ASR (OpenAI-compatible chat/completions + input_audio) with language + keyterm biasing and an echo guard
- Live streaming over the DashScope realtime WebSocket (fun-asr-realtime / paraformer-realtime-v2) with interim text and an input-level waveform, behind voice.protocol=dashscope-realtime
- Rich VoiceIndicator UI (state, level meter, live partial transcript)
- Cross-platform prebuilds via prebuildify + node-gyp-build and a CI matrix

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* feat(cli): route voice ASR by model

* feat(cli): polish voice realtime parity

* ci: fix voice workflow checks

* fix(cli): address voice review blockers

* fix(cli): harden voice transcription failures

* fix(cli): address voice PR review blockers

* fix(cli): harden voice review follow-ups

* fix(voice): handle realtime review blockers

* fix(cli): add voice command i18n keys

* fix(cli): address voice review blockers

* fix(cli): address voice review follow-ups

* fix(voice): harden review edge cases

* fix(voice): address release and realtime review blockers

* fix(voice): address realtime suggestion followups

* fix(voice): handle realtime review followups

* fix(voice): address realtime review blockers

* fix(voice): address review feedback

* fix(voice): address recorder review feedback

* docs(voice): document ssrf guard boundary

* fix(voice): address review feedback

* fix(voice): preserve warm recorder session safety

* fix(voice): address stream review suggestions

* fix(voice): address multi-round review findings

Realtime/streaming:
- Salvage an already-committed transcript when the WebSocket closes right
  after finish() instead of rejecting the whole dictation
  (qwenAsrRealtimeSession, voiceStreamSession) + regression tests.

useVoiceInput state machine:
- Single-shot finalize guard so a tap-stop racing the silence auto-stop can't
  double-stop the recorder and surface a spurious failure.
- Reset mountedRef on (re)mount so StrictMode (DEBUG) can't freeze the voice UI.
- Widen the hold-mode first-press release window above common key-repeat delays.

Model selection:
- Reject ids with no ASR transport at /model --voice and in the model dialog via
  a new isSelectableVoiceModel; move resolveVoiceTransport into voiceModel so the
  record-time config resolver stays transport-agnostic (+ tests).

macOS mic permission:
- Surface the not-determined state in voice warmup so the first dictation isn't
  silently lost behind the TCC dialog.

Native packaging:
- Make the audio-capture native install non-fatal (falls back to SoX/arecord) so
  a voice-only build failure can't break npm ci.
- Download audio-capture prebuilds before building standalone release archives.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(voice): address review blockers

* fix(voice): update batch recording audio level

* fix(voice): tap-mode transcript loss, stream leaks, and dead keyterm cleanup

- Tap-mode dictation submitted a stale empty buffer.text, wiping the just-
  inserted transcript and sending nothing: thread the resulting prompt text
  through onSubmit(text) instead of reading buffer.text back synchronously
  after the async insert (useVoiceInput, InputPrompt).
- Streaming finalize leaked the WebSocket session when recorder.drain() threw:
  abort the session before propagating the error.
- voiceStreamSession: reject the connect promise when 'task-finished' arrives
  before 'task-started' instead of hanging forever in 'transcribing'.
- Remove dead keyterm enrichment (project/branch/recent-file paths were
  unreachable after the privacy fix) and the now-inert CJK echo guard, plus the
  tests that asserted that removed behavior; fix the misleading "OpenAI prompt
  field" comment.
- Add the missing 'Voice Model' and macOS mic-permission i18n keys to
  en/zh/zh-TW (they were falling back to English; check-i18n doesn't flag keys
  absent from en).

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(voice): reject partial stream transcripts

* fix(cli): let dialogs consume voice keys first

* fix(voice): reject incomplete qwen realtime transcripts

* fix(voice): handle stream review blockers

* fix(voice): salvage qwen realtime transcript on close

* fix(voice): sanitize streamed transcript text

* test(audio): run audio capture tests in CI

* fix(voice): address review blockers

* fix(voice): report stream close while recording

* fix(voice): reduce keyterm echo false positives

* fix(voice): handle realtime close diagnostics

* fix(voice): address realtime review blockers

* fix(lint): allow legacy voice filenames

* chore(cli): rename voice files to kebab-case

---------

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-06-22 14:33:36 +08:00

110 lines
3.5 KiB
JavaScript

/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { execSync } from 'node:child_process';
import { existsSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const root = join(__dirname, '..');
// npm install if node_modules was removed (e.g. via npm run clean or scripts/clean.js)
if (!existsSync(join(root, 'node_modules'))) {
execSync('npm install', { stdio: 'inherit', cwd: root });
}
// build all workspaces/packages in dependency order
execSync('npm run generate', { stdio: 'inherit', cwd: root });
// --cli-only: skip packages not needed by the CLI bundle
// (webui, sdk, web-shell, vscode-ide-companion are for IDE/web use only)
const cliOnly = process.argv.includes('--cli-only');
// Build in dependency order:
// 1. core (foundation package, includes test-utils)
// 2. web-templates (embeddable web templates - used by cli)
// 3. channel-base (base channel infrastructure - used by channel adapters and cli)
// 4. channel adapters (depend on channel-base)
// 5. audio-capture (native microphone backend used by cli)
// 6. acp-bridge (depends on core - used by cli)
// 7. cli (depends on core, acp-bridge, web-templates, channel packages)
// 8. webui (shared UI components - used by vscode companion)
// 9. sdk (build-time devDep on acp-bridge for shared constants)
// 10. web-shell (depends on webui and sdk)
// 11. vscode-ide-companion (depends on webui)
const buildOrder = [
'packages/core',
'packages/web-templates',
'packages/channels/base',
'packages/channels/telegram',
'packages/channels/weixin',
'packages/channels/dingtalk',
'packages/channels/feishu',
'packages/channels/qqbot',
'packages/channels/plugin-example',
'packages/audio-capture',
'packages/acp-bridge',
'packages/cli',
...(cliOnly
? []
: [
'packages/webui',
'packages/sdk-typescript',
'packages/web-shell',
'packages/vscode-ide-companion',
]),
];
for (const workspace of buildOrder) {
const command =
workspace === 'packages/audio-capture'
? `npm run build:ts --workspace=${workspace}`
: `npm run build --workspace=${workspace}`;
execSync(command, { stdio: 'inherit', cwd: root });
// After cli is built, generate the JSON Schema for settings
// so the vscode-ide-companion extension can provide IntelliSense
if (workspace === 'packages/cli') {
execSync('node --import tsx/esm scripts/generate-settings-schema.ts', {
stdio: 'inherit',
cwd: root,
});
}
}
// also build container image if sandboxing is enabled
// skip (-s) npm install + build since we did that above
try {
execSync('node scripts/sandbox_command.js -q', {
stdio: 'inherit',
cwd: root,
});
if (
process.env.BUILD_SANDBOX === '1' ||
process.env.BUILD_SANDBOX === 'true'
) {
execSync('node scripts/build_sandbox.js -s', {
stdio: 'inherit',
cwd: root,
});
}
} catch {
// ignore
}