mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-07-09 17:19:02 +00:00
* feat(voice): voice dictation with native capture, streaming, and biasing
Add voice dictation for the prompt input:
- /voice [hold|tap|off|status] command + general.voice.{enabled,mode,language,protocol} settings; push-to-talk via Space, /model --voice to pick the model
- Native microphone capture (@qwen-code/audio-capture, miniaudio N-API) with arecord/SoX fallback, silence auto-stop, cold-start warm-up, and macOS permission query
- Batch transcription via DashScope Qwen-ASR (OpenAI-compatible chat/completions + input_audio) with language + keyterm biasing and an echo guard
- Live streaming over the DashScope realtime WebSocket (fun-asr-realtime / paraformer-realtime-v2) with interim text and an input-level waveform, behind voice.protocol=dashscope-realtime
- Rich VoiceIndicator UI (state, level meter, live partial transcript)
- Cross-platform prebuilds via prebuildify + node-gyp-build and a CI matrix
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* feat(cli): route voice ASR by model
* feat(cli): polish voice realtime parity
* ci: fix voice workflow checks
* fix(cli): address voice review blockers
* fix(cli): harden voice transcription failures
* fix(cli): address voice PR review blockers
* fix(cli): harden voice review follow-ups
* fix(voice): handle realtime review blockers
* fix(cli): add voice command i18n keys
* fix(cli): address voice review blockers
* fix(cli): address voice review follow-ups
* fix(voice): harden review edge cases
* fix(voice): address release and realtime review blockers
* fix(voice): address realtime suggestion followups
* fix(voice): handle realtime review followups
* fix(voice): address realtime review blockers
* fix(voice): address review feedback
* fix(voice): address recorder review feedback
* docs(voice): document ssrf guard boundary
* fix(voice): address review feedback
* fix(voice): preserve warm recorder session safety
* fix(voice): address stream review suggestions
* fix(voice): address multi-round review findings
Realtime/streaming:
- Salvage an already-committed transcript when the WebSocket closes right
after finish() instead of rejecting the whole dictation
(qwenAsrRealtimeSession, voiceStreamSession) + regression tests.
useVoiceInput state machine:
- Single-shot finalize guard so a tap-stop racing the silence auto-stop can't
double-stop the recorder and surface a spurious failure.
- Reset mountedRef on (re)mount so StrictMode (DEBUG) can't freeze the voice UI.
- Widen the hold-mode first-press release window above common key-repeat delays.
Model selection:
- Reject ids with no ASR transport at /model --voice and in the model dialog via
a new isSelectableVoiceModel; move resolveVoiceTransport into voiceModel so the
record-time config resolver stays transport-agnostic (+ tests).
macOS mic permission:
- Surface the not-determined state in voice warmup so the first dictation isn't
silently lost behind the TCC dialog.
Native packaging:
- Make the audio-capture native install non-fatal (falls back to SoX/arecord) so
a voice-only build failure can't break npm ci.
- Download audio-capture prebuilds before building standalone release archives.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(voice): address review blockers
* fix(voice): update batch recording audio level
* fix(voice): tap-mode transcript loss, stream leaks, and dead keyterm cleanup
- Tap-mode dictation submitted a stale empty buffer.text, wiping the just-
inserted transcript and sending nothing: thread the resulting prompt text
through onSubmit(text) instead of reading buffer.text back synchronously
after the async insert (useVoiceInput, InputPrompt).
- Streaming finalize leaked the WebSocket session when recorder.drain() threw:
abort the session before propagating the error.
- voiceStreamSession: reject the connect promise when 'task-finished' arrives
before 'task-started' instead of hanging forever in 'transcribing'.
- Remove dead keyterm enrichment (project/branch/recent-file paths were
unreachable after the privacy fix) and the now-inert CJK echo guard, plus the
tests that asserted that removed behavior; fix the misleading "OpenAI prompt
field" comment.
- Add the missing 'Voice Model' and macOS mic-permission i18n keys to
en/zh/zh-TW (they were falling back to English; check-i18n doesn't flag keys
absent from en).
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(voice): reject partial stream transcripts
* fix(cli): let dialogs consume voice keys first
* fix(voice): reject incomplete qwen realtime transcripts
* fix(voice): handle stream review blockers
* fix(voice): salvage qwen realtime transcript on close
* fix(voice): sanitize streamed transcript text
* test(audio): run audio capture tests in CI
* fix(voice): address review blockers
* fix(voice): report stream close while recording
* fix(voice): reduce keyterm echo false positives
* fix(voice): handle realtime close diagnostics
* fix(voice): address realtime review blockers
* fix(lint): allow legacy voice filenames
* chore(cli): rename voice files to kebab-case
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
185 lines
6.6 KiB
JavaScript
185 lines
6.6 KiB
JavaScript
/**
|
|
* @license
|
|
* Copyright 2025 Google LLC
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*/
|
|
|
|
import path from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
import { createRequire } from 'node:module';
|
|
import { writeFileSync, rmSync, readFileSync } from 'node:fs';
|
|
import { wasmLoader } from 'esbuild-plugin-wasm';
|
|
|
|
let esbuild;
|
|
try {
|
|
esbuild = (await import('esbuild')).default;
|
|
} catch (_error) {
|
|
console.warn('esbuild not available, skipping bundle step');
|
|
process.exit(0);
|
|
}
|
|
|
|
const __filename = fileURLToPath(import.meta.url);
|
|
const __dirname = path.dirname(__filename);
|
|
const require = createRequire(import.meta.url);
|
|
const pkg = require(path.resolve(__dirname, 'package.json'));
|
|
|
|
// Clean dist directory (cross-platform)
|
|
rmSync(path.resolve(__dirname, 'dist'), { recursive: true, force: true });
|
|
|
|
/**
|
|
* Resolve `import X from '*.wasm?binary'` imports to an inline Uint8Array.
|
|
*
|
|
* The `?binary` suffix is a build-time hint: at bundle time (esbuild) the WASM
|
|
* bytes are embedded as base64 and exported as a default Uint8Array, so no
|
|
* external vendor files are needed at runtime. In source / transpiled mode
|
|
* the dynamic import throws and the caller falls back to reading from
|
|
* node_modules via `require.resolve`.
|
|
*/
|
|
const wasmBinaryPlugin = {
|
|
name: 'wasm-binary',
|
|
setup(build) {
|
|
build.onResolve({ filter: /\.wasm\?binary$/ }, (args) => {
|
|
const specifier = args.path.replace(/\?binary$/, '');
|
|
const localRequire = createRequire(
|
|
path.resolve(args.resolveDir || __dirname, '_dummy_.js'),
|
|
);
|
|
return {
|
|
path: localRequire.resolve(specifier),
|
|
namespace: 'wasm-binary',
|
|
};
|
|
});
|
|
build.onLoad({ filter: /.*/, namespace: 'wasm-binary' }, (args) => {
|
|
const contents = readFileSync(args.path);
|
|
return { contents, loader: 'binary' };
|
|
});
|
|
},
|
|
};
|
|
|
|
const external = [
|
|
'@lydell/node-pty',
|
|
'node-pty',
|
|
'@lydell/node-pty-darwin-arm64',
|
|
'@lydell/node-pty-darwin-x64',
|
|
'@lydell/node-pty-linux-x64',
|
|
'@lydell/node-pty-win32-arm64',
|
|
'@lydell/node-pty-win32-x64',
|
|
'@qwen-code/audio-capture',
|
|
'@teddyzhu/clipboard',
|
|
'@teddyzhu/clipboard-darwin-arm64',
|
|
'@teddyzhu/clipboard-darwin-x64',
|
|
'@teddyzhu/clipboard-linux-x64-gnu',
|
|
'@teddyzhu/clipboard-linux-arm64-gnu',
|
|
'@teddyzhu/clipboard-win32-x64-msvc',
|
|
'@teddyzhu/clipboard-win32-arm64-msvc',
|
|
];
|
|
|
|
// Name of the directory under `dist/` that esbuild emits shared chunks into.
|
|
// MUST stay in sync with `BUNDLE_CHUNK_DIR` in
|
|
// `packages/core/src/utils/bundlePaths.ts`, whose `resolveBundleDir` helper
|
|
// strips this exact segment when modules look up sibling assets at runtime.
|
|
// Renaming here without renaming there silently breaks bundled-binary lookup
|
|
// in skill-manager / ripgrepUtils / i18n / extensions/new.
|
|
const BUNDLE_CHUNK_DIR = 'chunks';
|
|
|
|
const mainBuild = esbuild.build({
|
|
entryPoints: { cli: 'packages/cli/index.ts' },
|
|
bundle: true,
|
|
outdir: 'dist',
|
|
entryNames: '[name]',
|
|
chunkNames: `${BUNDLE_CHUNK_DIR}/[name]-[hash]`,
|
|
splitting: true,
|
|
platform: 'node',
|
|
format: 'esm',
|
|
target: 'node22',
|
|
external,
|
|
packages: 'bundle',
|
|
inject: [path.resolve(__dirname, 'scripts/esbuild-shims.js')],
|
|
banner: {
|
|
js: `// Force strict mode and setup for ESM
|
|
"use strict";`,
|
|
},
|
|
alias: {
|
|
'is-in-ci': path.resolve(__dirname, 'packages/cli/src/patches/is-in-ci.ts'),
|
|
'@qwen-code/web-templates': path.resolve(
|
|
__dirname,
|
|
'packages/web-templates/src/index.ts',
|
|
),
|
|
// Resolve to userland punycode instead of deprecated node:punycode built-in
|
|
punycode: require.resolve('punycode/'),
|
|
},
|
|
define: {
|
|
'process.env.CLI_VERSION': JSON.stringify(pkg.version),
|
|
// react-reconciler ≥0.33 (ink 7) gates its dev build behind NODE_ENV
|
|
// and calls performance.measure() on every render, leaking
|
|
// PerformanceMeasure objects into the global measureEntryBuffer.
|
|
// Setting production here tree-shakes the entire dev build (~15k lines).
|
|
'process.env.NODE_ENV': JSON.stringify('production'),
|
|
// Make global available for compatibility
|
|
global: 'globalThis',
|
|
// Redirect free __dirname/__filename references to the shim so that
|
|
// vendored libraries that emit their own `var __dirname` locals don't
|
|
// collide with our injected bindings when code-splitting is enabled.
|
|
//
|
|
// CONTRIBUTOR WARNING: this rewrite applies to *all* source files, so
|
|
// any bare `__dirname` / `__filename` in our own code resolves to the
|
|
// shim chunk's on-disk location (i.e. `dist/chunks/`), NOT the source
|
|
// file's own directory. To get a per-file path, declare a local shadow
|
|
// at the top of the module:
|
|
//
|
|
// import { fileURLToPath } from 'node:url';
|
|
// const __filename = fileURLToPath(import.meta.url);
|
|
// const __dirname = path.dirname(__filename);
|
|
//
|
|
// esbuild leaves the local binding alone (it's a declared identifier,
|
|
// not a free reference). For sibling-asset lookups in modules that may
|
|
// be hoisted into a shared chunk, prefer
|
|
// `resolveBundleDir(import.meta.url)` from
|
|
// `packages/core/src/utils/bundlePaths.ts` — it both produces a
|
|
// per-file path and strips the chunk segment when the module ends up
|
|
// under `dist/chunks/`.
|
|
__dirname: '__qwen_dirname',
|
|
__filename: '__qwen_filename',
|
|
},
|
|
loader: { '.node': 'file' },
|
|
plugins: [wasmBinaryPlugin, wasmLoader({ mode: 'embedded' })],
|
|
metafile: true,
|
|
write: true,
|
|
keepNames: true,
|
|
});
|
|
|
|
// fzf index worker — runs in its own worker_threads worker that
|
|
// `fzfWorkerHandle.ts` spawns via `new Worker(new URL('./fzfWorker.js', ...))`.
|
|
// Must exist as a standalone file next to `dist/cli.js` so the URL resolves
|
|
// at runtime; we bundle it self-contained (no chunk splitting) so fzf is
|
|
// inlined and the worker doesn't need to walk back into node_modules from
|
|
// the published tarball. `prepare-package.js` whitelists `fzfWorker.js` in
|
|
// the dist `files` array.
|
|
const workerBuild = esbuild.build({
|
|
entryPoints: ['packages/core/src/utils/filesearch/fzfWorker.ts'],
|
|
bundle: true,
|
|
outfile: 'dist/fzfWorker.js',
|
|
platform: 'node',
|
|
format: 'esm',
|
|
target: 'node22',
|
|
external,
|
|
packages: 'bundle',
|
|
// fzf is CJS — needs the same require()-shim the main bundle uses for
|
|
// CJS interop in ESM output.
|
|
inject: [path.resolve(__dirname, 'scripts/esbuild-shims.js')],
|
|
banner: {
|
|
js: `"use strict";`,
|
|
},
|
|
write: true,
|
|
keepNames: true,
|
|
});
|
|
|
|
Promise.all([mainBuild, workerBuild])
|
|
.then(([{ metafile }]) => {
|
|
if (process.env.DEV === 'true') {
|
|
writeFileSync('./dist/esbuild.json', JSON.stringify(metafile, null, 2));
|
|
}
|
|
})
|
|
.catch((error) => {
|
|
console.error('esbuild build failed:', error);
|
|
process.exitCode = 1;
|
|
});
|