mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-08-11 01:36:35 +00:00
* feat(review): say so when the bundle is older than the review it runs Every `qwen review …` step runs the BUILT bundle, not the working tree. So editing a review command, or switching to a branch that contains one, changes nothing about the run until someone rebuilds -- and the failure is silent and total: the run behaves like the last build, and every conclusion drawn from it is a conclusion about that build. Measured on 2026-08-02, dogfooding /review against #8368 from a checkout whose bundle was fourteen hours old. Three things were invalidated at once and none announced itself: `drive` and `mock-provider` had merged that morning and were absent from the binary, so "the agent never reached for them" measured nothing; and #8345's guard against scoring a mutant `survived` when its own collocated test was red had merged too, so the run reproduced the bug it fixed and filed three findings the current code holds as `inconclusive`. The round was discarded and re-run after a rebuild. `parse-args` is the first command of every review, which makes it the only place a notice reaches a reader before they act on a result. It names the file that is ahead, by how much, what actually runs from the bundle, and the command to rebuild -- "rebuild" without evidence is advice nobody can check. mtime, not git: the question is whether this bundle was built from this source, and a git comparison answers a different one. A margin absorbs a checkout, which writes everything at once in no guaranteed order. An installed package has no sources beside it, finds nothing to compare, and stays silent -- a check that cannot see the files must not accuse the build. Also documents `findings --test-delta` for users: it can lower a severity, and therefore change what the verdict is computed from, so it belongs beside `--outcomes` rather than only in the skill. * fix(review): watch the file every subcommand is registered in `packages/cli/src/commands/review.ts` is where all 30-odd subcommands are imported and registered, and it sits beside the directory rather than in it -- so a new command, or a changed dispatch, was exactly the change this check could not see. A root may now be a single file, which is what that one is. Confirmed end to end: with `review.ts` three hours ahead of a fresh bundle, the warning names it. Also two comments that did not match the code: symlinks of every kind are skipped, not only directories (`isFile()` is false for a symlinked file too), and the module now says what `QWEN_CODE_CLI` already covers -- talking to a different program -- so it is clear this guards the other half, the right program built before the change. * fix(review): compare content, because a timestamp check cried wolf The first version compared the bundle's mtime against the newest review source, and it was wrong in the direction that matters most. `git checkout` rewrites every file that differs between two commits, so returning to the branch a bundle was built from re-stamps exactly those files and the check calls a byte-for-byte correct bundle stale. Measured: with the sources untouched and the bundle two minutes older, it warned. A line that fires when nothing is wrong teaches its reader to skip the line, which would have made this worse than absent. The build now stamps a digest of the review sources it bundled into `dist/review-sources.sha256`, and the check re-derives that digest from the tree and compares. No margin to tune, no clock to trust, and no answer but the true one. Verified end to end across all five cases: a clean tree is silent, a source touched but unchanged is silent, and a real change under any of the three roots -- the command directory, the `review.ts` that registers them, the bundled skill -- warns. The digest is now one rule stated twice, since the build script cannot import the package it runs before building. `scripts/tests/review-source-digest.test.ts` holds the two equal, on this repo and on a synthetic tree that exercises the file-shaped root; a package test may not reach into `scripts/`, so it lives on the side of the boundary that may. Paths are folded relative to the repo root with separators normalised, and the file list is sorted -- `readdir` order is a property of the filesystem, so without it a bundle built in CI and a tree cloned locally would hash the same source differently and every run would warn. * fix(review): a diagnostic must not kill the run, and tests are not the bundle Two Criticals and five suggestions from review, all verified before changing anything. `writeStderrLine` throws on EPIPE, so stderr piped to `head` would have killed the review before it parsed a single argument -- a warning that destroys the run it was warning about, and the opposite of this change's own invariant. `writeStderrLineSafe` is the convention for diagnostics in this subsystem and is what it calls now. `reviewSourceRoots` builds paths with the platform `join`, and the test asserted forward-slash literals, so all three elements would have failed on the merge queue's Windows leg -- which the pull_request event never runs, so the green CI here proved nothing about it. Test files left the digest. esbuild follows imports from the CLI entry and no test is reachable that way, so folding them in fired the warning for an edit that cannot change a byte of the bundle -- the false positive this module already rejected once. 112 files became 61, and a test-only edit is now silent while a production one still warns. The handler wiring is tested at last, against a real temp tree rather than a mock of the reads under test: the derivation from `process.argv[1]`, the stamp read, and the warning. All three mutations the review named -- dropping the call, reading the stamp from the wrong directory, collapsing repoRoot to distDir -- now redden it. Also: the stamp's filename is pinned across the boundary it crosses (the build wrote a literal while the check read `DIGEST_FILE`, so a one-sided rename would have silenced the feature with every test green); the digest is computed only when there is a stamp to compare it against, instead of hashing a hundred files for a value the first guard discards; the `rebuildCommand` parameter no caller ever set is gone; and the build script's comment no longer claims a code-sharing relationship that does not exist. * fix(review): fixtures are not in the bundle either The same false positive, a third time and one directory over. Excluding tests from the digest was right and incomplete: `review/__fixtures__` holds four files — three responder modules and a captured comment — that a test loads at runtime, from no import the bundler follows. Measured against `dist`: none of the four appears in it, so editing one changed the digest while the bundle stayed byte-identical and the warning claimed a review command had changed. Both walks skip the directory now, and the parity test's synthetic tree grows a fixture and a `.spec.tsx` so the two implementations are held equal on the whole exclusion, not just the part the first case exercised. Reverting one side reddens the local case AND both parity cases, which is what that guard is for. Verified the other direction too, since an exclusion can overshoot: every review source that reaches `dist` is still covered. `DESIGN.md` and `SKILL.md` both ship and both remain in the digest — checked, not assumed, after two rounds of this exact mistake. Six cases end to end after a rebuild: a clean tree, a test edit and a fixture edit are silent; a production edit, a `review.ts` edit and a `DESIGN.md` edit each warn. * fix(review): allowlist the stamp, and stop guessing what the bundle holds The Critical first: `create-standalone-package.js` fails on any top-level dist entry outside its allowlist, and `review-sources.sha256` was on neither list. The next release would have aborted the standalone archive on all five targets, and no PR-time job runs the packager, which is why this suite is green. Allowlisted -- shipping it is harmless, since a standalone install has no `packages/` to compare against and the check stays silent there. `lib/test-utils.ts` was in the digest: test support with a production-looking name, imported by two test files and nothing else. That is the fourth patch to one rule -- `.test.ts`, then `__fixtures__/`, then this, plus `.DS_Store` -- and each was found by a reviewer after it shipped. So the rule stops being a list somebody remembers to extend: a new test asserts the property the list approximates, that every file the digest folds in is reachable from production code and nothing reachable is left out. Dropping `test-utils.ts` from the exclusion reddens it, which is the fifth instance failing in CI instead of in a review. Three branches that no test reached, each with a mutant the review measured surviving the whole suite: the walk's symlink skip (a directory cycle would send the first command of every review into unbounded recursion), the read-failure path (hashing the survivors of a concurrent checkout would accuse a tree that is merely mid-change), and the build's stamp call site (removing it left the scripts suite green while `npm run bundle` silently stopped writing the stamp). All three now redden. And `unmeasured` had no reader, so the one edge this check cannot measure but can see -- sources present, stamp absent -- passed in silence. That is the state of every existing checkout the moment this ships, and it is exactly the silent failure the change was written to end. It now says so, while an installed package, which has no sources either, still says nothing. * fix(review): the guard was shallower than the property it claimed The guard added last round asserts that every file in the digest is reachable from production code. It did not: a file imported by nothing passed, because the filter also required some test to import it; only `.ts` was inspected, so a test-only `.tsx` or `.mts` helper walked through; and it read static imports only, while this directory has nine `await import('./…')` edges. It asserts the property now — every extension, orphans included, dynamic edges seen — and the tree has no violators, so the strictness cost nothing today and is there for the next file. `__snapshots__` joins the exclusions. `vitest --update` regenerating a snapshot would have moved the digest with the bundle byte-identical; none exists under the review roots today only by chance, and 120 `toMatchSnapshot()` calls live elsewhere in this package. Three couplings that no test held: - the allowlist entry that fixed the release-breaking R2-1 -- reverting those five lines left the whole scripts suite green, and the next failure would have been a release aborting on all five targets. `isAllowedDistEntry` is exported and the stamp's own name is asserted against it, so a one-sided rename fails here instead; - the `.DS_Store` member of `NOT_BUNDLED_FILE`, absent from the repo and so from the parity tree -- one-sided removal stayed green while a macOS checkout would digest differently on the two sides forever; - each `unmeasured` reason. Swapping the two arguments at the single call site kept all 76 tests green while telling a pre-stamp checkout its sources were missing. And two comments that said the opposite of the code beneath them: the digest is computed unconditionally on purpose (the pre-stamp notice needs it), and `NOT_BUNDLED_FILE` helpers are deliberately not importers, since nothing reaches the bundle through a file the bundle does not contain. The two stderr diagnostics are documented for users, beside the sibling paragraph this PR already added. * fix(review): measure only the layout that can carry a stamp `npm start` launches `node <root>/packages/cli`, and node sets `argv[1]` to that directory -- so the derivation found sources under `<root>` with no stamp beside them and printed "could not check" on every review, forever, with advice that could never make it stop. That is the fires-when-nothing-is-wrong failure this change argues against, on the path `start.js` sets `QWEN_CODE_CLI` to precisely so reviews reach that build. Only a `<root>/dist/cli.js` layout is measured now; anything else has no stamp to find and no way to grow one. The build-side digest could kill `npm run bundle` where the check side degrades gracefully: a file vanishing mid-walk threw out of the hash loop, and the stamp is the copier's last step, so the build would fail with every asset already in place. Caught and skipped -- a missing stamp is `unmeasured`, which the runtime already treats as an acceptable answer. The skill now says what to do with the warning, which is the half that makes it reach a human: `parse-args` runs inside an agent's shell tool, the user reads the agent's summary rather than raw stderr, and a line nobody repeats is a line nobody sees -- which is how the 2026-08-02 round went wrong in the first place. It also records that the instruction cannot help the run that needs it, since the skill comes from the same bundle. And the scope is stated where silence could be over-read: the digest covers the review commands, the file that registers them, and the bundled skill -- not the shared helpers those import. A quiet run means the review code matches the bundle, not that the tree does. * fix(review): refuse to certify a bundle the copier may not describe The stamp described the tree as the COPIER saw it, and the copier runs after esbuild -- so a source edited in between, or `copy_bundle_assets.js` run on its own (it self-executes), wrote a digest certifying a `cli.js` built from something else. Silence then means "verified fresh" when it is not, and that is the only direction here where a quiet run is affirmatively wrong rather than merely uninformative: every other gap degrades to `unmeasured`. Timestamps are the wrong tool for judging staleness and the right one for judging whether this stamp can be honest at all, so the build refuses when any source is newer than the bundle it would attest to, and says why. Driven for real: touching a review source and running the copier alone now prints "skipped the source digest rather than certify a bundle it may not describe". `it('counts the same files')` compared nothing -- it asserted `> 50` on the build side while the check side exposes no count, so the title claimed a parity the body never checked, and the margin over the real 56 made it a future false alarm in `scripts/` for an unrelated change. Removed; the digest parity already holds the file set. "Root is a file" was inferred from `readdirSync` raising ENOTDIR, an assumption about every platform's libuv on the one root that is a file -- `review.ts`, where "a new subcommand was registered" lives. `statSync(root).isFile()` says it instead. And the check itself moves out of the handler into `bundleStalenessNotices`, which is where the rest of it already lived. `parse-args` is about parsing arguments again, the wording is testable without the yargs harness, and a second caller -- an agent resuming a review never runs step 1 -- is one line. * fix(review): align the twin walk, and stop a test from passing on nothing The build side still inferred "this root is a file" from `readdirSync` raising ENOTDIR, one commit after the check side stopped doing exactly that and said why. A platform that maps the case differently would drop `commands/review.ts` from one digest and not the other, and a byte-for-byte correct bundle would warn on every review forever, on that platform alone, with rebuilding reproducing the same one-sided walk. Both sides ask `statSync(...).isFile()` now. Fixing one half of a pair and not the other is the mistake this file keeps making. The filename parity test had been passing on nothing since the previous commit: it matched `writeFileSync(join(distDir, '…'))` against the script's source, the literal moved into a `stampPath` variable, and the regex returned `undefined` so the assertion compared against nothing. It runs the build against a fixture now and reads the name off `dist/`, so it measures what the build does instead of what its source looks like. Renaming the stamp on one side reddens it. Also from review: the duplicated comment block in `parse-args`; an unreadable source now says the check could not run rather than passing in the same silence as an installed package, which is what the docstring already promised; the "could not check" line no longer asserts that the checkout predates the feature, since the build has three refusal paths and one of them means the opposite; every refusal removes an existing stamp, because leaving an older attestation beside a newer bundle is a weaker form of the certifying it refuses; and `drive` calls the check, which the module comment argued for and the diff had not done -- a resumed review never runs step 1, and that is where the long work starts. * fix(review): pin the regex group the parity tree missed, and say source, not command * fix(review): allowlist what the bundle holds, and cover the drive notice (#8390) * fix(review): treat unreadable review sources as unmeasured (#8390) * test(review): pin the stamp guard mutations that survived the suite (#8390) * fix(review): close staleness-check gaps and pin the round-4 survivors (#8390) * fix(review): close round-5 staleness gaps for parity, refusals, and partial checkouts (#8390) * fix(review): close round-6 gaps in the clause classifier, symlink layout, and pin honesty (#8390) * fix(review): close round-7 gaps in the closure oracle, parity pin, and refusal pins (#8390) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(review): close round-8 gaps from the maintainer review (#8390) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(review): close round-9 nits from the maintainer review (#8390) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(review): pin the lease root in the synthetic digest parity case (#8390) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Autofix <autofix@qwen-code.dev> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
806 lines
23 KiB
JavaScript
806 lines
23 KiB
JavaScript
#!/usr/bin/env node
|
|
|
|
/**
|
|
* @license
|
|
* Copyright 2025 Qwen Team
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*/
|
|
|
|
import { execFileSync } from 'node:child_process';
|
|
import crypto from 'node:crypto';
|
|
import fs from 'node:fs';
|
|
import os from 'node:os';
|
|
import path from 'node:path';
|
|
import { createRequire } from 'node:module';
|
|
import { pipeline } from 'node:stream/promises';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
const __filename = fileURLToPath(import.meta.url);
|
|
const __dirname = path.dirname(__filename);
|
|
const rootDir = path.resolve(__dirname, '..');
|
|
const distDir = path.join(rootDir, 'dist');
|
|
|
|
const TARGETS = new Map([
|
|
[
|
|
'darwin-arm64',
|
|
{ outputExtension: 'tar.gz', nodeExecutable: ['bin', 'node'] },
|
|
],
|
|
[
|
|
'darwin-x64',
|
|
{ outputExtension: 'tar.gz', nodeExecutable: ['bin', 'node'] },
|
|
],
|
|
[
|
|
'linux-arm64',
|
|
{ outputExtension: 'tar.gz', nodeExecutable: ['bin', 'node'] },
|
|
],
|
|
['linux-x64', { outputExtension: 'tar.gz', nodeExecutable: ['bin', 'node'] }],
|
|
['win-x64', { outputExtension: 'zip', nodeExecutable: ['node.exe'] }],
|
|
]);
|
|
|
|
// Standalone target -> prebuildify platform-arch dir name (process.platform
|
|
// based, so Windows is 'win32'). Only this archive's matching prebuild is
|
|
// bundled, keeping each archive lean and correct-arch.
|
|
const TARGET_PREBUILD_DIR = new Map([
|
|
['darwin-arm64', 'darwin-arm64'],
|
|
['darwin-x64', 'darwin-x64'],
|
|
['linux-arm64', 'linux-arm64'],
|
|
['linux-x64', 'linux-x64'],
|
|
['win-x64', 'win32-x64'],
|
|
]);
|
|
|
|
const TARGET_CLIPBOARD_PACKAGE = new Map([
|
|
['darwin-arm64', '@teddyzhu/clipboard-darwin-arm64'],
|
|
['darwin-x64', '@teddyzhu/clipboard-darwin-x64'],
|
|
['linux-arm64', '@teddyzhu/clipboard-linux-arm64-gnu'],
|
|
['linux-x64', '@teddyzhu/clipboard-linux-x64-gnu'],
|
|
['win-x64', '@teddyzhu/clipboard-win32-x64-msvc'],
|
|
]);
|
|
|
|
const DIST_REQUIRED_PATHS = [
|
|
'cli.js',
|
|
'cli-entry.js',
|
|
'chunks',
|
|
'vendor',
|
|
'bundled/qc-helper/docs',
|
|
];
|
|
const DIST_ALLOWED_ENTRIES = new Set([
|
|
'cli.js',
|
|
// bin wrapper emitted by prepare-package.js. Standalone shims use it for
|
|
// `qwen serve` so daemon startup gets the same fast path as npm installs.
|
|
'cli-entry.js',
|
|
// fzf fuzzy-search worker; esbuild emits it as a standalone entry that must
|
|
// sit next to cli.js so `new URL('./fzfWorker.js', ...)` resolves at runtime.
|
|
'fzfWorker.js',
|
|
'chunks',
|
|
'vendor',
|
|
'bundled',
|
|
'package.json',
|
|
'README.md',
|
|
'LICENSE',
|
|
// Digest of the review sources this bundle was built from, stamped by
|
|
// copy_bundle_assets.js. Harmless to ship: a standalone install lays the
|
|
// dist entries out under `lib/`, and the staleness check only applies to
|
|
// a `<root>/dist/cli.js` layout — it never reads the stamp there.
|
|
'review-sources.sha256',
|
|
'locales',
|
|
'examples',
|
|
// Web Shell SPA served at the daemon root by `qwen serve` (index.html +
|
|
// assets/). Copied into dist/web-shell/ by copy_bundle_assets.js when the
|
|
// web-shell workspace has been built; optional, so it's allowed but not
|
|
// required.
|
|
'web-shell',
|
|
]);
|
|
const DIST_ALLOWED_ENTRY_PATTERNS = [
|
|
/^sandbox-macos-(permissive|restrictive)-(open|closed|proxied)\.sb$/,
|
|
];
|
|
// Emitted into dist/ by prepare-package.js for npm publishing only;
|
|
// standalone archives must not copy them into lib/.
|
|
const DIST_NPM_PACKAGE_ONLY_ENTRIES = new Set(['postinstall.js', 'patches']);
|
|
const ROOT_REQUIRED_PATHS = ['README.md', 'LICENSE'];
|
|
|
|
if (isMainModule()) {
|
|
try {
|
|
await main();
|
|
} catch (error) {
|
|
console.error(error instanceof Error ? error.message : error);
|
|
process.exitCode = 1;
|
|
}
|
|
}
|
|
|
|
async function main() {
|
|
const args = parseArgs(process.argv.slice(2));
|
|
|
|
if (args.help) {
|
|
printUsage();
|
|
return;
|
|
}
|
|
|
|
const target = args.target;
|
|
if (!target || !TARGETS.has(target)) {
|
|
fail(`--target must be one of: ${Array.from(TARGETS.keys()).join(', ')}`);
|
|
}
|
|
|
|
if (!args.nodeArchive) {
|
|
fail('--node-archive is required');
|
|
}
|
|
|
|
const nodeArchive = path.resolve(args.nodeArchive);
|
|
if (!fs.existsSync(nodeArchive)) {
|
|
fail(`Node.js archive not found: ${nodeArchive}`);
|
|
}
|
|
|
|
assertRequiredInputs();
|
|
|
|
const version = args.version || readPackageVersion();
|
|
const outDir = path.resolve(args.outDir || path.join(distDir, 'standalone'));
|
|
fs.mkdirSync(outDir, { recursive: true });
|
|
|
|
const targetConfig = TARGETS.get(target);
|
|
const outputName = `qwen-code-${target}.${targetConfig.outputExtension}`;
|
|
const outputPath = path.join(outDir, outputName);
|
|
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'qwen-standalone-'));
|
|
|
|
try {
|
|
const packageRoot = path.join(tempRoot, 'qwen-code');
|
|
const runtimeExtractDir = path.join(tempRoot, 'runtime');
|
|
fs.mkdirSync(packageRoot, { recursive: true });
|
|
fs.mkdirSync(runtimeExtractDir, { recursive: true });
|
|
|
|
copyRuntimeAssets(packageRoot, outDir);
|
|
copyNativeAddon(packageRoot, target);
|
|
copyClipboardAddon(packageRoot, target, args.nativeModulesDir);
|
|
extractNodeArchive(nodeArchive, runtimeExtractDir);
|
|
const nodeDir = path.join(packageRoot, 'node');
|
|
copyExtractedNode(runtimeExtractDir, nodeDir);
|
|
validateNodeRuntime(target, nodeDir);
|
|
writeShims(packageRoot);
|
|
writeManifest(packageRoot, {
|
|
version,
|
|
target,
|
|
nodeArchive: path.basename(nodeArchive),
|
|
});
|
|
|
|
if (fs.existsSync(outputPath)) {
|
|
fs.rmSync(outputPath, { force: true });
|
|
}
|
|
createArchive(targetConfig.outputExtension, outputPath, tempRoot);
|
|
if (!args.skipChecksums) {
|
|
await writeSha256Sums(outDir);
|
|
}
|
|
|
|
console.log(`Created ${path.relative(rootDir, outputPath)}`);
|
|
if (!args.skipChecksums) {
|
|
console.log(
|
|
`Updated ${path.relative(rootDir, path.join(outDir, 'SHA256SUMS'))}`,
|
|
);
|
|
}
|
|
} finally {
|
|
fs.rmSync(tempRoot, { recursive: true, force: true });
|
|
}
|
|
}
|
|
|
|
function isMainModule() {
|
|
return process.argv[1] && path.resolve(process.argv[1]) === __filename;
|
|
}
|
|
|
|
function parseArgs(argv) {
|
|
const args = {
|
|
help: false,
|
|
nativeModulesDir: undefined,
|
|
outDir: undefined,
|
|
nodeArchive: undefined,
|
|
skipChecksums: false,
|
|
target: undefined,
|
|
version: undefined,
|
|
};
|
|
|
|
for (let index = 0; index < argv.length; index += 1) {
|
|
const arg = argv[index];
|
|
switch (arg) {
|
|
case '--help':
|
|
case '-h':
|
|
args.help = true;
|
|
break;
|
|
case '--target':
|
|
args.target = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
break;
|
|
case '--node-archive':
|
|
args.nodeArchive = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
break;
|
|
case '--native-modules-dir':
|
|
args.nativeModulesDir = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
break;
|
|
case '--out-dir':
|
|
args.outDir = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
break;
|
|
case '--version':
|
|
args.version = readOptionValue(argv, index, arg);
|
|
index += 1;
|
|
break;
|
|
case '--skip-checksums':
|
|
args.skipChecksums = true;
|
|
break;
|
|
default:
|
|
fail(`Unknown option: ${arg}`);
|
|
}
|
|
}
|
|
|
|
return args;
|
|
}
|
|
|
|
function readOptionValue(argv, index, optionName) {
|
|
const value = argv[index + 1];
|
|
if (!value || value.startsWith('-')) {
|
|
fail(`${optionName} requires a value`);
|
|
}
|
|
return value;
|
|
}
|
|
|
|
function printUsage() {
|
|
console.log(`Qwen Code standalone package builder
|
|
|
|
Usage:
|
|
npm run package:standalone -- --target TARGET --node-archive PATH [OPTIONS]
|
|
|
|
Options:
|
|
--target TARGET One of: ${Array.from(TARGETS.keys()).join(', ')}
|
|
--node-archive PATH Downloaded Node.js runtime archive.
|
|
--native-modules-dir DIR
|
|
Staged native node_modules directory. Missing
|
|
clipboard packages are fatal when this is supplied.
|
|
--out-dir DIR Output directory. Defaults to dist/standalone.
|
|
--version VERSION Qwen Code version. Defaults to package.json version.
|
|
--skip-checksums Do not update SHA256SUMS. Used by release packaging.
|
|
-h, --help Show this help message.`);
|
|
}
|
|
|
|
function assertRequiredInputs() {
|
|
if (!fs.existsSync(distDir)) {
|
|
fail('dist/ directory not found. Run "npm run bundle" first.');
|
|
}
|
|
|
|
for (const relativePath of DIST_REQUIRED_PATHS) {
|
|
const fullPath = path.join(distDir, relativePath);
|
|
if (!fs.existsSync(fullPath)) {
|
|
fail(
|
|
`Required dist asset missing: ${fullPath}. ` +
|
|
'Run "npm run bundle" and "npm run prepare:package" first.',
|
|
);
|
|
}
|
|
}
|
|
|
|
for (const relativePath of ROOT_REQUIRED_PATHS) {
|
|
const fullPath = path.join(rootDir, relativePath);
|
|
if (!fs.existsSync(fullPath)) {
|
|
fail(`Required repository file missing: ${fullPath}`);
|
|
}
|
|
}
|
|
}
|
|
|
|
function readPackageVersion() {
|
|
const packageJsonPath = path.join(rootDir, 'package.json');
|
|
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
|
|
return packageJson.version;
|
|
}
|
|
|
|
function copyRuntimeAssets(packageRoot, outDir) {
|
|
const libDir = path.join(packageRoot, 'lib');
|
|
const skippedDistEntry = topLevelDistEntryForPath(outDir);
|
|
fs.mkdirSync(libDir, { recursive: true });
|
|
|
|
for (const entry of fs.readdirSync(distDir)) {
|
|
// Standalone rebuilds a clean, target-trimmed lib/node_modules via the
|
|
// native addon copy steps. If a local dist/node_modules exists from older
|
|
// packaging output or manual testing, copying it would drag in unrelated
|
|
// packages or every platform's native prebuild.
|
|
if (
|
|
entry === skippedDistEntry ||
|
|
entry === '.DS_Store' ||
|
|
entry === 'node_modules' ||
|
|
DIST_NPM_PACKAGE_ONLY_ENTRIES.has(entry)
|
|
) {
|
|
continue;
|
|
}
|
|
if (!isAllowedDistEntry(entry)) {
|
|
fail(`Unexpected dist asset: ${path.join(distDir, entry)}`);
|
|
}
|
|
fs.cpSync(path.join(distDir, entry), path.join(libDir, entry), {
|
|
recursive: true,
|
|
dereference: true,
|
|
verbatimSymlinks: false,
|
|
});
|
|
}
|
|
assertNoSymlinks(libDir, 'Copied runtime assets still contain symlinks.');
|
|
|
|
for (const fileName of ROOT_REQUIRED_PATHS) {
|
|
fs.copyFileSync(
|
|
path.join(rootDir, fileName),
|
|
path.join(packageRoot, fileName),
|
|
);
|
|
}
|
|
|
|
fs.copyFileSync(
|
|
path.join(rootDir, 'package.json'),
|
|
path.join(packageRoot, 'package.json'),
|
|
);
|
|
}
|
|
|
|
// Bundle the @qwen-code/audio-capture native addon (compiled JS + only this
|
|
// target's prebuild + its runtime dep node-gyp-build) into lib/node_modules so
|
|
// streaming voice works in standalone installs. The addon is esbuild-external
|
|
// and resolved at runtime via import('@qwen-code/audio-capture') from
|
|
// lib/cli.js, so lib/node_modules is where Node looks. Without it, standalone
|
|
// users fall back to SoX/arecord (batch only) — #5502 follow-up #5590.
|
|
function copyNativeAddon(packageRoot, target) {
|
|
const prebuildDirName = TARGET_PREBUILD_DIR.get(target);
|
|
const addonSrc = path.join(rootDir, 'packages', 'audio-capture');
|
|
const prebuildSrc = path.join(addonSrc, 'prebuilds', prebuildDirName);
|
|
if (!hasNativePrebuild(prebuildSrc)) {
|
|
if (process.env.QWEN_STANDALONE_REQUIRE_AUDIO_CAPTURE_PREBUILD === '1') {
|
|
fail(
|
|
`Required audio-capture prebuild is missing for ${prebuildDirName}: ${prebuildSrc}`,
|
|
);
|
|
}
|
|
// No prebuild for this target (e.g. a local build without the release
|
|
// artifacts). Ship without the addon: voice degrades to the SoX/arecord
|
|
// fallback, streaming is unavailable. The release pipeline downloads
|
|
// prebuilds before packaging, so release archives do bundle it.
|
|
console.warn(
|
|
`[standalone] no audio-capture prebuild for ${prebuildDirName}; ` +
|
|
'bundling without the native addon (streaming voice unavailable; ' +
|
|
'batch via SoX still works).',
|
|
);
|
|
return;
|
|
}
|
|
|
|
const nodeRequire = createRequire(import.meta.url);
|
|
const nodeGypBuildSrc = path.dirname(
|
|
nodeRequire.resolve('node-gyp-build/package.json'),
|
|
);
|
|
|
|
const modulesDir = path.join(packageRoot, 'lib', 'node_modules');
|
|
const addonDest = path.join(modulesDir, '@qwen-code', 'audio-capture');
|
|
fs.mkdirSync(addonDest, { recursive: true });
|
|
|
|
// Trimmed manifest: keep type/exports so ESM resolution works; drop the
|
|
// install hook (no npm runs inside the archive).
|
|
const addonPkg = JSON.parse(
|
|
fs.readFileSync(path.join(addonSrc, 'package.json'), 'utf8'),
|
|
);
|
|
delete addonPkg.scripts;
|
|
delete addonPkg.devDependencies;
|
|
fs.writeFileSync(
|
|
path.join(addonDest, 'package.json'),
|
|
JSON.stringify(addonPkg, null, 2) + '\n',
|
|
);
|
|
|
|
const copyOpts = {
|
|
recursive: true,
|
|
dereference: true,
|
|
verbatimSymlinks: false,
|
|
};
|
|
fs.cpSync(path.join(addonSrc, 'dist'), path.join(addonDest, 'dist'), {
|
|
...copyOpts,
|
|
filter: (src) => !/\.test\.(d\.)?[mc]?[jt]s(\.map)?$/.test(src),
|
|
});
|
|
fs.cpSync(
|
|
prebuildSrc,
|
|
path.join(addonDest, 'prebuilds', prebuildDirName),
|
|
copyOpts,
|
|
);
|
|
// node-gyp-build is the addon's only runtime dependency (zero-dep itself).
|
|
fs.cpSync(nodeGypBuildSrc, path.join(modulesDir, 'node-gyp-build'), copyOpts);
|
|
|
|
assertNoSymlinks(modulesDir, 'Bundled native addon still contains symlinks.');
|
|
}
|
|
|
|
function copyClipboardAddon(packageRoot, target, nativeModulesDir) {
|
|
const modulesSrc = path.resolve(
|
|
nativeModulesDir || path.join(rootDir, 'node_modules'),
|
|
);
|
|
const nativePackage = TARGET_CLIPBOARD_PACKAGE.get(target);
|
|
const packageNames = ['@teddyzhu/clipboard', nativePackage];
|
|
const packageSources = packageNames.map((packageName) =>
|
|
path.join(modulesSrc, packageName),
|
|
);
|
|
const nativePackageSrc = packageSources[1];
|
|
const hasRequiredFiles =
|
|
packageSources.every((packageSrc) =>
|
|
fs.existsSync(path.join(packageSrc, 'package.json')),
|
|
) &&
|
|
fs.readdirSync(nativePackageSrc).some((entry) => entry.endsWith('.node'));
|
|
|
|
if (!hasRequiredFiles) {
|
|
const message = `clipboard packages for ${target} are missing from ${modulesSrc}`;
|
|
if (nativeModulesDir) {
|
|
fail(`Required ${message}`);
|
|
}
|
|
console.warn(
|
|
`[standalone] ${message}; bundling without clipboard image support.`,
|
|
);
|
|
return;
|
|
}
|
|
|
|
const modulesDest = path.join(packageRoot, 'lib', 'node_modules');
|
|
const copyOpts = {
|
|
recursive: true,
|
|
dereference: true,
|
|
verbatimSymlinks: false,
|
|
};
|
|
for (let index = 0; index < packageNames.length; index += 1) {
|
|
fs.cpSync(
|
|
packageSources[index],
|
|
path.join(modulesDest, packageNames[index]),
|
|
copyOpts,
|
|
);
|
|
}
|
|
|
|
assertNoSymlinks(
|
|
modulesDest,
|
|
'Bundled clipboard addon still contains symlinks.',
|
|
);
|
|
}
|
|
|
|
function hasNativePrebuild(prebuildDir) {
|
|
return (
|
|
fs.existsSync(prebuildDir) &&
|
|
fs.readdirSync(prebuildDir).some((entry) => entry.endsWith('.node'))
|
|
);
|
|
}
|
|
|
|
function topLevelDistEntryForPath(candidatePath) {
|
|
const relative = path.relative(distDir, candidatePath);
|
|
if (
|
|
relative === '' ||
|
|
relative.startsWith('..') ||
|
|
path.isAbsolute(relative)
|
|
) {
|
|
return undefined;
|
|
}
|
|
|
|
return relative.split(path.sep)[0];
|
|
}
|
|
|
|
function isAllowedDistEntry(entry) {
|
|
return (
|
|
DIST_ALLOWED_ENTRIES.has(entry) ||
|
|
DIST_ALLOWED_ENTRY_PATTERNS.some((pattern) => pattern.test(entry))
|
|
);
|
|
}
|
|
|
|
function extractNodeArchive(nodeArchive, extractDir) {
|
|
if (nodeArchive.endsWith('.zip')) {
|
|
extractZipArchive(nodeArchive, extractDir);
|
|
return;
|
|
}
|
|
|
|
if (
|
|
nodeArchive.endsWith('.tar.gz') ||
|
|
nodeArchive.endsWith('.tgz') ||
|
|
nodeArchive.endsWith('.tar.xz')
|
|
) {
|
|
run('tar', ['-xf', nodeArchive, '-C', extractDir]);
|
|
return;
|
|
}
|
|
|
|
fail(
|
|
`Unsupported Node.js archive format: ${nodeArchive}. Expected .zip, .tar.gz, .tgz, or .tar.xz.`,
|
|
);
|
|
}
|
|
|
|
function extractZipArchive(nodeArchive, extractDir) {
|
|
if (process.platform === 'win32') {
|
|
run(
|
|
'powershell',
|
|
[
|
|
'-NoProfile',
|
|
'-ExecutionPolicy',
|
|
'Bypass',
|
|
'-Command',
|
|
'Expand-Archive -LiteralPath $env:QWEN_NODE_ARCHIVE -DestinationPath $env:QWEN_EXTRACT_DIR -Force',
|
|
],
|
|
{
|
|
env: {
|
|
...process.env,
|
|
QWEN_NODE_ARCHIVE: nodeArchive,
|
|
QWEN_EXTRACT_DIR: extractDir,
|
|
},
|
|
},
|
|
);
|
|
return;
|
|
}
|
|
|
|
run('unzip', ['-q', nodeArchive, '-d', extractDir]);
|
|
}
|
|
|
|
function copyExtractedNode(extractDir, nodeDir) {
|
|
const entries = fs
|
|
.readdirSync(extractDir)
|
|
.filter((entry) => entry !== '.DS_Store');
|
|
if (entries.length === 0) {
|
|
fail('Node.js archive did not contain any files.');
|
|
}
|
|
|
|
const sourceRoot =
|
|
entries.length === 1 &&
|
|
fs.statSync(path.join(extractDir, entries[0])).isDirectory()
|
|
? path.join(extractDir, entries[0])
|
|
: extractDir;
|
|
|
|
// Official Unix Node.js archives include internal npm/npx symlinks.
|
|
// The installer rejects symlinks in final archives, so keep safe internal
|
|
// targets by copying their referents during a single checked traversal.
|
|
copyNodeRuntimeEntry(sourceRoot, nodeDir, {
|
|
realRoot: fs.realpathSync(sourceRoot),
|
|
sourceRoot,
|
|
activeDirectories: new Set(),
|
|
});
|
|
}
|
|
|
|
function copyNodeRuntimeEntry(source, destination, state) {
|
|
const lstat = fs.lstatSync(source);
|
|
|
|
if (lstat.isSymbolicLink()) {
|
|
copyNodeRuntimeEntry(
|
|
resolveRuntimeSymlink(source, state),
|
|
destination,
|
|
state,
|
|
);
|
|
return;
|
|
}
|
|
|
|
if (lstat.isDirectory()) {
|
|
const realSource = fs.realpathSync(source);
|
|
if (state.activeDirectories.has(realSource)) {
|
|
fail(
|
|
`Node.js runtime contains a symlink cycle at ${displayRuntimePath(
|
|
state,
|
|
source,
|
|
)}`,
|
|
);
|
|
}
|
|
|
|
state.activeDirectories.add(realSource);
|
|
fs.mkdirSync(destination, { recursive: true });
|
|
fs.chmodSync(destination, lstat.mode);
|
|
for (const entry of fs.readdirSync(source)) {
|
|
copyNodeRuntimeEntry(
|
|
path.join(source, entry),
|
|
path.join(destination, entry),
|
|
state,
|
|
);
|
|
}
|
|
state.activeDirectories.delete(realSource);
|
|
return;
|
|
}
|
|
|
|
if (lstat.isFile()) {
|
|
fs.copyFileSync(source, destination);
|
|
fs.chmodSync(destination, lstat.mode);
|
|
return;
|
|
}
|
|
|
|
fail(`Unsupported Node.js runtime entry type: ${source}`);
|
|
}
|
|
|
|
function resolveRuntimeSymlink(source, state) {
|
|
const target = fs.readlinkSync(source);
|
|
const resolvedTarget = path.resolve(path.dirname(source), target);
|
|
let realTarget;
|
|
try {
|
|
realTarget = fs.realpathSync(resolvedTarget);
|
|
} catch (error) {
|
|
const errorCode =
|
|
error && typeof error === 'object' && 'code' in error
|
|
? error.code
|
|
: undefined;
|
|
const reason =
|
|
errorCode === 'ELOOP' ? 'a symlink cycle' : 'a missing target';
|
|
fail(
|
|
`Node.js runtime symlink points to ${reason}: ${displayRuntimePath(
|
|
state,
|
|
source,
|
|
)} -> ${target}`,
|
|
);
|
|
}
|
|
|
|
if (!isPathInside(state.realRoot, realTarget)) {
|
|
fail(
|
|
`Node.js runtime symlink escapes the archive: ${displayRuntimePath(
|
|
state,
|
|
source,
|
|
)} -> ${target}`,
|
|
);
|
|
}
|
|
|
|
return resolvedTarget;
|
|
}
|
|
|
|
function displayRuntimePath(state, source) {
|
|
return path.relative(state.sourceRoot, source) || '.';
|
|
}
|
|
|
|
function assertNoSymlinks(root, message) {
|
|
for (const entry of walkDirectory(root)) {
|
|
if (fs.lstatSync(entry).isSymbolicLink()) {
|
|
fail(`${message} First symlink: ${path.relative(root, entry)}`);
|
|
}
|
|
}
|
|
}
|
|
|
|
function* walkDirectory(root) {
|
|
for (const entry of fs.readdirSync(root)) {
|
|
const fullPath = path.join(root, entry);
|
|
yield fullPath;
|
|
if (fs.lstatSync(fullPath).isDirectory()) {
|
|
yield* walkDirectory(fullPath);
|
|
}
|
|
}
|
|
}
|
|
|
|
function isPathInside(root, candidate) {
|
|
const relative = path.relative(root, candidate);
|
|
return (
|
|
relative === '' ||
|
|
(!relative.startsWith('..') && !path.isAbsolute(relative))
|
|
);
|
|
}
|
|
|
|
function validateNodeRuntime(target, nodeDir) {
|
|
const targetConfig = TARGETS.get(target);
|
|
const executablePath = path.join(nodeDir, ...targetConfig.nodeExecutable);
|
|
const displayPath = targetConfig.nodeExecutable.join('/');
|
|
|
|
if (!fs.existsSync(executablePath)) {
|
|
fail(`Node.js runtime for ${target} must contain ${displayPath}.`);
|
|
}
|
|
|
|
if (target !== 'win-x64') {
|
|
const mode = fs.statSync(executablePath).mode;
|
|
if ((mode & 0o111) === 0) {
|
|
fail(
|
|
`Node.js runtime for ${target} must provide executable ${displayPath}.`,
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
function writeShims(packageRoot) {
|
|
const binDir = path.join(packageRoot, 'bin');
|
|
fs.mkdirSync(binDir, { recursive: true });
|
|
|
|
const unixShim = `#!/usr/bin/env sh
|
|
set -e
|
|
ROOT="$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)"
|
|
QWEN_CODE_LAUNCHER_PATH="$ROOT/bin/qwen" exec "$ROOT/node/bin/node" "$ROOT/lib/cli-entry.js" "$@"
|
|
`;
|
|
const unixShimPath = path.join(binDir, 'qwen');
|
|
fs.writeFileSync(unixShimPath, unixShim);
|
|
fs.chmodSync(unixShimPath, 0o755);
|
|
|
|
const windowsShim = `@echo off
|
|
setlocal
|
|
set "ROOT=%~dp0.."
|
|
set "QWEN_CODE_LAUNCHER_PATH=%ROOT%\\bin\\qwen.cmd"
|
|
"%ROOT%\\node\\node.exe" "%ROOT%\\lib\\cli-entry.js" %*
|
|
exit /b %ERRORLEVEL%
|
|
`;
|
|
fs.writeFileSync(path.join(binDir, 'qwen.cmd'), windowsShim);
|
|
}
|
|
|
|
function writeManifest(packageRoot, manifest) {
|
|
const manifestPath = path.join(packageRoot, 'manifest.json');
|
|
fs.writeFileSync(
|
|
manifestPath,
|
|
JSON.stringify(
|
|
{
|
|
name: '@qwen-code/qwen-code',
|
|
version: manifest.version,
|
|
target: manifest.target,
|
|
nodeArchive: manifest.nodeArchive,
|
|
createdAt: new Date().toISOString(),
|
|
},
|
|
null,
|
|
2,
|
|
) + '\n',
|
|
);
|
|
}
|
|
|
|
function createArchive(outputExtension, outputPath, cwd) {
|
|
if (outputExtension === 'zip') {
|
|
createZipArchive(outputPath, cwd);
|
|
return;
|
|
}
|
|
|
|
run('tar', ['-czf', outputPath, '-C', cwd, 'qwen-code']);
|
|
}
|
|
|
|
function createZipArchive(outputPath, cwd) {
|
|
if (process.platform === 'win32') {
|
|
run(
|
|
'powershell',
|
|
[
|
|
'-NoProfile',
|
|
'-ExecutionPolicy',
|
|
'Bypass',
|
|
'-Command',
|
|
'Compress-Archive -LiteralPath $env:QWEN_PACKAGE_ROOT -DestinationPath $env:QWEN_OUTPUT_PATH -Force',
|
|
],
|
|
{
|
|
env: {
|
|
...process.env,
|
|
QWEN_PACKAGE_ROOT: path.join(cwd, 'qwen-code'),
|
|
QWEN_OUTPUT_PATH: outputPath,
|
|
},
|
|
},
|
|
);
|
|
return;
|
|
}
|
|
|
|
run('zip', ['-qr', outputPath, 'qwen-code'], { cwd });
|
|
}
|
|
|
|
async function writeSha256Sums(outDir) {
|
|
const entries = fs
|
|
.readdirSync(outDir)
|
|
.filter(
|
|
(entry) =>
|
|
entry.startsWith('qwen-code-') &&
|
|
(entry.endsWith('.tar.gz') || entry.endsWith('.zip')),
|
|
)
|
|
.sort();
|
|
|
|
if (entries.length === 0) {
|
|
fail(
|
|
`No qwen-code archives found in ${outDir}; refusing to write empty SHA256SUMS.`,
|
|
);
|
|
}
|
|
|
|
const lines = [];
|
|
for (const entry of entries) {
|
|
const filePath = path.join(outDir, entry);
|
|
const hash = await sha256File(filePath);
|
|
lines.push(`${hash} ${entry}`);
|
|
}
|
|
|
|
fs.writeFileSync(path.join(outDir, 'SHA256SUMS'), `${lines.join('\n')}\n`);
|
|
}
|
|
|
|
async function sha256File(filePath) {
|
|
const hash = crypto.createHash('sha256');
|
|
await pipeline(fs.createReadStream(filePath), hash);
|
|
return hash.digest('hex');
|
|
}
|
|
|
|
function run(command, args, options = {}) {
|
|
try {
|
|
execFileSync(command, args, {
|
|
stdio: 'inherit',
|
|
...options,
|
|
});
|
|
} catch (error) {
|
|
const detail =
|
|
error && typeof error === 'object' && 'message' in error
|
|
? `: ${error.message}`
|
|
: '';
|
|
fail(`Command failed: ${command} ${args.join(' ')}${detail}`);
|
|
}
|
|
}
|
|
|
|
function fail(message) {
|
|
throw new Error(`Error: ${message}`);
|
|
}
|
|
|
|
export {
|
|
TARGET_CLIPBOARD_PACKAGE,
|
|
TARGETS,
|
|
writeSha256Sums,
|
|
// Exported so a test can hold the allowlist and the build's stamp together:
|
|
// the packager aborts on any dist entry it does not know, and nothing else
|
|
// would notice a one-sided rename until a release was cut.
|
|
isAllowedDistEntry,
|
|
};
|