qwen-code/scripts/copy_bundle_assets.js
Shaojin Wen 715ef938f5
feat(cli): serve the Web Shell UI from qwen serve (#5392)
* feat(cli): serve the Web Shell UI from `qwen serve`

`qwen serve` now serves the built Web Shell SPA at its root on the same
origin as the API, so a released binary exposes the browser terminal
without the dev-only Vite server (the `npm run dev:daemon` two-process
setup is unchanged for development).

- New `webShellStatic.ts` mounts `/`, `/assets/*` and an SPA deep-link
  fallback. The fallback uses the same document-navigation discriminator
  as the Vite dev proxy so it never shadows API JSON 404s.
- The static shell is registered BEFORE bearerAuth (a browser can't attach
  a token to a `<script>` subresource or an address-bar navigation; the
  shell carries no secrets and every API route stays token-gated). HTML
  responses set CSP + X-Frame-Options + Referrer-Policy + no-cache.
- `--open` launches the browser at the daemon URL (with `?token=` when set)
  once the listener is up, guarded by `shouldLaunchBrowser()`.
- `--no-web` opts out for an API-only daemon.
- Bundle / npm publish / standalone packaging now ship `dist/web-shell/`.
  Missing assets degrade to API-only with a breadcrumb, never a hard fail.

Tests: +6 cases in server.test.ts (root shell, assets, SPA fallback,
non-navigation 404 passthrough, security headers, --no-web off).

* fix(cli): address review on Web Shell serving

Review fixes for #5392 (qwen-code-ci-bot):

- [Critical] SPA fallback no longer shadows /health or /demo on non-loopback
  binds — those paths fall through to their own routes / bearerAuth instead
  of receiving index.html.
- [Critical] --open trims the bearer token before putting it in the browser
  URL, matching runQwenServe's own trimming, so a trailing newline from
  `$(cat token.txt)` no longer makes every API call 401.
- --open is wrapped in its own try/catch so a failed browser launch can't
  take down the already-listening daemon; it normalizes wildcard binds
  (0.0.0.0 / ::) to loopback, and only fires when the UI is actually mounted
  (new RunHandle.webShellMounted).
- resolveWebShellDir() now requires BOTH index.html and assets/, so a partial
  build degrades to API-only instead of serving a shell whose chunks 404.
- runQwenServe logs a positive "Web Shell UI served from <dir>" breadcrumb,
  and warns that on a non-loopback bind without --allow-origin the shell is
  read-only (same-origin POSTs are blocked by the CORS wall).
- Document the --open token-in-process-list exposure in help text + a stderr
  note when a token is forwarded.
- Tests: POST method guard, sec-fetch navigation signal, /health not shadowed,
  sendFile 500 path, plus isDocumentNavigation and resolveWebShellDir units.

* fix(cli): harden Web Shell asset resolution and send-error logging

Second-round review (claude /qreview on the initial commit):

- resolveWebShellDir() now walks up from this module to find a sibling
  packages/web-shell/dist, covering the transpiled layouts the previous
  fixed `..` depth missed — per-package `tsc` output and the integration
  daemon harness (packages/cli/dist/index.js), which would otherwise resolve
  to nonexistent paths and silently run API-only.
- sendFile failures are no longer silent: log the error (matching the /demo
  handler — previously the only 5xx path that emitted nothing) and res.end()
  a half-streamed response instead of leaving the client on a 200 with a
  partial body.

The remaining comment (open-browser inside the boot try) was already fixed
in 2487c90, where the --open block gained its own try/catch.

* fix(cli): pass --open token via URL fragment + add auth-contract tests

Third-round review (qwen3.7-max /review):

- --open now puts the token in the URL fragment (#token=) instead of a query
  param, and the Web Shell reads it from the fragment first (falling back to
  ?token= for the dev launcher / hand-built URLs). A fragment is never sent to
  the server, so the token stays out of access logs and Referer headers. It is
  still visible in the browser-launcher's argv, so the stderr note stays and a
  one-time-code exchange remains the real fix for multi-user hosts (follow-up).
- Add a server test pinning the "shell served before bearerAuth, API still
  token-gated" contract (GET / → 200 without auth, /capabilities → 401 with a
  token set), plus front-end getDaemonToken fragment/query precedence tests.

The token-trim comment in this pass was already addressed in 2487c90.

* fix(cli): read --open token from RunHandle.resolvedToken; doc + test polish

Fourth-round review (qwen3.7-max /review), all suggestions:

- --open now reads the server's resolved (trimmed) token from
  RunHandle.resolvedToken instead of re-deriving it from argv/env. Removes the
  duplicated QWEN_SERVER_TOKEN literal + trim logic and any drift risk; the
  browser token is by construction what the daemon authenticates against.
- Simplify webShellMounted to !!webShellDir (serveWebShell===false already
  forces webShellDir to undefined, so the extra conjunct was dead).
- Docs: the --open row now documents the #token= fragment transport (was
  ?token=) and why a fragment is used.
- Tests: add removeDaemonTokenFromUrl coverage (strip from fragment / query /
  both, preserve non-token hash params, no-op when absent) and the missing
  afterEach import.

* fix(cli): register Web Shell SPA fallback after API routes

Fifth-round review (claude /qreview):

- The SPA fallback no longer sits before bearerAuth. It now runs after every
  API route (just before the error handler), so authed routes — and their
  401s — always win, and only genuine 404 misses fall through to the shell.
  A navigation with an attacker-controlled `Accept: text/html` to
  /capabilities (or /health on a non-loopback bind) no longer coaxes the 200
  shell out of a gated endpoint, and the fragile exact-match /health,/demo
  denylist (which trailing-slash variants slipped past) is gone.
  registerWebShell is split into mountWebShellAssets (/, /assets — still
  pre-auth so a browser can load the shell + subresources without a header)
  and mountWebShellSpaFallback (post-auth). The contract test now sends
  Accept: text/html to /capabilities and asserts 401 — it would have been 200
  before this change (the test was passing only because it omitted Accept).
- verifyBundleArtifacts (the publish gate) now requires dist/web-shell, so a
  build that skipped the web-shell workspace (e.g. npm ci --ignore-scripts
  bypassing the root prepare) fails packaging loudly instead of silently
  shipping an API-only CLI whose GET / 404s.

* fix(cli): return a clean 404 for missing Web Shell assets

Sixth-round review (qwen3.7-max /review):

A missing /assets/* (e.g. a stale hashed chunk after a redeploy renamed it)
now returns 404 instead of falling through to the SPA fallback and answering a
browser navigation with a 200 index.html. Implemented with an explicit /assets
404 handler after express.static rather than serve-static's `fallthrough:
false` — the latter forwards a 404 error to the catch-all error handler, which
would turn it into a 500. Test added.

* test(cli): cover --open + Web Shell signals; add shell security headers

Seventh-round review (qwen-code-ci-bot):

- [Critical] Extract the --open browser-launch logic into the exported
  maybeOpenWebShellBrowser() and unit-test it: --open / webShellMounted /
  shouldLaunchBrowser gating, wildcard-host -> loopback rewrite, token in the
  URL fragment (not query), and the never-throws error catch.
- [Critical] Assert RunHandle.webShellMounted (false under --no-web) and
  resolvedToken (trimmed / undefined) in runQwenServe.test.ts; also cover
  --web/--no-web and --open arg parsing.
- Drop dead code: target.hostname === '::' is unreachable (Node's URL returns
  the IPv6 wildcard as '[::]', which is already handled).
- Add defense-in-depth headers to the shell response: base-uri 'none' in the
  CSP (does not fall back to default-src), X-Content-Type-Options: nosniff,
  and a restrictive Permissions-Policy.
- Add serve-debug-gated logging for /assets 404s and SPA-fallback hits so a
  white-screen shell / routing misconfig has a diagnostic trail.

* fix(test): satisfy the Web Shell release gate in package-assets fixture

Eighth-round review (claude /qreview) — this is the actual CI failure.

The verifyBundleArtifacts Web Shell gate (requiring dist/web-shell, added in
this PR) broke scripts/tests/package-assets.test.js, which merge-main pulled
in: its createBundleArtifacts fixture only created cli.js / vendor / bundled,
so preparePackage exited 1 at the new gate before the test's assertions ran —
red on all three Test jobs. Add the web-shell artifacts (index.html +
assets/) to the fixture. The gate itself is intentional (it stops an API-only
package from shipping).
2026-06-19 19:41:33 +08:00

193 lines
6.7 KiB
JavaScript

/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { copyFileSync, existsSync, mkdirSync, statSync } from 'node:fs';
import { dirname, join, basename, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { glob } from 'glob';
import fs from 'node:fs';
const __dirname = dirname(fileURLToPath(import.meta.url));
const defaultRoot = join(__dirname, '..');
export function copyBundleAssets({ root = defaultRoot } = {}) {
const distDir = join(root, 'dist');
const coreVendorDir = join(root, 'packages', 'core', 'vendor');
// Create the dist directory if it doesn't exist
if (!existsSync(distDir)) {
mkdirSync(distDir);
}
// Find and copy all .sb files from packages to the root of the dist directory
const sbFiles = glob.sync('packages/**/*.sb', { cwd: root });
for (const file of sbFiles) {
copyFileSync(join(root, file), join(distDir, basename(file)));
}
console.log('Copied sandbox profiles to dist/');
// Copy vendor directory (contains ripgrep binaries)
console.log('Copying vendor directory...');
if (existsSync(coreVendorDir)) {
const destVendorDir = join(distDir, 'vendor');
copyRecursiveSync(coreVendorDir, destVendorDir);
console.log('Copied vendor directory to dist/');
} else {
console.warn(`Warning: Vendor directory not found at ${coreVendorDir}`);
}
// Copy bundled skills (e.g. /review) so they are available at runtime.
// In the esbuild bundle, import.meta.url resolves to dist/cli.js, so
// SkillManager looks for bundled skills at dist/bundled/.
const bundledSkillsDir = join(
root,
'packages',
'core',
'src',
'skills',
'bundled',
);
if (existsSync(bundledSkillsDir)) {
const destBundledDir = join(distDir, 'bundled');
copyRecursiveSync(bundledSkillsDir, destBundledDir);
console.log('Copied bundled skills to dist/bundled/');
} else {
console.warn(
`Warning: Bundled skills directory not found at ${bundledSkillsDir}`,
);
}
// Copy user docs into qc-helper bundled skill so it can reference them at runtime.
// The qc-helper skill reads docs from a `docs/` subdirectory relative to its own
// directory. In the esbuild bundle this becomes dist/bundled/qc-helper/docs/.
const userDocsDir = join(root, 'docs', 'users');
if (existsSync(userDocsDir)) {
const destDocsDir = join(distDir, 'bundled', 'qc-helper', 'docs');
copyRecursiveSync(userDocsDir, destDocsDir);
console.log('Copied docs/users/ to dist/bundled/qc-helper/docs/');
} else {
console.warn(`Warning: User docs directory not found at ${userDocsDir}`);
}
// Copy builtin locales so bundled dist/cli.js can load UI translations at runtime.
// Published packages already include these via prepare-package.js; bundle output
// should mirror that behavior for local `node dist/cli.js` runs.
const localesDir = join(root, 'packages', 'cli', 'src', 'i18n', 'locales');
if (existsSync(localesDir)) {
const destLocalesDir = join(distDir, 'locales');
copyRecursiveSync(localesDir, destLocalesDir);
console.log('Copied builtin locales to dist/locales/');
} else {
console.warn(`Warning: Locales directory not found at ${localesDir}`);
}
// Copy extension templates so bundled dist/cli.js can scaffold
// `/extensions new` from the runtime examples directory.
const extensionExamplesDir = join(
root,
'packages',
'cli',
'src',
'commands',
'extensions',
'examples',
);
if (existsSync(extensionExamplesDir)) {
const destExtensionExamplesDir = join(distDir, 'examples');
copyRecursiveSync(extensionExamplesDir, destExtensionExamplesDir);
console.log('Copied extension examples to dist/examples/');
} else {
console.warn(
`Warning: Extension examples directory not found at ${extensionExamplesDir}`,
);
}
// Copy the built Web Shell SPA (index.html + assets/) so the bundled
// `qwen serve` can serve the browser UI at its root path. The library
// build outputs (dist/index.js, dist/types) are for npm consumers and are
// intentionally NOT copied. Source only exists after the web-shell
// workspace is built (npm run build); when absent (e.g. a --cli-only
// build, or bundling without a prior full build) we warn and skip so the
// bundle step never fails — the daemon then runs API-only at runtime.
const webShellDistDir = join(root, 'packages', 'web-shell', 'dist');
const webShellIndexHtml = join(webShellDistDir, 'index.html');
const webShellAssetsDir = join(webShellDistDir, 'assets');
if (existsSync(webShellIndexHtml) && existsSync(webShellAssetsDir)) {
const destWebShellDir = join(distDir, 'web-shell');
mkdirSync(destWebShellDir, { recursive: true });
copyFileSync(webShellIndexHtml, join(destWebShellDir, 'index.html'));
copyRecursiveSync(webShellAssetsDir, join(destWebShellDir, 'assets'));
console.log('Copied Web Shell UI to dist/web-shell/');
} else {
console.warn(
`Warning: Web Shell assets not found at ${webShellDistDir}; ` +
'dist/web-shell/ will be absent and `qwen serve` runs API-only. ' +
'Run a full `npm run build` before bundling to include the UI.',
);
}
console.log('\n✅ All bundle assets copied to dist/');
}
if (isDirectRun()) {
copyBundleAssets();
}
function isDirectRun() {
return process.argv[1]
? fileURLToPath(import.meta.url) === resolve(process.argv[1])
: false;
}
/**
* Recursively copy directory
*/
function copyRecursiveSync(src, dest) {
if (!existsSync(src)) {
return;
}
const stats = statSync(src);
if (stats.isDirectory()) {
if (!existsSync(dest)) {
mkdirSync(dest, { recursive: true });
}
const entries = fs.readdirSync(src);
for (const entry of entries) {
// Skip .DS_Store files
if (entry === '.DS_Store') {
continue;
}
const srcPath = join(src, entry);
const destPath = join(dest, entry);
copyRecursiveSync(srcPath, destPath);
}
} else {
copyFileSync(src, dest);
// Preserve execute permissions for binaries
const srcStats = statSync(src);
if (srcStats.mode & 0o111) {
fs.chmodSync(dest, srcStats.mode);
}
}
}