qwen-code/scripts/tests
Shaojin Wen b286875e72
fix(review): harden the pipeline against four live-run failures (#9086)
* fix(review): harden the pipeline against four live-run failures

Measured on three parallel PR reviews (qwen3.8-max, 2026-08-13, PRs
#9013/#9014/#9045) run via `qwen review run`:

- run.ts: pin the composed-verdict and report scans to the run's own
  target. The generic newest-composed scan captured a concurrent run's
  artifact — two of the three runs republished a neighbour PR's verdict
  (one reported REQUEST_CHANGES for a review whose own report said
  Comment). Also keep re-reading while the child runs: a coverage
  re-check legitimately recomposed a verdict 12 minutes after the first
  write, and the first-snapshot capture would republish the superseded
  one.

- budget.ts: drop placeholder gaps whose completion word carries a
  trailing budget adverbial. Three "none — all checks … completed
  within budget" non-answers reached two posted bodies because the
  completion idiom required the completion word to end the text.

- coverage.ts: label a non-chunk agent by the brief codename found
  anywhere in its launch prompt. Launchers prepend context lines, so the
  first-line label gave twelve finders one shared PR-summary sentence,
  and every budget-gap disclosure rendered as the same truncated PR
  quote instead of a name.

- copy_bundle_assets.js: emit dist/cli.js with a shebang and the execute
  bit. shellContextEnv blanks a QWEN_CODE_CLI a POSIX shell cannot exec,
  so every review subcommand issued from a session launched off the
  bundle silently fell back to the PATH's global install — all three
  runs executed the machine's auto-updated release instead of the tree
  they were launched from.

* test(review): follow the codename label into compose-review's fixtures

The backtick-collapse fixture's first line was itself the brief codename
shape, so the new codename extraction labels it `agent security` and the
first-line assertion no longer holds. Keep the sanitization intent on a
prose-only first line, and pin the codename behaviour — a prepended
context line must not reintroduce the shared-PR-quote label — as its own
case.

* fix(review): classify the run target with the child's own parser

Review feedback on the target pin: prNumberFromTarget re-derived PR
classification with a narrower regex than parse-args — /pull/<n>/files
URLs went unpinned, 0042 pinned pr-0042- while the child writes pr-42-,
and docs/pull/42 pinned a file target as a PR — so a completed (and
posted) review could be reported as one that produced no verdict.
Delegate to parseReviewArgs, whose verdict is what the child names its
artifacts from, and pin the divergent shapes as tests.

Also gate the bundle's shebang/exec-bit block with a package-assets
case (mode asserted off-win32, double-run must not stack shebangs), and
document the accepted same-target residual race on composedPatternFor.

* fix(review): pin run artifacts by exact target identity, share the identity-line parser

Round-2 review feedback, all six findings:

- run.ts: replace name-shape pins with the exact composed filename each
  target class produces (pr-<n> / file basename / the fixed 'local' token,
  per the skill's --out template). The (?!pr-\d+-) lookahead rejected a
  file run's own artifact whenever the reviewed file was named
  pr-<digits>-…, the PR branch's .* wildcard claimed that same artifact,
  and the pooled null class let concurrent file and no-target runs
  cross-capture each other's verdicts. Target classification now comes
  from classifyRunTarget (parse-args' verdict, basename for files).

- run.ts: newestArtifactSince returns {path, mtime}, so the capture poll
  reuses the scan's own stat instead of re-statting the path — the
  scan-vs-sweep window (and its untestable catch branch) is gone
  structurally.

- lib/agent-identity.ts: one parser for the identity line agent-prompt
  bakes into every launch, shared by cost-ledger's row labels and
  coverage's disclosure labels — the two hand-rolled copies could drift,
  and coverage's copy dropped the (round N) and owned-file suffixes,
  folding reverse-audit rounds into indistinguishable disclosure lines.
  cost-ledger still feeds it only the first line (quoted identity lines
  below must never be credited); coverage scans for the first
  line-anchored identity line (launchers prepend context lines).

- lib/budget.ts: one vocabulary for the budget-idiom family — 'below'
  joins the completion tail's position words, and the stayed idiom takes
  the same qualifiers ('stayed inside the tool-call budget').

- run.test.ts: handler-level assertion that the report scan is pinned
  (a strictly newer neighbour report must not become this run's
  reportPath), alongside the pattern-level cases for every shape the
  review probed.

* fix(review): round-3 review polish — CRLF identity lines, producer-side no-gap rule, edge-case pins

- agent-identity: tolerate a trailing CR (CRLF-recorded prompts fed
  through \n-splitting callers failed every parse and fell back to
  first-line prose); scan the launch prompt with one multiline match
  instead of materializing a line array per agent record; pin the
  round-over-file precedence with a both-suffixes test.

- agent-prompt: state the no-gap rule at the producer — write NO
  'Budget gap:' line when nothing was cut short — instead of leaving
  each agent to improvise a 'none' phrasing the consumer-side
  placeholder filter must chase forever.

- run.ts: strip trailing path separators before taking a file target's
  basename (a tab-completed 'src/' pinned 'qwen-review--composed.json',
  which no child artifact carries — fail-closed exit 1 on a completed
  review); pin file-run reports by their filename slot so a file named
  'pr-1234.md' claims its own report; document the two collision
  classes the basename-keyed pin defines (same-basename files,
  basenames spelling 'local'/'pr-<n>').

- budget.test: pin the parenthesis-form exception keep case beside the
  dash form.

* fix(review): round-4 polish — one budget vocabulary, named pin expectation, honest prompt claim

- budget.ts: spell the budget-position vocabulary once (BUDGET_QUALIFIED /
  COMPLETION_TAIL) and build PLACEHOLDER_GAP_RE from it — the literal
  carried three hand-copies that had already drifted twice in two review
  rounds; the space-separated 'tool call' form is pinned in both branches.

- agent-prompt.ts: the no-gap rule now states what actually happens to a
  'none' disclosure — at best filtered, and any unrecognized wording is
  published as a phantom coverage gap — instead of claiming the parser
  treats it as a gap, which was the negation of the filter shipped beside
  it.

- run.ts: derive the composed pin from composedNameFor and name the
  expected filename in both the no-verdict prose and the JSON result
  (expectedComposedName) — a naming drift between the pin and the skill's
  template was undiagnosable once Step 9 swept the near-miss.

- compose-review.ts: publicAgentSubject's provenance note now describes
  the codename labels coverage prefers, with first-line prose as the
  fallback.

* test(review): guard the no-verdict diagnostic and the pinned capture; align the chunk-role grammar

- run.test.ts: assert the no-verdict report names the artifact it waited
  for, in prose and as expectedComposedName — mutation-verified: dropping
  the suffix now fails.

- run.test.ts: force the neighbour's composed artifact strictly NEWER in
  the concurrent-run fixture. With it older, an unpinned newest-composed
  scan landed on the right file anyway and the regression passed;
  mutation-verified: reverting composedPatternFor to the generic scan now
  fails the handler test, not only the pattern units.

- agent-identity.ts: CHUNK_ROLE_RE takes coverage's CHUNK_RE shape
  (whitespace-tolerant, case-insensitive) so a hand-edited 'Chunk 3 of 7'
  cannot resolve as a chunk owner in the posted body and a role agent in
  the ledger row.

* fix(review): keep the bundle's write time across the shebang rewrite; hold the pins to the skill

Round-6 review feedback, all five findings:

- copy_bundle_assets.js: preserve dist/cli.js's atime/mtime across the
  shebang rewrite. stampReviewSourceDigest reads that mtime as the build
  time, so a bumped one certifies a bundle as newer than review sources
  edited before it and the staleness warning the skill's Step 0 stops on
  never fires. A full bundle stamps before reaching here, but a
  standalone run of this script — a flow the gate's own comment
  contemplates — was exposed.

- package-assets.test.js: pin both halves the block owes. The mtime is
  asserted against a fixture built 60s in the past, and the second run
  now arrives at mode 0644 so the exec bit must be re-set — demoting the
  chmod inside the shebang guard previously stayed green.

- run-skill-parity.test.ts: new. composedNameFor and reportPatternFor
  encode the bundled skill's Step 6 --out template and Step 8 report
  stems, and were pinned only against self-referential literals. This
  reads the templates out of SKILL.md and renders them per target class,
  so a skill-side edit fails next to the code that must follow it
  instead of silently in a later review.

- cost-ledger.test.ts: pin the first-line-only invariant — a launch
  whose prepended context sits above the identity line keeps the
  transcript's own id, never a label lifted from below. Consolidating
  both callers on labelFromLaunchPrompt now fails.

- agent-identity.test.ts: assert the two entry points genuinely differ
  on that prompt, so neither caller's policy can be collapsed into the
  other unnoticed.

* fix(review): stop the mtime assertion from pinning libuv's timespec truncation

The assertion compared the recorded mtime against the Date handed to
utimesSync, so it also pinned libuv's double-seconds → timespec
conversion: about half of all millisecond values read back 1 ns low
(X - 0.001), and builtAt is a fresh Date.now() - 60_000 every run — a
~50% coin flip that would have landed intermittent reds on unrelated
PRs through test:ci.

Capture what the filesystem actually recorded after the setup and
compare against that; the invariant under test is only whether the
shebang rewrite moves the stored time. 10/10 green through the CI entry
point, and it keeps its teeth: removing the production
fs.utimesSync(cliEntry, atime, mtime) restore fails it 3/3.
2026-08-14 04:38:49 +00:00
..
ai-release-notes-workflow.test.js fix(release): keep notes anchored and cap the release body (#8199) 2026-07-31 09:55:38 +00:00
audit-runtime-critical.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
build-and-publish-image-workflow.test.js
capture-tmux-ci.test.js ci: install tmux and zip tooling on the Linux test lane, and pin it (#8792) 2026-08-11 05:47:39 +00:00
check-build-status.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
check-i18n.test.ts fix(cli): localize approval mode UI labels (#6592) 2026-07-11 00:07:03 +00:00
check-voice-guard-sync.test.js feat(voice): support trusted private ASR base URLs (#8350) 2026-08-06 14:04:57 +00:00
chrome-extension-package.test.js
ci-flaky-rerun-workflow.test.js fix(ci): stop a slow patrol classifier from killing every flaky rerun (#7358) 2026-07-21 02:33:51 +00:00
ci-flaky-rerun.test.js feat(ci): auto-open a deflake fix issue for confirmed flaky tests (#7231) 2026-07-19 16:49:29 +00:00
clean-package-build-artifacts.test.js test(core): stabilize file history eviction test (#6637) 2026-07-10 06:39:52 +00:00
cli-entry.test.js fix(cli): preserve Qwen Review startup version in footers (#8431) 2026-08-04 14:58:56 +00:00
comment-attachment-guard-workflow.test.js ci: route trusted-author fork PRs and no-checkout jobs to the ECS pool (#8502) 2026-08-04 03:48:24 +00:00
desktop-oss-workflow.test.js fix(desktop): harden release pipeline (#9009) 2026-08-12 16:38:12 +00:00
dev.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
e2e-workflow.test.js fix(ci): keep the post-merge E2E signal on main alive (#7795) 2026-07-28 11:54:52 +00:00
generate-changelog.test.js feat(release): generate AI-assisted release notes (#6756) 2026-07-12 13:00:22 +00:00
generate-release-notes.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
get-release-version-python-sdk.test.js
get-release-version.test.js fix(release): bump preview base past published stable (#7978) 2026-07-29 23:21:00 +00:00
install-script.test.js fix(install): avoid Get-FileHash for Windows checksums (#9112) 2026-08-14 01:12:08 +00:00
integration-vitest-config.test.ts fix(tests): apply integration worker limits to forks (#8689) 2026-08-08 00:53:33 +00:00
issue-triage-ownership-workflow.test.js ci: remove broken legacy scheduled PR triage workflow (#8434) 2026-08-03 10:20:42 +00:00
lint.test.js fix(ci): cache downloaded linters on ECS runners (#9001) 2026-08-13 05:13:23 +00:00
live-host-oss-workflow.test.js fix(ci): restore Live Host release mirroring (#8917) 2026-08-11 07:08:26 +00:00
main-ci-failure-issue-workflow.test.js fix(ci): keep the post-merge E2E signal on main alive (#7795) 2026-07-28 11:54:52 +00:00
no-ak-integration-ci.test.js fix(ci): reduce ENOSPC and load-sensitive test flakes (#8982) 2026-08-13 16:44:39 +00:00
package-assets.test.js fix(review): harden the pipeline against four live-run failures (#9086) 2026-08-14 04:38:49 +00:00
package-scripts.test.js fix(tests): apply integration worker limits to forks (#8689) 2026-08-08 00:53:33 +00:00
pr-force-push-reminder-workflow.test.js
pr-self-report-label.test.js fix(ci): route workflow label mutations through REST (#8761) 2026-08-09 15:05:15 +00:00
qwen-autofix-fork-bridge-workflow.test.js feat(autofix): bridge fork-PR reviews into the credentialed review lane (#8676) 2026-08-07 16:11:48 +00:00
qwen-autofix-workflow.test.js fix(autofix): hold autofix rounds while review-pr is in flight (#8899) 2026-08-14 02:13:02 +00:00
qwen-fleet-shepherd-workflow.test.js feat(autofix): bridge fork-PR reviews into the credentialed review lane (#8676) 2026-08-07 16:11:48 +00:00
qwen-pr-review-workflow.test.js perf(ci): tighten the automatic review kill switch for micro diffs (#8774) 2026-08-10 04:01:36 +00:00
qwen-repo-hygiene-workflow.test.js fix(ci): route workflow label mutations through REST (#8761) 2026-08-09 15:05:15 +00:00
qwen-resolve-workflow.test.js fix(ci): keep the review workflow under the expression-length limit (#8720) 2026-08-08 05:53:35 +00:00
qwen-triage-finalize-workflow.test.js fix(ci): rename triage status marker to avoid duplicate-guard collision (#7723) 2026-07-26 15:51:22 +00:00
qwen-triage-workflow.test.js perf(ci): make the triage budget operator-tunable and raise it (#8810) 2026-08-10 12:40:21 +00:00
release-helpers.test.js
release-sdk-workflow.test.js fix(ci): skip empty SDK release PR (#6861) 2026-07-14 13:19:42 +00:00
release-workflow.test.js feat(review): say so when the bundle is older than the review it runs (#8390) 2026-08-07 03:21:26 +00:00
review-source-digest.test.ts feat(review): say so when the bundle is older than the review it runs (#8390) 2026-08-07 03:21:26 +00:00
review-worktree-cleanup-workflow.test.js fix(ci): clean review worktrees after cancellation (#8474) 2026-08-05 02:39:36 +00:00
sandbox-command.test.js
sdk-java-workflow.test.js ci: route trusted-author fork PRs and no-checkout jobs to the ECS pool (#8502) 2026-08-04 03:48:24 +00:00
sdk-node-exporter-stub.test.js perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol (#7276) 2026-07-21 07:35:30 +00:00
security-workflows.test.js chore(ci): Add security hygiene: CODEOWNERS for release workflows, least-privilege permissions, security checks and Scorecard (#9008) 2026-08-14 01:22:53 +00:00
serve-fast-path-bundle-check.test.js feat(ci): fail the startup bundle check when the CLI entry is hoisted into a chunk (#8203) 2026-07-31 08:57:57 +00:00
start.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
test-setup.ts
update-ecs-runner-qwen-workflow.test.js fix(ci): reconcile ECS runner updater on workflow changes (#8373) 2026-08-02 09:25:17 +00:00
upload-aliyun-oss-assets.test.js
verify-capture.test.js fix(ci): avoid verify capture color conflict (#8236) 2026-07-31 14:15:40 +00:00
vitest.config.ts ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
workflow-helpers.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
workspaces.test.js