qwen-code/scripts/tests
Shaojin Wen a123d0030a
ci(review): prepare evidence-image tooling for GitHub-triggered reviews (#8454)
* ci(review): prepare evidence-image tooling for GitHub-triggered reviews

Reviews triggered on GitHub cannot attach images today for three
reasons; this wires the two that live in the workflow:

- Install tmux and freeze (pinned, checksum-verified) before the
  review runs. Both are optional by contract — the evidence ladder
  degrades honestly without them (png -> ans-only -> refused, recorded
  in the capture manifest) — so the step never fails the review; it
  only decides which rung the runner can reach. tmux mirrors the
  tolerant install qwen-autofix.yml already uses; freeze falls back to
  ~/.local/bin when passwordless sudo is absent.

- Pass QWEN_REVIEW_ASSETS_REPO from a repository variable to the
  review step. Publishing stays OPT-IN by design: with the variable
  unset the env is empty and publish-assets refuses (parseAssetsRepo
  trims and rejects empty), so nothing changes until a maintainer sets
  the variable. When set, evidence images land on commit-pinned
  pr-assets/<pr>-review branches — already covered by the visuals
  cleanup workflow — pushed with the same CI_BOT_PAT the step uses.

The third reason is release lag: the capture producer (capture-tui,
#8388) has to merge and ship in a release before rendering claims can
generate images on CI at all. This change is inert until then.

* fix(ci): capture-tools step review fixes — enforced tolerance, version pin, cached fallback

R1-1: the never-fails contract is now enforced twice — continue-on-error
at the YAML level (the belt) and set +e with a trailing exit 0 inside
(the suspenders); under the runner's default bash -e several statements
(mktemp, install, sudo install with an empty path) could previously
abort the step and fail the review the comment promised never to fail.

R1-6: probe the VERSION, not just the binary — on a persistent
self-hosted runner an installed freeze made any FREEZE_VERSION/SHA
bump a silent no-op; the pin now forces a refresh when the cached
binary does not match.

Cached-fallback fix: put ~/.local/bin on PATH (and GITHUB_PATH) before
the probe — a sudo-less runner otherwise re-downloads the tarball on
every review run forever.

R1-3: the step comment says capture-tui is UPCOMING (#8388, not in the
released CLI) and names qwen review drive as today's tmux consumer, so
the step cannot be mistaken for stale dead weight and deleted from
under the follow-up.

R1-4: the retention comment scopes the cleanup-workflow claim to the
same-repository designation; a fork or scratch destination manages its
own retention (docs updated to match, plus a note documenting the
repository VARIABLE a maintainer sets to enable publishing).

R1-5: the step's real bash now runs in the workflow behavioural
harness under bash -e with stubbed sudo/apt/curl/sha256sum/tar/uname:
worst-runner and checksum-reject scenarios exit 0 installing nothing,
the no-sudo happy path pins the ~/.local/bin + GITHUB_PATH pairing,
and the version-pin probe is pinned from both sides (wrong version
re-downloads, matching version skips). Real freeze/sudo on a
developer machine are shadowed so the tests are deterministic and can
never install to /usr/local/bin.

Nit: both sudo guards now check sudo -n true.

* fix(ci): capture-tools step review fixes — step-owned tool dir, anchored probe, honest failures

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(ci): capture-tools test harness — shadow tmux, don't blank its PATH dir

The harness dropped every host PATH directory that ships a tmux so the
step's apt branch would depend on the scenario, not on the machine
hosting the suite. On GitHub-hosted ubuntu runners tmux lives in
/usr/bin, so the filter blanked /usr/bin wholesale — bash, grep, mkdir,
and tar included — and execFileSync('bash') died of ENOENT: all seven
capture-tools tests failed in the Test (ubuntu-latest Node 22.x) job
while passing on tmux-less dev machines.

Replace the directory-level drop with an entry-level shadow: each
tmux-bearing directory is mirrored (symlinks) into a scratch dir minus
the tmux entry, in place, preserving PATH order and the empty-entry
stripping the old filter did. Hosts without tmux take the map through
unchanged, and Windows (no tmux in its PATH, no symlink branch) keeps
its current behavior exactly.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(ci): capture-tools test review fixes — faithful stubs, env shape pins, missing-branch scenarios

Round-3 review findings: the harness executed several of the step's
branches but asserted nothing about them, so probe-verified mutants
(dropped tmux guard, deleted warning/degradation messages, malformed or
missing FREEZE_VERSION/FREEZE_SHA256, wrong hash variable, dropped URL
`v` prefix or curl `-L`, severed tarball paths, broken regex boundary,
leaked mktemp dir) all shipped green.

- Make the curl/sha256sum/tar stubs model their real contracts: exact
  pinned URL, pinned checksum over a file curl actually wrote, existing
  -xzf operand
- Pin FREEZE_VERSION/FREEZE_SHA256 shape in captureToolsSource
- Pin the full curl flag set and the three-site tarball path agreement
- Assert the stale-renderer warning (fires on degraded re-download,
  silent on the happy path) and the tmux-unavailable message
- Pin TMPDIR and assert the mktemp cleanup leaves it empty
- Add the two missing scenarios: tmux-present skips apt, cached version
  extending the pin with a leading digit re-downloads

Verified by 13 mutation probes: every named mutant now turns the suite
red (13/13 killed), baseline 34/34 green.

* fix(ci): capture-tools step review fixes — hash-verified cache, per-run PATH promotion

* fix(ci): capture-tools step review fixes — verified-bytes-only installs, step timeout

Review findings on the capture-tools step:

- Drop the PATH-trust branch: a freeze already on PATH was accepted on
  its own --version and executed to probe it — exactly the self-report
  the FREEZE_BIN_SHA256 comment declares attacker-controllable, from
  dirs writable between jobs on both runner classes. The checksummed
  download always runs now; the cache makes it free after the first
  run.
- Guard $tools_bin in the download branch: with mktemp failing, the
  unguarded install resolved to /freeze — harmless unprivileged, but a
  root-in-container self-hosted runner writes it and reports success
  with nothing on PATH.
- Copy-then-verify the cache: install into the fresh per-run dir FIRST,
  verify THOSE bytes, delete both copies on mismatch — the verified
  bytes are the bytes later steps execute, closing the check-then-copy
  race for free. This makes the separate pre-verify block redundant;
  it is deleted.
- Add timeout-minutes: 5 — continue-on-error bounds failure, not
  duration, and a stalled `sudo apt-get update` mirror had no other
  bound under the 300-minute job cap.
- Report block: say the resolved freeze is likely broken when its
  --version produces nothing, instead of echoing a blank line and
  calling it stale; the mismatch wording is direction-neutral now.

Tests: replace the PATH-trust scenario with a planted-PATH one (marker
outside the scenario dir proves the plant never executes), add the
mktemp-failure scenario (the install stub succeeds like root would, so
the unguarded mutant is caught) and the promoted-dir 0700 assertion;
re-anchor the two digit-boundary tests on the report's warning. 41/41
green; both fix mutants verified killed.

* fix(ci): capture-tools review fixes — stale-dir cleanup, pinned guards

Address round-5 review:
- R5-1 (Critical): the per-run qwen-review-tools.* dir under RUNNER_TEMP
  was never removed; RUNNER_TEMP survives across jobs on the shared pool,
  so every review run accumulated one dir + one Go binary, unbounded.
  'Clean stale agent state' now removes stale dirs before the install
  step creates the current run's dir, matching the qwen-triage.yml
  convention. The harness comment claiming the dirs were runner-cleaned
  is corrected.
- R5-6: the cache re-verification rejection branch now logs why it
  deletes the cached binary instead of degrading silently.
- R5-7: bump-checklist note beside the freeze pins — the harness stubs
  key on the same env values, so a transposed hash pair must be caught
  against the real release artifacts at bump time.
- R5-2/R5-3/R5-4/R5-5: four unpinned step properties now pinned (the
  if: guard, the sudo -n probe flag, install-after-context ordering, and
  the cache branch's tools_bin guard via a new mktemp-fails scenario);
  six mutation probes confirm each pin kills its mutant.

* fix(ci): capture-tools review fixes — curl budget, swept scratch dir, wiring pins

* fix(ci): capture-tools review fixes — harness mutation pins, pin-pair self-check

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(ci): capture-tools review fixes — shadow-farm cleanup, backoff budget term

* fix(ci): capture-tools review fixes — report probes only installed freeze, age-gated sweep

---------

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: qwen-code-dev-bot <qwen-code-dev-bot@users.noreply.github.com>
2026-08-05 13:16:56 +00:00
..
ai-release-notes-workflow.test.js fix(release): keep notes anchored and cap the release body (#8199) 2026-07-31 09:55:38 +00:00
audit-runtime-critical.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
build-and-publish-image-workflow.test.js ci(autofix): restore sandbox image flow (#6261) 2026-07-03 15:30:58 +00:00
check-build-status.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
check-i18n.test.ts fix(cli): localize approval mode UI labels (#6592) 2026-07-11 00:07:03 +00:00
chrome-extension-package.test.js fix(ci): cover release integration regressions (#5994) 2026-06-29 11:54:11 +00:00
ci-flaky-rerun-workflow.test.js fix(ci): stop a slow patrol classifier from killing every flaky rerun (#7358) 2026-07-21 02:33:51 +00:00
ci-flaky-rerun.test.js feat(ci): auto-open a deflake fix issue for confirmed flaky tests (#7231) 2026-07-19 16:49:29 +00:00
clean-package-build-artifacts.test.js test(core): stabilize file history eviction test (#6637) 2026-07-10 06:39:52 +00:00
cli-entry.test.js fix(cli): preserve Qwen Review startup version in footers (#8431) 2026-08-04 14:58:56 +00:00
comment-attachment-guard-workflow.test.js ci: route trusted-author fork PRs and no-checkout jobs to the ECS pool (#8502) 2026-08-04 03:48:24 +00:00
dev.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
e2e-workflow.test.js fix(ci): keep the post-merge E2E signal on main alive (#7795) 2026-07-28 11:54:52 +00:00
generate-changelog.test.js feat(release): generate AI-assisted release notes (#6756) 2026-07-12 13:00:22 +00:00
generate-release-notes.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
get-release-version-python-sdk.test.js feat(sdk-python): add network timeouts to release version helper (#3833) 2026-05-05 19:25:00 +08:00
get-release-version.test.js fix(release): bump preview base past published stable (#7978) 2026-07-29 23:21:00 +00:00
install-script.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
issue-triage-ownership-workflow.test.js ci: remove broken legacy scheduled PR triage workflow (#8434) 2026-08-03 10:20:42 +00:00
lint.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
main-ci-failure-issue-workflow.test.js fix(ci): keep the post-merge E2E signal on main alive (#7795) 2026-07-28 11:54:52 +00:00
no-ak-integration-ci.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
package-assets.test.js refactor(core): move review skill incident narratives to DESIGN.md (#8499) 2026-08-04 12:41:18 +00:00
package-scripts.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
pr-force-push-reminder-workflow.test.js ci(autofix): run agents on dedicated ECS runners (#6207) 2026-07-03 07:40:07 +00:00
pr-self-report-label.test.js ci: label a PR that closes an issue its own author opened (#7630) 2026-07-24 05:50:47 +00:00
qwen-autofix-workflow.test.js perf(autofix): build the review CLI bundle once per scan and fan it out to legs (#8548) 2026-08-05 05:16:49 +00:00
qwen-fleet-shepherd-workflow.test.js feat(autofix): label-driven takeover and release; fix forced-dispatch green no-op (#7165) 2026-07-19 07:35:13 +00:00
qwen-pr-review-workflow.test.js ci(review): prepare evidence-image tooling for GitHub-triggered reviews (#8454) 2026-08-05 13:16:56 +00:00
qwen-repo-hygiene-workflow.test.js feat(ci): add repo-hygiene skill and weekly patrol workflow (#7908) 2026-08-02 09:45:38 +00:00
qwen-resolve-workflow.test.js test(ci): align resolve-workflow timeout expectations with externalized variables (#8460 follow-up) (#8485) 2026-08-05 02:33:04 +00:00
qwen-triage-finalize-workflow.test.js fix(ci): rename triage status marker to avoid duplicate-guard collision (#7723) 2026-07-26 15:51:22 +00:00
qwen-triage-workflow.test.js ci: auto-update ECS runners on stable publish and harden update job (#8343) 2026-08-02 03:33:52 +00:00
release-helpers.test.js refactor: extract shared release helper utilities (#3834) 2026-05-05 10:15:17 +08:00
release-sdk-workflow.test.js fix(ci): skip empty SDK release PR (#6861) 2026-07-14 13:19:42 +00:00
release-workflow.test.js fix(release): reuse configured Apple signing secrets (#8574) 2026-08-05 09:20:17 +00:00
review-worktree-cleanup-workflow.test.js fix(ci): clean review worktrees after cancellation (#8474) 2026-08-05 02:39:36 +00:00
sandbox-command.test.js fix(scripts): avoid shell injection in sandbox command detection (#6108) 2026-07-01 16:20:40 +08:00
sdk-java-workflow.test.js ci: route trusted-author fork PRs and no-checkout jobs to the ECS pool (#8502) 2026-08-04 03:48:24 +00:00
sdk-node-exporter-stub.test.js perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol (#7276) 2026-07-21 07:35:30 +00:00
serve-fast-path-bundle-check.test.js feat(ci): fail the startup bundle check when the CLI entry is hoisted into a chunk (#8203) 2026-07-31 08:57:57 +00:00
start.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
test-setup.ts feat(installer): add standalone archive installation (#3776) 2026-05-11 13:25:48 +08:00
update-ecs-runner-qwen-workflow.test.js fix(ci): reconcile ECS runner updater on workflow changes (#8373) 2026-08-02 09:25:17 +00:00
upload-aliyun-oss-assets.test.js feat(installer): add standalone hosted install and uninstall flow (#3828) 2026-05-21 11:57:10 +08:00
verify-capture.test.js fix(ci): avoid verify capture color conflict (#8236) 2026-07-31 14:15:40 +00:00
vitest.config.ts ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
workflow-helpers.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
workspaces.test.js feat(desktop): Add desktop app package with Qwen ACP SDK integration (#3778) 2026-06-11 21:57:20 +08:00