qwen-code/scripts/tests
Shaojin Wen 1f9318f974
feat(triage): stop in-agent CI polling, finalize evidence and approval after CI completes (#7693)
* feat(triage): stop in-agent CI polling, finalize evidence and approval after CI completes

The triage agent's Stage 2b polled pending checks for up to 10 minutes, but
this repo's unit suite runs ~30 minutes, so the poll always burned its full
budget, gave up with 'CI still running', and Stage 3 could then approve before
the suite finished (observed on a PR approved 12 minutes before its Test job
completed).

Split the wait out of the agent entirely:

- pr-workflow.md Stage 2b now forbids polling: fetch check-runs once, report
  pending checks honestly, and wrap the CI table in qwen-triage-ci region
  markers keyed to the reviewed SHA.
- Stage 3 defers a clean-verdict approval when checks are still pending: the
  comment carries an approve-on-green marker instead of an immediate APPROVE.
- New qwen-triage-finalize.yml fires on workflow_run completion of 'Qwen Code
  CI' / 'E2E Tests' and, with plain bash over the API (no model, no checkout),
  rewrites the marked table region with the settled results and posts the
  commit-pinned approval only when every check landed green — failing closed
  on red checks, a moved head, or a closed/draft PR, and flipping the triage
  status comment to say which way it resolved.

Markers are honored only in comments authored by the bot identity itself, and
check names (attacker-influenced on fork PRs) go through the same HTML-escape
chain the skill mandates for file paths.

Stage comments now land ~10 minutes sooner and the approval, when deferred,
lands at CI completion with full evidence instead of before it.

* fix(triage): address finalize review — broken red gate, table truncation, dead trigger

Review findings on the finalize workflow, all reproduced before fixing:

- Blocker 1: the RED jq used the array-first membership form, where | rebinds
  . and .conclusion indexes an array — jq exits 5 every run, RED comes back
  empty, [ "" -gt 0 ] errors, and control falls through to the approve path:
  a red CI auto-approved. The gate now binds the conclusion before the
  membership test (IN(...)), and the counters are numeric-validated so any
  future jq failure reads as 'cannot attest', never 'approve'.
- Blocker 2: the table rendered raw check-runs — on a real PR (96 runs, 35
  names, 68 skipped) alphabetical sort + head -60 truncated away every actual
  test job. table_rows now dedups per name (latest run), drops skipped rows,
  and sorts running/non-green first so the cap can only cut green rows.
  Replayed against the same PR: 96 rows -> 16, unit suite present.
- The approval gate now reads workflow runs filtered to event=pull_request
  (deduped per workflow) instead of head-SHA check-runs, which also carry
  long-running bot orchestration jobs that would wedge PENDING above zero at
  the exact moment the last CI workflow fires — silently dropping the
  deferred approval forever. The skill's Stage 3 PENDING count matches.
- E2E Tests had no pull_request trigger (dead entry); the workflows list is
  now exactly the six pull_request-triggered workflows, so the last finisher
  always re-fires the job.
- Head/state re-check moved before the red/deferred verdicts so a
  cancel-in-progress firing on a stale SHA cannot stamp a red status over
  the new head's comment; the still-deferred branch now updates the status
  comment instead of staying invisible.
- replace_region fails closed when the end-marker text only precedes the
  begin marker (awk END guard) — previously that shape truncated the comment
  body, eating the signature and reviewed-commit footer.
- Region content is deterministic (no run URL) so the no-op cmp works;
  empty run list or unavailable gate skips approval; comment wording fixed
  (workflow_run jobs are attributed to the default branch, so the self-check
  exclusion is belt-and-braces, not load-bearing).

Tests now execute the decision logic, not just grep for it: gate_counts and
table_rows run against fixtures covering every conclusion class, non-PR
events, re-run dedup, skipped filtering, ordering, and both marker-order
failure shapes. 30/30 passing.

* fix(triage): keep a stale finalize firing from clobbering the newer review's status comment

The status comment is deliberately not SHA-scoped (the triage workflow
creates it unscoped; scoping only the finalize side would orphan the
pairing), so a finalize firing for an old SHA that loses the race against a
newer head's green approval would overwrite the  status with a stale
warning. Guard the stale path: when the current head already carries bot
sha= markers (a re-review owns the status comment), stay silent; when the
head moved with no re-review yet — triage does not auto-rerun on
synchronize — the stale note is accurate and still posts. Closed/draft PRs
now just log instead of flipping the status.

* fix(triage): close the guardrail bypass and align the finalize table with the gate

Second review round, all four findings reproduced or confirmed before fixing:

- The approve-on-green marker was emitted in Step 1 while the fork-refactor
  GUARD only ran in Step 2 — a marker that slipped out on a fork refactor
  would have been honored by the finalize job on green CI, bypassing the
  guardrail entirely. GUARD now computes in Step 1 and gates the marker's
  emission, and the finalize job re-asserts it structurally from the PR
  state it already fetched (null head.repo = deleted fork = blocked), with
  a 'guarded' status message instead of an approval.
- table_rows now restricts check-runs to the suites of the same deduped
  event=pull_request workflow runs the gate trusts. Without it, 5 of 8
  rendered rows on this PR's own head were bot plumbing presented as CI
  evidence; with it, 115 raw check-runs reduce to exactly the 3 CI rows.
- A firing that saw PENDING>0 after the approval landed flipped the status
  comment back to 'deferred' with nothing to ever right it; the
  already-approved branch now repairs the status.
- Zero surviving table rows (failed runs fetch, missing suite ids) skips
  the region rewrite instead of blanking the agent's table, and
  replace_region refuses an empty region file (an unchecked getline would
  have deleted the region and its markers unrecoverably).

Nits: the house github.repository guard on the job, the table header
matches the skill template, and the run-URL stays out of the region so the
no-op cmp keeps working.
2026-07-25 00:58:16 +00:00
..
ai-release-notes-workflow.test.js ci(release): finalize stable releases asynchronously (#6868) 2026-07-15 00:30:51 +00:00
build-and-publish-image-workflow.test.js ci(autofix): restore sandbox image flow (#6261) 2026-07-03 15:30:58 +00:00
check-build-status.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
check-i18n.test.ts fix(cli): localize approval mode UI labels (#6592) 2026-07-11 00:07:03 +00:00
chrome-extension-package.test.js fix(ci): cover release integration regressions (#5994) 2026-06-29 11:54:11 +00:00
ci-flaky-rerun-workflow.test.js fix(ci): stop a slow patrol classifier from killing every flaky rerun (#7358) 2026-07-21 02:33:51 +00:00
ci-flaky-rerun.test.js feat(ci): auto-open a deflake fix issue for confirmed flaky tests (#7231) 2026-07-19 16:49:29 +00:00
clean-package-build-artifacts.test.js test(core): stabilize file history eviction test (#6637) 2026-07-10 06:39:52 +00:00
cli-entry.test.js fix(cli): update npm installs safely in background (#7322) 2026-07-21 06:32:24 +00:00
comment-attachment-guard-workflow.test.js ci: add suspicious comment attachment guard (#6599) 2026-07-10 09:40:54 +00:00
dev.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
generate-changelog.test.js feat(release): generate AI-assisted release notes (#6756) 2026-07-12 13:00:22 +00:00
generate-release-notes.test.js feat(release): generate AI-assisted release notes (#6756) 2026-07-12 13:00:22 +00:00
get-release-version-python-sdk.test.js feat(sdk-python): add network timeouts to release version helper (#3833) 2026-05-05 19:25:00 +08:00
get-release-version.test.js fix(scripts): handle missing NPM dist-tags gracefully in release versioning (#6476) (#6481) 2026-07-08 12:21:14 +00:00
install-script.test.js fix(cli): avoid updating active CLI processes (#6874) 2026-07-15 00:33:17 +00:00
issue-triage-ownership-workflow.test.js fix(ci): consolidate issue triage ownership (#7180) 2026-07-19 10:58:40 +00:00
lint.test.js revert: remove local PR verification gate (#7031) 2026-07-16 11:24:38 +00:00
main-ci-failure-issue-workflow.test.js feat(release): generate AI-assisted release notes (#6756) 2026-07-12 13:00:22 +00:00
no-ak-integration-ci.test.js feat(core): propagate trusted daemon invocation context (#7279) 2026-07-23 06:49:11 +00:00
package-assets.test.js fix(release): raise prepared package size limit to 96 MB (#6687) (#6691) 2026-07-11 01:01:59 +00:00
package-scripts.test.js refactor(autofix): extract review verification runner (#7644) 2026-07-24 16:46:24 +00:00
pr-force-push-reminder-workflow.test.js ci(autofix): run agents on dedicated ECS runners (#6207) 2026-07-03 07:40:07 +00:00
pr-self-report-label.test.js ci: label a PR that closes an issue its own author opened (#7630) 2026-07-24 05:50:47 +00:00
qwen-autofix-workflow.test.js refactor(autofix): extract review verification runner (#7644) 2026-07-24 16:46:24 +00:00
qwen-fleet-shepherd-workflow.test.js feat(autofix): label-driven takeover and release; fix forced-dispatch green no-op (#7165) 2026-07-19 07:35:13 +00:00
qwen-pr-review-workflow.test.js fix(web-shell): restore scheduled task reference interactions (#7313) 2026-07-21 07:10:26 +00:00
qwen-resolve-workflow.test.js fix(ci): serialise the two workflows that push to a PR head branch (#7392) 2026-07-21 07:40:56 +00:00
qwen-triage-finalize-workflow.test.js feat(triage): stop in-agent CI polling, finalize evidence and approval after CI completes (#7693) 2026-07-25 00:58:16 +00:00
qwen-triage-workflow.test.js ci(triage): surface live progress via an early status comment (#7654) 2026-07-24 06:27:53 +00:00
release-helpers.test.js refactor: extract shared release helper utilities (#3834) 2026-05-05 10:15:17 +08:00
release-sdk-workflow.test.js fix(ci): skip empty SDK release PR (#6861) 2026-07-14 13:19:42 +00:00
sandbox-command.test.js fix(scripts): avoid shell injection in sandbox command detection (#6108) 2026-07-01 16:20:40 +08:00
sdk-node-exporter-stub.test.js perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol (#7276) 2026-07-21 07:35:30 +00:00
serve-fast-path-bundle-check.test.js perf(startup): lazy-load Google GenAI SDK on first use (#7512) 2026-07-23 02:07:39 +00:00
start.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
test-setup.ts feat(installer): add standalone archive installation (#3776) 2026-05-11 13:25:48 +08:00
update-ecs-runner-qwen-workflow.test.js fix(ci): update qwen in the runner's active npm prefix (#7689) 2026-07-24 19:11:41 +00:00
upload-aliyun-oss-assets.test.js feat(installer): add standalone hosted install and uninstall flow (#3828) 2026-05-21 11:57:10 +08:00
vitest.config.ts refactor(auth): unify provider config in core, simplify /auth as "Connect a Provider" (#4287) 2026-05-20 23:48:52 +08:00
workspaces.test.js feat(desktop): Add desktop app package with Qwen ACP SDK integration (#3778) 2026-06-11 21:57:20 +08:00