qwen-code/scripts/tests
Shaojin Wen 9f2342d323
fix(ci): stop the fallback comment from denying a review it already posted (#9462)
* fix(ci): stop the fallback comment from denying a review it already posted

The review job can fail AFTER posting its review — the CLI exiting
silently, a cleanup step dying — and both fallback sites then announce
that review as one that could not be posted, retry instruction attached.
Measured on PR #9342: the review posted at 11:56:34Z, review-pr failed
at 12:00:53Z ("Qwen review completed but produced no output"), and the
comment landed at 12:01:00Z saying the pipeline "failed before a review
could be posted. … retry with @qwen-code /review" — a fresh ~3-hour
review, asked for beside the review that had just landed. The autofix
takeover loop reads the same feed a human does.

Both sites now check, before composing a body, whether a review this run
posted is already on the PR. The check is scoped three ways so a stale
review can never buy silence on a genuinely dead pipeline: the bot's own
account, the head this run reviewed, and a submission at or after this
run started. Where the proof is unavailable — no start time, no head, a
failed listing — the guard declines to fire and the comment posts, the
same call the head-moved guard already makes.

The job-level step now reads state and headRefOid in one `gh pr view`
(the in-job step already did), which is where its head value comes from.

Tests run the steps' real bash over review fixtures, because the guard
IS a filter: silence when this run posted the review, and posting for
each near-miss on its own — an earlier run's review at the same head,
another account's, one of a different head, a PENDING one, none at all,
an unavailable start time, and a failed reviews listing. One existing
assertion tightened: "no `gh run view`" was the proxy for "no head
comparison on comment runs", and the new guard asks that same command
for startedAt on every event, so it now pins the head lookups
themselves. The stub's state,headRefOid branch learned the pr_closed
scenario its state-only sibling already knew.

* fix(ci): anchor the already-posted guard on the run's creation, and say when it cannot run

Round 1's two blockers, both re-verified against this repo's own run data.

The time anchor reset on job re-runs. `gh run view --json startedAt`
returns the LATEST attempt's start while the run id stays the same — the
dedup above relies on that stability — so a re-run pushed attempt 1's
review outside "this run": runs 32219268680 (created 05:23:57Z,
startedAt 05:51:26Z) and 32218596441 (05:13:04Z → 05:22:05Z) both show
the ~9-28 minute shift. Attempt 1 posts its review, the job fails after
the post, someone re-runs it, attempt 2 fails before posting — and the
guard, anchored on attempt 2's start, lets the contradictory comment
through. Exactly the shape this PR exists to stop, on the path most
likely to reach it. Both sites now anchor on `createdAt`, which is
attempt-stable; a review submitted after the run was created still
cannot belong to an earlier run, so the stale-review protection is
unchanged.

The guard also swallowed its own lookup failures. A transient failure
in either call emptied the value, the guard declined, and the false
comment posted with nothing in the log separating "the guard ran,
nothing matched" from "the lookup died" — while every sibling lookup in
these steps announces its failures. Both unavailable paths now emit a
`:⚠️:` and a step-summary line before posting. No behavior
change: posting was, and remains, the fail-open direction.

Tests: a re-run fixture per site, where the stub answers `createdAt` and
`startedAt` with DIFFERENT values and attempt 1's review sits between
them — reverting either site to `startedAt` fails exactly these two; and
a per-site assertion that both unavailable paths announce themselves.

Also from round 1, both verified before taking: the stub's standalone
`*state*)` branch is dead (no `--json state` call remains in either
extracted step) and is removed, so its scenarios cannot be edited into a
no-op; and the harness now substitutes `${{ vars.* }}` before running
the in-job script, which bash rejected as a bad substitution — the
assignment was skipped, `MAX_TIMEOUT_MINUTES` stayed unset, and eight
error lines rode every suite run, so "the step's real bash" was not
quite true for that line.

* fix(ci): read the head this run reviewed, and claim only what the guard proved

Round 2's six, all taken.

The fallback JOB compared review commit ids against the PR's head at
fallback time, not the head the run reviewed. On every trigger but
pull_request_target the head-moved guard above deliberately does not
run, so a push landing between the post and this step leaves that value
pointing at bytes no review ever covered: the match fails and the
contradictory comment posts anyway — the #9342 shape, re-opened for the
trigger + post + push + fail-after-post interleaving. `review-pr` now
publishes the head its review step recorded as a job output, and the
guard reads it, falling back to the fresh head only when the job died
before that step (a run that posted nothing either). The in-job twin
needs none of this — its unconditional head-moved check exits first —
and that asymmetry is now pinned per site rather than left to be
rediscovered.

Both skip messages claimed "this run already posted a review". Reviews
carry no run id, so the window (bot account + head + submitted at or
after this run was created) also matches an overlapping sibling run's
review, which this workflow's own concurrency note says can happen. The
suppression is right either way — a review IS sitting above the comment
— but the oncall reading the summary was told something the guard never
proved; both now say what it did.

The guard's opening paragraphs still described the round-1 `startedAt`
anchor while the code (and the paragraph below it, and the runtime
warning) said creation. A maintainer reading top-down got the anchor
that re-runs break — the defect round 1 removed.

Test stub: `gh run view` now answers by running the caller's own --jq
over an object carrying both timestamps, instead of a `case` on "$*"
that matched substrings in order. A combined
`--json createdAt,startedAt --jq '.startedAt'` was answered from the
createdAt branch, leaving the re-run pin green for a guard reading the
attempt-scoped field — the exact regression it exists to catch.

* fix(ci): attribute the guard by time alone — the head is not a stable run attribute

Round 3's blocker, and the second time the head clause re-opened the
contradiction this PR exists to close. Two entrances this round, both
after a "Re-run failed jobs": attempt 2 dies before the review step
writes its head, so the guard falls back to a head attempt 1 never
reviewed; or a push lands and attempt 2 records the NEW head — in both,
attempt 1's own review no longer matches `.commit_id`, and the fallback
posts "failed before a review could be posted … retry" beneath the
review the same run had posted.

Rather than patch the head lookup a third time, the head clause is
gone. What the guard proves is now narrower and stable: a bot review of
this PR was submitted while this run was alive — bot account plus the
attempt-stable `createdAt` window. That closes both entrances at once
and takes the round-2 cross-job wiring with it (review-pr's
`expected_head_sha` output and the env line that read it), so there is
no untested chain left whose silent breakage would restore the
fresh-head comparison. The job-level step no longer needs the PR head
either and reverts to its state-only query; the test stub's
state-only branch, removed in round 1 as dead, has a caller again.

The comment blocks now state the guarantee the concurrency model
actually supports. They claimed a review inside the window "cannot
belong to an earlier run", but per-run concurrency groups deliberately
allow overlapping runs on the same head, so an earlier-created run's
review can match and this run's failure then goes unannounced. That is
accepted, and said plainly: the silence coincides with a bot review a
reader can see — the very state that makes the comment's claim false —
while the bot-author and creation-time clauses still rule out silence
with no review at all.

Tests: the moved-head case flips from "posts" to "silences" and is
pinned per site (a review on ANY head inside the window silences);
re-introducing a head clause fails exactly that test; and a structural
pin asserts the wiring is absent rather than merely unused.

* test(ci): skip the guard's jq-driven cases where jq is absent, instead of failing them

The stub answers the guard's reviews and run-view lookups by running the
caller's own `--jq` filter — that filter IS the thing under test — so
those cases need jq on PATH. A reviewer running the suite on Windows
without jq saw them as failures of the guard rather than as untested,
which is the wrong signal in the wrong direction.

Probed once per run and skipped honestly. Measured with a jq that exits
127: the file goes from 31 failures to 26 failures plus 13 skips — the
26 are the retry-loop cases, which have parsed the review log with jq
since long before this change and are equally untestable without it.
GitHub's windows-latest image ships jq, so CI coverage is unchanged
either way; what changes is what a jq-less machine reports.

* docs(ci): remove the head-keyed leftovers the guard no longer has

Round 5's four, all leftovers of the round-3 design change rather than
new behavior.

The job-level block still explained why it compared against the head
this run reviewed — naming `pr_head`, "the reviewed head's review" and a
`review-pr` job output, none of which survive: the shipped filter is
author scope plus the creation-time window, and the wiring was deleted
with the head clause. A maintainer reading it would look for a
comparison that is not there. The in-job block stated the
createdAt-not-startedAt rationale twice, once with the measured run ids
and once without; the measured one stays.

Same in the tests: the stub's comment listed a head clause the filter
deliberately does not have (`attributes by TIME, not by head` is the
test that pins its absence), and the harness still declared and injected
`reviewedHead`/`REVIEWED_HEAD_SHA`, which nothing reads since the wiring
went — a knob that looks live and cannot be.

* docs(ci): drop the duplicated anchor rationale and the last stale-head leftovers

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(ci): exclude foreign same-account approvals from the already-posted guard

* fix(ci): attribute the already-posted guard by composed-review markers

The foreign-approval exclusion list shipped incomplete: the triage skill's
commit-pinned APPROVE body also posts under the same account, matches the
guard's author and window clauses, and silenced the fallback for a
genuinely dead run — the failure shape this guard exists to stop. The
producer set is open, so no exclusion list can be finished; every miss
fails in the dangerous direction.

Match positively instead: a review silences the fallback only if its body
carries what only this pipeline's composed reviews carry — the
"via Qwen Code /review" attribution footer or the invisible
qwen-review-ledger marker. Every composed body carries at least one (a
zero-findings APPROVE included); no foreign approval carries either. A
marker that ever changes shape stops the guard firing and the comment
posts — the pre-guard status quo, not a masked dead run.

---------

Co-authored-by: qwen-code-ci-bot <qwen-code-ci-bot@users.noreply.github.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-08-20 23:51:05 +00:00
..
ai-release-notes-workflow.test.js fix(release): keep notes anchored and cap the release body (#8199) 2026-07-31 09:55:38 +00:00
audit-runtime-critical.test.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
build-and-publish-image-workflow.test.js ci(autofix): restore sandbox image flow (#6261) 2026-07-03 15:30:58 +00:00
capture-tmux-ci.test.js ci: install tmux and zip tooling on the Linux test lane, and pin it (#8792) 2026-08-11 05:47:39 +00:00
check-build-status.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
check-i18n.test.ts fix(cli): localize approval mode UI labels (#6592) 2026-07-11 00:07:03 +00:00
check-voice-guard-sync.test.js feat(voice): support trusted private ASR base URLs (#8350) 2026-08-06 14:04:57 +00:00
chrome-extension-package.test.js fix(ci): cover release integration regressions (#5994) 2026-06-29 11:54:11 +00:00
ci-flaky-rerun-workflow.test.js fix(ci): stop dropping agent settings in resolve and follow-up workflows (#9252) 2026-08-16 03:27:00 +00:00
ci-flaky-rerun.test.js feat(ci): auto-open a deflake fix issue for confirmed flaky tests (#7231) 2026-07-19 16:49:29 +00:00
clean-package-build-artifacts.test.js test(core): stabilize file history eviction test (#6637) 2026-07-10 06:39:52 +00:00
cli-entry.test.js fix(cli): preserve Qwen Review startup version in footers (#8431) 2026-08-04 14:58:56 +00:00
comment-attachment-guard-workflow.test.js ci: route trusted-author fork PRs and no-checkout jobs to the ECS pool (#8502) 2026-08-04 03:48:24 +00:00
cross-package-contracts.test.js refactor: centralize cross-package contracts (#9497) 2026-08-20 06:24:41 +00:00
desktop-oss-workflow.test.js fix(desktop): harden release pipeline (#9009) 2026-08-12 16:38:12 +00:00
dev.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
e2e-workflow.test.js fix(ci): keep the post-merge E2E signal on main alive (#7795) 2026-07-28 11:54:52 +00:00
generate-changelog.test.js feat(release): user-facing bilingual digest for release notes (#9216) 2026-08-17 00:12:04 +00:00
generate-release-notes.test.js feat(release): user-facing bilingual digest for release notes (#9216) 2026-08-17 00:12:04 +00:00
get-release-version-python-sdk.test.js feat(sdk-python): add network timeouts to release version helper (#3833) 2026-05-05 19:25:00 +08:00
get-release-version.test.js fix(ci): force-push release branch so retries replace failed attempts (#9076) (#9082) 2026-08-16 16:33:14 +00:00
install-script.test.js fix(install): avoid Get-FileHash for Windows checksums (#9112) 2026-08-14 01:12:08 +00:00
integration-vitest-config.test.ts fix(tests): apply integration worker limits to forks (#8689) 2026-08-08 00:53:33 +00:00
issue-triage-ownership-workflow.test.js ci: remove broken legacy scheduled PR triage workflow (#8434) 2026-08-03 10:20:42 +00:00
lint.test.js fix(ci): cache downloaded linters on ECS runners (#9001) 2026-08-13 05:13:23 +00:00
live-host-oss-workflow.test.js fix(ci): restore Live Host release mirroring (#8917) 2026-08-11 07:08:26 +00:00
main-ci-failure-issue-workflow.test.js fix(ci): keep the post-merge E2E signal on main alive (#7795) 2026-07-28 11:54:52 +00:00
no-ak-integration-ci.test.js feat(web-shell): add transcript contract prevalidation (#9388) 2026-08-19 14:13:12 +00:00
package-assets.test.js fix(review): harden the pipeline against four live-run failures (#9086) 2026-08-14 04:38:49 +00:00
package-scripts.test.js chore(ci): Drop NPM_TOKEN in favor of npm Trusted Publishing (#9552) 2026-08-20 08:31:41 +00:00
pr-force-push-reminder-workflow.test.js ci(autofix): run agents on dedicated ECS runners (#6207) 2026-07-03 07:40:07 +00:00
pr-self-report-label.test.js feat(autofix): escalate stopped takeover PRs and age out unanswered pauses (#8960) 2026-08-15 17:32:23 +00:00
qwen-autofix-fork-bridge-workflow.test.js feat(autofix): bridge fork-PR reviews into the credentialed review lane (#8676) 2026-08-07 16:11:48 +00:00
qwen-autofix-workflow.test.js fix(ci): make autofix finding replies idempotent (#9463) 2026-08-20 14:10:26 +00:00
qwen-fleet-shepherd-workflow.test.js fix(ci): stop counting wedged queued runs as in-flight in the shepherd (#9518) 2026-08-20 13:37:22 +00:00
qwen-pr-review-workflow.test.js fix(ci): stop the fallback comment from denying a review it already posted (#9462) 2026-08-20 23:51:05 +00:00
qwen-repo-hygiene-workflow.test.js fix(ci): route workflow label mutations through REST (#8761) 2026-08-09 15:05:15 +00:00
qwen-resolve-workflow.test.js fix(ci): keep the review workflow under the expression-length limit (#8720) 2026-08-08 05:53:35 +00:00
qwen-triage-finalize-workflow.test.js fix(ci): rename triage status marker to avoid duplicate-guard collision (#7723) 2026-07-26 15:51:22 +00:00
qwen-triage-workflow.test.js fix(ci): heal a symlinked workspace instead of wedging the runner on it (#9498) 2026-08-20 13:54:36 +00:00
release-helpers.test.js refactor: extract shared release helper utilities (#3834) 2026-05-05 10:15:17 +08:00
release-sdk-workflow.test.js fix(ci): skip empty SDK release PR (#6861) 2026-07-14 13:19:42 +00:00
release-workflow.test.js fix(ci): force-push release branch so retries replace failed attempts (#9076) (#9082) 2026-08-16 16:33:14 +00:00
review-source-digest.test.ts refactor(cli): consolidate shared helpers ahead of the legacy audit skill (#9345) 2026-08-19 14:53:44 +00:00
review-worktree-cleanup-workflow.test.js fix(ci): clean review worktrees after cancellation (#8474) 2026-08-05 02:39:36 +00:00
sandbox-command.test.js fix(scripts): avoid shell injection in sandbox command detection (#6108) 2026-07-01 16:20:40 +08:00
sdk-java-workflow.test.js ci: route trusted-author fork PRs and no-checkout jobs to the ECS pool (#8502) 2026-08-04 03:48:24 +00:00
sdk-node-exporter-stub.test.js perf(telemetry): lazy-load the SDK and split OTLP exporter chains by protocol (#7276) 2026-07-21 07:35:30 +00:00
security-workflows.test.js chore(ci): Add security hygiene: CODEOWNERS for release workflows, least-privilege permissions, security checks and Scorecard (#9008) 2026-08-14 01:22:53 +00:00
serve-ab-workflow.test.js fix(ci): heal a symlinked workspace instead of wedging the runner on it (#9498) 2026-08-20 13:54:36 +00:00
serve-fast-path-bundle-check.test.js feat(ci): fail the startup bundle check when the CLI entry is hoisted into a chunk (#8203) 2026-07-31 08:57:57 +00:00
start.test.js fix(review): report what the transcripts prove; build the roster in one call (#7033) 2026-07-18 00:43:57 +00:00
test-setup.ts feat(installer): add standalone archive installation (#3776) 2026-05-11 13:25:48 +08:00
update-ecs-runner-qwen-workflow.test.js fix(ci): reconcile ECS runner updater on workflow changes (#8373) 2026-08-02 09:25:17 +00:00
upload-aliyun-oss-assets.test.js feat(installer): add standalone hosted install and uninstall flow (#3828) 2026-05-21 11:57:10 +08:00
verify-capture.test.js fix(ci): avoid verify capture color conflict (#8236) 2026-07-31 14:15:40 +00:00
vitest-global-setup.test.js fix(devx): fail with actionable message when unit-test build prerequisites are missing (#9149) (#9171) 2026-08-18 13:19:09 +00:00
vitest.config.ts fix(ci): back-port the checkout-heal wipe guard to the triage and serve-ab wipes (#9277) 2026-08-18 06:49:40 +00:00
workflow-helpers.js ci: run Windows merge queue tests on ECS (#8386) 2026-08-05 12:14:42 +00:00
workflow-size.test.js fix(ci): keep qwen-autofix.yml under GitHub's 500 KB start-runs limit (#9517) 2026-08-20 01:56:45 +00:00
workspaces.test.js feat(desktop): Add desktop app package with Qwen ACP SDK integration (#3778) 2026-06-11 21:57:20 +08:00