3 commits

Author	SHA1	Message	Date
Shaojin Wen	f4802031d0	perf(ci): run docs-only automatic reviews at medium effort (#8648) ... * perf(ci): run docs-only automatic reviews at medium effort A 1-line docs PR costs the same 57-180 minute high-effort review as a code change, and on a diff with zero source lines the passes medium drops - the adversarial personas and the reverse audit - have no failure mode to hunt. Counterfactual analysis over six dissected CI runs showed the one case where those passes caught a real Critical was a source PR, which this gate never touches: classification reuses the Test workflow's conservative classify-profile.mjs (docs/**.md(x) + root prose only; markdown under any src/ tree stays full, matching the review skill's own source rule), and any fetch or classifier failure falls back to the full review. Only the automatic pull_request_target review downgrades; every explicit request (workflow_dispatch, @qwen-code /review) keeps full high effort. Because an effective --comment forces high and medium never posts, the downgrade drops --comment and a new step relays the review CLI's verbatim "Review complete:" line - its machine-readable completion contract - as a single PR comment, with a pointer for requesting the full review. The docs-only budget is the size-aware timeout halved with a 90-minute floor. * perf(ci): address review feedback on the docs-only medium gate All nine review suggestions, each verified before fixing: - review_requested is an explicit ask: the AUTO_REVIEW flag now excludes that action (authorize write-permission-checks its requester), so a maintainer requesting the bot's review gets the full high-effort run. - The fetch-and-classify wrapper is extracted to .github/scripts/ci/classify-pr-profile.sh and consumed by both ci.yml and the review gate, so the classifier's input contract lives in one place; distinct exit codes preserve each caller's fallback messages. - Neither completion-line fallback mints the reserved "Review complete: " prefix anymore, and the relayed line passes a strict not-posted disposition allowlist - on this never-posts path any posted-form disposition is false by definition (the measured phantom APPROVE posted), so it falls back to a neutral non-scrapable form. - The relay upserts by its marker (mirroring the queued-acknowledgement step) instead of stacking a comment per push, retries the POST/PATCH three times, and never fails the job - a failed relay after a successful review must not trip the failure fallback into announcing a review failure that never happened. - The Chinese relay copy no longer parses as "发行" and renders high-effort as 高强度 rather than 高档. - The qwen-review docs-only-medium marker is registered in all six BOT_COMMENT_FILTER sites in qwen-autofix.yml, so clean docs-only relays cannot select PRs into autofix rounds as actionable feedback. - The gate's behavioral invariants are pinned in scripts/tests/qwen-pr-review-workflow.test.js by executing the extracted bash: prompt-branch order (--effort medium instead of --comment), the halve-with-90-minute-floor arithmetic, the completion-line allowlist including the phantom shapes, AUTO_REVIEW exclusivity, the six-site marker registration, and the shared-wrapper routing in both workflows. * perf(ci): harden the docs-only gate against round-2 review findings Thirteen findings across two review passes; every fix is executed by a test rather than asserted as text where the finding was behavioral. - The relay marker exclusion in qwen-autofix.yml is author-scoped at all six filter sites: only the relay bot's own marker comment is filtered, so a human quoting the marker stays actionable feedback. - classify-pr-profile.sh guards the 3,000-file listing cap (any mismatch against the PR's declared changed_files classifies full), uses mktemp+trap instead of a fixed path on the shared persistent pool, and ships its own node:test suite (renamed source→docs pins the projection contract; exit codes 2/3 pinned) registered in HELPER_TESTS. - classify-profile.mjs restricts reserved root prose basenames to inert extensions - README.js / SECURITY.ts / LICENSE.sh classify full. - The completion-line allowlist binds to pr-<number> and to the only verdict a medium run can produce (Comment, not posted) - a stale line for another PR or an Approve-shaped injection falls back to neutral. - A dedicated review_completed output gates the relay: the state/head guards exit 0 without running the review, and outcome==success alone would have announced a review that never ran. - The relay upsert filters by the authenticated bot login, re-resolves the comment id on every attempt, and falls back to POST when the PATCH target is gone - a participant posting the marker can no longer capture the upsert, a transient listing failure no longer mints duplicates. - The gate and relay are now executed under stubbed executables in qwen-pr-review-workflow.test.js (docs_only/full/failure/explicit scenarios; POST/PATCH/never-fail branches), the AUTO_REVIEW pin covers both guard halves, and the marker contract is pinned producer-side and filter-side. * perf(ci): fix the medium Request-changes swallow and the stale docs badge Round-3 review findings (2 Critical, 8 test-gap Suggestions), each fix executed by a test where the finding was behavioral: - The completion-line allowlist accepts `Request changes, not posted` - compose-review caps only Approve at medium, so a docs-only run that verifies a Critical legitimately emits Request changes, and the old Comment-only allowlist swallowed exactly the blocker-finding outcome into the neutral fallback. Target binding to pr-<number> is unchanged and now pinned by a test, as is the last-line selection over a stale or injected earlier completion line. - A stale docs-only badge can no longer outlive its revision: the full automatic review path now supersedes the bot-authored marker comment (strikethrough + superseded note) via a new --update-only mode that never mints a badge where none existed. - The marker+author upsert protocol is extracted to .github/scripts/upsert-bot-comment.sh - one implementation shared by the relay and the supersede step (the per-step copies had already drifted), with its own node:test suite covering the author scope, the per-attempt re-resolution (deleted-mid-retry falls back to POST), and the --update-only no-op; registered in HELPER_TESTS. - The classify-pr-profile gh stub now applies the wrapper's own --jq argument with real jq over API-shaped fixtures, so the projection contract is genuinely under test (negative control: dropping `status` turns the renamed-source scenario red). - New pins: review_completed wiring end to end (run-step emit + both consumers' if clauses), the auto_review output->env wiring at both links, and both AUTO_REVIEW guard halves. * perf(ci): never let a failed lookup mint or keep a stale docs badge Round-4 review findings (1 Critical, 7 Suggestions): - The upsert script no longer conflates failed lookups with empty results: the authenticated login, the listing, and the jq extraction are all resolved inside the retry loop as one prerequisite chain, an attempt whose prerequisites failed retries instead of falling through to POST (the shape that minted a permanent duplicate badge off one transient 5xx), and --update-only exits 1 on a failed lookup so the supersede warning fires instead of a false no-op success. New tests pin the failed-listing-then-PATCH path, the persistent identity failure, the update-only failure exit, and the update-only PATCH. - Supersede now covers every path that owes the correction: a FAILED full review and an EXPLICIT requested review (the badge's own CTA) both retire the badge, gated only on docs_only_medium == 'false' - empty on runs that failed before classifying, so a badge is never superseded on ignorance. The body is cause-neutral: it asserts only that the badge described an earlier revision. - The marker literal is defined once per step (MARKER variable, the qwen-triage convention) and shared between body and lookup argument; a pin requires the definition and --update-only on the supersede invocation. - New behavioral pins: the Approve verdict stays rejected by the allowlist, github_ci_only never downgrades (CI helpers are executable), and review_completed's emit position is asserted AFTER the closed-PR and stale-head guards (the hoist mutant survived position-independent contains checks). * perf(ci): make docs_only_medium three-valued and pin the untested guards Round-5 review findings (1 Critical, 6 Suggestions): - docs_only_medium no longer conflates "determined not docs-only" with "never determined": the output is three-valued ('' when the classification failed or never ran), so a transient classifier failure or a dispatch dry-run can no longer retire a still-accurate badge. The supersede condition names its two licensed paths explicitly - a POSITIVE not-docs-only determination (without requiring review success), or an explicit comment-mode review that completed (the badge's CTA; report-mode dry runs retire nothing). - The count-mismatch fallback in classify-pr-profile.sh logs to stderr, so a systematic divergence is distinguishable from every PR genuinely classifying full. - Six probed surviving mutants now each turn a test red: the supersede body is executed (existing-badge PATCH and the never-fail guard), ci.yml's rc-handling fragment is executed (exit 0/2/3 with the full fallback), the changed_files fetch failure exits 2, duplicate badges PATCH the last (newest) comment, and the relay's POSTed body must carry the marker that keys both the upsert and the supersede. * perf(ci): bind the docs badge to the reviewed head and retire it on failure MDX pages are executable (imported components, expressions), so the classifier no longer treats them as inert docs-only changes. The relay and supersede writes re-read the live PR state/head immediately before the mutation and skip unless the PR is still open at the reviewed SHA, the badge body names that SHA, a failed docs-only review now retires the singleton badge instead of leaving the previous revision's outcome visible, and the retired wording is cause-neutral (an explicit review can complete on the very head the badge describes). Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> --------- Co-authored-by: verify <verify@local> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> Co-authored-by: qwen-code-dev-bot <qwen-code-dev-bot@users.noreply.github.com>	2026-08-07 17:00:32 +00:00
Shaojin Wen	43c35e533b	fix(ci): keep the post-merge E2E signal on main alive (#7795) ... * fix(ci): stop cancelling in-progress E2E runs on main Merges land on main roughly every 18 minutes (median) while a full E2E run takes about 40, so cancelling the in-flight run on every push starved the suite: across the last 100 push runs, 67 were cancelled and only 25 ever reported a result. The nightly regression was the only reliable signal, and each cancelled run still burned roughly 10 minutes on three runners before dying. Turning cancellation off for main hands the coalescing to GitHub's concurrency queue, which keeps at most one pending run per group and cancels the previously pending one. The queue therefore collapses to the newest tree on its own while the in-flight run always finishes, so every result covers the batch of commits merged since the last one — bisect that range when it goes red. Dev branches keep cancelling superseded runs, where only the latest push matters. * fix(ci): dedupe main CI failure issues by failing test The autofix issue for a red main was deduped on the commit SHA, so a standing failure opened a brand new issue on every merge: one broken E2E test on 2026-07-26 produced six duplicate issues in twelve hours, each pointing the autofix agent at an unrelated commit. The failing tests are now identified from the logs of the failed jobs and used as the dedupe key, with one marker per test so a failure set that grows still matches the issue that already tracks part of it. Later commits hitting the same failure are appended to that issue as recurrences (bounded, newest first) instead of opening another one, and notes written by a human or the agent are preserved when the machine-owned trailer is refreshed. Runs with no identifiable test — an install or build break — keep the previous per-commit behaviour. * fix(ci): keep the failure analysis out of the bot-PAT job Identifying the failing tests means running a helper from the repository, and the workflow deliberately checked out nothing so that the job holding the bot PAT could never execute repository code — an invariant its own test enforces. Rather than weaken it, the work is split: a read-only job checks out the tree, reads the failed run's logs, finds any issue that already tracks the failure and renders the title and body, handing both over as outputs. The job with the PAT keeps checking out nothing and only writes what it was given. The invariant test now asserts that separation — the PAT job runs no repository code and holds only issue write — instead of banning checkout everywhere in the file. * fix(ci): harden main CI failure dedupe per review (#7795) - Match the `[run <id>]` link text instead of the run URL when deduping recurrences: `/301` is a substring of `/3010`, so the URL match silently deleted an unrelated run's line. - Rebuild the machine-owned "## Also failing" section from the live failure set on every merge so a test that has since been fixed drops out instead of being listed forever. - Assert the analyze checkout is SHA-pinned with persist-credentials disabled and pin the failed-job log-download paths in the workflow test. - Add an e2e workflow test guarding the cancel-in-progress expression that keeps in-progress runs on main from being cancelled. * fix(ci): bound the issue body and randomize the heredoc delimiter (#7795) * fix(ci): test the runCli --existing merge path (#7795) * fix(ci): address review feedback on e2e signal PR (#7795) - Assert the full cancel-in-progress expression including && so a mutation to || is caught by the e2e-workflow test - Filter the capped-summary line from missingTests so it is not rendered as a fake bullet under Also failing --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev-bot@users.noreply.github.com> Co-authored-by: qwen-code-ci-bot <qwen-code-ci-bot@users.noreply.github.com> Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com>	2026-07-28 11:54:52 +00:00
易良	183ad54b11	ci: add lightweight PR profiles (#6186) ... * ci: add lightweight PR profiles * fix(ci): harden lightweight profile classification * fix(ci): harden lightweight profile edge cases * fix(ci): report classifier invocation failures * fix(ci): tighten docs-only path matching * ci: restrict light profiles to same-repo PRs --------- Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>	2026-07-03 07:40:25 +00:00