mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-08-06 23:35:34 +00:00
ac67de2e46
* fix(review): stop the reverse-audit loop while there is still time to report Measured on CI run #8368 (+1699 lines): the iterative reverse audit ran to its 5-round cap, each round a per-chunk fan-out whose findings then went back through verification, and the loop consumed 3.5 of the job's 4 budgeted hours. The outer GNU-timeout kill arrived while round 5's findings were still being verified. The review died holding every confirmed finding it had; nothing reached the pull request. The loop's rounds are driven by the orchestrator, but every round begins at the same place: agent-prompt building the round's prompts. So the builder becomes the loop's clock. When the environment carries a review deadline (QWEN_REVIEW_DEADLINE_EPOCH, exported per attempt by the review workflow) and the remaining time is inside the reserve kept for the last verification, compose-review and submission (default 60 minutes, QWEN_REVIEW_DEADLINE_RESERVE_SECONDS to override), a reverse-audit round is refused: a BUDGET line on stderr, exit code 4, no prompt built and no record written. The message carries the exact unreviewedDimensions entry to file, so the disclosure that caps the verdict is the CLI's text, and Step 6 proceeds with the findings already confirmed. Local runs have no deadline and are untouched. A malformed deadline fails open — the outer kill still bounds the run, and a broken variable must degrade to today's behaviour rather than wedge every budgeted review at round 1. The verifier is deliberately not gated: the reserve exists so it can run. * fixup: scale the deadline reserve to the externally-chosen budget The budget is not this workflow's to assume: it arrives from a repository variable, a workflow input, or a /review --timeout=N comment. A fixed 60-minute reserve would consume most of a 70-minute budget and refuse the audit loop outright on a 30-minute one. The workflow now passes a reserve of a quarter of the attempt, floored at 10 minutes and capped at 60; the CLI constant remains only the fallback for a caller that sets a deadline without a reserve. * review feedback: admit the round only if IT fits, and cap deterministically Three findings from review, all taken: 1. The gate budgeted for the tail but not for the round it admits — the terminal round is by construction the one that starts closest to the boundary, so the killed-mid-verification failure survived one round wide. The gate now requires remaining >= round + reserve, where the round's cost is the previous round's, measured admission-to-admission from a stamp the builder writes (one per round; a same-round rebuild is not a round), falling back to a 30-minute constant for round 1, which starts with the most headroom. 2. The refusal was deterministic; the disclosure that caps the verdict was prose the orchestrator had to carry. The builder now records a budget-stop marker beside the prompt records and compose-review synthesizes the unreviewedDimensions entry from it — deduped against a relayed copy — so a run that drops the sentence still cannot approve past a truncated audit. 3. Exit code 4 is documented in the command's describe. Also restores the Step 5 bullet the previous commit's edit displaced (new findings merge into the cumulative list before the next round). * review feedback: pin the budget gate's all-chunks refusal and ordering Cover the two behaviours the review noted were only asserted on the bare --findings form: an exhausted budget refuses the loop's real --all-chunks round before ANY of the per-chunk records is written, and a malformed call (--round 0) still gets its validation error first — exit 4 is for a well-formed round the budget refuses, never a replacement error. Also name what the code already does: reserve=0 is the deliberate escape hatch (the gate shrinks to the round estimate alone), and the workflow's 3600s cap mirrors DEFAULT_RESERVE_SECONDS. * docs(review): describe the soft-deadline env vars for time-budgeted runs The review noted the two new variables appeared in no user-facing doc; the reserve in particular is an operator-facing knob. State what each does, the fail-open posture, and how the refusal surfaces in the verdict. * fix(cli): align budget-stop disclosure with the gate's refusal (#8468) A round-1 budget refusal left no reverse-audit records, so the Step 4/5 floor reported the deliberate stop as a rogue/unlaunched audit with a rebuild FIX the same gate deterministically rejects; the refusal's own disclosure was swallowed by the caller-echo dedup. The floor now stands down when the budget-stop marker exists, and compose-review renders the disclosure structurally, bilingually, from the marker. Also: `--role reverse-audit` requires `--round <k>` (an unlabeled admission stamps an entry no estimate can attribute), the budget gate runs after the plan/findings reads (a broken plan or unreadable findings deserves its own error, and nothing is stamped ahead of a buildable call), and the gate's admission boundary, measured-cost behaviour, and the workflow env contract are pinned by tests. * review: a budget stop excuses only the round it refused The budget-stop suppression keyed on the marker's existence alone, so every reverse-audit gap shape went silent once any round was refused — including the shapes that describe rounds which RAN before the budget hit. A hand-written round-1 launch is exactly as undelivered when round 3 later hits the budget, and suppressing its disclosure let 'stopped before round 3' imply the rounds that did run were faithful. Exactly one shape is by design under a marker: not-built — the refusal writes no record, so an audit with no records IS the audit the gate stopped, and its FIX (rebuild the round) would be refused by the same gate. The suppression now names that shape and no other; a rewritten, unlaunched or brief-unread round keeps its disclosure and its repair. The new test pins the operative halves: the verdict stays capped, the marker's disclosure posts, and the operator channel carries the rewritten round's exact repair. (The posted body collapses same-subject disclosures — both say 'reverse audit' — so the author sees the stop; repairs are acted on from stderr, where the rewritten fix rides.) * fix(review): fence budget state per run, and let gate errors beat budget stops Address the round-2 review threads on the reverse-audit budget gate: - Fence budget-rounds.json and budget-stop.json by the plan's own mtime. Every run rewrites the plan at its Step 1 capture, so records older than the plan belong to a previous run of the same PR: a run killed before cleanup no longer prices the next run's rounds off stale stamps (an hours-old stamp read as an hours-long round refused round 1 of a fresh budget) and no longer caps a later run's verdict on a stop that did not happen in it (R2-1, R2-2). - Refuse a structurally unbuildable plan (no chunks[], duplicate or non-integer ids) with its own error ahead of the budget gate, so the same corruption gets the same diagnosis whatever the clock says, and no budget-stop marker is written over a corrupt plan (R2-5). - Stamp a round admitted only after its build succeeds: a build that throws leaves no stamp, so the next round's cost is never measured from a build that produced nothing and floored to 600s (R2-6). - Keep the budget entry's 'reverse audit' subject out of the caller-echo prefix filter: other reverse-audit scopes the orchestrator disclosed (a twice-whiffed chunk from the rounds that DID run) are no longer silently dropped in the marker's shadow; the marker's own relays stay deduped by the phrase splice (R2-7). - Render --round unbracketed in the reverse-audit rebuild fix — the CLI refuses a round-less reverse-audit call, so the paste-and-run repair must not present the flag as optional (R2-14). - Document the deliberate one-verification overlap between the measured round estimate and the tail reserve, at both definitions (R2-13). - Test hardening, each assertion mutation-probed to fail its named mutant: a reshaped relay only the marker-phrase splice dedups (R2-8); the stamp's round label and the verifier's no-stamp invariant (R2-9); whole-line, unit-arithmetic and reserve-cap pins on the CI wiring contract (R2-10); the first-wins stamp survivor (R2-11); the reserve=0 escape hatch (R2-12). --------- Co-authored-by: verify <verify@local> Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> |
||
|---|---|---|
| .. | ||
| assets | ||
| design | ||
| developers | ||
| e2e-tests | ||
| images | ||
| plans | ||
| superpowers | ||
| users | ||
| verification/abort-controller-refactor | ||
| _meta.ts | ||
| index.md | ||