mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-08-04 13:51:13 +00:00
9bfe608be9
* feat(review): borrowed-verification trio — test-plan check, base-tree A/B, per-hunk probes - qwen review test-plan: rule on the PR Test Plan's checkable claims (paths, npm scripts, test counts) against the reviewed tree; contradictions and differing counts are disclosed via compose-review, never capping. - qwen review base-tree: build the merge base in a sibling worktree so the verifier can A/B a comparative claim instead of reading it; swept by cleanup. - test-efficacy: third probe kind — reverse-apply one hunk at a time and re-run the affected tests, attributing a still-green suite to the specific change nothing gates; shares the mutants' budget window, runs last. * fix(review): survive real runner output — ANSI-laced and trimmed-away summaries Both measured on a live /review of QwenLM/qwen-code#8176 with the built CLI: - test-plan's observedTestCounts strips SGR sequences before matching; a color-enabled pipe interleaves them BETWEEN tokens, and the count claim fell to 'unchecked' with the summary right there in the report. - build-test's trimOutput rescues runner summary lines from the omitted middle (like module-resolution errors): a failing suite's tail is all failure details and npm epilogue, which pushed the one-line summary out of the kept text entirely. * fix(review): address the eight findings from live review of this branch All measured in the review (QwenLM/qwen-code#8215 review comment): - test-plan: linear-time bold-heading scan (the old pattern backtracked catastrophically on an unclosed ** line an untrusted PR body controls); a flag preceding the npm script yields no claim instead of a false 'no package defines this script'. - test-efficacy: a hunk probe's restore recreates the parent directory a reverse-applied 'new file' hunk removed (the ENOENT from finally lost the verdict and marked every remaining hunk inconclusive); hunks get their own skippedForBaseline instead of mislabeling a red baseline as a budget skip; splitDiffIntoHunks re-captures the file header at every diff --git boundary; a hunk-survived finding notes when it restates an inert file-level revert at hunk granularity. - base-tree: idempotent fast path keyed on a build marker + HEAD check — concurrent verifier shards reuse one built tree instead of sweeping it out from under each other mid-A/B (a fabricated base-side difference with a deterministic source tag was the worst case); cost wording is now 'an install and a build' everywhere it was 'one extra build'. * fix(cli): never score a hunk survived when its own test left the baseline (#8215) A per-hunk probe reported `survived` whenever the green baseline probes still passed with the hunk reverted. When the hunk's own collocated test dropped out of the baseline (a probe-tree import error collects nothing), the remaining green probes prove only that THEY do not cover the hunk, so the verdict is now `inconclusive` — the same dropped-test asymmetry the mutants already hold. Also scope the hunk-survived cross-reference note to the hunk's own collocated test, and let test-plan match a workspace-scoped run of the plan's bare command instead of falling through to the manifest on an exact-string miss. * fix(review): silence-bias hardening from four live review rounds of this branch The two blocking findings, reproduced on this PR's own Test Plan: - test-plan files no false contradicted notes: npm rulings move from a four-verb denylist to an allowlist (the run form + npm's script aliases — the ~fifty other builtins each used to become 'no package defines this script'); a slash token is claimed as a repo path only with evidence (an extension or ./ prefix), never when it is a flag's value (--repo owner/repo) or under the review's own temp root; HEADING_LINE_RE drops the same quadratic shape its bold sibling was rewritten to remove. - base-tree gets a real mutual-exclusion lock around sweep+add+build (mkdirSync test-and-set; the loser returns busy instead of deleting the tree the winner is mid-install in), and a failed build writes a settled marker so later shards stop re-paying the install to relearn 'unavailable'. Also: Agent 7's brief now names hunk-survived and the hunks.* counters (it is the report's only consumer, and the finding class was invisible); hunk findings anchor at the first ADDED line instead of up to three context lines above the change. * feat(review): four round-2 borrowings — measured failure attribution, round ledger, richer mutants, doc parity Shaped by a live round-2 maintainer re-verification (QwenLM/qwen-code#7818): - qwen review test-delta: rerun the PR side's failed test commands on the built merge base and diff the failing FILE SETS — netNew is the PR's own failure by measurement (whatever files the diff touches), shared is pre-existing by measurement. Counts are never compared: a flaky suite fails different test names between runs of the same tree. An unfinished base rerun attributes nothing. - Round ledger: the incremental review cache persists confirmed findings under round-scoped ids (R1-2); a high-effort re-review rules on each (fixed / still stands / cannot tell) and opens its report with the table, the way a human round-2 comment opens with 'M1 is fixed'. - Three replacement mutation operators in test-efficacy: drop '?? fallback', force a comparison-bearing guard condition to 'true', drop a '+ CONST' term. Each survivor maps to one crisp untested-protection sentence. A line whose raw text and literal-blanked code view disagree yields no candidate — an edit index computed across the two views once spliced 'iftrue 0)' into a guard, and a mangled mutant reads as inconclusive while quietly spending a cap slot. - Quality brief: documentation-parity lens — a new user-facing surface whose siblings are documented is a Suggestion naming the sibling precedent; no documented sibling, no finding. * fix(review): address review feedback — false-positive hardening, binary diff guard, error convention (#8215) * fix(review): address review feedback — base-tree availability gate, test-plan false positives, hunk-probe ranges (#8215) - base-tree: only stamp a base tree available when runBuildTest actually compiled something (ok AND npm toolchain AND a non-empty build). An `unsupported` handoff or an empty npm scope returns ok:true having built nothing; marking that tree available let an A/B read the absence of a build as a behavioural difference. - cleanup: sweep the stale base-tree build lock a killed builder leaves behind. - test-plan: read the root manifest's scripts directly so a root-only script survives when the root defines no build/test; bail on the inline --root=./dir rebasing form; stop treating a positional after an inline --flag=value as the flag's value; prefer a failed scoped run when ruling a bare command; anchor the npm script alias to a full token so `yarn test:unit` is not truncated. - test-efficacy: exclude `\ No newline at end of file` from the startLine offset count; compute the mutant-overlap range from the header's new-side span so it no longer overshoots into a closely following hunk. * fix(review): address review feedback — diff-header false positives, stale prompt enumeration, added-file hunk probes (#8215) * fix(review): address review feedback — cd-base exclusion, Test Files count guard, base-tree error handling, probe delegation (#8215) * fix(review): port the collocated-dropout test to the post-#8050 runner seam Merging main brought #8050's Windows-portability refactor, which resolves the probe runner through vitest/package.json's bin — a node_modules/.bin fake is dead weight it never reads. The 8215-only collocated-dropout test still installed the old .bin fake, so the REAL vitest ran its fixtures, price.test.ts genuinely passed, and the hunk scored survived. The test now overrides the fake package's vitest.mjs like every post-refactor test. * fix(review): bound the summary rescue, apply the ATX heading rule, sweep stale build locks The three 8215-layer findings from the latest review, fixed at this layer (they were first patched further up the stack, where the reviewer of THIS PR cannot see them): - trimOutput's summary rescue is capped at 40 lines — uncapped, 40k lines of 'Test <n>: …' prose voided the trim entirely (measured 1.6MB in, 1.6MB out) and the bounded-output contract is the whole point. - A '#' with no following whitespace is prose, not a heading (the ATX rule GitHub applies): '#8176', '#tag', an unfenced '#!/bin/bash' no longer end the Test Plan section mid-body; the bare-#-run crash on the closing scan is guarded. - A base-tree build lock older than 30 minutes is a corpse left by a killed builder — swept and rebuilt instead of reporting busy for the rest of the review. * fix(review): EEXIST-only lock busy, bun test alias, chained cd bail, fence backreference Four live findings from the latest inline review round (the rest of the round was already fixed upstream by the takeover bot - verified by probing head behavior rather than re-reading the threads): - base-tree's lock catch distinguishes EEXIST (a concurrent builder, busy) from EPERM/EROFS/ENOSPC (this run's own failure, reported as such, not as a busy that will never clear). - "bun test" is bun's built-in runner, not a package-script alias: it runs whether or not any manifest defines test, so ruling it against the scripts table filed a false contradicted. - A chained cd matches the leading-cd shape but the single-hop resolver joined file tokens against the FIRST directory; it now bails like the exotic-cd case. - codeSpans' fence regex closes on its own marker via backreference; a tilde fence line inside a backtick block ended the span early and lines after it were lost to extraction. * fix(review): close the ten open findings on this PR - guard-true tested for a comparison anywhere after `if (`, including the then-body, so `if (ready) emit(a !== b);` admitted a mutant on the comparison-less condition the gate exists to exclude. It now tests the condition span only. - The `survived` detail said "when it changes" for legacy DELETION mutants too; it now matches the operator. - test-delta's `unparsed` required both sides to parse zero files, so a PR-side failure whose FAIL lines the trim scattered was silently dropped whenever the base rerun happened to parse. netNew/shared come from the PR side, so the PR side alone decides. - failingFilesOf now matches Windows path shapes (backslashes, C:) - a missed parse is an unattributed failure, not a loud error. - The replacement branch of runOneMutant (write-file -> run-probe -> classify) had no end-to-end test; one now drives a coalesce operator through the real handler and asserts the mutated line, the verdict, the operator-specific wording, and that the shared tree is untouched. - Two tests were vacuous with respect to what they promised: the baseline-dir test never asserted the cwd (its helper swallowed the argument - fixed at the helper), and the one-candidate-per-line test used an input that never triggered the replacement path, so the `continue` under test was not load-bearing. - Reattached the orphaned selectMutants JSDoc; reworded the SKILL line. * fix(review): the test helper's cwd parameter is required, not optional CI's `tsc --build` failed on test-delta.test.ts: the exec seam always passes a cwd, but the helper's signature marked it optional, so pushing it into a string[] was `string | undefined`. Missed locally because vitest runs through esbuild, which strips types without checking them - the suite was green while the build was red. The gate to run before pushing a type-level change is `npx tsc --build`, not the test suite. * fix(review): a base rerun that could not RUN attributes nothing Two Criticals from the latest review, both reachable on the brief's own happy path: - baseUnusable covered only timeouts. Every other way the base side can fail to run - an unbuilt base tree, a missing install, a workspace the PR ADDED (npm test --workspace cannot resolve on base), an ENOBUFS truncation - exits non-zero with zero FAIL lines, which this code read as a green base. Every PR-side failure then became netNew: the strongest evidence the command emits, manufactured from a base that never ran a test. It now attributes nothing and says why. - Timeout detection was the weaker substring form the sibling explicitly rejects; an external SIGTERM (container stop, cancelled job) set neither an ETIMEDOUT message nor an exit code and fed straight into the above. build-test now exports spawnTimedOut and test-delta asks the same question rather than re-deriving it. Also: the base output is trimmed (it precedes the verdict fields in the report the agent reads, so an untrimmed megabyte truncates exactly what the command produces); the guard-true gate no longer reads an arrow function's => as a comparison (every predicate guard was a candidate - the if (ready) noise the gate exists to exclude); the term-drop message no longer calls a string concatenation a reserve term; the unparsed note describes its own PR-side-only condition; and the ledger's Step 6/Step 8 now agree that a still-standing finding keeps its id. * fix(review): compare failing files by a normalised, project-keyed identity Critical: the two sides run in DIFFERENT roots (the PR worktree and the base tree), and netNew/shared compared the parsed paths verbatim - so an absolute-path runner turned every pre-existing failure into a fabricated Critical, with the authority of a measurement behind it. Paths are now normalised against each run's own root (and backslashes to /, so a Windows path compares with its POSIX-printed twin), which is why test-delta gained --pr-worktree. The identity also keeps the vitest project token: dropping it collapsed same-named test files across workspaces, so a PR-caused failure in one package could read as pre-existing because another package has a file by the same name - the worse failure direction. Also from the same review, all of them reachable on the brief's own path: - The base rerun now inherits build-test's stdio: ['ignore','pipe','pipe'] ("a build that asks a question is a build that hangs until the deadline") and its trimOutput, which matters because entries[].base precedes the verdict fields in the report the agent reads. - The brief gates on base-tree's `available`, not just its `path`: a tree that was created but did not build populates path too, and measuring against it turns an infrastructure failure into Criticals. - A programmatic caller omitting `timeout` no longer sends NaN into spawnSync. - MutantCandidate is a discriminated union, so an operator without its replacement line - which would delete a line while reporting "with its ?? fallback dropped" - is unrepresentable. - The comparison class no longer requires a trailing space (if (a<b) is the same guard, just unformatted) and matches a brace-less else if. - DeltaEntry.unparsed's doc now describes the PR-side-only condition it actually implements. * fix(review): restore the whole-command budget, keep generics out of guard-true Round-1 findings from a fresh review of this PR: - test-delta had no aggregate deadline: --timeout is PER command and defaults to 300s, so three failed commands is 900s against Agent 7's 600s ceiling - killed with NO report at all, discarding the base-tree install and build just paid for. TOTAL_BUDGET_MS mirrors the one test-efficacy reserves; commands it cannot fit are disclosed. - guard-true matched generic calls: `if (isRecord<string>(v))` produced a mutant, and a type-guard predicate is exactly the `if (ready)` shape whose survivors the gate calls noise. The trailing \s is required, not an accidental asymmetry with [!=]== - telling `a<b` from `fn<T>(x)` needs a parser, and the gate is silence-biased by design. - --pr-worktree had no contract test, and its failure mode is the worst here: arriving undefined, root stripping silently stops and every pre-existing failure becomes a fabricated netNew. The new test feeds parseSync's output straight into runTestDelta and asserts an attribution only reachable when both roots were stripped (verified red against the snake_case field shape that shipped once already). - Merged the two consecutive doc comments on prWorktree. * fix(review): a budget-shortened deadline is not the same fact as a slow rerun Round-2 finding on the budget just restored: `Math.min(perCommandMs, remaining)` can hand a rerun far less than --timeout, and if it dies there the note said only "timed out - infrastructure, not evidence". True, but it sends the reader hunting a hang that is really an exhausted budget - and unlike a real timeout, a rerun with budget to spare would still measure it. The note now names those commands separately and says so. Verified red against removing the tracking line. * fix(review): restore the replacement sub-cap, stranded on a downstream branch Round-3 finding, and the third instance of one class: an 8218-layer fix committed on the 8261 branch, four PRs above the code it belongs to. Measured over 40 real commits, the replacement operators produce ~24x the deletion pool (215 vs 9 candidates; guard-true drives it). Every mutant run drains the same window hunk probes draw from LAST, so uncapped, most diffs with any replacement candidates leave hunk probing zero runs - the hunk-survived finding class silently stops firing and nothing says so. Three slots, and what the sub-cap drops is counted in skippedForCap rather than lost. Also swept the other direction: diffed every review file against its 8261 copy to confirm nothing else 8218-layer is stranded up there. The remaining divergence is 8261's own (the positive control, its lenses). * docs(review): complete the "delta cannot rule" enumeration in both places Round-4 finding. The brief and SKILL.md each listed three cases where test-delta attributes nothing - unparsed, timed-out base, no merge base - but the code has five: the later rounds of this PR added "a base rerun that failed without naming any failing file" (it did not measure the base) and "a command the whole-command budget could not fit". Two enumerations of the same set with different membership, in the two places an agent reads. That is the sibling-enumeration lesson this skill teaches, applied to its own prose for the second time: the fix is not just adding the missing members but saying that the report names each case with its own reason rather than folding them into one. * docs(review): the rationale named only one of the ways base goes unmeasured Third and last copy of the enumeration the previous commit fixed. The bullet's headline already generalised - "base attributes nothing it did not finish" - but its body named only the timeout, so a reader learning the contract from the rationale would conclude timeouts are the only unusable case. Name the set, and say why the report keeps the reasons apart: "we could not measure" and "we measured nothing" are different facts to the author, and only one of them is about their PR. * fix(review): a file-count label stops counting at the end of its line `Test Files 45 passed` filing its 45 as a differing TEST count was fixed once, with a lookbehind on the bare-count pattern. That only ever rejected the all-green shape. The moment any file fails, the runner prints `Test Files 1 failed | 44 passed (45)`, the label is no longer adjacent to the number, and 44 comes through as a test count - so the note reads "claimed 44, observed 1323" on exactly the runs whose summary someone would paste. Adjacency was the wrong invariant; the line is. Masking from the label to end-of-line is distance-independent and picks up jest's `Test Suites: 1 failed, 44 passed, 45 total` at no cost. The label keeps its `Test` word on purpose: a first cut matched a bare `files` and blanked the line in "expect all four files and 471 tests to pass", silencing a real claim. An existing test caught that, which is the argument for the rule being as narrow as it is - anything that suppresses claims is worth exactly its narrowness. Also drops the now-dead lookbehind, which a reviewer had separately (and wrongly) called a JS syntax error; variable-length lookbehind is legal in V8, and the module parsed fine. It goes because the line rule subsumes it, not because it was broken. * fix(review): parse the base rerun before its output is trimmed `trimOutput` rescues module errors and runner summaries out of the omitted middle, not the per-file FAIL lines this command reads. A base suite whose failure section overruns the tail budget therefore lost failing files into the gap - and a SHORT base set is the dangerous direction, because netNew is the PR side minus the base side. Every file the trim hid came back as a Critical attributed to this PR by "measurement". Parse the raw text, report the bounded one. The PR side cannot be fixed here: it is read out of build-test's stored output, trimmed before this command existed. That loss runs the other way - it understates `shared`, never invents a netNew - so the entry carries `prTruncated` and the note says the list may be partial. A silence-biased gap is still a gap the author should hear about. Also names both selection caps in the mutant-skip diagnostic. The count accumulates replacement sub-cap drops, and with 2 deletions and 6 replacements the total is exactly MAX_MUTANTS: the main cap never fires, yet 3 are dropped, and the message sent the reader looking for a pool of 11 candidates that never existed. * fix(review): rerun only the command shapes build-test emits This command reads a report off disk and then executes the strings in it with `shell: true`, in the base worktree. Nothing else in the pipeline re-executes a value it read back from a file, so nothing else has to care where that value came from - this does, and the provenance is worse than it looks: the command is `npm test --workspace="<dir>"`, the workspace token is a directory, and a directory is a name a pull request chooses. `packages/x";curl …|sh;"` is a legal path in git and on Linux and it round-trips through the report into a shell. Restricting to the emitter's own grammar costs nothing real, since that grammar is the two shapes build-test produces. A command outside it is skipped and disclosed, the same treatment everything else this command cannot do already gets, so a future shape degrades to "judge it by the diff" rather than to arbitrary execution. --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code <qwen-code@users.noreply.github.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com> Co-authored-by: Qwen Code Autofix <qwen-code-autofix@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| src | ||
| .gitignore | ||
| index.ts | ||
| package.json | ||
| test-setup.ts | ||
| tsconfig.json | ||
| vitest.config.ts | ||