Commit graph

6 commits

Author SHA1 Message Date
iamtoruk
52eb9fdb8d optimize: re-measure applied fixes on every run, with --auto-revert
Every still-applied journal entry now comes back with a verdict on the
next optimize run: worked (>=70% of its window-scaled estimate
realized), partial, no-effect (printed with its undo command), or
measuring while it is younger than the 3-day window. The verdicts come
off the rows act report already computes, so there is one
reconciliation, not two; the AppliedFix type and its formatter live in
act/types.ts so the optimize renderer can use them without importing
report.ts back into optimize.ts.

--auto-revert undoes the no-effect entries through the same code path as
codeburn act undo. It never touches partial or measuring entries, and
never a claude-md-rule - those land in whatever directory the user
happened to be in, the same reason --yes skips them.

--apply now names when the re-measure happens, and --format json carries
appliedFixes[] (add-only).
2026-08-18 03:15:07 -07:00
ozymandiashh
db018f755f fix(act): dedup defer baselines against MCP rows, report pending honestly
deferRow now sums only the servers no applied mcp-remove / mcp-project-scope
record already measures, so a defer row and an MCP row can never claim the
same server's schema tokens over the same post-apply sessions and
totalRealizedTokens stays a disjoint sum. Conservative by design: the defer
row drops a claimed server for its whole window, and when every server is
claimed it reports not measurable instead of guessing.

deferredSessions === 0 now reports the new 'pending' status instead of
asserting 'reverted': the note already said the cause was ambiguous, and
--json consumers could not tell a not-yet-restarted client from a genuine
revert. The table renders it as 'not yet in effect'.
2026-08-04 02:22:47 +03:00
AVSR Pavan Kumar
77b41ce63b feat(act): measure realized savings for defer-* actions in act report
The defer-enable / defer-alwaysload / defer-threshold plan kinds (part 2
of the deferral-coverage work) applied and undid correctly but were
invisible to `act report` — it returned "not measurable: no baseline
captured at apply time", because report.ts never captured a baseline for
them or knew how to compute their realized delta.

Wire them in, mirroring the mcp-remove path since deferral has the same
effect (MCP tool-def schema leaves the upfront prefix):

- needsConfigBaseline + captureBaseline now cover the defer-* kinds.
  Servers are the named set for defer-alwaysload, or the observed MCP
  surface for defer-enable / defer-threshold (which re-enable deferral
  across everything). Per-session tokens use observed tool counts, or the
  5-tools x 400 fallback, exactly like mcp-remove.
- A new deferRow computes realized savings as per-session prefix tokens
  times the post-apply sessions where deferral actually became active —
  detected by the same deferred-tools-inventory signal the
  mcp-deferral-off detector uses. Sessions begun before the client
  restarted still run deferral-off and are excluded; if none benefited,
  the row reports "not yet in effect" with zero realized rather than
  claiming a saving that has not taken hold.

Records applied before this change (no baseline) keep the existing
"no baseline captured at apply time" note, so nothing regresses.

11 tests cover the measured, partial, not-yet-in-effect, no-sessions,
empty-baseline, and missing-baseline paths, plus baseline capture for
each kind. Verified live end to end: apply captures the baseline, and
act report reports realized savings scaled to the sessions that adopted
deferral.
2026-07-30 10:13:06 +05:30
ozymandiashh
491c9cbfc8
feat: implement act apply-model and report baseline tripwire (#607) (#616)
* feat: implement act apply-model and report baseline tripwire (#607)

* feat: implement act apply-model and report baseline tripwire (#607)

* fix(act): scope model-default tripwire to the applied project

Review fixes for #616:
- modelDefaultRow and the under-20-edit-turns gate now aggregate only the
  target project's sessions (derived from changes[0].path, separators
  normalized before dirname), matching the per-project baseline captured at
  apply time instead of comparing against all projects.
- baseline.candidateModel labels the candidate explicitly, with a
  backward-compatible fallback to metrics key order for existing journals.
- measured model-default rows render a correlation marker instead of a
  formatTokens(0) token claim.
- zero-matching-projects now reports an honest project-not-found note; clean
  rows route through confidenceFor like every other kind.
- tests: tripwire fires on a same-project regression that global aggregation
  would mask (fails on pre-fix code), clean and not-measurable cases,
  Windows-separator journal paths with an excluded masking project.
2026-07-16 11:10:08 -07:00
AgentSeal
fea66a80d3 fix(act): report scale, corrupt-journal safety, and under-claim gaps (#606)
Bug-hunt follow-ups on the realized-savings report:

- Scale the displayed estimate to the measured window so the Estimated and
  Realized columns are comparable: mcp/archive use the per-session baseline
  times the post-window session count, read-edit uses deficitThen times the
  same edits denominator as realized (so realized never exceeds it). The
  at-apply estimate stays in --json as estimatedAtApply next to
  estimatedForWindow; the footer states the scaling and that mcp/archive
  realized figures are derived from session counts, not independently
  measured.
- Never crash on a corrupt journal: records without a string status or a
  parseable string `at` are skipped and surfaced ("N malformed records
  skipped"); the optimize header computation is additionally wrapped so any
  error just drops the header.
- Zero post-window sessions now reads "not measurable: no sessions in the
  window yet" instead of measured-zero.
- The optimize header sums only normal-confidence measured rows (under-claim);
  low-confidence rows stay visible in act report only.
- Tests: floor discipline on non-integer mcp and read-edit products (a ceil
  mutation fails), the project-scope keeper subtraction, the archive
  estimate==realized tautology, malformed-journal robustness, and the
  low-confidence header exclusion.
2026-07-03 13:36:36 +02:00
AgentSeal
ad471f3d8c feat(act): realized savings measurement (codeburn act report) (#606)
Capture a trailing-14-day before-baseline when a fix is applied and
re-measure it against the post-apply window so optimize can show realized
numbers next to estimates.

- ActionBaseline (windowDays, capturedAt, estimatedTokens, sessions, metrics)
  persisted by runAction; captured in the optimize --apply flow and at guard
  install time.
- codeburn act report [--json]: applied, not-undone actions older than 3 days,
  re-running the detectors over apply-date-to-now (capped 30 days). Per-kind
  realized deltas: MCP/archive tokens-per-session times saved sessions with
  reverted-by-user detection; read-edit deficit reduction; guard yield split
  labeled correlation. Bash cap is marked not measurable (result sizes are not
  retained). Low confidence under 20 post-window sessions or past a 2x volume
  shift. Realized numbers rounded down, estimate kept visible.
- optimize gains one header line only when a measured action exists, and
  appends "(previously applied <date>, re-flagged)" to re-triggered findings.
  No change for users with no applied actions.

Reuses scanAndDetect helpers over a date-bounded range; exports the token
constants and read/edit tool sets rather than duplicating the math.
2026-07-03 13:04:37 +02:00