codeburn/tests
AVSR Pavan Kumar b6bf233f46
feat(act): defer-* plan kinds for native deferral config (part 2 of #614) (#631)
* feat(optimize): mcp-deferral-gaps finding family

Add three read-only detectors for MCP tool-def deferral coverage gaps:

- mcp-deferral-off: sessions carrying MCP tool-def overhead with zero
  ToolSearch invocations and no deferred_tools_delta inventory across
  the window. Cause attribution, each with its own message and fix:
  stale ENABLE_TOOL_SEARCH=false (settings env, any scope, or shell
  profile), non-first-party ANTHROPIC_BASE_URL (reported as unknown
  proxy — never assumes capability), Vertex AI config, all observed
  Claude Code versions predating default-on tool search (v2.1.7), or
  a generic deferral-appears-inactive fallback.
- mcp-alwaysload-hygiene: alwaysLoad-pinned servers whose observed
  call rate is below 1 call per 5 sessions (named constant). Notes the
  5s startup-blocking cost of alwaysLoad in the explanation.
- mcp-defer-threshold: ENABLE_TOOL_SEARCH=auto/auto:N overrides whose
  threshold the estimated def volume never reaches, so tools load
  upfront; recommends the tightest auto:N that would defer.

Detection only: no FindingApply payloads, no act-layer changes, no
file mutation. Users without deferral gaps see zero new output —
deferral-active evidence (ToolSearch calls or inventory) suppresses
the findings, and each detector gates on conservative named
thresholds. Config readers take an injectable homeDir (PlanContext
style) for hermetic tests.

Discrepancies vs the design notes, codebase/live-docs win:
- The Claude Code changelog records default-on (2.1.7), not first
  ship, so the version cause uses the default-on boundary.
- Per-tool "anthropic/alwaysLoad" lives in server-served tool _meta,
  not static config; server-level only, limitation documented.
- Live docs confirm auto default threshold 10% of context window and
  the alwaysLoad v2.1.121+ requirement.

Refs getagentseal/codeburn#614

* feat(act): defer-* plan kinds for native deferral config

Wire the mcp-deferral-gaps findings to the act machinery with three
new plan kinds, all through the existing ConfigDocs/runAction path
(journaled, backed up, stale-guarded, dry-run previewable, undo
restores byte-identical files):

- defer-enable: removes a stale ENABLE_TOOL_SEARCH=false from the
  settings env of the scope where the finding recorded it. Refusal
  paths render as manual notes instead of plans: shell-profile lines
  (codeburn only appends marker blocks to shell files, never edits
  user lines), unknown proxies (setting the override blind makes
  requests fail outright on proxies that don't forward tool_reference
  blocks — the note says to verify first), Vertex, and old versions.
  A proxy-verified cause (set by the part-3 verifier) produces a real
  ENABLE_TOOL_SEARCH=true plan in user settings.
- defer-alwaysload: strips alwaysLoad: true from the named servers in
  the exact config files the finding recorded. Gated on an injectable
  installed-version probe (default: claude --version); below v2.1.121,
  unparseable, or probe failure all refuse with a note naming the
  required version. Preview notes the removed up-to-5s startup block.
- defer-threshold: rewrites the auto override to the recommended
  auto:N, or deletes it when the finding says the default already
  defers (removeOverride).

Findings now carry FindingApply payloads (path, scope, cause, servers,
recommended N); detector text is unchanged, so plain optimize renders
byte-identically to before. Every plan links its findingId into the
ActionRecord and states that changes take effect on the next session
(the config is read at Claude Code start).

Discrepancy vs the design notes: no existing version-check helper was
found in guard/ or act/ to reuse, so the detector's parseVersion/
versionPredates are exported and shared instead of adding a parallel
comparator.

Refs getagentseal/codeburn#614

---------

Co-authored-by: AgentSeal <hello@agentseal.org>
2026-07-16 19:01:54 -07:00
..
fixtures Merge origin/main into feat/desktop-app 2026-07-16 07:52:28 -07:00
providers copilot: extract skill invocations so Skills & Agents populates (#654) (#697) 2026-07-16 12:02:34 -07:00
security fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
setup feat: add CodeWhale provider support (#674) 2026-07-16 03:12:28 -07:00
sharing feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
act-journal.test.ts fix(guard): dedupe streaming message copies in the incremental cost fold 2026-07-03 12:32:46 +02:00
act-model-defaults.test.ts feat: implement act apply-model and report baseline tripwire (#607) (#616) 2026-07-16 11:10:08 -07:00
act-report.test.ts feat: implement act apply-model and report baseline tripwire (#607) (#616) 2026-07-16 11:10:08 -07:00
act-undo.test.ts fix(act): support directory moves for archive actions 2026-07-03 10:14:48 +02:00
antigravity-statusline.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
audit-report.test.ts feat: add codeburn audit token-source view (#578) 2026-07-01 13:02:29 +02:00
bash-commands.test.ts fix(bash): attribute wrapped commands to the real tool, not the wrapper (#658) 2026-07-10 06:36:07 -07:00
blob-to-text.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
budget.test.ts feat(budget): spend budgets with overview display and scriptable --check (#562) 2026-07-16 11:10:16 -07:00
cache-persistence-coldstart.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
classifier.test.ts File-aware retry detection with typed ToolCall (#379) 2026-05-22 01:21:42 -07:00
cli-budget.test.ts feat(budget): spend budgets with overview display and scriptable --check (#562) 2026-07-16 11:10:16 -07:00
cli-date.test.ts fix(web): reject invalid dashboard periods without exiting (#554) 2026-06-28 19:35:46 +02:00
cli-deepseek-v4-pricing.test.ts fix: DeepSeek v4 Claude pricing through stale runtime cache (#367) 2026-05-21 00:34:53 -07:00
cli-devin-model-variants.test.ts fix(devin): report friendly GPT model names with effort tier (#585) 2026-07-01 16:01:49 +02:00
cli-emitters.test.ts feat(cli): add compare and sessions JSON emitters 2026-07-11 12:16:45 -07:00
cli-export-date-range.test.ts feat(export): support custom date ranges 2026-05-05 23:18:48 -07:00
cli-json-daily.test.ts Add multi-day calendar selection with custom popover UI (#393) 2026-05-24 09:49:59 -07:00
cli-model-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
cli-plan.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
cli-price-override.test.ts feat(pricing): add user price overrides for models (#390) (#560) 2026-06-28 19:07:29 +02:00
cli-provider-validation.test.ts fix(cli): validate --provider and make non-TTY report deterministic (#501) 2026-06-18 12:39:22 +02:00
cli-proxy-path.test.ts test(cli-proxy-path): raise timeout for spawn-based tests to fix load flake (#504) 2026-06-18 12:57:44 +02:00
cli-settings-json.test.ts feat(cli): price-override --list --format json 2026-07-16 03:45:36 -07:00
cli-status-menubar.test.ts feat(cli): scan progress protocol, incremental cache saves, --no-timeline 2026-07-16 10:51:18 -07:00
codex-cache-invalidation.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
codex-credits.test.ts feat(codex): compute Codex credit usage (#408, #495) (#510) 2026-06-18 17:03:46 +02:00
compare-stats.test.ts fix(cli): compare cache-hit rate excludes cache writes 2026-07-16 01:14:41 -07:00
content-utils.test.ts fix(parser): tolerate string message content; isolate per-file parse failures (#441) (#450) 2026-06-06 04:01:12 +02:00
currency-rounding.test.ts fix(currency): resolve RON symbol to "lei" across all surfaces 2026-07-06 03:43:03 +03:00
daily-cache.test.ts fix: audit batch — prefetch one-shot, turn-anchored day bucketing, warm-path collapse 2026-07-16 15:37:03 -07:00
dashboard.test.ts feat(dashboard): make Daily Activity history directly scrollable (#672) 2026-07-16 04:10:08 -07:00
date-range-filter.test.ts Add multi-day calendar selection with custom popover UI (#393) 2026-05-24 09:49:59 -07:00
day-aggregator-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
day-aggregator.test.ts fix: audit batch — prefetch one-shot, turn-anchored day bucketing, warm-path collapse 2026-07-16 15:37:03 -07:00
defer-plans.test.ts feat(act): defer-* plan kinds for native deferral config (part 2 of #614) (#631) 2026-07-16 19:01:54 -07:00
devices-json.test.ts fix(cli): surface mDNS scan errors, robust spend-flow rollup + date label, harden tests 2026-07-10 16:20:41 -07:00
doctor.test.ts feat(doctor): per-provider detection diagnostics (#685) 2026-07-16 10:38:08 -07:00
export.test.ts feat(export): subagentType and model on per-record exports; --by-agent follow-ups (#712) 2026-07-16 16:35:33 -07:00
fetch-utils.test.ts fix(network): add timeouts to critical-path fetches (#445) (#448) 2026-06-06 03:25:36 +02:00
format-mark-estimated.test.ts fix(report): surface estimated costs distinctly (#688) 2026-07-16 10:46:21 -07:00
fs-utils.test.ts Raise streaming session cap to 4GB so large Codex sessions are counted (#569) 2026-06-26 01:16:56 +02:00
granular-history.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
guard-hooks.test.ts feat(claude): account for /advisor escalations under the advisor model (#659) 2026-07-16 07:42:49 -07:00
guard-install.test.ts fix(guard): dedupe streaming message copies in the incremental cost fold 2026-07-03 12:32:46 +02:00
hydration-interrupt-convergence.test.ts fix: hydration completeness markers, cold-only splash, honest switching, prefetch 2026-07-16 14:05:11 -07:00
kiro-cache-invalidation.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
local-model-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
mcp-coverage.test.ts Add MCP project profile advisor (#356) 2026-05-24 01:57:59 -07:00
mcp-deferral.test.ts feat(act): defer-* plan kinds for native deferral config (part 2 of #614) (#631) 2026-07-16 19:01:54 -07:00
mcp-redact.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
mcp-server.test.ts feat(mcp): get_usage + get_savings tools with annotations, schemas, coalescing 2026-06-02 02:16:10 -07:00
mcp-tables.test.ts feat(mcp): markdown table renderers for usage and savings 2026-06-02 02:13:51 -07:00
menubar-installer.test.ts Fix menubar install release lookup 2026-07-08 21:36:20 +02:00
menubar-json.test.ts Merge origin/main into feat/desktop-app 2026-07-16 07:52:28 -07:00
menubar-report-parity.test.ts fix: hydration completeness markers, cold-only splash, honest switching, prefetch 2026-07-16 14:05:11 -07:00
menubar-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
minimax.test.ts chore(pricing): refresh LiteLLM snapshot; MiniMax-M3 standard tier 2026-07-02 06:17:01 +02:00
model-efficiency.test.ts feat(report): add per-model efficiency metrics 2026-05-05 23:36:59 -07:00
models-hoist.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
models-report.test.ts feat(export): subagentType and model on per-record exports; --by-agent follow-ups (#712) 2026-07-16 16:35:33 -07:00
models.test.ts fix(report): only exact zero-rate overrides suppress the unpriced flag 2026-07-09 21:22:23 +02:00
optimize-apply.test.ts fix(act): refuse to apply a plan whose target changed since it was built 2026-07-03 11:20:59 +02:00
optimize-fs.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
optimize-zero-edit-savings.test.ts optimize: bound zero-edit low-worth savings estimate to a recovery fraction (#640) (#691) 2026-07-16 11:09:44 -07:00
optimize.test.ts optimize: bound zero-edit low-worth savings estimate to a recovery fraction (#640) (#691) 2026-07-16 11:09:44 -07:00
otel-cache-aggregation.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
overview.test.ts feat(budget): spend budgets with overview display and scriptable --check (#562) 2026-07-16 11:10:16 -07:00
parser-advisor-usage.test.ts feat(claude): account for /advisor escalations under the advisor model (#659) 2026-07-16 07:42:49 -07:00
parser-antigravity-timestamp.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
parser-claude-cwd.test.ts Show real Claude project leaf names; stop stray-.git over-grouping (#540) 2026-06-21 22:42:19 +02:00
parser-compact-entry.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
parser-estimated-cost.test.ts fix(report): surface estimated costs distinctly (#688) 2026-07-16 10:46:21 -07:00
parser-filter.test.ts test: cover filterProjectsByName include/exclude semantics 2026-04-16 15:49:57 -07:00
parser-gemini-cache.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
parser-hydration-lock.test.ts fix: audit batch — prefetch one-shot, turn-anchored day bucketing, warm-path collapse 2026-07-16 15:37:03 -07:00
parser-large-json-scanner-parity.test.ts parser: dedupe the twin string/Buffer large-JSONL scanners (#698) 2026-07-16 12:23:06 -07:00
parser-large-json-scanner.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
parser-large-session.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-local-savings.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-mcp-inventory.test.ts feat(optimize): MCP tool coverage detector with cache-aware costing 2026-05-05 04:13:04 +03:00
parser-number-helpers.test.ts feat: add devin provider (#444) 2026-06-09 21:58:31 +02:00
parser-proxy-codex-only.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-proxy-merge.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-proxy-pricing.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-skip-line.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
parser-subagent-collection.test.ts feat(report): Claude-scoped agent-type breakdown 2026-06-10 11:45:28 +02:00
parser.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
plan-usage.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
plans.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
pricing-fallback-data.test.ts feat(pricing): automatic gap-fill from models.dev and OpenRouter (#457) 2026-06-09 21:17:23 +02:00
provider-registry.test.ts feat: add CodeWhale provider support (#674) 2026-07-16 03:12:28 -07:00
provider-turn-grouping.test.ts kiro: preserve credit-based cost through the session cache 2026-07-14 22:35:51 +00:00
scan-progress.test.ts fix(parser): keep scan-progress out of the interactive dashboard and compare TUIs 2026-07-10 00:50:35 +02:00
session-cache.test.ts feat(app): refresh cadence, instant switching, cache coherence, telemetry v1 2026-07-16 11:47:05 -07:00
sessions-report.test.ts feat(cli): add compare and sessions JSON emitters 2026-07-11 12:16:45 -07:00
spend-flow.test.ts fix(cli): surface mDNS scan errors, robust spend-flow rollup + date label, harden tests 2026-07-10 16:20:41 -07:00
sync-e2e.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync-headless-e2e.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync-infra-e2e.test.ts test(sync): unit, mock-IdP e2e, and developer-only integration tests 2026-07-12 16:05:34 +00:00
sync-ledger-otlp.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync-push.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
token-estimate.test.ts Extract shared token estimation helper, migrate all CHARS_PER_TOKEN sites (#644) (#677) 2026-07-16 11:58:22 -07:00
usage-aggregator.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
web-dashboard.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
yield-overlap-attribution.test.ts yield: single-owner commit attribution, ambiguous bucket, heuristic caveat (#692) 2026-07-16 13:09:26 -07:00
yield-repo-grouping.test.ts fix(yield): group commit attribution by canonical repo, not launch path 2026-07-16 18:58:49 -07:00
yield.test.ts yield: single-owner commit attribution, ambiguous bucket, heuristic caveat (#692) 2026-07-16 13:09:26 -07:00