codeburn/tests
Resham Joshi 98051b4263
fix(copilot): bill CLI input and cache tokens from session.shutdown (#684)
* fix(copilot): bill CLI input and cache tokens from session.shutdown

Copilot CLI sessions (~/.copilot/session-state/*/events.jsonl) recorded
only outputTokens, because the parser read assistant.message events and
ignored the session.shutdown rollup. That rollup is the sole on-disk
source of a CLI session's input, cache-read and cache-write tokens, so
cost was underreported (in the reported case 35k+ cache-read tokens were
dropped entirely).

Handle session.shutdown in the CLI (non-transcript) JSONL path and emit
one supplementary call per model from modelMetrics.<model>.usage. The
per-turn assistant.message events keep owning output tokens, tools and
turn structure; the shutdown call contributes only the input and cache
tokens they lack, with output excluded, so no dimension double-counts and
the combined cost equals the full CLI-measured cost. usage.inputTokens is
cache-inclusive (input + cache_read + cache_write), so the cache
components are subtracted to recover the uncached input calculateCost
expects. These are real counts written by the CLI, so costIsEstimated is
false. Sessions without session.shutdown (crashed or still running) keep
the exact previous output-only behavior, and the VS Code, JetBrains and
OTel paths are untouched.

Bump the copilot entry in PROVIDER_PARSE_VERSIONS so already-cached CLI
sessions re-parse and pick up the corrected tokens and cost.

* fix(copilot): never emit an empty-timestamp shutdown call

A session.shutdown without its own timestamp and without sessionStartTime
produced a call with an empty timestamp, which the date-range filters in
parser.ts drop silently, erasing exactly the tokens this feature bills.
Fall back to the last stamped event in the session.

* fix(cache): durable orphans survive provider fingerprint changes

Bumping a durable provider's PROVIDER_PARSE_VERSIONS changed its env
fingerprint, and getOrCreateProviderSection replaced the whole section on
mismatch. For copilot OTel orphans the discarded cache entry is the only
remaining record of pruned spans, so the first post-upgrade run silently
and permanently erased month-to-date history. Carry forward exactly the
entries whose source no longer exists; present sources still drop and
re-parse under the new fingerprint, which is the point of a bump.

Regression test forges a pre-upgrade fingerprint on a pruned-source
orphan and asserts it stays counted and cached across the transition.
2026-07-16 10:37:58 -07:00
..
fixtures Merge origin/main into feat/desktop-app 2026-07-16 07:52:28 -07:00
providers fix(copilot): bill CLI input and cache tokens from session.shutdown (#684) 2026-07-16 10:37:58 -07:00
security fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
setup feat: add CodeWhale provider support (#674) 2026-07-16 03:12:28 -07:00
sharing feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
act-journal.test.ts fix(guard): dedupe streaming message copies in the incremental cost fold 2026-07-03 12:32:46 +02:00
act-report.test.ts fix(act): report scale, corrupt-journal safety, and under-claim gaps (#606) 2026-07-03 13:36:36 +02:00
act-undo.test.ts fix(act): support directory moves for archive actions 2026-07-03 10:14:48 +02:00
antigravity-statusline.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
audit-report.test.ts feat: add codeburn audit token-source view (#578) 2026-07-01 13:02:29 +02:00
bash-commands.test.ts fix(bash): attribute wrapped commands to the real tool, not the wrapper (#658) 2026-07-10 06:36:07 -07:00
blob-to-text.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
classifier.test.ts File-aware retry detection with typed ToolCall (#379) 2026-05-22 01:21:42 -07:00
cli-date.test.ts fix(web): reject invalid dashboard periods without exiting (#554) 2026-06-28 19:35:46 +02:00
cli-deepseek-v4-pricing.test.ts fix: DeepSeek v4 Claude pricing through stale runtime cache (#367) 2026-05-21 00:34:53 -07:00
cli-devin-model-variants.test.ts fix(devin): report friendly GPT model names with effort tier (#585) 2026-07-01 16:01:49 +02:00
cli-emitters.test.ts feat(cli): add compare and sessions JSON emitters 2026-07-11 12:16:45 -07:00
cli-export-date-range.test.ts feat(export): support custom date ranges 2026-05-05 23:18:48 -07:00
cli-json-daily.test.ts Add multi-day calendar selection with custom popover UI (#393) 2026-05-24 09:49:59 -07:00
cli-model-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
cli-plan.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
cli-price-override.test.ts feat(pricing): add user price overrides for models (#390) (#560) 2026-06-28 19:07:29 +02:00
cli-provider-validation.test.ts fix(cli): validate --provider and make non-TTY report deterministic (#501) 2026-06-18 12:39:22 +02:00
cli-proxy-path.test.ts test(cli-proxy-path): raise timeout for spawn-based tests to fix load flake (#504) 2026-06-18 12:57:44 +02:00
cli-settings-json.test.ts feat(cli): price-override --list --format json 2026-07-16 03:45:36 -07:00
cli-status-menubar.test.ts feat(menubar): Claude config selector, hardened (supersedes #635) 2026-07-10 01:44:51 +02:00
codex-cache-invalidation.test.ts fix(codex): attribute CLI-wrapped MCP calls (mcp-cli), and re-parse cached sessions (#478) 2026-07-10 12:11:59 +02:00
codex-credits.test.ts feat(codex): compute Codex credit usage (#408, #495) (#510) 2026-06-18 17:03:46 +02:00
compare-stats.test.ts fix(cli): compare cache-hit rate excludes cache writes 2026-07-16 01:14:41 -07:00
content-utils.test.ts fix(parser): tolerate string message content; isolate per-file parse failures (#441) (#450) 2026-06-06 04:01:12 +02:00
currency-rounding.test.ts fix(currency): resolve RON symbol to "lei" across all surfaces 2026-07-06 03:43:03 +03:00
daily-cache.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
dashboard.test.ts feat(dashboard): make Daily Activity history directly scrollable (#672) 2026-07-16 04:10:08 -07:00
date-range-filter.test.ts Add multi-day calendar selection with custom popover UI (#393) 2026-05-24 09:49:59 -07:00
day-aggregator-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
day-aggregator.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
devices-json.test.ts fix(cli): surface mDNS scan errors, robust spend-flow rollup + date label, harden tests 2026-07-10 16:20:41 -07:00
export.test.ts feat(export): surface MCP server usage in JSON and CSV exports (#514) 2026-06-18 23:13:48 +02:00
fetch-utils.test.ts fix(network): add timeouts to critical-path fetches (#445) (#448) 2026-06-06 03:25:36 +02:00
fs-utils.test.ts Raise streaming session cap to 4GB so large Codex sessions are counted (#569) 2026-06-26 01:16:56 +02:00
granular-history.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
guard-hooks.test.ts feat(claude): account for /advisor escalations under the advisor model (#659) 2026-07-16 07:42:49 -07:00
guard-install.test.ts fix(guard): dedupe streaming message copies in the incremental cost fold 2026-07-03 12:32:46 +02:00
kiro-cache-invalidation.test.ts fix(kiro): harden ws-session parsing and cover the stale-cache path (#619) 2026-07-05 15:02:25 +02:00
local-model-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
mcp-coverage.test.ts Add MCP project profile advisor (#356) 2026-05-24 01:57:59 -07:00
mcp-redact.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
mcp-server.test.ts feat(mcp): get_usage + get_savings tools with annotations, schemas, coalescing 2026-06-02 02:16:10 -07:00
mcp-tables.test.ts feat(mcp): markdown table renderers for usage and savings 2026-06-02 02:13:51 -07:00
menubar-installer.test.ts Fix menubar install release lookup 2026-07-08 21:36:20 +02:00
menubar-json.test.ts Merge origin/main into feat/desktop-app 2026-07-16 07:52:28 -07:00
menubar-savings.test.ts feat(cli): track local-model cost savings against a paid baseline (#421) 2026-06-01 11:06:39 +03:00
minimax.test.ts chore(pricing): refresh LiteLLM snapshot; MiniMax-M3 standard tier 2026-07-02 06:17:01 +02:00
model-efficiency.test.ts feat(report): add per-model efficiency metrics 2026-05-05 23:36:59 -07:00
models-hoist.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
models-report.test.ts feat(codex): compute Codex credit usage (#408, #495) (#510) 2026-06-18 17:03:46 +02:00
models.test.ts fix(report): only exact zero-rate overrides suppress the unpriced flag 2026-07-09 21:22:23 +02:00
optimize-apply.test.ts fix(act): refuse to apply a plan whose target changed since it was built 2026-07-03 11:20:59 +02:00
optimize-fs.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
optimize.test.ts fix(optimize): exclude a young project's founding session from cost-outlier detection (#666) 2026-07-16 02:03:06 -07:00
otel-cache-aggregation.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
overview.test.ts feat(report): surface unpriced models across overview, JSON, dashboard, and MCP 2026-07-09 21:04:56 +02:00
parser-advisor-usage.test.ts feat(claude): account for /advisor escalations under the advisor model (#659) 2026-07-16 07:42:49 -07:00
parser-antigravity-timestamp.test.ts fix(antigravity): stamp mtime fallback at emission, tighten path classification (#612) 2026-07-16 07:49:54 -07:00
parser-claude-cwd.test.ts Show real Claude project leaf names; stop stray-.git over-grouping (#540) 2026-06-21 22:42:19 +02:00
parser-compact-entry.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
parser-filter.test.ts test: cover filterProjectsByName include/exclude semantics 2026-04-16 15:49:57 -07:00
parser-gemini-cache.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-large-json-scanner.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
parser-large-session.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-local-savings.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-mcp-inventory.test.ts feat(optimize): MCP tool coverage detector with cache-aware costing 2026-05-05 04:13:04 +03:00
parser-number-helpers.test.ts feat: add devin provider (#444) 2026-06-09 21:58:31 +02:00
parser-proxy-codex-only.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-proxy-merge.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-proxy-pricing.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
parser-skip-line.test.ts Merge main into feat/multi-provider-plans, remove dead tautology 2026-05-16 10:42:03 -07:00
parser-subagent-collection.test.ts feat(report): Claude-scoped agent-type breakdown 2026-06-10 11:45:28 +02:00
parser.test.ts fix(copilot): bill CLI input and cache tokens from session.shutdown (#684) 2026-07-16 10:37:58 -07:00
plan-usage.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
plans.test.ts fix: fix and improve test isolation and collision with environment (#530) 2026-06-20 13:42:10 +02:00
pricing-fallback-data.test.ts feat(pricing): automatic gap-fill from models.dev and OpenRouter (#457) 2026-06-09 21:17:23 +02:00
provider-registry.test.ts feat: add CodeWhale provider support (#674) 2026-07-16 03:12:28 -07:00
provider-turn-grouping.test.ts kiro: preserve credit-based cost through the session cache 2026-07-14 22:35:51 +00:00
scan-progress.test.ts fix(parser): keep scan-progress out of the interactive dashboard and compare TUIs 2026-07-10 00:50:35 +02:00
session-cache.test.ts fix(kiro): parse context.messages, add .kiro-server path, extract usageSummary tools, workspace-sessions 2026-07-05 01:28:33 +00:00
sessions-report.test.ts feat(cli): add compare and sessions JSON emitters 2026-07-11 12:16:45 -07:00
spend-flow.test.ts fix(cli): surface mDNS scan errors, robust spend-flow rollup + date label, harden tests 2026-07-10 16:20:41 -07:00
sync-e2e.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync-headless-e2e.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync-infra-e2e.test.ts test(sync): unit, mock-IdP e2e, and developer-only integration tests 2026-07-12 16:05:34 +00:00
sync-ledger-otlp.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync-push.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
sync.test.ts fix(sync): address PR review — https enforcement, keychain test isolation, golden ID pins 2026-07-13 16:45:39 +00:00
usage-aggregator.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
web-dashboard.test.ts feat(dashboard): add granular session and model timelines (#673) 2026-07-16 03:04:54 -07:00
yield.test.ts Add JSON output for optimize and yield 2026-06-18 12:13:17 +02:00