A warm launch rewrote the entire session cache whenever any provider
appended a few KB: on a 6 GB corpus that is a 155 MB stringify + fsync
every run. The on-disk cache is now a version-suffixed directory holding
one shard per provider plus a small envelope, and a save rewrites only
the providers marked dirty.
- Dirtiness is tracked per provider (markCacheDirty) instead of one
global flag, so an appended Claude session no longer republishes
Codex, Copilot and the rest.
- Shards carry a nonce in their filename and the envelope is renamed
last, so a save is published at a single point: readers never see a
half-updated set, and a writer that loses the refresh ownership fence
leaves the canonical shards untouched.
- A shard that fails validation is treated as an absent provider rather
than rejecting the whole cache, so one malformed turn costs one
provider's re-parse instead of every provider's history.
- v7 migrates losslessly: the blob is re-laid-out into shards and
removed only once that save publishes. Nothing re-parses.
- Cold-parse progress saves now trigger every N files parsed rather than
every 5s, so a slow cold parse no longer rewrites the growing cache on
a wall clock.
Every CLI spawn on a large corpus pays seconds of fixed cost before any
query work: node boot, a 100MB+ session-cache JSON.parse, the discovery +
fingerprint sweep, and serve-time classification. The desktop app spawns
one CLI per panel fetch, so it pays that cost per panel.
codeburn serve --stdio is the same CLI kept warm: the app holds one child,
sends {id, args} per line, and gets the command's stdout back. Three layers
make it fast, each disabled outside serve so one-shot runs stay byte-exact:
- loadCache memo (session-cache.ts): the parsed cache object is reused
while a stat() shows the file unchanged; saveCache updates it
write-through. A rewrite by another process still forces a fresh read.
- burst reuse (parser.ts, CODEBURN_PARSE_BURST_MS, serve sets 10s): panel
bursts anchor their range ends at their own new Date(), so the exact-key
memo never hits in real traffic; within the window a re-anchored range is
served by trimming the previous parse instead of re-running discovery.
- fresh commander program per request (main.ts buildProgram factory),
because commander option state is sticky across parses.
The server allows only the app's read queries (status/overview/models/
sessions/compare/yield/spend/optimize/audit), refuses everything else
(client falls back to a spawn), serializes requests, and converts
process.exit into a caught signal. The app starts the child once at
startup; requests route through it only when warm, cold-start keeps the
spawn path with its progress events, any serve failure falls back to a
spawn, and three child deaths disable serve for the app run.
Measured on a real 17B-token corpus: panel fetches drop from ~7.4s per
spawn to 5-900ms warm (sessions/spend 5ms, status 898ms). One-shot CLI
output verified byte-identical against the pre-branch baseline.
Round 2 of the independent review proved five things by mutation: it broke the
behavior and the tests stayed green. Every one is now pinned.
The most important invariant in this change was the least guarded. Copilot must
have NO entry in PROVIDER_ENV_VARS - declaring any of its nine reads moves its
fingerprint and re-opens the durable history-loss path - but only one of the
nine was covered, so declaring any of the other eight passed the whole suite.
Now the absence of the entry is asserted directly, and all nine vars are
table-tested for fingerprint stability.
Doctor stops blaming parse-only overrides for a failed discovery.
CODEBURN_CURSOR_MAX_BUBBLES caps how many bubbles Cursor parses and
KIMI_MODEL_NAME renames an attributed model; neither relocates anything, so
"NOTHING FOUND (override CODEBURN_CURSOR_MAX_BUBBLES set...)" pointed the user
at the wrong thing. Both join NON_DISCOVERY_ENV_VARS, which exists for exactly
this, and both still appear in Details - only the verdict's blame line changes.
The secret-redaction and ambient-suppression tests are table-driven over both
names each covers, since removing either second name (VERCEL_OIDC_TOKEN,
LOCALAPPDATA) previously leaked or surfaced it with every test still passing.
The changelog no longer claims a one-time re-parse for the Vercel gateway: it
is a network provider re-fetched on every writable run, so its declaration is a
read-only-path correction, not a migration. Fourteen file-backed providers
migrate once.
Five findings from a cross-model review of the previous two commits, each
verified on the code before acting:
Copilot is no longer declared. Declaring anything for it changes its
fingerprint, and getOrCreateProviderSection keeps only cached entries whose
source path is gone - but OTel discovery returns one source per DB file
(copilot.ts:1935) and that DB keeps existing, so the entry would be dropped and
re-parsed, destroying conversations Copilot has since pruned from the DB that
only the cache still holds. Trading a staleness bug for a data-loss bug is a
bad trade; copilot waits for the durable carry-forward to merge instead of
drop, and its reads are allowlisted with that reason.
The Vercel gateway credentials ARE declared, reversing the previous commit's
reasoning, which was wrong: servedSources is seeded with every discovered
source (parser.ts:2875) before the network branch, and the network re-fetch
(parser.ts:2888) only runs when !readOnly, so a read-only refresh serves the
cached report and an undeclared credential keeps reporting the previous
account's usage after a swap. Doctor redacts credential values so a key can
never reach terminal output or the JSON report.
AMBIENT_ENV_VARS narrows to APPDATA and LOCALAPPDATA. Windows sets those for
every process so they carry no intent, but the XDG vars are opt-in and do:
suppressing them made doctor answer a deliberately relocated XDG_DATA_HOME with
"tool likely not installed", which is worse than the noise it avoided.
The guard's allowlist is keyed by file and var, not var alone - a var
allowlisted for one file silenced every other file's undeclared read of it.
Cursor drops its stale XDG_DATA_HOME declaration, which it never reads; its
fingerprint already changes here, so this costs no extra migration.
cursor-agent keeps its equally stale one, since removing it would force a
re-parse to fix nothing.
The nine undeclared overrides in #920 all slipped through the same way: the
declaration lives in one file and the read in another, and nothing tied them
together. Add the static guard the issue asked for - every process.env read in
src/providers is either declared in PROVIDER_ENV_VARS for the provider(s) that
file serves, or allowlisted with a reason. It resolves bracket literals, dot
access and `process.env[CONST]` indirection (open-design's ENV_DIR), and fails
loudly on any read it cannot resolve to a name rather than skipping it, since a
silently skipped read is how this class of defect survives. A read-bearing
provider file missing from the file-to-provider map fails too, so a new
provider cannot join without being mapped. A second assertion catches a
PROVIDER_ENV_VARS key that is not a registered provider name, which declares
nothing and fails just as silently.
Plus the direct regression: each of the nine reported (provider, var) pairs
must move the fingerprint, with codex/CODEX_HOME as the control the issue used,
and the round trip asserted so the hash stays a pure function of the
environment.
Hermes, Cursor, OpenCode and copilot OTel sources all live in SQLite
databases that their agents keep open in WAL mode for the life of the
process. Committed writes park in <db>-wal until a checkpoint, so the
main file's stat can sit hours or days behind the newest committed data.
fingerprintFile only statted the main file, which broke two ways:
- The date-range mtime pre-filter in parseProviderSources read the stale
mtime as "nothing in range" and skipped the source entirely. Every
Hermes session committed after the last checkpoint vanished from
reports: the today-parse skipped the db (mtime < local midnight) while
the backfill only keeps days through yesterday. Exactly the "17
sessions in the DB, 14 reported, the 3 from today missing" report in
issue #913.
- reconcileFile saw an unchanged fingerprint between checkpoints and kept
serving stale cached turns for sessions that had since grown.
Fold the -wal sibling into the fingerprint: newest mtime wins and sizes
add, so both WAL growth and a checkpoint (db grows, wal truncates) move
the fingerprint. -shm is deliberately ignored (it mutates on reads).
Bare SQLite paths get the fold only when the extension says database, so
JSONL transcript fingerprints (offset-based append detection) are
untouched.
Refs #913
Performance and coherence:
- Settings > General 'Refresh every' (Manual/30s/1m/3m/5m/10m), live via
RefreshCadenceContext; Manual polls only on demand
- usePolled memoKey LRU: provider/period switches paint the last-good
result instantly with a switching hairline while refreshing quietly
- quota TTL 5min + honest rate-limited copy on 429 backoff
- version-suffixed cache files (session-cache.v5.json, daily-cache.v12,
auto-minted on future bumps); legacy files never written or deleted,
adopted once when versions match: old and new binaries coexist
without clobbering (field-observed menubar-vs-desktop ping-pong)
- advisory hydration lock: concurrent cold starts share one scan
(wait-then-read-warm), stale/dead locks self-heal, never a
correctness gate
Telemetry v1 + onboarding (desktop only, per owner decisions):
- first-launch onboarding (3 feature screens + consent); region-split
default (EU/EEA/UK/CH off, elsewhere on, unknown off); nothing sends
before consent completion or while off; dev builds never send
- anonymous install UUID, rotated on opt-out; day-granularity events,
cost buckets only; whitelisted names; 200-event queue, 5min flush
- Settings > Privacy live toggle replaces the static claim
Zero computed-number changes. App 316/316, root 1805.
Wire contract targets api.codeburn.app/v1/telemetry (Worker follows).
* fix(report): surface estimated costs distinctly
Providers that estimate tokens or price (kiro, cursor, warp, copilot,
grok, hermes, codewhale, and the codex proxy path) set costIsEstimated
on their parsed calls, but the flag died at the parser boundary: it was
never carried onto ParsedApiCall or into the session aggregates, so a
figure whose tokens were synthesized from content length rendered with
the same authority as a metered one.
Plumb the truth through, mirroring the savingsUSD/isLocalSavings pattern:
a call-level boolean (ParsedApiCall.isEstimated, CachedCall.isEstimated,
persisted so it survives the session-cache round trip) and an additive
aggregate amount (estimatedCostUSD on the model breakdown, session, and
project totals, plus PeriodData and the menubar payload). The amount is
carried rather than a bare boolean so a row that is mostly metered with a
small estimated slice is not indistinguishable from a fully guessed one.
Display: report (TUI) and overview per-model rows prefix the cost with a
tilde and print one legend line; the MCP tables carry the same marker and
legend, and the machine surfaces (report --json, MCP get_usage, menubar /
web payload) expose estimatedCostUSD. Totals math is unchanged; the flag
is display/metadata only.
Bump PROVIDER_PARSE_VERSIONS for every provider that sets the flag so
already-cached sessions reparse once and pick it up. Copilot is excluded:
it is a durable provider, so changing its env fingerprint would discard
OTel cache entries whose source rows may already be pruned.
Also fix the cross-provider project merge, which summed totalCostUSD but
dropped merged-in projects' totalEstimatedCostUSD, undercounting the
project/period estimated total (the same latent gap still affects
totalSavingsUSD, left untouched here).
* test(parser): pin estimated dollars through the cross-provider merge
The merge fix for dropped totalEstimatedCostUSD was not covered: deleting
the summing line left every test green. Extract the merge into an exported
mergeProjectsByCrossProviderKey (no behavior change) and pin both the
measured-plus-estimated and both-estimated merge cases.
* docs(parser): honest merge-comment scope and load-bearing overwrite note
Re-review nits: the merge doc claimed all additive totals are summed there
while totalSavingsUSD still is not (pre-existing gap, tracked separately)
and totalProxiedCostUSD is re-derived post-merge; say so. Mark the
buildPeriodData overwrite in usage-aggregator as load-bearing for the
estimated marker so nobody optimizes it away trusting the daily cache.
Four fixes for the Kiro IDE provider:
1. Add 'entries' to extractText() key list — Kiro IDE stores message
content in context.messages[].entries (not .content), causing the
parser to extract 0 chars from every execution file.
2. Check data.context[key] for conversation arrays in parseModernExecution
— current Kiro builds store messages at data.context.messages, not at
the top-level data.messages path the parser was checking.
3. Scan both ~/.kiro-server/data/... AND ~/.config/Kiro/... on Linux —
remote dev boxes use .kiro-server while local installs use .config/Kiro.
Both can have data simultaneously; the old code short-circuited on the
first path found.
4. Discover and parse workspace-sessions/<base64>/*.json files — newer
Kiro builds write session state here with history[].message format.
Skips stub entries (executionId refs + 'On it.' only) to avoid
double-counting with execution files parsed separately.
5. Add kiro: 'ide-parsing-v1' to PROVIDER_PARSE_VERSIONS for automatic
cache invalidation — users upgrading from the broken parser will get
a fresh re-parse without manually clearing session-cache.json.
Bonus: Extract tool names from usageSummary[].usedTools, add chatSessionId
to session ID resolution, add Kiro-specific tool name mappings.
AI-Origin: human
* fix: fix and improve test isolation and collision with environment
* docs: remove unnecessary comment
* test(env-isolation): clear CODEBURN_FORCE_MACOS_MAJOR and pin TZ
Two env vars read in src/ were not isolated: CODEBURN_FORCE_MACOS_MAJOR
(now cleared so it cannot leak between tests) and TZ (now pinned to UTC,
since clearing it falls back to the OS zone and would shift date buckets
versus a clean CI runner).
---------
Co-authored-by: AgentSeal <hello@agentseal.org>
Follow-up to #450. When a session file throws during parse it was excluded but
left uncached, so every refresh (~4x/min in the menubar) re-read and re-parsed
it, and only the first failing file per provider was ever surfaced.
- Add a negative-result marker: a failed file is cached as { fingerprint,
turns: [], failed: true }. reconcileFile treats it as 'unchanged' at the same
fingerprint, so it's skipped (no re-read) until the file changes. Empty turns
=> contributes no usage.
- Warn per offending file (with its path), capped at 5 per provider per run,
instead of once-per-provider — so a systemic break surfaces more than one file
without flooding. Cached markers keep it quiet across refreshes.
Tests: marker round-trips through save/load; reconcile stays 'unchanged' at the
same fingerprint and re-parses when the file changes.
The Cursor provider encodes workspace context into source paths using a
`#cursor-ws=<tag>` suffix (e.g. `state.vscdb#cursor-ws=__orphan__`).
`fingerprintFile` only had a fallback for `:` separators (OpenCode
sessions), so Cursor sources silently returned null on macOS/Linux where
paths contain no colons, causing them to be skipped entirely.
Add a `#` fallback before the existing `:` check. The first `stat()`
on the full path still succeeds for real files containing `#`, so there
is no regression for legitimate paths.
Includes 4 new test cases covering both separators, the combined case,
and the null case for non-existent base files.
Cache normalized turns/calls to ~/.cache/codeburn/session-cache.json so
the CLI skips re-parsing unchanged JSONL files on subsequent runs.
File reconciliation uses dev+ino+mtime+size fingerprinting; cost,
classification, and summaries are recomputed at query time. Atomic
writes via temp+fsync+rename, deep structural validation on load,
per-provider env fingerprinting, and best-effort save so cache failures
never break the CLI. ~6x speedup on warm cache.
