A warm launch rewrote the entire session cache whenever any provider
appended a few KB: on a 6 GB corpus that is a 155 MB stringify + fsync
every run. The on-disk cache is now a version-suffixed directory holding
one shard per provider plus a small envelope, and a save rewrites only
the providers marked dirty.
- Dirtiness is tracked per provider (markCacheDirty) instead of one
global flag, so an appended Claude session no longer republishes
Codex, Copilot and the rest.
- Shards carry a nonce in their filename and the envelope is renamed
last, so a save is published at a single point: readers never see a
half-updated set, and a writer that loses the refresh ownership fence
leaves the canonical shards untouched.
- A shard that fails validation is treated as an absent provider rather
than rejecting the whole cache, so one malformed turn costs one
provider's re-parse instead of every provider's history.
- v7 migrates losslessly: the blob is re-laid-out into shards and
removed only once that save publishes. Nothing re-parses.
- Cold-parse progress saves now trigger every N files parsed rather than
every 5s, so a slow cold parse no longer rewrites the growing cache on
a wall clock.
The remaining warm-serve cost was the per-request discovery sweep
(stat-ing thousands of session files) plus re-aggregation, even when
nothing on disk had changed. Serve now watches every provider's
probeRoots() via fs.watch (FSEvents-backed recursive watches on macOS)
and injects a quiet-since validator into the parser: while the watched
roots are quiet, a previous parse stays reusable past the burst window,
and an output-level memo returns identical panel queries verbatim - so a
fetch with no data changes skips the sweep AND the aggregation.
Safety rails, in order: a parse is validated-reusable only if the
watchers were armed before it ran; any filesystem event ends reuse
instantly; a 5-minute hard cap self-heals a missed event; a root that
fails to watch just goes uncovered (shorter reuse, never staleness);
outside serve the validator is never installed and behavior is
byte-identical. During an active AI session the session roots fire
constantly, so reuse correctly stays inside the 10s burst window - the
extended reuse serves the idle-browsing case it was built for.
The one watched path inside the cache dir is antigravity's statusline
file specifically, so serve's own cache writes never self-invalidate.
Every CLI spawn on a large corpus pays seconds of fixed cost before any
query work: node boot, a 100MB+ session-cache JSON.parse, the discovery +
fingerprint sweep, and serve-time classification. The desktop app spawns
one CLI per panel fetch, so it pays that cost per panel.
codeburn serve --stdio is the same CLI kept warm: the app holds one child,
sends {id, args} per line, and gets the command's stdout back. Three layers
make it fast, each disabled outside serve so one-shot runs stay byte-exact:
- loadCache memo (session-cache.ts): the parsed cache object is reused
while a stat() shows the file unchanged; saveCache updates it
write-through. A rewrite by another process still forces a fresh read.
- burst reuse (parser.ts, CODEBURN_PARSE_BURST_MS, serve sets 10s): panel
bursts anchor their range ends at their own new Date(), so the exact-key
memo never hits in real traffic; within the window a re-anchored range is
served by trimming the previous parse instead of re-running discovery.
- fresh commander program per request (main.ts buildProgram factory),
because commander option state is sticky across parses.
The server allows only the app's read queries (status/overview/models/
sessions/compare/yield/spend/optimize/audit), refuses everything else
(client falls back to a spawn), serializes requests, and converts
process.exit into a caught signal. The app starts the child once at
startup; requests route through it only when warm, cold-start keeps the
spawn path with its progress events, any serve failure falls back to a
spawn, and three child deaths disable serve for the app run.
Measured on a real 17B-token corpus: panel fetches drop from ~7.4s per
spawn to 5-900ms warm (sessions/spend 5ms, status 898ms). One-shot CLI
output verified byte-identical against the pre-branch baseline.
- cli-durable-totals: the live fixture session was stamped at noon today,
so every before-noon run saw it in the future; the provider-filtered
path drops future instants while the all-provider path keeps the whole
day, failing the parity assertion. Relative-and-clamped timestamps,
the same fix project-filter-durable-totals got in 1596220.
- parser (copilot, 2 cases): the fixture's fixed 2026-05-01 dates crossed
copilot's durable 90-day age-out on 2026-07-30, so the first parse
pruned the freshly-cached session. Relative timestamps.
- parser-incremental-append: unlink-then-create let ext4 hand the freed
inode straight back, breaking the new-inode premise. The replacement
is now created beside the original and renamed over it.
- parser-proxy-pricing: normalizeProxyPath folds case only on darwin and
win32, deliberately; the test now asserts the platform-correct
behavior on both kinds of filesystem instead of hardcoding macOS.
- cli-status-menubar: the config-source filter case does real multi-parse
work and needs more than the 5s default on shared runners; 30s cap.
Claude is scanned via scanProjectDirs instead of parseProviderSources, and
that call had no provider-filter guard. On a --provider <other> run
discoverAllSessions correctly returns no claude sources, so claudeDirs is
empty, but scanProjectDirs still ran: its orphan pass reads the whole cached
claude section and treats every file as no-longer-discovered, re-injecting
PR-bearing entries (and in read-only mode every cached entry) into the result.
The headline stayed correct because it comes from the provider-sliced daily
cache, so only the live-parse panels were wrong. By Model then listed
Anthropic models under --provider cursor while the total showed cursor alone.
Guard the scan with claudeInScope, mirroring the guard the durable-orphan
loop already applies. Deliberately not a claudeDirs.length check: when claude
is in scope but every transcript has been pruned, the orphan pass is what
keeps PR-attributed spend from vanishing.
Skill attribution was gated on the turn classifying as 'general', but
classifyByToolPattern checks hasSkill last, so any turn invoking the Skill
tool alongside Read/Bash/Edit/MCP got a real category and its skill was
silently dropped. Skill turns virtually always carry other tools, which is
why skills came back empty in all 25 usage snapshots (#741) while mcpServers
populated.
subCategory is now set for any turn with Skill tool_use; category assignment
is untouched. The claude parse version is bumped so cached session summaries
(which lack skill attribution) re-parse on upgrade.
Performance and coherence:
- Settings > General 'Refresh every' (Manual/30s/1m/3m/5m/10m), live via
RefreshCadenceContext; Manual polls only on demand
- usePolled memoKey LRU: provider/period switches paint the last-good
result instantly with a switching hairline while refreshing quietly
- quota TTL 5min + honest rate-limited copy on 429 backoff
- version-suffixed cache files (session-cache.v5.json, daily-cache.v12,
auto-minted on future bumps); legacy files never written or deleted,
adopted once when versions match: old and new binaries coexist
without clobbering (field-observed menubar-vs-desktop ping-pong)
- advisory hydration lock: concurrent cold starts share one scan
(wait-then-read-warm), stale/dead locks self-heal, never a
correctness gate
Telemetry v1 + onboarding (desktop only, per owner decisions):
- first-launch onboarding (3 feature screens + consent); region-split
default (EU/EEA/UK/CH off, elsewhere on, unknown off); nothing sends
before consent completion or while off; dev builds never send
- anonymous install UUID, rotated on opt-out; day-granularity events,
cost buckets only; whitelisted names; 200-event queue, 5min flush
- Settings > Privacy live toggle replaces the static claim
Zero computed-number changes. App 316/316, root 1805.
Wire contract targets api.codeburn.app/v1/telemetry (Worker follows).
* fix(copilot): bill CLI input and cache tokens from session.shutdown
Copilot CLI sessions (~/.copilot/session-state/*/events.jsonl) recorded
only outputTokens, because the parser read assistant.message events and
ignored the session.shutdown rollup. That rollup is the sole on-disk
source of a CLI session's input, cache-read and cache-write tokens, so
cost was underreported (in the reported case 35k+ cache-read tokens were
dropped entirely).
Handle session.shutdown in the CLI (non-transcript) JSONL path and emit
one supplementary call per model from modelMetrics.<model>.usage. The
per-turn assistant.message events keep owning output tokens, tools and
turn structure; the shutdown call contributes only the input and cache
tokens they lack, with output excluded, so no dimension double-counts and
the combined cost equals the full CLI-measured cost. usage.inputTokens is
cache-inclusive (input + cache_read + cache_write), so the cache
components are subtracted to recover the uncached input calculateCost
expects. These are real counts written by the CLI, so costIsEstimated is
false. Sessions without session.shutdown (crashed or still running) keep
the exact previous output-only behavior, and the VS Code, JetBrains and
OTel paths are untouched.
Bump the copilot entry in PROVIDER_PARSE_VERSIONS so already-cached CLI
sessions re-parse and pick up the corrected tokens and cost.
* fix(copilot): never emit an empty-timestamp shutdown call
A session.shutdown without its own timestamp and without sessionStartTime
produced a call with an empty timestamp, which the date-range filters in
parser.ts drop silently, erasing exactly the tokens this feature bills.
Fall back to the last stamped event in the session.
* fix(cache): durable orphans survive provider fingerprint changes
Bumping a durable provider's PROVIDER_PARSE_VERSIONS changed its env
fingerprint, and getOrCreateProviderSection replaced the whole section on
mismatch. For copilot OTel orphans the discarded cache entry is the only
remaining record of pruned spans, so the first post-upgrade run silently
and permanently erased month-to-date history. Carry forward exactly the
entries whose source no longer exists; present sources still drop and
re-parse under the new fingerprint, which is the point of a bump.
Regression test forges a pre-upgrade fingerprint on a pruned-source
orphan and asserts it stays counted and cached across the transition.
* fix: fix and improve test isolation and collision with environment
* docs: remove unnecessary comment
* test(env-isolation): clear CODEBURN_FORCE_MACOS_MAJOR and pin TZ
Two env vars read in src/ were not isolated: CODEBURN_FORCE_MACOS_MAJOR
(now cleared so it cannot leak between tests) and TZ (now pinned to UTC,
since clearing it falls back to the OS zone and would shift date buckets
versus a clean CI runner).
---------
Co-authored-by: AgentSeal <hello@agentseal.org>
