Resolves the conflicts #946 accumulated while it was in validation. Eight
files conflicted; the session-store accounting is unchanged.
src/daily-cache.ts — version collision. This PR minted 25 when main was at
24; #1056 (`codex-auto-review` pricing) then spent 25 on main. The bump moves
to 26/MIN 26 and daily-cache.v26.json, with main's full comment ladder kept as
the foundation and this PR's paragraph rewritten to name 26 and record the
collision. PENDING_REDERIVE_PROVIDERS and the B1 migration semantics from
b6481c19 carry over intact, retargeted at 26.
src/models.ts, src/parser.ts, src/audit-report.ts, src/models-report.ts —
#1075/#1078 replaced the per-site "reasoning is already inside output" tests
with billableOutputTokens() and REASONING_INCLUDED_IN_OUTPUT. This PR had
added copilot to that case at three sites independently. Union: all three
sites take main's helper call verbatim, and copilot joins claude and codex in
the set — same accounting this PR shipped, now through main's single source of
truth. It also reaches parser.ts activeGeneratedTokens (a fourth site, from
#1079), which is the same correction: a copilot supplementary call carries
reasoning with output 0, so counting it as generated repeats the per-turn
output. The audit legend already said so on this side.
src/providers/copilot.ts — comment-only. #1054's lastEventTimestamp-first
shutdown fallback was derived from this branch, so the code was already
identical on both sides: the `shutdownTimestamp` expression and the
`copilot:<sid>:shutdown:<model>:<n>` key are byte-for-byte main's. Both
rationales are kept (leg-collapse on date, and residual anchoring).
src/session-cache.ts — PROVIDER_PARSE_VERSIONS.copilot takes this PR's
`-session-store-v3` suffix; main's #1051 note about why a fingerprint change
is expensive is kept above it. Codex keeps main's #1092 suffix chain untouched.
src/main.ts — #1067 deleted the unreachable live dailyMap fallback that this
PR had taught behavioral weight. Main's deletion wins; the now-unused
isBehavioralTurn import goes with it.
tests/parser.test.ts — import union.
Also: scripts/upgrade-path/run.mjs NEW_DAILY_CACHE -> daily-cache.v26.json,
CODEBURN_COPILOT_SESSION_STORE_DB added to the #1064 env-isolation CLEARED
list, and the CHANGELOG entry's stale "v21" corrected to v26.
Verified: tsc clean; 3132 tests pass across 223 files; test:locks 26/26;
verify:upgrade PASSED, re-deriving daily-cache.v26.json and holding durable
copilot history across the bump. #1054's regression ("keeps three stampless
shutdown legs as :n keys with lastEventTimestamp") passes on the merged tree.
Real-corpus A/B against origin/main over 2026-07-01..2026-08-22: codex, grok,
kimicode and opencode byte-identical in export, audit and models; claude drifts
only monotonically with run order (a live session writing transcripts, confirmed
by interleaving four runs). This machine has no copilot data, so the copilot
recovery semantics rest on the suites and the upgrade-path corpus.
@vidoluco's round-6 re-validation: a cache generated by 0.9.20, copied and then
migrated by this branch, dropped exactly ONE call a virgin cache serves -
mai-code-1-flash-picker, 324 input / 91,136 cache read / $0.01, two minutes
after the labelled compaction row in its session. Every other day matched.
Root cause is the durable union merge, not the reconciliation. `compactedAt` is
a capture-only field this branch adds to the shutdown-ROLLUP call, and 0.9.20
already cached that call under the same (deliberately stable) dedup key. The
union appended only UNSEEN keys, so the bump could never reach it. The migrated
cache then ran the compaction-anchored residual math with no anchor - the
pre-anchor behaviour exactly: the interval opens at -Infinity, the
PRE-compaction rows are subtracted from a rollup that never counted them, and
the residual clamps to zero and disappears.
The union now REPLACES a cached call with the freshly-derived one wherever the
key matches. Nothing is deleted: keys the re-read did not produce are exactly
the rows the source has pruned, and those still carry forward untouched with
the old parser's fields. Keys it did produce were just derived from the live
source in this very pass, so the fresh call is by construction the better one.
This is also what makes the round-5 compaction-output fix reach an existing
cache - that row's key is stable too, so append-only would have pinned it at
output 0 forever. Both directions are pinned by (c6).
Two defects from @kelchm's two-machine validation of #946.
1. The daily-cache migration carried the copilot slice of every existing day
entry verbatim instead of re-deriving it, so `overview` on a migrated cache
kept serving pre-store numbers (2,980,804 tokens for a day whose fresh
derivation is 74,811,412; 21,608 calls vs 16,326) while export/models/audit
served the corrected ones off the same cache.
Root cause is the partial-survival guard (e9d922ca), not the migration: it
keeps a settled baseline slice whenever the fresh derivation reports FEWER
calls, reading that as aged-out sources. This change's whole point is that
copilot's supplementary accounting calls stop counting as api calls, so
every store-era day shrinks in calls and got pinned to its pre-store value.
The guard's own TRADE-OFF note predicted exactly this case.
A migration from an older cache version now grants copilot ONE
guarded-shrink-exempt re-derivation (`pendingRederive`), spent by the first
COMPLETE parse. It fires only where that parse actually produced a slice for
the (day, provider), so a day whose sources are gone still carries forward
whole - never-lose is unchanged in both directions - and no other provider
is exempt.
The daily cache moves 21 -> 25: 21-24 landed on main during validation, and
only a number the validators' own daily-cache.v21.json cannot claim gets
their stale carried slices re-derived.
2. The `initiator='compaction'` store row was ingested with outputTokens 0. Its
output has no assistant.message anywhere in events.jsonl, so that row is the
only place those tokens exist - dropping them was the sole token discrepancy
(-3,085) across a 30-session store-matched comparison. It is now read (on the
same optional-select rung as `initiator`, which is what identifies the row),
counted and priced; every other row keeps output 0 because a per-turn call
owns it. The copilot parse version moves to session-store-v3 so cached v2
rows re-parse; the dedup key is deliberately unchanged.
Settle window stays at 24 h - the three-machine evidence for tightening it is
recorded next to the constant, but shortening it is a product call.
Maintainer review on #1054: two-journal collision is unreachable
through discovery. Keying by timestamp+journal collapsed a 3-leg
stampless journal onto one row, and the :n strip re-sent ledger
keys. Keep occurrence keys. Take lastEventTimestamp before
sessionStartTime for the call timestamp only. No migration.
Merge resolution
- daily-cache: main shipped v20 with #1040, so 21 is now the first FREE number
rather than one dodging an unmerged head. Both version notes kept, ours on
top. The re-derivation seed stays one below current (20), now the shipped
predecessor rather than a draft.
- upgrade-path: expected daily cache filename stays v21.
The rollup is cumulative across resume legs — already handled, now pinned
Measured on a real 3-leg session (CLI 1.0.80): every counter in a leg includes
the legs before it, and the last leg of a complete session equals its store-row
total to the nano. The parser has always emitted `cumulative - previous
cumulative` per model, so the interval arithmetic downstream already consumes
per-leg claims; it was the PR body and the provider doc that described the raw
journal as per-leg and misled the reader. Doc corrected, and (c3) pins it end
to end: a complete 3-leg session serves 900 (its last leg / store total), not
1,400 (the sum of its legs); an uncovered one serves 900 exactly once; and a
leg reporting LESS than its predecessor is taken as a fresh epoch rather than
clamped to a negative delta, so an older per-leg CLI never loses a leg.
initiator = 'compaction' replaces the timestamp heuristic where it exists
The summarization request does write its own assistant_usage_events row, and
newer stores label it. The label is now read — schema-adaptively, and the
enrichment select is a graduated chain so a store carrying the billing columns
but not `initiator` keeps its billing metadata instead of falling all the way
back to the base select. Two uses: the row is subtracted from the leg it
belongs to even though it commits before the compaction stamp, and it is kept
out of per-turn pairing since it has no assistant.message to pair with.
Optional twice over — absent on older stores, NULL on 1,504 of 2,509 rows on a
real one — so (c5) pins the labelled path at 350, the identical UNLABELLED
fixture at 400 (the documented one-request-per-compaction over-serve, which no
timestamp rule can close: the request that triggered the compaction completes
immediately before it too), and a compaction row never stealing a pairing
partner.
Attribution invariant, standing guard for the 1.8x report
(c4) runs `codeburn audit`'s own two numbers — attributed vs recomputed —
over all four combinations of the three representations a session can be
written in, at the magnitudes of a real reported day (gpt-5.6-terra, 146 rows,
input 17,792 / cache write 501,395 / cache read 12,097,364 / output 63,344 /
reasoning 24,831, billed $4.47). Every shape reconciles at 1.000, and the
covered case lands on $4.4687 = what the tokens price at = what GitHub billed,
with output 63,344 rather than the pre-fix 88,175 that re-priced reasoning on
top of itself.
upgrade-path: a day KEY is not the unit of never-lose
A parse change that re-dates a call to its true day legitimately empties one
day and fills its neighbour, token for token — observed on a real cache where
2026-08-08's single call moved to 08-07 exactly. The aging step now compares a
+/-1 day WINDOW rather than the day alone, so only a window that shrinks is a
loss, and an emptied day key is reported rather than failed.
RECONCILE_SETTLE_MS keeps 24h, with the measurement that argues against it
recorded at the constant and in the provider doc: across 91 real sessions zero
rows landed after shutdown (median -0.1s, max -0.0s). One machine, one CLI
version — the number to beat is seconds, not hours, once a second machine
agrees.
Extra High HOLD on #1054: timestamp-only keys still collided across
journals, and the provider-wide rollup-ts-v1 bump dropped present
OTel sources, erasing conversations already pruned from the DB.
Identity is now (session, model, timestamp, journal basename).
Legacy :n keys migrate on that JSONL file only.
B-1 The freeze was one-directional and left the reverse hole wide open. Rows
sync; at the 90-day durable age-out the cached rows are pruned, the rollup
stops being dropped and serves again under a key that was never sent; the
session is long past settle, so it pushes an aggregate on top of the
per-request spans the receiver already holds.
Reframed around the thing that actually matters: a copilot session's
input/cache leaves this machine in one of two SHAPES — the raw rollup
(`copilot:<sid>:shutdown:`), or reconciled output (rows plus
`:shutdown-residual:`, which are disjoint by construction and together are
exactly the rollup re-expressed). The receiver must never hold both. Whichever
shape a session was first synced in, it stays in, and the other is frozen for
that session permanently — in both directions. Growth WITHIN the sent shape is
untouched, because same-shape output is additive, never substitutive: a
resumed session's new rows and residuals still push if rows were sent, a new
leg's rollup still pushes if rollups were.
This also corrects the previous commit, which classed a residual as aggregate
and so froze it for a session whose rows had gone out — the residual is
reconciled output and belongs with the rows.
B-2 A session with an unparseable timestamp was held forever while the CLI
promised it would "push once it settles". Nothing could ever settle it. Settle
now reads the newest moment a session can be SHOWN active; a session with
nothing datable at all is sent rather than held. Mixed sessions still settle on
their datable stamps.
B-3 A stamp implausibly far in the future is broken data, not evidence the
session is live, so it no longer counts toward that newest moment. One
year-ahead row can no longer hold a month-old session hostage. Ordinary clock
skew is absorbed by a one-hour grace in both directions and still reads as
live.
B-4 The residual dedup key carried the leg's POSITION. Legs sort across every
cached file for a session, so an earlier leg arriving later renumbers every
residual after it — and a renamed key is a span the receiver takes a second
time, since there is no retraction for a usage span. Keyed by the leg's own
instant instead: append-only files mean it never moves, and the equal-timestamp
coalescing above makes it unique per leg. Residuals are new in this change and
have never shipped, so no CACHE_VERSION concern.
B-5 The dry-run "already synced" count now subtracts frozen too, matching the
nothing-to-push line.
Tests: symmetric freeze from the rows side (rollup frozen, residual and new
rows still sent, per-turn untouched); all-unparseable sent, partly-unparseable
still held; a year-ahead stamp ignored while a one-minute skew still holds;
residual keys named after their leg instant with the positional names gone. The
key-prefix pin picks up the residual's new tail. (c2) pins the property rather
than one insertion scenario, because a second file's leg currently collides on
the rollup's own dedup key before it can reach the residual sweep — the
reachable repro would prove nothing about the next one, and that is stated at
the test.
Flakes: (f2) and (sc) re-bucket each run 10x isolated and 10x with four busy
loops pinning cores — 40/40 clean. Neither has clock or ordering dependence at
its margins: afterEach removes both tmpHome and the cache dir, and the two
relative timestamps in the re-bucket test are 35 days apart so they cannot
share a month. The likeliest cause of the transient failures is this branch's
own commits rewriting src/parser.ts under a concurrent vitest. (f2) gains a
self-check that its fingerprint sentinel really differs from the computed one,
which is the one way it could have passed while exercising nothing.
The CLI's session.shutdown rollup RESETS its counters at a successful
in-session compaction, so a leg containing one describes only its
post-compaction requests. Running the store-row subtraction from the previous
leg cancelled that leg's usage against the whole pre-compaction conversation
and left the residual short by exactly that much. With a complete store the
floor hid it; with a partial snapshot it was a permanent undercount, and a day
sealed on it never corrected. That is the half of the reviewer's blocker 2 the
read-ordering fence could not reach: no timing rule helps when the rows are
simply not there yet, and rollup > rows-in-interval is indistinguishable from
legitimate partial coverage.
Anchor at the compaction instead. Pre-compaction rows still SERVE; they just
stop cancelling usage the rollup never claimed. The maintainer's repro (row A
100 pre-compaction, sole rollup 200 for post-compaction request B, B's row
absent) now totals 300 instead of 200, and still totals 300 once B lands.
Event names taken from @github/copilot 1.0.80, not guessed:
session.compaction_start (nothing needed from it) and
session.compaction_complete, whose `success` is read and every other field
ignored — the payload also carries the summarization call's own usage, token
counts, model and trigger. Only success:true anchors; failed or absent falls
back to the previous leg; last compaction in a leg wins. The stamp rides on
the cached rollup call as `compactedAt` (add-only, validated optional).
Accepted, bounded over-serve, documented at the doc site: if the summarization
call writes its own store row it sits just before the compaction stamp, so it
falls outside the interval while the post-reset rollup counts it — one request
per compaction can serve twice. Not subtracted from compactionTokensUsed,
because the request that TRIGGERED the compaction also completes immediately
before it: any grace window wide enough to catch the summarization row also
catches a real request and turns an over-serve into a loss. The row writer is
native, so the bundle cannot settle it; a real store can.
(c1) pins all four shapes: the repro at 300 (and again through the cache), no
double once the row lands, a failed compaction ignored, and a leg with no
compaction unchanged. The existing 107 copilot tests stay green.
A session.shutdown rollup is reconciled against the store rows written up to
it, and the two live in different files. A session that shuts down mid-pass
appends its rollup to events.jsonl and its last row to the store, so whichever
we read first is the one that can be short — and the short direction
under-serves a compacted session, whose sole rollup covers only its
post-compaction requests while the missing row's usage has nowhere else to
come from. A day finalized on that pass keeps the undercount, because a sealed
day is only ever re-derived by a version bump.
Two halves, one rule: the durable record is read last.
- Changed stores parse after every other changed source of their provider, so
the row set is a superset of anything a rollup we read can claim. The short
direction becomes unreachable; only the harmless one is left (a row whose
journal partner has not landed, which serves its own tokens and pairs next
pass).
- A store served from cache is never re-read, so ordering cannot help it. If
it moves between its classification stat and the end of the pass, the rows
the journals were just reconciled against are stale — hold the daily
watermark instead of sealing. It lifts by itself on the next refresh, which
sees the store as changed and re-reads it last.
(f3) pins both: the flagged source parses last whatever the discovery order,
and a cache-served store mutated from inside another source's parse reports
partial hydration, then clears.
Not covered, deliberately: a store that is behind its journal for reasons no
local evidence can show (rows pruned, a backdated fixture). rollup > rows in a
leg's interval is indistinguishable from legitimate partial coverage — the
store adopted mid-session has exactly that shape permanently — so fencing on
it would hold the watermark forever. Anchoring a leg's interval at the
compaction that reset it is the real fix and needs the compaction event's name
from a machine that has Copilot data.
A parse-version or env-fingerprint change rebuilds the provider section, and
the durable carry-forward only kept entries whose source PATH had disappeared.
A durable SQLite source does not disappear: copilot's agent-traces.db and
session-store.db keep their file while the CLI prunes rows out of them. So
"the path exists" was being read as "the source can re-derive this", and the
bump deleted, permanently, exactly the history only the cache still held.
Carry every durable entry forward. For a source still on disk, park an
impossible fingerprint on it so the new parse version re-reads it in full; the
durable union merge then appends only turns whose dedup keys are not already
cached, which adds what the source still holds without duplicating it or
deleting what it has lost. The persisted `durable` flag is carried too, so the
orphan bootstrap does not have to wait for the section to be re-stamped.
The contract this rests on is stated at the site: a parse-version bump must not
re-key calls the source can still re-derive, or the union counts them twice.
(f2) pins both directions on the reviewer's shape — a pruned-but-extant DB
keeps its history across a bump, served AND persisted, and an intact DB
re-read under the same bump does not double.
Resolved against main's newer invariants:
- daily-cache: DAILY_CACHE_VERSION/MIN_SUPPORTED -> 21. Main shipped 19
(#1015 grok); 18 was burned by this branch's earlier public head and 20 is
claimed by the unmerged #1040, and isMigratableCache would adopt either as
finalized without re-deriving. Header keeps main's v19 note and adds v21's.
- session-cache: PROVIDER_PARSE_VERSIONS keeps main's grok/dsh entries and
appends this branch's `-session-store-v2` to main's copilot value.
- parser: durable age-out takes main's orphan-only rule (#992).
retainWhilePresent is now redundant under it rather than load-bearing.
- audit-report: both imports (isBehavioralCall + sanitizeModelForDisplay).
- tests/parser (e), (l), (sc): re-pinned to the orphan-only age-out. A
still-discovered >90d copilot events.jsonl now keeps serving its per-turn
output alongside the store rows; the (sc) age-out subject is now an orphan.
- daily-cache-version-rederivation: seed 20 so the adjacent-draft-version
case is what the test pins.
- scripts/upgrade-path: expected daily cache filename -> v21.
The Copilot CLI and the GitHub Copilot desktop app both write
~/.copilot/session-store.db unconditionally; its assistant_usage_events
table holds one row per API request. Until now input/cache tokens for
these surfaces came only from the session.shutdown rollups in
events.jsonl, which are written only on clean shutdown (a crash loses
the whole leg's input/cache accounting) and lump each session leg into
one per-model total. The rollup also RESETS its counters at in-session
compaction (traced on a clean single-process 107-request session whose
sole rollup covered exactly its five post-compaction requests), so even
cleanly-closed long sessions were truncated; on a long-history machine
the store recovered ~35% of real Copilot spend lost to crashes and
compaction resets. The DB rows are per-request, crash-proof, and carry
real timestamps.
The store's input_tokens is cache-INCLUSIVE (input + cache_read +
cache_write), the same convention as the shutdown rollups — verified
against each row's token_details_json and by reconciling per-session
sums against the CLI's own footers and rollups across two machines
(1,380+ rows, 8 models, CLI 1.0.70–1.0.79, schema_version 6): every
divergence was a rollup gap. Emitted calls mirror the shutdown-call
contract: input/cache/reasoning only, output 0 — per-turn output stays
owned by the events.jsonl assistant.message calls.
Rollup-vs-store precedence is RECONCILED at serve time, per
(session, model), and only there. Both representations always parse and
cache; parseProviderSources aggregates the cached calls and, wherever
store rows exist for a (session, model), drops the rollup calls and
serves the rows plus per-leg RESIDUAL calls: each rollup leg subtracts
only the rows in its own interval — rows commit strictly before their
leg's shutdown line, so a leg at time T covers exactly the rows in
(previous leg's T, T] — and any remainder (per token component, floored
at zero) serves once at that leg's own timestamp. A store missing
requests a leg covered — adopted mid-session, rows pruned before ever
being read — therefore still serves that tail exactly once ON THAT
LEG'S DAY, a crash-tail row the rollup never saw can never cancel it,
and a complete store serves pure per-request granularity with every
residual retired to zero. The decision reads only cached contents, never discovery:
deleting or resetting the store changes nothing served, so finalized
daily history can never flip on an absence epoch; cached rows of a
deleted store remain the record until the 90-day orphan age-out (which
exempts still-discovered paths). The serve set is the one coherent
snapshot — nothing a writer does between discovery and a parse can
change what one pass sees — and read-time precedence heals persisted
duplication (stale epochs, runtimes without node:sqlite, restored
files) instead of preserving it, following the buildDurablePeriod
pattern.
Store rows and rollups carry supplementary accounting weight. A rollup
(or its residual) is aggregate accounting, never a request: zero
api-call/model-call/turn weight, tokens and cost fully retained. A
store row is one real request, but when it pairs with a served per-turn
call it is supplementary too; rows pair with same-model per-turn calls
by timestamp adjacency (monotone matching, tight 2-minute window — the
two are written at the same completion moment, and a wide window would
let a crash-only row pair against a neighbor whose own row is missing),
computed once over the FULL serve set so a date-range boundary that
separates a row from its call cannot double the request across adjacent
day queries. Only the unpaired rows — store-only requests, exactly
where crash-lost requests sit — count. Supplementary-only turns fold
into the nearest behavioral turn within 30 minutes; with no behavioral
turn to fold into they stay separate weightless turns, each on its own
day, with apiCalls 0 — and the session emission gate admits
usage-bearing zero-call sessions. The weight
propagates into the daily cache: aggregateProjectsIntoDays applies the
same rule to every calls counter and category-turn count it seals, so
v19 history and live summaries can never disagree about what was a
request.
A changed source whose read defers on the busy shape (locked, EACCES,
corrupt mid-replace — discovery still emits the source; only true
absence or a schema mismatch reads as absent) now marks session
hydration incomplete, so the daily backfill holds its watermark instead
of finalizing a day the deferred rows never reached; an unchanged
unreadable store defers nothing. The verdict travels with its result —
the 180s memo and the serve burst-reuse restore the hydration verdict
their cached data was parsed under, so a memoized partial parse cannot
inherit a later parse's complete — and a discovered source whose
FINGERPRINT cannot be read (EACCES on a present file) defers instead of
silently skipping, while a genuinely deleted file stays a silent skip. Copilot reasoning tokens are no longer
double-billed at the report layer: they are a subset of the output the
per-turn calls already price, and copilot joins claude in the
reasoning-inside-output case of the query-time cost recompute.
Store dedup keys are content-discriminated —
copilot-store:<sid>:<rowId>:<fnv1a64(created_at|tokens|model)> —
because AUTOINCREMENT prevents id reuse only within one database
lifetime: a same-path DB reset reusing row ids now mints new keys
instead of the durable union swallowing the new usage, while a
byte-identical re-insert still collapses (64-bit: 32-bit FNV
collisions between plausible token tuples are constructible). Every
call of a session serves under one project label resolved at serve
time — the session-state-derived label when the serve set knows it,
else the store rows' own — so neither rows cached before events.jsonl
existed nor an events.jsonl orphaned by a session-state prune can
split the session across two grouping keys.
CODEBURN_COPILOT_SESSION_STORE_DB is read but deliberately NOT
fingerprinted, per the #927 ruling (any copilot fingerprint change
drops cached entries whose path still exists, destroying pruned history
only the cache holds); the read is allowlisted in the #927 guard, and
serve-time reconciliation makes repointing safe without a fingerprint —
the new store's rows parse on sight and the old path's entries persist
as durable orphans. The copilot parse version appends session-store-v2
and the daily cache bumps v17 → v19: per-day attribution, call counts
and costs all change against pre-store builds. 19, not 18: an earlier
pushed head of this PR already claimed v18 under different accounting,
and the carry-forward would adopt those days as finalized without
re-deriving them.
Verified by A/B on snapshots of two real stores, a live SIGKILL crash
test (row present, no rollup, tokens recovered exactly), live resumes
whose warm-cache deltas matched new rows to the token, upgrade-healing
at 4,800-session scale, and serve-level regressions pinning every
maintainer finding from six review rounds: the rows-then-shutdown race,
stale-cache healing, age-out exemption, absence-epoch identity,
progressive row landing with residual retirement, behavioral weight
across all four pinned scenarios, the hydration fence, project
unification in both directions, the same-path reset, mixed
coverage (crash tail vs covered-leg gap), multi-leg residual day
attribution, range-invariant pairing, memo-scoped hydration verdicts,
and the fingerprint-failure fence.
A warm launch rewrote the entire session cache whenever any provider
appended a few KB: on a 6 GB corpus that is a 155 MB stringify + fsync
every run. The on-disk cache is now a version-suffixed directory holding
one shard per provider plus a small envelope, and a save rewrites only
the providers marked dirty.
- Dirtiness is tracked per provider (markCacheDirty) instead of one
global flag, so an appended Claude session no longer republishes
Codex, Copilot and the rest.
- Shards carry a nonce in their filename and the envelope is renamed
last, so a save is published at a single point: readers never see a
half-updated set, and a writer that loses the refresh ownership fence
leaves the canonical shards untouched.
- A shard that fails validation is treated as an absent provider rather
than rejecting the whole cache, so one malformed turn costs one
provider's re-parse instead of every provider's history.
- v7 migrates losslessly: the blob is re-laid-out into shards and
removed only once that save publishes. Nothing re-parses.
- Cold-parse progress saves now trigger every N files parsed rather than
every 5s, so a slow cold parse no longer rewrites the growing cache on
a wall clock.
The remaining warm-serve cost was the per-request discovery sweep
(stat-ing thousands of session files) plus re-aggregation, even when
nothing on disk had changed. Serve now watches every provider's
probeRoots() via fs.watch (FSEvents-backed recursive watches on macOS)
and injects a quiet-since validator into the parser: while the watched
roots are quiet, a previous parse stays reusable past the burst window,
and an output-level memo returns identical panel queries verbatim - so a
fetch with no data changes skips the sweep AND the aggregation.
Safety rails, in order: a parse is validated-reusable only if the
watchers were armed before it ran; any filesystem event ends reuse
instantly; a 5-minute hard cap self-heals a missed event; a root that
fails to watch just goes uncovered (shorter reuse, never staleness);
outside serve the validator is never installed and behavior is
byte-identical. During an active AI session the session roots fire
constantly, so reuse correctly stays inside the 10s burst window - the
extended reuse serves the idle-browsing case it was built for.
The one watched path inside the cache dir is antigravity's statusline
file specifically, so serve's own cache writes never self-invalidate.
Every CLI spawn on a large corpus pays seconds of fixed cost before any
query work: node boot, a 100MB+ session-cache JSON.parse, the discovery +
fingerprint sweep, and serve-time classification. The desktop app spawns
one CLI per panel fetch, so it pays that cost per panel.
codeburn serve --stdio is the same CLI kept warm: the app holds one child,
sends {id, args} per line, and gets the command's stdout back. Three layers
make it fast, each disabled outside serve so one-shot runs stay byte-exact:
- loadCache memo (session-cache.ts): the parsed cache object is reused
while a stat() shows the file unchanged; saveCache updates it
write-through. A rewrite by another process still forces a fresh read.
- burst reuse (parser.ts, CODEBURN_PARSE_BURST_MS, serve sets 10s): panel
bursts anchor their range ends at their own new Date(), so the exact-key
memo never hits in real traffic; within the window a re-anchored range is
served by trimming the previous parse instead of re-running discovery.
- fresh commander program per request (main.ts buildProgram factory),
because commander option state is sticky across parses.
The server allows only the app's read queries (status/overview/models/
sessions/compare/yield/spend/optimize/audit), refuses everything else
(client falls back to a spawn), serializes requests, and converts
process.exit into a caught signal. The app starts the child once at
startup; requests route through it only when warm, cold-start keeps the
spawn path with its progress events, any serve failure falls back to a
spawn, and three child deaths disable serve for the app run.
Measured on a real 17B-token corpus: panel fetches drop from ~7.4s per
spawn to 5-900ms warm (sessions/spend 5ms, status 898ms). One-shot CLI
output verified byte-identical against the pre-branch baseline.
- cli-durable-totals: the live fixture session was stamped at noon today,
so every before-noon run saw it in the future; the provider-filtered
path drops future instants while the all-provider path keeps the whole
day, failing the parity assertion. Relative-and-clamped timestamps,
the same fix project-filter-durable-totals got in 1596220.
- parser (copilot, 2 cases): the fixture's fixed 2026-05-01 dates crossed
copilot's durable 90-day age-out on 2026-07-30, so the first parse
pruned the freshly-cached session. Relative timestamps.
- parser-incremental-append: unlink-then-create let ext4 hand the freed
inode straight back, breaking the new-inode premise. The replacement
is now created beside the original and renamed over it.
- parser-proxy-pricing: normalizeProxyPath folds case only on darwin and
win32, deliberately; the test now asserts the platform-correct
behavior on both kinds of filesystem instead of hardcoding macOS.
- cli-status-menubar: the config-source filter case does real multi-parse
work and needs more than the 5s default on shared runners; 30s cap.
Claude is scanned via scanProjectDirs instead of parseProviderSources, and
that call had no provider-filter guard. On a --provider <other> run
discoverAllSessions correctly returns no claude sources, so claudeDirs is
empty, but scanProjectDirs still ran: its orphan pass reads the whole cached
claude section and treats every file as no-longer-discovered, re-injecting
PR-bearing entries (and in read-only mode every cached entry) into the result.
The headline stayed correct because it comes from the provider-sliced daily
cache, so only the live-parse panels were wrong. By Model then listed
Anthropic models under --provider cursor while the total showed cursor alone.
Guard the scan with claudeInScope, mirroring the guard the durable-orphan
loop already applies. Deliberately not a claudeDirs.length check: when claude
is in scope but every transcript has been pruned, the orphan pass is what
keeps PR-attributed spend from vanishing.
Skill attribution was gated on the turn classifying as 'general', but
classifyByToolPattern checks hasSkill last, so any turn invoking the Skill
tool alongside Read/Bash/Edit/MCP got a real category and its skill was
silently dropped. Skill turns virtually always carry other tools, which is
why skills came back empty in all 25 usage snapshots (#741) while mcpServers
populated.
subCategory is now set for any turn with Skill tool_use; category assignment
is untouched. The claude parse version is bumped so cached session summaries
(which lack skill attribution) re-parse on upgrade.
Performance and coherence:
- Settings > General 'Refresh every' (Manual/30s/1m/3m/5m/10m), live via
RefreshCadenceContext; Manual polls only on demand
- usePolled memoKey LRU: provider/period switches paint the last-good
result instantly with a switching hairline while refreshing quietly
- quota TTL 5min + honest rate-limited copy on 429 backoff
- version-suffixed cache files (session-cache.v5.json, daily-cache.v12,
auto-minted on future bumps); legacy files never written or deleted,
adopted once when versions match: old and new binaries coexist
without clobbering (field-observed menubar-vs-desktop ping-pong)
- advisory hydration lock: concurrent cold starts share one scan
(wait-then-read-warm), stale/dead locks self-heal, never a
correctness gate
Telemetry v1 + onboarding (desktop only, per owner decisions):
- first-launch onboarding (3 feature screens + consent); region-split
default (EU/EEA/UK/CH off, elsewhere on, unknown off); nothing sends
before consent completion or while off; dev builds never send
- anonymous install UUID, rotated on opt-out; day-granularity events,
cost buckets only; whitelisted names; 200-event queue, 5min flush
- Settings > Privacy live toggle replaces the static claim
Zero computed-number changes. App 316/316, root 1805.
Wire contract targets api.codeburn.app/v1/telemetry (Worker follows).
* fix(copilot): bill CLI input and cache tokens from session.shutdown
Copilot CLI sessions (~/.copilot/session-state/*/events.jsonl) recorded
only outputTokens, because the parser read assistant.message events and
ignored the session.shutdown rollup. That rollup is the sole on-disk
source of a CLI session's input, cache-read and cache-write tokens, so
cost was underreported (in the reported case 35k+ cache-read tokens were
dropped entirely).
Handle session.shutdown in the CLI (non-transcript) JSONL path and emit
one supplementary call per model from modelMetrics.<model>.usage. The
per-turn assistant.message events keep owning output tokens, tools and
turn structure; the shutdown call contributes only the input and cache
tokens they lack, with output excluded, so no dimension double-counts and
the combined cost equals the full CLI-measured cost. usage.inputTokens is
cache-inclusive (input + cache_read + cache_write), so the cache
components are subtracted to recover the uncached input calculateCost
expects. These are real counts written by the CLI, so costIsEstimated is
false. Sessions without session.shutdown (crashed or still running) keep
the exact previous output-only behavior, and the VS Code, JetBrains and
OTel paths are untouched.
Bump the copilot entry in PROVIDER_PARSE_VERSIONS so already-cached CLI
sessions re-parse and pick up the corrected tokens and cost.
* fix(copilot): never emit an empty-timestamp shutdown call
A session.shutdown without its own timestamp and without sessionStartTime
produced a call with an empty timestamp, which the date-range filters in
parser.ts drop silently, erasing exactly the tokens this feature bills.
Fall back to the last stamped event in the session.
* fix(cache): durable orphans survive provider fingerprint changes
Bumping a durable provider's PROVIDER_PARSE_VERSIONS changed its env
fingerprint, and getOrCreateProviderSection replaced the whole section on
mismatch. For copilot OTel orphans the discarded cache entry is the only
remaining record of pruned spans, so the first post-upgrade run silently
and permanently erased month-to-date history. Carry forward exactly the
entries whose source no longer exists; present sources still drop and
re-parse under the new fingerprint, which is the point of a bump.
Regression test forges a pre-upgrade fingerprint on a pruned-source
orphan and asserts it stays counted and cached across the transition.
* fix: fix and improve test isolation and collision with environment
* docs: remove unnecessary comment
* test(env-isolation): clear CODEBURN_FORCE_MACOS_MAJOR and pin TZ
Two env vars read in src/ were not isolated: CODEBURN_FORCE_MACOS_MAJOR
(now cleared so it cannot leak between tests) and TZ (now pinned to UTC,
since clearing it falls back to the OS zone and would shift date buckets
versus a clean CI runner).
---------
Co-authored-by: AgentSeal <hello@agentseal.org>
