Every per-file markCacheDirty call site now names the file, so a parse,
a re-parse, a failure marker, an orphan eviction and the durable age-out
each dirty exactly the month they touched. The two section-level marks
(a fingerprint reset, the durable stamp) stay provider-wide.
parseAllSessions derives a month scope from its dateRange and threads it
through every loadCache call, so a today/week query stops reading the
months it cannot report on.
A provider's shard held its whole history, so one appended session
rewrote 95 MB. Each provider's files are now split by the UTC month of
their first turn - a bucket that is stable across appends, so a growing
session never migrates shards - and every shard records the newest month
it holds so a ranged load can skip the ones that cannot contribute.
Dirty tracking is per bucket: markCacheDirty takes an optional file path
and marks both the bucket the entry was last saved in and the one it is
in now. A save writes only dirty buckets, carries the refs of months it
never loaded, and merges the on-disk shard back in when a bucket is
dirty but was never loaded. v8 and v7 caches re-lay-out losslessly.
A memo hit must not re-render. The generated stamp is minted per render,
so asserting it is unchanged across the hit states that contract directly
instead of leaving it implied by byte equality.
scanProjectDirs ran every cached turn through cachedTurnToClassified —
per-call reconstruction plus the turn classifier's category / retries /
edit regexes — and only then applied the date slice, so a week view paid
to classify all of history to keep a few percent of it.
The keep/drop decision now runs on the raw CachedTurn (calls map 1:1 onto
assistantCalls, so callsInRange sees the same survivors as the classified
slicer) and only survivors are classified, still from their complete call
list. The branch and PR-set carries still walk the full ordered turn list,
and buildSpawnPrSets still reads the pre-slice turns.
Real corpus, warm one-shot status --format menubar-json --no-optimize,
identical payloads: today 1561ms -> 1318ms, week 1560ms -> 1410ms,
month 1788ms -> 1675ms.
Two live processes share one cache directory routinely (a one-shot CLI
beside the resident serve child, two menubar polls), and the shard layout
had two ways to lose data there.
- The atomic write used a FIXED temp name, so two writers publishing the
envelope — every save does — shared one `envelope.json.tmp` and
interleaved into a torn payload, or one deleted the shards the other's
envelope named. 39 of 40 rounds ended in a total cache loss. The temp
name carries a nonce again, as it did before the shard layout.
- A save reused a shard filename from its own load snapshot without
checking the file was still there. Another process republishing that
provider unlinks the old shard, so the stale writer published an
envelope naming a deleted file — read back as a corrupt provider and
dropped whole, including PR-linked orphans no re-parse can recover. A
reused shard is now existence-checked, and re-verified once more
immediately before the envelope is published; a vanished one is
rewritten from memory.
Also:
- Progress saves take a 30s floor beside the file counter. Only the
claude scan reports per file; every other provider calls saveProgress
once at its own boundary, so the counter alone never fired there.
- The unreferenced-shard sweep waits an hour (temps still 5 minutes): an
unreferenced shard may belong to a concurrent save whose envelope has
not landed yet.
- The sweep also retires the pre-v8 single-file temps in the parent
directory, which nothing writes anymore.
- The shard directory is created 0o700.
- The claude and provider paths mark the cache dirty where they DELETE a
stale entry, not only where they replace it: an unreadable file skips
the replace, and the deletion would otherwise live only in memory.
- Codex only treats a grown file as an append when the recorded boundary
still lands just after a newline, so a same-inode rewrite that happens
to end up larger re-parses instead of resuming mid-line.
Codex rollout files are append-only and the active ones run to hundreds
of MB, but any growth re-read the file from byte 0 because the cache
keyed only on mtime+size. The parser now records a restart point at every
task_started boundary — the byte offset plus the state the single-pass
decode carries across it — and a grown file with the same dev/ino picks
up from there.
The boundary sits at the task_started line itself, so the task it opens
is re-decoded from the tail; the entry stores how many calls were decoded
before that point so the resumed run starts from exactly those and cannot
double-count the open task. An unusable or absent snapshot falls back to
a full re-parse.
CODEX_CACHE_VERSION is deliberately not bumped: the new fields are
additive and absence-safe both ways, so a bump would discard a warm
cache for nothing.
A warm launch rewrote the entire session cache whenever any provider
appended a few KB: on a 6 GB corpus that is a 155 MB stringify + fsync
every run. The on-disk cache is now a version-suffixed directory holding
one shard per provider plus a small envelope, and a save rewrites only
the providers marked dirty.
- Dirtiness is tracked per provider (markCacheDirty) instead of one
global flag, so an appended Claude session no longer republishes
Codex, Copilot and the rest.
- Shards carry a nonce in their filename and the envelope is renamed
last, so a save is published at a single point: readers never see a
half-updated set, and a writer that loses the refresh ownership fence
leaves the canonical shards untouched.
- A shard that fails validation is treated as an absent provider rather
than rejecting the whole cache, so one malformed turn costs one
provider's re-parse instead of every provider's history.
- v7 migrates losslessly: the blob is re-laid-out into shards and
removed only once that save publishes. Nothing re-parses.
- Cold-parse progress saves now trigger every N files parsed rather than
every 5s, so a slow cold parse no longer rewrites the growing cache on
a wall clock.
The separator regex retried its leading \s* from every offset, which is
quadratic on the long whitespace-heavy commands agents emit; that one regex
was ~30% of a warm status run on a multi-GB corpus. Match the separator
alone and widen over whitespace by hand. Output is unchanged (differential
check over 42k real commands).
Clear the per-directory Codex and Antigravity memo maps in the resident RSS
guard; document the single cache-dir rule (XDG_CACHE_HOME no longer
consulted, ledger migrated); stop output-overflow terminations from spending
the resident's unexpected-death budget.
flatSlice returned strings within the bound unchanged, but provider adapters
pre-truncate with .slice(0, 500) before the cache site, so those views still
pinned their parent buffers. Always flatten; the round-trip is ~150ns per turn.
Use utf16le so lone surrogates survive the copy.
Cache the canonical-path Promise instead of the resolved value so calls in one
Promise.all batch share a single walk.
Document the one-time kiro re-parse and worktree regrouping.
The kiro provider was reading the full working directory from session
metadata (meta.cwd for CLI sessions, workspacePaths[0] for v2 IDE
sessions, workspaceDirectory for workspace sessions) but discarding it
via basename(), keeping only the leaf name for display. This meant
computeAttributionRecords could never resolve kiro sessions to a git
repo, so `codeburn sync push --attribution` produced 0 facts for all
kiro-originated sessions.
Now passes the full path as projectPath on emitted ParsedProviderCalls,
which buildRepoGroups uses to resolve git identity and correlate
commits with sessions via timestamp windows. The path stays local:
only the normalized origin remote egresses in attribution spans.
Behavior changes beyond attribution:
- kiro calls now flow through canonicalizeProviderCallProject, so kiro
sessions in LINKED GIT WORKTREES canonicalize to the main repository:
their report project name changes from the worktree dir name to the
main repo name (consistent with claude/codex behavior).
- workingDirectory is now populated on kiro calls.
- PROVIDER_PARSE_VERSIONS.kiro bumped (project-path-v1): cached entries
predate projectPath and are served without re-invoking the parser, so
without the bump this fix silently no-ops for every warm cache. The
bump forces a one-time cold kiro re-parse on upgrade.
ORDERING: this commit must land WITH (or after) the preceding
SlicedString OOM fix. The forced cold re-parse it triggers is exactly
the workload that OOM'd before that fix on multi-GB kiro stores.
Perf: per-call canonicalization added a measured +5% to cold parse
(.git-marker lstat walk per call). resolveCanonicalProjectPath is now
memoized on cwd (cleared with the session cache), removing the
redundant walks for all providers.
Tests: projectPath emission fixtures for all three session formats
(CLI, v2 IDE, workspace-session), fingerprint-change assertion, and a
regression test seeding a pre-bump cache entry and proving the re-parse
recovers projectPath.
AI-Origin: human
String.prototype.slice returns a V8 SlicedString: a view that retains a
reference to its ENTIRE parent string. The parsers store short previews
of message text (userMessage.slice(0, 500/2000)) in the long-lived
session cache. Session files routinely carry 100KB+ strings (agent-
injected system prompts, tool results), so every cached preview pinned
its full parent buffer for the life of the process.
Measured on 3.2GB of kiro CLI session files (6,659 files, largest 40MB):
cold parse, default heap, before: 4.33GB peak -> OOM crash
cold parse, 8GB heap, before: 5.67GB peak (kiro provider alone)
after kiro flatSlice: 0.91GB peak
after parser.ts cache sites too: 0.64GB peak
original failing command (cold,
default heap, all providers): 0.89GB peak -> completes
Warm runs were always fine (~0.29GB) because the cache's JSON round-trip
flattens the strings on load — which made this bug appear intermittent:
it only fired on a cold or invalidated cache.
Fix: flatSlice() in content-utils.ts forces a flat copy via Buffer
round-trip. Applied at the six kiro userMessage capture sites and the
three shared cache-building sites in parser.ts (protects all providers).
Regression test asserts the no-retention property via bounded heap
growth over 1000 large-parent slices.
AI-Origin: human
Classic has no fitting store category (per forum feedback) and strict is
electron-builder's well-supported path. One personal-files plug grants
read-only access to the dot-directories the supported tools write session
logs into; the app's own config and cache stay in the snap's private area.
- electron-builder snap target (classic confinement) with package:snap script
- build-snap.yml: builds on ubuntu, uploads artifact, optional store publish
gated on workflow_dispatch input + SNAPCRAFT_STORE_CREDENTIALS secret
- app/flathub: manifest repacking the released deb, desktop file, AppStream
metainfo, and submission runbook
cli-json-daily, spend-flow and cli-emitters spawn the real CLI per test
and blow the 5s default under full parallel suite load while passing in
isolation - the flake set #948 documented on unmodified main, observed
again locally (cli-json-daily) and in CI (cli-emitters on a green PR).
Same file-level remedy the CLI menubar suite already uses; the default
stays 5s for everything else.
- Punchcard tooltips flipped below the cursor on the top rows (the
overflow container clips anything above its own edge) and clamp
horizontally near the strip edges. Applied to both the app and dash
copies of the component.
- The menubar Workflow strip is removed (view, strip model, tests):
the popover is the compact surface and the workflow metrics live in
the desktop app, dash and TUI. The payload keeps emitting the block
(add-only contract; the other surfaces read it).
The release's new capabilities were unevenly surfaced: the Spend
punchcard existed only on the web dashboard, and the menubar decoded
nothing of the PR-attribution block the payload already carries.
- Desktop app: new Punchcard component (hour-of-day x weekday spend
matrix, ported from the dash and restyled to the app's tokens) on the
Spend page. Fed by a dedicated getTimeline bridge channel that fetches
the payload WITH history.timeline; every other fetch keeps the lean
--no-timeline path, and the serve child makes the extra fetch cheap.
Hides gracefully when the payload has no timeline (older CLI).
- Menubar: PullRequestsSection renders the top three PRs by attributed
spend under the Workflow strip; MenubarPayload now decodes the
pullRequests block (decodeIfPresent, so older payloads are unchanged).
Hidden when absent or empty.
Deliberately NOT ported: codex Tok/s (reads rollout files per session -
too heavy for payload cadence; stays a TUI/report analytics view) and
the punchcard in the menubar (a 7x24 matrix has no legible place in a
compact popover).
App suite 468 green (bridge channel pinned, mocks extended), swift
build + 156 tests green, CLI suite green.
The cwd evidence rule attributed ANY session sharing a checkout with a
PR-linked session, with no time bound - so a repo whose only captured PR
link was pasted once became a black hole: 129 of 131 sessions and a
month of unrelated work (~$7.4K direct, $11.2K displayed) attributed to
one PR, observed live on the desktop Pull requests tab.
Cwd anchors now carry the evidence sessions' own activity window (union
across evidence for the same PR set), and only sessions overlapping that
window plus a 6h pad inherit the PR. The rule's charter is 'a tool
session launched around PR work in this checkout', which is inherently a
same-working-stretch claim; the design's own philosophy (timestamps
narrow, never create) now applies to this rule too.
On the real corpus the row corrected to $450.87 / 21 sessions across the
PR's actual two-day working stretch. Regression pins both directions:
nearby same-cwd session inherits, weeks-later one never does; multiple
evidence sessions widen the window.
The remaining warm-serve cost was the per-request discovery sweep
(stat-ing thousands of session files) plus re-aggregation, even when
nothing on disk had changed. Serve now watches every provider's
probeRoots() via fs.watch (FSEvents-backed recursive watches on macOS)
and injects a quiet-since validator into the parser: while the watched
roots are quiet, a previous parse stays reusable past the burst window,
and an output-level memo returns identical panel queries verbatim - so a
fetch with no data changes skips the sweep AND the aggregation.
Safety rails, in order: a parse is validated-reusable only if the
watchers were armed before it ran; any filesystem event ends reuse
instantly; a 5-minute hard cap self-heals a missed event; a root that
fails to watch just goes uncovered (shorter reuse, never staleness);
outside serve the validator is never installed and behavior is
byte-identical. During an active AI session the session roots fire
constantly, so reuse correctly stays inside the 10s burst window - the
extended reuse serves the idle-browsing case it was built for.
The one watched path inside the cache dir is antigravity's statusline
file specifically, so serve's own cache writes never self-invalidate.
Two more surfaces adopt the resident-serve pattern the desktop app got:
- Web dashboard: every period tab is prefetched sequentially right after
startup, so the first click on 7d/30d/Month answers from the payload
cache instead of paying a full parse; stale-while-revalidate rebuilds
behind a served payload past 75% of the TTL so expiry never lands its
multi-second parse on a user's click. Lifetime prefetches last.
- Menubar: ServeConnection (Swift actor) holds one codeburn serve --stdio
child; status payload fetches route through it once warm, with the same
contract as the app client — cold start and every failure keep the
spawn path, three child deaths disable serve for the run, requests
time out by killing the child, app termination shuts it down, and a
pre-serve CLI (0.9.19) simply dies into permanent spawn fallback, so
mixed-version installs degrade gracefully.
swift build clean, swift test 156/156, CLI tsc clean; verified live with
both the Electron app's and the menubar's serve children resident and
answering.
