vrr/codeburn
2
0
Fork 0
mirror of https://github.com/AgentSeal/codeburn.git synced 2026-08-21 06:24:32 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge remote-tracking branch 'origin/main' into pr1001-review

Browse source
This commit is contained in:
iamtoruk 2026-08-17 10:49:29 -07:00
commit 56e291fc7c
67 changed files with 8298 additions and 868 deletions
Download patch file Download diff file Expand all files Collapse all files

17
CHANGELOG.md
View file

@ -2,6 +2,23 @@
## Unreleased
### Changed
- **Codex rollouts parse across worker threads too, and the workload gate now takes bytes or files.** Codex is the bigger half of a real cold parse — a 4 GB rollout corpus against 1.8 GB of Claude sessions — and it was still decoding one file at a time. A whole-file rollout decode now runs on the same pool, against an empty dedup set, and comes back with the calls, the dedup keys it claimed, and the codex-cache entry it would have written; the parent installs all three in the serial loop's order, so `codex-results.json` and every payload come out byte-identical to a serial run. Cross-file state stays where it was: a forked rollout replaying its parent's token_count history collides on the parent's keys and is re-parsed in-process, and no worker ever touches the cache module's per-directory state. Files the Codex cache can serve exactly or resume into from a byte offset never reach a worker — they read a few KB and the resume state belongs to the parent. The workload gate is now pending BYTES alone (200 MB), not file count: 250 pending files holding under a megabyte between them spawned threads that made the run ~5% slower, while a few hundred huge rollouts were being turned away. The count takes `max(pendingFiles / 50, pendingBytes / 200 MB)`, and the per-thread memory budget is derived per parse as `clamp(256 MB, 2 × average pending file + 128 MB, 1 GB)` rather than a flat 256 MB — a 260 MB rollout peaks near 430 MB in its worker and scales linearly with the pool, so the flat figure over-subscribed exactly the workload this adds. The decision is per provider, and at most one pool is alive at a time.
- **A large cold Claude parse now runs across worker threads.** Reading, decoding and line-parsing a session JSONL is per-file work that never touches anything shared, so it moves onto `worker_threads`; each worker ships its parsed turns back as a JSON string and the parent installs them in the exact order the serial loop would. Everything with cross-file state — the streaming-message dedup, canonical project paths, spawn links, PR correlation, progress saves — stays on the main thread, and a file whose message ids were already claimed by an earlier file (or whose worker failed) is simply re-parsed in-process, so the session cache and every payload are identical either way. On a 6 GB corpus a cold `status` drops from 27.5s to 14.8s with peak RSS up 2.27 GB → 2.52 GB. Threads only engage for a genuinely large cold parse: never with under 200 MB behind the pending whole-file re-parses, 2 or fewer cores, or under 4 GB of available memory — so warm and incremental runs are untouched and spawn nothing. Otherwise the count is `min(cores - 1, min(0.25 × available, 2 GB) / 256 MB, pendingFiles / 50)`, where available is `process.availableMemory()` (cgroup-aware in containers) rather than free memory, which on macOS reports free pages and would switch the feature on and off between runs. `CODEBURN_PARSE_WORKERS=0` forces the serial parse and `CODEBURN_PARSE_WORKERS=N` forces N (capped at the core count), both bypassing every gate; `CODEBURN_VERBOSE=1` prints the resolved count and why.
- **A warm launch rewrites only the month that changed, and a ranged query reads only the months it can report on.** Per-provider shards still meant one appended session republished that provider's entire history — 95 MB for Claude on a 6 GB corpus. Each provider's shard is now split again by the UTC month of the cached session's FIRST turn, a bucket that never moves as a session grows, so an append rewrites one month. Every shard records the newest month it holds, which lets `--period today/week` skip the shards that cannot contribute a turn to the range; the skipped months stay on disk untouched across the save, and providers whose cache is the only surviving record (durable) or whose parse fingerprint moved are always read in full. Remaining shards are read concurrently. Existing v8 and v7 caches are re-laid-out losslessly on first load and the old layout removed once the new one is published: nothing re-parses.
- **A warm launch rewrites only the provider that changed.** The session cache was a single blob, so any provider appending a few KB republished the whole thing — 147 MB of stringify + fsync on a 6 GB corpus, ~18% of a warm run. It is now a version-suffixed directory holding one shard per provider plus a small envelope, written per provider and published by a single envelope rename. An existing v7 cache is re-laid-out losslessly on first load and the old file removed once the new layout is on disk: nothing re-parses. One unreadable shard now costs that provider a re-parse instead of discarding every provider's history, and partial saves during a cold parse are triggered every 2000 files rather than every 5 seconds, so a slow cold parse no longer rewrites the growing cache on a wall clock.
- **An appended Codex rollout parses only its tail.** Rollout files are append-only and the active ones run to hundreds of MB, but the Codex result cache keyed on mtime + size alone, so any growth re-read the file from byte 0. Each entry now records a restart point at the last task boundary — byte offset plus the state the single-pass decode carries across it — and a grown file with the same inode resumes there, producing output identical to a full re-parse. An entry without a usable restart point simply re-parses in full once and gains one.
- **A date-ranged report classifies only the turns it keeps.** Every cached turn went through the turn classifier — category, retries, edit detection, and a full reconstruction of its API calls — before the date slice discarded most of them, so a week view paid to classify all of history to keep a few percent of it. The keep/drop decision is now taken on the raw cached turn and only the survivors are classified, still from their complete call list, with the branch and pull-request carries still walking the full ordered turn list. Output is byte-identical.
- **One rule for every cache file.** `CODEBURN_CACHE_DIR` when set, otherwise `~/.cache/codeburn`. `XDG_CACHE_HOME` is no longer consulted; the sync ledger, the only file that ever honored it, is merged into the canonical location on first read and the legacy copy is retired, so nothing is re-uploaded after the move. (#972)
### Fixed (Desktop & Menubar)
- **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972)
### Fixed
- **Bash command splitting was quadratic on long whitespace-heavy commands.** The separator regex retried its leading `\s*` from every offset; matching the separator alone and widening over whitespace by hand makes cold parse ~24% and warm ~40% faster on large corpora, output unchanged.
- **Cold parse no longer retains full message bodies through cached previews.** `flatSlice` skipped its Buffer round-trip for strings already within the bound, but provider adapters pre-truncate user-message previews with `.slice(0, 500)` before the cache-site call — those pre-sliced views are still V8 SlicedStrings pinning their large parent, so the retention that OOM'd cold parses of large histories survived. The round-trip now always runs.
- **Kiro sessions carry the real `projectPath`** (CLI meta.cwd, v2 `workspacePaths[0]`, workspace sessions' `workspaceDirectory`), so git-repo attribution can resolve them; previously they were attribution-blind. Bumps the kiro parse version, so the first run after upgrade re-parses kiro history once, and kiro sessions in linked git worktrees now group under the main repo.
## 0.9.20 - 2026-08-10
### Added

596
app/electron/cli.test.ts
View file

@ -1,10 +1,10 @@
// @vitest-environment node
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, chmodSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join, isAbsolute, relative, win32, posix } from 'node:path'
import { spawnCli, spawnCliAction, spawnEnvFor, spawnSpecFor, killAll, CliError, nodeManagerDirs, notFoundStage, resolveCodeburnPath, resolveTarget } from './cli'
import { spawnCli, spawnCliAction, spawnEnvFor, spawnSpecFor, startServe, killAll, shutdownAll, CliError, nodeManagerDirs, notFoundStage, resolveCodeburnPath, resolveTarget } from './cli'
let dir: string
const originalBin = process.env.CODEBURN_BIN
@ -23,6 +23,61 @@ function fakeBin(name: string, body: string): string {
return p
}
function readMaybe(path: string): string {
try { return readFileSync(path, 'utf8') } catch { return '' }
}
async function waitFor(condition: () => boolean, timeoutMs = 3_000): Promise<void> {
const deadline = Date.now() + timeoutMs
while (!condition()) {
if (Date.now() > deadline) throw new Error('waitFor timed out')
await new Promise(resolve => setTimeout(resolve, 10))
}
}
/** A protocol-faithful fake CLI whose serve child accepts requests before its
* delayed ready frame. Files expose process starts and heavy request executions
* without relying on timing or private ServeClient internals. */
function fakeResidentBin(): {
startsFile: string
heavyFile: string
oneShotsFile: string
actionsFile: string
serveEnvFile: string
} {
const startsFile = join(dir, 'serve-starts')
const heavyFile = join(dir, 'heavy-requests')
const oneShotsFile = join(dir, 'one-shot-reads')
const actionsFile = join(dir, 'actions')
const serveEnvFile = join(dir, 'serve-progress-env')
fakeBin(
'resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
const command = process.argv[2];
if (command === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const generation = fs.readFileSync(${JSON.stringify(startsFile)}, 'utf8').length;
fs.writeFileSync(${JSON.stringify(serveEnvFile)}, process.env.CODEBURN_PROGRESS || '');
const rl = readline.createInterface({ input: process.stdin });
rl.on('line', line => {
const request = JSON.parse(line);
fs.appendFileSync(${JSON.stringify(heavyFile)}, 'h');
const progress = 'CODEBURN_PROGRESS ' + JSON.stringify({ kind: 'provider', provider: 'claude', state: 'start', generation }) + '\\n';
process.stdout.write(JSON.stringify({ id: request.id, progress }) + '\\n');
process.stdout.write(JSON.stringify({ id: request.id, ok: true, output: JSON.stringify({ via: 'serve', generation, args: request.args }) }) + '\\n');
});
setTimeout(() => process.stdout.write(JSON.stringify({ ready: true, pid: process.pid }) + '\\n'), 100);
} else if (command === 'currency') {
fs.appendFileSync(${JSON.stringify(actionsFile)}, 'a');
process.stdout.write('currency updated');
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write(JSON.stringify({ via: 'spawn', command }));
}`,
)
return { startsFile, heavyFile, oneShotsFile, actionsFile, serveEnvFile }
}
/** Writes the repo CLI under this test's isolated dev-root override. */
function fakeDevRepoCli(): string {
const repoRoot = join(dir, 'dev-repo')
@ -39,6 +94,7 @@ beforeEach(() => {
})
afterEach(() => {
killAll()
if (originalBin === undefined) delete process.env.CODEBURN_BIN
else process.env.CODEBURN_BIN = originalBin
if (originalPathDirs === undefined) delete process.env.CODEBURN_PATH_DIRS
@ -357,19 +413,19 @@ describe('spawnCli coalescing (read-only)', () => {
expect(readFileSync(countFile, 'utf8')).toBe('x') // exactly one spawn
})
it('spawns again once the 5s result cache has expired', async () => {
vi.useFakeTimers({ toFake: ['Date'] })
try {
const countFile = join(dir, 'spawns')
fakeBin('counter-ttl.js', `require('fs').appendFileSync(${JSON.stringify(countFile)},'x'); process.stdout.write(JSON.stringify({ok:1}))`)
vi.setSystemTime(0)
await spawnCli(['status'])
vi.setSystemTime(6_000)
await spawnCli(['status'])
expect(readFileSync(countFile, 'utf8')).toBe('xx') // cache expired → new spawn
} finally {
vi.useRealTimers()
}
it('reflects an external config change on the next same-argv read', async () => {
const configFile = join(dir, 'external-config')
const countFile = join(dir, 'spawns')
writeFileSync(configFile, 'before')
fakeBin(
'external-config.js',
`const fs = require('node:fs'); fs.appendFileSync(${JSON.stringify(countFile)}, 'x'); process.stdout.write(JSON.stringify({ value: fs.readFileSync(${JSON.stringify(configFile)}, 'utf8') }))`,
)
await expect(spawnCli(['model-alias', '--list'])).resolves.toEqual({ value: 'before' })
writeFileSync(configFile, 'after')
await expect(spawnCli(['model-alias', '--list'])).resolves.toEqual({ value: 'after' })
expect(readFileSync(countFile, 'utf8')).toBe('xx')
})
it('never coalesces config-mutating action calls', async () => {
@ -379,14 +435,433 @@ describe('spawnCli coalescing (read-only)', () => {
expect(readFileSync(countFile, 'utf8')).toBe('xx') // two independent spawns
})
it('flushes the read cache when an action completes, so post-action refetches are fresh', async () => {
it('runs a fresh read after a config-mutating action', async () => {
const countFile = join(dir, 'spawns')
fakeBin('mixed.js', `require('fs').appendFileSync(${JSON.stringify(countFile)},'x'); process.stdout.write(JSON.stringify({ok:1}))`)
await spawnCli(['model-alias', '--list']) // primes the 5s cache
await spawnCliAction(['model-alias', 'a', 'b']) // config change → cache flush
await spawnCli(['model-alias', '--list']) // must NOT serve the pre-action cache
await spawnCli(['model-alias', '--list'])
await spawnCliAction(['model-alias', 'a', 'b'])
await spawnCli(['model-alias', '--list'])
expect(readFileSync(countFile, 'utf8')).toBe('xxx')
})
it('fences old in-flight reads across a mutation without deleting the new flight', async () => {
const configFile = join(dir, 'generation-config')
const startsFile = join(dir, 'generation-read-starts')
const releaseDir = join(dir, 'generation-release')
mkdirSync(releaseDir)
writeFileSync(configFile, 'old')
fakeBin(
'generation-fence.js',
`const fs = require('node:fs'); const path = require('node:path');
if (process.argv[3] === '--list') {
const value = fs.readFileSync(${JSON.stringify(configFile)}, 'utf8');
fs.appendFileSync(${JSON.stringify(startsFile)}, 'r');
const generation = fs.readFileSync(${JSON.stringify(startsFile)}, 'utf8').length;
const release = path.join(${JSON.stringify(releaseDir)}, String(generation));
const timer = setInterval(() => {
if (!fs.existsSync(release)) return;
clearInterval(timer);
process.stdout.write(JSON.stringify({ value, generation }));
}, 5);
} else {
fs.writeFileSync(${JSON.stringify(configFile)}, 'new');
process.stdout.write('updated');
}`,
)
const oldRead = spawnCli(['model-alias', '--list'])
await waitFor(() => readMaybe(startsFile) === 'r')
await expect(spawnCliAction(['model-alias', 'alias', 'model']))
.resolves.toMatchObject({ ok: true })
const newRead = spawnCli(['model-alias', '--list'])
await waitFor(() => readMaybe(startsFile) === 'rr')
writeFileSync(join(releaseDir, '1'), '')
await expect(oldRead).resolves.toEqual({ value: 'old', generation: 1 })
// Settling the superseded flight must not remove the current generation's
// entry: this identical call still shares read #2 instead of spawning #3.
const coalescedNewRead = spawnCli(['model-alias', '--list'])
await new Promise(resolve => setTimeout(resolve, 100))
expect(readMaybe(startsFile)).toBe('rr')
writeFileSync(join(releaseDir, '2'), '')
await expect(Promise.all([newRead, coalescedNewRead])).resolves.toEqual([
{ value: 'new', generation: 2 },
{ value: 'new', generation: 2 },
])
})
})
describe('resident serve single-flight', () => {
it('startServe is idempotent and creates only one resident child', async () => {
const files = fakeResidentBin()
startServe()
startServe()
const result = await spawnCli(['status', '--double-start'], { timeoutMs: 5_000 }) as { generation: number }
expect(result.generation).toBe(1)
expect(readMaybe(files.startsFile)).toBe('s')
expect(readMaybe(files.heavyFile)).toBe('h')
})
it('lazily starts a new resident after an unexpected death and one-shot fallback', async () => {
const startsFile = join(dir, 'serve-starts')
const oneShotsFile = join(dir, 'one-shot-reads')
fakeBin(
'dies-once-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
const command = process.argv[2];
if (command === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const generation = fs.readFileSync(${JSON.stringify(startsFile)}, 'utf8').length;
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', line => {
const request = JSON.parse(line);
if (generation === 1) process.exit(1);
process.stdout.write(JSON.stringify({ id: request.id, ok: true, output: JSON.stringify({ via: 'serve', generation }) }) + '\\n');
});
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write(JSON.stringify({ via: 'spawn' }));
}`,
)
startServe()
await expect(spawnCli(['status', '--first'], { timeoutMs: 5_000 }))
.resolves.toEqual({ via: 'spawn' })
await expect(spawnCli(['models', '--second'], { timeoutMs: 5_000 }))
.resolves.toEqual({ via: 'serve', generation: 2 })
expect(readMaybe(startsFile)).toBe('ss')
expect(readMaybe(oneShotsFile)).toBe('o')
})
it('gives the first resident status request the power-user cold timeout floor', async () => {
fakeBin(
'slow-cold-resident.js',
`const readline = require('node:readline');
if (process.argv[2] === 'serve') {
const rl = readline.createInterface({ input: process.stdin });
rl.on('line', line => {
const request = JSON.parse(line);
setTimeout(() => process.stdout.write(JSON.stringify({ id: request.id, ok: true, output: JSON.stringify({ via: 'serve' }) }) + '\\n'), 80);
});
} else {
process.stdout.write(JSON.stringify({ via: 'spawn' }));
}`,
)
startServe()
await expect(spawnCli(['status', '--cold-floor'], { timeoutMs: 20 }))
.resolves.toEqual({ via: 'serve' })
})
it('starts a queued resident timeout only after the request ahead settles', async () => {
fakeBin(
'serial-resident.js',
`const readline = require('node:readline');
if (process.argv[2] === 'serve') {
const rl = readline.createInterface({ input: process.stdin });
(async () => {
for await (const line of rl) {
const request = JSON.parse(line);
if (request.args.includes('--slow')) await new Promise(resolve => setTimeout(resolve, 400));
process.stdout.write(JSON.stringify({ id: request.id, ok: true, output: JSON.stringify({ via: 'serve', args: request.args }) }) + '\\n');
}
})();
} else {
process.stdout.write(JSON.stringify({ via: 'spawn' }));
}`,
)
startServe()
await expect(spawnCli(['status', '--warm'], { timeoutMs: 5_000 }))
.resolves.toMatchObject({ via: 'serve' })
const slow = spawnCli(['sessions', '--slow'], { timeoutMs: 1_000 })
const queued = spawnCli(['models', '--queued'], { timeoutMs: 200 })
const [slowResult, queuedResult] = await Promise.all([slow, queued])
expect(slowResult).toMatchObject({ via: 'serve' })
expect(queuedResult).toMatchObject({ via: 'serve' })
})
it('uses the first real request as the only heavy execution, even before ready', async () => {
const files = fakeResidentBin()
startServe()
const result = await spawnCli(['status', '--format', 'menubar-json'], {
timeoutMs: 5_000,
extraEnv: { CODEBURN_PROGRESS: '1' },
}) as { via: string; generation: number }
expect(result).toMatchObject({ via: 'serve', generation: 1 })
expect(readMaybe(files.startsFile)).toBe('s')
expect(readMaybe(files.heavyFile)).toBe('h')
expect(readMaybe(files.oneShotsFile)).toBe('')
expect(readMaybe(files.serveEnvFile)).toBe('1')
})
it('forwards serve progress frames through the read onStderr callback', async () => {
fakeResidentBin()
startServe()
const chunks: string[] = []
await spawnCli(['status'], {
timeoutMs: 5_000,
extraEnv: { CODEBURN_PROGRESS: '1' },
onStderr: chunk => { chunks.push(chunk) },
})
expect(chunks.join('')).toBe('CODEBURN_PROGRESS {"kind":"provider","provider":"claude","state":"start","generation":1}\n')
})
it('rejects and terminates a resident that emits an oversized valid JSON frame', async () => {
const startsFile = join(dir, 'oversized-frame-starts')
const oneShotsFile = join(dir, 'oversized-frame-one-shots')
fakeBin(
'oversized-frame-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
if (process.argv[2] === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const generation = fs.readFileSync(${JSON.stringify(startsFile)}, 'utf8').length;
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', line => {
const request = JSON.parse(line);
const output = generation === 1
? JSON.stringify({ value: 'x'.repeat(16 * 1024 * 1024 + 1024) })
: JSON.stringify({ generation });
process.stdout.write(JSON.stringify({ id: request.id, ok: true, output }) + '\\n');
});
setInterval(() => {}, 1000);
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write('{}');
}`,
)
startServe()
await expect(spawnCli(['status', '--oversized-frame'], { timeoutMs: 5_000 }))
.rejects.toMatchObject({ kind: 'too-large' } satisfies Partial<CliError>)
await expect(spawnCli(['status', '--after-oversized-frame'], { timeoutMs: 5_000 }))
.resolves.toEqual({ generation: 2 })
expect(readMaybe(startsFile)).toBe('ss')
expect(readMaybe(oneShotsFile)).toBe('')
})
it('keeps serve enabled after more overflows than the resident death budget', async () => {
const startsFile = join(dir, 'overflow-budget-starts')
const oneShotsFile = join(dir, 'overflow-budget-one-shots')
fakeBin(
'always-oversized-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
if (process.argv[2] === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', line => {
const request = JSON.parse(line);
const output = JSON.stringify({ value: 'x'.repeat(16 * 1024 * 1024 + 1024) });
process.stdout.write(JSON.stringify({ id: request.id, ok: true, output }) + '\\n');
});
setInterval(() => {}, 1000);
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write('{}');
}`,
)
startServe()
for (let attempt = 0; attempt < 4; attempt += 1) {
await expect(spawnCli(['status', '--overflow', String(attempt)], { timeoutMs: 5_000 }))
.rejects.toMatchObject({ kind: 'too-large' } satisfies Partial<CliError>)
}
// An overflow kill is deliberate, so it never spends the unexpected-death
// budget: the fourth request still reaches a resident, not a one-shot.
expect(readMaybe(startsFile)).toBe('ssss')
expect(readMaybe(oneShotsFile)).toBe('')
})
it('rejects and terminates a resident whose protocol line never terminates', async () => {
const startsFile = join(dir, 'unterminated-line-starts')
const oneShotsFile = join(dir, 'unterminated-line-one-shots')
fakeBin(
'unterminated-line-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
if (process.argv[2] === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', () => process.stdout.write('x'.repeat(16 * 1024 * 1024 + 1024)));
setInterval(() => {}, 1000);
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write('{}');
}`,
)
startServe()
await expect(spawnCli(['status', '--unterminated-line'], { timeoutMs: 5_000 }))
.rejects.toMatchObject({ kind: 'too-large' } satisfies Partial<CliError>)
expect(readMaybe(startsFile)).toBe('s')
expect(readMaybe(oneShotsFile)).toBe('')
})
it('keeps requests with any non-progress env override on the one-shot path', async () => {
const files = fakeResidentBin()
startServe()
const result = await spawnCli(['status'], {
timeoutMs: 5_000,
extraEnv: { CODEBURN_PROGRESS: '1', CODEBURN_TEST_MODE: 'isolated' },
}) as { via: string }
expect(result.via).toBe('spawn')
expect(readMaybe(files.heavyFile)).toBe('')
expect(readMaybe(files.oneShotsFile)).toBe('o')
})
it('treats empty and undefined-only env overrides as serve-compatible', async () => {
const files = fakeResidentBin()
startServe()
const empty = await spawnCli(['status', '--empty-env'], {
timeoutMs: 5_000,
extraEnv: {},
}) as { via: string }
const undefinedOnly = await spawnCli(['models', '--undefined-env'], {
timeoutMs: 5_000,
extraEnv: { CODEBURN_PROGRESS: undefined },
}) as { via: string }
expect(empty.via).toBe('serve')
expect(undefinedOnly.via).toBe('serve')
expect(readMaybe(files.heavyFile)).toBe('hh')
expect(readMaybe(files.oneShotsFile)).toBe('')
})
it('restarts the resident child after a successful config mutation', async () => {
const files = fakeResidentBin()
startServe()
const before = await spawnCli(['status'], { timeoutMs: 5_000 }) as { generation: number }
const action = await spawnCliAction(['currency', 'EUR'], { timeoutMs: 5_000 })
const after = await spawnCli(['status'], { timeoutMs: 5_000 }) as { generation: number }
expect(action).toMatchObject({ ok: true, stdout: 'currency updated', code: 0 })
expect(before.generation).toBe(1)
expect(after.generation).toBe(2)
expect(readMaybe(files.startsFile)).toBe('ss')
expect(readMaybe(files.heavyFile)).toBe('hh')
expect(readMaybe(files.actionsFile)).toBe('a')
})
it('preserves the unexpected-death budget across mutation restarts', async () => {
const startsFile = join(dir, 'serve-starts')
const oneShotsFile = join(dir, 'one-shot-reads')
fakeBin(
'crashing-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
const command = process.argv[2];
if (command === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', () => process.exit(1));
} else if (command === 'currency') {
process.stdout.write('currency updated');
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write(JSON.stringify({ via: 'spawn' }));
}`,
)
startServe()
for (let attempt = 0; attempt < 3; attempt += 1) {
await expect(spawnCli(['status', '--attempt', String(attempt)], { timeoutMs: 5_000 }))
.resolves.toEqual({ via: 'spawn' })
await expect(spawnCliAction(['currency', attempt % 2 === 0 ? 'EUR' : 'USD'], { timeoutMs: 5_000 }))
.resolves.toMatchObject({ ok: true })
}
// A mutation may replace a healthy child, but it must not erase real crash
// history and resurrect serve after the third unexpected death.
expect(readMaybe(startsFile)).toBe('sss')
await expect(spawnCli(['status', '--after-budget'], { timeoutMs: 5_000 }))
.resolves.toEqual({ via: 'spawn' })
expect(readMaybe(startsFile)).toBe('sss')
expect(readMaybe(oneShotsFile)).toBe('oooo')
})
it('stops lazy crash recovery after three consecutive resident deaths', async () => {
const startsFile = join(dir, 'serve-starts')
const oneShotsFile = join(dir, 'one-shot-reads')
fakeBin(
'always-crashing-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
if (process.argv[2] === 'serve') {
fs.appendFileSync(${JSON.stringify(startsFile)}, 's');
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', () => process.exit(1));
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write(JSON.stringify({ via: 'spawn' }));
}`,
)
startServe()
for (let attempt = 0; attempt < 4; attempt += 1) {
await expect(spawnCli(['status', '--lazy-crash', String(attempt)], { timeoutMs: 5_000 }))
.resolves.toEqual({ via: 'spawn' })
}
expect(readMaybe(startsFile)).toBe('sss')
expect(readMaybe(oneShotsFile)).toBe('oooo')
})
it('does not spawn a one-shot fallback after killAll destroys serve', async () => {
const requestSeenFile = join(dir, 'request-seen')
const oneShotsFile = join(dir, 'one-shot-reads')
fakeBin(
'shutdown-resident.js',
`const fs = require('node:fs'); const readline = require('node:readline');
if (process.argv[2] === 'serve') {
const rl = readline.createInterface({ input: process.stdin });
rl.once('line', () => { fs.writeFileSync(${JSON.stringify(requestSeenFile)}, '1'); });
} else {
fs.appendFileSync(${JSON.stringify(oneShotsFile)}, 'o');
process.stdout.write('{}');
}`,
)
startServe()
const pending = spawnCli(['status', '--shutdown'], { timeoutMs: 60_000 })
for (let attempt = 0; attempt < 400 && !readMaybe(requestSeenFile); attempt += 1) {
await new Promise(resolve => setTimeout(resolve, 5))
}
const requestSeen = readMaybe(requestSeenFile)
killAll()
expect(requestSeen).toBe('1')
await expect(pending).rejects.toMatchObject({ kind: 'nonzero' })
await new Promise(resolve => setTimeout(resolve, 25))
expect(readMaybe(oneShotsFile)).toBe('')
})
it('keeps the warm resident child after a successful export', async () => {
const files = fakeResidentBin()
startServe()
const before = await spawnCli(['status'], { timeoutMs: 5_000 }) as { generation: number }
const action = await spawnCliAction(['export', '-f', 'json', '-o', join(dir, 'usage.json')], { timeoutMs: 5_000 })
// A different panel query proves which resident generation handled the
// next served read without relying on same-request coalescing.
const after = await spawnCli(['models', '--format', 'json'], { timeoutMs: 5_000 }) as { generation: number }
expect(action.ok).toBe(true)
expect(before.generation).toBe(1)
expect(after.generation).toBe(1)
expect(readMaybe(files.startsFile)).toBe('s')
expect(readMaybe(files.heavyFile)).toBe('hh')
})
})
describe('killAll', () => {
@ -398,6 +873,43 @@ describe('killAll', () => {
killAll()
await expect(pending).rejects.toMatchObject({ kind: 'nonzero' })
})
it('terminal shutdown rejects new read and action races without spawning', async () => {
const startsFile = join(dir, 'starts')
fakeBin(
'shutdown-guard.js',
`require('node:fs').appendFileSync(${JSON.stringify(startsFile)}, 'x'); process.stdout.write('{}')`,
)
shutdownAll()
startServe()
await expect(spawnCli(['status', '--after-shutdown']))
.rejects.toMatchObject({ kind: 'nonzero' })
await expect(spawnCliAction(['currency', 'EUR']))
.resolves.toMatchObject({ ok: false, code: null })
expect(readMaybe(startsFile)).toBe('')
})
it('terminal shutdown cancels read and action slots admitted before their spawn microtask', async () => {
const startsFile = join(dir, 'starts-after-admission')
fakeBin(
'shutdown-after-admission.js',
`require('node:fs').appendFileSync(${JSON.stringify(startsFile)}, process.argv[2] + '\\n'); if (process.argv[2] === 'status') process.stdout.write('{}'); else process.stdout.write('updated')`,
)
// Both calls synchronously acquire the two free scheduler slots. Their
// actual spawn resumes in a microtask, which is exactly the before-quit race.
const read = spawnCli(['status', '--admitted'])
const action = spawnCliAction(['currency', 'EUR'])
shutdownAll()
await Promise.all([
expect(read).rejects.toMatchObject({ kind: 'nonzero' }),
expect(action).resolves.toMatchObject({ ok: false, code: null }),
])
expect(readMaybe(startsFile)).toBe('')
})
})
describe('spawnCli concurrency scheduler', () => {
@ -525,6 +1037,52 @@ describe('spawnCli concurrency scheduler', () => {
await delay(50)
expect(startedList(startedFile)).not.toContain('sessions') // never spawned
})
it('limits six simultaneous resident-failure fallbacks to two one-shot children', async () => {
const startedFile = join(dir, 'fallback-started')
const activeDir = join(dir, 'fallback-active'); mkdirSync(activeDir)
const activeCountsFile = join(dir, 'fallback-active-counts')
const releaseDir = join(dir, 'fallback-release'); mkdirSync(releaseDir)
fakeBin(
'failing-resident-with-blocked-fallbacks.js',
`const fs = require('node:fs'); const path = require('node:path'); const readline = require('node:readline');
if (process.argv[2] === 'serve') {
const rl = readline.createInterface({ input: process.stdin });
rl.on('line', line => {
const request = JSON.parse(line);
process.stdout.write(JSON.stringify({ id: request.id, ok: false, error: 'resident failed' }) + '\\n');
});
} else {
const id = process.argv[3];
fs.appendFileSync(${JSON.stringify(startedFile)}, id + '\\n');
const activeFile = path.join(${JSON.stringify(activeDir)}, String(process.pid));
fs.writeFileSync(activeFile, '');
fs.appendFileSync(${JSON.stringify(activeCountsFile)}, fs.readdirSync(${JSON.stringify(activeDir)}).length + '\\n');
const releaseFile = path.join(${JSON.stringify(releaseDir)}, id);
const timer = setInterval(() => {
if (!fs.existsSync(releaseFile)) return;
clearInterval(timer);
fs.unlinkSync(activeFile);
process.stdout.write(JSON.stringify({ via: 'spawn', id }));
}, 5);
}`,
)
startServe()
const requests = Array.from({ length: 6 }, (_, index) =>
spawnCli(['status', `fallback-${index}`], { timeoutMs: 5_000 }),
)
await waitUntil(() => startedList(startedFile).length >= 2)
await delay(150)
const admittedBeforeRelease = startedList(startedFile)
for (let index = 0; index < 6; index += 1) release(releaseDir, `fallback-${index}`)
await Promise.all(requests)
const activeCounts = startedList(activeCountsFile).map(Number)
expect(admittedBeforeRelease).toHaveLength(2)
expect(Math.max(...activeCounts)).toBeLessThanOrEqual(2)
})
})
describe('spawnCliAction', () => {

321
app/electron/cli.ts
View file

@ -54,11 +54,12 @@ export class CliError extends Error {
}
const DEFAULT_TIMEOUT_MS = 45_000
// The first status query may hydrate a power-user cache from scratch. Every
// resident request admitted before that succeeds shares this floor so a later
// short request cannot kill the child while it waits behind the cold scan.
export const DESKTOP_COLD_TIMEOUT_MS = 10 * 60_000
// A runaway CLI (or a compromised binary) must not exhaust main-process memory.
const MAX_OUTPUT_BYTES = 16 * 1024 * 1024
// Same-cadence pollers fire near-identical read spawns; share one child and hold
// its result briefly so six overview hooks don't launch six processes at once.
const COALESCE_TTL_MS = 5_000
// A cold-cache CLI spawn costs seconds at ~120% CPU; letting every poll +
// prefetch launch at once saturates the machine. Cap how many children run
// concurrently — the rest queue and drain as slots free (interactive first).
@ -67,7 +68,10 @@ const MAX_CONCURRENT_CLI = 2
// Every live child so `before-quit` can reap them (Electron does not on macOS).
const activeChildren = new Set<ChildProcess>()
const readInflight = new Map<string, Promise<unknown>>()
const readCache = new Map<string, { at: number; value: unknown }>()
// Successful mutations advance the epoch before their promise resolves. A read
// begun against older config may still settle for its original caller, but can
// never be reused by the post-mutation refetch or delete that newer flight.
let readGeneration = 0
// Concurrency scheduler. `running` counts spawned (not queued) children; waiters
// hold the slot-grant resolver for a queued spawn. Two queues so interactive
@ -76,6 +80,7 @@ type SlotWaiter = { resolve: () => void; reject: (err: unknown) => void }
let running = 0
const interactiveQueue: SlotWaiter[] = []
const backgroundQueue: SlotWaiter[] = []
let shuttingDown = false
/** Grant free slots to queued waiters, interactive first, up to the cap. */
function pumpSlots(): void {
@ -101,9 +106,8 @@ function releaseSlot(): void {
pumpSlots()
}
/** SIGKILL every in-flight child and cancel anything still queued for a slot.
* Wired to Electron's `before-quit`. */
export function killAll(): void {
/** Reap every child and cancel anything still queued for a slot. */
function reapAll(): void {
serveClient?.destroy()
serveClient = null
for (const child of activeChildren) child.kill('SIGKILL')
@ -117,6 +121,19 @@ export function killAll(): void {
for (const waiter of waiting) waiter.reject(new CliError('nonzero', 'codeburn cancelled'))
}
/** Test/dev cleanup that permits a later fresh start in this same process. */
export function killAll(): void {
shuttingDown = false
reapAll()
}
/** Terminal app shutdown: reap current work and reject any IPC race that arrives
* while Electron is still flushing telemetry before the final quit pass. */
export function shutdownAll(): void {
shuttingDown = true
reapAll()
}
// Homebrew + common Node version managers, mirroring mac/CodeburnCLI.swift so a
// GUI-launched app (minimal PATH) still finds a globally-installed `codeburn`.
export function nodeManagerDirs(): string[] {
@ -378,6 +395,25 @@ function runCli(spec: SpawnSpec, cmdLabel: string, timeoutMs: number, onStderr?:
})
}
/** Run a one-shot read under the global child cap. A slot grant resumes on a
* microtask, so terminal shutdown must be checked again immediately before the
* synchronous spawn call. */
async function runScheduledCli(
spec: SpawnSpec,
cmdLabel: string,
timeoutMs: number,
priority: SpawnPriority,
onStderr?: (chunk: string) => void,
): Promise<unknown> {
await acquireSlot(priority)
try {
if (shuttingDown) throw new CliError('nonzero', 'codeburn is shutting down')
return await runCli(spec, cmdLabel, timeoutMs, onStderr)
} finally {
releaseSlot()
}
}
/**
* Spawn `codeburn <args>` with plain argv (never a shell), collect stdout, and
* decode it as JSON. Rejects with a structured {@link CliError}:
@ -387,8 +423,9 @@ function runCli(spec: SpawnSpec, cmdLabel: string, timeoutMs: number, onStderr?:
* timeout the process was killed after `timeoutMs`
* too-large stdout+stderr exceeded {@link MAX_OUTPUT_BYTES}
*
* Read-only, so concurrent identical calls share one child and a 5s result cache
* absorbs same-cadence pollers. Never use this for config-mutating commands.
* Read-only, so concurrent identical calls share one child. Settled results are
* never cached here because config can also change outside the desktop app.
* Never use this for config-mutating commands.
*/
// ── Resident serve child ────────────────────────────────────────────────
// The heavy read queries (one per panel) each pay seconds of CLI startup on
@ -397,75 +434,141 @@ function runCli(spec: SpawnSpec, cmdLabel: string, timeoutMs: number, onStderr?:
// stdio and the cache stays parsed in the child. Routing rules keep this
// strictly an optimization:
// - only SERVE_ROUTED commands (the app's JSON panel queries) are eligible;
// - requests route through serve only once the child is READY AND WARM, so
// the cold-start path keeps its spawn (with its stderr progress events);
// - the first real panel request is also the cache warm-up, so startup never
// runs an artificial warm-up query beside a duplicate one-shot child;
// - progress frames from serve are forwarded through the same onStderr hook
// used by a one-shot cold start;
// - any serve failure falls back to a normal spawn for that call;
// - three child deaths permanently disable serve for this app run.
const SERVE_ROUTED = new Set(['status', 'models', 'sessions', 'compare', 'yield', 'spend', 'optimize', 'audit'])
const SERVE_REQUEST_TIMEOUT_MS = 60_000
const SERVE_MAX_RESTARTS = 3
class ServeClient {
private child: ReturnType<typeof spawn> | null = null
private pending = new Map<number, { resolve: (v: unknown) => void; reject: (e: Error) => void; timer: NodeJS.Timeout }>()
private pending = new Map<number, {
resolve: (v: unknown) => void
reject: (e: Error) => void
timer: NodeJS.Timeout
warmsServe: boolean
decodedBytes: number
onStderr?: (chunk: string) => void
}>()
private nextId = 1
private ready = false
private warm = false
private deaths = 0
private buffer = ''
private bufferBytes = 0
private warmed = false
private destroyed = false
private requestTail: Promise<void> = Promise.resolve()
constructor(private readonly spec: SpawnSpec) {}
isWarmAndReady(): boolean { return this.ready && this.warm && this.child !== null }
isRunning(): boolean { return this.child !== null }
disabled(): boolean { return this.deaths >= SERVE_MAX_RESTARTS }
isDestroyed(): boolean { return this.destroyed }
start(): void {
if (this.child || this.disabled()) return
if (this.child || this.disabled() || this.destroyed) return
const child = spawn(this.spec.bin, [...this.spec.args], { shell: false, stdio: ['pipe', 'pipe', 'ignore'], env: this.spec.env })
this.child = child
child.stdout!.setEncoding('utf8')
child.stdout!.on('data', (chunk: string) => this.onData(chunk))
const onGone = () => this.onDeath()
child.stdout!.on('data', (chunk: string) => {
// A replaced child's stream can drain after its exit callback. Never let
// those stale bytes repopulate the shared line buffer for the new child.
if (this.child === child) this.onData(child, chunk)
})
const onGone = () => this.onDeath(child)
child.on('exit', onGone)
child.on('error', onGone)
// Background warm-up: one cheap query makes the child parse the session
// cache once; every later panel fetch reuses the in-memory copy.
void this.request(['status', '--format', 'menubar-json', '--period', 'today'], SERVE_REQUEST_TIMEOUT_MS)
.then(() => { this.warm = true })
.catch(() => { /* warm-up failure just leaves routing on the spawn path */ })
}
private onData(chunk: string): void {
private onData(child: ReturnType<typeof spawn>, chunk: string): void {
this.buffer += chunk
this.bufferBytes += Buffer.byteLength(chunk)
let idx: number
while ((idx = this.buffer.indexOf('\n')) >= 0) {
const line = this.buffer.slice(0, idx).trim()
const rawLine = this.buffer.slice(0, idx)
this.buffer = this.buffer.slice(idx + 1)
const rawLineBytes = Buffer.byteLength(rawLine)
this.bufferBytes = Math.max(0, this.bufferBytes - rawLineBytes - 1)
if (rawLineBytes > MAX_OUTPUT_BYTES) {
this.terminateForOverflow(child)
return
}
const line = rawLine.trim()
if (!line) continue
let msg: { id?: number; ready?: boolean; ok?: boolean; refused?: boolean; output?: string; error?: string }
let msg: { id?: number; ready?: boolean; progress?: string; ok?: boolean; refused?: boolean; output?: string; error?: string }
try { msg = JSON.parse(line) } catch { continue }
if (msg.ready) { this.ready = true; continue }
if (msg.ready) continue
if (typeof msg.id !== 'number') continue
const waiter = this.pending.get(msg.id)
if (!waiter) continue
if (typeof msg.progress === 'string') {
if (!this.consumeDecodedOutput(child, waiter, msg.progress)) return
if (waiter.onStderr) {
try { waiter.onStderr(msg.progress) } catch { /* progress consumers never own the request */ }
}
continue
}
const terminalOutput = typeof msg.output === 'string' ? msg.output : typeof msg.error === 'string' ? msg.error : ''
if (!this.consumeDecodedOutput(child, waiter, terminalOutput)) return
this.pending.delete(msg.id)
clearTimeout(waiter.timer)
if (msg.ok && typeof msg.output === 'string') {
if (waiter.warmsServe) this.warmed = true
try { waiter.resolve(JSON.parse(msg.output)) }
catch { waiter.reject(new CliError('bad-json', 'codeburn produced output that was not valid JSON')) }
} else {
waiter.reject(new CliError('nonzero', msg.error ?? 'serve request failed'))
}
}
// Complete lines are bounded above before parsing. Bound the partial frame
// too, otherwise a child that never emits '\n' can grow this buffer forever.
if (this.bufferBytes > MAX_OUTPUT_BYTES) this.terminateForOverflow(child)
}
private onDeath(): void {
const child = this.child
private consumeDecodedOutput(
child: ReturnType<typeof spawn>,
waiter: { decodedBytes: number },
output: string,
): boolean {
waiter.decodedBytes += Buffer.byteLength(output)
if (waiter.decodedBytes <= MAX_OUTPUT_BYTES) return true
this.terminateForOverflow(child)
return false
}
private terminateForOverflow(child: ReturnType<typeof spawn>): void {
if (this.child !== child) return
const error = new CliError('too-large', `codeburn serve produced more than ${MAX_OUTPUT_BYTES} bytes`)
// Detach synchronously before SIGKILL. A new request may start the next
// generation immediately; the old child's eventual exit must not reject it.
this.child = null
this.ready = false
this.warm = false
this.deaths += 1
if (child) activeChildren.delete(child as never)
this.buffer = ''
this.bufferBytes = 0
this.warmed = false
// Deliberate termination, not a crash: it must not spend the unexpected-death
// budget, or three oversized payloads would disable serve for the app run.
activeChildren.delete(child as never)
for (const [, waiter] of this.pending) {
clearTimeout(waiter.timer)
waiter.reject(error)
}
this.pending.clear()
child.kill('SIGKILL')
}
private onDeath(child: ReturnType<typeof spawn>, countsTowardBudget = true): void {
// Both `error` and `exit` can fire for one child, and destroy() performs the
// same cleanup synchronously. Only the currently-owned child may transition
// this client or reject its pending requests.
if (this.child !== child) return
this.child = null
this.buffer = ''
this.bufferBytes = 0
this.warmed = false
if (countsTowardBudget) this.deaths += 1
activeChildren.delete(child as never)
for (const [, waiter] of this.pending) {
clearTimeout(waiter.timer)
waiter.reject(new CliError('nonzero', 'codeburn serve exited'))
@ -473,10 +576,32 @@ class ServeClient {
this.pending.clear()
}
request(args: string[], timeoutMs: number): Promise<unknown> {
restartAfterMutation(): void {
const child = this.child
if (child) {
// This is an intentional replacement, not a crash. Detach first so the
// later exit event cannot consume the unexpected-death budget.
this.onDeath(child, false)
child.kill('SIGKILL')
}
this.start()
}
request(args: string[], timeoutMs: number, onStderr?: (chunk: string) => void): Promise<unknown> {
// The stdio server is deliberately serial. Mirror that contract client-side
// so queued calls do not start their timers while a cold request is still
// hydrating the cache in front of them.
const run = () => this.requestNow(args, timeoutMs, onStderr)
const result = this.requestTail.then(run, run)
this.requestTail = result.then(() => undefined, () => undefined)
return result
}
private requestNow(args: string[], timeoutMs: number, onStderr?: (chunk: string) => void): Promise<unknown> {
const child = this.child
if (!child?.stdin) return Promise.reject(new CliError('nonzero', 'serve not running'))
const id = this.nextId++
const effectiveTimeoutMs = this.warmed ? timeoutMs : Math.max(timeoutMs, DESKTOP_COLD_TIMEOUT_MS)
return new Promise<unknown>((resolve, reject) => {
const timer = setTimeout(() => {
// A hung request would block the serialized queue behind it; kill the
@ -484,8 +609,15 @@ class ServeClient {
this.pending.delete(id)
reject(new CliError('timeout', 'codeburn serve timed out'))
child.kill('SIGKILL')
}, timeoutMs)
this.pending.set(id, { resolve, reject, timer })
}, effectiveTimeoutMs)
this.pending.set(id, {
resolve,
reject,
timer,
warmsServe: args[0] === 'status',
decodedBytes: 0,
...(onStderr ? { onStderr } : {}),
})
child.stdin!.write(JSON.stringify({ id, args }) + '\n', (err) => {
if (err) {
this.pending.delete(id)
@ -497,68 +629,113 @@ class ServeClient {
}
destroy(): void {
this.destroyed = true
this.deaths = SERVE_MAX_RESTARTS
this.child?.kill('SIGKILL')
this.onDeath()
const child = this.child
if (!child) return
this.onDeath(child, false)
child.kill('SIGKILL')
}
}
let serveClient: ServeClient | null = null
/** Start the resident serve child and its warm-up query. Called once from app
* startup (never from the spawn path, so unit tests of the scheduler and the
* cold-start flow are byte-identical without it). Safe to call repeatedly. */
export function startServeWarmup(): void {
/** Start the resident serve child without issuing a query. The first real panel
* request is accepted immediately (even before the ready frame) and performs
* the one cold-cache hydration while streaming progress back to the splash. */
export function startServe(): void {
if (shuttingDown) return
const target = resolveTarget()
if (!target) return
if (serveClient?.disabled()) return
if (!serveClient) serveClient = new ServeClient(spawnSpecFor(target, ['serve', '--stdio']))
if (!serveClient) {
const spec = spawnSpecFor(target, ['serve', '--stdio'])
spec.env = { ...spec.env, CODEBURN_PROGRESS: '1' }
serveClient = new ServeClient(spec)
}
serveClient.start()
}
function restartServeAfterMutation(): void {
// CLI-only consumers never started serve, so do not create a surprise daemon
// for them. In Electron, replace the resident child immediately so its parser
// and output memos cannot survive a successful config mutation. Reusing the
// client preserves its app-lifetime budget of unexpected child deaths.
if (!serveClient) return
serveClient.restartAfterMutation()
}
function actionInvalidatesServe(args: string[]): boolean {
// Export only writes the caller-selected artifact. Every other current
// Electron action changes config or device state, and future actions restart
// by default until they are explicitly proven state-preserving.
return args[0] !== 'export'
}
function isServeCompatibleEnv(extraEnv?: NodeJS.ProcessEnv): boolean {
if (!extraEnv) return true
const entries = Object.entries(extraEnv).filter(([, value]) => value !== undefined)
if (entries.length === 0) return true
return entries.length === 1 && entries[0]![0] === 'CODEBURN_PROGRESS' && entries[0]![1] === '1'
}
export function spawnCli(
args: string[],
opts: { timeoutMs?: number; onStderr?: (chunk: string) => void; extraEnv?: NodeJS.ProcessEnv; priority?: SpawnPriority } = {},
): Promise<unknown> {
if (shuttingDown) return Promise.reject(new CliError('nonzero', 'codeburn is shutting down'))
const target = resolveTarget()
if (!target) return Promise.reject(new CliError('not-found', 'codeburn CLI not found', notFoundStage()))
const spec = spawnSpecFor(target, args)
if (opts.extraEnv) spec.env = { ...spec.env, ...opts.extraEnv }
const key = JSON.stringify([spec.bin, ...spec.args])
const cached = readCache.get(key)
if (cached && Date.now() - cached.at < COALESCE_TTL_MS) return Promise.resolve(cached.value)
const generation = readGeneration
const key = JSON.stringify([generation, spec.bin, ...spec.args])
const existing = readInflight.get(key)
// A same-cadence re-poll during a slow cold warmup coalesces onto the one
// in-flight child (which already carries onStderr); no second cold parse.
// Coalesce/cache hits settle here, BEFORE queueing, so they never hold a slot.
// Coalesced calls settle here, BEFORE queueing, so they never hold a slot.
if (existing) return existing
// Serve fast-path: warm resident child answers the panel query without a
// spawn. The child is started once at app startup (startServeWarmup); until
// it is warm, every call keeps the plain spawn path.
if (SERVE_ROUTED.has(args[0] ?? '') && !opts.extraEnv) {
const priority = opts.priority ?? 'interactive'
// Serve fast-path: the child is started once at app startup. It accepts the
// first real query before its ready frame, making that request the single
// cache warm-up. CODEBURN_PROGRESS is compatible because startServe sets it
// on the resident child; any other per-call env needs an isolated one-shot.
if (SERVE_ROUTED.has(args[0] ?? '') && isServeCompatibleEnv(opts.extraEnv)) {
const serve = serveClient
if (serve?.isWarmAndReady()) {
const flight = serve.request(args, opts.timeoutMs ?? DEFAULT_TIMEOUT_MS)
.catch(() => runCli(spec, args[0] ?? '', opts.timeoutMs ?? DEFAULT_TIMEOUT_MS, opts.onStderr))
.then(value => { readCache.set(key, { at: Date.now(), value }); return value })
// Recover lazily from an unexpected child death. start() is synchronous and
// idempotent, and the client's lifetime death budget prevents an endlessly
// crashing binary from being respawned on every poll.
if (serve && !serve.isRunning() && !serve.disabled()) serve.start()
if (serve?.isRunning()) {
const flight = serve.request(args, opts.timeoutMs ?? DEFAULT_TIMEOUT_MS, opts.onStderr)
.catch(err => {
// App shutdown is terminal: never turn rejected resident requests
// into brand-new one-shot children after killAll() has reaped them.
if (serve.isDestroyed() || (err instanceof CliError && err.kind === 'too-large')) throw err
return runScheduledCli(
spec,
args[0] ?? '',
opts.timeoutMs ?? DEFAULT_TIMEOUT_MS,
priority,
opts.onStderr,
)
})
.finally(() => { readInflight.delete(key) })
readInflight.set(key, flight)
return flight
}
}
const priority = opts.priority ?? 'interactive'
const flight = (async () => {
await acquireSlot(priority)
try {
return await runCli(spec, args[0] ?? '', opts.timeoutMs ?? DEFAULT_TIMEOUT_MS, opts.onStderr)
} finally {
releaseSlot()
}
})()
.then(value => { readCache.set(key, { at: Date.now(), value }); return value })
const flight = runScheduledCli(
spec,
args[0] ?? '',
opts.timeoutMs ?? DEFAULT_TIMEOUT_MS,
priority,
opts.onStderr,
)
.finally(() => { readInflight.delete(key) })
readInflight.set(key, flight)
return flight
@ -568,6 +745,7 @@ export function spawnCli(
* Mutations count as interactive, so they take a run slot ahead of any queued
* background warm — a Settings save is never stuck behind speculative prefetch. */
export function spawnCliAction(args: string[], opts: { timeoutMs?: number } = {}): Promise<ActionResult> {
if (shuttingDown) return Promise.resolve({ ok: false, stdout: '', stderr: 'codeburn is shutting down', code: null })
const timeoutMs = opts.timeoutMs ?? DEFAULT_TIMEOUT_MS
const target = resolveTarget()
if (!target) return Promise.resolve({ ok: false, stdout: '', stderr: 'codeburn CLI not found', code: null })
@ -580,6 +758,7 @@ export function spawnCliAction(args: string[], opts: { timeoutMs?: number } = {}
return { ok: false, stdout: '', stderr: 'codeburn cancelled', code: null }
}
try {
if (shuttingDown) return { ok: false, stdout: '', stderr: 'codeburn is shutting down', code: null }
return await runAction(spec, args, timeoutMs)
} finally {
releaseSlot()
@ -600,9 +779,13 @@ function runAction(spec: SpawnSpec, args: string[], timeoutMs: number): Promise<
settled = true
clearTimeout(timer)
activeChildren.delete(child)
// The action may have changed config the read cache still reflects; a
// Settings refetch fires immediately after, so serve it fresh data.
readCache.clear()
if (result.ok && actionInvalidatesServe(args)) {
// Fence coalescing before the action promise resolves. An immediate
// same-argv refetch belongs to the new config generation even while an
// older read is still running.
readGeneration += 1
restartServeAfterMutation()
}
resolve(result)
}

14
app/electron/main.ts
View file

@ -1,7 +1,7 @@
import { app, BrowserWindow, dialog, ipcMain, Menu, nativeTheme, shell, type MenuItemConstructorOptions } from 'electron'
import path from 'node:path'
import { CliError, killAll, resolveCodeburnPath, spawnCli, spawnCliAction, startServeWarmup, type ActionResult, type SpawnPriority } from './cli'
import { CliError, DESKTOP_COLD_TIMEOUT_MS, resolveCodeburnPath, shutdownAll, spawnCli, spawnCliAction, startServe, type ActionResult, type SpawnPriority } from './cli'
import { getQuota, sanitizeError } from './quota'
import { Telemetry } from './telemetry'
import { createUpdateChecker, type UpdateChecker, type UpdateStatus } from './updates'
@ -77,7 +77,7 @@ export type Envelope<T = unknown> = { ok: true; value: T } | { ok: false; error:
// slowness. Give the first (cold) overview a long window; revert to the default
// once it succeeds. Sections gate their own first poll on this one resolving so
// the cold hydration runs ONCE, not once per section in parallel.
const WARMUP_TIMEOUT_MS = 10 * 60_000
const WARMUP_TIMEOUT_MS = DESKTOP_COLD_TIMEOUT_MS
// Wire marker for CLI scan-progress lines (src/parser.ts: PROGRESS_LINE_PREFIX).
const PROGRESS_LINE_PREFIX = 'CODEBURN_PROGRESS '
// IPC channel carrying cold-start scan-progress events to the splash.
@ -564,15 +564,15 @@ function bootstrap(): void {
app.on('before-quit', createBeforeQuitHandler({
getTelemetry: () => telemetryInstance,
killAll,
killAll: shutdownAll,
quit: () => app.quit(),
}))
void app.whenReady().then(() => {
// Start the resident serve child early so its warm-up (one cache parse)
// finishes during the first panels' cold spawns; every fetch after that
// answers from the warm child in milliseconds.
startServeWarmup()
// Start the resident child early, but issue no artificial warm-up query:
// the first real overview request is the single cache hydration and streams
// its progress through serve. Every later panel reuses that parsed cache.
startServe()
// Consent-gated anonymous telemetry (desktop only). Nothing transmits until
// the onboarding consent screen is completed and the toggle is on; EU/EEA/
// UK/CH installs default the toggle off. Dev builds never send.

7
app/scripts/stage-cli.mjs
View file

@ -11,6 +11,7 @@
// build/cli/package.json (root package.json: {version}, type:module)
// build/cli/dist/cli.js (Node-version-guard launcher → ./main.js)
// build/cli/dist/main.js (the bundle)
// build/cli/dist/parse-worker.js (the parse worker thread's own entry)
// build/cli/node_modules/ (production dependency closure)
//
// The production closure is copied out of the already-installed root
@ -29,7 +30,7 @@ const dist = join(root, 'dist')
const rootModules = join(root, 'node_modules')
const stage = join(appDir, 'build', 'cli')
for (const f of ['cli.js', 'main.js']) {
for (const f of ['cli.js', 'main.js', 'parse-worker.js']) {
if (!existsSync(join(dist, f))) {
throw new Error(`stage-cli: ${join(dist, f)} is missing — build the root CLI first`)
}
@ -41,6 +42,10 @@ mkdirSync(join(stage, 'dist'), { recursive: true })
copyFileSync(join(root, 'package.json'), join(stage, 'package.json'))
copyFileSync(join(dist, 'cli.js'), join(stage, 'dist', 'cli.js'))
copyFileSync(join(dist, 'main.js'), join(stage, 'dist', 'main.js'))
// The cold-parse worker pool resolves this as a sibling of the bundle it runs
// from, so it has to be staged alongside main.js or a packaged app silently
// loses every parse thread.
copyFileSync(join(dist, 'parse-worker.js'), join(stage, 'dist', 'parse-worker.js'))
// Desktop-app launch shim (the app spawns this, not cli.js). The packaged app
// runs the CLI with Electron's own binary as Node (ELECTRON_RUN_AS_NODE=1).

61
docs/architecture.md
View file

@ -69,6 +69,67 @@ output formatter (Ink TUI, JSON, or menubar-json)
`src/parser.ts` is the central aggregator. Public exports: `parseAllSessions`, `filterProjectsByName`, `extractMcpInventory`. It owns the dedup `Set` (`seenKeys`) that is passed into every provider parser so a turn that surfaces in two providers (Claude logs vs. Cursor mirror, for instance) is counted once.
### Parallel Cold Parse
A cold parse spends most of its time on work that is per-file and pure: reading a
session JSONL or a Codex rollout, decoding it, and turning each line into a
journal entry. `src/parse-workers.ts` moves that onto `worker_threads` when the
pending workload is big enough to pay for them. Each worker runs the same
per-file function the serial path runs — `parseClaudeFileFull` for a Claude
session, `parseCodexFileFull` for a Codex rollout — against an empty dedup set,
and ships the result back as a JSON string together with every dedup key it
claimed. The parent installs results in the same order the serial loop would, and
everything with cross-file state (the dedup sets, canonical project paths, spawn
links, PR correlation, the Codex result cache) stays on the main thread. A file
whose keys were already claimed by an earlier file, or whose worker failed, is
re-parsed in-process — so the output is identical to the serial path either way.
That overlap check is what makes a forked Codex rollout safe: it replays its
parent's token_count history under the parent's key namespace, collides, and is
re-parsed against the real dedup set.
A Codex worker never touches `src/codex-cache.ts`: it returns the cache entry it
would have written and the parent writes it, in install order, so
`flushCodexCache` publishes exactly what a serial parse would. Only whole-file
parses go off-thread; the append/incremental paths (a Claude append, a Codex
byte-offset resume) are untouched and stay in-process. The decision is made per
provider — the Claude scan and the provider loop run one after the other, so at
most one pool is alive — and the pool is terminated when its scan ends, so the
resident `serve` child never accumulates threads.
The pool is off by default for anything that is not a large cold parse:
| Gate | Serial when |
|---|---|
| Pending bytes | under 200 MB behind the pending whole-file parses |
| Cores | `availableParallelism() <= 2` |
| Memory | under 4 GB available |
Otherwise the worker count is
`min(cores - 1, min(0.25 * available, 2 GB) / perWorker, max(pendingFiles / 50, pendingBytes / 200 MB))`.
Files and bytes each earn threads on their own, so a few hundred multi-hundred-MB
Codex rollouts parallelize as well as a few thousand small Claude transcripts. The
gate is bytes only, deliberately: 250 pending files holding under a megabyte
between them spawn threads that make the run ~5% slower, and a file count only
starts paying for itself around 400.
`perWorker` is the per-thread memory budget, derived per parse as
`clamp(256 MB, 2 x (pendingBytes / pendingFiles) + 128 MB, 1 GB)`. A flat figure
was wrong in both directions: small Claude transcripts peak well under 256 MB,
while a 260 MB Codex rollout peaks near 430 MB in its worker and scales linearly
with the pool. The budget also covers the parent, which buffers up to `pool.size`
finished results while it installs one.
"Available" is `process.availableMemory()`, falling back to `os.totalmem()`. It is
deliberately not `os.freemem()`: on macOS that counts free pages rather than
available memory and reads as a few hundred MB on an idle 128 GB machine, so a
gate built on it switches the feature on and off between runs. On Linux outside a
memory-limited cgroup, `availableMemory()` reports free memory and can still
under-report on a busy host — which fails safe, to fewer threads or none.
`CODEBURN_PARSE_WORKERS` overrides the decision and skips every gate above:
`0` forces the serial parse, `N` forces N workers (capped at the core count).
`CODEBURN_VERBOSE=1` prints the resolved worker count and the reason for it.
### Cache Layers
Three caches under `~/.cache/codeburn/` (override with `CODEBURN_CACHE_DIR`):

1
docs/providers/kiro.md
View file

@ -59,6 +59,7 @@ The stores are disjoint (v2 sessions use `sess_`-prefixed IDs in a separate dire
- Token counts are estimated via char count (`CHARS_PER_TOKEN = 4`).
- **Credits are the cost source; tokens stay estimated.** Kiro bills in credits ($20/mo for 1,000; overage $0.04/credit). CLI (`metering_usage`), v1 executions (`usageSummary[].usage`), and v2 (`usage_summary.promptTurnSummaries[].usage`) turns record real credits, converted to USD at `USD_PER_KIRO_CREDIT = 0.04` (the public overage rate — the same never-understate approach as Codebuff). Turns without credit data fall back to token-estimated cost (`costIsEstimated: true`); legacy `.chat` and workspace-session records carry no usage data, so they are always token-estimated. Note: an earlier CLI implementation summed credit values directly as dollars, overstating cost 25×. Token *counts* remain char-estimated everywhere (input undercounts: only visible transcript text is seen, not the full resent context; v2's `session_metadata.contextUsage.usagePercentage` × context window is a better input proxy if ever needed). v2 does keep the real `modelId`, so unlike the v1 execution-file path it is not mislabeled `kiro-auto`.
- **Cost is frozen at parse time.** Kiro is on the `costUSD` pass-through allowlist in `providerCallToCachedCall` (alongside mistral-vibe, devin, hermes, …), so its credit-based cost survives the session cache instead of being re-priced from estimated tokens — token re-pricing understated/overstated real kiro spend by up to 16× per model. The tradeoff, shared with all allowlisted providers: `codeburn price-override` and `model-alias` do not affect kiro dollar amounts (token *counts* are unaffected). Historical caches from before this change re-parse via the `CACHE_VERSION` bump to 5.
- **`projectPath` for git attribution.** The parser now records the session's working directory as `projectPath` (CLI `meta.cwd`, v2 `workspacePaths[0]`, workspace sessions' `workspaceDirectory`), which sync attribution needs to resolve the git repo. The `project-path-v1` parse-version bump re-parses cached kiro history once; sessions in linked git worktrees now group under the main repo.
## When fixing a bug here

5
mac/Sources/CodeBurnMenubar/CodeBurnApp.swift
View file

@ -127,9 +127,8 @@ final class AppDelegate: NSObject, NSApplicationDelegate, NSPopoverDelegate, NSM
// interaction (popover open, wake) refreshes immediately.
restorePersistedCurrency()
// Resident serve child: payload fetches answer from a warm CLI once
// its warm-up completes; until then (and on any failure) fetches keep
// the spawn path. See ServeConnection.
// Start the resident CLI early without an artificial query. The first
// real status refresh becomes its only cold warm-up. See ServeConnection.
Task { await ServeConnection.shared.ensureStarted() }
// #868 experiment: restore only the activation half of the #147 fix.
// Packaged builds ship LSUIElement=true, so the policy is .accessory

6
mac/Sources/CodeBurnMenubar/CurrencyState.swift
View file

@ -77,11 +77,7 @@ actor FXRateCache {
private var loaded = false
private var cacheFilePath: String {
let base = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0]
return base
.appendingPathComponent("codeburn-mac", isDirectory: true)
.appendingPathComponent("fx-rates.json")
.path
return (CodeBurnCacheDirectory.resolve() as NSString).appendingPathComponent("fx-rates.json")
}
private func loadIfNeeded() {

18
mac/Sources/CodeBurnMenubar/Data/CodeBurnCacheDirectory.swift Normal file
View file

@ -0,0 +1,18 @@
import Foundation
/// Resolves the on-disk directory shared by the CLI, desktop app and menubar.
enum CodeBurnCacheDirectory {
static func resolve(
environment: [String: String] = ProcessInfo.processInfo.environment,
homeDirectory: URL = FileManager.default.homeDirectoryForCurrentUser
) -> String {
if let override = environment["CODEBURN_CACHE_DIR"],
!override.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return override
}
return homeDirectory
.appendingPathComponent(".cache", isDirectory: true)
.appendingPathComponent("codeburn", isDirectory: true)
.path
}
}

69
mac/Sources/CodeBurnMenubar/Data/DataClient.swift
View file

@ -123,21 +123,68 @@ struct DataClient {
subcommand: [String],
qualityOfService: QualityOfService = .userInitiated
) async throws -> ProcessResult {
// Serve fast path: a warm resident `codeburn serve` child answers the
// status payload without a spawn (no node boot, no session-cache
// reload). Any serve failure falls back to the spawn path below, so
// this is strictly an optimization; it also takes no spawn slot.
try await runCLI(
subcommand: subcommand,
serveRequest: { args in
try await ServeConnection.shared.request(args: args)
},
spawnFallback: {
await spawnLimiter.acquire()
defer { Task { await spawnLimiter.release() } }
let process = CodeburnCLI.makeProcess(
subcommand: subcommand,
qualityOfService: qualityOfService
)
return try await runProcess(
process,
timeoutSeconds: spawnTimeoutSeconds,
label: subcommand.joined(separator: " ")
)
}
)
}
/// Internal seam for behavior-shaped lifecycle tests. Production supplies
/// the shared resident and globally limited one-shot closures above.
static func runCLI(
subcommand: [String],
serveRequest: ([String]) async throws -> Data,
spawnFallback: () async throws -> ProcessResult
) async throws -> ProcessResult {
// Serve path: the first real status payload warms the resident child,
// then later payloads reuse it (no node boot or session-cache reload).
// Transport/protocol failures fall back to the spawn path below, so
// the resident remains an optimization. Resource-policy failures stay
// terminal and cannot bypass the resident output ceiling.
if ServeConnection.isEligible(subcommand) {
if let stdout = try? await ServeConnection.shared.requestIfWarm(args: subcommand) {
do {
let stdout = try await serveRequest(subcommand)
return ProcessResult(stdout: stdout, stderr: "", exitCode: 0)
} catch let error as CancellationError {
// Cancellation is control flow from the refresh owner. Starting
// a fallback process here would turn cancelled work into a new
// expensive cold parse and delay task teardown.
throw error
} catch {
if let terminalError = terminalServeError(error) {
throw terminalError
}
// Resident serve is only an optimization. Protocol, child, and
// timeout failures retain the established one-shot fallback,
// unless a sibling teardown raced this task's cancellation.
try Task.checkCancellation()
}
}
await spawnLimiter.acquire()
defer { Task { await spawnLimiter.release() } }
let process = CodeburnCLI.makeProcess(subcommand: subcommand, qualityOfService: qualityOfService)
return try await runProcess(process,
timeoutSeconds: spawnTimeoutSeconds,
label: subcommand.joined(separator: " "))
return try await spawnFallback()
}
/// Some resident failures are terminal resource-policy decisions, not
/// transport failures. Retrying those through the one-shot path would redo
/// the cold scan and could bypass the resident's stricter output ceiling.
static func terminalServeError(_ error: Error) -> DataClientError? {
guard let failure = error as? ServeConnection.ServeRequestFailed,
failure.reason == .outputTooLarge else { return nil }
return .outputTooLarge
}
/// Runs an already-configured process to completion, draining its output and

5
mac/Sources/CodeBurnMenubar/Data/MenubarStatusCache.swift
View file

@ -9,8 +9,9 @@ struct MenubarStatusCache {
/// Default location under `~/.cache/codeburn/`.
static func standard() -> MenubarStatusCache {
let home = FileManager.default.homeDirectoryForCurrentUser.path
return MenubarStatusCache(statusPath: "\(home)/.cache/codeburn/menubar-status.json")
let cacheDir = CodeBurnCacheDirectory.resolve()
let path = (cacheDir as NSString).appendingPathComponent("menubar-status.json")
return MenubarStatusCache(statusPath: path)
}
struct BadgeRead {

546
mac/Sources/CodeBurnMenubar/Data/ServeConnection.swift
View file

@ -1,3 +1,4 @@
import Darwin
import Foundation
/// A resident `codeburn serve --stdio` child, held so payload fetches skip the
@ -6,53 +7,118 @@ import Foundation
/// replies are `{id, ok, output}`. Mirrors the desktop app's client contract:
///
/// - Only `status` payload queries route here; anything else spawns as before.
/// - Requests route through serve only once the child is READY and WARM (one
/// completed query), so cold start behaves exactly as today.
/// - Any failure falls back to the spawn path for that call; three child
/// deaths disable serve for this app run.
/// - The first real status request is also the warm-up. It may be written
/// before the child announces READY; the pipe buffers it until serve reads
/// stdin, avoiding a second one-shot process that parses the same cache.
/// - Transport/protocol failures fall back to the spawn path for that call;
/// resource-policy failures remain terminal. Three child deaths disable
/// serve for this app run.
/// - The child's stdin closing (app quit, even SIGKILL) ends the server loop
/// on the CLI side, so no orphan survives the menubar.
actor ServeConnection {
static let shared = ServeConnection()
typealias ProcessFactory = ([String], QualityOfService) -> Process
typealias TimeoutSleep = @Sendable (UInt64) async throws -> Void
private struct QueuedRequest {
let token: Int
let args: [String]
let continuation: CheckedContinuation<Data, Error>
}
private struct ActiveRequest {
let token: Int
let id: Int
let args: [String]
let child: Process
}
private var process: Process?
private var stdinHandle: FileHandle?
private var nextId = 1
private var nextRequestToken = 1
private var queuedRequests: [QueuedRequest] = []
private var activeRequest: ActiveRequest?
private var pending: [Int: CheckedContinuation<Data, Error>] = [:]
private var ready = false
private var warm = false
private var requestTimeouts: [Int: Task<Void, Never>] = [:]
private var timeoutOwners: [Int: Process] = [:]
private var responseBytes: [Int: Int] = [:]
private var deaths = 0
private var buffer = Data()
private var receivedTerminalResponse = false
private var outputTasks: [ObjectIdentifier: Task<Void, Never>] = [:]
private var terminationTasks: [ObjectIdentifier: Task<Void, Never>] = [:]
private let makeProcess: ProcessFactory
private let timeoutSleep: TimeoutSleep
private let terminationGraceSleep: TimeoutSleep
private let responseLimitBytes: Int
private static let maxDeaths = 3
private static let requestTimeoutSeconds: UInt64 = 60
static let maxResponseBytes = 16 * 1024 * 1024
private static let stdoutReadChunkBytes = 64 * 1024
private static let terminationGraceNanoseconds: UInt64 = 1_000_000_000
private static let coldRequestTimeoutNanoseconds: UInt64 = 10 * 60 * 1_000_000_000
private static let warmRequestTimeoutNanoseconds: UInt64 = 60 * 1_000_000_000
struct ServeUnavailable: Error {}
struct ServeRequestFailed: Error { let message: String }
enum FailureReason: Sendable, Equatable {
case generic
case outputTooLarge
}
struct ServeRequestFailed: Error, Sendable {
let message: String
let reason: FailureReason
init(message: String, reason: FailureReason = .generic) {
self.message = message
self.reason = reason
}
}
init(
makeProcess: @escaping ProcessFactory = CodeburnCLI.makeProcess,
timeoutSleep: @escaping TimeoutSleep = { nanoseconds in
try await Task<Never, Never>.sleep(nanoseconds: nanoseconds)
},
terminationGraceSleep: @escaping TimeoutSleep = { nanoseconds in
try await Task<Never, Never>.sleep(nanoseconds: nanoseconds)
},
responseLimitBytes: Int = ServeConnection.maxResponseBytes
) {
self.makeProcess = makeProcess
self.timeoutSleep = timeoutSleep
self.terminationGraceSleep = terminationGraceSleep
precondition(responseLimitBytes > 0)
self.responseLimitBytes = responseLimitBytes
}
static func isEligible(_ subcommand: [String]) -> Bool {
subcommand.first == "status"
}
/// Kick the child off (idempotent). Called from app startup; fetches keep
/// spawning until the warm-up completes.
/// Kick the child off (idempotent). Called from app startup and again by
/// the first request in case the startup task has not run yet.
func ensureStarted() {
guard process == nil, deaths < Self.maxDeaths else { return }
let child = CodeburnCLI.makeProcess(subcommand: ["serve", "--stdio"], qualityOfService: .utility)
// This single resident serves both background and user-visible status
// requests. Its cold hydration replaces the old interactive one-shot,
// so keep the child at the same user-initiated QoS as visible fetches.
let child = makeProcess(["serve", "--stdio"], .userInitiated)
let stdinPipe = Pipe()
let stdinWriter = stdinPipe.fileHandleForWriting
// Suppress SIGPIPE only for this connection's write end. A process-wide
// SIG_IGN leaks into unrelated libraries and children; F_SETNOSIGPIPE
// keeps a closed child stdin on the normal throwable EPIPE path.
guard Darwin.fcntl(stdinWriter.fileDescriptor, F_SETNOSIGPIPE, 1) == 0 else {
deaths = Self.maxDeaths
return
}
let stdoutPipe = Pipe()
let stdoutReader = stdoutPipe.fileHandleForReading
child.standardInput = stdinPipe
child.standardOutput = stdoutPipe
child.standardError = FileHandle.nullDevice
stdoutPipe.fileHandleForReading.readabilityHandler = { handle in
let data = handle.availableData
guard !data.isEmpty else { return }
Task { await ServeConnection.shared.consume(data) }
}
child.terminationHandler = { _ in
stdoutPipe.fileHandleForReading.readabilityHandler = nil
Task { await ServeConnection.shared.childDied() }
}
do {
try child.run()
} catch {
@ -60,112 +126,404 @@ actor ServeConnection {
return
}
process = child
stdinHandle = stdinPipe.fileHandleForWriting
Task {
// Warm-up: one cheap query makes the child parse the session cache
// once; every later payload answers from the warm in-memory copy.
_ = try? await self.send(args: ["status", "--format", "menubar-json", "--period", "today", "--no-optimize"])
await self.markWarm()
stdinHandle = stdinWriter
let generation = ObjectIdentifier(child)
// One blocking reader owns this generation's stdout. It never reads a
// second bounded chunk until the actor has consumed the first, giving
// the 16 MiB protocol limit real backpressure instead of accumulating
// an unbounded callback/AsyncStream backlog. EOF is observed only after
// the pipe's final bytes, so child death cannot overtake a split reply.
outputTasks[generation] = Task.detached { [weak self] in
var bytes = [UInt8](repeating: 0, count: Self.stdoutReadChunkBytes)
while !Task.isCancelled {
let count = Darwin.read(stdoutReader.fileDescriptor, &bytes, bytes.count)
if count > 0 {
guard let self else { break }
await self.consume(Data(bytes[0..<count]), from: child)
} else if count == -1, errno == EINTR {
continue
} else {
break
}
}
await self?.outputStreamEnded(for: child)
await self?.outputStreamFinished(for: child)
}
}
/// The fast path `runCLI` consults: throws ServeUnavailable unless the
/// child is warm, so callers can fall back to a spawn without waiting.
func requestIfWarm(args: [String]) async throws -> Data {
guard ready, warm, process != nil else { throw ServeUnavailable() }
return try await send(args: args)
/// Send the first real payload through the resident child. A request does
/// not need to wait for the READY frame: stdin is safe to write as soon as
/// Process.run() succeeds, and serve serializes it after initialization.
func request(args: [String]) async throws -> Data {
try Task.checkCancellation()
ensureStarted()
guard process != nil else { throw ServeUnavailable() }
let token = nextRequestToken
nextRequestToken += 1
let response = try await withTaskCancellationHandler {
try await withCheckedThrowingContinuation { continuation in
queuedRequests.append(QueuedRequest(
token: token,
args: args,
continuation: continuation
))
startNextRequestIfPossible()
}
} onCancel: {
Task { await self.cancelRequest(token: token) }
}
try Task.checkCancellation()
return response
}
func shutdown() {
deaths = Self.maxDeaths
process?.terminate()
failAllPending()
for task in terminationTasks.values { task.cancel() }
terminationTasks.removeAll()
cancelAllTimeouts()
failAllRequests()
process = nil
stdinHandle = nil
buffer = Data()
receivedTerminalResponse = false
}
// MARK: - internals
private func markWarm() {
if process != nil { warm = true }
}
private func startNextRequestIfPossible() {
guard activeRequest == nil, !queuedRequests.isEmpty else { return }
ensureStarted()
guard let stdinHandle, let child = process else {
failQueuedRequests(error: ServeUnavailable())
return
}
// A Process can report not-running just before its termination callback
// reaches the ordered event stream. Keep the request queued for that
// event instead of writing to a generation which is already exiting.
guard child.isRunning else { return }
private func send(args: [String]) async throws -> Data {
guard let stdinHandle, let child = process else { throw ServeUnavailable() }
let request = queuedRequests.removeFirst()
let id = nextId
nextId += 1
let request: [String: Any] = ["id": id, "args": args]
let line = try JSONSerialization.data(withJSONObject: request)
return try await withThrowingTaskGroup(of: Data.self) { group in
group.addTask {
try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Data, Error>) in
Task { await self.registerPending(id: id, continuation: continuation) }
do {
try stdinHandle.write(contentsOf: line + Data("\n".utf8))
} catch {
Task { await self.rejectPending(id: id, error: ServeRequestFailed(message: "stdin write failed")) }
}
}
}
group.addTask {
try await Task.sleep(nanoseconds: Self.requestTimeoutSeconds * 1_000_000_000)
// A hung request would block the serialized queue behind it:
// kill the child so everything falls back to spawns.
await self.rejectPending(id: id, error: ServeRequestFailed(message: "serve timeout"))
child.terminate()
throw ServeRequestFailed(message: "serve timeout")
}
let result = try await group.next()!
group.cancelAll()
return result
let line: Data
do {
line = try JSONSerialization.data(withJSONObject: ["id": id, "args": request.args])
} catch {
request.continuation.resume(throwing: error)
startNextRequestIfPossible()
return
}
// The previous response can resume its caller just before EOF reaches
// this actor. Avoid admitting a successor to an already-reaped child;
// the reader's ordered EOF path will start it on a replacement.
guard child.isRunning else {
queuedRequests.insert(request, at: 0)
outputStreamEnded(for: child)
return
}
// Select and arm the timeout only when this request becomes the sole
// protocol request in flight. A queued request must not spend its own
// budget while its predecessor is still hydrating or draining.
let timeoutNanoseconds = receivedTerminalResponse
? Self.warmRequestTimeoutNanoseconds
: Self.coldRequestTimeoutNanoseconds
activeRequest = ActiveRequest(
token: request.token,
id: id,
args: request.args,
child: child
)
pending[id] = request.continuation
responseBytes[id] = 0
do {
try stdinHandle.write(contentsOf: line + Data("\n".utf8))
armTimeout(id: id, child: child, nanoseconds: timeoutNanoseconds)
} catch {
// The previous terminal frame can resume its caller just before
// EOF detaches that generation. Preserve this never-admitted
// request and retry it on the replacement instead of surfacing a
// transient EPIPE to the UI.
pending.removeValue(forKey: id)
responseBytes.removeValue(forKey: id)
activeRequest = nil
queuedRequests.insert(request, at: 0)
outputStreamEnded(for: child)
}
}
private func registerPending(id: Int, continuation: CheckedContinuation<Data, Error>) {
pending[id] = continuation
private func cancelRequest(token: Int) {
if let index = queuedRequests.firstIndex(where: { $0.token == token }) {
let request = queuedRequests.remove(at: index)
request.continuation.resume(throwing: CancellationError())
return
}
guard let activeRequest, activeRequest.token == token,
let continuation = pending.removeValue(forKey: activeRequest.id) else { return }
continuation.resume(throwing: CancellationError())
// Caller cancellation abandons only this response. The serialized serve
// child may still be doing the expensive first hydration, and killing it
// here lets tab switches and UI watchdogs restart that work indefinitely.
// Its independent request timeout remains armed: a command that never
// returns is still reaped, so it cannot wedge every later serialized call.
}
private func rejectPending(id: Int, error: Error) {
private func armTimeout(id: Int, child: Process, nanoseconds: UInt64) {
let sleep = timeoutSleep
timeoutOwners[id] = child
requestTimeouts[id] = Task.detached { [weak self] in
do {
try await sleep(nanoseconds)
} catch {
return
}
await self?.requestTimedOut(id: id)
}
}
private func requestTimedOut(id: Int) {
guard let child = timeoutOwners.removeValue(forKey: id) else { return }
requestTimeouts.removeValue(forKey: id)
responseBytes.removeValue(forKey: id)
if let continuation = pending.removeValue(forKey: id) {
continuation.resume(throwing: error)
continuation.resume(throwing: ServeRequestFailed(message: "serve timeout"))
}
}
private func consume(_ data: Data) {
buffer.append(data)
while let newline = buffer.firstIndex(of: UInt8(ascii: "\n")) {
let lineData = buffer.subdata(in: buffer.startIndex..<newline)
buffer.removeSubrange(buffer.startIndex...newline)
guard !lineData.isEmpty,
let object = try? JSONSerialization.jsonObject(with: lineData) as? [String: Any] else { continue }
if object["ready"] as? Bool == true {
ready = true
continue
}
guard let id = object["id"] as? Int, let continuation = pending.removeValue(forKey: id) else { continue }
if object["ok"] as? Bool == true, let output = object["output"] as? String {
continuation.resume(returning: Data(output.utf8))
} else {
let message = object["error"] as? String ?? "serve request failed"
continuation.resume(throwing: ServeRequestFailed(message: message))
}
// The waiter may already have been abandoned by caller cancellation.
// Timeout ownership is deliberately independent of that continuation:
// kill only the exact generation that received the timed-out request.
guard process === child else {
if activeRequest?.id == id { activeRequest = nil }
startNextRequestIfPossible()
return
}
}
private func childDied() {
// Retire the timed-out generation synchronously. Its stdout may never
// reach EOF (for example, a stuck child can ignore SIGTERM or a
// descendant can retain the pipe), so waiting for the reader would also
// spend every queued caller's timeout before it can even be admitted.
process = nil
stdinHandle = nil
ready = false
warm = false
buffer.removeAll()
buffer = Data()
receivedTerminalResponse = false
deaths += 1
failAllPending()
if activeRequest?.id == id { activeRequest = nil }
cancelTimeouts(ownedBy: child)
terminateTimedOutChild(child)
// The waiter was removed above and cannot be requeued by stale EOF.
// A queued read starts on a replacement immediately, subject to the
// ordinary three-death budget.
startNextRequestIfPossible()
}
private func failAllPending() {
private func cancelTimeout(id: Int) {
timeoutOwners.removeValue(forKey: id)
requestTimeouts.removeValue(forKey: id)?.cancel()
responseBytes.removeValue(forKey: id)
}
private func cancelTimeouts(ownedBy child: Process) {
let ids = timeoutOwners.compactMap { id, owner in owner === child ? id : nil }
for id in ids { cancelTimeout(id: id) }
}
private func cancelAllTimeouts() {
for task in requestTimeouts.values { task.cancel() }
requestTimeouts.removeAll()
timeoutOwners.removeAll()
responseBytes.removeAll()
}
private func outputStreamFinished(for child: Process) {
outputTasks.removeValue(forKey: ObjectIdentifier(child))
if !child.isRunning {
terminationTasks.removeValue(forKey: ObjectIdentifier(child))?.cancel()
}
}
private func terminateTimedOutChild(_ child: Process) {
guard child.isRunning else { return }
child.terminate()
let generation = ObjectIdentifier(child)
let sleep = terminationGraceSleep
terminationTasks[generation] = Task.detached { [weak self] in
do {
try await sleep(Self.terminationGraceNanoseconds)
} catch {
// Cancellation means the owner stopped waiting: either shutdown
// (which must not orphan a SIGTERM-ignoring generation) or the
// child already died and the stream finished. Escalate either
// way; the isRunning guard makes the dead-child case a no-op.
await self?.forceKillAfterGrace(child)
return
}
await self?.forceKillAfterGrace(child)
}
}
private func forceKillAfterGrace(_ child: Process) {
terminationTasks.removeValue(forKey: ObjectIdentifier(child))
guard child.isRunning else { return }
_ = Darwin.kill(child.processIdentifier, SIGKILL)
}
private func outputStreamEnded(for child: Process) {
guard process === child else { return }
// EOF/read failure is a transport death even if the process has not
// reaped yet. Terminate that exact generation so a child which closed
// stdout cannot survive after the actor starts its replacement.
if child.isRunning { child.terminate() }
childDied(child)
}
// Internal so the generation guard can be exercised deterministically by
// tests without relying on Foundation callback scheduling at process exit.
func consume(_ data: Data, from child: Process) {
// A readability callback can already have queued its actor Task when the
// old process exits. If a replacement starts first, those late bytes must
// not repopulate the shared line buffer or mark the new child as warm.
guard process === child else { return }
var remaining = data[data.startIndex..<data.endIndex]
while !remaining.isEmpty {
if let newline = remaining.firstIndex(of: UInt8(ascii: "\n")) {
let fragment = remaining[remaining.startIndex..<newline]
guard fragment.count <= responseLimitBytes - buffer.count else {
outputOverflowed(child)
return
}
buffer.append(contentsOf: fragment)
let lineData = buffer
buffer = Data()
consumeLine(lineData, from: child)
guard process === child else { return }
remaining = remaining[remaining.index(after: newline)..<remaining.endIndex]
} else {
guard remaining.count <= responseLimitBytes - buffer.count else {
outputOverflowed(child)
return
}
buffer.append(contentsOf: remaining)
return
}
}
}
private func consumeLine(_ lineData: Data, from child: Process) {
guard !lineData.isEmpty,
let object = try? JSONSerialization.jsonObject(with: lineData) as? [String: Any] else { return }
if object["ready"] as? Bool == true {
return
}
guard let id = object["id"] as? Int,
responseBytes[id] != nil else { return }
// Desktop asks serve to stream cold-scan stderr as progress frames.
// Menubar has no progress UI, but must leave the request pending
// until the terminal response arrives if such a frame is emitted.
if let progress = object["progress"] as? String {
guard accountResponseBytes(Data(progress.utf8).count, id: id, child: child) else { return }
return
}
let succeeded = object["ok"] as? Bool == true
let payload = succeeded
? (object["output"] as? String)
: (object["error"] as? String)
if let payload {
guard accountResponseBytes(Data(payload.utf8).count, id: id, child: child) else { return }
}
// A refused/failed command can finish before any cache hydration.
// Only a successful terminal proves the resident is warm. Keep
// this before the waiter lookup so a successful orphan response
// still records the child as warm without resuming anything.
if succeeded { receivedTerminalResponse = true }
cancelTimeout(id: id)
let continuation = pending.removeValue(forKey: id)
if activeRequest?.id == id { activeRequest = nil }
if let continuation {
if succeeded, let output = object["output"] as? String {
continuation.resume(returning: Data(output.utf8))
} else {
let message = object["error"] as? String ?? "serve request failed"
continuation.resume(throwing: ServeRequestFailed(message: message))
}
}
// This also advances after an orphan terminal response whose caller
// was cancelled: cancellation removes only the waiter, not the
// active protocol lifecycle.
startNextRequestIfPossible()
}
private func accountResponseBytes(_ count: Int, id: Int, child: Process) -> Bool {
guard let current = responseBytes[id],
count <= responseLimitBytes - current else {
outputOverflowed(child)
return false
}
responseBytes[id] = current + count
return true
}
private func outputOverflowed(_ child: Process) {
guard process === child else { return }
// Detach this exact generation before terminating it. Its eventual exit
// and any already-scheduled stdout callbacks are then stale and cannot
// consume a second death or corrupt a replacement generation.
process = nil
stdinHandle = nil
buffer = Data()
receivedTerminalResponse = false
deaths += 1
cancelTimeouts(ownedBy: child)
failAllRequests(error: ServeRequestFailed(
message: "serve output exceeded \(responseLimitBytes) bytes",
reason: .outputTooLarge
))
if child.isRunning { child.terminate() }
}
private func childDied(_ child: Process) {
guard process === child else { return }
process = nil
stdinHandle = nil
buffer.removeAll()
receivedTerminalResponse = false
deaths += 1
cancelTimeouts(ownedBy: child)
if let activeRequest, activeRequest.child === child {
if let continuation = pending.removeValue(forKey: activeRequest.id) {
// Only read-only status requests enter this connection. If a
// generation exits after admission but before its terminal
// reply, retain the waiter and retry on the replacement rather
// than racing it into a one-shot fallback. A timed-out or
// cancelled waiter is already absent and is never retried.
queuedRequests.insert(QueuedRequest(
token: activeRequest.token,
args: activeRequest.args,
continuation: continuation
), at: 0)
}
self.activeRequest = nil
}
// Requests which were never written survive an ordinary child crash.
// They begin on a replacement only after this ordered death event.
startNextRequestIfPossible()
}
private func failAllRequests(
error: Error = ServeRequestFailed(message: "serve exited")
) {
for (_, continuation) in pending {
continuation.resume(throwing: ServeRequestFailed(message: "serve exited"))
continuation.resume(throwing: error)
}
pending.removeAll()
activeRequest = nil
failQueuedRequests(error: error)
}
private func failQueuedRequests(error: Error) {
let requests = queuedRequests
queuedRequests.removeAll()
for request in requests {
request.continuation.resume(throwing: error)
}
}
}

7
mac/Sources/CodeBurnMenubar/Data/SubscriptionSnapshotStore.swift
View file

@ -14,13 +14,8 @@ struct SubscriptionSnapshot: Codable, Sendable {
private let snapshotFilename = "subscription-snapshots.json"
private let pruneOlderThanSeconds: TimeInterval = 30 * 24 * 3600
private func snapshotsCacheDir() -> String {
return ProcessInfo.processInfo.environment["CODEBURN_CACHE_DIR"]
?? (NSHomeDirectory() as NSString).appendingPathComponent(".cache/codeburn")
}
private func snapshotsPath() -> String {
return (snapshotsCacheDir() as NSString).appendingPathComponent(snapshotFilename)
return (CodeBurnCacheDirectory.resolve() as NSString).appendingPathComponent(snapshotFilename)
}
private actor SnapshotLock {

2
mac/Sources/CodeBurnMenubar/Security/SafeFile.swift
View file

@ -2,7 +2,7 @@ import Foundation
/// Symlink-safe file I/O with atomic writes and optional cross-process flock.
///
/// Every cache file we touch (`~/Library/Caches/codeburn-mac/fx-rates.json`,
/// Every cache file we touch (`~/.cache/codeburn/fx-rates.json`,
/// `~/.cache/codeburn/subscription-snapshots.json`, `~/.config/codeburn/config.json`) is a
/// legitimate target for a local-symlink attack: if an attacker plants a symlink from one of
/// those paths to, say, `~/.ssh/config`, a naive `Data.write(to:)` blindly follows the link and

36
mac/Tests/CodeBurnMenubarTests/CodeBurnCacheDirectoryTests.swift Normal file
View file

@ -0,0 +1,36 @@
import Foundation
import Testing
@testable import CodeBurnMenubar
@Suite("CodeBurnCacheDirectory")
struct CodeBurnCacheDirectoryTests {
@Test("honors CODEBURN_CACHE_DIR override")
func honorsOverride() {
let resolved = CodeBurnCacheDirectory.resolve(
environment: ["CODEBURN_CACHE_DIR": "/tmp/codeburn-shared-cache"],
homeDirectory: URL(fileURLWithPath: "/Users/test")
)
#expect(resolved == "/tmp/codeburn-shared-cache")
}
@Test("falls back to the user's standard cache directory")
func fallsBackToStandardDirectory() {
let resolved = CodeBurnCacheDirectory.resolve(
environment: [:],
homeDirectory: URL(fileURLWithPath: "/Users/test", isDirectory: true)
)
#expect(resolved == "/Users/test/.cache/codeburn")
}
@Test("ignores an empty cache override")
func ignoresEmptyOverride() {
let resolved = CodeBurnCacheDirectory.resolve(
environment: ["CODEBURN_CACHE_DIR": " \n"],
homeDirectory: URL(fileURLWithPath: "/Users/test", isDirectory: true)
)
#expect(resolved == "/Users/test/.cache/codeburn")
}
}

1161
mac/Tests/CodeBurnMenubarTests/ServeConnectionTests.swift Normal file
View file

File diff suppressed because it is too large Load diff

3
package.json
View file

@ -8,7 +8,8 @@
"codeburn": "dist/cli.js"
},
"files": [
"dist"
"dist",
"!dist/parse-worker.js.map"
],
"scripts": {
"bundle-litellm": "node scripts/bundle-litellm.mjs",

7
src/antigravity-statusline.ts
View file

@ -3,6 +3,7 @@ import { randomBytes } from 'crypto'
import { dirname, join } from 'path'
import { homedir } from 'os'
import { getCodeburnCacheDir } from './cache-dir.js'
import {
recordAntigravityStatusLinePayload,
snapshotAntigravityStatusLinePayload,
@ -54,12 +55,8 @@ function settingsPath(): string {
?? join(homedir(), '.gemini', 'antigravity-cli', 'settings.json')
}
function codeburnCacheDir(): string {
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
}
function previousStatusLinePath(): string {
return join(codeburnCacheDir(), 'antigravity-statusline-previous.json')
return join(getCodeburnCacheDir(), 'antigravity-statusline-previous.json')
}
async function readSettings(): Promise<Settings> {

18
src/bash-utils.ts
View file

@ -1,6 +1,8 @@
import { basename } from 'path'
import stripAnsi from 'strip-ansi'
const WHITESPACE = /\s/
function stripQuotedStrings(command: string): string {
return command.replace(/"[^"]*"|'[^']*'/g, match => ' '.repeat(match.length))
}
@ -19,12 +21,22 @@ export function extractBashCommands(rawCommand: string): string[] {
const command = stripAnsi(rawCommand)
const stripped = stripQuotedStrings(command)
const separatorRegex = /\s*(?:&&|;|\|)\s*/g
// Match the separator alone, then widen over surrounding whitespace by hand.
// /\s*(?:&&|;|\|)\s*/ retried its leading \s* from every offset, quadratic on
// long whitespace-heavy commands. Widening is required (not cosmetic): stripQuotedStrings
// blanks quoted text, and segments are sliced from the original string.
const separatorRegex = /(?:&&|;|\|)/g
const separators: Array<{ start: number; end: number }> = []
let match: RegExpExecArray | null
while ((match = separatorRegex.exec(stripped)) !== null) {
separators.push({ start: match.index, end: match.index + match[0].length })
let start = match.index
while (start > 0 && WHITESPACE.test(stripped[start - 1]!)) start--
let end = match.index + match[0].length
while (end < stripped.length && WHITESPACE.test(stripped[end]!)) end++
const prevEnd = separators[separators.length - 1]?.end ?? 0
separators.push({ start: Math.max(start, prevEnd), end })
separatorRegex.lastIndex = end
}
const ranges: Array<[number, number]> = []
@ -93,7 +105,7 @@ const GIT_READ_SUBCOMMANDS = new Set([
export function isReadShapedBashCommand(rawCommand: string): boolean {
if (!rawCommand || !rawCommand.trim()) return false
const stripped = stripQuotedStrings(stripAnsi(rawCommand))
const segments = stripped.split(/\s*(?:&&|;|\|)\s*/)
const segments = stripped.split(/(?:&&|;|\|)/)
let sawCommand = false
for (const segment of segments) {
const trimmed = segment.trim()

13
src/cache-dir.ts Normal file
View file

@ -0,0 +1,13 @@
import { homedir } from 'os'
import { join } from 'path'
/**
* Resolve CodeBurn's shared cache directory at call time.
*
* Reading the environment on every call matters for embedded consumers and
* tests that change CODEBURN_CACHE_DIR after importing the CLI modules.
*/
export function getCodeburnCacheDir(): string {
const override = process.env['CODEBURN_CACHE_DIR']
return override?.trim() ? override : join(homedir(), '.cache', 'codeburn')
}

9
src/cache-refresh-lock.ts
View file

@ -1,9 +1,10 @@
import { createHash, randomBytes } from 'crypto'
import { existsSync } from 'fs'
import { mkdir, open, readFile, stat, unlink, utimes, writeFile } from 'fs/promises'
import { homedir } from 'os'
import { join } from 'path'
import { getCodeburnCacheDir } from './cache-dir.js'
const LOCK_FILE = 'session-refresh.lock'
const TAKEOVER_FILE = `${LOCK_FILE}.takeover`
const DEFAULT_HEARTBEAT_MS = 10_000
@ -46,10 +47,6 @@ const defaultClock: RefreshLockClock = {
wallNow: () => Date.now(),
}
function defaultCacheDir(): string {
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
}
function delay(ms: number): Promise<void> {
return new Promise(resolve => { setTimeout(resolve, ms) })
}
@ -197,7 +194,7 @@ export async function acquireCacheRefreshLock(options: RefreshLockOptions = {}):
leaveSingleFlight()
}
const cacheDir = options.cacheDir ?? defaultCacheDir()
const cacheDir = options.cacheDir ?? getCodeburnCacheDir()
const clock = options.clock ?? defaultClock
const heartbeatMs = options.heartbeatMs ?? DEFAULT_HEARTBEAT_MS
const staleMs = options.staleMs ?? DEFAULT_STALE_MS

133
src/codex-cache.ts
View file

@ -1,9 +1,10 @@
import { readFile, mkdir, stat, open, rename, unlink } from 'fs/promises'
import { existsSync } from 'fs'
import { randomBytes } from 'crypto'
import { join } from 'path'
import { homedir } from 'os'
import { join, resolve } from 'path'
import { AsyncLocalStorage } from 'node:async_hooks'
import { getCodeburnCacheDir } from './cache-dir.js'
import type { ParsedProviderCall } from './providers/types.js'
// v4: attribute MCP calls emitted as event_msg/mcp_tool_call_end (issue #478).
@ -14,45 +15,85 @@ import type { ParsedProviderCall } from './providers/types.js'
// v6/v7: rich-session-capture — per-call locAdded/locRemoved/editFailed from
// patch_apply_end. Sessions cached under v5 lack these fields; re-parse to add.
// v8: persist native MCP timing and compact invocation attribution.
// Deliberately NOT bumped for the resume fields (dev/ino + resumeOffset/
// resumeState): they are additive and absence-safe in both directions, so a
// bump would only throw away a warm multi-hundred-MB cache to gain nothing. An
// entry without them simply re-parses in full once and gains them.
const CODEX_CACHE_VERSION = 8
const CACHE_FILE = 'codex-results.json'
type FileFingerprint = { mtimeMs: number; sizeBytes: number }
export type CodexFileFingerprint = { dev: number; ino: number; mtimeMs: number; sizeBytes: number }
type FileFingerprint = CodexFileFingerprint
type FileEntry = {
// Absent on entries written before the resume support landed.
dev?: number
ino?: number
mtimeMs: number
sizeBytes: number
project: string
calls: ParsedProviderCall[]
/** Byte offset of a complete-line boundary the parser can restart from. */
resumeOffset?: number
/** Opaque parser state captured at `resumeOffset` (shape owned by the Codex parser). */
resumeState?: unknown
/** How many of `calls` were decoded before `resumeOffset`. */
resumeCallCount?: number
}
/** An exact fingerprint match, or an append the parser can resume into. */
export type CodexCacheHit =
| { kind: 'exact'; calls: ParsedProviderCall[] }
| { kind: 'resume'; calls: ParsedProviderCall[]; offset: number; state: unknown; callCount: number }
type ResultCache = {
version: number
files: Record<string, FileEntry>
}
function getCacheDir(): string {
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
const cacheDirContext = new AsyncLocalStorage<string>()
function currentCacheDir(): string {
return cacheDirContext.getStore() ?? resolve(getCodeburnCacheDir())
}
function getCachePath(): string {
return join(getCacheDir(), CACHE_FILE)
// A parse can cross many async boundaries before the Codex provider publishes
// its incremental cache. Embedded hosts are allowed to change the process env
// between calls, so pin the call-time directory for the whole transaction
// instead of re-reading CODEBURN_CACHE_DIR at each cache operation.
export function withCodexCacheDirectory<T>(cacheDir: string, operation: () => T): T {
return cacheDirContext.run(resolve(cacheDir), operation)
}
let memCache: ResultCache | null = null
function getCachePath(cacheDir: string): string {
return join(cacheDir, CACHE_FILE)
}
async function loadCache(): Promise<ResultCache> {
if (memCache) return memCache
// Embedded consumers can change CODEBURN_CACHE_DIR without reloading this
// module. Keep each directory's in-memory state separate so a warm cache (or an
// unflushed update) from A can never be read from or written into B.
const memCaches = new Map<string, ResultCache>()
// Dropped by the resident RSS guard. Every write is published by
// flushCodexCache() in the parse's finally, so the next load re-reads disk.
export function clearCodexMemCaches(): void {
memCaches.clear()
}
async function loadCache(cacheDir: string): Promise<ResultCache> {
const inMemory = memCaches.get(cacheDir)
if (inMemory) return inMemory
try {
const raw = await readFile(getCachePath(), 'utf-8')
const raw = await readFile(getCachePath(cacheDir), 'utf-8')
const cache = JSON.parse(raw) as ResultCache
if (cache.version === CODEX_CACHE_VERSION && cache.files && typeof cache.files === 'object') {
memCache = cache
memCaches.set(cacheDir, cache)
return cache
}
} catch {}
memCache = { version: CODEX_CACHE_VERSION, files: {} }
return memCache
const empty = { version: CODEX_CACHE_VERSION, files: {} }
memCaches.set(cacheDir, empty)
return empty
}
function getEntry(cache: ResultCache, filePath: string, fp: FileFingerprint): FileEntry | null {
@ -64,14 +105,51 @@ function getEntry(cache: ResultCache, filePath: string, fp: FileFingerprint): Fi
return null
}
// A grown file is only assumed to be an APPEND if the recorded boundary still
// falls right after a newline. A same-inode rewrite (truncate + refill, or an
// in-place edit) that happens to end up larger would otherwise resume into the
// middle of an unrelated line. Reading one byte is cheaper than being wrong.
async function endsLineAt(filePath: string, offset: number): Promise<boolean> {
if (offset === 0) return true
try {
const handle = await open(filePath, 'r')
try {
const buf = Buffer.alloc(1)
const { bytesRead } = await handle.read(buf, 0, 1, offset - 1)
return bytesRead === 1 && buf[0] === 0x0a
} finally {
await handle.close()
}
} catch {
return false
}
}
export async function readCachedCodexResults(
filePath: string,
): Promise<ParsedProviderCall[] | null> {
): Promise<CodexCacheHit | null> {
try {
const s = await stat(filePath)
const cache = await loadCache()
const entry = getEntry(cache, filePath, { mtimeMs: s.mtimeMs, sizeBytes: s.size })
return entry?.calls ?? null
const cache = await loadCache(currentCacheDir())
const fp = { dev: s.dev, ino: s.ino, mtimeMs: s.mtimeMs, sizeBytes: s.size }
const entry = getEntry(cache, filePath, fp)
if (entry) return { kind: 'exact', calls: entry.calls }
// Rollouts are append-only: the same inode, grown past a boundary we
// recorded, can be picked up from that boundary instead of re-read whole.
const stale = cache.files[filePath]
if (
stale
&& stale.dev === fp.dev
&& stale.ino === fp.ino
&& stale.resumeOffset !== undefined
&& stale.resumeState !== undefined
&& stale.resumeCallCount !== undefined
&& fp.sizeBytes > stale.sizeBytes
&& stale.resumeOffset <= fp.sizeBytes
&& await endsLineAt(filePath, stale.resumeOffset)
) {
return { kind: 'resume', calls: stale.calls, offset: stale.resumeOffset, state: stale.resumeState, callCount: stale.resumeCallCount }
}
} catch {}
return null
}
@ -81,8 +159,8 @@ export async function getCachedCodexProject(
): Promise<string | null> {
try {
const s = await stat(filePath)
const cache = await loadCache()
const entry = getEntry(cache, filePath, { mtimeMs: s.mtimeMs, sizeBytes: s.size })
const cache = await loadCache(currentCacheDir())
const entry = getEntry(cache, filePath, { dev: s.dev, ino: s.ino, mtimeMs: s.mtimeMs, sizeBytes: s.size })
return entry?.project ?? null
} catch {}
return null
@ -93,7 +171,7 @@ export async function fingerprintFile(
): Promise<FileFingerprint | null> {
try {
const s = await stat(filePath)
return { mtimeMs: s.mtimeMs, sizeBytes: s.size }
return { dev: s.dev, ino: s.ino, mtimeMs: s.mtimeMs, sizeBytes: s.size }
} catch {
return null
}
@ -104,19 +182,25 @@ export async function writeCachedCodexResults(
project: string,
calls: ParsedProviderCall[],
fingerprint: FileFingerprint,
resume?: { offset: number; state: unknown; callCount: number },
): Promise<void> {
try {
const cache = await loadCache()
const cache = await loadCache(currentCacheDir())
cache.files[filePath] = {
dev: fingerprint.dev,
ino: fingerprint.ino,
mtimeMs: fingerprint.mtimeMs,
sizeBytes: fingerprint.sizeBytes,
project,
calls,
...(resume ? { resumeOffset: resume.offset, resumeState: resume.state, resumeCallCount: resume.callCount } : {}),
}
} catch {}
}
export async function flushCodexCache(): Promise<void> {
const cacheDir = currentCacheDir()
const memCache = memCaches.get(cacheDir)
if (!memCache) return
try {
// Evict entries for files that no longer exist on disk
@ -129,9 +213,8 @@ export async function flushCodexCache(): Promise<void> {
}
}
const dir = getCacheDir()
if (!existsSync(dir)) await mkdir(dir, { recursive: true })
const finalPath = getCachePath()
if (!existsSync(cacheDir)) await mkdir(cacheDir, { recursive: true })
const finalPath = getCachePath(cacheDir)
const tempPath = `${finalPath}.${randomBytes(8).toString('hex')}.tmp`
const payload = JSON.stringify(memCache)
const handle = await open(tempPath, 'w', 0o600)

32
src/content-utils.ts
View file

@ -24,3 +24,35 @@ export function normalizeContentBlocks<T extends { type?: string; text?: string
if (typeof content === 'string') return [{ type: 'text', text: content } as T]
return []
}
/// Take a bounded prefix of a string as a FLAT copy.
///
/// `String.prototype.slice` returns a V8 SlicedString — a view object that
/// retains a reference to its ENTIRE parent string. Session files routinely
/// carry 100KB+ message strings (agent-injected system prompts, tool
/// results); storing a short `.slice()` of each in a long-lived structure
/// (the session cache) pins every parent buffer for the life of the process.
/// Across thousands of session files this balloons a cold parse of a few GB
/// of JSONL into an out-of-memory crash (~5.5GB peak observed), while a warm
/// run — whose strings were flattened by the cache's JSON round-trip — needs
/// only ~300MB for the same data.
///
/// Round-tripping through a Buffer forces a fresh flat string with no parent
/// reference. This always runs, even when `s` is already within `max`:
/// callers may pass an already-sliced view (provider adapters pre-truncate
/// with `.slice(0, 500)` before the cache-site call), and that view is
/// itself a SlicedString pinning its own large parent.
export function flatSlice(s: string, max: number): string {
return Buffer.from(s.slice(0, max), 'utf16le').toString('utf16le')
}
/// Force a FLAT copy of a string regardless of length.
///
/// Companion to `flatSlice` for strings that are ALREADY short but were
/// produced as views over a large parent — regex match groups
/// (`match[1]` retains the entire subject string) and `trim()` results
/// both come back as V8 SlicedStrings. Use this when storing such values
/// in long-lived structures; use `flatSlice` when also bounding length.
export function flatString(s: string): string {
return Buffer.from(s, 'utf16le').toString('utf16le')
}

21
src/currency.ts
View file

@ -1,7 +1,7 @@
import { readFile, writeFile, mkdir } from 'fs/promises'
import { join } from 'path'
import { homedir } from 'os'
import { getCodeburnCacheDir } from './cache-dir.js'
import { readConfig } from './config.js'
import { fetchWithTimeout } from './fetch-utils.js'
@ -72,15 +72,8 @@ export function roundForActiveCurrency(value: number): number {
return Math.round(value * factor) / factor
}
function getCacheDir(): string {
// Honor the same relocation override every other cache module uses
// (session-cache, daily-cache, codex-cache, models); this was the one
// straggler still hardcoding the default path.
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
}
function getRateCachePath(): string {
return join(getCacheDir(), 'exchange-rate.json')
return join(getCodeburnCacheDir(), 'exchange-rate.json')
}
async function fetchRate(code: string): Promise<number> {
@ -111,7 +104,7 @@ async function loadCachedRate(code: string): Promise<number | null> {
}
async function cacheRate(code: string, rate: number): Promise<void> {
await mkdir(getCacheDir(), { recursive: true })
await mkdir(getCodeburnCacheDir(), { recursive: true })
await writeFile(getRateCachePath(), JSON.stringify({ timestamp: Date.now(), code, rate }))
}
@ -138,7 +131,13 @@ async function getExchangeRate(code: string): Promise<number> {
export async function loadCurrency(): Promise<void> {
const config = await readConfig()
if (!config.currency) return
if (!config.currency) {
// A long-lived `serve` process may previously have loaded a non-USD
// currency. Removing the config entry is the USD reset contract, so reset
// the module state as well as letting the output memo invalidate.
active = USD
return
}
const code = config.currency.code.toUpperCase()
const rate = await getExchangeRate(code)

10
src/cursor-cache.ts
View file

@ -1,8 +1,8 @@
import { readFile, writeFile, mkdir, rename, stat, unlink } from 'fs/promises'
import { join } from 'path'
import { homedir } from 'os'
import { randomBytes } from 'crypto'
import { getCodeburnCacheDir } from './cache-dir.js'
import type { ParsedProviderCall } from './providers/types.js'
// Bumped to 3 for the workspace-aware breakdown change: the cursor parser
@ -31,12 +31,8 @@ type ResultCache = {
const CACHE_FILE = 'cursor-results.json'
function getCacheDir(): string {
return join(homedir(), '.cache', 'codeburn')
}
function getCachePath(): string {
return join(getCacheDir(), CACHE_FILE)
return join(getCodeburnCacheDir(), CACHE_FILE)
}
async function getDbFingerprint(dbPath: string): Promise<{ mtimeMs: number; size: number } | null> {
@ -86,7 +82,7 @@ export async function writeCachedResults(
const fp = await getDbFingerprint(dbPath)
if (!fp) return
const dir = getCacheDir()
const dir = getCodeburnCacheDir()
await mkdir(dir, { recursive: true }).catch(() => {})
const cache: ResultCache = {
version: CURSOR_CACHE_VERSION,

13
src/daily-cache.ts
View file

@ -1,8 +1,9 @@
import { randomBytes } from 'crypto'
import { existsSync } from 'fs'
import { mkdir, open, readdir, readFile, rename, stat, unlink } from 'fs/promises'
import { homedir } from 'os'
import { join } from 'path'
import { getCodeburnCacheDir } from './cache-dir.js'
import type { DateRange, ProjectSummary } from './types.js'
// Bumped to 17: copilot CLI sessions were misclassified as VS Code transcripts
@ -176,10 +177,6 @@ export type DailyCache = {
watermarkTrusted?: boolean
}
function getCacheDir(): string {
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
}
/** IANA name of the current local timezone (respects the TZ env var). Days are
* bucketed by local midnight, so this tags the cache for TZ-change invalidation. */
export function currentTzKey(): string {
@ -187,7 +184,7 @@ export function currentTzKey(): string {
}
function getCachePath(): string {
return join(getCacheDir(), DAILY_CACHE_FILENAME)
return join(getCodeburnCacheDir(), DAILY_CACHE_FILENAME)
}
/** Absolute path of the active (version-suffixed) daily cache file. */
@ -379,7 +376,7 @@ function isAdoptableCache(parsed: unknown): parsed is AdoptableCache {
/// bump lossless: the new version starts from the union of everything every
/// previous version ever recorded, then re-derives what sources still support.
async function adoptOlderDailyCaches(): Promise<DailyCache> {
const dir = getCacheDir()
const dir = getCodeburnCacheDir()
let names: string[] = []
try {
names = await readdir(dir)
@ -449,7 +446,7 @@ async function adoptOlderDailyCaches(): Promise<DailyCache> {
}
export async function saveDailyCache(cache: DailyCache): Promise<void> {
const dir = getCacheDir()
const dir = getCodeburnCacheDir()
if (!existsSync(dir)) await mkdir(dir, { recursive: true })
const finalPath = getCachePath()
const tempPath = `${finalPath}.${randomBytes(8).toString('hex')}.tmp`

46
src/models.ts
View file

@ -1,8 +1,9 @@
import { readFile, writeFile, mkdir } from 'fs/promises'
import { join } from 'path'
import { homedir } from 'os'
import snapshotData from './data/litellm-snapshot.json'
import fallbackData from './data/pricing-fallback.json'
import { getCodeburnCacheDir } from './cache-dir.js'
import snapshotData from './data/litellm-snapshot.json' with { type: 'json' }
import fallbackData from './data/pricing-fallback.json' with { type: 'json' }
import { fetchWithTimeout } from './fetch-utils.js'
export type ModelCosts = {
@ -143,13 +144,8 @@ function getLowercasePricingIndex(): Map<string, ModelCosts> {
return lowercasePricingIndex
}
function getCacheDir(): string {
if (process.env['CODEBURN_CACHE_DIR']) return process.env['CODEBURN_CACHE_DIR']
return join(homedir(), '.cache', 'codeburn')
}
function getCachePath(): string {
return join(getCacheDir(), 'litellm-pricing.json')
return join(getCodeburnCacheDir(), 'litellm-pricing.json')
}
/// Clamp a per-token rate to a sane non-negative value. Defense in depth
@ -202,7 +198,7 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
if (stripped !== name && !pricing.has(stripped)) pricing.set(stripped, costs)
}
await mkdir(getCacheDir(), { recursive: true })
await mkdir(getCodeburnCacheDir(), { recursive: true })
await writeFile(getCachePath(), JSON.stringify({
timestamp: Date.now(),
data: Object.fromEntries(pricing),
@ -999,3 +995,33 @@ export function getShortModelName(model: string): string {
}
return canonical
}
// Pricing is process-global state assembled at CLI startup from the cached
// LiteLLM snapshot plus user config. A parse worker thread starts with none of
// it, and re-running loadPricing() there would mean N more disk reads (or, on a
// cold pricing cache, N network fetches). Ship the resolved state across
// instead, so every thread prices a call exactly as the main thread would.
export type PricingSnapshot = {
pricing: Map<string, ModelCosts>
aliases: Record<string, string>
priceOverrides: Record<string, PriceOverrideRates>
localModelSavings: Record<string, string>
}
export function snapshotPricingState(): PricingSnapshot {
return {
pricing: pricingCache,
aliases: userAliases,
priceOverrides: userPriceOverridesConfig,
localModelSavings: userLocalModelSavings,
}
}
export function restorePricingState(snapshot: PricingSnapshot): void {
pricingCache = snapshot.pricing
sortedPricingKeys = null
lowercasePricingIndex = null
setModelAliases(snapshot.aliases)
setPriceOverrides(snapshot.priceOverrides)
setLocalModelSavings(snapshot.localModelSavings)
}

34
src/parse-worker.ts Normal file
View file

@ -0,0 +1,34 @@
import { parentPort, workerData } from 'worker_threads'
import { restorePricingState, type PricingSnapshot } from './models.js'
import type { ParseJob } from './parse-workers.js'
import { parseClaudeFileFull } from './parser.js'
import { parseCodexFileFull } from './providers/codex.js'
const port = parentPort
if (!port) throw new Error('parse-worker must be started as a worker thread')
restorePricingState((workerData as { pricing: PricingSnapshot }).pricing)
// The parsed turns go back as a JSON string rather than as a live object graph:
// structured-cloning a whole corpus of turns costs more than the parallel parse
// saves, while a string is a single copy the parent re-parses at memcpy speed.
// `msgIds` / `keys` is every dedup key this file claimed; the parent uses it to
// prove no earlier file already owned one before installing the result. A Codex
// job also carries back the cache entry it would have written, because the cache
// module's per-directory state belongs to the parent, not to a thread.
port.on('message', (msg: ParseJob) => {
void (async () => {
try {
const seen = new Set<string>()
if (msg.kind === 'codex') {
const parsed = await parseCodexFileFull(msg.source, seen)
port.postMessage({ json: JSON.stringify({ ...parsed, keys: [...seen], path: msg.source.path }) })
return
}
const parsed = await parseClaudeFileFull(msg.filePath, seen)
port.postMessage({ json: parsed === null ? null : JSON.stringify({ ...parsed, msgIds: [...seen], path: msg.filePath }) })
} catch (err) {
port.postMessage({ error: err instanceof Error ? err.message : String(err) })
}
})()
})

245
src/parse-workers.ts Normal file
View file

@ -0,0 +1,245 @@
import { availableParallelism, totalmem } from 'os'
import { Worker } from 'worker_threads'
import { snapshotPricingState } from './models.js'
import type { ClaudeFileParse } from './parser.js'
import type { SessionSource } from './providers/types.js'
// A worker holds one file's entries plus its serialized result, and the parent
// buffers up to `pool.size` finished results while it installs one — both are in
// this budget. A flat 256 MB was measured wrong on Codex: a 260 MB rollout peaks
// near 430 MB per worker and scales linearly with the pool. So derive it from the
// average pending file instead, floored at the small-transcript figure and capped
// at 1 GB. Going over the budget is what turns a parallel parse into a swapping one.
const MIN_PER_WORKER_RSS_BYTES = 256 * 1024 * 1024
const MAX_PER_WORKER_RSS_BYTES = 1024 * 1024 * 1024
const PER_WORKER_RSS_OVERHEAD_BYTES = 128 * 1024 * 1024
const MEMORY_BUDGET_CAP_BYTES = 2 * 1024 * 1024 * 1024
const MIN_AVAILABLE_BYTES = 4 * 1024 * 1024 * 1024
const MIN_FILES_PER_WORKER = 50
const MIN_BYTES_PER_WORKER = 200 * 1024 * 1024
// Below this a parse is warm/incremental and the thread startup + result transfer
// costs more than the parallelism buys. Bytes, not file count: 250 pending files
// holding under a megabyte between them spawn threads that make the run ~5%
// SLOWER, and the file count only starts paying for itself around 400.
const MIN_PENDING_BYTES = 200 * 1024 * 1024
export type ParseWorkerDecision = { workers: number; reason: string }
export type SystemCapacity = { cores: number; availableBytes: number }
// `process.availableMemory()` respects a container's cgroup / rlimit, which is the
// case this gate exists for; where it is absent it falls back to total RAM. Free
// memory is deliberately NOT used: on macOS `os.freemem()` counts free pages, not
// available memory, and reads as a few hundred MB on an idle 128 GB machine — a
// gate built on it turns the feature on and off at random.
function currentSystemCapacity(): SystemCapacity {
return {
cores: availableParallelism(),
availableBytes: typeof process.availableMemory === 'function' ? process.availableMemory() : totalmem(),
}
}
/// Decide how many parse worker threads a pending workload earns. Returning 0
/// means "parse serially" — the only behaviour before this existed, and still
/// the behaviour for every warm run, every small corpus, and every low-spec box.
export function decideParseWorkers(
pending: { files: number; bytes: number },
sys: SystemCapacity = currentSystemCapacity(),
env: NodeJS.ProcessEnv = process.env,
): ParseWorkerDecision {
// Every reason carries the full decision input, so a support log line explains
// itself without a second run.
const inputs = `${sys.cores} cores, ${Math.round(sys.availableBytes / 1e9 * 10) / 10} GB available, ${pending.files} pending files / ${Math.round(pending.bytes / 1e6)} MB`
const override = env['CODEBURN_PARSE_WORKERS']
if (override !== undefined && override !== '') {
const n = Number(override)
if (!Number.isFinite(n) || n < 0) return { workers: 0, reason: `invalid CODEBURN_PARSE_WORKERS=${override}` }
const capped = Math.min(Math.floor(n), sys.cores)
return { workers: capped, reason: `${capped === 0 ? 'forced serial' : 'forced'} by CODEBURN_PARSE_WORKERS=${override}; ${inputs}` }
}
// Workload gates first, so a warm run's log line says "warm", not whatever the
// machine happened to look like at that moment.
if (pending.bytes < MIN_PENDING_BYTES) return { workers: 0, reason: `below ${Math.round(MIN_PENDING_BYTES / 1e6)} MB pending; ${inputs}` }
if (sys.cores <= 2) return { workers: 0, reason: `too few cores; ${inputs}` }
if (sys.availableBytes < MIN_AVAILABLE_BYTES) return { workers: 0, reason: `below ${Math.round(MIN_AVAILABLE_BYTES / 1e9)} GB available memory; ${inputs}` }
const memoryBudget = Math.min(0.25 * sys.availableBytes, MEMORY_BUDGET_CAP_BYTES)
const perWorker = Math.min(
MAX_PER_WORKER_RSS_BYTES,
Math.max(MIN_PER_WORKER_RSS_BYTES, 2 * (pending.bytes / Math.max(1, pending.files)) + PER_WORKER_RSS_OVERHEAD_BYTES),
)
// Files and bytes each earn threads on their own: a few hundred huge rollouts
// are as parallelisable as a few thousand small transcripts, and gating the
// count on files alone would hand a 6 GB / 60-file workload a single thread.
const workers = Math.min(
sys.cores - 1,
Math.floor(memoryBudget / perWorker),
Math.max(
Math.floor(pending.files / MIN_FILES_PER_WORKER),
Math.floor(pending.bytes / MIN_BYTES_PER_WORKER),
),
)
return { workers, reason: inputs }
}
// In dist the entry is the bundled sibling of this module and a worker can load
// it directly. Running from source (tsx, vitest) the entry is TypeScript, and a
// worker thread inherits none of the parent's loader hooks — so register tsx's
// inside the thread before importing. tsx is a devDependency, which is exactly
// the only situation where the entry can be a .ts file at all.
function workerBootstrap(entryUrl: string): { source: string | URL; eval: boolean } {
if (!entryUrl.endsWith('.ts')) return { source: new URL(entryUrl), eval: false }
return {
eval: true,
// Chained, not awaited: the eval scope is CommonJS, and a top-level await of
// the entry re-enters it as a require(esm) cycle.
source: `
process.noDeprecation = true
import('tsx/esm/api').then(tsx => { tsx.register(); return import(${JSON.stringify(entryUrl)}) })
`,
}
}
function workerEntryUrl(): string {
const ext = import.meta.url.endsWith('.ts') ? '.ts' : '.js'
return new URL(`./parse-worker${ext}`, import.meta.url).href
}
/// One whole-file parse for a worker to run. Both kinds carry exactly what the
/// serial per-file parse takes, so the worker can run that same function.
export type ParseJob =
| { kind: 'claude'; filePath: string }
| { kind: 'codex'; source: SessionSource }
export type ParseWorkerResult<T> =
| { ok: true; parsed: T | null }
| { ok: false; error: string }
export type ClaudeWorkerParse = ClaudeFileParse & { msgIds: string[]; path: string }
type Task = { job: ParseJob; resolve: (r: ParseWorkerResult<unknown>) => void }
type WorkerMessage = { json?: string | null; error?: string }
export class ParseWorkerPool {
private readonly workers: Worker[] = []
private readonly idle: Worker[] = []
private readonly inflight = new Map<Worker, Task>()
private readonly queue: Task[] = []
private closed = false
constructor(size: number) {
const boot = workerBootstrap(workerEntryUrl())
const workerData = { pricing: snapshotPricingState() }
try {
for (let i = 0; i < size; i++) {
const worker = new Worker(boot.source, { eval: boot.eval, workerData })
worker.on('message', (msg: WorkerMessage) => this.settle(worker, msg))
worker.on('error', (err: Error) => this.settle(worker, { error: err.message }, true))
worker.on('exit', () => this.drop(worker))
this.workers.push(worker)
this.idle.push(worker)
}
} catch (err) {
for (const w of this.workers) void w.terminate()
this.workers.length = 0
this.idle.length = 0
throw err
}
}
get size(): number {
return this.workers.length
}
/// Parse one file off-thread. Never rejects: a worker-side failure (or a dead
/// pool) comes back as `ok: false` so the caller can fall back to an in-process
/// parse and never lose a file to a crashed thread.
submit<T>(job: ParseJob): Promise<ParseWorkerResult<T>> {
return new Promise<ParseWorkerResult<T>>((resolve) => {
if (this.closed || this.workers.length === 0) {
resolve({ ok: false, error: 'parse worker pool unavailable' })
return
}
this.queue.push({ job, resolve: resolve as (r: ParseWorkerResult<unknown>) => void })
this.pump()
})
}
async close(): Promise<void> {
this.closed = true
const pending = [...this.queue]
this.queue.length = 0
for (const task of pending) task.resolve({ ok: false, error: 'parse worker pool closed' })
await Promise.all(this.workers.map(w => w.terminate()))
this.workers.length = 0
this.idle.length = 0
this.inflight.clear()
}
private pump(): void {
while (this.queue.length > 0 && this.idle.length > 0) {
const worker = this.idle.pop()!
const task = this.queue.shift()!
this.inflight.set(worker, task)
worker.postMessage(task.job)
}
}
private settle(worker: Worker, msg: WorkerMessage, fatal = false): void {
const task = this.inflight.get(worker)
this.inflight.delete(worker)
if (task) {
if (msg.error !== undefined) task.resolve({ ok: false, error: msg.error })
else task.resolve({ ok: true, parsed: msg.json == null ? null : JSON.parse(msg.json) })
}
if (fatal) return
if (!this.closed) {
this.idle.push(worker)
this.pump()
}
}
// A thread that died takes its queue slot with it; the remaining files are
// handed back for a serial parse rather than being lost.
private drop(worker: Worker): void {
const i = this.workers.indexOf(worker)
if (i >= 0) this.workers.splice(i, 1)
const j = this.idle.indexOf(worker)
if (j >= 0) this.idle.splice(j, 1)
const task = this.inflight.get(worker)
if (task) {
this.inflight.delete(worker)
task.resolve({ ok: false, error: 'parse worker exited' })
}
if (this.workers.length === 0) {
const pending = [...this.queue]
this.queue.length = 0
for (const t of pending) t.resolve({ ok: false, error: 'all parse workers exited' })
}
}
}
/// Yield results for `jobs` in the SAME order they were given, no matter which
/// worker finishes first. Keeps exactly `pool.size` files in flight, so at most
/// that many parsed results are buffered while the caller installs one.
export async function* parseFilesInOrder<T>(
pool: ParseWorkerPool,
jobs: readonly ParseJob[],
): AsyncGenerator<ParseWorkerResult<T>, void, void> {
const inflight: Array<Promise<ParseWorkerResult<T>>> = []
let next = 0
const fill = (): void => {
while (inflight.length < Math.max(1, pool.size) && next < jobs.length) {
inflight.push(pool.submit<T>(jobs[next++]!))
}
}
fill()
for (let i = 0; i < jobs.length; i++) {
const result = await inflight.shift()!
fill()
yield result
}
}

765
src/parser.ts
View file

File diff suppressed because it is too large Load diff

85
src/providers/antigravity.ts
View file

@ -1,11 +1,12 @@
import { readdir, readFile, mkdir, stat, open, rename, unlink } from 'fs/promises'
import { execFile } from 'child_process'
import { randomBytes } from 'crypto'
import { basename, join } from 'path'
import { basename, join, resolve } from 'path'
import { homedir } from 'os'
import { fileURLToPath } from 'url'
import https from 'https'
import { getCodeburnCacheDir } from '../cache-dir.js'
import { calculateCost } from '../models.js'
import { isSqliteAvailable, isSqliteBusyError, openDatabase } from '../sqlite.js'
import type { ProbeRoot, Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
@ -161,8 +162,17 @@ type AntigravityGenMetadataRow = {
const cachedServers = new Map<string, ServerInfo | null>()
const cachedModelMaps = new Map<string, ModelMap>()
let memCache: AntigravityCache | null = null
let cacheDirty = false
type AntigravityCacheState = { cache: AntigravityCache; dirty: boolean }
const cacheStates = new Map<string, AntigravityCacheState>()
// Dropped by the resident RSS guard. A dirty state holds cascades not yet on
// disk, so it stays resident until its own flush publishes it.
export function clearAntigravityCacheStates(): void {
for (const [dir, state] of cacheStates) {
if (!state.dirty) cacheStates.delete(dir)
}
}
let httpsAgent: https.Agent | undefined
const protoTextDecoder = new TextDecoder('utf-8', { fatal: false })
@ -175,16 +185,16 @@ function getAgent(): https.Agent {
return httpsAgent
}
function getCacheDir(): string {
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
function currentCacheDir(): string {
return resolve(getCodeburnCacheDir())
}
function getCachePath(): string {
return join(getCacheDir(), 'antigravity-results.json')
function getCachePath(cacheDir: string): string {
return join(cacheDir, 'antigravity-results.json')
}
export function getAntigravityStatusLineEventsPath(): string {
return join(getCacheDir(), 'antigravity-statusline.jsonl')
return join(getCodeburnCacheDir(), 'antigravity-statusline.jsonl')
}
function execFileText(command: string, args: string[], timeout = 3000): Promise<string> {
@ -323,22 +333,30 @@ export function extractAntigravityGeneratorMetadata(resp: unknown): GeneratorMet
return Array.isArray(metadata) ? metadata : []
}
async function loadCache(): Promise<AntigravityCache> {
if (memCache) return memCache
async function loadCache(cacheDir: string): Promise<AntigravityCacheState> {
const inMemory = cacheStates.get(cacheDir)
if (inMemory) return inMemory
try {
const raw = await readFile(getCachePath(), 'utf-8')
const raw = await readFile(getCachePath(cacheDir), 'utf-8')
const cache = JSON.parse(raw) as AntigravityCache
if (cache.version === CACHE_VERSION && cache.cascades && typeof cache.cascades === 'object') {
memCache = cache
return cache
const state = { cache, dirty: false }
cacheStates.set(cacheDir, state)
return state
}
} catch { /* no cache or invalid */ }
memCache = { version: CACHE_VERSION, cascades: {} }
return memCache
const state: AntigravityCacheState = {
cache: { version: CACHE_VERSION, cascades: {} },
dirty: false,
}
cacheStates.set(cacheDir, state)
return state
}
async function flushCache(liveCascadeIds?: Set<string>): Promise<void> {
if (!memCache) return
async function flushCache(liveCascadeIds?: Set<string>, cacheDir = currentCacheDir()): Promise<void> {
const state = cacheStates.get(cacheDir)
if (!state) return
const memCache = state.cache
// If the caller supplied liveCascadeIds, we must run the eviction step
// even when no cascade was added or updated this run; otherwise deleted
// .pb files would persist in the cache forever once it stops getting
@ -348,16 +366,14 @@ async function flushCache(liveCascadeIds?: Set<string>): Promise<void> {
for (const id of Object.keys(memCache.cascades)) {
if (!liveCascadeIds.has(id)) {
delete memCache.cascades[id]
cacheDirty = true
state.dirty = true
}
}
}
if (!cacheDirty) return
if (!state.dirty) return
try {
const dir = getCacheDir()
await mkdir(dir, { recursive: true })
const finalPath = getCachePath()
await mkdir(cacheDir, { recursive: true })
const finalPath = getCachePath(cacheDir)
const tempPath = `${finalPath}.${randomBytes(8).toString('hex')}.tmp`
const handle = await open(tempPath, 'w', 0o600)
try {
@ -371,7 +387,7 @@ async function flushCache(liveCascadeIds?: Set<string>): Promise<void> {
} catch {
try { await unlink(tempPath) } catch { /* cleanup */ }
}
cacheDirty = false
state.dirty = false
} catch { /* best-effort */ }
}
@ -1009,7 +1025,7 @@ export async function recordAntigravityStatusLinePayload(input: unknown): Promis
if (!event) return false
const path = getAntigravityStatusLineEventsPath()
await mkdir(getCacheDir(), { recursive: true, mode: 0o700 })
await mkdir(getCodeburnCacheDir(), { recursive: true, mode: 0o700 })
const fd = await open(path, 'a', 0o600)
try {
await fd.appendFile(`${JSON.stringify(event)}\n`, { encoding: 'utf-8' })
@ -1173,7 +1189,9 @@ export async function snapshotAntigravityStatusLinePayload(input: unknown): Prom
const s = await stat(source.path).catch(() => null)
if (!s) return false
const cache = await loadCache()
const cacheDir = currentCacheDir()
const state = await loadCache(cacheDir)
const cache = state.cache
const cached = cache.cascades[cascadeId]
if (cached && cached.mtimeMs === s.mtimeMs && cached.sizeBytes === s.size && cached.calls.length > 0) {
return true
@ -1195,8 +1213,8 @@ export async function snapshotAntigravityStatusLinePayload(input: unknown): Prom
sizeBytes: s.size,
calls: snapshotCalls,
}
cacheDirty = true
await flushCache()
state.dirty = true
await flushCache(undefined, cacheDir)
return cache.cascades[cascadeId]!.calls.length > 0
} catch {
return false
@ -1300,7 +1318,8 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
}
const cascadeId = antigravityCascadeIdFromPath(source.path)
const cache = await loadCache()
const state = await loadCache(currentCacheDir())
const cache = state.cache
const s = await stat(source.path).catch(() => null)
if (!s) return
@ -1331,7 +1350,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
sizeBytes: s.size,
calls: sqliteResults,
}
cacheDirty = true
state.dirty = true
for (const call of sqliteResults) {
if (seenKeys.has(call.deduplicationKey)) continue
@ -1384,7 +1403,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
sizeBytes: s.size,
calls: results,
}
cacheDirty = true
state.dirty = true
for (const call of results) {
if (seenKeys.has(call.deduplicationKey)) continue
@ -1445,8 +1464,8 @@ export function createAntigravityProvider(): Provider {
}
}
export async function flushAntigravityCache(liveCascadeIds?: Set<string>): Promise<void> {
await flushCache(liveCascadeIds)
export async function flushAntigravityCache(liveCascadeIds?: Set<string>, cacheDir?: string): Promise<void> {
await flushCache(liveCascadeIds, cacheDir ? resolve(cacheDir) : currentCacheDir())
}
export const antigravity = createAntigravityProvider()

200
src/providers/codex.ts
View file

@ -6,7 +6,7 @@ import { homedir } from 'os'
import { readSessionLines } from '../fs-utils.js'
import { calculateCost } from '../models.js'
import { readCachedCodexResults, writeCachedCodexResults, getCachedCodexProject, fingerprintFile } from '../codex-cache.js'
import { readCachedCodexResults, writeCachedCodexResults, getCachedCodexProject, fingerprintFile, type CodexFileFingerprint } from '../codex-cache.js'
import { normalizeContentBlocks } from '../content-utils.js'
import { estimateTokensFromChars } from '../token-estimate.js'
import type { ToolCall } from '../types.js'
@ -569,52 +569,131 @@ function resolveModel(info: CodexEntry['payload'], sessionModel?: string): strin
return firstModelString(info?.model, info?.info?.model, info?.info?.model_name, sessionModel) ?? 'gpt-5'
}
function createParser(source: SessionSource, seenKeys: Set<string>): SessionParser {
// Everything the single-pass decode carries across a `task_started` boundary.
// A rollout is append-only, so recording this at such a boundary (where the
// previous task has been flushed to `results` and the per-task accumulators are
// empty) lets a later run restart there and produce byte-identical output
// instead of re-reading the whole file. Every field the loop below reads from a
// PRIOR line must appear here, or a resumed decode silently diverges.
type CodexResumeState = {
sessionModel?: string
sessionId: string
sessionCwd?: string
forkedFromId: string
forkCutoff: string
prevCumulativeTotal: number | null
prevInput: number
prevCached: number
prevOutput: number
prevReasoning: number
pendingTools: string[]
pendingToolSequence: ToolCall[][]
pendingUserMessage: string
pendingOutputChars: number
pendingLocAdded: number
pendingLocRemoved: number
pendingEditFailed: number
estCounter: number
turnCounter: number
currentTurnId: string
taskStartedAt?: number
}
// The state comes back off our own JSON cache; a truncated or hand-edited file
// must fall back to a full re-parse rather than decode against nonsense.
function isResumeState(value: unknown): value is CodexResumeState {
if (!value || typeof value !== 'object') return false
const v = value as Record<string, unknown>
return typeof v['sessionId'] === 'string'
&& typeof v['forkedFromId'] === 'string'
&& typeof v['forkCutoff'] === 'string'
&& (v['prevCumulativeTotal'] === null || typeof v['prevCumulativeTotal'] === 'number')
&& typeof v['prevInput'] === 'number'
&& typeof v['prevCached'] === 'number'
&& typeof v['prevOutput'] === 'number'
&& typeof v['prevReasoning'] === 'number'
&& Array.isArray(v['pendingTools'])
&& Array.isArray(v['pendingToolSequence'])
&& typeof v['pendingUserMessage'] === 'string'
&& typeof v['pendingOutputChars'] === 'number'
&& typeof v['pendingLocAdded'] === 'number'
&& typeof v['pendingLocRemoved'] === 'number'
&& typeof v['pendingEditFailed'] === 'number'
&& typeof v['estCounter'] === 'number'
&& typeof v['turnCounter'] === 'number'
&& typeof v['currentTurnId'] === 'string'
}
/** What the serial path would have written to the codex cache for one file. */
export type CodexCacheWrite = {
project: string
fingerprint: CodexFileFingerprint
resume?: { offset: number; state: unknown; callCount: number }
}
// When `capture` is passed the parse is a whole-file decode that never touches
// the codex cache: no hit lookup (so no resume), and the entry it would have
// written comes back through `capture` for the caller to install. That is what
// lets a worker thread run this exact decode without owning the cache module's
// per-directory state.
function createParser(source: SessionSource, seenKeys: Set<string>, capture?: { write?: CodexCacheWrite }): SessionParser {
return {
async *parse(): AsyncGenerator<ParsedProviderCall> {
const cached = await readCachedCodexResults(source.path)
if (cached) {
for (const call of cached) {
const hit = capture ? null : await readCachedCodexResults(source.path)
if (hit?.kind === 'exact') {
for (const call of hit.calls) {
if (seenKeys.has(call.deduplicationKey)) continue
seenKeys.add(call.deduplicationKey)
yield call
}
return
}
const resume = hit && isResumeState(hit.state)
? { offset: hit.offset, state: hit.state, calls: hit.calls.slice(0, hit.callCount) }
: null
const fp = await fingerprintFile(source.path)
if (!fp) return
let sessionModel: string | undefined
let sessionId = ''
let sessionCwd: string | undefined
let forkedFromId = ''
let forkCutoff = ''
let sessionModel: string | undefined = resume?.state.sessionModel
let sessionId = resume?.state.sessionId ?? ''
let sessionCwd: string | undefined = resume?.state.sessionCwd
let forkedFromId = resume?.state.forkedFromId ?? ''
let forkCutoff = resume?.state.forkCutoff ?? ''
// Null sentinel rather than `0` so the FIRST event is never confused
// with a duplicate. A session that only emits last_token_usage (no
// total_token_usage) reports cumulativeTotal=0 on every event; with a
// 0-initialized prev, the first event would have matched and been
// dropped. Once we've observed any event, we record its cumulative
// total and dedup on equality regardless of whether it is zero.
let prevCumulativeTotal: number | null = null
let prevInput = 0
let prevCached = 0
let prevOutput = 0
let prevReasoning = 0
let pendingTools: string[] = []
let pendingToolSequence: ToolCall[][] = []
let pendingUserMessage = ''
let pendingOutputChars = 0
let prevCumulativeTotal: number | null = resume?.state.prevCumulativeTotal ?? null
let prevInput = resume?.state.prevInput ?? 0
let prevCached = resume?.state.prevCached ?? 0
let prevOutput = resume?.state.prevOutput ?? 0
let prevReasoning = resume?.state.prevReasoning ?? 0
let pendingTools: string[] = resume ? [...resume.state.pendingTools] : []
let pendingToolSequence: ToolCall[][] = resume ? [...resume.state.pendingToolSequence] : []
let pendingUserMessage = resume?.state.pendingUserMessage ?? ''
let pendingOutputChars = resume?.state.pendingOutputChars ?? 0
// Rich-session-capture: edit LOC deltas and failed-patch count accumulated
// across a turn's patch_apply_end events, flushed onto the turn's call.
let pendingLocAdded = 0
let pendingLocRemoved = 0
let pendingEditFailed = 0
let estCounter = 0
let turnCounter = 0
let currentTurnId = `${sessionId}:t0`
let pendingLocAdded = resume?.state.pendingLocAdded ?? 0
let pendingLocRemoved = resume?.state.pendingLocRemoved ?? 0
let pendingEditFailed = resume?.state.pendingEditFailed ?? 0
let estCounter = resume?.state.estCounter ?? 0
let turnCounter = resume?.state.turnCounter ?? 0
let currentTurnId = resume?.state.currentTurnId ?? `${sessionId}:t0`
let sawAnyLine = false
const results: ParsedProviderCall[] = []
// Calls already decoded before the resume boundary. They pass through the
// same cross-provider dedup a full decode would have applied to them.
if (resume) {
for (const call of resume.calls) {
if (seenKeys.has(call.deduplicationKey)) continue
seenKeys.add(call.deduplicationKey)
results.push(call)
}
}
// Calls decoded since the last task_started, held back so task_complete can
// stamp active/toolWait timing before they are appended to results. Emitting
// a task only once its timing is known keeps single-pass and split/resume
@ -623,15 +702,25 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
let pendingTaskCalls: ParsedProviderCall[] = []
let taskGeneratedTokens = 0
let taskToolIntervals: Array<[number, number]> = []
let taskStartedAt: number | undefined
let taskStartedAt: number | undefined = resume?.state.taskStartedAt
const openToolStarts = new Map<string, number>()
// Resume point for the NEXT run, refreshed at every task boundary.
const tracker = { lastCompleteLineOffset: resume?.offset ?? 0 }
let resumeOffset = resume?.offset ?? 0
let resumeState: CodexResumeState | null = resume?.state ?? null
let resumeCallCount = results.length
// Stream the session file line by line. Heavy Codex sessions can exceed
// 250 MB on disk; reading the entire file into a string would either hit
// the readSessionFile cap or push V8 toward its 512 MB string limit
// after split('\n'). readSessionLines streams raw buffers and hands
// huge lines to the compact parser without full string conversion.
for await (const rawLine of readSessionLines(source.path, undefined, { largeLineAsBuffer: true })) {
for await (const rawLine of readSessionLines(source.path, undefined, {
largeLineAsBuffer: true,
byteOffsetTracker: tracker,
...(resume ? { startByteOffset: resume.offset } : {}),
})) {
sawAnyLine = true
const entry = parseCodexLine(rawLine)
if (!entry) continue
@ -684,6 +773,33 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
const startedAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
taskStartedAt = Number.isFinite(startedAt) ? startedAt : undefined
openToolStarts.clear()
// Everything decoded so far is now in `results` and the per-task
// accumulators are empty: a clean restart point for an appended tail.
resumeOffset = tracker.lastCompleteLineOffset
resumeCallCount = results.length
resumeState = {
...(sessionModel !== undefined ? { sessionModel } : {}),
sessionId,
...(sessionCwd !== undefined ? { sessionCwd } : {}),
forkedFromId,
forkCutoff,
prevCumulativeTotal,
prevInput,
prevCached,
prevOutput,
prevReasoning,
pendingTools: [...pendingTools],
pendingToolSequence: [...pendingToolSequence],
pendingUserMessage,
pendingOutputChars,
pendingLocAdded,
pendingLocRemoved,
pendingEditFailed,
estCounter,
turnCounter,
currentTurnId,
...(taskStartedAt !== undefined ? { taskStartedAt } : {}),
}
continue
}
@ -997,13 +1113,23 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
// If the stream yielded nothing the file was unreadable, oversized, or
// empty. Skip cache write so a transient failure can't pin an empty
// result set against a fingerprint that would otherwise be re-parsed.
if (!sawAnyLine) return
// result set against a fingerprint that would otherwise be re-parsed. On a
// resume the earlier calls are still valid output, so serve them - but
// still leave the cache entry alone.
if (!sawAnyLine) {
if (resume) for (const call of results) yield call
return
}
// Flush the final task, which has no following task_started to trigger it.
results.push(...pendingTaskCalls)
await writeCachedCodexResults(source.path, source.project, results, fp)
const resumeWrite = resumeState ? { offset: resumeOffset, state: resumeState, callCount: resumeCallCount } : undefined
if (capture) {
capture.write = { project: source.project, fingerprint: fp, ...(resumeWrite ? { resume: resumeWrite } : {}) }
} else {
await writeCachedCodexResults(source.path, source.project, results, fp, resumeWrite)
}
for (const call of results) {
yield call
@ -1012,6 +1138,20 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
}
}
export type CodexFullParse = { calls: ParsedProviderCall[]; write?: CodexCacheWrite }
/// Decode one rollout end to end, exactly as the serial path does for a file
/// with no cache entry, without reading or writing the codex cache. `seenKeys`
/// is the dedup set the decode runs against — pass an empty one off-thread and
/// let the caller prove no earlier file claimed any of the keys before
/// installing the result.
export async function parseCodexFileFull(source: SessionSource, seenKeys: Set<string>): Promise<CodexFullParse> {
const capture: { write?: CodexCacheWrite } = {}
const calls: ParsedProviderCall[] = []
for await (const call of createParser(source, seenKeys, capture).parse()) calls.push(call)
return { calls, ...(capture.write ? { write: capture.write } : {}) }
}
export function createCodexProvider(codexDir?: string): Provider {
const dir = getCodexDir(codexDir)

23
src/providers/kiro.ts
View file

@ -5,6 +5,7 @@ import { basename, dirname, extname, join } from 'path'
import { homedir } from 'os'
import { readSessionFile } from '../fs-utils.js'
import { flatSlice, flatString } from '../content-utils.js'
import { calculateCost } from '../models.js'
import { estimateTokensFromChars } from '../token-estimate.js'
import type { ToolCall } from '../types.js'
@ -98,7 +99,10 @@ function extractToolNames(content: string): string[] {
let match
while ((match = regex.exec(content)) !== null) {
const name = match[1]!.trim()
tools.push(toolNameMap[name] ?? name)
// flatString: regex match groups are V8 SlicedStrings that retain the
// ENTIRE subject string — storing them in the session cache would pin
// every scanned assistant-content buffer. Mapped names are flat literals.
tools.push(toolNameMap[name] ?? flatString(name))
}
return tools
}
@ -217,7 +221,7 @@ function parseChatFile(data: KiroChatFile, sessionId: string, project: string, s
if (msg.role === 'human') {
if (msg.content.startsWith('<identity>')) continue
inputChars += msg.content.length
pendingUserMessage = msg.content.slice(0, 500)
pendingUserMessage = flatSlice(msg.content, 500)
}
if (msg.role === 'bot') {
const msgTools = extractToolNames(msg.content)
@ -296,7 +300,7 @@ function parseModernExecution(data: KiroModernExecution, sourcePath: string, see
if (directInput) {
inputChars += directInput.length
pendingUserMessage = directInput.slice(0, 500)
pendingUserMessage = flatSlice(directInput, 500)
}
if (directOutput) {
@ -328,7 +332,7 @@ function parseModernExecution(data: KiroModernExecution, sourcePath: string, see
if (role === 'human' || role === 'user') {
if (!text) continue
inputChars += text.length
pendingUserMessage = text.slice(0, 500)
pendingUserMessage = flatSlice(text, 500)
} else if (role === 'bot' || role === 'assistant' || role === 'ai' || role === 'model') {
if (text) outputChars += text.length
if (text || tools.length > 0) hasOutputActivity = true
@ -506,6 +510,7 @@ function parseCliSession(meta: KiroCliSessionMeta, entries: KiroCliEntry[], seen
userMessage: pendingUserMessage,
sessionId,
project,
...(meta.cwd ? { projectPath: meta.cwd } : {}),
})
turnIndex++
}
@ -526,7 +531,7 @@ function parseCliSession(meta: KiroCliSessionMeta, entries: KiroCliEntry[], seen
for (const item of content) {
const rec = asRecord(item)
if (rec && rec['kind'] === 'text' && typeof rec['data'] === 'string') {
pendingUserMessage = (rec['data'] as string).slice(0, 500)
pendingUserMessage = flatSlice(rec['data'] as string, 500)
inputChars += (rec['data'] as string).length
}
}
@ -605,7 +610,7 @@ async function parseWorkspaceSession(record: Record<string, unknown>, source: Se
const text = extractText(msg['content'])
if (role === 'user' && text) {
inputChars += text.length
pendingUserMessage = text.slice(0, 500)
pendingUserMessage = flatSlice(text, 500)
} else if (role === 'assistant' && !execBacked && text && text !== 'On it.') {
// An item carrying an executionId is execution-backed: its content is
// counted from the execution file, so counting it here would double-count.
@ -662,6 +667,9 @@ async function parseWorkspaceSession(record: Record<string, unknown>, source: Se
deduplicationKey: dedupKey,
userMessage: pendingUserMessage,
sessionId,
...(typeof record['workspaceDirectory'] === 'string' && record['workspaceDirectory']
? { projectPath: record['workspaceDirectory'] as string }
: {}),
})
return results
@ -774,6 +782,7 @@ async function parseV2Session(source: SessionSource, seenKeys: Set<string>): Pro
userMessage: turnUserMessage,
sessionId,
project: source.project,
...(meta.workspacePaths?.[0] ? { projectPath: meta.workspacePaths[0] } : {}),
})
}
}
@ -794,7 +803,7 @@ async function parseV2Session(source: SessionSource, seenKeys: Set<string>): Pro
// for the upcoming turn_start.
if (inTurn) flushTurn()
const text = typeof payload['content'] === 'string' ? payload['content'] as string : extractText(payload['content'])
pendingUserMessage = text.slice(0, 500)
pendingUserMessage = flatSlice(text, 500)
pendingUserChars = text.length
} else if (type === 'turn_start') {
if (inTurn) flushTurn()

332
src/serve.ts
View file

@ -1,8 +1,11 @@
import { watch, type FSWatcher } from 'fs'
import { stat } from 'fs/promises'
import { readFile, stat } from 'fs/promises'
import { createHash } from 'crypto'
import { createInterface } from 'readline'
import type { Command } from 'commander'
import { getConfigFilePath } from './config.js'
import type { ParseReuseValidation } from './parser.js'
// ---------------------------------------------------------------------------
// codeburn serve --stdio: a resident query server for the desktop app.
@ -28,16 +31,77 @@ import type { Command } from 'commander'
// already guards between processes.
// ---------------------------------------------------------------------------
// First-token allowlist of the app's heavy read queries. Deliberately absent:
// every config mutation (currency, model-alias set, budget, price-override,
// proxy-path, plan), export (writes files), share/devices (network + pairing
// state), menubar/web/mcp/guard/sync/act (process management or writes).
// Past this resident-set size the serve loop drops its in-memory memos and
// re-parses on the next request. 3GB leaves generous room for the largest
// observed corpora while bounding a pathological one.
const SERVE_MAX_RSS_BYTES = 3 * 1024 * 1024 * 1024
const SERVE_COMMANDS = new Set(['status', 'overview', 'models', 'sessions', 'compare', 'yield', 'spend', 'optimize', 'audit'])
type OutputMemoEntry = {
createdAt: number
validatedFrom: number
output: string
configFingerprint: string
}
// Kept as a small seam so the ordering contract can be tested without relying
// on filesystem watcher scheduling: an event arriving while a parse is in
// flight must be newer than the memo produced by that parse.
export function createOutputMemoEntry(
parseStartedAt: number,
parseCompletedAt: number,
output: string,
configFingerprint: string,
): OutputMemoEntry {
return { createdAt: parseCompletedAt, validatedFrom: parseStartedAt, output, configFingerprint }
}
type ServeOptionKind = 'flag' | 'value'
// This is intentionally a positive, command-specific option schema rather
// than a shared denylist. If a command later gains a write-capable option it
// remains a normal one-shot CLI action until it is explicitly reviewed here.
// The entries mirror the Commander definitions in main.ts. In particular,
// optimize omits its apply-only surface (--apply, --yes, --dry-run, --only).
const SERVE_OPTIONS: Readonly<Record<string, Readonly<Record<string, ServeOptionKind>>>> = {
status: {
'--format': 'value', '--scope': 'value', '--provider': 'value', '--project': 'value',
'--exclude': 'value', '--period': 'value', '--day': 'value', '--from': 'value',
'--to': 'value', '--days': 'value', '--no-optimize': 'flag', '--no-timeline': 'flag',
'--claude-config-source': 'value',
},
overview: {
'-p': 'value', '--period': 'value', '--from': 'value', '--to': 'value',
'--provider': 'value', '--project': 'value', '--exclude': 'value', '--no-color': 'flag',
},
models: {
'-p': 'value', '--period': 'value', '--from': 'value', '--to': 'value',
'--provider': 'value', '--task': 'value', '--by-task': 'flag', '--by-agent': 'flag',
'--top': 'value', '--min-cost': 'value', '--no-totals': 'flag', '--format': 'value',
},
sessions: {
'-p': 'value', '--period': 'value', '--from': 'value', '--to': 'value',
'--provider': 'value', '--format': 'value', '--by-pr': 'flag', '--no-pager': 'flag',
},
compare: {
'-p': 'value', '--period': 'value', '--provider': 'value', '--format': 'value',
'--model-a': 'value', '--model-b': 'value',
},
yield: {
'-p': 'value', '--period': 'value', '--provider': 'value', '--format': 'value',
},
spend: {
'-p': 'value', '--period': 'value', '--from': 'value', '--to': 'value',
'--provider': 'value', '--format': 'value',
},
optimize: {
'-p': 'value', '--period': 'value', '--from': 'value', '--to': 'value',
'--provider': 'value', '--format': 'value', '--json': 'flag',
},
audit: {
'-p': 'value', '--period': 'value', '--from': 'value', '--to': 'value',
'--provider': 'value', '--format': 'value',
},
}
type ServeRequest = { id: string | number; args: string[] }
@ -50,30 +114,73 @@ function isServeRequest(value: unknown): value is ServeRequest {
function allowed(args: string[]): boolean {
const first = args[0]
if (!first || !SERVE_COMMANDS.has(first)) return false
// No request may smuggle a second positional that turns a read into
// something else; the allowed commands take flags only.
return args.slice(1).every((a, i, all) => a.startsWith('-') || (i > 0 && all[i - 1]!.startsWith('--')))
if (!first) return false
const options = SERVE_OPTIONS[first]
if (!options) return false
// Served commands have no positional arguments. Long options may use the
// standard --name=value form; otherwise every value must immediately
// follow an option declared as value-bearing in that command's schema.
for (let i = 1; i < args.length; i++) {
const token = args[i]!
const separator = token.startsWith('--') ? token.indexOf('=') : -1
const option = separator >= 0 ? token.slice(0, separator) : token
const inlineValue = separator >= 0
const kind = options[option]
if (!kind) return false
if (kind === 'flag') {
if (inlineValue) return false
continue
}
if (inlineValue) continue
const value = args[++i]
if (value === undefined || value.startsWith('-')) return false
}
return true
}
class ExitSignal extends Error {
constructor(public readonly code: number) { super(`exit ${code}`) }
}
/// Run one argv through a fresh program, capturing everything the command
/// writes to stdout. process.exit inside a handler is converted to a thrown
/// ExitSignal so a failing request can never take the server down.
async function runCaptured(buildProgram: () => Command, args: string[]): Promise<{ output: string; code: number }> {
function chunkToString(chunk: unknown, encoding: unknown): string {
if (typeof chunk === 'string') return chunk
if (chunk instanceof Uint8Array) {
return Buffer.from(chunk).toString(typeof encoding === 'string' ? encoding as BufferEncoding : 'utf8')
}
return String(chunk)
}
function finishWrite(rest: unknown[]): void {
const callback = rest[rest.length - 1]
if (typeof callback === 'function') (callback as () => void)()
}
/// Run one argv through a fresh program, capturing command stdout for the
/// final response and forwarding command stderr as progress. process.exit
/// inside a handler is converted to a thrown ExitSignal so a failing request
/// can never take the server down.
async function runCaptured(
buildProgram: () => Command,
args: string[],
onProgress: (progress: string) => void,
): Promise<{ output: string; code: number }> {
const chunks: string[] = []
const originalWrite = process.stdout.write.bind(process.stdout)
const originalErrorWrite = process.stderr.write.bind(process.stderr)
const originalExit = process.exit.bind(process)
process.stdout.write = ((chunk: unknown, ...rest: unknown[]) => {
chunks.push(typeof chunk === 'string' ? chunk : String(chunk))
const last = rest[rest.length - 1]
if (typeof last === 'function') (last as () => void)()
chunks.push(chunkToString(chunk, rest[0]))
finishWrite(rest)
return true
}) as typeof process.stdout.write
process.stderr.write = ((chunk: unknown, ...rest: unknown[]) => {
const progress = chunkToString(chunk, rest[0])
if (progress) onProgress(progress)
finishWrite(rest)
return true
}) as typeof process.stderr.write
process.exit = ((code?: number) => { throw new ExitSignal(code ?? 0) }) as typeof process.exit
try {
@ -86,19 +193,57 @@ async function runCaptured(buildProgram: () => Command, args: string[]): Promise
throw err
} finally {
process.stdout.write = originalWrite
process.stderr.write = originalErrorWrite
process.exit = originalExit
}
}
/// A cheap per-request fingerprint for the configuration that affects query
/// rendering and aggregation. Hashing the small config file tracks effective
/// content rather than filesystem churn: a byte-identical rewrite keeps the
/// memo hot, while any real change invalidates immediately. A missing config
/// is a stable state; every other read failure fails closed (no memo reuse).
async function getConfigFingerprint(): Promise<string | null> {
const path = getConfigFilePath()
try {
const content = await readFile(path)
const digest = createHash('sha256').update(content).digest('hex')
return `${path}\u0000sha256:${digest}`
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') return `${path}\u0000missing`
return null
}
}
/// Watch every provider's probe roots (the same paths codeburn doctor reports
/// as "where discovery looks") so the parse-reuse validator can answer "did
/// any session data change since T?" without a stat sweep. macOS fs.watch
/// rides FSEvents and supports recursive directory watches; a root that fails
/// to watch is simply not covered, which only shortens reuse (the burst
/// window and the hard cap still apply), never staleness.
async function startRootWatchers(): Promise<{ startedAt: number; lastEventAt: () => number; close: () => void }> {
/// rides FSEvents and supports recursive directory watches. A probe failure or
/// a watch failure for an existing root disables event-driven reuse for this
/// generation; a root absent at setup is rechecked by the parser's hard cap.
type RootWatcherState = {
startedAt: number
lastEventAt: () => number
healthy: () => boolean
close: () => void
}
export function classifyRootReuse(
sinceTs: number,
state: { startedAt: number; lastEventAt: number; healthy: boolean },
): ParseReuseValidation {
// A known event is conclusive even if watcher coverage degraded afterward.
// Unknown means only that no dirty evidence exists and cleanliness cannot be
// established for the whole interval.
if (state.lastEventAt >= sinceTs) return 'dirty'
if (!state.healthy || sinceTs < state.startedAt) return 'unknown'
return 'clean'
}
async function startRootWatchers(): Promise<RootWatcherState | null> {
let lastEventAt = 0
const startedAt = Date.now()
let healthy = true
let closed = false
const watchers: FSWatcher[] = []
try {
const { getAllProviders } = await import('./providers/index.js')
@ -108,21 +253,54 @@ async function startRootWatchers(): Promise<{ startedAt: number; lastEventAt: ()
if (!provider.probeRoots) continue
try {
for (const root of await provider.probeRoots()) roots.add(root.path)
} catch { /* a failing probe just goes unwatched */ }
} catch {
// An unknown probe result could hide an existing input root, so no
// global all-roots-quiet claim is safe for this watcher generation.
healthy = false
}
}
for (const root of roots) {
let info: Awaited<ReturnType<typeof stat>>
try {
info = await stat(root)
} catch (err) {
// An absent discovery root contains no sessions at arm time. If it is
// created later there is no child watcher to see that creation, so the
// parser's hard reuse cap remains the eventual revalidation backstop.
// Other stat failures mean an existing input could be uncovered.
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') healthy = false
continue
}
try {
const info = await stat(root)
const watcher = watch(root, { recursive: info.isDirectory() }, () => { lastEventAt = Date.now() })
watcher.on('error', () => { /* dropped watcher = shorter reuse, never staleness */ })
watcher.on('error', () => { healthy = false })
watchers.push(watcher)
} catch { /* nonexistent root: nothing to watch */ }
} catch {
// stat proved this input exists, so failing to arm it invalidates the
// global quiet predicate even when other roots remain watched.
healthy = false
}
}
} catch { /* watcherless serve still works via the burst window */ }
} catch {
// Discovery itself failed. Existing watchers are still closed normally,
// but they cannot validate reuse for an incomplete root set.
healthy = false
}
if (watchers.length === 0) return null
// Coverage begins only after at least one watcher has been successfully
// armed. A parse performed while provider probing/stat/watch setup was in
// flight must not be blessed retroactively as watched.
const startedAt = Date.now()
return {
startedAt,
lastEventAt: () => lastEventAt,
close: () => { for (const w of watchers) w.close() },
healthy: () => healthy && !closed,
close: () => {
if (closed) return
closed = true
for (const w of watchers) w.close()
},
}
}
@ -136,27 +314,49 @@ export async function runStdioServe(buildProgram: () => Command): Promise<void>
// parse stays valid past the burst window (capped in parser.ts, so a missed
// filesystem event self-heals within minutes). This is what turns a warm
// no-change fetch into a no-op instead of a stat sweep.
let rootsQuietSince: ((sinceTs: number) => boolean) | null = null
void startRootWatchers().then(async (w) => {
let rootReuseValidation: ((sinceTs: number) => ParseReuseValidation) | null = null
// Mutable object properties keep cleanup visible to TypeScript even though
// setup assigns them from an asynchronous continuation.
const watcherLifecycle: {
state: RootWatcherState | null
resetValidator: (() => void) | null
} = { state: null, resetValidator: null }
const watcherSetup = startRootWatchers().then(async (w) => {
watcherLifecycle.state = w
if (!w) return
const { setParseReuseValidator } = await import('./parser.js')
// Clean means: the watchers were already armed when the parse happened,
// and no filesystem event has landed since. lastEventAt of 0 is a quiet
// system (clean for anything parsed after arming), not an unknown.
const quiet = (sinceTs: number): boolean => sinceTs >= w.startedAt && w.lastEventAt() < sinceTs
rootsQuietSince = quiet
setParseReuseValidator(quiet)
}).catch(() => { /* watcherless serve still works via the burst window */ })
const validate = (sinceTs: number): ParseReuseValidation => classifyRootReuse(sinceTs, {
startedAt: w.startedAt,
lastEventAt: w.lastEventAt(),
healthy: w.healthy(),
})
rootReuseValidation = validate
setParseReuseValidator(validate)
watcherLifecycle.resetValidator = () => setParseReuseValidator(null)
}).catch(() => {
watcherLifecycle.state?.close()
watcherLifecycle.state = null
})
// Output-level memo: an identical panel query while the roots are quiet
// returns the previous stdout verbatim - the aggregation work is skipped
// too, not just the parse. Invalidation is the same event-or-cap rule the
// parse reuse uses.
// too, not just the parse. Session data uses the same event-or-cap rule as
// parse reuse; config.json is fingerprinted on every request because it can
// change rendering without touching a provider root.
const OUTPUT_MEMO_CAP_MS = 5 * 60 * 1000
const outputMemo = new Map<string, { at: number; output: string }>()
const outputMemo = new Map<string, OutputMemoEntry>()
let observedConfigFingerprint: string | null | undefined
if (process.stdin.isTTY) {
process.stderr.write('codeburn serve speaks JSON over stdio and exists for the desktop app to hold warm.\nNothing interactive happens here; press Ctrl+C to exit.\n')
}
const write = (value: unknown): void => { process.stdout.write(JSON.stringify(value) + '\n') }
// Keep the protocol transport anchored to the real stdout. runCaptured()
// temporarily replaces process.stdout.write to collect command output; a
// dynamic lookup here would swallow progress frames into the final payload.
const protocolWrite = process.stdout.write.bind(process.stdout)
const write = (value: unknown): void => { protocolWrite(JSON.stringify(value) + '\n') }
write({ ready: true, pid: process.pid })
// Strict serialization: each request chains on the previous one.
@ -183,18 +383,39 @@ export async function runStdioServe(buildProgram: () => Command): Promise<void>
write({ id: request.id, ok: false, refused: true, error: 'command not served' })
return
}
const configFingerprint = await getConfigFingerprint()
if (observedConfigFingerprint !== undefined && configFingerprint !== observedConfigFingerprint) {
outputMemo.clear()
}
observedConfigFingerprint = configFingerprint
// A permission or transient read failure must shorten reuse, never make
// an old result look current.
if (configFingerprint === null) outputMemo.clear()
const memoKey = request.args.join('\u0000')
const memoHit = outputMemo.get(memoKey)
if (memoHit && Date.now() - memoHit.at < OUTPUT_MEMO_CAP_MS && rootsQuietSince?.(memoHit.at)) {
if (
configFingerprint !== null
&& memoHit?.configFingerprint === configFingerprint
&& Date.now() - memoHit.createdAt < OUTPUT_MEMO_CAP_MS
&& rootReuseValidation?.(memoHit.validatedFrom) === 'clean'
) {
write({ id: request.id, ok: true, output: memoHit.output })
return
}
try {
const { output, code } = await runCaptured(buildProgram, request.args)
const parseStartedAt = Date.now()
const { output, code } = await runCaptured(
buildProgram,
request.args,
progress => write({ id: request.id, progress }),
)
if (code === 0) {
outputMemo.set(memoKey, { at: Date.now(), output })
if (configFingerprint !== null) {
outputMemo.set(memoKey, createOutputMemoEntry(parseStartedAt, Date.now(), output, configFingerprint))
}
if (outputMemo.size > 32) {
const oldest = [...outputMemo.entries()].sort((a, b) => a[1].at - b[1].at)[0]
const oldest = [...outputMemo.entries()].sort((a, b) => a[1].createdAt - b[1].createdAt)[0]
if (oldest) outputMemo.delete(oldest[0])
}
write({ id: request.id, ok: true, output })
@ -212,16 +433,35 @@ export async function runStdioServe(buildProgram: () => Command): Promise<void>
if (process.memoryUsage().rss > SERVE_MAX_RSS_BYTES) {
const { clearSessionCache } = await import('./parser.js')
const { clearLoadCacheMemo } = await import('./session-cache.js')
const { clearCodexMemCaches } = await import('./codex-cache.js')
const { clearAntigravityCacheStates } = await import('./providers/antigravity.js')
clearSessionCache()
clearLoadCacheMemo()
clearCodexMemCaches()
clearAntigravityCacheStates()
if (typeof globalThis.gc === 'function') globalThis.gc()
}
})
})
// The app owns this process: stdin closing means the app is gone.
await new Promise<void>((resolve) => {
rl.on('close', resolve)
process.stdin.on('end', resolve)
// The app owns this process: stdin closing (or failing) means the app is
// gone. Always release FSEvents handles and the module-global validator;
// otherwise an existing Claude root keeps a naturally closed child alive.
const transportClosed = new Promise<void>((resolve) => {
rl.once('close', resolve)
process.stdin.once('end', resolve)
process.stdin.once('error', resolve)
})
try {
await transportClosed
} finally {
rl.close()
await watcherSetup
rootReuseValidation = null
try {
watcherLifecycle.resetValidator?.()
} finally {
watcherLifecycle.state?.close()
}
}
}

809
src/session-cache.ts
View file

@ -1,9 +1,9 @@
import { readFile, stat, open, rename, unlink, readdir, mkdir } from 'fs/promises'
import { readFile, stat, open, rename, unlink, readdir, mkdir, rm } from 'fs/promises'
import { existsSync, readFileSync, unlinkSync } from 'fs'
import { createHash, randomBytes } from 'crypto'
import { join } from 'path'
import { homedir } from 'os'
import { getCodeburnCacheDir } from './cache-dir.js'
import type { ToolCall } from './types.js'
// ── Types ──────────────────────────────────────────────────────────────
@ -157,19 +157,37 @@ export type SessionCache = {
// INVARIANT: a version bump must extend `PRIOR_CACHE_VERSIONS` (the adoption path
// below) to EVERY prior version that can still exist on disk, or expired-PR
// history from the immediately preceding build silently vanishes.
export const CACHE_VERSION = 7
// v8: on-disk layout only - the single blob became a directory of per-provider
// shards plus a small envelope, so a launch that only touched one provider
// rewrites just that provider's file. The turn shape is unchanged, so a v7 file
// migrates losslessly (migrateSingleFileCache) rather than re-parsing.
// v9: on-disk layout only - a provider's shard split further by the UTC month of
// each cached file, so one appended session rewrites one month instead of the
// provider's whole (100MB-scale) history, and a ranged query loads only the
// months it can possibly report on. Turn shape unchanged, so v8 and v7 both
// migrate losslessly.
export const CACHE_VERSION = 9
// The cache filename is version-suffixed so different binaries (e.g. an old
// launchd menubar on a prior release and a newer desktop app) each own a
// distinct file and can never clobber each other's incompatible schema. Bumping
// CACHE_VERSION automatically mints a fresh filename, superseding the migration
// dance the legacy unversioned file used to need.
const CACHE_FILE = `session-cache.v${CACHE_VERSION}.json`
// The cache directory is version-suffixed for the same reason the file used to
// be: different binaries (an old launchd menubar, a newer desktop app) each own
// a distinct layout and can never clobber each other's incompatible schema.
const CACHE_DIR_NAME = `session-cache.v${CACHE_VERSION}`
// The v8 shard directory, read once by the lossless v8 -> v9 re-layout.
const PRIOR_SHARD_DIR_NAME = 'session-cache.v8'
// Written LAST on every save: it names the shard file of every provider-month, so
// the rename that publishes it is the single point at which a save becomes visible.
const ENVELOPE_FILE = 'envelope.json'
// The pre-versioning filename. Never written or deleted anymore — old binaries
// still own it. On first load we adopt-copy it once (see loadCache) when the
// versioned file is absent and the legacy file's version matches ours.
const LEGACY_CACHE_FILE = 'session-cache.json'
const TEMP_FILE_MAX_AGE_MS = 5 * 60 * 1000
// A shard the published envelope does not name is either superseded garbage or
// a CONCURRENT writer's shard that its envelope has not published yet. The
// second case is why this guard is an order of magnitude above the temp-file
// one: sweeping a live save's shard out from under it would publish an envelope
// naming a file that no longer exists. No save takes an hour.
const UNREFERENCED_SHARD_MAX_AGE_MS = 60 * 60 * 1000
// Env vars that change what a provider discovers or how its sessions parse.
// computeEnvFingerprint hashes exactly these to decide when a provider's cache
@ -271,7 +289,12 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
hermes: 'reasoning-output-accounting-v1-est-cost',
'lingtai-tui': 'token-ledger-registry-activity-v3',
'ibm-bob': 'worktree-project-grouping-v1',
kiro: 'ide-parsing-v1-est-cost',
// project-path-v1: the parser now records the session's full working
// directory as projectPath (CLI meta.cwd, v2 workspacePaths[0], workspace
// sessions' workspaceDirectory), which sync attribution needs to resolve
// the git repo. Cached entries from before the bump lack projectPath and
// would serve attribution-blind sessions forever without a re-parse.
kiro: 'ide-parsing-v1-est-cost-project-path-v1',
opencode: 'session-model-v1',
quickdesk: 'emf-sqlite-v2-est-cost',
kimicode: 'wire-usage-v1-est-cost',
@ -281,23 +304,142 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
antigravity: 'worktree-project-grouping-v5',
}
// ── Cache Dir ──────────────────────────────────────────────────────────
function getCacheDir(): string {
return process.env['CODEBURN_CACHE_DIR'] ?? join(homedir(), '.cache', 'codeburn')
}
function getCachePath(): string {
return join(getCacheDir(), CACHE_FILE)
}
function getLegacyCachePath(): string {
return join(getCacheDir(), LEGACY_CACHE_FILE)
return join(getCodeburnCacheDir(), LEGACY_CACHE_FILE)
}
/** Absolute path of the active (version-suffixed) session cache file. */
export function sessionCachePath(): string {
return getCachePath()
/** Absolute path of the active (version-suffixed) session cache directory. */
export function sessionCacheDir(): string {
return join(getCodeburnCacheDir(), CACHE_DIR_NAME)
}
// `until` is the UTC month of the newest turn any file in the shard holds. The
// shard's own key is the month of the OLDEST (a file is bucketed by its first
// turn), so the pair bounds every turn the shard can contribute and a ranged
// load can skip the shard outright when the two do not overlap the query.
type ShardRef = { name: string; until: string }
type EnvelopeProvider = {
envFingerprint: string
durable?: boolean
/** month (`YYYY-MM`, or `0000-00` for turn-less files) -> shard */
shards: Record<string, ShardRef>
}
type CacheEnvelope = {
version: number
complete?: boolean
nonce: string
providers: Record<string, EnvelopeProvider>
}
// Files with no turns (failure markers, empty sessions) have no month to bucket
// by. They live in one always-loaded bucket, which is also what makes the only
// possible re-bucketing safe: a file leaves this bucket the first time it gains
// a turn, and the bucket it leaves is guaranteed to be in memory.
const UNDATED_BUCKET = '0000-00'
// Sentinel inside `dirtyBuckets`: every bucket of the provider is dirty.
const ALL_BUCKETS = '*'
function monthKey(timestamp: string | undefined): string | null {
if (!timestamp) return null
const ms = Date.parse(timestamp)
if (Number.isNaN(ms)) return null
const d = new Date(ms)
return `${d.getUTCFullYear()}-${String(d.getUTCMonth() + 1).padStart(2, '0')}`
}
/** The UTC month span a cached file covers: `bucket` is its OLDEST turn's month
* (the shard it lives in), `until` its NEWEST (how far forward the shard can
* contribute). Both scan every turn rather than reading turns[0]/turns[-1]:
* several providers emit turns out of chronological order (cursor composers by
* ROWID, goose/crush/copilot by a DESC ordering), and a `until < bucket` span
* is empty, which makes the shard unreachable at EVERY scope. */
export function cacheFileSpan(file: CachedFile): { bucket: string; until: string } {
let bucket: string | null = null
let until: string | null = null
for (const turn of file.turns) {
const month = monthKey(turn.timestamp)
if (month === null) continue
if (bucket === null || month < bucket) bucket = month
if (until === null || month > until) until = month
}
return bucket === null ? { bucket: UNDATED_BUCKET, until: UNDATED_BUCKET } : { bucket, until: until! }
}
/** The shard bucket a cached file belongs to. Derived from the file's own turns,
* so an APPEND never moves it: appending can only extend `until`. */
export function cacheBucketMonth(file: CachedFile): string {
return cacheFileSpan(file).bucket
}
// Save bookkeeping, held beside the cache rather than on it so it never lands in
// a shard's JSON or in a caller's deep-equality.
type CacheState = {
dirty: boolean
/** provider -> dirty months (or `ALL_BUCKETS`). */
dirtyBuckets: Map<string, Set<string>>
/** provider -> the shard refs the last load/save published. */
shards: Map<string, Record<string, ShardRef>>
/** provider -> months held in memory; `null` when the whole provider loaded. */
loaded: Map<string, Set<string> | null>
/** provider -> the envFingerprint the published envelope recorded. */
fingerprints: Map<string, string>
/** `provider\0path` -> the bucket the entry was loaded/saved under, so a
* delete or a re-bucketing can dirty the bucket it is leaving. */
bucketOf: Map<string, string>
/** The load scope this cache was read under, for the cross-request memo. */
scope: string
}
const cacheStates = new WeakMap<SessionCache, CacheState>()
function stateOf(cache: SessionCache): CacheState {
let state = cacheStates.get(cache)
if (!state) {
state = {
dirty: false,
dirtyBuckets: new Map(),
shards: new Map(),
loaded: new Map(),
fingerprints: new Map(),
bucketOf: new Map(),
scope: 'all',
}
cacheStates.set(cache, state)
}
return state
}
function markBucketDirty(state: CacheState, provider: string, bucket: string): void {
state.dirty = true
let buckets = state.dirtyBuckets.get(provider)
if (!buckets) { buckets = new Set(); state.dirtyBuckets.set(provider, buckets) }
buckets.add(bucket)
}
function isBucketDirty(state: CacheState, provider: string, bucket: string): boolean {
const buckets = state.dirtyBuckets.get(provider)
return buckets !== undefined && (buckets.has(ALL_BUCKETS) || buckets.has(bucket))
}
/** Record that `provider`'s section changed, so the next save rewrites the
* affected shards. Pass `filePath` whenever the change is scoped to one cached
* file — both the bucket it was last saved in and the bucket it is in now are
* marked, so a delete, a rewrite and a re-bucketing are all covered whichever
* order the caller mutates and marks in. Omitting it dirties every bucket. */
export function markCacheDirty(cache: SessionCache, provider: string, filePath?: string): void {
const state = stateOf(cache)
if (filePath === undefined) { markBucketDirty(state, provider, ALL_BUCKETS); return }
const prior = state.bucketOf.get(`${provider}\0${filePath}`)
if (prior !== undefined) markBucketDirty(state, provider, prior)
const file = cache.providers[provider]?.files[filePath]
if (file) markBucketDirty(state, provider, cacheBucketMonth(file))
// A path with neither a prior bucket nor a live entry (deleted before this
// process ever saw it) still has to move `dirty`, or the save is skipped.
state.dirty = true
}
/** True when any provider section changed since the last save. */
export function isCacheDirty(cache: SessionCache): boolean {
return stateOf(cache).dirty
}
// ── Env Fingerprint ────────────────────────────────────────────────────
@ -442,6 +584,12 @@ function validateCachedFile(f: unknown): f is CachedFile {
&& (o['turns'] as unknown[]).every(validateTurn)
}
// A shard's payload: the provider's `files` map, restricted to one month.
function validateFiles(v: unknown): v is Record<string, CachedFile> {
if (!v || typeof v !== 'object' || Array.isArray(v)) return false
return Object.values(v as Record<string, unknown>).every(validateCachedFile)
}
function validateProviderSection(s: unknown): s is ProviderSection {
if (!s || typeof s !== 'object') return false
const o = s as Record<string, unknown>
@ -450,10 +598,11 @@ function validateProviderSection(s: unknown): s is ProviderSection {
return Object.values(o['files'] as Record<string, unknown>).every(validateCachedFile)
}
function validateCache(raw: unknown): raw is SessionCache {
// Full validation of a single-file (pre-v8) cache blob at `version`.
function validateCache(raw: unknown, version: number): raw is SessionCache {
if (!raw || typeof raw !== 'object') return false
const o = raw as Record<string, unknown>
if (o['version'] !== CACHE_VERSION) return false
if (o['version'] !== version) return false
if (!o['providers'] || typeof o['providers'] !== 'object' || Array.isArray(o['providers'])) return false
return Object.values(o['providers'] as Record<string, unknown>).every(validateProviderSection)
}
@ -466,7 +615,7 @@ function validateCache(raw: unknown): raw is SessionCache {
// CACHE_VERSION bump MUST extend this list to every prior version that can still
// exist on disk, or that history silently vanishes. (v5 was missed on the 5->6
// bump; v6 on the 6->7 bump; both are listed here.)
const PRIOR_CACHE_VERSIONS = [6, 5] as const
const PRIOR_CACHE_VERSIONS = [7, 6, 5] as const
function priorCacheFile(version: number): string {
return `session-cache.v${version}.json`
@ -492,7 +641,7 @@ function isCacheEnvelope(raw: unknown, version: number): raw is { version: numbe
// sources. The daily cache (durable cost history) is not touched.
async function adoptPriorCache(version: number): Promise<SessionCache | null> {
try {
const raw = await readFile(join(getCacheDir(), priorCacheFile(version)), 'utf-8')
const raw = await readFile(join(getCodeburnCacheDir(), priorCacheFile(version)), 'utf-8')
const parsed = JSON.parse(raw)
if (!isCacheEnvelope(parsed, version)) return null
const migrated: SessionCache = { version: CACHE_VERSION, providers: {}, complete: false }
@ -542,54 +691,239 @@ async function adoptNewestPriorCache(): Promise<SessionCache | null> {
return merged
}
// In-process memo of the parsed cache, keyed by the file identity that last
// produced it. On a 100MB+ corpus the JSON.parse of the session cache is
// seconds of work per load; a resident process (codeburn serve) pays it once
// and revalidates with a stat() per request. A rewrite by ANOTHER process
// moves mtime/size and forces a reload, so cross-process freshness is
// preserved; saveCache updates the memo write-through so the object handed
// out stays the canonical one after a refresh.
let cacheMemo: { path: string; mtimeMs: number; size: number; cache: SessionCache } | null = null
// In-process memo of the parsed cache, keyed by the envelope nonce that last
// produced it. On a 100MB+ corpus the JSON.parse of the shards is seconds of
// work per load; a resident process (codeburn serve) pays it once and
// revalidates by re-reading the (tiny) envelope per request. A save by ANOTHER
// process mints a new nonce and forces a reload, so cross-process freshness is
// preserved; saveCache updates the memo write-through so the object handed out
// stays the canonical one after a refresh.
let cacheMemo: { dir: string; nonce: string; scope: string; cache: SessionCache } | null = null
export function clearLoadCacheMemo(): void {
cacheMemo = null
}
export async function loadCache(): Promise<SessionCache> {
const path = getCachePath()
/** Months (UTC `YYYY-MM`, inclusive) a query can possibly report on. The load
* widens this by one month BELOW `fromMonth` and none above (see
* shardInScope): every cross-range carry in the report reads BACKWARDS from the
* first in-range turn, never forwards, so there is nothing above the range to
* reach for. */
export type CacheLoadScope = { fromMonth: string; toMonth: string }
export function monthScopeForRange(start: Date, end: Date): CacheLoadScope {
return { fromMonth: monthKey(start.toISOString())!, toMonth: monthKey(end.toISOString())! }
}
function previousMonth(month: string): string {
const [y, m] = month.split('-').map(Number) as [number, number]
return m === 1 ? `${y - 1}-12` : `${y}-${String(m - 1).padStart(2, '0')}`
}
// A shard is in scope when its [bucket .. until] span overlaps the query. One
// extra month of slack BELOW the range (and none above — every carry reads
// backwards) covers the cross-range carries that read turns from before the
// window: the pre-range PR set / git branch a session carries into its first
// in-range turn (both resolved from the same file, so they only need the file
// loaded at all), and the out-of-range subagent-spawn ANCHOR whose in-range
// child folds into it. LIMITATION: an anchor whose last turn is two or more
// months before its child's is not loaded, so that child attributes without the
// parent's PR set. One month of slack is the deliberate ceiling; widening it
// gives back the read savings the scope exists for.
// The undated bucket has no span and is always loaded.
function shardInScope(bucket: string, until: string, scope: CacheLoadScope): boolean {
if (bucket === UNDATED_BUCKET) return true
return bucket <= scope.toMonth && until >= previousMonth(scope.fromMonth)
}
function isShardRef(v: unknown): v is ShardRef {
if (!v || typeof v !== 'object') return false
const o = v as Record<string, unknown>
return typeof o['name'] === 'string' && typeof o['until'] === 'string'
}
function isEnvelope(raw: unknown): raw is CacheEnvelope {
if (!raw || typeof raw !== 'object') return false
const o = raw as Record<string, unknown>
if (o['version'] !== CACHE_VERSION || typeof o['nonce'] !== 'string') return false
const providers = o['providers']
if (!providers || typeof providers !== 'object' || Array.isArray(providers)) return false
return Object.values(providers as Record<string, unknown>).every(p => {
if (!p || typeof p !== 'object') return false
const e = p as Record<string, unknown>
if (typeof e['envFingerprint'] !== 'string') return false
if (!e['shards'] || typeof e['shards'] !== 'object' || Array.isArray(e['shards'])) return false
return Object.values(e['shards'] as Record<string, unknown>).every(isShardRef)
})
}
async function readEnvelope(dir: string): Promise<CacheEnvelope | null> {
try {
const info = await stat(path)
if (cacheMemo && cacheMemo.path === path && cacheMemo.mtimeMs === info.mtimeMs && cacheMemo.size === info.size) {
return cacheMemo.cache
}
const raw = await readFile(path, 'utf-8')
const parsed = JSON.parse(raw)
if (!validateCache(parsed)) return afterMissingVersionedCache()
cacheMemo = { path, mtimeMs: info.mtimeMs, size: info.size, cache: parsed }
return parsed
const parsed = JSON.parse(await readFile(join(dir, ENVELOPE_FILE), 'utf-8'))
return isEnvelope(parsed) ? parsed : null
} catch {
return afterMissingVersionedCache()
return null
}
}
// The current versioned file is absent/unreadable. Prefer adopting the newest
// prior versioned file's expired-source PR orphans (v6 before v5); failing that,
// fall back to the legacy unversioned file. Either way the versioned file is
// minted on the next save.
async function afterMissingVersionedCache(): Promise<SessionCache> {
// A shard that is missing or malformed costs exactly the provider-months it
// held, not the provider and never the whole cache: those files re-parse while
// every other month keeps serving.
async function loadShard(path: string): Promise<Record<string, CachedFile> | null> {
try {
const parsed = JSON.parse(await readFile(path, 'utf-8'))
return validateFiles(parsed) ? parsed : null
} catch {
return null
}
}
/**
* Read the cache. With a `scope`, only the shards whose months can contribute a
* turn to that range are read — everything else stays on disk and is carried
* across the next save untouched (see saveCache). Durable providers and any
* provider whose recorded fingerprint no longer matches are always read in
* full: the first because its cache is the only surviving record of pruned
* usage, the second because a fingerprint change discards the whole section and
* must see every entry it is discarding.
*/
export async function loadCache(scope?: CacheLoadScope): Promise<SessionCache> {
const dir = sessionCacheDir()
const envelope = await readEnvelope(dir)
if (!envelope) return afterMissingShardCache()
const scopeKey = scope ? `${scope.fromMonth}..${scope.toMonth}` : 'all'
if (cacheMemo && cacheMemo.dir === dir && cacheMemo.nonce === envelope.nonce
&& (cacheMemo.scope === 'all' || cacheMemo.scope === scopeKey)) return cacheMemo.cache
const cache: SessionCache = { version: CACHE_VERSION, providers: {}, complete: envelope.complete === true }
const state = stateOf(cache)
const reads: Promise<void>[] = []
for (const [provider, meta] of Object.entries(envelope.providers)) {
const section: ProviderSection = {
envFingerprint: meta.envFingerprint,
files: {},
...(meta.durable ? { durable: true } : {}),
}
// Recorded even when every shard is skipped or unreadable: the section is
// what tells the next save which provider these carried-forward shard refs
// belong to, and what stops the reconcile from re-parsing under a
// fingerprint the envelope already agrees with.
cache.providers[provider] = section
const full = !scope || meta.durable === true || meta.envFingerprint !== computeEnvFingerprint(provider)
const loaded: Set<string> | null = full ? null : new Set()
// Shards are read concurrently but merged in envelope order, so the result
// never depends on which read finished first. A path that somehow ended up
// in two shards resolves to the FRESHEST fingerprint and dirties both
// buckets, so the next save prunes the loser instead of letting it linger.
const pending: { bucket: string; files: Promise<Record<string, CachedFile> | null> }[] = []
for (const [bucket, ref] of Object.entries(meta.shards)) {
if (loaded && !shardInScope(bucket, ref.until, scope!)) continue
loaded?.add(bucket)
pending.push({ bucket, files: loadShard(join(dir, ref.name)) })
}
reads.push((async () => {
for (const { bucket, files: read } of pending) {
const files = await read
// Unreadable: the bucket counts as loaded-and-empty and is marked
// dirty, so the re-parsed files replace it instead of the stale shard
// being carried forward forever.
if (!files) { markBucketDirty(state, provider, bucket); continue }
for (const [path, file] of Object.entries(files)) {
const key = `${provider}\0${path}`
const seenIn = state.bucketOf.get(key)
if (seenIn !== undefined) {
markBucketDirty(state, provider, seenIn)
markBucketDirty(state, provider, bucket)
if (section.files[path]!.fingerprint.mtimeMs >= file.fingerprint.mtimeMs) continue
}
state.bucketOf.set(key, bucket)
section.files[path] = file
}
}
})())
state.loaded.set(provider, loaded)
state.shards.set(provider, meta.shards)
state.fingerprints.set(provider, meta.envFingerprint)
}
await Promise.all(reads)
state.scope = scopeKey
cacheMemo = { dir, nonce: envelope.nonce, scope: scopeKey, cache }
return cache
}
// The shard directory is absent/unreadable. Prefer a LOSSLESS re-layout of the
// newest prior layout that is present (v8 provider shards, then the v7 single
// file — both hold the current turn shape, so nothing re-parses); failing that,
// adopt the prior versions' expired-source PR orphans, then the legacy
// unversioned file. Either way the shard directory is minted on the next save.
async function afterMissingShardCache(): Promise<SessionCache> {
const relaid = await migrateProviderShardCache() ?? await migrateSingleFileCache()
if (relaid) return relaid
const prior = await adoptNewestPriorCache()
if (prior) return prior
// validateCache requires version === CACHE_VERSION, so a different-version
// legacy file is ignored (left intact). We copy it into the versioned file once
// via saveCache; the legacy file is never modified.
// validateCache requires the version to match, so a different-version legacy
// file is ignored (left intact). We copy it into the shard layout once via
// saveCache; the legacy file is never modified.
return adoptLegacyCache()
}
// One-time, lossless re-layout of the v8 per-provider shard directory: v9
// changed the on-disk LAYOUT only, so every entry moves across verbatim (just
// re-bucketed by month in memory) and nothing re-parses. The v8 directory is
// removed only once the v9 save has published.
async function migrateProviderShardCache(): Promise<SessionCache | null> {
const dir = join(getCodeburnCacheDir(), PRIOR_SHARD_DIR_NAME)
let envelope: { complete?: boolean; shards: Record<string, string> }
try {
const parsed = JSON.parse(await readFile(join(dir, ENVELOPE_FILE), 'utf-8')) as Record<string, unknown>
if (parsed['version'] !== 8 || !parsed['shards'] || typeof parsed['shards'] !== 'object') return null
envelope = parsed as { complete?: boolean; shards: Record<string, string> }
} catch {
return null
}
const cache: SessionCache = { version: CACHE_VERSION, providers: {}, complete: envelope.complete === true }
await Promise.all(Object.entries(envelope.shards).map(async ([provider, name]) => {
try {
const parsed = JSON.parse(await readFile(join(dir, name), 'utf-8'))
if (validateProviderSection(parsed)) cache.providers[provider] = parsed
} catch { /* one unreadable v8 shard costs that provider, as it already did */ }
}))
return publishRelaidCache(cache, () => rm(dir, { recursive: true, force: true }))
}
// One-time, lossless re-layout of the v7 single-file cache. v7 never wrote a
// shard directory, so it is migrated straight to v9 without minting a v8 in
// between.
async function migrateSingleFileCache(): Promise<SessionCache | null> {
const v7Path = join(getCodeburnCacheDir(), priorCacheFile(7))
let parsed: unknown
try {
parsed = JSON.parse(await readFile(v7Path, 'utf-8'))
} catch {
return null
}
if (!validateCache(parsed, 7)) return null
return publishRelaidCache(
{ version: CACHE_VERSION, providers: parsed.providers, complete: parsed.complete === true },
() => unlink(v7Path),
)
}
// Every section is marked dirty so the save writes each month's shard; the old
// layout is retired only once that save has published.
async function publishRelaidCache(cache: SessionCache, retire: () => Promise<unknown>): Promise<SessionCache> {
for (const provider of Object.keys(cache.providers)) markCacheDirty(cache, provider)
const published = await saveCache(cache).catch(() => false)
if (published) await retryCacheFileMutation(async () => { await retire() })
return cache
}
async function adoptLegacyCache(): Promise<SessionCache> {
try {
const raw = await readFile(getLegacyCachePath(), 'utf-8')
const parsed = JSON.parse(raw)
if (!validateCache(parsed)) return emptyCache()
if (!validateCache(parsed, CACHE_VERSION)) return emptyCache()
for (const provider of Object.keys(parsed.providers)) markCacheDirty(parsed, provider)
await saveCache(parsed).catch(() => {})
return parsed
} catch {
@ -597,15 +931,19 @@ async function adoptLegacyCache(): Promise<SessionCache> {
}
}
export async function saveCache(cache: SessionCache, verifyStillOwner?: () => Promise<boolean>): Promise<boolean> {
const dir = getCacheDir()
if (!existsSync(dir)) await mkdir(dir, { recursive: true })
// Shard filenames carry a fresh nonce on every write, so a save never overwrites
// the file the currently-published envelope points at: readers keep seeing a
// consistent set until the envelope rename publishes the new one, and a writer
// that loses the ownership fence leaves the canonical shards untouched.
function shardFileName(provider: string, bucket: string): string {
return `${provider.replace(/[^A-Za-z0-9_-]/g, '_')}.${bucket}.${randomBytes(8).toString('hex')}.json`
}
const finalPath = getCachePath()
// The temp name carries a nonce: two processes writing the SAME final path
// (the envelope, every save) would otherwise share one temp file and interleave
// their writes into a torn or foreign payload.
async function writeFileAtomic(finalPath: string, payload: string): Promise<void> {
const tempPath = `${finalPath}.${randomBytes(8).toString('hex')}.tmp`
delete (cache as { _dirty?: boolean })._dirty
const payload = JSON.stringify(cache)
const handle = await open(tempPath, 'w', 0o600)
try {
await handle.writeFile(payload, { encoding: 'utf-8' })
@ -613,44 +951,275 @@ export async function saveCache(cache: SessionCache, verifyStillOwner?: () => Pr
} finally {
await handle.close()
}
try {
// The warm refresh transaction passes an ownership fence. It must be the
// final operation before publication so a displaced writer cannot replace
// the canonical cache with its stale snapshot.
if (verifyStillOwner && !await verifyStillOwner()) {
await retryCacheFileMutation(() => unlink(tempPath))
return false
}
let renamed = false
for (let attempt = 0; attempt < 3; attempt++) {
try {
await rename(tempPath, finalPath)
renamed = true
break
return
} catch (err) {
const code = (err as NodeJS.ErrnoException).code
if ((code !== 'EPERM' && code !== 'EBUSY') || attempt === 2) throw err
await new Promise(resolve => { setTimeout(resolve, 10 * (attempt + 1)) })
}
}
if (!renamed) throw new Error('session cache rename failed')
// Write-through: the object just published IS the freshest state; capture
// the post-rename file identity so the next loadCache in this process
// reuses it instead of re-parsing what it just wrote.
try {
const info = await stat(finalPath)
cacheMemo = { path: finalPath, mtimeMs: info.mtimeMs, size: info.size, cache }
} catch {
cacheMemo = null
}
return true
} catch (err) {
await retryCacheFileMutation(() => unlink(tempPath))
throw err
}
}
function bucketFiles(section: ProviderSection): { groups: Map<string, Record<string, CachedFile>>; until: Map<string, string> } {
const groups = new Map<string, Record<string, CachedFile>>()
const until = new Map<string, string>()
for (const [path, file] of Object.entries(section.files)) {
const span = cacheFileSpan(file)
let group = groups.get(span.bucket)
if (!group) { group = {}; groups.set(span.bucket, group) }
group[path] = file
const seen = until.get(span.bucket)
if (seen === undefined || span.until > seen) until.set(span.bucket, span.until)
}
return { groups, until }
}
function untilMonth(files: Record<string, CachedFile>): string {
let until = UNDATED_BUCKET
for (const file of Object.values(files)) {
const month = cacheFileSpan(file).until
if (month > until) until = month
}
return until
}
// What a save has decided about one provider, carried across the ownership
// fence so every shard READ that a save needs happens as late as possible (see
// the phase-two comment in saveCache).
type ProviderPlan = {
section: ProviderSection
groups: Map<string, Record<string, CachedFile>>
loaded: Set<string> | null
priorRefs: Record<string, ShardRef>
reset: boolean
/** Paths that may ALSO still sit in a shard this run never loaded. */
moved: Set<string>
/** Buckets whose payload has to be merged with the published shard first. */
deferred: string[]
/** bucket -> the shard name the merge was built from, for the retry below. */
mergedFrom: Map<string, string | undefined>
refs: Record<string, ShardRef>
}
export async function saveCache(cache: SessionCache, verifyStillOwner?: () => Promise<boolean>): Promise<boolean> {
const dir = sessionCacheDir()
if (!existsSync(dir)) await mkdir(dir, { recursive: true, mode: 0o700 })
const state = stateOf(cache)
const written = new Set<string>()
const plans = new Map<string, ProviderPlan>()
const writeShard = async (provider: string, bucket: string, files: Record<string, CachedFile>): Promise<ShardRef> => {
const name = shardFileName(provider, bucket)
await writeFileAtomic(join(dir, name), JSON.stringify(files))
written.add(name)
return { name, until: untilMonth(files) }
}
// Overlay this run's entries for `bucket` onto the published shard `from`,
// minus any path that has since moved to another month.
const mergeShard = async (provider: string, plan: ProviderPlan, bucket: string, from: string | undefined): Promise<ShardRef> => {
const files = plan.groups.get(bucket)!
const onDisk = from ? await loadShard(join(dir, from)) : null
if (!onDisk) return writeShard(provider, bucket, files)
for (const path of plan.moved) delete onDisk[path]
return writeShard(provider, bucket, { ...onDisk, ...files })
}
try {
// ── Phase one: everything that can be written from memory alone ──────
for (const [provider, section] of Object.entries(cache.providers)) {
const priorRefs = state.shards.get(provider) ?? {}
const loaded = state.loaded.get(provider) ?? null
// A fingerprint change discards the section outright (see
// getOrCreateProviderSection), so the months it did not load must be
// dropped rather than carried — they hold entries under the old
// fingerprint. loadCache never scopes such a provider, so `loaded` is
// null here in practice; the guard is what makes that safe to rely on.
const priorFingerprint = state.fingerprints.get(provider)
const reset = priorFingerprint !== undefined && priorFingerprint !== section.envFingerprint
const { groups } = bucketFiles(section)
const plan: ProviderPlan = { section, groups, loaded, priorRefs, reset, moved: new Set(), deferred: [], mergedFrom: new Map(), refs: {} }
plans.set(provider, plan)
// An entry whose bucket this run never loaded may ALSO still exist, under
// an older month, in a shard we are about to carry across verbatim — a
// re-parse that shifted the file's oldest turn, or (the common #441 path)
// a parse failure that left a turn-less marker with no month at all. Left
// alone, the path would live in two shards at once and a later load could
// resolve to the stale copy. Both cases are rare, so the prune they
// trigger below reads shards it otherwise would not.
if (loaded) {
for (const [path, file] of Object.entries(section.files)) {
if (state.bucketOf.has(`${provider}\0${path}`)) continue
const bucket = cacheFileSpan(file).bucket
if (!loaded.has(bucket) || bucket === UNDATED_BUCKET) plan.moved.add(path)
}
}
for (const [bucket, files] of groups) {
const prior = priorRefs[bucket]
// `priorRefs` is this process's snapshot from its last load or save.
// ANOTHER process may have republished that shard since, unlinking the
// file we are about to name — so reuse is conditional on the file still
// being there, and a vanished one is rewritten from memory.
if (prior && !isBucketDirty(state, provider, bucket) && existsSync(join(dir, prior.name))) {
plan.refs[bucket] = prior
continue
}
// Dirty but never loaded: memory holds only the entries this run wrote
// into the bucket, so the published shard's other entries have to be
// merged back in or the save would drop them. Deferred to phase two so
// the read happens against the CURRENT shard, not a stale name.
if (loaded && !loaded.has(bucket) && prior) { plan.deferred.push(bucket); continue }
plan.refs[bucket] = await writeShard(provider, bucket, files)
}
}
// The warm refresh transaction passes an ownership fence. It must be the
// final operation before publication so a displaced writer cannot replace
// the canonical cache with its stale snapshot. Shards written above are
// unreferenced until the envelope names them, so a lost fence publishes
// nothing.
if (verifyStillOwner && !await verifyStillOwner()) {
for (const name of written) await retryCacheFileMutation(() => unlink(join(dir, name)))
return false
}
// ── Phase two: everything that has to read the published shards ──────
// Re-read the envelope first. Between our load and now, another process may
// have republished any month we are carrying or merging into; adopting its
// CURRENT name is what keeps a carried orphan (an expired transcript's PR
// spend, unrecoverable by any re-parse) from being dropped just because the
// name we remembered was retired. It also shrinks the read-modify-write
// window for a merge down to the publish itself. That window is not zero:
// two processes merging into the same unloaded month can still interleave,
// and the loser's entries are re-derived on the next parse rather than lost
// for good — a full lock here would cost every save the contention.
const live = await readEnvelope(dir)
for (const [provider, plan] of plans) {
const liveShards = live?.providers[provider]?.shards ?? {}
const currentName = (bucket: string): string | undefined => {
const name = liveShards[bucket]?.name ?? plan.priorRefs[bucket]?.name
return name && existsSync(join(dir, name)) ? name : undefined
}
for (const bucket of plan.deferred) {
plan.refs[bucket] = await mergeShard(provider, plan, bucket, currentName(bucket))
plan.mergedFrom.set(bucket, currentName(bucket))
}
// Months this run never loaded keep their published shard. This is the
// invariant that makes a scoped load safe to save from. A month another
// process published while we held a partial view is adopted for the same
// reason: dropping it would delete history we never even saw.
if (!plan.loaded || plan.reset) continue
const carried = new Set([...Object.keys(plan.priorRefs), ...Object.keys(liveShards)])
for (const bucket of carried) {
if (plan.refs[bucket] || plan.groups.has(bucket) || plan.loaded.has(bucket)) continue
const name = currentName(bucket)
if (!name) continue
const ref = { name, until: (liveShards[bucket] ?? plan.priorRefs[bucket])!.until }
if (plan.moved.size === 0) { plan.refs[bucket] = ref; continue }
// A path that moved into another month must not survive here too.
const onDisk = await loadShard(join(dir, name))
if (!onDisk || !Object.keys(onDisk).some(p => plan.moved.has(p))) { plan.refs[bucket] = ref; continue }
for (const path of plan.moved) delete onDisk[path]
if (Object.keys(onDisk).length > 0) plan.refs[bucket] = await writeShard(provider, bucket, onDisk)
}
}
// One optimistic retry: if another process republished a month we merged
// into while we were reading it, our shard was built on a superseded
// pre-image and would drop that process's entries. Redoing the merge from
// the current shard narrows the read-modify-write window from a shard read
// down to the envelope publish below. It does not close it — a save that
// loses the remaining race has its entries re-derived by the next parse
// (the reconcile sees no cache entry and re-reads the file), never silently
// dropped for good. A lock here would tax every save for a rare interleave.
const settled = await readEnvelope(dir)
for (const [provider, plan] of plans) {
for (const [bucket, mergedFrom] of plan.mergedFrom) {
const now = settled?.providers[provider]?.shards[bucket]?.name
if (!now || now === mergedFrom || !existsSync(join(dir, now))) continue
plan.refs[bucket] = await mergeShard(provider, plan, bucket, now)
}
}
// Last look before publishing: a concurrent save may have unlinked a shard
// in the moment since. An envelope must never name a file that is already
// gone — that reads back as a corrupt month and drops its history.
const providers: Record<string, EnvelopeProvider> = {}
for (const [provider, plan] of plans) {
const shards: Record<string, ShardRef> = {}
for (const [bucket, ref] of Object.entries(plan.refs)) {
if (written.has(ref.name) || existsSync(join(dir, ref.name))) { shards[bucket] = ref; continue }
const files = plan.groups.get(bucket)
// A carried month whose file vanished and whose content was never in
// memory cannot be rewritten; dropping the reference is the only honest
// option, and the sweep retires the name.
if (files) shards[bucket] = await writeShard(provider, bucket, files)
}
providers[provider] = {
envFingerprint: plan.section.envFingerprint,
...(plan.section.durable ? { durable: true } : {}),
shards,
}
}
const envelope: CacheEnvelope = {
version: CACHE_VERSION,
complete: cache.complete === true,
nonce: randomBytes(8).toString('hex'),
providers,
}
await writeFileAtomic(join(dir, ENVELOPE_FILE), JSON.stringify(envelope))
// Shards the new envelope no longer references are garbage; a reader that
// already opened one keeps reading it, and any failure here is swept later
// by cleanupOrphanedTempFiles.
const retired: string[] = []
for (const [provider, priorRefs] of state.shards) {
const kept = providers[provider]?.shards ?? {}
for (const [bucket, ref] of Object.entries(priorRefs)) {
if (kept[bucket]?.name !== ref.name) retired.push(ref.name)
}
}
state.dirty = false
state.dirtyBuckets.clear()
state.shards.clear()
state.fingerprints.clear()
state.bucketOf.clear()
// `loaded` deliberately survives: a merged-and-rewritten shard is complete
// on disk but still partial in memory, so the next save has to merge again.
for (const [provider, meta] of Object.entries(providers)) {
state.shards.set(provider, meta.shards)
state.fingerprints.set(provider, meta.envFingerprint)
for (const [path, file] of Object.entries(cache.providers[provider]!.files)) {
state.bucketOf.set(`${provider}\0${path}`, cacheFileSpan(file).bucket)
}
}
// Write-through: the object just published IS the freshest state, so the
// next loadCache in this process reuses it instead of re-parsing. Its scope
// is whatever was loaded, not `all` — a save never widens what is in memory.
cacheMemo = { dir, nonce: envelope.nonce, scope: state.scope, cache }
for (const name of retired) await retryCacheFileMutation(() => unlink(join(dir, name)))
return true
} catch (err) {
for (const name of written) await retryCacheFileMutation(() => unlink(join(dir, name)))
throw err
}
}
async function retryCacheFileMutation(operation: () => Promise<void>): Promise<boolean> {
for (let attempt = 0; attempt < 3; attempt++) {
try {
@ -801,26 +1370,58 @@ export function mergeCallByDedupKey(
// ── Temp Cleanup ───────────────────────────────────────────────────────
async function unlinkIfOlderThan(path: string, maxAgeMs: number, now: number): Promise<void> {
try {
const s = await stat(path)
if (now - s.mtimeMs > maxAgeMs) await unlink(path)
} catch {}
}
// Sweeps our own shard directory: interrupted temp writes, plus shards the
// published envelope no longer references. Also retires the single-file layout's
// leftover temps in the parent directory, which nothing writes anymore.
export async function cleanupOrphanedTempFiles(): Promise<void> {
const dir = getCacheDir()
const now = Date.now()
const parent = getCodeburnCacheDir()
// `session-cache.v<n>.json.<nonce>.tmp` from a pre-v8 binary interrupted
// mid-write. Age-guarded, so an old binary's in-flight write is left alone.
try {
for (const entry of await readdir(parent)) {
if (!/^session-cache\.v\d+\.json\..*\.tmp$/.test(entry)) continue
await unlinkIfOlderThan(join(parent, entry), TEMP_FILE_MAX_AGE_MS, now)
}
} catch {}
const dir = sessionCacheDir()
if (!existsSync(dir)) return
try {
const entries = await readdir(dir)
const now = Date.now()
const referenced = new Set<string>([ENVELOPE_FILE])
const envelope = await readEnvelope(dir)
if (envelope) {
for (const meta of Object.values(envelope.providers)) {
for (const ref of Object.values(meta.shards)) referenced.add(ref.name)
}
// A published v9 envelope means the re-layout completed. Its retirement of
// the old layout is a separate, unsynchronised step, so a crash in between
// leaves 100MB+ of superseded cache behind forever. Age-guarded for the
// same reason the shard sweep is: an OLD binary may still be writing there.
await unlinkIfOlderThan(join(getCodeburnCacheDir(), priorCacheFile(7)), UNREFERENCED_SHARD_MAX_AGE_MS, now)
const v8Dir = join(getCodeburnCacheDir(), PRIOR_SHARD_DIR_NAME)
try {
const s = await stat(join(v8Dir, ENVELOPE_FILE))
if (now - s.mtimeMs > UNREFERENCED_SHARD_MAX_AGE_MS) await rm(v8Dir, { recursive: true, force: true })
} catch {}
}
// Only our own (versioned) temp files. Legacy `session-cache.json.*.tmp`
// temps belong to old binaries mid-write and must not be touched.
const prefix = `${CACHE_FILE}.`
for (const entry of entries) {
if (!entry.startsWith(prefix) || !entry.endsWith('.tmp')) continue
try {
const fullPath = join(dir, entry)
const s = await stat(fullPath)
if (now - s.mtimeMs > TEMP_FILE_MAX_AGE_MS) {
await unlink(fullPath)
}
} catch {}
try {
for (const entry of await readdir(dir)) {
if (entry.endsWith('.tmp')) {
await unlinkIfOlderThan(join(dir, entry), TEMP_FILE_MAX_AGE_MS, now)
continue
}
if (!envelope || referenced.has(entry)) continue
await unlinkIfOlderThan(join(dir, entry), UNREFERENCED_SHARD_MAX_AGE_MS, now)
}
} catch {}
}
@ -846,7 +1447,7 @@ export type HydrationHandle = { waited: boolean; release: () => Promise<void> }
const NOOP_HANDLE: HydrationHandle = { waited: false, release: async () => {} }
function lockPath(): string {
return join(getCacheDir(), HYDRATION_LOCK_FILE)
return join(getCodeburnCacheDir(), HYDRATION_LOCK_FILE)
}
// Our own pid never counts as a foreign holder: a same-process lock is either
@ -869,7 +1470,7 @@ async function readLockRecord(): Promise<LockRecord | null> {
async function writeOurLock(): Promise<boolean> {
try {
const dir = getCacheDir()
const dir = getCodeburnCacheDir()
if (!existsSync(dir)) await mkdir(dir, { recursive: true })
const handle = await open(lockPath(), 'wx', 0o600)
try { await handle.writeFile(JSON.stringify({ pid: process.pid, at: Date.now() }), { encoding: 'utf-8' }) }

104
src/sync/ledger.ts
View file

@ -6,8 +6,8 @@
*/
import { existsSync, mkdirSync, readFileSync, writeFileSync, unlinkSync, renameSync } from 'fs'
import { join } from 'path'
import { homedir } from 'os'
import { join, resolve } from 'path'
import { getCodeburnCacheDir } from '../cache-dir.js'
export interface LedgerEntry {
key: string // deduplicationKey
@ -16,34 +16,69 @@ export interface LedgerEntry {
const SIX_MONTHS_MS = 180 * 24 * 60 * 60 * 1000
function cacheDir(): string {
// Honor XDG_CACHE_HOME — the ledger is reconstructible state, not config
const xdg = process.env.XDG_CACHE_HOME
const base = xdg && xdg.trim() ? xdg : join(homedir(), '.cache')
return join(base, 'codeburn')
function ledgerCacheDir(): string {
return getCodeburnCacheDir()
}
function ledgerPath(): string {
return join(cacheDir(), 'sync-ledger.json')
return join(ledgerCacheDir(), 'sync-ledger.json')
}
export function readLedger(): LedgerEntry[] {
const path = ledgerPath()
if (!existsSync(path)) return []
// Before the shared cache resolver existed, sync alone wrote beneath
// XDG_CACHE_HOME. Treat that location as a one-time migration source only;
// CODEBURN_CACHE_DIR (when non-empty) is authoritative and must never import
// from an unrelated XDG tree.
function legacyXdgLedgerPath(): string | null {
if (process.env.CODEBURN_CACHE_DIR?.trim()) return null
const xdg = process.env.XDG_CACHE_HOME
if (!xdg?.trim()) return null
const legacy = join(xdg, 'codeburn', 'sync-ledger.json')
return resolve(legacy) === resolve(ledgerPath()) ? null : legacy
}
function readLedgerFile(path: string): LedgerEntry[] | null {
try {
const raw = readFileSync(path, 'utf-8')
const entries = JSON.parse(raw) as unknown
if (!Array.isArray(entries)) return []
const entries = JSON.parse(readFileSync(path, 'utf-8')) as unknown
if (!Array.isArray(entries)) return null
return entries.filter(
(e): e is LedgerEntry => typeof e === 'object' && e !== null && typeof e.key === 'string'
)
} catch {
return []
return null
}
}
export function readLedger(): LedgerEntry[] {
const path = ledgerPath()
const legacyPath = legacyXdgLedgerPath()
const canonicalEntries = existsSync(path) ? readLedgerFile(path) : null
if (!legacyPath || !existsSync(legacyPath)) return canonicalEntries ?? []
const legacyEntries = readLedgerFile(legacyPath)
if (!legacyEntries) return canonicalEntries ?? []
// Canonical wins for duplicate keys, but retain every key that exists only
// in the historical ledger so an upgrade cannot re-upload old calls.
const merged = [...(canonicalEntries ?? [])]
const keys = new Set(merged.map(entry => entry.key))
for (const entry of legacyEntries) {
if (keys.has(entry.key)) continue
keys.add(entry.key)
merged.push(entry)
}
// Publish the canonical copy before retiring the legacy source. If the
// write fails, keep and return the old ledger so deduplication still works.
try {
writeLedger(merged)
try { unlinkSync(legacyPath) } catch { /* canonical copy already wins */ }
} catch {
return merged
}
return merged
}
export function writeLedger(entries: LedgerEntry[]): void {
const dir = cacheDir()
const dir = ledgerCacheDir()
mkdirSync(dir, { recursive: true })
// Atomic write: a crash mid-write must not corrupt the ledger — a corrupt
// ledger reads as empty and the next push re-sends the whole window.
@ -77,11 +112,34 @@ export function ledgerKeySet(): Set<string> {
return new Set(readLedger().map(e => e.key))
}
/** Clear the ledger (for sync reset). Returns the number of entries removed. */
export function clearLedger(): number {
const path = ledgerPath()
if (!existsSync(path)) return 0
const count = readLedger().length
unlinkSync(path)
return count
function isMissingFileError(error: unknown): boolean {
return typeof error === 'object' && error !== null && 'code' in error && error.code === 'ENOENT'
}
/** Clear every eligible ledger (for sync reset). Returns the number of unique
* entries removed. This deliberately bypasses readLedger(): reset must delete
* canonical and legacy files independently, never migrate one into the other. */
export function clearLedger(): number {
const canonicalPath = ledgerPath()
const legacyPath = legacyXdgLedgerPath()
const targets = [canonicalPath, ...(legacyPath ? [legacyPath] : [])].map(path => ({
path,
entries: readLedgerFile(path) ?? [],
}))
const removedKeys = new Set<string>()
let deletionError: unknown
// Attempt every target even if one unlink fails. A retry then has only the
// actual remainder to remove, while ENOENT is the idempotent success case.
for (const target of targets) {
try {
unlinkSync(target.path)
for (const entry of target.entries) removedKeys.add(entry.key)
} catch (error) {
if (!isMissingFileError(error) && deletionError === undefined) deletionError = error
}
}
if (deletionError !== undefined) throw deletionError
return removedKeys.size
}

153
tests/bash-commands.test.ts
View file

@ -1,4 +1,6 @@
import { describe, it, expect } from 'vitest'
import { basename } from 'path'
import stripAnsi from 'strip-ansi'
import { extractBashCommands, isReadShapedBashCommand } from '../src/bash-utils.js'
import { BASH_TOOLS } from '../src/classifier.js'
@ -118,6 +120,157 @@ describe('BASH_TOOLS', () => {
it('rejects unknown tools', () => { expect(BASH_TOOLS.has('Read')).toBe(false) })
})
// Regression coverage for the quadratic -> linear separator-matching rewrite.
// The old regex (/\s*(?:&&|;|\|)\s*/g, and the equivalent split form) is kept
// here verbatim as a reference so new/old output can be diffed on tricky inputs.
describe('separator regex fix: parity with pre-fix implementation', () => {
function stripQuotedStringsRef(command: string): string {
return command.replace(/"[^"]*"|'[^']*'/g, match => ' '.repeat(match.length))
}
const COMMAND_PREFIXES_REF = new Set([
'sudo', 'doas',
'npx', 'bunx',
'time',
'nice', 'nohup', 'stdbuf',
'rtk',
])
const READ_ONLY_BASH_REF = new Set([
'rg', 'grep', 'egrep', 'fgrep', 'ag',
'cat', 'head', 'tail', 'less', 'more',
'ls', 'find', 'fd', 'tree',
'wc', 'stat', 'file', 'du', 'df',
'which', 'type', 'pwd', 'printenv', 'env',
'readlink', 'realpath', 'basename', 'dirname',
'jq', 'diff',
])
const GIT_READ_SUBCOMMANDS_REF = new Set([
'log', 'diff', 'status', 'show', 'blame', 'grep',
'shortlog', 'describe', 'rev-parse', 'ls-files',
])
function extractBashCommandsOld(rawCommand: string): string[] {
if (!rawCommand || !rawCommand.trim()) return []
const command = stripAnsi(rawCommand)
const stripped = stripQuotedStringsRef(command)
const separatorRegex = /\s*(?:&&|;|\|)\s*/g
const separators: Array<{ start: number; end: number }> = []
let match: RegExpExecArray | null
while ((match = separatorRegex.exec(stripped)) !== null) {
separators.push({ start: match.index, end: match.index + match[0].length })
}
const ranges: Array<[number, number]> = []
let cursor = 0
for (const sep of separators) {
ranges.push([cursor, sep.start])
cursor = sep.end
}
ranges.push([cursor, command.length])
const commands: string[] = []
for (const [start, end] of ranges) {
const segment = command.slice(start, end).trim()
if (!segment) continue
const tokens = segment.split(/\s+/)
let i = 0
while (i < tokens.length) {
if (/^\w+=/.test(tokens[i]!)) { i++; continue }
const next = tokens[i + 1]
if (
next !== undefined &&
COMMAND_PREFIXES_REF.has(basename(tokens[i]!)) &&
!next.startsWith('-') &&
!/["']/.test(next)
) { i++; continue }
break
}
const base = i < tokens.length ? basename(tokens[i]!) : ''
if (base && base !== 'cd' && base !== 'true' && base !== 'false') {
commands.push(base)
}
}
return commands
}
function isReadShapedBashCommandOld(rawCommand: string): boolean {
if (!rawCommand || !rawCommand.trim()) return false
const stripped = stripQuotedStringsRef(stripAnsi(rawCommand))
const segments = stripped.split(/\s*(?:&&|;|\|)\s*/)
let sawCommand = false
for (const segment of segments) {
const trimmed = segment.trim()
if (!trimmed) continue
const tokens = trimmed.split(/\s+/)
let i = 0
while (i < tokens.length && (/^\w+=/.test(tokens[i]!) || COMMAND_PREFIXES_REF.has(basename(tokens[i]!)))) i++
const base = i < tokens.length ? basename(tokens[i]!) : ''
if (!base) continue
sawCommand = true
if (base === 'git') {
const sub = tokens[i + 1]
if (!sub || !GIT_READ_SUBCOMMANDS_REF.has(sub)) return false
continue
}
if (!READ_ONLY_BASH_REF.has(base)) return false
}
return sawCommand
}
function buildWhitespaceHeavyCommand(): string {
const parts: string[] = []
for (let i = 0; i < 10; i++) parts.push('git' + ' '.repeat(2000) + 'status')
return parts.join(' && ')
}
const TRICKY_INPUTS: string[] = [
'echo "a && b" && ls',
"foo 'x;y';bar",
'cat <<EOF\n some text with lots of whitespace \n\n\nEOF\n && ls -la',
'echo a\r\n&&\tls',
'foo\t;\tbar',
'a ; ; b',
'a|b',
'&& ls',
'ls &&',
'; ls ;',
buildWhitespaceHeavyCommand(),
'echo a && ls',
'foo
;
bar',
'',
' ',
'ls && pwd',
'git status',
]
it('extractBashCommands matches the old implementation across tricky separator inputs', () => {
for (const input of TRICKY_INPUTS) {
expect(extractBashCommands(input)).toEqual(extractBashCommandsOld(input))
}
})
it('isReadShapedBashCommand matches the old implementation across tricky separator inputs', () => {
for (const input of TRICKY_INPUTS) {
expect(isReadShapedBashCommand(input)).toBe(isReadShapedBashCommandOld(input))
}
})
it('runs the whitespace-heavy command in well under 50ms (old form is quadratic)', () => {
const big = buildWhitespaceHeavyCommand()
const t0 = Date.now()
extractBashCommands(big)
expect(Date.now() - t0).toBeLessThan(50)
})
})
describe('isReadShapedBashCommand (#941)', () => {
it('accepts single read commands and read-only git subcommands', () => {
expect(isReadShapedBashCommand('rg -n "x" src/')).toBe(true)

25
tests/cache-dir.test.ts Normal file
View file

@ -0,0 +1,25 @@
import { afterEach, describe, expect, it } from 'vitest'
import { join } from 'path'
import { homedir } from 'os'
import { getCodeburnCacheDir } from '../src/cache-dir.js'
describe('getCodeburnCacheDir', () => {
const original = process.env['CODEBURN_CACHE_DIR']
afterEach(() => {
if (original === undefined) delete process.env['CODEBURN_CACHE_DIR']
else process.env['CODEBURN_CACHE_DIR'] = original
})
it('resolves an explicit override at call time', () => {
process.env['CODEBURN_CACHE_DIR'] = '/tmp/codeburn-one'
expect(getCodeburnCacheDir()).toBe('/tmp/codeburn-one')
process.env['CODEBURN_CACHE_DIR'] = '/tmp/codeburn-two'
expect(getCodeburnCacheDir()).toBe('/tmp/codeburn-two')
})
it.each(['', ' ', '\n\t'])('treats a blank override as absent (%j)', value => {
process.env['CODEBURN_CACHE_DIR'] = value
expect(getCodeburnCacheDir()).toBe(join(homedir(), '.cache', 'codeburn'))
})
})

277
tests/cache-directory-switch.test.ts Normal file
View file

@ -0,0 +1,277 @@
import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'fs/promises'
import { existsSync } from 'fs'
import { tmpdir } from 'os'
import { join } from 'path'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import {
clearCodexMemCaches,
fingerprintFile,
flushCodexCache,
readCachedCodexResults,
writeCachedCodexResults,
} from '../src/codex-cache.js'
import {
clearAntigravityCacheStates,
createAntigravityProvider,
flushAntigravityCache,
} from '../src/providers/antigravity.js'
import type { ParsedProviderCall } from '../src/providers/types.js'
const originalCacheDir = process.env['CODEBURN_CACHE_DIR']
const originalHome = process.env['HOME']
const originalCodexHome = process.env['CODEX_HOME']
let root: string
function call(provider: string, marker: string): ParsedProviderCall {
return {
provider,
model: marker,
inputTokens: 1,
outputTokens: 1,
cacheCreationInputTokens: 0,
cacheReadInputTokens: 0,
cachedInputTokens: 0,
reasoningTokens: 0,
webSearchRequests: 0,
costUSD: 0,
tools: [],
bashCommands: [],
timestamp: '2026-08-12T00:00:00.000Z',
speed: 'standard',
deduplicationKey: `${provider}:${marker}`,
userMessage: '',
sessionId: marker,
}
}
async function seedAntigravityCache(
cacheDir: string,
sourcePath: string,
marker: string,
): Promise<void> {
const sourceStat = await stat(sourcePath)
await mkdir(cacheDir, { recursive: true })
await writeFile(join(cacheDir, 'antigravity-results.json'), JSON.stringify({
version: 5,
cascades: {
shared: {
mtimeMs: sourceStat.mtimeMs,
sizeBytes: sourceStat.size,
calls: [call('antigravity', marker)],
},
},
}))
}
async function readAntigravityModel(sourcePath: string): Promise<string | undefined> {
const parser = createAntigravityProvider().createSessionParser({
path: sourcePath,
project: 'fixture',
provider: 'antigravity',
}, new Set())
for await (const parsed of parser.parse()) return parsed.model
return undefined
}
beforeEach(async () => {
root = await mkdtemp(join(tmpdir(), 'codeburn-cache-switch-'))
})
afterEach(async () => {
if (originalCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']
else process.env['CODEBURN_CACHE_DIR'] = originalCacheDir
if (originalHome === undefined) delete process.env['HOME']
else process.env['HOME'] = originalHome
if (originalCodexHome === undefined) delete process.env['CODEX_HOME']
else process.env['CODEX_HOME'] = originalCodexHome
await rm(root, { recursive: true, force: true })
})
describe('call-time CODEBURN_CACHE_DIR isolation', () => {
it('keeps Codex reads and writes keyed by the active cache directory', async () => {
const sourcePath = join(root, 'rollout.jsonl')
const cacheA = join(root, 'cache-a')
const cacheB = join(root, 'cache-b')
await writeFile(sourcePath, '{}\n')
const fingerprint = await fingerprintFile(sourcePath)
expect(fingerprint).not.toBeNull()
process.env['CODEBURN_CACHE_DIR'] = cacheA
await writeCachedCodexResults(sourcePath, 'project-a', [call('codex', 'from-a')], fingerprint!)
await flushCodexCache()
process.env['CODEBURN_CACHE_DIR'] = cacheB
expect(await readCachedCodexResults(sourcePath)).toBeNull()
await writeCachedCodexResults(sourcePath, 'project-b', [call('codex', 'from-b')], fingerprint!)
await flushCodexCache()
const diskB = JSON.parse(await readFile(join(cacheB, 'codex-results.json'), 'utf8'))
expect(diskB.files[sourcePath].calls.map((entry: ParsedProviderCall) => entry.model)).toEqual(['from-b'])
process.env['CODEBURN_CACHE_DIR'] = cacheA
expect((await readCachedCodexResults(sourcePath))?.calls.map(entry => entry.model)).toEqual(['from-a'])
})
it('does not flush dirty Codex state from A into B', async () => {
const sourceA = join(root, 'a.jsonl')
const sourceB = join(root, 'b.jsonl')
const cacheA = join(root, 'cache-a-dirty')
const cacheB = join(root, 'cache-b-dirty')
await writeFile(sourceA, 'a\n')
await writeFile(sourceB, 'b\n')
process.env['CODEBURN_CACHE_DIR'] = cacheA
await writeCachedCodexResults(sourceA, 'project-a', [call('codex', 'dirty-a')], (await fingerprintFile(sourceA))!)
process.env['CODEBURN_CACHE_DIR'] = cacheB
await writeCachedCodexResults(sourceB, 'project-b', [call('codex', 'dirty-b')], (await fingerprintFile(sourceB))!)
await flushCodexCache()
const diskB = JSON.parse(await readFile(join(cacheB, 'codex-results.json'), 'utf8'))
expect(Object.keys(diskB.files)).toEqual([sourceB])
process.env['CODEBURN_CACHE_DIR'] = cacheA
await flushCodexCache()
const diskA = JSON.parse(await readFile(join(cacheA, 'codex-results.json'), 'utf8'))
expect(Object.keys(diskA.files)).toEqual([sourceA])
})
it('pins Codex reads, dirty writes, and flushes to the parse call-time directory', async () => {
const home = join(root, 'parse-home')
const codexHome = join(root, 'parse-codex-home')
const sessionDir = join(codexHome, 'sessions', '2026', '08', '12')
const cacheA = join(root, 'parse-cache-a')
const cacheB = join(root, 'parse-cache-b')
await mkdir(sessionDir, { recursive: true })
await mkdir(home, { recursive: true })
const sourcePath = join(sessionDir, 'rollout-cache-dir-switch.jsonl')
await writeFile(sourcePath, [
JSON.stringify({
type: 'session_meta',
timestamp: '2026-08-12T10:00:00.000Z',
payload: {
cwd: '/Users/test/cache-dir-transaction',
originator: 'codex-cli',
session_id: 'cache-dir-transaction',
model: 'gpt-5.3-codex',
},
}),
JSON.stringify({
type: 'event_msg',
timestamp: '2026-08-12T10:01:00.000Z',
payload: {
type: 'token_count',
info: {
model: 'gpt-5.3-codex',
last_token_usage: {
input_tokens: 10,
cached_input_tokens: 0,
output_tokens: 5,
reasoning_output_tokens: 0,
total_tokens: 15,
},
total_token_usage: {
input_tokens: 10,
cached_input_tokens: 0,
output_tokens: 5,
reasoning_output_tokens: 0,
total_tokens: 15,
},
},
},
}),
].join('\n') + '\n')
process.env['HOME'] = home
process.env['CODEX_HOME'] = codexHome
process.env['CODEBURN_CACHE_DIR'] = cacheA
const { clearSessionCache, parseAllSessions } = await import('../src/parser.js')
clearSessionCache()
// parseAllSessions reaches its first await before any Codex cache access.
// Switching the host env immediately after invocation deterministically
// exercises every later read/write/flush under the captured A transaction.
const parsing = parseAllSessions(undefined, 'codex')
process.env['CODEBURN_CACHE_DIR'] = cacheB
const projects = await parsing
expect(projects.some(project => project.sessions.some(session =>
session.turns.some(turn => turn.assistantCalls.some(entry => entry.provider === 'codex'))
))).toBe(true)
expect(existsSync(join(cacheA, 'codex-results.json'))).toBe(true)
expect(existsSync(join(cacheB, 'codex-results.json'))).toBe(false)
const diskA = JSON.parse(await readFile(join(cacheA, 'codex-results.json'), 'utf8'))
expect(diskA.files[sourcePath].calls).toHaveLength(1)
clearSessionCache()
})
it('loads Antigravity cache entries from the active directory after A to B', async () => {
const sourcePath = join(root, 'shared.pb')
const cacheA = join(root, 'agy-cache-a')
const cacheB = join(root, 'agy-cache-b')
await writeFile(sourcePath, 'fixture')
await seedAntigravityCache(cacheA, sourcePath, 'from-a')
await seedAntigravityCache(cacheB, sourcePath, 'from-b')
process.env['CODEBURN_CACHE_DIR'] = cacheA
expect(await readAntigravityModel(sourcePath)).toBe('from-a')
process.env['CODEBURN_CACHE_DIR'] = cacheB
expect(await readAntigravityModel(sourcePath)).toBe('from-b')
})
it('does not flush dirty Antigravity state from A into B', async () => {
const sourcePath = join(root, 'shared.pb')
const cacheA = join(root, 'agy-cache-a-dirty')
const cacheB = join(root, 'agy-cache-b-dirty')
await writeFile(sourcePath, 'fixture')
await seedAntigravityCache(cacheA, sourcePath, 'from-a')
await seedAntigravityCache(cacheB, sourcePath, 'from-b')
process.env['CODEBURN_CACHE_DIR'] = cacheA
expect(await readAntigravityModel(sourcePath)).toBe('from-a')
// The provider parse transaction captures A. Even if the host changes its
// call-time env before the deferred flush, eviction/publication stays on A.
process.env['CODEBURN_CACHE_DIR'] = cacheB
await flushAntigravityCache(new Set(), cacheA)
expect(existsSync(join(cacheB, 'antigravity-results.json'))).toBe(true)
const diskB = JSON.parse(await readFile(join(cacheB, 'antigravity-results.json'), 'utf8'))
expect(diskB.cascades.shared.calls[0].model).toBe('from-b')
const diskA = JSON.parse(await readFile(join(cacheA, 'antigravity-results.json'), 'utf8'))
expect(diskA.cascades).toEqual({})
})
it('drops clean per-directory memos when the resident RSS guard clears them', async () => {
const codexSource = join(root, 'guard.jsonl')
const antigravitySource = join(root, 'shared.pb')
const cacheDir = join(root, 'guard-cache')
await writeFile(codexSource, '{}\n')
await writeFile(antigravitySource, 'fixture')
await seedAntigravityCache(cacheDir, antigravitySource, 'before')
process.env['CODEBURN_CACHE_DIR'] = cacheDir
await writeCachedCodexResults(codexSource, 'project', [call('codex', 'before')], (await fingerprintFile(codexSource))!)
await flushCodexCache()
expect(await readAntigravityModel(antigravitySource)).toBe('before')
// Another process republishes both cache files. Without the clear, the
// resident keeps serving its warm copies.
await seedAntigravityCache(cacheDir, antigravitySource, 'after')
const codexDisk = JSON.parse(await readFile(join(cacheDir, 'codex-results.json'), 'utf8'))
codexDisk.files[codexSource].calls[0].model = 'after'
await writeFile(join(cacheDir, 'codex-results.json'), JSON.stringify(codexDisk))
expect((await readCachedCodexResults(codexSource))?.calls.map(entry => entry.model)).toEqual(['before'])
expect(await readAntigravityModel(antigravitySource)).toBe('before')
clearCodexMemCaches()
clearAntigravityCacheStates()
expect((await readCachedCodexResults(codexSource))?.calls.map(entry => entry.model)).toEqual(['after'])
expect(await readAntigravityModel(antigravitySource)).toBe('after')
})
})

5
tests/cache-persistence-coldstart.test.ts
View file

@ -4,7 +4,8 @@ import { join } from 'path'
import { tmpdir } from 'os'
import { parseAllSessions, clearSessionCache } from '../src/parser.js'
import { CACHE_VERSION, sessionCachePath } from '../src/session-cache.js'
import { CACHE_VERSION } from '../src/session-cache.js'
import { readCacheOnDisk } from './fixtures/session-cache-io.js'
let tmpDir: string
let cacheDir: string
@ -48,7 +49,7 @@ describe('cold-start cache persistence', () => {
const projects = await parseAllSessions()
expect(projects.length).toBeGreaterThan(0)
const raw = JSON.parse(await readFile(sessionCachePath(), 'utf-8'))
const raw = await readCacheOnDisk()
expect(raw.version).toBe(CACHE_VERSION)
const claudeFiles = Object.keys(raw.providers?.claude?.files ?? {})
expect(claudeFiles.length).toBeGreaterThan(0)

4
tests/cache-refresh-lock.test.ts
View file

@ -8,7 +8,7 @@ import {
type RefreshLockClock,
} from '../src/cache-refresh-lock.js'
import { clearSessionCache, parseAllSessions } from '../src/parser.js'
import { emptyCache, loadCache, saveCache, sessionCachePath } from '../src/session-cache.js'
import { emptyCache, loadCache, saveCache, sessionCacheDir } from '../src/session-cache.js'
const dirs: string[] = []
@ -202,7 +202,7 @@ describe('warm session-cache refresh lock', () => {
await result.handle.release()
expect(JSON.parse(await readFile(lockPath(dir), 'utf-8')).token).toBe('successor')
expect(sessionCachePath()).toContain(dir)
expect(sessionCacheDir()).toContain(dir)
})
// retry shields environmental fd/CPU starvation in a saturated full-suite

7
tests/codex-cache-invalidation.test.ts
View file

@ -13,7 +13,7 @@ import { createHash } from 'crypto'
import { join } from 'path'
import { clearSessionCache, parseAllSessions } from '../src/parser.js'
import { sessionCachePath } from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
const testRoot = vi.hoisted(() => {
const root = `${process.env['TMPDIR'] || '/tmp'}/codex-stale-repro-${process.pid}-${Date.now()}`
@ -75,8 +75,7 @@ describe('codex parser change invalidates stale session-cache (#478/#513)', () =
// release: pre-fix envFingerprint, unchanged file fingerprint, cached
// turns lack the mcp__ tool. Also reset codex-results.json to v4 so the
// provider (if it runs at all) must genuinely re-parse.
const cachePath = sessionCachePath()
const cache = JSON.parse(await readFile(cachePath, 'utf8'))
const cache = await readCacheOnDisk() as any
cache.providers.codex.envFingerprint = preFixFingerprint()
for (const f of Object.values(cache.providers.codex.files) as any[]) {
for (const turn of f.turns) {
@ -88,7 +87,7 @@ describe('codex parser change invalidates stale session-cache (#478/#513)', () =
}
}
}
await writeFile(cachePath, JSON.stringify(cache))
await writeCacheOnDisk(cache)
const codexCachePath = join(CACHE_DIR, 'codex-results.json')
const codexCache = JSON.parse(await readFile(codexCachePath, 'utf8'))
codexCache.version = 4

4
tests/fixtures/cache-refresh-worker.ts vendored
View file

@ -3,7 +3,7 @@ import { mkdir, readFile, writeFile } from 'fs/promises'
import { join } from 'path'
import { acquireCacheRefreshLock } from '../../src/cache-refresh-lock.js'
import { loadCache, saveCache } from '../../src/session-cache.js'
import { loadCache, markCacheDirty, saveCache } from '../../src/session-cache.js'
const [cacheDir, barrierDir, id, sourcePath, bypass = 'false'] = process.argv.slice(2)
if (!cacheDir || !barrierDir || !id || !sourcePath) throw new Error('missing worker argument')
@ -33,7 +33,7 @@ try {
mcpInventory: [],
turns: [],
}
;(cache as { _dirty?: boolean })._dirty = true
markCacheDirty(cache, 'regression')
await writeFile(join(barrierDir, `${id}.parsed`), '')
await waitFor(`${id}.save`)
const published = await saveCache(cache, refresh?.handle.verifyStillOwner)

35
tests/fixtures/session-cache-io.ts vendored Normal file
View file

@ -0,0 +1,35 @@
// Test-side IO for the sharded session cache: the on-disk form is a directory
// (envelope + one shard per provider), so tests read and write it through the
// real load/save path instead of touching a single JSON file.
import { readFile, readdir } from 'fs/promises'
import { join } from 'path'
import {
clearLoadCacheMemo,
loadCache,
markCacheDirty,
saveCache,
sessionCacheDir,
type SessionCache,
} from '../../src/session-cache.js'
/** The cache exactly as it is on disk, bypassing the in-process memo. */
export async function readCacheOnDisk(): Promise<SessionCache> {
clearLoadCacheMemo()
return loadCache()
}
/** Publish `cache`, rewriting every provider's shard. */
export async function writeCacheOnDisk(cache: SessionCache): Promise<void> {
for (const provider of Object.keys(cache.providers)) markCacheDirty(cache, provider)
await saveCache(cache)
clearLoadCacheMemo()
}
/** Byte-level snapshot of the whole cache directory (names + contents). */
export async function cacheDirSnapshot(): Promise<string> {
const dir = sessionCacheDir()
const names = (await readdir(dir)).sort()
const parts = await Promise.all(names.map(async name => `${name}:${await readFile(join(dir, name), 'utf-8')}`))
return parts.join('\n')
}

117
tests/flat-slice.test.ts Normal file
View file

@ -0,0 +1,117 @@
/**
* Tests for flatSlice — the SlicedString-retention fix.
*
* Background: `String.prototype.slice` returns a V8 SlicedString that
* retains its entire parent string. Storing short slices of large session
* strings (100KB+ agent prompts) in the long-lived session cache pinned
* gigabytes of parent buffers during cold parses, OOMing the default heap
* (issue observed at ~5.5GB peak for 3.2GB of kiro session files; ~300MB
* after flattening).
*/
import { describe, it, expect } from 'vitest'
import { flatSlice, flatString } from '../src/content-utils.js'
describe('flatSlice', () => {
it('returns the prefix for strings over the bound', () => {
const big = 'x'.repeat(10_000)
const out = flatSlice(big, 500)
expect(out.length).toBe(500)
expect(out).toBe(big.slice(0, 500))
})
it('returns the string itself when within the bound', () => {
const small = 'hello world'
expect(flatSlice(small, 500)).toBe(small)
})
it('handles multi-byte characters without corruption', () => {
// Emoji + CJK near the boundary — Buffer round-trip must not produce
// invalid UTF-8 replacement chars for chars fully inside the slice.
const s = '🐾'.repeat(300) // each emoji is 2 UTF-16 code units
const out = flatSlice(s, 500)
expect(out).toBe(s.slice(0, 500))
})
it('preserves a lone surrogate at a mid-pair cut', () => {
// A cut landing between the high and low surrogate of a pair leaves a
// lone surrogate. utf16le round-trips code units byte-for-byte, so the
// lone surrogate survives intact (unlike utf-8, which would replace it
// with U+FFFD).
const s = 'ab' + '🐾'.repeat(300) // odd offset puts every emoji across even boundaries
const out = flatSlice(s, 501) // cuts mid-pair
expect(out.length).toBe(501)
expect(out.slice(0, 500)).toBe(s.slice(0, 500)) // content before the cut intact
expect(out.charCodeAt(500)).toBe(s.charCodeAt(500)) // lone surrogate preserved
})
it('does not retain the parent of an already-sliced view', () => {
// The bug this early-return removal fixes: provider adapters pre-truncate
// with .slice(0, 500) before the cache-site flatSlice call, so a naive
// "already within bound" early return would skip flattening and leave
// the SlicedString pinning its 100KB parent.
const before = process.memoryUsage().heapUsed
const kept: string[] = []
for (let i = 0; i < 1000; i++) {
const parent = (i % 10).toString().repeat(100_000) + i
const preSliced = parent.slice(0, 500)
kept.push(flatSlice(preSliced, 2000))
}
if (typeof global.gc === 'function') global.gc()
const after = process.memoryUsage().heapUsed
const growthMB = (after - before) / 1048576
expect(kept.length).toBe(1000)
expect(growthMB).toBeLessThan(50)
})
it('does not retain the parent string (heap growth stays bounded)', () => {
// Property test for the retention fix: keep 1000 short prefixes of
// 1000 distinct 100KB strings. With plain .slice() each prefix pins its
// 100KB parent (~200MB in UTF-16 total). With flatSlice, retained data
// is ~1000 × 500 chars ≈ 1MB. Assert heap growth is far below the
// retention scenario. Threshold is generous (50MB) to be CI-safe while
// still failing decisively if retention returns (>190MB). When the test
// runner exposes gc (vitest under --expose-gc), force a collection so
// transient parent garbage doesn't inflate the measurement.
const before = process.memoryUsage().heapUsed
const kept: string[] = []
for (let i = 0; i < 1000; i++) {
// Distinct content so V8 cannot intern/share the parents.
const parent = (i % 10).toString().repeat(100_000)
kept.push(flatSlice(parent + i, 500))
}
if (typeof global.gc === 'function') global.gc()
const after = process.memoryUsage().heapUsed
const growthMB = (after - before) / 1048576
expect(kept.length).toBe(1000)
expect(growthMB).toBeLessThan(50)
})
})
describe('flatString', () => {
it('returns an equal string for any input', () => {
expect(flatString('')).toBe('')
expect(flatString('hello')).toBe('hello')
expect(flatString('🐾 multi-byte ✓')).toBe('🐾 multi-byte ✓')
})
it('does not retain the parent of a regex match group', () => {
// match[1] is a SlicedString retaining the entire subject. flatString
// must break that link: keep 1000 short match groups of distinct 100KB
// subjects and assert bounded heap growth (same thresholds as the
// flatSlice retention test).
const before = process.memoryUsage().heapUsed
const kept: string[] = []
for (let i = 0; i < 1000; i++) {
const subject = `<name>tool_${i}</name>` + (i % 10).toString().repeat(100_000)
const m = /<name>([^<]+)<\/name>/.exec(subject)
kept.push(flatString(m![1]!))
}
if (typeof global.gc === 'function') global.gc()
const after = process.memoryUsage().heapUsed
const growthMB = (after - before) / 1048576
expect(kept.length).toBe(1000)
expect(growthMB).toBeLessThan(50)
})
})

8
tests/hydration-interrupt-convergence.test.ts
View file

@ -6,7 +6,7 @@ import { afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'
import { loadPricing, setLocalModelSavings, setModelAliases } from '../src/models.js'
import { buildMenubarPayloadForRange } from '../src/usage-aggregator.js'
import { clearSessionCache } from '../src/parser.js'
import { sessionCachePath } from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
import { dailyCachePath } from '../src/daily-cache.js'
import type { DateRange } from '../src/types.js'
@ -91,9 +91,9 @@ describe('interrupted hydration converges to the uninterrupted result', () => {
// (a) Session cache: present but NOT marked complete — an interrupted cold
// start's throttled partial save.
const sessionRaw = JSON.parse(await readFile(sessionCachePath(), 'utf-8'))
const sessionRaw = await readCacheOnDisk()
sessionRaw.complete = false
await writeFile(sessionCachePath(), JSON.stringify(sessionRaw), 'utf-8')
await writeCacheOnDisk(sessionRaw)
// (b) Daily cache: frozen with the older days dropped but `lastComputedDate`
// advanced to yesterday and NO completeness marker — the exact freeze that
@ -118,7 +118,7 @@ describe('interrupted hydration converges to the uninterrupted result', () => {
expect(healed.current.calls).toBe(reference.current.calls)
// And the on-disk markers are now durably complete, so the next launch is warm.
expect(JSON.parse(await readFile(sessionCachePath(), 'utf-8')).complete).toBe(true)
expect((await readCacheOnDisk()).complete).toBe(true)
expect(JSON.parse(await readFile(dailyCachePath(), 'utf-8')).complete).toBe(true)
})
})

4
tests/kiro-cache-invalidation.test.ts
View file

@ -25,9 +25,9 @@ import {
CACHE_VERSION,
computeEnvFingerprint,
fingerprintFile,
sessionCachePath,
type SessionCache,
} from '../src/session-cache.js'
import { writeCacheOnDisk } from './fixtures/session-cache-io.js'
// The kiro provider singleton captures homedir() when its module is first
// imported, so HOME must point at the test root before ../src/parser.js is
@ -99,7 +99,7 @@ async function seedCache(execPath: string, envFingerprint: string): Promise<void
},
}
await mkdir(CACHE_DIR, { recursive: true })
await writeFile(sessionCachePath(), JSON.stringify(cache))
await writeCacheOnDisk(cache)
}
async function parseKiroCalls() {

205
tests/kiro-projectpath.test.ts Normal file
View file

@ -0,0 +1,205 @@
/**
* Tests for kiro projectPath emission (sync attribution support).
*
* The kiro provider historically reduced the session's working directory to
* `basename(cwd)` for display and discarded the full path. Sync attribution
* (`codeburn sync push --attribution`) needs the full path on
* `ParsedProviderCall.projectPath` to resolve the git repo — without it,
* every kiro session is attribution-blind.
*
* Also covers the cache side: projectPath is persisted via CachedCall, so
* entries cached BEFORE the parser learned to emit it must re-parse. That is
* driven by the PROVIDER_PARSE_VERSIONS.kiro bump (project-path-v1); a cache
* seeded at the pre-bump fingerprint must be discarded.
*/
import { mkdir, writeFile, rm } from 'node:fs/promises'
import { createHash } from 'node:crypto'
import { join } from 'node:path'
import { describe, it, expect, beforeEach, afterAll, vi } from 'vitest'
import { clearSessionCache, parseAllSessions } from '../src/parser.js'
import {
CACHE_VERSION,
computeEnvFingerprint,
fingerprintFile,
type SessionCache,
} from '../src/session-cache.js'
import { writeCacheOnDisk } from './fixtures/session-cache-io.js'
// The kiro provider reads homedir()/env at call time in discovery; HOME must
// point at the test root before ../src/parser.js is evaluated (see the
// equivalent note in kiro-cache-invalidation.test.ts).
const testRoot = vi.hoisted(() => {
const root = `${process.env['TMPDIR'] || '/tmp'}/kiro-projpath-${process.pid}-${Date.now()}`
process.env['HOME'] = `${root}/home`
process.env['USERPROFILE'] = `${root}/home`
return root
})
const HOME = join(testRoot, 'home')
const CACHE_DIR = join(testRoot, 'cache')
const KIRO_SESSIONS = join(HOME, '.kiro', 'sessions')
const CLI_DIR = join(KIRO_SESSIONS, 'cli')
const CLI_CWD = '/local/home/testuser/workplace/my-project'
const V2_WORKSPACE = '/local/home/testuser/workplace/ide-project'
beforeEach(() => {
process.env['HOME'] = HOME
process.env['USERPROFILE'] = HOME
process.env['CODEBURN_CACHE_DIR'] = CACHE_DIR
delete process.env['KIRO_HOME']
clearSessionCache()
})
afterAll(async () => {
await rm(testRoot, { recursive: true, force: true })
})
/** Write a minimal kiro CLI session: <id>.jsonl entries + companion .json meta. */
async function seedCliSession(id: string, cwd: string): Promise<string> {
await mkdir(CLI_DIR, { recursive: true })
const jsonlPath = join(CLI_DIR, `${id}.jsonl`)
const entries = [
{ kind: 'Prompt', data: { content: [{ kind: 'text', data: 'add a feature' }] } },
{ kind: 'AssistantMessage', data: { content: [{ kind: 'text', data: 'Done — added the feature and tests.' }] } },
]
await writeFile(jsonlPath, entries.map(e => JSON.stringify(e)).join('\n'))
await writeFile(join(CLI_DIR, `${id}.json`), JSON.stringify({
session_id: id,
cwd,
created_at: '2026-08-01T10:00:00Z',
updated_at: '2026-08-01T10:05:00Z',
session_state: {
rts_model_state: { model_info: { model_id: 'auto' } },
conversation_metadata: {
user_turn_metadatas: [
{ end_timestamp: '2026-08-01T10:05:00Z', metering_usage: [] },
],
},
},
}))
return jsonlPath
}
/** Write a minimal v2 IDE session: sessions/<hash>/sess_<id>/{session.json,messages.jsonl}. */
async function seedV2Session(id: string, workspacePath: string): Promise<void> {
const sessDir = join(KIRO_SESSIONS, 'f'.repeat(32), `sess_${id}`)
await mkdir(sessDir, { recursive: true })
await writeFile(join(sessDir, 'session.json'), JSON.stringify({
id,
modelId: 'auto',
workspacePaths: [workspacePath],
createdAt: '2026-08-01T11:00:00Z',
}))
const events = [
{ timestamp: '2026-08-01T11:00:00Z', payload: { type: 'user', content: 'fix the bug' } },
{ timestamp: '2026-08-01T11:00:01Z', payload: { type: 'turn_start', executionId: 'x1' } },
{ timestamp: '2026-08-01T11:00:05Z', payload: { type: 'assistant', content: 'Fixed the bug in handler.ts by checking null first.' } },
{ timestamp: '2026-08-01T11:00:06Z', payload: { type: 'turn_end', executionId: 'x1' } },
]
await writeFile(join(sessDir, 'messages.jsonl'), events.map(e => JSON.stringify(e)).join('\n'))
}
function kiroAgentDir(): string {
if (process.platform === 'darwin') {
return join(HOME, 'Library', 'Application Support', 'Kiro', 'User', 'globalStorage', 'kiro.kiroagent')
}
if (process.platform === 'win32') {
return join(HOME, 'AppData', 'Roaming', 'Kiro', 'User', 'globalStorage', 'kiro.kiroagent')
}
return join(HOME, '.config', 'Kiro', 'User', 'globalStorage', 'kiro.kiroagent')
}
/** Write a minimal IDE workspace-session:
* <agentDir>/workspace-sessions/<base64(workspacePath), '='→'_'>/<sessionId>.json */
async function seedWorkspaceSession(id: string, workspaceDirectory: string): Promise<void> {
const encoded = Buffer.from(workspaceDirectory, 'utf-8').toString('base64').replace(/=/g, '_')
const dir = join(kiroAgentDir(), 'workspace-sessions', encoded)
await mkdir(dir, { recursive: true })
await writeFile(join(dir, `${id}.json`), JSON.stringify({
sessionId: id,
selectedModel: 'auto',
workspaceDirectory,
history: [
{ message: { role: 'user', content: 'refactor the config loader' } },
{ message: { role: 'assistant', content: 'Refactored the loader into three small functions with tests.' } },
],
}))
}
async function kiroCalls() {
const projects = await parseAllSessions(undefined, 'kiro')
return projects.flatMap(p => p.sessions.map(s => ({ project: p.project, projectPath: p.projectPath, session: s })))
}
describe('kiro projectPath emission', () => {
it('CLI session: projectPath is the full meta.cwd, project the basename', async () => {
await seedCliSession('cli-001', CLI_CWD)
const rows = await kiroCalls()
const row = rows.find(r => r.project === 'my-project')
expect(row).toBeDefined()
expect(row!.projectPath).toBe(CLI_CWD)
})
it('v2 IDE session: projectPath is workspacePaths[0]', async () => {
await seedV2Session('v2-001', V2_WORKSPACE)
const rows = await kiroCalls()
const row = rows.find(r => r.project === 'ide-project')
expect(row).toBeDefined()
expect(row!.projectPath).toBe(V2_WORKSPACE)
})
it('workspace session: projectPath is workspaceDirectory', async () => {
const WS_DIR = '/local/home/testuser/workplace/ws-project'
await seedWorkspaceSession('ws-001', WS_DIR)
const rows = await kiroCalls()
const row = rows.find(r => r.project === 'ws-project')
expect(row).toBeDefined()
expect(row!.projectPath).toBe(WS_DIR)
})
})
describe('kiro projectPath cache invalidation (project-path-v1 bump)', () => {
// The fingerprint a cache written by the PREVIOUS release carries: same env
// vars, but the parser version before the project-path-v1 bump.
function preBumpFingerprint(): string {
const parts = [`KIRO_HOME=${process.env['KIRO_HOME'] ?? ''}`, 'parser=ide-parsing-v1-est-cost']
return createHash('sha256').update(parts.join('\0')).digest('hex').slice(0, 16)
}
it('the bump changed the env fingerprint', () => {
expect(computeEnvFingerprint('kiro')).not.toBe(preBumpFingerprint())
})
it('a pre-bump cache entry (no projectPath) is re-parsed and gains projectPath', async () => {
const jsonlPath = await seedCliSession('cli-002', CLI_CWD)
// Seed a cache exactly as the pre-bump release would have left it:
// correct file fingerprint, pre-bump env fingerprint, turns WITHOUT
// projectPath on the cached calls.
const fp = await fingerprintFile(jsonlPath)
if (!fp) throw new Error('failed to fingerprint seeded session file')
const cache: SessionCache = {
version: CACHE_VERSION,
providers: {
kiro: {
envFingerprint: preBumpFingerprint(),
files: {
[jsonlPath]: { fingerprint: fp, mcpInventory: [], turns: [] },
},
},
},
}
await mkdir(CACHE_DIR, { recursive: true })
await writeCacheOnDisk(cache)
clearSessionCache()
const rows = await kiroCalls()
const row = rows.find(r => r.project === 'my-project')
expect(row).toBeDefined()
expect(row!.projectPath).toBe(CLI_CWD)
})
})

484
tests/parse-workers.test.ts Normal file
View file

@ -0,0 +1,484 @@
import { spawnSync } from 'node:child_process'
import { appendFile, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { createHash } from 'node:crypto'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { decideParseWorkers, ParseWorkerPool, parseFilesInOrder, type ClaudeWorkerParse } from '../src/parse-workers.js'
import { clearSessionCache, parseAllSessions, parseClaudeFileFull } from '../src/parser.js'
import { parseCodexFileFull, type CodexFullParse } from '../src/providers/codex.js'
import type { SessionSource } from '../src/providers/types.js'
// Two full cold CLI parses of a multi-hundred-file corpus, plus in-process parses
// that spawn real threads.
vi.setConfig({ testTimeout: 60_000 })
const BIG_SYSTEM = { cores: 16, availableBytes: 32 * 1024 ** 3 }
const BIG_PENDING = { files: 5000, bytes: 6 * 1024 ** 3 }
const NO_ENV = {} as NodeJS.ProcessEnv
describe('decideParseWorkers', () => {
it('scales with cores, memory budget and pending file count', () => {
// 15 (cores-1) vs 8 (2 GB budget / 256 MB) vs 100 (5000/50) -> memory cap wins
expect(decideParseWorkers(BIG_PENDING, BIG_SYSTEM, NO_ENV).workers).toBe(8)
// Fewer cores than the memory budget allows -> cores-1 wins
expect(decideParseWorkers(BIG_PENDING, { cores: 6, availableBytes: 32 * 1024 ** 3 }, NO_ENV).workers).toBe(5)
// 8 GB reaches the same cap as 32 GB: a quarter of it is the 2 GB budget
expect(decideParseWorkers(BIG_PENDING, { cores: 16, availableBytes: 8 * 1024 ** 3 }, NO_ENV).workers).toBe(8)
// Under that, the quarter-of-available budget is the binding constraint
expect(decideParseWorkers(BIG_PENDING, { cores: 16, availableBytes: 6 * 1024 ** 3 }, NO_ENV).workers).toBe(6)
// The smallest machine that clears every gate still only earns 2 threads
expect(decideParseWorkers({ files: 200, bytes: 300 * 1024 ** 2 }, { cores: 3, availableBytes: 4 * 1024 ** 3 }, NO_ENV).workers).toBe(2)
// Big average file: the per-worker budget scales with it (2 x 260 MB + 128 MB),
// so the same 2 GB buys 3 threads instead of the 8 a flat 256 MB would.
expect(decideParseWorkers({ files: 250, bytes: 65 * 1024 ** 3 }, BIG_SYSTEM, NO_ENV).workers).toBe(3)
// 300 files earn 6, but the 6 GB behind them earn 30 — bytes win, then the
// memory budget caps it. A Codex corpus is exactly this shape.
expect(decideParseWorkers({ files: 300, bytes: 6 * 1024 ** 3 }, BIG_SYSTEM, NO_ENV).workers).toBe(8)
// Few enough files AND bytes that MIN_FILES_PER_WORKER is the binding constraint
expect(decideParseWorkers({ files: 300, bytes: 700 * 1024 ** 2 }, BIG_SYSTEM, NO_ENV).workers).toBe(6)
})
it('stays serial on low-spec machines and on warm/small parses', () => {
expect(decideParseWorkers(BIG_PENDING, { cores: 2, availableBytes: 32 * 1024 ** 3 }, NO_ENV).workers).toBe(0)
// A 4 GB box: availableMemory() always reads a little under the nominal size
expect(decideParseWorkers(BIG_PENDING, { cores: 16, availableBytes: 3.9 * 1024 ** 3 }, NO_ENV).workers).toBe(0)
// Warm/incremental: the byte gate is not reached
expect(decideParseWorkers({ files: 12, bytes: 10 * 1024 ** 2 }, BIG_SYSTEM, NO_ENV).workers).toBe(0)
})
it('gates on bytes alone, so a thin corpus never spawns threads it cannot pay for', () => {
// 250 files holding under a megabyte between them: threads made this ~5% slower
expect(decideParseWorkers({ files: 250, bytes: 917 * 1024 }, BIG_SYSTEM, NO_ENV).workers).toBe(0)
expect(decideParseWorkers({ files: 5000, bytes: 10 * 1024 ** 2 }, BIG_SYSTEM, NO_ENV).workers).toBe(0)
expect(decideParseWorkers({ files: 250, bytes: 917 * 1024 }, BIG_SYSTEM, NO_ENV).reason)
.toContain('below 210 MB pending')
// 150 rollouts over the byte gate: far under any file-count threshold, and the
// biggest workload there is
expect(decideParseWorkers({ files: 150, bytes: 4 * 1024 ** 3 }, BIG_SYSTEM, NO_ENV).workers).toBe(8)
})
it('honours CODEBURN_PARSE_WORKERS, which also bypasses the auto gates', () => {
expect(decideParseWorkers(BIG_PENDING, BIG_SYSTEM, { CODEBURN_PARSE_WORKERS: '0' }).workers).toBe(0)
expect(decideParseWorkers(BIG_PENDING, BIG_SYSTEM, { CODEBURN_PARSE_WORKERS: '4' }).workers).toBe(4)
// Capped by the core count
expect(decideParseWorkers(BIG_PENDING, { cores: 4, availableBytes: 32 * 1024 ** 3 }, { CODEBURN_PARSE_WORKERS: '32' }).workers).toBe(4)
// A tiny fixture corpus still gets threads when forced — that is what makes
// the determinism test below able to exercise them at all.
expect(decideParseWorkers({ files: 3, bytes: 1000 }, BIG_SYSTEM, { CODEBURN_PARSE_WORKERS: '3' }).workers).toBe(3)
expect(decideParseWorkers(BIG_PENDING, BIG_SYSTEM, { CODEBURN_PARSE_WORKERS: 'nonsense' }).workers).toBe(0)
})
it('reports the decision inputs in every reason, gate or not', () => {
for (const d of [
decideParseWorkers(BIG_PENDING, BIG_SYSTEM, NO_ENV),
decideParseWorkers({ files: 12, bytes: 1000 }, BIG_SYSTEM, NO_ENV),
decideParseWorkers(BIG_PENDING, BIG_SYSTEM, { CODEBURN_PARSE_WORKERS: '2' }),
]) {
expect(d.reason).toContain('16 cores')
expect(d.reason).toContain('GB available')
expect(d.reason).toContain('pending files')
}
})
})
type Turn = { id: string; t: number }
function sessionLines(project: string, session: string, turns: Turn[]): string {
const lines: string[] = []
for (const { id, t } of turns) {
const ts = new Date(Date.UTC(2026, 4, 4 + (t % 5), 9, t % 60, 0)).toISOString()
const gitBranch = t % 3 === 0 ? 'main' : 'feature'
lines.push(JSON.stringify({
type: 'user', sessionId: session, timestamp: ts, cwd: `/tmp/proj${project}`, gitBranch,
message: { role: 'user', content: `task ${t} in ${project}` },
}))
lines.push(JSON.stringify({
type: 'assistant', sessionId: session, timestamp: ts, cwd: `/tmp/proj${project}`, gitBranch,
message: {
id, type: 'message', role: 'assistant', model: 'claude-sonnet-4-5',
content: [
{ type: 'text', text: 'x'.repeat(200) },
{ type: 'tool_use', id: `tu-${id}`, name: 'Edit', input: { file_path: '/tmp/x', old_string: 'a', new_string: 'b' } },
],
usage: { input_tokens: 400 + t, output_tokens: 40 + t, cache_read_input_tokens: 9 },
},
}))
}
return lines.join('\n') + '\n'
}
const range = (n: number, from = 0): number[] => Array.from({ length: n }, (_, i) => i + from)
async function writeCorpus(claudeDir: string, projects: number, filesPerProject: number): Promise<string[]> {
const written: string[] = []
for (let p = 0; p < projects; p++) {
const dir = join(claudeDir, 'projects', `-tmp-proj${p}`)
await mkdir(dir, { recursive: true })
for (let f = 0; f < filesPerProject; f++) {
const session = `${p}${f}`.padStart(8, '0') + '-aaaa-bbbb-cccc-000000000000'
const path = join(dir, `${session}.jsonl`)
await writeFile(path, sessionLines(String(p), session, range(12).map(t => ({ id: `msg-${p}-${session}-${t}`, t }))))
written.push(path)
}
}
return written
}
/// A resumed Claude session: the new transcript restates the original's assistant
/// messages verbatim (same message ids) before adding its own. Cross-file dedup
/// means whichever file is installed FIRST keeps those turns and the other loses
/// them, so this fixture is only stable if worker results are installed in the
/// serial order — and it is the only fixture that drives the discard/re-parse path,
/// since a worker parses against an empty dedup set and cannot see the overlap.
async function writeResumedPair(claudeDir: string, tag: string, originalName: string, resumedName: string): Promise<void> {
const dir = join(claudeDir, 'projects', `-tmp-${tag}`)
await mkdir(dir, { recursive: true })
const shared = range(6).map(t => ({ id: `${tag}-m${t}`, t }))
await writeFile(join(dir, `${originalName}.jsonl`), sessionLines(tag, originalName, shared))
await writeFile(
join(dir, `${resumedName}.jsonl`),
sessionLines(tag, resumedName, [...shared, ...range(4, 6).map(t => ({ id: `${tag}-n${t}`, t }))]),
)
}
type CodexTask = { n: number; at: string }
/// One Codex rollout: session_meta plus a complete task cycle per entry. A
/// token_count dedup key is namespaced by the FORK PARENT when there is one and
/// keyed on the cumulative token breakdown, so a fork restating a parent's tasks
/// emits exactly the parent's keys — the cross-file overlap that only the
/// install-order check can resolve. The replayed tasks are timestamped well past
/// `metaTs + 5s` on purpose: inside that window the parser drops replays outright
/// and the dedup path would never be reached.
function codexRollout(sessionId: string, cwd: string, tasks: CodexTask[], forkedFrom?: string, metaTs = '2026-05-04T09:00:00.000Z'): string {
const lines = [JSON.stringify({
type: 'session_meta',
timestamp: metaTs,
payload: {
cwd, originator: 'codex-cli', session_id: sessionId, model: 'gpt-5.3-codex',
...(forkedFrom ? { forked_from_id: forkedFrom } : {}),
},
})]
lines.push(...codexTaskLines(tasks))
return lines.join('\n') + '\n'
}
function codexTaskLines(tasks: CodexTask[]): string[] {
const lines: string[] = []
for (const { n, at } of tasks) {
const ts = (s: number) => new Date(Date.parse(at) + s * 1000).toISOString()
lines.push(
JSON.stringify({ type: 'event_msg', timestamp: ts(0), payload: { type: 'task_started' } }),
JSON.stringify({ type: 'response_item', timestamp: ts(1), payload: { type: 'message', role: 'user', content: [{ type: 'input_text', text: `codex task ${n}` }] } }),
JSON.stringify({ type: 'response_item', timestamp: ts(2), payload: { type: 'function_call', name: 'shell', call_id: `c${n}`, arguments: JSON.stringify({ command: `ls ${n}` }) } }),
JSON.stringify({ type: 'response_item', timestamp: ts(3), payload: { type: 'function_call_output', call_id: `c${n}` } }),
JSON.stringify({ type: 'event_msg', timestamp: ts(4), payload: { type: 'patch_apply_end', success: true, changes: { [`/tmp/cx/f${n}.ts`]: { unified_diff: '@@ -1 +1,2 @@\n-old\n+new\n+extra\n' } } } }),
JSON.stringify({ type: 'response_item', timestamp: ts(5), payload: { type: 'message', role: 'assistant', content: [{ type: 'output_text', text: 'y'.repeat(120) }] } }),
JSON.stringify({
type: 'event_msg', timestamp: ts(6),
payload: {
type: 'token_count',
info: {
last_token_usage: { input_tokens: 100, cached_input_tokens: 20, output_tokens: 50, reasoning_output_tokens: 10, total_tokens: 180 },
total_token_usage: { input_tokens: 100 * n, cached_input_tokens: 20 * n, output_tokens: 50 * n, reasoning_output_tokens: 10 * n, total_tokens: 160 * n },
},
},
}),
JSON.stringify({ type: 'event_msg', timestamp: ts(7), payload: { type: 'task_complete', duration_ms: 4000 } }),
)
}
return lines
}
async function writeCodexRollout(codexHome: string, day: string, name: string, body: string): Promise<string> {
const dir = join(codexHome, 'sessions', '2026', '05', day)
await mkdir(dir, { recursive: true })
const path = join(dir, `rollout-${name}.jsonl`)
await writeFile(path, body)
return path
}
/// A parent rollout and a fork that replays its tasks before adding its own.
/// `parentFirst` flips which file is created first, since discovery follows
/// directory order: the shared keys must land on whichever file the SERIAL loop
/// reaches first, in either order.
async function writeForkedCodexPair(codexHome: string, day: string, tag: string, parentFirst: boolean): Promise<void> {
const shared = [1, 2, 3].map(n => ({ n, at: `2026-05-04T09:${String(10 + n).padStart(2, '0')}:00.000Z` }))
const parent = codexRollout(`${tag}-parent`, `/tmp/cx${tag}`, shared)
const fork = codexRollout(
`${tag}-fork`,
`/tmp/cx${tag}`,
[...shared.map(t => ({ ...t, at: `2026-05-04T10:${String(10 + t.n).padStart(2, '0')}:00.000Z` })), { n: 4, at: '2026-05-04T10:30:00.000Z' }],
`${tag}-parent`,
)
const order: Array<[string, string]> = parentFirst
? [[`${tag}-a-parent`, parent], [`${tag}-b-fork`, fork]]
: [[`${tag}-a-fork`, fork], [`${tag}-b-parent`, parent]]
for (const [name, body] of order) await writeCodexRollout(codexHome, day, name, body)
}
/// Cache shard file names carry a random nonce, so compare bodies keyed by
/// `<provider>.<month>` instead of by file name.
async function shardBodies(cacheDir: string): Promise<Record<string, string>> {
const dir = join(cacheDir, 'session-cache.v9')
const out: Record<string, string> = {}
for (const name of (await readdir(dir).catch(() => []))) {
if (name === 'envelope.json' || !name.endsWith('.json')) continue
const key = name.split('.').slice(0, 2).join('.')
out[key] = createHash('sha256').update(await readFile(join(dir, name))).digest('hex')
}
return out
}
/// The Codex incremental cache is a single JSON file; both runs read the same
/// rollouts, so it must come out identical byte for byte.
async function codexResults(cacheDir: string): Promise<string | null> {
return readFile(join(cacheDir, 'codex-results.json'), 'utf-8').catch(() => null)
}
function runCli(args: string[], home: string, extraEnv: Record<string, string>) {
return spawnSync(process.execPath, ['--import', 'tsx', 'src/cli.ts', ...args], {
cwd: process.cwd(),
env: {
...process.env,
CLAUDE_CONFIG_DIR: join(home, '.claude'),
CODEX_HOME: join(home, '.codex'),
CODEBURN_CACHE_DIR: join(home, '.cache', 'codeburn'),
HOME: home,
TZ: 'UTC',
...extraEnv,
},
encoding: 'utf-8',
timeout: 60_000,
})
}
function stripVolatile(payload: unknown): unknown {
if (Array.isArray(payload)) return payload.map(stripVolatile)
if (payload && typeof payload === 'object') {
return Object.fromEntries(
Object.entries(payload as Record<string, unknown>)
.filter(([k]) => !k.toLowerCase().startsWith('generated'))
.map(([k, v]) => [k, stripVolatile(v)]),
)
}
if (typeof payload === 'number') return Math.round(payload * 1e9) / 1e9
return payload
}
describe('parallel cold parse', () => {
let home: string
beforeEach(async () => {
home = await mkdtemp(join(tmpdir(), 'cb-cold-'))
})
afterEach(async () => {
await rm(home, { recursive: true, force: true })
})
/// Both runs read the SAME corpus, so the absolute paths embedded in the cache
/// shards match and the bodies can be compared byte for byte.
async function bothWays(extraParallelEnv: Record<string, string> = {}) {
const serialCache = join(home, 'cache-serial')
const parallelCache = join(home, 'cache-parallel')
const args = ['status', '--format', 'menubar-json']
const serial = runCli(args, home, { CODEBURN_PARSE_WORKERS: '0', CODEBURN_CACHE_DIR: serialCache })
const parallel = runCli(args, home, { CODEBURN_PARSE_WORKERS: '3', CODEBURN_CACHE_DIR: parallelCache, ...extraParallelEnv })
expect(serial.status, serial.stderr).toBe(0)
expect(parallel.status, parallel.stderr).toBe(0)
expect(stripVolatile(JSON.parse(parallel.stdout))).toEqual(stripVolatile(JSON.parse(serial.stdout)))
const serialShards = await shardBodies(serialCache)
expect(Object.keys(serialShards).length).toBeGreaterThan(0)
expect(await shardBodies(parallelCache)).toEqual(serialShards)
// Byte-compared, not deep-equalled: the Codex cache is written entry by entry
// in install order, so the key order is itself a claim about that order.
expect(await codexResults(parallelCache)).toEqual(await codexResults(serialCache))
return parallel
}
// The whole point of the feature: threads may only ever be a speed change.
it('produces an identical payload and byte-identical cache shards with and without workers', async () => {
await writeCorpus(join(home, '.claude'), 4, 12)
await bothWays()
})
// Resumed sessions in both filename orders: the restating file sorts after the
// original in one project and before it in the other, so install order decides
// which file keeps the shared turns either way. Out-of-order installation, or
// any attempt to patch overlapping turns out of a worker result instead of
// discarding the whole file, changes the answer.
it("matches the serial parse when files restate each other's message ids", async () => {
const claude = join(home, '.claude')
await writeResumedPair(claude, 'fwd', '00000000-aaaa-bbbb-cccc-000000000000', '99999999-aaaa-bbbb-cccc-000000000000')
await writeResumedPair(claude, 'rev', '99999999-dddd-bbbb-cccc-000000000000', '00000000-dddd-bbbb-cccc-000000000000')
const parallel = await bothWays({ CODEBURN_VERBOSE: '1' })
// Pin that the discard path actually ran rather than passing by luck.
const overlaps = [...parallel.stderr.matchAll(/(\d+)\/\d+ results re-parsed in-process on id overlap/g)]
.reduce((n, m) => n + Number(m[1]), 0)
expect(overlaps).toBeGreaterThan(0)
})
// Codex is the bigger half of a real cold parse, and its cross-file dedup is
// stronger than Claude's: a forked rollout replays its parent's token_count
// history under the PARENT's key namespace, so two files claim the same keys
// outright. Both fork orders are present, so install order decides which file
// keeps the shared tasks either way.
it('matches the serial parse for a mixed Claude + Codex corpus with forked rollouts', async () => {
await writeCorpus(join(home, '.claude'), 2, 6)
const codex = join(home, '.codex')
await writeForkedCodexPair(codex, '04', 'fwd', true)
await writeForkedCodexPair(codex, '05', 'rev', false)
for (const n of range(4)) {
await writeCodexRollout(codex, '06', `plain-${n}`, codexRollout(`plain-${n}`, `/tmp/cx${n}`, [1, 2].map(t => ({ n: t, at: `2026-05-06T0${n}:${t}0:00.000Z` }))))
}
const parallel = await bothWays({ CODEBURN_VERBOSE: '1' })
expect(parallel.stderr).toContain('codeburn: codex parse workers=3')
// Pin that the codex-cache comparison in bothWays was not vacuous.
expect(await codexResults(join(home, 'cache-parallel'))).toContain('rollout-')
// Pin that the codex discard path actually ran rather than passing by luck.
const codexDiscards = [...parallel.stderr.matchAll(/codex parse workers done, (\d+)\/\d+ results/g)]
.reduce((n, m) => n + Number(m[1]), 0)
expect(codexDiscards).toBeGreaterThan(0)
})
// Workers only ever run WHOLE-file decodes. A rollout that grew by a few KB is
// resumed from its last task boundary in-process: a thread hop would cost more
// than it saves, and the resume state lives in the parent's codex cache.
it('never hands a resumable rollout to a worker', async () => {
const codex = join(home, '.codex')
const path = await writeCodexRollout(codex, '04', 'grow', codexRollout('grow', '/tmp/cxg', [1, 2, 3].map(n => ({ n, at: `2026-05-04T09:${n}0:00.000Z` }))))
const cache = join(home, 'cache-inc')
const args = ['status', '--format', 'menubar-json']
const cold = runCli(args, home, { CODEBURN_PARSE_WORKERS: '3', CODEBURN_CACHE_DIR: cache, CODEBURN_VERBOSE: '1' })
expect(cold.status, cold.stderr).toBe(0)
expect(cold.stderr).toContain('codeburn: codex parse workers=3')
await appendFile(path, codexTaskLines([{ n: 4, at: '2026-05-04T09:40:00.000Z' }]).join('\n') + '\n')
const warm = runCli(args, home, { CODEBURN_PARSE_WORKERS: '3', CODEBURN_CACHE_DIR: cache, CODEBURN_VERBOSE: '1' })
expect(warm.status, warm.stderr).toBe(0)
expect(warm.stderr).toContain('codeburn: codex parse workers=0 (no full parses pending)')
})
})
describe('ParseWorkerPool', () => {
let home: string
let files: string[]
let codexPath: string
let codexSource: SessionSource
beforeEach(async () => {
clearSessionCache()
home = await mkdtemp(join(tmpdir(), 'cb-pool-'))
files = await writeCorpus(join(home, '.claude'), 2, 4)
codexPath = await writeCodexRollout(
join(home, '.codex'), '04', 'pool',
codexRollout('pool-1', '/tmp/cx', [1, 2].map(n => ({ n, at: `2026-05-04T09:${n}0:00.000Z` }))),
)
codexSource = { provider: 'codex', path: codexPath, project: 'tmp-cx' }
process.env['CLAUDE_CONFIG_DIR'] = join(home, '.claude')
// Isolated so a parse in this process can never walk the developer's own
// ~/.codex, and so the pool the Codex path opens is covered by the leak check.
process.env['CODEX_HOME'] = join(home, '.codex')
process.env['CODEBURN_CACHE_DIR'] = join(home, '.cache', 'codeburn')
})
afterEach(async () => {
clearSessionCache()
delete process.env['CODEBURN_PARSE_WORKERS']
delete process.env['CODEX_HOME']
await rm(home, { recursive: true, force: true })
})
function liveWorkers(): number {
return process.getActiveResourcesInfo().filter(r => r === 'Worker').length
}
it('returns results in submission order and terminates every thread on close', async () => {
const before = liveWorkers()
const pool = new ParseWorkerPool(3)
const results = []
for await (const r of parseFilesInOrder<ClaudeWorkerParse>(pool, files.map(filePath => ({ kind: 'claude' as const, filePath })))) results.push(r)
await pool.close()
expect(results).toHaveLength(files.length)
for (const r of results) expect(r.ok).toBe(true)
// Each fixture session's first turn names its own project, which pins the
// yielded order to the submitted order rather than to completion order.
const projects = results.map(r => (r.ok && r.parsed ? r.parsed.turns[0]?.userMessage : undefined))
expect(projects).toEqual(files.map((_, i) => `task 0 in ${Math.floor(i / 4)}`))
expect(liveWorkers()).toBe(before)
})
// A worker that cannot answer must hand the file back, never drop it: the
// caller's fallback is an in-process parse, and it has to land on the same
// result the worker would have produced.
it('reports failures instead of throwing, and the serial fallback matches', async () => {
const pool = new ParseWorkerPool(1)
const fromWorker = await pool.submit<ClaudeWorkerParse>({ kind: 'claude', filePath: files[0]! })
await pool.close()
const afterClose = await pool.submit({ kind: 'claude', filePath: files[1]! })
expect(afterClose.ok).toBe(false)
const serial = await parseClaudeFileFull(files[0]!, new Set<string>())
expect(fromWorker.ok).toBe(true)
if (!fromWorker.ok || !fromWorker.parsed) throw new Error('expected a parsed result')
const { msgIds, path, ...worker } = fromWorker.parsed
expect(msgIds.length).toBeGreaterThan(0)
// Echoed back so the parent can assert the positional worker/file pairing.
expect(path).toBe(files[0])
expect(worker).toEqual(JSON.parse(JSON.stringify(serial)))
})
// Same contract for a Codex rollout: the off-thread decode is the serial
// decode, including the cache entry the parent has to install, and a worker
// that cannot answer hands the file back for an in-process parse.
it('decodes a codex rollout off-thread exactly as the serial path does', async () => {
const before = liveWorkers()
const pool = new ParseWorkerPool(1)
const fromWorker = await pool.submit<CodexFullParse & { keys: string[] }>({ kind: 'codex', source: codexSource })
await pool.close()
expect(liveWorkers()).toBe(before)
const afterClose = await pool.submit({ kind: 'codex', source: codexSource })
expect(afterClose.ok).toBe(false)
const seen = new Set<string>()
const serial = await parseCodexFileFull(codexSource, seen)
if (!fromWorker.ok || !fromWorker.parsed) throw new Error('expected a parsed result')
const { keys, path, ...worker } = fromWorker.parsed
expect(keys.length).toBeGreaterThan(0)
expect(new Set(keys)).toEqual(seen)
// Echoed back so the parent can assert the positional worker/file pairing.
expect(path).toBe(codexPath)
expect(worker).toEqual(JSON.parse(JSON.stringify(serial)))
// The decode itself must never have touched the codex cache file.
expect(await codexResults(join(home, '.cache', 'codeburn'))).toBeNull()
})
// The resident `serve` child parses over and over in one process; a thread
// that outlives its parse would accumulate across requests.
it('leaves no live worker behind after back-to-back parses', async () => {
const before = liveWorkers()
process.env['CODEBURN_PARSE_WORKERS'] = '2'
await parseAllSessions()
expect(liveWorkers()).toBe(before)
clearSessionCache()
await parseAllSessions()
expect(liveWorkers()).toBe(before)
})
})

6
tests/parser-antigravity-timestamp.test.ts
View file

@ -7,7 +7,7 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { getDateRange } from '../src/cli-date.js'
import { clearSessionCache, parseAllSessions } from '../src/parser.js'
import { sessionCachePath } from '../src/session-cache.js'
import { readCacheOnDisk } from './fixtures/session-cache-io.js'
import { isSqliteAvailable } from '../src/sqlite.js'
import type { DateRange } from '../src/types.js'
@ -40,9 +40,7 @@ function createGenMetadataDb(dbPath: string, fixture: Fixture): void {
}
async function cachedAntigravityTurns(cacheDir: string, dbPath: string): Promise<Array<{ timestamp: string }>> {
const saved = JSON.parse(await readFile(sessionCachePath(), 'utf-8')) as {
providers: Record<string, { files: Record<string, { turns: Array<{ timestamp: string }> }> }>
}
const saved = await readCacheOnDisk()
return saved.providers['antigravity']?.files[dbPath]?.turns ?? []
}

6
tests/parser-cache-refresh-timeout.test.ts
View file

@ -8,7 +8,7 @@ vi.mock('../src/cache-refresh-lock.js', () => ({
}))
import { clearSessionCache, isSessionHydrationComplete, parseAllSessions } from '../src/parser.js'
import { sessionCachePath } from '../src/session-cache.js'
import { cacheDirSnapshot } from './fixtures/session-cache-io.js'
let root: string
let sessionPath: string
@ -52,12 +52,12 @@ describe('parseAllSessions warm refresh timeout', () => {
it('serves the prior complete snapshot and leaves the holder cache untouched', async () => {
await writeSession(50)
expect(output(await parseAllSessions(undefined, 'claude'))).toBe(50)
const before = await readFile(sessionCachePath(), 'utf-8')
const before = await cacheDirSnapshot()
await writeSession(5000)
clearSessionCache()
expect(output(await parseAllSessions(undefined, 'claude'))).toBe(50)
expect(await readFile(sessionCachePath(), 'utf-8')).toBe(before)
expect(await cacheDirSnapshot()).toBe(before)
})
// The snapshot a timed-out refresh serves is only as good as what has changed

111
tests/parser-classify-after-slice.test.ts Normal file
View file

@ -0,0 +1,111 @@
import { afterEach, beforeEach, expect, it } from 'vitest'
import { mkdtemp, mkdir, writeFile, rm } from 'fs/promises'
import { join } from 'path'
import { tmpdir } from 'os'
import { parseAllSessions, filterProjectsByDateRange, clearSessionCache } from '../src/parser.js'
import { loadPricing } from '../src/models.js'
import type { ClassifiedTurn, DateRange } from '../src/types.js'
// scanProjectDirs decides the date slice on the RAW cached turn and classifies
// only survivors. The classification itself must still see each surviving
// turn's COMPLETE call list, and the branch/PR carries must still run over the
// full ordered turn list — so this fixture puts the branch anchor and the PR
// reference before the range, and straddles the range start with a turn whose
// only Edit lands on the out-of-range side.
const SESSION = '22222222-2222-4222-8222-222222222222'
const CWD = '/tmp/slice-proj'
const BRANCH = 'feat/carry'
const PR = 'https://github.com/o/r/pull/42'
const RANGE: DateRange = {
start: new Date('2026-07-20T00:00:00.000Z'),
end: new Date('2026-07-20T23:59:59.999Z'),
}
let tmpDir: string
beforeEach(async () => {
clearSessionCache()
tmpDir = await mkdtemp(join(tmpdir(), 'slice-'))
process.env['CLAUDE_CONFIG_DIR'] = join(tmpDir, 'claude')
process.env['CODEBURN_CACHE_DIR'] = join(tmpDir, 'cache')
})
afterEach(async () => {
clearSessionCache()
delete process.env['CLAUDE_CONFIG_DIR']
delete process.env['CODEBURN_CACHE_DIR']
await rm(tmpDir, { recursive: true, force: true })
})
function user(ts: string, content: string): string {
return JSON.stringify({ type: 'user', sessionId: SESSION, timestamp: ts, cwd: CWD, gitBranch: BRANCH, message: { role: 'user', content } })
}
function assistant(ts: string, id: string, tools: string[]): string {
return JSON.stringify({
type: 'assistant', sessionId: SESSION, timestamp: ts, cwd: CWD, gitBranch: BRANCH,
message: {
id, type: 'message', role: 'assistant', model: 'claude-sonnet-4-5',
content: tools.map((name, i) => ({ type: 'tool_use', id: `${id}_${i}`, name, input: {} })),
usage: { input_tokens: 100, output_tokens: 50 },
},
})
}
async function writeTranscript(): Promise<void> {
const projDir = join(tmpDir, 'claude', 'projects', 'slice-proj')
await mkdir(projDir, { recursive: true })
await writeFile(join(projDir, `${SESSION}.jsonl`), [
// Before the range: the only turn carrying the branch (the cache elides an
// unchanged branch on later turns) and the only PR reference.
user('2026-07-19T09:00:00.000Z', `please finish ${PR}`),
assistant('2026-07-19T09:00:05.000Z', 'm1', ['Read']),
// Straddles the range start: the Edit is on the out-of-range call.
user('2026-07-19T23:50:00.000Z', 'keep going overnight'),
assistant('2026-07-19T23:50:10.000Z', 'm2', ['Edit']),
assistant('2026-07-20T00:10:00.000Z', 'm3', ['Read']),
// Fully inside the range.
user('2026-07-20T10:00:00.000Z', 'what changed?'),
assistant('2026-07-20T10:00:05.000Z', 'm4', ['Read']),
].join('\n') + '\n', 'utf-8')
}
function shape(turn: ClassifiedTurn): unknown {
return {
timestamp: turn.timestamp,
category: turn.category,
subCategory: turn.subCategory,
retries: turn.retries,
hasEdits: turn.hasEdits,
gitBranch: turn.gitBranch,
prRefs: turn.prRefs,
calls: turn.assistantCalls.map(c => c.timestamp),
}
}
it('slices before classifying without changing carried branch, PR, or turn classification', async () => {
await loadPricing()
await writeTranscript()
const sliced = await parseAllSessions(RANGE, 'claude')
// Reference: the old order — classify every turn from the full history, then
// apply the same range slice afterwards.
clearSessionCache()
const reference = filterProjectsByDateRange(await parseAllSessions(undefined, 'claude'), RANGE)
const session = sliced[0]!.sessions[0]!
expect(session.turns.map(shape)).toEqual(reference[0]!.sessions[0]!.turns.map(shape))
// The branch anchor and the PR reference both live before the range.
expect(session.everHadBranch).toBe(true)
expect(session.turns.every(t => t.gitBranch === BRANCH)).toBe(true)
expect(session.prRefsAtRangeStart).toEqual([PR])
// The straddling turn kept only its in-range call, but was classified from
// the complete call list — the Edit it dropped still counts.
const straddled = session.turns[0]!
expect(straddled.assistantCalls.map(c => c.timestamp)).toEqual(['2026-07-20T00:10:00.000Z'])
expect(straddled.hasEdits).toBe(true)
})

9
tests/parser-gemini-cache.test.ts
View file

@ -5,7 +5,8 @@ import { join } from 'path'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { clearSessionCache, parseAllSessions } from '../src/parser.js'
import { CACHE_VERSION, computeEnvFingerprint, sessionCachePath } from '../src/session-cache.js'
import { CACHE_VERSION, computeEnvFingerprint, type SessionCache } from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
import type { DateRange } from '../src/types.js'
let home: string
@ -54,7 +55,7 @@ describe('Gemini session cache migration', () => {
}))
const fileStat = await stat(sessionPath)
await writeFile(sessionCachePath(), JSON.stringify({
await writeCacheOnDisk({
version: CACHE_VERSION,
providers: {
gemini: {
@ -97,7 +98,7 @@ describe('Gemini session cache migration', () => {
},
},
},
}))
} as SessionCache)
const range: DateRange = {
start: new Date('2026-05-16T00:00:00.000Z'),
@ -117,7 +118,7 @@ describe('Gemini session cache migration', () => {
'gemini:gemini-session-1:g2',
])
const savedCache = JSON.parse(await readFile(sessionCachePath(), 'utf-8'))
const savedCache = await readCacheOnDisk() as any
const savedKeys = savedCache.providers.gemini.files[sessionPath].turns.flatMap((turn: { calls: Array<{ deduplicationKey: string }> }) =>
turn.calls.map(call => call.deduplicationKey),
)

19
tests/parser-hydration-lock.test.ts
View file

@ -6,12 +6,13 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { existsSync } from 'fs'
import { mkdir, mkdtemp, rm, unlink, writeFile, readFile } from 'fs/promises'
import { mkdir, mkdtemp, rm, unlink, writeFile } from 'fs/promises'
import { tmpdir } from 'os'
import { join } from 'path'
import { clearSessionCache, parseAllSessions } from '../src/parser.js'
import { sessionCachePath } from '../src/session-cache.js'
import { sessionCacheDir } from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
let tmpHome: string
let cacheDir: string
@ -71,17 +72,16 @@ describe('parseAllSessions hydration lock', () => {
await writeClaudeSession(50)
expect(totalOutput(await parseAllSessions(undefined, 'claude'))).toBe(50)
const warm = JSON.parse(await readFile(sessionCachePath(), 'utf-8'))
for (const section of Object.values(warm.providers) as Array<{ files: Record<string, { turns: Array<{ calls: Array<{ usage: { outputTokens: number } }> }> }> }>) {
const tampered = await readCacheOnDisk()
for (const section of Object.values(tampered.providers)) {
for (const file of Object.values(section.files)) {
for (const turn of file.turns) for (const call of turn.calls) call.usage.outputTokens = 999
}
}
const tampered = JSON.stringify(warm)
// Go cold: remove the versioned cache and drop the in-memory cache so the
// next parse genuinely cold-starts and consults the lock.
await unlink(sessionCachePath())
await rm(sessionCacheDir(), { recursive: true })
clearSessionCache()
// A fresh lock held by another live process (pid 1 is always alive and is
@ -97,7 +97,7 @@ describe('parseAllSessions hydration lock', () => {
// The "first process" finishes: it leaves the warm (tampered) cache behind
// and releases the lock. The waiter wakes, reloads, and serves the cache.
await writeFile(sessionCachePath(), tampered)
await writeCacheOnDisk(tampered)
await unlink(lockPath())
const result = await promise
@ -119,8 +119,9 @@ describe('parseAllSessions hydration lock', () => {
expect(totalOutput(result)).toBe(50)
// Lock released in the finally.
expect(existsSync(lockPath())).toBe(false)
// The parse warmed the versioned cache.
expect(existsSync(sessionCachePath())).toBe(true)
// The parse warmed the versioned cache: the envelope is what publishes it,
// so the directory merely existing proves nothing.
expect(existsSync(join(sessionCacheDir(), 'envelope.json'))).toBe(true)
})
it('ignores a fresh lock whose pid is dead', async () => {

13
tests/parser-incremental-append.test.ts
View file

@ -20,7 +20,7 @@ vi.mock('../src/fs-utils.js', async (importOriginal) => {
})
import { parseAllSessions, clearSessionCache } from '../src/parser.js'
import { sessionCachePath } from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
import type { ProjectSummary } from '../src/types.js'
let tmpDir: string
@ -129,8 +129,8 @@ describe('incremental append parsing', () => {
await writeFile(sessionPath, baseLines().join('\n') + '\n')
await parseWith(warmCache)
const cachedOffset: number = JSON.parse(await readFile(sessionCachePath(), 'utf-8'))
.providers.claude.files[sessionPath].lastCompleteLineOffset
const cachedOffset = (await readCacheOnDisk())
.providers['claude']!.files[sessionPath]!.lastCompleteLineOffset!
expect(cachedOffset).toBeGreaterThan(0)
// 2) append a new complete turn plus a torn (invalid JSON, no newline) tail.
@ -317,10 +317,9 @@ describe('incremental append parsing', () => {
await parseWith(warmCache)
// Corrupt the persisted offset to point far beyond the file, then grow it.
const cachePath = sessionCachePath()
const cache = JSON.parse(await readFile(cachePath, 'utf-8'))
cache.providers.claude.files[sessionPath].lastCompleteLineOffset = 10_000_000
await writeFile(cachePath, JSON.stringify(cache))
const cache = await readCacheOnDisk()
cache.providers['claude']!.files[sessionPath]!.lastCompleteLineOffset = 10_000_000
await writeCacheOnDisk(cache)
await appendFile(sessionPath,
userLine('2026-05-01T13:00:00.000Z', 'grow the file') + '\n' +

168
tests/parser.test.ts
View file

@ -14,7 +14,8 @@ import { createRequire } from 'node:module'
import { isSqliteAvailable } from '../src/sqlite.js'
import { clearSessionCache, parseAllSessions, setParseReuseValidator } from '../src/parser.js'
import { loadCache, saveCache, sessionCachePath } from '../src/session-cache.js'
import { loadCache, saveCache } from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
import type { SessionSource, SessionParser, ParsedProviderCall } from '../src/providers/types.js'
// ── Synthetic provider state ───────────────────────────────────────────────
@ -23,6 +24,7 @@ import type { SessionSource, SessionParser, ParsedProviderCall } from '../src/pr
let _synthSources: SessionSource[] = []
let _synthDurable = false
let _synthYields: ParsedProviderCall[] = []
let _synthOnParse: (() => void | Promise<void>) | null = null
vi.mock('../src/providers/index.js', async (importOriginal) => {
type Mod = typeof import('../src/providers/index.js')
@ -52,6 +54,7 @@ vi.mock('../src/providers/index.js', async (importOriginal) => {
createSessionParser(_s: SessionSource, _k: Set<string>): SessionParser {
return {
async *parse(): AsyncGenerator<ParsedProviderCall> {
await _synthOnParse?.()
for (const call of _synthYields) {
// Respect seenKeys so that when multiple sources share the same
// dedup key, only the first source yields it (mirrors real parsers).
@ -190,13 +193,16 @@ beforeEach(async () => {
_synthSources = []
_synthDurable = false
_synthYields = []
_synthOnParse = null
})
afterEach(async () => {
clearSessionCache()
setParseReuseValidator(null)
vi.unstubAllEnvs()
_synthSources = []
_synthOnParse = null
await rm(tmpHome, { recursive: true, force: true })
await rm(tmpCache, { recursive: true, force: true })
@ -444,12 +450,10 @@ describe('(f) durable orphans survive a parse-version bump', () => {
// Simulate the fingerprint a PREVIOUS release computed (any mismatching
// value takes the same code path as a real parse-version bump).
const { readFile, writeFile: writeFileFs } = await import('fs/promises')
const cachePath = sessionCachePath()
const disk = JSON.parse(await readFile(cachePath, 'utf-8')) as { providers: Record<string, { envFingerprint: string }> }
const disk = await readCacheOnDisk()
expect(disk.providers['copilot']).toBeDefined()
disk.providers['copilot']!.envFingerprint = '0000000000000000'
await writeFileFs(cachePath, JSON.stringify(disk), 'utf-8')
await writeCacheOnDisk(disk)
// First parse after the "upgrade": the orphan must still be counted and
// must survive in the rewritten cache, not be erased with the section.
@ -728,6 +732,75 @@ describe('(q) parse burst reuse (CODEBURN_PARSE_BURST_MS)', () => {
})
describe('(r) validated parse reuse (setParseReuseValidator)', () => {
it('falls back to the exact TTL when watcher coverage is unknown, but rejects dirty', async () => {
vi.stubEnv('CODEBURN_PARSE_BURST_MS', '0')
clearSessionCache()
const start = new Date(Date.now() - 60 * 60 * 1000)
const end = new Date()
const ts = new Date(Date.now() - 10 * 60 * 1000).toISOString()
const synthFile = join(tmpHome, 'synth-unknown-exact.txt')
await writeFile(synthFile, 'first input')
_synthSources = [{ path: synthFile, project: 'p', provider: 'test-synthetic' }]
_synthYields = [{
provider: 'test-synthetic', model: 'synth-model',
inputTokens: 1, outputTokens: 5, cacheCreationInputTokens: 0, cacheReadInputTokens: 0,
cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0,
costUSD: 0, costIsEstimated: false, tools: [], bashCommands: [], skills: [],
timestamp: ts, speed: 'standard', deduplicationKey: 'synth-unknown-exact-1', userMessage: 'hi', sessionId: 'sue-1',
}] as never
expect(totalOutput(await parseAllSessions({ start, end }, 'test-synthetic'))).toBe(5)
_synthYields = [..._synthYields, {
...( _synthYields[0] as object ), deduplicationKey: 'synth-unknown-exact-2', outputTokens: 7,
}] as never
await writeFile(synthFile, 'second input with changed fingerprint')
// An unhealthy/pre-arm watcher cannot extend freshness, but it must retain
// the normal exact-key TTL instead of forcing a full rescan every request.
setParseReuseValidator(() => 'unknown')
expect(totalOutput(await parseAllSessions({ start, end }, 'test-synthetic'))).toBe(5)
// The same entry must be rejected immediately once a real change is known.
setParseReuseValidator(() => 'dirty')
expect(totalOutput(await parseAllSessions({ start, end }, 'test-synthetic'))).toBe(12)
})
it('falls back to the short burst when watcher coverage is unknown, but dirty wins inside it', async () => {
vi.stubEnv('CODEBURN_PARSE_BURST_MS', '10000')
clearSessionCache()
const start = new Date(Date.now() - 60 * 60 * 1000)
const firstEnd = new Date()
const ts = new Date(Date.now() - 10 * 60 * 1000).toISOString()
const synthFile = join(tmpHome, 'synth-unknown-burst.txt')
await writeFile(synthFile, 'first input')
_synthSources = [{ path: synthFile, project: 'p', provider: 'test-synthetic' }]
_synthYields = [{
provider: 'test-synthetic', model: 'synth-model',
inputTokens: 1, outputTokens: 5, cacheCreationInputTokens: 0, cacheReadInputTokens: 0,
cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0,
costUSD: 0, costIsEstimated: false, tools: [], bashCommands: [], skills: [],
timestamp: ts, speed: 'standard', deduplicationKey: 'synth-unknown-burst-1', userMessage: 'hi', sessionId: 'sub-1',
}] as never
expect(totalOutput(await parseAllSessions({ start, end: firstEnd }, 'test-synthetic'))).toBe(5)
_synthYields = [..._synthYields, {
...( _synthYields[0] as object ), deduplicationKey: 'synth-unknown-burst-2', outputTokens: 7,
}] as never
await writeFile(synthFile, 'second input with changed fingerprint')
setParseReuseValidator(() => 'unknown')
expect(totalOutput(await parseAllSessions(
{ start, end: new Date(firstEnd.getTime() + 100) },
'test-synthetic',
))).toBe(5)
setParseReuseValidator(() => 'dirty')
expect(totalOutput(await parseAllSessions(
{ start, end: new Date(firstEnd.getTime() + 200) },
'test-synthetic',
))).toBe(12)
})
it('reuses past the burst window while the validator reports quiet, never when dirty', async () => {
vi.stubEnv('CODEBURN_PARSE_BURST_MS', '1')
clearSessionCache()
@ -750,14 +823,14 @@ describe('(r) validated parse reuse (setParseReuseValidator)', () => {
// 1ms burst window has certainly elapsed; with a quiet validator the
// previous parse is still served (world changed, result must not).
await new Promise(r => setTimeout(r, 5))
setParseReuseValidator(() => true)
setParseReuseValidator(() => 'clean')
_synthYields = [..._synthYields, { ...( _synthYields[0] as object ), deduplicationKey: 'synth-val-2', outputTokens: 7 }] as never
await writeFile(synthFile, 'placeholder v2')
const second = await parseAllSessions({ start, end: new Date(Date.now() + 500) }, 'test-synthetic')
expect(totalOutput(second)).toBe(5)
// A dirty validator ends the reuse: fresh parse sees the new call.
setParseReuseValidator(() => false)
setParseReuseValidator(() => 'dirty')
const third = await parseAllSessions({ start, end: new Date(Date.now() + 1000) }, 'test-synthetic')
expect(totalOutput(third)).toBe(12)
@ -766,4 +839,85 @@ describe('(r) validated parse reuse (setParseReuseValidator)', () => {
_synthSources = []
_synthYields = []
})
it('rejects an exact-key memo when a root event arrived during its parse', async () => {
clearSessionCache()
const start = new Date(Date.now() - 60 * 60 * 1000)
const end = new Date()
const ts = new Date(Date.now() - 10 * 60 * 1000).toISOString()
const synthFile = join(tmpHome, 'synth-exact-event-during-parse.txt')
await writeFile(synthFile, 'first input')
_synthSources = [{ path: synthFile, project: 'p', provider: 'test-synthetic' }]
_synthYields = [{
provider: 'test-synthetic', model: 'synth-model',
inputTokens: 1, outputTokens: 5, cacheCreationInputTokens: 0, cacheReadInputTokens: 0,
cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0,
costUSD: 0, costIsEstimated: false, tools: [], bashCommands: [], skills: [],
timestamp: ts, speed: 'standard', deduplicationKey: 'synth-exact-event-1', userMessage: 'hi', sessionId: 'see-1',
}] as never
let rootEventAt = 0
setParseReuseValidator(sinceTs => rootEventAt === 0 || rootEventAt < sinceTs ? 'clean' : 'dirty')
_synthOnParse = async () => {
await new Promise(resolve => setTimeout(resolve, 10))
rootEventAt = Date.now()
await new Promise(resolve => setTimeout(resolve, 10))
}
const first = await parseAllSessions({ start, end }, 'test-synthetic')
expect(totalOutput(first)).toBe(5)
_synthOnParse = null
_synthYields = [..._synthYields, {
...( _synthYields[0] as object ), deduplicationKey: 'synth-exact-event-2', outputTokens: 7,
}] as never
await writeFile(synthFile, 'second input')
const second = await parseAllSessions({ start, end }, 'test-synthetic')
expect(totalOutput(second)).toBe(12)
})
it('does not bless a root event that arrived while the cached parse was running', async () => {
vi.stubEnv('CODEBURN_PARSE_BURST_MS', '1')
clearSessionCache()
const start = new Date(Date.now() - 60 * 60 * 1000)
const firstEnd = new Date()
const ts = new Date(Date.now() - 10 * 60 * 1000).toISOString()
const synthFile = join(tmpHome, 'synth-event-during-parse.txt')
await writeFile(synthFile, 'first input')
_synthSources = [{ path: synthFile, project: 'p', provider: 'test-synthetic' }]
_synthYields = [{
provider: 'test-synthetic', model: 'synth-model',
inputTokens: 1, outputTokens: 5, cacheCreationInputTokens: 0, cacheReadInputTokens: 0,
cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0,
costUSD: 0, costIsEstimated: false, tools: [], bashCommands: [], skills: [],
timestamp: ts, speed: 'standard', deduplicationKey: 'synth-event-1', userMessage: 'hi', sessionId: 'se-1',
}] as never
let rootEventAt = 0
_synthOnParse = async () => {
// Bracket the controlled event so it is strictly after parse start and
// strictly before completion, independent of same-millisecond clocks.
await new Promise(resolve => setTimeout(resolve, 10))
rootEventAt = Date.now()
await new Promise(resolve => setTimeout(resolve, 10))
}
const first = await parseAllSessions({ start, end: firstEnd }, 'test-synthetic')
expect(totalOutput(first)).toBe(5)
expect(rootEventAt).toBeGreaterThan(0)
_synthOnParse = null
// Outside the 1ms burst, old code validated against cachePut completion
// and reused stale output because the in-parse event appeared older. The
// parse-start timestamp makes the validator reject reuse and rescan.
await new Promise(resolve => setTimeout(resolve, 5))
setParseReuseValidator(sinceTs => rootEventAt < sinceTs ? 'clean' : 'dirty')
_synthYields = [..._synthYields, {
...( _synthYields[0] as object ), deduplicationKey: 'synth-event-2', outputTokens: 7,
}] as never
await writeFile(synthFile, 'second input')
const second = await parseAllSessions(
{ start, end: new Date(firstEnd.getTime() + 500) },
'test-synthetic',
)
expect(totalOutput(second)).toBe(12)
})
})

31
tests/providers/claude-config-dirs.test.ts
View file

@ -390,6 +390,37 @@ describe('claude provider — config.json claudeConfigDirs (menubar-driven)', ()
expect(paths).toContain(join(personal, 'projects', '-Users-you-app'))
})
it('invalidates the exact parse memo when config.json adds a Claude discovery root', async () => {
const work = await makeConfigDir('claude-work', [])
const personal = await makeConfigDir('claude-personal', [])
const slug = '-Users-you-shared-app'
const cwd = '/Users/you/shared-app'
await writeSession(work, slug, 'sess-work', [
summaryLine('sess-work', cwd),
userLine('u1', 'sess-work', cwd, 'hi from work'),
assistantLine('a1', 'u1', 'sess-work', cwd),
])
await writeSession(personal, slug, 'sess-personal', [
summaryLine('sess-personal', cwd),
userLine('u2', 'sess-personal', cwd, 'hi from personal'),
assistantLine('a2', 'u2', 'sess-personal', cwd),
])
await writeConfigJson([work])
const first = await parseAllSessions(undefined, 'claude')
expect(first.flatMap(project => project.sessions).map(session => session.sessionId)).toEqual(['sess-work'])
// Same argv/date range and unchanged env: only the effective roots sourced
// from config.json differ. A resident process must not return the exact-key
// memo populated by the first call.
await writeConfigJson([work, personal])
const second = await parseAllSessions(undefined, 'claude')
expect(second.flatMap(project => project.sessions).map(session => session.sessionId).sort()).toEqual([
'sess-personal',
'sess-work',
])
})
it('lets env CLAUDE_CONFIG_DIRS override config.json', async () => {
const fromEnv = await makeConfigDir('claude-env', ['-Users-you-app'])
const fromFile = await makeConfigDir('claude-file', ['-Users-you-app'])

8
tests/providers/codewhale.test.ts
View file

@ -4,7 +4,7 @@ import { tmpdir } from 'os'
import { join } from 'path'
import { clearSessionCache, parseAllSessions } from '../../src/parser.js'
import { sessionCachePath } from '../../src/session-cache.js'
import { readCacheOnDisk } from '../fixtures/session-cache-io.js'
import { MAX_SESSION_FILE_BYTES } from '../../src/fs-utils.js'
import { codewhale, createCodeWhaleProvider } from '../../src/providers/codewhale.js'
import type { ParsedProviderCall } from '../../src/providers/types.js'
@ -275,10 +275,8 @@ describe('codewhale provider', () => {
expect(first[0]!.totalCostUSD).toBeCloseTo(0.75)
expect(second[0]!.totalCostUSD).toBeCloseTo(0.75)
const cache = JSON.parse(await readFile(sessionCachePath(), 'utf-8')) as {
providers: { codewhale: { envFingerprint: string } }
}
expect(cache.providers.codewhale.envFingerprint).toMatch(/^[a-f0-9]{16}$/)
const cache = await readCacheOnDisk()
expect(cache.providers['codewhale']!.envFingerprint).toMatch(/^[a-f0-9]{16}$/)
})
it('exposes canonical model and tool display names', () => {

287
tests/providers/codex-resume.test.ts Normal file
View file

@ -0,0 +1,287 @@
// Codex rollouts are append-only and the active ones are huge, so a run that
// re-read a grown session from byte 0 paid for the whole file to pick up a few
// KB. The parser now restarts from the last task boundary it recorded. What has
// to hold: the resumed decode is byte-identical to a full re-parse, and it
// really does start at an offset rather than quietly re-reading everything.
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
import { appendFile, mkdtemp, mkdir, rm, writeFile } from 'fs/promises'
import { join } from 'path'
import { tmpdir } from 'os'
const readLineCalls: Array<{ filePath: string; startByteOffset?: number }> = []
vi.mock('../../src/fs-utils.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('../../src/fs-utils.js')>()
return {
...actual,
readSessionLines: (filePath: string, skip?: unknown, options?: { startByteOffset?: number }) => {
readLineCalls.push({ filePath, startByteOffset: options?.startByteOffset })
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return (actual.readSessionLines as any)(filePath, skip, options)
},
}
})
import { clearCodexMemCaches, flushCodexCache, withCodexCacheDirectory } from '../../src/codex-cache.js'
import { createCodexProvider } from '../../src/providers/codex.js'
import type { ParsedProviderCall } from '../../src/providers/types.js'
let tmpDir: string
let sessionPath: string
beforeEach(async () => {
tmpDir = await mkdtemp(join(tmpdir(), 'codex-resume-'))
readLineCalls.length = 0
})
afterEach(async () => {
await rm(tmpDir, { recursive: true, force: true })
})
function meta(): string {
return JSON.stringify({
type: 'session_meta',
timestamp: '2026-04-14T10:00:00Z',
payload: { cwd: '/Users/test/proj', originator: 'codex-cli', session_id: 'sess-1', model: 'gpt-5.3-codex' },
})
}
// One complete task: user turn, tools, an edit, an MCP call, usage, completion.
function task(n: number, cumulative: { input: number; cached: number; output: number; reasoning: number }): string[] {
const at = (s: number) => `2026-04-14T10:${String(n).padStart(2, '0')}:${String(s).padStart(2, '0')}Z`
return [
JSON.stringify({ type: 'event_msg', timestamp: at(0), payload: { type: 'task_started' } }),
JSON.stringify({
type: 'response_item', timestamp: at(1),
payload: { type: 'message', role: 'user', content: [{ type: 'input_text', text: `task ${n}` }] },
}),
JSON.stringify({
type: 'response_item', timestamp: at(2),
payload: { type: 'function_call', name: 'shell', call_id: `c${n}`, arguments: JSON.stringify({ command: `ls ${n}` }) },
}),
JSON.stringify({
type: 'response_item', timestamp: at(3),
payload: { type: 'function_call_output', call_id: `c${n}` },
}),
JSON.stringify({
type: 'event_msg', timestamp: at(4),
payload: {
type: 'patch_apply_end', success: n % 2 === 0,
changes: { [`/Users/test/proj/f${n}.ts`]: { unified_diff: '@@ -1 +1,2 @@\n-old\n+new\n+extra\n' } },
},
}),
JSON.stringify({
type: 'event_msg', timestamp: at(5),
payload: { type: 'mcp_tool_call_end', call_id: `m${n}`, invocation: { server: 'github', tool: 'list' }, duration_ms: 120, result: { Ok: {} } },
}),
JSON.stringify({
type: 'response_item', timestamp: at(6),
payload: { type: 'message', role: 'assistant', content: [{ type: 'output_text', text: 'x'.repeat(40) }] },
}),
JSON.stringify({
type: 'event_msg', timestamp: at(7),
payload: {
type: 'token_count',
info: {
last_token_usage: { input_tokens: 100, cached_input_tokens: 20, output_tokens: 50, reasoning_output_tokens: 10, total_tokens: 180 },
total_token_usage: {
input_tokens: cumulative.input, cached_input_tokens: cumulative.cached,
output_tokens: cumulative.output, reasoning_output_tokens: cumulative.reasoning,
total_tokens: cumulative.input + cumulative.output + cumulative.reasoning,
},
},
},
}),
JSON.stringify({ type: 'event_msg', timestamp: at(8), payload: { type: 'task_complete', duration_ms: 5000 } }),
]
}
function tasks(from: number, to: number): string[] {
const lines: string[] = []
for (let n = from; n <= to; n++) {
lines.push(...task(n, { input: 100 * n, cached: 20 * n, output: 50 * n, reasoning: 10 * n }))
}
return lines
}
async function writeRollout(lines: string[]): Promise<string> {
const dir = join(tmpDir, 'sessions', '2026', '04', '14')
await mkdir(dir, { recursive: true })
const path = join(dir, 'rollout-sess-1.jsonl')
await writeFile(path, lines.join('\n') + '\n')
return path
}
async function parse(cacheDir: string, codexDir = tmpDir): Promise<ParsedProviderCall[]> {
clearCodexMemCaches()
return withCodexCacheDirectory(cacheDir, async () => {
const provider = createCodexProvider(codexDir)
const sources = await provider.discoverSessions()
const seenKeys = new Set<string>()
const calls: ParsedProviderCall[] = []
for (const source of sources) {
for await (const call of provider.createSessionParser!(source, seenKeys).parse()) calls.push(call)
}
await flushCodexCache()
clearCodexMemCaches()
return calls
})
}
describe('codex incremental resume', () => {
it('resumes at a task boundary and matches a full re-parse exactly', async () => {
const warmCache = join(tmpDir, 'cache-warm')
const coldCache = join(tmpDir, 'cache-cold')
sessionPath = await writeRollout([meta(), ...tasks(1, 3)])
const first = await parse(warmCache)
expect(first.length).toBe(3)
await appendFile(sessionPath, tasks(4, 6).join('\n') + '\n')
readLineCalls.length = 0
const resumed = await parse(warmCache)
const resumeReads = readLineCalls.filter(c => c.filePath === sessionPath)
// The parse re-entered the file at a boundary rather than at byte 0.
expect(resumeReads.some(c => (c.startByteOffset ?? 0) > 0)).toBe(true)
expect(resumeReads.every(c => (c.startByteOffset ?? 0) > 0)).toBe(true)
// Byte-for-byte agreement with a decode that never saw a cache.
const full = await parse(coldCache)
expect(resumed.length).toBe(6)
expect(JSON.stringify(resumed)).toBe(JSON.stringify(full))
})
it('stays exact across successive appends, resuming from a resumed state', async () => {
const warmCache = join(tmpDir, 'cache-warm')
sessionPath = await writeRollout([meta(), ...tasks(1, 2)])
await parse(warmCache)
await appendFile(sessionPath, tasks(3, 4).join('\n') + '\n')
await parse(warmCache)
// A tail with no task boundary at all: the next run restarts from the same
// boundary and re-decodes the open task.
await appendFile(sessionPath, tasks(5, 5).slice(1).join('\n') + '\n')
const resumed = await parse(warmCache)
const full = await parse(join(tmpDir, 'cache-cold'))
expect(JSON.stringify(resumed)).toBe(JSON.stringify(full))
})
it('serves an unchanged file from the cache without reading it', async () => {
const cacheDir = join(tmpDir, 'cache')
sessionPath = await writeRollout([meta(), ...tasks(1, 2)])
const first = await parse(cacheDir)
readLineCalls.length = 0
const second = await parse(cacheDir)
expect(readLineCalls.filter(c => c.filePath === sessionPath)).toHaveLength(0)
expect(JSON.stringify(second)).toBe(JSON.stringify(first))
})
it('falls back to a full re-parse when the stored resume state is unusable', async () => {
const cacheDir = join(tmpDir, 'cache')
sessionPath = await writeRollout([meta(), ...tasks(1, 2)])
await parse(cacheDir)
const cachePath = join(cacheDir, 'codex-results.json')
const { readFile } = await import('fs/promises')
const raw = JSON.parse(await readFile(cachePath, 'utf-8'))
raw.files[sessionPath].resumeState = { garbage: true }
await writeFile(cachePath, JSON.stringify(raw))
const { clearCodexMemCaches } = await import('../../src/codex-cache.js')
clearCodexMemCaches()
await appendFile(sessionPath, tasks(3, 3).join('\n') + '\n')
readLineCalls.length = 0
const resumed = await parse(cacheDir)
expect(readLineCalls.filter(c => c.filePath === sessionPath).every(c => (c.startByteOffset ?? 0) === 0)).toBe(true)
const full = await parse(join(tmpDir, 'cache-cold'))
expect(JSON.stringify(resumed)).toBe(JSON.stringify(full))
})
})
// The resume snapshot has to carry EVERY field the decode reads from an earlier
// line. Missing one shows up only when the split lands between the line that
// sets it and the line that reads it — so split at every line boundary and
// require the resumed decode to equal a full one each time.
const J = (o: unknown) => JSON.stringify(o)
const ts = (n: number, s: number) => `2026-04-14T${String(10 + Math.floor(n / 60)).padStart(2, '0')}:${String(n % 60).padStart(2, '0')}:${String(s).padStart(2, '0')}Z`
function richTask(n: number, opts: { tokens?: false | 'empty'; reasoning?: number; model?: string; noComplete?: boolean } = {}): string[] {
const lines: string[] = [J({ type: 'event_msg', timestamp: ts(n, 0), payload: { type: 'task_started' } })]
if (opts.model) lines.push(J({ type: 'turn_context', timestamp: ts(n, 0), payload: { model: opts.model } }))
lines.push(
J({ type: 'response_item', timestamp: ts(n, 1), payload: { type: 'message', role: 'user', content: [{ type: 'input_text', text: `please do task ${n} with some length` }] } }),
J({ type: 'response_item', timestamp: ts(n, 2), payload: { type: 'function_call', name: 'shell', call_id: `c${n}`, arguments: J({ command: `ls ${n}` }) } }),
J({ type: 'response_item', timestamp: ts(n, 3), payload: { type: 'function_call_output', call_id: `c${n}` } }),
J({ type: 'event_msg', timestamp: ts(n, 4), payload: { type: 'patch_apply_end', success: n % 3 !== 0, changes: { [`/Users/test/proj/f${n}.ts`]: { unified_diff: '@@ -1 +1,2 @@\n-old\n+new\n+extra\n' } } } }),
J({ type: 'event_msg', timestamp: ts(n, 5), payload: { type: 'mcp_tool_call_end', call_id: `m${n}`, invocation: { server: 'github', tool: 'list' }, duration_ms: 120, result: { Ok: {} } } }),
J({ type: 'response_item', timestamp: ts(n, 6), payload: { type: 'message', role: 'assistant', content: [{ type: 'output_text', text: 'y'.repeat(120) }] } }),
)
if (opts.tokens === 'empty') {
// No `info`: the estimated-usage path, which advances estCounter.
lines.push(J({ type: 'event_msg', timestamp: ts(n, 7), payload: { type: 'token_count' } }))
} else if (opts.tokens !== false) {
const c = { input: 100 * n, cached: 20 * n, output: 50 * n, reasoning: (opts.reasoning ?? 10) * n }
lines.push(J({ type: 'event_msg', timestamp: ts(n, 7), payload: { type: 'token_count', info: {
last_token_usage: { input_tokens: 100, cached_input_tokens: 20, output_tokens: 50, reasoning_output_tokens: opts.reasoning ?? 10, total_tokens: 180 },
total_token_usage: { input_tokens: c.input, cached_input_tokens: c.cached, output_tokens: c.output, reasoning_output_tokens: c.reasoning, total_tokens: c.input + c.output + c.reasoning },
} } }))
}
if (!opts.noComplete) lines.push(J({ type: 'event_msg', timestamp: ts(n, 8), payload: { type: 'task_complete', duration_ms: 5000 } }))
return lines
}
const RICH_LINES = [
J({ type: 'session_meta', timestamp: ts(0, 0), payload: { cwd: '/Users/test/proj', originator: 'codex-cli', session_id: 'sess-1', model: 'gpt-5.3-codex' } }),
...richTask(1),
...richTask(2, { tokens: 'empty' }),
...richTask(7, { tokens: 'empty' }),
...richTask(3, { model: 'gpt-5.3-codex-mini' }),
...richTask(4, { reasoning: 33 }),
...richTask(5, { noComplete: true }),
...richTask(6),
]
const FORK_LINES = [
J({ type: 'session_meta', timestamp: ts(0, 0), payload: { cwd: '/Users/test/proj', originator: 'codex-cli', session_id: 'sess-2', forked_from_id: 'sess-1', model: 'gpt-5.3-codex' } }),
// Parent history replayed inside the 5s fork cutoff: must stay skipped across a split.
...richTask(0).map(l => l.replace(/2026-04-14T10:00:0\d/g, '2026-04-14T10:00:01')),
...richTask(11),
...richTask(12, { tokens: 'empty' }),
]
describe('codex resume differential', () => {
async function rollout(lines: string[]): Promise<{ codexDir: string; path: string }> {
const codexDir = await mkdtemp(join(tmpdir(), 'codex-split-'))
const dir = join(codexDir, 'sessions', '2026', '04', '14')
await mkdir(dir, { recursive: true })
const path = join(dir, 'rollout-sess-1.jsonl')
await writeFile(path, lines.join('\n') + '\n')
return { codexDir, path }
}
async function assertEverySplitMatches(lines: string[]): Promise<void> {
const base = await rollout(lines)
const full = await parse(await mkdtemp(join(tmpdir(), 'codex-c-')), base.codexDir)
expect(full.length).toBeGreaterThan(0)
for (let k = 1; k < lines.length; k++) {
const split = await rollout(lines.slice(0, k))
const cacheDir = await mkdtemp(join(tmpdir(), 'codex-c-'))
await parse(cacheDir, split.codexDir)
await appendFile(split.path, lines.slice(k).join('\n') + '\n')
const resumed = await parse(cacheDir, split.codexDir)
expect(JSON.stringify(resumed), `split after line ${k}: ${lines[k - 1]!.slice(0, 80)}`).toBe(JSON.stringify(full))
}
}
it('matches a full re-parse at every line boundary of a rich session', async () => {
await assertEverySplitMatches(RICH_LINES)
}, 60_000)
it('matches a full re-parse at every line boundary of a forked session', async () => {
await assertEverySplitMatches(FORK_LINES)
}, 60_000)
})

33
tests/providers/cursor.test.ts
View file

@ -1,5 +1,5 @@
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises'
import { mkdtemp, mkdir, readFile, writeFile, rm } from 'node:fs/promises'
import { join } from 'node:path'
import { tmpdir } from 'node:os'
import { createRequire } from 'node:module'
@ -87,6 +87,37 @@ describe('cursor cache', () => {
const result = await readCachedResults('/nonexistent/path.db', new Date(0).toISOString())
expect(result).toBeNull()
})
it('honors CODEBURN_CACHE_DIR at call time', async () => {
const root = await mkdtemp(join(tmpdir(), 'cursor-cache-override-'))
const previousCacheDir = process.env['CODEBURN_CACHE_DIR']
const dbPath = join(root, 'state.vscdb')
const firstCacheDir = join(root, 'cache-a')
const secondCacheDir = join(root, 'cache-b')
const firstFloor = '2026-01-01T00:00:00.000Z'
const secondFloor = '2026-02-01T00:00:00.000Z'
await writeFile(dbPath, 'cursor-db-fixture')
try {
const { writeCachedResults } = await import('../../src/cursor-cache.js')
process.env['CODEBURN_CACHE_DIR'] = firstCacheDir
await writeCachedResults(dbPath, [], firstFloor)
process.env['CODEBURN_CACHE_DIR'] = secondCacheDir
await writeCachedResults(dbPath, [], secondFloor)
const firstPath = join(firstCacheDir, 'cursor-results.json')
const secondPath = join(secondCacheDir, 'cursor-results.json')
const first = JSON.parse(await readFile(firstPath, 'utf-8')) as { lookbackFloor: string }
const second = JSON.parse(await readFile(secondPath, 'utf-8')) as { lookbackFloor: string }
expect(first.lookbackFloor).toBe(firstFloor)
expect(second.lookbackFloor).toBe(secondFloor)
} finally {
if (previousCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']
else process.env['CODEBURN_CACHE_DIR'] = previousCacheDir
await rm(root, { recursive: true, force: true })
}
})
})
// Regression: Cursor renamed the per-workspace composer list key from

260
tests/serve-stdio.test.ts
View file

@ -1,6 +1,31 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest'
import { spawn, type ChildProcess } from 'child_process'
import { mkdir, readFile, writeFile } from 'fs/promises'
import { join } from 'path'
import { classifyRootReuse, createOutputMemoEntry } from '../src/serve.js'
it('timestamps a completed output memo before parsing begins', () => {
const parseStartedAt = 100
const rootEventDuringParseAt = 150
const parseCompletedAt = 200
const memo = createOutputMemoEntry(parseStartedAt, parseCompletedAt, 'output', 'config')
const rootsQuietSince = (sinceTs: number): boolean => rootEventDuringParseAt < sinceTs
// The old completion timestamp incorrectly made the in-parse event look
// older than the memo. The start timestamp keeps it visible to validation.
expect(rootsQuietSince(parseCompletedAt)).toBe(true)
expect(memo.createdAt).toBe(parseCompletedAt)
expect(memo.validatedFrom).toBe(parseStartedAt)
expect(rootsQuietSince(memo.validatedFrom)).toBe(false)
})
it('classifies watcher gaps as unknown without confusing them with dirty roots', () => {
expect(classifyRootReuse(100, { startedAt: 50, lastEventAt: 0, healthy: false })).toBe('unknown')
expect(classifyRootReuse(100, { startedAt: 150, lastEventAt: 0, healthy: true })).toBe('unknown')
expect(classifyRootReuse(100, { startedAt: 50, lastEventAt: 100, healthy: false })).toBe('dirty')
expect(classifyRootReuse(100, { startedAt: 50, lastEventAt: 100, healthy: true })).toBe('dirty')
expect(classifyRootReuse(100, { startedAt: 50, lastEventAt: 99, healthy: true })).toBe('clean')
})
// End-to-end protocol test for `codeburn serve --stdio` (the desktop app's
// resident query server). Runs the real entry through tsx against the
@ -10,6 +35,8 @@ describe('codeburn serve --stdio', () => {
let child: ChildProcess
let buffer = ''
const waiters = new Map<number, (msg: Record<string, unknown>) => void>()
const progressFrames = new Map<number, Array<Record<string, unknown>>>()
let configPath = ''
let readyResolve: () => void
const ready = new Promise<void>(resolve => { readyResolve = resolve })
@ -25,6 +52,23 @@ describe('codeburn serve --stdio', () => {
}
beforeAll(async () => {
const home = process.env['HOME']!
configPath = join(home, '.config', 'codeburn', 'config.json')
await mkdir(join(home, '.config', 'codeburn'), { recursive: true })
// Give the resident process one real provider root to arm. With no
// successfully armed roots, event-driven reuse correctly stays disabled.
await mkdir(join(home, '.claude', 'projects'), { recursive: true })
await writeFile(configPath, JSON.stringify({ currency: { code: 'USD' } }), 'utf8')
// Keep the EUR half of the config-freshness regression fully offline.
const cacheDir = join(home, '.cache', 'codeburn')
await mkdir(cacheDir, { recursive: true })
await writeFile(join(cacheDir, 'exchange-rate.json'), JSON.stringify({
timestamp: Date.now(),
code: 'EUR',
rate: 0.9,
}), 'utf8')
child = spawn(process.execPath, ['--import', 'tsx', join(__dirname, '..', 'src', 'cli.ts'), 'serve', '--stdio'], {
stdio: ['pipe', 'pipe', 'ignore'],
env: { ...process.env },
@ -40,6 +84,13 @@ describe('codeburn serve --stdio', () => {
let msg: Record<string, unknown>
try { msg = JSON.parse(line) } catch { continue }
if (msg['ready']) { readyResolve(); continue }
if (typeof msg['progress'] === 'string' && !('ok' in msg)) {
const id = msg['id'] as number
const frames = progressFrames.get(id) ?? []
frames.push(msg)
progressFrames.set(id, frames)
continue
}
const waiter = waiters.get(msg['id'] as number)
if (waiter) { waiters.delete(msg['id'] as number); waiter(msg) }
}
@ -82,9 +133,218 @@ describe('codeburn serve --stdio', () => {
expect(res['refused']).toBe(true)
})
it('refuses every optimize apply-only option without touching shell config or the action journal', async () => {
const home = process.env['HOME']!
const zshrc = join(home, '.zshrc')
const journal = join(home, '.config', 'codeburn', 'actions', 'journal.jsonl')
await writeFile(zshrc, '# user-owned\n', 'utf8')
// `optimize` is the only served command whose Commander definition also
// has mutation-capable options. The full request below used to execute a
// shell-config action inside the resident process.
const applied = await request(300, [
'optimize', '--apply', '--yes', '--only', 'bash-output-cap', '--period', 'today',
])
expect(applied).toMatchObject({ ok: false, refused: true })
// Keep the allowlist categorical: apply-only modifiers are not useful to
// a read query and must not become resident options on their own either.
for (const [id, args] of [
[301, ['optimize', '--yes']],
[302, ['optimize', '--dry-run']],
[303, ['optimize', '--only', 'bash-output-cap']],
] as const) {
expect(await request(id, [...args])).toMatchObject({ ok: false, refused: true })
}
expect(await readFile(zshrc, 'utf8')).toBe('# user-owned\n')
await expect(readFile(journal, 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
}, 60_000)
it('accepts the reviewed read-only option surface for every served command', async () => {
const commands: Array<[number, string[]]> = [
[310, ['status', '--format', 'json', '--period', 'today']],
[311, ['overview', '--period', 'today', '--no-color']],
[312, ['models', '--format', 'json', '--period', 'today', '--no-totals']],
[313, ['sessions', '--format', 'json', '--period', 'today', '--no-pager']],
[314, ['compare', '--format', 'json', '--period', 'today']],
[315, ['yield', '--format', 'json', '--period', 'today']],
[316, ['spend', '--format', 'flow-json', '--period', 'today']],
[317, ['optimize', '--format', 'json', '--period', 'today']],
[318, ['audit', '--format', 'json', '--period', 'today']],
]
for (const [id, args] of commands) {
expect(await request(id, args)).toMatchObject({ ok: true })
}
}, 60_000)
it('survives a malformed request line and keeps serving', async () => {
sendRaw('this is not json')
const res = await request(6, ['status', '--format', 'menubar-json', '--period', 'today'])
expect(res['ok']).toBe(true)
}, 60_000)
it('streams captured command stderr as protocol progress frames', async () => {
const res = await request(7, ['status', '--provider', 'definitely-not-a-real-provider'])
expect(res['ok']).toBe(false)
const frames = progressFrames.get(7) ?? []
expect(frames.length).toBeGreaterThan(0)
expect(frames.every(frame => Object.keys(frame).sort().join(',') === 'id,progress')).toBe(true)
expect(frames.map(frame => frame['progress']).join('')).toContain('unknown provider')
}, 60_000)
it('discovers a newly configured Claude root on identical resident argv', async () => {
const home = process.env['HOME']!
const rootA = join(home, 'claude-root-a')
const rootB = join(home, 'claude-root-b')
const slug = '-Users-test-shared-project'
const cwd = '/Users/test/shared-project'
const writeClaudeSession = async (root: string, sessionId: string, marker: string): Promise<void> => {
const projectDir = join(root, 'projects', slug)
await mkdir(projectDir, { recursive: true })
const lines = [
{
type: 'summary', summary: marker, leafUuid: `leaf-${marker}`, sessionId, cwd,
timestamp: '2026-08-12T10:00:00.000Z',
},
{
type: 'user', uuid: `user-${marker}`, sessionId, cwd,
timestamp: '2026-08-12T10:00:01.000Z', message: { role: 'user', content: marker },
},
{
type: 'assistant', uuid: `assistant-${marker}`, parentUuid: `user-${marker}`, sessionId, cwd,
timestamp: '2026-08-12T10:00:02.000Z',
message: {
id: `msg-${marker}`, type: 'message', role: 'assistant', model: 'claude-sonnet-4-6',
content: [{ type: 'text', text: 'reply' }], usage: { input_tokens: 100, output_tokens: 50 },
},
},
]
await writeFile(join(projectDir, `${sessionId}.jsonl`), lines.map(line => JSON.stringify(line)).join('\n'))
}
await writeClaudeSession(rootA, 'resident-session-a', 'a')
await writeClaudeSession(rootB, 'resident-session-b', 'b')
const args = ['sessions', '--period', 'lifetime', '--provider', 'claude', '--format', 'json', '--no-pager']
await writeFile(configPath, JSON.stringify({ claudeConfigDirs: [rootA] }), 'utf8')
const first = await request(200, args)
expect(first['ok']).toBe(true)
expect((JSON.parse(first['output'] as string) as Array<{ sessionId: string }>).map(row => row.sessionId)).toEqual([
'resident-session-a',
])
// Same command in the same process; only config.json adds root B.
await writeFile(configPath, JSON.stringify({ claudeConfigDirs: [rootA, rootB] }), 'utf8')
const second = await request(201, args)
expect(second['ok']).toBe(true)
expect((JSON.parse(second['output'] as string) as Array<{ sessionId: string }>).map(row => row.sessionId).sort()).toEqual([
'resident-session-a',
'resident-session-b',
])
// Keep the following currency-freshness regression self-contained.
await writeFile(configPath, JSON.stringify({ currency: { code: 'USD' } }), 'utf8')
}, 60_000)
it('invalidates identical-argv output memo immediately when config.json changes', async () => {
const args = ['status', '--format', 'menubar-json', '--period', 'week', '--no-optimize', '--no-timeline']
const usdConfig = JSON.stringify({ currency: { code: 'USD' } })
await writeFile(configPath, usdConfig, 'utf8')
let previous = await request(8, args)
expect(previous['ok']).toBe(true)
expect((JSON.parse(previous['output'] as string) as { currency: { code: string } }).currency.code).toBe('USD')
// Prove this argv is actually hitting the output memo before testing its
// invalidation. The root watchers arm asynchronously at serve startup, so
// allow a few requests until two byte-identical generated payloads arrive.
let memoized: Record<string, unknown> | null = null
for (let id = 9; id < 110; id++) {
await new Promise(resolve => setTimeout(resolve, 20))
const next = await request(id, args)
if (next['output'] === previous['output']) {
memoized = next
break
}
previous = next
}
expect(memoized).not.toBeNull()
// A byte-identical rewrite changes filesystem metadata but not effective
// configuration. The memo must survive it and return the exact generated
// payload, including the original volatile `generated` timestamp.
await new Promise(resolve => setTimeout(resolve, 20))
await writeFile(configPath, usdConfig, 'utf8')
const sameBytes = await request(110, args)
expect(sameBytes['ok']).toBe(true)
expect(sameBytes['output']).toBe(memoized!['output'])
// `generated` is minted per render, so an unchanged stamp is the proof
// that a memo hit returns the stored string instead of re-rendering.
const stamp = (res: Record<string, unknown>): string =>
(JSON.parse(res['output'] as string) as { generated: string }).generated
expect(stamp(sameBytes)).toBe(stamp(memoized!))
// Same byte length as USD: a size-only fingerprint would miss this.
await writeFile(configPath, JSON.stringify({ currency: { code: 'EUR' } }), 'utf8')
const fresh = await request(111, args)
expect(fresh['ok']).toBe(true)
expect((JSON.parse(fresh['output'] as string) as { currency: { code: string } }).currency.code).toBe('EUR')
expect(fresh['output']).not.toBe(memoized!['output'])
// Removing the configured currency is the USD reset contract. The serve
// process must reset its module-level currency state as well as invalidate
// the output memo, otherwise a long-lived child keeps rendering EUR.
await writeFile(configPath, '{}', 'utf8')
const reset = await request(112, args)
expect(reset['ok']).toBe(true)
expect((JSON.parse(reset['output'] as string) as {
currency: { code: string; rate: number }
}).currency).toMatchObject({ code: 'USD', rate: 1 })
}, 60_000)
it('exits on natural stdin EOF after arming a watcher for an existing Claude root', async () => {
const claudeRoot = join(process.env['HOME']!, 'claude-eof-root')
await mkdir(join(claudeRoot, 'projects'), { recursive: true })
const eofChild = spawn(process.execPath, ['--import', 'tsx', join(__dirname, '..', 'src', 'cli.ts'), 'serve', '--stdio'], {
stdio: ['pipe', 'pipe', 'ignore'],
env: { ...process.env, CLAUDE_CONFIG_DIR: claudeRoot },
})
let stdout = ''
const becameReady = new Promise<void>((resolve, reject) => {
eofChild.once('error', reject)
eofChild.stdout!.setEncoding('utf8')
eofChild.stdout!.on('data', (chunk: string) => {
stdout += chunk
if (stdout.split('\n').some(line => {
try { return (JSON.parse(line) as { ready?: boolean }).ready === true } catch { return false }
})) resolve()
})
eofChild.once('exit', (code, signal) => reject(new Error(`serve exited before ready: ${code ?? signal}`)))
})
const exited = new Promise<boolean>(resolve => eofChild.once('exit', () => resolve(true)))
let naturalExit = false
try {
await becameReady
// READY is intentionally emitted before provider probing; give the real
// watcher setup time to finish so the regression exercises its handle.
await new Promise(resolve => setTimeout(resolve, 500))
eofChild.stdin!.end()
naturalExit = await Promise.race([
exited,
new Promise<false>(resolve => setTimeout(() => resolve(false), 2_000)),
])
} finally {
if (!naturalExit) {
eofChild.kill('SIGKILL')
await exited
}
}
expect(naturalExit).toBe(true)
}, 10_000)
})

4
tests/session-cache-rich-capture.test.ts
View file

@ -12,8 +12,8 @@ import {
emptyCache,
loadCache,
saveCache,
sessionCachePath,
} from '../src/session-cache.js'
import { writeCacheOnDisk } from './fixtures/session-cache-io.js'
const TMP_DIR = join(tmpdir(), `codeburn-rich-cache-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`)
@ -113,7 +113,7 @@ describe('session cache round-trip for rich-capture fields', () => {
},
}
if (!existsSync(TMP_DIR)) await mkdir(TMP_DIR, { recursive: true })
await writeFile(sessionCachePath(), JSON.stringify(oldCache), 'utf-8')
await writeCacheOnDisk(oldCache)
const loaded = await loadCache()
const call = loaded.providers['claude']!.files['/x/old.jsonl']!.turns[0]!.calls[0]!

802
tests/session-cache-shards.test.ts Normal file
View file

@ -0,0 +1,802 @@
// Provider x month shard layout (CACHE_VERSION 9): the on-disk cache is a
// directory holding one envelope plus one shard per provider-month. What matters
// here is that the move off the older layouts loses nothing, that a file's
// bucket never moves when the session is appended to, that a save rewrites only
// the months that changed (including when the load was scoped to a subset of
// them), and that one unreadable shard costs exactly one month.
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { mkdir, readFile, readdir, rm, stat, utimes, writeFile } from 'fs/promises'
import { existsSync } from 'fs'
import { tmpdir } from 'os'
import { join } from 'path'
import {
CACHE_VERSION,
cacheBucketMonth,
cacheFileSpan,
computeEnvFingerprint,
cleanupOrphanedTempFiles,
clearLoadCacheMemo,
loadCache,
markCacheDirty,
monthScopeForRange,
saveCache,
sessionCacheDir,
type CachedFile,
type SessionCache,
} from '../src/session-cache.js'
let TMP_DIR: string
beforeEach(async () => {
TMP_DIR = join(tmpdir(), `codeburn-shard-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`)
process.env['CODEBURN_CACHE_DIR'] = TMP_DIR
await mkdir(TMP_DIR, { recursive: true })
clearLoadCacheMemo()
})
afterEach(async () => {
if (existsSync(TMP_DIR)) await rm(TMP_DIR, { recursive: true })
})
function turnAt(timestamp: string, key = 'msg-1'): CachedFile['turns'][number] {
const base = cachedFile().turns[0]!
return { ...base, timestamp, calls: [{ ...base.calls[0]!, timestamp, deduplicationKey: key }] }
}
function fileSpanning(first: string, last?: string): CachedFile {
return cachedFile({ turns: last ? [turnAt(first, 'a'), turnAt(last, 'b')] : [turnAt(first, 'a')] })
}
function cachedFile(overrides: Partial<CachedFile> = {}): CachedFile {
return {
fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 },
lastCompleteLineOffset: 128,
mcpInventory: ['mcp__github__list'],
turns: [{
timestamp: '2026-05-15T10:00:00Z',
sessionId: 'sess-1',
userMessage: 'do the thing',
calls: [{
provider: 'claude',
model: 'claude-sonnet-4-20250514',
usage: {
inputTokens: 1000,
outputTokens: 500,
cacheCreationInputTokens: 0,
cacheReadInputTokens: 0,
cachedInputTokens: 0,
reasoningTokens: 0,
webSearchRequests: 0,
cacheCreationOneHourTokens: 0,
},
costUSD: 0.01,
speed: 'standard',
timestamp: '2026-05-15T10:00:00Z',
tools: ['Read'],
bashCommands: [],
skills: [],
subagentTypes: [],
deduplicationKey: 'msg-1',
}],
}],
...overrides,
}
}
function v7Cache(): SessionCache {
return {
version: 7,
complete: true,
providers: {
claude: {
envFingerprint: 'claude-fp',
files: {
'/live/a.jsonl': cachedFile(),
'/live/b.jsonl': cachedFile({ turns: [] }),
// An orphaned PR-linked entry: its transcript is gone and can never
// re-parse, so the migration has to carry it across verbatim.
'/gone/pruned.jsonl': cachedFile({ prLinks: ['https://github.com/o/r/pull/1'] }),
},
},
codex: {
envFingerprint: 'codex-fp',
durable: true,
files: { '/live/rollout.jsonl': cachedFile() },
},
},
}
}
async function shardNames(): Promise<string[]> {
return (await readdir(sessionCacheDir())).sort()
}
async function envelope(): Promise<{ providers: Record<string, { shards: Record<string, { name: string; until: string }> }> }> {
return JSON.parse(await readFile(join(sessionCacheDir(), 'envelope.json'), 'utf-8'))
}
/** name -> bytes, for every shard on disk. */
async function shardBytes(): Promise<Map<string, string>> {
const dir = sessionCacheDir()
const out = new Map<string, string>()
for (const name of await shardNames()) out.set(name, await readFile(join(dir, name), 'utf-8'))
return out
}
describe('v7 -> shard migration', () => {
it('is lossless: every entry survives, shards replace the v7 file, reload matches', async () => {
const v7 = v7Cache()
const v7Path = join(TMP_DIR, 'session-cache.v7.json')
await writeFile(v7Path, JSON.stringify(v7))
const loaded = await loadCache()
// Same content, re-stamped at the current version.
expect(loaded).toEqual({ ...v7, version: CACHE_VERSION })
// Shards on disk, v7 blob removed.
expect(existsSync(v7Path)).toBe(false)
const names = await shardNames()
expect(names).toContain('envelope.json')
// claude's three entries split by month: two dated 2026-05, one turn-less.
expect(Object.keys((await envelope()).providers['claude']!.shards).sort()).toEqual(['0000-00', '2026-05'])
expect(Object.keys((await envelope()).providers['codex']!.shards)).toEqual(['2026-05'])
// A second load reads only the shards and produces the same cache.
clearLoadCacheMemo()
expect(await loadCache()).toEqual(loaded)
})
it('leaves a corrupt v7 file alone and starts fresh', async () => {
await writeFile(join(TMP_DIR, 'session-cache.v7.json'), '{broken')
const loaded = await loadCache()
expect(loaded.providers).toEqual({})
expect(existsSync(join(TMP_DIR, 'session-cache.v7.json'))).toBe(true)
})
})
describe('per-provider dirty tracking', () => {
it('rewrites only the provider that changed', async () => {
await writeFile(join(TMP_DIR, 'session-cache.v7.json'), JSON.stringify(v7Cache()))
const cache = await loadCache()
const dir = sessionCacheDir()
const before = new Map<string, string>()
for (const name of await shardNames()) before.set(name, await readFile(join(dir, name), 'utf-8'))
cache.providers['codex']!.files['/live/rollout.jsonl'] = cachedFile({ mcpInventory: ['changed'] })
markCacheDirty(cache, 'codex')
await saveCache(cache)
const after = await shardNames()
const claudeShard = [...before.keys()].find(n => n.startsWith('claude.'))!
// The untouched provider keeps its exact file, byte for byte.
expect(after).toContain(claudeShard)
expect(await readFile(join(dir, claudeShard), 'utf-8')).toBe(before.get(claudeShard))
// The changed provider is republished under a new name; the old one is gone.
const codexBefore = [...before.keys()].find(n => n.startsWith('codex.'))!
const codexAfter = after.find(n => n.startsWith('codex.'))!
expect(codexAfter).not.toBe(codexBefore)
expect(after).not.toContain(codexBefore)
clearLoadCacheMemo()
const reloaded = await loadCache()
expect(reloaded.providers['codex']!.files['/live/rollout.jsonl']!.mcpInventory).toEqual(['changed'])
expect(reloaded.providers['claude']).toEqual(cache.providers['claude'])
})
})
describe('corrupt shard isolation', () => {
it('drops only the unreadable month, keeping every other month and provider', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: 'claude-fp',
files: { '/live/may.jsonl': fileSpanning('2026-05-15T10:00:00Z'), '/live/jun.jsonl': fileSpanning('2026-06-15T10:00:00Z') },
},
codex: { envFingerprint: 'codex-fp', files: { '/live/r.jsonl': fileSpanning('2026-05-15T10:00:00Z') } },
},
}
markCacheDirty(cache, 'claude')
markCacheDirty(cache, 'codex')
await saveCache(cache)
const dir = sessionCacheDir()
await writeFile(join(dir, (await envelope()).providers['claude']!.shards['2026-05']!.name), '{"/x":{"turns":')
clearLoadCacheMemo()
const reloaded = await loadCache()
expect(Object.keys(reloaded.providers['claude']!.files)).toEqual(['/live/jun.jsonl'])
expect(reloaded.providers['codex']).toEqual(cache.providers['codex'])
// Self-heals: the unreadable month is republished from whatever re-parses
// into it rather than being carried forward corrupt forever.
reloaded.providers['claude']!.files['/live/may.jsonl'] = fileSpanning('2026-05-15T10:00:00Z')
markCacheDirty(reloaded, 'claude', '/live/may.jsonl')
await saveCache(reloaded)
clearLoadCacheMemo()
expect(Object.keys((await loadCache()).providers['claude']!.files).sort()).toEqual(['/live/jun.jsonl', '/live/may.jsonl'])
})
})
describe('cleanupOrphanedTempFiles', () => {
it('sweeps stale shard temps and unreferenced shards, keeping the live ones', async () => {
await saveCache({ version: CACHE_VERSION, complete: true, providers: {
claude: { envFingerprint: 'fp', files: { '/a.jsonl': cachedFile() } },
} })
const dir = sessionCacheDir()
const live = (await shardNames()).find(n => n.startsWith('claude.'))!
const backdate = async (path: string, minutes: number) => {
const at = new Date(Date.now() - minutes * 60 * 1000)
await utimes(path, at, at)
}
const oldTemp = join(dir, 'claude.deadbeef.json.tmp')
await writeFile(oldTemp, 'partial')
await backdate(oldTemp, 10)
const orphanShard = join(dir, 'codex.deadbeef.json')
await writeFile(orphanShard, '{}')
await backdate(orphanShard, 90)
// Unreferenced but fresh: this is what a CONCURRENT save's shard looks like
// before its envelope lands, so the sweep must leave it alone.
const inFlightShard = join(dir, 'codex.c0ffee00.json')
await writeFile(inFlightShard, '{}')
const recentTemp = join(dir, 'claude.feedface.json.tmp')
await writeFile(recentTemp, 'in flight')
// The live shard is far older than the temp cutoff; being referenced is what
// protects it, not its age.
await backdate(join(dir, live), 120)
await cleanupOrphanedTempFiles()
expect(existsSync(oldTemp)).toBe(false)
expect(existsSync(orphanShard)).toBe(false)
expect(existsSync(inFlightShard)).toBe(true)
expect(existsSync(recentTemp)).toBe(true)
expect(existsSync(join(dir, live))).toBe(true)
expect(existsSync(join(dir, 'envelope.json'))).toBe(true)
})
it('retires the pre-v8 single-file layout temps left in the parent directory', async () => {
await saveCache({ version: CACHE_VERSION, complete: true, providers: {} })
const legacyTemp = join(TMP_DIR, 'session-cache.v7.json.abc123.tmp')
await writeFile(legacyTemp, 'orphan from an older build')
const at = new Date(Date.now() - 10 * 60 * 1000)
await utimes(legacyTemp, at, at)
const freshLegacyTemp = join(TMP_DIR, 'session-cache.v7.json.def456.tmp')
await writeFile(freshLegacyTemp, 'an old binary mid-write')
await cleanupOrphanedTempFiles()
expect(existsSync(legacyTemp)).toBe(false)
expect(existsSync(freshLegacyTemp)).toBe(true)
})
})
// Two live processes share one cache directory routinely: a one-shot CLI beside
// the resident serve child, or two menubar polls. Neither may publish an
// envelope naming a file that is not there — that reads back as a corrupt
// provider and silently drops its history.
describe('concurrent writers', () => {
function seed(provider: string, tag: string, files: number): SessionCache {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
[provider]: {
envFingerprint: tag,
files: Object.fromEntries(
Array.from({ length: files }, (_, i) => [`/f/${provider}/${i}.jsonl`, cachedFile()]),
),
},
},
}
markCacheDirty(cache, provider)
return cache
}
async function assertReferentialIntegrity(expected: string[]): Promise<void> {
const dir = sessionCacheDir()
for (const meta of Object.values((await envelope()).providers)) {
for (const ref of Object.values(meta.shards)) {
expect(existsSync(join(dir, ref.name)), `envelope names a missing shard: ${ref.name}`).toBe(true)
}
}
clearLoadCacheMemo()
const loaded = await loadCache()
expect(Object.keys(loaded.providers).length).toBeGreaterThan(0)
for (const provider of expected) expect(loaded.providers[provider]).toBeDefined()
}
it('never publishes a dangling envelope when two saves race', async () => {
for (let round = 0; round < 15; round++) {
await Promise.allSettled([
saveCache(seed('claude', `a${round}`, 40)),
saveCache(seed('codex', `b${round}`, 40)),
])
// Whichever won, the published set has to be internally consistent and
// hold at least the provider that got there last.
await assertReferentialIntegrity([])
}
})
it('a stale writer rewrites a shard another process retired instead of orphaning it', async () => {
// Seed: claude holds an expired-source PR orphan no re-parse can recover.
const initial: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: { envFingerprint: 'fp', files: { '/gone/pruned.jsonl': cachedFile({ prLinks: ['https://github.com/o/r/pull/1'] }) } },
codex: { envFingerprint: 'fp', durable: true, files: { '/live/r.jsonl': cachedFile() } },
},
}
markCacheDirty(initial, 'claude')
markCacheDirty(initial, 'codex')
await saveCache(initial)
// Process B loads now, recording claude's current shard name.
clearLoadCacheMemo()
const b = await loadCache()
// Process A independently touches ONLY claude and republishes, retiring the
// shard file B is still holding a name for.
clearLoadCacheMemo()
const a = await loadCache()
a.providers['claude']!.files['/live/new.jsonl'] = cachedFile()
markCacheDirty(a, 'claude')
await saveCache(a)
// B now saves an unrelated codex change.
b.providers['codex']!.files['/live/r2.jsonl'] = cachedFile()
markCacheDirty(b, 'codex')
await saveCache(b)
await assertReferentialIntegrity(['claude', 'codex'])
clearLoadCacheMemo()
const final = await loadCache()
expect(final.providers['claude']!.files['/gone/pruned.jsonl']).toBeDefined()
expect(final.providers['codex']!.files['/live/r2.jsonl']).toBeDefined()
})
})
describe('month buckets', () => {
it('keeps a file in its first-turn month when the session is appended to', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: { claude: { envFingerprint: 'fp', files: { '/live/long.jsonl': fileSpanning('2026-05-15T10:00:00Z') } } },
}
markCacheDirty(cache, 'claude')
await saveCache(cache)
expect(Object.keys((await envelope()).providers['claude']!.shards)).toEqual(['2026-05'])
// Two months of appends later the bucket is unchanged; only `until` moves,
// which is what lets a ranged load still find this session.
const appended = fileSpanning('2026-05-15T10:00:00Z', '2026-07-02T10:00:00Z')
expect(cacheBucketMonth(appended)).toBe('2026-05')
cache.providers['claude']!.files['/live/long.jsonl'] = appended
markCacheDirty(cache, 'claude', '/live/long.jsonl')
await saveCache(cache)
const shards = (await envelope()).providers['claude']!.shards
expect(Object.keys(shards)).toEqual(['2026-05'])
expect(shards['2026-05']!.until).toBe('2026-07')
// ...and a July query still loads it, despite the May bucket key.
clearLoadCacheMemo()
const scoped = await loadCache(monthScopeForRange(new Date('2026-07-01T00:00:00Z'), new Date('2026-07-31T23:59:59Z')))
expect(scoped.providers['claude']!.files['/live/long.jsonl']).toBeDefined()
})
it('rewrites only the month that changed', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: 'fp',
files: {
'/live/mar.jsonl': fileSpanning('2026-03-10T10:00:00Z'),
'/live/apr.jsonl': fileSpanning('2026-04-10T10:00:00Z'),
'/live/may.jsonl': fileSpanning('2026-05-10T10:00:00Z'),
},
},
},
}
markCacheDirty(cache, 'claude')
await saveCache(cache)
const before = await shardBytes()
const untouched = [
(await envelope()).providers['claude']!.shards['2026-03']!.name,
(await envelope()).providers['claude']!.shards['2026-04']!.name,
]
cache.providers['claude']!.files['/live/may.jsonl'] = cachedFile({ turns: [turnAt('2026-05-10T10:00:00Z', 'a')], mcpInventory: ['changed'] })
markCacheDirty(cache, 'claude', '/live/may.jsonl')
await saveCache(cache)
const after = await shardBytes()
for (const name of untouched) expect(after.get(name)).toBe(before.get(name))
expect(after.has((await envelope()).providers['claude']!.shards['2026-05']!.name)).toBe(true)
})
it('dirties the month a deleted file was in', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: 'fp',
files: { '/live/mar.jsonl': fileSpanning('2026-03-10T10:00:00Z'), '/live/mar2.jsonl': fileSpanning('2026-03-11T10:00:00Z') },
},
},
}
markCacheDirty(cache, 'claude')
await saveCache(cache)
delete cache.providers['claude']!.files['/live/mar2.jsonl']
markCacheDirty(cache, 'claude', '/live/mar2.jsonl')
await saveCache(cache)
clearLoadCacheMemo()
expect(Object.keys((await loadCache()).providers['claude']!.files)).toEqual(['/live/mar.jsonl'])
})
})
describe('scoped load', () => {
async function seedThreeMonths(): Promise<void> {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: computeEnvFingerprint('claude'),
files: {
'/live/mar.jsonl': fileSpanning('2026-03-10T10:00:00Z'),
'/live/apr.jsonl': fileSpanning('2026-04-10T10:00:00Z'),
'/live/jun.jsonl': fileSpanning('2026-06-10T10:00:00Z'),
},
},
},
}
markCacheDirty(cache, 'claude')
await saveCache(cache)
}
const juneScope = monthScopeForRange(new Date('2026-06-01T00:00:00Z'), new Date('2026-06-30T23:59:59Z'))
it('reads only the months the range can report on, plus one of slack', async () => {
await seedThreeMonths()
clearLoadCacheMemo()
const scoped = await loadCache(juneScope)
// June is in range; May would be the slack month (absent here); March and
// April cannot contribute a June turn and stay on disk.
expect(Object.keys(scoped.providers['claude']!.files)).toEqual(['/live/jun.jsonl'])
})
it('save from a scoped load leaves the unloaded months byte-identical', async () => {
await seedThreeMonths()
const before = await shardBytes()
const kept = [
(await envelope()).providers['claude']!.shards['2026-03']!.name,
(await envelope()).providers['claude']!.shards['2026-04']!.name,
]
clearLoadCacheMemo()
const scoped = await loadCache(juneScope)
scoped.providers['claude']!.files['/live/jun2.jsonl'] = fileSpanning('2026-06-20T10:00:00Z')
markCacheDirty(scoped, 'claude', '/live/jun2.jsonl')
await saveCache(scoped)
const after = await shardBytes()
for (const name of kept) expect(after.get(name), `unloaded month rewritten: ${name}`).toBe(before.get(name))
clearLoadCacheMemo()
const full = await loadCache()
expect(Object.keys(full.providers['claude']!.files).sort())
.toEqual(['/live/apr.jsonl', '/live/jun.jsonl', '/live/jun2.jsonl', '/live/mar.jsonl'])
})
it('merges rather than replaces when a re-parse lands in an unloaded month', async () => {
await seedThreeMonths()
clearLoadCacheMemo()
const scoped = await loadCache(juneScope)
// A March session that was never loaded is re-parsed (its mtime moved) and
// written straight back into the March bucket.
scoped.providers['claude']!.files['/live/mar.jsonl'] = cachedFile({ turns: [turnAt('2026-03-10T10:00:00Z', 'z')], mcpInventory: ['reparsed'] })
markCacheDirty(scoped, 'claude', '/live/mar.jsonl')
await saveCache(scoped)
clearLoadCacheMemo()
const full = await loadCache()
expect(full.providers['claude']!.files['/live/mar.jsonl']!.mcpInventory).toEqual(['reparsed'])
expect(Object.keys(full.providers['claude']!.files).sort())
.toEqual(['/live/apr.jsonl', '/live/jun.jsonl', '/live/mar.jsonl'])
})
it('never scopes a provider whose fingerprint moved, or a durable one', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: { envFingerprint: 'stale-fp', files: { '/live/mar.jsonl': fileSpanning('2026-03-10T10:00:00Z') } },
copilot: { envFingerprint: computeEnvFingerprint('copilot'), durable: true, files: { '/live/otel.db': fileSpanning('2026-03-10T10:00:00Z') } },
},
}
markCacheDirty(cache, 'claude')
markCacheDirty(cache, 'copilot')
await saveCache(cache)
clearLoadCacheMemo()
const scoped = await loadCache(juneScope)
// Both would be skipped on month alone; both are read in full anyway, so the
// fingerprint reset and the durable orphan carry-forward see every entry.
expect(scoped.providers['claude']!.files['/live/mar.jsonl']).toBeDefined()
expect(scoped.providers['copilot']!.files['/live/otel.db']).toBeDefined()
})
})
describe('v8 -> v9 migration', () => {
it('re-buckets the v8 provider shards losslessly and retires the v8 directory', async () => {
const v8Dir = join(TMP_DIR, 'session-cache.v8')
await mkdir(v8Dir, { recursive: true })
const section = {
envFingerprint: 'claude-fp',
files: {
'/live/mar.jsonl': fileSpanning('2026-03-10T10:00:00Z'),
'/live/jun.jsonl': fileSpanning('2026-06-10T10:00:00Z'),
'/gone/pruned.jsonl': cachedFile({ prLinks: ['https://github.com/o/r/pull/1'] }),
},
}
await writeFile(join(v8Dir, 'claude.abc.json'), JSON.stringify(section))
await writeFile(join(v8Dir, 'envelope.json'), JSON.stringify({
version: 8, complete: true, nonce: 'n', shards: { claude: 'claude.abc.json' },
}))
const loaded = await loadCache()
expect(loaded.providers['claude']).toEqual(section)
expect(loaded.complete).toBe(true)
expect(existsSync(v8Dir)).toBe(false)
expect(Object.keys((await envelope()).providers['claude']!.shards).sort()).toEqual(['2026-03', '2026-05', '2026-06'])
clearLoadCacheMemo()
expect(await loadCache()).toEqual(loaded)
})
})
// Several providers emit turns in a non-chronological order (cursor composers by
// ROWID, goose / crush / copilot by a DESC ordering). Reading the span off
// turns[0]/turns[-1] then gives `until < bucket` — an empty span, unreachable at
// every scope.
describe('out-of-order turns', () => {
const outOfOrder = () => cachedFile({ turns: [turnAt('2026-08-10T10:00:00Z', 'a'), turnAt('2026-03-04T10:00:00Z', 'b')] })
it('spans oldest to newest whatever order the turns arrive in', () => {
const span = cacheFileSpan(outOfOrder())
expect(span).toEqual({ bucket: '2026-03', until: '2026-08' })
expect(cacheBucketMonth(outOfOrder())).toBe('2026-03')
})
it('stays reachable at the scope of either end', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: { claude: { envFingerprint: computeEnvFingerprint('claude'), files: { '/live/desc.jsonl': outOfOrder() } } },
}
markCacheDirty(cache, 'claude')
await saveCache(cache)
expect((await envelope()).providers['claude']!.shards['2026-03']!.until).toBe('2026-08')
for (const [from, to] of [['2026-03-01', '2026-03-31'], ['2026-08-01', '2026-08-31']] as const) {
clearLoadCacheMemo()
const scoped = await loadCache(monthScopeForRange(new Date(`${from}T00:00:00Z`), new Date(`${to}T23:59:59Z`)))
expect(scoped.providers['claude']!.files['/live/desc.jsonl'], `unreachable at ${from}`).toBeDefined()
}
})
})
// A path must never end up in two shards at once: on a later load the two copies
// race and the stale one can win, and nothing sweeps it because the envelope
// names both.
describe('re-bucketing out of an unloaded month', () => {
const juneScope = monthScopeForRange(new Date('2026-06-01T00:00:00Z'), new Date('2026-06-30T23:59:59Z'))
async function seed(): Promise<void> {
const cache: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: computeEnvFingerprint('claude'),
files: {
'/live/moving.jsonl': fileSpanning('2026-01-10T10:00:00Z'),
'/live/stay.jsonl': fileSpanning('2026-01-11T10:00:00Z'),
'/live/jun.jsonl': fileSpanning('2026-06-10T10:00:00Z'),
},
},
},
}
markCacheDirty(cache, 'claude')
await saveCache(cache)
}
/** Every shard's view of `path`, so a duplicate is visible directly. */
async function copiesOf(path: string): Promise<string[]> {
const dir = sessionCacheDir()
const found: string[] = []
for (const [bucket, ref] of Object.entries((await envelope()).providers['claude']!.shards)) {
const files = JSON.parse(await readFile(join(dir, ref.name), 'utf-8'))
if (files[path]) found.push(bucket)
}
return found.sort()
}
it('(a) drops the old copy when a re-parse moves the file to another month', async () => {
await seed()
clearLoadCacheMemo()
const scoped = await loadCache(juneScope)
expect(scoped.providers['claude']!.files['/live/moving.jsonl']).toBeUndefined()
// Re-parsed from byte 0 after a rewrite: its oldest turn is now in May.
scoped.providers['claude']!.files['/live/moving.jsonl'] = cachedFile({ turns: [turnAt('2026-05-02T10:00:00Z', 'new')], mcpInventory: ['reparsed'] })
markCacheDirty(scoped, 'claude', '/live/moving.jsonl')
await saveCache(scoped)
expect(await copiesOf('/live/moving.jsonl')).toEqual(['2026-05'])
clearLoadCacheMemo()
const full = await loadCache()
expect(full.providers['claude']!.files['/live/moving.jsonl']!.mcpInventory).toEqual(['reparsed'])
expect(full.providers['claude']!.files['/live/stay.jsonl']).toBeDefined()
})
it('(b) drops the old copy when a parse failure leaves a turn-less marker', async () => {
await seed()
clearLoadCacheMemo()
const scoped = await loadCache(juneScope)
// The #441 path: the file threw, so only a failure marker is cached.
scoped.providers['claude']!.files['/live/moving.jsonl'] = { fingerprint: { dev: 1, ino: 2, mtimeMs: 9, sizeBytes: 4 }, mcpInventory: [], turns: [], failed: true }
markCacheDirty(scoped, 'claude', '/live/moving.jsonl')
await saveCache(scoped)
expect(await copiesOf('/live/moving.jsonl')).toEqual(['0000-00'])
clearLoadCacheMemo()
const full = await loadCache()
expect(full.providers['claude']!.files['/live/moving.jsonl']!.failed).toBe(true)
expect(full.providers['claude']!.files['/live/stay.jsonl']).toBeDefined()
})
it('resolves a duplicate to the freshest copy and prunes it on the next save', async () => {
await seed()
// Forge the split state directly: the same path in two shards.
const dir = sessionCacheDir()
const env = await envelope()
const janName = env.providers['claude']!.shards['2026-01']!.name
const junName = env.providers['claude']!.shards['2026-06']!.name
const jun = JSON.parse(await readFile(join(dir, junName), 'utf-8'))
jun['/live/moving.jsonl'] = cachedFile({ turns: [turnAt('2026-06-02T10:00:00Z', 'fresh')], fingerprint: { dev: 1, ino: 2, mtimeMs: 999, sizeBytes: 4 }, mcpInventory: ['fresh'] })
await writeFile(join(dir, junName), JSON.stringify(jun))
clearLoadCacheMemo()
const full = await loadCache()
// Newest fingerprint wins, whichever shard finished reading first.
expect(full.providers['claude']!.files['/live/moving.jsonl']!.mcpInventory).toEqual(['fresh'])
await saveCache(full)
expect(await copiesOf('/live/moving.jsonl')).toEqual(['2026-06'])
expect(existsSync(join(dir, janName))).toBe(false)
})
})
describe('carried months under a concurrent writer', () => {
it('adopts the current shard rather than dropping an orphan-bearing month', async () => {
const orphan = cachedFile({ turns: [turnAt('2026-03-10T10:00:00Z', 'm')], prLinks: ['https://github.com/o/r/pull/1'] })
const initial: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: computeEnvFingerprint('claude'),
files: { '/gone/mar.jsonl': orphan, '/live/jun.jsonl': fileSpanning('2026-06-10T10:00:00Z') },
},
},
}
markCacheDirty(initial, 'claude')
await saveCache(initial)
// Process B loads June-scoped: March is carried, by the name it saw.
clearLoadCacheMemo()
const b = await loadCache(monthScopeForRange(new Date('2026-06-01T00:00:00Z'), new Date('2026-06-30T23:59:59Z')))
// Process A independently republishes March, retiring the file B remembers.
clearLoadCacheMemo()
const a = await loadCache(monthScopeForRange(new Date('2026-03-01T00:00:00Z'), new Date('2026-03-31T23:59:59Z')))
a.providers['claude']!.files['/gone/mar2.jsonl'] = cachedFile({ turns: [turnAt('2026-03-12T10:00:00Z', 'n')], prLinks: ['https://github.com/o/r/pull/2'] })
markCacheDirty(a, 'claude', '/gone/mar2.jsonl')
await saveCache(a)
// B saves its own unrelated June change.
b.providers['claude']!.files['/live/jun2.jsonl'] = fileSpanning('2026-06-20T10:00:00Z')
markCacheDirty(b, 'claude', '/live/jun2.jsonl')
await saveCache(b)
clearLoadCacheMemo()
const final = await loadCache()
// March survived under A's name, with both orphans; June has both files.
expect(Object.keys(final.providers['claude']!.files).sort())
.toEqual(['/gone/mar.jsonl', '/gone/mar2.jsonl', '/live/jun.jsonl', '/live/jun2.jsonl'])
for (const ref of Object.values((await envelope()).providers['claude']!.shards)) {
expect(existsSync(join(sessionCacheDir(), ref.name))).toBe(true)
}
})
// Two saves merging into the SAME unloaded month are a read-modify-write with
// no lock between them. In the product they are serialised by the warm refresh
// lock; this covers what survives when they are not. The optimistic retry in
// saveCache narrows the window to the envelope publish, and a loser's entries
// are re-derived by the next parse (the reconcile finds no cache entry and
// re-reads the file) rather than being lost for good.
it('two scoped saves merging into the same unloaded month keep the envelope sound', async () => {
const base: SessionCache = {
version: CACHE_VERSION,
complete: true,
providers: {
claude: {
envFingerprint: computeEnvFingerprint('claude'),
files: { '/live/mar.jsonl': fileSpanning('2026-03-10T10:00:00Z'), '/live/jun.jsonl': fileSpanning('2026-06-10T10:00:00Z') },
},
},
}
markCacheDirty(base, 'claude')
await saveCache(base)
const juneScope = monthScopeForRange(new Date('2026-06-01T00:00:00Z'), new Date('2026-06-30T23:59:59Z'))
clearLoadCacheMemo()
const p1 = await loadCache(juneScope)
clearLoadCacheMemo()
const p2 = await loadCache(juneScope)
// Both re-parse a different March session neither of them loaded.
p1.providers['claude']!.files['/live/mar-a.jsonl'] = fileSpanning('2026-03-20T10:00:00Z')
markCacheDirty(p1, 'claude', '/live/mar-a.jsonl')
p2.providers['claude']!.files['/live/mar-b.jsonl'] = fileSpanning('2026-03-21T10:00:00Z')
markCacheDirty(p2, 'claude', '/live/mar-b.jsonl')
await Promise.allSettled([saveCache(p1), saveCache(p2)])
clearLoadCacheMemo()
const final = await loadCache()
// The envelope is internally consistent and the pre-existing March session
// survived; a read-modify-write loser is re-derived by the next parse.
for (const ref of Object.values((await envelope()).providers['claude']!.shards)) {
expect(existsSync(join(sessionCacheDir(), ref.name))).toBe(true)
}
expect(final.providers['claude']!.files['/live/mar.jsonl']).toBeDefined()
expect(final.providers['claude']!.files['/live/jun.jsonl']).toBeDefined()
const landed = ['/live/mar-a.jsonl', '/live/mar-b.jsonl'].filter(p => final.providers['claude']!.files[p])
expect(landed.length).toBeGreaterThanOrEqual(1)
})
})
describe('retiring an orphaned prior layout', () => {
it('sweeps a v8 directory and a v7 file left behind by an interrupted re-layout', async () => {
await saveCache({ version: CACHE_VERSION, complete: true, providers: {
claude: { envFingerprint: 'fp', files: { '/a.jsonl': fileSpanning('2026-05-10T10:00:00Z') } },
} })
const v8Dir = join(TMP_DIR, 'session-cache.v8')
await mkdir(v8Dir, { recursive: true })
await writeFile(join(v8Dir, 'envelope.json'), JSON.stringify({ version: 8, nonce: 'n', shards: {} }))
await writeFile(join(v8Dir, 'claude.abc.json'), '{}')
const v7 = join(TMP_DIR, 'session-cache.v7.json')
await writeFile(v7, '{}')
// Fresh: an in-flight write by an older binary must be left alone.
await cleanupOrphanedTempFiles()
expect(existsSync(v8Dir)).toBe(true)
expect(existsSync(v7)).toBe(true)
const old = new Date(Date.now() - 90 * 60 * 1000)
await utimes(join(v8Dir, 'envelope.json'), old, old)
await utimes(v7, old, old)
await cleanupOrphanedTempFiles()
expect(existsSync(v8Dir)).toBe(false)
expect(existsSync(v7)).toBe(false)
})
})

39
tests/session-cache.test.ts
View file

@ -21,11 +21,12 @@ import {
mergeCallByDedupKey,
reconcileFile,
saveCache,
sessionCachePath,
sessionCacheDir,
} from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
// Version-suffixed filename (e.g. session-cache.v5.json) the cache now writes to.
const CACHE_FILE = () => basename(sessionCachePath())
// Version-suffixed directory (e.g. session-cache.v8) the cache now writes to.
const CACHE_DIR = () => basename(sessionCacheDir())
const TMP_DIR = join(tmpdir(), `codeburn-scache-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`)
@ -185,8 +186,7 @@ describe('loadCache / saveCache', () => {
it('atomic write does not leave partial file on error', async () => {
await saveCache(emptyCache())
const raw = await readFile(sessionCachePath(), 'utf-8')
expect(JSON.parse(raw)).toEqual(emptyCache())
expect(await readCacheOnDisk()).toEqual(emptyCache())
})
})
@ -196,14 +196,15 @@ describe('versioned cache file + legacy adoption', () => {
function validCache(): SessionCache {
return {
version: CACHE_VERSION,
complete: false,
providers: { claude: { envFingerprint: 'abc123', files: { '/path/to/session.jsonl': makeCachedFile() } } },
}
}
it('writes and reads the version-suffixed file, never the legacy name', async () => {
expect(basename(sessionCachePath())).toBe(`session-cache.v${CACHE_VERSION}.json`)
it('writes and reads the version-suffixed directory, never the legacy name', async () => {
expect(basename(sessionCacheDir())).toBe(`session-cache.v${CACHE_VERSION}`)
await saveCache(validCache())
expect(existsSync(sessionCachePath())).toBe(true)
expect(existsSync(sessionCacheDir())).toBe(true)
expect(existsSync(join(TMP_DIR, 'session-cache.json'))).toBe(false)
expect(await loadCache()).toEqual(validCache())
})
@ -213,9 +214,9 @@ describe('versioned cache file + legacy adoption', () => {
const legacy = join(TMP_DIR, 'session-cache.json')
await writeFile(legacy, JSON.stringify(validCache()))
// Versioned file absent → adopt-copy from legacy on first load.
// Versioned directory absent → adopt-copy from legacy on first load.
expect(await loadCache()).toEqual(validCache())
expect(existsSync(sessionCachePath())).toBe(true)
expect(existsSync(sessionCacheDir())).toBe(true)
// Legacy left intact (not deleted, not rewritten).
expect(existsSync(legacy)).toBe(true)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(validCache())
@ -224,7 +225,7 @@ describe('versioned cache file + legacy adoption', () => {
// file exists.
const mutated: SessionCache = { version: CACHE_VERSION, providers: { codex: { envFingerprint: 'zzz', files: {} } } }
await writeFile(legacy, JSON.stringify(mutated))
expect(await loadCache()).toEqual(validCache())
expect(await readCacheOnDisk()).toEqual(validCache())
})
it('ignores a different-version legacy file and never touches it', async () => {
@ -234,8 +235,8 @@ describe('versioned cache file + legacy adoption', () => {
await writeFile(legacy, JSON.stringify(stale))
expect((await loadCache()).providers).toEqual({})
// No versioned file adopted; legacy left byte-intact.
expect(existsSync(sessionCachePath())).toBe(false)
// No versioned directory adopted; legacy left byte-intact.
expect(existsSync(sessionCacheDir())).toBe(false)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(stale)
})
@ -246,9 +247,9 @@ describe('versioned cache file + legacy adoption', () => {
await writeFile(legacy, legacyContent)
await saveCache(validCache())
// The versioned file holds the new data; the legacy file is byte-untouched.
// The shards hold the new data; the legacy file is byte-untouched.
expect(await readFile(legacy, 'utf-8')).toBe(legacyContent)
expect(JSON.parse(await readFile(sessionCachePath(), 'utf-8'))).toEqual(validCache())
expect(await readCacheOnDisk()).toEqual(validCache())
})
})
@ -820,7 +821,7 @@ describe('loadCache validation', () => {
} } },
}
await writeRawCache(cache)
expect((await loadCache())).toEqual(cache)
expect(await loadCache()).toEqual(cache)
})
it('accepts a fully valid cache with all fields populated', async () => {
@ -845,7 +846,8 @@ describe('cleanupOrphanedTempFiles', () => {
it('removes .tmp files older than 5 minutes', async () => {
await mkdir(TMP_DIR, { recursive: true })
const oldTmp = join(TMP_DIR, `${CACHE_FILE()}.abc123.tmp`)
await mkdir(join(TMP_DIR, CACHE_DIR()), { recursive: true })
const oldTmp = join(TMP_DIR, CACHE_DIR(), 'claude.abc123.json.tmp')
await writeFile(oldTmp, 'stale')
const { utimes } = await import('fs/promises')
const oldTime = new Date(Date.now() - 10 * 60 * 1000)
@ -858,7 +860,8 @@ describe('cleanupOrphanedTempFiles', () => {
it('preserves recent .tmp files', async () => {
await mkdir(TMP_DIR, { recursive: true })
const recentTmp = join(TMP_DIR, `${CACHE_FILE()}.def456.tmp`)
await mkdir(join(TMP_DIR, CACHE_DIR()), { recursive: true })
const recentTmp = join(TMP_DIR, CACHE_DIR(), 'claude.def456.json.tmp')
await writeFile(recentTmp, 'recent')
await cleanupOrphanedTempFiles()

266
tests/sync-ledger-otlp.test.ts
View file

@ -2,7 +2,7 @@
* Unit tests for sync ledger and OTLP payload builder.
*/
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
import { mkdtemp, rm } from 'fs/promises'
import { join } from 'path'
import { tmpdir } from 'os'
@ -233,17 +233,21 @@ describe('batchCalls', () => {
describe('ledger', () => {
let tmpDir: string
const originalHome = process.env.HOME
const originalCacheDir = process.env.CODEBURN_CACHE_DIR
const originalXdgCacheDir = process.env.XDG_CACHE_HOME
beforeEach(async () => {
tmpDir = await mkdtemp(join(tmpdir(), 'codeburn-ledger-'))
process.env.HOME = tmpDir
// env-isolation.ts redirects XDG_CACHE_HOME to a per-worker sandbox shared
// across tests — the ledger honors XDG, so point it at the per-test dir.
process.env.XDG_CACHE_HOME = join(tmpDir, '.cache')
process.env.CODEBURN_CACHE_DIR = join(tmpDir, '.cache', 'codeburn')
})
afterEach(async () => {
process.env.HOME = originalHome
if (originalCacheDir === undefined) delete process.env.CODEBURN_CACHE_DIR
else process.env.CODEBURN_CACHE_DIR = originalCacheDir
if (originalXdgCacheDir === undefined) delete process.env.XDG_CACHE_HOME
else process.env.XDG_CACHE_HOME = originalXdgCacheDir
await rm(tmpDir, { recursive: true, force: true })
})
@ -308,6 +312,91 @@ describe('ledger', () => {
expect(clearLedger()).toBe(0)
})
it('clearLedger removes coexisting canonical and eligible legacy ledgers without adopting either', async () => {
const { clearLedger } = await import('../src/sync/ledger.js')
const { existsSync, mkdirSync, writeFileSync } = await import('fs')
const canonicalDir = join(tmpDir, '.cache', 'codeburn')
const xdgDir = join(tmpDir, 'xdg-clear-coexisting')
const legacyDir = join(xdgDir, 'codeburn')
delete process.env.CODEBURN_CACHE_DIR
process.env.XDG_CACHE_HOME = xdgDir
mkdirSync(canonicalDir, { recursive: true })
mkdirSync(legacyDir, { recursive: true })
writeFileSync(join(canonicalDir, 'sync-ledger.json'), JSON.stringify([
{ key: 'canonical', ts: '2026-07-02T00:00:00Z' },
{ key: 'duplicate', ts: '2026-07-03T00:00:00Z' },
]))
writeFileSync(join(legacyDir, 'sync-ledger.json'), JSON.stringify([
{ key: 'legacy', ts: '2026-07-01T00:00:00Z' },
{ key: 'duplicate', ts: '2025-01-01T00:00:00Z' },
]))
expect(clearLedger()).toBe(3)
expect(existsSync(join(canonicalDir, 'sync-ledger.json'))).toBe(false)
expect(existsSync(join(legacyDir, 'sync-ledger.json'))).toBe(false)
})
it('clearLedger attempts both targets, reports a real unlink failure, and can retry the remainder', async () => {
const fs = await import('fs')
const canonicalDir = join(tmpDir, '.cache', 'codeburn')
const canonicalPath = join(canonicalDir, 'sync-ledger.json')
const xdgDir = join(tmpDir, 'xdg-clear-retry')
const legacyDir = join(xdgDir, 'codeburn')
const legacyPath = join(legacyDir, 'sync-ledger.json')
delete process.env.CODEBURN_CACHE_DIR
process.env.XDG_CACHE_HOME = xdgDir
fs.mkdirSync(canonicalDir, { recursive: true })
fs.mkdirSync(legacyDir, { recursive: true })
fs.writeFileSync(canonicalPath, JSON.stringify([
{ key: 'canonical', ts: '2026-07-02T00:00:00Z' },
]))
fs.writeFileSync(legacyPath, JSON.stringify([
{ key: 'legacy', ts: '2026-07-01T00:00:00Z' },
]))
const attempts: string[] = []
let failCanonicalOnce = true
vi.doMock('fs', async () => {
const actual = await vi.importActual<typeof import('fs')>('fs')
return {
...actual,
unlinkSync: (path: fs.PathLike) => {
const value = String(path)
attempts.push(value)
if (value === canonicalPath && failCanonicalOnce) {
failCanonicalOnce = false
throw Object.assign(new Error('injected canonical unlink failure'), { code: 'EACCES' })
}
return actual.unlinkSync(path)
},
}
})
vi.resetModules()
try {
const { clearLedger } = await import('../src/sync/ledger.js')
expect(() => clearLedger()).toThrow('injected canonical unlink failure')
expect(attempts).toContain(canonicalPath)
expect(attempts).toContain(legacyPath)
expect(fs.existsSync(canonicalPath)).toBe(true)
expect(fs.existsSync(legacyPath)).toBe(false)
expect(JSON.parse(fs.readFileSync(canonicalPath, 'utf8'))).toEqual([
{ key: 'canonical', ts: '2026-07-02T00:00:00Z' },
])
// The successful legacy deletion is not replayed or migrated. Retrying
// removes only the canonical remainder; its missing peer is ENOENT-safe.
expect(clearLedger()).toBe(1)
expect(fs.existsSync(canonicalPath)).toBe(false)
expect(fs.existsSync(legacyPath)).toBe(false)
} finally {
vi.doUnmock('fs')
vi.resetModules()
}
})
it('corrupt ledger file reads as empty (crash-safe recovery)', async () => {
const { readLedger } = await import('../src/sync/ledger.js')
const { mkdirSync, writeFileSync } = await import('fs')
@ -328,21 +417,168 @@ describe('ledger', () => {
expect(existsSync(join(dir, 'sync-ledger.json.tmp'))).toBe(false)
})
it('honors XDG_CACHE_HOME when set', async () => {
it('honors CODEBURN_CACHE_DIR at call time', async () => {
const { writeLedger, readLedger } = await import('../src/sync/ledger.js')
const { existsSync } = await import('fs')
const { join } = await import('path')
const xdgDir = join(process.env.HOME!, 'xdg-cache')
const original = process.env.XDG_CACHE_HOME
const firstDir = join(tmpDir, 'cache-a')
const secondDir = join(tmpDir, 'cache-b')
process.env.CODEBURN_CACHE_DIR = firstDir
writeLedger([{ key: 'first', ts: '2026-07-01T00:00:00Z' }])
process.env.CODEBURN_CACHE_DIR = secondDir
writeLedger([{ key: 'second', ts: '2026-07-02T00:00:00Z' }])
expect(existsSync(join(firstDir, 'sync-ledger.json'))).toBe(true)
expect(existsSync(join(secondDir, 'sync-ledger.json'))).toBe(true)
expect(readLedger().map(e => e.key)).toEqual(['second'])
process.env.CODEBURN_CACHE_DIR = firstDir
expect(readLedger().map(e => e.key)).toEqual(['first'])
})
it('uses the shared default when both overrides are explicitly absent', async () => {
const { writeLedger } = await import('../src/sync/ledger.js')
const { existsSync } = await import('fs')
const { join } = await import('path')
delete process.env.CODEBURN_CACHE_DIR
delete process.env.XDG_CACHE_HOME
writeLedger([{ key: 'default', ts: '2026-07-01T00:00:00Z' }])
expect(existsSync(join(tmpDir, '.cache', 'codeburn', 'sync-ledger.json'))).toBe(true)
})
it('adopts an XDG-only legacy ledger into the canonical default and writes there thereafter', async () => {
const { appendToLedger, readLedger } = await import('../src/sync/ledger.js')
const { existsSync, mkdirSync, readFileSync, writeFileSync } = await import('fs')
const { join } = await import('path')
const xdgDir = join(tmpDir, 'xdg-cache')
const legacyDir = join(xdgDir, 'codeburn')
const canonicalDir = join(tmpDir, '.cache', 'codeburn')
delete process.env.CODEBURN_CACHE_DIR
process.env.XDG_CACHE_HOME = xdgDir
try {
writeLedger([{ key: 'xdg-entry', ts: '2026-07-01T00:00:00Z' }])
expect(existsSync(join(xdgDir, 'codeburn', 'sync-ledger.json'))).toBe(true)
expect(readLedger().map(e => e.key)).toEqual(['xdg-entry'])
} finally {
if (original === undefined) delete process.env.XDG_CACHE_HOME
else process.env.XDG_CACHE_HOME = original
}
mkdirSync(legacyDir, { recursive: true })
writeFileSync(join(legacyDir, 'sync-ledger.json'), JSON.stringify([
{ key: 'legacy', ts: '2026-07-01T00:00:00Z' },
]))
expect(readLedger().map(entry => entry.key)).toEqual(['legacy'])
expect(existsSync(join(canonicalDir, 'sync-ledger.json'))).toBe(true)
expect(existsSync(join(legacyDir, 'sync-ledger.json'))).toBe(false)
appendToLedger([{ key: 'canonical', ts: '2026-07-02T00:00:00Z' }])
expect(JSON.parse(readFileSync(join(canonicalDir, 'sync-ledger.json'), 'utf8')).map((entry: { key: string }) => entry.key)).toEqual([
'legacy',
'canonical',
])
expect(existsSync(join(legacyDir, 'sync-ledger.json'))).toBe(false)
})
it('does not adopt a legacy XDG ledger when CODEBURN_CACHE_DIR is explicitly set', async () => {
const { readLedger, writeLedger } = await import('../src/sync/ledger.js')
const { existsSync, mkdirSync, writeFileSync } = await import('fs')
const explicitDir = join(tmpDir, 'explicit-cache-precedence')
const xdgDir = join(tmpDir, 'xdg-cache-precedence')
const legacyDir = join(xdgDir, 'codeburn')
process.env.CODEBURN_CACHE_DIR = explicitDir
process.env.XDG_CACHE_HOME = xdgDir
mkdirSync(legacyDir, { recursive: true })
writeFileSync(join(legacyDir, 'sync-ledger.json'), JSON.stringify([
{ key: 'legacy', ts: '2026-07-01T00:00:00Z' },
]))
expect(readLedger()).toEqual([])
writeLedger([{ key: 'explicit', ts: '2026-07-02T00:00:00Z' }])
expect(readLedger().map(entry => entry.key)).toEqual(['explicit'])
expect(existsSync(join(explicitDir, 'sync-ledger.json'))).toBe(true)
expect(existsSync(join(legacyDir, 'sync-ledger.json'))).toBe(true)
})
it('merges an XDG legacy ledger into an existing canonical ledger once', async () => {
const { readLedger } = await import('../src/sync/ledger.js')
const { existsSync, mkdirSync, writeFileSync } = await import('fs')
const canonicalDir = join(tmpDir, '.cache', 'codeburn')
const xdgDir = join(tmpDir, 'xdg-cache-merge')
const legacyDir = join(xdgDir, 'codeburn')
delete process.env.CODEBURN_CACHE_DIR
process.env.XDG_CACHE_HOME = xdgDir
mkdirSync(canonicalDir, { recursive: true })
mkdirSync(legacyDir, { recursive: true })
writeFileSync(join(canonicalDir, 'sync-ledger.json'), JSON.stringify([
{ key: 'canonical', ts: '2026-07-02T00:00:00Z' },
{ key: 'duplicate', ts: '2026-07-03T00:00:00Z' },
]))
writeFileSync(join(legacyDir, 'sync-ledger.json'), JSON.stringify([
{ key: 'legacy', ts: '2026-07-01T00:00:00Z' },
{ key: 'duplicate', ts: '2025-01-01T00:00:00Z' },
]))
expect(readLedger()).toEqual([
{ key: 'canonical', ts: '2026-07-02T00:00:00Z' },
{ key: 'duplicate', ts: '2026-07-03T00:00:00Z' },
{ key: 'legacy', ts: '2026-07-01T00:00:00Z' },
])
expect(existsSync(join(legacyDir, 'sync-ledger.json'))).toBe(false)
// A later read is canonical-only and stable; XDG is no longer active.
expect(readLedger().map(entry => entry.key)).toEqual(['canonical', 'duplicate', 'legacy'])
})
it('prefers non-empty CODEBURN_CACHE_DIR over XDG_CACHE_HOME', async () => {
const { writeLedger } = await import('../src/sync/ledger.js')
const { existsSync } = await import('fs')
const { join } = await import('path')
const explicitDir = join(tmpDir, 'explicit-cache')
const xdgDir = join(tmpDir, 'xdg-cache')
process.env.CODEBURN_CACHE_DIR = explicitDir
process.env.XDG_CACHE_HOME = xdgDir
writeLedger([{ key: 'explicit', ts: '2026-07-01T00:00:00Z' }])
expect(existsSync(join(explicitDir, 'sync-ledger.json'))).toBe(true)
expect(existsSync(join(xdgDir, 'codeburn', 'sync-ledger.json'))).toBe(false)
})
it.each(['', ' '])('ignores empty CODEBURN_CACHE_DIR %j and writes to the canonical default', async explicit => {
const { writeLedger } = await import('../src/sync/ledger.js')
const { existsSync } = await import('fs')
const { join } = await import('path')
const xdgDir = join(tmpDir, `xdg-cache-${explicit.length}`)
process.env.CODEBURN_CACHE_DIR = explicit
process.env.XDG_CACHE_HOME = xdgDir
writeLedger([{ key: 'xdg-fallback', ts: '2026-07-01T00:00:00Z' }])
expect(existsSync(join(tmpDir, '.cache', 'codeburn', 'sync-ledger.json'))).toBe(true)
expect(existsSync(join(xdgDir, 'codeburn', 'sync-ledger.json'))).toBe(false)
})
it.each(['', ' '])('ignores empty XDG_CACHE_HOME %j and uses the shared default', async xdg => {
const { writeLedger } = await import('../src/sync/ledger.js')
const { existsSync } = await import('fs')
const { join } = await import('path')
delete process.env.CODEBURN_CACHE_DIR
process.env.XDG_CACHE_HOME = xdg
writeLedger([{ key: 'default-fallback', ts: '2026-07-01T00:00:00Z' }])
expect(existsSync(join(tmpDir, '.cache', 'codeburn', 'sync-ledger.json'))).toBe(true)
})
it.each(['', ' '])('uses the shared default when both overrides are empty and CODEBURN is %j', async explicit => {
const { writeLedger } = await import('../src/sync/ledger.js')
const { existsSync } = await import('fs')
const { join } = await import('path')
process.env.CODEBURN_CACHE_DIR = explicit
process.env.XDG_CACHE_HOME = ' '
writeLedger([{ key: 'default-fallback', ts: '2026-07-01T00:00:00Z' }])
expect(existsSync(join(tmpDir, '.cache', 'codeburn', 'sync-ledger.json'))).toBe(true)
})
})

2
tsup.config.ts
View file

@ -1,7 +1,7 @@
import { defineConfig } from 'tsup'
export default defineConfig({
entry: ['src/main.ts'],
entry: ['src/main.ts', 'src/parse-worker.ts'],
format: ['esm'],
target: 'node20',
outDir: 'dist',