codeburn/docs
iamtoruk 5bc78b8e9e perf(parse-workers): gate on pending bytes, size the per-worker budget per parse
Three fixes from review, all measured on this box.

The workload gate was files OR bytes. The files arm is wrong: 250 pending files
holding 117 KB between them spawned 5 threads and ran ~5% SLOWER than serial,
and a file count only starts paying for itself around 400. Gate on bytes alone;
the count still takes max(files / 50, bytes / 200 MB), so a few hundred huge
rollouts keep their threads.

The flat 256 MB per-worker memory budget was contradicted by the Codex workload:
a 260 MB rollout peaks near 430 MB in its worker, linearly across the pool. It is
now derived per parse as clamp(256 MB, 2 x average pending file + 128 MB, 1 GB),
which leaves a corpus of small Claude transcripts where it was and stops
over-subscribing on rollouts. The parent's buffer of up to pool.size finished
results is part of that peak and is named in the comment.

The worker/file pairing at both install sites was positional, guarded only by
position (Claude) or a path membership check (Codex). Each worker now echoes its
path and the parent asserts it, outside the per-file try: a misalignment would
install one session's turns under another's path -- a wrong number nobody would
ever notice -- so it fails the run rather than being swallowed as a parse
failure. On the Claude side that meant hoisting the whole worker-result block
above the try, which is safe because an append never consumes a result in either
its shortcut or its straddled-fallthrough case.
2026-08-17 02:53:19 -07:00
..
design Scope the root test script to tests/ so npm test runs from a clean install 2026-08-09 10:19:58 -05:00
providers fix(parser): flatten already-sliced previews and memoize canonical path walks 2026-08-16 01:49:10 -07:00
sync fix(sync): close credential-leak paths; session retraction; span/key/CLI hardening 2026-08-02 12:56:59 +00:00
architecture.md perf(parse-workers): gate on pending bytes, size the per-worker budget per parse 2026-08-17 02:53:19 -07:00