Commit graph

1401 commits

Author SHA1 Message Date
iamtoruk
527e58078d menubar: install and launch the Windows tray app from codeburn menubar
Shares the mac release resolution behind a per-platform spec (tag prefix, asset name, error text), so the Windows path reuses the pinned-version URL, the release-API fallback scan, the retrying download and the sha256 verify unchanged. Windows then runs msiexec out of %SystemRoot%\System32 with /i /passive /norestart, treats 3010 and 1602 as non-failures, and launches the exe named by the product's Uninstall registry key.
2026-08-18 06:55:09 -07:00
Resham Joshi
9905e8f8d0
Merge pull request #1024 from getagentseal/feat/windows-menubar
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (22) (push) Waiting to run
Tests / test (22.13.0) (push) Waiting to run
Windows Menubar CI / check (ubuntu-latest) (push) Waiting to run
Windows Menubar CI / check (windows-latest) (push) Waiting to run
windows: Windows menubar (Tauri tray app) at 0.9.20, with CI and release workflow
2026-08-18 04:45:46 -07:00
Resham Joshi
bfd21db37a
Merge pull request #1025 from getagentseal/test/vitest-global-timeout
test: raise vitest testTimeout to 30s
2026-08-18 04:41:18 -07:00
iamtoruk
76c63cbfd1 windows: cfg-gate the tray badge so the Linux build has no dead code
The spend badge is a second tray icon carrying the number as its bitmap, which
only the Tauri tray backend provides; Linux runs its own SNI tray and has no
equivalent. The module was compiled there anyway, so every item in it - and the
two tray ids in lib.rs - tripped dead_code under clippy -D warnings on the
ubuntu leg.

- `mod tray_badge` and TRAY_ID / BADGE_TRAY_ID are now cfg(not(linux)), so the
  code is absent on Linux rather than present and unused.
- `set_tray_badge` reports the badge as unsupported on Linux instead of
  returning a success that never happened.
- The frontend hides the control wherever it is unsupported, behind
  TRAY_BADGE_SUPPORTED in lib/platform.ts - one constant, three call sites
  (the settings row, the footer menu item, and the effect that would otherwise
  invoke the command).
- AppState's `linux_tray` field is dropped: it was written and never read, which
  is the same lint one file over. init_tray_linux already owns the handle.

Checked rather than reasoned: `cargo clippy --all-targets -- -D warnings` passes
on macOS, and a scratch copy of the crate with the linux/macos cfg arms swapped
(so a normal macOS clippy selects everything a Linux build would keep, and drops
everything it would drop) also passes. Reinstating the ungated `mod tray_badge`
in that copy reproduces the exact CI failure, so the check is real. It stubs
tray_linux.rs, whose ksni and png deps do not build here - that file is still
only covered by CI's ubuntu leg.
2026-08-18 04:37:27 -07:00
iamtoruk
30037b331f test: raise vitest testTimeout to 30s
The 5s default failed three unrelated real-I/O tests on the CI Node-22 runner
in two days (dashboard optimize scan, codex resume differential, OTel DB
prune) purely from runner load. Hung tests still fail at 30s.
2026-08-18 04:32:20 -07:00
iamtoruk
562bab65ba docs: surface the Windows menubar
README gains a Windows section next to the macOS and GNOME ones, a download
badge on the menubar card, and an honest note that the Tauri tray builds on
Linux but is unreleased there. docs/architecture.md picks up windows/ in the
surfaces diagram and gets a section covering the crate layout, the PATH and
System32 spawn rules, and the Claude quota parity.

`codeburn menubar` on Windows now points at the windows-v release page instead
of failing with "macOS only". The generalized installer from the source branch
is not brought over: it is 1182 commits behind this file and would drop the
proxy support, retry/backoff, checksum and bundle verification, and persistent
CLI path handling that landed since.
2026-08-18 04:14:05 -07:00
iamtoruk
120747d2fa ci: build and release the Windows menubar
windows-menubar-ci.yml runs on windows/** changes: tsc, clippy with -D warnings,
and cargo test on both windows-latest and ubuntu-latest, plus a release-profile
`tauri build --no-bundle` on Windows. The Linux leg exists because most of the
crate's cfg(windows) code cannot be compiled anywhere else, so the reverse -
keeping the ksni paths and every shared helper clean off Windows - has to be
checked somewhere too.

release-menubar-windows.yml mirrors release-menubar.yml: a `windows-v*` tag (or
a manual dispatch) builds the MSI and publishes it, with a sha256, to a
"Windows Menubar vX" release.

The two release-desktop-*.yml files on the source branch are deliberately not
imported: their tag names and "Desktop" release titles would collide with the
Electron app in app/, which already owns `desktop-v*`.
2026-08-18 04:11:32 -07:00
iamtoruk
7e57fb8d4f windows: fix the security and correctness findings from the import audit
Process spawning (Windows searches the current directory before PATH):

- reg.exe and cmd.exe are now spawned by absolute System32 path via
  cli::system_command, which also carries the CREATE_NO_WINDOW flag the three
  previous copies each set by hand. The tray badge re-ran `reg query` on every
  refresh, so this was the most reliably reachable planted-binary path.
- "Connect Claude" resolves the claude binary itself instead of handing a bare
  name to the console shell.
- CLI lookup ignores empty and relative PATH entries. `;;` or a trailing `;`
  used to yield PathBuf::from("").join("codeburn.cmd"), a current-directory
  lookup, at startup. The filter lives in one place (find_in_dirs) that every
  search - codeburn, claude, and the Linux terminal probe - goes through.
- The Linux terminal path re-validates the whole command against the argument
  allowlist before joining it into the string `bash -lc` parses; anything that
  fails falls through to the argv-only detached spawn.

CLI version gate:

- MIN_CLI_VERSION moves from 0.7.0 to 0.9.9, the first release accepting
  `status --format menubar-json --no-optimize` (every quiet refresh passes it)
  and emitting all the payload fields the popover reads.
- The gate is probed on mount, before the first fetch. It previously only ran
  when a fetch failed with the literal "CLI not found", so an old CLI produced
  a payload whose missing history.daily threw and blanked the popover.
- A successful fetch no longer flips an incompatible CLI to compatible, and the
  settings panel no longer probes on its own - App owns the verdict, so a
  transient probe failure there cannot drop a working app onto the setup screen.
- The payload reads App makes are optional now, so a surprising payload lands on
  an empty state rather than a blank window.

Refresh cadence, mirroring mac RefreshCadence.swift: 60s with optimize findings
while the popover is visible, 120s today/all without them while it is hidden,
and an immediate refresh on show when the visible key is stale. Every hide path
in lib.rs now goes through mark_hidden so the debounce stamp and the frontend
signal cannot drift. Previously it was 60s with optimize regardless of
visibility - about 2880 CLI spawns a day.

Claude quota (plan.rs) stops calling the token refresh endpoint. Claude's
refresh token is single-use and rotates, so spending it invalidated the token
Claude Code itself holds and broke the user's login. Parity with
ClaudeCredentialStore.refreshAfter401: re-read Claude's own store for a token it
has already rotated, and report a transient failure when there is not one yet.

Smaller:

- Snapshot writes refuse a symlinked target and are 0600 on unix, mirroring
  mac SafeFile; no home directory now means no snapshots rather than a file
  dropped in whatever directory the tray was launched from.
- The Windows config lock keeps its file handle open, so the stale sweep can
  only ever unlink a lock whose owner is gone. The doc comment now says what the
  lock actually buys (the CLI never takes it).
- "updated Xs ago" is only stamped by a fetch of the key on screen.
- External links go through tauri-plugin-opener instead of target=_blank.
- The tray badge font is loaded once into a OnceLock instead of read and parsed
  on every render.
- autostart's shared import and constant are cfg-gated so clippy is clean on
  every OS.

Adds unit tests for the PATH filter and the version gate.
2026-08-18 04:10:07 -07:00
iamtoruk
b199af182e windows: import the Tauri tray app as the Windows menubar
Brings the Tauri 2.x tray popover from #1022 onto main as windows/, mirroring
mac/. Product name, bundle identifier, and version line up with the macOS
menubar (org.agentseal.codeburn-menubar, 0.9.20); the crate is renamed off
"desktop" so it no longer collides with the Electron app in app/.

Linux (ksni) stays compiled and dev-usable but is documented as experimental:
gnome/ is the shipping Linux surface.

The five src/ CLI commits on that branch are dropped - they re-implement a
daily-bucketing fix main already carries.
2026-08-18 04:00:59 -07:00
Resham Joshi
a0ada25104
Merge pull request #1021 from getagentseal/feat/optimize-recurring-context
optimize: detect recurring context pasted at the start of sessions
2026-08-18 03:35:21 -07:00
iamtoruk
b249ba1ffe Merge remote-tracking branch 'origin/main' into feat/optimize-recurring-context
# Conflicts:
#	CHANGELOG.md
2026-08-18 03:31:16 -07:00
Resham Joshi
95d5fa6269
Merge pull request #1020 from getagentseal/feat/optimize-verify-after-apply
optimize: re-measure applied fixes and report worked / partial / no effect
2026-08-18 03:30:54 -07:00
iamtoruk
e3a55cdf20 optimize: count only human pastes as recurring context
An SDK session's opening prompt and a subagent's task prompt are written by
a program: they repeat by design and have no home in CLAUDE.md. Both are
flagged on the entry, but a user entry over the parser's large-line
threshold comes back without its root flags - routine for generated prompts,
which are exactly the long ones - so the markers are read off the ends of
the raw line, where the fields sit either side of the oversized message.
2026-08-18 03:27:10 -07:00
iamtoruk
f8bc9b2594 docs: cover what happens after optimize --apply
docs/optimize.md gains an "After you apply" section (the four verdicts,
--auto-revert, what is never auto-reverted); README and CHANGELOG follow.
2026-08-18 03:15:12 -07:00
iamtoruk
3936284811 optimize: show the applied-fix verdicts in the TUI and desktop app
Same section, compact: one line per still-applied fix with the verdict
glyph, and the undo command for the ones that measured nothing. The app
reads appliedFixes[] off the optimize JSON, tolerating its absence from
an older CLI.
2026-08-18 03:15:12 -07:00
iamtoruk
52eb9fdb8d optimize: re-measure applied fixes on every run, with --auto-revert
Every still-applied journal entry now comes back with a verdict on the
next optimize run: worked (>=70% of its window-scaled estimate
realized), partial, no-effect (printed with its undo command), or
measuring while it is younger than the 3-day window. The verdicts come
off the rows act report already computes, so there is one
reconciliation, not two; the AppliedFix type and its formatter live in
act/types.ts so the optimize renderer can use them without importing
report.ts back into optimize.ts.

--auto-revert undoes the no-effect entries through the same code path as
codeburn act undo. It never touches partial or measuring entries, and
never a claude-md-rule - those land in whatever directory the user
happened to be in, the same reason --yes skips them.

--apply now names when the re-measure happens, and --format json carries
appliedFixes[] (add-only).
2026-08-18 03:15:07 -07:00
iamtoruk
8d8848d805 optimize: detect recurring context pasted at the start of sessions
Groups sessions by their opening block (whitespace/ANSI-normalized, hashed
over the first 2 KB) and flags a block of at least 1.5 KB that opens five or
more sessions. Class nudge: CodeBurn will not move the user's own text into
CLAUDE.md, so the fix asks Claude to give the block a permanent home. Only
the repeats count as savings, sized from the block's bytes because provider
usage is per API call and cannot isolate the paste. The opener comes from
the session scan that already runs, so nothing extra is read.
2026-08-18 03:13:40 -07:00
Resham Joshi
117d66f7f7
Merge pull request #1019 from getagentseal/feat/optimize-finding-class
optimize: classify findings as fix/nudge/keep and mark measured vs estimated
2026-08-18 02:44:47 -07:00
iamtoruk
0aed84f4ca Merge remote-tracking branch 'origin/main' into feat/optimize-finding-class
# Conflicts:
#	tests/dashboard.test.ts
2026-08-18 02:40:53 -07:00
iamtoruk
5660909801 optimize: per-group subtotals in every finding render
Each class header now carries its own token/dollar subtotal and finding
count, so the apply-able slice is never mistaken for the whole board; the
headline savings line names that slice explicitly. CLI and TUI share one
classHeaderLine helper, the desktop app reads the same numbers from the
new summary.byClass in --format json (add-only; the three subtotals sum to
findingCount and potentialSavingsTokens).

Also scopes the SHELL_PROFILE_SCOPE comment to what is actually true: the
MCP deferral plans refuse to rewrite a shell profile, but bash-output-cap
appends its own marker block to one.
2026-08-18 02:40:00 -07:00
Resham Joshi
027bc9d1f5
Merge pull request #1018 from getagentseal/test/dashboard-optimize-refresh-flake
test(dashboard): wait for the Optimize scan on real event-loop turns
2026-08-18 02:36:09 -07:00
iamtoruk
6267c49c25 docs: document optimize classes, provenance, and what --apply writes
Adds docs/optimize.md (what optimize scans, the three classes, the exact
files --apply may touch plus undo, measured vs estimated, the health
grade bands, the --yes CLAUDE.md guardrail), links it from the README
waste section, and corrects the detector count in docs/architecture.md
(14 -> 19).
2026-08-18 02:27:01 -07:00
iamtoruk
7c54cf85c2 optimize: classify findings as fix/nudge/keep and mark measured vs estimated
Every finding now resolves to a class (apply-able fix, habit nudge, or
informational keep) and a basis (measured from provider-counted usage, or
estimated from a schema/heuristic model), both from one table next to the
FindingId union. The class follows the plan layer: an id is 'fix' only when
buildPlan routes it, and an instance drops to 'nudge' when it lacks the
payload or cause its builder needs.

CLI, TUI and the desktop app group findings under Fix now / Habits / FYI
with continuous numbering; the CLI header reports 'N measured · M
estimated' in place of the blanket 'Estimates only.' footer. The JSON
report gains class + basis per finding and summary.measuredSavingsUSD;
existing fields are unchanged. The menubar's top three follow the same
order, since every surface reads the sorted findings list.

Sessions whose cost the provider never reported leave the cost-outliers
peer math; when nothing else is priced the comparison falls back to them
and the finding reports itself as estimated instead of disappearing.
2026-08-18 02:26:57 -07:00
iamtoruk
e2007c5e2f test(dashboard): wait for the Optimize scan on real event-loop turns, not fake-timer hops
The Optimize scan does real fs I/O (readdir/stat) that only resolves on a
real event-loop turn, but the wait loop counted 20 vi.advanceTimersByTimeAsync
hops under full fake timers, which flush fake timers + microtasks but never
give real I/O a chance to complete. Under load that read a stale
"Scanning Today..." frame. Scope fake timers to just what the 60s
auto-refresh interval needs, leave setImmediate/Date real, and wait on a
real wall-clock deadline instead of a fixed hop count.
2026-08-18 02:10:24 -07:00
Resham Joshi
d5b3720079
Merge pull request #1001 from MiloMMIN/feat/dsh-provider
feat: add DeepSeek Harness (dsh) provider
2026-08-18 01:54:48 -07:00
Emre K.
7e421b14bb
Add unpriced filter to models report (#985)
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (push) Waiting to run
2026-08-18 04:38:19 +03:00
Dongmin,Yu
cbb09d0170
fix(optimize): scope transcript-derived findings to the selected provider (#1003)
* fix(optimize): scope transcript-derived findings to the selected provider

scanSessions() always ran discoverAllSessions('claude'), so every finding it
feeds was computed from Claude transcripts regardless of --provider, while the
header (sessions, calls, cost) came from the already-filtered projects. Under
--provider codex the two described different providers, and the Claude-derived
numbers read as the selected provider's.

Skip the scan when the filter excludes Claude, and skip the detectors it feeds
rather than handing them an empty scan: emptiness reads as "never invoked", so
an empty scan turned every skill, agent and command into a reported ghost.
Findings derived from projects (MCP tool coverage, capability reliability,
low-worth sessions, context bloat, outliers, model recommendations) already
filter correctly and still run.

The result cache key now carries the provider, since the provider decides
whether the scan runs at all.

* fix(optimize): thread the provider through the apply, aggregator and TUI scans

The previous commit fixed one of four scanAndDetect callers. The other three
carry a provider filter and dropped it:

- act/optimize-apply.ts: `optimize --apply` branches in main.ts before the
  code that threads it, so a Codex-scoped run planned applies off Claude
  findings. This is the worst of the three because `unused-skills` is
  appliable and its plan moves directories out of ~/.claude/skills.
- usage-aggregator.ts: AggregateOpts.provider was already honoured for the
  usage half but not for the optimize half, so the menubar, desktop and web
  surfaces carried the same mismatch.
- dashboard.tsx: `p` cycles activeProvider and `o` opens optimize off the
  same state, so the TUI could show Claude findings under a Codex view.
  activeProvider joins the callback deps; reloadData already clears
  optimizeResult on a provider switch, so no extra invalidation is needed.

Covered by a dry-run test on the apply path: under `provider: 'codex'` a fake
home holding an uninvoked skill must plan nothing, and under 'claude' the same
fixture must still plan the archive.
2026-08-18 04:30:11 +03:00
Aditya Vikram Singh
eadc99ef99
fix(models): correct Sonnet 4 thinking alias (#982)
Co-authored-by: Aditya Vikram Singh <247195684+avs-io@users.noreply.github.com>
2026-08-18 03:58:50 +03:00
iamtoruk
09965f93ae fix(dsh): cap zstd decode, coerce usage fields, scope the snap read
Security audit follow-ups on the DeepSeek Harness provider.

- **Decompression bomb.** Every zstd frame was decoded with no output bound, so
  a 16 KB crafted log expanded to ~916 MB of RSS (a 65 KB one declares 2 GB).
  Each frame now decodes under a 64 MB per-call cap, and the caps chain into a
  running per-file budget of MAX_SESSION_FILE_BYTES: a frame is given only the
  bytes the file has left, so node throws ERR_BUFFER_TOO_LARGE without
  allocating past the cap. The throw propagates out of the existing skip path,
  which discards the whole file rather than counting the frames read before the
  bomb, so a crafted tail cannot poison a partial total. The discovery header
  read takes the same per-frame cap. Measured on a 65 KB / 2 GB bomb: 916 MB
  -> 67 MB peak, zero calls emitted, one notice.
  Lines are still materialized eagerly; the byte budget bounds that, and making
  the read lazy would change readEventLines' contract for no further bound.
- **Usage type confusion.** Token fields were read with `?? 0` and never
  type-checked, so a string or array inputTokens flowed into the global totals
  and the persisted cache, where `0 + [1, 2]` becomes "01,2". They now go
  through numberOrZero (copilot.ts semantics: finite, positive, else 0).
  All-zero calls are still skipped.
- **Snap over-scope.** The personal-files read entry is `$HOME/.dsh/sessions`
  rather than all of `$HOME/.dsh`; the provider reads nothing else.
- **Third-party notice.** scanZstdFrames is transcribed from
  @deepseek-ai/dsh-session-persistence-jsonl. The published npm package is
  BSD-3-Clause (Copyright (c) 2026, DeepSeek) while the monorepo source
  declares MIT for the same package; THIRD_PARTY_NOTICES.md reproduces the
  stricter of the two and ships via package.json `files`.
2026-08-17 17:28:41 -07:00
iamtoruk
ffd9213126 test(dsh): skip zstd container tests where node:zlib lacks zstd, cover both Node lines in CI
zstd landed in node:zlib in 22.15; the package floor and CI pin are 22.13, so
the runtime already degrades with a notice there. The tests compressed
fixtures at runtime and failed outright. Fixture writes now fall back to
plain jsonl below 22.15 so parsing semantics still run, container-specific
tests skip, and the Tests workflow runs on both 22.13.0 and latest 22.x.
2026-08-17 14:32:55 -07:00
iamtoruk
cf75d7c735 ci: retrigger after GitHub outage 2026-08-17 14:17:47 -07:00
iamtoruk
eb8ceeb867 fix(dsh): bound the two remaining unbounded reads and the version notice
- The unsupported-version notice is keyed on the version rather than the path:
  a DSH format bump makes every session unreadable at once, and one stderr
  line per session log is noise.
- The discovery header read falls back to reading the whole file when a 256 KB
  head does not cover one full zstd frame. A fork's first write batch carries
  the entire inherited seed, so that is reachable on a real log; it now takes
  the same oversize guard as the parse read.
2026-08-17 11:01:12 -07:00
iamtoruk
d9a9486b6d docs(dsh): finish the provider registration checklist
docs/providers/NEW_PROVIDER.md items the PR had not reached yet, plus the two
surfaces that are functional rather than cosmetic:

- docs/providers/dsh.md and its row in the provider index, documenting the
  storage layout, the JSONL-backend-only scope (the opt-in SQLite persistence
  backend is not read), and that DSH is a developer preview whose format
  version 0 implies no compatibility.
- CHANGELOG entry under Unreleased.
- README provider count 40 -> 41 and a data-locations row.
- app/package.json: $HOME/.dsh in the snap personal-files allowlist, without
  which the Linux snap build cannot read DSH sessions at all.
- UsageDataChangeGuard: the DSH sessions root, without which the menubar never
  notices a new session and does not refresh.
- Bumps the dsh parse version, since the parser's attribution changed.
2026-08-17 10:59:58 -07:00
iamtoruk
4fa16a2293 fix(dsh): correct usage attribution against the real session format
Reviewed src/providers/dsh.ts against deepseek-harness @ 99f6f02f and fixed
what the format says but the parser did not:

- A forked session's log replays its parent's events verbatim, and codeburn
  parses the parent's own log as its own session, so every inherited call was
  billed twice. The header's parentSession + seedLength mark that prefix;
  events with seq < seedLength are now skipped.
- The model now comes from the reporting assistant/message's own
  message.source, which is what actually served the step. request/header only
  describes the request DSH was about to make, and is the fallback.
- user/message also carries agent-injected context (runtime snapshots, skill
  bodies) under source.kind 'plugin'; only a typed prompt becomes the preview,
  and it is bounded to 500 chars like every other provider rather than holding
  a whole injected system prompt per turn.
- A log stamped with a session format version other than 0 is skipped with a
  notice. The format is pinned at 0 upstream with no compatibility implied, so
  reading a bumped format under today's assumptions would report confident
  wrong numbers.
- Timestamps go through the seconds-vs-milliseconds guard and fall back to the
  header createdAt, so a call can no longer carry an empty timestamp and land
  in the undated cache shard.
- The compressed read buffers the whole log to scan its frames, so it now takes
  the same oversize guard readSessionFile applies to the uncompressed variant.
- The zstd-unavailable notice fired once per session log; each distinct notice
  is now emitted once.
- Emit workingDirectory beside projectPath, as codex does.

Tests add the upstream examples/acp-agent snapshot as a fixture, covering the
real record shapes: packed reasoning-chunks/tool-call-chunks storage rows, a
plugin-injected user/message beside the typed one, and both the streamed usage
chunk and the final assistant/message usage for the same step. Plus the same
snapshot re-encoded as multi-frame zstd with a torn tail (identical output), a
forked session, an unsupported format version, and unparsable lines.
2026-08-17 10:59:52 -07:00
iamtoruk
56e291fc7c Merge remote-tracking branch 'origin/main' into pr1001-review 2026-08-17 10:49:29 -07:00
Resham Joshi
da3d903887
Merge pull request #1010 from getagentseal/perf/parallel-codex-parse
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (push) Waiting to run
perf(parser): parallelize the cold Codex rollout parse on the same worker pool
2026-08-17 02:56:56 -07:00
iamtoruk
5bc78b8e9e perf(parse-workers): gate on pending bytes, size the per-worker budget per parse
Three fixes from review, all measured on this box.

The workload gate was files OR bytes. The files arm is wrong: 250 pending files
holding 117 KB between them spawned 5 threads and ran ~5% SLOWER than serial,
and a file count only starts paying for itself around 400. Gate on bytes alone;
the count still takes max(files / 50, bytes / 200 MB), so a few hundred huge
rollouts keep their threads.

The flat 256 MB per-worker memory budget was contradicted by the Codex workload:
a 260 MB rollout peaks near 430 MB in its worker, linearly across the pool. It is
now derived per parse as clamp(256 MB, 2 x average pending file + 128 MB, 1 GB),
which leaves a corpus of small Claude transcripts where it was and stops
over-subscribing on rollouts. The parent's buffer of up to pool.size finished
results is part of that peak and is named in the comment.

The worker/file pairing at both install sites was positional, guarded only by
position (Claude) or a path membership check (Codex). Each worker now echoes its
path and the parent asserts it, outside the per-file try: a misalignment would
install one session's turns under another's path -- a wrong number nobody would
ever notice -- so it fails the run rather than being swallowed as a parse
failure. On the Claude side that meant hoisting the whole worker-result block
above the try, which is safe because an append never consumes a result in either
its shortcut or its straddled-fallthrough case.
2026-08-17 02:53:19 -07:00
iamtoruk
2d873c2290 docs: describe the Codex half of the parallel cold parse 2026-08-17 02:30:20 -07:00
iamtoruk
5b3b993f06 test(parse-workers): cover the Codex worker path end to end
A mixed Claude + Codex corpus with forked rollouts in both creation orders,
asserting an identical payload, byte-identical cache shards and a byte-identical
codex-results.json between CODEBURN_PARSE_WORKERS=0 and =3, with the codex
discard count pinned above zero so the overlap path is really exercised.

Plus: a resumable rollout never reaching a worker (the decision line reports no
full parses pending after an append), the off-thread decode matching
parseCodexFileFull exactly including the cache entry it hands back, no cache file
written by the decode itself, no leaked threads, and the files-OR-bytes gate.
2026-08-17 02:30:20 -07:00
iamtoruk
76460bcb35 perf(parser): parallelize the cold Codex rollout parse on the same worker pool
Codex is the bigger half of a real cold parse (4 GB of rollouts against 1.8 GB
of Claude sessions) and was still decoding one file at a time.

A whole-file rollout decode now runs on the #1008 pool. parseCodexFileFull is
the serial decode with the codex cache switched off: no hit lookup, and the
entry it would have written comes back to the parent instead. The worker runs it
against an EMPTY dedup set and returns the calls, the keys it claimed, and that
entry; the parent installs all three in the serial loop's order, so an empty key
intersection is the proof that a serial parse would have dropped nothing either.
On overlap the whole file is discarded and re-parsed in-process -- which is what
makes a forked rollout safe, since it replays its parent's token_count history
under the parent's key namespace and collides outright.

Nothing cross-file moves off the main thread: the dedup set, canonical project
paths and the codex cache's per-directory state all stay in the parent, and a
file the cache can serve exactly or resume into from a byte offset never reaches
a worker. The decision is per provider -- the Claude scan and the provider loop
run one after the other, so at most one pool is alive -- and the pool is
terminated when its scan ends.

The workload gate is now files OR bytes rather than both, and the count takes
max(files / 50, bytes / 200 MB): a corpus of a few hundred multi-hundred-MB
rollouts is as parallelisable as a few thousand small transcripts, and would
otherwise have earned one thread or none.
2026-08-17 02:30:14 -07:00
Resham Joshi
9421b41369
Merge pull request #1009 from getagentseal/test/codex-resume-timeout
test(codex): explicit timeout for the every-boundary resume differential
2026-08-17 01:59:06 -07:00
iamtoruk
1acdecdebf test(codex): give the every-boundary differential an explicit timeout
110 resume splits take ~1.3s locally but exceeded vitest's 5s default on the
CI runner once the parallel suite also hosts the parse-worker tests.
2026-08-17 01:48:04 -07:00
Resham Joshi
569030e9dd
Merge pull request #1008 from getagentseal/perf/parallel-cold-parse
perf(parser): parallelize the cold Claude parse across worker threads, hardware-adaptively
2026-08-17 01:47:21 -07:00
iamtoruk
ef636472f4 review: pin the discard invariant in comment, test and verbose output
The comment at the install site claimed only that an overlapping worker result
'is discarded'. State why the empty-set result is installable at all — an empty
id intersection is proof a serial parse would have dropped nothing — and why the
tempting shortcut is wrong: parsedTurnsToCachedTurns delta-encodes gitBranch
across turns, so dropping one turn changes whether a LATER turn carries a
gitBranch key. Overlap discards the whole file, never individual turns.

Tests: the end-to-end determinism check now runs both parses over the SAME
corpus, so cache shard BODIES are compared byte for byte instead of just their
keys, and a new resumed-session fixture (a transcript restating another file's
message ids, in both filename orders) makes install order decide the answer.
Verified by mutation: removing the discard guard fails it, and yielding worker
results out of order fails it.

CODEBURN_VERBOSE now reports how many worker results were re-parsed in-process
on id overlap, which is what the new test asserts on. The worker bundle's source
map is excluded from the published package (-1.8 MB).
2026-08-17 01:43:51 -07:00
iamtoruk
b99744bf93 fix(parser): gate parse workers on available memory, not free memory
os.freemem() reports free pages on macOS, not available memory: on an idle
128 GB machine it reads a few hundred MB, so the 2 GB gate switched the worker
pool on and off between runs on the platform the desktop app ships to. The gate
and the budget now use process.availableMemory() (cgroup/rlimit-aware in a
container), falling back to os.totalmem(): serial under 4 GB available, budget
min(0.25 * available, 2 GB). An 8 GB box earns 8 threads, a 4 GB box none.

The verbose line now carries every decision input — cores, available GB, pending
files and bytes — on both the gate and the go path, so one support log explains
itself.
2026-08-17 01:23:42 -07:00
iamtoruk
da68055c9e docs: document the parallel cold parse and CODEBURN_PARSE_WORKERS 2026-08-17 01:19:25 -07:00
iamtoruk
3d6d0ab40a test(parser): cover parse-worker policy, ordering and determinism
Pins the gates that keep threads off low-spec machines and warm runs, that a
forced worker count bypasses them, that results come back in submission order,
that a dead pool reports failure instead of throwing (and the serial fallback
lands on the same result), that no thread outlives a parse across back-to-back
parses, and that a cold CLI parse with and without workers produces the same
payload and the same cache shards.
2026-08-17 01:19:25 -07:00
iamtoruk
cb9fa71623 perf(parser): parallelize the cold Claude parse across worker threads
Reading, decoding and line-parsing a Claude session JSONL is per-file work that
touches nothing shared, so it moves onto worker_threads for a large cold parse.
parseClaudeFileFull() is the extracted unit both sides run; a worker runs it
against an empty dedup set and returns the result as a JSON string, and the
parent installs results in the order the serial loop would. Everything with
cross-file state stays on the main thread, and a file whose message ids were
already claimed (or whose worker failed) re-parses in-process, so the output is
identical to the serial path.

Thread count is decided per parse: never with <=2 cores, under 2 GB free memory,
fewer than 200 pending whole-file re-parses or under 200 MB behind them, so warm
and incremental runs spawn nothing. CODEBURN_PARSE_WORKERS overrides it. The pool
is terminated when the parse ends, so the resident serve child accumulates no
threads.
2026-08-17 01:19:20 -07:00
Resham Joshi
980199b5a1
Merge pull request #1007 from getagentseal/perf/cache-month-shards
perf(cache): shard the session cache by provider and month, load only the months a query needs
2026-08-17 00:51:46 -07:00
iamtoruk
148e7b2151 fix(cache): close three data-integrity holes in the month-shard layout
A file's shard span was read off turns[0]/turns[-1], but several providers
emit turns non-chronologically (cursor by ROWID, goose/crush/copilot by a
DESC ordering). That produced until < bucket - an empty span, so the shard
was unreachable at every scope and its sessions re-parsed every run.
cacheFileSpan now takes the min and max month over all turns.

An entry re-bucketing out of a month the run never loaded (a re-parse that
moved its oldest turn, or the #441 failure marker that has no turns at all)
left the old copy in the carried shard, so one path lived in two shards and
a later load could resolve to the stale one. A save now prunes those paths
from the shards it carries, and a load merges shards in envelope order,
resolving any duplicate to the freshest fingerprint and dirtying both
buckets so the next save retires the loser.

A carried month whose shard another writer had republished was dropped from
the envelope outright, losing expired-transcript PR orphans no re-parse can
recover. The envelope is now re-read just before publishing and the current
shard name adopted; a ref is dropped only when that envelope lacks it too.
The same re-read moves every merge read after the ownership fence and gives
the merge one optimistic retry, so the read-modify-write window shrinks to
the publish itself.

Also: retire an orphaned v8 directory / v7 file left by an interrupted
re-layout, age-guarded, once a v9 envelope is published.
2026-08-17 00:42:39 -07:00