Commit graph

4 commits

Author SHA1 Message Date
iamtoruk
0e11b51516 fix(cache): make the shard layout safe against a second live writer
Two live processes share one cache directory routinely (a one-shot CLI
beside the resident serve child, two menubar polls), and the shard layout
had two ways to lose data there.

- The atomic write used a FIXED temp name, so two writers publishing the
  envelope — every save does — shared one `envelope.json.tmp` and
  interleaved into a torn payload, or one deleted the shards the other's
  envelope named. 39 of 40 rounds ended in a total cache loss. The temp
  name carries a nonce again, as it did before the shard layout.
- A save reused a shard filename from its own load snapshot without
  checking the file was still there. Another process republishing that
  provider unlinks the old shard, so the stale writer published an
  envelope naming a deleted file — read back as a corrupt provider and
  dropped whole, including PR-linked orphans no re-parse can recover. A
  reused shard is now existence-checked, and re-verified once more
  immediately before the envelope is published; a vanished one is
  rewritten from memory.

Also:
- Progress saves take a 30s floor beside the file counter. Only the
  claude scan reports per file; every other provider calls saveProgress
  once at its own boundary, so the counter alone never fired there.
- The unreferenced-shard sweep waits an hour (temps still 5 minutes): an
  unreferenced shard may belong to a concurrent save whose envelope has
  not landed yet.
- The sweep also retires the pre-v8 single-file temps in the parent
  directory, which nothing writes anymore.
- The shard directory is created 0o700.
- The claude and provider paths mark the cache dirty where they DELETE a
  stale entry, not only where they replace it: an unreadable file skips
  the replace, and the deletion would otherwise live only in memory.
- Codex only treats a grown file as an append when the recorded boundary
  still lands just after a newline, so a same-inode rewrite that happens
  to end up larger re-parses instead of resuming mid-line.
2026-08-16 19:23:39 -07:00
iamtoruk
0bfdfa372a perf(cache): shard the session cache per provider
A warm launch rewrote the entire session cache whenever any provider
appended a few KB: on a 6 GB corpus that is a 155 MB stringify + fsync
every run. The on-disk cache is now a version-suffixed directory holding
one shard per provider plus a small envelope, and a save rewrites only
the providers marked dirty.

- Dirtiness is tracked per provider (markCacheDirty) instead of one
  global flag, so an appended Claude session no longer republishes
  Codex, Copilot and the rest.
- Shards carry a nonce in their filename and the envelope is renamed
  last, so a save is published at a single point: readers never see a
  half-updated set, and a writer that loses the refresh ownership fence
  leaves the canonical shards untouched.
- A shard that fails validation is treated as an absent provider rather
  than rejecting the whole cache, so one malformed turn costs one
  provider's re-parse instead of every provider's history.
- v7 migrates losslessly: the blob is re-laid-out into shards and
  removed only once that save publishes. Nothing re-parses.
- Cold-parse progress saves now trigger every N files parsed rather than
  every 5s, so a slow cold parse no longer rewrites the growing cache on
  a wall clock.
2026-08-16 19:03:12 -07:00
iamtoruk
ae18b1fd91 fix: audit batch — prefetch one-shot, turn-anchored day bucketing, warm-path collapse
- prefetch fires exactly once per provider per session; memo sized to
  detected providers so base keys never evict (was: 12 full-history
  parses every 30s forever from LRU thrash re-triggering the effect)
- day-aggregator buckets whole turns by user-message timestamp,
  matching the report/headline path: trend bars and provider breakdown
  now reconcile exactly (midnight-straddling turns were split);
  DAILY_CACHE_VERSION 13 with an IANA tz guard that rehydrates when
  the cache crosses timezones
- overview's redundant second all-provider scan collapsed (~11% faster
  warm); numbers parity-guarded
- lock takeover cleans stale hydrating.lock; SIGINT/SIGTERM release it
- scanner skips EPERM/EACCES provider dirs (progress shows skipped)
  instead of aborting; onboarding hints Full Disk Access
- rateLimited quota copy render tests; export -f json emits valid
  empty JSON when no data

Root 2023 (+4), app 327 (+3), typechecks clean.
2026-07-16 15:37:03 -07:00
iamtoruk
300e949951 feat(app): refresh cadence, instant switching, cache coherence, telemetry v1
Performance and coherence:
- Settings > General 'Refresh every' (Manual/30s/1m/3m/5m/10m), live via
  RefreshCadenceContext; Manual polls only on demand
- usePolled memoKey LRU: provider/period switches paint the last-good
  result instantly with a switching hairline while refreshing quietly
- quota TTL 5min + honest rate-limited copy on 429 backoff
- version-suffixed cache files (session-cache.v5.json, daily-cache.v12,
  auto-minted on future bumps); legacy files never written or deleted,
  adopted once when versions match: old and new binaries coexist
  without clobbering (field-observed menubar-vs-desktop ping-pong)
- advisory hydration lock: concurrent cold starts share one scan
  (wait-then-read-warm), stale/dead locks self-heal, never a
  correctness gate

Telemetry v1 + onboarding (desktop only, per owner decisions):
- first-launch onboarding (3 feature screens + consent); region-split
  default (EU/EEA/UK/CH off, elsewhere on, unknown off); nothing sends
  before consent completion or while off; dev builds never send
- anonymous install UUID, rotated on opt-out; day-granularity events,
  cost buckets only; whitelisted names; 200-event queue, 5min flush
- Settings > Privacy live toggle replaces the static claim

Zero computed-number changes. App 316/316, root 1805.

Wire contract targets api.codeburn.app/v1/telemetry (Worker follows).
2026-07-16 11:47:05 -07:00