Commit graph

4 commits

Author SHA1 Message Date
iamtoruk
2fb078bdfb Fix V8 OOM crash on 30-day period with Buffer-based line reader and large-line parser
Three-layer fix for V8 heap exhaustion when parsing heavy session data:

1. Buffer-based readSessionLines (fs-utils.ts): Replace readline with raw
   Buffer streaming using Buffer.indexOf(0x0a). Eliminates ConsString trees
   that caused OOM when regex-flattening 100MB+ lines. Two-state machine
   (ACCUMULATING/SCANNING) skips old lines at ~2KB cost instead of 200MB.

2. Large-line streaming parser (parser.ts): Hand-written JSON scanner for
   lines >32KB extracts only cost/token/tool fields without JSON.parse,
   avoiding full object graph allocation. Dual string/Buffer paths.

3. Dashboard memory management (dashboard.tsx): Disable auto-refresh for
   heavy periods (30d/month/all), clear old dataset before reload via
   nextTick to allow GC, prevent overlapping reloads with mutex, lazy
   optimize scanning on keypress instead of useEffect.

Also fixes three race conditions in dashboard reload deduplication:
- Early return after nextTick bypassing finally block (permanent mutex lock)
- A->B->A period switching dropping final reload (stale pending)
- Stale pendingReloadRef not cleared when in-flight matches request
2026-05-15 23:15:26 -07:00
Łukasz Majcher
5e49f17e64 fix: switch scanJsonlFile and parseSessionFile to readSessionLines to prevent OOM
readViaStream (used for files ≥8 MB) reconstructs the full file as a
single string via chunks.join('\n'), giving the same peak allocation as
readFile. Callers then call content.split('\n'), creating a second copy.
With FILE_READ_CONCURRENCY=16 and files up to 128 MB this can exhaust
the V8 heap (~6 GB theoretical peak).

readSessionLines already exists as a proper async generator that yields
one line at a time. Switch both hot-path callers to iterate it directly
so the full file string is never held in memory.

Adds two tests: a spy test confirming readSessionLines is called (not
readSessionFile), and a 500-entry correctness test.

Fixes #131
2026-04-22 10:11:13 +00:00
Ninym
bd71377fdd docs(optimize): remove references to non-existent .claudeignore
Claude Code does not document or implement a .claudeignore feature.
The junk-reads detector's fix is now a CLAUDE.md instruction asking
Claude to avoid generated/dependency directories. The separate
detectMissingClaudeignore finding and its tests are removed; checking
for the presence of a non-existent file has no signal.

Closes #61.
2026-04-17 08:32:07 +02:00
AgentSeal
c02f63235a test(optimize): add 34 filesystem-mocking tests
Covers previously untested detectors and helpers with real temp-dir
fixtures (not stubs) to verify behavior against actual file I/O.

New coverage:
- detectMissingClaudeignore: project with/without junk dirs and
  .claudeignore, impact scaling.
- detectBloatedClaudeMd: plain oversized file, @-import expansion,
  circular import safety, email/npm-scope @-token filtering.
- loadMcpConfigs: project reads, colon-to-underscore normalization,
  malformed JSON tolerance.
- detectUnusedMcp: 24-hour grace period, config vs invocation merge.
- detectBashBloat: env var unset, configured under/over limit.
- detectGhostCommands: path prefixes are not commands, <command-name>
  tag parsing.
- scanJsonlFile: missing file, tool_use parsing, malformed line
  skipping, date-range filter.
- scanAndDetect: empty projects returns healthy result.
- estimateContextBudget: system base, MCP tools, memory files.
- discoverProjectCwd: empty dir, no jsonl, cwd extraction.

Uses vi.mock to redirect os.homedir() to a disposable temp directory,
so tests do not read the tester's real ~/.claude. 34 tests, <30ms
wall time. Total suite now 160 tests.
2026-04-16 09:33:03 -07:00