parser: keep promptSource on lines over 32 KB

parseLargeJsonl dropped promptSource for exactly the lines SDK-generated
prompts live on, so the recurring-context detector regex-scanned the ends of
the raw line for it. Add the field to LARGE_ROOT_FIELDS (tiny scalar,
add-only, isSidechain already there) and delete the workaround: it read only
2 KB from each end, so a flag further in was missed.

No cache change: optimize scans the raw JSONL each run, so promptSource never
has to persist on CachedFile.

Fixes #1030. With #994 this closes #1023.

CHANGELOG.md

@@ -4,7 +4,7 @@
 
 ### Added
 - **`codeburn models --unpriced`.** The dashboard warns about models that price at $0 and points at `codeburn model-alias`, but the list itself was hard to get out of the TUI. This filters the plain-stdout `models` report to exactly those rows, reusing `findUnpricedModels` so local, free, aliased and price-overridden models are treated the same way the warning treats them, and defaulting that mode's min-cost to 0 so $0 rows are not pre-filtered away. Thanks @kocaemre. (#969)
-- **`optimize` spots the same long block pasted at the start of many sessions.** The new `recurring-context` detector groups sessions by their opening block — normalized for whitespace and ANSI, hashed over the first 2 KB — and reports a block of at least 1.5 KB that opens 5 or more sessions, with the top three by tokens, their session counts and the project each is confined to. It is a habit, not an apply-able fix: CodeBurn will not move your own text into `CLAUDE.md` for you, so the finding asks Claude to give the block a permanent home (a `CLAUDE.md` rule, or a file read on demand) and hand back a one-line pointer to open sessions with instead. Savings count the repeats only, never the first paste, and are marked `estimated`: provider usage is counted per API call, where the pasted block is mixed in with the system prompt, tool schemas and `CLAUDE.md`, so the block is sized from its own bytes. Injected system reminders and slash-command wrappers are not pastes and are skipped, and neither is a prompt a program wrote — an SDK session or a subagent task — read from the entry's flags, or off the ends of the raw line when the entry is too large for the parser to keep them. The opening block comes from the session scan that already runs, so nothing extra is read from disk.
+- **`optimize` spots the same long block pasted at the start of many sessions.** The new `recurring-context` detector groups sessions by their opening block — normalized for whitespace and ANSI, hashed over the first 2 KB — and reports a block of at least 1.5 KB that opens 5 or more sessions, with the top three by tokens, their session counts and the project each is confined to. It is a habit, not an apply-able fix: CodeBurn will not move your own text into `CLAUDE.md` for you, so the finding asks Claude to give the block a permanent home (a `CLAUDE.md` rule, or a file read on demand) and hand back a one-line pointer to open sessions with instead. Savings count the repeats only, never the first paste, and are marked `estimated`: provider usage is counted per API call, where the pasted block is mixed in with the system prompt, tool schemas and `CLAUDE.md`, so the block is sized from its own bytes. Injected system reminders and slash-command wrappers are not pastes and are skipped, and neither is a prompt a program wrote — an SDK session or a subagent task — read from the entry's flags, which survive the parser's large-line path. The opening block comes from the session scan that already runs, so nothing extra is read from disk.
 - **Applied fixes get re-measured on every `optimize` run, and told plainly whether they worked.** After `codeburn optimize --apply`, every still-applied fix comes back in an `Applied fixes` section on subsequent `codeburn optimize` runs, carrying the verdict `act report` already computes from the same reconciliation: `worked` (at least 70% of its window-scaled estimate realized), `partial` (something, but under that), `no-effect` (no measured reduction, printed with the exact `codeburn act undo <id>` that puts it back), or `measuring` for anything younger than the 3-day measurement window. The numbers are measured — provider-counted usage over the post-apply window — not re-estimated. `--apply` now says when the re-measure will happen, `--format json` gains `appliedFixes[]` (add-only), and the same section appears in the dashboard TUI and the desktop app. New `codeburn optimize --auto-revert` undoes the fixes that measured no reduction at all through the same code path as `codeburn act undo`; it never touches `partial` or still-measuring fixes, and never auto-reverts a `CLAUDE.md` rule (it prints the undo command instead), matching the `--yes` guardrail.
 - **Optimize findings say what to do with them and where their number came from.** Every finding now carries a class and a basis, and every surface groups by it: `Fix now (apply-able)` for findings `codeburn optimize --apply` can write itself, `Habits` for the behavioural ones, `FYI` for informational ones whose cost may be justified. A finding only counts as apply-able when a plan can actually be built for that instance, so an `mcp-deferral-off` caused by Vertex policy or a shell-profile override is grouped as a habit rather than promising a fix that does not exist. Alongside it, each finding is marked `measured` (summed from provider-counted usage) or `estimated` (a schema-size or recovery-fraction model), with the split reported in the header as `N measured · M estimated` in place of the blanket "Estimates only." footer. Sessions whose cost the provider never reported are kept out of the `cost-outliers` peer comparison, and a provider that only ever estimates gets the finding marked `estimated` rather than dropped. `--format json` gains `class` and `basis` per finding plus `summary.measuredSavingsUSD` (existing fields unchanged), and the new `docs/optimize.md` covers what is scanned, exactly what `--apply` may write, and how to read the health grade.
 

src/optimize.ts

@@ -663,22 +663,12 @@ function normalizeOpener(text: string): string {
   return stripAnsi(text).replace(/\s+/g, ' ').trim()
 }
 
-const MACHINE_PROMPT_HEAD_BYTES = 2048
-const MACHINE_PROMPT_PATTERN = /"promptSource"\s*:\s*"sdk"|"isSidechain"\s*:\s*true/
-
 /// True when a program wrote this prompt rather than a person pasting it: an
 /// SDK caller, or a parent agent writing a subagent's task. Either repeats by
-/// design and has no home in CLAUDE.md. A user entry over the parser's
-/// large-line threshold — routine for generated prompts — comes back without
-/// its root flags, so those are read off the raw line instead: the ends of it,
-/// since the fields sit either side of the message that made the line large.
-function isMachineWrittenPrompt(entry: Record<string, unknown>, line: string | Buffer): boolean {
-  if (entry['promptSource'] === 'sdk' || entry['isSidechain'] === true) return true
-  const edge = (start: number, end: number): string =>
-    typeof line === 'string' ? line.slice(start, end) : line.subarray(start, end).toString('utf-8')
-  const head = edge(0, MACHINE_PROMPT_HEAD_BYTES)
-  const tail = edge(Math.max(MACHINE_PROMPT_HEAD_BYTES, line.length - MACHINE_PROMPT_HEAD_BYTES), line.length)
-  return MACHINE_PROMPT_PATTERN.test(head) || MACHINE_PROMPT_PATTERN.test(tail)
+/// design and has no home in CLAUDE.md. Both flags survive the parser's
+/// large-line path, which is where generated prompts routinely land.
+function isMachineWrittenPrompt(entry: Record<string, unknown>): boolean {
+  return entry['promptSource'] === 'sdk' || entry['isSidechain'] === true
 }
 
 /// A session's opening block, or null when it is too small to matter or is
@@ -768,7 +758,7 @@ export async function scanJsonlFile(
         userMessages.push(msgContent.slice(0, OPTIMIZE_TEXT_CAP))
         if (!sawUserText) {
           sawUserText = true
-          const opener = isMachineWrittenPrompt(entry, line) ? null : toSessionOpener(msgContent, project)
+          const opener = isMachineWrittenPrompt(entry) ? null : toSessionOpener(msgContent, project)
           if (opener) openers.push(opener)
         }
       } else if (Array.isArray(msgContent)) {
@@ -781,7 +771,7 @@ export async function scanJsonlFile(
             remaining -= text.length
             if (!sawUserText) {
               sawUserText = true
-              const opener = isMachineWrittenPrompt(entry, line) ? null : toSessionOpener(block.text, project)
+              const opener = isMachineWrittenPrompt(entry) ? null : toSessionOpener(block.text, project)
               if (opener) openers.push(opener)
             }
           }

src/parser.ts

@@ -557,7 +557,7 @@ function extractObjectFields(
   return captured
 }
 
-const LARGE_ROOT_FIELDS = ['type', 'timestamp', 'sessionId', 'cwd', 'gitBranch', 'attachment', 'message', 'isSidechain'] as const
+const LARGE_ROOT_FIELDS = ['type', 'timestamp', 'sessionId', 'cwd', 'gitBranch', 'attachment', 'message', 'isSidechain', 'promptSource'] as const
 const LARGE_ASSISTANT_MESSAGE_FIELDS = ['model', 'usage', 'id', 'content'] as const
 
 function parseLargeJsonl(line: string | Buffer): JournalEntry | null {
@@ -578,10 +578,12 @@ function parseLargeJsonl(line: string | Buffer): JournalEntry | null {
   const sessionId = readJsonString(source, root['sessionId'])
   const cwd = readJsonString(source, root['cwd'])
   const gitBranch = readJsonString(source, root['gitBranch'])
+  const promptSource = readJsonString(source, root['promptSource'])
   if (timestamp !== undefined) entry.timestamp = timestamp
   if (sessionId !== undefined) entry.sessionId = sessionId
   if (cwd !== undefined) entry.cwd = cwd
   if (gitBranch !== undefined) entry.gitBranch = gitBranch
+  if (promptSource !== undefined) entry.promptSource = promptSource
   const addedNames = extractLargeAddedNames(source, root['attachment'])
   if (addedNames.length > 0) {
     ;(entry as Record<string, unknown>)['attachment'] = { type: 'deferred_tools_delta', addedNames }

tests/optimize-fs.test.ts

@@ -551,17 +551,15 @@ describe('detectRecurringContext', () => {
     expect(detectRecurringContext(openers)).toBeNull()
   })
 
-  // Over 32 KB the JSONL parser returns a reduced entry without the root
-  // flags, so the markers have to be read off the raw line.
-  it('skips machine-written prompts too large for the parser to keep flags on', async () => {
+  // Over 32 KB the JSONL parser returns a reduced entry; the root flags are
+  // part of that reduction, so the markers survive.
+  it('skips machine-written prompts on lines too large for a full parse', async () => {
     const root = makeFixtureRoot()
     const now = new Date().toISOString()
     const huge = BRIEF + 'x'.repeat(40_000)
     const openers: SessionOpener[] = []
     for (let i = 0; i < 6; i++) {
       const filePath = join(root, `huge-${i}.jsonl`)
-      // Field order matters: the flags land past the head, behind the very
-      // message that made the line large.
       writeFile(filePath, JSON.stringify({
         isSidechain: false, type: 'user', message: { content: huge }, timestamp: now, promptSource: 'sdk',
       }))

tests/parser-large-json-scanner.test.ts

@@ -44,7 +44,25 @@ function largeAssistantLine(): string {
   })
 }
 
+// The fields sit either side of the message that makes the line large, which
+// is where a generated prompt puts them in the wild.
+function largeMachineWrittenLine(): string {
+  return JSON.stringify({
+    isSidechain: true,
+    type: 'user',
+    message: { role: 'user', content: 'brief ' + 'x'.repeat(40_000) },
+    timestamp: '2026-05-01T00:00:00Z',
+    promptSource: 'sdk',
+  })
+}
+
 describe('large JSONL compact scanner', () => {
+  it('keeps the flags marking a program-written prompt', () => {
+    const parsed = parseJsonlLine(largeMachineWrittenLine())
+    expect(parsed?.promptSource).toBe('sdk')
+    expect(parsed?.isSidechain).toBe(true)
+  })
+
   it('extracts user text from array content without full JSON.parse', () => {
     const parsed = parseJsonlLine(largeUserLine())
     expect(parsed?.type).toBe('user')