codeburn/tests/optimize-fs.test.ts
Resham Joshi 5c162600e8
fix(optimize): stop archiving used commands, match MCP servers the way Claude Code names them, count each saving once (#1898)
* fix(optimize): count slash commands named with their leading slash as used

A slash command shows up in the transcript as <command-name>/review</command-name>.
commandNames kept the slash, so "/review" never matched the defined command
"review": every command the user actually ran was reported unused and the apply
plan archived it. The leading slash is now stripped.

commandNames produces stored facts, so FACTS_VERSION goes to 2 and the
optimize facts store rebuilds once on the next run.

* fix(optimize): name MCP servers the way Claude Code namespaces their tools

Claude Code turns every character outside letters, digits, "_" and "-" into
"_" when it builds tool names, so a server "docs.local" is called as
mcp__docs_local__*. Config names were only mapped ":" -> "_", so such a server
was reported as never called and offered for removal, counted twice by the
deferral check, and shown with 0 calls by the alwaysLoad check. One helper,
mcpNamespace, now applies the full rule in the config loader, the local server
names and the remove plan's key lookup.

* fix(optimize): charge a never-called MCP server's schema once

A configured server that was never called is in unused-mcp, and the same
schema was charged again by mcp-deferral-off (unpinned) or
mcp-alwaysload-hygiene (pinned), so potential savings and the share of spend
summed it twice. The servers unused-mcp reports are now passed to both checks
and left out there.

* fix(act): recommend a default model only for Claude Code projects

The model-default plan writes "model" into <project>/.claude/settings.json,
which only Claude Code reads. A Codex project got a gpt model recommended and
apply-model wrote it there, so Claude Code in that project started on a model
it does not know. recommendModelDefault now returns nothing for a non-Claude
provider.

* fix(optimize): read the bash output cap from the settings env block first

Claude Code takes BASH_MAX_OUTPUT_LENGTH from the env block of its settings
files. The check only read shell profiles (lines with export) and the process
env, so a cap of 10000 set in settings.json still reported the 30K default and
the apply plan appended an export to ~/.zshrc that the settings value
overrides. The check now uses the same lookup as the deferral checks: settings
env from the most specific scope, then shell profiles.

* fix(act): plan each optimize fix right before applying it

optimize --apply built every plan up front, each carrying the hash of the
config file as first read. When two fixes edit the same file (remove one MCP
server and unpin another in .mcp.json, or two edits to ~/.claude.json), the
first apply changed the file and the second was refused as "changed since the
plan was built", exit code 1. Each plan is now rebuilt from the file on disk
just before it runs; the baseline captured for it is kept, and the stale-plan
check still guards edits made while the plan is being applied.

* fix(optimize): fast mode on background agents saves dollars, not tokens

Switching subagent calls from fast to standard speed bills the same tokens at
a lower rate. The finding claimed every output token of those calls as tokens
saved, which inflated the token headline and pushed it up the urgency ranking.
tokensSaved is now 0; the dollar figure is unchanged and still carries it.

* fix(optimize): charge an unused MCP server only to Claude sessions

unused-mcp reads servers from Claude Code config, but multiplied their schema
by every session of every provider in the period, so under --provider all a
handful of Claude sessions next to hundreds of Codex or Cursor sessions
produced a large token and dollar figure. It now counts and prices only Claude
sessions, as the deferral checks already do.

* fix(optimize): stop reading MCP servers from ~/.claude/settings.json

Claude Code reads MCP servers from ~/.claude.json and .mcp.json, not from the
mcpServers key of ~/.claude/settings.json or settings.local.json. The MCP
checks read both user settings files, while the remove plan never edits them,
so a server listed only there showed as an unused server with savings under
"Fix now" and apply skipped it as "not found in editable config". The loader
now leaves those two files out; its home directory parameter, and the one on
the alwaysLoad check that only fed it, go with them.

* fix(act): match MCP servers in the report by their namespace

The report's baseline keeps config names such as "docs.local", while sessions
name the server as Claude Code does in tool names ("docs_local"). A removed
server that came back never matched, so act report counted savings instead of
reading it as reverted. sessionLoadsAny now compares through mcpNamespace.

* fix(optimize): keep every config name that maps to one MCP namespace

Claude Code turns "a.b" and "a:b" into the same tool namespace "a_b", so the
MCP config loader keyed both under one entry and kept only the newest name.
unused-mcp then named one of them, and the remove plan left the other config
entry in place. The entry now keeps every config name; unused-mcp and the
alwaysLoad check pass each one to their plans.

* fix(act): read a "__" server name whole from report inventory entries

Inventory entries are mcp__<server>__<tool>, and the report took the server
as the second "__" segment, so a server named "a__b" was read as "a" and a
re-added "a__b" never showed as reverted. The tool is now cut off from the
right, leaving the full server name.

* fix(act): resolve MCP namespaces against the known servers

mcp__a__b__c can be server "a__b" with tool "c" or server "a" with tool
"b__c", and both fixed rules (first segment, or cut from the right) pick the
wrong server for some names. A tool name is now split at the longest server
namespace found in the MCP config (plus the servers being measured in act
report), falling back to the first segment only when none matches. This
applies to act report's inventory reads and to the call names unused-mcp and
mcp-deferral-off count.

* fix(act): replan only for files an earlier fix in this run changed

Rebuilding every plan right before it ran took a fresh hash of the current
file, so under --apply --yes a config file edited outside codeburn between
selection and apply was rewritten anyway, where the stale-plan guard used to
refuse it. The run now records the hash of every file it writes. A plan is
rebuilt only when one of its files differs from the plan solely because an
earlier fix in this run wrote it; any other change keeps the original plan,
and runAction rejects it with "changed since the plan was built".

* docs(optimize): note why an MCP tool name with two known prefixes is ambiguous
2026-11-06 09:27:46 +01:00

951 lines
41 KiB
TypeScript

import { describe, it, expect, afterAll, afterEach, beforeAll, beforeEach, vi } from 'vitest'
import { Writable } from 'node:stream'
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, utimesSync } from 'fs'
import { tmpdir } from 'os'
import { join } from 'path'
import * as fsUtils from '../src/fs-utils.js'
vi.mock('os', async () => {
const actual = await vi.importActual<typeof import('os')>('os')
const fs = await vi.importActual<typeof import('fs')>('fs')
const fakeHome = fs.mkdtempSync(actual.tmpdir() + '/codeburn-home-')
fs.mkdirSync(fakeHome + '/.claude', { recursive: true })
process.env['CODEBURN_TEST_FAKE_HOME'] = fakeHome
return { ...actual, homedir: () => fakeHome }
})
const FAKE_HOME_FOR_MOCK = process.env['CODEBURN_TEST_FAKE_HOME']!
import {
detectBloatedClaudeMd,
detectUnusedMcp,
detectMcpDeferralOff,
detectMcpAlwaysLoadHygiene,
mcpNamespace,
detectBashBloat,
detectIdleCompactionBlocked,
commandNames,
detectGhostCommands,
detectDuplicateReads,
detectJunkReads,
detectLowReadEditRatio,
loadMcpConfigs,
localMcpServerNames,
scanJsonlFile,
scanAndDetect,
detectRecurringContext,
renderOptimize,
findingPricing,
type SessionOpener,
type ToolCall,
} from '../src/optimize.js'
import { setPriceOverrides } from '../src/models.js'
import {
estimateContextBudget,
discoverProjectCwd,
} from '../src/context-budget.js'
import type { ProjectSummary } from '../src/types.js'
import { runOptimizeApply } from '../src/act/optimize-apply.js'
import { planFor } from '../src/act/plans.js'
// ============================================================================
// Helpers for filesystem fixtures
// ============================================================================
const FIXTURE_ROOTS: string[] = [FAKE_HOME_FOR_MOCK]
function makeFixtureRoot(): string {
const dir = mkdtempSync(join(tmpdir(), 'codeburn-test-'))
FIXTURE_ROOTS.push(dir)
return dir
}
function writeFile(path: string, content: string): void {
mkdirSync(join(path, '..'), { recursive: true })
writeFileSync(path, content)
}
function touchOld(path: string, daysAgo: number): void {
const past = new Date(Date.now() - daysAgo * 24 * 60 * 60 * 1000)
utimesSync(path, past, past)
}
afterAll(() => {
for (const dir of FIXTURE_ROOTS) {
rmSync(dir, { recursive: true, force: true })
}
})
// ============================================================================
// detectBloatedClaudeMd (including @-import expansion)
// ============================================================================
describe('detectBloatedClaudeMd', () => {
it('flags a CLAUDE.md with more than 200 lines', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
const content = Array.from({ length: 300 }, (_, i) => `line ${i}`).join('\n')
writeFile(join(projectDir, 'CLAUDE.md'), content)
const finding = detectBloatedClaudeMd(new Set([projectDir]))
expect(finding).not.toBeNull()
})
it('expands @-imports and counts transitive load', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(
join(projectDir, 'CLAUDE.md'),
'line 1\nline 2\n@./rules.md\n@./conventions.md\n',
)
writeFile(join(projectDir, 'rules.md'), Array.from({ length: 120 }, (_, i) => `rule ${i}`).join('\n'))
writeFile(join(projectDir, 'conventions.md'), Array.from({ length: 120 }, (_, i) => `conv ${i}`).join('\n'))
const finding = detectBloatedClaudeMd(new Set([projectDir]))
expect(finding).not.toBeNull()
expect(finding!.explanation).toContain('2 @-imports')
})
it('does not flag a lean CLAUDE.md under 200 lines with no imports', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, 'CLAUDE.md'), 'just a few\nlines\nhere\n')
expect(detectBloatedClaudeMd(new Set([projectDir]))).toBeNull()
})
it('does not recurse infinitely on circular @-imports', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, 'CLAUDE.md'), '@./a.md\n')
writeFile(join(projectDir, 'a.md'), '@./b.md\n')
writeFile(join(projectDir, 'b.md'), '@./a.md\n')
expect(() => detectBloatedClaudeMd(new Set([projectDir]))).not.toThrow()
})
it('ignores @ tokens that are not paths (emails, npm scopes)', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(
join(projectDir, 'CLAUDE.md'),
Array.from({ length: 250 }, (_, i) =>
i === 10 ? '@user@example.com' :
i === 20 ? '@org/package' :
`line ${i}`
).join('\n'),
)
const finding = detectBloatedClaudeMd(new Set([projectDir]))
expect(finding).not.toBeNull()
// "with N @-imports" suffix appears only when non-zero imports were resolved
expect(finding!.explanation).not.toMatch(/with \d+ @-import/)
})
})
// ============================================================================
// loadMcpConfigs + detectUnusedMcp
// ============================================================================
describe('loadMcpConfigs', () => {
it('returns empty map when no configs exist', () => {
const root = makeFixtureRoot()
const servers = loadMcpConfigs([root])
expect(servers.size).toBe(0)
})
it('reads servers from project .mcp.json', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({
mcpServers: { foo: { command: 'foo' }, bar: { command: 'bar' } },
}))
const servers = loadMcpConfigs([projectDir])
expect(servers.has('foo')).toBe(true)
expect(servers.has('bar')).toBe(true)
})
it('normalizes server names by replacing colons with underscores', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({
mcpServers: { 'plugin:context7:context7': { command: 'ctx' } },
}))
const servers = loadMcpConfigs([projectDir])
expect(servers.has('plugin_context7_context7')).toBe(true)
expect(servers.get('plugin_context7_context7')!.originals).toEqual(['plugin:context7:context7'])
})
it('normalizes every character outside letters, digits, _ and - the way tool names do', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { 'docs.local': { command: 'x' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
expect([...loadMcpConfigs([projectDir]).keys()]).toEqual(['docs_local'])
const calls = [{ name: 'mcp__docs_local__search', input: {}, sessionId: 's1', project: 'p' }] as unknown as ToolCall[]
expect(detectUnusedMcp(calls, [], new Set([projectDir]), [])).toBeNull()
})
it('keeps every config name that maps to one namespace, so removal reaches each file', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { 'a.b': { command: 'x' } } }))
writeFile(join(projectDir, '.claude', 'settings.json'), JSON.stringify({ mcpServers: { 'a:b': { command: 'y' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
touchOld(join(projectDir, '.claude', 'settings.json'), 30)
expect(loadMcpConfigs([projectDir]).get('a_b')!.originals.sort()).toEqual(['a.b', 'a:b'])
const finding = detectUnusedMcp([], [], new Set([projectDir]), [])!
const plan = planFor(finding, { homeDir: root, cwd: projectDir })!
expect(plan.changes.map(c => c.path).sort()).toEqual([join(projectDir, '.claude', 'settings.json'), join(projectDir, '.mcp.json')])
})
it('credits a call to the longest configured server, "a__b" over "a"', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { a: { command: 'x' }, a__b: { command: 'y' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
const unusedAfter = (name: string) => {
const calls = [{ name, input: {}, sessionId: 's1', project: 'p' }] as unknown as ToolCall[]
const apply = detectUnusedMcp(calls, [], new Set([projectDir]), [])!.apply
return apply?.kind === 'mcp-remove' ? apply.servers : []
}
expect(unusedAfter('mcp__a__b__c')).toEqual(['a'])
expect(unusedAfter('mcp__a__c')).toEqual(['a__b'])
})
it('handles malformed JSON without crashing', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), '{ not valid json')
expect(() => loadMcpConfigs([projectDir])).not.toThrow()
expect(loadMcpConfigs([projectDir]).size).toBe(0)
})
})
describe('localMcpServerNames', () => {
it('adds the ~/.claude.json top-level and per-project servers to the config names', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { fromMcpJson: {} } }))
writeFile(join(FAKE_HOME_FOR_MOCK, '.claude.json'), JSON.stringify({
mcpServers: { 'claude_ai_homegrown': {}, 'plugin:ctx:ctx': {} },
projects: { [projectDir]: { mcpServers: { scoped: {} } } },
}))
const names = localMcpServerNames([projectDir])
expect([...names].sort()).toEqual(['claude_ai_homegrown', 'fromMcpJson', 'plugin_ctx_ctx', 'scoped'])
})
it('ignores mcpServers in ~/.claude/settings.json, which Claude Code does not read', () => {
const home = makeFixtureRoot()
writeFile(join(home, '.claude', 'settings.json'), JSON.stringify({ mcpServers: { fromSettings: {} } }))
expect(localMcpServerNames([], home).size).toBe(0)
})
it('contributes no names when ~/.claude.json cannot be parsed', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(FAKE_HOME_FOR_MOCK, '.claude.json'), '{ not valid json')
expect(localMcpServerNames([projectDir]).size).toBe(0)
})
})
describe('detectUnusedMcp', () => {
it('flags servers configured but never called', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({
mcpServers: { ghost: { command: 'x' } },
}))
const configFile = join(projectDir, '.mcp.json')
touchOld(configFile, 30)
const finding = detectUnusedMcp([], [], new Set([projectDir]))
expect(finding).not.toBeNull()
expect(finding!.explanation).toContain('ghost')
})
it('counts a session split across projects once', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { ghost: { command: 'x' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
const slice = (primary: boolean) => ({ sessionId: 's1', mcpBreakdown: {}, turns: [{ assistantCalls: [{ provider: 'claude' }] }], projectSplit: { primaryProject: 'a', primaryProjectPath: '/a', primary } })
const projects = [
{ project: 'a', projectPath: '/a', sessions: [slice(true)] },
{ project: 'b', projectPath: '/b', sessions: [slice(false)] },
] as unknown as ProjectSummary[]
const split = detectUnusedMcp([], projects, new Set([projectDir]), [])
const single = detectUnusedMcp([], projects.slice(0, 1), new Set([projectDir]), [])
expect(split!.tokensSaved).toBe(single!.tokensSaved)
})
it('charges only Claude sessions', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { ghost: { command: 'x' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
const session = (sessionId: string, provider: string) => ({ sessionId, mcpBreakdown: {}, turns: [{ assistantCalls: [{ provider }] }] })
const sessions = [session('c1', 'claude'), ...Array.from({ length: 9 }, (_, i) => session(`x${i}`, 'codex'))]
const projects = [{ project: 'a', projectPath: '/a', sessions }] as unknown as ProjectSummary[]
expect(detectUnusedMcp([], projects, new Set([projectDir]), [])!.tokensSaved).toBe(2000)
})
it('leaves a never-called server to unused-mcp alone, so its schema is charged once', () => {
for (const idle of [{ command: 'x' }, { command: 'x', alwaysLoad: true }]) {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { idle, used: { command: 'y' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
const session = (sessionId: string) => ({ sessionId, mcpBreakdown: { used: { calls: 3 } }, turns: [{ assistantCalls: [{ provider: 'claude' }] }] })
const projects = [{ project: 'a', projectPath: '/a', sessions: ['s1', 's2', 's3'].map(session) }] as unknown as ProjectSummary[]
const cwds = new Set([projectDir])
const unused = detectUnusedMcp([], projects, cwds, [])!
const servers = new Set(unused.apply!.kind === 'mcp-remove' ? unused.apply!.servers.map(mcpNamespace) : [])
const findings = [
unused,
detectMcpDeferralOff([], projects, cwds, [], FAKE_HOME_FOR_MOCK, undefined, servers),
detectMcpAlwaysLoadHygiene(projects, cwds, [], [], undefined, servers),
]
expect(findings.reduce((sum, f) => sum + (f?.tokensSaved ?? 0), 0)).toBe(2 * 2000 * 3)
}
})
describe('measured dollars', () => {
beforeAll(() => setPriceOverrides({ 'measured-test-model': { input: 4, output: 20 } }))
afterAll(() => setPriceOverrides({}))
it('prices the schema at each session model input rate, unpriced sessions at the blended rate', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({ mcpServers: { ghost: { command: 'x' } } }))
touchOld(join(projectDir, '.mcp.json'), 30)
const session = (sessionId: string, model: string) => ({ sessionId, mcpBreakdown: {}, turns: [{ assistantCalls: [{ model, provider: 'claude' }] }] })
const projects = [
{ project: 'a', projectPath: '/a', sessions: [session('s1', 'measured-test-model'), session('s2', 'unpriced-test-model')] },
] as unknown as ProjectSummary[]
const finding = detectUnusedMcp([], projects, new Set([projectDir]), [], findingPricing(projects, 1e-6))!
expect(finding.tokensSaved).toBe(4000)
expect(finding.savingsUSD).toBeCloseTo(2000 * 4e-6 + 2000 * 1e-6, 12)
})
})
it('does not flag servers configured within 24 hours', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({
mcpServers: { freshly_added: { command: 'x' } },
}))
expect(detectUnusedMcp([], [], new Set([projectDir]))).toBeNull()
})
it('does not flag servers that were called', () => {
const root = makeFixtureRoot()
const projectDir = join(root, 'myapp')
mkdirSync(projectDir, { recursive: true })
writeFile(join(projectDir, '.mcp.json'), JSON.stringify({
mcpServers: { used: { command: 'x' } },
}))
touchOld(join(projectDir, '.mcp.json'), 30)
const calls: ToolCall[] = [
{ name: 'mcp__used__some_tool', input: {}, sessionId: 's1', project: 'p1' },
]
expect(detectUnusedMcp(calls, [], new Set([projectDir]))).toBeNull()
})
})
// ============================================================================
// detectBashBloat
// ============================================================================
describe('detectBashBloat', () => {
it('flags when env var is unset (uses default 30K)', () => {
const finding = detectBashBloat()
expect(finding).not.toBeNull()
expect(finding!.impact).toBe('medium')
})
it('does not flag when env var is at recommended 15K', () => {
process.env['BASH_MAX_OUTPUT_LENGTH'] = '15000'
expect(detectBashBloat()).toBeNull()
})
it('does not flag when env var is below recommended', () => {
process.env['BASH_MAX_OUTPUT_LENGTH'] = '10000'
expect(detectBashBloat()).toBeNull()
})
it('does not flag when the settings env block sets the cap', () => {
delete process.env['BASH_MAX_OUTPUT_LENGTH']
const cwd = makeFixtureRoot()
writeFile(join(cwd, '.claude', 'settings.json'), JSON.stringify({ env: { BASH_MAX_OUTPUT_LENGTH: '10000' } }))
expect(detectBashBloat([cwd])).toBeNull()
})
it('flags when env var is above 15K', () => {
process.env['BASH_MAX_OUTPUT_LENGTH'] = '50000'
const finding = detectBashBloat()
expect(finding).not.toBeNull()
})
})
// ============================================================================
// detectIdleCompactionBlocked
// ============================================================================
describe('detectIdleCompactionBlocked', () => {
const calls = (version: string) => [{ cacheCreationTokens: 0, version }]
const withOneHourWrites = (tokens: number) => [
{ sessions: [{ turns: [{ assistantCalls: [{ provider: 'claude', cacheCreationOneHourTokens: tokens }] }] }] },
] as unknown as ProjectSummary[]
const oneHour = withOneHourWrites(5000)
function setup(user: Record<string, unknown>, project?: Record<string, unknown>): { home: string; cwds: Set<string> } {
const home = makeFixtureRoot()
const cwd = makeFixtureRoot()
writeFile(join(home, '.claude', 'settings.json'), JSON.stringify(user))
if (project) writeFile(join(cwd, '.claude', 'settings.json'), JSON.stringify(project))
return { home, cwds: new Set([cwd]) }
}
it('fires when the window equals the default threshold', () => {
const { home, cwds } = setup({ autoCompactWindow: 200000 })
const finding = detectIdleCompactionBlocked(oneHour, cwds, calls('2.1.290'), home)
expect(finding?.id).toBe('idle-compaction-blocked')
expect(finding!.tokensSaved).toBe(0)
expect(finding!.fix.type === 'paste' && finding!.fix.text).toBe('"CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS": "100000"')
})
it('fires when the window is below the threshold', () => {
const { home, cwds } = setup({ autoCompactWindow: 150000 })
const finding = detectIdleCompactionBlocked(oneHour, cwds, calls('2.1.311'), home)
expect(finding!.explanation).toContain('"autoCompactWindow": 150000 in user settings')
expect(finding!.explanation).toContain('(its default)')
})
it('is silent when the window is above the threshold', () => {
const { home, cwds } = setup({ autoCompactWindow: 150000, env: { CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS: '100000' } })
expect(detectIdleCompactionBlocked(oneHour, cwds, calls('2.1.311'), home)).toBeNull()
})
it('is silent before Claude Code v2.1.290', () => {
const { home, cwds } = setup({ autoCompactWindow: 150000 })
expect(detectIdleCompactionBlocked(oneHour, cwds, calls('2.1.289'), home)).toBeNull()
})
it('is silent when idle compaction is turned off', () => {
const { home, cwds } = setup({ autoCompactWindow: 150000, idleCompaction: false })
expect(detectIdleCompactionBlocked(oneHour, cwds, calls('2.1.311'), home)).toBeNull()
})
it('is silent when the window only wrote to the five-minute cache', () => {
const { home, cwds } = setup({ autoCompactWindow: 150000 })
expect(detectIdleCompactionBlocked(withOneHourWrites(0), cwds, calls('2.1.311'), home)).toBeNull()
})
it('takes project settings over user settings', () => {
const above = setup({ autoCompactWindow: 150000 }, { autoCompactWindow: 300000 })
expect(detectIdleCompactionBlocked(oneHour, above.cwds, calls('2.1.311'), above.home)).toBeNull()
const below = setup({ autoCompactWindow: 300000 }, { autoCompactWindow: 150000 })
expect(detectIdleCompactionBlocked(oneHour, below.cwds, calls('2.1.311'), below.home)!.explanation).toContain('in project settings')
})
})
// ============================================================================
// detectGhostCommands (the pure-function ghost detector)
// ============================================================================
describe('detectGhostCommands', () => {
it('returns null when no commands are defined', async () => {
expect(await detectGhostCommands([])).toBeNull()
})
it('does not match /tmp or /usr or other path prefixes as command usage', async () => {
const messages = [
'check /tmp/debug.log',
'look at /usr/local/bin',
'rm -rf /var/cache',
]
expect(await detectGhostCommands(messages.flatMap(commandNames))).toBeNull()
})
it('matches <command-name> tags in user messages', async () => {
const messages = ['<command-name>review</command-name>']
expect(await detectGhostCommands(messages.flatMap(commandNames))).toBeNull()
})
it('names a <command-name> tag without its leading slash', () => {
expect(commandNames('<command-name>/review</command-name>')).toEqual(['review'])
})
})
// ============================================================================
// scanJsonlFile
// ============================================================================
describe('scanJsonlFile', () => {
it('returns empty result for nonexistent file', async () => {
const result = await scanJsonlFile('/nonexistent/path.jsonl', 'p1', undefined)
expect(result.calls).toEqual([])
expect(result.cwds).toEqual([])
expect(result.apiCalls).toEqual([])
expect(result.commandNames).toEqual([])
})
it('parses tool_use blocks from assistant entries', async () => {
const root = makeFixtureRoot()
const filePath = join(root, 'session.jsonl')
const now = new Date().toISOString()
const lines = [
JSON.stringify({
type: 'assistant',
timestamp: now,
message: {
content: [{ type: 'tool_use', name: 'Read', input: { file_path: '/x/foo.ts' } }],
},
}),
]
writeFile(filePath, lines.join('\n'))
const result = await scanJsonlFile(filePath, 'p1', undefined)
expect(result.calls).toHaveLength(1)
expect(result.calls[0].name).toBe('Read')
})
it('marks tool calls from sidechain transcript entries', async () => {
const root = makeFixtureRoot()
const filePath = join(root, 'agent-reviewer.jsonl')
const now = new Date().toISOString()
writeFile(filePath, JSON.stringify({
type: 'assistant', isSidechain: true, timestamp: now,
message: { content: [{ type: 'tool_use', name: 'Edit', input: { file_path: '/x/foo.ts' } }] },
}))
const result = await scanJsonlFile(filePath, 'p1', undefined)
expect(result.calls).toHaveLength(1)
expect(result.calls[0]!.isSidechain).toBe(true)
})
it('classifies every tool call in a transcript when a later large entry marks it as sidechain', async () => {
const root = makeFixtureRoot()
const filePath = join(root, 'agent-reviewer.jsonl')
const now = new Date().toISOString()
const assistant = (name: string, isSidechain?: boolean, padding = '') => JSON.stringify({
type: 'assistant',
...(isSidechain === true ? { isSidechain: true } : {}),
timestamp: now,
cwd: '/x',
padding,
message: {
model: 'claude-sonnet-4-5',
usage: { cache_creation_input_tokens: 1 },
content: [{ type: 'tool_use', name, input: { file_path: `/x/${name}.ts` } }],
},
})
writeFile(filePath, [
JSON.stringify({ type: 'user', timestamp: now, cwd: '/x', message: { content: 'delegate this with /review' } }),
assistant('Read'),
assistant('Edit', true, 'x'.repeat(40_000)),
assistant('Bash'),
].join('\n'))
const result = await scanJsonlFile(filePath, 'p1', undefined)
expect(result.calls.map(call => [call.name, call.isSidechain])).toEqual([
['Read', true],
['Edit', true],
['Bash', true],
])
expect(result.apiCalls).toHaveLength(3)
expect(result.cwds).toHaveLength(4)
expect(result.commandNames).toEqual(['review'])
})
it('keeps sidechain calls out of duplicate reads but in junk reads and the read:edit ratio', () => {
const sidechain = { sessionId: 'agent-reviewer', project: 'p1', isSidechain: true }
const editCalls = Array.from({ length: 10 }, (_, index) => ({
name: 'Edit', input: { file_path: `/src/${index}.ts` }, ...sidechain,
}))
const junkReads = Array.from({ length: 6 }, () => ({
name: 'Read', input: { file_path: '/app/node_modules/pkg/index.js' }, ...sidechain,
}))
const repeatReads = Array.from({ length: 6 }, () => ({
name: 'Read', input: { file_path: '/app/src/a.ts' }, ...sidechain,
}))
// A subagent editing without reading, or reading into node_modules, is the
// same waste as the parent doing it, and the CLAUDE.md rule both suggest
// binds subagents too - so the full call population feeds them.
expect(detectLowReadEditRatio(editCalls)?.id).toBe('read-edit-ratio')
expect(detectJunkReads(junkReads)?.id).toBe('build-folder-reads')
// A re-read is only waste when the context already held the file; a
// sidechain starts fresh and has to read it.
expect(detectDuplicateReads(repeatReads)).toBeNull()
})
it('skips malformed JSONL lines without crashing', async () => {
const root = makeFixtureRoot()
const filePath = join(root, 'session.jsonl')
writeFile(filePath, 'this is not json\n{broken\n{"type":"assistant","message":{"content":[]}}\n')
const result = await scanJsonlFile(filePath, 'p1', undefined)
expect(result.calls).toEqual([])
})
it('uses readSessionLines (streaming) rather than readSessionFile (full-string load)', async () => {
const readSessionLinesSpy = vi.spyOn(fsUtils, 'readSessionLines')
const readSessionFileSpy = vi.spyOn(fsUtils, 'readSessionFile')
const root = makeFixtureRoot()
const filePath = join(root, 'session.jsonl')
const now = new Date().toISOString()
writeFile(filePath, JSON.stringify({
type: 'assistant', timestamp: now,
message: { content: [{ type: 'tool_use', name: 'Bash', input: {} }] },
}))
await scanJsonlFile(filePath, 'p1', undefined)
// The tracker rides along so a later read can resume after the last
// complete line; what this test guards is that the file is streamed.
expect(readSessionLinesSpy).toHaveBeenCalledWith(filePath, undefined, {
largeLineAsBuffer: true,
byteOffsetTracker: { lastCompleteLineOffset: expect.any(Number) },
})
expect(readSessionFileSpy).not.toHaveBeenCalled()
readSessionLinesSpy.mockRestore()
readSessionFileSpy.mockRestore()
})
it('processes all entries in a large multi-line file without truncation', async () => {
const root = makeFixtureRoot()
const filePath = join(root, 'session.jsonl')
const now = new Date().toISOString()
const ENTRY_COUNT = 500
const lines = Array.from({ length: ENTRY_COUNT }, (_, i) =>
JSON.stringify({
type: 'assistant',
timestamp: now,
message: { content: [{ type: 'tool_use', name: 'Read', input: { file_path: `/file-${i}.ts` } }] },
}),
)
writeFile(filePath, lines.join('\n'))
const result = await scanJsonlFile(filePath, 'p1', undefined)
expect(result.calls).toHaveLength(ENTRY_COUNT)
})
it('respects date-range filter for assistant entries', async () => {
const root = makeFixtureRoot()
const filePath = join(root, 'session.jsonl')
const old = '2020-01-01T00:00:00Z'
const now = new Date().toISOString()
writeFile(filePath, [
JSON.stringify({
type: 'assistant', timestamp: old,
message: { content: [{ type: 'tool_use', name: 'Read', input: { file_path: '/old' } }] },
}),
JSON.stringify({
type: 'assistant', timestamp: now,
message: { content: [{ type: 'tool_use', name: 'Read', input: { file_path: '/new' } }] },
}),
].join('\n'))
const today = new Date()
const start = new Date(today.getFullYear(), today.getMonth(), today.getDate())
const result = await scanJsonlFile(filePath, 'p1', { start, end: today })
expect(result.calls).toHaveLength(1)
expect((result.calls[0].input as Record<string, unknown>).file_path).toBe('/new')
})
})
// ============================================================================
// detectRecurringContext
// ============================================================================
describe('detectRecurringContext', () => {
// ~2.2 KB, comfortably over the 1.5 KB floor.
const BRIEF = 'PROJECT BRIEF\n' + 'Ship the invoice importer behind a flag. '.repeat(53)
const TOKENS_PER_CHAR = 0.25
type Opener = { text: string; project?: string }
async function openersFor(sessions: Opener[]): Promise<SessionOpener[]> {
const root = makeFixtureRoot()
const now = new Date().toISOString()
const openers: SessionOpener[] = []
for (const [i, { text, project }] of sessions.entries()) {
const filePath = join(root, `s${i}.jsonl`)
writeFile(filePath, JSON.stringify({ type: 'user', timestamp: now, message: { content: text } }))
const result = await scanJsonlFile(filePath, project ?? 'my-app', undefined)
openers.push(...result.openers)
}
return openers
}
const repeat = (n: number, text = BRIEF, project?: string): Opener[] =>
Array.from({ length: n }, () => ({ text, project }))
it('flags a block that opens the session threshold worth of sessions', async () => {
const finding = detectRecurringContext(await openersFor(repeat(5)))
expect(finding).not.toBeNull()
expect(finding!.id).toBe('recurring-context')
expect(finding!.title).toBe(`Same ${(BRIEF.length / 1024).toFixed(1)} KB block pasted at the start of 5 sessions`)
// Only the four repeats are recoverable; the first paste is the honest cost.
expect(finding!.tokensSaved).toBe(Math.round(4 * BRIEF.length * TOKENS_PER_CHAR))
expect(finding!.fix.type).toBe('paste')
expect(finding!.fix.type === 'paste' && finding!.fix.destination).toBe('prompt')
})
it('renders the finding with its preview and destination header', async () => {
const finding = detectRecurringContext(await openersFor(repeat(5)))!
const out = renderOptimize([finding], 0.00001, '30 Days', 10, 5, 100, 80, 'B', [], [])
.replace(/\u001b\[[0-9;]*m/g, '')
expect(out).toContain(finding.title)
expect(out).toContain('PROJECT BRIEF Ship the invoice importer')
expect(out).toContain('Ask Claude in the current session')
expect(out).toContain('Move it into CLAUDE.md if it is a standing rule')
})
it('stays quiet below the session threshold', async () => {
expect(detectRecurringContext(await openersFor(repeat(4)))).toBeNull()
})
it('ignores short openers however often they repeat', async () => {
expect(detectRecurringContext(await openersFor(repeat(20, 'continue')))).toBeNull()
expect(detectRecurringContext(await openersFor(repeat(20, 'yes')))).toBeNull()
})
it('groups sessions whose block differs only in whitespace', async () => {
const reflowed = BRIEF.replace(/ /g, ' ').replace(/\n/g, '\n\n')
const finding = detectRecurringContext(await openersFor([...repeat(3), ...repeat(2, reflowed)]))
expect(finding).not.toBeNull()
expect(finding!.title).toContain('start of 5 sessions')
})
it('ignores injected system reminders and slash-command wrappers', async () => {
expect(detectRecurringContext(await openersFor(repeat(6, `<system-reminder>${BRIEF}</system-reminder>`)))).toBeNull()
expect(detectRecurringContext(await openersFor(repeat(6, `<command-name>/brief</command-name>${BRIEF}`)))).toBeNull()
})
it('skips prompts a program wrote: SDK sessions and subagent transcripts', async () => {
const root = makeFixtureRoot()
const now = new Date().toISOString()
const openers: SessionOpener[] = []
for (const [i, entry] of [{ promptSource: 'sdk' }, { isSidechain: true }].entries()) {
for (let j = 0; j < 6; j++) {
const filePath = join(root, `machine-${i}-${j}.jsonl`)
writeFile(filePath, JSON.stringify({ type: 'user', timestamp: now, ...entry, message: { content: BRIEF } }))
openers.push(...(await scanJsonlFile(filePath, 'my-app', undefined)).openers)
}
}
expect(openers).toEqual([])
expect(detectRecurringContext(openers)).toBeNull()
})
// Over 32 KB the JSONL parser returns a reduced entry; the root flags are
// part of that reduction, so the markers survive.
it('skips machine-written prompts on lines too large for a full parse', async () => {
const root = makeFixtureRoot()
const now = new Date().toISOString()
const huge = BRIEF + 'x'.repeat(40_000)
const openers: SessionOpener[] = []
for (let i = 0; i < 6; i++) {
const filePath = join(root, `huge-${i}.jsonl`)
writeFile(filePath, JSON.stringify({
isSidechain: false, type: 'user', message: { content: huge }, timestamp: now, promptSource: 'sdk',
}))
openers.push(...(await scanJsonlFile(filePath, 'my-app', undefined)).openers)
}
expect(openers).toEqual([])
})
it('only counts the first message of a session as its opener', async () => {
const root = makeFixtureRoot()
const now = new Date().toISOString()
const openers: SessionOpener[] = []
for (let i = 0; i < 6; i++) {
const filePath = join(root, `late-${i}.jsonl`)
writeFile(filePath, [
JSON.stringify({ type: 'user', timestamp: now, message: { content: 'go on' } }),
JSON.stringify({ type: 'user', timestamp: now, message: { content: BRIEF } }),
].join('\n'))
openers.push(...(await scanJsonlFile(filePath, 'my-app', undefined)).openers)
}
expect(detectRecurringContext(openers)).toBeNull()
})
it('names the project when the block is confined to one, and counts them otherwise', async () => {
const single = detectRecurringContext(await openersFor(repeat(5, BRIEF, '-Users-me-Projects-codeburn')))
expect(single!.explanation).toContain('5 sessions in codeburn')
const spread = detectRecurringContext(await openersFor([
...repeat(3, BRIEF, '-Users-me-Projects-codeburn'),
...repeat(2, BRIEF, '-Users-me-Projects-dash'),
]))
expect(spread!.explanation).toContain('5 sessions in 2 projects')
})
})
// ============================================================================
// scanAndDetect (top-level integration)
// ============================================================================
describe('scanAndDetect', () => {
it('returns healthy result for empty projects', async () => {
const result = await scanAndDetect([])
expect(result.findings).toEqual([])
expect(result.healthScore).toBe(100)
expect(result.healthGrade).toBe('A')
expect(result.costRate).toBe(0)
})
// The session scan only ever reads Claude Code transcripts, so under a
// non-Claude --provider it used to report Claude-derived findings beside a
// header scoped to the other provider - e.g. `optimize --provider codex`
// printing a read/edit ratio counted from Claude sessions.
describe('provider scoping', () => {
// These fixtures live in the shared fake home, so they have to come back
// out: later suites in this file assert on an otherwise empty ~/.claude.
const CLAUDE_DIR = join(FAKE_HOME_FOR_MOCK, '.claude')
afterEach(() => {
for (const sub of ['projects', 'skills']) {
rmSync(join(CLAUDE_DIR, sub), { recursive: true, force: true })
}
})
function claudeSessionWithEditHeavyTurns(): void {
const projectDir = join(CLAUDE_DIR, 'projects', 'provider-scope')
mkdirSync(projectDir, { recursive: true })
const now = new Date().toISOString()
const entry = (name: string, file: string) => JSON.stringify({
type: 'assistant', timestamp: now,
message: { content: [{ type: 'tool_use', name, input: { file_path: file } }] },
})
const lines = [entry('Read', '/src/a.ts')]
for (let i = 0; i < 12; i++) lines.push(entry('Edit', `/src/f${i}.ts`))
writeFileSync(join(projectDir, 'session.jsonl'), lines.join('\n'))
}
// scanAndDetect memoises on (provider, range, project fingerprint) for 60s,
// and the cache is module-level, so tests that differ only in what is on
// disk would serve each other's results. `seed` moves the fingerprint so
// each case scans for real.
function projectFixture(seed: number): ProjectSummary {
return {
project: 'provider-scope',
projectPath: '/tmp/provider-scope',
sessions: [],
totalCostUSD: 1,
totalApiCalls: 13 + seed,
} as unknown as ProjectSummary
}
it('reports transcript-derived findings when scoped to claude', async () => {
claudeSessionWithEditHeavyTurns()
const result = await scanAndDetect([projectFixture(1)], undefined, 'claude')
expect(result.findings.map(f => f.id)).toContain('read-edit-ratio')
})
it('omits transcript-derived findings when scoped to another provider', async () => {
claudeSessionWithEditHeavyTurns()
mkdirSync(join(CLAUDE_DIR, 'skills', 'never-invoked'), { recursive: true })
writeFileSync(join(CLAUDE_DIR, 'skills', 'never-invoked', 'SKILL.md'), '# skill\n')
const result = await scanAndDetect([projectFixture(2)], undefined, 'codex')
const ids = result.findings.map(f => f.id)
expect(ids).not.toContain('read-edit-ratio')
// An unmeasured skill must not be reported as an unused one: the scan
// returns nothing under this filter, which is not evidence of disuse.
expect(ids).not.toContain('unused-skills')
})
// The apply path reaches scanAndDetect through its own entry point, so it
// needs its own guard: `unused-skills` is appliable, and its plan moves
// directories out of ~/.claude/skills. Reporting a Codex-labelled finding
// is a wrong number; offering to archive every skill off one is a wrong
// number with side effects.
async function applyDryRun(provider: string): Promise<string> {
const chunks: string[] = []
const output = new Writable({ write(c, _e, cb) { chunks.push(String(c)); cb() } })
const errorOutput = new Writable({ write(_c, _e, cb) { cb() } })
await runOptimizeApply([projectFixture(3)], undefined, { provider, dryRun: true, output, errorOutput })
return chunks.join('')
}
it('plans no applies from Claude findings when scoped to another provider', async () => {
claudeSessionWithEditHeavyTurns()
mkdirSync(join(CLAUDE_DIR, 'skills', 'never-invoked'), { recursive: true })
writeFileSync(join(CLAUDE_DIR, 'skills', 'never-invoked', 'SKILL.md'), '# skill\n')
const codex = await applyDryRun('codex')
expect(codex).toContain('No appliable config-class fixes')
expect(codex).not.toContain('never-invoked')
const claude = await applyDryRun('claude')
expect(claude).toContain('never-invoked')
})
})
})
// ============================================================================
// context-budget
// ============================================================================
describe('estimateContextBudget', () => {
it('returns only system base when project has no config', async () => {
const root = makeFixtureRoot()
const budget = await estimateContextBudget(root)
expect(budget.total).toBeGreaterThan(0)
expect(budget.mcpTools.count).toBe(0)
expect(budget.skills.count).toBe(0)
})
it('includes MCP tools from project .mcp.json', async () => {
const root = makeFixtureRoot()
writeFile(join(root, '.mcp.json'), JSON.stringify({
mcpServers: { a: { command: 'x' }, b: { command: 'x' } },
}))
const budget = await estimateContextBudget(root)
expect(budget.mcpTools.count).toBeGreaterThan(0)
})
it('includes memory file tokens from CLAUDE.md', async () => {
const root = makeFixtureRoot()
writeFile(join(root, 'CLAUDE.md'), 'Project context for Claude.\n')
const budget = await estimateContextBudget(root)
expect(budget.memory.count).toBeGreaterThan(0)
expect(budget.memory.tokens).toBeGreaterThan(0)
})
})
describe('discoverProjectCwd', () => {
it('returns null for empty directory', async () => {
const root = makeFixtureRoot()
expect(await discoverProjectCwd(root)).toBeNull()
})
it('returns null for directory with no jsonl files', async () => {
const root = makeFixtureRoot()
writeFile(join(root, 'readme.txt'), 'hi')
expect(await discoverProjectCwd(root)).toBeNull()
})
it('extracts cwd from the first jsonl entry', async () => {
const root = makeFixtureRoot()
const entry = JSON.stringify({ type: 'assistant', cwd: '/Users/test/project', timestamp: new Date().toISOString() })
writeFile(join(root, 'session.jsonl'), entry + '\n')
expect(await discoverProjectCwd(root)).toBe('/Users/test/project')
})
})