codeburn/src/parser.ts
Carlo Fuselli b067136147
Some checks are pending
CI / semgrep (push) Waiting to run
fix: group Cowork sessions by space name and unify cross-provider project entries (#398)
* feat(claude): group Cowork sessions by space name instead of sanitized path

Claude Desktop local-agent-mode (Cowork) sessions were appearing as
separate projects with cryptic sanitized-path names like
`-Users-carlo-Library-Application-Support-Claude-...`. This meant
sessions from the same Cowork space (e.g. PhD_Thesis) showed up as
dozens of unrelated entries.

Changes:
- `discoverSessions()` detects paths inside the desktop sessions dir
  (`~/Library/Application Support/Claude/local-agent-mode-sessions`)
  and resolves the human-readable space name by reading the sibling
  `local_<id>.json` (for spaceId) and `spaces.json` (id → name map)
- `CODEBURN_DESKTOP_SESSIONS_DIR` env var overrides the desktop path
  for testability
- Sessions without a spaceId fall back to `userSelectedFolders[0]`
  basename, then the session title (instead of the unintelligible slug)
- `isCoworkOutputPath()` in parser.ts prevents the ephemeral
  `outputs/` cwd from being used as the canonical project key,
  which would have given every session a unique grouping key
- Parse-version bump (`cowork-space-grouping-v1`) invalidates stale
  cache so all files re-scan with the new logic on first run
- 5 new integration tests covering spaceId grouping, cross-space
  separation, folder fallback, title fallback, and no-metadata fallback

* test: use generic Project1/Project2/SubFolder names in cowork tests

* fix(parser): detect container-mode Cowork sessions by file path, not just cwd

Cowork sessions run in two modes:
- Local-mode: cwd is an ephemeral per-session outputs/ dir inside the
  desktop sessions directory — already handled.
- Container-mode: session runs inside Docker, so cwd is a container-local
  path like /sessions/<adjective-name> that does not exist on the host.
  The JSONL file still lives inside the desktop sessions directory, so we
  detect these by also checking the file path.

Rename isCoworkOutputPath → isCoworkSession and check both cwd and filePath.
Without this, container-mode sessions leaked their Docker cwd as the
canonical project path and showed as e.g. "sessions/trusting-inspiring-ri"
instead of grouping under the Cowork space name.

Also add a test covering the container-mode case and use generic names
(Project1, Project2, SubFolder) throughout the Cowork test suite.

* feat(parser): merge same-project sessions across AI providers by path

When a repository has sessions from multiple tools (e.g. both Claude Code
and Codex), parseAllSessions now groups them into a single ProjectSummary
instead of showing duplicate entries.

The previous outer merge keyed on p.project (a string name), so Claude's
"LibricicloAgain" and Codex's "Users-carlo-Desktop-LibricicloAgain" were
treated as different projects despite referring to the same directory.

The root cause is that Codex's sanitizeProject() strips the leading '/'
from cwds, producing "Users/carlo/Desktop/foo" instead of
"/Users/carlo/Desktop/foo". The new crossProviderKey normalises both by
stripping leading slashes and lowercasing before comparison. Non-path
identifiers (like Cowork space names: "PhD_Thesis") have no '/' and fall
back to comparing the project name, so they are unaffected.

* feat(parser): resolve other-provider worktrees before cross-provider merge

Codex (and similar tools) store worktrees under paths like
~/.codex/worktrees/<hash>/Repo. These were not being resolved to their
main-repo path because canonicalizeProviderCallProject only operates on
call.projectPath, which Codex doesn't set — the project is inferred from
the session source slug instead.

Fix: before the cross-provider merge in parseAllSessions, run worktree
resolution on each non-Claude ProjectSummary's projectPath. Restore the
missing leading '/' (Codex strips it via sanitizeProject) before calling
resolveCanonicalProjectPath so the stat() can find the .git file. If the
path is a worktree, replace project name and path with the canonical
main-repo values so the downstream crossProviderKey matches.

* feat(dashboard): show more projects in By Project panel to fill panel height

The panel was hardcoded to 8 projects regardless of the Daily Activity
panel height alongside it. Pass the same `days` value used by Daily Activity
so the two panels stay in sync: 14 for week/6-months views, 31 for
month/30-days, 20 for all-time, and a minimum of 8 for the today view.

* feat(parser): resolve subdirectory sessions to their git repo root

When a cwd like ~/Repo/src/subdir is recorded, it should group with the
rest of the ~/Repo project. Previously resolveCanonicalProjectPath only
checked for a .git entry at the exact cwd, so subdirectory sessions
created separate ProjectSummary entries.

Walk up the directory tree until a .git entry is found. Guard against
foreign paths (a Windows path like C:\… on macOS cannot be walked on
the local filesystem) by only walking paths that look like absolute
paths on the current platform.

This also folds non-Claude provider subdirectory sessions into the parent
repo via resolveOtherProjectWorktrees, which now triggers whenever
canonical.path differs from the input path (not only for git worktrees).

* Address reviewer feedback: deduplicate getDesktopSessionsDir and fix test

Export getDesktopSessionsDir from claude.ts and import it in parser.ts
instead of maintaining an identical copy in both files.

Update the 'merges by sanitized slug' test to reflect the new path-aware
merge behavior: sessions with the same slug but different cwds now stay
separate (crossProviderKey keys by path, not slug), which is correct since
they represent different repos.
2026-05-25 13:41:53 -07:00

2185 lines
83 KiB
TypeScript

import { lstat, readFile, readdir, stat } from 'fs/promises'
import { basename, dirname, join, resolve, sep } from 'path'
import { readSessionLines } from './fs-utils.js'
import { calculateCost, getShortModelName } from './models.js'
import { discoverAllSessions, getProvider } from './providers/index.js'
import { flushCodexCache } from './codex-cache.js'
import { antigravityCascadeIdFromPath, flushAntigravityCache, shouldReparseAntigravitySource } from './providers/antigravity.js'
import { getDesktopSessionsDir } from './providers/claude.js'
import { isSqliteBusyError } from './sqlite.js'
import {
type CachedCall,
type CachedFile,
type CachedTurn,
type ProviderSection,
type SessionCache,
cleanupOrphanedTempFiles,
computeEnvFingerprint,
fingerprintFile,
loadCache,
reconcileFile,
saveCache,
} from './session-cache.js'
import type { ParsedProviderCall } from './providers/types.js'
import type {
AssistantMessageContent,
ClassifiedTurn,
ContentBlock,
DateRange,
JournalEntry,
ParsedApiCall,
ParsedTurn,
ProjectSummary,
SessionSummary,
TokenUsage,
ToolCall,
ToolUseBlock,
} from './types.js'
import { classifyTurn, BASH_TOOLS, EDIT_TOOLS } from './classifier.js'
import { extractBashCommands } from './bash-utils.js'
function unsanitizePath(dirName: string): string {
return dirName.replace(/-/g, '/')
}
function normalizeProjectPathKey(projectPath: string): string {
const normalized = projectPath.trim().replace(/\\/g, '/')
return (normalized.replace(/\/+$/, '') || normalized).toLowerCase()
}
function projectNameFromPath(projectPath: string, fallback: string): string {
const normalized = projectPath.trim().replace(/\\/g, '/').replace(/\/+$/, '')
return normalized.split('/').filter(Boolean).pop() ?? fallback
}
// Returns true for sessions whose canonical project key must NOT be derived
// from the cwd. Cowork sessions come in two flavours:
// 1. Local-mode: cwd is an ephemeral per-session outputs/ dir inside the
// desktop sessions directory (detected by checking the cwd).
// 2. Container-mode: the session runs inside a Docker container so cwd is
// something like /sessions/<adjective-name> — not a real path on the host.
// We detect these by checking the JSONL file path instead: if the file
// lives inside the desktop sessions directory, the cwd is container-local
// and must not become the canonical project key.
// In both cases the grouping key comes from the Cowork space name resolved in
// claude.ts::discoverSessions().
function isCoworkSession(cwd: string, filePath: string): boolean {
const base = resolve(getDesktopSessionsDir())
const inBase = (p: string) => p.startsWith(base + sep) || p.startsWith(base + '/')
return inBase(resolve(cwd)) || inBase(resolve(filePath))
}
async function resolveCanonicalProjectPath(cwd: string): Promise<{ path: string; isWorktree: boolean }> {
const trimmed = cwd.trim()
if (!trimmed) return { path: cwd, isWorktree: false }
// Walk up the directory tree to find the nearest .git entry. This handles
// three cases: (1) cwd IS the repo root (.git is a directory), (2) cwd is a
// git worktree (.git is a file pointing back to the main repo), (3) cwd is a
// subdirectory of a repo — we resolve up to the repo root so subdirectory
// sessions group with the rest of the project even when the subdir no longer
// exists on disk.
// Guard against foreign paths (e.g. a Windows path recorded on a machine
// that now runs macOS): only walk paths that look like absolute paths on the
// current platform. A relative or foreign-format path cannot be walked on
// the current filesystem without risking false positives.
const isAbsoluteOnCurrentPlatform = process.platform === 'win32'
? /^[a-zA-Z]:[/\\]/.test(trimmed)
: trimmed.startsWith('/')
if (!isAbsoluteOnCurrentPlatform) return { path: cwd, isWorktree: false }
let dir = trimmed
while (true) {
const gitEntry = join(dir, '.git')
const entryStat = await lstat(gitEntry).catch(() => null)
if (entryStat?.isDirectory()) return { path: dir, isWorktree: false }
if (entryStat?.isFile()) {
const gitFile = await readFile(gitEntry, 'utf-8').catch(() => null)
if (gitFile === null) return { path: dir, isWorktree: false }
const match = gitFile.match(/^gitdir:\s*(.+?)\s*$/m)
if (!match?.[1]) return { path: dir, isWorktree: false }
const gitDir = resolve(dir, match[1])
const normalizedGitDir = gitDir.replace(/\\/g, '/')
const worktreeMarker = '/.git/worktrees/'
const markerIndex = normalizedGitDir.lastIndexOf(worktreeMarker)
if (markerIndex === -1) return { path: dir, isWorktree: false }
return { path: normalizedGitDir.slice(0, markerIndex), isWorktree: true }
}
const parent = dirname(dir)
if (parent === dir) return { path: cwd, isWorktree: false }
dir = parent
}
}
const LARGE_JSONL_LINE_BYTES = 32 * 1024
export function parseJsonlLine(line: string | Buffer): JournalEntry | null {
if (Buffer.isBuffer(line)) {
if (line.length > LARGE_JSONL_LINE_BYTES) return parseLargeJsonlBuffer(line)
try {
return JSON.parse(line.toString('utf-8')) as JournalEntry
} catch {
return null
}
}
if (line.length > LARGE_JSONL_LINE_BYTES) return parseLargeJsonlLine(line)
try {
return JSON.parse(line) as JournalEntry
} catch {
return null
}
}
const RAW_HEAD_BYTES = 2048
type JsonValueBounds = {
start: number
end: number
kind: 'string' | 'object' | 'array' | 'scalar'
}
function findJsonStringEnd(source: string, start: number, limit = source.length): number {
for (let i = start + 1; i < limit; i++) {
const ch = source.charCodeAt(i)
if (ch === 0x5c) {
i++
continue
}
if (ch === 0x22) return i
}
return -1
}
function findJsonContainerEnd(source: string, start: number, open: number, close: number, limit = source.length): number {
let depth = 0
let inString = false
for (let i = start; i < limit; i++) {
const ch = source.charCodeAt(i)
if (inString) {
if (ch === 0x5c) {
i++
} else if (ch === 0x22) {
inString = false
}
continue
}
if (ch === 0x22) {
inString = true
} else if (ch === open) {
depth++
} else if (ch === close) {
depth--
if (depth === 0) return i
}
}
return -1
}
function findJsonValueBounds(source: string, start: number, limit = source.length): JsonValueBounds | null {
let i = start
while (i < limit && /\s/.test(source[i]!)) i++
if (i >= limit) return null
const ch = source.charCodeAt(i)
if (ch === 0x22) {
const end = findJsonStringEnd(source, i, limit)
return end === -1 ? null : { start: i, end: end + 1, kind: 'string' }
}
if (ch === 0x7b) {
const end = findJsonContainerEnd(source, i, 0x7b, 0x7d, limit)
return end === -1 ? null : { start: i, end: end + 1, kind: 'object' }
}
if (ch === 0x5b) {
const end = findJsonContainerEnd(source, i, 0x5b, 0x5d, limit)
return end === -1 ? null : { start: i, end: end + 1, kind: 'array' }
}
let end = i
while (end < limit) {
const c = source.charCodeAt(end)
if (c === 0x2c || c === 0x7d || c === 0x5d || /\s/.test(source[end]!)) break
end++
}
return { start: i, end, kind: 'scalar' }
}
function findObjectFieldValue(source: string, objectStart: number, objectEnd: number, field: string): JsonValueBounds | null {
if (source.charCodeAt(objectStart) !== 0x7b) return null
let i = objectStart + 1
while (i < objectEnd - 1) {
while (i < objectEnd && /\s/.test(source[i]!)) i++
if (source.charCodeAt(i) === 0x2c) {
i++
continue
}
if (source.charCodeAt(i) !== 0x22) {
i++
continue
}
const keyEnd = findJsonStringEnd(source, i, objectEnd)
if (keyEnd === -1) return null
const key = source.slice(i + 1, keyEnd)
i = keyEnd + 1
while (i < objectEnd && /\s/.test(source[i]!)) i++
if (source.charCodeAt(i) !== 0x3a) continue
const value = findJsonValueBounds(source, i + 1, objectEnd)
if (!value) return null
if (key === field) return value
i = value.end
}
return null
}
function readJsonString(source: string, bounds: JsonValueBounds | null, cap = Number.POSITIVE_INFINITY): string | undefined {
if (!bounds || bounds.kind !== 'string') return undefined
let out = ''
for (let i = bounds.start + 1; i < bounds.end - 1 && out.length < cap; i++) {
const ch = source[i]!
if (ch !== '\\') {
out += ch
continue
}
const next = source[++i]
if (!next) break
if (next === 'n') out += '\n'
else if (next === 'r') out += '\r'
else if (next === 't') out += '\t'
else if (next === 'b') out += '\b'
else if (next === 'f') out += '\f'
else if (next === 'u' && i + 4 < bounds.end) {
const hex = source.slice(i + 1, i + 5)
const code = Number.parseInt(hex, 16)
if (Number.isFinite(code)) out += String.fromCharCode(code)
i += 4
} else {
out += next
}
}
return out
}
function readJsonNumberField(source: string, objectBounds: JsonValueBounds | null, field: string): number | undefined {
if (!objectBounds || objectBounds.kind !== 'object') return undefined
const bounds = findObjectFieldValue(source, objectBounds.start, objectBounds.end, field)
if (!bounds) return undefined
const value = Number(source.slice(bounds.start, bounds.end))
return Number.isFinite(value) ? value : undefined
}
function parseLargeUsage(source: string, usageBounds: JsonValueBounds | null) {
const usage: AssistantMessageContent['usage'] = {
input_tokens: readJsonNumberField(source, usageBounds, 'input_tokens') ?? 0,
output_tokens: readJsonNumberField(source, usageBounds, 'output_tokens') ?? 0,
cache_creation_input_tokens: readJsonNumberField(source, usageBounds, 'cache_creation_input_tokens'),
cache_read_input_tokens: readJsonNumberField(source, usageBounds, 'cache_read_input_tokens'),
}
if (usageBounds?.kind === 'object') {
const cacheCreation = findObjectFieldValue(source, usageBounds.start, usageBounds.end, 'cache_creation')
const ephemeral5m = readJsonNumberField(source, cacheCreation, 'ephemeral_5m_input_tokens')
const ephemeral1h = readJsonNumberField(source, cacheCreation, 'ephemeral_1h_input_tokens')
if (ephemeral5m !== undefined || ephemeral1h !== undefined) {
;(usage as AssistantMessageContent['usage']).cache_creation = {
...(ephemeral5m !== undefined ? { ephemeral_5m_input_tokens: ephemeral5m } : {}),
...(ephemeral1h !== undefined ? { ephemeral_1h_input_tokens: ephemeral1h } : {}),
}
}
const serverToolUse = findObjectFieldValue(source, usageBounds.start, usageBounds.end, 'server_tool_use')
const webSearch = readJsonNumberField(source, serverToolUse, 'web_search_requests')
const webFetch = readJsonNumberField(source, serverToolUse, 'web_fetch_requests')
if (webSearch !== undefined || webFetch !== undefined) {
;(usage as AssistantMessageContent['usage']).server_tool_use = {
...(webSearch !== undefined ? { web_search_requests: webSearch } : {}),
...(webFetch !== undefined ? { web_fetch_requests: webFetch } : {}),
}
}
const speed = readJsonString(source, findObjectFieldValue(source, usageBounds.start, usageBounds.end, 'speed'))
if (speed === 'standard' || speed === 'fast') usage.speed = speed
}
return usage
}
function extractLargeToolBlocks(source: string, contentBounds: JsonValueBounds | null): ToolUseBlock[] {
if (!contentBounds || contentBounds.kind !== 'array') return []
const tools: ToolUseBlock[] = []
let i = contentBounds.start + 1
while (i < contentBounds.end - 1 && tools.length < MAX_TOOL_BLOCKS) {
while (i < contentBounds.end && /\s/.test(source[i]!)) i++
if (source.charCodeAt(i) === 0x2c) {
i++
continue
}
if (source.charCodeAt(i) !== 0x7b) {
i++
continue
}
const objectEnd = findJsonContainerEnd(source, i, 0x7b, 0x7d, contentBounds.end)
if (objectEnd === -1) break
const objectBounds = { start: i, end: objectEnd + 1, kind: 'object' as const }
const blockType = readJsonString(source, findObjectFieldValue(source, objectBounds.start, objectBounds.end, 'type'))
if (blockType === 'tool_use') {
const name = readJsonString(source, findObjectFieldValue(source, objectBounds.start, objectBounds.end, 'name')) ?? ''
const id = readJsonString(source, findObjectFieldValue(source, objectBounds.start, objectBounds.end, 'id')) ?? ''
const inputBounds = findObjectFieldValue(source, objectBounds.start, objectBounds.end, 'input')
const input: Record<string, unknown> = {}
if (inputBounds?.kind === 'object') {
if (name === 'Skill') {
const skill = readJsonString(source, findObjectFieldValue(source, inputBounds.start, inputBounds.end, 'skill'), 200)
const skillName = readJsonString(source, findObjectFieldValue(source, inputBounds.start, inputBounds.end, 'name'), 200)
if (skill !== undefined) input['skill'] = skill
if (skillName !== undefined) input['name'] = skillName
} else if (name === 'Read' || name === 'FileReadTool' || EDIT_TOOLS.has(name)) {
const filePath = readJsonString(source, findObjectFieldValue(source, inputBounds.start, inputBounds.end, 'file_path'), BASH_COMMAND_CAP)
if (filePath !== undefined) input['file_path'] = filePath
} else if (name === 'Agent' || name === 'Task') {
const subagentType = readJsonString(source, findObjectFieldValue(source, inputBounds.start, inputBounds.end, 'subagent_type'), 200)
if (subagentType !== undefined) input['subagent_type'] = subagentType
} else if (BASH_TOOLS.has(name)) {
const command = readJsonString(source, findObjectFieldValue(source, inputBounds.start, inputBounds.end, 'command'), BASH_COMMAND_CAP)
if (command !== undefined) input['command'] = command
}
}
tools.push({ type: 'tool_use', id, name, input })
}
i = objectEnd + 1
}
return tools
}
function extractLargeUserText(source: string, contentBounds: JsonValueBounds | null): string | undefined {
if (!contentBounds) return undefined
if (contentBounds.kind === 'string') return readJsonString(source, contentBounds, USER_TEXT_CAP)
if (contentBounds.kind !== 'array') return undefined
let text = ''
let i = contentBounds.start + 1
while (i < contentBounds.end - 1 && text.length < USER_TEXT_CAP) {
while (i < contentBounds.end && /\s/.test(source[i]!)) i++
if (source.charCodeAt(i) === 0x2c) {
i++
continue
}
if (source.charCodeAt(i) !== 0x7b) {
i++
continue
}
const objectEnd = findJsonContainerEnd(source, i, 0x7b, 0x7d, contentBounds.end)
if (objectEnd === -1) break
const objectBounds = { start: i, end: objectEnd + 1, kind: 'object' as const }
const type = readJsonString(source, findObjectFieldValue(source, objectBounds.start, objectBounds.end, 'type'))
if (type === 'text' || type === 'input_text') {
const part = readJsonString(
source,
findObjectFieldValue(source, objectBounds.start, objectBounds.end, 'text'),
USER_TEXT_CAP - text.length,
)
if (part) text += (text ? ' ' : '') + part
}
i = objectEnd + 1
}
return text || undefined
}
function extractLargeAddedNames(source: string, attachmentBounds: JsonValueBounds | null): string[] {
if (!attachmentBounds || attachmentBounds.kind !== 'object') return []
const attachmentType = readJsonString(source, findObjectFieldValue(source, attachmentBounds.start, attachmentBounds.end, 'type'))
if (attachmentType !== 'deferred_tools_delta') return []
const addedNames = findObjectFieldValue(source, attachmentBounds.start, attachmentBounds.end, 'addedNames')
if (!addedNames || addedNames.kind !== 'array') return []
const names: string[] = []
let i = addedNames.start + 1
while (i < addedNames.end - 1 && names.length < MAX_ADDED_NAMES) {
while (i < addedNames.end && /\s/.test(source[i]!)) i++
if (source.charCodeAt(i) === 0x2c) {
i++
continue
}
if (source.charCodeAt(i) !== 0x22) {
i++
continue
}
const end = findJsonStringEnd(source, i, addedNames.end)
if (end === -1) break
const name = readJsonString(source, { start: i, end: end + 1, kind: 'string' }, 500)
if (name) names.push(name)
i = end + 1
}
return names
}
function parseLargeJsonlLine(line: string): JournalEntry | null {
const rootEnd = findJsonContainerEnd(line, 0, 0x7b, 0x7d)
if (rootEnd === -1) return null
const rootStart = 0
const rootLimit = rootEnd + 1
const type = readJsonString(line, findObjectFieldValue(line, rootStart, rootLimit, 'type'))
if (!type) return null
const entry: JournalEntry = { type }
const timestamp = readJsonString(line, findObjectFieldValue(line, rootStart, rootLimit, 'timestamp'))
const sessionId = readJsonString(line, findObjectFieldValue(line, rootStart, rootLimit, 'sessionId'))
const cwd = readJsonString(line, findObjectFieldValue(line, rootStart, rootLimit, 'cwd'))
if (timestamp !== undefined) entry.timestamp = timestamp
if (sessionId !== undefined) entry.sessionId = sessionId
if (cwd !== undefined) entry.cwd = cwd
const addedNames = extractLargeAddedNames(line, findObjectFieldValue(line, rootStart, rootLimit, 'attachment'))
if (addedNames.length > 0) {
;(entry as Record<string, unknown>)['attachment'] = { type: 'deferred_tools_delta', addedNames }
}
if (type === 'user') {
const message = findObjectFieldValue(line, rootStart, rootLimit, 'message')
if (message?.kind === 'object') {
const content = findObjectFieldValue(line, message.start, message.end, 'content')
const text = extractLargeUserText(line, content)
if (text !== undefined) entry.message = { role: 'user', content: text }
}
return entry
}
if (type !== 'assistant') return entry
const message = findObjectFieldValue(line, rootStart, rootLimit, 'message')
if (message?.kind !== 'object') return entry
const model = readJsonString(line, findObjectFieldValue(line, message.start, message.end, 'model'))
const usageBounds = findObjectFieldValue(line, message.start, message.end, 'usage')
if (!model || usageBounds?.kind !== 'object') return entry
const id = readJsonString(line, findObjectFieldValue(line, message.start, message.end, 'id'))
const contentBounds = findObjectFieldValue(line, message.start, message.end, 'content')
entry.message = {
type: 'message',
role: 'assistant',
model,
...(id !== undefined ? { id } : {}),
content: extractLargeToolBlocks(line, contentBounds),
usage: parseLargeUsage(line, usageBounds),
}
return entry
}
type BufferJsonValueBounds = {
start: number
end: number
kind: 'string' | 'object' | 'array' | 'scalar'
}
function isJsonWhitespaceByte(ch: number | undefined): boolean {
return ch === 0x20 || ch === 0x0a || ch === 0x0d || ch === 0x09
}
function findJsonStringEndBuffer(source: Buffer, start: number, limit = source.length): number {
for (let i = start + 1; i < limit; i++) {
const ch = source[i]
if (ch === 0x5c) {
i++
continue
}
if (ch === 0x22) return i
}
return -1
}
function findJsonContainerEndBuffer(source: Buffer, start: number, open: number, close: number, limit = source.length): number {
let depth = 0
let inString = false
for (let i = start; i < limit; i++) {
const ch = source[i]
if (inString) {
if (ch === 0x5c) {
i++
} else if (ch === 0x22) {
inString = false
}
continue
}
if (ch === 0x22) {
inString = true
} else if (ch === open) {
depth++
} else if (ch === close) {
depth--
if (depth === 0) return i
}
}
return -1
}
function findJsonValueBoundsBuffer(source: Buffer, start: number, limit = source.length): BufferJsonValueBounds | null {
let i = start
while (i < limit && isJsonWhitespaceByte(source[i])) i++
if (i >= limit) return null
const ch = source[i]
if (ch === 0x22) {
const end = findJsonStringEndBuffer(source, i, limit)
return end === -1 ? null : { start: i, end: end + 1, kind: 'string' }
}
if (ch === 0x7b) {
const end = findJsonContainerEndBuffer(source, i, 0x7b, 0x7d, limit)
return end === -1 ? null : { start: i, end: end + 1, kind: 'object' }
}
if (ch === 0x5b) {
const end = findJsonContainerEndBuffer(source, i, 0x5b, 0x5d, limit)
return end === -1 ? null : { start: i, end: end + 1, kind: 'array' }
}
let end = i
while (end < limit) {
const c = source[end]
if (c === 0x2c || c === 0x7d || c === 0x5d || isJsonWhitespaceByte(c)) break
end++
}
return { start: i, end, kind: 'scalar' }
}
function bufferKeyEquals(source: Buffer, keyStart: number, keyEnd: number, field: string): boolean {
if (keyEnd - keyStart !== field.length) return false
return source.subarray(keyStart, keyEnd).equals(Buffer.from(field))
}
function findObjectFieldValueBuffer(source: Buffer, objectStart: number, objectEnd: number, field: string): BufferJsonValueBounds | null {
if (source[objectStart] !== 0x7b) return null
let i = objectStart + 1
while (i < objectEnd - 1) {
while (i < objectEnd && isJsonWhitespaceByte(source[i])) i++
if (source[i] === 0x2c) {
i++
continue
}
if (source[i] !== 0x22) {
i++
continue
}
const keyEnd = findJsonStringEndBuffer(source, i, objectEnd)
if (keyEnd === -1) return null
const keyStart = i + 1
i = keyEnd + 1
while (i < objectEnd && isJsonWhitespaceByte(source[i])) i++
if (source[i] !== 0x3a) continue
const value = findJsonValueBoundsBuffer(source, i + 1, objectEnd)
if (!value) return null
if (bufferKeyEquals(source, keyStart, keyEnd, field)) return value
i = value.end
}
return null
}
function appendBufferJsonSegment(source: Buffer, start: number, end: number, current: string, cap: number): string {
if (start >= end || current.length >= cap) return current
const remaining = cap - current.length
const cappedEnd = Number.isFinite(cap) ? Math.min(end, start + remaining * 4) : end
return current + source.subarray(start, cappedEnd).toString('utf-8').slice(0, remaining)
}
function readJsonStringBuffer(source: Buffer, bounds: BufferJsonValueBounds | null, cap = Number.POSITIVE_INFINITY): string | undefined {
if (!bounds || bounds.kind !== 'string') return undefined
let out = ''
let segmentStart = bounds.start + 1
for (let i = bounds.start + 1; i < bounds.end - 1 && out.length < cap; i++) {
const ch = source[i]
if (ch !== 0x5c) continue
out = appendBufferJsonSegment(source, segmentStart, i, out, cap)
if (out.length >= cap) break
const next = source[++i]
if (next === undefined) break
if (next === 0x6e) out += '\n'
else if (next === 0x72) out += '\r'
else if (next === 0x74) out += '\t'
else if (next === 0x62) out += '\b'
else if (next === 0x66) out += '\f'
else if (next === 0x75 && i + 4 < bounds.end) {
const hex = source.subarray(i + 1, i + 5).toString('ascii')
const code = Number.parseInt(hex, 16)
if (Number.isFinite(code)) out += String.fromCharCode(code)
i += 4
} else {
out += String.fromCharCode(next)
}
segmentStart = i + 1
}
return appendBufferJsonSegment(source, segmentStart, bounds.end - 1, out, cap)
}
function readJsonNumberFieldBuffer(source: Buffer, objectBounds: BufferJsonValueBounds | null, field: string): number | undefined {
if (!objectBounds || objectBounds.kind !== 'object') return undefined
const bounds = findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, field)
if (!bounds) return undefined
const value = Number(source.subarray(bounds.start, bounds.end).toString('ascii'))
return Number.isFinite(value) ? value : undefined
}
function parseLargeUsageBuffer(source: Buffer, usageBounds: BufferJsonValueBounds | null) {
const usage: AssistantMessageContent['usage'] = {
input_tokens: readJsonNumberFieldBuffer(source, usageBounds, 'input_tokens') ?? 0,
output_tokens: readJsonNumberFieldBuffer(source, usageBounds, 'output_tokens') ?? 0,
cache_creation_input_tokens: readJsonNumberFieldBuffer(source, usageBounds, 'cache_creation_input_tokens'),
cache_read_input_tokens: readJsonNumberFieldBuffer(source, usageBounds, 'cache_read_input_tokens'),
}
if (usageBounds?.kind === 'object') {
const cacheCreation = findObjectFieldValueBuffer(source, usageBounds.start, usageBounds.end, 'cache_creation')
const ephemeral5m = readJsonNumberFieldBuffer(source, cacheCreation, 'ephemeral_5m_input_tokens')
const ephemeral1h = readJsonNumberFieldBuffer(source, cacheCreation, 'ephemeral_1h_input_tokens')
if (ephemeral5m !== undefined || ephemeral1h !== undefined) {
;(usage as AssistantMessageContent['usage']).cache_creation = {
...(ephemeral5m !== undefined ? { ephemeral_5m_input_tokens: ephemeral5m } : {}),
...(ephemeral1h !== undefined ? { ephemeral_1h_input_tokens: ephemeral1h } : {}),
}
}
const serverToolUse = findObjectFieldValueBuffer(source, usageBounds.start, usageBounds.end, 'server_tool_use')
const webSearch = readJsonNumberFieldBuffer(source, serverToolUse, 'web_search_requests')
const webFetch = readJsonNumberFieldBuffer(source, serverToolUse, 'web_fetch_requests')
if (webSearch !== undefined || webFetch !== undefined) {
;(usage as AssistantMessageContent['usage']).server_tool_use = {
...(webSearch !== undefined ? { web_search_requests: webSearch } : {}),
...(webFetch !== undefined ? { web_fetch_requests: webFetch } : {}),
}
}
const speed = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, usageBounds.start, usageBounds.end, 'speed'))
if (speed === 'standard' || speed === 'fast') usage.speed = speed
}
return usage
}
function extractLargeToolBlocksBuffer(source: Buffer, contentBounds: BufferJsonValueBounds | null): ToolUseBlock[] {
if (!contentBounds || contentBounds.kind !== 'array') return []
const tools: ToolUseBlock[] = []
let i = contentBounds.start + 1
while (i < contentBounds.end - 1 && tools.length < MAX_TOOL_BLOCKS) {
while (i < contentBounds.end && isJsonWhitespaceByte(source[i])) i++
if (source[i] === 0x2c) {
i++
continue
}
if (source[i] !== 0x7b) {
i++
continue
}
const objectEnd = findJsonContainerEndBuffer(source, i, 0x7b, 0x7d, contentBounds.end)
if (objectEnd === -1) break
const objectBounds = { start: i, end: objectEnd + 1, kind: 'object' as const }
const blockType = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, 'type'))
if (blockType === 'tool_use') {
const name = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, 'name')) ?? ''
const id = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, 'id')) ?? ''
const inputBounds = findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, 'input')
const input: Record<string, unknown> = {}
if (inputBounds?.kind === 'object') {
if (name === 'Skill') {
const skill = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, inputBounds.start, inputBounds.end, 'skill'), 200)
const skillName = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, inputBounds.start, inputBounds.end, 'name'), 200)
if (skill !== undefined) input['skill'] = skill
if (skillName !== undefined) input['name'] = skillName
} else if (name === 'Read' || name === 'FileReadTool' || EDIT_TOOLS.has(name)) {
const filePath = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, inputBounds.start, inputBounds.end, 'file_path'), BASH_COMMAND_CAP)
if (filePath !== undefined) input['file_path'] = filePath
} else if (name === 'Agent' || name === 'Task') {
const subagentType = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, inputBounds.start, inputBounds.end, 'subagent_type'), 200)
if (subagentType !== undefined) input['subagent_type'] = subagentType
} else if (BASH_TOOLS.has(name)) {
const command = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, inputBounds.start, inputBounds.end, 'command'), BASH_COMMAND_CAP)
if (command !== undefined) input['command'] = command
}
}
tools.push({ type: 'tool_use', id, name, input })
}
i = objectEnd + 1
}
return tools
}
function extractLargeUserTextBuffer(source: Buffer, contentBounds: BufferJsonValueBounds | null): string | undefined {
if (!contentBounds) return undefined
if (contentBounds.kind === 'string') return readJsonStringBuffer(source, contentBounds, USER_TEXT_CAP)
if (contentBounds.kind !== 'array') return undefined
let text = ''
let i = contentBounds.start + 1
while (i < contentBounds.end - 1 && text.length < USER_TEXT_CAP) {
while (i < contentBounds.end && isJsonWhitespaceByte(source[i])) i++
if (source[i] === 0x2c) {
i++
continue
}
if (source[i] !== 0x7b) {
i++
continue
}
const objectEnd = findJsonContainerEndBuffer(source, i, 0x7b, 0x7d, contentBounds.end)
if (objectEnd === -1) break
const objectBounds = { start: i, end: objectEnd + 1, kind: 'object' as const }
const type = readJsonStringBuffer(source, findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, 'type'))
if (type === 'text' || type === 'input_text') {
const part = readJsonStringBuffer(
source,
findObjectFieldValueBuffer(source, objectBounds.start, objectBounds.end, 'text'),
USER_TEXT_CAP - text.length,
)
if (part) text += (text ? ' ' : '') + part
}
i = objectEnd + 1
}
return text || undefined
}
function extractLargeAddedNamesBuffer(source: Buffer, attachmentBounds: BufferJsonValueBounds | null): string[] {
if (!attachmentBounds || attachmentBounds.kind !== 'object') return []
const attachmentType = readJsonStringBuffer(
source,
findObjectFieldValueBuffer(source, attachmentBounds.start, attachmentBounds.end, 'type'),
)
if (attachmentType !== 'deferred_tools_delta') return []
const addedNames = findObjectFieldValueBuffer(source, attachmentBounds.start, attachmentBounds.end, 'addedNames')
if (!addedNames || addedNames.kind !== 'array') return []
const names: string[] = []
let i = addedNames.start + 1
while (i < addedNames.end - 1 && names.length < MAX_ADDED_NAMES) {
while (i < addedNames.end && isJsonWhitespaceByte(source[i])) i++
if (source[i] === 0x2c) {
i++
continue
}
if (source[i] !== 0x22) {
i++
continue
}
const end = findJsonStringEndBuffer(source, i, addedNames.end)
if (end === -1) break
const name = readJsonStringBuffer(source, { start: i, end: end + 1, kind: 'string' }, 500)
if (name) names.push(name)
i = end + 1
}
return names
}
function parseLargeJsonlBuffer(line: Buffer): JournalEntry | null {
let rootStart = 0
while (rootStart < line.length && isJsonWhitespaceByte(line[rootStart])) rootStart++
if (line[rootStart] !== 0x7b) return null
const rootEnd = findJsonContainerEndBuffer(line, rootStart, 0x7b, 0x7d)
if (rootEnd === -1) return null
const rootLimit = rootEnd + 1
const type = readJsonStringBuffer(line, findObjectFieldValueBuffer(line, rootStart, rootLimit, 'type'))
if (!type) return null
const entry: JournalEntry = { type }
const timestamp = readJsonStringBuffer(line, findObjectFieldValueBuffer(line, rootStart, rootLimit, 'timestamp'))
const sessionId = readJsonStringBuffer(line, findObjectFieldValueBuffer(line, rootStart, rootLimit, 'sessionId'))
const cwd = readJsonStringBuffer(line, findObjectFieldValueBuffer(line, rootStart, rootLimit, 'cwd'))
if (timestamp !== undefined) entry.timestamp = timestamp
if (sessionId !== undefined) entry.sessionId = sessionId
if (cwd !== undefined) entry.cwd = cwd
const addedNames = extractLargeAddedNamesBuffer(line, findObjectFieldValueBuffer(line, rootStart, rootLimit, 'attachment'))
if (addedNames.length > 0) {
;(entry as Record<string, unknown>)['attachment'] = { type: 'deferred_tools_delta', addedNames }
}
if (type === 'user') {
const message = findObjectFieldValueBuffer(line, rootStart, rootLimit, 'message')
if (message?.kind === 'object') {
const content = findObjectFieldValueBuffer(line, message.start, message.end, 'content')
const text = extractLargeUserTextBuffer(line, content)
if (text !== undefined) entry.message = { role: 'user', content: text }
}
return entry
}
if (type !== 'assistant') return entry
const message = findObjectFieldValueBuffer(line, rootStart, rootLimit, 'message')
if (message?.kind !== 'object') return entry
const model = readJsonStringBuffer(line, findObjectFieldValueBuffer(line, message.start, message.end, 'model'))
const usageBounds = findObjectFieldValueBuffer(line, message.start, message.end, 'usage')
if (!model || usageBounds?.kind !== 'object') return entry
const id = readJsonStringBuffer(line, findObjectFieldValueBuffer(line, message.start, message.end, 'id'))
const contentBounds = findObjectFieldValueBuffer(line, message.start, message.end, 'content')
entry.message = {
type: 'message',
role: 'assistant',
model,
...(id !== undefined ? { id } : {}),
content: extractLargeToolBlocksBuffer(line, contentBounds),
usage: parseLargeUsageBuffer(line, usageBounds),
}
return entry
}
function getTopLevelRawJsonStringField(head: string, field: string): string | null {
let i = 0
while (i < head.length && /\s/.test(head[i]!)) i++
if (head.charCodeAt(i) !== 0x7b) return null
i++
while (i < head.length) {
while (i < head.length && /\s/.test(head[i]!)) i++
if (head.charCodeAt(i) === 0x2c) {
i++
continue
}
if (head.charCodeAt(i) === 0x7d) return null
if (head.charCodeAt(i) !== 0x22) return null
const keyEnd = findJsonStringEnd(head, i)
if (keyEnd === -1) return null
const key = head.slice(i + 1, keyEnd)
i = keyEnd + 1
while (i < head.length && /\s/.test(head[i]!)) i++
if (head.charCodeAt(i) !== 0x3a) return null
const value = findJsonValueBounds(head, i + 1)
if (!value) return null
if (key === field) return readJsonString(head, value) ?? null
i = value.end
}
return null
}
export function shouldSkipLine(line: string, threshold: string): boolean {
const head = line.length > RAW_HEAD_BYTES ? line.slice(0, RAW_HEAD_BYTES) : line
const type = getTopLevelRawJsonStringField(head, 'type')
if (type !== 'user' && type !== 'assistant') return false
const ts = getTopLevelRawJsonStringField(head, 'timestamp')
if (!ts || ts.length < 10) return false
return ts < threshold
}
const USER_TEXT_CAP = 2000
const BASH_COMMAND_CAP = 2000
const MAX_TOOL_BLOCKS = 500
const MAX_ADDED_NAMES = 1000
export function compactEntry(raw: JournalEntry): JournalEntry {
const entry: JournalEntry = { type: raw.type }
if (raw.timestamp !== undefined) entry.timestamp = raw.timestamp
if (raw.sessionId !== undefined) entry.sessionId = raw.sessionId
if (raw.cwd !== undefined) entry.cwd = raw.cwd
const att = (raw as Record<string, unknown>)['attachment']
if (att && typeof att === 'object') {
const a = att as Record<string, unknown>
if (a['type'] === 'deferred_tools_delta' && Array.isArray(a['addedNames'])) {
const names: string[] = []
for (let i = 0; i < Math.min(a['addedNames'].length, MAX_ADDED_NAMES); i++) {
const n = a['addedNames'][i]
if (typeof n === 'string') names.push(n)
}
;(entry as Record<string, unknown>)['attachment'] = { type: 'deferred_tools_delta', addedNames: names }
}
}
if (!raw.message) return entry
if (raw.message.role === 'user') {
const content = raw.message.content
if (typeof content === 'string') {
entry.message = { role: 'user', content: content.slice(0, USER_TEXT_CAP) }
} else if (Array.isArray(content)) {
let remaining = USER_TEXT_CAP
const blocks: { type: 'text'; text: string }[] = []
for (const b of content) {
if (remaining <= 0) break
if (!b || typeof b !== 'object' || b.type !== 'text') continue
const text = (b as { text?: unknown }).text
if (typeof text !== 'string') continue
const sliced = text.slice(0, remaining)
blocks.push({ type: 'text', text: sliced })
remaining -= sliced.length
}
entry.message = { role: 'user', content: blocks }
}
return entry
}
const msg = raw.message as AssistantMessageContent
if (!msg.usage || !msg.model) return entry
const rawContent = msg.content
const contentArr = Array.isArray(rawContent) ? rawContent : []
const toolBlocks = contentArr.filter((b): b is ToolUseBlock => b != null && typeof b === 'object' && b.type === 'tool_use')
const compactContent: ContentBlock[] = toolBlocks.slice(0, MAX_TOOL_BLOCKS).map(tb => {
let input: Record<string, unknown> = {}
if (tb.name === 'Skill') {
const ri = (tb.input ?? {}) as Record<string, unknown>
if (typeof ri['skill'] === 'string') input['skill'] = (ri['skill'] as string).slice(0, 200)
if (typeof ri['name'] === 'string') input['name'] = (ri['name'] as string).slice(0, 200)
} else if (tb.name === 'Read' || tb.name === 'FileReadTool' || EDIT_TOOLS.has(tb.name)) {
const ri = (tb.input ?? {}) as Record<string, unknown>
if (typeof ri['file_path'] === 'string') input['file_path'] = (ri['file_path'] as string).slice(0, BASH_COMMAND_CAP)
} else if (tb.name === 'Agent' || tb.name === 'Task') {
const ri = (tb.input ?? {}) as Record<string, unknown>
if (typeof ri['subagent_type'] === 'string') input['subagent_type'] = (ri['subagent_type'] as string).slice(0, 200)
} else if (BASH_TOOLS.has(tb.name)) {
const ri = (tb.input ?? {}) as Record<string, unknown>
if (typeof ri['command'] === 'string') {
input['command'] = (ri['command'] as string).slice(0, BASH_COMMAND_CAP)
}
}
return { type: 'tool_use' as const, id: tb.id ?? '', name: tb.name, input }
})
const u = msg.usage
const compactUsage: AssistantMessageContent['usage'] = {
input_tokens: u.input_tokens,
output_tokens: u.output_tokens,
}
if (u.cache_creation_input_tokens) compactUsage.cache_creation_input_tokens = u.cache_creation_input_tokens
if (u.cache_creation) {
compactUsage.cache_creation = {
...(u.cache_creation.ephemeral_5m_input_tokens ? { ephemeral_5m_input_tokens: u.cache_creation.ephemeral_5m_input_tokens } : {}),
...(u.cache_creation.ephemeral_1h_input_tokens ? { ephemeral_1h_input_tokens: u.cache_creation.ephemeral_1h_input_tokens } : {}),
}
}
if (u.cache_read_input_tokens) compactUsage.cache_read_input_tokens = u.cache_read_input_tokens
if (u.server_tool_use) {
compactUsage.server_tool_use = {
...(u.server_tool_use.web_search_requests ? { web_search_requests: u.server_tool_use.web_search_requests } : {}),
...(u.server_tool_use.web_fetch_requests ? { web_fetch_requests: u.server_tool_use.web_fetch_requests } : {}),
}
}
if (u.speed) compactUsage.speed = u.speed
entry.message = {
type: 'message',
role: 'assistant',
model: msg.model,
usage: compactUsage,
content: compactContent,
...(msg.id ? { id: msg.id } : {}),
}
return entry
}
function extractToolNames(content: ContentBlock[]): string[] {
return content
.filter((b): b is ToolUseBlock => b.type === 'tool_use')
.map(b => b.name)
}
function extractMcpTools(tools: string[]): string[] {
return tools.filter(t => t.startsWith('mcp__'))
}
function extractSkillNames(content: ContentBlock[]): string[] {
return content
.filter((b): b is ToolUseBlock => b.type === 'tool_use' && b.name === 'Skill')
.map(b => {
const input = (b.input ?? {}) as Record<string, unknown>
const raw = input['skill'] ?? input['name']
return typeof raw === 'string' ? raw.trim() : ''
})
.filter(name => name.length > 0)
}
function extractSubagentTypes(content: ContentBlock[]): string[] {
return content
.filter((b): b is ToolUseBlock => b.type === 'tool_use' && (b.name === 'Agent' || b.name === 'Task'))
.map(b => {
const input = (b.input ?? {}) as Record<string, unknown>
const raw = input['subagent_type']
return typeof raw === 'string' ? raw.trim() : ''
})
.filter(name => name.length > 0)
}
function extractCoreTools(tools: string[]): string[] {
return tools.filter(t => !t.startsWith('mcp__'))
}
function extractBashCommandsFromContent(content: ContentBlock[]): string[] {
return content
.filter((b): b is ToolUseBlock => b.type === 'tool_use' && BASH_TOOLS.has((b as ToolUseBlock).name))
.flatMap(b => {
const command = (b.input as Record<string, unknown>)?.command
return typeof command === 'string' ? extractBashCommands(command) : []
})
}
function getUserMessageText(entry: JournalEntry): string {
if (!entry.message || entry.message.role !== 'user') return ''
const content = entry.message.content
if (typeof content === 'string') return content
if (Array.isArray(content)) {
return content
.filter((b): b is { type: 'text'; text: string } => b.type === 'text')
.map(b => b.text)
.join(' ')
}
return ''
}
function getMessageId(entry: JournalEntry): string | null {
if (entry.type !== 'assistant') return null
const msg = entry.message as AssistantMessageContent | undefined
return msg?.id ?? null
}
function positiveNumber(n: number | undefined): number {
return n !== undefined && Number.isFinite(n) && n > 0 ? n : 0
}
function extractClaudeCacheCreation(usage: AssistantMessageContent['usage']): { totalTokens: number; oneHourTokens: number } {
const legacyTotal = positiveNumber(usage.cache_creation_input_tokens)
const cacheCreation = usage.cache_creation
const fiveMinuteTokens = positiveNumber(cacheCreation?.ephemeral_5m_input_tokens)
const oneHourTokens = positiveNumber(cacheCreation?.ephemeral_1h_input_tokens)
const splitTotal = fiveMinuteTokens + oneHourTokens
if (splitTotal === 0) return { totalTokens: legacyTotal, oneHourTokens: 0 }
// Valid Claude usage reports the legacy total and split total as equal.
// Keep the larger value so malformed partial splits do not drop tokens.
const totalTokens = Math.max(legacyTotal, splitTotal)
return {
totalTokens,
oneHourTokens: Math.min(oneHourTokens, totalTokens),
}
}
function parseApiCall(entry: JournalEntry): ParsedApiCall | null {
if (entry.type !== 'assistant') return null
const msg = entry.message as AssistantMessageContent | undefined
if (!msg?.usage || !msg?.model) return null
const usage = msg.usage
const cacheCreation = extractClaudeCacheCreation(usage)
const tokens: TokenUsage = {
inputTokens: usage.input_tokens ?? 0,
outputTokens: usage.output_tokens ?? 0,
cacheCreationInputTokens: cacheCreation.totalTokens,
cacheReadInputTokens: usage.cache_read_input_tokens ?? 0,
cachedInputTokens: 0,
reasoningTokens: 0,
webSearchRequests: usage.server_tool_use?.web_search_requests ?? 0,
}
const tools = extractToolNames(msg.content ?? [])
const skills = extractSkillNames(msg.content ?? [])
const subagentTypes = extractSubagentTypes(msg.content ?? [])
const costUSD = calculateCost(
msg.model,
tokens.inputTokens,
tokens.outputTokens,
tokens.cacheCreationInputTokens,
tokens.cacheReadInputTokens,
tokens.webSearchRequests,
usage.speed ?? 'standard',
cacheCreation.oneHourTokens,
)
const bashCmds = extractBashCommandsFromContent(msg.content ?? [])
const toolSeq: ToolCall[][] = (msg.content ?? [])
.filter((b): b is ToolUseBlock => b.type === 'tool_use')
.map(b => {
const call: ToolCall = { tool: b.name }
const inp = (b.input ?? {}) as Record<string, unknown>
if (typeof inp['file_path'] === 'string') call.file = inp['file_path'] as string
if (typeof inp['command'] === 'string') call.command = inp['command'] as string
return [call]
})
return {
provider: 'claude',
model: msg.model,
usage: tokens,
costUSD,
tools,
mcpTools: extractMcpTools(tools),
skills,
subagentTypes,
hasAgentSpawn: tools.includes('Agent'),
hasPlanMode: tools.includes('EnterPlanMode'),
speed: usage.speed ?? 'standard',
timestamp: entry.timestamp ?? '',
bashCommands: bashCmds,
deduplicationKey: msg.id ?? `claude:${entry.timestamp}`,
cacheCreationOneHourTokens: cacheCreation.oneHourTokens || undefined,
toolSequence: toolSeq.length > 0 ? toolSeq : undefined,
}
}
function dedupeStreamingMessageIds(entries: JournalEntry[]): JournalEntry[] {
const firstIdxById = new Map<string, number>()
const lastIdxById = new Map<string, number>()
for (let i = 0; i < entries.length; i++) {
const id = getMessageId(entries[i]!)
if (!id) continue
if (!firstIdxById.has(id)) firstIdxById.set(id, i)
lastIdxById.set(id, i)
}
if (lastIdxById.size === 0) return entries
const result: JournalEntry[] = []
for (let i = 0; i < entries.length; i++) {
const id = getMessageId(entries[i]!)
if (id && lastIdxById.get(id) !== i) continue
if (id && firstIdxById.get(id) !== i) {
const firstTs = entries[firstIdxById.get(id)!]!.timestamp
result.push({ ...entries[i]!, timestamp: firstTs ?? entries[i]!.timestamp })
continue
}
result.push(entries[i]!)
}
return result
}
function groupIntoTurns(entries: JournalEntry[], seenMsgIds: Set<string>): ParsedTurn[] {
const turns: ParsedTurn[] = []
let currentUserMessage = ''
let currentCalls: ParsedApiCall[] = []
let currentTimestamp = ''
let currentSessionId = ''
for (const entry of entries) {
if (entry.type === 'user') {
const text = getUserMessageText(entry)
if (text.trim()) {
if (currentCalls.length > 0) {
turns.push({
userMessage: currentUserMessage,
assistantCalls: currentCalls,
timestamp: currentTimestamp,
sessionId: currentSessionId,
})
}
currentUserMessage = text
currentCalls = []
currentTimestamp = entry.timestamp ?? ''
currentSessionId = entry.sessionId ?? ''
}
} else if (entry.type === 'assistant') {
const msgId = getMessageId(entry)
if (msgId && seenMsgIds.has(msgId)) continue
if (msgId) seenMsgIds.add(msgId)
const call = parseApiCall(entry)
if (call) currentCalls.push(call)
}
}
if (currentCalls.length > 0) {
turns.push({
userMessage: currentUserMessage,
assistantCalls: currentCalls,
timestamp: currentTimestamp,
sessionId: currentSessionId,
})
}
return turns
}
/**
* Extract MCP tool inventory observed across a session's JSONL entries.
*
* Claude Code emits `attachment.type === "deferred_tools_delta"` entries whose
* `addedNames` array lists every tool currently available at that turn (built-in
* tools plus all `mcp__<server>__<tool>` names exposed by configured MCP
* servers). Tool inventory can change mid-session if the user reloads MCP
* config, so we union every occurrence rather than trusting only the first.
*
* Built-in tools are filtered out: only `mcp__*` identifiers survive.
*/
// Fully-qualified MCP tool name shape: `mcp__<server>__<tool>`. Both server
// and tool segments must be non-empty. Names like `mcp__server` (no tool
// segment) or `mcp__server__` (trailing empty tool) would silently pollute
// the inventory and break downstream `split('__')` consumers, so they're
// rejected here.
function isMcpToolName(name: string): boolean {
if (!name.startsWith('mcp__')) return false
const rest = name.slice(5) // strip `mcp__`
const sep = rest.indexOf('__')
if (sep <= 0) return false // missing or empty server
if (sep >= rest.length - 2) return false // missing or empty tool
return true
}
export function extractMcpInventory(entries: JournalEntry[]): string[] {
const inventory = new Set<string>()
for (const entry of entries) {
const att = entry['attachment']
if (!att || typeof att !== 'object') continue
const a = att as { type?: unknown; addedNames?: unknown }
if (a.type !== 'deferred_tools_delta') continue
if (!Array.isArray(a.addedNames)) continue
for (const name of a.addedNames) {
if (typeof name !== 'string') continue
if (!isMcpToolName(name)) continue
inventory.add(name)
}
}
if (inventory.size === 0) return []
return Array.from(inventory).sort()
}
function extractCanonicalCwd(entries: JournalEntry[]): string | undefined {
for (const entry of entries) {
if (typeof entry.cwd !== 'string') continue
const cwd = entry.cwd.trim()
if (cwd) return cwd
}
return undefined
}
function buildSessionSummary(
sessionId: string,
project: string,
turns: ClassifiedTurn[],
mcpInventory?: string[],
): SessionSummary {
const modelBreakdown: SessionSummary['modelBreakdown'] = Object.create(null)
const toolBreakdown: SessionSummary['toolBreakdown'] = Object.create(null)
const mcpBreakdown: SessionSummary['mcpBreakdown'] = Object.create(null)
const bashBreakdown: SessionSummary['bashBreakdown'] = Object.create(null)
const categoryBreakdown: SessionSummary['categoryBreakdown'] = Object.create(null)
const skillBreakdown: SessionSummary['skillBreakdown'] = Object.create(null)
const subagentBreakdown: SessionSummary['subagentBreakdown'] = Object.create(null)
let totalCost = 0
let totalInput = 0
let totalOutput = 0
let totalCacheRead = 0
let totalCacheWrite = 0
let apiCalls = 0
let firstTs = ''
let lastTs = ''
for (const turn of turns) {
const turnCost = turn.assistantCalls.reduce((s, c) => s + c.costUSD, 0)
if (!categoryBreakdown[turn.category]) {
categoryBreakdown[turn.category] = { turns: 0, costUSD: 0, retries: 0, editTurns: 0, oneShotTurns: 0 }
}
categoryBreakdown[turn.category].turns++
categoryBreakdown[turn.category].costUSD += turnCost
if (turn.hasEdits) {
categoryBreakdown[turn.category].editTurns++
categoryBreakdown[turn.category].retries += turn.retries
if (turn.retries === 0) categoryBreakdown[turn.category].oneShotTurns++
}
if (turn.subCategory) {
const skillKey = turn.subCategory
if (!skillBreakdown[skillKey]) {
skillBreakdown[skillKey] = { turns: 0, costUSD: 0, editTurns: 0, oneShotTurns: 0 }
}
skillBreakdown[skillKey].turns++
skillBreakdown[skillKey].costUSD += turnCost
if (turn.hasEdits) {
skillBreakdown[skillKey].editTurns++
if (turn.retries === 0) skillBreakdown[skillKey].oneShotTurns++
}
}
for (const call of turn.assistantCalls) {
totalCost += call.costUSD
totalInput += call.usage.inputTokens
totalOutput += call.usage.outputTokens
totalCacheRead += call.usage.cacheReadInputTokens
totalCacheWrite += call.usage.cacheCreationInputTokens
apiCalls++
const modelKey = getShortModelName(call.model)
if (!modelBreakdown[modelKey]) {
modelBreakdown[modelKey] = {
calls: 0,
costUSD: 0,
tokens: { inputTokens: 0, outputTokens: 0, cacheCreationInputTokens: 0, cacheReadInputTokens: 0, cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0 },
}
}
modelBreakdown[modelKey].calls++
modelBreakdown[modelKey].costUSD += call.costUSD
modelBreakdown[modelKey].tokens.inputTokens += call.usage.inputTokens
modelBreakdown[modelKey].tokens.outputTokens += call.usage.outputTokens
modelBreakdown[modelKey].tokens.cacheReadInputTokens += call.usage.cacheReadInputTokens
modelBreakdown[modelKey].tokens.cacheCreationInputTokens += call.usage.cacheCreationInputTokens
for (const tool of extractCoreTools(call.tools)) {
toolBreakdown[tool] = toolBreakdown[tool] ?? { calls: 0 }
toolBreakdown[tool].calls++
}
for (const mcp of call.mcpTools) {
const server = mcp.split('__')[1] ?? mcp
mcpBreakdown[server] = mcpBreakdown[server] ?? { calls: 0 }
mcpBreakdown[server].calls++
}
for (const cmd of call.bashCommands) {
bashBreakdown[cmd] = bashBreakdown[cmd] ?? { calls: 0 }
bashBreakdown[cmd].calls++
}
for (const sat of call.subagentTypes) {
subagentBreakdown[sat] = subagentBreakdown[sat] ?? { calls: 0, costUSD: 0 }
subagentBreakdown[sat].calls++
subagentBreakdown[sat].costUSD += call.costUSD
}
if (!firstTs || call.timestamp < firstTs) firstTs = call.timestamp
if (!lastTs || call.timestamp > lastTs) lastTs = call.timestamp
}
}
return {
sessionId,
project,
firstTimestamp: firstTs || turns[0]?.timestamp || '',
lastTimestamp: lastTs || turns[turns.length - 1]?.timestamp || '',
totalCostUSD: totalCost,
totalInputTokens: totalInput,
totalOutputTokens: totalOutput,
totalCacheReadTokens: totalCacheRead,
totalCacheWriteTokens: totalCacheWrite,
apiCalls,
turns,
modelBreakdown,
toolBreakdown,
mcpBreakdown,
bashBreakdown,
categoryBreakdown,
skillBreakdown,
subagentBreakdown,
...(mcpInventory && mcpInventory.length > 0 ? { mcpInventory } : {}),
}
}
async function parseSessionFile(
filePath: string,
project: string,
seenMsgIds: Set<string>,
dateRange?: DateRange,
): Promise<{ session: SessionSummary; canonicalCwd?: string } | null> {
// Skip files whose mtime is older than the range start. A session file
// can only contain entries up to its last-modified time; if that predates
// the requested range, nothing in this file can match.
if (dateRange) {
try {
const s = await stat(filePath)
if (s.mtimeMs < dateRange.start.getTime()) return null
} catch { /* fall through to normal read; missing stat shouldn't break parsing */ }
}
const entries: JournalEntry[] = []
let hasLines = false
// When a dateRange is given, skip user/assistant lines whose timestamp
// is older than range.start - 24h without calling JSON.parse. Huge lines
// that cannot be skipped are yielded as Buffers and compact-parsed without
// converting the whole line into a V8 string.
const earlySkipThreshold = dateRange
? new Date(dateRange.start.getTime() - 86_400_000).toISOString()
: null
const skipFn = earlySkipThreshold
? (head: string) => shouldSkipLine(head, earlySkipThreshold)
: undefined
for await (const line of readSessionLines(filePath, skipFn, { largeLineAsBuffer: true })) {
hasLines = true
const entry = parseJsonlLine(line)
if (entry) entries.push(compactEntry(entry))
}
if (!hasLines) return null
if (entries.length === 0) return null
const sessionId = basename(filePath, '.jsonl')
const dedupedEntries = dedupeStreamingMessageIds(entries)
let turns = groupIntoTurns(dedupedEntries, seenMsgIds)
if (dateRange) {
// Bucket a turn by the timestamp of its first assistant call (when the cost was
// actually incurred). Filtering entries directly produced orphan assistant calls
// when a user message sat in one day and the response landed in another -- those
// got pushed as turns with empty timestamps, which some code paths counted and
// others dropped, producing inconsistent Today totals.
turns = turns.filter(turn => {
if (turn.assistantCalls.length === 0) return false
const firstCallTs = turn.assistantCalls[0]!.timestamp
if (!firstCallTs) return false
const ts = new Date(firstCallTs)
return ts >= dateRange.start && ts <= dateRange.end
})
if (turns.length === 0) return null
}
const classified = turns.map(classifyTurn)
// Inventory is extracted from the full entry stream, not just the
// turns we kept after date filtering: tool availability is set up
// once at the start of a session (with possible mid-session reloads),
// and we want to reflect what was loaded even if the user only ran
// turns inside a narrow date window.
const mcpInventory = extractMcpInventory(entries)
const canonicalCwd = extractCanonicalCwd(entries)
return {
session: buildSessionSummary(sessionId, project, classified, mcpInventory),
...(canonicalCwd ? { canonicalCwd } : {}),
}
}
async function collectJsonlFiles(dirPath: string): Promise<string[]> {
const files = await readdir(dirPath).catch(() => [])
const jsonlFiles = new Set(files.filter(f => f.endsWith('.jsonl')).map(f => join(dirPath, f)))
const directSubagentsPath = join(dirPath, 'subagents')
const directSubFiles = await readdir(directSubagentsPath).catch(() => [])
for (const sf of directSubFiles) {
if (sf.endsWith('.jsonl')) jsonlFiles.add(join(directSubagentsPath, sf))
}
for (const entry of files) {
if (entry.endsWith('.jsonl')) continue
const subagentsPath = join(dirPath, entry, 'subagents')
const subFiles = await readdir(subagentsPath).catch(() => [])
for (const sf of subFiles) {
if (sf.endsWith('.jsonl')) jsonlFiles.add(join(subagentsPath, sf))
}
}
return [...jsonlFiles]
}
async function scanProjectDirs(
dirs: Array<{ path: string; name: string }>,
seenMsgIds: Set<string>,
diskCache: SessionCache,
dateRange?: DateRange,
): Promise<ProjectSummary[]> {
const section = getOrCreateProviderSection(diskCache, 'claude')
const allDiscoveredFiles = new Set<string>()
type FileInfo = { dirName: string; fp: NonNullable<Awaited<ReturnType<typeof fingerprintFile>>> }
const unchangedFiles: Array<{ filePath: string; dirName: string; cached: CachedFile }> = []
const changedFiles: Array<{ filePath: string; info: FileInfo }> = []
for (const { path: dirPath, name: dirName } of dirs) {
const jsonlFiles = await collectJsonlFiles(dirPath)
for (const filePath of jsonlFiles) {
allDiscoveredFiles.add(filePath)
const fp = await fingerprintFile(filePath)
if (!fp) continue
const action = reconcileFile(fp, section.files[filePath])
if (action.action === 'unchanged') {
unchangedFiles.push({ filePath, dirName, cached: section.files[filePath]! })
} else {
changedFiles.push({ filePath, info: { dirName, fp } })
}
}
}
// Pre-seed dedup set from cached (unchanged) files
for (const { cached } of unchangedFiles) {
for (const turn of cached.turns) {
for (const call of turn.calls) {
seenMsgIds.add(call.deduplicationKey)
}
}
}
for (const { filePath, info } of changedFiles) {
delete section.files[filePath]
const tracker = { lastCompleteLineOffset: 0 }
const entries = await parseClaudeEntries(filePath, tracker)
if (!entries) continue
const turns = groupIntoTurns(dedupeStreamingMessageIds(entries), seenMsgIds)
const cwd = extractCanonicalCwd(entries)
const canonical = (cwd && !isCoworkSession(cwd, filePath)) ? await resolveCanonicalProjectPath(cwd) : undefined
section.files[filePath] = {
fingerprint: info.fp,
lastCompleteLineOffset: tracker.lastCompleteLineOffset,
canonicalCwd: canonical?.path,
canonicalProjectName: canonical?.isWorktree ? projectNameFromPath(canonical.path, info.dirName) : undefined,
mcpInventory: extractMcpInventory(entries),
turns: turns.map(parsedTurnToCachedTurn),
}
;(diskCache as { _dirty?: boolean })._dirty = true
}
if (dirs.length > 0) {
for (const cachedPath of Object.keys(section.files)) {
if (!allDiscoveredFiles.has(cachedPath)) {
delete section.files[cachedPath]
;(diskCache as { _dirty?: boolean })._dirty = true
}
}
}
const projectMap = new Map<string, { project: string; projectPath: string; sessions: SessionSummary[]; dirNames: Set<string> }>()
const allFiles = [
...unchangedFiles.map(f => ({ filePath: f.filePath, dirName: f.dirName })),
...changedFiles.map(f => ({ filePath: f.filePath, dirName: f.info.dirName })),
]
for (const { filePath, dirName } of allFiles) {
const cachedFile = section.files[filePath]
if (!cachedFile || cachedFile.turns.length === 0) continue
let classifiedTurns = cachedFile.turns.map(cachedTurnToClassified)
if (dateRange) {
classifiedTurns = classifiedTurns.filter(turn => {
if (turn.assistantCalls.length === 0) return false
const firstCallTs = turn.assistantCalls[0]!.timestamp
if (!firstCallTs) return false
const ts = new Date(firstCallTs)
return ts >= dateRange.start && ts <= dateRange.end
})
}
if (classifiedTurns.length === 0) continue
const sessionId = basename(filePath, '.jsonl')
const projectPath = cachedFile.canonicalCwd ?? unsanitizePath(dirName)
const projectName = cachedFile.canonicalProjectName ?? dirName
const mcpInv = cachedFile.mcpInventory.length > 0 ? cachedFile.mcpInventory : undefined
const session = buildSessionSummary(sessionId, projectName, classifiedTurns, mcpInv)
if (session.apiCalls > 0) {
const projectKey = cachedFile.canonicalCwd
? normalizeProjectPathKey(cachedFile.canonicalCwd)
: `slug:${dirName}`
const existing = projectMap.get(projectKey)
if (existing) {
existing.sessions.push(session)
existing.dirNames.add(dirName)
} else {
projectMap.set(projectKey, { project: projectName, projectPath, sessions: [session], dirNames: new Set([dirName]) })
}
}
}
// Fold slug-keyed entries into cwd-keyed entries
const cwdKeyByDirName = new Map<string, string>()
for (const [key, entry] of projectMap) {
if (key.startsWith('slug:')) continue
for (const dirName of entry.dirNames) {
if (!cwdKeyByDirName.has(dirName)) cwdKeyByDirName.set(dirName, key)
}
}
for (const [key, entry] of [...projectMap]) {
if (!key.startsWith('slug:')) continue
const cwdKey = cwdKeyByDirName.get(entry.project)
if (!cwdKey) continue
const target = projectMap.get(cwdKey)!
target.sessions.push(...entry.sessions)
projectMap.delete(key)
}
const projects: ProjectSummary[] = []
for (const { project, projectPath, sessions } of projectMap.values()) {
projects.push({
project,
projectPath,
sessions,
totalCostUSD: sessions.reduce((s, sess) => s + sess.totalCostUSD, 0),
totalApiCalls: sessions.reduce((s, sess) => s + sess.apiCalls, 0),
})
}
return projects
}
function providerCallToTurn(call: ParsedProviderCall): ParsedTurn {
const tools = call.tools
const usage: TokenUsage = {
inputTokens: call.inputTokens,
outputTokens: call.outputTokens,
cacheCreationInputTokens: call.cacheCreationInputTokens,
cacheReadInputTokens: call.cacheReadInputTokens,
cachedInputTokens: call.cachedInputTokens,
reasoningTokens: call.reasoningTokens,
webSearchRequests: call.webSearchRequests,
}
const apiCall: ParsedApiCall = {
provider: call.provider,
model: call.model,
usage,
costUSD: call.costUSD,
tools,
mcpTools: extractMcpTools(tools),
skills: [],
subagentTypes: [],
hasAgentSpawn: tools.includes('Agent'),
hasPlanMode: tools.includes('EnterPlanMode'),
speed: call.speed,
timestamp: call.timestamp,
bashCommands: call.bashCommands,
deduplicationKey: call.deduplicationKey,
}
return {
userMessage: call.userMessage,
assistantCalls: [apiCall],
timestamp: call.timestamp,
sessionId: call.sessionId,
}
}
// ── Cache Conversion ───────────────────────────────────────────────────
function providerCallToCachedCall(call: ParsedProviderCall): CachedCall {
return {
provider: call.provider,
model: call.model,
usage: {
inputTokens: call.inputTokens,
outputTokens: call.outputTokens,
cacheCreationInputTokens: call.cacheCreationInputTokens,
cacheReadInputTokens: call.cacheReadInputTokens,
cachedInputTokens: call.cachedInputTokens,
reasoningTokens: call.reasoningTokens,
webSearchRequests: call.webSearchRequests,
cacheCreationOneHourTokens: 0,
},
costUSD: (call.provider === 'mistral-vibe' || call.provider === 'antigravity') ? call.costUSD : undefined,
speed: call.speed,
timestamp: call.timestamp,
tools: call.tools,
bashCommands: call.bashCommands,
skills: [],
subagentTypes: [],
deduplicationKey: call.deduplicationKey,
project: call.project,
projectPath: call.projectPath,
toolSequence: call.toolSequence,
}
}
async function canonicalizeProviderCallProject(call: ParsedProviderCall): Promise<ParsedProviderCall> {
if (!call.projectPath) return call
const canonical = await resolveCanonicalProjectPath(call.projectPath)
if (!canonical.isWorktree) return call
return {
...call,
project: projectNameFromPath(canonical.path, call.project ?? canonical.path),
projectPath: canonical.path,
}
}
function apiCallToCachedCall(call: ParsedApiCall): CachedCall {
return {
provider: call.provider,
model: call.model,
usage: { ...call.usage, cacheCreationOneHourTokens: call.cacheCreationOneHourTokens ?? 0 },
speed: call.speed,
timestamp: call.timestamp,
tools: call.tools,
bashCommands: call.bashCommands,
skills: call.skills,
subagentTypes: call.subagentTypes,
deduplicationKey: call.deduplicationKey,
toolSequence: call.toolSequence,
}
}
function parsedTurnToCachedTurn(turn: ParsedTurn): CachedTurn {
return {
timestamp: turn.timestamp,
sessionId: turn.sessionId,
userMessage: turn.userMessage.slice(0, 2000),
calls: turn.assistantCalls.map(apiCallToCachedCall),
}
}
function providerCallToCachedTurn(call: ParsedProviderCall): CachedTurn {
return {
timestamp: call.timestamp,
sessionId: call.sessionId,
userMessage: call.userMessage.slice(0, 2000),
calls: [providerCallToCachedCall(call)],
}
}
function providerCallsToCachedTurns(calls: ParsedProviderCall[]): CachedTurn[] {
const turns: CachedTurn[] = []
const grouped = new Map<string, CachedTurn>()
for (const call of calls) {
if (!call.turnId) {
turns.push(providerCallToCachedTurn(call))
continue
}
const key = `${call.sessionId}\0${call.turnId}`
let turn = grouped.get(key)
if (!turn) {
turn = {
timestamp: call.timestamp,
sessionId: call.sessionId,
userMessage: call.userMessage.slice(0, 2000),
calls: [],
}
grouped.set(key, turn)
turns.push(turn)
}
turn.calls.push(providerCallToCachedCall(call))
}
return turns
}
function cachedCallToApiCall(call: CachedCall): ParsedApiCall {
const u = call.usage
const outputForCost = call.provider === 'claude'
? u.outputTokens
: u.outputTokens + u.reasoningTokens
const costUSD = calculateCost(
call.model, u.inputTokens, outputForCost,
u.cacheCreationInputTokens, u.cacheReadInputTokens,
u.webSearchRequests, call.speed, u.cacheCreationOneHourTokens,
)
return {
provider: call.provider,
model: call.model,
usage: {
inputTokens: u.inputTokens,
outputTokens: u.outputTokens,
cacheCreationInputTokens: u.cacheCreationInputTokens,
cacheReadInputTokens: u.cacheReadInputTokens,
cachedInputTokens: u.cachedInputTokens,
reasoningTokens: u.reasoningTokens,
webSearchRequests: u.webSearchRequests,
},
costUSD: call.costUSD ?? costUSD,
tools: call.tools,
mcpTools: extractMcpTools(call.tools),
skills: call.skills,
subagentTypes: call.subagentTypes ?? [],
hasAgentSpawn: call.tools.includes('Agent'),
hasPlanMode: call.tools.includes('EnterPlanMode'),
speed: call.speed,
timestamp: call.timestamp,
bashCommands: call.bashCommands,
deduplicationKey: call.deduplicationKey,
cacheCreationOneHourTokens: u.cacheCreationOneHourTokens || undefined,
toolSequence: call.toolSequence,
}
}
function cachedTurnToClassified(turn: CachedTurn): ClassifiedTurn {
const parsed: ParsedTurn = {
userMessage: turn.userMessage,
assistantCalls: turn.calls.map(cachedCallToApiCall),
timestamp: turn.timestamp,
sessionId: turn.sessionId,
}
return classifyTurn(parsed)
}
// ── Cache-Aware Parsing Helpers ────────────────────────────────────────
async function parseClaudeEntries(
filePath: string,
tracker: { lastCompleteLineOffset: number },
): Promise<JournalEntry[] | null> {
const entries: JournalEntry[] = []
let hasLines = false
for await (const line of readSessionLines(filePath, undefined, {
largeLineAsBuffer: true,
byteOffsetTracker: tracker,
})) {
hasLines = true
const entry = parseJsonlLine(line)
if (entry) entries.push(compactEntry(entry))
}
if (!hasLines || entries.length === 0) return null
return entries
}
function getOrCreateProviderSection(cache: SessionCache, provider: string): ProviderSection {
const envFp = computeEnvFingerprint(provider)
const existing = cache.providers[provider]
if (existing && existing.envFingerprint === envFp) return existing
const section = { envFingerprint: envFp, files: {} }
cache.providers[provider] = section
return section
}
function cachedFileNeedsProviderReparse(providerName: string, sourcePath: string, cached: CachedFile): boolean {
// Antigravity data comes from the live server, not from the conversation file.
// A 0-turn cache entry may just mean the server was unavailable last run.
if (providerName === 'antigravity') return shouldReparseAntigravitySource(sourcePath, cached.turns.length)
if (providerName !== 'gemini') return false
return cached.turns.some(turn =>
turn.calls.some(call => call.deduplicationKey === `gemini:${turn.sessionId}`),
)
}
const warnedProviderReadFailures = new Set<string>()
function warnProviderReadFailureOnce(providerName: string, err: unknown): void {
const key = `${providerName}:sqlite-busy`
if (warnedProviderReadFailures.has(key)) return
warnedProviderReadFailures.add(key)
if (isSqliteBusyError(err)) {
process.stderr.write(
`codeburn: skipped ${providerName} data because its SQLite database is temporarily locked; will retry on the next refresh.\n`
)
}
}
async function parseProviderSources(
providerName: string,
sources: Array<{ path: string; project: string }>,
seenKeys: Set<string>,
diskCache: SessionCache,
dateRange?: DateRange,
): Promise<ProjectSummary[]> {
const provider = await getProvider(providerName)
if (!provider) return []
const section = getOrCreateProviderSection(diskCache, providerName)
const allDiscoveredFiles = new Set<string>()
type SourceInfo = { source: { path: string; project: string }; fp: NonNullable<Awaited<ReturnType<typeof fingerprintFile>>> }
const unchangedSources: Array<{ source: { path: string; project: string }; cached: CachedFile }> = []
const changedSources: SourceInfo[] = []
for (const source of sources) {
allDiscoveredFiles.add(source.path)
const fp = await fingerprintFile(source.path)
if (!fp) continue
const cached = section.files[source.path]
const action = reconcileFile(fp, cached)
if (action.action === 'unchanged' && cached && !cachedFileNeedsProviderReparse(providerName, source.path, cached)) {
unchangedSources.push({ source, cached })
} else {
changedSources.push({ source, fp })
}
}
// Parser dedup: cross-provider keys + cached file keys.
// Separate from seenKeys so parsing doesn't suppress query-time output.
const parserDedup = new Set(seenKeys)
for (const { cached } of unchangedSources) {
for (const turn of cached.turns) {
for (const call of turn.calls) {
parserDedup.add(call.deduplicationKey)
}
}
}
// Parse changed files, update cache
let didParse = false
try {
for (const { source, fp } of changedSources) {
if (dateRange) {
if (fp.mtimeMs < dateRange.start.getTime()) continue
}
// Clear stale entry before parse — if parse fails, file is excluded
delete section.files[source.path]
const parser = provider.createSessionParser(
{ path: source.path, project: source.project, provider: providerName },
parserDedup,
)
try {
const providerCalls: ParsedProviderCall[] = []
for await (const call of parser.parse()) {
providerCalls.push(call)
}
const canonicalCalls = await Promise.all(providerCalls.map(canonicalizeProviderCallProject))
const turns = providerCallsToCachedTurns(canonicalCalls)
section.files[source.path] = { fingerprint: fp, mcpInventory: [], turns }
didParse = true
;(diskCache as { _dirty?: boolean })._dirty = true
} catch (err) {
if (isSqliteBusyError(err)) {
warnProviderReadFailureOnce(providerName, err)
continue
}
throw err
}
}
} finally {
if (didParse && providerName === 'codex') await flushCodexCache()
if (didParse && providerName === 'antigravity') {
const liveIds = new Set(sources.map(s => antigravityCascadeIdFromPath(s.path)))
await flushAntigravityCache(liveIds)
}
}
if (sources.length > 0) {
for (const cachedPath of Object.keys(section.files)) {
if (!allDiscoveredFiles.has(cachedPath)) {
delete section.files[cachedPath]
;(diskCache as { _dirty?: boolean })._dirty = true
}
}
}
// Query-time: derive SessionSummary from all cached turns.
// Uses seenKeys (shared across providers) for cross-provider dedup.
const sessionMap = new Map<string, { project: string; projectPath?: string; turns: ClassifiedTurn[] }>()
for (const source of sources) {
const cachedFile = section.files[source.path]
if (!cachedFile) continue
for (const turn of cachedFile.turns) {
const hasDup = turn.calls.some(c => seenKeys.has(c.deduplicationKey))
if (hasDup) continue
for (const c of turn.calls) seenKeys.add(c.deduplicationKey)
if (dateRange) {
const callTs = turn.calls[0]?.timestamp
if (!callTs) continue
const ts = new Date(callTs)
if (ts < dateRange.start || ts > dateRange.end) continue
}
const classified = cachedTurnToClassified(turn)
const project = turn.calls[0]?.project ?? source.project
const key = `${providerName}:${turn.sessionId}:${project}`
const existing = sessionMap.get(key)
if (existing) {
existing.turns.push(classified)
if (!existing.projectPath && turn.calls[0]?.projectPath) {
existing.projectPath = turn.calls[0]!.projectPath
}
} else {
sessionMap.set(key, { project, projectPath: turn.calls[0]?.projectPath, turns: [classified] })
}
}
}
const projectMap = new Map<string, { projectPath?: string; sessions: SessionSummary[] }>()
for (const [key, { project, projectPath, turns }] of sessionMap) {
const sessionId = key.split(':')[1] ?? key
const session = buildSessionSummary(sessionId, project, turns)
if (session.apiCalls > 0) {
const existing = projectMap.get(project)
if (existing) {
existing.sessions.push(session)
if (!existing.projectPath && projectPath) existing.projectPath = projectPath
} else {
projectMap.set(project, { projectPath, sessions: [session] })
}
}
}
const projects: ProjectSummary[] = []
for (const [dirName, { projectPath, sessions }] of projectMap) {
projects.push({
project: dirName,
projectPath: projectPath ?? unsanitizePath(dirName),
sessions,
totalCostUSD: sessions.reduce((s, sess) => s + sess.totalCostUSD, 0),
totalApiCalls: sessions.reduce((s, sess) => s + sess.apiCalls, 0),
})
}
return projects
}
const CACHE_TTL_MS = 180_000
const MAX_CACHE_ENTRIES = 10
const sessionCache = new Map<string, { data: ProjectSummary[]; ts: number }>()
function cacheKey(dateRange?: DateRange, providerFilter?: string): string {
const s = dateRange ? `${dateRange.start.getTime()}:${dateRange.end.getTime()}` : 'none'
// Include the Claude config-dir env so a config change in a long-lived
// process (menubar / GNOME extension / test workers) does not return
// stale data keyed under a previous configuration.
const claudeEnv = (process.env['CLAUDE_CONFIG_DIRS'] ?? '') + '|' + (process.env['CLAUDE_CONFIG_DIR'] ?? '')
return `${s}:${providerFilter ?? 'all'}:${claudeEnv}`
}
export function clearSessionCache(): void {
sessionCache.clear()
}
function cachePut(key: string, data: ProjectSummary[]) {
const now = Date.now()
for (const [k, v] of sessionCache) {
if (now - v.ts > CACHE_TTL_MS) sessionCache.delete(k)
}
if (sessionCache.size >= MAX_CACHE_ENTRIES) {
const oldest = [...sessionCache.entries()].sort((a, b) => a[1].ts - b[1].ts)[0]
if (oldest) sessionCache.delete(oldest[0])
}
sessionCache.set(key, { data, ts: now })
}
export function filterProjectsByName(
projects: ProjectSummary[],
include?: string[],
exclude?: string[],
): ProjectSummary[] {
let result = projects
if (include && include.length > 0) {
const patterns = include.map(s => s.toLowerCase())
result = result.filter(p => {
const name = p.project.toLowerCase()
const path = p.projectPath.toLowerCase()
return patterns.some(pat => name.includes(pat) || path.includes(pat))
})
}
if (exclude && exclude.length > 0) {
const patterns = exclude.map(s => s.toLowerCase())
result = result.filter(p => {
const name = p.project.toLowerCase()
const path = p.projectPath.toLowerCase()
return !patterns.some(pat => name.includes(pat) || path.includes(pat))
})
}
return result
}
function turnIsInDateRange(turn: ClassifiedTurn, dateRange: DateRange): boolean {
if (turn.assistantCalls.length === 0) return false
const firstCallTs = turn.assistantCalls[0]!.timestamp
if (!firstCallTs) return false
const ts = new Date(firstCallTs)
return ts >= dateRange.start && ts <= dateRange.end
}
function turnDayString(turn: ClassifiedTurn): string | null {
if (turn.assistantCalls.length === 0) return null
const ts = turn.assistantCalls[0]!.timestamp
if (!ts) return null
const d = new Date(ts)
const y = d.getFullYear()
const m = String(d.getMonth() + 1).padStart(2, '0')
const day = String(d.getDate()).padStart(2, '0')
return `${y}-${m}-${day}`
}
export function filterProjectsByDays(projects: ProjectSummary[], days: Set<string>): ProjectSummary[] {
const filtered: ProjectSummary[] = []
for (const project of projects) {
const sessions: SessionSummary[] = []
for (const session of project.sessions) {
const turns = session.turns.filter(turn => {
const ds = turnDayString(turn)
return ds !== null && days.has(ds)
})
if (turns.length === 0) continue
sessions.push(buildSessionSummary(session.sessionId, session.project, turns, session.mcpInventory))
}
if (sessions.length === 0) continue
filtered.push({
project: project.project,
projectPath: project.projectPath,
sessions,
totalCostUSD: sessions.reduce((s, sess) => s + sess.totalCostUSD, 0),
totalApiCalls: sessions.reduce((s, sess) => s + sess.apiCalls, 0),
})
}
return filtered.sort((a, b) => b.totalCostUSD - a.totalCostUSD)
}
export function filterProjectsByDateRange(projects: ProjectSummary[], dateRange: DateRange): ProjectSummary[] {
const filtered: ProjectSummary[] = []
for (const project of projects) {
const sessions: SessionSummary[] = []
for (const session of project.sessions) {
const turns = session.turns.filter(turn => turnIsInDateRange(turn, dateRange))
if (turns.length === 0) continue
sessions.push(buildSessionSummary(session.sessionId, session.project, turns, session.mcpInventory))
}
if (sessions.length === 0) continue
filtered.push({
project: project.project,
projectPath: project.projectPath,
sessions,
totalCostUSD: sessions.reduce((s, sess) => s + sess.totalCostUSD, 0),
totalApiCalls: sessions.reduce((s, sess) => s + sess.apiCalls, 0),
})
}
return filtered.sort((a, b) => b.totalCostUSD - a.totalCostUSD)
}
export async function parseAllSessions(dateRange?: DateRange, providerFilter?: string): Promise<ProjectSummary[]> {
const key = cacheKey(dateRange, providerFilter)
const cached = sessionCache.get(key)
if (cached && Date.now() - cached.ts < CACHE_TTL_MS) return cached.data
const diskCache = await loadCache()
await cleanupOrphanedTempFiles()
const seenMsgIds = new Set<string>()
const seenKeys = new Set<string>()
const allSources = await discoverAllSessions(providerFilter)
const claudeSources = allSources.filter(s => s.provider === 'claude')
const nonClaudeSources = allSources.filter(s => s.provider !== 'claude')
const claudeDirs = claudeSources.map(s => ({ path: s.path, name: s.project }))
const claudeProjects = await scanProjectDirs(claudeDirs, seenMsgIds, diskCache, dateRange)
const providerGroups = new Map<string, Array<{ path: string; project: string }>>()
for (const source of nonClaudeSources) {
const existing = providerGroups.get(source.provider) ?? []
existing.push({ path: source.path, project: source.project })
providerGroups.set(source.provider, existing)
}
const otherProjects: ProjectSummary[] = []
for (const [providerName, sources] of providerGroups) {
const projects = await parseProviderSources(providerName, sources, seenKeys, diskCache, dateRange)
otherProjects.push(...projects)
}
if ((diskCache as { _dirty?: boolean })._dirty) {
try { await saveCache(diskCache) } catch {}
}
// Merge across providers by normalised project path so the same repository
// is not double-counted when it was worked on with more than one tool
// (e.g. both Claude Code and Codex). Two sub-problems:
//
// 1. Codex's sanitizeProject strips the leading '/' from cwds, so
// "Users/carlo/foo" and "/Users/carlo/foo" must compare equal. We
// normalise by stripping leading slashes before keying.
//
// 2. Codex worktrees (e.g. ~/.codex/worktrees/e55f/Repo) are not resolved
// to their main-repo path by canonicalizeProviderCallProject because that
// function only operates on call.projectPath, which Codex doesn't set.
// Resolve at the ProjectSummary level here: prepend '/' if needed to get
// an absolute path, then run the same worktree-detection logic.
const resolvedOtherProjects = await Promise.all(otherProjects.map(async p => {
const absPath = p.projectPath.startsWith('/') || p.projectPath.startsWith('\\')
? p.projectPath
: '/' + p.projectPath
const canonical = await resolveCanonicalProjectPath(absPath)
// Skip if path is unchanged: same location, not a worktree, not a subdir
if (!canonical.isWorktree && canonical.path === absPath.replace(/[/\\]+$/, '')) return p
return { ...p, project: projectNameFromPath(canonical.path, p.project), projectPath: canonical.path }
}))
const crossProviderKey = (p: ProjectSummary): string => {
const path = p.projectPath.replace(/\\/g, '/').replace(/^\/+/, '').toLowerCase()
return path.includes('/') ? path : p.project.toLowerCase()
}
const mergedMap = new Map<string, ProjectSummary>()
for (const p of [...claudeProjects, ...resolvedOtherProjects]) {
const key = crossProviderKey(p)
const existing = mergedMap.get(key)
if (existing) {
existing.sessions.push(...p.sessions)
existing.totalCostUSD += p.totalCostUSD
existing.totalApiCalls += p.totalApiCalls
} else {
mergedMap.set(key, { ...p })
}
}
const result = Array.from(mergedMap.values()).sort((a, b) => b.totalCostUSD - a.totalCostUSD)
cachePut(key, result)
return result
}