mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-08-21 14:34:32 +00:00
Merge pull request #994 from avs-io/codex/issue-974-sidechains
fix(optimize): exclude sidechains from behavioral signals
This commit is contained in:
Resham Joshi
GitHub
commit
215f5011a1
16 changed files with 586 additions and 31 deletions
|
|
@ -29,6 +29,7 @@
|
|||
|
||||
### Fixed
|
||||
- **Old durable sources remain visible while they still exist.** The 90-day session-cache age-out now applies only after a durable source disappears from discovery, so an unchanged older Copilot source keeps reporting usage and reuses its persisted fingerprint instead of being reparsed and immediately discarded. (#987) On long-lived machines this makes previously dropped history reappear, so lifetime totals can jump once after upgrading.
|
||||
- **`optimize` no longer treats subagent transcripts as your sessions.** Claude Code writes each subagent's transcript to its own `subagents/agent-*.jsonl` file with `isSidechain: true` on every entry, and optimize counted each one as a user-started session. That inflated the session count in the header and fed the session-level detectors a population that fails their tests by construction: a sidechain is handed a large context and returns a short answer (context-heavy), and it never commits or opens a PR because its parent does (low-worth). Excluded from sidechains now: the header session count, the `low-worth-sessions`, `context-bloat`, `cost-outliers` and `capability-reliability` detectors, the coaching notes, the file-churn table, the median time-to-first-edit, the worst one-shot category, and the model-default recommendation - plus `duplicate-reads`, because a subagent starts on a fresh context and re-reading what its parent read is a necessary read, not a repeat. Everything else keeps the full population: `build-folder-reads` and `read-edit-ratio` still count calls made inside a sidechain, since reading `node_modules` or editing without reading is the same waste whoever does it and the `CLAUDE.md` rule they suggest binds subagents too, and so do the MCP, cache-bloat, ghost-command and configuration-overhead findings. Classification is sticky across the whole file, so calls that appear before the first marked entry are reclassified too, and `isSidechain` now survives the compact parser's 32 KB large-line path and warm-cache range rebuilds. Nothing is deleted from spend: sidechain tokens, calls and cost stay in every total and in `status`, and the optimize result cache keys on sidechain identity so a run cannot be served a pre-fix result. Absent markers still read as user-started, so no cache re-parse is needed. (#974)
|
||||
- **`optimize` no longer offers `claude mcp remove` for claude.ai connectors, and its MCP schema-cost estimate is per session.** A `claude_ai_*` namespace that no readable local MCP config claims is a claude.ai connector, managed through `/mcp` or claude.ai Settings rather than as a local MCP server (a local server that carries the prefix keeps its removal command and gains a same-name connector note); low-coverage findings now render them as a manual follow-up and build `--apply` plans only for exact local server names found in readable MCP config, so mixed findings remove only the local subset and the "apply-able" subtotal counts only that subset. The same change replaces the old global schema-cost cap with per-session, per-server proportional attribution — a more accurate model that lowers `mcp-low-coverage` estimates for everyone, connectors or not (on a large corpus roughly by half). (#975, #991)
|
||||
- **Bash command splitting was quadratic on long whitespace-heavy commands.** The separator regex retried its leading `\s*` from every offset; matching the separator alone and widening over whitespace by hand makes cold parse ~24% and warm ~40% faster on large corpora, output unchanged.
|
||||
- **Cold parse no longer retains full message bodies through cached previews.** `flatSlice` skipped its Buffer round-trip for strings already within the bound, but provider adapters pre-truncate user-message previews with `.slice(0, 500)` before the cache-site call — those pre-sliced views are still V8 SlicedStrings pinning their large parent, so the retention that OOM'd cold parses of large histories survived. The round-trip now always runs.
|
||||
|
|
|
|||
|
|
@ -157,6 +157,14 @@ codeburn optimize --format json # setup health + findings as JSON
|
|||
|
||||
`codeburn optimize` scans your sessions and your `~/.claude/` setup for waste patterns:
|
||||
|
||||
For Claude Code, the optimize session count, the per-session findings, coaching,
|
||||
and model-default recommendations use user-started (main) sessions. Subagent
|
||||
sidechain transcripts are excluded from that population because their delegated
|
||||
context and delivery behavior are structurally different, and so is the re-read
|
||||
finding, since a subagent starts on a fresh context. Findings about how Claude
|
||||
uses tools (junk reads, read:edit ratio) and every spend, MCP, and
|
||||
configuration-overhead finding keep counting them.
|
||||
|
||||
- Files Claude re-reads across sessions (same content, same context, over and over)
|
||||
- Low Read:Edit ratio (editing without reading leads to retries and wasted tokens)
|
||||
- Wasted bash output (uncapped `BASH_MAX_OUTPUT_LENGTH`, trailing noise)
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ import { readFile } from 'node:fs/promises'
|
|||
import { join } from 'node:path'
|
||||
|
||||
import { aggregateModelStats, type ModelStats } from '../compare-stats.js'
|
||||
import { withUserStartedSessions } from '../session-population.js'
|
||||
import type { ProjectSummary } from '../types.js'
|
||||
import { sha256File } from './backup.js'
|
||||
import type { ActionPlan } from './types.js'
|
||||
|
|
@ -73,15 +74,16 @@ function isDebuggingHeavy(project: ProjectSummary): boolean {
|
|||
}
|
||||
|
||||
export function recommendModelDefault(project: ProjectSummary, opts: { now?: Date } = {}): ModelDefaultRecommendation | null {
|
||||
const behavioralProject = withUserStartedSessions(project)
|
||||
const now = opts.now ?? new Date()
|
||||
const stats = aggregateModelStats([project])
|
||||
const stats = aggregateModelStats([behavioralProject])
|
||||
.filter(s => s.model !== '<synthetic>' && s.editTurns >= MIN_EDIT_TURNS)
|
||||
.sort((a, b) => b.editTurns - a.editTurns || b.editCost - a.editCost)
|
||||
|
||||
const current = stats[0]
|
||||
if (!current) return null
|
||||
|
||||
const providers = providerByModel(project)
|
||||
const providers = providerByModel(behavioralProject)
|
||||
const provider = providers.get(current.model)
|
||||
if (!provider || !isRecent(current.lastSeen, now)) return null
|
||||
|
||||
|
|
@ -89,7 +91,7 @@ export function recommendModelDefault(project: ProjectSummary, opts: { now?: Dat
|
|||
const currentCost = costPerEdit(current)
|
||||
if (!Number.isFinite(currentCost) || currentCost <= 0) return null
|
||||
|
||||
const debuggingHeavy = isDebuggingHeavy(project)
|
||||
const debuggingHeavy = isDebuggingHeavy(behavioralProject)
|
||||
const tolerance = debuggingHeavy ? 0 : ONE_SHOT_TOLERANCE
|
||||
|
||||
const candidates = stats
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
import chalk from 'chalk'
|
||||
import stripAnsi from 'strip-ansi'
|
||||
import { isReadShapedBashCommand } from './bash-utils.js'
|
||||
import { createHash } from 'crypto'
|
||||
import { readdir, stat } from 'fs/promises'
|
||||
import { existsSync, statSync } from 'fs'
|
||||
import { basename, join } from 'path'
|
||||
import { homedir } from 'os'
|
||||
|
||||
import { isReadShapedBashCommand } from './bash-utils.js'
|
||||
import { readSessionLines, readSessionFileSync } from './fs-utils.js'
|
||||
import { discoverAllSessions } from './providers/index.js'
|
||||
import { parseJsonlLine, shouldSkipLine } from './parser.js'
|
||||
|
|
@ -15,6 +15,7 @@ import { formatCost } from './currency.js'
|
|||
import { formatTokens } from './format.js'
|
||||
import { recommendModelDefault, type ModelDefaultRecommendation } from './act/model-defaults.js'
|
||||
import { appliedFixGlyph, formatAppliedFix, type AppliedFix } from './act/types.js'
|
||||
import { isUserStartedSession, userStartedProjects } from './session-population.js'
|
||||
import { aggregateFileChurn, buildCoachingNotes, scanUserCorrections, medianTimeToFirstEditMs, worstOneShotCategory, type ReworkedFile } from './workflow-insights.js'
|
||||
|
||||
// ============================================================================
|
||||
|
|
@ -542,6 +543,7 @@ export type ToolCall = {
|
|||
sessionId: string
|
||||
project: string
|
||||
recent?: boolean
|
||||
isSidechain?: boolean
|
||||
}
|
||||
|
||||
export type ApiCallMeta = {
|
||||
|
|
@ -726,6 +728,7 @@ export async function scanJsonlFile(
|
|||
const openers: SessionOpener[] = []
|
||||
const sessionId = basename(filePath, '.jsonl')
|
||||
let lastVersion = ''
|
||||
let fileIsSidechain = false
|
||||
// The opening block is the first user message carrying text; anything
|
||||
// later in the session is not what the user opens with.
|
||||
let sawUserText = false
|
||||
|
|
@ -744,6 +747,11 @@ export async function scanJsonlFile(
|
|||
if (!parsed) continue
|
||||
const entry = parsed as Record<string, unknown>
|
||||
|
||||
if (entry.isSidechain === true && !fileIsSidechain) {
|
||||
fileIsSidechain = true
|
||||
for (const call of calls) call.isSidechain = true
|
||||
}
|
||||
|
||||
if (entry.version && typeof entry.version === 'string') lastVersion = entry.version
|
||||
|
||||
const ts = typeof entry.timestamp === 'string' ? entry.timestamp : undefined
|
||||
|
|
@ -804,6 +812,7 @@ export async function scanJsonlFile(
|
|||
sessionId,
|
||||
project,
|
||||
recent,
|
||||
isSidechain: fileIsSidechain,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -999,6 +1008,11 @@ export function detectJunkReads(calls: ToolCall[], dateRange?: DateRange): Waste
|
|||
}
|
||||
|
||||
export function detectDuplicateReads(calls: ToolCall[], dateRange?: DateRange): WasteFinding | null {
|
||||
// A sidechain re-reading what its parent read is not a repeat: a subagent
|
||||
// starts on a fresh context and has to read it. Junk reads and the
|
||||
// read:edit ratio keep the full call population - that waste is waste
|
||||
// whoever does it, and the CLAUDE.md rule they suggest binds subagents too.
|
||||
calls = calls.filter(call => call.isSidechain !== true)
|
||||
const sessionFiles = new Map<string, Map<string, { count: number; recent: number }>>()
|
||||
|
||||
for (const call of calls) {
|
||||
|
|
@ -1923,6 +1937,7 @@ function findCapabilityReliabilityCandidates(projects: ProjectSummary[]): Capabi
|
|||
}
|
||||
|
||||
export function detectCapabilityReliability(projects: ProjectSummary[]): WasteFinding | null {
|
||||
projects = userStartedProjects(projects)
|
||||
const candidates = findCapabilityReliabilityCandidates(projects)
|
||||
if (candidates.length === 0) return null
|
||||
|
||||
|
|
@ -2947,6 +2962,22 @@ function sessionTokenTotal(session: ProjectSummary['sessions'][number]): number
|
|||
+ session.totalCacheWriteTokens
|
||||
}
|
||||
|
||||
// Sidechain transcripts are real usage, so they stay in project totals and in
|
||||
// token/cost calibration. They are not user-started sessions, however, and
|
||||
// should never enter optimize heuristics whose unit is a human work session.
|
||||
// Keep that distinction local to optimize instead of deleting sidechains from
|
||||
// ProjectSummary, which would under-report the work delegated to subagents.
|
||||
function isOptimizeSession(session: ProjectSummary['sessions'][number]): boolean {
|
||||
return isUserStartedSession(session)
|
||||
}
|
||||
|
||||
function optimizeSessionCount(projects: ProjectSummary[]): number {
|
||||
return projects.reduce(
|
||||
(total, project) => total + project.sessions.filter(isOptimizeSession).length,
|
||||
0,
|
||||
)
|
||||
}
|
||||
|
||||
function sessionEffectiveContextTokens(session: ProjectSummary['sessions'][number]): number {
|
||||
return session.totalInputTokens
|
||||
+ session.totalCacheReadTokens * CACHE_READ_DISCOUNT
|
||||
|
|
@ -3055,6 +3086,7 @@ export function findLowWorthCandidates(projects: ProjectSummary[]): LowWorthCand
|
|||
|
||||
for (const project of projects) {
|
||||
for (const session of project.sessions) {
|
||||
if (!isOptimizeSession(session)) continue
|
||||
if (session.totalCostUSD < WORTH_IT_MIN_COST_USD) continue
|
||||
if (sessionDeliveryCommand(session)) continue
|
||||
|
||||
|
|
@ -3155,7 +3187,7 @@ export function findContextBloatCandidates(projects: ProjectSummary[]): ContextB
|
|||
const candidates: ContextBloatCandidate[] = []
|
||||
|
||||
for (const project of projects) {
|
||||
const sessions = [...project.sessions].sort((a, b) =>
|
||||
const sessions = project.sessions.filter(isOptimizeSession).sort((a, b) =>
|
||||
new Date(a.firstTimestamp).getTime() - new Date(b.firstTimestamp).getTime()
|
||||
)
|
||||
let previousInputTokens: number | null = null
|
||||
|
|
@ -3273,7 +3305,7 @@ export function detectSessionOutliers(projects: ProjectSummary[], excludedSessio
|
|||
let usedEstimatedCosts = false
|
||||
|
||||
for (const project of projects) {
|
||||
const costed = project.sessions.filter(s => s.totalCostUSD > 0)
|
||||
const costed = project.sessions.filter(s => isOptimizeSession(s) && s.totalCostUSD > 0)
|
||||
const exact = costed.filter(s => (s.totalEstimatedCostUSD ?? 0) === 0)
|
||||
const sessions = exact.length >= MIN_SESSIONS_FOR_OUTLIER ? exact : costed
|
||||
const fellBack = sessions.length > exact.length
|
||||
|
|
@ -3339,7 +3371,7 @@ function findYoungProjectFirstSessionIds(projects: ProjectSummary[]): Set<string
|
|||
const firstSessionIds = new Set<string>()
|
||||
|
||||
for (const project of projects) {
|
||||
const costed = project.sessions.filter(s => s.totalCostUSD > 0)
|
||||
const costed = project.sessions.filter(s => isOptimizeSession(s) && s.totalCostUSD > 0)
|
||||
if (costed.length >= YOUNG_PROJECT_SESSION_LIMIT) continue
|
||||
|
||||
let firstSession: ProjectSummary['sessions'][number] | null = null
|
||||
|
|
@ -3458,15 +3490,25 @@ export function cacheKey(projects: ProjectSummary[], dateRange: DateRange | unde
|
|||
// stale findings when cost/tokens moved (e.g. a re-price) while call count
|
||||
// held - reachable in the long-lived menubar process within the 60s TTL.
|
||||
// Cost is scaled to whole micro-dollars so float jitter cannot thrash the key.
|
||||
let calls = 0, cost = 0, savings = 0, proxied = 0
|
||||
let calls = 0, cost = 0, savings = 0, proxied = 0, sessions = 0, sidechains = 0
|
||||
const sidechainIdentities: string[] = []
|
||||
for (const p of projects) {
|
||||
calls += p.totalApiCalls
|
||||
cost += p.totalCostUSD
|
||||
savings += p.totalSavingsUSD
|
||||
proxied += p.totalProxiedCostUSD
|
||||
sessions += p.sessions.length
|
||||
for (const session of p.sessions) {
|
||||
if (session.isSidechain !== true) continue
|
||||
sidechains++
|
||||
sidechainIdentities.push(`${p.projectPath}\0${session.sessionId}`)
|
||||
}
|
||||
}
|
||||
const sidechainDigest = createHash('sha256')
|
||||
.update(sidechainIdentities.sort().join('\0'))
|
||||
.digest('base64url')
|
||||
// Costs scaled to whole micro-dollars so float jitter cannot thrash the key.
|
||||
const fingerprint = `${projects.length}:${calls}:${Math.round(cost * 1e6)}:${Math.round(savings * 1e6)}:${Math.round(proxied * 1e6)}`
|
||||
const fingerprint = `${projects.length}:${sessions}:${sidechains}:${sidechainDigest}:${calls}:${Math.round(cost * 1e6)}:${Math.round(savings * 1e6)}:${Math.round(proxied * 1e6)}`
|
||||
// The provider decides whether the Claude session scan runs at all, so two
|
||||
// filters that happen to share a project fingerprint must not share a result.
|
||||
return `${provider ?? 'all'}:${dr}:${fingerprint}`
|
||||
|
|
@ -3486,6 +3528,7 @@ export async function scanAndDetect(
|
|||
if (cached && Date.now() - cached.ts < RESULT_CACHE_TTL_MS) return cached.data
|
||||
|
||||
const costRate = computeInputCostRate(projects)
|
||||
const behavioralProjects = userStartedProjects(projects)
|
||||
const scanCoversClaude = providerCoversClaude(provider)
|
||||
const { toolCalls, projectCwds, apiCalls, userMessages, openers } = await scanSessions(dateRange, provider)
|
||||
const mcpCoverage = aggregateMcpCoverage(projects)
|
||||
|
|
@ -3494,13 +3537,13 @@ export async function scanAndDetect(
|
|||
// Priority order for the per-session findings: low-worth → context-bloat →
|
||||
// outliers. Each later detector excludes sessions already named by an
|
||||
// earlier one so a single session is not listed in three findings.
|
||||
const lowWorthSessionIds = new Set(findLowWorthCandidates(projects).map(c => c.sessionId))
|
||||
const lowWorthSessionIds = new Set(findLowWorthCandidates(behavioralProjects).map(c => c.sessionId))
|
||||
const contextBloatVisibleIds = new Set(
|
||||
findContextBloatCandidates(projects)
|
||||
findContextBloatCandidates(behavioralProjects)
|
||||
.filter(c => !lowWorthSessionIds.has(c.sessionId))
|
||||
.map(c => c.sessionId),
|
||||
)
|
||||
const firstSessionIds = findYoungProjectFirstSessionIds(projects)
|
||||
const firstSessionIds = findYoungProjectFirstSessionIds(behavioralProjects)
|
||||
const outlierExclusions = new Set([...lowWorthSessionIds, ...contextBloatVisibleIds, ...firstSessionIds])
|
||||
// Detectors fed by the session scan or by `~/.claude` config only mean
|
||||
// anything when the run covers Claude. Under a different `--provider` they
|
||||
|
|
@ -3521,10 +3564,10 @@ export async function scanAndDetect(
|
|||
claudeOnly(() => detectMcpDeferralOff(toolCalls, projects, projectCwds, apiCalls)),
|
||||
claudeOnly(() => detectMcpAlwaysLoadHygiene(projects, projectCwds, apiCalls, mcpCoverage)),
|
||||
claudeOnly(() => detectMcpDeferThreshold(projects, projectCwds)),
|
||||
() => detectCapabilityReliability(projects),
|
||||
() => detectLowWorthSessions(projects),
|
||||
() => detectContextBloat(projects, lowWorthSessionIds),
|
||||
() => detectSessionOutliers(projects, outlierExclusions),
|
||||
() => detectCapabilityReliability(behavioralProjects),
|
||||
() => detectLowWorthSessions(behavioralProjects),
|
||||
() => detectContextBloat(behavioralProjects, lowWorthSessionIds),
|
||||
() => detectSessionOutliers(behavioralProjects, outlierExclusions),
|
||||
claudeOnly(() => detectBloatedClaudeMd(projectCwds)),
|
||||
claudeOnly(() => detectBashBloat()),
|
||||
claudeOnly(() => detectRecurringContext(openers)),
|
||||
|
|
@ -3550,7 +3593,7 @@ export async function scanAndDetect(
|
|||
const { score, grade } = computeHealth(findings)
|
||||
|
||||
const modelRecommendations: ModelDefaultRecommendation[] = []
|
||||
for (const project of projects) {
|
||||
for (const project of behavioralProjects) {
|
||||
const rec = recommendModelDefault(project, { now: dateRange?.end })
|
||||
if (rec) modelRecommendations.push(rec)
|
||||
}
|
||||
|
|
@ -3717,7 +3760,7 @@ export function renderOptimize(
|
|||
const issueSuffix = findings.length > 0 ? `, ${findings.length} issue${findings.length > 1 ? 's' : ''}` : ''
|
||||
const measured = findings.filter(f => findingBasis(f) === 'measured').length
|
||||
lines.push(' ' + [
|
||||
`${sessionCount} sessions`,
|
||||
`${sessionCount} session${sessionCount === 1 ? '' : 's'}`,
|
||||
`${callCount.toLocaleString()} calls`,
|
||||
chalk.hex(GOLD)(formatCost(periodCost)),
|
||||
`Health: ${chalk.bold.hex(GRADE_COLORS[healthGrade])(healthGrade)}${chalk.dim(` (${healthScore}/100${issueSuffix})`)}`,
|
||||
|
|
@ -3823,7 +3866,7 @@ export async function runOptimize(
|
|||
|
||||
const result = await scanAndDetect(projects, dateRange, opts.provider)
|
||||
const { findings, costRate, healthScore, healthGrade } = result
|
||||
const sessions = projects.flatMap(p => p.sessions)
|
||||
const sessionCount = optimizeSessionCount(projects)
|
||||
const periodCost = projects.reduce((s, p) => s + p.totalCostUSD, 0)
|
||||
const callCount = projects.reduce((s, p) => s + p.totalApiCalls, 0)
|
||||
|
||||
|
|
@ -3833,7 +3876,7 @@ export async function runOptimize(
|
|||
}
|
||||
|
||||
const { topReworkedFiles, coachingNotes } = buildWorkflowReport(projects)
|
||||
const output = renderOptimize(findings, costRate, periodLabel, periodCost, sessions.length, callCount, healthScore, healthGrade, topReworkedFiles, coachingNotes, opts.appliedHeader, opts.previouslyApplied, result.modelRecommendations, opts.appliedFixes)
|
||||
const output = renderOptimize(findings, costRate, periodLabel, periodCost, sessionCount, callCount, healthScore, healthGrade, topReworkedFiles, coachingNotes, opts.appliedHeader, opts.previouslyApplied, result.modelRecommendations, opts.appliedFixes)
|
||||
console.log(output)
|
||||
}
|
||||
|
||||
|
|
@ -3844,7 +3887,6 @@ export function buildOptimizeJsonReport(
|
|||
dateRange?: DateRange,
|
||||
appliedFixes: AppliedFix[] = [],
|
||||
): OptimizeJsonReport {
|
||||
const sessions = projects.flatMap(p => p.sessions)
|
||||
const periodCostUSD = projects.reduce((s, p) => s + p.totalCostUSD, 0)
|
||||
const calls = projects.reduce((s, p) => s + p.totalApiCalls, 0)
|
||||
const potentialSavingsTokens = result.findings.reduce((s, f) => s + f.tokensSaved, 0)
|
||||
|
|
@ -3864,7 +3906,7 @@ export function buildOptimizeJsonReport(
|
|||
healthGrade: result.healthGrade,
|
||||
findingCount: result.findings.length,
|
||||
periodCostUSD,
|
||||
sessions: sessions.length,
|
||||
sessions: optimizeSessionCount(projects),
|
||||
calls,
|
||||
potentialSavingsTokens,
|
||||
potentialSavingsCostUSD,
|
||||
|
|
|
|||
|
|
@ -557,7 +557,7 @@ function extractObjectFields(
|
|||
return captured
|
||||
}
|
||||
|
||||
const LARGE_ROOT_FIELDS = ['type', 'timestamp', 'sessionId', 'cwd', 'gitBranch', 'attachment', 'message'] as const
|
||||
const LARGE_ROOT_FIELDS = ['type', 'timestamp', 'sessionId', 'cwd', 'gitBranch', 'attachment', 'message', 'isSidechain'] as const
|
||||
const LARGE_ASSISTANT_MESSAGE_FIELDS = ['model', 'usage', 'id', 'content'] as const
|
||||
|
||||
function parseLargeJsonl(line: string | Buffer): JournalEntry | null {
|
||||
|
|
@ -571,6 +571,9 @@ function parseLargeJsonl(line: string | Buffer): JournalEntry | null {
|
|||
if (!type) return null
|
||||
|
||||
const entry: JournalEntry = { type }
|
||||
if (root['isSidechain']?.kind === 'scalar' && source.slice(root['isSidechain'].start, root['isSidechain'].end) === 'true') {
|
||||
entry.isSidechain = true
|
||||
}
|
||||
const timestamp = readJsonString(source, root['timestamp'])
|
||||
const sessionId = readJsonString(source, root['sessionId'])
|
||||
const cwd = readJsonString(source, root['cwd'])
|
||||
|
|
@ -2350,6 +2353,7 @@ async function scanProjectDirs(
|
|||
// on a resumed session) and derive the agent id from the `agent-<agentId>`
|
||||
// filename. A sidechain whose parent id was never captured stays standalone.
|
||||
if (cachedFile.isSidechain) {
|
||||
session.isSidechain = true
|
||||
if (cachedFile.parentSessionId) session.parentSessionId = cachedFile.parentSessionId
|
||||
session.agentId = sessionId.startsWith('agent-') ? sessionId.slice('agent-'.length) : sessionId
|
||||
}
|
||||
|
|
@ -3575,6 +3579,7 @@ function carryLinkageFields(rebuilt: SessionSummary, original: SessionSummary):
|
|||
if (original.prLinks?.length) rebuilt.prLinks = original.prLinks
|
||||
if (original.prAttributionSource) rebuilt.prAttributionSource = original.prAttributionSource
|
||||
if (original.workingDirectory) rebuilt.workingDirectory = original.workingDirectory
|
||||
if (original.isSidechain) rebuilt.isSidechain = true
|
||||
// prRefsAtRangeStart is NOT copied here: a narrower slice needs it recomputed at
|
||||
// the new boundary (see recomputeRangeStartPrRefs), not the wide range's value.
|
||||
if (original.parentSessionId) rebuilt.parentSessionId = original.parentSessionId
|
||||
|
|
|
|||
|
|
@ -100,7 +100,7 @@ export type CachedFile = {
|
|||
// is re-parsed only when the file changes (fingerprint differs). Carries no
|
||||
// turns, so it contributes no usage. (issue #441 follow-up)
|
||||
failed?: boolean
|
||||
// Rich-session-capture, Claude session-level (capture-only; no report yet).
|
||||
// Rich-session-capture, Claude session-level.
|
||||
// `title` is the LAST `ai-title` entry's text; `prLinks` accumulates every
|
||||
// `pr-link` entry's URL. `isSidechain` is true when any entry is a sidechain:
|
||||
// parentUuid references an intra-file entry uuid, not another session id, so it
|
||||
|
|
|
|||
34
src/session-population.ts
Normal file
34
src/session-population.ts
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
import type { ProjectSummary, SessionSummary } from './types.js'
|
||||
|
||||
/**
|
||||
* Sidechains are real usage, but they are not user-started work sessions.
|
||||
* Behavioral consumers should use this predicate or the projected project
|
||||
* view below; accounting and configuration consumers should use the originals.
|
||||
*/
|
||||
export function isUserStartedSession(session: SessionSummary): boolean {
|
||||
return session.isSidechain !== true
|
||||
}
|
||||
|
||||
export function withUserStartedSessions(project: ProjectSummary): ProjectSummary {
|
||||
const sessions = project.sessions.filter(isUserStartedSession)
|
||||
if (sessions.length === project.sessions.length) return project
|
||||
|
||||
const totalCostUSD = sessions.reduce((sum, session) => sum + session.totalCostUSD, 0)
|
||||
return {
|
||||
...project,
|
||||
sessions,
|
||||
totalCostUSD,
|
||||
totalSavingsUSD: sessions.reduce((sum, session) => sum + session.totalSavingsUSD, 0),
|
||||
totalEstimatedCostUSD: project.totalEstimatedCostUSD === undefined
|
||||
? undefined
|
||||
: sessions.reduce((sum, session) => sum + (session.totalEstimatedCostUSD ?? 0), 0),
|
||||
totalApiCalls: sessions.reduce((sum, session) => sum + session.apiCalls, 0),
|
||||
// Proxy coverage is project-scoped and applies to every retained session
|
||||
// whenever it applies to the source project.
|
||||
totalProxiedCostUSD: project.totalProxiedCostUSD > 0 ? totalCostUSD : 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function userStartedProjects(projects: ProjectSummary[]): ProjectSummary[] {
|
||||
return projects.map(withUserStartedSessions)
|
||||
}
|
||||
|
|
@ -205,6 +205,11 @@ export type SessionSummary = {
|
|||
/// correlations performed after all saved sessions have been parsed.
|
||||
prAttributionSource?: 'transcript' | 'explicit-reference' | 'working-directory' | 'launcher-prompt'
|
||||
source?: SessionSourceMetadata
|
||||
/// Claude Code only: true when this record is a subagent (sidechain)
|
||||
/// transcript rather than a user-started parent session. Sidechain spend is
|
||||
/// real and remains in every cost/token/call aggregate; consumers that reason
|
||||
/// about human session populations may exclude it explicitly.
|
||||
isSidechain?: boolean
|
||||
// Claude Code only: agent type of a subagent transcript session
|
||||
// (`workflow-subagent`, `Explore`, `general-purpose`, …); undefined for
|
||||
// ordinary sessions. Drives the Claude-scoped agent-type breakdown.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { homedir } from 'os'
|
||||
|
||||
import { EDIT_TOOLS } from './classifier.js'
|
||||
import { userStartedProjects } from './session-population.js'
|
||||
import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
|
||||
|
||||
// User-side mirror of compare-stats.ts scanSelfCorrections (which scans the
|
||||
|
|
@ -44,7 +45,7 @@ export type UserCorrectionStats = {
|
|||
export function scanUserCorrections(projects: ProjectSummary[]): UserCorrectionStats {
|
||||
let corrections = 0
|
||||
let userTurns = 0
|
||||
for (const project of projects) {
|
||||
for (const project of userStartedProjects(projects)) {
|
||||
for (const session of project.sessions) {
|
||||
// A correction is a FOLLOW-UP by definition: the session's opening
|
||||
// prompt cannot be correcting this assistant, however correction-shaped
|
||||
|
|
@ -95,7 +96,7 @@ export function sessionTimeToFirstEditMs(session: ProjectSummary['sessions'][num
|
|||
|
||||
export function medianTimeToFirstEditMs(projects: ProjectSummary[]): number | null {
|
||||
const samples: number[] = []
|
||||
for (const project of projects) {
|
||||
for (const project of userStartedProjects(projects)) {
|
||||
for (const session of project.sessions) {
|
||||
const ms = sessionTimeToFirstEditMs(session)
|
||||
if (ms !== null) samples.push(ms)
|
||||
|
|
@ -140,7 +141,7 @@ export function aggregateFileChurn(projects: ProjectSummary[], limit = 15): Rewo
|
|||
type Acc = { path: string; sessions: Set<string>; edits: number }
|
||||
const byPath = new Map<string, Acc>()
|
||||
|
||||
for (const project of projects) {
|
||||
for (const project of userStartedProjects(projects)) {
|
||||
for (const session of project.sessions) {
|
||||
for (const turn of session.turns) {
|
||||
for (const call of turn.assistantCalls) {
|
||||
|
|
@ -191,7 +192,7 @@ export const MIN_ONE_SHOT_EDIT_TURNS = 5
|
|||
/// model-efficiency and the report's category one-shot figures.
|
||||
export function worstOneShotCategory(projects: ProjectSummary[], minEditTurns = MIN_ONE_SHOT_EDIT_TURNS): CategoryOneShot | null {
|
||||
const acc = new Map<string, { editTurns: number; oneShotTurns: number }>()
|
||||
for (const project of projects) {
|
||||
for (const project of userStartedProjects(projects)) {
|
||||
for (const session of project.sessions) {
|
||||
for (const [cat, d] of Object.entries(session.categoryBreakdown)) {
|
||||
const e = acc.get(cat) ?? { editTurns: 0, oneShotTurns: 0 }
|
||||
|
|
|
|||
|
|
@ -212,6 +212,13 @@ describe('model default recommendations', () => {
|
|||
|
||||
expect(recommendModelDefault(project, { now: NOW })).toBeNull()
|
||||
})
|
||||
|
||||
it('never recommends an actionable default from sidechain-only behavior', () => {
|
||||
const project = recommendationProject()
|
||||
project.sessions[0]!.isSidechain = true
|
||||
|
||||
expect(recommendModelDefault(project, { now: NOW })).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('model default apply plan', () => {
|
||||
|
|
|
|||
|
|
@ -21,6 +21,9 @@ import {
|
|||
detectUnusedMcp,
|
||||
detectBashBloat,
|
||||
detectGhostCommands,
|
||||
detectDuplicateReads,
|
||||
detectJunkReads,
|
||||
detectLowReadEditRatio,
|
||||
loadMcpConfigs,
|
||||
localMcpServerNames,
|
||||
scanJsonlFile,
|
||||
|
|
@ -327,6 +330,78 @@ describe('scanJsonlFile', () => {
|
|||
expect(result.calls[0].name).toBe('Read')
|
||||
})
|
||||
|
||||
it('marks tool calls from sidechain transcript entries', async () => {
|
||||
const root = makeFixtureRoot()
|
||||
const filePath = join(root, 'agent-reviewer.jsonl')
|
||||
const now = new Date().toISOString()
|
||||
writeFile(filePath, JSON.stringify({
|
||||
type: 'assistant', isSidechain: true, timestamp: now,
|
||||
message: { content: [{ type: 'tool_use', name: 'Edit', input: { file_path: '/x/foo.ts' } }] },
|
||||
}))
|
||||
|
||||
const result = await scanJsonlFile(filePath, 'p1', undefined)
|
||||
|
||||
expect(result.calls).toHaveLength(1)
|
||||
expect(result.calls[0]!.isSidechain).toBe(true)
|
||||
})
|
||||
|
||||
it('classifies every tool call in a transcript when a later large entry marks it as sidechain', async () => {
|
||||
const root = makeFixtureRoot()
|
||||
const filePath = join(root, 'agent-reviewer.jsonl')
|
||||
const now = new Date().toISOString()
|
||||
const assistant = (name: string, isSidechain?: boolean, padding = '') => JSON.stringify({
|
||||
type: 'assistant',
|
||||
...(isSidechain === true ? { isSidechain: true } : {}),
|
||||
timestamp: now,
|
||||
cwd: '/x',
|
||||
padding,
|
||||
message: {
|
||||
model: 'claude-sonnet-4-5',
|
||||
usage: { cache_creation_input_tokens: 1 },
|
||||
content: [{ type: 'tool_use', name, input: { file_path: `/x/${name}.ts` } }],
|
||||
},
|
||||
})
|
||||
writeFile(filePath, [
|
||||
JSON.stringify({ type: 'user', timestamp: now, cwd: '/x', message: { content: 'delegate this' } }),
|
||||
assistant('Read'),
|
||||
assistant('Edit', true, 'x'.repeat(40_000)),
|
||||
assistant('Bash'),
|
||||
].join('\n'))
|
||||
|
||||
const result = await scanJsonlFile(filePath, 'p1', undefined)
|
||||
|
||||
expect(result.calls.map(call => [call.name, call.isSidechain])).toEqual([
|
||||
['Read', true],
|
||||
['Edit', true],
|
||||
['Bash', true],
|
||||
])
|
||||
expect(result.apiCalls).toHaveLength(3)
|
||||
expect(result.cwds).toHaveLength(4)
|
||||
expect(result.userMessages).toEqual(['delegate this'])
|
||||
})
|
||||
|
||||
it('keeps sidechain calls out of duplicate reads but in junk reads and the read:edit ratio', () => {
|
||||
const sidechain = { sessionId: 'agent-reviewer', project: 'p1', isSidechain: true }
|
||||
const editCalls = Array.from({ length: 10 }, (_, index) => ({
|
||||
name: 'Edit', input: { file_path: `/src/${index}.ts` }, ...sidechain,
|
||||
}))
|
||||
const junkReads = Array.from({ length: 6 }, () => ({
|
||||
name: 'Read', input: { file_path: '/app/node_modules/pkg/index.js' }, ...sidechain,
|
||||
}))
|
||||
const repeatReads = Array.from({ length: 6 }, () => ({
|
||||
name: 'Read', input: { file_path: '/app/src/a.ts' }, ...sidechain,
|
||||
}))
|
||||
|
||||
// A subagent editing without reading, or reading into node_modules, is the
|
||||
// same waste as the parent doing it, and the CLAUDE.md rule both suggest
|
||||
// binds subagents too - so the full call population feeds them.
|
||||
expect(detectLowReadEditRatio(editCalls)?.id).toBe('read-edit-ratio')
|
||||
expect(detectJunkReads(junkReads)?.id).toBe('build-folder-reads')
|
||||
// A re-read is only waste when the context already held the file; a
|
||||
// sidechain starts fresh and has to read it.
|
||||
expect(detectDuplicateReads(repeatReads)).toBeNull()
|
||||
})
|
||||
|
||||
it('skips malformed JSONL lines without crashing', async () => {
|
||||
const root = makeFixtureRoot()
|
||||
const filePath = join(root, 'session.jsonl')
|
||||
|
|
|
|||
341
tests/optimize-sidechains.test.ts
Normal file
341
tests/optimize-sidechains.test.ts
Normal file
|
|
@ -0,0 +1,341 @@
|
|||
import { describe, expect, it, vi } from 'vitest'
|
||||
|
||||
vi.mock('../src/providers/index.js', async (importOriginal) => {
|
||||
type ProvidersModule = typeof import('../src/providers/index.js')
|
||||
const actual = await importOriginal<ProvidersModule>()
|
||||
return {
|
||||
...actual,
|
||||
async discoverAllSessions() {
|
||||
return []
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
import {
|
||||
buildOptimizeJsonReport,
|
||||
cacheKey,
|
||||
computeInputCostRate,
|
||||
detectCapabilityReliability,
|
||||
detectSessionOutliers,
|
||||
findContextBloatCandidates,
|
||||
findLowWorthCandidates,
|
||||
runOptimize,
|
||||
scanAndDetect,
|
||||
type OptimizeResult,
|
||||
} from '../src/optimize.js'
|
||||
import type { ClassifiedTurn, ProjectSummary, SessionSummary } from '../src/types.js'
|
||||
|
||||
function behavioralTurn(
|
||||
model: string,
|
||||
index: number,
|
||||
options: { retries?: number; costUSD?: number; userMessage?: string } = {},
|
||||
): ClassifiedTurn {
|
||||
const timestamp = new Date(Date.parse('2026-08-01T10:00:00.000Z') + index * 1_000).toISOString()
|
||||
return {
|
||||
userMessage: options.userMessage ?? 'edit the code',
|
||||
timestamp,
|
||||
sessionId: 'agent-behavior',
|
||||
category: 'feature',
|
||||
retries: options.retries ?? 0,
|
||||
hasEdits: true,
|
||||
assistantCalls: [{
|
||||
provider: 'claude',
|
||||
model,
|
||||
usage: {
|
||||
inputTokens: 100,
|
||||
outputTokens: 50,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 0,
|
||||
cachedInputTokens: 0,
|
||||
reasoningTokens: 0,
|
||||
webSearchRequests: 0,
|
||||
},
|
||||
costUSD: options.costUSD ?? 1,
|
||||
tools: ['Edit'],
|
||||
mcpTools: [],
|
||||
skills: [],
|
||||
subagentTypes: [],
|
||||
hasAgentSpawn: false,
|
||||
hasPlanMode: false,
|
||||
speed: 'standard',
|
||||
timestamp,
|
||||
bashCommands: [],
|
||||
deduplicationKey: `${model}-${index}`,
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
function session(
|
||||
sessionId: string,
|
||||
overrides: Partial<SessionSummary> = {},
|
||||
): SessionSummary {
|
||||
return {
|
||||
sessionId,
|
||||
project: 'app',
|
||||
firstTimestamp: '2026-08-01T10:00:00.000Z',
|
||||
lastTimestamp: '2026-08-01T10:30:00.000Z',
|
||||
totalCostUSD: 1,
|
||||
totalSavingsUSD: 0,
|
||||
totalInputTokens: 1_000,
|
||||
totalOutputTokens: 1_000,
|
||||
totalReasoningTokens: 0,
|
||||
totalCacheReadTokens: 0,
|
||||
totalCacheWriteTokens: 0,
|
||||
apiCalls: 1,
|
||||
turns: [],
|
||||
modelBreakdown: {},
|
||||
toolBreakdown: {},
|
||||
mcpBreakdown: {},
|
||||
bashBreakdown: {},
|
||||
categoryBreakdown: {} as SessionSummary['categoryBreakdown'],
|
||||
skillBreakdown: {},
|
||||
subagentBreakdown: {},
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function sidechain(
|
||||
sessionId: string,
|
||||
overrides: Partial<SessionSummary> = {},
|
||||
): SessionSummary {
|
||||
return session(sessionId, {
|
||||
isSidechain: true,
|
||||
parentSessionId: 'parent-session',
|
||||
agentId: sessionId.replace(/^agent-/, ''),
|
||||
...overrides,
|
||||
})
|
||||
}
|
||||
|
||||
function project(sessions: SessionSummary[]): ProjectSummary {
|
||||
return {
|
||||
project: 'app',
|
||||
projectPath: '/tmp/app',
|
||||
sessions,
|
||||
totalCostUSD: sessions.reduce((sum, item) => sum + item.totalCostUSD, 0),
|
||||
totalSavingsUSD: sessions.reduce((sum, item) => sum + item.totalSavingsUSD, 0),
|
||||
totalApiCalls: sessions.reduce((sum, item) => sum + item.apiCalls, 0),
|
||||
totalProxiedCostUSD: 0,
|
||||
}
|
||||
}
|
||||
|
||||
describe('optimize sidechain population (issue #974)', () => {
|
||||
it('does not recommend a model default from sidechain-only edit behavior', async () => {
|
||||
const sonnetTurns = Array.from({ length: 35 }, (_, index) =>
|
||||
behavioralTurn('claude-sonnet-4-20250514', index, {
|
||||
retries: index >= 32 ? 1 : 0,
|
||||
costUSD: 2,
|
||||
}))
|
||||
const haikuTurns = Array.from({ length: 32 }, (_, index) =>
|
||||
behavioralTurn('claude-haiku-3-5-20241022', index + 35, {
|
||||
retries: index >= 29 ? 1 : 0,
|
||||
costUSD: 0.9,
|
||||
}))
|
||||
const child = sidechain('agent-behavior', { turns: [...sonnetTurns, ...haikuTurns] })
|
||||
const projects = [project([child])]
|
||||
|
||||
const result = await scanAndDetect(projects, {
|
||||
start: new Date('2026-08-01T00:00:00.000Z'),
|
||||
end: new Date('2026-08-02T00:00:00.000Z'),
|
||||
})
|
||||
|
||||
expect(result.modelRecommendations).toEqual([])
|
||||
})
|
||||
|
||||
it('does not emit coaching from sidechain-only correction behavior', () => {
|
||||
const turns = Array.from({ length: 66 }, (_, index) =>
|
||||
behavioralTurn('claude-sonnet-4-20250514', index, {
|
||||
userMessage: index === 0 ? 'review the code' : 'you missed the edge case',
|
||||
}))
|
||||
const child = sidechain('agent-corrections', {
|
||||
totalCostUSD: 9,
|
||||
totalInputTokens: 6_600,
|
||||
totalOutputTokens: 3_300,
|
||||
apiCalls: 66,
|
||||
turns,
|
||||
})
|
||||
const projects = [project([child])]
|
||||
const result: OptimizeResult = {
|
||||
findings: [],
|
||||
costRate: computeInputCostRate(projects),
|
||||
healthScore: 100,
|
||||
healthGrade: 'A',
|
||||
modelRecommendations: [],
|
||||
}
|
||||
|
||||
const report = buildOptimizeJsonReport(projects, 'fixture', result)
|
||||
|
||||
expect(report.coachingNotes).toEqual([])
|
||||
expect(report.summary.periodCostUSD).toBe(9)
|
||||
expect(report.summary.calls).toBe(66)
|
||||
})
|
||||
|
||||
it('does not report retry-heavy capabilities from sidechain-only edits', () => {
|
||||
const turns = Array.from({ length: 5 }, (_, index) => {
|
||||
const item = behavioralTurn('claude-sonnet-4-20250514', index, {
|
||||
retries: index < 3 ? 1 : 0,
|
||||
})
|
||||
item.assistantCalls[0]!.skills = ['reviewer']
|
||||
return item
|
||||
})
|
||||
const child = sidechain('agent-capability', { turns })
|
||||
|
||||
expect(detectCapabilityReliability([project([child])])).toBeNull()
|
||||
})
|
||||
|
||||
it('keeps sidechain spend out of the low-worth candidate population', () => {
|
||||
const parent = session('parent', {
|
||||
totalCostUSD: 4,
|
||||
turns: [],
|
||||
})
|
||||
const child = sidechain('agent-child', {
|
||||
totalCostUSD: 12,
|
||||
turns: [],
|
||||
})
|
||||
|
||||
expect(findLowWorthCandidates([project([parent, child])]).map(item => item.sessionId))
|
||||
.toEqual(['parent'])
|
||||
})
|
||||
|
||||
it('does not use a sidechain as a context-heavy candidate or growth baseline', () => {
|
||||
const baseline = session('parent-baseline', {
|
||||
firstTimestamp: '2026-08-01T10:00:00.000Z',
|
||||
totalInputTokens: 20_000,
|
||||
totalOutputTokens: 2_000,
|
||||
})
|
||||
const child = sidechain('agent-child', {
|
||||
firstTimestamp: '2026-08-02T10:00:00.000Z',
|
||||
totalInputTokens: 200_000,
|
||||
totalOutputTokens: 100,
|
||||
})
|
||||
const candidate = session('parent-candidate', {
|
||||
firstTimestamp: '2026-08-03T10:00:00.000Z',
|
||||
totalInputTokens: 100_000,
|
||||
totalOutputTokens: 2_000,
|
||||
})
|
||||
|
||||
const candidates = findContextBloatCandidates([project([baseline, child, candidate])])
|
||||
|
||||
expect(candidates.map(item => item.sessionId)).toEqual(['parent-candidate'])
|
||||
expect(candidates[0]!.growthRatio).toBe(5)
|
||||
})
|
||||
|
||||
it('does not let a sidechain satisfy the peer-sample minimum for cost outliers', () => {
|
||||
const sessions = [
|
||||
session('parent-cheap', { totalCostUSD: 1 }),
|
||||
session('parent-expensive', { totalCostUSD: 10 }),
|
||||
sidechain('agent-cheap', { totalCostUSD: 1 }),
|
||||
]
|
||||
|
||||
expect(detectSessionOutliers([project(sessions)])).toBeNull()
|
||||
})
|
||||
|
||||
it('never reports an expensive sidechain as a parent-session cost outlier', () => {
|
||||
const sessions = [
|
||||
session('parent-1', { totalCostUSD: 1 }),
|
||||
session('parent-2', { totalCostUSD: 1 }),
|
||||
session('parent-3', { totalCostUSD: 1 }),
|
||||
sidechain('agent-expensive', { totalCostUSD: 100 }),
|
||||
]
|
||||
|
||||
expect(detectSessionOutliers([project(sessions)])).toBeNull()
|
||||
})
|
||||
|
||||
it('counts only parent sessions while conserving sidechain cost, calls, and tokens', () => {
|
||||
const projects = [project([
|
||||
session('parent', {
|
||||
totalCostUSD: 3,
|
||||
totalInputTokens: 100,
|
||||
totalOutputTokens: 20,
|
||||
apiCalls: 2,
|
||||
}),
|
||||
sidechain('agent-child', {
|
||||
totalCostUSD: 7,
|
||||
totalInputTokens: 900,
|
||||
totalOutputTokens: 80,
|
||||
apiCalls: 4,
|
||||
}),
|
||||
])]
|
||||
const result: OptimizeResult = {
|
||||
findings: [],
|
||||
costRate: computeInputCostRate(projects),
|
||||
healthScore: 100,
|
||||
healthGrade: 'A',
|
||||
}
|
||||
|
||||
const report = buildOptimizeJsonReport(projects, 'fixture', result)
|
||||
|
||||
expect(report.summary.sessions).toBe(1)
|
||||
expect(report.summary.periodCostUSD).toBe(10)
|
||||
expect(report.summary.calls).toBe(6)
|
||||
// Input-cost calibration keeps all spend and all input/cache tokens:
|
||||
// ($10 * 0.7) / (100 + 900) tokens.
|
||||
expect(report.summary.costRateUSD).toBeCloseTo(0.007, 12)
|
||||
})
|
||||
|
||||
it('keeps sidechain spend out of every per-session finding in the optimize pipeline', async () => {
|
||||
const projects = [project([
|
||||
sidechain('agent-only', {
|
||||
totalCostUSD: 100,
|
||||
totalInputTokens: 1_000_000,
|
||||
totalOutputTokens: 100,
|
||||
}),
|
||||
])]
|
||||
|
||||
const result = await scanAndDetect(projects, {
|
||||
start: new Date('2026-08-01T00:00:00.000Z'),
|
||||
end: new Date('2026-08-02T00:00:00.000Z'),
|
||||
})
|
||||
|
||||
expect(result.findings.map(finding => finding.id)).not.toContain('low-worth-sessions')
|
||||
expect(result.findings.map(finding => finding.id)).not.toContain('context-heavy-sessions')
|
||||
expect(result.findings.map(finding => finding.id)).not.toContain('cost-outliers')
|
||||
})
|
||||
|
||||
it('uses the parent-session count in the text optimize headline', async () => {
|
||||
const projects = [project([
|
||||
session('parent', {
|
||||
totalCostUSD: 1,
|
||||
bashBreakdown: { 'git commit -m shipped': { calls: 1 } },
|
||||
}),
|
||||
sidechain('agent-child', {
|
||||
totalCostUSD: 2,
|
||||
bashBreakdown: { 'git commit -m irrelevant': { calls: 1 } },
|
||||
}),
|
||||
])]
|
||||
const log = vi.spyOn(console, 'log').mockImplementation(() => undefined)
|
||||
const stderr = vi.spyOn(process.stderr, 'write').mockImplementation(() => true)
|
||||
try {
|
||||
await runOptimize(projects, 'fixture', {
|
||||
start: new Date('2026-08-01T00:00:00.000Z'),
|
||||
end: new Date('2026-08-02T00:00:00.000Z'),
|
||||
})
|
||||
const output = String(log.mock.calls.at(-1)?.[0] ?? '')
|
||||
expect(output).toContain('1 session')
|
||||
expect(output).not.toContain('2 sessions')
|
||||
} finally {
|
||||
log.mockRestore()
|
||||
stderr.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('separates cached optimize results when only sidechain classification changes', () => {
|
||||
const range = {
|
||||
start: new Date('2026-08-01T00:00:00.000Z'),
|
||||
end: new Date('2026-08-02T00:00:00.000Z'),
|
||||
}
|
||||
const parentOnly = project([session('same')])
|
||||
const sidechainOnly = project([sidechain('same')])
|
||||
|
||||
expect(parentOnly.totalCostUSD).toBe(sidechainOnly.totalCostUSD)
|
||||
expect(parentOnly.totalApiCalls).toBe(sidechainOnly.totalApiCalls)
|
||||
expect(cacheKey([parentOnly], range)).not.toBe(cacheKey([sidechainOnly], range))
|
||||
|
||||
const firstSidechain = project([session('first'), sidechain('second')])
|
||||
const secondSidechain = project([sidechain('first'), session('second')])
|
||||
expect(firstSidechain.sessions.length).toBe(secondSidechain.sessions.length)
|
||||
expect(firstSidechain.totalCostUSD).toBe(secondSidechain.totalCostUSD)
|
||||
expect(firstSidechain.sessions.filter(item => item.isSidechain).length)
|
||||
.toBe(secondSidechain.sessions.filter(item => item.isSidechain).length)
|
||||
expect(cacheKey([firstSidechain], range)).not.toBe(cacheKey([secondSidechain], range))
|
||||
})
|
||||
})
|
||||
|
|
@ -21,6 +21,7 @@ function largeUserLine(): string {
|
|||
function largeAssistantLine(): string {
|
||||
return JSON.stringify({
|
||||
type: 'assistant',
|
||||
isSidechain: true,
|
||||
sessionId: 's1',
|
||||
timestamp: '2026-05-01T00:00:01Z',
|
||||
cwd: '/repo',
|
||||
|
|
@ -55,6 +56,7 @@ describe('large JSONL compact scanner', () => {
|
|||
|
||||
it('extracts capped tool inputs needed by optimize', () => {
|
||||
const parsed = parseJsonlLine(Buffer.from(largeAssistantLine()))
|
||||
expect(parsed?.isSidechain).toBe(true)
|
||||
const msg = parsed?.message
|
||||
expect(msg?.role).toBe('assistant')
|
||||
if (msg?.role !== 'assistant') return
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ function assistantLine(sessionId: string, timestamp: string, messageId: string,
|
|||
|
||||
function messageFirstLargeAssistantLine(sessionId: string, timestamp: string, messageId: string): string {
|
||||
const hugeText = 'y'.repeat(3_000_000)
|
||||
return `{"parentUuid":"u1","isSidechain":false,"message":{"model":"claude-sonnet-4-5","id":"${messageId}","type":"message","role":"assistant","content":[{"type":"text","text":"${hugeText}"},{"type":"tool_use","id":"tu-large","name":"Edit","input":{"file_path":"/tmp/x","old_string":"a","new_string":"b"}}],"usage":{"input_tokens":1000,"output_tokens":100,"cache_read_input_tokens":5000}},"uuid":"a1","timestamp":"${timestamp}","type":"assistant","sessionId":"${sessionId}","cwd":"/projects/app"}`
|
||||
return `{"parentUuid":"u1","isSidechain":true,"message":{"model":"claude-sonnet-4-5","id":"${messageId}","type":"message","role":"assistant","content":[{"type":"text","text":"${hugeText}"},{"type":"tool_use","id":"tu-large","name":"Edit","input":{"file_path":"/tmp/x","old_string":"a","new_string":"b"}}],"usage":{"input_tokens":1000,"output_tokens":100,"cache_read_input_tokens":5000}},"uuid":"a1","timestamp":"${timestamp}","type":"assistant","sessionId":"${sessionId}","cwd":"/projects/app"}`
|
||||
}
|
||||
|
||||
function attachmentLine(sessionId: string, timestamp: string): string {
|
||||
|
|
@ -227,6 +227,7 @@ describe('parseAllSessions with large Claude fixture', () => {
|
|||
expect(projects.length).toBeGreaterThan(0)
|
||||
|
||||
const sess = projects[0]!.sessions[0]!
|
||||
expect(sess.isSidechain).toBe(true)
|
||||
expect(sess.apiCalls).toBe(1)
|
||||
expect(sess.totalInputTokens).toBe(1000)
|
||||
expect(sess.totalOutputTokens).toBe(100)
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import { join } from 'path'
|
|||
import { tmpdir } from 'os'
|
||||
|
||||
import { parseAllSessions, filterProjectsByDays, clearSessionCache } from '../src/parser.js'
|
||||
import { clearLoadCacheMemo } from '../src/session-cache.js'
|
||||
import { loadPricing } from '../src/models.js'
|
||||
import { aggregateByPr, prLinkedTotals } from '../src/sessions-report.js'
|
||||
|
||||
|
|
@ -64,8 +65,16 @@ describe('subagent fold across a date-range boundary', () => {
|
|||
const projects = await parseAllSessions(range, 'claude')
|
||||
|
||||
// The child session is present (its work is in range) as a standalone session.
|
||||
const childPresent = projects.some(p => p.sessions.some(s => s.sessionId === `agent-${AGENT}`))
|
||||
expect(childPresent).toBe(true)
|
||||
const child = projects.flatMap(p => p.sessions).find(s => s.sessionId === `agent-${AGENT}`)
|
||||
expect(child).toBeDefined()
|
||||
expect(child!.isSidechain).toBe(true)
|
||||
// Drop both in-process memo layers so the second parse reloads the persisted
|
||||
// session cache. The marker must survive that warm-disk path too, not only
|
||||
// the cold transcript parse.
|
||||
clearSessionCache()
|
||||
clearLoadCacheMemo()
|
||||
const warmProjects = await parseAllSessions(range, 'claude')
|
||||
expect(warmProjects.flatMap(p => p.sessions).find(s => s.sessionId === `agent-${AGENT}`)?.isSidechain).toBe(true)
|
||||
// The anchor parent (0 in-range turns) must NOT contaminate the sessions list;
|
||||
// it lives in subagentAnchors only.
|
||||
const anchorInSessions = projects.some(p => p.sessions.some(s => s.sessionId === PARENT))
|
||||
|
|
@ -101,6 +110,7 @@ describe('subagent fold across a date-range boundary', () => {
|
|||
const dayFiltered = filterProjectsByDays(projects, new Set(['2026-07-20']))
|
||||
expect(dayFiltered.some(p => p.sessions.some(s => s.sessionId === PARENT))).toBe(false) // parent no longer a session
|
||||
expect(dayFiltered.some(p => (p.subagentAnchors ?? []).some(s => s.sessionId === PARENT))).toBe(true) // kept as anchor
|
||||
expect(dayFiltered.flatMap(p => p.sessions).find(s => s.sessionId === `agent-${AGENT}`)?.isSidechain).toBe(true)
|
||||
|
||||
// The child's spend still folds to the PR through the anchor.
|
||||
const row = aggregateByPr(dayFiltered).find(r => r.url === PR)
|
||||
|
|
|
|||
|
|
@ -353,4 +353,25 @@ describe('review-findings regressions', () => {
|
|||
// 95 local calls + 5 unpriced cloud calls: coverage must be 0, not 0.95.
|
||||
expect(computePricingCoverage(5, 5)).toBe(0)
|
||||
})
|
||||
|
||||
it('excludes sidechain-only work from every human workflow signal', () => {
|
||||
const editCall = call({
|
||||
tools: ['Edit'],
|
||||
timestamp: '2026-06-01T10:06:00Z',
|
||||
toolSequence: [[{ tool: 'Edit', file: '/home/u/app/src/a.ts' }]],
|
||||
})
|
||||
const sidechain = session('agent-reviewer', [
|
||||
turn({ userMessage: 'review the change', timestamp: '2026-06-01T10:00:00Z' }),
|
||||
turn({ userMessage: 'you missed the edge case', calls: [editCall], timestamp: '2026-06-01T10:06:00Z' }),
|
||||
turn({ userMessage: 'that is still wrong', timestamp: '2026-06-01T10:07:00Z' }),
|
||||
turn({ userMessage: 'revert that change', timestamp: '2026-06-01T10:08:00Z' }),
|
||||
], { feature: cat(10, 0) } as SessionSummary['categoryBreakdown'])
|
||||
sidechain.isSidechain = true
|
||||
const projects = [project([sidechain])]
|
||||
|
||||
expect(scanUserCorrections(projects)).toEqual({ corrections: 0, userTurns: 0, correctionRate: null })
|
||||
expect(medianTimeToFirstEditMs(projects)).toBeNull()
|
||||
expect(aggregateFileChurn(projects)).toEqual([])
|
||||
expect(worstOneShotCategory(projects)).toBeNull()
|
||||
})
|
||||
})
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue