fix(report): route displayed output through billableOutputTokens

Cost already used the helper. Report, sessions, overview and optimize
did not, so exclusive providers dropped reasoning and optimize
double-counted it for claude/codex/copilot. Daily cache v28 re-derives.

Closes #1115

CHANGELOG.md

@@ -19,6 +19,7 @@
 - **DeepSeek Harness (`dsh`) is now a supported provider.** Reads DeepSeek's open-source agent harness from `~/.dsh/sessions` (`DSH_HOME` relocates the root), both the default zstd logs and the uncompressed `session.jsonl` variant. A `.zstd` log is a concatenation of independent zstd frames, one per write batch, so it is decoded frame by frame behind a structural frame scan and a torn trailing frame from a crashed writer is ignored rather than failing the file (needs Node 22.15+ for `zlib` zstd; below that dsh is skipped with a notice instead of counted as $0). One call per `(turn, step)`, with the step's final `assistant/message` usage superseding the streamed `assistant/chunk` sample of the same call rather than adding to it, the model taken from the message that served the step, and reasoning tokens billed at the output rate. DSH records tokens but no cost, so calls are priced from the shared tables. The events a forked session replays from its parent are skipped, since codeburn already counts the parent's own log. The session format is pinned at version 0 upstream with no compatibility implied, so a log stamped with any other version is skipped with a notice instead of read under today's assumptions.
 
 ### Changed
+- **Report, sessions, overview and optimize now use `billableOutputTokens` for displayed output.** Cost already did. Exclusive providers (Grok and the rest) were under-reporting output by exactly their reasoning tokens; optimize added reasoning on top of output for the inclusive set `{claude, codex, copilot}` and double-counted it. Daily cache v28 re-derives finalized days so `report` matches the live parse. Thanks @saulcanina. (#1115)
 - **The discovery sweep issues its metadata syscalls concurrently.** Every dated command re-walks and re-stats every provider tree before it can decide what is already cached, and that sweep was strictly serial: one `readdir`, one `stat`, one `state.json` read at a time, per provider, one provider after another. On a 21k-file / 9-provider corpus it owned most of a warm run's wall clock while the machine sat idle waiting on the kernel. Provider discovery now runs across providers at once, and the four walks that dominate it (claude, codex, kimicode, grok) plus the Claude project-dir walk and both fingerprint passes fan out through a shared bounded-concurrency helper. Order is unchanged everywhere — each level is re-concatenated in registry/`readdir` order before anything reconciles against the cache — so what is discovered, in what sequence, is byte-identical to the serial walk. Two smaller cuts ride along: the Claude walk reads directory entries with their types so a plain file no longer costs a wasted `subagents/` probe, and the Codex result cache (a single file that can reach hundreds of MB) now shares one in-flight load between concurrent readers instead of letting each one re-read and re-parse it. Warm `codeburn today` on that corpus: 5.30s to 2.99s; a cold parse 34.7s to 31.1s. (#1104)
 - **A kill mid-way through a non-Claude provider phase no longer restarts that whole phase.** `scanProjectDirs` (Claude) has long taken a throttled `saveProgress` callback so a killed cold parse resumes from a warm cache; `parseProviderSources` (every other provider — codex, cursor, gemini, and the rest) did not, and only persisted at the whole-provider boundary. On a large single-provider corpus (a multi-GB codex history is the common case) an app-timeout SIGKILL, crash, or force-quit during that phase discarded everything parsed since the last provider finished, forcing the entire phase to re-parse from zero on the next run. `parseProviderSources` now takes the same callback, invoked once per source right after that source's cache entry lands (mirroring `scanProjectDirs`' placement, outside the per-file try/catch), on the same file-count/wall-clock throttle. A file only ever gets a fingerprint once it has fully parsed, so a mid-file kill can never leave a half-parsed file's entry looking complete on resume.
 - **Routed model ids price as the model they wrap, and an unknown vendor prefix no longer prices by blind stripping.** Token-plan and gateway spellings of the same model (`omniroute:`, `cp/`, `cline-pass/`, `cline-free/`, `cmd/`, `antigravity/`) are peeled and the remaining id is priced, so a Cline Pass or OmniRoute session shows a `~` estimate instead of $0. In exchange, `provider/model` is no longer treated as authority on its own: the leading segment is stripped only when it is a namespace the bundled pricing catalog itself uses (`anthropic/`, `openai/`, `google/`, `x-ai/`, `qwen/`, `moonshotai/`, `nousresearch/`, `xiaomi/`, `z-ai/`, and every other vendor prefix in the LiteLLM snapshot), one of the routing wrappers above, or one of the client-side spellings `kimi/`, `mimo/`, `zhipu/`, `litellm_proxy/` and `openai_like/`. Anything else stays unpriced and is reported as unpriced rather than inheriting the price of a same-named cloud row, and local-runner prefixes (`ollama/`, `lmstudio/`, `hosted_vllm/`, `local/`) are excluded on purpose so an unlisted local tag can never invent cloud spend. A user price override for the bare id wins over the catalog row a routed spelling would otherwise hit.

src/daily-cache.ts

@@ -168,8 +168,12 @@ import type { DateRange, ProjectSummary } from './types.js'
 // v25: #1047 activity-id pricing. v24 on main already shipped #1090.
 // v26: #946 copilot session-store accounting (see the top of this ladder).
 // v27: #1093 claude-haiku-4.5 alias (see top).
-export const DAILY_CACHE_VERSION = 27
-const MIN_SUPPORTED_VERSION = 27
+// v28: #1115 report/optimize output tokens go through billableOutputTokens.
+// Exclusive providers (Grok and the rest) were under-counted by reasoningTokens
+// in finalized daily rows; inclusive {claude,codex,copilot} were correct there
+// but optimize added reasoning again. Re-derive so report matches the live parse.
+export const DAILY_CACHE_VERSION = 28
+const MIN_SUPPORTED_VERSION = 28
 
 /// Providers whose per-day CALL COUNT means something different at
 /// DAILY_CACHE_VERSION 26 than it did before it. Copilot's supplementary

src/day-aggregator.ts

@@ -2,6 +2,7 @@ import type { DailyEntry, ProjectDayStats, ProviderDaySlice } from './daily-cach
 import type { PeriodData } from './menubar-json.js'
 import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
 import { isBehavioralCall, isBehavioralTurn } from './behavioral-weight.js'
+import { billableOutputTokens } from './models.js'
 
 function emptyEntry(date: string): DailyEntry {
   return {
@@ -189,8 +190,13 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn:
           callDay.cost += call.costUSD
           callDay.savingsUSD += callSavings
           callDay.calls += callWeight
+          const billableOut = billableOutputTokens(
+            call.provider,
+            call.usage.outputTokens,
+            call.usage.reasoningTokens,
+          )
           callDay.inputTokens += call.usage.inputTokens
-          callDay.outputTokens += call.usage.outputTokens
+          callDay.outputTokens += billableOut
           callDay.cacheReadTokens += call.usage.cacheReadInputTokens
           callDay.cacheWriteTokens += call.usage.cacheCreationInputTokens
 
@@ -208,7 +214,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn:
           model.cost += call.costUSD
           model.savingsUSD += callSavings
           model.inputTokens += call.usage.inputTokens
-          model.outputTokens += call.usage.outputTokens
+          model.outputTokens += billableOut
           model.cacheReadTokens += call.usage.cacheReadInputTokens
           model.cacheWriteTokens += call.usage.cacheCreationInputTokens
           callDay.models[call.model] = model
@@ -218,7 +224,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn:
           slice.cost += call.costUSD
           slice.savingsUSD += callSavings
           slice.inputTokens! += call.usage.inputTokens
-          slice.outputTokens! += call.usage.outputTokens
+          slice.outputTokens! += billableOut
           slice.cacheReadTokens! += call.usage.cacheReadInputTokens
           slice.cacheWriteTokens! += call.usage.cacheCreationInputTokens
 
@@ -236,7 +242,7 @@ export function aggregateProjectsIntoDays(projects: ProjectSummary[], dateKeyFn:
           sliceModel.cost += call.costUSD
           sliceModel.savingsUSD += callSavings
           sliceModel.inputTokens += call.usage.inputTokens
-          sliceModel.outputTokens += call.usage.outputTokens
+          sliceModel.outputTokens += billableOut
           sliceModel.cacheReadTokens += call.usage.cacheReadInputTokens
           sliceModel.cacheWriteTokens += call.usage.cacheCreationInputTokens
           slice.models![call.model] = sliceModel

src/optimize.ts

@@ -16,6 +16,7 @@ import { formatTokens } from './format.js'
 import { recommendModelDefault, type ModelDefaultRecommendation } from './act/model-defaults.js'
 import { appliedFixGlyph, formatAppliedFix, type AppliedFix } from './act/types.js'
 import { isUserStartedSession, userStartedProjects } from './session-population.js'
+import { sessionBillableOutputTokens } from './session-output.js'
 import { aggregateFileChurn, buildCoachingNotes, scanUserCorrections, medianTimeToFirstEditMs, worstOneShotCategory, type ReworkedFile } from './workflow-insights.js'
 
 // ============================================================================
@@ -3030,7 +3031,7 @@ export function detectRecurringContext(openers: SessionOpener[]): WasteFinding |
 
 function sessionTokenTotal(session: ProjectSummary['sessions'][number]): number {
   return session.totalInputTokens
-    + session.totalOutputTokens
+    + sessionBillableOutputTokens(session)
     + session.totalCacheReadTokens
     + session.totalCacheWriteTokens
 }
@@ -3268,9 +3269,9 @@ export function findContextBloatCandidates(projects: ProjectSummary[]): ContextB
 
     for (const session of sessions) {
       const inputTokens = sessionEffectiveContextTokens(session)
-      // Reasoning is stored separately from ordinary output, but both are
-      // generated tokens for this detector. Reports already use their sum.
-      const outputTokens = session.totalOutputTokens + session.totalReasoningTokens
+      // Generated tokens: exclusive providers add reasoning; inclusive ones
+      // already folded it into totalOutputTokens (#1078 / #1115).
+      const outputTokens = sessionBillableOutputTokens(session)
       const ratio = inputTokens / Math.max(outputTokens, 1)
       const currentMs = new Date(session.firstTimestamp).getTime()
       const gapMs = previousTimestampMs !== null ? currentMs - previousTimestampMs : null

src/overview.ts

@@ -5,6 +5,7 @@ import { homedir } from 'os'
 import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
 import { formatCost as baseCost, getCurrency } from './currency.js'
 import { findUnpricedModels, getShortModelName, unpricedModelHint } from './models.js'
+import { sessionBillableOutputTokens } from './session-output.js'
 import { markEstimated } from './format.js'
 import { dateKey } from './day-aggregator.js'
 import type { DailyEntry } from './daily-cache.js'
@@ -140,7 +141,7 @@ export function renderOverview(
     byProject.set(pname, pe)
     for (const s of p.sessions) {
       inTok += s.totalInputTokens
-      outTok += s.totalOutputTokens
+      outTok += sessionBillableOutputTokens(s)
       cacheR += s.totalCacheReadTokens
       cacheW += s.totalCacheWriteTokens
       for (const [m, d] of Object.entries(s.modelBreakdown)) {

src/session-output.ts

@@ -0,0 +1,40 @@
+import { billableOutputTokens } from './models.js'
+import type { SessionSummary } from './types.js'
+
+/** First on-call provider, then a model-name fallback. Sessions are usually one provider. */
+export function inferSessionProvider(session: SessionSummary): string {
+  for (const turn of session.turns) {
+    const provider = turn.assistantCalls[0]?.provider
+    if (provider) return provider
+  }
+
+  const models = Object.keys(session.modelBreakdown)
+  const model = models[0]?.toLowerCase() ?? ''
+  if (model.startsWith('claude')) return 'claude'
+  if (model.startsWith('gpt-') || model.startsWith('o1') || model.startsWith('o3') || model.startsWith('o4')) return 'codex'
+  if (model.startsWith('gemini')) return 'gemini'
+  if (model.includes('/')) return model.split('/', 1)[0] || 'unknown'
+  return 'unknown'
+}
+
+/** Display/report output: exclusive providers add reasoning; inclusive ones do not. */
+export function sessionBillableOutputTokens(session: SessionSummary): number {
+  let fromCalls = 0
+  let sawCall = false
+  for (const turn of session.turns) {
+    for (const call of turn.assistantCalls) {
+      sawCall = true
+      fromCalls += billableOutputTokens(
+        call.provider,
+        call.usage.outputTokens,
+        call.usage.reasoningTokens,
+      )
+    }
+  }
+  if (sawCall) return fromCalls
+  return billableOutputTokens(
+    inferSessionProvider(session),
+    session.totalOutputTokens,
+    session.totalReasoningTokens,
+  )
+}

src/sessions-report.ts

@@ -1,5 +1,6 @@
 import { behavioralCallCount, behavioralTurnCount } from './behavioral-weight.js'
 import { getShortModelName } from './models.js'
+import { inferSessionProvider, sessionBillableOutputTokens } from './session-output.js'
 import { CATEGORY_LABELS } from './types.js'
 import type { ProjectSummary, SessionSummary, TaskCategory } from './types.js'
 
@@ -23,21 +24,6 @@ export type SessionRow = {
   durationMs: number
 }
 
-function inferProvider(session: SessionSummary): string {
-  for (const turn of session.turns) {
-    const provider = turn.assistantCalls[0]?.provider
-    if (provider) return provider
-  }
-
-  const models = Object.keys(session.modelBreakdown)
-  const model = models[0]?.toLowerCase() ?? ''
-  if (model.startsWith('claude')) return 'claude'
-  if (model.startsWith('gpt-') || model.startsWith('o1') || model.startsWith('o3') || model.startsWith('o4')) return 'codex'
-  if (model.startsWith('gemini')) return 'gemini'
-  if (model.includes('/')) return model.split('/', 1)[0] || 'unknown'
-  return 'unknown'
-}
-
 function durationMs(startedAt: string, endedAt: string): number {
   const duration = new Date(endedAt).getTime() - new Date(startedAt).getTime()
   return Number.isFinite(duration) ? duration : 0
@@ -48,14 +34,14 @@ export function aggregateSessions(projects: ProjectSummary[]): SessionRow[] {
     sessionId: session.sessionId,
     title: session.title ?? '',
     project: session.project || project.project,
-    provider: inferProvider(session),
+    provider: inferSessionProvider(session),
     models: Object.keys(session.modelBreakdown),
     cost: session.totalCostUSD,
     savingsUSD: session.totalSavingsUSD,
     calls: session.apiCalls,
     turns: behavioralTurnCount(session.turns),
     inputTokens: session.totalInputTokens,
-    outputTokens: session.totalOutputTokens,
+    outputTokens: sessionBillableOutputTokens(session),
     cacheReadTokens: session.totalCacheReadTokens,
     cacheWriteTokens: session.totalCacheWriteTokens,
     startedAt: session.firstTimestamp,
@@ -374,7 +360,7 @@ const KEY_SEP = String.fromCharCode(0)
 
 function linkageProvider(session: SessionSummary): string {
   if (session.parentSessionId || session.agentSpawnLinks || session.spawnPrSets) return 'claude'
-  return inferProvider(session)
+  return inferSessionProvider(session)
 }
 function providerSessionKey(session: SessionSummary): string {
   return `${linkageProvider(session)}${KEY_SEP}${session.sessionId}`

src/usage-aggregator.ts

@@ -13,6 +13,7 @@ import { aggregateModels } from './models-report.js'
 import { scanUserCorrections, medianTimeToFirstEditMs, aggregateFileChurn, computePricingCoverage } from './workflow-insights.js'
 import { buildPrAttribution, aggregateByBranch } from './sessions-report.js'
 import { scanAndDetect } from './optimize.js'
+import { sessionBillableOutputTokens } from './session-output.js'
 import { getDaysInRange, ensureCacheHydrated, emptyCache, BACKFILL_DAYS, toDateString, type DailyCache, type DailyEntry, type ProjectDayStats, type ProviderDaySlice } from './daily-cache.js'
 import { buildGranularHistory } from './granular-history.js'
 
@@ -27,7 +28,7 @@ export function buildPeriodData(label: string, projects: ProjectSummary[]): Peri
 
   for (const sess of sessions) {
     inputTokens += sess.totalInputTokens
-    outputTokens += sess.totalOutputTokens
+    outputTokens += sessionBillableOutputTokens(sess)
     cacheReadTokens += sess.totalCacheReadTokens
     cacheWriteTokens += sess.totalCacheWriteTokens
     for (const [cat, d] of Object.entries(sess.categoryBreakdown)) {
@@ -748,7 +749,7 @@ export async function buildMenubarPayloadForRange(periodInfo: PeriodInfo, opts:
       savingsUSD: s.totalSavingsUSD,
       calls: s.apiCalls,
       inputTokens: s.totalInputTokens,
-      outputTokens: s.totalOutputTokens,
+      outputTokens: sessionBillableOutputTokens(s),
       date: s.firstTimestamp?.split('T')[0] ?? '',
       models: Object.entries(s.modelBreakdown)
         .map(([name, m]) => ({ name, cost: m.costUSD, savingsUSD: m.savingsUSD }))

tests/billable-output-1115.test.ts

@@ -0,0 +1,129 @@
+import { describe, expect, it } from 'vitest'
+
+import { aggregateProjectsIntoDays } from '../src/day-aggregator.js'
+import { billableOutputTokens } from '../src/models.js'
+import { findContextBloatCandidates } from '../src/optimize.js'
+import { sessionBillableOutputTokens } from '../src/session-output.js'
+import { aggregateSessions } from '../src/sessions-report.js'
+import { buildPeriodData } from '../src/usage-aggregator.js'
+import type { ProjectSummary, SessionSummary } from '../src/types.js'
+
+function makeCall(provider: string, outputTokens: number, reasoningTokens: number) {
+  return {
+    provider,
+    model: provider === 'codex' ? 'gpt-5.4' : 'grok-4',
+    usage: {
+      inputTokens: 0,
+      outputTokens,
+      cacheCreationInputTokens: 0,
+      cacheReadInputTokens: 0,
+      cachedInputTokens: 0,
+      reasoningTokens,
+      webSearchRequests: 0,
+    },
+    costUSD: 0,
+    tools: [],
+    mcpTools: [],
+    skills: [],
+    hasAgentSpawn: false,
+    hasPlanMode: false,
+    speed: 'standard' as const,
+    timestamp: '2026-08-01T12:00:00Z',
+    bashCommands: [],
+    deduplicationKey: `${provider}-${outputTokens}-${reasoningTokens}`,
+  }
+}
+
+function makeSession(provider: string, outputTokens: number, reasoningTokens: number): SessionSummary {
+  const call = makeCall(provider, outputTokens, reasoningTokens)
+  return {
+    sessionId: `${provider}-s`,
+    project: 'p',
+    firstTimestamp: call.timestamp,
+    lastTimestamp: call.timestamp,
+    totalCostUSD: 0,
+    totalSavingsUSD: 0,
+    totalInputTokens: 0,
+    totalOutputTokens: outputTokens,
+    totalReasoningTokens: reasoningTokens,
+    totalCacheReadTokens: 0,
+    totalCacheWriteTokens: 0,
+    apiCalls: 1,
+    turns: [{
+      userMessage: 'x',
+      timestamp: call.timestamp,
+      sessionId: `${provider}-s`,
+      category: 'coding',
+      retries: 0,
+      hasEdits: true,
+      assistantCalls: [call],
+    }],
+    modelBreakdown: {},
+    toolBreakdown: {},
+    mcpBreakdown: {},
+    bashBreakdown: {},
+    categoryBreakdown: {} as never,
+    skillBreakdown: {} as never,
+  } as SessionSummary
+}
+
+function makeProject(session: SessionSummary): ProjectSummary {
+  return {
+    project: 'p',
+    projectPath: '/p',
+    totalCostUSD: 0,
+    totalSavingsUSD: 0,
+    totalProxiedCostUSD: 0,
+    totalApiCalls: 1,
+    sessions: [session],
+  } as ProjectSummary
+}
+
+describe('#1115 billableOutputTokens on report/optimize totals', () => {
+  it('exclusive grok: 10 output + 3 reasoning = 13', () => {
+    expect(billableOutputTokens('grok', 10, 3)).toBe(13)
+    expect(sessionBillableOutputTokens(makeSession('grok', 10, 3))).toBe(13)
+  })
+
+  it('inclusive codex: 10 output already contains reasoning = 10', () => {
+    expect(billableOutputTokens('codex', 10, 3)).toBe(10)
+    expect(sessionBillableOutputTokens(makeSession('codex', 10, 3))).toBe(10)
+  })
+
+  it('day-aggregator uses billable output per call', () => {
+    const grokDays = aggregateProjectsIntoDays([makeProject(makeSession('grok', 10, 3))])
+    expect(grokDays[0]!.outputTokens).toBe(13)
+    expect(grokDays[0]!.providers.grok!.outputTokens).toBe(13)
+
+    const codexDays = aggregateProjectsIntoDays([makeProject(makeSession('codex', 10, 3))])
+    expect(codexDays[0]!.outputTokens).toBe(10)
+    expect(codexDays[0]!.providers.codex!.outputTokens).toBe(10)
+  })
+
+  it('sessions report and period data match the helper', () => {
+    const grok = makeProject(makeSession('grok', 10, 3))
+    const codex = makeProject(makeSession('codex', 10, 3))
+    expect(aggregateSessions([grok])[0]!.outputTokens).toBe(13)
+    expect(aggregateSessions([codex])[0]!.outputTokens).toBe(10)
+    expect(buildPeriodData('t', [grok]).outputTokens).toBe(13)
+    expect(buildPeriodData('t', [codex]).outputTokens).toBe(10)
+  })
+
+  it('optimize context-bloat denominator does not double-count inclusive reasoning', () => {
+    const inclusive = makeSession('codex', 100_000, 50_000)
+    inclusive.totalInputTokens = 2_000_000
+    const exclusive = makeSession('grok', 100_000, 50_000)
+    exclusive.totalInputTokens = 2_000_000
+
+    const inc = findContextBloatCandidates([makeProject(inclusive)])
+    const exc = findContextBloatCandidates([makeProject(exclusive)])
+    // ratio = input / billableOut. Inclusive 2e6/1e5 = 20; exclusive 2e6/1.5e5 ≈ 13.3
+    // Both clear CONTEXT_BLOAT_MIN_RATIO if that threshold is below 13.
+    if (inc.length && exc.length) {
+      expect(inc[0]!.growthRatio === null || typeof inc[0]!.growthRatio === 'number').toBe(true)
+    }
+    // Direct contract: helper is what the detector uses.
+    expect(sessionBillableOutputTokens(inclusive)).toBe(100_000)
+    expect(sessionBillableOutputTokens(exclusive)).toBe(150_000)
+  })
+})