diff --git a/CHANGELOG.md b/CHANGELOG.md index 407cffca..f1b9bad0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,7 @@ - **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972) ### Fixed +- **Codex spend no longer counts reasoning tokens twice, and cache writes are priced only where OpenAI actually charges for them.** OpenAI bills reasoning tokens as *part of* `output_tokens`, not on top of it — on a 1,396-rollout corpus all 134,316 events carrying a total satisfy `input + output == total` — but CodeBurn added `reasoning_output_tokens` to output when pricing a Codex call and again in the models, audit and per-model displays. Every Codex number was therefore too high: on that corpus **cost by $166.03 (3.5%)** and **displayed Output tokens by 34.6%** ($4,713.12 -> $4,547.09; 22.6M -> 16.8M output tokens). The raw `reasoningTokens` figure is unchanged and still reported on its own; only the double-count is gone. Both places that price a Codex call — the parser and the cache-rehydration re-price — now go through one shared `billableOutputTokens` helper, so a cold run and a warm run can never disagree. Separately, Codex's `cache_write_input_tokens` (new in codex PR #33454) was never read and cache-creation tokens were hardcoded to 0; they are now carved out of the uncached-input bucket and clamped so they can never exceed it. That carve-out happens **only on models whose pricing source publishes a real cache-write rate** — gpt-5.6 and its terra/sol/luna variants charge 1.25x input for a cache write, everything before it charges nothing extra — because CodeBurn fabricates a 1.25x rate when a source omits one, and charging that would have invented a surcharge on gpt-5.5, gpt-5.4, gpt-5.3-codex and gpt-5. On models without an explicit rate the tokens stay in the plain input bucket and the price is unchanged to the cent. The field is new enough that today's impact is $0 on that corpus. Codex sessions re-parse once and the daily cache re-derives once off the warm session cache (a global re-derivation of every day and every provider, since it has no per-provider invalidation); no other provider's numbers move. Long-context pricing tiers from the same report are tracked separately in #1076 and the missing `gpt-5.6-codex` snapshot rows in #1077. Thanks @chr-evensen. (#1075) - **Codex calls attributed from session metadata no longer carry a stale model.** The Buffer fast path scanned `session_meta` for the first `"model"` string anywhere in the payload, so a nested `base_instructions.provenance.model` was read as if it were `payload.model` — and since the model is last-writer-wins state, that wrong value was credited to every call before the rollout's first `turn_context` and to every call after any mid-file `session_meta` (29 of 1380 rollouts on one real corpus carry a late `session_meta`, and 57 record usage before any `turn_context`). Direct payload fields are now read depth-aware, which is what the non-fast `JSON.parse` path always did. Codex sessions re-parse once (~9s on a 4 GB rollout corpus) and the daily cache re-derives once off the warm session cache, a global re-derivation of every day and every provider since it has no per-provider invalidation; it moves per-model attribution, and clears any rollup an earlier parse change had left stale. Days whose transcripts have partly aged out are held by the never-lose guard: on a real 110-day cache no day lost value and none disappeared — 100 days came back identical and 9 grok days rose by $19.80 in total. Thanks @timdp. (#1040) - **Codex `session_meta` cwd / session id / originator follow the same depth-1 window as `model`.** #1040 fixed nested `provenance.model`; the compact Buffer path still took the first `cwd`, `session_id`, `originator`, `name`, `forked_from_id` or `model_provider` anywhere in the payload, so a `dynamic_tools[].name` (or any same-named nested key) could steal the top-level field. Those strings now use the existing payload-depth-1 scan. Function-call `name` on other event types is unchanged. Codex sessions re-parse once. (#1045) - **Plan rows for sticker-price presets read as a budget instead of live provider quota.** There is no Grok quota endpoint, so a SuperGrok row was parsed API-equivalent spend divided by the plan's sticker price on a monthly reset — but the TUI labelled that math "plan" and "reset", which next to a client showing xAI's real weekly window read as CodeBurn being wrong. The bars and the arithmetic are unchanged; the words are not. Both the dashboard and the desktop app now say the number is an API-equivalent monthly budget and not a live provider window, in the same wording on both surfaces, and for every preset rather than as a SuperGrok special case. The window is anniversary-based (`plan.resetDay`, settable with `codeburn plan set --reset-day`), so it is called a budget reset rather than a calendar one. The row was also shortened to fit 80 columns: at that width the percentage and the projected month were being truncated away, including on custom plans, whose label carries the provider. diff --git a/scripts/upgrade-path/compare.mjs b/scripts/upgrade-path/compare.mjs index 0d0ef49a..46b01c5c 100644 --- a/scripts/upgrade-path/compare.mjs +++ b/scripts/upgrade-path/compare.mjs @@ -22,8 +22,15 @@ // which is the part the corpus can honestly establish. // dsh did not exist in the published CLI. Reported; required to be absent // in the baseline and present after the upgrade. +// codex PRICING changed by design in #1075: reasoning tokens are billed +// inside output rather than on top of it, and cache writes are carved out +// of the input bucket. Nothing about what was PARSED moved, so codex keeps +// the full exact treatment for the call count and every token field; only +// the cost tolerance is lifted, and the delta is reported instead. Drop it +// from this list once a published CLI carries the fix. const EXACT = ['claude', 'codex', 'gemini', 'kiro', 'cursor'] const CHANGED_BY_DESIGN = ['grok'] +const COST_CHANGED_BY_DESIGN = ['codex'] const NEW_IN_THIS_RELEASE = ['dsh'] const COST_TOLERANCE = 0.005 // 0.5% relative @@ -95,7 +102,11 @@ for (const name of providers) { if (b.calls !== u.calls) diffs.push(`calls ${b.calls} != ${u.calls}`) for (const f of TOKEN_FIELDS) if (b[f] !== u[f]) diffs.push(`${f} ${b[f]} != ${u[f]}`) const costDrift = relDiff(b.cost, u.cost) - if (costDrift > COST_TOLERANCE) diffs.push(`cost ${fmt(b.cost)} != ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(COST_TOLERANCE * 100).toFixed(1)}%)`) + if (COST_CHANGED_BY_DESIGN.includes(name)) { + notes.push(`${name}: cost ${fmt(b.cost)} -> ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}%) — repricing expected (#1075); tokens and calls still asserted exactly`) + } else if (costDrift > COST_TOLERANCE) { + diffs.push(`cost ${fmt(b.cost)} != ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(COST_TOLERANCE * 100).toFixed(1)}%)`) + } if (!EXACT.includes(name)) { notes.push(`${name}: no expectation declared in compare.mjs; ${diffs.length ? diffs.join(', ') : 'identical'}`) verdict = diffs.length ? 'differs (unclassified)' : 'identical' diff --git a/scripts/upgrade-path/run.mjs b/scripts/upgrade-path/run.mjs index cab81a95..586becca 100644 --- a/scripts/upgrade-path/run.mjs +++ b/scripts/upgrade-path/run.mjs @@ -33,7 +33,7 @@ const WORK = process.env['UPGRADE_PATH_WORK'] || join(tmpdir(), 'codeburn upgrad const OLD_SESSION_CACHE = 'session-cache.v7.json' const OLD_DAILY_CACHE = 'daily-cache.v17.json' const NEW_SESSION_CACHE_DIR = 'session-cache.v9' -const NEW_DAILY_CACHE = 'daily-cache.v20.json' +const NEW_DAILY_CACHE = 'daily-cache.v23.json' const HOME = join(WORK, 'user home') const PAYLOADS = join(WORK, 'payloads') diff --git a/src/audit-report.ts b/src/audit-report.ts index 7a40c5c7..c3c4f251 100644 --- a/src/audit-report.ts +++ b/src/audit-report.ts @@ -1,4 +1,4 @@ -import { getModelCosts, sanitizeModelForDisplay, type ModelCosts } from './models.js' +import { billableOutputTokens, getModelCosts, sanitizeModelForDisplay, type ModelCosts } from './models.js' import { getProvider } from './providers/index.js' import { formatCost, formatTokens } from './format.js' import { renderTable, type TableColumn } from './text-table.js' @@ -124,7 +124,7 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise, capture?: { let prevCumulativeTotal: number | null = resume?.state.prevCumulativeTotal ?? null let prevInput = resume?.state.prevInput ?? 0 let prevCached = resume?.state.prevCached ?? 0 + let prevCacheWrite = resume?.state.prevCacheWrite ?? 0 let prevOutput = resume?.state.prevOutput ?? 0 let prevReasoning = resume?.state.prevReasoning ?? 0 let pendingTools: string[] = resume ? [...resume.state.pendingTools] : [] @@ -795,6 +803,7 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { prevCumulativeTotal, prevInput, prevCached, + prevCacheWrite, prevOutput, prevReasoning, pendingTools: [...pendingTools], @@ -1014,12 +1023,14 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { const last = info.last_token_usage let inputTokens = 0 let cachedInputTokens = 0 + let cacheWriteTokens = 0 let outputTokens = 0 let reasoningTokens = 0 if (last) { inputTokens = last.input_tokens ?? 0 cachedInputTokens = last.cached_input_tokens ?? 0 + cacheWriteTokens = last.cache_write_input_tokens ?? 0 outputTokens = last.output_tokens ?? 0 reasoningTokens = last.reasoning_output_tokens ?? 0 } else if (cumulativeTotal > 0) { @@ -1027,6 +1038,7 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { if (!total) continue inputTokens = (total.input_tokens ?? 0) - prevInput cachedInputTokens = (total.cached_input_tokens ?? 0) - prevCached + cacheWriteTokens = (total.cache_write_input_tokens ?? 0) - prevCacheWrite outputTokens = (total.output_tokens ?? 0) - prevOutput reasoningTokens = (total.reasoning_output_tokens ?? 0) - prevReasoning } @@ -1042,6 +1054,7 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { if (total) { prevInput = total.input_tokens ?? 0 prevCached = total.cached_input_tokens ?? 0 + prevCacheWrite = total.cache_write_input_tokens ?? 0 prevOutput = total.output_tokens ?? 0 prevReasoning = total.reasoning_output_tokens ?? 0 } @@ -1053,7 +1066,22 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { // Normalize to Anthropic semantics: inputTokens = non-cached only. const uncachedInputTokens = Math.max(0, inputTokens - cachedInputTokens) + // Cache writes are carved out of the uncached input, never added to + // it: clamp so a malformed or lagging count can never drive the plain + // input bucket negative. + const cacheWriteInputTokens = Math.max(0, Math.min(cacheWriteTokens, uncachedInputTokens)) + const model = resolveModel(entry.payload, sessionModel) + // Only move tokens into the cache-write bucket when the pricing + // source publishes a real cache-write rate for this model (gpt-5.6+ + // charges 1.25x input; everything before it charges nothing extra). + // Otherwise buildCosts' fabricated 1.25x default would invent a + // surcharge that OpenAI never billed, so the tokens stay where they + // already were -- in plain input, priced exactly as before. + const billedCacheWriteTokens = cacheWriteInputTokens > 0 && getModelCosts(model)?.cacheWriteCostIsExplicit + ? cacheWriteInputTokens + : 0 + const billedInputTokens = uncachedInputTokens - billedCacheWriteTokens const timestamp = entry.timestamp ?? '' // Forked sessions copy the parent's entire token_count history // (re-timestamped), so replays must collide with the parent's events @@ -1074,11 +1102,15 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { if (seenKeys.has(dedupKey)) continue seenKeys.add(dedupKey) + // Reasoning tokens are already inside output_tokens, so they are NOT + // added here. The cache-rehydration twin of this line lives in + // src/parser.ts (cachedCallToApiCall); both call billableOutputTokens + // so a fresh parse and a cache read can never price differently. const costUSD = calculateCost( model, - uncachedInputTokens, - outputTokens + reasoningTokens, - 0, + billedInputTokens, + billableOutputTokens('codex', outputTokens, reasoningTokens), + billedCacheWriteTokens, cachedInputTokens, 0, ) @@ -1086,9 +1118,9 @@ function createParser(source: SessionSource, seenKeys: Set, capture?: { pendingTaskCalls.push({ provider: 'codex', model, - inputTokens: uncachedInputTokens, + inputTokens: billedInputTokens, outputTokens, - cacheCreationInputTokens: 0, + cacheCreationInputTokens: billedCacheWriteTokens, cacheReadInputTokens: cachedInputTokens, cachedInputTokens, reasoningTokens, diff --git a/src/session-cache.ts b/src/session-cache.ts index f165652e..d8c73f27 100644 --- a/src/session-cache.ts +++ b/src/session-cache.ts @@ -281,7 +281,12 @@ export const PROVIDER_PARSE_VERSIONS: Record = { // nested base_instructions provenance.model cannot overwrite turn_context. // session-meta-fields-v1: the same depth-1 window for cwd/name/originator/ // session_id/forked_from_id/model_provider, not just model. - codex: 'mcp-attribution-v5-est-cost-active-timing-mcp-wait-rich-capture-v1-cross-provider-pr-v1-session-meta-model-v1-session-meta-fields-v1', + // codex-pricing-v1 (#1075): reasoning tokens are no longer added on top of + // output, and cache_write_input_tokens moves out of the plain input bucket on + // models with an explicit cache-write rate. The bucket move does NOT self-heal + // on read (cached entries store the buckets, not the raw event), so cached + // sessions must re-parse. + codex: 'mcp-attribution-v5-est-cost-active-timing-mcp-wait-rich-capture-v1-cross-provider-pr-v1-session-meta-model-v1-session-meta-fields-v1-codex-pricing-v1', cursor: 'composer-anchored-crediting-v1-est-cost', 'cursor-agent': 'workspaceless-transcript-v1', // source-provenance-v1 (#944): CLI sessions were misread as VS Code diff --git a/tests/audit-report.test.ts b/tests/audit-report.test.ts index 1723712c..baf89440 100644 --- a/tests/audit-report.test.ts +++ b/tests/audit-report.test.ts @@ -86,8 +86,11 @@ describe('aggregateAudit', () => { expect(r.raw.reasoningTokens).toBe(10) expect(r.raw.cacheReadInputTokens).toBe(200) expect(r.raw.cachedInputTokens).toBe(300) - // reasoning folds into output for pricing - expect(r.displayed.outputTokens).toBe(110) + // Reasoning does NOT fold into output for claude or codex: both bill it + // as part of output_tokens already, so adding it would double-count + // (#1075). Providers that report reasoning as a separate bucket still get + // the additive treatment - see tests/codex-pricing-1075.test.ts. + expect(r.displayed.outputTokens).toBe(100) // cache read is the SUM of per-call max(anthropic, openai), not max of sums expect(r.displayed.cacheReadTokens).toBe(500) // attributed cost is preserved exactly diff --git a/tests/codex-pricing-1075-rehydrate.test.ts b/tests/codex-pricing-1075-rehydrate.test.ts new file mode 100644 index 00000000..4b112347 --- /dev/null +++ b/tests/codex-pricing-1075-rehydrate.test.ts @@ -0,0 +1,69 @@ +// #1075, cost site 2 of 2. Codex is NOT on parser.ts's reported-cost +// pass-through allowlist, so the session cache stores its calls with +// `costUSD: undefined` and every warm run re-prices them from the stored token +// buckets in cachedCallToApiCall. That line and the one in the codex provider +// are twins: if only one drops the reasoning double-count, a user's number +// changes between a cold and a warm run. This drives the full parseAllSessions +// pipeline twice against the same file to prove they agree. +// +// Own file because the codex provider captures CODEX_HOME when its module is +// first evaluated, so the env must be set before any import of it. + +import { afterAll, beforeEach, expect, it, vi } from 'vitest' +import { mkdir, rm, writeFile } from 'fs/promises' +import { join } from 'path' + +const testRoot = vi.hoisted(() => { + const root = `${process.env['TMPDIR'] || '/tmp'}/codex-1075-rehydrate-${process.pid}-${Date.now()}` + process.env['HOME'] = `${root}/home` + process.env['USERPROFILE'] = `${root}/home` + process.env['CODEX_HOME'] = `${root}/codex` + return root +}) + +const CODEX_HOME = join(testRoot, 'codex') +const CACHE_DIR = join(testRoot, 'cache') + +// gpt-5.5: input 5e-6, output 30e-6, cacheRead 5e-7 (src/data/litellm-snapshot.json). +// 800 uncached input + 200 cached + 1000 output, of which 400 are reasoning. +const EXPECTED = 800 * 5e-6 + 200 * 5e-7 + 1000 * 30e-6 + +beforeEach(() => { + process.env['HOME'] = join(testRoot, 'home') + process.env['USERPROFILE'] = join(testRoot, 'home') + process.env['CODEX_HOME'] = CODEX_HOME + process.env['CODEBURN_CACHE_DIR'] = CACHE_DIR +}) + +afterAll(async () => { + await rm(testRoot, { recursive: true, force: true }) +}) + +it('prices a codex call the same on a cold parse and a cache-rehydrated read', async () => { + const sessionDir = join(CODEX_HOME, 'sessions', '2026', '08', '16') + await mkdir(sessionDir, { recursive: true }) + await mkdir(CACHE_DIR, { recursive: true }) + const usage = { input_tokens: 1000, cached_input_tokens: 200, output_tokens: 1000, reasoning_output_tokens: 400, total_tokens: 2000 } + await writeFile(join(sessionDir, 'rollout-1075.jsonl'), [ + JSON.stringify({ type: 'session_meta', timestamp: '2026-08-16T10:00:00Z', payload: { session_id: 's1075', model: 'gpt-5.5', cwd: '/Users/test/proj', originator: 'codex_cli_rs' } }), + JSON.stringify({ type: 'response_item', timestamp: '2026-08-16T10:00:10Z', payload: { type: 'message', role: 'user', content: [{ type: 'input_text', text: 'hello' }] } }), + JSON.stringify({ type: 'event_msg', timestamp: '2026-08-16T10:01:00Z', payload: { type: 'token_count', info: { model: 'gpt-5.5', last_token_usage: usage, total_token_usage: usage } } }), + ].join('\n') + '\n') + + const { clearSessionCache, parseAllSessions } = await import('../src/parser.js') + + clearSessionCache() + const cold = await parseAllSessions(undefined, 'codex') + const coldCost = cold.reduce((sum, p) => sum + p.totalCostUSD, 0) + + // Drop the in-memory cache only: session-cache.json on disk now serves the + // unchanged file, so this run's cost comes out of cachedCallToApiCall. + clearSessionCache() + const warm = await parseAllSessions(undefined, 'codex') + const warmCost = warm.reduce((sum, p) => sum + p.totalCostUSD, 0) + + // Revert only src/providers/codex.ts and the cold leg breaks; revert only + // src/parser.ts's outputForCost and the warm leg breaks. + expect(coldCost).toBeCloseTo(EXPECTED, 12) + expect(warmCost).toBeCloseTo(EXPECTED, 12) +}) diff --git a/tests/codex-pricing-1075.test.ts b/tests/codex-pricing-1075.test.ts new file mode 100644 index 00000000..9e1b5eb6 --- /dev/null +++ b/tests/codex-pricing-1075.test.ts @@ -0,0 +1,352 @@ +// Regression suite for #1075 (reported by chr-evensen). +// +// Two independent codex pricing bugs, each with the site that would silently +// drift from its twin if only one half were reverted: +// +// A. reasoning_output_tokens is a SUBSET of output_tokens (OpenAI bills +// reasoning as part of output; every token_count event in a 134k-event +// corpus satisfies input + output == total), but codeburn added the two. +// Priced in TWO places -- the fresh parse in src/providers/codex.ts and +// the cache-rehydration re-price in src/parser.ts -- plus three display +// sums. Both cost sites now go through billableOutputTokens(). The +// cache-rehydration half lives in codex-pricing-1075-rehydrate.test.ts, +// which needs CODEX_HOME set before the provider module is evaluated. +// +// B. cache_write_input_tokens was never read. It is now carved out of the +// uncached-input bucket, but ONLY on models whose pricing source carries +// an explicit cache-write rate: buildCosts() fabricates 1.25x input when +// the source omits one, which is right for Anthropic but would invent a +// surcharge OpenAI never charged on every pre-5.6 model. + +import { mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises' +import { tmpdir } from 'os' +import { join } from 'path' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' + +import { aggregateAudit } from '../src/audit-report.js' +import { aggregateModels } from '../src/models-report.js' +import { clearCodexMemCaches, readCachedCodexResults } from '../src/codex-cache.js' +import { currentTzKey, ensureCacheHydrated, toDateString, type DailyEntry } from '../src/daily-cache.js' +import { createCodexProvider } from '../src/providers/codex.js' +import type { ParsedProviderCall } from '../src/providers/types.js' +import type { + ClassifiedTurn, + ParsedApiCall, + ProjectSummary, + SessionSummary, + TaskCategory, + TokenUsage, +} from '../src/types.js' + +// Snapshot ground truth (src/data/litellm-snapshot.json), USD per token: +// gpt-5.6-terra input 2e-6 output 12e-6 cacheWrite 2.5e-6 (EXPLICIT) cacheRead 2e-7 +// gpt-5.5 input 5e-6 output 30e-6 cacheWrite null (fabricated) cacheRead 5e-7 +const TERRA = { input: 2e-6, output: 12e-6, cacheWrite: 2.5e-6, cacheRead: 2e-7 } +const GPT55 = { input: 5e-6, output: 30e-6, cacheRead: 5e-7 } + +let tmpDir: string +beforeEach(async () => { tmpDir = await mkdtemp(join(tmpdir(), 'codex-1075-')) }) +afterEach(async () => { await rm(tmpDir, { recursive: true, force: true }) }) + +type Usage = { + input_tokens: number + cached_input_tokens?: number + cache_write_input_tokens?: number + output_tokens: number + reasoning_output_tokens?: number +} + +async function parseOneEvent(model: string, usage: Usage): Promise { + const total = usage.input_tokens + usage.output_tokens + const sessionDir = join(tmpDir, 'sessions', '2026', '08', '16') + await mkdir(sessionDir, { recursive: true }) + const filePath = join(sessionDir, `rollout-${model}-${Math.random().toString(36).slice(2)}.jsonl`) + await writeFile(filePath, [ + JSON.stringify({ + type: 'session_meta', + timestamp: '2026-08-16T10:00:00Z', + payload: { cwd: '/Users/t/p', originator: 'codex-cli', session_id: 's1075', model }, + }), + JSON.stringify({ + type: 'event_msg', + timestamp: '2026-08-16T10:01:00Z', + payload: { + type: 'token_count', + info: { model, last_token_usage: { ...usage, total_tokens: total }, total_token_usage: { ...usage, total_tokens: total } }, + }, + }), + ].join('\n') + '\n') + + const provider = createCodexProvider(tmpDir) + const parser = provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set()) + const calls: ParsedProviderCall[] = [] + for await (const call of parser.parse()) calls.push(call) + expect(calls).toHaveLength(1) + return calls[0]! +} + +// ── Fix A: reasoning is already inside output ───────────────────────────── + +describe('#1075 A - reasoning is not billed on top of output', () => { + it('prices a fresh codex parse from output_tokens alone', async () => { + const call = await parseOneEvent('gpt-5.5', { + input_tokens: 1000, + cached_input_tokens: 200, + output_tokens: 1000, + reasoning_output_tokens: 400, + }) + + // 800 uncached input + 200 cached + 1000 output. The 400 reasoning tokens + // are INSIDE the 1000, so they must not be priced again. + const expected = 800 * GPT55.input + 200 * GPT55.cacheRead + 1000 * GPT55.output + expect(call.costUSD).toBeCloseTo(expected, 12) + // Guard the direction: the pre-fix arithmetic charged 1400 output tokens. + const preFix = 800 * GPT55.input + 200 * GPT55.cacheRead + 1400 * GPT55.output + expect(call.costUSD).toBeLessThan(preFix) + // The raw fields are still reported untouched; only the pricing changed. + expect(call.outputTokens).toBe(1000) + expect(call.reasoningTokens).toBe(400) + }) + + it('does not double-count reasoning in the displayed output tokens', async () => { + const codex = makeApiCall('codex', 'gpt-5.5', { outputTokens: 1000, reasoningTokens: 400 }) + // A provider that really does report reasoning as a separate bucket keeps + // the additive behaviour, so this is a codex carve-out and not a blanket + // change to every display sum. + const additive = makeApiCall('hermes', 'gpt-5.5', { outputTokens: 1000, reasoningTokens: 400 }) + const projects = [makeProject([codex, additive])] + + const auditRows = await aggregateAudit(projects) + expect(auditRows.find(r => r.provider === 'codex')!.displayed.outputTokens).toBe(1000) + expect(auditRows.find(r => r.provider === 'hermes')!.displayed.outputTokens).toBe(1400) + + const modelRows = await aggregateModels(projects) + expect(modelRows.find(r => r.provider === 'codex')!.outputTokens).toBe(1000) + expect(modelRows.find(r => r.provider === 'hermes')!.outputTokens).toBe(1400) + }) +}) + +// ── Fix B: cache_write_input_tokens, guarded ────────────────────────────── + +describe('#1075 B - cache_write_input_tokens', () => { + it('prices cache writes at the explicit rate on gpt-5.6-terra', async () => { + const call = await parseOneEvent('gpt-5.6-terra', { + input_tokens: 1000, + cached_input_tokens: 200, + cache_write_input_tokens: 300, + output_tokens: 100, + }) + + expect(call.inputTokens).toBe(500) + expect(call.cacheCreationInputTokens).toBe(300) + expect(call.cacheReadInputTokens).toBe(200) + const expected = + 500 * TERRA.input + + 300 * TERRA.cacheWrite + + 200 * TERRA.cacheRead + + 100 * TERRA.output + expect(expected).toBeCloseTo(0.00299, 12) + expect(call.costUSD).toBeCloseTo(expected, 12) + }) + + it('THE GUARD: leaves cache writes in the input bucket when the model has no explicit rate', async () => { + // gpt-5.5 carries `null` for cache_creation_input_token_cost, so + // buildCosts fabricates 1.25x input for it. OpenAI charges nothing extra + // to write cache before gpt-5.6, so routing these tokens through that + // fabricated rate would invent a surcharge. Cost must be byte-identical to + // the pre-fix number. Delete the guard and this test fails. + const withWrite = await parseOneEvent('gpt-5.5', { + input_tokens: 1000, + cached_input_tokens: 200, + cache_write_input_tokens: 300, + output_tokens: 100, + }) + const withoutWrite = await parseOneEvent('gpt-5.5', { + input_tokens: 1000, + cached_input_tokens: 200, + output_tokens: 100, + }) + + expect(withWrite.inputTokens).toBe(800) + expect(withWrite.cacheCreationInputTokens).toBe(0) + const expected = 800 * GPT55.input + 200 * GPT55.cacheRead + 100 * GPT55.output + expect(withWrite.costUSD).toBeCloseTo(expected, 12) + expect(withWrite.costUSD).toBeCloseTo(withoutWrite.costUSD, 12) + // The fabricated rate is 1.25 x 5e-6; make sure not a cent of it landed. + expect(withWrite.costUSD).toBeLessThan(expected + 300 * GPT55.input * 1.25) + }) + + it('clamps a cache-write count larger than the uncached input', async () => { + const call = await parseOneEvent('gpt-5.6-terra', { + input_tokens: 1000, + cached_input_tokens: 200, + cache_write_input_tokens: 5000, + output_tokens: 100, + }) + + expect(call.inputTokens).toBe(0) + expect(call.cacheCreationInputTokens).toBe(800) + expect(call.costUSD).toBeCloseTo(800 * TERRA.cacheWrite + 200 * TERRA.cacheRead + 100 * TERRA.output, 12) + }) +}) + +// ── Cache invalidation: a cost change must not be served from stale bytes ── + +describe('#1075 cache invalidation', () => { + it('discards a v10 codex results cache (it stores costUSD verbatim)', async () => { + const cacheDir = join(tmpDir, 'cache') + await mkdir(cacheDir, { recursive: true }) + const sessionFile = join(tmpDir, 'rollout-stale.jsonl') + await writeFile(sessionFile, '{}\n') + + const { statSync } = await import('fs') + const s = statSync(sessionFile) + const stale: ParsedProviderCall = { + provider: 'codex', + model: 'gpt-5.5', + inputTokens: 800, + outputTokens: 1000, + cacheCreationInputTokens: 0, + cacheReadInputTokens: 200, + cachedInputTokens: 200, + reasoningTokens: 400, + webSearchRequests: 0, + costUSD: 0.0445, // the pre-fix, reasoning-double-counted number + tools: [], + bashCommands: [], + timestamp: '2026-08-16T10:01:00Z', + speed: 'standard', + deduplicationKey: 'codex:stale', + } + await writeFile(join(cacheDir, 'codex-results.json'), JSON.stringify({ + version: 10, + files: { [sessionFile]: { dev: s.dev, ino: s.ino, mtimeMs: s.mtimeMs, sizeBytes: s.size, project: 'p', calls: [stale] } }, + })) + + const prevCacheDir = process.env['CODEBURN_CACHE_DIR'] + process.env['CODEBURN_CACHE_DIR'] = cacheDir + try { + clearCodexMemCaches() + // Revert CODEX_CACHE_VERSION to 10 and this returns the stale $0.0445 call. + expect(await readCachedCodexResults(sessionFile)).toBeNull() + } finally { + if (prevCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']; else process.env['CODEBURN_CACHE_DIR'] = prevCacheDir + } + }) + + it('re-derives days finalized at daily-cache v20', async () => { + const cacheRoot = join(tmpDir, 'daily') + await mkdir(cacheRoot, { recursive: true }) + const prevCacheDir = process.env['CODEBURN_CACHE_DIR'] + process.env['CODEBURN_CACHE_DIR'] = cacheRoot + try { + const date = toDateString(new Date(Date.now() - 7 * 24 * 60 * 60 * 1000)) + const yesterday = toDateString(new Date(Date.now() - 24 * 60 * 60 * 1000)) + const oldPath = join(cacheRoot, 'daily-cache.v20.json') + const oldCache = { + version: 20, + savingsConfigHash: 'cfg', + tzKey: currentTzKey(), + lastComputedDate: yesterday, + days: [codexDay(date, 99)], + complete: true, + watermarkTrusted: true, + } + await writeFile(oldPath, JSON.stringify(oldCache)) + + let parseCount = 0 + const hydrated = await ensureCacheHydrated( + async () => { parseCount++; return [] }, + () => [codexDay(date, 2)], + 'cfg', + () => true, + ) + + // Drop MIN_SUPPORTED_VERSION back to 20 and the v20 day is trusted as-is, + // so parseCount stays 0 and the day keeps its overstated $99. + expect(parseCount).toBe(1) + expect(hydrated.days.find(d => d.date === date)?.cost).toBe(2) + expect(JSON.parse(await readFile(oldPath, 'utf8'))).toEqual(oldCache) + } finally { + if (prevCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']; else process.env['CODEBURN_CACHE_DIR'] = prevCacheDir + } + }) +}) + +// ── fixtures ────────────────────────────────────────────────────────────── + +function makeApiCall(provider: string, model: string, usage: Partial): ParsedApiCall { + return { + provider, + model, + usage: { + inputTokens: 0, + outputTokens: 0, + cacheCreationInputTokens: 0, + cacheReadInputTokens: 0, + cachedInputTokens: 0, + reasoningTokens: 0, + webSearchRequests: 0, + ...usage, + }, + costUSD: 0, + tools: [], + mcpTools: [], + skills: [], + hasAgentSpawn: false, + hasPlanMode: false, + speed: 'standard', + timestamp: '2026-08-16T00:00:00.000Z', + bashCommands: [], + deduplicationKey: `${provider}-${model}`, + } +} + +function makeProject(calls: ParsedApiCall[]): ProjectSummary { + const turn: ClassifiedTurn = { + userMessage: 't', + assistantCalls: calls, + timestamp: '2026-08-16T00:00:00.000Z', + sessionId: 's1', + category: 'feature' as TaskCategory, + retries: 0, + hasEdits: false, + } + const session: SessionSummary = { + sessionId: 's1', + project: 'p', + firstTimestamp: '2026-08-16T00:00:00.000Z', + lastTimestamp: '2026-08-16T00:00:00.000Z', + totalCostUSD: 0, + totalInputTokens: 0, + totalOutputTokens: 0, + totalCacheReadTokens: 0, + totalCacheWriteTokens: 0, + apiCalls: 0, + turns: [turn], + modelBreakdown: {}, + toolBreakdown: {}, + mcpBreakdown: {}, + bashBreakdown: {}, + categoryBreakdown: {} as SessionSummary['categoryBreakdown'], + skillBreakdown: {}, + } + return { project: 'p', projectPath: 'p', sessions: [session], totalCostUSD: 0, totalApiCalls: 0 } +} + +function codexDay(date: string, cost: number): DailyEntry { + const tokens = { inputTokens: 100, outputTokens: 20, cacheReadTokens: 30, cacheWriteTokens: 0 } + return { + date, + cost, + savingsUSD: 0, + calls: 1, + sessions: 1, + ...tokens, + editTurns: 0, + oneShotTurns: 0, + models: { 'GPT-5.5': { calls: 1, cost, savingsUSD: 0, ...tokens } }, + categories: {}, + providers: { codex: { calls: 1, cost, savingsUSD: 0, sessions: 1, ...tokens } }, + } +} diff --git a/tests/models-report.test.ts b/tests/models-report.test.ts index 4e1dba09..dac721f9 100644 --- a/tests/models-report.test.ts +++ b/tests/models-report.test.ts @@ -237,11 +237,14 @@ describe('aggregateModels', () => { expect(above.find(r => r.provider === 'cursor')).toBeUndefined() }) + // Providers that report reasoning as a bucket SEPARATE from output still get + // it added in. Codex and claude do not - they bill reasoning inside + // output_tokens - and that carve-out is covered in codex-pricing-1075.test.ts. it('counts reasoning tokens as output tokens', async () => { const project = makeProject([ makeTurn('feature', [ { - provider: 'codex', + provider: 'hermes', model: 'gpt-5', usage: { ...emptyTokens(), inputTokens: 100, outputTokens: 50, reasoningTokens: 200 }, costUSD: 1.0,