Commit graph

3 commits

Author SHA1 Message Date
iamtoruk
f92949c081 docs(pricing): bound the codex repricing drift in verify:upgrade, fix stale comments
The COST_CHANGED_BY_DESIGN carve-out in compare.mjs left codex cost entirely
unasserted after #1075/#1078. It now requires the upgraded cost to be strictly
lower than baseline and within 25% of it, and the row verdict says "repriced"
instead of the misleading "identical (cost N% drift)".

grok.ts's comment on the reasoning/output split still claimed provider-side
splitting was the repo's only mechanism; it now also names
billableOutputTokens/REASONING_INCLUDED_IN_OUTPUT (models.ts), which is the
other half since #1078. usage-aggregator.ts's "folds reasoning into output"
comment was true pre-#1078 but is backwards for codex now (reasoning is
already inside output, not added to it).

Test exemplars for the "reasoning is additive" case used hermes, whose
upstream is OpenAI-shaped and may not stay a safe example; swapped to gemini,
which documents "thoughts" as genuinely separate output.

CHANGELOG's #1075 entry gets one line noting days whose codex transcripts
have aged out keep their pre-fix totals via the daily-cache never-lose guard,
matching the disclosure already given for #1040.
2026-08-21 12:49:05 -07:00
iamtoruk
fda7e8024d fix(codex): stop double-billing reasoning output, price cache writes at the explicit rate only
Reasoning tokens are a subset of output_tokens for OpenAI models, not an
extra bucket: on a 1,396-rollout corpus all 134,316 token_count events
carrying a total satisfy input + output == total. CodeBurn added
reasoning_output_tokens on top when pricing a codex call, in the
cache-rehydration re-price, and in the models/audit display sums. That
overstated codex cost by $166.03 (3.5%) and displayed output tokens by
34.6% on that corpus. Both cost sites and the display sums now go through
one shared billableOutputTokens() so a cold parse and a warm read cannot
drift apart.

cache_write_input_tokens (codex PR #33454) was never read and
cacheCreationInputTokens was hardcoded to 0. It is now carved out of the
uncached-input bucket and clamped to it, but routed to the cache-write
bucket ONLY when the pricing source publishes an explicit cache-write rate.
buildCosts fabricates 1.25x input when a source omits one, which is correct
for Anthropic and would have invented a surcharge OpenAI never charged on
gpt-5.5 / 5.4 / 5.3-codex / gpt-5. ModelCosts now carries
cacheWriteCostIsExplicit so that distinction survives getModelCosts.

A cost change invalidates persisted output: codex-results.json v10 -> v11
(stores costUSD verbatim), the codex parse version moves (the token-bucket
change does not self-heal on read), and the daily cache goes 20 -> 23 (21 is
claimed by the #946 landing branch and 22 by PR #1056). The upgrade-path
corpus asserts codex tokens and calls exactly and reports the repricing.

Closes #1075
2026-08-21 11:50:23 -07:00
iamtoruk
74d718fe03 ci: verify the 0.9.20 upgrade path on every platform
Everyone upgrading from the last published CLI crosses the session-cache
v7 -> v9 re-layout (#1005/#1007) and the daily-cache v17 -> v19
re-derivation (#1015) on their first run. That path was covered by unit
tests on one platform against caches the tests wrote themselves.

`npm run verify:upgrade` (and the matching matrix job over
{ubuntu, windows, macos} x node {22.13.0, 22}) instead installs the real
codeburn@0.9.20 into an isolated global prefix, points it at a generated
seven-provider corpus in a HOME whose path contains a space, and lets it
write a genuine session-cache.v7.json + daily-cache.v17.json. This build
then runs against that same cache dir, installed the same way, so
dist/parse-worker.js has to resolve from an entry point outside the
checkout.

It asserts the v7 file is retired, the v9 envelope and shards publish,
the daily history does not shrink, and per-provider calls/tokens/cost
match the baseline exactly for claude, codex, gemini, kiro and cursor.
grok is reported rather than asserted (its accounting changed in #1015)
and dsh is required to be new. It also smoke-tests serve --stdio against
the one-shot payloads, pins CODEBURN_PARSE_WORKERS to 0 and 3 and
requires identical shards and payloads either way, and checks that a
second run re-parses nothing.
2026-08-18 12:14:55 -07:00