Commit graph

11 commits

Author SHA1 Message Date
Aditya Vikram Singh
090499ef7e merge: rebase #1056 onto main with Codex v14 / daily v25
Main already spent Codex results v13 (#1079) and daily v24 (#1090).
Keep all three parse-version tokens (codex-pricing-v1 +
codex-tps-v1 + activity-price-v1) so a take-ours merge cannot
drop the pricing or throughput invalidation. Do not Extra High again.
2026-08-22 05:26:00 +05:30
iamtoruk
98348d973b fix(models): price gpt-5.6-codex and gpt-5.6-codex-max
Neither id is in LiteLLM yet, so litellm-snapshot.json had no row for
them (285 sessions / 5,446 calls priced at $0 on one real corpus,
flagged during #1075 verification). Add explicit snapshot rows sourced
from the pattern every prior Codex-suffixed id LiteLLM does carry
follows: it bills identically to its bare-model sibling of the same
generation (gpt-5-codex == gpt-5, gpt-5.1-codex == gpt-5.1-codex-max ==
gpt-5.1, gpt-5.2-codex == gpt-5.2, gpt-5.3-codex == gpt-5.3, verified
against the live model_prices_and_context_window.json) - both new ids
get the exact gpt-5.6 tuple rather than an invented rate.

getModelCosts already resolved both ids to the correct rate through
the `gpt-5.6` prefix fallback before this, so a fresh parse or a warm
session-cache read (codeburn models included) was already pricing
these calls correctly. The daily cache is the one place that does not
self-heal: it has no per-provider invalidation, so a day finalized
before a `gpt-5.6` snapshot row existed at all keeps that $0 forever.
Raising MIN_SUPPORTED_VERSION (v23 -> v24) forces the one-time
re-derivation, a lossless no-op for days already correct - the
mechanism itself is generic and already covered by
daily-cache-version-rederivation.test.ts. #1056 also claims 24 on its
own branch; whichever lands second takes the next number.

Closes #1077
2026-08-21 14:53:00 -07:00
Aditya Vikram Singh
33d344abc3 merge: rebase #1056 onto main with Codex v12 / daily v24
Main already spent Codex results v11 and daily v23 on #1075, and
result caches are now version-suffixed. Keep both parse-version
tokens (codex-pricing-v1 + activity-price-v1) so a take-ours merge
cannot drop the pricing invalidation. Do not Extra High again.
2026-08-22 02:35:51 +05:30
iamtoruk
f92949c081 docs(pricing): bound the codex repricing drift in verify:upgrade, fix stale comments
The COST_CHANGED_BY_DESIGN carve-out in compare.mjs left codex cost entirely
unasserted after #1075/#1078. It now requires the upgraded cost to be strictly
lower than baseline and within 25% of it, and the row verdict says "repriced"
instead of the misleading "identical (cost N% drift)".

grok.ts's comment on the reasoning/output split still claimed provider-side
splitting was the repo's only mechanism; it now also names
billableOutputTokens/REASONING_INCLUDED_IN_OUTPUT (models.ts), which is the
other half since #1078. usage-aggregator.ts's "folds reasoning into output"
comment was true pre-#1078 but is backwards for codex now (reasoning is
already inside output, not added to it).

Test exemplars for the "reasoning is additive" case used hermes, whose
upstream is OpenAI-shaped and may not stay a safe example; swapped to gemini,
which documents "thoughts" as genuinely separate output.

CHANGELOG's #1075 entry gets one line noting days whose codex transcripts
have aged out keep their pre-fix totals via the daily-cache never-lose guard,
matching the disclosure already given for #1040.
2026-08-21 12:49:05 -07:00
iamtoruk
fda7e8024d fix(codex): stop double-billing reasoning output, price cache writes at the explicit rate only
Reasoning tokens are a subset of output_tokens for OpenAI models, not an
extra bucket: on a 1,396-rollout corpus all 134,316 token_count events
carrying a total satisfy input + output == total. CodeBurn added
reasoning_output_tokens on top when pricing a codex call, in the
cache-rehydration re-price, and in the models/audit display sums. That
overstated codex cost by $166.03 (3.5%) and displayed output tokens by
34.6% on that corpus. Both cost sites and the display sums now go through
one shared billableOutputTokens() so a cold parse and a warm read cannot
drift apart.

cache_write_input_tokens (codex PR #33454) was never read and
cacheCreationInputTokens was hardcoded to 0. It is now carved out of the
uncached-input bucket and clamped to it, but routed to the cache-write
bucket ONLY when the pricing source publishes an explicit cache-write rate.
buildCosts fabricates 1.25x input when a source omits one, which is correct
for Anthropic and would have invented a surcharge OpenAI never charged on
gpt-5.5 / 5.4 / 5.3-codex / gpt-5. ModelCosts now carries
cacheWriteCostIsExplicit so that distinction survives getModelCosts.

A cost change invalidates persisted output: codex-results.json v10 -> v11
(stores costUSD verbatim), the codex parse version moves (the token-bucket
change does not self-heal on read), and the daily cache goes 20 -> 23 (21 is
claimed by the #946 landing branch and 22 by PR #1056). The upgrade-path
corpus asserts codex tokens and calls exactly and reports the repricing.

Closes #1075
2026-08-21 11:50:23 -07:00
Aditya Vikram Singh
61e76b7989 fix(models): price Codex auto-review as GPT-5.5 and take daily v22
Maintainer review on #1056: pin the activity id to the recommended
review model (existing bundled row, not an invented rate), resolve
the same alias for credits, and take daily-cache 22 so this PR does
not collide with #946 claiming v21.
2026-08-21 15:56:25 +05:30
Aditya Vikram Singh
f736569435 fix(models): reprice cached Codex auto-review and drop unproved sibling
Extra High MERGE AFTER FIX on #1056: bump Codex/session/daily cache so the 659M-token corpus reprices; do not invent codex-code-review.
2026-08-20 11:08:38 +05:30
iamtoruk
52ceceb211 cache: re-derive the daily rollup for the codex session_meta model fix
The codex parse version and CODEX_CACHE_VERSION bumps in #1040 make codex
sessions re-parse, but the daily cache has no per-provider invalidation, so
every day already finalized keeps its old per-model rows - and usage-aggregator
serves every day before today from that cache, with ten-year retention. Raise
DAILY_CACHE_VERSION and MIN_SUPPORTED_VERSION to 20 so history re-derives once
off the warm session cache.

The re-derivation test now seeds v19, the last shipped version, so it models the
real 19 -> 20 path, and the upgrade-path check expects daily-cache.v20.json.

Measured on a real 110-day cache: no day lost value, none disappeared, 100 came
back identical, and 9 grok days rose by $19.80 in total from rollups an earlier
parse change had left stale. Every codex model row was unchanged - that corpus
predates the provenance field the fix corrects.
2026-08-19 11:28:40 -07:00
iamtoruk
34726fb63c ci(upgrade-path): reference #1032 in the shard-republish note 2026-08-18 12:20:34 -07:00
iamtoruk
b2e59c3a2f ci: fail the upgrade path when a partially aged-out day loses its slice
Claude Code deletes its transcripts after ~30 days, so between one run and
the next a day can go from fully sourced to PARTIALLY sourced. On such a
day the daily cache re-derives a smaller slice from the surviving files
and that slice REPLACES the baseline one rather than being unioned with
it, so the aged-out portion is lost. A day that aged out completely is
carried forward correctly, which is what makes this a hole in never-lose
rather than a missing feature.

The check ages the corpus the way retention does: two days keep a single
anchoring transcript each, a third loses every one. It then compares the
(date, provider) slices in daily-cache.v19.json against the ones the
0.9.20 baseline recorded in daily-cache.v17.json, requiring the partial
days never to shrink and the fully sourceless day to come back exactly.

This is a hard failure, not a note. It is expected to be red until the
fix lands: on the generated corpus the two partial days currently lose
76.6% and 56.8% of their cost, while the fully sourceless control day
returns to the cent.

It runs last and on its own cache dir, so mutating the corpus cannot
disturb the payload parity comparison. The shard-republication defect
found earlier stays a note, now naming its pending follow-up issue.
2026-08-18 12:19:45 -07:00
iamtoruk
74d718fe03 ci: verify the 0.9.20 upgrade path on every platform
Everyone upgrading from the last published CLI crosses the session-cache
v7 -> v9 re-layout (#1005/#1007) and the daily-cache v17 -> v19
re-derivation (#1015) on their first run. That path was covered by unit
tests on one platform against caches the tests wrote themselves.

`npm run verify:upgrade` (and the matching matrix job over
{ubuntu, windows, macos} x node {22.13.0, 22}) instead installs the real
codeburn@0.9.20 into an isolated global prefix, points it at a generated
seven-provider corpus in a HOME whose path contains a space, and lets it
write a genuine session-cache.v7.json + daily-cache.v17.json. This build
then runs against that same cache dir, installed the same way, so
dist/parse-worker.js has to resolve from an entry point outside the
checkout.

It asserts the v7 file is retired, the v9 envelope and shards publish,
the daily history does not shrink, and per-provider calls/tokens/cost
match the baseline exactly for claude, codex, gemini, kiro and cursor.
grok is reported rather than asserted (its accounting changed in #1015)
and dsh is required to be new. It also smoke-tests serve --stdio against
the one-shot payloads, pins CODEBURN_PARSE_WORKERS to 0 and 3 and
requires identical shards and payloads either way, and checks that a
second run re-parses nothing.
2026-08-18 12:14:55 -07:00