mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-08-25 00:14:40 +00:00
Merge origin/main into feat/copilot-session-store
Resolves the conflicts #946 accumulated while it was in validation. Eight
files conflicted; the session-store accounting is unchanged.
src/daily-cache.ts — version collision. This PR minted 25 when main was at
24; #1056 (`codex-auto-review` pricing) then spent 25 on main. The bump moves
to 26/MIN 26 and daily-cache.v26.json, with main's full comment ladder kept as
the foundation and this PR's paragraph rewritten to name 26 and record the
collision. PENDING_REDERIVE_PROVIDERS and the B1 migration semantics from
b6481c19 carry over intact, retargeted at 26.
src/models.ts, src/parser.ts, src/audit-report.ts, src/models-report.ts —
#1075/#1078 replaced the per-site "reasoning is already inside output" tests
with billableOutputTokens() and REASONING_INCLUDED_IN_OUTPUT. This PR had
added copilot to that case at three sites independently. Union: all three
sites take main's helper call verbatim, and copilot joins claude and codex in
the set — same accounting this PR shipped, now through main's single source of
truth. It also reaches parser.ts activeGeneratedTokens (a fourth site, from
#1079), which is the same correction: a copilot supplementary call carries
reasoning with output 0, so counting it as generated repeats the per-turn
output. The audit legend already said so on this side.
src/providers/copilot.ts — comment-only. #1054's lastEventTimestamp-first
shutdown fallback was derived from this branch, so the code was already
identical on both sides: the `shutdownTimestamp` expression and the
`copilot:<sid>:shutdown:<model>:<n>` key are byte-for-byte main's. Both
rationales are kept (leg-collapse on date, and residual anchoring).
src/session-cache.ts — PROVIDER_PARSE_VERSIONS.copilot takes this PR's
`-session-store-v3` suffix; main's #1051 note about why a fingerprint change
is expensive is kept above it. Codex keeps main's #1092 suffix chain untouched.
src/main.ts — #1067 deleted the unreachable live dailyMap fallback that this
PR had taught behavioral weight. Main's deletion wins; the now-unused
isBehavioralTurn import goes with it.
tests/parser.test.ts — import union.
Also: scripts/upgrade-path/run.mjs NEW_DAILY_CACHE -> daily-cache.v26.json,
CODEBURN_COPILOT_SESSION_STORE_DB added to the #1064 env-isolation CLEARED
list, and the CHANGELOG entry's stale "v21" corrected to v26.
Verified: tsc clean; 3132 tests pass across 223 files; test:locks 26/26;
verify:upgrade PASSED, re-deriving daily-cache.v26.json and holding durable
copilot history across the bump. #1054's regression ("keeps three stampless
shutdown legs as :n keys with lastEventTimestamp") passes on the merged tree.
Real-corpus A/B against origin/main over 2026-07-01..2026-08-22: codex, grok,
kimicode and opencode byte-identical in export, audit and models; claude drifts
only monotonically with run order (a live session writing transcripts, confirmed
by interleaving four runs). This machine has no copilot data, so the copilot
recovery semantics rest on the suites and the upgrade-path corpus.
This commit is contained in:
commit
c30ebecff4
77 changed files with 3352 additions and 497 deletions
17
.github/workflows/tests.yml
vendored
17
.github/workflows/tests.yml
vendored
|
|
@ -33,14 +33,13 @@ jobs:
|
|||
# script since #948, so CI and a contributor's `npm test` can never drift.
|
||||
- name: Test suite (parallel)
|
||||
run: npm test
|
||||
# Single forked worker, so lock contention comes only from the child processes the
|
||||
# tests spawn deliberately. Quarantined (reports, never gates): the process
|
||||
# suite still races its own takeover window even serially on slow runners -
|
||||
# tracked in #904; drop continue-on-error once that race is settled.
|
||||
# Step-level timeout so a stalled lock suite fails soft here instead of
|
||||
# tripping the job's 15-minute budget, which kills the whole job as
|
||||
# "cancelled" and hides the parallel suite's green result.
|
||||
- name: Cache-lock suite (serial, quarantined)
|
||||
continue-on-error: true
|
||||
# Single forked worker, so lock contention comes only from the child
|
||||
# processes the tests spawn deliberately. Serial because the suite is
|
||||
# parallelism-sensitive (fails under full worker pressure, passes
|
||||
# serially). After #904 the loser of a stale-lock contest may be
|
||||
# timed-out or completed-by-other; this step gates CI.
|
||||
# Step-level timeout so a stalled lock suite fails this job instead of
|
||||
# eating the 15-minute budget and cancelling a green parallel suite.
|
||||
- name: Cache-lock suite (serial)
|
||||
timeout-minutes: 5
|
||||
run: npm run test:locks
|
||||
|
|
|
|||
12
CHANGELOG.md
12
CHANGELOG.md
|
|
@ -15,7 +15,7 @@
|
|||
- **A menubar app for Windows.** `windows/` is a Tauri 2 tray app — Rust binary, React popover — that puts today's spend in the notification area and mirrors the macOS menubar screen for screen: agent tabs, period switcher, Trend, Forecast, Pulse, Stats and Plan insights, activity and model breakdowns, optimize findings, CSV/JSON export, launch at login, currency, and theme. Windows has no menubar title, so the number lives in a second tray icon rendered from the system font at the panel's native icon size (Settings can turn it off; the tooltip always carries it). It reads everything through the CLI like the macOS and GNOME clients do, and gates on **codeburn 0.9.9 or newer** — the first release accepting `status --format menubar-json --no-optimize` — showing a setup screen with the install command until it finds one. Refresh follows popover visibility the way the macOS app does: 60 s with optimize findings while open, 2 minutes for today's total while closed, and immediately on open when what you are looking at has gone stale. The Claude quota view never spends Claude's single-use refresh token; on a 401 it re-reads Claude Code's own credential file for a token it has already rotated, matching the macOS client. Ships as an unsigned `.msi` from the `windows-v*` tag, which `codeburn menubar` now installs for you. The same crate still builds and runs a tray on Linux, but that stays experimental and unreleased — `gnome/` is the supported Linux surface.
|
||||
|
||||
### Added (CLI)
|
||||
- **Copilot input/cache tokens are read per request from `~/.copilot/session-store.db`.** Previously, codeburn relied on `session.shutdown` rollups from the Copilot CLI and GitHub Copilot desktop app. Those rollups are written only after a clean shutdown, stamp all usage on the shutdown day, and reset their counters at in-session compaction — so a crash could lose an entire session's input/cache usage, and even cleanly-closed long sessions were silently truncated. On one machine with long history, reading the per-request rows recovered about 35% of actual Copilot spend. Covered sessions now use per-request tokens with their real timestamps, counted exactly once against existing rollups and never added as extra calls or turns. Pre-store CLI sessions continue using the unchanged rollup path, and a locked or unreadable store defers only its own re-read instead of prematurely sealing daily history. Copilot reasoning tokens are also no longer double-billed: they are a subset of output already priced through the per-turn calls. This triggers a one-time re-parse, with the daily cache bumped to v21 to re-derive finalized days. Because that reconciliation keeps changing while a session is live, `codeburn sync push` holds a Copilot session back until it has been quiet for 24 hours and then sends it once, final — the sent-ledger is append-once, so a value sent mid-reconciliation could never be corrected at the receiver (#988). Sync also pins each Copilot session to whichever of the two shapes it was first synced in — the `session.shutdown` rollup, or per-request rows plus a residual — because a usage span cannot be retracted and sending the same tokens in the other shape would double them at the receiver permanently. That holds in both directions: a session synced before this release keeps its rollup and never sends rows, and a session synced as rows never sends the rollup that starts serving again once the 90-day age-out prunes them. `--dry-run` reports the frozen count, and `codeburn sync reset --confirm` re-pushes everything under the new breakdown for anyone who can clear the receiver too. (#946)
|
||||
- **Copilot input/cache tokens are read per request from `~/.copilot/session-store.db`.** Previously, codeburn relied on `session.shutdown` rollups from the Copilot CLI and GitHub Copilot desktop app. Those rollups are written only after a clean shutdown, stamp all usage on the shutdown day, and reset their counters at in-session compaction — so a crash could lose an entire session's input/cache usage, and even cleanly-closed long sessions were silently truncated. On one machine with long history, reading the per-request rows recovered about 35% of actual Copilot spend. Covered sessions now use per-request tokens with their real timestamps, counted exactly once against existing rollups and never added as extra calls or turns. Pre-store CLI sessions continue using the unchanged rollup path, and a locked or unreadable store defers only its own re-read instead of prematurely sealing daily history. Copilot reasoning tokens are also no longer double-billed: they are a subset of output already priced through the per-turn calls. This triggers a one-time re-parse, with the daily cache bumped to v26 to re-derive finalized days. Because that reconciliation keeps changing while a session is live, `codeburn sync push` holds a Copilot session back until it has been quiet for 24 hours and then sends it once, final — the sent-ledger is append-once, so a value sent mid-reconciliation could never be corrected at the receiver (#988). Sync also pins each Copilot session to whichever of the two shapes it was first synced in — the `session.shutdown` rollup, or per-request rows plus a residual — because a usage span cannot be retracted and sending the same tokens in the other shape would double them at the receiver permanently. That holds in both directions: a session synced before this release keeps its rollup and never sends rows, and a session synced as rows never sends the rollup that starts serving again once the 90-day age-out prunes them. `--dry-run` reports the frozen count, and `codeburn sync reset --confirm` re-pushes everything under the new breakdown for anyone who can clear the receiver too. (#946)
|
||||
- **DeepSeek Harness (`dsh`) is now a supported provider.** Reads DeepSeek's open-source agent harness from `~/.dsh/sessions` (`DSH_HOME` relocates the root), both the default zstd logs and the uncompressed `session.jsonl` variant. A `.zstd` log is a concatenation of independent zstd frames, one per write batch, so it is decoded frame by frame behind a structural frame scan and a torn trailing frame from a crashed writer is ignored rather than failing the file (needs Node 22.15+ for `zlib` zstd; below that dsh is skipped with a notice instead of counted as $0). One call per `(turn, step)`, with the step's final `assistant/message` usage superseding the streamed `assistant/chunk` sample of the same call rather than adding to it, the model taken from the message that served the step, and reasoning tokens billed at the output rate. DSH records tokens but no cost, so calls are priced from the shared tables. The events a forked session replays from its parent are skipped, since codeburn already counts the parent's own log. The session format is pinned at version 0 upstream with no compatibility implied, so a log stamped with any other version is skipped with a notice instead of read under today's assumptions.
|
||||
|
||||
### Changed
|
||||
|
|
@ -30,13 +30,23 @@
|
|||
- **A date-ranged report classifies only the turns it keeps.** Every cached turn went through the turn classifier — category, retries, edit detection, and a full reconstruction of its API calls — before the date slice discarded most of them, so a week view paid to classify all of history to keep a few percent of it. The keep/drop decision is now taken on the raw cached turn and only the survivors are classified, still from their complete call list, with the branch and pull-request carries still walking the full ordered turn list. Output is byte-identical.
|
||||
- **One rule for every cache file.** `CODEBURN_CACHE_DIR` when set, otherwise `~/.cache/codeburn`. `XDG_CACHE_HOME` is no longer consulted; the sync ledger, the only file that ever honored it, is merged into the canonical location on first read and the legacy copy is retired, so nothing is re-uploaded after the move. (#972)
|
||||
|
||||
### Changed (Linux packaging)
|
||||
- **The snap asks for the log directories it reads, not each tool's whole home.** The first Snap Store submission declared a `personal-files` read of every AI tool's root — `$HOME/.claude`, `$HOME/.codex`, `$HOME/.cursor` and the rest — and that interface is recursive, so it granted read of every credential file those roots hold. Each entry now names the subdirectory the provider actually opens (`.claude/projects`, `.codex/sessions`, `.cline/data`, `.vibe/logs/session`, `.dsh/sessions`, `.kiro/sessions`, `.quickwork/{profiles.json,sessions,metrics}`, `.config/Claude/local-agent-mode-sessions`, `.config/Open Design/{runs,data/runs,namespaces}`), two are single files (`.forge/.forge.db`, `.zcode/cli/db/db.sqlite`), and the editor entries name only the extension folders holding transcripts instead of the editor's whole configuration. Five providers that were missing entirely and would have shown no data are declared — opencode, crush, goose, kilo, kimi-code — and four roots stay roots only because the file the provider opens sits directly in them (`.config/github-copilot`, `.local/share/{opencode,crush,kilo}`). One credential file is now requested openly rather than implicitly: `.claude/.credentials.json`, read-only, for the live plan gauge. Codex's equivalent would need write access to the Codex CLI's own `auth.json` to rotate the token, so neither it nor a Codex root is declared and the Codex live gauge is disabled under `$SNAP`; Codex usage and cost are unaffected, they come from the session rollouts. Two consequences inside the snap: `.lingtai` is dropped, because its per-agent log directory needs a wildcard the interface has no form for, and `optimize`, `context-budget` and `act` no longer see the user-scope `~/.claude/settings.json`, `agents/`, `skills/` and `commands/` — project-scope copies still work through the `home` plug. Nothing outside the snap changes.
|
||||
|
||||
### Fixed (Desktop & Menubar)
|
||||
- **The menubar's copies of your Claude and Codex credentials move out of Application Support and into the login Keychain.** Connecting a provider used to leave the copied OAuth material in `~/Library/Application Support/CodeBurn/*-credentials.v1.json`, written world-readable (0644) because macOS ignores `.completeFileProtection` outside iOS. The copy now lives in a CodeBurn-owned login-Keychain item, and the first read after upgrading migrates the old file: it is reopened with `O_NOFOLLOW`, refused if it is a symlink or not owned by you, repaired to 0600 before a single secret byte is read, written to the Keychain, read back and compared, and only then unlinked — a failed or unverified write leaves the (now 0600) file in place so a retry can still find it, and the next read retries the cleanup. Where both a Keychain item and an old file exist, the one that expires later wins before anything is removed, so an item left behind by a much older build cannot displace a fresher token. Claude's entry no longer stores a refresh token at all — the CLI owns that grant and the menubar never spends it — and any refresh token in a historical blob is dropped on read. Disconnect only reports success once the material is actually gone; if the delete fails it says so and leaves the provider connected so you can retry. Keychain reads are non-interactive and are skipped outright while the login Keychain is locked, so a background quota refresh can never raise an unlock panel. (#1037)
|
||||
- **First launch no longer asks to control System Events.** The macOS menubar registered its login item by driving System Events over AppleScript, which made macOS put up an Automation consent dialog the first time the app ran. It now registers itself through `SMAppService.mainApp`, an in-process call that needs no Automation grant; there is no AppleScript fallback, so a failure logs and leaves the login item unset rather than bringing the prompt back. The same `codeburn.loginItemRegistered` guard still limits this to the first launch, so a login item you removed by hand stays removed. (#1026)
|
||||
- **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972)
|
||||
|
||||
### Fixed
|
||||
- **Subscription SKUs are classified from real product ids, and a false-positive built-in can be opted out.** `codex-auto-review` consumes ordinary Codex usage ([openai/codex#32224](https://github.com/openai/codex/issues/32224)) and is priced as GPT-5.5 on #1056, so treating it as $0 hid real spend — it left the flat-rate list. Warp's product id is `auto`, not the synthetic `warp`. `kimi-for-coding-highspeed` (the SKU #968 was filed around) is now honestly $0. `big-pickle` was dropped: it appears under OpenCode, not as a cited ClinePass codename. `codeburn model-flat-rate --remove` now opts out of a built-in, so a wrong classifier entry can warn again without waiting for a release. The daily-cache config hash now always includes the flat-rate section (even when empty), so the first run after upgrade re-derives every stored day once from the warm session cache. (#968, #1050)
|
||||
- **Codex MCP and skill usage is attributed from every shape Codex records a shell command in.** `mcp-cli call <server> <tool>` was only recognized when the command arrived as `function_call` arguments (#656). Codex has two other shapes for the same exec: its custom-tool transport records the shell tool as a `custom_tool_call` whose payload is an `input` program rather than `arguments`, and its item model repeats a finished command as `event_msg`/`item_completed` carrying a `CommandExecution` item with an argv `command`. Both reached the Bash counter and neither reached the matcher, so a CLI-wrapped MCP call stayed absent from the MCP breakdown exactly as before the fix. All three shapes now feed one classification pipeline. The same pipeline learns skills: Codex has no skill tool, so loading one is a shell read of the skill's `SKILL.md`, and those reads landed entirely under Bash with the Skills dimension empty. A read counts as a skill load only when the command segment starts with a file-reading binary (`cat`/`bat`/`sed`/`head`/`tail`/`less`/`more`) and the path it reads ends in `<name>/SKILL.md`; the skill is `<name>`, the same key `pi` derives for a native skill read (#588) and the same vocabulary the Claude parser records from the `Skill` tool. A `grep`/`rg`/`ls` that merely mentions a `SKILL.md` is a search near the file, not a skill load, and stays plain Bash. This is attribution only — no call, token or cost figure moves, and a command carried by both a response item and an item-model item is attributed once. On a 1,397-rollout corpus: Skills went from empty to 7 skills over 35 turns (55 attributions), Bash was unchanged at 42,170, and cost, calls, tokens, sessions, daily, models and projects came back identical. Cached Codex sessions re-parse once (`CODEX_CACHE_VERSION` 13 → 14 and the codex parse version both move; without them the fix is invisible on a warm cache). Thanks @chr-evensen. (#478)
|
||||
- **`gpt-5.6-codex` and `gpt-5.6-codex-max` now have their own pricing rows.** Neither id is in LiteLLM yet, and both were missing from the bundled snapshot — flagged during #1075 verification on a real corpus (285 sessions, 5,446 calls). `getModelCosts` already resolved both through the `gpt-5.6` prefix fallback, so live pricing was already correct once a session priced fresh; every prior Codex-suffixed id LiteLLM does carry bills identically to its bare-model sibling of the same generation (`gpt-5-codex` == `gpt-5`, `gpt-5.1-codex` == `gpt-5.1-codex-max` == `gpt-5.1`, `gpt-5.2-codex` == `gpt-5.2`, `gpt-5.3-codex` == `gpt-5.3`), which is the evidence both new rows mirror rather than inventing a rate. The gap that does not self-heal is the daily cache: it has no per-provider invalidation, so a day finalized while either id had no billable rate keeps that $0 forever. Raising `MIN_SUPPORTED_VERSION` (v23 -> v24) forces the one-time re-derivation, a lossless no-op for days already correct. (#1077)
|
||||
- **Mixed-version installs no longer thrash the Codex / Cursor / Antigravity result caches.** Daily and session caches already own a version-suffixed file so an old desktop binary and a newer CLI cannot clobber each other. The three per-provider result caches still used one unsuffixed filename with an internal version field, so a v10 and a v11 binary rewrote the same `codex-results.json` (and the Cursor / Antigravity siblings) on every run and each re-parsed its whole corpus. They now write `*-results.v<n>.json` the same way the daily cache does. The unsuffixed file is left for older binaries; a matching-version copy is adopted once and never overwritten. (#1082)
|
||||
- **Codex spend no longer counts reasoning tokens twice, and cache writes are priced only where OpenAI actually charges for them.** OpenAI bills reasoning tokens as *part of* `output_tokens`, not on top of it — on a 1,396-rollout corpus all 134,316 events carrying a total satisfy `input + output == total` — but CodeBurn added `reasoning_output_tokens` to output when pricing a Codex call and again in the models, audit and per-model displays. Every Codex number was therefore too high: on that corpus **cost by $166.03 (3.5%)** and **displayed Output tokens by 34.6%** ($4,713.12 -> $4,547.09; 22.6M -> 16.8M output tokens). The raw `reasoningTokens` figure is unchanged and still reported on its own; only the double-count is gone. Both places that price a Codex call — the parser and the cache-rehydration re-price — now go through one shared `billableOutputTokens` helper, so a cold run and a warm run can never disagree. Separately, Codex's `cache_write_input_tokens` (new in codex PR #33454) was never read and cache-creation tokens were hardcoded to 0; they are now carved out of the uncached-input bucket and clamped so they can never exceed it. That carve-out happens **only on models whose pricing source publishes a real cache-write rate** — gpt-5.6 and its terra/sol/luna variants charge 1.25x input for a cache write, everything before it charges nothing extra — because CodeBurn fabricates a 1.25x rate when a source omits one, and charging that would have invented a surcharge on gpt-5.5, gpt-5.4, gpt-5.3-codex and gpt-5. On models without an explicit rate the tokens stay in the plain input bucket and the price is unchanged to the cent. The field is new enough that today's impact is $0 on that corpus. Codex sessions re-parse once and the daily cache re-derives once off the warm session cache (a global re-derivation of every day and every provider, since it has no per-provider invalidation); no other provider's numbers move. Days whose Codex transcripts have since aged out are held by the same never-lose guard #1040 relies on: a re-derivation that finds fewer calls than the settled baseline keeps the older, pre-fix (double-counted) total rather than truncating it, so those days do not pick up the repricing until their sources are re-derived with equal or greater evidence. Long-context pricing tiers from the same report are tracked separately in #1076 and the missing `gpt-5.6-codex` snapshot rows in #1077. Thanks @chr-evensen. (#1075)
|
||||
- **Codex Tok/s no longer counts reasoning tokens twice or credits harness startup as model time.** Two distortions in the same metric, found and fixed together because they share the same cache-invalidation and test surface. (1) #1075 fixed the reasoning-token double-count for cost, but `activeGeneratedTokens`/`taskGeneratedTokens` in the Codex parser and `generatedTokens` in the `codex-tps` live-throughput reader still summed `outputTokens + reasoningTokens`; both now go through the same `billableOutputTokens('codex', …)` helper #1075 introduced, so the numerator can never drift from the billed one. (2) Codex fires `task_started` before it assembles the request, so the gap up to the first request-context event (`turn_context`, `world_state`, `event_msg/user_message`, or a `response_item/message`) was pure CLI/harness startup counted as active model time — the active window now starts at that first event instead, which matters most for one-shot `codex exec` sessions that pay the gap on every task. The duplicated tool-interval clip/merge/cap logic in `providers/codex.ts` and `codex-throughput.ts` is now one function (`mergeToolIntervals`, exported from `codex-throughput.ts`), which also closes a live trap where `task_complete`'s duration only parsed a plain number and silently dropped the `{secs,nanos}`/string forms `mcp_tool_call_end` already tolerated. (A third suspected distortion — fork-replay dedup dropping a token_count event's tokens from the numerator without shrinking the window to match — was investigated and retracted: the earlier `prevCumulativeTotal` guard already discards a repeated running total before dedup is ever reached, so a real Codex writer never produces a partial drop; the dedup site now carries a comment recording this so the trip isn't repeated.) Display only, no cost or token-count impact — verified byte-identical on the same real corpus. Combined effect on a real Codex corpus (original bug -> all fixes): GPT-5.5 37.8 -> 28.2 tok/s (-25.5%), Codex Auto Review 23.3 -> 20.0 (-14.2%), GPT-5.6 Sol 43.2 -> 33.9 (-21.4%), GPT-5.6 Luna 53.5 -> 49.0 (-8.5%), GPT-5.4 68.0 -> 43.6 (-35.9%), GPT-5.4 Mini 54.9 -> 55.6 (**+1.1%**, the harness-startup correction outweighing the reasoning-count correction for this model on this corpus). `activeGeneratedTokens`/`activeDurationMs`/`toolWaitMs` are stored verbatim in both the Codex result cache and the session cache rather than re-derived on read, so none of this self-heals: Codex sessions re-parse once (one cache-version bump covers both fixes, since they touch the same fields). The dashboard's per-model column stays labelled `Tok/s` — a wider label had zero room at the standard three-column layout, verified by breaking a real width-budget test — but the legend beneath it now reads "Effective Tok/s: generated tokens ÷ time the agent spent waiting on the model, tool execution excluded. Includes prefill, request assembly and reasoning. Not comparable to vendor decode-speed figures." (#1079, #1088)
|
||||
- **Codex calls attributed from session metadata no longer carry a stale model.** The Buffer fast path scanned `session_meta` for the first `"model"` string anywhere in the payload, so a nested `base_instructions.provenance.model` was read as if it were `payload.model` — and since the model is last-writer-wins state, that wrong value was credited to every call before the rollout's first `turn_context` and to every call after any mid-file `session_meta` (29 of 1380 rollouts on one real corpus carry a late `session_meta`, and 57 record usage before any `turn_context`). Direct payload fields are now read depth-aware, which is what the non-fast `JSON.parse` path always did. Codex sessions re-parse once (~9s on a 4 GB rollout corpus) and the daily cache re-derives once off the warm session cache, a global re-derivation of every day and every provider since it has no per-provider invalidation; it moves per-model attribution, and clears any rollup an earlier parse change had left stale. Days whose transcripts have partly aged out are held by the never-lose guard: on a real 110-day cache no day lost value and none disappeared — 100 days came back identical and 9 grok days rose by $19.80 in total. Thanks @timdp. (#1040)
|
||||
- **Codex `session_meta` cwd / session id / originator follow the same depth-1 window as `model`.** #1040 fixed nested `provenance.model`; the compact Buffer path still took the first `cwd`, `session_id`, `originator`, `name`, `forked_from_id` or `model_provider` anywhere in the payload, so a `dynamic_tools[].name` (or any same-named nested key) could steal the top-level field. Those strings now use the existing payload-depth-1 scan. Function-call `name` on other event types is unchanged. Codex sessions re-parse once. (#1045)
|
||||
- **Plan rows for sticker-price presets read as a budget instead of live provider quota.** There is no Grok quota endpoint, so a SuperGrok row was parsed API-equivalent spend divided by the plan's sticker price on a monthly reset — but the TUI labelled that math "plan" and "reset", which next to a client showing xAI's real weekly window read as CodeBurn being wrong. The bars and the arithmetic are unchanged; the words are not. Both the dashboard and the desktop app now say the number is an API-equivalent monthly budget and not a live provider window, in the same wording on both surfaces, and for every preset rather than as a SuperGrok special case. The window is anniversary-based (`plan.resetDay`, settable with `codeburn plan set --reset-day`), so it is called a budget reset rather than a calendar one. The row was also shortened to fit 80 columns: at that width the percentage and the projected month were being truncated away, including on custom plans, whose label carries the provider.
|
||||
- **MiMo sessions price from the LiteLLM Xiaomi rows, and MiMo v2 Flash no longer crashes the display path.** Hermes / Xiaomi token-plan sessions store the bare id (`mimo-v2.5-pro`, `mimo-v2.5`) while LiteLLM namespaces its row (`xiaomi/…`), so those models reported $0. They now alias to the existing snapshot rows — no invented rate, and `kimi-k3` still has none — which means a session Hermes left costless is priced from the shared tables and carries the estimated marker, exactly as `mimo-v2-flash` already did. The same change fixes a **pre-existing** crash that this alias did not introduce: the shipped `mimo-v2-flash -> xiaomi/mimo-v2-flash` alias already cycled through display-name resolution — strip the namespace, alias it back, take the leaf, repeat — so `getShortModelName` blew the stack on any real MiMo v2 Flash session and took every surface that names a model down with it, the `models` table included. Display-name resolution is now cycle-safe, and the `mimo-v2-flash` and `mimo-v2.5` rows are named rather than shown as raw slugs.
|
||||
- **A date-ranged run no longer republishes the month shards it never read.** A scoped load leaves an out-of-range month on disk, so the files it holds have no visible cache entry and the reconcile re-parses them — re-deriving the entry the shard already stores. That re-parse marked the unloaded month dirty, and the save merged and republished it under a fresh nonce name on every single run, byte-identical content and all, so a repeated `codeburn status --format json` churned old months (on a real corpus: claude/2026-03, cursor/2026-02 and warp/2026-03 renamed every run) and left the retired shards for the sweeper. A merge into an unloaded month that neither adds, changes nor removes an entry now keeps the published shard, so unchanged months keep their names and their bytes. (#1032)
|
||||
|
|
|
|||
|
|
@ -516,7 +516,7 @@ Sync sends token counts, costs, models, and projects, never prompts or code. Thi
|
|||
| `codeburn models --by-task` | Break each model into per-task-type rows |
|
||||
| `codeburn models --by-agent` | Break each model into per-agent rows: which agent drove which model's spend (`(main)` covers non-agent sessions; `--min-cost 0` shows sub-cent agents) |
|
||||
| `codeburn models --top 10` | Only the 10 most expensive models |
|
||||
| `codeburn models --unpriced` | Only models with usage that currently price at $0 — the copyable form of the unpriced-models warning. Shows raw model IDs (not friendly names) so they can be pasted into `model-alias`; JSON keeps them exact |
|
||||
| `codeburn models --unpriced` | Only models with usage that currently price at $0 — the copyable form of the unpriced-models warning. Shows raw model IDs (not friendly names). Per-token gaps go to `model-alias`; subscription / flat-rate SKUs go to `model-flat-rate`. JSON keeps IDs exact |
|
||||
| `codeburn models --format markdown` | Emit a paste-friendly markdown table |
|
||||
| `codeburn models --task feature` | Filter to feature-development work |
|
||||
| `codeburn models --provider claude` | Filter to a single provider |
|
||||
|
|
@ -608,10 +608,11 @@ Aliases are stored in `~/.config/codeburn/config.json` and applied at runtime be
|
|||
```bash
|
||||
codeburn price-override my-model --input 0.27 --output 1.10 # USD per 1M tokens
|
||||
codeburn model-savings "llama3.1:8b" gpt-4o # local model, counted as savings
|
||||
codeburn model-flat-rate auto-genius # subscription SKU, $0 is correct
|
||||
codeburn proxy-path ~/work/copilot-repo # subscription-covered project
|
||||
```
|
||||
|
||||
`price-override` sets exact rates for any model (input, output, cache read, cache creation), useful for private deployments or models LiteLLM prices wrong. `model-savings` maps a free local model to a paid baseline: the local calls stay $0, and the dashboard shows what the same tokens would have cost on the baseline. `proxy-path` marks a project routed through a subscription-backed proxy (e.g. Claude Code over GitHub Copilot), so its API-rate cost is reported as subscription-covered and your net out-of-pocket stays honest. All three support `--list` and `--remove`.
|
||||
`price-override` sets exact rates for any model (input, output, cache read, cache creation), useful for private deployments or models LiteLLM prices wrong. `model-savings` maps a free local model to a paid baseline: the local calls stay $0, and the dashboard shows what the same tokens would have cost on the baseline. `model-flat-rate` marks a subscription-billed product SKU so the unpriced warning stays quiet and `model-alias` is not suggested — aliasing those ids invents spend. `--remove` also opts out of a built-in SKU. `proxy-path` marks a project routed through a subscription-backed proxy (e.g. Claude Code over GitHub Copilot), so its API-rate cost is reported as subscription-covered and your net out-of-pocket stays honest. All four support `--list` and `--remove`.
|
||||
|
||||
### Filtering
|
||||
|
||||
|
|
@ -699,7 +700,7 @@ These are starting points, not verdicts. A 60% cache hit on a single experimenta
|
|||
| **Claude Code** | `~/.claude/projects/<sanitized-path>/<session-id>.jsonl` | Each assistant entry carries model name, token usage (input, output, cache read, cache write), `tool_use` blocks, and timestamps. |
|
||||
| **Claude (multiple config dirs)** | Set via `CLAUDE_CONFIG_DIRS` (e.g. `~/.claude-work:~/.claude-personal`) | Scans every listed directory and merges sessions into one row per project so totals reflect all your Claude usage. Use `:` on POSIX, `;` on Windows; overrides `CLAUDE_CONFIG_DIR`. Missing or unreadable directories are skipped. |
|
||||
| **Codex (OpenAI)** | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl`, `~/.codex/archived_sessions/rollout-*.jsonl` | Reads `token_count` events (per-call and cumulative usage) and `function_call` entries for tool tracking; attributes cost by project working directory. `codeburn report --provider codex` views Codex alone. |
|
||||
| **Cursor** | SQLite `state.vscdb` under `globalStorage`: macOS `~/Library/Application Support/Cursor/User/globalStorage/`, Linux `~/.config/Cursor/User/globalStorage/`, Windows `%APPDATA%/Cursor/User/globalStorage/`; results cached at `~/.cache/codeburn/cursor-results.json` | Input tokens come from Cursor's own per-conversation context meter (`composerData.promptTokenBreakdown`), credited once per conversation on a stable anchor; tool calls and shell commands are read from the agent stream (`agentKv`), and Composer house models are priced from Cursor's published rates. Output is a reply-text estimate and cache tokens are server-side only, so figures are marked estimated and undercount the Cursor admin console for long conversations. The cache auto-invalidates when the database changes; the first run on a large database can take a minute. |
|
||||
| **Cursor** | SQLite `state.vscdb` under `globalStorage`: macOS `~/Library/Application Support/Cursor/User/globalStorage/`, Linux `~/.config/Cursor/User/globalStorage/`, Windows `%APPDATA%/Cursor/User/globalStorage/`; results cached at `~/.cache/codeburn/cursor-results.v<n>.json` | Input tokens come from Cursor's own per-conversation context meter (`composerData.promptTokenBreakdown`), credited once per conversation on a stable anchor; tool calls and shell commands are read from the agent stream (`agentKv`), and Composer house models are priced from Cursor's published rates. Output is a reply-text estimate and cache tokens are server-side only, so figures are marked estimated and undercount the Cursor admin console for long conversations. The cache auto-invalidates when the database changes; the first run on a large database can take a minute. |
|
||||
| **OpenCode** | SQLite `~/.local/share/opencode/opencode*.db` or file-based `~/.local/share/opencode/storage/` (respects `XDG_DATA_HOME`; `OPENCODE_DATA_DIR`/`OPENCODE_DB_PREFIX` for renamed/forked builds) | Queries `session`, `message`, and `part` read-only and recalculates cost via LiteLLM (falling back to OpenCode's own cost field for unpriced models). Subtask sessions (`parent_id IS NOT NULL`) are excluded to avoid double counting; multiple channel databases are supported. |
|
||||
| **Gemini CLI** | `~/.gemini/tmp/<project>/chats/session-*.json` | One JSON file per session with real token counts (input, output, cached, thoughts) per message, so no estimation is needed. Input is reported inclusive of cached, so CodeBurn subtracts cached before pricing to avoid double charging. |
|
||||
| **Antigravity (CLI & IDE)** | Session files under `.gemini/` folders, plus the running language server | Pulls granular trajectory and pricing from the language server process. For the short-lived CLI, optionally install a status-line hook with `codeburn antigravity-hook install` so usage is captured between menubar refreshes. The IDE is detected via the `--app-data-dir antigravity-ide` flag on Windows. |
|
||||
|
|
|
|||
|
|
@ -8,6 +8,32 @@ const quota = (provider: 'claude' | 'codex'): QuotaProvider => ({
|
|||
})
|
||||
|
||||
describe('QuotaService', () => {
|
||||
// The snap declares no Codex credential path, because the live gauge would
|
||||
// need write access to the Codex CLI's own auth.json to rotate the token.
|
||||
// Under $SNAP the Codex fetch must not run at all; Claude is unaffected.
|
||||
it('skips the Codex live gauge under snap confinement', async () => {
|
||||
const previous = process.env['SNAP']
|
||||
process.env['SNAP'] = '/snap/codeburn/current'
|
||||
try {
|
||||
const claude = vi.fn(async () => ({ quota: quota('claude') }))
|
||||
const codex = vi.fn(async () => ({ quota: quota('codex') }))
|
||||
const service = new QuotaService({
|
||||
claude, codex, now: () => Date.parse('2026-08-14T00:00:00Z'),
|
||||
readFile: vi.fn(async () => null),
|
||||
writeFile: vi.fn(async () => {}),
|
||||
statePath: '/mock/backoff.json',
|
||||
})
|
||||
const [claudeQuota, codexQuota] = await service.getQuota({ force: true })
|
||||
expect(codex).not.toHaveBeenCalled()
|
||||
expect(claude).toHaveBeenCalledTimes(1)
|
||||
expect(codexQuota?.connection).toBe('disconnected')
|
||||
expect(claudeQuota?.connection).toBe('connected')
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env['SNAP']
|
||||
else process.env['SNAP'] = previous
|
||||
}
|
||||
})
|
||||
|
||||
it('persists provider 429 blocked-until and gates the next forced fetch', async () => {
|
||||
const writes: string[] = []
|
||||
const claude = vi.fn(async () => ({ quota: quota('claude'), retryAfterSeconds: 60 }))
|
||||
|
|
|
|||
|
|
@ -37,6 +37,19 @@ function unavailable(provider: ProviderName, connection: QuotaProvider['connecti
|
|||
return { provider, connection, primary: null, details: [], planLabel: null, footerLines: [] }
|
||||
}
|
||||
|
||||
/**
|
||||
* The Codex live gauge needs read-write access to the Codex CLI's own
|
||||
* `~/.codex/auth.json`, because refreshing the OAuth grant rotates the token
|
||||
* and writes it back. A store-distributed snap should not hold write access to
|
||||
* another vendor's credential file, so the snap's personal-files declaration
|
||||
* requests neither that file nor a Codex root, and the gauge is disabled here
|
||||
* to match. Codex usage and cost analytics are unaffected: those come from the
|
||||
* session rollouts under `~/.codex/sessions`, which the snap does read.
|
||||
*/
|
||||
function codexQuotaSupported(): boolean {
|
||||
return !process.env['SNAP']
|
||||
}
|
||||
|
||||
export class QuotaService {
|
||||
private readonly deps: QuotaDeps
|
||||
private cache: { at: number; value: QuotaProvider[] } | null = null
|
||||
|
|
@ -115,7 +128,10 @@ export class QuotaService {
|
|||
if (this.controllers[provider] === controller) this.controllers[provider] = undefined
|
||||
return retainOnFailure(result.quota)
|
||||
}
|
||||
const value = await Promise.all([run('claude'), run('codex')])
|
||||
const value = await Promise.all([
|
||||
run('claude'),
|
||||
codexQuotaSupported() ? run('codex') : Promise.resolve(unavailable('codex', 'disconnected')),
|
||||
])
|
||||
if (startingGenerations.claude === this.generations.claude && startingGenerations.codex === this.generations.codex) {
|
||||
this.cache = { at: this.deps.now(), value }
|
||||
}
|
||||
|
|
|
|||
|
|
@ -162,42 +162,70 @@
|
|||
"ai-agent-session-logs": {
|
||||
"interface": "personal-files",
|
||||
"read": [
|
||||
"$HOME/.claude",
|
||||
"$HOME/.cline",
|
||||
"$HOME/.codewhale",
|
||||
"$HOME/.codex",
|
||||
"$HOME/.copilot",
|
||||
"$HOME/.cursor",
|
||||
"$HOME/.deepseek",
|
||||
"$HOME/.claude/projects",
|
||||
"$HOME/.claude/.credentials.json",
|
||||
"$HOME/.codex/sessions",
|
||||
"$HOME/.codex/archived_sessions",
|
||||
"$HOME/.cline/data",
|
||||
"$HOME/.codewhale/sessions",
|
||||
"$HOME/.copilot/session-state",
|
||||
"$HOME/.cursor/projects",
|
||||
"$HOME/.cursor/ai-tracking",
|
||||
"$HOME/.deepseek/sessions",
|
||||
"$HOME/.dsh/sessions",
|
||||
"$HOME/.factory",
|
||||
"$HOME/.forge",
|
||||
"$HOME/.gemini",
|
||||
"$HOME/.grok",
|
||||
"$HOME/.hermes",
|
||||
"$HOME/.kimi",
|
||||
"$HOME/.kiro",
|
||||
"$HOME/.kiro-server",
|
||||
"$HOME/.lingtai",
|
||||
"$HOME/.lingtai-tui",
|
||||
"$HOME/.mux",
|
||||
"$HOME/.omp",
|
||||
"$HOME/.openclaude",
|
||||
"$HOME/.pi",
|
||||
"$HOME/.quickwork",
|
||||
"$HOME/.qwen",
|
||||
"$HOME/.vibe",
|
||||
"$HOME/.zcode",
|
||||
"$HOME/.config/Claude",
|
||||
"$HOME/.config/Code",
|
||||
"$HOME/.config/Code - Insiders",
|
||||
"$HOME/.config/Cursor",
|
||||
"$HOME/.config/Kiro",
|
||||
"$HOME/.config/Open Design",
|
||||
"$HOME/.config/VSCodium",
|
||||
"$HOME/.factory/sessions",
|
||||
"$HOME/.gemini/tmp",
|
||||
"$HOME/.gemini/antigravity",
|
||||
"$HOME/.gemini/antigravity-cli",
|
||||
"$HOME/.gemini/antigravity-ide",
|
||||
"$HOME/.grok/sessions",
|
||||
"$HOME/.hermes/profiles",
|
||||
"$HOME/.hermes/state.db",
|
||||
"$HOME/.kimi/sessions",
|
||||
"$HOME/.kimi-code/sessions",
|
||||
"$HOME/.kiro/sessions",
|
||||
"$HOME/.mux/sessions",
|
||||
"$HOME/.mux/config.json",
|
||||
"$HOME/.omp/agent",
|
||||
"$HOME/.openclaude/projects",
|
||||
"$HOME/.openclaw/agents",
|
||||
"$HOME/.pi/agent",
|
||||
"$HOME/.qwen/projects",
|
||||
"$HOME/.quickwork/profiles.json",
|
||||
"$HOME/.quickwork/sessions",
|
||||
"$HOME/.quickwork/metrics",
|
||||
"$HOME/.vibe/logs/session",
|
||||
"$HOME/.forge/.forge.db",
|
||||
"$HOME/.zcode/cli/db/db.sqlite",
|
||||
"$HOME/.config/Claude/local-agent-mode-sessions",
|
||||
"$HOME/.config/Code/User/globalStorage/saoudrizwan.claude-dev",
|
||||
"$HOME/.config/Code/User/globalStorage/rooveterinaryinc.roo-cline",
|
||||
"$HOME/.config/Code/User/globalStorage/kilocode.kilo-code",
|
||||
"$HOME/.config/Code/User/globalStorage/GitHub.copilot-chat",
|
||||
"$HOME/.config/Code/User/globalStorage/emptyWindowChatSessions",
|
||||
"$HOME/.config/Code/User/workspaceStorage",
|
||||
"$HOME/.config/Code - Insiders/User/globalStorage/saoudrizwan.claude-dev",
|
||||
"$HOME/.config/Code - Insiders/User/globalStorage/rooveterinaryinc.roo-cline",
|
||||
"$HOME/.config/Code - Insiders/User/globalStorage/kilocode.kilo-code",
|
||||
"$HOME/.config/Code - Insiders/User/workspaceStorage",
|
||||
"$HOME/.config/VSCodium/User/globalStorage/saoudrizwan.claude-dev",
|
||||
"$HOME/.config/VSCodium/User/globalStorage/rooveterinaryinc.roo-cline",
|
||||
"$HOME/.config/VSCodium/User/globalStorage/kilocode.kilo-code",
|
||||
"$HOME/.config/VSCodium/User/workspaceStorage",
|
||||
"$HOME/.config/Cursor/User/globalStorage/state.vscdb",
|
||||
"$HOME/.config/Kiro/User/globalStorage/kiro.kiroagent",
|
||||
"$HOME/.config/Kiro/User/workspaceStorage",
|
||||
"$HOME/.config/Open Design/runs",
|
||||
"$HOME/.config/Open Design/data/runs",
|
||||
"$HOME/.config/Open Design/namespaces",
|
||||
"$HOME/.kiro-server/data/User/globalStorage/kiro.kiroagent",
|
||||
"$HOME/.config/github-copilot",
|
||||
"$HOME/.config/manicode",
|
||||
"$HOME/.local/share/zed"
|
||||
"$HOME/.config/manicode/projects",
|
||||
"$HOME/.local/share/zed/threads",
|
||||
"$HOME/.local/share/opencode",
|
||||
"$HOME/.local/share/crush",
|
||||
"$HOME/.local/share/goose/sessions",
|
||||
"$HOME/.local/share/kilo"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
|
|||
45
app/scripts/snap-grants.test.ts
Normal file
45
app/scripts/snap-grants.test.ts
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
import { describe, it, expect } from 'vitest'
|
||||
import { readFileSync } from 'fs'
|
||||
import { join } from 'path'
|
||||
|
||||
// Snap Store review rejected the first submission because every entry was a
|
||||
// tool's whole root, and personal-files read is recursive: granting $HOME/.claude
|
||||
// granted .claude/.credentials.json with it. Each entry must name the log
|
||||
// directory (or file) the provider actually opens, never the root above it.
|
||||
// The exceptions below are roots only because the provider reads a file sitting
|
||||
// directly in them, so no narrower path exists without wildcards.
|
||||
const ROOT_GRANTS_WITH_NO_NARROWER_FORM = new Set([
|
||||
'$HOME/.config/github-copilot', // JetBrains stores nest under a variable <ide>/<kind>/<storeId>
|
||||
'$HOME/.local/share/opencode', // opencode*.db sits in the data dir itself
|
||||
'$HOME/.local/share/crush', // projects.json sits in the data dir itself
|
||||
'$HOME/.local/share/kilo', // kilo*.db sits in the data dir itself
|
||||
])
|
||||
|
||||
const XDG_PARENTS = ['.config', '.local']
|
||||
|
||||
function readGrants(): string[] {
|
||||
const pkg = JSON.parse(readFileSync(join(__dirname, '..', 'package.json'), 'utf8'))
|
||||
const plug = pkg.build.snap.plugs.find((p: unknown) => typeof p === 'object')
|
||||
return plug['ai-agent-session-logs'].read
|
||||
}
|
||||
|
||||
describe('snap personal-files declaration', () => {
|
||||
it('names a log path under each tool root, never the root itself', () => {
|
||||
const bare: string[] = []
|
||||
for (const entry of readGrants()) {
|
||||
if (ROOT_GRANTS_WITH_NO_NARROWER_FORM.has(entry)) continue
|
||||
const segments = entry.replace('$HOME/', '').split('/')
|
||||
const depth = XDG_PARENTS.includes(segments[0] ?? '') ? 3 : 2
|
||||
if (segments.length < depth) bare.push(entry)
|
||||
}
|
||||
expect(bare).toEqual([])
|
||||
})
|
||||
|
||||
it('requests read only, and one credential file explicitly', () => {
|
||||
const pkg = JSON.parse(readFileSync(join(__dirname, '..', 'package.json'), 'utf8'))
|
||||
const plug = pkg.build.snap.plugs.find((p: unknown) => typeof p === 'object')['ai-agent-session-logs']
|
||||
expect(Object.keys(plug).sort()).toEqual(['interface', 'read'])
|
||||
expect(readGrants().filter(e => e.includes('credential') || e.includes('auth.json')))
|
||||
.toEqual(['$HOME/.claude/.credentials.json'])
|
||||
})
|
||||
})
|
||||
|
|
@ -136,8 +136,8 @@ Three caches under `~/.cache/codeburn/` (override with `CODEBURN_CACHE_DIR`):
|
|||
|
||||
| File | Owner | Invalidation |
|
||||
|---|---|---|
|
||||
| `codex-results.json` | `src/codex-cache.ts` | `mtimeMs + sizeBytes` per Codex `.jsonl`. |
|
||||
| `cursor-results.json` | `src/cursor-cache.ts` | `mtimeMs + sizeBytes` of the Cursor SQLite db. |
|
||||
| `codex-results.v<n>.json` | `src/codex-cache.ts` | `mtimeMs + sizeBytes` per Codex `.jsonl`. Unsuffixed `codex-results.json` is adopted when versions match and never overwritten. |
|
||||
| `cursor-results.v<n>.json` | `src/cursor-cache.ts` | `mtimeMs + sizeBytes` of the Cursor SQLite db. Unsuffixed `cursor-results.json` is adopted when versions match and never overwritten. |
|
||||
| `daily-cache.json` | `src/daily-cache.ts` | Tracks `lastComputedDate`; new days are backfilled, old days are reused. |
|
||||
|
||||
All three use atomic write (temp file + `rename`) and write with mode `0o600`. All three carry a numeric `version` field; bumping it forces a recompute next run.
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ Protobuf. Cascade and response objects map to `ParsedProviderCall` directly.
|
|||
|
||||
## Caching
|
||||
|
||||
Custom file cache at `$CODEBURN_CACHE_DIR/antigravity-results.json` (defaults to `~/.cache/codeburn/`). The cache is also used as the data source when the RPC endpoint is unavailable, not just as an optimization. Bumping the cache version forces a recompute.
|
||||
Custom file cache at `$CODEBURN_CACHE_DIR/antigravity-results.v<n>.json` (defaults to `~/.cache/codeburn/`). The unsuffixed `antigravity-results.json` is left for older binaries; a matching-version copy is adopted once and never overwritten. The cache is also used as the data source when the RPC endpoint is unavailable, not just as an optimization. Bumping the cache version forces a recompute.
|
||||
|
||||
## Deduplication
|
||||
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ The first line read is capped at 1 MB (`FIRST_LINE_READ_CAP`). Codex CLI 0.128+
|
|||
|
||||
## Caching
|
||||
|
||||
`src/codex-cache.ts` writes `~/.cache/codeburn/codex-results.json` (or `$CODEBURN_CACHE_DIR/codex-results.json`). Each entry is keyed by absolute file path and validated against `mtimeMs + sizeBytes`. Cached entries are returned wholesale.
|
||||
`src/codex-cache.ts` writes `~/.cache/codeburn/codex-results.v<n>.json` (or `$CODEBURN_CACHE_DIR/codex-results.v<n>.json`). The unsuffixed `codex-results.json` is left for older binaries; a matching-version copy is adopted once and never overwritten. Each entry is keyed by absolute file path and validated against `mtimeMs + sizeBytes`. Cached entries are returned wholesale.
|
||||
|
||||
A session that yielded zero parseable lines does **not** write to the cache (`codex.ts:419`); this prevents a transient read failure from pinning an empty result against a fingerprint.
|
||||
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ The parser tries both and dedupes via `seenKeys`.
|
|||
|
||||
## Caching
|
||||
|
||||
`src/cursor-cache.ts` writes `~/.cache/codeburn/cursor-results.json` (override with `$CODEBURN_CACHE_DIR`). The fingerprint is `dbMtimeMs + dbSizeBytes` of `state.vscdb`. Atomic write via temp + rename.
|
||||
`src/cursor-cache.ts` writes `~/.cache/codeburn/cursor-results.v<n>.json` (override with `$CODEBURN_CACHE_DIR`). The unsuffixed `cursor-results.json` is left for older binaries; a matching-version copy is adopted once and never overwritten. The fingerprint is `dbMtimeMs + dbSizeBytes` of `state.vscdb`. Atomic write via temp + rename.
|
||||
|
||||
## Deduplication
|
||||
|
||||
|
|
|
|||
|
|
@ -106,7 +106,6 @@ A pseudonymous `device_id` distinguishes your machines without revealing hostnam
|
|||
|
||||
| Field | Example | Description |
|
||||
|---|---|---|
|
||||
| `ai.session_id` | `abc123…` | Session (shares the usage spans' traceId) |
|
||||
| `ai.project` | `my-app` | Project name |
|
||||
| `git.repo` | `github.com/acme/widget` | Normalized `origin` remote (credentials and ports stripped) |
|
||||
| `git.pr_links` | `["…/pull/12"]` | PR URLs captured for the session |
|
||||
|
|
@ -120,7 +119,7 @@ A pseudonymous `device_id` distinguishes your machines without revealing hostnam
|
|||
| `git.in_main` | `true` | Whether the commit landed in the main branch |
|
||||
| `git.was_reverted` | `false` | Whether a later commit reverted it |
|
||||
|
||||
Attribution is **inferred** (timestamp-window correlation, the same heuristic as `codeburn yield`); the resource attribute `codeburn.attribution_methodology: timestamp-window` marks it as such. State transitions (a commit merging to main, or being reverted) are re-sent automatically on later pushes — receivers should upsert commits by `(git.repo, git.sha)` and session spans by `ai.session_id` (latest state wins). When a commit migrates to a later-parsed session with a tighter window, the losing session re-emits with `git.commit_count: 0` (a retraction), so summing `git.commit_count` across upserted session rows never double-counts. Retractions fire only when the commit was won by another session — commits that merely age out of the `--since` window are not retracted, so a previously-synced count stays correct. Session spans also re-emit when an ongoing session's window grows, keeping the span end time current.
|
||||
Attribution is **inferred** (timestamp-window correlation, the same heuristic as `codeburn yield`); the resource attribute `codeburn.attribution_methodology: timestamp-window` marks it as such. State transitions (a commit merging to main, or being reverted) are re-sent automatically on later pushes — receivers should upsert commits by `(git.repo, git.sha)` and session spans by `traceId` (the same id usage spans already carry; latest state wins). When a commit migrates to a later-parsed session with a tighter window, the losing session re-emits with `git.commit_count: 0` (a retraction), so summing `git.commit_count` across upserted session rows never double-counts. Retractions fire only when the commit was won by another session — commits that merely age out of the `--since` window are not retracted, so a previously-synced count stays correct. Session spans also re-emit when an ongoing session's window grows, keeping the span end time current.
|
||||
|
||||
With `--attribution`, normalized repo remote URLs, commit SHAs, commit timestamps (span start times), PR URLs, and the merged/reverted booleans leave your machine — plus the same pseudonymous `codeburn.device_id` resource attribute the usage spans carry. PR links are rebuilt client-side from scheme + host + path only (userinfo, query strings, and fragments are dropped; https, `/org/repo/pull/N` path, bounded length, max 20 per session), and the repo identity itself passes a strict hostname/path allow-list before sending — malformed or transport-helper remotes (`ext::…`, `codecommit::…`) are rejected outright rather than parsed. Precisely what is and is not sent:
|
||||
|
||||
|
|
|
|||
|
|
@ -44,6 +44,16 @@ const MANUAL_ENTRIES = {
|
|||
'deepseek-v4-pro': [4.35e-7, 8.7e-7, 0, 3.625e-9],
|
||||
// Mythos 5 launch pricing; not yet in LiteLLM or the models.dev/OpenRouter gap-fill (Fable is).
|
||||
'claude-mythos-5': [10e-6, 50e-6, 12.5e-6, 1e-6],
|
||||
// gpt-5.6-codex / gpt-5.6-codex-max (#1077): not yet in LiteLLM. Every prior
|
||||
// Codex-suffixed id LiteLLM DOES carry bills identically to its bare-model
|
||||
// sibling of the same generation - gpt-5-codex == gpt-5, gpt-5.1-codex ==
|
||||
// gpt-5.1-codex-max == gpt-5.1, gpt-5.2-codex == gpt-5.2, gpt-5.3-codex ==
|
||||
// gpt-5.3 (all four input/output/cache-write/cache-read rates identical,
|
||||
// verified against the live model_prices_and_context_window.json). Mirroring
|
||||
// that pattern onto gpt-5.6 rather than inventing a number: both ids get the
|
||||
// exact gpt-5.6 tuple (Sol-tier: $5/$30 per million, 1.25x cache-write).
|
||||
'gpt-5.6-codex': [5e-6, 3e-5, 6.25e-6, 5e-7],
|
||||
'gpt-5.6-codex-max': [5e-6, 3e-5, 6.25e-6, 5e-7],
|
||||
}
|
||||
|
||||
const snapshot = {}
|
||||
|
|
|
|||
|
|
@ -22,11 +22,26 @@
|
|||
// which is the part the corpus can honestly establish.
|
||||
// dsh did not exist in the published CLI. Reported; required to be absent
|
||||
// in the baseline and present after the upgrade.
|
||||
// codex PRICING changed by design in #1075: reasoning tokens are billed
|
||||
// inside output rather than on top of it, and cache writes are carved out
|
||||
// of the input bucket. Nothing about what was PARSED moved, so codex keeps
|
||||
// the full exact treatment for the call count and every token field; the
|
||||
// cost tolerance is instead replaced with REPRICE_TOLERANCE — the
|
||||
// upgraded cost must be strictly lower than the baseline and within 25%
|
||||
// of it, since #1075 only ever removes a double-count and never raises
|
||||
// cost — and the delta is reported instead. Drop it from this list once a
|
||||
// published CLI carries the fix.
|
||||
const EXACT = ['claude', 'codex', 'gemini', 'kiro', 'cursor']
|
||||
const CHANGED_BY_DESIGN = ['grok']
|
||||
const COST_CHANGED_BY_DESIGN = ['codex']
|
||||
const NEW_IN_THIS_RELEASE = ['dsh']
|
||||
|
||||
const COST_TOLERANCE = 0.005 // 0.5% relative
|
||||
// #1075 only ever LOWERS codex cost (double-counted reasoning removed, cache
|
||||
// writes carved out of the input bucket) and by a bounded amount on any real
|
||||
// corpus; a rise, or a drop past this bound, means something beyond the known
|
||||
// repricing changed.
|
||||
const REPRICE_TOLERANCE = 0.25 // 25% relative
|
||||
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
|
|
@ -95,13 +110,25 @@ for (const name of providers) {
|
|||
if (b.calls !== u.calls) diffs.push(`calls ${b.calls} != ${u.calls}`)
|
||||
for (const f of TOKEN_FIELDS) if (b[f] !== u[f]) diffs.push(`${f} ${b[f]} != ${u[f]}`)
|
||||
const costDrift = relDiff(b.cost, u.cost)
|
||||
if (costDrift > COST_TOLERANCE) diffs.push(`cost ${fmt(b.cost)} != ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(COST_TOLERANCE * 100).toFixed(1)}%)`)
|
||||
let repriced = false
|
||||
if (COST_CHANGED_BY_DESIGN.includes(name)) {
|
||||
if (u.cost > b.cost) diffs.push(`cost ${fmt(b.cost)} -> ${fmt(u.cost)} rose; #1075 should only lower codex cost`)
|
||||
else if (costDrift > REPRICE_TOLERANCE) diffs.push(`cost ${fmt(b.cost)} -> ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(REPRICE_TOLERANCE * 100).toFixed(0)}% expected bound for #1075)`)
|
||||
else {
|
||||
repriced = true
|
||||
notes.push(`${name}: cost ${fmt(b.cost)} -> ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}%) — repricing expected (#1075); tokens and calls still asserted exactly`)
|
||||
}
|
||||
} else if (costDrift > COST_TOLERANCE) {
|
||||
diffs.push(`cost ${fmt(b.cost)} != ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(COST_TOLERANCE * 100).toFixed(1)}%)`)
|
||||
}
|
||||
if (!EXACT.includes(name)) {
|
||||
notes.push(`${name}: no expectation declared in compare.mjs; ${diffs.length ? diffs.join(', ') : 'identical'}`)
|
||||
verdict = diffs.length ? 'differs (unclassified)' : 'identical'
|
||||
} else if (diffs.length) {
|
||||
failures.push(`${name}: ${diffs.join(', ')}`)
|
||||
verdict = 'DIFFERS'
|
||||
} else if (repriced) {
|
||||
verdict = `repriced (cost ${(costDrift * 100).toFixed(3)}% drift)`
|
||||
} else {
|
||||
verdict = costDrift === 0 ? 'identical' : `identical (cost ${(costDrift * 100).toFixed(3)}% drift)`
|
||||
}
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ const WORK = process.env['UPGRADE_PATH_WORK'] || join(tmpdir(), 'codeburn upgrad
|
|||
const OLD_SESSION_CACHE = 'session-cache.v7.json'
|
||||
const OLD_DAILY_CACHE = 'daily-cache.v17.json'
|
||||
const NEW_SESSION_CACHE_DIR = 'session-cache.v9'
|
||||
const NEW_DAILY_CACHE = 'daily-cache.v25.json'
|
||||
const NEW_DAILY_CACHE = 'daily-cache.v26.json'
|
||||
|
||||
const HOME = join(WORK, 'user home')
|
||||
const PAYLOADS = join(WORK, 'payloads')
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import { isBehavioralCall } from './behavioral-weight.js'
|
||||
import { getModelCosts, sanitizeModelForDisplay, type ModelCosts } from './models.js'
|
||||
import { billableOutputTokens, fallbackRawModelDisplayName, getModelCosts, getShortModelName, sanitizeModelForDisplay, type ModelCosts } from './models.js'
|
||||
import { getProvider } from './providers/index.js'
|
||||
import { formatCost, formatTokens } from './format.js'
|
||||
import { renderTable, type TableColumn } from './text-table.js'
|
||||
|
|
@ -115,8 +115,8 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise<AuditR
|
|||
const entry = {
|
||||
displayName: p?.displayName ?? name,
|
||||
formatModel: p
|
||||
? (m: string) => sanitizeModelForDisplay(p.modelDisplayName(m))
|
||||
: sanitizeModelForDisplay,
|
||||
? (m: string) => sanitizeModelForDisplay(fallbackRawModelDisplayName(p.modelDisplayName(m), m))
|
||||
: (m: string) => sanitizeModelForDisplay(getShortModelName(m)),
|
||||
}
|
||||
providerCache.set(name, entry)
|
||||
return entry
|
||||
|
|
@ -130,9 +130,7 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise<AuditR
|
|||
// in parser.ts), so folding them in would display phantom output for its store rows/rollups.
|
||||
const displayed = {
|
||||
inputTokens: bucket.raw.inputTokens,
|
||||
outputTokens: bucket.provider === 'copilot'
|
||||
? bucket.raw.outputTokens
|
||||
: bucket.raw.outputTokens + bucket.raw.reasoningTokens,
|
||||
outputTokens: billableOutputTokens(bucket.provider, bucket.raw.outputTokens, bucket.raw.reasoningTokens),
|
||||
cacheWriteTokens: bucket.raw.cacheCreationInputTokens,
|
||||
cacheReadTokens: bucket.cacheReadDisplayed,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { readFile } from 'fs/promises'
|
||||
import { homedir } from 'os'
|
||||
import { join } from 'path'
|
||||
|
||||
|
|
@ -11,3 +12,18 @@ export function getCodeburnCacheDir(): string {
|
|||
const override = process.env['CODEBURN_CACHE_DIR']
|
||||
return override?.trim() ? override : join(homedir(), '.cache', 'codeburn')
|
||||
}
|
||||
|
||||
/** A versioned cache file is the only source when it exists. Legacy adoption is ENOENT-only. */
|
||||
export type ExistingTextFile =
|
||||
| { status: 'absent' }
|
||||
| { status: 'unreadable' }
|
||||
| { status: 'ok'; text: string }
|
||||
|
||||
export async function readExistingTextFile(path: string): Promise<ExistingTextFile> {
|
||||
try {
|
||||
return { status: 'ok', text: await readFile(path, 'utf-8') }
|
||||
} catch (err) {
|
||||
const code = err && typeof err === 'object' && 'code' in err ? (err as { code?: string }).code : undefined
|
||||
return { status: code === 'ENOENT' ? 'absent' : 'unreadable' }
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { randomBytes } from 'crypto'
|
|||
import { join, resolve } from 'path'
|
||||
import { AsyncLocalStorage } from 'node:async_hooks'
|
||||
|
||||
import { getCodeburnCacheDir } from './cache-dir.js'
|
||||
import { getCodeburnCacheDir, readExistingTextFile } from './cache-dir.js'
|
||||
import type { ParsedProviderCall } from './providers/types.js'
|
||||
|
||||
// v4: attribute MCP calls emitted as event_msg/mcp_tool_call_end (issue #478).
|
||||
|
|
@ -21,8 +21,33 @@ import type { ParsedProviderCall } from './providers/types.js'
|
|||
// entry without them simply re-parses in full once and gains them.
|
||||
// v9: parse large session_meta records structurally so nested provenance.model
|
||||
// cannot overwrite the model selected by turn_context.
|
||||
const CODEX_CACHE_VERSION = 9
|
||||
const CACHE_FILE = 'codex-results.json'
|
||||
// v10: same depth-1 window for the rest of session_meta's raw string fields
|
||||
// (cwd/name/originator/session_id/forked_from_id/model_provider).
|
||||
// v11: codex pricing fix (#1075) - reasoning is no longer added on top of
|
||||
// output, and cache_write_input_tokens is carved out of the input bucket. This
|
||||
// file stores each call's costUSD and token buckets verbatim, so entries
|
||||
// written by v10 carry the old (overstated) cost and must be re-derived.
|
||||
// v13: codex throughput fix (#1079) - activeGeneratedTokens was summing
|
||||
// output + reasoning, the same double-count Fix A removed from cost. This
|
||||
// file stores activeGeneratedTokens/activeDurationMs/toolWaitMs verbatim (not
|
||||
// re-derived on read), so v11 entries carry the overstated numerator and must
|
||||
// re-parse. Not 12: v12 is claimed by feat/core-extraction's own port of this
|
||||
// throughput feature (PR #1086), so reusing it would let two incompatible
|
||||
// schemas share a filename.
|
||||
// v14: MCP + Skill attribution for the shapes the classic `function_call` path
|
||||
// never reached (#478) - the `exec` custom tool's `input` program and the item
|
||||
// model's `item_completed`/`CommandExecution` item - plus SKILL.md reads landing
|
||||
// in `skills`. This file stores each call's `tools`/`toolSequence`/`skills`
|
||||
// verbatim (they are passed through on read, never re-derived), so v13 entries
|
||||
// keep the old, MCP- and skill-less attribution until they re-parse.
|
||||
// v15: builtin alias prices `codex-auto-review` (#1047). Exact-hit cache
|
||||
// entries still hold the pre-alias $0; bump so unchanged rollouts reprice.
|
||||
// Must be max(main v14 #1092, this)+1 — #1092 spent v14 on MCP/skills.
|
||||
export const CODEX_CACHE_VERSION = 15
|
||||
export const CODEX_LEGACY_CACHE_FILE = 'codex-results.json'
|
||||
export function codexCacheFileName(version = CODEX_CACHE_VERSION): string {
|
||||
return `codex-results.v${version}.json`
|
||||
}
|
||||
|
||||
export type CodexFileFingerprint = { dev: number; ino: number; mtimeMs: number; sizeBytes: number }
|
||||
type FileFingerprint = CodexFileFingerprint
|
||||
|
|
@ -68,7 +93,15 @@ export function withCodexCacheDirectory<T>(cacheDir: string, operation: () => T)
|
|||
}
|
||||
|
||||
function getCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, CACHE_FILE)
|
||||
return join(cacheDir, codexCacheFileName())
|
||||
}
|
||||
|
||||
function getLegacyCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, CODEX_LEGACY_CACHE_FILE)
|
||||
}
|
||||
|
||||
function isCurrentCache(cache: ResultCache): boolean {
|
||||
return cache.version === CODEX_CACHE_VERSION && !!cache.files && typeof cache.files === 'object'
|
||||
}
|
||||
|
||||
// Embedded consumers can change CODEBURN_CACHE_DIR without reloading this
|
||||
|
|
@ -85,15 +118,33 @@ export function clearCodexMemCaches(): void {
|
|||
async function loadCache(cacheDir: string): Promise<ResultCache> {
|
||||
const inMemory = memCaches.get(cacheDir)
|
||||
if (inMemory) return inMemory
|
||||
const empty = { version: CODEX_CACHE_VERSION, files: {} }
|
||||
const versioned = await readExistingTextFile(getCachePath(cacheDir))
|
||||
if (versioned.status === 'ok') {
|
||||
try {
|
||||
const cache = JSON.parse(versioned.text) as ResultCache
|
||||
if (isCurrentCache(cache)) {
|
||||
memCaches.set(cacheDir, cache)
|
||||
return cache
|
||||
}
|
||||
} catch {}
|
||||
memCaches.set(cacheDir, empty)
|
||||
return empty
|
||||
}
|
||||
if (versioned.status === 'unreadable') {
|
||||
memCaches.set(cacheDir, empty)
|
||||
return empty
|
||||
}
|
||||
// Versioned file is absent (ENOENT). Adopt the unsuffixed file only when its
|
||||
// version matches — old binaries still own that path; we never write or delete it.
|
||||
try {
|
||||
const raw = await readFile(getCachePath(cacheDir), 'utf-8')
|
||||
const raw = await readFile(getLegacyCachePath(cacheDir), 'utf-8')
|
||||
const cache = JSON.parse(raw) as ResultCache
|
||||
if (cache.version === CODEX_CACHE_VERSION && cache.files && typeof cache.files === 'object') {
|
||||
if (isCurrentCache(cache)) {
|
||||
memCaches.set(cacheDir, cache)
|
||||
return cache
|
||||
}
|
||||
} catch {}
|
||||
const empty = { version: CODEX_CACHE_VERSION, files: {} }
|
||||
memCaches.set(cacheDir, empty)
|
||||
return empty
|
||||
}
|
||||
|
|
|
|||
|
|
@ -20,10 +20,19 @@ const CREDITS_PER_MILLION: Record<string, CodexCreditRate> = {
|
|||
'gpt-5.4-mini': { input: 18.75, cachedInput: 1.875, output: 113 },
|
||||
}
|
||||
|
||||
// Activity surfaces keep their product id on the call (display stays
|
||||
// "Codex Auto Review"). Credits must follow the same underlying model
|
||||
// BUILTIN_ALIASES uses for USD. Keep this table in lockstep with
|
||||
// `codex-auto-review` in src/models.ts.
|
||||
const ACTIVITY_CREDIT_MODELS: Record<string, string> = {
|
||||
'codex-auto-review': 'gpt-5.5',
|
||||
}
|
||||
|
||||
/// Resolve the credit rate for a Codex model name, tolerating suffix variants
|
||||
/// (e.g. "gpt-5.5-codex"). Returns null when the model has no known credit rate.
|
||||
export function codexCreditRate(model: string): CodexCreditRate | null {
|
||||
const m = model.toLowerCase()
|
||||
const mapped = ACTIVITY_CREDIT_MODELS[model] ?? ACTIVITY_CREDIT_MODELS[model.toLowerCase()]
|
||||
const m = (mapped ?? model).toLowerCase()
|
||||
if (m.includes('5.4') && m.includes('mini')) return CREDITS_PER_MILLION['gpt-5.4-mini']!
|
||||
if (m.includes('5.4')) return CREDITS_PER_MILLION['gpt-5.4']!
|
||||
if (m.includes('5.5')) return CREDITS_PER_MILLION['gpt-5.5']!
|
||||
|
|
@ -36,9 +45,9 @@ export type CodexCreditTokens = {
|
|||
inputTokens: number
|
||||
/// Cache-read (cached input) tokens, billed at the cheaper cached rate.
|
||||
cachedReadTokens: number
|
||||
/// Billable output tokens: reasoning is already included (billableOutputTokens
|
||||
/// in models.ts), so callers must not add it on top here.
|
||||
outputTokens: number
|
||||
/// Reasoning tokens are billed as output, matching CodeBurn's cost model.
|
||||
reasoningTokens?: number
|
||||
}
|
||||
|
||||
/// Credits consumed for one Codex usage record. Returns null when the model has
|
||||
|
|
@ -48,10 +57,9 @@ export function codexCredits(model: string, tokens: CodexCreditTokens): number |
|
|||
if (!rate) return null
|
||||
const safe = (n: number) => (Number.isFinite(n) && n > 0 ? n : 0)
|
||||
const PER_MILLION = 1_000_000
|
||||
const output = safe(tokens.outputTokens) + safe(tokens.reasoningTokens ?? 0)
|
||||
return (
|
||||
(safe(tokens.inputTokens) / PER_MILLION) * rate.input +
|
||||
(safe(tokens.cachedReadTokens) / PER_MILLION) * rate.cachedInput +
|
||||
(output / PER_MILLION) * rate.output
|
||||
(safe(tokens.outputTokens) / PER_MILLION) * rate.output
|
||||
)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
import { open, stat } from 'node:fs/promises'
|
||||
import { StringDecoder } from 'node:string_decoder'
|
||||
|
||||
import { billableOutputTokens } from './models.js'
|
||||
|
||||
export type CodexThroughputPoint = {
|
||||
timestamp: string
|
||||
model?: string
|
||||
|
|
@ -105,7 +107,12 @@ function durationMs(payload: RolloutLine['payload']): number | undefined {
|
|||
return undefined
|
||||
}
|
||||
|
||||
function mergeToolIntervals(intervals: Array<[number, number]>, durationMs: number, taskStartedAt?: number, taskCompletedAt?: number): number {
|
||||
// Shared with src/providers/codex.ts (#1088 BUG-8): both clip a task's tool
|
||||
// intervals to its [taskStartedAt, taskStartedAt + durationMs] window, merge
|
||||
// overlaps, and cap the sum at durationMs. Was copy-pasted inline in
|
||||
// providers/codex.ts and had already drifted (duration parsing there accepted
|
||||
// only a plain `duration_ms` number); one copy now, called from both.
|
||||
export function mergeToolIntervals(intervals: Array<[number, number]>, durationMs: number, taskStartedAt?: number, taskCompletedAt?: number): number {
|
||||
const windowStart = taskStartedAt ?? (taskCompletedAt !== undefined ? taskCompletedAt - durationMs : undefined)
|
||||
const windowEnd = windowStart !== undefined ? windowStart + durationMs : undefined
|
||||
const clipped = intervals.map(([start, end]) => [
|
||||
|
|
@ -400,7 +407,9 @@ export class CodexThroughputReader {
|
|||
state.previousOutput = total?.output_tokens ?? state.previousOutput
|
||||
state.previousReasoning = total?.reasoning_output_tokens ?? state.previousReasoning
|
||||
}
|
||||
const generatedTokens = outputTokens + reasoningTokens
|
||||
// Reasoning is already inside output_tokens (#1075/#1078); same numerator
|
||||
// as the cost path so live Tok/s can't drift from billed tokens (#1079).
|
||||
const generatedTokens = billableOutputTokens('codex', outputTokens, reasoningTokens)
|
||||
if (generatedTokens <= 0) return
|
||||
const timestampMs = Date.parse(entry.timestamp)
|
||||
if (!Number.isFinite(timestampMs)) return
|
||||
|
|
|
|||
|
|
@ -44,6 +44,15 @@ export type CodeburnConfig = {
|
|||
// can show "saved $X by running locally". Distinct from modelAliases which
|
||||
// rewrites actual spend.
|
||||
localModelSavings?: Record<string, string>
|
||||
// Model ids whose $0 cost is correct because they are billed as a
|
||||
// subscription / flat-rate product, not missing LiteLLM rows. Distinct from
|
||||
// modelAliases (which invent per-token spend) and localModelSavings
|
||||
// (counterfactual local baseline). See `codeburn model-flat-rate`.
|
||||
flatRateModels?: string[]
|
||||
// Opt-outs from the built-in flat-rate classifier. `model-flat-rate --remove`
|
||||
// on a built-in SKU records the id here so a false positive can warn again
|
||||
// without waiting for a release.
|
||||
flatRateModelsRemoved?: string[]
|
||||
// Spend budgets are stored in the configured display currency, not USD.
|
||||
budget?: {
|
||||
daily?: number
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { readFile, writeFile, mkdir, rename, stat, unlink } from 'fs/promises'
|
|||
import { join } from 'path'
|
||||
import { randomBytes } from 'crypto'
|
||||
|
||||
import { getCodeburnCacheDir } from './cache-dir.js'
|
||||
import { getCodeburnCacheDir, readExistingTextFile } from './cache-dir.js'
|
||||
import type { ParsedProviderCall } from './providers/types.js'
|
||||
|
||||
// Bumped to 3 for the workspace-aware breakdown change: the cursor parser
|
||||
|
|
@ -19,7 +19,11 @@ import type { ParsedProviderCall } from './providers/types.js'
|
|||
// (cursor:composer-input:<id>) with per-conversation source selection, the
|
||||
// agent stream regained tool/system context and stream-only sessions, and
|
||||
// tool names are canonicalized. v5 results mix crediting regimes.
|
||||
const CURSOR_CACHE_VERSION = 6
|
||||
export const CURSOR_CACHE_VERSION = 6
|
||||
export const CURSOR_LEGACY_CACHE_FILE = 'cursor-results.json'
|
||||
export function cursorCacheFileName(version = CURSOR_CACHE_VERSION): string {
|
||||
return `cursor-results.v${version}.json`
|
||||
}
|
||||
|
||||
type ResultCache = {
|
||||
version?: number
|
||||
|
|
@ -29,10 +33,30 @@ type ResultCache = {
|
|||
calls: ParsedProviderCall[]
|
||||
}
|
||||
|
||||
const CACHE_FILE = 'cursor-results.json'
|
||||
|
||||
function getCachePath(): string {
|
||||
return join(getCodeburnCacheDir(), CACHE_FILE)
|
||||
return join(getCodeburnCacheDir(), cursorCacheFileName())
|
||||
}
|
||||
|
||||
function getLegacyCachePath(): string {
|
||||
return join(getCodeburnCacheDir(), CURSOR_LEGACY_CACHE_FILE)
|
||||
}
|
||||
|
||||
function isCurrentHit(cache: ResultCache, fp: { mtimeMs: number; size: number }, requestedFloor: string): boolean {
|
||||
return (
|
||||
cache.version === CURSOR_CACHE_VERSION
|
||||
&& cache.dbMtimeMs === fp.mtimeMs
|
||||
&& cache.dbSizeBytes === fp.size
|
||||
&& typeof cache.lookbackFloor === 'string'
|
||||
&& cache.lookbackFloor <= requestedFloor
|
||||
)
|
||||
}
|
||||
|
||||
async function readCacheFile(path: string): Promise<ResultCache | null> {
|
||||
try {
|
||||
const cache = JSON.parse(await readFile(path, 'utf-8')) as ResultCache
|
||||
if (cache && typeof cache === 'object') return cache
|
||||
} catch {}
|
||||
return null
|
||||
}
|
||||
|
||||
async function getDbFingerprint(dbPath: string): Promise<{ mtimeMs: number; size: number } | null> {
|
||||
|
|
@ -52,18 +76,20 @@ export async function readCachedResults(
|
|||
const fp = await getDbFingerprint(dbPath)
|
||||
if (!fp) return null
|
||||
|
||||
const raw = await readFile(getCachePath(), 'utf-8')
|
||||
const cache = JSON.parse(raw) as ResultCache
|
||||
|
||||
if (
|
||||
cache.version === CURSOR_CACHE_VERSION &&
|
||||
cache.dbMtimeMs === fp.mtimeMs &&
|
||||
cache.dbSizeBytes === fp.size &&
|
||||
typeof cache.lookbackFloor === 'string' &&
|
||||
cache.lookbackFloor <= requestedFloor
|
||||
) {
|
||||
return cache.calls
|
||||
const versioned = await readExistingTextFile(getCachePath())
|
||||
if (versioned.status === 'ok') {
|
||||
try {
|
||||
const cache = JSON.parse(versioned.text) as ResultCache
|
||||
if (cache && typeof cache === 'object' && isCurrentHit(cache, fp, requestedFloor)) return cache.calls
|
||||
} catch {}
|
||||
return null
|
||||
}
|
||||
if (versioned.status === 'unreadable') return null
|
||||
|
||||
// Versioned file is absent (ENOENT). Adopt the unsuffixed file only when its
|
||||
// version and fingerprint match — old binaries still own that path.
|
||||
const legacy = await readCacheFile(getLegacyCachePath())
|
||||
if (legacy && isCurrentHit(legacy, fp, requestedFloor)) return legacy.calls
|
||||
return null
|
||||
} catch {
|
||||
return null
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ import { join } from 'path'
|
|||
import { getCodeburnCacheDir } from './cache-dir.js'
|
||||
import type { DateRange, ProjectSummary } from './types.js'
|
||||
|
||||
// Bumped to 25: copilot input/cache tokens for sessions covered by the CLI's
|
||||
// Bumped to 26: copilot input/cache tokens for sessions covered by the CLI's
|
||||
// session-store.db move from one shutdown-rollup lump (stamped at session end)
|
||||
// to per-request DB rows with real timestamps, supplementary accounting calls
|
||||
// (rollups, residuals, paired rows) stop counting as api/model calls, and
|
||||
|
|
@ -15,16 +15,29 @@ import type { DateRange, ProjectSummary } from './types.js'
|
|||
// costs all move, so days finalized under an earlier version would disagree
|
||||
// with the live parse.
|
||||
//
|
||||
// Why 25 and not 21 (the number this change first claimed): 20 is #1040 (codex
|
||||
// Why 26 and not 21 (the number this change first claimed): 20 is #1040 (codex
|
||||
// model attribution), 18 was burned by an earlier public head of THIS change
|
||||
// under different accounting, and 21-24 landed on main while this branch was in
|
||||
// validation. 25 is the first free number on main's ladder — and it is load
|
||||
// under different accounting, and 21-25 landed on main while this branch was in
|
||||
// validation — 25 in particular was spent by #1056 (`codex-auto-review`
|
||||
// pricing) after this PR had already minted it, which is why the number moves
|
||||
// again here. 26 is the first free number on main's ladder — and it is load
|
||||
// bearing beyond the collision: the branch's own validators already hold
|
||||
// daily-cache.v21.json files whose copilot slices were carried stale by the
|
||||
// bug PENDING_REDERIVE_PROVIDERS fixes below, and only a number those files
|
||||
// cannot claim gets them re-derived. isMigratableCache/adoptOlderDailyCaches
|
||||
// carry a same-or-newer version forward as FINALIZED without re-deriving it,
|
||||
// so a number can never mean two accountings.
|
||||
// so a number can never mean two accountings. (feat/core-extraction also sits
|
||||
// at 26 and reconciles at its final merge by keeping the max.)
|
||||
//
|
||||
// Bumped to 25: `codex-auto-review` now prices as the recommended GPT-5.5
|
||||
// row (#1047). Days already finalized under v24 keep that id at $0
|
||||
// forever unless MIN_SUPPORTED_VERSION moves: the daily cache has no
|
||||
// per-provider invalidation. The Codex parse version and CODEX_CACHE_VERSION
|
||||
// move with this so the lower caches reprice first; this pass then re-derives
|
||||
// ALL days from the warm session cache (seconds, not a full re-parse).
|
||||
// adoptOlderDailyCaches keeps the superseded file as the baseline. v21 is
|
||||
// #946, v22 was this PR's earlier claim, v23 is #1075, v24 is #1090.
|
||||
//
|
||||
// Bumped to 20: the Codex fast-path read a nested
|
||||
// `base_instructions.provenance.model` out of `session_meta` as if it were
|
||||
// `payload.model` (#1040), so every call a rollout attributed from session
|
||||
|
|
@ -129,11 +142,32 @@ import type { DateRange, ProjectSummary } from './types.js'
|
|||
// that older binaries skipped. v8 added local-model savings to the daily
|
||||
// rollup; the `savingsConfigHash` field is invalidated separately when the
|
||||
// user changes their `localModelSavings` mapping.
|
||||
export const DAILY_CACHE_VERSION = 25
|
||||
const MIN_SUPPORTED_VERSION = 25
|
||||
// v23: codex pricing fix (#1075) - reasoning tokens were billed on top of
|
||||
// output (they are a subset of it) and cache_write_input_tokens was ignored, so
|
||||
// days finalized at v20 carry codex costs overstated by ~3.5% and codex output
|
||||
// tokens overstated by ~34.6%. Raising MIN_SUPPORTED_VERSION forces the
|
||||
// one-time re-derivation.
|
||||
// It takes 23, not 21: v21 is claimed by the #946 landing branch and v22 by
|
||||
// PR #1056, so those numbers are spoken for and reusing one would let two
|
||||
// incompatible schemas share a filename. (feat/core-extraction sits at 26 and
|
||||
// reconciles at its final merge by keeping the max.)
|
||||
//
|
||||
// v24: gpt-5.6-codex / gpt-5.6-codex-max pricing (#1077) - added as explicit
|
||||
// litellm-snapshot.json rows. getModelCosts already resolved both ids to the
|
||||
// correct rate via the `gpt-5.6` prefix fallback before this landed, so a day
|
||||
// finalized on any binary that had a `gpt-5.6` snapshot row already carries
|
||||
// the right cost; this bump only matters for a day finalized before THAT (a
|
||||
// window where neither existed). The daily cache has no per-provider
|
||||
// invalidation, so there is no way to tell those days apart from here -
|
||||
// raising MIN_SUPPORTED_VERSION forces the one-time re-derivation for
|
||||
// everyone, which is a lossless no-op for days already correct.
|
||||
// v25: #1047 activity-id pricing. v24 on main already shipped #1090.
|
||||
// v26: #946 copilot session-store accounting (see the top of this ladder).
|
||||
export const DAILY_CACHE_VERSION = 26
|
||||
const MIN_SUPPORTED_VERSION = 26
|
||||
|
||||
/// Providers whose per-day CALL COUNT means something different at
|
||||
/// DAILY_CACHE_VERSION 25 than it did before it. Copilot's supplementary
|
||||
/// DAILY_CACHE_VERSION 26 than it did before it. Copilot's supplementary
|
||||
/// accounting calls (rollups, residuals, store rows paired with a per-turn
|
||||
/// call) stopped counting as api calls here, so a settled day's re-derivation
|
||||
/// legitimately reports FEWER calls than the cache holds — which is exactly
|
||||
|
|
|
|||
|
|
@ -747,6 +747,11 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
|
|||
const anyEstimated = Object.values(modelTotals).some(d => d.estimatedCostUSD > 0)
|
||||
const sorted = Object.entries(modelTotals).sort(([, a], [, b]) => b.costUSD - a.costUSD)
|
||||
const costLabels = sorted.map(([, data]) => markEstimated(formatCost(data.costUSD), data.estimatedCostUSD > 0))
|
||||
// #1088: this column has zero width slack left at the standard 3-column
|
||||
// breakpoint (verified: widening the header even one character clips
|
||||
// 'cache'/'1-shot' and drops the value column entirely), so the header stays
|
||||
// "Tok/s" -- the legend below carries the "Effective Tok/s" framing and the
|
||||
// caveat that it is not a vendor decode-speed figure.
|
||||
const headers = ['cost', 'cache', 'calls', '1-shot', 'Tok/s']
|
||||
const values = sorted.map(([model, data], index) => {
|
||||
const totalInput = data.freshInput + data.cacheRead + data.cacheWrite
|
||||
|
|
@ -807,7 +812,7 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
|
|||
{anyEstimated && (
|
||||
<Text dimColor wrap="truncate-end">~ estimated cost (priced from estimated tokens)</Text>
|
||||
)}
|
||||
<Text dimColor wrap="truncate-end">~ Tok/s: generated tokens / active time; tool wait excluded</Text>
|
||||
<Text dimColor wrap="truncate-end">~ Effective Tok/s: generated tokens ÷ time the agent spent waiting on the model, tool execution excluded. Includes prefill, request assembly and reasoning. Not comparable to vendor decode-speed figures.</Text>
|
||||
</Panel>
|
||||
)
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -64,6 +64,7 @@ type SessionLabelEntry = {
|
|||
key: string
|
||||
info: SessionLabelInfo
|
||||
baseLabel: string
|
||||
idFirst: string
|
||||
}
|
||||
|
||||
function nonNegative(value: number): number {
|
||||
|
|
@ -174,19 +175,41 @@ function preferredSessionTitle(titleCandidates: Map<string, SessionTitleCandidat
|
|||
return cleaned[0]?.title
|
||||
}
|
||||
|
||||
// Chart legend is `max-w-40` at 10px ≈ 24–32 glyphs. A title that is unique in
|
||||
// that window can lead; otherwise the short id must stay in the prefix or
|
||||
// truncated series look identical (the #997 class). Count code points, matching
|
||||
// the title cap — a UTF-16 slice can split an emoji and collide two titles.
|
||||
const VISIBLE_LEGEND_PREFIX = 24
|
||||
|
||||
function visibleLegendPrefix(label: string): string {
|
||||
return Array.from(label).slice(0, VISIBLE_LEGEND_PREFIX).join('')
|
||||
}
|
||||
|
||||
function buildSessionLabels(inputs: Map<string, SessionLabelInfo>): Map<string, string> {
|
||||
// Stable raw-key order makes the residual used-label guard independent of
|
||||
// project/session discovery order when a title happens to match another
|
||||
// label shape.
|
||||
const entries: SessionLabelEntry[] = [...inputs.entries()].map(([key, info]) => {
|
||||
const sessionLabel = preferredSessionTitle(info.titleCandidates)
|
||||
?? shortProjectLabel(info.projectPath, preferredProjectName(info.projectNames))
|
||||
return {
|
||||
key,
|
||||
info,
|
||||
baseLabel: `${shortSessionId(info.sessionId)} (${info.provider}) · ${sessionLabel}`,
|
||||
}
|
||||
const draft: SessionLabelEntry[] = [...inputs.entries()].map(([key, info]) => {
|
||||
const title = preferredSessionTitle(info.titleCandidates)
|
||||
const project = shortProjectLabel(info.projectPath, preferredProjectName(info.projectNames))
|
||||
const shortId = shortSessionId(info.sessionId)
|
||||
const idFirst = title
|
||||
? `${shortId} (${info.provider}) · ${title}`
|
||||
: `${shortId} (${info.provider}) · ${project}`
|
||||
const titleFirst = title ? `${title} (${info.provider})` : idFirst
|
||||
return { key, info, baseLabel: titleFirst, idFirst }
|
||||
}).sort((a, b) => a.key < b.key ? -1 : a.key > b.key ? 1 : 0)
|
||||
|
||||
const titlePrefixCounts = new Map<string, number>()
|
||||
for (const entry of draft) {
|
||||
const prefix = visibleLegendPrefix(entry.baseLabel)
|
||||
titlePrefixCounts.set(prefix, (titlePrefixCounts.get(prefix) ?? 0) + 1)
|
||||
}
|
||||
const entries: SessionLabelEntry[] = draft.map(entry => {
|
||||
const prefix = visibleLegendPrefix(entry.baseLabel)
|
||||
const titleLeads = (titlePrefixCounts.get(prefix) ?? 0) === 1
|
||||
return { ...entry, baseLabel: titleLeads ? entry.baseLabel : entry.idFirst }
|
||||
})
|
||||
const byBaseLabel = new Map<string, SessionLabelEntry[]>()
|
||||
for (const entry of entries) {
|
||||
const group = byBaseLabel.get(entry.baseLabel) ?? []
|
||||
|
|
|
|||
175
src/main.ts
175
src/main.ts
|
|
@ -2,7 +2,7 @@ import { isAbsolute } from 'path'
|
|||
import { Command, Option } from 'commander'
|
||||
import { installMenubarApp } from './menubar-installer.js'
|
||||
import { exportCsv, exportJson, type PeriodExport } from './export.js'
|
||||
import { findUnpricedModels, loadPricing, sanitizeModelForDisplay, setModelAliases, setPriceOverrides, setLocalModelSavings, setProxyPaths, normalizeProxyPath } from './models.js'
|
||||
import { findUnpricedModels, loadPricing, sanitizeModelForDisplay, setModelAliases, setPriceOverrides, setLocalModelSavings, setFlatRateModels, setFlatRateRemoved, setProxyPaths, normalizeProxyPath, unpricedModelHint, isBuiltInFlatRateModel, isSameFlatRateModel } from './models.js'
|
||||
import { parseAllSessions, filterProjectsByName, filterProjectsByDateRange, clearSessionCache, setInteractiveScanUI } from './parser.js'
|
||||
import { allProviderNames, getAllProviders } from './providers/index.js'
|
||||
import { getProvider } from './providers/index.js'
|
||||
|
|
@ -10,7 +10,7 @@ import { convertCost, formatCost } from './currency.js'
|
|||
import { renderStatusBar } from './format.js'
|
||||
import { toDateString } from './daily-cache.js'
|
||||
import { dateKey } from './day-aggregator.js'
|
||||
import { isBehavioralCall, isBehavioralTurn } from './behavioral-weight.js'
|
||||
import { isBehavioralCall } from './behavioral-weight.js'
|
||||
import { CATEGORY_LABELS, type DateRange, type ProjectSummary, type TaskCategory } from './types.js'
|
||||
import type { AppliedFix } from './act/types.js'
|
||||
import { aggregateModelEfficiency } from './model-efficiency.js'
|
||||
|
|
@ -466,6 +466,8 @@ program.hook('preAction', async (thisCommand) => {
|
|||
setModelAliases(config.modelAliases ?? {})
|
||||
setPriceOverrides(config.priceOverrides ?? {})
|
||||
setLocalModelSavings(config.localModelSavings ?? {})
|
||||
setFlatRateModels(config.flatRateModels ?? [])
|
||||
setFlatRateRemoved(config.flatRateModelsRemoved ?? [])
|
||||
setProxyPaths(config.proxyPaths ?? [])
|
||||
if (thisCommand.opts<{ verbose?: boolean }>().verbose) {
|
||||
process.env['CODEBURN_VERBOSE'] = '1'
|
||||
|
|
@ -473,7 +475,7 @@ program.hook('preAction', async (thisCommand) => {
|
|||
await loadCurrency()
|
||||
})
|
||||
|
||||
function buildJsonReport(projects: ProjectSummary[], period: string, periodKey: string, durable?: DurablePeriod) {
|
||||
function buildJsonReport(projects: ProjectSummary[], period: string, periodKey: string, durable: DurablePeriod) {
|
||||
const sessions = projects.flatMap(p => p.sessions)
|
||||
const { code } = getCurrency()
|
||||
|
||||
|
|
@ -481,73 +483,28 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey:
|
|||
// session files have expired still count), matching the menubar exactly. The
|
||||
// proxied/net split is a surviving-session concept (subscription attribution
|
||||
// isn't stored per day), so it stays live; net is taken off the durable total.
|
||||
const totalCostUSD = durable ? durable.data.cost : projects.reduce((s, p) => s + p.totalCostUSD, 0)
|
||||
const totalSavingsUSD = durable ? durable.data.savingsUSD : projects.reduce((s, p) => s + p.totalSavingsUSD, 0)
|
||||
const totalEstimatedUSD = durable ? (durable.data.estimatedCostUSD ?? 0) : projects.reduce((s, p) => s + (p.totalEstimatedCostUSD ?? 0), 0)
|
||||
const totalCostUSD = durable.data.cost
|
||||
const totalSavingsUSD = durable.data.savingsUSD
|
||||
const totalEstimatedUSD = durable.data.estimatedCostUSD ?? 0
|
||||
// Subscription-covered (proxied) portion of totalCostUSD, and the resulting
|
||||
// out-of-pocket figure. `cost` stays the full billable/would-be amount.
|
||||
const totalProxiedUSD = projects.reduce((s, p) => s + p.totalProxiedCostUSD, 0)
|
||||
const netCostUSD = totalCostUSD - totalProxiedUSD
|
||||
const totalCalls = durable ? durable.data.calls : projects.reduce((s, p) => s + p.totalApiCalls, 0)
|
||||
const totalSessions = durable ? durable.data.sessions : projects.reduce((s, p) => s + p.sessions.length, 0)
|
||||
const totalInput = durable ? durable.data.inputTokens : sessions.reduce((s, sess) => s + sess.totalInputTokens, 0)
|
||||
const totalOutput = durable ? durable.data.outputTokens : sessions.reduce((s, sess) => s + sess.totalOutputTokens, 0)
|
||||
const totalCacheRead = durable ? durable.data.cacheReadTokens : sessions.reduce((s, sess) => s + sess.totalCacheReadTokens, 0)
|
||||
const totalCacheWrite = durable ? durable.data.cacheWriteTokens : sessions.reduce((s, sess) => s + sess.totalCacheWriteTokens, 0)
|
||||
const totalCalls = durable.data.calls
|
||||
const totalSessions = durable.data.sessions
|
||||
const totalInput = durable.data.inputTokens
|
||||
const totalOutput = durable.data.outputTokens
|
||||
const totalCacheRead = durable.data.cacheReadTokens
|
||||
const totalCacheWrite = durable.data.cacheWriteTokens
|
||||
// Match src/menubar-json.ts:cacheHitPercent: reads over reads+fresh-input. cache_write
|
||||
// counts tokens being stored, not served, so it doesn't belong in the denominator.
|
||||
const cacheHitDenom = totalInput + totalCacheRead
|
||||
const cacheHitPercent = cacheHitDenom > 0 ? Math.round((totalCacheRead / cacheHitDenom) * 1000) / 10 : 0
|
||||
|
||||
// Per-day rollup. Mirrors parser.ts categoryBreakdown semantics so a
|
||||
// consumer summing daily[].editTurns over a period gets the same total as
|
||||
// sum(activities[].editTurns) for that period: every turn counts once for
|
||||
// `turns`, edit turns count for `editTurns`, edit turns with zero retries
|
||||
// count for `oneShotTurns`. Issue #279 — daily-resolution efficiency
|
||||
// dashboards need this without re-deriving from activity-level rollups.
|
||||
const dailyMap: Record<string, { cost: number; savings: number; calls: number; turns: number; editTurns: number; oneShotTurns: number }> = {}
|
||||
for (const sess of sessions) {
|
||||
for (const turn of sess.turns) {
|
||||
// Prefer the user-message timestamp on the turn; fall back to the first
|
||||
// assistant-call timestamp when the user line is missing (continuation
|
||||
// sessions where the JSONL begins mid-conversation). Previously these
|
||||
// turns dropped from daily but stayed in activities, breaking the
|
||||
// sum(daily[].editTurns) === sum(activities[].editTurns) invariant.
|
||||
const ts = turn.timestamp || turn.assistantCalls[0]?.timestamp
|
||||
if (!ts) { continue }
|
||||
const day = dateKey(ts)
|
||||
if (!dailyMap[day]) { dailyMap[day] = { cost: 0, savings: 0, calls: 0, turns: 0, editTurns: 0, oneShotTurns: 0 } }
|
||||
// Turn weight follows day-aggregator.ts exactly: a turn whose calls are
|
||||
// all supplementary accounting (copilot rollup / paired store rows) is
|
||||
// not a behavioral exchange, so it adds cost below but no turn/edit
|
||||
// weight here — otherwise this fallback disagrees with durable.days.
|
||||
if (isBehavioralTurn(turn)) {
|
||||
dailyMap[day].turns += 1
|
||||
if (turn.hasEdits) {
|
||||
dailyMap[day].editTurns += 1
|
||||
if (turn.retries === 0) dailyMap[day].oneShotTurns += 1
|
||||
}
|
||||
}
|
||||
for (const call of turn.assistantCalls) {
|
||||
// Cost/savings/calls bucket under each call's OWN day — the same
|
||||
// per-call rule as the durable day set (day-aggregator.ts), so this
|
||||
// fallback and durable.days never diverge on a midnight-straddling
|
||||
// turn (issue #852). Turn counts/edit stats stay anchored on the
|
||||
// turn's day above. An unparseable call timestamp falls back to the
|
||||
// turn's day rather than producing a garbage date key.
|
||||
const callDay = Number.isNaN(new Date(call.timestamp).getTime()) ? day : dateKey(call.timestamp)
|
||||
if (!dailyMap[callDay]) { dailyMap[callDay] = { cost: 0, savings: 0, calls: 0, turns: 0, editTurns: 0, oneShotTurns: 0 } }
|
||||
dailyMap[callDay].cost += call.costUSD
|
||||
dailyMap[callDay].savings += call.savingsUSD ?? 0
|
||||
dailyMap[callDay].calls += isBehavioralCall(call) ? 1 : 0
|
||||
}
|
||||
}
|
||||
}
|
||||
// Daily rows come from the same durable day set as the headline so they sum
|
||||
// to it, carried days included. The live per-turn rollup (dailyMap) is only
|
||||
// the fallback for callers that pass no durable period.
|
||||
const daily = durable
|
||||
? durable.days.map(d => {
|
||||
// to it, carried days included. Both JSON call sites always pass durable
|
||||
// (#1067); the live dailyMap fallback was unreachable and is gone.
|
||||
const daily = durable.days.map(d => {
|
||||
const turns = Object.values(d.categories).reduce((s, c) => s + c.turns, 0)
|
||||
return {
|
||||
date: d.date,
|
||||
|
|
@ -562,21 +519,6 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey:
|
|||
: null,
|
||||
}
|
||||
})
|
||||
: Object.entries(dailyMap).sort().map(([date, d]) => ({
|
||||
date,
|
||||
cost: convertCost(d.cost),
|
||||
savings: convertCost(d.savings),
|
||||
calls: d.calls,
|
||||
turns: d.turns,
|
||||
editTurns: d.editTurns,
|
||||
oneShotTurns: d.oneShotTurns,
|
||||
// Pre-computed convenience for dashboards that don't want to do the math.
|
||||
// null when there are no edit turns (the rate is undefined, not zero —
|
||||
// a day where the user only had Q&A turns shouldn't read as 0% one-shot).
|
||||
oneShotRate: d.editTurns > 0
|
||||
? Math.round((d.oneShotTurns / d.editTurns) * 1000) / 10
|
||||
: null,
|
||||
}))
|
||||
|
||||
const projectList = projects.map(p => ({
|
||||
name: p.project,
|
||||
|
|
@ -1588,6 +1530,87 @@ program
|
|||
console.log(` Config: ${getConfigFilePath()}\n`)
|
||||
})
|
||||
|
||||
program
|
||||
.command('model-flat-rate [model]')
|
||||
.description('Mark a model as subscription / flat-rate billed. $0 is the correct cost and the unpriced warning is silenced. Do not use model-alias for these — that maps them onto another model\'s per-token rate and invents spend (e.g. codeburn model-flat-rate auto-genius).')
|
||||
.option('--remove <model>', 'Remove a flat-rate mark, including a built-in SKU')
|
||||
.option('--list', 'List configured flat-rate models and built-in opt-outs')
|
||||
.action(async (model?: string, opts?: { remove?: string; list?: boolean }) => {
|
||||
const config = await readConfig()
|
||||
const marked = [...(config.flatRateModels ?? [])]
|
||||
const removed = [...(config.flatRateModelsRemoved ?? [])]
|
||||
|
||||
if (opts?.list || (!model && !opts?.remove)) {
|
||||
if (marked.length === 0 && removed.length === 0) {
|
||||
console.log('\n No flat-rate models configured.')
|
||||
console.log(` Config: ${getConfigFilePath()}`)
|
||||
console.log(' Add one with: codeburn model-flat-rate <model>\n')
|
||||
} else {
|
||||
if (marked.length > 0) {
|
||||
console.log('\n Flat-rate / subscription models:')
|
||||
for (const name of marked) {
|
||||
console.log(` ${name}`)
|
||||
}
|
||||
}
|
||||
if (removed.length > 0) {
|
||||
console.log('\n Built-in flat-rate opt-outs (unpriced warning fires again):')
|
||||
for (const name of removed) {
|
||||
console.log(` ${name}`)
|
||||
}
|
||||
}
|
||||
console.log(` Config: ${getConfigFilePath()}\n`)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if (opts?.remove) {
|
||||
const target = opts.remove
|
||||
const idx = marked.indexOf(target)
|
||||
const builtIn = isBuiltInFlatRateModel(target)
|
||||
const alreadyOptedOut = removed.some(id => isSameFlatRateModel(id, target))
|
||||
if (idx < 0 && (!builtIn || alreadyOptedOut)) {
|
||||
console.error(`\n No flat-rate mark found for: ${target}\n`)
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
if (idx >= 0) {
|
||||
marked.splice(idx, 1)
|
||||
config.flatRateModels = marked.length > 0 ? marked : undefined
|
||||
}
|
||||
if (builtIn && !alreadyOptedOut) {
|
||||
removed.push(target)
|
||||
config.flatRateModelsRemoved = removed
|
||||
}
|
||||
await saveConfig(config)
|
||||
console.log(`\n Removed flat-rate mark: ${target}`)
|
||||
if (builtIn) {
|
||||
console.log(' Built-in SKU opted out; the unpriced warning will fire again until you re-add it.')
|
||||
}
|
||||
console.log()
|
||||
return
|
||||
}
|
||||
|
||||
if (!model) {
|
||||
console.error('\n Usage: codeburn model-flat-rate <model>\n')
|
||||
process.exitCode = 1
|
||||
return
|
||||
}
|
||||
|
||||
if (!marked.includes(model)) marked.push(model)
|
||||
config.flatRateModels = marked
|
||||
const remainingOptOuts = removed.filter(id => !isSameFlatRateModel(id, model))
|
||||
config.flatRateModelsRemoved = remainingOptOuts.length > 0 ? remainingOptOuts : undefined
|
||||
await saveConfig(config)
|
||||
|
||||
if (config.modelAliases && Object.hasOwn(config.modelAliases, model)) {
|
||||
console.log(`\n Note: ${model} is also in modelAliases (-> ${config.modelAliases[model]}).`)
|
||||
console.log(' The alias still invents per-token spend. Remove it if $0 is the correct cost.')
|
||||
}
|
||||
|
||||
console.log(`\n Flat-rate mark saved: ${model}`)
|
||||
console.log(` Config: ${getConfigFilePath()}\n`)
|
||||
})
|
||||
|
||||
program
|
||||
.command('proxy-path [path]')
|
||||
.description('Mark a project directory as routed through a subscription-backed LLM proxy (e.g. Claude Code over GitHub Copilot). Sessions whose canonical path is under it keep their full API-rate cost as the "would-be" figure, but that amount is reported as subscription-covered so the report can show net out-of-pocket (e.g. codeburn proxy-path ~/work/copilot-repo). Actual API-key sessions elsewhere are untouched.')
|
||||
|
|
@ -2173,7 +2196,7 @@ program
|
|||
process.stdout.write(renderTable(renderRows, { byTask: !!opts.byTask, byAgent: !!opts.byAgent, showTotals: opts.totals !== false }) + '\n')
|
||||
// Never advise aliasing unconditionally: a subscription or flat-rate model
|
||||
// is correctly $0, and mapping it onto another model's rate invents spend.
|
||||
if (opts.unpriced) process.stdout.write('If a model is billed per token, map it with: codeburn model-alias "<model>" <known-model>. Subscription or flat-rate models are correctly $0.\n')
|
||||
if (opts.unpriced) process.stdout.write(unpricedModelHint() + '\n')
|
||||
} else {
|
||||
process.stderr.write(`codeburn: unknown --format "${opts.format}". Choose table, markdown, json, or csv.\n`)
|
||||
process.exit(1)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import { formatCost, formatTokens, markEstimated } from '../format.js'
|
||||
import type { MenubarPayload } from '../menubar-json.js'
|
||||
import { unpricedModelHint } from '../models.js'
|
||||
|
||||
const ESTIMATED_LEGEND = '_~ estimated cost (priced from estimated tokens)_'
|
||||
const isEstimated = (m: { estimatedCostUSD?: number }) => (m.estimatedCostUSD ?? 0) > 0
|
||||
|
|
@ -23,7 +24,7 @@ export function renderSummaryTable(p: MenubarPayload): string {
|
|||
`**${c.label}** — ${formatCost(c.cost)} · ${c.calls} calls · ${c.sessions} sessions`,
|
||||
`cache hit ${pct(c.cacheHitPercent)} · one-shot ${oneShot(c.oneShotRate)} · in ${formatTokens(c.inputTokens)} / out ${formatTokens(c.outputTokens)}`,
|
||||
...(unpriced.length > 0
|
||||
? [`⚠ ${unpriced.length} model${unpriced.length === 1 ? '' : 's'} unpriced, counted at $0: ${unpriced.map(u => `${u.model} (${u.calls} calls)`).join(', ')}. Cost above understates real spend; fix with \`codeburn model-alias\` or \`codeburn price-override\`.`]
|
||||
? [`⚠ ${unpriced.length} model${unpriced.length === 1 ? '' : 's'} unpriced, counted at $0: ${unpriced.map(u => `${u.model} (${u.calls} calls)`).join(', ')}. ${unpricedModelHint()}`]
|
||||
: []),
|
||||
'',
|
||||
'_Top models_',
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import stripAnsi from 'strip-ansi'
|
|||
import { isBehavioralCall } from './behavioral-weight.js'
|
||||
import { codexCredits } from './codex-credits.js'
|
||||
import { formatCost, formatTokens } from './format.js'
|
||||
import { sanitizeModelForDisplay } from './models.js'
|
||||
import { billableOutputTokens, fallbackRawModelDisplayName, getShortModelName, resolveCanonicalModelId, sanitizeModelForDisplay } from './models.js'
|
||||
import { getProvider } from './providers/index.js'
|
||||
import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
|
||||
|
||||
|
|
@ -29,8 +29,12 @@ export type ModelReportRow = {
|
|||
savingsBaselineModel: string
|
||||
calls: number
|
||||
/// Codex credit consumption (issues #408/#495). null for non-Codex models or
|
||||
/// Codex models without a known credit rate.
|
||||
/// Codex models without a known credit rate. A merged row that mixed rated
|
||||
/// and unrated buckets stores the partial sum of the rated ones.
|
||||
credits: number | null
|
||||
/// True when `credits` is a partial sum because some contributing buckets
|
||||
/// had no known credit rate.
|
||||
creditsIncomplete?: boolean
|
||||
topCategory?: TaskCategory
|
||||
topCategoryCost?: number
|
||||
topCategoryShare?: number
|
||||
|
|
@ -74,9 +78,10 @@ function bucketKey(provider: string, model: string, category: TaskCategory | nul
|
|||
}
|
||||
|
||||
/// Walks every parsed turn, attributes each assistant call to a
|
||||
/// (provider, model, category, agent) bucket, and returns rows keyed by
|
||||
/// (provider, model) by default, (provider, model, category) under `byTask`, or
|
||||
/// (provider, model, agent) under `byAgent`.
|
||||
/// (provider, raw-model, category, agent) bucket, then merges buckets that
|
||||
/// resolve to the same provider + alias-resolved canonical id. Display names
|
||||
/// stay cosmetic. Returned rows are keyed by (provider, canonical id) by
|
||||
/// default, plus category under `byTask` or agent under `byAgent`.
|
||||
///
|
||||
/// Default view: rows sorted by cost descending.
|
||||
/// byTask / byAgent view: rows grouped by (provider, model) so the renderer can
|
||||
|
|
@ -121,11 +126,7 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat
|
|||
buckets.set(key, bucket)
|
||||
}
|
||||
bucket.inputTokens += call.usage.inputTokens
|
||||
// Copilot reasoning tokens are already INSIDE outputTokens (same rule as cachedCallToApiCall
|
||||
// in parser.ts), so folding them in would display phantom output for its store rows/rollups.
|
||||
bucket.outputTokens += provider === 'copilot'
|
||||
? call.usage.outputTokens
|
||||
: call.usage.outputTokens + call.usage.reasoningTokens
|
||||
bucket.outputTokens += billableOutputTokens(provider, call.usage.outputTokens, call.usage.reasoningTokens)
|
||||
bucket.cacheWriteTokens += call.usage.cacheCreationInputTokens
|
||||
// cacheReadInputTokens (Anthropic vocab) and cachedInputTokens (OpenAI vocab)
|
||||
// are two names for the same thing. Providers populate one or set both to the
|
||||
|
|
@ -162,74 +163,133 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat
|
|||
const entry = {
|
||||
displayName: p?.displayName ?? name,
|
||||
formatModel: p
|
||||
? (m: string) => sanitizeModelForDisplay(p.modelDisplayName(m))
|
||||
: sanitizeModelForDisplay,
|
||||
? (m: string) => sanitizeModelForDisplay(fallbackRawModelDisplayName(p.modelDisplayName(m), m))
|
||||
: (m: string) => sanitizeModelForDisplay(getShortModelName(m)),
|
||||
}
|
||||
providerCache.set(name, entry)
|
||||
return entry
|
||||
}
|
||||
|
||||
const rows: ModelReportRow[] = []
|
||||
const rowsByKey = new Map<string, ModelReportRow>()
|
||||
const foldedCategoryCost = new Map<string, Map<CategoryKey, number>>()
|
||||
const foldedTotalCost = new Map<string, number>()
|
||||
const foldedRawSeen = new Set<string>()
|
||||
// Empty string on the row means both "none seen" and "conflict". Track
|
||||
// distinct non-empty baselines separately so a later bucket cannot
|
||||
// repopulate a conflict that was already detected.
|
||||
const baselinesByKey = new Map<string, Set<string>>()
|
||||
|
||||
for (const bucket of buckets.values()) {
|
||||
const meta = await resolveProvider(bucket.provider)
|
||||
const modelDisplayName = meta.formatModel(bucket.model)
|
||||
const canonicalId = resolveCanonicalModelId(bucket.model)
|
||||
const resolvedKey = bucketKey(bucket.provider, canonicalId, bucket.category, bucket.agentType)
|
||||
const foldKey = `${bucket.provider} ${canonicalId}`
|
||||
const total = bucket.inputTokens + bucket.outputTokens + bucket.cacheWriteTokens + bucket.cacheReadTokens
|
||||
const row: ModelReportRow = {
|
||||
provider: bucket.provider,
|
||||
providerDisplayName: meta.displayName,
|
||||
model: bucket.model,
|
||||
modelDisplayName: meta.formatModel(bucket.model),
|
||||
category: bucket.category,
|
||||
agentType: bucket.agentType,
|
||||
inputTokens: bucket.inputTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
cacheWriteTokens: bucket.cacheWriteTokens,
|
||||
cacheReadTokens: bucket.cacheReadTokens,
|
||||
totalTokens: total,
|
||||
costUSD: bucket.costUSD,
|
||||
savingsUSD: bucket.savingsUSD,
|
||||
savingsBaselineModel: bucket.savingsBaselineModel,
|
||||
calls: bucket.calls,
|
||||
// outputTokens already includes reasoning (folded in above), and for Codex
|
||||
// inputTokens is non-cached with cacheReadTokens holding cached input, which
|
||||
// is exactly what the credit rates expect.
|
||||
credits: bucket.provider === 'codex'
|
||||
? codexCredits(bucket.model, {
|
||||
inputTokens: bucket.inputTokens,
|
||||
cachedReadTokens: bucket.cacheReadTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
})
|
||||
: null,
|
||||
// Credits are per raw id (aliases can have different rates). Sum the
|
||||
// rated buckets and flag the row incomplete when any contributor is
|
||||
// unrated — nulling the whole merge would zero a real menubar total.
|
||||
// outputTokens is already the billable output (for Codex that includes
|
||||
// reasoning, so nothing is added on top), and inputTokens is non-cached
|
||||
// with cacheReadTokens holding cached input - exactly what the credit
|
||||
// rates expect.
|
||||
const bucketCredits = bucket.provider === 'codex'
|
||||
? codexCredits(bucket.model, {
|
||||
inputTokens: bucket.inputTokens,
|
||||
cachedReadTokens: bucket.cacheReadTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
})
|
||||
: null
|
||||
|
||||
const baselines = baselinesByKey.get(resolvedKey) ?? new Set<string>()
|
||||
if (bucket.savingsBaselineModel) baselines.add(bucket.savingsBaselineModel)
|
||||
baselinesByKey.set(resolvedKey, baselines)
|
||||
const resolvedBaseline = baselines.size === 1 ? [...baselines][0]! : ''
|
||||
|
||||
const existing = rowsByKey.get(resolvedKey)
|
||||
if (existing) {
|
||||
existing.inputTokens += bucket.inputTokens
|
||||
existing.outputTokens += bucket.outputTokens
|
||||
existing.cacheWriteTokens += bucket.cacheWriteTokens
|
||||
existing.cacheReadTokens += bucket.cacheReadTokens
|
||||
existing.totalTokens += total
|
||||
existing.costUSD += bucket.costUSD
|
||||
existing.savingsUSD += bucket.savingsUSD
|
||||
existing.calls += bucket.calls
|
||||
existing.savingsBaselineModel = resolvedBaseline
|
||||
const existingRated = existing.credits !== null
|
||||
const incomingRated = bucketCredits !== null
|
||||
if (incomingRated) existing.credits = (existing.credits ?? 0) + bucketCredits
|
||||
if (existingRated !== incomingRated) existing.creditsIncomplete = true
|
||||
} else {
|
||||
rowsByKey.set(resolvedKey, {
|
||||
provider: bucket.provider,
|
||||
providerDisplayName: meta.displayName,
|
||||
model: bucket.model,
|
||||
modelDisplayName,
|
||||
category: bucket.category,
|
||||
agentType: bucket.agentType,
|
||||
inputTokens: bucket.inputTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
cacheWriteTokens: bucket.cacheWriteTokens,
|
||||
cacheReadTokens: bucket.cacheReadTokens,
|
||||
totalTokens: total,
|
||||
costUSD: bucket.costUSD,
|
||||
savingsUSD: bucket.savingsUSD,
|
||||
savingsBaselineModel: resolvedBaseline,
|
||||
calls: bucket.calls,
|
||||
credits: bucketCredits,
|
||||
})
|
||||
}
|
||||
|
||||
if (!opts.byTask && !opts.byAgent) {
|
||||
const perCat = perModelCategoryCost.get(`${bucket.provider} ${bucket.model}`)
|
||||
if (perCat && perCat.size > 0) {
|
||||
let topCat: TaskCategory = 'general'
|
||||
let topCost = -1
|
||||
let totalCost = 0
|
||||
for (const [cat, cost] of perCat.entries()) {
|
||||
totalCost += cost
|
||||
if (cost > topCost) {
|
||||
topCost = cost
|
||||
topCat = cat
|
||||
}
|
||||
const rawKey = `${bucket.provider} ${bucket.model}`
|
||||
if (!foldedRawSeen.has(rawKey)) {
|
||||
foldedRawSeen.add(rawKey)
|
||||
const rawCat = perModelCategoryCost.get(rawKey)
|
||||
if (rawCat) {
|
||||
let folded = foldedCategoryCost.get(foldKey)
|
||||
if (!folded) {
|
||||
folded = new Map()
|
||||
foldedCategoryCost.set(foldKey, folded)
|
||||
}
|
||||
row.topCategory = topCat
|
||||
row.topCategoryCost = topCost
|
||||
row.topCategoryShare = totalCost > 0 ? topCost / totalCost : 0
|
||||
for (const [cat, cost] of rawCat) {
|
||||
folded.set(cat, (folded.get(cat) ?? 0) + cost)
|
||||
}
|
||||
}
|
||||
foldedTotalCost.set(
|
||||
foldKey,
|
||||
(foldedTotalCost.get(foldKey) ?? 0) + (perModelTotalCost.get(rawKey) ?? 0),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const rows = [...rowsByKey.values()]
|
||||
for (const row of rows) {
|
||||
if (opts.byTask || opts.byAgent) continue
|
||||
const perCat = foldedCategoryCost.get(`${row.provider} ${resolveCanonicalModelId(row.model)}`)
|
||||
if (!perCat || perCat.size === 0) continue
|
||||
let topCat: TaskCategory = 'general'
|
||||
let topCost = -1
|
||||
let totalCost = 0
|
||||
for (const [cat, cost] of perCat.entries()) {
|
||||
totalCost += cost
|
||||
if (cost > topCost) {
|
||||
topCost = cost
|
||||
topCat = cat
|
||||
}
|
||||
}
|
||||
|
||||
rows.push(row)
|
||||
row.topCategory = topCat
|
||||
row.topCategoryCost = topCost
|
||||
row.topCategoryShare = totalCost > 0 ? topCost / totalCost : 0
|
||||
}
|
||||
|
||||
if (opts.byTask || opts.byAgent) {
|
||||
rows.sort((a, b) => {
|
||||
const aTotal = perModelTotalCost.get(`${a.provider} ${a.model}`) ?? 0
|
||||
const bTotal = perModelTotalCost.get(`${b.provider} ${b.model}`) ?? 0
|
||||
const aTotal = foldedTotalCost.get(`${a.provider} ${resolveCanonicalModelId(a.model)}`) ?? 0
|
||||
const bTotal = foldedTotalCost.get(`${b.provider} ${resolveCanonicalModelId(b.model)}`) ?? 0
|
||||
if (aTotal !== bTotal) return bTotal - aTotal
|
||||
if (a.provider !== b.provider) return a.provider.localeCompare(b.provider)
|
||||
if (a.model !== b.model) return a.model.localeCompare(b.model)
|
||||
if (a.modelDisplayName !== b.modelDisplayName) return a.modelDisplayName.localeCompare(b.modelDisplayName)
|
||||
return (b.costUSD + b.savingsUSD) - (a.costUSD + a.savingsUSD)
|
||||
})
|
||||
} else {
|
||||
|
|
@ -489,7 +549,7 @@ export function renderTable(
|
|||
const rowEntries: RowCells[] = []
|
||||
let prevProviderModel = ''
|
||||
for (const row of rows) {
|
||||
const groupKey = `${row.provider} ${row.model}`
|
||||
const groupKey = `${row.provider} ${row.modelDisplayName}`
|
||||
const isNewGroup = !grouped || groupKey !== prevProviderModel
|
||||
prevProviderModel = groupKey
|
||||
const allCells = defaultColumns(byTask, byAgent, showSaved).map(col => {
|
||||
|
|
@ -611,6 +671,7 @@ export function renderJson(rows: ModelReportRow[]): string {
|
|||
savingsUSD: r.savingsUSD,
|
||||
savingsBaselineModel: r.savingsBaselineModel,
|
||||
credits: r.credits,
|
||||
creditsIncomplete: r.creditsIncomplete === true,
|
||||
})),
|
||||
null,
|
||||
2,
|
||||
|
|
|
|||
202
src/models.ts
202
src/models.ts
|
|
@ -13,6 +13,32 @@ export type ModelCosts = {
|
|||
cacheReadCostPerToken: number
|
||||
webSearchCostPerRequest: number
|
||||
fastMultiplier: number
|
||||
/// True only when the pricing source carried a real cache-write rate. When
|
||||
/// absent/false, `cacheWriteCostPerToken` is the fabricated `1.25 x input`
|
||||
/// default, which is right for Anthropic-style pricing but would invent a
|
||||
/// surcharge on providers that charge nothing extra to write cache. Callers
|
||||
/// that decide WHICH bucket to put tokens in (rather than what to multiply
|
||||
/// them by) must consult this before routing tokens to the cache-write
|
||||
/// bucket. Optional so an incomplete literal defaults to the safe answer.
|
||||
cacheWriteCostIsExplicit?: boolean
|
||||
}
|
||||
|
||||
/// Providers whose reported `reasoningTokens` are a SUBSET of `outputTokens`
|
||||
/// rather than a separate bucket to add on top. OpenAI bills reasoning as part
|
||||
/// of output (every codex `token_count` event satisfies input + output ==
|
||||
/// total), and Anthropic folds thinking into output the same way, so summing
|
||||
/// the two double-counts both the cost and the displayed output tokens. Copilot
|
||||
/// is the same case: its per-request token_details_json prices input/cache/output
|
||||
/// and nothing else, and its supplementary store-row/shutdown calls carry
|
||||
/// reasoningTokens with outputTokens 0 while the per-turn assistant.message call
|
||||
/// bills the full output, so adding reasoning on top bills it twice.
|
||||
const REASONING_INCLUDED_IN_OUTPUT = new Set(['claude', 'codex', 'copilot'])
|
||||
|
||||
/// Output tokens to bill and display for one call. Single source of truth so
|
||||
/// the pricing sites and the display sums can never disagree about whether a
|
||||
/// provider's reasoning tokens are already inside its output count (#1075).
|
||||
export function billableOutputTokens(provider: string, outputTokens: number, reasoningTokens: number): number {
|
||||
return REASONING_INCLUDED_IN_OUTPUT.has(provider) ? outputTokens : outputTokens + reasoningTokens
|
||||
}
|
||||
|
||||
type PriceOverrideRates = {
|
||||
|
|
@ -37,6 +63,11 @@ type SnapshotEntry = [number, number, number | null, number | null, (number | nu
|
|||
|
||||
const LITELLM_URL = 'https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json'
|
||||
const CACHE_TTL_MS = 24 * 60 * 60 * 1000
|
||||
// Bump whenever a ModelCosts field changes pricing behavior (cacheWriteCostIsExplicit,
|
||||
// added in #1075/#1078). A cache written under an older/missing version is treated as a
|
||||
// miss instead of read verbatim, so a stale on-disk file can't reintroduce a killed bug
|
||||
// for up to CACHE_TTL_MS after an upgrade.
|
||||
const CACHE_SCHEMA_VERSION = 2
|
||||
const WEB_SEARCH_COST = 0.01
|
||||
const ONE_HOUR_CACHE_WRITE_MULTIPLIER_FROM_FIVE_MINUTE_RATE = 1.6
|
||||
|
||||
|
|
@ -71,6 +102,7 @@ function buildCosts(
|
|||
cacheReadCostPerToken: cacheRead ?? input * 0.1,
|
||||
webSearchCostPerRequest: WEB_SEARCH_COST,
|
||||
fastMultiplier: fast ?? 1,
|
||||
cacheWriteCostIsExplicit: cacheWrite !== null && cacheWrite !== undefined,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -200,6 +232,7 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
|
|||
|
||||
await mkdir(getCodeburnCacheDir(), { recursive: true })
|
||||
await writeFile(getCachePath(), JSON.stringify({
|
||||
version: CACHE_SCHEMA_VERSION,
|
||||
timestamp: Date.now(),
|
||||
data: Object.fromEntries(pricing),
|
||||
}))
|
||||
|
|
@ -210,7 +243,8 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
|
|||
async function loadCachedPricing(): Promise<Map<string, ModelCosts> | null> {
|
||||
try {
|
||||
const raw = await readFile(getCachePath(), 'utf-8')
|
||||
const cached = JSON.parse(raw) as { timestamp: number; data: Record<string, ModelCosts> }
|
||||
const cached = JSON.parse(raw) as { version?: number; timestamp: number; data: Record<string, ModelCosts> }
|
||||
if (cached.version !== CACHE_SCHEMA_VERSION) return null
|
||||
if (Date.now() - cached.timestamp > CACHE_TTL_MS) return null
|
||||
return new Map(Object.entries(cached.data))
|
||||
} catch {
|
||||
|
|
@ -282,6 +316,16 @@ const BUILTIN_ALIASES: Record<string, string> = {
|
|||
'openclaw-auto': 'claude-sonnet-4-5',
|
||||
'warp-auto-efficient': 'gpt-5.3-codex',
|
||||
'warp-auto-powerful': 'claude-opus-4-6',
|
||||
// Codex activity ids are product surfaces, not subscription SKUs and not
|
||||
// LiteLLM rows. OpenAI's tracker (openai/codex#32224) says auto review
|
||||
// consumes normal model usage. Public evidence: review_model defaults to
|
||||
// the session model; GPT-5.5 is the currently recommended review model.
|
||||
// Price as that existing bundled row. Do not invent a rate. Do not treat
|
||||
// the id as honestly $0 — it draws from the same credit pool. Display
|
||||
// stays on autoModelNames (same class as cursor-auto / copilot-openai-auto).
|
||||
// Only alias ids observed in Codex source / real rollouts. Do not infer
|
||||
// `codex-code-review` from the activity name "code review".
|
||||
'codex-auto-review': 'gpt-5.5',
|
||||
'grok-build': 'grok-build-0.1',
|
||||
'GPT-5.3 Codex (low reasoning)': 'gpt-5.3-codex',
|
||||
'GPT-5.3 Codex (medium reasoning)': 'gpt-5.3-codex',
|
||||
|
|
@ -529,6 +573,117 @@ export function getLocalModelSavingsConfigHash(): string {
|
|||
return parts.join('\u0002')
|
||||
}
|
||||
|
||||
// Subscription / flat-rate product SKUs. $0 is the correct cost; aliasing
|
||||
// them onto a per-token row fabricates spend (#968). Distinct from
|
||||
// model-savings (counterfactual local baseline) and from a zero-rate
|
||||
// price-override (user-declared free). Built-in families plus a user hatch.
|
||||
let userFlatRateModels = new Set<string>()
|
||||
let userFlatRateLeaves = new Set<string>()
|
||||
let userFlatRateRemoved = new Set<string>()
|
||||
let userFlatRateRemovedLeaves = new Set<string>()
|
||||
|
||||
function flatRateLeaf(model: string): string {
|
||||
const trimmed = model.trim().replace(/@.*$/, '').replace(/-\d{8}$/, '')
|
||||
const leaf = trimmed.includes('/') ? trimmed.slice(trimmed.lastIndexOf('/') + 1) : trimmed
|
||||
return leaf.toLowerCase()
|
||||
}
|
||||
|
||||
function fillFlatRateSet(
|
||||
models: Iterable<string>,
|
||||
): { ids: Set<string>; leaves: Set<string> } {
|
||||
const ids = new Set<string>()
|
||||
const leaves = new Set<string>()
|
||||
for (const model of models) {
|
||||
if (!model || typeof model !== 'string') continue
|
||||
ids.add(model)
|
||||
const leaf = flatRateLeaf(model)
|
||||
if (leaf) leaves.add(leaf)
|
||||
}
|
||||
return { ids, leaves }
|
||||
}
|
||||
|
||||
export function setFlatRateModels(models: Iterable<string>): void {
|
||||
const filled = fillFlatRateSet(models)
|
||||
userFlatRateModels = filled.ids
|
||||
userFlatRateLeaves = filled.leaves
|
||||
}
|
||||
|
||||
export function setFlatRateRemoved(models: Iterable<string>): void {
|
||||
const filled = fillFlatRateSet(models)
|
||||
userFlatRateRemoved = filled.ids
|
||||
userFlatRateRemovedLeaves = filled.leaves
|
||||
}
|
||||
|
||||
export function getFlatRateModelsConfigHash(): string {
|
||||
const added = [...userFlatRateModels].sort().join('\u0002')
|
||||
const removed = [...userFlatRateRemoved].sort().join('\u0002')
|
||||
if (!removed) return added
|
||||
return `${added}\u0003${removed}`
|
||||
}
|
||||
|
||||
export function getFlatRateModels(): string[] {
|
||||
return [...userFlatRateModels]
|
||||
}
|
||||
|
||||
export function getFlatRateRemoved(): string[] {
|
||||
return [...userFlatRateRemoved]
|
||||
}
|
||||
|
||||
export function isSameFlatRateModel(a: string, b: string): boolean {
|
||||
if (!a || !b) return false
|
||||
if (a === b) return true
|
||||
const leaf = flatRateLeaf(a)
|
||||
return leaf.length > 0 && leaf === flatRateLeaf(b)
|
||||
}
|
||||
|
||||
function isUserFlatRateModel(model: string): boolean {
|
||||
if (userFlatRateModels.has(model)) return true
|
||||
const leaf = flatRateLeaf(model)
|
||||
return leaf.length > 0 && userFlatRateLeaves.has(leaf)
|
||||
}
|
||||
|
||||
function isFlatRateRemoved(model: string): boolean {
|
||||
if (userFlatRateRemoved.has(model)) return true
|
||||
const leaf = flatRateLeaf(model)
|
||||
return leaf.length > 0 && userFlatRateRemovedLeaves.has(leaf)
|
||||
}
|
||||
|
||||
/// Product SKUs billed as a subscription, not missing LiteLLM rows.
|
||||
/// Match raw ids and path-prefixed ids (`cline-pass/auto-genius`). Display
|
||||
/// names from getShortModelName are matched only when the aggregation key
|
||||
/// is not the raw leaf (Warp Auto *, Grok Composer *).
|
||||
export function isBuiltInFlatRateModel(model: string): boolean {
|
||||
const leaf = flatRateLeaf(model)
|
||||
// Warp's product SKU is the bare id `auto`. Kiro rewrites its own `auto`
|
||||
// to `kiro-auto` before pricing, so this leaf does not swallow Kiro.
|
||||
if (
|
||||
leaf === 'auto'
|
||||
|| leaf === 'auto-genius'
|
||||
|| leaf === 'kimi-for-coding-highspeed'
|
||||
) return true
|
||||
if (leaf.startsWith('grok-composer-')) return true
|
||||
if (leaf.startsWith('warp-auto-')) return true
|
||||
const display = model.trim()
|
||||
if (/^grok composer\b/i.test(display)) return true
|
||||
if (/^warp auto\b/i.test(display)) return true
|
||||
return false
|
||||
}
|
||||
|
||||
export function isFlatRateModel(model: string): boolean {
|
||||
if (!model) return false
|
||||
if (isFlatRateRemoved(model)) return false
|
||||
return isUserFlatRateModel(model) || isBuiltInFlatRateModel(model)
|
||||
}
|
||||
|
||||
/// Shared unpriced-warning copy. Never tell the user to alias unconditionally:
|
||||
/// mapping a subscription SKU onto a priced row invents spend. Optional `model`
|
||||
/// interpolates the sanitized id so the verbose calculateCost path names the
|
||||
/// same two hatches.
|
||||
export function unpricedModelHint(model = '<model>'): string {
|
||||
const safe = model.replace(/[\x00-\x1F\x7F-\x9F]/g, '?').slice(0, 200)
|
||||
return `If a model is billed per token, map it with: codeburn model-alias "${safe}" <known-model>. If $0 is correct (subscription / flat-rate): codeburn model-flat-rate "${safe}".`
|
||||
}
|
||||
|
||||
/// Stable hash of the model-alias map, for the same staleness class as the
|
||||
/// hashes below: a resident process (codeburn serve) must not serve memoized
|
||||
/// parse results priced under aliases the user has since changed.
|
||||
|
|
@ -625,6 +780,19 @@ function getCanonicalName(model: string): string {
|
|||
)
|
||||
}
|
||||
|
||||
/// Alias-resolved identity for report merge. Display names stay cosmetic —
|
||||
/// prefix matches in SHORT_NAMES must not fold distinct SKUs into one row.
|
||||
/// Path-form ids (`accounts/fireworks/models/<slug>`, `cline-pass/<slug>`)
|
||||
/// peel to the leaf so they share a bucket with the bare slug.
|
||||
export function resolveCanonicalModelId(model: string): string {
|
||||
const viaUser = Object.hasOwn(userAliases, model) ? userAliases[model]! : model
|
||||
const aliased = resolveAlias(getCanonicalName(viaUser))
|
||||
if (!aliased.includes('/')) return aliased
|
||||
const leaf = aliased.slice(aliased.lastIndexOf('/') + 1)
|
||||
if (!leaf) return aliased
|
||||
return resolveAlias(getCanonicalName(leaf))
|
||||
}
|
||||
|
||||
// Namespaces the pricing catalog itself uses, plus the ones below. An unknown
|
||||
// `provider/model` must stay unpriced — do not treat `/` as authority. Derived
|
||||
// rather than hand-listed so a vendor LiteLLM already knows (`x-ai/`, `qwen/`,
|
||||
|
|
@ -857,14 +1025,18 @@ function exactPriceOverrideFor(model: string): ModelCosts | null {
|
|||
// correct cost, as are zero-rate USER overrides (explicitly declared free).
|
||||
/// Models whose $0 cost is CORRECT rather than a pricing gap, mirroring the
|
||||
/// exclusions findUnpricedModels applies: local-looking models, models mapped
|
||||
/// to a local-savings baseline, and models an exact zero-rate user override
|
||||
/// declares free. Used to keep their calls out of the pricing-coverage
|
||||
/// denominator — otherwise a 95%-ollama user reads high coverage while every
|
||||
/// genuinely cost-bearing call is unpriced.
|
||||
/// to a local-savings baseline, subscription / flat-rate product SKUs, and
|
||||
/// models an exact zero-rate user override declares free. Used to keep their
|
||||
/// calls out of the pricing-coverage denominator — otherwise a 95%-ollama
|
||||
/// user reads high coverage while every genuinely cost-bearing call is unpriced.
|
||||
export function isExpectedFreeModel(model: string): boolean {
|
||||
if (looksLikeLocalModel(model)) return true
|
||||
if (getLocalSavingsBaseline(model)) return true
|
||||
const costs = getModelCosts(model)
|
||||
// A builtin/user alias can still attach a billable rate to a subscription
|
||||
// SKU (warp-auto-* today). Those calls are priced, so they stay in the
|
||||
// coverage denominator. Only the $0 / no-rate case is expected-free.
|
||||
if (isFlatRateModel(model) && (!costs || !hasBillableRate(costs))) return true
|
||||
if (costs && !hasBillableRate(costs) && exactPriceOverrideFor(model)) return true
|
||||
return false
|
||||
}
|
||||
|
|
@ -881,6 +1053,7 @@ export function findUnpricedModels(
|
|||
if (row.cost > 0) continue
|
||||
if (looksLikeLocalModel(model)) continue
|
||||
if (getLocalSavingsBaseline(model)) continue
|
||||
if (isFlatRateModel(model)) continue
|
||||
const costs = getModelCosts(model)
|
||||
if (costs && hasBillableRate(costs)) continue
|
||||
if (costs && exactPriceOverrideFor(model)) continue
|
||||
|
|
@ -897,6 +1070,7 @@ function shouldWarnAboutUnknownModel(name: string): boolean {
|
|||
// actively misleading there. Users who need cost visibility for local
|
||||
// inference can still set an alias via `codeburn model-alias`.
|
||||
if (looksLikeLocalModel(name)) return false
|
||||
if (isFlatRateModel(name)) return false
|
||||
// The warning fired on every CLI invocation (including the default
|
||||
// dashboard) which made first launches look broken — three "no pricing
|
||||
// data" lines greet a user before the dashboard even draws. Now opt-in
|
||||
|
|
@ -929,10 +1103,9 @@ export function calculateCost(
|
|||
// payloads written by external tools, so a hostile or corrupt file
|
||||
// could embed terminal escape sequences here.
|
||||
const safeName = sanitizeModelForDisplay(model)
|
||||
const aliasHint = `Map it with: codeburn model-alias "${safeName}" <known-model>, or track local-model savings with: codeburn model-savings "${safeName}" <baseline-model>`
|
||||
process.stderr.write(
|
||||
`codeburn: no pricing data for model "${safeName}" — costs for this model will show $0. ` +
|
||||
`${aliasHint}, or update with: npx codeburn@latest.\n`
|
||||
`${unpricedModelHint(safeName)} Or track local-model savings with: codeburn model-savings "${safeName}" <baseline-model>, or update with: npx codeburn@latest.\n`,
|
||||
)
|
||||
}
|
||||
return 0
|
||||
|
|
@ -974,6 +1147,7 @@ const autoModelNames: Record<string, string> = {
|
|||
'openclaw-auto': 'OpenClaw (auto)',
|
||||
'qwen-auto': 'Qwen (auto)',
|
||||
'kimi-auto': 'Kimi (auto)',
|
||||
'codex-auto-review': 'Codex Auto Review',
|
||||
}
|
||||
|
||||
const SHORT_NAMES: Record<string, string> = {
|
||||
|
|
@ -1114,6 +1288,14 @@ export function getShortModelName(model: string): string {
|
|||
return shortModelName(model, new Set())
|
||||
}
|
||||
|
||||
/// Provider-first display name. Local labels win (Cursor estimated suffixes,
|
||||
/// provider tables that intentionally override the global map). If the provider
|
||||
/// echoed the raw id, it missed — fall back to the global resolver instead of
|
||||
/// showing `gpt-5.6-sol` / `accounts/fireworks/models/kimi-k2p6`.
|
||||
export function fallbackRawModelDisplayName(localLabel: string, rawModel: string): string {
|
||||
return localLabel === rawModel ? getShortModelName(rawModel) : localLabel
|
||||
}
|
||||
|
||||
function shortModelName(model: string, seen: Set<string>): string {
|
||||
if (autoModelNames[model]) return autoModelNames[model]
|
||||
if (seen.has(model)) {
|
||||
|
|
@ -1166,6 +1348,8 @@ export type PricingSnapshot = {
|
|||
aliases: Record<string, string>
|
||||
priceOverrides: Record<string, PriceOverrideRates>
|
||||
localModelSavings: Record<string, string>
|
||||
flatRateModels?: string[]
|
||||
flatRateModelsRemoved?: string[]
|
||||
}
|
||||
|
||||
export function snapshotPricingState(): PricingSnapshot {
|
||||
|
|
@ -1174,6 +1358,8 @@ export function snapshotPricingState(): PricingSnapshot {
|
|||
aliases: userAliases,
|
||||
priceOverrides: userPriceOverridesConfig,
|
||||
localModelSavings: userLocalModelSavings,
|
||||
flatRateModels: getFlatRateModels(),
|
||||
flatRateModelsRemoved: getFlatRateRemoved(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1185,4 +1371,6 @@ export function restorePricingState(snapshot: PricingSnapshot): void {
|
|||
setModelAliases(snapshot.aliases)
|
||||
setPriceOverrides(snapshot.priceOverrides)
|
||||
setLocalModelSavings(snapshot.localModelSavings)
|
||||
setFlatRateModels(snapshot.flatRateModels ?? [])
|
||||
setFlatRateRemoved(snapshot.flatRateModelsRemoved ?? [])
|
||||
}
|
||||
|
|
|
|||
140
src/optimize.ts
140
src/optimize.ts
|
|
@ -8,7 +8,7 @@ import { homedir } from 'os'
|
|||
|
||||
import { isReadShapedBashCommand } from './bash-utils.js'
|
||||
import { readSessionLines, readSessionFileSync } from './fs-utils.js'
|
||||
import { discoverAllSessions } from './providers/index.js'
|
||||
import { discoverAllSessions, providerDisplayName } from './providers/index.js'
|
||||
import { parseJsonlLine, shouldSkipLine } from './parser.js'
|
||||
import type { DateRange, ProjectSummary, SessionSummary } from './types.js'
|
||||
import { formatCost } from './currency.js'
|
||||
|
|
@ -249,6 +249,88 @@ export type PasteDestination =
|
|||
| 'shell-config' // append to ~/.zshrc / ~/.bashrc
|
||||
| 'manual' // instructions the user carries out directly
|
||||
|
||||
/// Provider-scoped remediation nouns. Cross-provider detectors and both
|
||||
/// render surfaces (CLI + TUI) share this so `--provider codex` cannot still
|
||||
/// tell the user to ask Claude or edit CLAUDE.md. Claude / unset / `all`
|
||||
/// keep the shipped CLAUDE.md copy. Only `codex` has a file CodeBurn already
|
||||
/// names (`AGENTS.md` in the Codex parser); every other provider stays on
|
||||
/// the generic "project instructions" rather than inventing a filename.
|
||||
export type OptimizeRemediationCopy = {
|
||||
agent: string
|
||||
instructionFile: string
|
||||
}
|
||||
|
||||
const REMEDIATION_INSTRUCTION_FILES: Record<string, string> = {
|
||||
claude: 'CLAUDE.md',
|
||||
codex: 'AGENTS.md',
|
||||
}
|
||||
|
||||
export function isDefaultClaudeProvider(provider?: string): boolean {
|
||||
return !provider || provider === 'all' || provider === 'claude'
|
||||
}
|
||||
|
||||
export function optimizeRemediationCopy(provider?: string): OptimizeRemediationCopy {
|
||||
const key = !provider || provider === 'all' ? 'claude' : provider.toLowerCase()
|
||||
return {
|
||||
agent: providerDisplayName(key),
|
||||
instructionFile: REMEDIATION_INSTRUCTION_FILES[key] ?? 'project instructions',
|
||||
}
|
||||
}
|
||||
|
||||
export function sessionOpenerLabel(copy: OptimizeRemediationCopy): string {
|
||||
return `Paste at the start of your NEXT expensive thread (one-time, do not add to ${copy.instructionFile}):`
|
||||
}
|
||||
|
||||
export function askAgentLabel(copy: OptimizeRemediationCopy, rest: string): string {
|
||||
return `Ask ${copy.agent} to ${rest}:`
|
||||
}
|
||||
|
||||
export function optimizePasteHeader(destination: PasteDestination | undefined, copy: OptimizeRemediationCopy): string {
|
||||
switch (destination) {
|
||||
case 'claude-md': return `Suggested ${copy.instructionFile} addition (permanent rule)`
|
||||
case 'session-opener': return `One-time session opener (do NOT add to ${copy.instructionFile})`
|
||||
case 'prompt': return `Ask ${copy.agent} in the current session`
|
||||
case 'shell-config': return 'Add to your shell config'
|
||||
case 'manual': return 'Manual action'
|
||||
default: return 'Suggested action'
|
||||
}
|
||||
}
|
||||
|
||||
/// Exact pre-#1049 TUI strings for unset/`all`/`claude`. Non-Claude providers
|
||||
/// reuse the CLI header table so the two surfaces cannot drift on new copy.
|
||||
export function optimizeTuiPasteHeader(destination: PasteDestination | undefined, provider?: string): string {
|
||||
if (isDefaultClaudeProvider(provider)) {
|
||||
switch (destination) {
|
||||
case 'claude-md': return '── Suggested CLAUDE.md addition (permanent rule) '.padEnd(64, '─')
|
||||
case 'session-opener': return '── One-time session opener (do not add to CLAUDE.md) '.padEnd(64, '─')
|
||||
case 'prompt': return '── Ask Claude in the current session '.padEnd(64, '─')
|
||||
case 'shell-config': return '── Add to your shell config '.padEnd(64, '─')
|
||||
case 'manual': return '── Manual action '.padEnd(64, '─')
|
||||
default: return '── Suggested action '.padEnd(64, '─')
|
||||
}
|
||||
}
|
||||
return `── ${optimizePasteHeader(destination, optimizeRemediationCopy(provider))} `.padEnd(64, '─')
|
||||
}
|
||||
|
||||
export function optimizeEmptyScanLines(provider?: string): [string, string, string] {
|
||||
if (isDefaultClaudeProvider(provider)) {
|
||||
return [
|
||||
'CodeBurn optimize scans your Claude Code sessions and config for',
|
||||
'token waste: junk directory reads, duplicate file reads, unused',
|
||||
'agents/skills/MCP servers, bloated CLAUDE.md, and more.',
|
||||
]
|
||||
}
|
||||
const copy = optimizeRemediationCopy(provider)
|
||||
// scanSessions is Claude-only. Naming the provider's instruction file here
|
||||
// would claim a scan that did not run (Health A / 100 under --provider
|
||||
// codex with every listed detector claudeOnly-disabled).
|
||||
return [
|
||||
`Session-scan detectors do not cover ${copy.agent} yet.`,
|
||||
'junk directory reads, duplicate file reads, unused agents/skills/MCP,',
|
||||
'and bloated instruction files currently scan Claude Code only.',
|
||||
]
|
||||
}
|
||||
|
||||
export type WasteAction =
|
||||
| { type: 'paste'; label: string; text: string; destination?: PasteDestination }
|
||||
| { type: 'command'; label: string; text: string }
|
||||
|
|
@ -1695,6 +1777,7 @@ function collectMcpProjectProfiles(
|
|||
export function detectMcpProfileAdvisor(
|
||||
projects: ProjectSummary[],
|
||||
coverage = aggregateMcpCoverage(projects),
|
||||
provider?: string,
|
||||
): WasteFinding | null {
|
||||
const candidates = collectMcpProjectProfiles(projects, coverage)
|
||||
if (candidates.length === 0) return null
|
||||
|
|
@ -1733,7 +1816,7 @@ export function detectMcpProfileAdvisor(
|
|||
fix: {
|
||||
type: 'paste',
|
||||
destination: 'prompt',
|
||||
label: 'Ask Claude to turn this into a project-scoped MCP profile:',
|
||||
label: askAgentLabel(optimizeRemediationCopy(provider), 'turn this into a project-scoped MCP profile'),
|
||||
text: [
|
||||
`Review these MCP profile recommendations before changing config (${preview.length} of ${candidates.length} shown):`,
|
||||
...preview.map(candidate => {
|
||||
|
|
@ -1926,7 +2009,7 @@ function findCapabilityReliabilityCandidates(projects: ProjectSummary[]): Capabi
|
|||
return candidates
|
||||
}
|
||||
|
||||
export function detectCapabilityReliability(projects: ProjectSummary[]): WasteFinding | null {
|
||||
export function detectCapabilityReliability(projects: ProjectSummary[], provider?: string): WasteFinding | null {
|
||||
projects = userStartedProjects(projects)
|
||||
const candidates = findCapabilityReliabilityCandidates(projects)
|
||||
if (candidates.length === 0) return null
|
||||
|
|
@ -1989,7 +2072,7 @@ export function detectCapabilityReliability(projects: ProjectSummary[]): WasteFi
|
|||
fix: {
|
||||
type: 'paste',
|
||||
destination: 'prompt',
|
||||
label: 'Ask Claude to audit the retry-heavy capability before changing config:',
|
||||
label: askAgentLabel(optimizeRemediationCopy(provider), 'audit the retry-heavy capability before changing config'),
|
||||
text: `Investigate these retry-correlated capabilities: ${names}. Compare edit turns with retries against one-shot edit turns, identify whether the MCP server or skill actually caused rework, then propose a scoped MCP config or skill-instruction change with session evidence. Do not remove a capability solely because it appears in this report.`,
|
||||
},
|
||||
}
|
||||
|
|
@ -3121,7 +3204,7 @@ export function findLowWorthCandidates(projects: ProjectSummary[]): LowWorthCand
|
|||
return candidates
|
||||
}
|
||||
|
||||
export function detectLowWorthSessions(projects: ProjectSummary[]): WasteFinding | null {
|
||||
export function detectLowWorthSessions(projects: ProjectSummary[], provider?: string): WasteFinding | null {
|
||||
const candidates = findLowWorthCandidates(projects)
|
||||
if (candidates.length === 0) return null
|
||||
|
||||
|
|
@ -3156,7 +3239,7 @@ export function detectLowWorthSessions(projects: ProjectSummary[]): WasteFinding
|
|||
fix: {
|
||||
type: 'paste',
|
||||
destination: 'session-opener',
|
||||
label: 'Paste at the start of your NEXT expensive thread (one-time, do not add to CLAUDE.md):',
|
||||
label: sessionOpenerLabel(optimizeRemediationCopy(provider)),
|
||||
text: LOW_WORTH_OPENER,
|
||||
},
|
||||
}
|
||||
|
|
@ -3231,7 +3314,7 @@ export function findContextBloatCandidates(projects: ProjectSummary[]): ContextB
|
|||
return candidates
|
||||
}
|
||||
|
||||
export function detectContextBloat(projects: ProjectSummary[], excludedSessionIds?: ReadonlySet<string>): WasteFinding | null {
|
||||
export function detectContextBloat(projects: ProjectSummary[], excludedSessionIds?: ReadonlySet<string>, provider?: string): WasteFinding | null {
|
||||
const candidates = findContextBloatCandidates(projects)
|
||||
.filter(c => !excludedSessionIds?.has(c.sessionId))
|
||||
if (candidates.length === 0) return null
|
||||
|
|
@ -3272,13 +3355,13 @@ export function detectContextBloat(projects: ProjectSummary[], excludedSessionId
|
|||
fix: {
|
||||
type: 'paste',
|
||||
destination: 'session-opener',
|
||||
label: 'Paste at the start of your NEXT expensive thread (one-time, do not add to CLAUDE.md):',
|
||||
label: sessionOpenerLabel(optimizeRemediationCopy(provider)),
|
||||
text: CONTEXT_HEAVY_OPENER,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
export function detectSessionOutliers(projects: ProjectSummary[], excludedSessionIds?: ReadonlySet<string>): WasteFinding | null {
|
||||
export function detectSessionOutliers(projects: ProjectSummary[], excludedSessionIds?: ReadonlySet<string>, provider?: string): WasteFinding | null {
|
||||
type Outlier = {
|
||||
project: string
|
||||
sessionId: string
|
||||
|
|
@ -3353,7 +3436,7 @@ export function detectSessionOutliers(projects: ProjectSummary[], excludedSessio
|
|||
fix: {
|
||||
type: 'paste',
|
||||
destination: 'session-opener',
|
||||
label: 'Paste at the start of your NEXT expensive thread (one-time, do not add to CLAUDE.md):',
|
||||
label: sessionOpenerLabel(optimizeRemediationCopy(provider)),
|
||||
text: 'Before making changes, summarize the smallest viable plan. Keep context narrow, avoid broad searches, and stop after the first working patch so I can review before continuing.',
|
||||
},
|
||||
}
|
||||
|
|
@ -3551,15 +3634,15 @@ export async function scanAndDetect(
|
|||
claudeOnly(() => detectDuplicateReads(toolCalls, dateRange)),
|
||||
claudeOnly(() => detectUnusedMcp(toolCalls, projects, projectCwds, mcpCoverage)),
|
||||
() => detectMcpToolCoverage(projects, mcpCoverage, localMcpServerNames(projectCwds)),
|
||||
() => detectMcpProfileAdvisor(projects, mcpCoverage),
|
||||
() => detectMcpProfileAdvisor(projects, mcpCoverage, provider),
|
||||
// mcp-deferral-gaps family (#614): detection only, no apply plans yet.
|
||||
claudeOnly(() => detectMcpDeferralOff(toolCalls, projects, projectCwds, apiCalls)),
|
||||
claudeOnly(() => detectMcpAlwaysLoadHygiene(projects, projectCwds, apiCalls, mcpCoverage)),
|
||||
claudeOnly(() => detectMcpDeferThreshold(projects, projectCwds)),
|
||||
() => detectCapabilityReliability(behavioralProjects),
|
||||
() => detectLowWorthSessions(behavioralProjects),
|
||||
() => detectContextBloat(behavioralProjects, lowWorthSessionIds),
|
||||
() => detectSessionOutliers(behavioralProjects, outlierExclusions),
|
||||
() => detectCapabilityReliability(behavioralProjects, provider),
|
||||
() => detectLowWorthSessions(behavioralProjects, provider),
|
||||
() => detectContextBloat(behavioralProjects, lowWorthSessionIds, provider),
|
||||
() => detectSessionOutliers(behavioralProjects, outlierExclusions, provider),
|
||||
claudeOnly(() => detectBloatedClaudeMd(projectCwds)),
|
||||
claudeOnly(() => detectBashBloat()),
|
||||
claudeOnly(() => detectRecurringContext(openers)),
|
||||
|
|
@ -3624,7 +3707,7 @@ function wrap(text: string, width: number, indent: string): string {
|
|||
/// destination. Issue #277: users were dropping one-time session openers
|
||||
/// into CLAUDE.md as permanent rules because the prompts had no labeled
|
||||
/// home in the output.
|
||||
function renderActionHeader(action: WasteAction): string {
|
||||
function renderActionHeader(action: WasteAction, copy: OptimizeRemediationCopy): string {
|
||||
const headerWidth = PANEL_WIDTH - 4
|
||||
const fillTo = (label: string): string => {
|
||||
const inner = ` ${label} `
|
||||
|
|
@ -3637,18 +3720,11 @@ function renderActionHeader(action: WasteAction): string {
|
|||
case 'command':
|
||||
return fillTo('Run this command')
|
||||
case 'paste':
|
||||
switch (action.destination) {
|
||||
case 'claude-md': return fillTo('Suggested CLAUDE.md addition (permanent rule)')
|
||||
case 'session-opener': return fillTo('One-time session opener (do NOT add to CLAUDE.md)')
|
||||
case 'prompt': return fillTo('Ask Claude in the current session')
|
||||
case 'shell-config': return fillTo('Add to your shell config')
|
||||
case 'manual': return fillTo('Manual action')
|
||||
default: return fillTo('Suggested action')
|
||||
}
|
||||
return fillTo(optimizePasteHeader(action.destination, copy))
|
||||
}
|
||||
}
|
||||
|
||||
function renderFinding(n: number, f: WasteFinding, costRate: number): string[] {
|
||||
function renderFinding(n: number, f: WasteFinding, costRate: number, copy: OptimizeRemediationCopy): string[] {
|
||||
const lines: string[] = []
|
||||
const costSaved = f.tokensSaved * costRate
|
||||
const impactLabel = f.impact.charAt(0).toUpperCase() + f.impact.slice(1)
|
||||
|
|
@ -3674,7 +3750,7 @@ function renderFinding(n: number, f: WasteFinding, costRate: number): string[] {
|
|||
// permanent rules and one-time prompts are no longer interchangeable in
|
||||
// the output.
|
||||
const a = f.fix
|
||||
lines.push(chalk.hex(ORANGE)(` ${renderActionHeader(a)}`))
|
||||
lines.push(chalk.hex(ORANGE)(` ${renderActionHeader(a, copy)}`))
|
||||
lines.push(chalk.hex(DIM)(` ${a.label}`))
|
||||
if (a.type === 'file-content') {
|
||||
for (const line of a.content.split('\n')) lines.push(chalk.hex(CYAN)(` ${line}`))
|
||||
|
|
@ -3743,7 +3819,9 @@ export function renderOptimize(
|
|||
previouslyApplied?: Record<string, string>,
|
||||
modelRecommendations?: ModelDefaultRecommendation[],
|
||||
appliedFixes: AppliedFix[] = [],
|
||||
provider?: string,
|
||||
): string {
|
||||
const copy = optimizeRemediationCopy(provider)
|
||||
const lines: string[] = []
|
||||
lines.push('')
|
||||
lines.push(` ${chalk.bold.hex(ORANGE)('CodeBurn config health')}${chalk.dim(' ' + periodLabel)}`)
|
||||
|
|
@ -3766,9 +3844,9 @@ export function renderOptimize(
|
|||
if (findings.length === 0) {
|
||||
lines.push(chalk.hex(GREEN)(' Nothing to fix. Your setup is lean.'))
|
||||
lines.push('')
|
||||
lines.push(chalk.dim(' CodeBurn optimize scans your Claude Code sessions and config for'))
|
||||
lines.push(chalk.dim(' token waste: junk directory reads, duplicate file reads, unused'))
|
||||
lines.push(chalk.dim(' agents/skills/MCP servers, bloated CLAUDE.md, and more.'))
|
||||
for (const line of optimizeEmptyScanLines(provider)) {
|
||||
lines.push(chalk.dim(` ${line}`))
|
||||
}
|
||||
lines.push('')
|
||||
lines.push(...renderAppliedFixes(appliedFixes))
|
||||
lines.push(...renderWorkflowSection(reworkedFiles, coachingNotes))
|
||||
|
|
@ -3799,7 +3877,7 @@ export function renderOptimize(
|
|||
for (const f of group) {
|
||||
const appliedOn = previouslyApplied?.[f.id]
|
||||
const shown = appliedOn ? { ...f, title: `${f.title} (previously applied ${appliedOn}, re-flagged)` } : f
|
||||
lines.push(...renderFinding(++n, shown, costRate))
|
||||
lines.push(...renderFinding(++n, shown, costRate, copy))
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -3868,7 +3946,7 @@ export async function runOptimize(
|
|||
}
|
||||
|
||||
const { topReworkedFiles, coachingNotes } = buildWorkflowReport(projects)
|
||||
const output = renderOptimize(findings, costRate, periodLabel, periodCost, sessionCount, callCount, healthScore, healthGrade, topReworkedFiles, coachingNotes, opts.appliedHeader, opts.previouslyApplied, result.modelRecommendations, opts.appliedFixes)
|
||||
const output = renderOptimize(findings, costRate, periodLabel, periodCost, sessionCount, callCount, healthScore, healthGrade, topReworkedFiles, coachingNotes, opts.appliedHeader, opts.previouslyApplied, result.modelRecommendations, opts.appliedFixes, opts.provider)
|
||||
console.log(output)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { homedir } from 'os'
|
|||
|
||||
import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
|
||||
import { formatCost as baseCost, getCurrency } from './currency.js'
|
||||
import { findUnpricedModels, getShortModelName } from './models.js'
|
||||
import { findUnpricedModels, getShortModelName, unpricedModelHint } from './models.js'
|
||||
import { markEstimated } from './format.js'
|
||||
import { dateKey } from './day-aggregator.js'
|
||||
import type { DailyEntry } from './daily-cache.js'
|
||||
|
|
@ -224,7 +224,7 @@ export function renderOverview(
|
|||
.join(', ')
|
||||
const more = unpriced.length > 3 ? ` +${unpriced.length - 3} more` : ''
|
||||
out.push(kv('Unpriced', c.yellow(`${unpriced.length} model${unpriced.length === 1 ? '' : 's'} at $0: `) + shown + more))
|
||||
out.push(kv('', c.dim('Fix: codeburn model-alias "<model>" <known-model>')))
|
||||
out.push(kv('', c.dim(unpricedModelHint())))
|
||||
}
|
||||
if (opts.budget) {
|
||||
const label = opts.budget.tier === 'daily'
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { existsSync } from 'fs'
|
|||
import { lstat, readFile, readdir, stat } from 'fs/promises'
|
||||
import { basename, dirname, join, resolve, sep } from 'path'
|
||||
import { readSessionLines } from './fs-utils.js'
|
||||
import { calculateCost, calculateLocalModelSavings, getShortModelName, isProxiedPath, getProxyPathsConfigHash, getModelAliasesConfigHash, getPriceOverridesConfigHash, getLocalModelSavingsConfigHash } from './models.js'
|
||||
import { billableOutputTokens, calculateCost, calculateLocalModelSavings, getShortModelName, isProxiedPath, getProxyPathsConfigHash, getModelAliasesConfigHash, getPriceOverridesConfigHash, getLocalModelSavingsConfigHash } from './models.js'
|
||||
import { resolveSubagentAttribution, sessionIdentity } from './sessions-report.js'
|
||||
import { normalizeContentBlocks, flatSlice, flatString } from './content-utils.js'
|
||||
import { discoverAllSessions, getProvider } from './providers/index.js'
|
||||
|
|
@ -1779,7 +1779,7 @@ function buildSessionSummary(
|
|||
modelBreakdown[modelKey].tokens.reasoningTokens += call.usage.reasoningTokens
|
||||
if (call.activeDurationMs !== undefined) {
|
||||
modelBreakdown[modelKey].activeDurationMs = (modelBreakdown[modelKey].activeDurationMs ?? 0) + call.activeDurationMs
|
||||
modelBreakdown[modelKey].activeGeneratedTokens = (modelBreakdown[modelKey].activeGeneratedTokens ?? 0) + (call.activeGeneratedTokens ?? call.usage.outputTokens + call.usage.reasoningTokens)
|
||||
modelBreakdown[modelKey].activeGeneratedTokens = (modelBreakdown[modelKey].activeGeneratedTokens ?? 0) + (call.activeGeneratedTokens ?? billableOutputTokens(call.provider, call.usage.outputTokens, call.usage.reasoningTokens))
|
||||
modelBreakdown[modelKey].toolWaitMs = (modelBreakdown[modelKey].toolWaitMs ?? 0) + (call.toolWaitMs ?? 0)
|
||||
}
|
||||
|
||||
|
|
@ -2644,16 +2644,11 @@ function providerCallsToCachedTurns(calls: ParsedProviderCall[]): CachedTurn[] {
|
|||
|
||||
function cachedCallToApiCall(call: CachedCall): ParsedApiCall {
|
||||
const u = call.usage
|
||||
// Claude thinking and Copilot reasoning tokens are already INSIDE
|
||||
// outputTokens (Copilot's own per-request token_details_json prices
|
||||
// input/cache/output and nothing else, and its reasoning counts are a
|
||||
// subset of the output count), so adding them here would bill them twice —
|
||||
// for copilot literally so: its session-store/shutdown supplementary calls
|
||||
// carry reasoningTokens with outputTokens 0 while the per-turn call bills
|
||||
// the full output. Other providers report reasoning separately from output.
|
||||
const outputForCost = call.provider === 'claude' || call.provider === 'copilot'
|
||||
? u.outputTokens
|
||||
: u.outputTokens + u.reasoningTokens
|
||||
// Cache-rehydration twin of the fresh-parse pricing in
|
||||
// src/providers/codex.ts (and every other provider's parser): both go
|
||||
// through billableOutputTokens so a cached read and a cold parse can never
|
||||
// disagree about whether reasoning is already inside output (#1075).
|
||||
const outputForCost = billableOutputTokens(call.provider, u.outputTokens, u.reasoningTokens)
|
||||
const costUSD = calculateCost(
|
||||
call.model, u.inputTokens, outputForCost,
|
||||
u.cacheCreationInputTokens, u.cacheReadInputTokens,
|
||||
|
|
@ -4152,6 +4147,9 @@ function cacheKey(dateRange: DateRange | undefined, providerFilter: string | und
|
|||
// Pricing-affecting config participates so a memoized parse (exact-key or
|
||||
// burst-reused in a resident serve process) can never present costs priced
|
||||
// under aliases/overrides/savings the user has since changed.
|
||||
// Flat-rate marks do not change parse-time cost (still $0 without a LiteLLM
|
||||
// row); findUnpricedModels / coverage apply them at render time, so they
|
||||
// stay out of this serve-memo key on purpose.
|
||||
return `${s}:${providerFilter ?? 'all'}:${claudeRoots}:${getProxyPathsConfigHash()}:${getModelAliasesConfigHash()}:${getPriceOverridesConfigHash()}:${getLocalModelSavingsConfigHash()}`
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ import { homedir } from 'os'
|
|||
import { fileURLToPath } from 'url'
|
||||
import https from 'https'
|
||||
|
||||
import { getCodeburnCacheDir } from '../cache-dir.js'
|
||||
import { getCodeburnCacheDir, readExistingTextFile } from '../cache-dir.js'
|
||||
import { calculateCost } from '../models.js'
|
||||
import { isSqliteAvailable, isSqliteBusyError, openDatabase } from '../sqlite.js'
|
||||
import type { ProbeRoot, Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
|
||||
|
|
@ -50,6 +50,11 @@ function conversationRoots(): readonly AntigravityConversationRoot[] {
|
|||
]
|
||||
}
|
||||
const CACHE_VERSION = 5
|
||||
export const ANTIGRAVITY_CACHE_VERSION = CACHE_VERSION
|
||||
export const ANTIGRAVITY_LEGACY_CACHE_FILE = 'antigravity-results.json'
|
||||
export function antigravityCacheFileName(version = CACHE_VERSION): string {
|
||||
return `antigravity-results.v${version}.json`
|
||||
}
|
||||
|
||||
const RPC_TIMEOUT_MS = 5000
|
||||
const MAX_RESPONSE_BYTES = 16 * 1024 * 1024
|
||||
|
|
@ -190,7 +195,15 @@ function currentCacheDir(): string {
|
|||
}
|
||||
|
||||
function getCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, 'antigravity-results.json')
|
||||
return join(cacheDir, antigravityCacheFileName())
|
||||
}
|
||||
|
||||
function getLegacyCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, ANTIGRAVITY_LEGACY_CACHE_FILE)
|
||||
}
|
||||
|
||||
function isCurrentCache(cache: AntigravityCache): boolean {
|
||||
return cache.version === CACHE_VERSION && !!cache.cascades && typeof cache.cascades === 'object'
|
||||
}
|
||||
|
||||
export function getAntigravityStatusLineEventsPath(): string {
|
||||
|
|
@ -336,15 +349,42 @@ export function extractAntigravityGeneratorMetadata(resp: unknown): GeneratorMet
|
|||
async function loadCache(cacheDir: string): Promise<AntigravityCacheState> {
|
||||
const inMemory = cacheStates.get(cacheDir)
|
||||
if (inMemory) return inMemory
|
||||
const versioned = await readExistingTextFile(getCachePath(cacheDir))
|
||||
if (versioned.status === 'ok') {
|
||||
try {
|
||||
const cache = JSON.parse(versioned.text) as AntigravityCache
|
||||
if (isCurrentCache(cache)) {
|
||||
const state = { cache, dirty: false }
|
||||
cacheStates.set(cacheDir, state)
|
||||
return state
|
||||
}
|
||||
} catch { /* present but invalid */ }
|
||||
const invalid: AntigravityCacheState = {
|
||||
cache: { version: CACHE_VERSION, cascades: {} },
|
||||
dirty: false,
|
||||
}
|
||||
cacheStates.set(cacheDir, invalid)
|
||||
return invalid
|
||||
}
|
||||
if (versioned.status === 'unreadable') {
|
||||
const invalid: AntigravityCacheState = {
|
||||
cache: { version: CACHE_VERSION, cascades: {} },
|
||||
dirty: false,
|
||||
}
|
||||
cacheStates.set(cacheDir, invalid)
|
||||
return invalid
|
||||
}
|
||||
// Versioned file is absent (ENOENT). Adopt the unsuffixed file only when its
|
||||
// version matches — old binaries still own that path; we never write or delete it.
|
||||
try {
|
||||
const raw = await readFile(getCachePath(cacheDir), 'utf-8')
|
||||
const raw = await readFile(getLegacyCachePath(cacheDir), 'utf-8')
|
||||
const cache = JSON.parse(raw) as AntigravityCache
|
||||
if (cache.version === CACHE_VERSION && cache.cascades && typeof cache.cascades === 'object') {
|
||||
if (isCurrentCache(cache)) {
|
||||
const state = { cache, dirty: false }
|
||||
cacheStates.set(cacheDir, state)
|
||||
return state
|
||||
}
|
||||
} catch { /* no cache or invalid */ }
|
||||
} catch { /* no legacy cache or invalid */ }
|
||||
const state: AntigravityCacheState = {
|
||||
cache: { version: CACHE_VERSION, cascades: {} },
|
||||
dirty: false,
|
||||
|
|
|
|||
|
|
@ -5,8 +5,9 @@ import { basename, join } from 'path'
|
|||
import { homedir } from 'os'
|
||||
|
||||
import { readSessionLines } from '../fs-utils.js'
|
||||
import { calculateCost } from '../models.js'
|
||||
import { billableOutputTokens, calculateCost, getModelCosts } from '../models.js'
|
||||
import { readCachedCodexResults, writeCachedCodexResults, getCachedCodexProject, fingerprintFile, type CodexFileFingerprint } from '../codex-cache.js'
|
||||
import { mergeToolIntervals } from '../codex-throughput.js'
|
||||
import { normalizeContentBlocks } from '../content-utils.js'
|
||||
import { estimateTokensFromChars } from '../token-estimate.js'
|
||||
import type { ToolCall } from '../types.js'
|
||||
|
|
@ -66,11 +67,7 @@ const toolNameMap: Record<string, string> = {
|
|||
// false-positive, an accepted tradeoff for the common case. \s+ and the token
|
||||
// class don't overlap, so there is no catastrophic backtracking.
|
||||
const MCP_CLI_CALL = /(?<![\w.-])mcp-cli(?:\s+(?!call\b)[^\s;|&]+)*\s+call\s+(\S+)\s+(\S+)/
|
||||
function mcpToolFromShellCommand(command: unknown): string | null {
|
||||
const text = typeof command === 'string'
|
||||
? command
|
||||
: Array.isArray(command) ? command.filter(x => typeof x === 'string').join(' ') : ''
|
||||
if (!text) return null
|
||||
function mcpToolFromShellCommand(text: string): string | null {
|
||||
const m = MCP_CLI_CALL.exec(text)
|
||||
if (!m) return null
|
||||
const server = m[1]!.replace(/['"]/g, '')
|
||||
|
|
@ -79,6 +76,37 @@ function mcpToolFromShellCommand(command: unknown): string | null {
|
|||
return `mcp__${server}__${tool}`
|
||||
}
|
||||
|
||||
// Codex has no dedicated skill tool: loading a skill is an ordinary shell exec
|
||||
// that reads the skill's `SKILL.md`, so the usage landed entirely under Bash and
|
||||
// the Skills dimension stayed empty (#478). Recognize only a *read* of a
|
||||
// SKILL.md, deliberately narrowly:
|
||||
// - the command segment must start with a file-reading binary (cat/sed/head/
|
||||
// tail/less/more/bat) — a `grep`/`rg`/`ls` that merely mentions a SKILL.md
|
||||
// is a search near the file, not a skill load, and stays plain Bash;
|
||||
// - the path it reads must END in `<name>/SKILL.md`, so `<name>` is the skill.
|
||||
// The name is the parent directory, the same key `src/providers/pi.ts` derives
|
||||
// for a native skill read (#588) and the same vocabulary the Claude parser
|
||||
// records from the `Skill` tool, so the Skills breakdown stays cross-provider
|
||||
// coherent. Substring matching, so a command that merely quotes the phrase can
|
||||
// false-positive — the same accepted tradeoff as MCP_CLI_CALL above.
|
||||
const SKILL_MD_READ = /(?<![\w.-])(?:cat|bat|sed|head|tail|less|more)\b[^;|&]*?[\s'"]([^\s;|&'"]*[\\/]([^\\/;|&'"]+)[\\/]SKILL\.md)\b/
|
||||
function skillFromShellCommand(text: string): string | null {
|
||||
const m = SKILL_MD_READ.exec(text)
|
||||
const name = m?.[2]?.trim()
|
||||
return name ? name : null
|
||||
}
|
||||
|
||||
/// Flatten every shape Codex records a shell command in to one string:
|
||||
/// `function_call` arguments carry it as a plain string, the item model's
|
||||
/// `CommandExecution` item as an argv array (`["/bin/zsh","-lc","..."]`), and
|
||||
/// the `exec` custom tool as the JS program that calls `tools.exec_command`.
|
||||
/// One text, one classification pipeline (#478).
|
||||
function shellCommandText(command: unknown): string {
|
||||
if (typeof command === 'string') return command
|
||||
if (Array.isArray(command)) return command.filter(x => typeof x === 'string').join(' ')
|
||||
return ''
|
||||
}
|
||||
|
||||
// Count added/removed lines from a Codex `patch_apply_end` change's
|
||||
// `unified_diff`. A leading '+' is an added line and '-' a removed line; the
|
||||
// '+++'/'---' file headers and '@@' hunk headers are excluded. Numbers only —
|
||||
|
|
@ -126,6 +154,10 @@ type CodexEntry = {
|
|||
type CodexTokenUsage = {
|
||||
input_tokens?: number
|
||||
cached_input_tokens?: number
|
||||
/// Portion of `input_tokens` that was WRITTEN to the prompt cache this call
|
||||
/// (codex PR #33454). Like `cached_input_tokens`, it is carved out of
|
||||
/// `input_tokens`, not added on top.
|
||||
cache_write_input_tokens?: number
|
||||
output_tokens?: number
|
||||
reasoning_output_tokens?: number
|
||||
total_tokens?: number
|
||||
|
|
@ -323,6 +355,7 @@ function getRawTokenUsage(head: string, field: 'last_token_usage' | 'total_token
|
|||
return {
|
||||
input_tokens: getRawJsonNumberField(body, 'input_tokens'),
|
||||
cached_input_tokens: getRawJsonNumberField(body, 'cached_input_tokens'),
|
||||
cache_write_input_tokens: getRawJsonNumberField(body, 'cache_write_input_tokens'),
|
||||
output_tokens: getRawJsonNumberField(body, 'output_tokens'),
|
||||
reasoning_output_tokens: getRawJsonNumberField(body, 'reasoning_output_tokens'),
|
||||
total_tokens: getRawJsonNumberField(body, 'total_tokens'),
|
||||
|
|
@ -431,12 +464,16 @@ function parseCodexLine(line: string | Buffer): CodexEntry | null {
|
|||
? getRawDurationMs(getRawPayloadFieldWindow(line, 'duration') ?? '')
|
||||
: undefined
|
||||
const timingDuration = payloadDuration ?? getRawDurationMs(pHead) ?? getRawDurationMs(timingTail)
|
||||
// session_meta can contain base_instructions.provenance.model. Only inspect
|
||||
// direct payload fields there, or a nested provenance model would overwrite
|
||||
// the model selected by the latest turn_context.
|
||||
const compactModel = type === 'session_meta'
|
||||
? getRawJsonStringField(getRawPayloadFieldWindow(line, 'model') ?? '', 'model')
|
||||
: getRawJsonStringField(pHead, 'model')
|
||||
// session_meta can embed same-name keys under base_instructions /
|
||||
// dynamic_tools (including provenance.model). A depth-agnostic scan of the
|
||||
// compact head steals the first nested hit and can overwrite turn_context.
|
||||
// Restrict every session_meta string field to payload depth 1. Other event
|
||||
// types keep the cheap first-match scan.
|
||||
const payloadString = (field: string): string | undefined =>
|
||||
type === 'session_meta'
|
||||
? getRawJsonStringField(getRawPayloadFieldWindow(line, field) ?? '', field)
|
||||
: getRawJsonStringField(pHead, field)
|
||||
const compactModel = payloadString('model')
|
||||
const compactModelName = getRawJsonStringField(pHead, 'model_name')
|
||||
const compactLastUsage = getRawTokenUsage(pHead, 'last_token_usage')
|
||||
const compactTotalUsage = getRawTokenUsage(pHead, 'total_token_usage')
|
||||
|
|
@ -451,13 +488,13 @@ function parseCodexLine(line: string | Buffer): CodexEntry | null {
|
|||
payload: {
|
||||
type: payloadType,
|
||||
role,
|
||||
cwd: getRawJsonStringField(pHead, 'cwd'),
|
||||
model_provider: getRawJsonStringField(pHead, 'model_provider'),
|
||||
originator: getRawJsonStringField(pHead, 'originator'),
|
||||
session_id: getRawJsonStringField(pHead, 'session_id'),
|
||||
forked_from_id: getRawJsonStringField(pHead, 'forked_from_id'),
|
||||
cwd: payloadString('cwd'),
|
||||
model_provider: payloadString('model_provider'),
|
||||
originator: payloadString('originator'),
|
||||
session_id: payloadString('session_id'),
|
||||
forked_from_id: payloadString('forked_from_id'),
|
||||
model: compactModel,
|
||||
name: getRawJsonStringField(pHead, 'name'),
|
||||
name: payloadString('name'),
|
||||
invocation,
|
||||
call_id: getRawJsonStringField(pHead, 'call_id'),
|
||||
turn_id: getRawJsonStringField(pHead, 'turn_id'),
|
||||
|
|
@ -589,10 +626,14 @@ type CodexResumeState = {
|
|||
prevCumulativeTotal: number | null
|
||||
prevInput: number
|
||||
prevCached: number
|
||||
prevCacheWrite: number
|
||||
prevOutput: number
|
||||
prevReasoning: number
|
||||
pendingTools: string[]
|
||||
pendingToolSequence: ToolCall[][]
|
||||
/// Optional so a resume state written before skills attribution existed still
|
||||
/// decodes (isResumeState does not require it); absent reads as "no skills".
|
||||
pendingSkills?: string[]
|
||||
pendingUserMessage: string
|
||||
pendingOutputChars: number
|
||||
pendingLocAdded: number
|
||||
|
|
@ -602,6 +643,12 @@ type CodexResumeState = {
|
|||
turnCounter: number
|
||||
currentTurnId: string
|
||||
taskStartedAt?: number
|
||||
// #1088 BUG-1: timestamp of the first request-context event (turn_context,
|
||||
// world_state, event_msg/user_message, or response_item/message) seen since
|
||||
// the last task_started. Codex fires task_started before it assembles the
|
||||
// request, so the gap up to this event is CLI/harness startup, not model
|
||||
// wait -- the active window for Tok/s starts here, not at task_started.
|
||||
taskActiveStartedAt?: number
|
||||
}
|
||||
|
||||
// The state comes back off our own JSON cache; a truncated or hand-edited file
|
||||
|
|
@ -615,6 +662,7 @@ function isResumeState(value: unknown): value is CodexResumeState {
|
|||
&& (v['prevCumulativeTotal'] === null || typeof v['prevCumulativeTotal'] === 'number')
|
||||
&& typeof v['prevInput'] === 'number'
|
||||
&& typeof v['prevCached'] === 'number'
|
||||
&& typeof v['prevCacheWrite'] === 'number'
|
||||
&& typeof v['prevOutput'] === 'number'
|
||||
&& typeof v['prevReasoning'] === 'number'
|
||||
&& Array.isArray(v['pendingTools'])
|
||||
|
|
@ -674,10 +722,51 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
let prevCumulativeTotal: number | null = resume?.state.prevCumulativeTotal ?? null
|
||||
let prevInput = resume?.state.prevInput ?? 0
|
||||
let prevCached = resume?.state.prevCached ?? 0
|
||||
let prevCacheWrite = resume?.state.prevCacheWrite ?? 0
|
||||
let prevOutput = resume?.state.prevOutput ?? 0
|
||||
let prevReasoning = resume?.state.prevReasoning ?? 0
|
||||
let pendingTools: string[] = resume ? [...resume.state.pendingTools] : []
|
||||
let pendingToolSequence: ToolCall[][] = resume ? [...resume.state.pendingToolSequence] : []
|
||||
let pendingSkills: string[] = resume?.state.pendingSkills ? [...resume.state.pendingSkills] : []
|
||||
// Attribution names already emitted from a `response_item` tool call.
|
||||
// Codex's item model repeats a finished shell command as
|
||||
// `item_completed`/`CommandExecution`; a rollout that carries BOTH shapes
|
||||
// must attribute the call once, and one that carries ONLY the item must
|
||||
// still attribute it. Counting per derived name (not per command text,
|
||||
// which the two shapes spell differently - argv array vs plain string)
|
||||
// makes repeats of the same call line up one-for-one. The response item is
|
||||
// always written first (the model requests the call, the item completes
|
||||
// it), so the item side only ever cancels an attribution, never adds a
|
||||
// duplicate.
|
||||
const itemDedup = new Map<string, number>()
|
||||
|
||||
/// The single classification pipeline every Codex shell-command shape
|
||||
/// feeds (#478): `function_call` arguments, the `exec` custom tool's JS
|
||||
/// `input`, and the item model's `CommandExecution` item. It adds MCP and
|
||||
/// Skill attribution only - the exec itself is counted as Bash by its
|
||||
/// response item, and `viaItem` calls never add a tool of their own, so
|
||||
/// call/token/cost totals cannot move.
|
||||
const attributeShellCommand = (text: string, viaItem: boolean): void => {
|
||||
if (!text) return
|
||||
const claim = (name: string): boolean => {
|
||||
const pending = itemDedup.get(name) ?? 0
|
||||
if (!viaItem) { itemDedup.set(name, pending + 1); return true }
|
||||
if (pending === 0) return true
|
||||
itemDedup.set(name, pending - 1)
|
||||
return false
|
||||
}
|
||||
const mcpTool = mcpToolFromShellCommand(text)
|
||||
if (mcpTool && claim(mcpTool)) {
|
||||
pendingTools.push(mcpTool)
|
||||
pendingToolSequence.push([{ tool: mcpTool }])
|
||||
}
|
||||
const skill = skillFromShellCommand(text)
|
||||
if (skill && claim(`skill:${skill}`)) {
|
||||
pendingSkills.push(skill)
|
||||
pendingTools.push('Skill')
|
||||
pendingToolSequence.push([{ tool: 'Skill', file: skill }])
|
||||
}
|
||||
}
|
||||
let pendingUserMessage = resume?.state.pendingUserMessage ?? ''
|
||||
let pendingOutputChars = resume?.state.pendingOutputChars ?? 0
|
||||
// Rich-session-capture: edit LOC deltas and failed-patch count accumulated
|
||||
|
|
@ -708,6 +797,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
let taskGeneratedTokens = 0
|
||||
let taskToolIntervals: Array<[number, number]> = []
|
||||
let taskStartedAt: number | undefined = resume?.state.taskStartedAt
|
||||
let taskActiveStartedAt: number | undefined = resume?.state.taskActiveStartedAt
|
||||
const openToolStarts = new Map<string, number>()
|
||||
|
||||
// Resume point for the NEXT run, refreshed at every task boundary.
|
||||
|
|
@ -751,6 +841,22 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
continue
|
||||
}
|
||||
|
||||
// #1088 BUG-1: the first request-context event since task_started marks
|
||||
// where model-request assembly actually began. Checked before any of
|
||||
// these types `continue` below, and unconditionally (like turn_context's
|
||||
// model capture above) so a forked replay's own request-context events
|
||||
// still mark it -- matching how those events are already read regardless
|
||||
// of isForkReplay, which only filters task boundaries and tool events.
|
||||
if (taskActiveStartedAt === undefined && (
|
||||
entry.type === 'turn_context'
|
||||
|| entry.type === 'world_state'
|
||||
|| (entry.type === 'event_msg' && entry.payload?.type === 'user_message')
|
||||
|| (entry.type === 'response_item' && entry.payload?.type === 'message')
|
||||
)) {
|
||||
const ctxAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
if (Number.isFinite(ctxAt)) taskActiveStartedAt = ctxAt
|
||||
}
|
||||
|
||||
if (entry.type === 'turn_context' && typeof entry.payload?.model === 'string') {
|
||||
sessionModel = entry.payload.model
|
||||
continue
|
||||
|
|
@ -765,6 +871,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
entry.payload?.type === 'custom_tool_call' ||
|
||||
entry.payload?.type === 'custom_tool_call_output' ||
|
||||
entry.payload?.type === 'mcp_tool_call_end' ||
|
||||
entry.payload?.type === 'item_completed' ||
|
||||
entry.payload?.type === 'patch_apply_end'
|
||||
)) continue
|
||||
|
||||
|
|
@ -777,6 +884,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
taskToolIntervals = []
|
||||
const startedAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
taskStartedAt = Number.isFinite(startedAt) ? startedAt : undefined
|
||||
taskActiveStartedAt = undefined
|
||||
openToolStarts.clear()
|
||||
// Everything decoded so far is now in `results` and the per-task
|
||||
// accumulators are empty: a clean restart point for an appended tail.
|
||||
|
|
@ -791,10 +899,12 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
prevCumulativeTotal,
|
||||
prevInput,
|
||||
prevCached,
|
||||
prevCacheWrite,
|
||||
prevOutput,
|
||||
prevReasoning,
|
||||
pendingTools: [...pendingTools],
|
||||
pendingToolSequence: [...pendingToolSequence],
|
||||
pendingSkills: [...pendingSkills],
|
||||
pendingUserMessage,
|
||||
pendingOutputChars,
|
||||
pendingLocAdded,
|
||||
|
|
@ -804,6 +914,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
turnCounter,
|
||||
currentTurnId,
|
||||
...(taskStartedAt !== undefined ? { taskStartedAt } : {}),
|
||||
...(taskActiveStartedAt !== undefined ? { taskActiveStartedAt } : {}),
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
|
@ -822,13 +933,21 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
if (typeof fp === 'string') call.file = fp
|
||||
const cmd = args['command'] ?? args['cmd']
|
||||
if (typeof cmd === 'string') call.command = cmd
|
||||
// Attribute a CLI-wrapped MCP call (e.g. `mcp-cli call server tool`)
|
||||
// to the MCP breakdown too; the exec still counts as Bash above.
|
||||
const mcpTool = mcpToolFromShellCommand(cmd)
|
||||
if (mcpTool) {
|
||||
pendingTools.push(mcpTool)
|
||||
pendingToolSequence.push([{ tool: mcpTool }])
|
||||
}
|
||||
attributeShellCommand(shellCommandText(cmd), false)
|
||||
}
|
||||
// A `custom_tool_call` has no `arguments`: the shell tool's payload is
|
||||
// the `input` program that calls `tools.exec_command({cmd: ...})`. The
|
||||
// MCP/skill matchers never saw it, so a CLI-wrapped MCP call or a
|
||||
// SKILL.md read made through Codex's custom-tool transport counted only
|
||||
// as Bash (#478). Restricted to the shell tool: `apply_patch` is a
|
||||
// custom tool too and its `input` is file content, where a documented
|
||||
// command would read as a real invocation. `call.command` is
|
||||
// deliberately NOT set from it either - that field feeds the retry
|
||||
// heuristic's read-shaped-command test, and a JS program is not a
|
||||
// shell command.
|
||||
if (mapped === 'Bash') {
|
||||
const input = (entry.payload as Record<string, unknown>)['input']
|
||||
if (typeof input === 'string') attributeShellCommand(input, false)
|
||||
}
|
||||
const callId = entry.payload.call_id
|
||||
const started = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
|
|
@ -847,26 +966,33 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
}
|
||||
|
||||
if (entry.type === 'event_msg' && entry.payload?.type === 'task_complete') {
|
||||
const durationMs = entry.payload.duration_ms
|
||||
// #1088 BUG-8: task_complete's duration can arrive as {secs,nanos} or
|
||||
// a string too (mcp_tool_call_end already tolerates both, below, via
|
||||
// this same durationValueMs helper); read it the same permissive way
|
||||
// instead of only the plain-number `duration_ms` field.
|
||||
const durationMs = durationValueMs(entry.payload.duration_ms) ?? durationValueMs(entry.payload.duration)
|
||||
if (typeof durationMs === 'number' && durationMs > 0 && taskGeneratedTokens > 0 && pendingTaskCalls.length > 0) {
|
||||
const completedAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
const windowStart = taskStartedAt ?? (Number.isFinite(completedAt) ? completedAt - durationMs : undefined)
|
||||
const windowEnd = windowStart !== undefined ? windowStart + durationMs : undefined
|
||||
const clipped = taskToolIntervals.map(([start, end]) => [
|
||||
windowStart !== undefined ? Math.max(start, windowStart) : start,
|
||||
windowEnd !== undefined ? Math.min(end, windowEnd) : end,
|
||||
] as [number, number]).filter(([start, end]) => end > start)
|
||||
const merged = clipped.sort((a, b) => a[0] - b[0]).reduce<Array<[number, number]>>((acc, interval) => {
|
||||
const previous = acc.at(-1)
|
||||
if (previous && interval[0] <= previous[1]) previous[1] = Math.max(previous[1], interval[1])
|
||||
else acc.push([...interval])
|
||||
return acc
|
||||
}, [])
|
||||
const toolWaitMs = Math.min(durationMs, merged.reduce((sum, interval) => sum + interval[1] - interval[0], 0))
|
||||
const activeMs = durationMs - toolWaitMs
|
||||
// #1088 BUG-1: Codex fires task_started before it assembles the
|
||||
// request, so the gap up to the first request-context event is
|
||||
// CLI/harness startup, not model wait. The active window starts
|
||||
// there instead of at task_started; task completion (windowEnd,
|
||||
// inside mergeToolIntervals) is unchanged.
|
||||
const activeWindowStart = taskActiveStartedAt ?? taskStartedAt
|
||||
const startupGapMs = activeWindowStart !== undefined && taskStartedAt !== undefined
|
||||
? activeWindowStart - taskStartedAt
|
||||
: 0
|
||||
const effectiveDurationMs = Math.max(0, durationMs - startupGapMs)
|
||||
// #1088 BUG-8: shared with codex-throughput.ts's live estimate
|
||||
// instead of a second inline copy of the same clip/merge/cap.
|
||||
const toolWaitMs = mergeToolIntervals(taskToolIntervals, effectiveDurationMs, activeWindowStart, Number.isFinite(completedAt) ? completedAt : undefined)
|
||||
const activeMs = effectiveDurationMs - toolWaitMs
|
||||
if (activeMs <= 0) continue
|
||||
for (const call of pendingTaskCalls) {
|
||||
const generated = call.outputTokens + call.reasoningTokens
|
||||
// Reasoning is already inside output_tokens (#1075/#1078); the
|
||||
// throughput numerator must agree with the cost numerator or
|
||||
// Tok/s reads high for reasoning-heavy calls (#1079).
|
||||
const generated = billableOutputTokens('codex', call.outputTokens, call.reasoningTokens)
|
||||
if (generated <= 0) continue
|
||||
call.activeGeneratedTokens = generated
|
||||
call.activeDurationMs = activeMs * (generated / taskGeneratedTokens)
|
||||
|
|
@ -919,6 +1045,21 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
continue
|
||||
}
|
||||
|
||||
// Codex's item model records a finished shell command as
|
||||
// `event_msg`/`item_completed` carrying a `CommandExecution` item whose
|
||||
// `command` is the argv array (`["/bin/zsh","-lc","..."]`) - a shape the
|
||||
// `function_call`-only tool path never reached, so a CLI-wrapped MCP call
|
||||
// or a SKILL.md read made under it stayed invisible (#478). Attribution
|
||||
// only: the exec's own Bash count comes from its response item, exactly
|
||||
// as `exec_command_end` is left alone for the classic transport.
|
||||
if (entry.type === 'event_msg' && entry.payload?.type === 'item_completed') {
|
||||
const item = (entry.payload as Record<string, unknown>)['item'] as Record<string, unknown> | undefined
|
||||
if (item && item['type'] === 'CommandExecution') {
|
||||
attributeShellCommand(shellCommandText(item['command']), true)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (entry.type === 'response_item' && entry.payload?.type === 'message' && entry.payload?.role === 'user') {
|
||||
const texts = normalizeContentBlocks(entry.payload.content)
|
||||
.filter(c => c.type === 'input_text')
|
||||
|
|
@ -955,7 +1096,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
const timestamp = entry.timestamp ?? ''
|
||||
const dedupKey = `codex:${sessionId}:${timestamp}:est${estCounter++}`
|
||||
|
||||
if (seenKeys.has(dedupKey)) { pendingTools = []; pendingToolSequence = []; pendingUserMessage = ''; pendingOutputChars = 0; pendingLocAdded = 0; pendingLocRemoved = 0; pendingEditFailed = 0; continue }
|
||||
if (seenKeys.has(dedupKey)) { pendingTools = []; pendingToolSequence = []; pendingSkills = []; pendingUserMessage = ''; pendingOutputChars = 0; pendingLocAdded = 0; pendingLocRemoved = 0; pendingEditFailed = 0; continue }
|
||||
seenKeys.add(dedupKey)
|
||||
|
||||
const costUSD = calculateCost(model, estInput, estOutput, 0, 0, 0)
|
||||
|
|
@ -979,6 +1120,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
deduplicationKey: dedupKey,
|
||||
turnId: currentTurnId,
|
||||
toolSequence: pendingToolSequence.length > 0 ? pendingToolSequence : undefined,
|
||||
...(pendingSkills.length > 0 ? { skills: pendingSkills } : {}),
|
||||
userMessage: pendingUserMessage,
|
||||
sessionId,
|
||||
...(sessionCwd ? { projectPath: sessionCwd, workingDirectory: sessionCwd } : {}),
|
||||
|
|
@ -990,6 +1132,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
|
||||
pendingTools = []
|
||||
pendingToolSequence = []
|
||||
pendingSkills = []
|
||||
pendingUserMessage = ''
|
||||
pendingOutputChars = 0
|
||||
pendingLocAdded = 0
|
||||
|
|
@ -1010,12 +1153,14 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
const last = info.last_token_usage
|
||||
let inputTokens = 0
|
||||
let cachedInputTokens = 0
|
||||
let cacheWriteTokens = 0
|
||||
let outputTokens = 0
|
||||
let reasoningTokens = 0
|
||||
|
||||
if (last) {
|
||||
inputTokens = last.input_tokens ?? 0
|
||||
cachedInputTokens = last.cached_input_tokens ?? 0
|
||||
cacheWriteTokens = last.cache_write_input_tokens ?? 0
|
||||
outputTokens = last.output_tokens ?? 0
|
||||
reasoningTokens = last.reasoning_output_tokens ?? 0
|
||||
} else if (cumulativeTotal > 0) {
|
||||
|
|
@ -1023,6 +1168,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
if (!total) continue
|
||||
inputTokens = (total.input_tokens ?? 0) - prevInput
|
||||
cachedInputTokens = (total.cached_input_tokens ?? 0) - prevCached
|
||||
cacheWriteTokens = (total.cache_write_input_tokens ?? 0) - prevCacheWrite
|
||||
outputTokens = (total.output_tokens ?? 0) - prevOutput
|
||||
reasoningTokens = (total.reasoning_output_tokens ?? 0) - prevReasoning
|
||||
}
|
||||
|
|
@ -1038,6 +1184,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
if (total) {
|
||||
prevInput = total.input_tokens ?? 0
|
||||
prevCached = total.cached_input_tokens ?? 0
|
||||
prevCacheWrite = total.cache_write_input_tokens ?? 0
|
||||
prevOutput = total.output_tokens ?? 0
|
||||
prevReasoning = total.reasoning_output_tokens ?? 0
|
||||
}
|
||||
|
|
@ -1049,7 +1196,22 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
// Normalize to Anthropic semantics: inputTokens = non-cached only.
|
||||
const uncachedInputTokens = Math.max(0, inputTokens - cachedInputTokens)
|
||||
|
||||
// Cache writes are carved out of the uncached input, never added to
|
||||
// it: clamp so a malformed or lagging count can never drive the plain
|
||||
// input bucket negative.
|
||||
const cacheWriteInputTokens = Math.max(0, Math.min(cacheWriteTokens, uncachedInputTokens))
|
||||
|
||||
const model = resolveModel(entry.payload, sessionModel)
|
||||
// Only move tokens into the cache-write bucket when the pricing
|
||||
// source publishes a real cache-write rate for this model (gpt-5.6+
|
||||
// charges 1.25x input; everything before it charges nothing extra).
|
||||
// Otherwise buildCosts' fabricated 1.25x default would invent a
|
||||
// surcharge that OpenAI never billed, so the tokens stay where they
|
||||
// already were -- in plain input, priced exactly as before.
|
||||
const billedCacheWriteTokens = cacheWriteInputTokens > 0 && getModelCosts(model)?.cacheWriteCostIsExplicit
|
||||
? cacheWriteInputTokens
|
||||
: 0
|
||||
const billedInputTokens = uncachedInputTokens - billedCacheWriteTokens
|
||||
const timestamp = entry.timestamp ?? ''
|
||||
// Forked sessions copy the parent's entire token_count history
|
||||
// (re-timestamped), so replays must collide with the parent's events
|
||||
|
|
@ -1067,14 +1229,22 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
// key would spuriously diverge on a replay and double-count it.
|
||||
const dedupKey = `codex:${forkedFromId || sessionId}:${cumulativeTotal}:${total?.input_tokens ?? 0}:${total?.cached_input_tokens ?? 0}:${total?.output_tokens ?? 0}:${total?.reasoning_output_tokens ?? 0}`
|
||||
|
||||
// A drop here can only be a byte-identical replay: the
|
||||
// prevCumulativeTotal guard above already discards a repeated
|
||||
// running total, so nothing reaching this point ever loses real
|
||||
// tokens -- no active-time rescaling needed (#1088 investigation).
|
||||
if (seenKeys.has(dedupKey)) continue
|
||||
seenKeys.add(dedupKey)
|
||||
|
||||
// Reasoning tokens are already inside output_tokens, so they are NOT
|
||||
// added here. The cache-rehydration twin of this line lives in
|
||||
// src/parser.ts (cachedCallToApiCall); both call billableOutputTokens
|
||||
// so a fresh parse and a cache read can never price differently.
|
||||
const costUSD = calculateCost(
|
||||
model,
|
||||
uncachedInputTokens,
|
||||
outputTokens + reasoningTokens,
|
||||
0,
|
||||
billedInputTokens,
|
||||
billableOutputTokens('codex', outputTokens, reasoningTokens),
|
||||
billedCacheWriteTokens,
|
||||
cachedInputTokens,
|
||||
0,
|
||||
)
|
||||
|
|
@ -1082,9 +1252,9 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
pendingTaskCalls.push({
|
||||
provider: 'codex',
|
||||
model,
|
||||
inputTokens: uncachedInputTokens,
|
||||
inputTokens: billedInputTokens,
|
||||
outputTokens,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheCreationInputTokens: billedCacheWriteTokens,
|
||||
cacheReadInputTokens: cachedInputTokens,
|
||||
cachedInputTokens,
|
||||
reasoningTokens,
|
||||
|
|
@ -1097,6 +1267,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
deduplicationKey: dedupKey,
|
||||
turnId: currentTurnId,
|
||||
toolSequence: pendingToolSequence.length > 0 ? pendingToolSequence : undefined,
|
||||
...(pendingSkills.length > 0 ? { skills: pendingSkills } : {}),
|
||||
userMessage: pendingUserMessage,
|
||||
sessionId,
|
||||
...(sessionCwd ? { projectPath: sessionCwd, workingDirectory: sessionCwd } : {}),
|
||||
|
|
@ -1104,10 +1275,11 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
...(pendingLocRemoved ? { locRemoved: pendingLocRemoved } : {}),
|
||||
...(pendingEditFailed ? { editFailed: pendingEditFailed } : {}),
|
||||
})
|
||||
taskGeneratedTokens += outputTokens + reasoningTokens
|
||||
taskGeneratedTokens += billableOutputTokens('codex', outputTokens, reasoningTokens)
|
||||
|
||||
pendingTools = []
|
||||
pendingToolSequence = []
|
||||
pendingSkills = []
|
||||
pendingUserMessage = ''
|
||||
pendingOutputChars = 0
|
||||
pendingLocAdded = 0
|
||||
|
|
|
|||
|
|
@ -783,8 +783,9 @@ function createJsonlParser(
|
|||
// CUMULATIVE per-model totals. Emitting each rollup whole would need the
|
||||
// cache to update a prior call in place — the durable merge is
|
||||
// append-only by dedup key — so we emit per-leg DELTAS keyed by
|
||||
// occurrence instead: re-parses of a growing file append only the new
|
||||
// leg, and each leg lands on its own timestamp.
|
||||
// occurrence (`:n`): re-parses of a growing file append only the new
|
||||
// leg, and each leg lands on its own timestamp. Discovery only yields
|
||||
// `<sid>/events.jsonl`, so two journals cannot share a session id.
|
||||
const prevShutdownUsage = new Map<string, ShutdownModelUsage>()
|
||||
const shutdownCountByModel = new Map<string, number>()
|
||||
|
||||
|
|
@ -897,6 +898,13 @@ function createJsonlParser(
|
|||
const modelMetrics = shutdownData.modelMetrics
|
||||
if (!isRecord(modelMetrics)) continue
|
||||
|
||||
// Prefer lastEventTimestamp over sessionStartTime. sessionStartTime
|
||||
// is identical for every stampless leg of a resumed session, so
|
||||
// using it for the call timestamp (or, previously, the key) collapsed
|
||||
// those legs onto one date. lastEventTimestamp is the last stamped
|
||||
// event in this journal — distinct per leg when intervening events
|
||||
// are stamped, and still a real time when they are not.
|
||||
//
|
||||
// Fallback order matters for accounting, not just display: this
|
||||
// stamp anchors the leg's interval in the serve-time reconciliation,
|
||||
// which subtracts the store rows written up to it. A shutdown
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { readdir, readFile, stat } from 'fs/promises'
|
|||
import { join, basename } from 'path'
|
||||
import { homedir } from 'os'
|
||||
|
||||
import { calculateCost } from '../models.js'
|
||||
import { calculateCost, getShortModelName } from '../models.js'
|
||||
import { openDatabase, type SqliteDatabase } from '../sqlite.js'
|
||||
import { normalizeContentBlocks } from '../content-utils.js'
|
||||
import { estimateTokensFromChars } from '../token-estimate.js'
|
||||
|
|
@ -506,7 +506,7 @@ export function createCursorAgentProvider(baseDirOverride?: string): Provider {
|
|||
|
||||
modelDisplayName(model: string): string {
|
||||
if (model === 'cursor-agent-auto') return 'Cursor (auto)'
|
||||
const label = modelDisplayNames[model] ?? model
|
||||
const label = modelDisplayNames[model] ?? getShortModelName(model)
|
||||
return `${label} (est.)`
|
||||
},
|
||||
|
||||
|
|
|
|||
|
|
@ -405,12 +405,15 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
|
|||
inputTokens: parsed.usage.input,
|
||||
// Grok reports reasoning INSIDE outputTokens, but the repo contract is
|
||||
// the opposite: ParsedProviderCall.reasoningTokens is exclusive of
|
||||
// outputTokens, and every consumer sums the two (parser.ts's
|
||||
// cachedCallToApiCall for cost, modelBreakdown for tokens, and the
|
||||
// models/audit reports). tests/providers/kiro.test.ts states it
|
||||
// outright. So split it here rather than special-casing grok in five
|
||||
// downstream places: subtracting reasoning makes `output + reasoning`
|
||||
// reconstruct exactly the number Grok reported.
|
||||
// outputTokens. Downstream consumers reconstitute the billable total
|
||||
// through billableOutputTokens() (models.ts): it adds reasoning back on
|
||||
// top for grok and every other provider, except the
|
||||
// REASONING_INCLUDED_IN_OUTPUT set (claude, codex) whose reasoning is
|
||||
// already inside output_tokens and must not be added again.
|
||||
// tests/providers/kiro.test.ts states the exclusive contract outright.
|
||||
// So split it here rather than special-casing grok in every downstream
|
||||
// site: subtracting reasoning makes `output + reasoning` reconstruct
|
||||
// exactly the number Grok reported.
|
||||
outputTokens: parsed.usage.output - reasoningTokens,
|
||||
cacheCreationInputTokens: parsed.usage.cacheCreation,
|
||||
cacheReadInputTokens: parsed.usage.cacheRead,
|
||||
|
|
|
|||
|
|
@ -199,6 +199,29 @@ const coreProviders: Provider[] = [claude, cline, clineCli, codewhale, codebuff,
|
|||
// when an optional module fails to load. Must stay in sync with getAllProviders.
|
||||
const lazyProviderNames = ['antigravity', 'forge', 'goose', 'cursor', 'opencode', 'cursor-agent', 'crush', 'warp', 'vercel-gateway', 'zcode', 'zed']
|
||||
|
||||
// Display names for lazy providers. Must match the `displayName` on the
|
||||
// loaded Provider object; `providerDisplayName` + getAllProviders() test
|
||||
// is the drift check.
|
||||
const lazyProviderDisplayNames: Record<string, string> = {
|
||||
antigravity: 'Antigravity',
|
||||
forge: 'Forge',
|
||||
goose: 'Goose',
|
||||
cursor: 'Cursor',
|
||||
opencode: 'OpenCode',
|
||||
'cursor-agent': 'Cursor Agent',
|
||||
crush: 'Crush',
|
||||
warp: 'Warp',
|
||||
'vercel-gateway': 'Vercel AI Gateway',
|
||||
zcode: 'ZCode',
|
||||
zed: 'Zed',
|
||||
}
|
||||
|
||||
export function providerDisplayName(name: string): string {
|
||||
const core = coreProviders.find(p => p.name === name)
|
||||
if (core) return core.displayName
|
||||
return lazyProviderDisplayNames[name] ?? name
|
||||
}
|
||||
|
||||
// Canonical set of every provider name (core + lazy), used to validate the
|
||||
// --provider CLI flag. Computed lazily so importing this module never depends on
|
||||
// every provider object being defined at load time (e.g. under test mocks).
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { getShortModelName } from '../models.js'
|
||||
import type { DateRange } from '../types.js'
|
||||
import type { Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
|
||||
import { fetchWithTimeout } from '../fetch-utils.js'
|
||||
|
|
@ -124,7 +125,7 @@ export const vercelGateway: Provider = {
|
|||
|
||||
modelDisplayName(model: string): string {
|
||||
const slash = model.indexOf('/')
|
||||
return slash >= 0 ? model.slice(slash + 1) : model
|
||||
return getShortModelName(slash >= 0 ? model.slice(slash + 1) : model)
|
||||
},
|
||||
|
||||
toolDisplayName(rawTool: string): string {
|
||||
|
|
|
|||
|
|
@ -299,12 +299,35 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
|
|||
// lockstep so the pre-session-cache layer re-parses too.)
|
||||
// session-meta-model-v1: parse large session_meta records structurally so a
|
||||
// nested base_instructions provenance.model cannot overwrite turn_context.
|
||||
codex: 'mcp-attribution-v5-est-cost-active-timing-mcp-wait-rich-capture-v1-cross-provider-pr-v1-session-meta-model-v1',
|
||||
// session-meta-fields-v1: the same depth-1 window for cwd/name/originator/
|
||||
// session_id/forked_from_id/model_provider, not just model. (#1055)
|
||||
// codex-pricing-v1 (#1075): reasoning tokens are no longer added on top of
|
||||
// output, and cache_write_input_tokens moves out of the plain input bucket on
|
||||
// models with an explicit cache-write rate. The bucket move does NOT self-heal
|
||||
// on read (cached entries store the buckets, not the raw event), so cached
|
||||
// sessions must re-parse.
|
||||
// codex-tps-v1 (#1079): activeGeneratedTokens summed output + reasoning, the
|
||||
// same double-count codex-pricing-v1 removed from cost. Cached entries store
|
||||
// activeGeneratedTokens/activeDurationMs/toolWaitMs verbatim (cachedCallToApiCall
|
||||
// passes them through without recomputing), so this does NOT self-heal either.
|
||||
// codex-mcp-skills-v1 (#478): CLI-wrapped MCP calls and SKILL.md reads made
|
||||
// through the `exec` custom tool or the item model's `CommandExecution` item
|
||||
// were counted as Bash only. Cached sessions store tools/toolSequence/skills
|
||||
// verbatim, so they must re-parse to gain the attribution.
|
||||
// activity-price-v1: `codex-auto-review` now prices via the recommended
|
||||
// review model. session-cache.json would otherwise keep the pre-alias $0.
|
||||
// Compose all four — a take-ours merge would drop #1075, #1079, or #1092.
|
||||
codex: 'mcp-attribution-v5-est-cost-active-timing-mcp-wait-rich-capture-v1-cross-provider-pr-v1-session-meta-model-v1-session-meta-fields-v1-codex-pricing-v1-codex-tps-v1-codex-mcp-skills-v1-activity-price-v1',
|
||||
cursor: 'composer-anchored-crediting-v1-est-cost',
|
||||
'cursor-agent': 'workspaceless-transcript-v1',
|
||||
// source-provenance-v1 (#944): CLI sessions were misread as VS Code
|
||||
// transcripts (both carry producer 'copilot-agent'), skipping the shutdown
|
||||
// input/cache rollup; this bump re-parses them so the missing tokens land.
|
||||
// #1051 did NOT bump this on its own. A fingerprint change drops every present
|
||||
// Copilot source (parser.ts getOrCreateProviderSection) and would erase
|
||||
// conversations already pruned from a still-present OTel DB. Old `:n`
|
||||
// shutdown keys migrate via cachedFileNeedsProviderReparse + a durable
|
||||
// strip of legacy shutdown calls on that JSONL file only.
|
||||
// session-store-v2: input/cache for sessions covered by session-store.db
|
||||
// moved from shutdown-rollup calls to per-request DB rows. This bump
|
||||
// re-parses pre-store caches so the DB rows land; the rollup calls stay
|
||||
|
|
|
|||
|
|
@ -154,7 +154,7 @@ export const COMMIT_ATTRIBUTION_SPAN_NAME = 'codeburn.commit'
|
|||
* encodes the mutable state (inMain/wasReverted for commits; repo, PR links,
|
||||
* and commit set for sessions), so a state TRANSITION mints a new key and the
|
||||
* updated fact is re-sent on the next push — the receiver upserts by
|
||||
* (repo, sha) / (session). Identical states dedupe via the sent-ledger.
|
||||
* (repo, sha) / traceId. Identical states dedupe via the sent-ledger.
|
||||
*/
|
||||
export type AttributionItem = {
|
||||
kind: 'session' | 'commit'
|
||||
|
|
@ -256,7 +256,6 @@ export function buildAttributionOtlpPayload(items: AttributionItem[]): OtlpPaylo
|
|||
const endNano = (rawEndNano > minEndNano ? rawEndNano : minEndNano).toString()
|
||||
|
||||
const attributes: OtlpAttribute[] = [
|
||||
{ key: 'ai.session_id', value: { stringValue: item.sessionId } },
|
||||
{ key: 'ai.project', value: { stringValue: item.project } },
|
||||
]
|
||||
if (item.repo) {
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory, type DateRange
|
|||
import { isBehavioralCall } from './behavioral-weight.js'
|
||||
import { type PeriodData, type ProviderCost, type BreakdownArrays, type MenubarPayload, type ClaudeConfigSelector, buildMenubarPayload } from './menubar-json.js'
|
||||
import { parseAllSessions, filterProjectsByName, filterProjectsByDays, filterProjectsByClaudeConfigSource, isSessionHydrationComplete } from './parser.js'
|
||||
import { findUnpricedModels, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName, isExpectedFreeModel } from './models.js'
|
||||
import { findUnpricedModels, getFlatRateModelsConfigHash, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName, isExpectedFreeModel } from './models.js'
|
||||
import { getAllProviders, safeDiscoverSessions } from './providers/index.js'
|
||||
import { claude, getClaudeConfigDirs, getDesktopSessionsDirs } from './providers/claude.js'
|
||||
import { stat } from 'node:fs/promises'
|
||||
|
|
@ -83,8 +83,8 @@ export function buildPeriodData(label: string, projects: ProjectSummary[]): Peri
|
|||
export function getDailyCacheConfigHash(): string {
|
||||
const savingsHash = getLocalModelSavingsConfigHash()
|
||||
const overridesHash = getPriceOverridesConfigHash()
|
||||
if (!overridesHash) return savingsHash
|
||||
return `localModelSavings=${savingsHash}\u0002priceOverrides=${overridesHash}`
|
||||
const flatRateHash = getFlatRateModelsConfigHash()
|
||||
return `localModelSavings=${savingsHash}\u0002priceOverrides=${overridesHash}\u0002flatRateModels=${flatRateHash}`
|
||||
}
|
||||
|
||||
async function hydrateCache(): Promise<DailyCache> {
|
||||
|
|
@ -610,9 +610,9 @@ export async function buildMenubarPayloadForRange(periodInfo: PeriodInfo, opts:
|
|||
}
|
||||
claudeConfigs = claudeConfigs ?? await claudeConfigSelector(scanProjects, null)
|
||||
|
||||
// Codex credits for the period. Reuses the models aggregation (folds reasoning
|
||||
// into output, keeps non-cached input + cached-read separate) so the figure
|
||||
// matches the official credit rates.
|
||||
// Codex credits for the period. Reuses the models aggregation (billable output
|
||||
// already includes reasoning for codex, keeps non-cached input + cached-read
|
||||
// separate) so the figure matches the official credit rates.
|
||||
const modelRows = await aggregateModels(scanProjects)
|
||||
currentData.codexCredits = modelRows.reduce(
|
||||
(sum, r) => sum + (r.provider === 'codex' && r.credits != null ? r.credits : 0),
|
||||
|
|
|
|||
|
|
@ -86,8 +86,11 @@ describe('aggregateAudit', () => {
|
|||
expect(r.raw.reasoningTokens).toBe(10)
|
||||
expect(r.raw.cacheReadInputTokens).toBe(200)
|
||||
expect(r.raw.cachedInputTokens).toBe(300)
|
||||
// reasoning folds into output for pricing
|
||||
expect(r.displayed.outputTokens).toBe(110)
|
||||
// Reasoning does NOT fold into output for claude or codex: both bill it
|
||||
// as part of output_tokens already, so adding it would double-count
|
||||
// (#1075). Providers that report reasoning as a separate bucket still get
|
||||
// the additive treatment - see tests/codex-pricing-1075.test.ts.
|
||||
expect(r.displayed.outputTokens).toBe(100)
|
||||
// cache read is the SUM of per-call max(anthropic, openai), not max of sums
|
||||
expect(r.displayed.cacheReadTokens).toBe(500)
|
||||
// attributed cost is preserved exactly
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
|||
|
||||
import {
|
||||
clearCodexMemCaches,
|
||||
codexCacheFileName,
|
||||
fingerprintFile,
|
||||
flushCodexCache,
|
||||
readCachedCodexResults,
|
||||
writeCachedCodexResults,
|
||||
} from '../src/codex-cache.js'
|
||||
import {
|
||||
antigravityCacheFileName,
|
||||
clearAntigravityCacheStates,
|
||||
createAntigravityProvider,
|
||||
flushAntigravityCache,
|
||||
|
|
@ -53,7 +55,7 @@ async function seedAntigravityCache(
|
|||
): Promise<void> {
|
||||
const sourceStat = await stat(sourcePath)
|
||||
await mkdir(cacheDir, { recursive: true })
|
||||
await writeFile(join(cacheDir, 'antigravity-results.json'), JSON.stringify({
|
||||
await writeFile(join(cacheDir, antigravityCacheFileName()), JSON.stringify({
|
||||
version: 5,
|
||||
cascades: {
|
||||
shared: {
|
||||
|
|
@ -107,7 +109,7 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
await writeCachedCodexResults(sourcePath, 'project-b', [call('codex', 'from-b')], fingerprint!)
|
||||
await flushCodexCache()
|
||||
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, 'codex-results.json'), 'utf8'))
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, codexCacheFileName()), 'utf8'))
|
||||
expect(diskB.files[sourcePath].calls.map((entry: ParsedProviderCall) => entry.model)).toEqual(['from-b'])
|
||||
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheA
|
||||
|
|
@ -129,12 +131,12 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
await writeCachedCodexResults(sourceB, 'project-b', [call('codex', 'dirty-b')], (await fingerprintFile(sourceB))!)
|
||||
await flushCodexCache()
|
||||
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, 'codex-results.json'), 'utf8'))
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, codexCacheFileName()), 'utf8'))
|
||||
expect(Object.keys(diskB.files)).toEqual([sourceB])
|
||||
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheA
|
||||
await flushCodexCache()
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, 'codex-results.json'), 'utf8'))
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, codexCacheFileName()), 'utf8'))
|
||||
expect(Object.keys(diskA.files)).toEqual([sourceA])
|
||||
})
|
||||
|
||||
|
|
@ -200,9 +202,9 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
expect(projects.some(project => project.sessions.some(session =>
|
||||
session.turns.some(turn => turn.assistantCalls.some(entry => entry.provider === 'codex'))
|
||||
))).toBe(true)
|
||||
expect(existsSync(join(cacheA, 'codex-results.json'))).toBe(true)
|
||||
expect(existsSync(join(cacheB, 'codex-results.json'))).toBe(false)
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, 'codex-results.json'), 'utf8'))
|
||||
expect(existsSync(join(cacheA, codexCacheFileName()))).toBe(true)
|
||||
expect(existsSync(join(cacheB, codexCacheFileName()))).toBe(false)
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, codexCacheFileName()), 'utf8'))
|
||||
expect(diskA.files[sourcePath].calls).toHaveLength(1)
|
||||
clearSessionCache()
|
||||
})
|
||||
|
|
@ -238,10 +240,10 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
process.env['CODEBURN_CACHE_DIR'] = cacheB
|
||||
await flushAntigravityCache(new Set(), cacheA)
|
||||
|
||||
expect(existsSync(join(cacheB, 'antigravity-results.json'))).toBe(true)
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, 'antigravity-results.json'), 'utf8'))
|
||||
expect(existsSync(join(cacheB, antigravityCacheFileName()))).toBe(true)
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, antigravityCacheFileName()), 'utf8'))
|
||||
expect(diskB.cascades.shared.calls[0].model).toBe('from-b')
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, 'antigravity-results.json'), 'utf8'))
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, antigravityCacheFileName()), 'utf8'))
|
||||
expect(diskA.cascades).toEqual({})
|
||||
})
|
||||
|
||||
|
|
@ -261,9 +263,9 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
// Another process republishes both cache files. Without the clear, the
|
||||
// resident keeps serving its warm copies.
|
||||
await seedAntigravityCache(cacheDir, antigravitySource, 'after')
|
||||
const codexDisk = JSON.parse(await readFile(join(cacheDir, 'codex-results.json'), 'utf8'))
|
||||
const codexDisk = JSON.parse(await readFile(join(cacheDir, codexCacheFileName()), 'utf8'))
|
||||
codexDisk.files[codexSource].calls[0].model = 'after'
|
||||
await writeFile(join(cacheDir, 'codex-results.json'), JSON.stringify(codexDisk))
|
||||
await writeFile(join(cacheDir, codexCacheFileName()), JSON.stringify(codexDisk))
|
||||
|
||||
expect((await readCachedCodexResults(codexSource))?.calls.map(entry => entry.model)).toEqual(['before'])
|
||||
expect(await readAntigravityModel(antigravitySource)).toBe('before')
|
||||
|
|
|
|||
|
|
@ -79,7 +79,12 @@ describe('warm refresh child-process regression', () => {
|
|||
const loserOutcome = await waitForAny(barriers, [
|
||||
`${loser}.timed-out`, `${loser}.parsed`, `${loser}.completed-by-other`, `${loser}.unavailable`,
|
||||
])
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).toBe(`${loser}.timed-out`)
|
||||
// Exactly one owner publishes. The loser of a stale-lock contest is not
|
||||
// the owner: `timed-out` is the usual wait-out, but missing+missing during
|
||||
// the winner's unlink-guard/create-successor gap is honestly
|
||||
// `completed-by-other` (#904). Do not require timed-out.
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).not.toBe(`${loser}.parsed`)
|
||||
expect([`${loser}.timed-out`, `${loser}.completed-by-other`, `${loser}.unavailable`]).toContain(loserOutcome)
|
||||
await writeFile(join(barriers, `${winner}.save`), '')
|
||||
await Promise.all([waitForExit(a), waitForExit(b)])
|
||||
await expect(stat(join(cacheDir, 'session-refresh.lock.takeover'))).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
|
|
@ -113,7 +118,12 @@ describe('warm refresh child-process regression', () => {
|
|||
const loserOutcome = await waitForAny(barriers, [
|
||||
`${loser}.timed-out`, `${loser}.parsed`, `${loser}.completed-by-other`, `${loser}.unavailable`,
|
||||
])
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).toBe(`${loser}.timed-out`)
|
||||
// Exactly one owner publishes. The loser of a stale-lock contest is not
|
||||
// the owner: `timed-out` is the usual wait-out, but missing+missing during
|
||||
// the winner's unlink-guard/create-successor gap is honestly
|
||||
// `completed-by-other` (#904). Do not require timed-out.
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).not.toBe(`${loser}.parsed`)
|
||||
expect([`${loser}.timed-out`, `${loser}.completed-by-other`, `${loser}.unavailable`]).toContain(loserOutcome)
|
||||
await writeFile(join(barriers, `${winner}.save`), '')
|
||||
await Promise.all([waitForExit(a), waitForExit(b)])
|
||||
await expect(stat(join(cacheDir, 'session-refresh.lock.takeover'))).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
|
|
|
|||
105
tests/cli-model-flat-rate.test.ts
Normal file
105
tests/cli-model-flat-rate.test.ts
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
import { mkdtemp, readFile, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { spawnSync } from 'node:child_process'
|
||||
|
||||
import { describe, it, expect } from 'vitest'
|
||||
|
||||
const CLI_TIMEOUT_MS = 30_000
|
||||
|
||||
function runCli(args: string[], home: string) {
|
||||
return spawnSync(process.execPath, ['--import', 'tsx', 'src/cli.ts', ...args], {
|
||||
cwd: process.cwd(),
|
||||
env: {
|
||||
...process.env,
|
||||
HOME: home,
|
||||
USERPROFILE: home,
|
||||
HOMEPATH: home,
|
||||
HOMEDRIVE: '',
|
||||
},
|
||||
encoding: 'utf-8',
|
||||
})
|
||||
}
|
||||
|
||||
function readConfig(home: string): Promise<Record<string, unknown>> {
|
||||
return readFile(join(home, '.config', 'codeburn', 'config.json'), 'utf-8')
|
||||
.then(raw => JSON.parse(raw) as Record<string, unknown>)
|
||||
}
|
||||
|
||||
describe('codeburn model-flat-rate command', () => {
|
||||
it('saves, lists, and removes a flat-rate mark', async () => {
|
||||
const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-'))
|
||||
try {
|
||||
const set = runCli(['model-flat-rate', 'zz-my-pass-sku'], home)
|
||||
expect(set.status).toBe(0)
|
||||
expect(set.stdout).toContain('Flat-rate mark saved: zz-my-pass-sku')
|
||||
|
||||
const saved = await readConfig(home)
|
||||
expect(saved.flatRateModels).toEqual(['zz-my-pass-sku'])
|
||||
|
||||
const list = runCli(['model-flat-rate', '--list'], home)
|
||||
expect(list.status).toBe(0)
|
||||
expect(list.stdout).toContain('zz-my-pass-sku')
|
||||
|
||||
const remove = runCli(['model-flat-rate', '--remove', 'zz-my-pass-sku'], home)
|
||||
expect(remove.status).toBe(0)
|
||||
|
||||
const after = await readConfig(home)
|
||||
expect(after.flatRateModels).toBeUndefined()
|
||||
expect(after.flatRateModelsRemoved).toBeUndefined()
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true })
|
||||
}
|
||||
}, CLI_TIMEOUT_MS)
|
||||
|
||||
it('warns when the same model is also configured in modelAliases', async () => {
|
||||
const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-'))
|
||||
try {
|
||||
expect(runCli(['model-alias', 'auto-genius', 'gpt-4o'], home).status).toBe(0)
|
||||
const set = runCli(['model-flat-rate', 'auto-genius'], home)
|
||||
expect(set.status).toBe(0)
|
||||
expect(set.stdout).toContain('also in modelAliases')
|
||||
expect(set.stdout).toContain('invents per-token spend')
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true })
|
||||
}
|
||||
}, CLI_TIMEOUT_MS)
|
||||
|
||||
it('rejects a remove for an unknown mark', async () => {
|
||||
const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-'))
|
||||
try {
|
||||
const result = runCli(['model-flat-rate', '--remove', 'unknown-sku'], home)
|
||||
expect(result.status).toBe(1)
|
||||
expect(result.stderr).toContain('No flat-rate mark found')
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true })
|
||||
}
|
||||
}, CLI_TIMEOUT_MS)
|
||||
|
||||
it('opts out of a built-in SKU so the unpriced warning can fire again', async () => {
|
||||
const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-'))
|
||||
try {
|
||||
const remove = runCli(['model-flat-rate', '--remove', 'auto-genius'], home)
|
||||
expect(remove.status).toBe(0)
|
||||
expect(remove.stdout).toContain('Removed flat-rate mark: auto-genius')
|
||||
expect(remove.stdout).toContain('Built-in SKU opted out')
|
||||
|
||||
const saved = await readConfig(home)
|
||||
expect(saved.flatRateModels).toBeUndefined()
|
||||
expect(saved.flatRateModelsRemoved).toEqual(['auto-genius'])
|
||||
|
||||
const list = runCli(['model-flat-rate', '--list'], home)
|
||||
expect(list.status).toBe(0)
|
||||
expect(list.stdout).toContain('auto-genius')
|
||||
expect(list.stdout).toContain('Built-in flat-rate opt-outs')
|
||||
|
||||
const restore = runCli(['model-flat-rate', 'auto-genius'], home)
|
||||
expect(restore.status).toBe(0)
|
||||
const after = await readConfig(home)
|
||||
expect(after.flatRateModels).toEqual(['auto-genius'])
|
||||
expect(after.flatRateModelsRemoved).toBeUndefined()
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true })
|
||||
}
|
||||
}, CLI_TIMEOUT_MS)
|
||||
})
|
||||
|
|
@ -114,9 +114,7 @@ describe('codeburn models --unpriced public CLI', () => {
|
|||
expect(result.stdout).toContain('acme/unknown-beta-969')
|
||||
expect(result.stdout).not.toContain('claude-opus-4-6')
|
||||
expect(result.stdout).toContain('If a model is billed per token, map it with: codeburn model-alias "<model>" <known-model>')
|
||||
// #968: aliasing a subscription-billed model fabricates spend, so the
|
||||
// hint must never read as an unconditional instruction.
|
||||
expect(result.stdout).toContain('Subscription or flat-rate models are correctly $0.')
|
||||
expect(result.stdout).toContain('codeburn model-flat-rate')
|
||||
expect(result.stdout).not.toContain('Fix: codeburn model-alias')
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -88,7 +88,8 @@ describe('codex parser change invalidates stale session-cache (#478/#513)', () =
|
|||
}
|
||||
}
|
||||
await writeCacheOnDisk(cache)
|
||||
const codexCachePath = join(CACHE_DIR, 'codex-results.json')
|
||||
const { codexCacheFileName } = await import('../src/codex-cache.js')
|
||||
const codexCachePath = join(CACHE_DIR, codexCacheFileName())
|
||||
const codexCache = JSON.parse(await readFile(codexCachePath, 'utf8'))
|
||||
codexCache.version = 4
|
||||
for (const f of Object.values(codexCache.files) as any[]) {
|
||||
|
|
|
|||
|
|
@ -17,6 +17,12 @@ describe('codexCreditRate', () => {
|
|||
expect(codexCreditRate('gpt-4o')).toBeNull()
|
||||
expect(codexCreditRate('claude-opus-4-8')).toBeNull()
|
||||
})
|
||||
|
||||
it('resolves the auto-review activity id to the same rate as GPT-5.5', () => {
|
||||
expect(codexCreditRate('codex-auto-review')).toEqual(codexCreditRate('gpt-5.5'))
|
||||
expect(codexCreditRate('codex-auto-review')).not.toBeNull()
|
||||
expect(codexCreditRate('CODEX-AUTO-REVIEW')).toEqual(codexCreditRate('gpt-5.5'))
|
||||
})
|
||||
})
|
||||
|
||||
describe('codexCredits', () => {
|
||||
|
|
@ -32,11 +38,6 @@ describe('codexCredits', () => {
|
|||
expect(codexCredits('gpt-5.5', { inputTokens: 0, cachedReadTokens: 1_000_000, outputTokens: 0 })).toBe(12.5)
|
||||
})
|
||||
|
||||
it('folds reasoning tokens into the output rate', () => {
|
||||
// 500k output + 500k reasoning = 1M output-billed => 750 credits.
|
||||
expect(codexCredits('gpt-5.5', { inputTokens: 0, cachedReadTokens: 0, outputTokens: 500_000, reasoningTokens: 500_000 })).toBe(750)
|
||||
})
|
||||
|
||||
it('sums a mixed record (gpt-5.4)', () => {
|
||||
// 2M input (125) + 1M cached (6.25) + 0.5M output (187.5) = 318.75
|
||||
const credits = codexCredits('gpt-5.4', { inputTokens: 2_000_000, cachedReadTokens: 1_000_000, outputTokens: 500_000 })
|
||||
|
|
@ -50,4 +51,8 @@ describe('codexCredits', () => {
|
|||
it('returns null for an unknown model', () => {
|
||||
expect(codexCredits('gpt-4o', { inputTokens: 1_000_000, cachedReadTokens: 0, outputTokens: 0 })).toBeNull()
|
||||
})
|
||||
|
||||
it('charges auto-review at the GPT-5.5 credit rate, not null', () => {
|
||||
expect(codexCredits('codex-auto-review', { inputTokens: 1_000_000, cachedReadTokens: 0, outputTokens: 0 })).toBe(125)
|
||||
})
|
||||
})
|
||||
|
|
|
|||
69
tests/codex-pricing-1075-rehydrate.test.ts
Normal file
69
tests/codex-pricing-1075-rehydrate.test.ts
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
// #1075, cost site 2 of 2. Codex is NOT on parser.ts's reported-cost
|
||||
// pass-through allowlist, so the session cache stores its calls with
|
||||
// `costUSD: undefined` and every warm run re-prices them from the stored token
|
||||
// buckets in cachedCallToApiCall. That line and the one in the codex provider
|
||||
// are twins: if only one drops the reasoning double-count, a user's number
|
||||
// changes between a cold and a warm run. This drives the full parseAllSessions
|
||||
// pipeline twice against the same file to prove they agree.
|
||||
//
|
||||
// Own file because the codex provider captures CODEX_HOME when its module is
|
||||
// first evaluated, so the env must be set before any import of it.
|
||||
|
||||
import { afterAll, beforeEach, expect, it, vi } from 'vitest'
|
||||
import { mkdir, rm, writeFile } from 'fs/promises'
|
||||
import { join } from 'path'
|
||||
|
||||
const testRoot = vi.hoisted(() => {
|
||||
const root = `${process.env['TMPDIR'] || '/tmp'}/codex-1075-rehydrate-${process.pid}-${Date.now()}`
|
||||
process.env['HOME'] = `${root}/home`
|
||||
process.env['USERPROFILE'] = `${root}/home`
|
||||
process.env['CODEX_HOME'] = `${root}/codex`
|
||||
return root
|
||||
})
|
||||
|
||||
const CODEX_HOME = join(testRoot, 'codex')
|
||||
const CACHE_DIR = join(testRoot, 'cache')
|
||||
|
||||
// gpt-5.5: input 5e-6, output 30e-6, cacheRead 5e-7 (src/data/litellm-snapshot.json).
|
||||
// 800 uncached input + 200 cached + 1000 output, of which 400 are reasoning.
|
||||
const EXPECTED = 800 * 5e-6 + 200 * 5e-7 + 1000 * 30e-6
|
||||
|
||||
beforeEach(() => {
|
||||
process.env['HOME'] = join(testRoot, 'home')
|
||||
process.env['USERPROFILE'] = join(testRoot, 'home')
|
||||
process.env['CODEX_HOME'] = CODEX_HOME
|
||||
process.env['CODEBURN_CACHE_DIR'] = CACHE_DIR
|
||||
})
|
||||
|
||||
afterAll(async () => {
|
||||
await rm(testRoot, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
it('prices a codex call the same on a cold parse and a cache-rehydrated read', async () => {
|
||||
const sessionDir = join(CODEX_HOME, 'sessions', '2026', '08', '16')
|
||||
await mkdir(sessionDir, { recursive: true })
|
||||
await mkdir(CACHE_DIR, { recursive: true })
|
||||
const usage = { input_tokens: 1000, cached_input_tokens: 200, output_tokens: 1000, reasoning_output_tokens: 400, total_tokens: 2000 }
|
||||
await writeFile(join(sessionDir, 'rollout-1075.jsonl'), [
|
||||
JSON.stringify({ type: 'session_meta', timestamp: '2026-08-16T10:00:00Z', payload: { session_id: 's1075', model: 'gpt-5.5', cwd: '/Users/test/proj', originator: 'codex_cli_rs' } }),
|
||||
JSON.stringify({ type: 'response_item', timestamp: '2026-08-16T10:00:10Z', payload: { type: 'message', role: 'user', content: [{ type: 'input_text', text: 'hello' }] } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-08-16T10:01:00Z', payload: { type: 'token_count', info: { model: 'gpt-5.5', last_token_usage: usage, total_token_usage: usage } } }),
|
||||
].join('\n') + '\n')
|
||||
|
||||
const { clearSessionCache, parseAllSessions } = await import('../src/parser.js')
|
||||
|
||||
clearSessionCache()
|
||||
const cold = await parseAllSessions(undefined, 'codex')
|
||||
const coldCost = cold.reduce((sum, p) => sum + p.totalCostUSD, 0)
|
||||
|
||||
// Drop the in-memory cache only: session-cache.json on disk now serves the
|
||||
// unchanged file, so this run's cost comes out of cachedCallToApiCall.
|
||||
clearSessionCache()
|
||||
const warm = await parseAllSessions(undefined, 'codex')
|
||||
const warmCost = warm.reduce((sum, p) => sum + p.totalCostUSD, 0)
|
||||
|
||||
// Revert only src/providers/codex.ts and the cold leg breaks; revert only
|
||||
// src/parser.ts's outputForCost and the warm leg breaks.
|
||||
expect(coldCost).toBeCloseTo(EXPECTED, 12)
|
||||
expect(warmCost).toBeCloseTo(EXPECTED, 12)
|
||||
})
|
||||
353
tests/codex-pricing-1075.test.ts
Normal file
353
tests/codex-pricing-1075.test.ts
Normal file
|
|
@ -0,0 +1,353 @@
|
|||
// Regression suite for #1075 (reported by chr-evensen).
|
||||
//
|
||||
// Two independent codex pricing bugs, each with the site that would silently
|
||||
// drift from its twin if only one half were reverted:
|
||||
//
|
||||
// A. reasoning_output_tokens is a SUBSET of output_tokens (OpenAI bills
|
||||
// reasoning as part of output; every token_count event in a 134k-event
|
||||
// corpus satisfies input + output == total), but codeburn added the two.
|
||||
// Priced in TWO places -- the fresh parse in src/providers/codex.ts and
|
||||
// the cache-rehydration re-price in src/parser.ts -- plus three display
|
||||
// sums. Both cost sites now go through billableOutputTokens(). The
|
||||
// cache-rehydration half lives in codex-pricing-1075-rehydrate.test.ts,
|
||||
// which needs CODEX_HOME set before the provider module is evaluated.
|
||||
//
|
||||
// B. cache_write_input_tokens was never read. It is now carved out of the
|
||||
// uncached-input bucket, but ONLY on models whose pricing source carries
|
||||
// an explicit cache-write rate: buildCosts() fabricates 1.25x input when
|
||||
// the source omits one, which is right for Anthropic but would invent a
|
||||
// surcharge OpenAI never charged on every pre-5.6 model.
|
||||
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises'
|
||||
import { tmpdir } from 'os'
|
||||
import { join } from 'path'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
|
||||
import { aggregateAudit } from '../src/audit-report.js'
|
||||
import { aggregateModels } from '../src/models-report.js'
|
||||
import { clearCodexMemCaches, readCachedCodexResults } from '../src/codex-cache.js'
|
||||
import { currentTzKey, ensureCacheHydrated, toDateString, type DailyEntry } from '../src/daily-cache.js'
|
||||
import { createCodexProvider } from '../src/providers/codex.js'
|
||||
import type { ParsedProviderCall } from '../src/providers/types.js'
|
||||
import type {
|
||||
ClassifiedTurn,
|
||||
ParsedApiCall,
|
||||
ProjectSummary,
|
||||
SessionSummary,
|
||||
TaskCategory,
|
||||
TokenUsage,
|
||||
} from '../src/types.js'
|
||||
|
||||
// Snapshot ground truth (src/data/litellm-snapshot.json), USD per token:
|
||||
// gpt-5.6-terra input 2e-6 output 12e-6 cacheWrite 2.5e-6 (EXPLICIT) cacheRead 2e-7
|
||||
// gpt-5.5 input 5e-6 output 30e-6 cacheWrite null (fabricated) cacheRead 5e-7
|
||||
const TERRA = { input: 2e-6, output: 12e-6, cacheWrite: 2.5e-6, cacheRead: 2e-7 }
|
||||
const GPT55 = { input: 5e-6, output: 30e-6, cacheRead: 5e-7 }
|
||||
|
||||
let tmpDir: string
|
||||
beforeEach(async () => { tmpDir = await mkdtemp(join(tmpdir(), 'codex-1075-')) })
|
||||
afterEach(async () => { await rm(tmpDir, { recursive: true, force: true }) })
|
||||
|
||||
type Usage = {
|
||||
input_tokens: number
|
||||
cached_input_tokens?: number
|
||||
cache_write_input_tokens?: number
|
||||
output_tokens: number
|
||||
reasoning_output_tokens?: number
|
||||
}
|
||||
|
||||
async function parseOneEvent(model: string, usage: Usage): Promise<ParsedProviderCall> {
|
||||
const total = usage.input_tokens + usage.output_tokens
|
||||
const sessionDir = join(tmpDir, 'sessions', '2026', '08', '16')
|
||||
await mkdir(sessionDir, { recursive: true })
|
||||
const filePath = join(sessionDir, `rollout-${model}-${Math.random().toString(36).slice(2)}.jsonl`)
|
||||
await writeFile(filePath, [
|
||||
JSON.stringify({
|
||||
type: 'session_meta',
|
||||
timestamp: '2026-08-16T10:00:00Z',
|
||||
payload: { cwd: '/Users/t/p', originator: 'codex-cli', session_id: 's1075', model },
|
||||
}),
|
||||
JSON.stringify({
|
||||
type: 'event_msg',
|
||||
timestamp: '2026-08-16T10:01:00Z',
|
||||
payload: {
|
||||
type: 'token_count',
|
||||
info: { model, last_token_usage: { ...usage, total_tokens: total }, total_token_usage: { ...usage, total_tokens: total } },
|
||||
},
|
||||
}),
|
||||
].join('\n') + '\n')
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const parser = provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set())
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of parser.parse()) calls.push(call)
|
||||
expect(calls).toHaveLength(1)
|
||||
return calls[0]!
|
||||
}
|
||||
|
||||
// ── Fix A: reasoning is already inside output ─────────────────────────────
|
||||
|
||||
describe('#1075 A - reasoning is not billed on top of output', () => {
|
||||
it('prices a fresh codex parse from output_tokens alone', async () => {
|
||||
const call = await parseOneEvent('gpt-5.5', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
output_tokens: 1000,
|
||||
reasoning_output_tokens: 400,
|
||||
})
|
||||
|
||||
// 800 uncached input + 200 cached + 1000 output. The 400 reasoning tokens
|
||||
// are INSIDE the 1000, so they must not be priced again.
|
||||
const expected = 800 * GPT55.input + 200 * GPT55.cacheRead + 1000 * GPT55.output
|
||||
expect(call.costUSD).toBeCloseTo(expected, 12)
|
||||
// Guard the direction: the pre-fix arithmetic charged 1400 output tokens.
|
||||
const preFix = 800 * GPT55.input + 200 * GPT55.cacheRead + 1400 * GPT55.output
|
||||
expect(call.costUSD).toBeLessThan(preFix)
|
||||
// The raw fields are still reported untouched; only the pricing changed.
|
||||
expect(call.outputTokens).toBe(1000)
|
||||
expect(call.reasoningTokens).toBe(400)
|
||||
})
|
||||
|
||||
it('does not double-count reasoning in the displayed output tokens', async () => {
|
||||
const codex = makeApiCall('codex', 'gpt-5.5', { outputTokens: 1000, reasoningTokens: 400 })
|
||||
// A provider that really does report reasoning as a separate bucket keeps
|
||||
// the additive behaviour, so this is a codex carve-out and not a blanket
|
||||
// change to every display sum. Gemini documents "thoughts" as genuinely
|
||||
// separate from output (src/providers/gemini.ts), unlike codex/claude.
|
||||
const additive = makeApiCall('gemini', 'gemini-2.5-pro', { outputTokens: 1000, reasoningTokens: 400 })
|
||||
const projects = [makeProject([codex, additive])]
|
||||
|
||||
const auditRows = await aggregateAudit(projects)
|
||||
expect(auditRows.find(r => r.provider === 'codex')!.displayed.outputTokens).toBe(1000)
|
||||
expect(auditRows.find(r => r.provider === 'gemini')!.displayed.outputTokens).toBe(1400)
|
||||
|
||||
const modelRows = await aggregateModels(projects)
|
||||
expect(modelRows.find(r => r.provider === 'codex')!.outputTokens).toBe(1000)
|
||||
expect(modelRows.find(r => r.provider === 'gemini')!.outputTokens).toBe(1400)
|
||||
})
|
||||
})
|
||||
|
||||
// ── Fix B: cache_write_input_tokens, guarded ──────────────────────────────
|
||||
|
||||
describe('#1075 B - cache_write_input_tokens', () => {
|
||||
it('prices cache writes at the explicit rate on gpt-5.6-terra', async () => {
|
||||
const call = await parseOneEvent('gpt-5.6-terra', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
cache_write_input_tokens: 300,
|
||||
output_tokens: 100,
|
||||
})
|
||||
|
||||
expect(call.inputTokens).toBe(500)
|
||||
expect(call.cacheCreationInputTokens).toBe(300)
|
||||
expect(call.cacheReadInputTokens).toBe(200)
|
||||
const expected =
|
||||
500 * TERRA.input +
|
||||
300 * TERRA.cacheWrite +
|
||||
200 * TERRA.cacheRead +
|
||||
100 * TERRA.output
|
||||
expect(expected).toBeCloseTo(0.00299, 12)
|
||||
expect(call.costUSD).toBeCloseTo(expected, 12)
|
||||
})
|
||||
|
||||
it('THE GUARD: leaves cache writes in the input bucket when the model has no explicit rate', async () => {
|
||||
// gpt-5.5 carries `null` for cache_creation_input_token_cost, so
|
||||
// buildCosts fabricates 1.25x input for it. OpenAI charges nothing extra
|
||||
// to write cache before gpt-5.6, so routing these tokens through that
|
||||
// fabricated rate would invent a surcharge. Cost must be byte-identical to
|
||||
// the pre-fix number. Delete the guard and this test fails.
|
||||
const withWrite = await parseOneEvent('gpt-5.5', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
cache_write_input_tokens: 300,
|
||||
output_tokens: 100,
|
||||
})
|
||||
const withoutWrite = await parseOneEvent('gpt-5.5', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
output_tokens: 100,
|
||||
})
|
||||
|
||||
expect(withWrite.inputTokens).toBe(800)
|
||||
expect(withWrite.cacheCreationInputTokens).toBe(0)
|
||||
const expected = 800 * GPT55.input + 200 * GPT55.cacheRead + 100 * GPT55.output
|
||||
expect(withWrite.costUSD).toBeCloseTo(expected, 12)
|
||||
expect(withWrite.costUSD).toBeCloseTo(withoutWrite.costUSD, 12)
|
||||
// The fabricated rate is 1.25 x 5e-6; make sure not a cent of it landed.
|
||||
expect(withWrite.costUSD).toBeLessThan(expected + 300 * GPT55.input * 1.25)
|
||||
})
|
||||
|
||||
it('clamps a cache-write count larger than the uncached input', async () => {
|
||||
const call = await parseOneEvent('gpt-5.6-terra', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
cache_write_input_tokens: 5000,
|
||||
output_tokens: 100,
|
||||
})
|
||||
|
||||
expect(call.inputTokens).toBe(0)
|
||||
expect(call.cacheCreationInputTokens).toBe(800)
|
||||
expect(call.costUSD).toBeCloseTo(800 * TERRA.cacheWrite + 200 * TERRA.cacheRead + 100 * TERRA.output, 12)
|
||||
})
|
||||
})
|
||||
|
||||
// ── Cache invalidation: a cost change must not be served from stale bytes ──
|
||||
|
||||
describe('#1075 cache invalidation', () => {
|
||||
it('discards a v10 codex results cache (it stores costUSD verbatim)', async () => {
|
||||
const cacheDir = join(tmpDir, 'cache')
|
||||
await mkdir(cacheDir, { recursive: true })
|
||||
const sessionFile = join(tmpDir, 'rollout-stale.jsonl')
|
||||
await writeFile(sessionFile, '{}\n')
|
||||
|
||||
const { statSync } = await import('fs')
|
||||
const s = statSync(sessionFile)
|
||||
const stale: ParsedProviderCall = {
|
||||
provider: 'codex',
|
||||
model: 'gpt-5.5',
|
||||
inputTokens: 800,
|
||||
outputTokens: 1000,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 200,
|
||||
cachedInputTokens: 200,
|
||||
reasoningTokens: 400,
|
||||
webSearchRequests: 0,
|
||||
costUSD: 0.0445, // the pre-fix, reasoning-double-counted number
|
||||
tools: [],
|
||||
bashCommands: [],
|
||||
timestamp: '2026-08-16T10:01:00Z',
|
||||
speed: 'standard',
|
||||
deduplicationKey: 'codex:stale',
|
||||
}
|
||||
await writeFile(join(cacheDir, 'codex-results.json'), JSON.stringify({
|
||||
version: 10,
|
||||
files: { [sessionFile]: { dev: s.dev, ino: s.ino, mtimeMs: s.mtimeMs, sizeBytes: s.size, project: 'p', calls: [stale] } },
|
||||
}))
|
||||
|
||||
const prevCacheDir = process.env['CODEBURN_CACHE_DIR']
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheDir
|
||||
try {
|
||||
clearCodexMemCaches()
|
||||
// Revert CODEX_CACHE_VERSION to 10 and this returns the stale $0.0445 call.
|
||||
expect(await readCachedCodexResults(sessionFile)).toBeNull()
|
||||
} finally {
|
||||
if (prevCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']; else process.env['CODEBURN_CACHE_DIR'] = prevCacheDir
|
||||
}
|
||||
})
|
||||
|
||||
it('re-derives days finalized at daily-cache v20', async () => {
|
||||
const cacheRoot = join(tmpDir, 'daily')
|
||||
await mkdir(cacheRoot, { recursive: true })
|
||||
const prevCacheDir = process.env['CODEBURN_CACHE_DIR']
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
try {
|
||||
const date = toDateString(new Date(Date.now() - 7 * 24 * 60 * 60 * 1000))
|
||||
const yesterday = toDateString(new Date(Date.now() - 24 * 60 * 60 * 1000))
|
||||
const oldPath = join(cacheRoot, 'daily-cache.v20.json')
|
||||
const oldCache = {
|
||||
version: 20,
|
||||
savingsConfigHash: 'cfg',
|
||||
tzKey: currentTzKey(),
|
||||
lastComputedDate: yesterday,
|
||||
days: [codexDay(date, 99)],
|
||||
complete: true,
|
||||
watermarkTrusted: true,
|
||||
}
|
||||
await writeFile(oldPath, JSON.stringify(oldCache))
|
||||
|
||||
let parseCount = 0
|
||||
const hydrated = await ensureCacheHydrated(
|
||||
async () => { parseCount++; return [] },
|
||||
() => [codexDay(date, 2)],
|
||||
'cfg',
|
||||
() => true,
|
||||
)
|
||||
|
||||
// Drop MIN_SUPPORTED_VERSION back to 20 and the v20 day is trusted as-is,
|
||||
// so parseCount stays 0 and the day keeps its overstated $99.
|
||||
expect(parseCount).toBe(1)
|
||||
expect(hydrated.days.find(d => d.date === date)?.cost).toBe(2)
|
||||
expect(JSON.parse(await readFile(oldPath, 'utf8'))).toEqual(oldCache)
|
||||
} finally {
|
||||
if (prevCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']; else process.env['CODEBURN_CACHE_DIR'] = prevCacheDir
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
// ── fixtures ──────────────────────────────────────────────────────────────
|
||||
|
||||
function makeApiCall(provider: string, model: string, usage: Partial<TokenUsage>): ParsedApiCall {
|
||||
return {
|
||||
provider,
|
||||
model,
|
||||
usage: {
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 0,
|
||||
cachedInputTokens: 0,
|
||||
reasoningTokens: 0,
|
||||
webSearchRequests: 0,
|
||||
...usage,
|
||||
},
|
||||
costUSD: 0,
|
||||
tools: [],
|
||||
mcpTools: [],
|
||||
skills: [],
|
||||
hasAgentSpawn: false,
|
||||
hasPlanMode: false,
|
||||
speed: 'standard',
|
||||
timestamp: '2026-08-16T00:00:00.000Z',
|
||||
bashCommands: [],
|
||||
deduplicationKey: `${provider}-${model}`,
|
||||
}
|
||||
}
|
||||
|
||||
function makeProject(calls: ParsedApiCall[]): ProjectSummary {
|
||||
const turn: ClassifiedTurn = {
|
||||
userMessage: 't',
|
||||
assistantCalls: calls,
|
||||
timestamp: '2026-08-16T00:00:00.000Z',
|
||||
sessionId: 's1',
|
||||
category: 'feature' as TaskCategory,
|
||||
retries: 0,
|
||||
hasEdits: false,
|
||||
}
|
||||
const session: SessionSummary = {
|
||||
sessionId: 's1',
|
||||
project: 'p',
|
||||
firstTimestamp: '2026-08-16T00:00:00.000Z',
|
||||
lastTimestamp: '2026-08-16T00:00:00.000Z',
|
||||
totalCostUSD: 0,
|
||||
totalInputTokens: 0,
|
||||
totalOutputTokens: 0,
|
||||
totalCacheReadTokens: 0,
|
||||
totalCacheWriteTokens: 0,
|
||||
apiCalls: 0,
|
||||
turns: [turn],
|
||||
modelBreakdown: {},
|
||||
toolBreakdown: {},
|
||||
mcpBreakdown: {},
|
||||
bashBreakdown: {},
|
||||
categoryBreakdown: {} as SessionSummary['categoryBreakdown'],
|
||||
skillBreakdown: {},
|
||||
}
|
||||
return { project: 'p', projectPath: 'p', sessions: [session], totalCostUSD: 0, totalApiCalls: 0 }
|
||||
}
|
||||
|
||||
function codexDay(date: string, cost: number): DailyEntry {
|
||||
const tokens = { inputTokens: 100, outputTokens: 20, cacheReadTokens: 30, cacheWriteTokens: 0 }
|
||||
return {
|
||||
date,
|
||||
cost,
|
||||
savingsUSD: 0,
|
||||
calls: 1,
|
||||
sessions: 1,
|
||||
...tokens,
|
||||
editTurns: 0,
|
||||
oneShotTurns: 0,
|
||||
models: { 'GPT-5.5': { calls: 1, cost, savingsUSD: 0, ...tokens } },
|
||||
categories: {},
|
||||
providers: { codex: { calls: 1, cost, savingsUSD: 0, sessions: 1, ...tokens } },
|
||||
}
|
||||
}
|
||||
|
|
@ -21,8 +21,38 @@ describe('Codex throughput prototype', () => {
|
|||
|
||||
const points = await readCodexThroughput(path)
|
||||
expect(points).toHaveLength(2)
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 50, elapsedSeconds: 5, generatedTokensPerSecond: 10, activeDurationSeconds: 7, activeGeneratedTokensPerSecond: 21.428571428571427, toolWaitSeconds: 3, model: 'gpt-5.6-sol' })
|
||||
expect(renderCodexThroughput(points, path)).toContain('21.4 generated tokens/sec')
|
||||
// Reasoning is a subset of output_tokens (#1075/#1078), not additive: the
|
||||
// checkpoints report output 80/40 and reasoning 20/10, so the generated
|
||||
// numerator is output alone (80, then 40), matching the cost path.
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 40, elapsedSeconds: 5, generatedTokensPerSecond: 8, activeDurationSeconds: 7, activeGeneratedTokensPerSecond: (80 + 40) / 7, toolWaitSeconds: 3, model: 'gpt-5.6-sol' })
|
||||
expect(renderCodexThroughput(points, path)).toContain('17.1 generated tokens/sec')
|
||||
})
|
||||
|
||||
it('REGRESSION (#1079): does not add reasoning tokens on top of output for Tok/s', async () => {
|
||||
// Reasoning tokens are a SUBSET of output_tokens (#1075/#1078), not a
|
||||
// separate bucket. A single checkpoint reporting output=60, reasoning=40
|
||||
// must drive Tok/s off 60, not 100 -- summing them would double-count 40
|
||||
// tokens that are already inside the 60. If this ever reverts to
|
||||
// `outputTokens + reasoningTokens`, activeGeneratedTokensPerSecond becomes
|
||||
// 10 (100 tokens / 10s) instead of 6 (60 tokens / 10s).
|
||||
const dir = await mkdtemp(join(tmpdir(), 'codeburn-tps-regression-'))
|
||||
const path = join(dir, 'rollout.jsonl')
|
||||
await writeFile(path, [
|
||||
JSON.stringify({ type: 'session_meta', timestamp: '2026-07-25T00:00:00.000Z', payload: { model: 'gpt-5.5' } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-07-25T00:00:00.000Z', payload: { type: 'task_started' } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-07-25T00:00:10.000Z', payload: { type: 'token_count', info: { last_token_usage: { output_tokens: 60, reasoning_output_tokens: 40 }, total_token_usage: { total_tokens: 100, output_tokens: 60, reasoning_output_tokens: 40 } } } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-07-25T00:00:10.000Z', payload: { type: 'task_complete', duration_ms: 10000 } }),
|
||||
].join('\n'))
|
||||
|
||||
const points = await readCodexThroughput(path)
|
||||
expect(points).toHaveLength(1)
|
||||
expect(points[0]).toMatchObject({
|
||||
outputTokens: 60,
|
||||
reasoningTokens: 40,
|
||||
generatedTokens: 60,
|
||||
taskGeneratedTokens: 60,
|
||||
activeGeneratedTokensPerSecond: 6,
|
||||
})
|
||||
})
|
||||
|
||||
it('parses only appended complete lines while watching a growing rollout', async () => {
|
||||
|
|
@ -36,7 +66,9 @@ describe('Codex throughput prototype', () => {
|
|||
await appendFile(path, first.slice(40) + '\n' + second + '\n')
|
||||
const points = await reader.update(path)
|
||||
expect(points).toHaveLength(2)
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 5, generatedTokensPerSecond: 5 })
|
||||
// second checkpoint: output 4 + reasoning 1 -> billable numerator is 4
|
||||
// (reasoning already inside output_tokens), not the additive 5.
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 4, generatedTokensPerSecond: 4 })
|
||||
})
|
||||
|
||||
it('ignores replayed pre-fork checkpoints before estimating new work', async () => {
|
||||
|
|
|
|||
|
|
@ -463,18 +463,19 @@ describe('buildPeriodDataFromDays', () => {
|
|||
})
|
||||
|
||||
describe('daily-cache ↔ report daily-bucket parity', () => {
|
||||
// The daily cache (history.daily + provider breakdown) and the live report /
|
||||
// headline (main.ts daily rollup) must bucket days by the SAME rule, or their
|
||||
// per-day totals drift and their period sums diverge from current.cost at
|
||||
// window boundaries — the V1 audit's constant -$3.45/-81-calls finding. Both
|
||||
// are now PER-CALL for cost/savings/calls (issue #852) with turn-level stats
|
||||
// still turn-anchored: this asserts per-day equality against a reference
|
||||
// that mirrors main.ts buildJsonReport's dailyMap fallback (each call on its
|
||||
// own date), plus the invariant history.daily Σ == report.daily Σ == total
|
||||
// call cost.
|
||||
// The daily cache (history.daily + provider breakdown) and JSON-report
|
||||
// daily[] rows (durable.days from buildDurablePeriod) must bucket days by the
|
||||
// SAME rule, or their per-day totals drift and their period sums diverge from
|
||||
// current.cost at window boundaries — the V1 audit's constant -$3.45/-81-calls
|
||||
// finding. Both are now PER-CALL for cost/savings/calls (issue #852) with
|
||||
// turn-level stats still turn-anchored: this asserts per-day equality against
|
||||
// an independent per-call oracle for the durable day aggregation used by
|
||||
// durable.days (each call on its own date), plus the invariant
|
||||
// history.daily Σ == report.daily Σ == total call cost.
|
||||
|
||||
// Mirrors the live report/headline daily rollup fallback in src/main.ts
|
||||
// (cost/savings/calls bucket under each call's own date).
|
||||
// Independent per-call reference for durable.days (cost/savings/calls bucket
|
||||
// under each call's own date). Not a live buildJsonReport fallback — that
|
||||
// path was deleted in #1067.
|
||||
function reportDailyByDate(projects: ProjectSummary[]): Record<string, number> {
|
||||
const byDate: Record<string, number> = {}
|
||||
for (const p of projects) {
|
||||
|
|
|
|||
29
tests/env-isolation-declarations.test.ts
Normal file
29
tests/env-isolation-declarations.test.ts
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
// Static guard: every PROVIDER_ENV_VARS entry must be CLEARED or REDIRECTED
|
||||
// by tests/setup/env-isolation.ts. A data-dir override that is fingerprinted
|
||||
// for cache invalidation but not isolated in tests leaks the developer's real
|
||||
// sessions into fixture parses — green on CI (no HERMES_HOME), red on a
|
||||
// Hermes-shell laptop. The named hole was HERMES_HOME; the class is every
|
||||
// sibling override that session-cache already knows about.
|
||||
//
|
||||
// The lists are imported from the same module applyIsolation() uses. Extra
|
||||
// High #1064: scraping setup-file source treated a comment containing
|
||||
// `'HERMES_HOME'` as isolation (false-green). Runtime membership cannot.
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import { PROVIDER_ENV_VARS } from '../src/session-cache.js'
|
||||
import { CLEARED, REDIRECTED } from './setup/env-isolation-vars.js'
|
||||
|
||||
describe('env-isolation covers PROVIDER_ENV_VARS', () => {
|
||||
it('clears or redirects every provider data-dir override so a developer shell cannot leak real sessions into fixtures', () => {
|
||||
const isolated = new Set<string>([...CLEARED, ...REDIRECTED])
|
||||
|
||||
const missing: string[] = []
|
||||
for (const [provider, vars] of Object.entries(PROVIDER_ENV_VARS)) {
|
||||
for (const varName of vars) {
|
||||
if (!isolated.has(varName)) missing.push(`${provider}:${varName}`)
|
||||
}
|
||||
}
|
||||
|
||||
expect(missing).toEqual([])
|
||||
})
|
||||
})
|
||||
|
|
@ -120,7 +120,7 @@ describe('granular history', () => {
|
|||
])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries.map(series => series.label)).toEqual([
|
||||
'sessio…3456 (claude) · Refactor billing module',
|
||||
'Refactor billing module (claude)',
|
||||
'sessio…3457 (claude) · repos/demo',
|
||||
'sessio…3458 (claude) · repos/demo',
|
||||
'sessio…3459 (claude) · repos/demo',
|
||||
|
|
@ -157,7 +157,7 @@ describe('granular history', () => {
|
|||
// 160px at the chart's 10px font fits roughly 31-32 lowercase glyphs;
|
||||
// compare a conservative prefix that must be visible in that budget.
|
||||
const visibleCharacterBudget = 24
|
||||
const visiblePrefixes = history.sessionSeries.map(series => series.label.slice(0, visibleCharacterBudget))
|
||||
const visiblePrefixes = history.sessionSeries.map(series => Array.from(series.label).slice(0, visibleCharacterBudget).join(''))
|
||||
expect(new Set(visiblePrefixes).size).toBe(2)
|
||||
expect(history.sessionSeries.map(series => series.label)).toEqual(expect.arrayContaining([
|
||||
expect.stringMatching(/^a1b2c3…7f01 \(claude\) · /),
|
||||
|
|
@ -165,6 +165,25 @@ describe('granular history', () => {
|
|||
]))
|
||||
})
|
||||
|
||||
it('counts the visible title-lead window in code points, not UTF-16 units', () => {
|
||||
const timestamp = '2026-07-15T12:05:00.000Z'
|
||||
const start = new Date('2026-07-15T00:00:00.000Z')
|
||||
const end = new Date('2026-07-15T23:59:59.999Z')
|
||||
const emojiPrefix = '😀'.repeat(12)
|
||||
const history = buildGranularHistory([project([
|
||||
{ id: 'emoji-alpha-aaaaaa', title: `${emojiPrefix} alpha work`, calls: [apiCall({ timestamp, cost: 1 })] },
|
||||
{ id: 'emoji-beta-bbbbbb', title: `${emojiPrefix} beta work`, calls: [apiCall({ timestamp, cost: 2 })] },
|
||||
])], { start, end }, end)
|
||||
|
||||
// 12 emoji = 12 glyphs / 24 UTF-16 units. A unit slice collides both
|
||||
// titles on the emoji run and would id-first; a code-point window still
|
||||
// sees " alpha" vs " beta" and can title-lead.
|
||||
expect(history.sessionSeries.map(series => series.label).sort()).toEqual([
|
||||
`${emojiPrefix} alpha work (claude)`,
|
||||
`${emojiPrefix} beta work (claude)`,
|
||||
].sort())
|
||||
})
|
||||
|
||||
it('sanitises control characters and ANSI escapes in session titles', () => {
|
||||
const timestamp = '2026-07-15T12:05:00.000Z'
|
||||
const start = new Date('2026-07-15T00:00:00.000Z')
|
||||
|
|
@ -175,7 +194,7 @@ describe('granular history', () => {
|
|||
calls: [apiCall({ timestamp, cost: 1 })],
|
||||
}])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries[0]?.label).toBe('sessio…3456 (claude) · Refactor billing module')
|
||||
expect(history.sessionSeries[0]?.label).toBe('Refactor billing module (claude)')
|
||||
expect(history.sessionSeries[0]?.label).not.toContain('\x1b')
|
||||
expect(history.sessionSeries[0]?.label).not.toContain('\x00')
|
||||
})
|
||||
|
|
@ -190,7 +209,7 @@ describe('granular history', () => {
|
|||
calls: [apiCall({ timestamp, cost: 1 })],
|
||||
}])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries[0]?.label).toBe('sessio…3456 (claude) · ' + 'x'.repeat(80))
|
||||
expect(history.sessionSeries[0]?.label).toBe('x'.repeat(80) + ' (claude)')
|
||||
})
|
||||
|
||||
it('caps session titles by code point without splitting an emoji', () => {
|
||||
|
|
@ -205,9 +224,8 @@ describe('granular history', () => {
|
|||
}])], { start, end }, end)
|
||||
|
||||
const label = history.sessionSeries[0]?.label ?? ''
|
||||
const titlePart = label.slice(label.indexOf(' · ') + 3)
|
||||
expect(titlePart).toBe('x'.repeat(79) + '😀')
|
||||
expect([...titlePart]).toEqual([...('x'.repeat(79) + '😀')])
|
||||
expect(label).toBe('x'.repeat(79) + '😀' + ' (claude)')
|
||||
expect([...label]).toEqual([...'x'.repeat(79), '😀', ' ', '(', 'c', 'l', 'a', 'u', 'd', 'e', ')'])
|
||||
})
|
||||
|
||||
it('prefers a title from any duplicate session summary sharing a key', () => {
|
||||
|
|
@ -230,7 +248,7 @@ describe('granular history', () => {
|
|||
])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries).toHaveLength(1)
|
||||
expect(history.sessionSeries[0]?.label).toBe('sessio…3456 (claude) · Z recovered session title')
|
||||
expect(history.sessionSeries[0]?.label).toBe('Z recovered session title (claude)')
|
||||
})
|
||||
|
||||
it('fills idle buckets and keeps separate model and session lines from real call timestamps', () => {
|
||||
|
|
|
|||
|
|
@ -769,6 +769,29 @@ describe('detectMcpProfileAdvisor', () => {
|
|||
}
|
||||
})
|
||||
|
||||
it('scopes the remediation label to --provider codex', () => {
|
||||
const hotTurns = [makeTurn([
|
||||
makeCall({ tools: ['mcp__github__t0'], cacheCreation: 10_000 }),
|
||||
makeCall({ tools: ['mcp__github__t1'], cacheCreation: 10_000 }),
|
||||
])]
|
||||
const coldTurns = [makeTurn([makeCall({ cacheCreation: 10_000 })])]
|
||||
const projects = [
|
||||
projectNamed('api', [
|
||||
makeSession({ inventory: smallInventory, turns: hotTurns, mcpBreakdown: { github: { calls: 2 } } }),
|
||||
]),
|
||||
projectNamed('web', [
|
||||
makeSession({ inventory: smallInventory, turns: coldTurns, mcpBreakdown: { github: { calls: 0 } } }),
|
||||
]),
|
||||
projectNamed('docs', [
|
||||
makeSession({ inventory: smallInventory, turns: coldTurns, mcpBreakdown: { github: { calls: 0 } } }),
|
||||
]),
|
||||
]
|
||||
const finding = detectMcpProfileAdvisor(projects, undefined, 'codex')
|
||||
expect(finding!.fix.label).toBe('Ask Codex to turn this into a project-scoped MCP profile:')
|
||||
expect(finding!.fix.label).not.toContain('Claude')
|
||||
expect(finding!.fix.label).not.toContain('CLAUDE.md')
|
||||
})
|
||||
|
||||
it('does not flag servers used evenly across loaded projects', () => {
|
||||
const projects = ['api', 'web', 'docs'].map(name => projectNamed(name, [
|
||||
makeSession({
|
||||
|
|
|
|||
|
|
@ -25,6 +25,14 @@ describe('tables', () => {
|
|||
expect(t).toContain('Opus 4.8')
|
||||
expect(t).toContain('| Model | Cost | Calls |')
|
||||
})
|
||||
it('unpriced warning names the flat-rate hatch instead of only alias', () => {
|
||||
const p = payload()
|
||||
p.current.unpricedModels = [{ model: 'zz-mystery-paid-model-999', calls: 3, tokens: 1200 }]
|
||||
const t = renderSummaryTable(p)
|
||||
expect(t).toContain('zz-mystery-paid-model-999')
|
||||
expect(t).toContain('model-flat-rate')
|
||||
expect(t).not.toContain('fix with `codeburn model-alias`')
|
||||
})
|
||||
it('breakdown by provider lists providers', () => {
|
||||
expect(renderBreakdownTable(payload(), 'provider', 20)).toContain('claude code')
|
||||
})
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import chalk from 'chalk'
|
|||
import stripAnsi from 'strip-ansi'
|
||||
|
||||
import { aggregateModels, renderTable, renderMarkdown, renderJson, renderCsv, type ModelReportRow } from '../src/models-report.js'
|
||||
import { setModelAliases } from '../src/models.js'
|
||||
import type {
|
||||
ProjectSummary,
|
||||
SessionSummary,
|
||||
|
|
@ -38,6 +39,8 @@ function makeCall(opts: {
|
|||
reasoning?: number
|
||||
cacheWrite?: number
|
||||
cacheRead?: number
|
||||
savingsUSD?: number
|
||||
savingsBaselineModel?: string
|
||||
}): ParsedApiCall {
|
||||
return {
|
||||
provider: opts.provider,
|
||||
|
|
@ -60,6 +63,8 @@ function makeCall(opts: {
|
|||
timestamp: '2026-05-09T00:00:00.000Z',
|
||||
bashCommands: [],
|
||||
deduplicationKey: `${opts.provider}-${opts.model}-${opts.costUSD}`,
|
||||
savingsUSD: opts.savingsUSD,
|
||||
savingsBaselineModel: opts.savingsBaselineModel,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -167,6 +172,32 @@ describe('aggregateModels', () => {
|
|||
expect(byKey['codex:gpt-5.5']!.credits).toBeCloseTo(887.5, 6)
|
||||
expect(byKey['codex:gpt-5']!.credits).toBeNull()
|
||||
expect(byKey['claude:claude-sonnet-4-6']!.credits).toBeNull()
|
||||
expect(byKey['codex:gpt-5.5']!.creditsIncomplete).toBeFalsy()
|
||||
})
|
||||
|
||||
it('partial-sums Codex credits when a merge mixes rated and unrated ids', async () => {
|
||||
setModelAliases({ 'codex-house-sku': 'gpt-5.5' })
|
||||
try {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [
|
||||
makeCall({ provider: 'codex', model: 'gpt-5.5', input: 0, output: 1_000_000, costUSD: 9 }),
|
||||
]),
|
||||
makeTurn('feature', [
|
||||
makeCall({ provider: 'codex', model: 'codex-house-sku', input: 0, output: 1_000_000, costUSD: 9 }),
|
||||
]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.model).toBe('gpt-5.5')
|
||||
// gpt-5.5 output is 750 credits/M; the aliased house SKU has no rate.
|
||||
expect(rows[0]!.credits).toBeCloseTo(750, 6)
|
||||
expect(rows[0]!.creditsIncomplete).toBe(true)
|
||||
expect(rows[0]!.calls).toBe(2)
|
||||
const parsed = JSON.parse(renderJson(rows))
|
||||
expect(parsed[0].credits).toBeCloseTo(750, 6)
|
||||
expect(parsed[0].creditsIncomplete).toBe(true)
|
||||
} finally {
|
||||
setModelAliases({})
|
||||
}
|
||||
})
|
||||
|
||||
it('includes credits in the JSON output', async () => {
|
||||
|
|
@ -190,6 +221,110 @@ describe('aggregateModels', () => {
|
|||
expect(rows[0]!.cacheReadTokens).toBe(4000) // not 8000
|
||||
})
|
||||
|
||||
it('falls back from a provider local table miss to the global short name', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'cursor-agent', model: 'gpt-5.6-sol', costUSD: 2.5 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.modelDisplayName).toBe('GPT-5.6 Sol (est.)')
|
||||
expect(rows[0]!.model).toBe('gpt-5.6-sol')
|
||||
})
|
||||
|
||||
it('resolves Fireworks path-form ids through the global table', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'cline', model: 'accounts/fireworks/models/kimi-k2p6', costUSD: 1 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.modelDisplayName).toBe('Kimi K2.6')
|
||||
expect(rows[0]!.model).toBe('accounts/fireworks/models/kimi-k2p6')
|
||||
})
|
||||
|
||||
it('merges two raw ids that resolve to the same alias-resolved canonical id', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('testing', [makeCall({
|
||||
provider: 'cline-cli', model: 'accounts/fireworks/models/glm-5p2',
|
||||
input: 800, output: 67, costUSD: 0.246,
|
||||
})]),
|
||||
makeTurn('conversation', [makeCall({
|
||||
provider: 'cline-cli', model: 'glm-5p2',
|
||||
input: 15, output: 1, costUSD: 0.019,
|
||||
})]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.provider).toBe('cline-cli')
|
||||
expect(rows[0]!.modelDisplayName).toBe('GLM-5.2')
|
||||
// First-seen raw id, not the lexicographically-smallest of the merge.
|
||||
expect(rows[0]!.model).toBe('accounts/fireworks/models/glm-5p2')
|
||||
expect(rows[0]!.inputTokens).toBe(815)
|
||||
expect(rows[0]!.outputTokens).toBe(68)
|
||||
expect(rows[0]!.costUSD).toBeCloseTo(0.265, 6)
|
||||
expect(rows[0]!.calls).toBe(2)
|
||||
expect(rows[0]!.topCategory).toBe('testing')
|
||||
expect(rows[0]!.topCategoryShare).toBeCloseTo(0.246 / 0.265, 3)
|
||||
})
|
||||
|
||||
it('does not pick the lexicographically-smallest raw id after a merge', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'kimi-k3', costUSD: 2 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'k3', costUSD: 1 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.model).toBe('kimi-k3')
|
||||
expect(rows[0]!.modelDisplayName).toBe('Kimi K3')
|
||||
expect(rows[0]!.calls).toBe(2)
|
||||
})
|
||||
|
||||
it('does not merge same-provider ids that only share a display name', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'gpt-5', costUSD: 3 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'gpt-5-untracked-xyz', costUSD: 1 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'cline-cli', model: 'glm-5p2', costUSD: 1 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'cline-cli', model: 'GLM-5.2', costUSD: 2 })]),
|
||||
])])
|
||||
const codex = rows.filter(r => r.provider === 'codex')
|
||||
expect(codex).toHaveLength(2)
|
||||
expect(codex.every(r => r.modelDisplayName === 'GPT-5')).toBe(true)
|
||||
expect(new Set(codex.map(r => r.model))).toEqual(new Set(['gpt-5', 'gpt-5-untracked-xyz']))
|
||||
|
||||
const cline = rows.filter(r => r.provider === 'cline-cli')
|
||||
expect(cline).toHaveLength(2)
|
||||
expect(cline.every(r => r.modelDisplayName === 'GLM-5.2')).toBe(true)
|
||||
expect(new Set(cline.map(r => r.model))).toEqual(new Set(['glm-5p2', 'GLM-5.2']))
|
||||
})
|
||||
|
||||
it('clears a merged savings baseline when three raw ids disagree', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({
|
||||
provider: 'codex', model: 'k3',
|
||||
costUSD: 1, savingsUSD: 2, savingsBaselineModel: 'gpt-4o',
|
||||
})]),
|
||||
makeTurn('feature', [makeCall({
|
||||
provider: 'codex', model: 'kimi-k3',
|
||||
costUSD: 1, savingsUSD: 2, savingsBaselineModel: 'claude-sonnet-4-6',
|
||||
})]),
|
||||
makeTurn('feature', [makeCall({
|
||||
provider: 'codex', model: 'k3-agent',
|
||||
costUSD: 1, savingsUSD: 2, savingsBaselineModel: 'gpt-5',
|
||||
})]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.provider).toBe('codex')
|
||||
expect(rows[0]!.modelDisplayName).toBe('Kimi K3')
|
||||
expect(rows[0]!.savingsUSD).toBe(6)
|
||||
expect(rows[0]!.savingsBaselineModel).toBe('')
|
||||
expect(rows[0]!.calls).toBe(3)
|
||||
})
|
||||
|
||||
it('does not merge the same display name across providers', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'cline-cli', model: 'glm-5p2', costUSD: 1 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'hermes', model: 'glm-5p2', costUSD: 2 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(2)
|
||||
expect(new Set(rows.map(r => r.provider))).toEqual(new Set(['cline-cli', 'hermes']))
|
||||
expect(rows.every(r => r.modelDisplayName === 'GLM-5.2')).toBe(true)
|
||||
})
|
||||
|
||||
it('reports the dominant task type with its cost share in default mode', async () => {
|
||||
const project = makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'claude', model: 'claude-sonnet-4-6', costUSD: 6.0, input: 100, output: 20 })]),
|
||||
|
|
@ -239,12 +374,15 @@ describe('aggregateModels', () => {
|
|||
expect(above.find(r => r.provider === 'cursor')).toBeUndefined()
|
||||
})
|
||||
|
||||
// Providers that report reasoning as a bucket SEPARATE from output still get
|
||||
// it added in. Codex and claude do not - they bill reasoning inside
|
||||
// output_tokens - and that carve-out is covered in codex-pricing-1075.test.ts.
|
||||
it('counts reasoning tokens as output tokens', async () => {
|
||||
const project = makeProject([
|
||||
makeTurn('feature', [
|
||||
{
|
||||
provider: 'codex',
|
||||
model: 'gpt-5',
|
||||
provider: 'gemini',
|
||||
model: 'gemini-2.5-pro',
|
||||
usage: { ...emptyTokens(), inputTokens: 100, outputTokens: 50, reasoningTokens: 200 },
|
||||
costUSD: 1.0,
|
||||
tools: [],
|
||||
|
|
|
|||
|
|
@ -7,17 +7,25 @@ import {
|
|||
findUnpricedModels,
|
||||
getModelCosts,
|
||||
getShortModelName,
|
||||
resolveCanonicalModelId,
|
||||
calculateCost,
|
||||
loadPricing,
|
||||
setModelAliases,
|
||||
setPriceOverrides,
|
||||
setLocalModelSavings,
|
||||
setFlatRateModels,
|
||||
setFlatRateRemoved,
|
||||
isExpectedFreeModel,
|
||||
isFlatRateModel,
|
||||
getLocalModelSavingsConfigHash,
|
||||
getPriceOverridesConfigHash,
|
||||
getModelAliasesConfigHash,
|
||||
getFlatRateModelsConfigHash,
|
||||
parseLiteLLMEntry,
|
||||
unpricedModelHint,
|
||||
} from '../src/models.js'
|
||||
import { getDailyCacheConfigHash } from '../src/usage-aggregator.js'
|
||||
import snapshotData from '../src/data/litellm-snapshot.json' with { type: 'json' }
|
||||
|
||||
beforeAll(async () => {
|
||||
await loadPricing()
|
||||
|
|
@ -27,6 +35,8 @@ afterEach(() => {
|
|||
setModelAliases({})
|
||||
setPriceOverrides({})
|
||||
setLocalModelSavings({})
|
||||
setFlatRateModels([])
|
||||
setFlatRateRemoved([])
|
||||
})
|
||||
|
||||
describe('getModelCosts', () => {
|
||||
|
|
@ -81,6 +91,32 @@ describe('getModelCosts', () => {
|
|||
expect(getModelCosts('z-ai/glm-5.3')!.inputCostPerToken).toBe(sibling!.inputCostPerToken)
|
||||
})
|
||||
|
||||
it('prices gpt-5.6-codex and gpt-5.6-codex-max, sourced directly from the snapshot (#1077)', () => {
|
||||
// Directly checks the bundled snapshot data (not just the resolved lookup),
|
||||
// so this fails if the litellm-snapshot.json entries are ever reverted even
|
||||
// though getModelCosts would still resolve both ids via the `gpt-5.6` prefix
|
||||
// fallback - explicit rows are still correct and match every other Codex
|
||||
// generation LiteLLM ships (gpt-5-codex, gpt-5.1-codex, gpt-5.1-codex-max,
|
||||
// gpt-5.2-codex, gpt-5.3-codex all carry their base model's exact rate).
|
||||
const snapshot = snapshotData as Record<string, unknown>
|
||||
expect(snapshot['gpt-5.6-codex']).toEqual(snapshot['gpt-5.6'])
|
||||
expect(snapshot['gpt-5.6-codex-max']).toEqual(snapshot['gpt-5.6'])
|
||||
|
||||
const codex = getModelCosts('gpt-5.6-codex')
|
||||
const codexMax = getModelCosts('gpt-5.6-codex-max')
|
||||
expect(codex).not.toBeNull()
|
||||
expect(codexMax).not.toBeNull()
|
||||
expect(codex!.inputCostPerToken).toBe(5e-6)
|
||||
expect(codex!.outputCostPerToken).toBe(3e-5)
|
||||
expect(codex!.cacheWriteCostPerToken).toBe(6.25e-6)
|
||||
expect(codex!.cacheReadCostPerToken).toBe(5e-7)
|
||||
expect(codex!.cacheWriteCostIsExplicit).toBe(true)
|
||||
expect(codexMax).toEqual(codex)
|
||||
|
||||
expect(calculateCost('gpt-5.6-codex', 1_000_000, 1_000_000, 0, 0, 0)).toBeGreaterThan(0)
|
||||
expect(calculateCost('gpt-5.6-codex-max', 1_000_000, 1_000_000, 0, 0, 0)).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
// A price override on a synthetic bare id can only be reached if the leading
|
||||
// segment was stripped, so these assert the namespace allowlist itself without
|
||||
// pinning to any real model's presence in (or absence from) the snapshot.
|
||||
|
|
@ -123,6 +159,22 @@ describe('getModelCosts', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('resolveCanonicalModelId', () => {
|
||||
it('aliases, peels path-form ids, and leaves display-only collisions distinct', () => {
|
||||
expect(resolveCanonicalModelId('k3')).toBe('kimi-k3')
|
||||
expect(resolveCanonicalModelId('k3-agent')).toBe('kimi-k3')
|
||||
expect(resolveCanonicalModelId('kimi-k3')).toBe('kimi-k3')
|
||||
expect(resolveCanonicalModelId('accounts/fireworks/models/glm-5p2')).toBe('glm-5p2')
|
||||
expect(resolveCanonicalModelId('glm-5p2')).toBe('glm-5p2')
|
||||
expect(resolveCanonicalModelId('GLM-5.2')).toBe('glm-5p1')
|
||||
expect(resolveCanonicalModelId('gpt-5-fast')).toBe('gpt-5')
|
||||
expect(resolveCanonicalModelId('gpt-5-untracked-xyz')).toBe('gpt-5-untracked-xyz')
|
||||
expect(resolveCanonicalModelId('claude-opus-4.6')).toBe('claude-opus-4-6')
|
||||
expect(resolveCanonicalModelId('kimi-code')).toBe('kimi-k2-thinking')
|
||||
expect(resolveCanonicalModelId('cline-pass/kimi-k3')).toBe('kimi-k3')
|
||||
})
|
||||
})
|
||||
|
||||
describe('getShortModelName', () => {
|
||||
it('maps gpt-4o-mini correctly (not gpt-4o)', () => {
|
||||
expect(getShortModelName('gpt-4o-mini-2024-07-18')).toBe('GPT-4o Mini')
|
||||
|
|
@ -486,6 +538,17 @@ describe('user price overrides', () => {
|
|||
expect(secondCombined).not.toBe(baseline)
|
||||
expect(secondCombined).not.toBe(firstCombined)
|
||||
})
|
||||
|
||||
it('includes flat-rate marks in the daily cache config hash', () => {
|
||||
setLocalModelSavings({})
|
||||
setPriceOverrides({})
|
||||
setFlatRateModels([])
|
||||
const baseline = getDailyCacheConfigHash()
|
||||
setFlatRateModels(['zz-flat-hash'])
|
||||
expect(getDailyCacheConfigHash()).not.toBe(baseline)
|
||||
setFlatRateModels([])
|
||||
expect(getDailyCacheConfigHash()).toBe(baseline)
|
||||
})
|
||||
})
|
||||
|
||||
describe('calculateCost - OMP names produce non-zero cost', () => {
|
||||
|
|
@ -609,6 +672,8 @@ describe('Cursor model variants resolve to pricing', () => {
|
|||
['claude-4.6-haiku', 'claude-haiku-4-5'],
|
||||
// Cursor auto proxy
|
||||
['cursor-auto', 'claude-sonnet-4-5'],
|
||||
// Codex activity surface (official rate card, observed raw id)
|
||||
['codex-auto-review', 'gpt-5.5'],
|
||||
// OpenAI variants Cursor emits
|
||||
['gpt-5', 'gpt-5'],
|
||||
['gpt-5-fast', 'gpt-5'],
|
||||
|
|
@ -644,6 +709,29 @@ describe('Cursor model variants resolve to pricing', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('Codex activity ids (#1047)', () => {
|
||||
it('keeps the activity label instead of collapsing to the underlying model name', () => {
|
||||
expect(getShortModelName('codex-auto-review')).toBe('Codex Auto Review')
|
||||
})
|
||||
|
||||
it('prices as the exact bundled GPT-5.5 object, not an invented rate', () => {
|
||||
expect(getModelCosts('codex-auto-review')).toBe(getModelCosts('gpt-5.5'))
|
||||
const auto = calculateCost('codex-auto-review', 1_000_000, 1_000_000, 0, 0, 0)
|
||||
const gpt55 = calculateCost('gpt-5.5', 1_000_000, 1_000_000, 0, 0, 0)
|
||||
expect(auto).toBeGreaterThan(0)
|
||||
expect(auto).toBe(gpt55)
|
||||
})
|
||||
|
||||
it('does not invent a family or an unobserved sibling id', () => {
|
||||
expect(getModelCosts('codex-code-review')).toBeNull()
|
||||
expect(getModelCosts('codex-cloud-task')).toBeNull()
|
||||
expect(getModelCosts('codex-automation')).toBeNull()
|
||||
expect(getModelCosts('code-review')).toBeNull()
|
||||
expect(getModelCosts('auto-review')).toBeNull()
|
||||
expect(calculateCost('codex-code-review', 1_000_000, 1_000_000, 0, 0, 0)).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('Cursor house model pricing', () => {
|
||||
const cases: Array<[string, { input: number; output: number; cacheWrite: number; cacheRead: number }]> = [
|
||||
['composer-2.5', { input: 0.5, output: 2.5, cacheWrite: 0.5, cacheRead: 0.2 }],
|
||||
|
|
@ -753,6 +841,7 @@ describe('DeepSeek v4 models resolve to pricing', () => {
|
|||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await mkdir(cacheRoot, { recursive: true })
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
version: 2, // must match models.ts's CACHE_SCHEMA_VERSION or the cache is treated as a miss
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'gpt-4o-mini': {
|
||||
|
|
@ -778,6 +867,42 @@ describe('DeepSeek v4 models resolve to pricing', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('pricing cache schema version (#1075/#1078 follow-up)', () => {
|
||||
it('discards a cache written by a pre-#1078 binary instead of reading its missing cacheWriteCostIsExplicit as false', async () => {
|
||||
const cacheRoot = await mkdtemp(join(tmpdir(), 'codeburn-pricing-cache-'))
|
||||
try {
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await mkdir(cacheRoot, { recursive: true })
|
||||
// Shape of a cache file written before #1078 added `version` and
|
||||
// `cacheWriteCostIsExplicit`: no version field, and entries missing the
|
||||
// key despite carrying a real (non-default) cache-write rate.
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'gpt-5.6': {
|
||||
inputCostPerToken: 5e-6,
|
||||
outputCostPerToken: 3e-5,
|
||||
cacheWriteCostPerToken: 6.25e-6,
|
||||
cacheReadCostPerToken: 5e-7,
|
||||
webSearchCostPerRequest: 0.01,
|
||||
fastMultiplier: 1,
|
||||
},
|
||||
},
|
||||
}), 'utf-8')
|
||||
|
||||
await loadPricing()
|
||||
|
||||
// Pre-fix, loadCachedPricing had no version check: it would read this
|
||||
// cache verbatim, and gpt-5.6's missing key would resolve to undefined
|
||||
// (falsy) here instead of the true its LiteLLM entry actually carries.
|
||||
expect(getModelCosts('gpt-5.6')!.cacheWriteCostIsExplicit).toBe(true)
|
||||
} finally {
|
||||
await rm(cacheRoot, { recursive: true, force: true })
|
||||
await loadPricing()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('provider pricing suffix variants', () => {
|
||||
const cases: Array<[string, string]> = [
|
||||
['GLM-4.7-TEE', 'glm-4.7'],
|
||||
|
|
@ -915,6 +1040,7 @@ describe('findUnpricedModels', () => {
|
|||
try {
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
version: 2, // must match models.ts's CACHE_SCHEMA_VERSION or the cache is treated as a miss
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'zz-zero-stub-model': {
|
||||
|
|
@ -980,6 +1106,71 @@ describe('findUnpricedModels', () => {
|
|||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
})
|
||||
|
||||
it('skips subscription / flat-rate product SKUs where $0 is correct', () => {
|
||||
const rows = [
|
||||
{ model: 'auto-genius', calls: 898, cost: 0, tokens: 35_300_000 },
|
||||
{ model: 'cline-pass/auto-genius', calls: 4, cost: 0, tokens: 33_900 },
|
||||
{ model: 'auto', calls: 449, cost: 0, tokens: 17_700_000 },
|
||||
{ model: 'kimi-for-coding-highspeed', calls: 12, cost: 0, tokens: 3_400_000 },
|
||||
{ model: 'moonshot/kimi-for-coding-highspeed', calls: 2, cost: 0, tokens: 80_000 },
|
||||
{ model: 'grok-composer-2.5-fast', calls: 10, cost: 0, tokens: 1_900_000 },
|
||||
{ model: 'Grok Composer 2.5 Fast', calls: 10, cost: 0, tokens: 1_900_000 },
|
||||
{ model: 'Warp Auto (efficient)', calls: 3, cost: 0, tokens: 50_000 },
|
||||
{ model: 'warp', calls: 449, cost: 0, tokens: 17_700_000 },
|
||||
{ model: 'codex-auto-review', calls: 940, cost: 0, tokens: 7_200_000 },
|
||||
{ model: 'Codex Auto Review', calls: 2, cost: 0, tokens: 100 },
|
||||
{ model: 'big-pickle', calls: 4, cost: 0, tokens: 33_900 },
|
||||
{ model: 'zz-mystery-paid-model-999', calls: 3, cost: 0, tokens: 1200 },
|
||||
]
|
||||
expect(findUnpricedModels(rows)).toEqual([
|
||||
{ model: 'warp', calls: 449, tokens: 17_700_000 },
|
||||
// Note: NOT 'codex-auto-review' — #1056 aliases it to gpt-5.5, so it
|
||||
// now resolves a billable rate and is filtered out here (a $0 row for
|
||||
// it is stale data, not evidence of missing pricing). It still left
|
||||
// the flat-rate list, verified separately in the "Codex activity ids
|
||||
// (#1047)" describe block below.
|
||||
{ model: 'big-pickle', calls: 4, tokens: 33_900 },
|
||||
{ model: 'zz-mystery-paid-model-999', calls: 3, tokens: 1200 },
|
||||
{ model: 'Codex Auto Review', calls: 2, tokens: 100 },
|
||||
])
|
||||
})
|
||||
|
||||
it('skips a user-declared flat-rate model, including path-prefixed siblings', () => {
|
||||
const model = 'zz-my-pass-codename'
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toHaveLength(1)
|
||||
setFlatRateModels([model])
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
expect(findUnpricedModels([{ model: `vendor/${model}`, calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
expect(findUnpricedModels([{ model: 'zz-other-unknown', calls: 1, cost: 0, tokens: 10 }])).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('does not treat a priced sibling as expected-free just because a family is flat-rate', () => {
|
||||
// warp-auto-* is a subscription SKU, but main already aliases it onto a
|
||||
// billable row. Coverage must still count those priced calls.
|
||||
expect(isFlatRateModel('warp-auto-efficient')).toBe(true)
|
||||
expect(getModelCosts('warp-auto-efficient')).not.toBeNull()
|
||||
expect(isExpectedFreeModel('warp-auto-efficient')).toBe(false)
|
||||
expect(isExpectedFreeModel('auto-genius')).toBe(true)
|
||||
expect(isExpectedFreeModel('auto')).toBe(true)
|
||||
expect(isExpectedFreeModel('kimi-for-coding-highspeed')).toBe(true)
|
||||
expect(isExpectedFreeModel('warp')).toBe(false)
|
||||
expect(isExpectedFreeModel('codex-auto-review')).toBe(false)
|
||||
expect(isExpectedFreeModel('zz-mystery-paid-model-999')).toBe(false)
|
||||
})
|
||||
|
||||
it('lets --remove opt out of a built-in so a false positive can warn again', () => {
|
||||
expect(findUnpricedModels([{ model: 'auto-genius', calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
setFlatRateRemoved(['auto-genius'])
|
||||
expect(isFlatRateModel('auto-genius')).toBe(false)
|
||||
expect(findUnpricedModels([{ model: 'auto-genius', calls: 1, cost: 0, tokens: 10 }])).toEqual([
|
||||
{ model: 'auto-genius', calls: 1, tokens: 10 },
|
||||
])
|
||||
expect(findUnpricedModels([{ model: 'cline-pass/auto-genius', calls: 1, cost: 0, tokens: 10 }])).toEqual([
|
||||
{ model: 'cline-pass/auto-genius', calls: 1, tokens: 10 },
|
||||
])
|
||||
expect(isFlatRateModel('auto')).toBe(true)
|
||||
})
|
||||
|
||||
it('sorts by tokens, then calls', () => {
|
||||
const unpriced = findUnpricedModels([
|
||||
{ model: 'zz-small', calls: 9, cost: 0, tokens: 10 },
|
||||
|
|
@ -1019,3 +1210,97 @@ describe('getModelAliasesConfigHash', () => {
|
|||
setModelAliases({})
|
||||
})
|
||||
})
|
||||
|
||||
describe('getFlatRateModelsConfigHash', () => {
|
||||
it('is empty for no marks, changes with content, ignores insertion order', () => {
|
||||
setFlatRateModels([])
|
||||
expect(getFlatRateModelsConfigHash()).toBe('')
|
||||
setFlatRateModels(['auto-genius'])
|
||||
const one = getFlatRateModelsConfigHash()
|
||||
expect(one).not.toBe('')
|
||||
setFlatRateModels(['warp', 'auto-genius'])
|
||||
const two = getFlatRateModelsConfigHash()
|
||||
expect(two).not.toBe(one)
|
||||
setFlatRateModels(['auto-genius', 'warp'])
|
||||
expect(getFlatRateModelsConfigHash()).toBe(two)
|
||||
setFlatRateModels([])
|
||||
})
|
||||
|
||||
it('changes when a built-in is opted out', () => {
|
||||
setFlatRateModels([])
|
||||
setFlatRateRemoved([])
|
||||
const baseline = getFlatRateModelsConfigHash()
|
||||
setFlatRateRemoved(['auto-genius'])
|
||||
expect(getFlatRateModelsConfigHash()).not.toBe(baseline)
|
||||
setFlatRateRemoved([])
|
||||
expect(getFlatRateModelsConfigHash()).toBe(baseline)
|
||||
})
|
||||
})
|
||||
|
||||
describe('pricing snapshot carries flat-rate marks', () => {
|
||||
it('restorePricingState reapplies user flat-rate marks', async () => {
|
||||
const { snapshotPricingState, restorePricingState } = await import('../src/models.js')
|
||||
setFlatRateModels(['zz-snapshot-flat'])
|
||||
const snap = snapshotPricingState()
|
||||
expect(snap.flatRateModels).toEqual(['zz-snapshot-flat'])
|
||||
setFlatRateModels([])
|
||||
expect(isFlatRateModel('zz-snapshot-flat')).toBe(false)
|
||||
restorePricingState(snap)
|
||||
expect(isFlatRateModel('zz-snapshot-flat')).toBe(true)
|
||||
setFlatRateModels([])
|
||||
})
|
||||
|
||||
it('restorePricingState reapplies built-in opt-outs', async () => {
|
||||
const { snapshotPricingState, restorePricingState } = await import('../src/models.js')
|
||||
setFlatRateRemoved(['auto-genius'])
|
||||
const snap = snapshotPricingState()
|
||||
expect(snap.flatRateModelsRemoved).toEqual(['auto-genius'])
|
||||
setFlatRateRemoved([])
|
||||
expect(isFlatRateModel('auto-genius')).toBe(true)
|
||||
restorePricingState(snap)
|
||||
expect(isFlatRateModel('auto-genius')).toBe(false)
|
||||
setFlatRateRemoved([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('unpricedModelHint', () => {
|
||||
it('never tells the user to alias unconditionally', () => {
|
||||
expect(unpricedModelHint()).toContain('If a model is billed per token')
|
||||
expect(unpricedModelHint()).toContain('model-flat-rate')
|
||||
expect(unpricedModelHint()).not.toContain('Fix: codeburn model-alias')
|
||||
})
|
||||
|
||||
it('names both hatches for a concrete unknown SKU', () => {
|
||||
const hint = unpricedModelHint('zz-new-subscription-pass-sku')
|
||||
expect(hint).toContain('codeburn model-alias "zz-new-subscription-pass-sku"')
|
||||
expect(hint).toContain('codeburn model-flat-rate "zz-new-subscription-pass-sku"')
|
||||
expect(hint).toContain('If a model is billed per token')
|
||||
expect(hint).toContain('If $0 is correct')
|
||||
})
|
||||
})
|
||||
|
||||
describe('calculateCost verbose unknown-model warning', () => {
|
||||
it('does not present model-alias as the only fix for an unknown SKU', () => {
|
||||
const previous = process.env['CODEBURN_VERBOSE']
|
||||
process.env['CODEBURN_VERBOSE'] = '1'
|
||||
const chunks: string[] = []
|
||||
const originalWrite = process.stderr.write.bind(process.stderr)
|
||||
process.stderr.write = ((chunk: string | Uint8Array, ...args: unknown[]) => {
|
||||
chunks.push(typeof chunk === 'string' ? chunk : Buffer.from(chunk).toString())
|
||||
return (originalWrite as (chunk: string | Uint8Array, ...rest: unknown[]) => boolean)(chunk, ...args)
|
||||
}) as typeof process.stderr.write
|
||||
try {
|
||||
expect(calculateCost('zz-new-subscription-pass-sku', 10, 10, 0, 0, 0)).toBe(0)
|
||||
} finally {
|
||||
process.stderr.write = originalWrite
|
||||
if (previous === undefined) delete process.env['CODEBURN_VERBOSE']
|
||||
else process.env['CODEBURN_VERBOSE'] = previous
|
||||
}
|
||||
const text = chunks.join('')
|
||||
expect(text).toContain('zz-new-subscription-pass-sku')
|
||||
expect(text).toContain('If a model is billed per token')
|
||||
expect(text).toContain('model-flat-rate')
|
||||
expect(text).toContain('model-alias')
|
||||
expect(text).not.toMatch(/Map it with: codeburn model-alias/)
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -28,6 +28,12 @@ import {
|
|||
buildOptimizeJsonReport,
|
||||
renderOptimize,
|
||||
findingBasis,
|
||||
optimizeRemediationCopy,
|
||||
optimizePasteHeader,
|
||||
optimizeTuiPasteHeader,
|
||||
optimizeEmptyScanLines,
|
||||
sessionOpenerLabel,
|
||||
askAgentLabel,
|
||||
type FindingId,
|
||||
type ToolCall,
|
||||
type ApiCallMeta,
|
||||
|
|
@ -1443,3 +1449,137 @@ describe('renderOptimize applied-fixes section', () => {
|
|||
expect(render([], [])).not.toContain('Applied fixes')
|
||||
})
|
||||
})
|
||||
|
||||
describe('provider-scoped remediation copy (#1044)', () => {
|
||||
const strip = (s: string): string => s.replace(/\u001b\[[0-9;]*m/g, '')
|
||||
|
||||
function promptFinding(label: string): WasteFinding {
|
||||
return {
|
||||
id: 'retry-heavy-capabilities',
|
||||
title: 'retry-heavy',
|
||||
explanation: 'why',
|
||||
impact: 'medium',
|
||||
tokensSaved: 1000,
|
||||
fix: { type: 'paste', destination: 'prompt', label, text: 'audit' },
|
||||
}
|
||||
}
|
||||
|
||||
function openerFinding(label: string): WasteFinding {
|
||||
return {
|
||||
id: 'low-worth-sessions',
|
||||
title: 'low-worth',
|
||||
explanation: 'why',
|
||||
impact: 'low',
|
||||
tokensSaved: 1000,
|
||||
fix: { type: 'paste', destination: 'session-opener', label, text: 'open' },
|
||||
}
|
||||
}
|
||||
|
||||
it('keeps the exact shipped Claude / CLAUDE.md strings', () => {
|
||||
for (const provider of [undefined, 'all', 'claude'] as const) {
|
||||
const copy = optimizeRemediationCopy(provider)
|
||||
expect(copy).toEqual({ agent: 'Claude', instructionFile: 'CLAUDE.md' })
|
||||
expect(optimizePasteHeader('prompt', copy)).toBe('Ask Claude in the current session')
|
||||
expect(optimizePasteHeader('session-opener', copy)).toBe('One-time session opener (do NOT add to CLAUDE.md)')
|
||||
expect(sessionOpenerLabel(copy)).toBe('Paste at the start of your NEXT expensive thread (one-time, do not add to CLAUDE.md):')
|
||||
expect(askAgentLabel(copy, 'audit the retry-heavy capability before changing config'))
|
||||
.toBe('Ask Claude to audit the retry-heavy capability before changing config:')
|
||||
expect(optimizeEmptyScanLines(provider)).toEqual([
|
||||
'CodeBurn optimize scans your Claude Code sessions and config for',
|
||||
'token waste: junk directory reads, duplicate file reads, unused',
|
||||
'agents/skills/MCP servers, bloated CLAUDE.md, and more.',
|
||||
])
|
||||
expect(optimizeTuiPasteHeader('session-opener', provider))
|
||||
.toBe('── One-time session opener (do not add to CLAUDE.md) '.padEnd(64, '─'))
|
||||
expect(optimizeTuiPasteHeader('prompt', provider))
|
||||
.toBe('── Ask Claude in the current session '.padEnd(64, '─'))
|
||||
}
|
||||
const empty = strip(renderOptimize([], 0, 'lifetime', 0, 0, 0, 100, 'A', [], []))
|
||||
expect(empty).toContain('CodeBurn optimize scans your Claude Code sessions and config for')
|
||||
expect(empty).toContain('bloated CLAUDE.md')
|
||||
expect(empty).not.toContain('Claude sessions')
|
||||
})
|
||||
|
||||
it('uses Codex / AGENTS.md for --provider codex', () => {
|
||||
const copy = optimizeRemediationCopy('codex')
|
||||
expect(copy).toEqual({ agent: 'Codex', instructionFile: 'AGENTS.md' })
|
||||
expect(optimizePasteHeader('prompt', copy)).toBe('Ask Codex in the current session')
|
||||
expect(optimizePasteHeader('session-opener', copy)).toBe('One-time session opener (do NOT add to AGENTS.md)')
|
||||
expect(sessionOpenerLabel(copy)).toContain('AGENTS.md')
|
||||
expect(sessionOpenerLabel(copy)).not.toContain('CLAUDE.md')
|
||||
expect(optimizeEmptyScanLines('codex')[0]).toBe('Session-scan detectors do not cover Codex yet.')
|
||||
expect(optimizeEmptyScanLines('codex')[2]).toContain('currently scan Claude Code only')
|
||||
expect(optimizeEmptyScanLines('codex').join(' ')).not.toContain('scans your Codex')
|
||||
expect(optimizeEmptyScanLines('codex').join(' ')).not.toContain('bloated AGENTS.md')
|
||||
expect(optimizeTuiPasteHeader('prompt', 'codex')).toContain('Ask Codex in the current session')
|
||||
expect(optimizeTuiPasteHeader('session-opener', 'codex')).toContain('do NOT add to AGENTS.md')
|
||||
expect(optimizeTuiPasteHeader('session-opener', 'codex')).not.toContain('CLAUDE.md')
|
||||
})
|
||||
|
||||
it('uses the canonical Provider.displayName, not a title-cased id', () => {
|
||||
expect(optimizeRemediationCopy('hermes').agent).toBe('Hermes Agent')
|
||||
expect(optimizeRemediationCopy('cursor').agent).toBe('Cursor')
|
||||
expect(optimizeRemediationCopy('cursor').instructionFile).toBe('project instructions')
|
||||
expect(optimizePasteHeader('session-opener', optimizeRemediationCopy('cursor')))
|
||||
.toBe('One-time session opener (do NOT add to project instructions)')
|
||||
})
|
||||
|
||||
it('scopes every cross-provider detector label, including JSON', () => {
|
||||
const lowWorth = detectLowWorthSessions([
|
||||
projectWithLowWorthSessions([lowWorthSession(4, 0, { turns: [lowWorthTurn({ hasEdits: false })] })]),
|
||||
], 'codex')
|
||||
const context = detectContextBloat([
|
||||
projectWithContextSessions([contextSession(0, {
|
||||
totalInputTokens: 90_000,
|
||||
totalCacheReadTokens: 30_000,
|
||||
totalOutputTokens: 2_000,
|
||||
})]),
|
||||
], undefined, 'codex')
|
||||
const outliers = detectSessionOutliers([projectWithSessions([1, 1, 1, 10])], undefined, 'codex')
|
||||
const turns = Array.from({ length: 5 }, (_, i) => reliabilityTurn(i, {
|
||||
retries: i < 3 ? 1 : 0,
|
||||
call: { tools: ['Edit', 'Skill'], skills: ['reviewer'] },
|
||||
}))
|
||||
const retry = detectCapabilityReliability([projectWithReliabilityTurns(turns)], 'codex')
|
||||
|
||||
const findings = [lowWorth, context, outliers, retry]
|
||||
expect(findings.every(Boolean)).toBe(true)
|
||||
for (const finding of findings) {
|
||||
expect(finding!.fix.label).not.toContain('Claude')
|
||||
expect(finding!.fix.label).not.toContain('CLAUDE.md')
|
||||
}
|
||||
expect(lowWorth!.fix.label).toContain('AGENTS.md')
|
||||
expect(context!.fix.label).toContain('AGENTS.md')
|
||||
expect(outliers!.fix.label).toContain('AGENTS.md')
|
||||
expect(retry!.fix.label).toBe('Ask Codex to audit the retry-heavy capability before changing config:')
|
||||
|
||||
const json = buildOptimizeJsonReport(
|
||||
[projectWithSessions([1])],
|
||||
'lifetime',
|
||||
{ findings: findings as WasteFinding[], costRate: 0.00001, healthScore: 80, healthGrade: 'B', modelRecommendations: [] },
|
||||
)
|
||||
expect(json.findings).toHaveLength(4)
|
||||
for (const row of json.findings) {
|
||||
expect(row.fix.label).not.toContain('Claude')
|
||||
expect(row.fix.label).not.toContain('CLAUDE.md')
|
||||
}
|
||||
})
|
||||
|
||||
it('renders destination headers from the selected provider, not the finding text', () => {
|
||||
const out = strip(renderOptimize(
|
||||
[promptFinding('Ask Claude to audit the retry-heavy capability before changing config:')],
|
||||
0.00001, 'lifetime', 10, 5, 100, 80, 'B', [], [],
|
||||
undefined, undefined, undefined, [], 'codex',
|
||||
))
|
||||
expect(out).toContain('Ask Codex in the current session')
|
||||
expect(out).not.toContain('Ask Claude in the current session')
|
||||
|
||||
const opener = strip(renderOptimize(
|
||||
[openerFinding('Paste at the start of your NEXT expensive thread (one-time, do not add to CLAUDE.md):')],
|
||||
0.00001, 'lifetime', 10, 5, 100, 80, 'B', [], [],
|
||||
undefined, undefined, undefined, [], 'codex',
|
||||
))
|
||||
expect(opener).toContain('do NOT add to AGENTS.md')
|
||||
expect(opener).not.toContain('do NOT add to CLAUDE.md')
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -240,6 +240,23 @@ describe('renderOverview unpriced models', () => {
|
|||
expect(out).toContain('1 model at $0')
|
||||
expect(out).toContain('zz-mystery-paid-model-999')
|
||||
expect(out).toContain('codeburn model-alias')
|
||||
expect(out).toContain('model-flat-rate')
|
||||
expect(out).not.toContain('Fix: codeburn model-alias')
|
||||
})
|
||||
|
||||
it('stays silent for subscription SKUs whose $0 is correct', () => {
|
||||
const out = renderOverview([makeProject({
|
||||
project: 'pass',
|
||||
projectPath: '/Users/test/pass',
|
||||
cost: 0,
|
||||
calls: 4,
|
||||
model: 'auto-genius',
|
||||
provider: 'cline-cli',
|
||||
tokens: { input: 1000, output: 200, cacheR: 0, cacheW: 0 },
|
||||
})], { label: 'June 2026', color: false })
|
||||
|
||||
expect(out).not.toContain('Unpriced')
|
||||
expect(out).not.toContain('model-alias')
|
||||
})
|
||||
|
||||
it('stays silent when every model is priced', () => {
|
||||
|
|
|
|||
|
|
@ -236,7 +236,8 @@ async function shardBodies(cacheDir: string): Promise<Record<string, string>> {
|
|||
/// The Codex incremental cache is a single JSON file; both runs read the same
|
||||
/// rollouts, so it must come out identical byte for byte.
|
||||
async function codexResults(cacheDir: string): Promise<string | null> {
|
||||
return readFile(join(cacheDir, 'codex-results.json'), 'utf-8').catch(() => null)
|
||||
const { codexCacheFileName } = await import('../src/codex-cache.js')
|
||||
return readFile(join(cacheDir, codexCacheFileName()), 'utf-8').catch(() => null)
|
||||
}
|
||||
|
||||
function runCli(args: string[], home: string, extraEnv: Record<string, string>) {
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ import { calculateCost } from '../src/models.js'
|
|||
import { aggregateProjectsIntoDays } from '../src/day-aggregator.js'
|
||||
import { DAILY_CACHE_VERSION, currentTzKey, ensureCacheHydrated, saveDailyCache } from '../src/daily-cache.js'
|
||||
import { clearSessionCache, isSessionHydrationComplete, parseAllSessions, setParseReuseValidator } from '../src/parser.js'
|
||||
import { CACHE_VERSION, clearLoadCacheMemo, computeEnvFingerprint, loadCache, saveCache } from '../src/session-cache.js'
|
||||
import { CACHE_VERSION, clearLoadCacheMemo, computeEnvFingerprint, loadCache, PROVIDER_PARSE_VERSIONS, saveCache } from '../src/session-cache.js'
|
||||
import { cacheDirSnapshot, readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
|
||||
import type { SessionSource, SessionParser, ParsedProviderCall } from '../src/providers/types.js'
|
||||
|
||||
|
|
|
|||
12
tests/provider-display-name.test.ts
Normal file
12
tests/provider-display-name.test.ts
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
import { describe, expect, it } from 'vitest'
|
||||
import { getAllProviders, providerDisplayName } from '../src/providers/index.js'
|
||||
|
||||
describe('providerDisplayName', () => {
|
||||
it('matches every loaded Provider.displayName', async () => {
|
||||
const loaded = await getAllProviders()
|
||||
expect(loaded.length).toBeGreaterThan(20)
|
||||
for (const provider of loaded) {
|
||||
expect(providerDisplayName(provider.name)).toBe(provider.displayName)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
|
@ -183,7 +183,8 @@ describe('codex incremental resume', () => {
|
|||
sessionPath = await writeRollout([meta(), ...tasks(1, 2)])
|
||||
await parse(cacheDir)
|
||||
|
||||
const cachePath = join(cacheDir, 'codex-results.json')
|
||||
const { codexCacheFileName } = await import('../../src/codex-cache.js')
|
||||
const cachePath = join(cacheDir, codexCacheFileName())
|
||||
const { readFile } = await import('fs/promises')
|
||||
const raw = JSON.parse(await readFile(cachePath, 'utf-8'))
|
||||
raw.files[sessionPath].resumeState = { garbage: true }
|
||||
|
|
|
|||
|
|
@ -1,9 +1,11 @@
|
|||
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'fs/promises'
|
||||
import { mkdtemp, mkdir, writeFile, rm, stat } from 'fs/promises'
|
||||
import { join } from 'path'
|
||||
import { tmpdir } from 'os'
|
||||
|
||||
import { createCodexProvider } from '../../src/providers/codex.js'
|
||||
import { clearCodexMemCaches, CODEX_CACHE_VERSION, codexCacheFileName } from '../../src/codex-cache.js'
|
||||
import { calculateCost } from '../../src/models.js'
|
||||
import type { ParsedProviderCall } from '../../src/providers/types.js'
|
||||
|
||||
let tmpDir: string
|
||||
|
|
@ -605,6 +607,47 @@ describe('codex provider - JSONL parsing', () => {
|
|||
expect(calls.map(call => call.model)).toEqual(['gpt-5.6-luna', 'gpt-5.6-luna'])
|
||||
})
|
||||
|
||||
it('reads session_meta cwd/session_id/originator at payload depth 1, not the first nested same-name key', async () => {
|
||||
const largeSessionMeta = JSON.stringify({
|
||||
type: 'session_meta',
|
||||
timestamp: '2026-04-14T10:00:00Z',
|
||||
payload: {
|
||||
dynamic_tools: [{
|
||||
name: 'shadow-tool',
|
||||
cwd: '/shadow/cwd',
|
||||
originator: 'shadow-originator',
|
||||
session_id: 'shadow-session',
|
||||
forked_from_id: 'shadow-fork',
|
||||
model_provider: 'shadow-provider',
|
||||
}],
|
||||
base_instructions: { text: 'x'.repeat(40_000) },
|
||||
cwd: '/Users/test/real-project',
|
||||
originator: 'codex-cli',
|
||||
session_id: 'sess-real',
|
||||
model: 'gpt-5.6-luna',
|
||||
model_provider: 'openai',
|
||||
name: 'real-session-name',
|
||||
},
|
||||
})
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-nested-keys.jsonl', [
|
||||
largeSessionMeta,
|
||||
functionCall('exec_command'),
|
||||
tokenCount({ timestamp: '2026-04-14T10:01:00Z', last: { input: 100, output: 50 }, total: { total: 150 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]!.sessionId).toBe('sess-real')
|
||||
expect(calls[0]!.workingDirectory).toBe('/Users/test/real-project')
|
||||
expect(calls[0]!.projectPath).toBe('/Users/test/real-project')
|
||||
expect(calls[0]!.model).toBe('gpt-5.6-luna')
|
||||
expect(calls[0]!.tools).toEqual(['Bash'])
|
||||
})
|
||||
|
||||
it('extracts token usage from last_token_usage', async () => {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-parse.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-parse', model: 'gpt-5.3-codex' }),
|
||||
|
|
@ -681,11 +724,58 @@ describe('codex provider - JSONL parsing', () => {
|
|||
reasoningTokens: 20,
|
||||
tools: ['Bash'],
|
||||
activeDurationMs: 7000,
|
||||
activeGeneratedTokens: 120,
|
||||
// Reasoning (20) is a subset of output_tokens (100), not additive
|
||||
// (#1075/#1078/#1079): the billable/throughput numerator is 100, not 120.
|
||||
activeGeneratedTokens: 100,
|
||||
toolWaitMs: 3000,
|
||||
})
|
||||
})
|
||||
|
||||
it('REGRESSION (#1088 BUG-1): excludes the task_started -> first request-context gap from active time', async () => {
|
||||
// Codex fires task_started before it assembles the request; the 7s gap to
|
||||
// the first request-context event (here, the user message) is CLI/harness
|
||||
// startup, not model wait, and must not count toward active time. If this
|
||||
// ever reverts to windowStart = taskStartedAt, activeDurationMs becomes
|
||||
// 20000 (the full duration_ms) instead of 13000 (20000 - the 7s gap).
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-startup-gap.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-startup-gap', model: 'gpt-5.5' }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:00Z', payload: { type: 'task_started' } }),
|
||||
userMessage('run the tool', '2026-04-14T10:00:07Z'),
|
||||
tokenCount({ timestamp: '2026-04-14T10:00:20Z', last: { output: 100 }, total: { output: 100, total: 100 } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:20Z', payload: { type: 'task_complete', duration_ms: 20_000 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]).toMatchObject({ activeGeneratedTokens: 100, activeDurationMs: 13_000, toolWaitMs: 0 })
|
||||
})
|
||||
|
||||
it('#1088 BUG-8: reads a task_complete duration reported as {secs,nanos}, not only a plain number', async () => {
|
||||
// mcp_tool_call_end already tolerates {secs,nanos} and string durations
|
||||
// (durationValueMs); task_complete only read the plain-number duration_ms
|
||||
// field, so a task_complete reported the object form was silently dropped
|
||||
// (no active timing at all) instead of parsed.
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-object-duration.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-object-duration', model: 'gpt-5.5' }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:00Z', payload: { type: 'task_started' } }),
|
||||
userMessage('run the tool', '2026-04-14T10:00:00Z'),
|
||||
tokenCount({ timestamp: '2026-04-14T10:00:10Z', last: { output: 100 }, total: { output: 100, total: 100 } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:10Z', payload: { type: 'task_complete', duration: { secs: 10, nanos: 0 } } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]).toMatchObject({ activeGeneratedTokens: 100, activeDurationMs: 10_000 })
|
||||
})
|
||||
|
||||
it('keeps estimated output parsing for large token lines without usage info', async () => {
|
||||
// Some rollout variants put token_count metadata beyond the compact head
|
||||
// or omit `info` entirely. The line must still reach the character-based
|
||||
|
|
@ -942,6 +1032,79 @@ describe('codex provider - JSONL parsing', () => {
|
|||
])
|
||||
})
|
||||
|
||||
// #478 follow-up: the shapes the `function_call` path never reached. Fixtures
|
||||
// are synthesized from the real shapes seen in Codex rollouts (a `custom_tool_call`
|
||||
// whose payload is an `input` JS program, and an item-model `item_completed`
|
||||
// carrying a `CommandExecution` item with an argv `command`); no real session
|
||||
// content is used.
|
||||
it('attributes MCP + Skill usage from the exec custom tool and the item model', async () => {
|
||||
const customExec = (input: string, callId: string) => JSON.stringify({
|
||||
type: 'response_item',
|
||||
timestamp: '2026-04-14T10:00:30Z',
|
||||
payload: { type: 'custom_tool_call', call_id: callId, name: 'exec', input },
|
||||
})
|
||||
const commandExecutionItem = (command: string[]) => JSON.stringify({
|
||||
type: 'event_msg',
|
||||
timestamp: '2026-04-14T10:00:40Z',
|
||||
payload: { type: 'item_completed', item: { type: 'CommandExecution', command, exit_code: 0 } },
|
||||
})
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-exec-items.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-exec-items', model: 'gpt-5.5' }),
|
||||
userMessage('use the MCP CLI and load a skill'),
|
||||
// custom-tool transport: MCP call and a skill read, both inside the JS program.
|
||||
customExec('const r = await tools.exec_command({cmd:"mcp-cli call github get_issue \'{}\'"}); text(r.output);', 'c1'),
|
||||
customExec('const r = await tools.exec_command({cmd:"sed -n \'1,200p\' /Users/x/.codex/skills/control-in-app-browser/SKILL.md"}); text(r.output);', 'c2'),
|
||||
// Negatives: a lookup subcommand, and a grep that merely mentions a SKILL.md.
|
||||
customExec('const r = await tools.exec_command({cmd:"mcp-cli info github"}); text(r.output);', 'c3'),
|
||||
customExec('const r = await tools.exec_command({cmd:"grep -rn TODO /Users/x/.codex/skills/deploy/SKILL.md"}); text(r.output);', 'c4'),
|
||||
// item model, no matching response item: must attribute on its own.
|
||||
commandExecutionItem(['/bin/zsh', '-lc', "mcp-cli call optimizely-cms-mcp help '{}'"]),
|
||||
commandExecutionItem(['/bin/zsh', '-lc', 'cat /Users/x/.codex/skills/graphify/SKILL.md']),
|
||||
commandExecutionItem(['/bin/zsh', '-lc', 'ls -la']),
|
||||
tokenCount({ timestamp: '2026-04-14T10:01:00Z', last: { input: 300, output: 100 }, total: { total: 400 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
const tools = calls[0]!.tools
|
||||
// Attribution only: the four custom-tool execs stay Bash, and the item-model
|
||||
// entries add no tool of their own, so the Bash count is unchanged at 4.
|
||||
expect(tools.filter(t => t === 'Bash')).toHaveLength(4)
|
||||
expect(tools.filter(t => t.startsWith('mcp__')).sort()).toEqual([
|
||||
'mcp__github__get_issue',
|
||||
'mcp__optimizely-cms-mcp__help',
|
||||
])
|
||||
expect(calls[0]!.skills?.slice().sort()).toEqual(['control-in-app-browser', 'graphify'])
|
||||
expect(tools.filter(t => t === 'Skill')).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('counts a command carried by BOTH the response item and the item model once', async () => {
|
||||
const cmd = "mcp-cli call github get_issue '{}'"
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-both-shapes.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-both-shapes', model: 'gpt-5.5' }),
|
||||
userMessage('call it twice'),
|
||||
JSON.stringify({ type: 'response_item', timestamp: '2026-04-14T10:00:30Z', payload: { type: 'function_call', name: 'exec_command', arguments: JSON.stringify({ cmd }) } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:31Z', payload: { type: 'item_completed', item: { type: 'CommandExecution', command: ['/bin/zsh', '-lc', cmd] } } }),
|
||||
JSON.stringify({ type: 'response_item', timestamp: '2026-04-14T10:00:32Z', payload: { type: 'function_call', name: 'exec_command', arguments: JSON.stringify({ cmd }) } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:33Z', payload: { type: 'item_completed', item: { type: 'CommandExecution', command: ['/bin/zsh', '-lc', cmd] } } }),
|
||||
tokenCount({ timestamp: '2026-04-14T10:01:00Z', last: { input: 300, output: 100 }, total: { total: 400 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
// Two execs, two MCP attributions - not four.
|
||||
expect(calls[0]!.tools.filter(t => t === 'Bash')).toHaveLength(2)
|
||||
expect(calls[0]!.tools.filter(t => t === 'mcp__github__get_issue')).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('normalizes Codex subagent tool calls to Agent', async () => {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-agent.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-agent', model: 'gpt-5.5' }),
|
||||
|
|
@ -1156,3 +1319,77 @@ describe('codex provider - forked session dedupe', () => {
|
|||
expect(tokens).toBe(300)
|
||||
})
|
||||
})
|
||||
|
||||
describe('codex auto-review pricing (#1047)', () => {
|
||||
it('parses auto-review as itself and prices it as GPT-5.5', async () => {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-auto-review.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-auto', model: 'codex-auto-review' }),
|
||||
userMessage('review the PR'),
|
||||
tokenCount({
|
||||
timestamp: '2026-04-14T10:01:00Z',
|
||||
last: { input: 1_000_000, output: 1_000_000 },
|
||||
total: { total: 2_000_000 },
|
||||
}),
|
||||
])
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set()).parse()) {
|
||||
calls.push(call)
|
||||
}
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]!.model).toBe('codex-auto-review')
|
||||
expect(calls[0]!.costUSD).toBe(calculateCost('gpt-5.5', 1_000_000, 1_000_000, 0, 0, 0))
|
||||
})
|
||||
|
||||
it('discards a warm v11 versioned $0 exact hit so unchanged rollouts reprice', async () => {
|
||||
const cacheDir = join(tmpDir, 'cache')
|
||||
await mkdir(cacheDir, { recursive: true })
|
||||
const prev = process.env['CODEBURN_CACHE_DIR']
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheDir
|
||||
try {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-stale-auto.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-stale-auto', model: 'codex-auto-review' }),
|
||||
userMessage('review the PR'),
|
||||
tokenCount({
|
||||
timestamp: '2026-04-14T10:01:00Z',
|
||||
last: { input: 1_000_000, output: 1_000_000 },
|
||||
total: { total: 2_000_000 },
|
||||
}),
|
||||
])
|
||||
const st = await stat(filePath)
|
||||
// Main's #1075 already owns v11. A colliding v11 $0 file must not be
|
||||
// treated as current after this PR takes v12.
|
||||
expect(CODEX_CACHE_VERSION).toBeGreaterThan(11)
|
||||
await writeFile(join(cacheDir, codexCacheFileName(11)), JSON.stringify({
|
||||
version: 11,
|
||||
files: {
|
||||
[filePath]: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
sizeBytes: st.size,
|
||||
project: 'test',
|
||||
calls: [{
|
||||
model: 'codex-auto-review',
|
||||
costUSD: 0,
|
||||
inputTokens: 1_000_000,
|
||||
outputTokens: 1_000_000,
|
||||
deduplicationKey: 'stale',
|
||||
}],
|
||||
},
|
||||
},
|
||||
}))
|
||||
clearCodexMemCaches()
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set()).parse()) {
|
||||
calls.push(call)
|
||||
}
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]!.costUSD).toBeGreaterThan(0)
|
||||
expect(calls[0]!.costUSD).toBe(calculateCost('gpt-5.5', 1_000_000, 1_000_000, 0, 0, 0))
|
||||
} finally {
|
||||
clearCodexMemCaches()
|
||||
if (prev === undefined) delete process.env['CODEBURN_CACHE_DIR']
|
||||
else process.env['CODEBURN_CACHE_DIR'] = prev
|
||||
}
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -721,9 +721,9 @@ describe('copilot provider - session.shutdown token/cost rollup', () => {
|
|||
it('emits per-leg deltas for a resumed session with cumulative shutdown rollups', async () => {
|
||||
// Numbers from a real resumed CLI 1.0.78 session (3 legs via --resume):
|
||||
// each leg appends a session.shutdown whose modelMetrics are CUMULATIVE.
|
||||
// Emitting deltas keyed by occurrence keeps a growing file append-only
|
||||
// under the durable union-by-key cache merge — re-parsing after each
|
||||
// resume adds only the new leg, never double-counting earlier ones.
|
||||
// Emitting deltas keyed by shutdown timestamp keeps a growing file
|
||||
// append-only under the durable union-by-key cache merge — re-parsing
|
||||
// after each resume adds only the new leg, never double-counting earlier ones.
|
||||
const legs = [
|
||||
{ inputTokens: 24672, outputTokens: 17, cacheReadTokens: 0, cacheWriteTokens: 24670 },
|
||||
{ inputTokens: 74463, outputTokens: 149, cacheReadTokens: 49489, cacheWriteTokens: 24968 },
|
||||
|
|
@ -818,6 +818,54 @@ describe('copilot provider - session.shutdown token/cost rollup', () => {
|
|||
expect(second).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('keeps three stampless shutdown legs as :n keys with lastEventTimestamp, not sessionStartTime', async () => {
|
||||
// sessionStartTime is identical on every leg. Putting it in the key (or
|
||||
// preferring it over lastEventTimestamp for the call timestamp) collapses
|
||||
// a 3-leg journal onto one row. Discovery only yields events.jsonl, so
|
||||
// two-journal fixtures are unreachable; this is the reachable class.
|
||||
const lastEvent = '2026-08-01T10:00:15Z'
|
||||
const sessionStartTime = 1784102040274
|
||||
const stamplessShutdown = (usage: {
|
||||
inputTokens: number
|
||||
outputTokens: number
|
||||
cacheReadTokens: number
|
||||
cacheWriteTokens: number
|
||||
}) => JSON.stringify({
|
||||
type: 'session.shutdown',
|
||||
data: {
|
||||
shutdownType: 'routine',
|
||||
sessionStartTime,
|
||||
modelMetrics: {
|
||||
'claude-sonnet-5': {
|
||||
requests: { count: 1, cost: 1 },
|
||||
usage: { ...usage, reasoningTokens: 0 },
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
const eventsPath = await createSessionDir('sess-stampless', [
|
||||
modelChange('claude-sonnet-5'),
|
||||
assistantMessage({ messageId: 'msg-1', outputTokens: 10, timestamp: lastEvent }),
|
||||
stamplessShutdown({ inputTokens: 3000, outputTokens: 10, cacheReadTokens: 1000, cacheWriteTokens: 500 }),
|
||||
stamplessShutdown({ inputTokens: 7000, outputTokens: 20, cacheReadTokens: 3000, cacheWriteTokens: 1000 }),
|
||||
stamplessShutdown({ inputTokens: 10000, outputTokens: 30, cacheReadTokens: 5000, cacheWriteTokens: 1500 }),
|
||||
])
|
||||
const calls = await collectCalls({ path: eventsPath, project: 'myproject', provider: 'copilot', sourceType: 'jsonl' })
|
||||
const shutdowns = calls.filter(c => c.deduplicationKey.includes(':shutdown:'))
|
||||
expect(shutdowns.map(c => c.deduplicationKey)).toEqual([
|
||||
'copilot:sess-stampless:shutdown:claude-sonnet-5:1',
|
||||
'copilot:sess-stampless:shutdown:claude-sonnet-5:2',
|
||||
'copilot:sess-stampless:shutdown:claude-sonnet-5:3',
|
||||
])
|
||||
expect(shutdowns.map(c => c.timestamp)).toEqual([lastEvent, lastEvent, lastEvent])
|
||||
expect(shutdowns[0]!.inputTokens).toBe(1500)
|
||||
expect(shutdowns[1]!.inputTokens).toBe(1500)
|
||||
expect(shutdowns[2]!.inputTokens).toBe(500)
|
||||
expect(shutdowns.reduce((a, c) => a + c.inputTokens, 0)).toBe(3500)
|
||||
expect(shutdowns.reduce((a, c) => a + c.cacheReadInputTokens, 0)).toBe(5000)
|
||||
expect(shutdowns.reduce((a, c) => a + c.cacheCreationInputTokens, 0)).toBe(1500)
|
||||
})
|
||||
|
||||
it('falls back to the last stamped event when shutdown carries no timestamp at all', async () => {
|
||||
// A shutdown with neither its own timestamp nor sessionStartTime must not
|
||||
// yield an empty-timestamp call: the date-range filters in parser.ts drop
|
||||
|
|
|
|||
|
|
@ -79,6 +79,7 @@ describe('cursor-agent provider', () => {
|
|||
|
||||
expect(provider.modelDisplayName('claude-5-future-model')).toBe('claude-5-future-model (est.)')
|
||||
expect(provider.modelDisplayName('gpt-9')).toBe('gpt-9 (est.)')
|
||||
expect(provider.modelDisplayName('gpt-5.6-sol')).toBe('GPT-5.6 Sol (est.)')
|
||||
})
|
||||
|
||||
it('returns identity for tool display name', () => {
|
||||
|
|
|
|||
|
|
@ -106,8 +106,9 @@ describe('cursor cache', () => {
|
|||
process.env['CODEBURN_CACHE_DIR'] = secondCacheDir
|
||||
await writeCachedResults(dbPath, [], secondFloor)
|
||||
|
||||
const firstPath = join(firstCacheDir, 'cursor-results.json')
|
||||
const secondPath = join(secondCacheDir, 'cursor-results.json')
|
||||
const { cursorCacheFileName } = await import('../../src/cursor-cache.js')
|
||||
const firstPath = join(firstCacheDir, cursorCacheFileName())
|
||||
const secondPath = join(secondCacheDir, cursorCacheFileName())
|
||||
const first = JSON.parse(await readFile(firstPath, 'utf-8')) as { lookbackFloor: string }
|
||||
const second = JSON.parse(await readFile(secondPath, 'utf-8')) as { lookbackFloor: string }
|
||||
expect(first.lookbackFloor).toBe(firstFloor)
|
||||
|
|
|
|||
|
|
@ -66,6 +66,11 @@ describe('vercel-gateway provider', () => {
|
|||
expect(calls[0]?.costUSD).toBe(1.25)
|
||||
expect(calls[0]?.model).toBe('anthropic/claude-sonnet-4.6')
|
||||
})
|
||||
|
||||
it('resolves vendor/slug ids through the global short-name table', () => {
|
||||
expect(vercelGateway.modelDisplayName('openai/gpt-5.6-terra')).toBe('GPT-5.6 Terra')
|
||||
expect(vercelGateway.modelDisplayName('accounts/fireworks/models/kimi-k2p6')).toBe('Kimi K2.6')
|
||||
})
|
||||
})
|
||||
|
||||
describe('vercel-gateway end-to-end (parseAllSessions network path)', () => {
|
||||
|
|
|
|||
317
tests/result-cache-suffix.test.ts
Normal file
317
tests/result-cache-suffix.test.ts
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises'
|
||||
import { existsSync } from 'fs'
|
||||
import { tmpdir } from 'os'
|
||||
import { join } from 'path'
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
|
||||
import {
|
||||
CODEX_CACHE_VERSION,
|
||||
CODEX_LEGACY_CACHE_FILE,
|
||||
clearCodexMemCaches,
|
||||
codexCacheFileName,
|
||||
fingerprintFile,
|
||||
flushCodexCache,
|
||||
readCachedCodexResults,
|
||||
writeCachedCodexResults,
|
||||
} from '../src/codex-cache.js'
|
||||
import {
|
||||
CURSOR_CACHE_VERSION,
|
||||
CURSOR_LEGACY_CACHE_FILE,
|
||||
cursorCacheFileName,
|
||||
readCachedResults,
|
||||
writeCachedResults,
|
||||
} from '../src/cursor-cache.js'
|
||||
import {
|
||||
ANTIGRAVITY_CACHE_VERSION,
|
||||
ANTIGRAVITY_LEGACY_CACHE_FILE,
|
||||
antigravityCacheFileName,
|
||||
clearAntigravityCacheStates,
|
||||
createAntigravityProvider,
|
||||
} from '../src/providers/antigravity.js'
|
||||
import type { ParsedProviderCall } from '../src/providers/types.js'
|
||||
|
||||
const originalCacheDir = process.env['CODEBURN_CACHE_DIR']
|
||||
let root: string
|
||||
|
||||
function call(provider: string, marker: string): ParsedProviderCall {
|
||||
return {
|
||||
provider,
|
||||
model: marker,
|
||||
inputTokens: 1,
|
||||
outputTokens: 1,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 0,
|
||||
cachedInputTokens: 0,
|
||||
reasoningTokens: 0,
|
||||
webSearchRequests: 0,
|
||||
costUSD: 0,
|
||||
tools: [],
|
||||
bashCommands: [],
|
||||
timestamp: '2026-08-22T00:00:00.000Z',
|
||||
speed: 'standard',
|
||||
deduplicationKey: `${provider}:${marker}`,
|
||||
userMessage: '',
|
||||
sessionId: marker,
|
||||
}
|
||||
}
|
||||
|
||||
async function readAntigravityModel(sourcePath: string): Promise<string | undefined> {
|
||||
const parser = createAntigravityProvider().createSessionParser({
|
||||
path: sourcePath,
|
||||
project: 'fixture',
|
||||
provider: 'antigravity',
|
||||
}, new Set())
|
||||
for await (const parsed of parser.parse()) return parsed.model
|
||||
return undefined
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
root = await mkdtemp(join(tmpdir(), 'codeburn-result-suffix-'))
|
||||
process.env['CODEBURN_CACHE_DIR'] = root
|
||||
clearCodexMemCaches()
|
||||
clearAntigravityCacheStates()
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
if (originalCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']
|
||||
else process.env['CODEBURN_CACHE_DIR'] = originalCacheDir
|
||||
clearCodexMemCaches()
|
||||
clearAntigravityCacheStates()
|
||||
await rm(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
describe('unsuffixed result caches coexist with mixed-version binaries', () => {
|
||||
it('publishes Codex to the versioned file and leaves an older unsuffixed file intact', async () => {
|
||||
const sourcePath = join(root, 'rollout.jsonl')
|
||||
await writeFile(sourcePath, '{}\n')
|
||||
const fingerprint = await fingerprintFile(sourcePath)
|
||||
expect(fingerprint).not.toBeNull()
|
||||
|
||||
const legacyPath = join(root, CODEX_LEGACY_CACHE_FILE)
|
||||
await writeFile(legacyPath, JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION - 1,
|
||||
files: { [sourcePath]: { mtimeMs: 1, sizeBytes: 1, project: 'old', calls: [call('codex', 'old-binary')] } },
|
||||
}))
|
||||
|
||||
await writeCachedCodexResults(sourcePath, 'project', [call('codex', 'new-binary')], fingerprint!)
|
||||
await flushCodexCache()
|
||||
|
||||
expect(existsSync(join(root, codexCacheFileName()))).toBe(true)
|
||||
expect(existsSync(legacyPath)).toBe(true)
|
||||
const versioned = JSON.parse(await readFile(join(root, codexCacheFileName()), 'utf8'))
|
||||
const legacy = JSON.parse(await readFile(legacyPath, 'utf8'))
|
||||
expect(versioned.version).toBe(CODEX_CACHE_VERSION)
|
||||
expect(versioned.files[sourcePath].calls[0].model).toBe('new-binary')
|
||||
expect(legacy.version).toBe(CODEX_CACHE_VERSION - 1)
|
||||
expect(legacy.files[sourcePath].calls[0].model).toBe('old-binary')
|
||||
})
|
||||
|
||||
it('adopts a matching-version unsuffixed Codex cache and ignores a mismatched one', async () => {
|
||||
const sourcePath = join(root, 'rollout.jsonl')
|
||||
await writeFile(sourcePath, '{}\n')
|
||||
const fingerprint = await fingerprintFile(sourcePath)
|
||||
expect(fingerprint).not.toBeNull()
|
||||
|
||||
await writeFile(join(root, CODEX_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION,
|
||||
files: {
|
||||
[sourcePath]: {
|
||||
dev: fingerprint!.dev,
|
||||
ino: fingerprint!.ino,
|
||||
mtimeMs: fingerprint!.mtimeMs,
|
||||
sizeBytes: fingerprint!.sizeBytes,
|
||||
project: 'adopted',
|
||||
calls: [call('codex', 'adopt-me')],
|
||||
},
|
||||
},
|
||||
}))
|
||||
|
||||
expect((await readCachedCodexResults(sourcePath))?.calls.map(entry => entry.model)).toEqual(['adopt-me'])
|
||||
expect(existsSync(join(root, CODEX_LEGACY_CACHE_FILE))).toBe(true)
|
||||
expect(existsSync(join(root, codexCacheFileName()))).toBe(false)
|
||||
|
||||
clearCodexMemCaches()
|
||||
await writeFile(join(root, CODEX_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION - 1,
|
||||
files: {
|
||||
[sourcePath]: {
|
||||
dev: fingerprint!.dev,
|
||||
ino: fingerprint!.ino,
|
||||
mtimeMs: fingerprint!.mtimeMs,
|
||||
sizeBytes: fingerprint!.sizeBytes,
|
||||
project: 'stale',
|
||||
calls: [call('codex', 'do-not-adopt')],
|
||||
},
|
||||
},
|
||||
}))
|
||||
expect(await readCachedCodexResults(sourcePath)).toBeNull()
|
||||
})
|
||||
|
||||
it('publishes Cursor to the versioned file and leaves an older unsuffixed file intact', async () => {
|
||||
const dbPath = join(root, 'state.vscdb')
|
||||
await writeFile(dbPath, 'cursor-db')
|
||||
const floor = '2026-01-01T00:00:00.000Z'
|
||||
const legacyPath = join(root, CURSOR_LEGACY_CACHE_FILE)
|
||||
await writeFile(legacyPath, JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION - 1,
|
||||
dbMtimeMs: 1,
|
||||
dbSizeBytes: 1,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'old-binary')],
|
||||
}))
|
||||
|
||||
await writeCachedResults(dbPath, [call('cursor', 'new-binary')], floor)
|
||||
|
||||
expect(existsSync(join(root, cursorCacheFileName()))).toBe(true)
|
||||
expect(existsSync(legacyPath)).toBe(true)
|
||||
const versioned = JSON.parse(await readFile(join(root, cursorCacheFileName()), 'utf8'))
|
||||
const legacy = JSON.parse(await readFile(legacyPath, 'utf8'))
|
||||
expect(versioned.version).toBe(CURSOR_CACHE_VERSION)
|
||||
expect(versioned.calls[0].model).toBe('new-binary')
|
||||
expect(legacy.version).toBe(CURSOR_CACHE_VERSION - 1)
|
||||
expect(legacy.calls[0].model).toBe('old-binary')
|
||||
})
|
||||
|
||||
it('adopts a matching-version unsuffixed Cursor cache and ignores a mismatched one', async () => {
|
||||
const dbPath = join(root, 'state.vscdb')
|
||||
await writeFile(dbPath, 'cursor-db')
|
||||
const { stat } = await import('fs/promises')
|
||||
const fp = await stat(dbPath)
|
||||
const floor = '2026-01-01T00:00:00.000Z'
|
||||
|
||||
await writeFile(join(root, CURSOR_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION,
|
||||
dbMtimeMs: fp.mtimeMs,
|
||||
dbSizeBytes: fp.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'adopt-me')],
|
||||
}))
|
||||
expect((await readCachedResults(dbPath, floor))?.map(entry => entry.model)).toEqual(['adopt-me'])
|
||||
expect(existsSync(join(root, cursorCacheFileName()))).toBe(false)
|
||||
|
||||
await writeFile(join(root, CURSOR_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION - 1,
|
||||
dbMtimeMs: fp.mtimeMs,
|
||||
dbSizeBytes: fp.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'do-not-adopt')],
|
||||
}))
|
||||
expect(await readCachedResults(dbPath, floor)).toBeNull()
|
||||
})
|
||||
|
||||
it('publishes Antigravity to the versioned file and leaves an older unsuffixed file intact', async () => {
|
||||
const sourcePath = join(root, 'shared.pb')
|
||||
await writeFile(sourcePath, 'fixture')
|
||||
const { stat } = await import('fs/promises')
|
||||
const sourceStat = await stat(sourcePath)
|
||||
const legacyPath = join(root, ANTIGRAVITY_LEGACY_CACHE_FILE)
|
||||
await mkdir(root, { recursive: true })
|
||||
await writeFile(legacyPath, JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION - 1,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'old-binary')] },
|
||||
},
|
||||
}))
|
||||
|
||||
await writeFile(join(root, antigravityCacheFileName()), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'new-binary')] },
|
||||
},
|
||||
}))
|
||||
|
||||
expect(await readAntigravityModel(sourcePath)).toBe('new-binary')
|
||||
const legacy = JSON.parse(await readFile(legacyPath, 'utf8'))
|
||||
expect(legacy.cascades.shared.calls[0].model).toBe('old-binary')
|
||||
})
|
||||
|
||||
it('adopts a matching-version unsuffixed Antigravity cache and ignores a mismatched one', async () => {
|
||||
const sourcePath = join(root, 'shared.pb')
|
||||
await writeFile(sourcePath, 'fixture')
|
||||
const { stat } = await import('fs/promises')
|
||||
const sourceStat = await stat(sourcePath)
|
||||
|
||||
await writeFile(join(root, ANTIGRAVITY_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'adopt-me')] },
|
||||
},
|
||||
}))
|
||||
expect(await readAntigravityModel(sourcePath)).toBe('adopt-me')
|
||||
expect(existsSync(join(root, antigravityCacheFileName()))).toBe(false)
|
||||
|
||||
clearAntigravityCacheStates()
|
||||
await writeFile(join(root, ANTIGRAVITY_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION - 1,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'do-not-adopt')] },
|
||||
},
|
||||
}))
|
||||
expect(await readAntigravityModel(sourcePath)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('never serves matching legacy when a versioned file is present but invalid', async () => {
|
||||
const sourcePath = join(root, 'rollout.jsonl')
|
||||
const dbPath = join(root, 'state.vscdb')
|
||||
const pbPath = join(root, 'shared.pb')
|
||||
await writeFile(sourcePath, '{}\n')
|
||||
await writeFile(dbPath, 'cursor-db')
|
||||
await writeFile(pbPath, 'fixture')
|
||||
const fingerprint = await fingerprintFile(sourcePath)
|
||||
const { stat } = await import('fs/promises')
|
||||
const dbStat = await stat(dbPath)
|
||||
const pbStat = await stat(pbPath)
|
||||
const floor = '2026-01-01T00:00:00.000Z'
|
||||
expect(fingerprint).not.toBeNull()
|
||||
|
||||
await writeFile(join(root, CODEX_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION,
|
||||
files: {
|
||||
[sourcePath]: {
|
||||
dev: fingerprint!.dev,
|
||||
ino: fingerprint!.ino,
|
||||
mtimeMs: fingerprint!.mtimeMs,
|
||||
sizeBytes: fingerprint!.sizeBytes,
|
||||
project: 'legacy',
|
||||
calls: [call('codex', 'legacy-served')],
|
||||
},
|
||||
},
|
||||
}))
|
||||
await writeFile(join(root, CURSOR_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION,
|
||||
dbMtimeMs: dbStat.mtimeMs,
|
||||
dbSizeBytes: dbStat.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'legacy-served')],
|
||||
}))
|
||||
await writeFile(join(root, ANTIGRAVITY_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION,
|
||||
cascades: {
|
||||
shared: { mtimeMs: pbStat.mtimeMs, sizeBytes: pbStat.size, calls: [call('antigravity', 'legacy-served')] },
|
||||
},
|
||||
}))
|
||||
|
||||
await writeFile(join(root, codexCacheFileName()), '{not-json')
|
||||
await writeFile(join(root, cursorCacheFileName()), '{not-json')
|
||||
await writeFile(join(root, antigravityCacheFileName()), '{not-json')
|
||||
expect(await readCachedCodexResults(sourcePath)).toBeNull()
|
||||
expect(await readCachedResults(dbPath, floor)).toBeNull()
|
||||
expect(await readAntigravityModel(pbPath)).toBeUndefined()
|
||||
|
||||
clearCodexMemCaches()
|
||||
clearAntigravityCacheStates()
|
||||
await writeFile(join(root, codexCacheFileName()), JSON.stringify({ version: CODEX_CACHE_VERSION - 1, files: {} }))
|
||||
await writeFile(join(root, cursorCacheFileName()), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION - 1,
|
||||
dbMtimeMs: dbStat.mtimeMs,
|
||||
dbSizeBytes: dbStat.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'wrong-version')],
|
||||
}))
|
||||
await writeFile(join(root, antigravityCacheFileName()), JSON.stringify({ version: ANTIGRAVITY_CACHE_VERSION - 1, cascades: {} }))
|
||||
expect(await readCachedCodexResults(sourcePath)).toBeNull()
|
||||
expect(await readCachedResults(dbPath, floor)).toBeNull()
|
||||
expect(await readAntigravityModel(pbPath)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
80
tests/setup/env-isolation-vars.ts
Normal file
80
tests/setup/env-isolation-vars.ts
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
// Side-effect-free lists for tests/setup/env-isolation.ts.
|
||||
// Imported by the setup file (which applies them) and by
|
||||
// tests/env-isolation-declarations.test.ts (which asserts coverage).
|
||||
// Do not put applyIsolation() here — importing this module from a test
|
||||
// must not re-sandbox the process or register another beforeEach.
|
||||
//
|
||||
// A comment containing 'HERMES_HOME' is not isolation. The declaration
|
||||
// test imports these arrays, so a commented-out name cannot false-green.
|
||||
|
||||
export const REDIRECTED = [
|
||||
'HOME',
|
||||
'XDG_CONFIG_HOME',
|
||||
'XDG_DATA_HOME',
|
||||
'XDG_CACHE_HOME',
|
||||
'XDG_STATE_HOME',
|
||||
'APPDATA',
|
||||
'LOCALAPPDATA',
|
||||
] as const
|
||||
|
||||
export const CLEARED = [
|
||||
// Provider session-discovery dirs
|
||||
'CLAUDE_CONFIG_DIR',
|
||||
'CLAUDE_CONFIG_DIRS',
|
||||
'CLINE_DIR',
|
||||
'CLINE_DATA_DIR',
|
||||
'CLINE_SESSION_DATA_DIR',
|
||||
'CODEX_HOME',
|
||||
'CODEWHALE_HOME',
|
||||
'CRUSH_GLOBAL_DATA',
|
||||
'CODEBUFF_DATA_DIR',
|
||||
'DSH_HOME',
|
||||
'FACTORY_DIR',
|
||||
'GOOSE_PATH_ROOT',
|
||||
'GROK_HOME',
|
||||
'HERMES_HOME',
|
||||
'KIRO_HOME',
|
||||
'KIMI_CODE_HOME',
|
||||
'KIMI_SHARE_DIR',
|
||||
'LINGTAI_HOME',
|
||||
'LINGTAI_TUI_GLOBAL_DIR',
|
||||
'LINGTAI_TUI_HOME',
|
||||
'MUX_ROOT',
|
||||
'OPENCODE_DATA_DIR',
|
||||
'OPENCODE_DB_PREFIX',
|
||||
'QUICKWORK_HOME',
|
||||
'QWEN_DATA_DIR',
|
||||
'VIBE_HOME',
|
||||
'WARP_DB_PATH',
|
||||
'ZS_DATA_DIR',
|
||||
// codeburn override dirs / paths
|
||||
'CODEBURN_CACHE_DIR',
|
||||
'CODEBURN_COPILOT_GLOBAL_STORAGE_DIR',
|
||||
'CODEBURN_COPILOT_JETBRAINS_DIR',
|
||||
'CODEBURN_COPILOT_OTEL_DB',
|
||||
'CODEBURN_COPILOT_SESSION_STATE_DIR',
|
||||
'CODEBURN_COPILOT_SESSION_STORE_DB',
|
||||
'CODEBURN_COPILOT_WS_STORAGE_DIR',
|
||||
'CODEBURN_DESKTOP_SESSIONS_DIR',
|
||||
'CODEBURN_MUX_DIR',
|
||||
'CODEBURN_OPEN_DESIGN_DIR',
|
||||
'CODEBURN_OPENCLAUDE_DIR',
|
||||
'CODEBURN_ANTIGRAVITY_SETTINGS_PATH',
|
||||
// codeburn behavior toggles (set by the dev to tweak local runs)
|
||||
'CODEBURN_COPILOT_DISABLE_OTEL',
|
||||
'CODEBURN_TZ',
|
||||
'CODEBURN_VERBOSE',
|
||||
'CODEBURN_CURSOR_MAX_BUBBLES',
|
||||
'CODEBURN_FORCE_MACOS_MAJOR',
|
||||
// Provider model/credential overrides
|
||||
'KIMI_MODEL_NAME',
|
||||
'AI_GATEWAY_API_KEY',
|
||||
'VERCEL_OIDC_TOKEN',
|
||||
// Read by detectBashBloat - a dev's real shell limit must not bleed in
|
||||
'BASH_MAX_OUTPUT_LENGTH',
|
||||
] as const
|
||||
|
||||
// Snapshotted from the dev's shell and restored every test. These can't be
|
||||
// wiped (Node needs PATH for spawn / module resolution, dashboard/table layout
|
||||
// reads COLUMNS) but a test that mutates them shouldn't leak.
|
||||
export const PRESERVED = ['PATH', 'COLUMNS'] as const
|
||||
|
|
@ -1,12 +1,13 @@
|
|||
// Vitest setup file: isolates every test from the developer's shell environment.
|
||||
//
|
||||
// codeburn discovers sessions through a long list of provider-specific env
|
||||
// vars (CLAUDE_CONFIG_DIR, CODEX_HOME, CRUSH_GLOBAL_DATA, …) and via HOME /
|
||||
// XDG_* / APPDATA / LOCALAPPDATA. Without this file, any value set in the
|
||||
// developer's shell (e.g. CLAUDE_CONFIG_DIRS=/Users/me/.claude:…) bleeds into
|
||||
// vars (CLAUDE_CONFIG_DIR, CODEX_HOME, HERMES_HOME, CRUSH_GLOBAL_DATA, …) and
|
||||
// via HOME / XDG_* / APPDATA / LOCALAPPDATA. Without this file, any value set
|
||||
// in the developer's shell (e.g. HERMES_HOME=/Users/me/.hermes) bleeds into
|
||||
// fixture-based tests: the parser reads the developer's REAL sessions instead
|
||||
// of the temp-dir fixture, producing nonsense totals and false failures that
|
||||
// pass on a clean CI runner.
|
||||
// pass on a clean CI runner. tests/env-isolation-declarations.test.ts fails
|
||||
// closed if PROVIDER_ENV_VARS grows a data-dir override that is not listed.
|
||||
//
|
||||
// What this file does:
|
||||
// 1. Mints an empty sandbox temp dir once per worker.
|
||||
|
|
@ -33,68 +34,10 @@ import { tmpdir } from 'os'
|
|||
import { join } from 'path'
|
||||
import { beforeEach } from 'vitest'
|
||||
|
||||
import { CLEARED, PRESERVED, REDIRECTED } from './env-isolation-vars.js'
|
||||
|
||||
const sandbox = mkdtempSync(join(tmpdir(), 'codeburn-test-env-'))
|
||||
|
||||
const REDIRECTED = [
|
||||
'HOME',
|
||||
'XDG_CONFIG_HOME',
|
||||
'XDG_DATA_HOME',
|
||||
'XDG_CACHE_HOME',
|
||||
'XDG_STATE_HOME',
|
||||
'APPDATA',
|
||||
'LOCALAPPDATA',
|
||||
] as const
|
||||
|
||||
const CLEARED = [
|
||||
// Provider session-discovery dirs
|
||||
'CLAUDE_CONFIG_DIR',
|
||||
'CLAUDE_CONFIG_DIRS',
|
||||
'CLINE_DIR',
|
||||
'CLINE_DATA_DIR',
|
||||
'CLINE_SESSION_DATA_DIR',
|
||||
'CODEX_HOME',
|
||||
'CODEWHALE_HOME',
|
||||
'CRUSH_GLOBAL_DATA',
|
||||
'CODEBUFF_DATA_DIR',
|
||||
'FACTORY_DIR',
|
||||
'GOOSE_PATH_ROOT',
|
||||
'GROK_HOME',
|
||||
'KIRO_HOME',
|
||||
'KIMI_SHARE_DIR',
|
||||
'MUX_ROOT',
|
||||
'OPENCODE_DATA_DIR',
|
||||
'OPENCODE_DB_PREFIX',
|
||||
'QWEN_DATA_DIR',
|
||||
'VIBE_HOME',
|
||||
'WARP_DB_PATH',
|
||||
'ZS_DATA_DIR',
|
||||
// codeburn override dirs / paths
|
||||
'CODEBURN_CACHE_DIR',
|
||||
'CODEBURN_COPILOT_JETBRAINS_DIR',
|
||||
'CODEBURN_COPILOT_OTEL_DB',
|
||||
'CODEBURN_COPILOT_SESSION_STATE_DIR',
|
||||
'CODEBURN_COPILOT_WS_STORAGE_DIR',
|
||||
'CODEBURN_DESKTOP_SESSIONS_DIR',
|
||||
'CODEBURN_MUX_DIR',
|
||||
'CODEBURN_ANTIGRAVITY_SETTINGS_PATH',
|
||||
// codeburn behavior toggles (set by the dev to tweak local runs)
|
||||
'CODEBURN_COPILOT_DISABLE_OTEL',
|
||||
'CODEBURN_TZ',
|
||||
'CODEBURN_VERBOSE',
|
||||
'CODEBURN_CURSOR_MAX_BUBBLES',
|
||||
'CODEBURN_FORCE_MACOS_MAJOR',
|
||||
// Provider model/credential overrides
|
||||
'KIMI_MODEL_NAME',
|
||||
'AI_GATEWAY_API_KEY',
|
||||
'VERCEL_OIDC_TOKEN',
|
||||
// Read by detectBashBloat - a dev's real shell limit must not bleed in
|
||||
'BASH_MAX_OUTPUT_LENGTH',
|
||||
] as const
|
||||
|
||||
// Snapshotted from the dev's shell and restored every test. These can't be
|
||||
// wiped (Node needs PATH for spawn / module resolution, dashboard/table layout
|
||||
// reads COLUMNS) but a test that mutates them shouldn't leak.
|
||||
const PRESERVED = ['PATH', 'COLUMNS'] as const
|
||||
const preservedSnapshot = new Map<string, string | undefined>()
|
||||
for (const key of PRESERVED) preservedSnapshot.set(key, process.env[key])
|
||||
|
||||
|
|
|
|||
|
|
@ -553,7 +553,7 @@ describe('buildAttributionOtlpPayload', () => {
|
|||
expect(sessionSpan.spanId).not.toBe(commitSpan.spanId)
|
||||
|
||||
const sessionAttrs = attrMap(sessionSpan.attributes)
|
||||
expect(sessionAttrs['ai.session_id']).toEqual({ stringValue: 'sess-1' })
|
||||
expect(sessionAttrs['ai.session_id']).toBeUndefined()
|
||||
expect(sessionAttrs['ai.project']).toEqual({ stringValue: 'app' })
|
||||
expect(sessionAttrs['git.repo']).toEqual({ stringValue: 'github.com/acme/widget' })
|
||||
expect(sessionAttrs['git.commit_count']).toEqual({ intValue: '1' })
|
||||
|
|
@ -565,12 +565,24 @@ describe('buildAttributionOtlpPayload', () => {
|
|||
expect(sessionSpan.endTimeUnixNano).toBe((BigInt(new Date('2026-01-01T11:00:00.000Z').getTime()) * 1_000_000n).toString())
|
||||
|
||||
const commitAttrs = attrMap(commitSpan.attributes)
|
||||
expect(commitAttrs['ai.session_id']).toBeUndefined()
|
||||
expect(commitAttrs['git.sha']).toEqual({ stringValue: 'a'.repeat(40) })
|
||||
expect(commitAttrs['git.in_main']).toEqual({ boolValue: true })
|
||||
expect(commitAttrs['git.was_reverted']).toEqual({ boolValue: false })
|
||||
expect(commitAttrs['git.repo']).toEqual({ stringValue: 'github.com/acme/widget' })
|
||||
})
|
||||
|
||||
it('keeps session identity off the wire; join is the shared traceId', () => {
|
||||
const items = flattenAttributionRecords([makeRecord()])
|
||||
const payload = buildAttributionOtlpPayload(items)
|
||||
const spans = payload.resourceSpans[0]!.scopeSpans[0]!.spans
|
||||
expect(spans.length).toBeGreaterThan(0)
|
||||
for (const span of spans) {
|
||||
expect(attrMap(span.attributes)['ai.session_id']).toBeUndefined()
|
||||
expect(span.traceId).toBe(deriveTraceId('sess-1'))
|
||||
}
|
||||
})
|
||||
|
||||
it('omits git.repo when null and pr_links when empty', () => {
|
||||
const items = flattenAttributionRecords([makeRecord({ repo: null, prLinks: [], commits: [] })])
|
||||
const payload = buildAttributionOtlpPayload(items)
|
||||
|
|
|
|||
|
|
@ -156,6 +156,7 @@ describe('buildOtlpPayload', () => {
|
|||
expect(attrMap['ai.cost_usd']).toEqual({ doubleValue: 0.05 })
|
||||
expect(attrMap['ai.project']).toEqual({ stringValue: 'my-project' })
|
||||
expect(attrMap['ai.speed']).toEqual({ stringValue: 'standard' })
|
||||
expect(attrMap['ai.session_id']).toBeUndefined()
|
||||
})
|
||||
|
||||
it('includes tools as array attribute', () => {
|
||||
|
|
|
|||
|
|
@ -350,6 +350,7 @@ describe('review-findings regressions', () => {
|
|||
expect(isExpectedFreeModel('qwen3.6:35b-a3b-bf16')).toBe(true)
|
||||
expect(isExpectedFreeModel('llama-3-8b-q4')).toBe(true)
|
||||
expect(isExpectedFreeModel('claude-opus-4-8')).toBe(false)
|
||||
expect(isExpectedFreeModel('auto-genius')).toBe(true)
|
||||
// 95 local calls + 5 unpriced cloud calls: coverage must be 0, not 0.95.
|
||||
expect(computePricingCoverage(5, 5)).toBe(0)
|
||||
})
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue