mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-10-19 18:23:41 +00:00
Merge origin/main into fix/968-subscription-flat-rate
Resolves conflicts against main after #1078/#1084/#1088/#1090/#1092/ #1053/#1056 landed since this branch's last upstream merge. - CHANGELOG.md: kept both entries (this PR's #968/#1050 note plus main's #1079/#1088, #1082, #1075 notes that had moved into the same "### Fixed" slot). - Everything else (src/models.ts, main.ts, usage-aggregator.ts, daily-cache.ts, tests/*) merged cleanly with no conflict markers; git's recursive merge combined #1050's flat-rate classifier changes with #1056's codex-auto-review -> gpt-5.5 alias without overlap. Verified the codex-auto-review / #1056 interaction post-merge: isBuiltInFlatRateModel no longer matches codex-auto-review (dropped per #1050), while MODEL_ALIASES still aliases it to gpt-5.5 (#1056), so it prices at GPT-5.5 rates rather than $0. Covered by the existing tests/models.test.ts "Codex activity ids (#1047)" describe block and the "does not treat a priced sibling as expected-free" case. getFlatRateModelsConfigHash's output is folded into getDailyCacheConfigHash's template literal unconditionally (flatRateModels=<hash>), so the flat-rate section always participates in the daily-cache invalidation hash even when empty -- no DAILY_CACHE_VERSION bump needed for this change.
This commit is contained in:
iamtoruk
commit
f9c99a0651
57 changed files with 2152 additions and 311 deletions
17
.github/workflows/tests.yml
vendored
17
.github/workflows/tests.yml
vendored
|
|
@ -33,14 +33,13 @@ jobs:
|
|||
# script since #948, so CI and a contributor's `npm test` can never drift.
|
||||
- name: Test suite (parallel)
|
||||
run: npm test
|
||||
# Single forked worker, so lock contention comes only from the child processes the
|
||||
# tests spawn deliberately. Quarantined (reports, never gates): the process
|
||||
# suite still races its own takeover window even serially on slow runners -
|
||||
# tracked in #904; drop continue-on-error once that race is settled.
|
||||
# Step-level timeout so a stalled lock suite fails soft here instead of
|
||||
# tripping the job's 15-minute budget, which kills the whole job as
|
||||
# "cancelled" and hides the parallel suite's green result.
|
||||
- name: Cache-lock suite (serial, quarantined)
|
||||
continue-on-error: true
|
||||
# Single forked worker, so lock contention comes only from the child
|
||||
# processes the tests spawn deliberately. Serial because the suite is
|
||||
# parallelism-sensitive (fails under full worker pressure, passes
|
||||
# serially). After #904 the loser of a stale-lock contest may be
|
||||
# timed-out or completed-by-other; this step gates CI.
|
||||
# Step-level timeout so a stalled lock suite fails this job instead of
|
||||
# eating the 15-minute budget and cancelling a green parallel suite.
|
||||
- name: Cache-lock suite (serial)
|
||||
timeout-minutes: 5
|
||||
run: npm run test:locks
|
||||
|
|
|
|||
|
|
@ -39,6 +39,11 @@
|
|||
|
||||
### Fixed
|
||||
- **Subscription SKUs are classified from real product ids, and a false-positive built-in can be opted out.** `codex-auto-review` consumes ordinary Codex usage ([openai/codex#32224](https://github.com/openai/codex/issues/32224)) and is priced as GPT-5.5 on #1056, so treating it as $0 hid real spend — it left the flat-rate list. Warp's product id is `auto`, not the synthetic `warp`. `kimi-for-coding-highspeed` (the SKU #968 was filed around) is now honestly $0. `big-pickle` was dropped: it appears under OpenCode, not as a cited ClinePass codename. `codeburn model-flat-rate --remove` now opts out of a built-in, so a wrong classifier entry can warn again without waiting for a release. The daily-cache config hash now always includes the flat-rate section (even when empty), so the first run after upgrade re-derives every stored day once from the warm session cache. (#968, #1050)
|
||||
- **Codex MCP and skill usage is attributed from every shape Codex records a shell command in.** `mcp-cli call <server> <tool>` was only recognized when the command arrived as `function_call` arguments (#656). Codex has two other shapes for the same exec: its custom-tool transport records the shell tool as a `custom_tool_call` whose payload is an `input` program rather than `arguments`, and its item model repeats a finished command as `event_msg`/`item_completed` carrying a `CommandExecution` item with an argv `command`. Both reached the Bash counter and neither reached the matcher, so a CLI-wrapped MCP call stayed absent from the MCP breakdown exactly as before the fix. All three shapes now feed one classification pipeline. The same pipeline learns skills: Codex has no skill tool, so loading one is a shell read of the skill's `SKILL.md`, and those reads landed entirely under Bash with the Skills dimension empty. A read counts as a skill load only when the command segment starts with a file-reading binary (`cat`/`bat`/`sed`/`head`/`tail`/`less`/`more`) and the path it reads ends in `<name>/SKILL.md`; the skill is `<name>`, the same key `pi` derives for a native skill read (#588) and the same vocabulary the Claude parser records from the `Skill` tool. A `grep`/`rg`/`ls` that merely mentions a `SKILL.md` is a search near the file, not a skill load, and stays plain Bash. This is attribution only — no call, token or cost figure moves, and a command carried by both a response item and an item-model item is attributed once. On a 1,397-rollout corpus: Skills went from empty to 7 skills over 35 turns (55 attributions), Bash was unchanged at 42,170, and cost, calls, tokens, sessions, daily, models and projects came back identical. Cached Codex sessions re-parse once (`CODEX_CACHE_VERSION` 13 → 14 and the codex parse version both move; without them the fix is invisible on a warm cache). Thanks @chr-evensen. (#478)
|
||||
- **`gpt-5.6-codex` and `gpt-5.6-codex-max` now have their own pricing rows.** Neither id is in LiteLLM yet, and both were missing from the bundled snapshot — flagged during #1075 verification on a real corpus (285 sessions, 5,446 calls). `getModelCosts` already resolved both through the `gpt-5.6` prefix fallback, so live pricing was already correct once a session priced fresh; every prior Codex-suffixed id LiteLLM does carry bills identically to its bare-model sibling of the same generation (`gpt-5-codex` == `gpt-5`, `gpt-5.1-codex` == `gpt-5.1-codex-max` == `gpt-5.1`, `gpt-5.2-codex` == `gpt-5.2`, `gpt-5.3-codex` == `gpt-5.3`), which is the evidence both new rows mirror rather than inventing a rate. The gap that does not self-heal is the daily cache: it has no per-provider invalidation, so a day finalized while either id had no billable rate keeps that $0 forever. Raising `MIN_SUPPORTED_VERSION` (v23 -> v24) forces the one-time re-derivation, a lossless no-op for days already correct. (#1077)
|
||||
- **Mixed-version installs no longer thrash the Codex / Cursor / Antigravity result caches.** Daily and session caches already own a version-suffixed file so an old desktop binary and a newer CLI cannot clobber each other. The three per-provider result caches still used one unsuffixed filename with an internal version field, so a v10 and a v11 binary rewrote the same `codex-results.json` (and the Cursor / Antigravity siblings) on every run and each re-parsed its whole corpus. They now write `*-results.v<n>.json` the same way the daily cache does. The unsuffixed file is left for older binaries; a matching-version copy is adopted once and never overwritten. (#1082)
|
||||
- **Codex spend no longer counts reasoning tokens twice, and cache writes are priced only where OpenAI actually charges for them.** OpenAI bills reasoning tokens as *part of* `output_tokens`, not on top of it — on a 1,396-rollout corpus all 134,316 events carrying a total satisfy `input + output == total` — but CodeBurn added `reasoning_output_tokens` to output when pricing a Codex call and again in the models, audit and per-model displays. Every Codex number was therefore too high: on that corpus **cost by $166.03 (3.5%)** and **displayed Output tokens by 34.6%** ($4,713.12 -> $4,547.09; 22.6M -> 16.8M output tokens). The raw `reasoningTokens` figure is unchanged and still reported on its own; only the double-count is gone. Both places that price a Codex call — the parser and the cache-rehydration re-price — now go through one shared `billableOutputTokens` helper, so a cold run and a warm run can never disagree. Separately, Codex's `cache_write_input_tokens` (new in codex PR #33454) was never read and cache-creation tokens were hardcoded to 0; they are now carved out of the uncached-input bucket and clamped so they can never exceed it. That carve-out happens **only on models whose pricing source publishes a real cache-write rate** — gpt-5.6 and its terra/sol/luna variants charge 1.25x input for a cache write, everything before it charges nothing extra — because CodeBurn fabricates a 1.25x rate when a source omits one, and charging that would have invented a surcharge on gpt-5.5, gpt-5.4, gpt-5.3-codex and gpt-5. On models without an explicit rate the tokens stay in the plain input bucket and the price is unchanged to the cent. The field is new enough that today's impact is $0 on that corpus. Codex sessions re-parse once and the daily cache re-derives once off the warm session cache (a global re-derivation of every day and every provider, since it has no per-provider invalidation); no other provider's numbers move. Days whose Codex transcripts have since aged out are held by the same never-lose guard #1040 relies on: a re-derivation that finds fewer calls than the settled baseline keeps the older, pre-fix (double-counted) total rather than truncating it, so those days do not pick up the repricing until their sources are re-derived with equal or greater evidence. Long-context pricing tiers from the same report are tracked separately in #1076 and the missing `gpt-5.6-codex` snapshot rows in #1077. Thanks @chr-evensen. (#1075)
|
||||
- **Codex Tok/s no longer counts reasoning tokens twice or credits harness startup as model time.** Two distortions in the same metric, found and fixed together because they share the same cache-invalidation and test surface. (1) #1075 fixed the reasoning-token double-count for cost, but `activeGeneratedTokens`/`taskGeneratedTokens` in the Codex parser and `generatedTokens` in the `codex-tps` live-throughput reader still summed `outputTokens + reasoningTokens`; both now go through the same `billableOutputTokens('codex', …)` helper #1075 introduced, so the numerator can never drift from the billed one. (2) Codex fires `task_started` before it assembles the request, so the gap up to the first request-context event (`turn_context`, `world_state`, `event_msg/user_message`, or a `response_item/message`) was pure CLI/harness startup counted as active model time — the active window now starts at that first event instead, which matters most for one-shot `codex exec` sessions that pay the gap on every task. The duplicated tool-interval clip/merge/cap logic in `providers/codex.ts` and `codex-throughput.ts` is now one function (`mergeToolIntervals`, exported from `codex-throughput.ts`), which also closes a live trap where `task_complete`'s duration only parsed a plain number and silently dropped the `{secs,nanos}`/string forms `mcp_tool_call_end` already tolerated. (A third suspected distortion — fork-replay dedup dropping a token_count event's tokens from the numerator without shrinking the window to match — was investigated and retracted: the earlier `prevCumulativeTotal` guard already discards a repeated running total before dedup is ever reached, so a real Codex writer never produces a partial drop; the dedup site now carries a comment recording this so the trip isn't repeated.) Display only, no cost or token-count impact — verified byte-identical on the same real corpus. Combined effect on a real Codex corpus (original bug -> all fixes): GPT-5.5 37.8 -> 28.2 tok/s (-25.5%), Codex Auto Review 23.3 -> 20.0 (-14.2%), GPT-5.6 Sol 43.2 -> 33.9 (-21.4%), GPT-5.6 Luna 53.5 -> 49.0 (-8.5%), GPT-5.4 68.0 -> 43.6 (-35.9%), GPT-5.4 Mini 54.9 -> 55.6 (**+1.1%**, the harness-startup correction outweighing the reasoning-count correction for this model on this corpus). `activeGeneratedTokens`/`activeDurationMs`/`toolWaitMs` are stored verbatim in both the Codex result cache and the session cache rather than re-derived on read, so none of this self-heals: Codex sessions re-parse once (one cache-version bump covers both fixes, since they touch the same fields). The dashboard's per-model column stays labelled `Tok/s` — a wider label had zero room at the standard three-column layout, verified by breaking a real width-budget test — but the legend beneath it now reads "Effective Tok/s: generated tokens ÷ time the agent spent waiting on the model, tool execution excluded. Includes prefill, request assembly and reasoning. Not comparable to vendor decode-speed figures." (#1079, #1088)
|
||||
- **Codex calls attributed from session metadata no longer carry a stale model.** The Buffer fast path scanned `session_meta` for the first `"model"` string anywhere in the payload, so a nested `base_instructions.provenance.model` was read as if it were `payload.model` — and since the model is last-writer-wins state, that wrong value was credited to every call before the rollout's first `turn_context` and to every call after any mid-file `session_meta` (29 of 1380 rollouts on one real corpus carry a late `session_meta`, and 57 record usage before any `turn_context`). Direct payload fields are now read depth-aware, which is what the non-fast `JSON.parse` path always did. Codex sessions re-parse once (~9s on a 4 GB rollout corpus) and the daily cache re-derives once off the warm session cache, a global re-derivation of every day and every provider since it has no per-provider invalidation; it moves per-model attribution, and clears any rollup an earlier parse change had left stale. Days whose transcripts have partly aged out are held by the never-lose guard: on a real 110-day cache no day lost value and none disappeared — 100 days came back identical and 9 grok days rose by $19.80 in total. Thanks @timdp. (#1040)
|
||||
- **Codex `session_meta` cwd / session id / originator follow the same depth-1 window as `model`.** #1040 fixed nested `provenance.model`; the compact Buffer path still took the first `cwd`, `session_id`, `originator`, `name`, `forked_from_id` or `model_provider` anywhere in the payload, so a `dynamic_tools[].name` (or any same-named nested key) could steal the top-level field. Those strings now use the existing payload-depth-1 scan. Function-call `name` on other event types is unchanged. Codex sessions re-parse once. (#1045)
|
||||
- **Plan rows for sticker-price presets read as a budget instead of live provider quota.** There is no Grok quota endpoint, so a SuperGrok row was parsed API-equivalent spend divided by the plan's sticker price on a monthly reset — but the TUI labelled that math "plan" and "reset", which next to a client showing xAI's real weekly window read as CodeBurn being wrong. The bars and the arithmetic are unchanged; the words are not. Both the dashboard and the desktop app now say the number is an API-equivalent monthly budget and not a live provider window, in the same wording on both surfaces, and for every preset rather than as a SuperGrok special case. The window is anniversary-based (`plan.resetDay`, settable with `codeburn plan set --reset-day`), so it is called a budget reset rather than a calendar one. The row was also shortened to fit 80 columns: at that width the percentage and the projected month were being truncated away, including on custom plans, whose label carries the provider.
|
||||
|
|
|
|||
|
|
@ -700,7 +700,7 @@ These are starting points, not verdicts. A 60% cache hit on a single experimenta
|
|||
| **Claude Code** | `~/.claude/projects/<sanitized-path>/<session-id>.jsonl` | Each assistant entry carries model name, token usage (input, output, cache read, cache write), `tool_use` blocks, and timestamps. |
|
||||
| **Claude (multiple config dirs)** | Set via `CLAUDE_CONFIG_DIRS` (e.g. `~/.claude-work:~/.claude-personal`) | Scans every listed directory and merges sessions into one row per project so totals reflect all your Claude usage. Use `:` on POSIX, `;` on Windows; overrides `CLAUDE_CONFIG_DIR`. Missing or unreadable directories are skipped. |
|
||||
| **Codex (OpenAI)** | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl`, `~/.codex/archived_sessions/rollout-*.jsonl` | Reads `token_count` events (per-call and cumulative usage) and `function_call` entries for tool tracking; attributes cost by project working directory. `codeburn report --provider codex` views Codex alone. |
|
||||
| **Cursor** | SQLite `state.vscdb` under `globalStorage`: macOS `~/Library/Application Support/Cursor/User/globalStorage/`, Linux `~/.config/Cursor/User/globalStorage/`, Windows `%APPDATA%/Cursor/User/globalStorage/`; results cached at `~/.cache/codeburn/cursor-results.json` | Input tokens come from Cursor's own per-conversation context meter (`composerData.promptTokenBreakdown`), credited once per conversation on a stable anchor; tool calls and shell commands are read from the agent stream (`agentKv`), and Composer house models are priced from Cursor's published rates. Output is a reply-text estimate and cache tokens are server-side only, so figures are marked estimated and undercount the Cursor admin console for long conversations. The cache auto-invalidates when the database changes; the first run on a large database can take a minute. |
|
||||
| **Cursor** | SQLite `state.vscdb` under `globalStorage`: macOS `~/Library/Application Support/Cursor/User/globalStorage/`, Linux `~/.config/Cursor/User/globalStorage/`, Windows `%APPDATA%/Cursor/User/globalStorage/`; results cached at `~/.cache/codeburn/cursor-results.v<n>.json` | Input tokens come from Cursor's own per-conversation context meter (`composerData.promptTokenBreakdown`), credited once per conversation on a stable anchor; tool calls and shell commands are read from the agent stream (`agentKv`), and Composer house models are priced from Cursor's published rates. Output is a reply-text estimate and cache tokens are server-side only, so figures are marked estimated and undercount the Cursor admin console for long conversations. The cache auto-invalidates when the database changes; the first run on a large database can take a minute. |
|
||||
| **OpenCode** | SQLite `~/.local/share/opencode/opencode*.db` or file-based `~/.local/share/opencode/storage/` (respects `XDG_DATA_HOME`; `OPENCODE_DATA_DIR`/`OPENCODE_DB_PREFIX` for renamed/forked builds) | Queries `session`, `message`, and `part` read-only and recalculates cost via LiteLLM (falling back to OpenCode's own cost field for unpriced models). Subtask sessions (`parent_id IS NOT NULL`) are excluded to avoid double counting; multiple channel databases are supported. |
|
||||
| **Gemini CLI** | `~/.gemini/tmp/<project>/chats/session-*.json` | One JSON file per session with real token counts (input, output, cached, thoughts) per message, so no estimation is needed. Input is reported inclusive of cached, so CodeBurn subtracts cached before pricing to avoid double charging. |
|
||||
| **Antigravity (CLI & IDE)** | Session files under `.gemini/` folders, plus the running language server | Pulls granular trajectory and pricing from the language server process. For the short-lived CLI, optionally install a status-line hook with `codeburn antigravity-hook install` so usage is captured between menubar refreshes. The IDE is detected via the `--app-data-dir antigravity-ide` flag on Windows. |
|
||||
|
|
|
|||
|
|
@ -136,8 +136,8 @@ Three caches under `~/.cache/codeburn/` (override with `CODEBURN_CACHE_DIR`):
|
|||
|
||||
| File | Owner | Invalidation |
|
||||
|---|---|---|
|
||||
| `codex-results.json` | `src/codex-cache.ts` | `mtimeMs + sizeBytes` per Codex `.jsonl`. |
|
||||
| `cursor-results.json` | `src/cursor-cache.ts` | `mtimeMs + sizeBytes` of the Cursor SQLite db. |
|
||||
| `codex-results.v<n>.json` | `src/codex-cache.ts` | `mtimeMs + sizeBytes` per Codex `.jsonl`. Unsuffixed `codex-results.json` is adopted when versions match and never overwritten. |
|
||||
| `cursor-results.v<n>.json` | `src/cursor-cache.ts` | `mtimeMs + sizeBytes` of the Cursor SQLite db. Unsuffixed `cursor-results.json` is adopted when versions match and never overwritten. |
|
||||
| `daily-cache.json` | `src/daily-cache.ts` | Tracks `lastComputedDate`; new days are backfilled, old days are reused. |
|
||||
|
||||
All three use atomic write (temp file + `rename`) and write with mode `0o600`. All three carry a numeric `version` field; bumping it forces a recompute next run.
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ Protobuf. Cascade and response objects map to `ParsedProviderCall` directly.
|
|||
|
||||
## Caching
|
||||
|
||||
Custom file cache at `$CODEBURN_CACHE_DIR/antigravity-results.json` (defaults to `~/.cache/codeburn/`). The cache is also used as the data source when the RPC endpoint is unavailable, not just as an optimization. Bumping the cache version forces a recompute.
|
||||
Custom file cache at `$CODEBURN_CACHE_DIR/antigravity-results.v<n>.json` (defaults to `~/.cache/codeburn/`). The unsuffixed `antigravity-results.json` is left for older binaries; a matching-version copy is adopted once and never overwritten. The cache is also used as the data source when the RPC endpoint is unavailable, not just as an optimization. Bumping the cache version forces a recompute.
|
||||
|
||||
## Deduplication
|
||||
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ The first line read is capped at 1 MB (`FIRST_LINE_READ_CAP`). Codex CLI 0.128+
|
|||
|
||||
## Caching
|
||||
|
||||
`src/codex-cache.ts` writes `~/.cache/codeburn/codex-results.json` (or `$CODEBURN_CACHE_DIR/codex-results.json`). Each entry is keyed by absolute file path and validated against `mtimeMs + sizeBytes`. Cached entries are returned wholesale.
|
||||
`src/codex-cache.ts` writes `~/.cache/codeburn/codex-results.v<n>.json` (or `$CODEBURN_CACHE_DIR/codex-results.v<n>.json`). The unsuffixed `codex-results.json` is left for older binaries; a matching-version copy is adopted once and never overwritten. Each entry is keyed by absolute file path and validated against `mtimeMs + sizeBytes`. Cached entries are returned wholesale.
|
||||
|
||||
A session that yielded zero parseable lines does **not** write to the cache (`codex.ts:419`); this prevents a transient read failure from pinning an empty result against a fingerprint.
|
||||
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ The parser tries both and dedupes via `seenKeys`.
|
|||
|
||||
## Caching
|
||||
|
||||
`src/cursor-cache.ts` writes `~/.cache/codeburn/cursor-results.json` (override with `$CODEBURN_CACHE_DIR`). The fingerprint is `dbMtimeMs + dbSizeBytes` of `state.vscdb`. Atomic write via temp + rename.
|
||||
`src/cursor-cache.ts` writes `~/.cache/codeburn/cursor-results.v<n>.json` (override with `$CODEBURN_CACHE_DIR`). The unsuffixed `cursor-results.json` is left for older binaries; a matching-version copy is adopted once and never overwritten. The fingerprint is `dbMtimeMs + dbSizeBytes` of `state.vscdb`. Atomic write via temp + rename.
|
||||
|
||||
## Deduplication
|
||||
|
||||
|
|
|
|||
|
|
@ -106,7 +106,6 @@ A pseudonymous `device_id` distinguishes your machines without revealing hostnam
|
|||
|
||||
| Field | Example | Description |
|
||||
|---|---|---|
|
||||
| `ai.session_id` | `abc123…` | Session (shares the usage spans' traceId) |
|
||||
| `ai.project` | `my-app` | Project name |
|
||||
| `git.repo` | `github.com/acme/widget` | Normalized `origin` remote (credentials and ports stripped) |
|
||||
| `git.pr_links` | `["…/pull/12"]` | PR URLs captured for the session |
|
||||
|
|
@ -120,7 +119,7 @@ A pseudonymous `device_id` distinguishes your machines without revealing hostnam
|
|||
| `git.in_main` | `true` | Whether the commit landed in the main branch |
|
||||
| `git.was_reverted` | `false` | Whether a later commit reverted it |
|
||||
|
||||
Attribution is **inferred** (timestamp-window correlation, the same heuristic as `codeburn yield`); the resource attribute `codeburn.attribution_methodology: timestamp-window` marks it as such. State transitions (a commit merging to main, or being reverted) are re-sent automatically on later pushes — receivers should upsert commits by `(git.repo, git.sha)` and session spans by `ai.session_id` (latest state wins). When a commit migrates to a later-parsed session with a tighter window, the losing session re-emits with `git.commit_count: 0` (a retraction), so summing `git.commit_count` across upserted session rows never double-counts. Retractions fire only when the commit was won by another session — commits that merely age out of the `--since` window are not retracted, so a previously-synced count stays correct. Session spans also re-emit when an ongoing session's window grows, keeping the span end time current.
|
||||
Attribution is **inferred** (timestamp-window correlation, the same heuristic as `codeburn yield`); the resource attribute `codeburn.attribution_methodology: timestamp-window` marks it as such. State transitions (a commit merging to main, or being reverted) are re-sent automatically on later pushes — receivers should upsert commits by `(git.repo, git.sha)` and session spans by `traceId` (the same id usage spans already carry; latest state wins). When a commit migrates to a later-parsed session with a tighter window, the losing session re-emits with `git.commit_count: 0` (a retraction), so summing `git.commit_count` across upserted session rows never double-counts. Retractions fire only when the commit was won by another session — commits that merely age out of the `--since` window are not retracted, so a previously-synced count stays correct. Session spans also re-emit when an ongoing session's window grows, keeping the span end time current.
|
||||
|
||||
With `--attribution`, normalized repo remote URLs, commit SHAs, commit timestamps (span start times), PR URLs, and the merged/reverted booleans leave your machine — plus the same pseudonymous `codeburn.device_id` resource attribute the usage spans carry. PR links are rebuilt client-side from scheme + host + path only (userinfo, query strings, and fragments are dropped; https, `/org/repo/pull/N` path, bounded length, max 20 per session), and the repo identity itself passes a strict hostname/path allow-list before sending — malformed or transport-helper remotes (`ext::…`, `codecommit::…`) are rejected outright rather than parsed. Precisely what is and is not sent:
|
||||
|
||||
|
|
|
|||
|
|
@ -44,6 +44,16 @@ const MANUAL_ENTRIES = {
|
|||
'deepseek-v4-pro': [4.35e-7, 8.7e-7, 0, 3.625e-9],
|
||||
// Mythos 5 launch pricing; not yet in LiteLLM or the models.dev/OpenRouter gap-fill (Fable is).
|
||||
'claude-mythos-5': [10e-6, 50e-6, 12.5e-6, 1e-6],
|
||||
// gpt-5.6-codex / gpt-5.6-codex-max (#1077): not yet in LiteLLM. Every prior
|
||||
// Codex-suffixed id LiteLLM DOES carry bills identically to its bare-model
|
||||
// sibling of the same generation - gpt-5-codex == gpt-5, gpt-5.1-codex ==
|
||||
// gpt-5.1-codex-max == gpt-5.1, gpt-5.2-codex == gpt-5.2, gpt-5.3-codex ==
|
||||
// gpt-5.3 (all four input/output/cache-write/cache-read rates identical,
|
||||
// verified against the live model_prices_and_context_window.json). Mirroring
|
||||
// that pattern onto gpt-5.6 rather than inventing a number: both ids get the
|
||||
// exact gpt-5.6 tuple (Sol-tier: $5/$30 per million, 1.25x cache-write).
|
||||
'gpt-5.6-codex': [5e-6, 3e-5, 6.25e-6, 5e-7],
|
||||
'gpt-5.6-codex-max': [5e-6, 3e-5, 6.25e-6, 5e-7],
|
||||
}
|
||||
|
||||
const snapshot = {}
|
||||
|
|
|
|||
|
|
@ -22,11 +22,26 @@
|
|||
// which is the part the corpus can honestly establish.
|
||||
// dsh did not exist in the published CLI. Reported; required to be absent
|
||||
// in the baseline and present after the upgrade.
|
||||
// codex PRICING changed by design in #1075: reasoning tokens are billed
|
||||
// inside output rather than on top of it, and cache writes are carved out
|
||||
// of the input bucket. Nothing about what was PARSED moved, so codex keeps
|
||||
// the full exact treatment for the call count and every token field; the
|
||||
// cost tolerance is instead replaced with REPRICE_TOLERANCE — the
|
||||
// upgraded cost must be strictly lower than the baseline and within 25%
|
||||
// of it, since #1075 only ever removes a double-count and never raises
|
||||
// cost — and the delta is reported instead. Drop it from this list once a
|
||||
// published CLI carries the fix.
|
||||
const EXACT = ['claude', 'codex', 'gemini', 'kiro', 'cursor']
|
||||
const CHANGED_BY_DESIGN = ['grok']
|
||||
const COST_CHANGED_BY_DESIGN = ['codex']
|
||||
const NEW_IN_THIS_RELEASE = ['dsh']
|
||||
|
||||
const COST_TOLERANCE = 0.005 // 0.5% relative
|
||||
// #1075 only ever LOWERS codex cost (double-counted reasoning removed, cache
|
||||
// writes carved out of the input bucket) and by a bounded amount on any real
|
||||
// corpus; a rise, or a drop past this bound, means something beyond the known
|
||||
// repricing changed.
|
||||
const REPRICE_TOLERANCE = 0.25 // 25% relative
|
||||
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
|
|
@ -95,13 +110,25 @@ for (const name of providers) {
|
|||
if (b.calls !== u.calls) diffs.push(`calls ${b.calls} != ${u.calls}`)
|
||||
for (const f of TOKEN_FIELDS) if (b[f] !== u[f]) diffs.push(`${f} ${b[f]} != ${u[f]}`)
|
||||
const costDrift = relDiff(b.cost, u.cost)
|
||||
if (costDrift > COST_TOLERANCE) diffs.push(`cost ${fmt(b.cost)} != ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(COST_TOLERANCE * 100).toFixed(1)}%)`)
|
||||
let repriced = false
|
||||
if (COST_CHANGED_BY_DESIGN.includes(name)) {
|
||||
if (u.cost > b.cost) diffs.push(`cost ${fmt(b.cost)} -> ${fmt(u.cost)} rose; #1075 should only lower codex cost`)
|
||||
else if (costDrift > REPRICE_TOLERANCE) diffs.push(`cost ${fmt(b.cost)} -> ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(REPRICE_TOLERANCE * 100).toFixed(0)}% expected bound for #1075)`)
|
||||
else {
|
||||
repriced = true
|
||||
notes.push(`${name}: cost ${fmt(b.cost)} -> ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}%) — repricing expected (#1075); tokens and calls still asserted exactly`)
|
||||
}
|
||||
} else if (costDrift > COST_TOLERANCE) {
|
||||
diffs.push(`cost ${fmt(b.cost)} != ${fmt(u.cost)} (${(costDrift * 100).toFixed(3)}% > ${(COST_TOLERANCE * 100).toFixed(1)}%)`)
|
||||
}
|
||||
if (!EXACT.includes(name)) {
|
||||
notes.push(`${name}: no expectation declared in compare.mjs; ${diffs.length ? diffs.join(', ') : 'identical'}`)
|
||||
verdict = diffs.length ? 'differs (unclassified)' : 'identical'
|
||||
} else if (diffs.length) {
|
||||
failures.push(`${name}: ${diffs.join(', ')}`)
|
||||
verdict = 'DIFFERS'
|
||||
} else if (repriced) {
|
||||
verdict = `repriced (cost ${(costDrift * 100).toFixed(3)}% drift)`
|
||||
} else {
|
||||
verdict = costDrift === 0 ? 'identical' : `identical (cost ${(costDrift * 100).toFixed(3)}% drift)`
|
||||
}
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ const WORK = process.env['UPGRADE_PATH_WORK'] || join(tmpdir(), 'codeburn upgrad
|
|||
const OLD_SESSION_CACHE = 'session-cache.v7.json'
|
||||
const OLD_DAILY_CACHE = 'daily-cache.v17.json'
|
||||
const NEW_SESSION_CACHE_DIR = 'session-cache.v9'
|
||||
const NEW_DAILY_CACHE = 'daily-cache.v20.json'
|
||||
const NEW_DAILY_CACHE = 'daily-cache.v25.json'
|
||||
|
||||
const HOME = join(WORK, 'user home')
|
||||
const PAYLOADS = join(WORK, 'payloads')
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
import { getModelCosts, sanitizeModelForDisplay, type ModelCosts } from './models.js'
|
||||
import { billableOutputTokens, fallbackRawModelDisplayName, getModelCosts, getShortModelName, sanitizeModelForDisplay, type ModelCosts } from './models.js'
|
||||
import { getProvider } from './providers/index.js'
|
||||
import { formatCost, formatTokens } from './format.js'
|
||||
import { renderTable, type TableColumn } from './text-table.js'
|
||||
|
|
@ -112,8 +112,8 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise<AuditR
|
|||
const entry = {
|
||||
displayName: p?.displayName ?? name,
|
||||
formatModel: p
|
||||
? (m: string) => sanitizeModelForDisplay(p.modelDisplayName(m))
|
||||
: sanitizeModelForDisplay,
|
||||
? (m: string) => sanitizeModelForDisplay(fallbackRawModelDisplayName(p.modelDisplayName(m), m))
|
||||
: (m: string) => sanitizeModelForDisplay(getShortModelName(m)),
|
||||
}
|
||||
providerCache.set(name, entry)
|
||||
return entry
|
||||
|
|
@ -124,7 +124,7 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise<AuditR
|
|||
const meta = await resolveProvider(bucket.provider)
|
||||
const displayed = {
|
||||
inputTokens: bucket.raw.inputTokens,
|
||||
outputTokens: bucket.raw.outputTokens + bucket.raw.reasoningTokens,
|
||||
outputTokens: billableOutputTokens(bucket.provider, bucket.raw.outputTokens, bucket.raw.reasoningTokens),
|
||||
cacheWriteTokens: bucket.raw.cacheCreationInputTokens,
|
||||
cacheReadTokens: bucket.cacheReadDisplayed,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { readFile } from 'fs/promises'
|
||||
import { homedir } from 'os'
|
||||
import { join } from 'path'
|
||||
|
||||
|
|
@ -11,3 +12,18 @@ export function getCodeburnCacheDir(): string {
|
|||
const override = process.env['CODEBURN_CACHE_DIR']
|
||||
return override?.trim() ? override : join(homedir(), '.cache', 'codeburn')
|
||||
}
|
||||
|
||||
/** A versioned cache file is the only source when it exists. Legacy adoption is ENOENT-only. */
|
||||
export type ExistingTextFile =
|
||||
| { status: 'absent' }
|
||||
| { status: 'unreadable' }
|
||||
| { status: 'ok'; text: string }
|
||||
|
||||
export async function readExistingTextFile(path: string): Promise<ExistingTextFile> {
|
||||
try {
|
||||
return { status: 'ok', text: await readFile(path, 'utf-8') }
|
||||
} catch (err) {
|
||||
const code = err && typeof err === 'object' && 'code' in err ? (err as { code?: string }).code : undefined
|
||||
return { status: code === 'ENOENT' ? 'absent' : 'unreadable' }
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { randomBytes } from 'crypto'
|
|||
import { join, resolve } from 'path'
|
||||
import { AsyncLocalStorage } from 'node:async_hooks'
|
||||
|
||||
import { getCodeburnCacheDir } from './cache-dir.js'
|
||||
import { getCodeburnCacheDir, readExistingTextFile } from './cache-dir.js'
|
||||
import type { ParsedProviderCall } from './providers/types.js'
|
||||
|
||||
// v4: attribute MCP calls emitted as event_msg/mcp_tool_call_end (issue #478).
|
||||
|
|
@ -23,8 +23,31 @@ import type { ParsedProviderCall } from './providers/types.js'
|
|||
// cannot overwrite the model selected by turn_context.
|
||||
// v10: same depth-1 window for the rest of session_meta's raw string fields
|
||||
// (cwd/name/originator/session_id/forked_from_id/model_provider).
|
||||
const CODEX_CACHE_VERSION = 10
|
||||
const CACHE_FILE = 'codex-results.json'
|
||||
// v11: codex pricing fix (#1075) - reasoning is no longer added on top of
|
||||
// output, and cache_write_input_tokens is carved out of the input bucket. This
|
||||
// file stores each call's costUSD and token buckets verbatim, so entries
|
||||
// written by v10 carry the old (overstated) cost and must be re-derived.
|
||||
// v13: codex throughput fix (#1079) - activeGeneratedTokens was summing
|
||||
// output + reasoning, the same double-count Fix A removed from cost. This
|
||||
// file stores activeGeneratedTokens/activeDurationMs/toolWaitMs verbatim (not
|
||||
// re-derived on read), so v11 entries carry the overstated numerator and must
|
||||
// re-parse. Not 12: v12 is claimed by feat/core-extraction's own port of this
|
||||
// throughput feature (PR #1086), so reusing it would let two incompatible
|
||||
// schemas share a filename.
|
||||
// v14: MCP + Skill attribution for the shapes the classic `function_call` path
|
||||
// never reached (#478) - the `exec` custom tool's `input` program and the item
|
||||
// model's `item_completed`/`CommandExecution` item - plus SKILL.md reads landing
|
||||
// in `skills`. This file stores each call's `tools`/`toolSequence`/`skills`
|
||||
// verbatim (they are passed through on read, never re-derived), so v13 entries
|
||||
// keep the old, MCP- and skill-less attribution until they re-parse.
|
||||
// v15: builtin alias prices `codex-auto-review` (#1047). Exact-hit cache
|
||||
// entries still hold the pre-alias $0; bump so unchanged rollouts reprice.
|
||||
// Must be max(main v14 #1092, this)+1 — #1092 spent v14 on MCP/skills.
|
||||
export const CODEX_CACHE_VERSION = 15
|
||||
export const CODEX_LEGACY_CACHE_FILE = 'codex-results.json'
|
||||
export function codexCacheFileName(version = CODEX_CACHE_VERSION): string {
|
||||
return `codex-results.v${version}.json`
|
||||
}
|
||||
|
||||
export type CodexFileFingerprint = { dev: number; ino: number; mtimeMs: number; sizeBytes: number }
|
||||
type FileFingerprint = CodexFileFingerprint
|
||||
|
|
@ -70,7 +93,15 @@ export function withCodexCacheDirectory<T>(cacheDir: string, operation: () => T)
|
|||
}
|
||||
|
||||
function getCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, CACHE_FILE)
|
||||
return join(cacheDir, codexCacheFileName())
|
||||
}
|
||||
|
||||
function getLegacyCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, CODEX_LEGACY_CACHE_FILE)
|
||||
}
|
||||
|
||||
function isCurrentCache(cache: ResultCache): boolean {
|
||||
return cache.version === CODEX_CACHE_VERSION && !!cache.files && typeof cache.files === 'object'
|
||||
}
|
||||
|
||||
// Embedded consumers can change CODEBURN_CACHE_DIR without reloading this
|
||||
|
|
@ -87,15 +118,33 @@ export function clearCodexMemCaches(): void {
|
|||
async function loadCache(cacheDir: string): Promise<ResultCache> {
|
||||
const inMemory = memCaches.get(cacheDir)
|
||||
if (inMemory) return inMemory
|
||||
const empty = { version: CODEX_CACHE_VERSION, files: {} }
|
||||
const versioned = await readExistingTextFile(getCachePath(cacheDir))
|
||||
if (versioned.status === 'ok') {
|
||||
try {
|
||||
const cache = JSON.parse(versioned.text) as ResultCache
|
||||
if (isCurrentCache(cache)) {
|
||||
memCaches.set(cacheDir, cache)
|
||||
return cache
|
||||
}
|
||||
} catch {}
|
||||
memCaches.set(cacheDir, empty)
|
||||
return empty
|
||||
}
|
||||
if (versioned.status === 'unreadable') {
|
||||
memCaches.set(cacheDir, empty)
|
||||
return empty
|
||||
}
|
||||
// Versioned file is absent (ENOENT). Adopt the unsuffixed file only when its
|
||||
// version matches — old binaries still own that path; we never write or delete it.
|
||||
try {
|
||||
const raw = await readFile(getCachePath(cacheDir), 'utf-8')
|
||||
const raw = await readFile(getLegacyCachePath(cacheDir), 'utf-8')
|
||||
const cache = JSON.parse(raw) as ResultCache
|
||||
if (cache.version === CODEX_CACHE_VERSION && cache.files && typeof cache.files === 'object') {
|
||||
if (isCurrentCache(cache)) {
|
||||
memCaches.set(cacheDir, cache)
|
||||
return cache
|
||||
}
|
||||
} catch {}
|
||||
const empty = { version: CODEX_CACHE_VERSION, files: {} }
|
||||
memCaches.set(cacheDir, empty)
|
||||
return empty
|
||||
}
|
||||
|
|
|
|||
|
|
@ -20,10 +20,19 @@ const CREDITS_PER_MILLION: Record<string, CodexCreditRate> = {
|
|||
'gpt-5.4-mini': { input: 18.75, cachedInput: 1.875, output: 113 },
|
||||
}
|
||||
|
||||
// Activity surfaces keep their product id on the call (display stays
|
||||
// "Codex Auto Review"). Credits must follow the same underlying model
|
||||
// BUILTIN_ALIASES uses for USD. Keep this table in lockstep with
|
||||
// `codex-auto-review` in src/models.ts.
|
||||
const ACTIVITY_CREDIT_MODELS: Record<string, string> = {
|
||||
'codex-auto-review': 'gpt-5.5',
|
||||
}
|
||||
|
||||
/// Resolve the credit rate for a Codex model name, tolerating suffix variants
|
||||
/// (e.g. "gpt-5.5-codex"). Returns null when the model has no known credit rate.
|
||||
export function codexCreditRate(model: string): CodexCreditRate | null {
|
||||
const m = model.toLowerCase()
|
||||
const mapped = ACTIVITY_CREDIT_MODELS[model] ?? ACTIVITY_CREDIT_MODELS[model.toLowerCase()]
|
||||
const m = (mapped ?? model).toLowerCase()
|
||||
if (m.includes('5.4') && m.includes('mini')) return CREDITS_PER_MILLION['gpt-5.4-mini']!
|
||||
if (m.includes('5.4')) return CREDITS_PER_MILLION['gpt-5.4']!
|
||||
if (m.includes('5.5')) return CREDITS_PER_MILLION['gpt-5.5']!
|
||||
|
|
@ -36,9 +45,9 @@ export type CodexCreditTokens = {
|
|||
inputTokens: number
|
||||
/// Cache-read (cached input) tokens, billed at the cheaper cached rate.
|
||||
cachedReadTokens: number
|
||||
/// Billable output tokens: reasoning is already included (billableOutputTokens
|
||||
/// in models.ts), so callers must not add it on top here.
|
||||
outputTokens: number
|
||||
/// Reasoning tokens are billed as output, matching CodeBurn's cost model.
|
||||
reasoningTokens?: number
|
||||
}
|
||||
|
||||
/// Credits consumed for one Codex usage record. Returns null when the model has
|
||||
|
|
@ -48,10 +57,9 @@ export function codexCredits(model: string, tokens: CodexCreditTokens): number |
|
|||
if (!rate) return null
|
||||
const safe = (n: number) => (Number.isFinite(n) && n > 0 ? n : 0)
|
||||
const PER_MILLION = 1_000_000
|
||||
const output = safe(tokens.outputTokens) + safe(tokens.reasoningTokens ?? 0)
|
||||
return (
|
||||
(safe(tokens.inputTokens) / PER_MILLION) * rate.input +
|
||||
(safe(tokens.cachedReadTokens) / PER_MILLION) * rate.cachedInput +
|
||||
(output / PER_MILLION) * rate.output
|
||||
(safe(tokens.outputTokens) / PER_MILLION) * rate.output
|
||||
)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
import { open, stat } from 'node:fs/promises'
|
||||
import { StringDecoder } from 'node:string_decoder'
|
||||
|
||||
import { billableOutputTokens } from './models.js'
|
||||
|
||||
export type CodexThroughputPoint = {
|
||||
timestamp: string
|
||||
model?: string
|
||||
|
|
@ -105,7 +107,12 @@ function durationMs(payload: RolloutLine['payload']): number | undefined {
|
|||
return undefined
|
||||
}
|
||||
|
||||
function mergeToolIntervals(intervals: Array<[number, number]>, durationMs: number, taskStartedAt?: number, taskCompletedAt?: number): number {
|
||||
// Shared with src/providers/codex.ts (#1088 BUG-8): both clip a task's tool
|
||||
// intervals to its [taskStartedAt, taskStartedAt + durationMs] window, merge
|
||||
// overlaps, and cap the sum at durationMs. Was copy-pasted inline in
|
||||
// providers/codex.ts and had already drifted (duration parsing there accepted
|
||||
// only a plain `duration_ms` number); one copy now, called from both.
|
||||
export function mergeToolIntervals(intervals: Array<[number, number]>, durationMs: number, taskStartedAt?: number, taskCompletedAt?: number): number {
|
||||
const windowStart = taskStartedAt ?? (taskCompletedAt !== undefined ? taskCompletedAt - durationMs : undefined)
|
||||
const windowEnd = windowStart !== undefined ? windowStart + durationMs : undefined
|
||||
const clipped = intervals.map(([start, end]) => [
|
||||
|
|
@ -400,7 +407,9 @@ export class CodexThroughputReader {
|
|||
state.previousOutput = total?.output_tokens ?? state.previousOutput
|
||||
state.previousReasoning = total?.reasoning_output_tokens ?? state.previousReasoning
|
||||
}
|
||||
const generatedTokens = outputTokens + reasoningTokens
|
||||
// Reasoning is already inside output_tokens (#1075/#1078); same numerator
|
||||
// as the cost path so live Tok/s can't drift from billed tokens (#1079).
|
||||
const generatedTokens = billableOutputTokens('codex', outputTokens, reasoningTokens)
|
||||
if (generatedTokens <= 0) return
|
||||
const timestampMs = Date.parse(entry.timestamp)
|
||||
if (!Number.isFinite(timestampMs)) return
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { readFile, writeFile, mkdir, rename, stat, unlink } from 'fs/promises'
|
|||
import { join } from 'path'
|
||||
import { randomBytes } from 'crypto'
|
||||
|
||||
import { getCodeburnCacheDir } from './cache-dir.js'
|
||||
import { getCodeburnCacheDir, readExistingTextFile } from './cache-dir.js'
|
||||
import type { ParsedProviderCall } from './providers/types.js'
|
||||
|
||||
// Bumped to 3 for the workspace-aware breakdown change: the cursor parser
|
||||
|
|
@ -19,7 +19,11 @@ import type { ParsedProviderCall } from './providers/types.js'
|
|||
// (cursor:composer-input:<id>) with per-conversation source selection, the
|
||||
// agent stream regained tool/system context and stream-only sessions, and
|
||||
// tool names are canonicalized. v5 results mix crediting regimes.
|
||||
const CURSOR_CACHE_VERSION = 6
|
||||
export const CURSOR_CACHE_VERSION = 6
|
||||
export const CURSOR_LEGACY_CACHE_FILE = 'cursor-results.json'
|
||||
export function cursorCacheFileName(version = CURSOR_CACHE_VERSION): string {
|
||||
return `cursor-results.v${version}.json`
|
||||
}
|
||||
|
||||
type ResultCache = {
|
||||
version?: number
|
||||
|
|
@ -29,10 +33,30 @@ type ResultCache = {
|
|||
calls: ParsedProviderCall[]
|
||||
}
|
||||
|
||||
const CACHE_FILE = 'cursor-results.json'
|
||||
|
||||
function getCachePath(): string {
|
||||
return join(getCodeburnCacheDir(), CACHE_FILE)
|
||||
return join(getCodeburnCacheDir(), cursorCacheFileName())
|
||||
}
|
||||
|
||||
function getLegacyCachePath(): string {
|
||||
return join(getCodeburnCacheDir(), CURSOR_LEGACY_CACHE_FILE)
|
||||
}
|
||||
|
||||
function isCurrentHit(cache: ResultCache, fp: { mtimeMs: number; size: number }, requestedFloor: string): boolean {
|
||||
return (
|
||||
cache.version === CURSOR_CACHE_VERSION
|
||||
&& cache.dbMtimeMs === fp.mtimeMs
|
||||
&& cache.dbSizeBytes === fp.size
|
||||
&& typeof cache.lookbackFloor === 'string'
|
||||
&& cache.lookbackFloor <= requestedFloor
|
||||
)
|
||||
}
|
||||
|
||||
async function readCacheFile(path: string): Promise<ResultCache | null> {
|
||||
try {
|
||||
const cache = JSON.parse(await readFile(path, 'utf-8')) as ResultCache
|
||||
if (cache && typeof cache === 'object') return cache
|
||||
} catch {}
|
||||
return null
|
||||
}
|
||||
|
||||
async function getDbFingerprint(dbPath: string): Promise<{ mtimeMs: number; size: number } | null> {
|
||||
|
|
@ -52,18 +76,20 @@ export async function readCachedResults(
|
|||
const fp = await getDbFingerprint(dbPath)
|
||||
if (!fp) return null
|
||||
|
||||
const raw = await readFile(getCachePath(), 'utf-8')
|
||||
const cache = JSON.parse(raw) as ResultCache
|
||||
|
||||
if (
|
||||
cache.version === CURSOR_CACHE_VERSION &&
|
||||
cache.dbMtimeMs === fp.mtimeMs &&
|
||||
cache.dbSizeBytes === fp.size &&
|
||||
typeof cache.lookbackFloor === 'string' &&
|
||||
cache.lookbackFloor <= requestedFloor
|
||||
) {
|
||||
return cache.calls
|
||||
const versioned = await readExistingTextFile(getCachePath())
|
||||
if (versioned.status === 'ok') {
|
||||
try {
|
||||
const cache = JSON.parse(versioned.text) as ResultCache
|
||||
if (cache && typeof cache === 'object' && isCurrentHit(cache, fp, requestedFloor)) return cache.calls
|
||||
} catch {}
|
||||
return null
|
||||
}
|
||||
if (versioned.status === 'unreadable') return null
|
||||
|
||||
// Versioned file is absent (ENOENT). Adopt the unsuffixed file only when its
|
||||
// version and fingerprint match — old binaries still own that path.
|
||||
const legacy = await readCacheFile(getLegacyCachePath())
|
||||
if (legacy && isCurrentHit(legacy, fp, requestedFloor)) return legacy.calls
|
||||
return null
|
||||
} catch {
|
||||
return null
|
||||
|
|
|
|||
|
|
@ -6,6 +6,15 @@ import { join } from 'path'
|
|||
import { getCodeburnCacheDir } from './cache-dir.js'
|
||||
import type { DateRange, ProjectSummary } from './types.js'
|
||||
|
||||
// Bumped to 25: `codex-auto-review` now prices as the recommended GPT-5.5
|
||||
// row (#1047). Days already finalized under v24 keep that id at $0
|
||||
// forever unless MIN_SUPPORTED_VERSION moves: the daily cache has no
|
||||
// per-provider invalidation. The Codex parse version and CODEX_CACHE_VERSION
|
||||
// move with this so the lower caches reprice first; this pass then re-derives
|
||||
// ALL days from the warm session cache (seconds, not a full re-parse).
|
||||
// adoptOlderDailyCaches keeps the superseded file as the baseline. v21 is
|
||||
// #946, v22 was this PR's earlier claim, v23 is #1075, v24 is #1090.
|
||||
//
|
||||
// Bumped to 20: the Codex fast-path read a nested
|
||||
// `base_instructions.provenance.model` out of `session_meta` as if it were
|
||||
// `payload.model` (#1040), so every call a rollout attributed from session
|
||||
|
|
@ -110,8 +119,28 @@ import type { DateRange, ProjectSummary } from './types.js'
|
|||
// that older binaries skipped. v8 added local-model savings to the daily
|
||||
// rollup; the `savingsConfigHash` field is invalidated separately when the
|
||||
// user changes their `localModelSavings` mapping.
|
||||
export const DAILY_CACHE_VERSION = 20
|
||||
const MIN_SUPPORTED_VERSION = 20
|
||||
// v23: codex pricing fix (#1075) - reasoning tokens were billed on top of
|
||||
// output (they are a subset of it) and cache_write_input_tokens was ignored, so
|
||||
// days finalized at v20 carry codex costs overstated by ~3.5% and codex output
|
||||
// tokens overstated by ~34.6%. Raising MIN_SUPPORTED_VERSION forces the
|
||||
// one-time re-derivation.
|
||||
// It takes 23, not 21: v21 is claimed by the #946 landing branch and v22 by
|
||||
// PR #1056, so those numbers are spoken for and reusing one would let two
|
||||
// incompatible schemas share a filename. (feat/core-extraction sits at 26 and
|
||||
// reconciles at its final merge by keeping the max.)
|
||||
//
|
||||
// v24: gpt-5.6-codex / gpt-5.6-codex-max pricing (#1077) - added as explicit
|
||||
// litellm-snapshot.json rows. getModelCosts already resolved both ids to the
|
||||
// correct rate via the `gpt-5.6` prefix fallback before this landed, so a day
|
||||
// finalized on any binary that had a `gpt-5.6` snapshot row already carries
|
||||
// the right cost; this bump only matters for a day finalized before THAT (a
|
||||
// window where neither existed). The daily cache has no per-provider
|
||||
// invalidation, so there is no way to tell those days apart from here -
|
||||
// raising MIN_SUPPORTED_VERSION forces the one-time re-derivation for
|
||||
// everyone, which is a lossless no-op for days already correct.
|
||||
// v25: #1047 activity-id pricing. v24 on main already shipped #1090.
|
||||
export const DAILY_CACHE_VERSION = 25
|
||||
const MIN_SUPPORTED_VERSION = 25
|
||||
// Version-suffixed so different binaries each own a distinct file and never
|
||||
// clobber an incompatible schema. Bumping the version mints a fresh filename;
|
||||
// adoptOlderDailyCaches then unions days out of every previous file (including
|
||||
|
|
|
|||
|
|
@ -746,6 +746,11 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
|
|||
const anyEstimated = Object.values(modelTotals).some(d => d.estimatedCostUSD > 0)
|
||||
const sorted = Object.entries(modelTotals).sort(([, a], [, b]) => b.costUSD - a.costUSD)
|
||||
const costLabels = sorted.map(([, data]) => markEstimated(formatCost(data.costUSD), data.estimatedCostUSD > 0))
|
||||
// #1088: this column has zero width slack left at the standard 3-column
|
||||
// breakpoint (verified: widening the header even one character clips
|
||||
// 'cache'/'1-shot' and drops the value column entirely), so the header stays
|
||||
// "Tok/s" -- the legend below carries the "Effective Tok/s" framing and the
|
||||
// caveat that it is not a vendor decode-speed figure.
|
||||
const headers = ['cost', 'cache', 'calls', '1-shot', 'Tok/s']
|
||||
const values = sorted.map(([model, data], index) => {
|
||||
const totalInput = data.freshInput + data.cacheRead + data.cacheWrite
|
||||
|
|
@ -806,7 +811,7 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
|
|||
{anyEstimated && (
|
||||
<Text dimColor wrap="truncate-end">~ estimated cost (priced from estimated tokens)</Text>
|
||||
)}
|
||||
<Text dimColor wrap="truncate-end">~ Tok/s: generated tokens / active time; tool wait excluded</Text>
|
||||
<Text dimColor wrap="truncate-end">~ Effective Tok/s: generated tokens ÷ time the agent spent waiting on the model, tool execution excluded. Includes prefill, request assembly and reasoning. Not comparable to vendor decode-speed figures.</Text>
|
||||
</Panel>
|
||||
)
|
||||
}
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -64,6 +64,7 @@ type SessionLabelEntry = {
|
|||
key: string
|
||||
info: SessionLabelInfo
|
||||
baseLabel: string
|
||||
idFirst: string
|
||||
}
|
||||
|
||||
function nonNegative(value: number): number {
|
||||
|
|
@ -174,19 +175,41 @@ function preferredSessionTitle(titleCandidates: Map<string, SessionTitleCandidat
|
|||
return cleaned[0]?.title
|
||||
}
|
||||
|
||||
// Chart legend is `max-w-40` at 10px ≈ 24–32 glyphs. A title that is unique in
|
||||
// that window can lead; otherwise the short id must stay in the prefix or
|
||||
// truncated series look identical (the #997 class). Count code points, matching
|
||||
// the title cap — a UTF-16 slice can split an emoji and collide two titles.
|
||||
const VISIBLE_LEGEND_PREFIX = 24
|
||||
|
||||
function visibleLegendPrefix(label: string): string {
|
||||
return Array.from(label).slice(0, VISIBLE_LEGEND_PREFIX).join('')
|
||||
}
|
||||
|
||||
function buildSessionLabels(inputs: Map<string, SessionLabelInfo>): Map<string, string> {
|
||||
// Stable raw-key order makes the residual used-label guard independent of
|
||||
// project/session discovery order when a title happens to match another
|
||||
// label shape.
|
||||
const entries: SessionLabelEntry[] = [...inputs.entries()].map(([key, info]) => {
|
||||
const sessionLabel = preferredSessionTitle(info.titleCandidates)
|
||||
?? shortProjectLabel(info.projectPath, preferredProjectName(info.projectNames))
|
||||
return {
|
||||
key,
|
||||
info,
|
||||
baseLabel: `${shortSessionId(info.sessionId)} (${info.provider}) · ${sessionLabel}`,
|
||||
}
|
||||
const draft: SessionLabelEntry[] = [...inputs.entries()].map(([key, info]) => {
|
||||
const title = preferredSessionTitle(info.titleCandidates)
|
||||
const project = shortProjectLabel(info.projectPath, preferredProjectName(info.projectNames))
|
||||
const shortId = shortSessionId(info.sessionId)
|
||||
const idFirst = title
|
||||
? `${shortId} (${info.provider}) · ${title}`
|
||||
: `${shortId} (${info.provider}) · ${project}`
|
||||
const titleFirst = title ? `${title} (${info.provider})` : idFirst
|
||||
return { key, info, baseLabel: titleFirst, idFirst }
|
||||
}).sort((a, b) => a.key < b.key ? -1 : a.key > b.key ? 1 : 0)
|
||||
|
||||
const titlePrefixCounts = new Map<string, number>()
|
||||
for (const entry of draft) {
|
||||
const prefix = visibleLegendPrefix(entry.baseLabel)
|
||||
titlePrefixCounts.set(prefix, (titlePrefixCounts.get(prefix) ?? 0) + 1)
|
||||
}
|
||||
const entries: SessionLabelEntry[] = draft.map(entry => {
|
||||
const prefix = visibleLegendPrefix(entry.baseLabel)
|
||||
const titleLeads = (titlePrefixCounts.get(prefix) ?? 0) === 1
|
||||
return { ...entry, baseLabel: titleLeads ? entry.baseLabel : entry.idFirst }
|
||||
})
|
||||
const byBaseLabel = new Map<string, SessionLabelEntry[]>()
|
||||
for (const entry of entries) {
|
||||
const group = byBaseLabel.get(entry.baseLabel) ?? []
|
||||
|
|
|
|||
80
src/main.ts
80
src/main.ts
|
|
@ -474,7 +474,7 @@ program.hook('preAction', async (thisCommand) => {
|
|||
await loadCurrency()
|
||||
})
|
||||
|
||||
function buildJsonReport(projects: ProjectSummary[], period: string, periodKey: string, durable?: DurablePeriod) {
|
||||
function buildJsonReport(projects: ProjectSummary[], period: string, periodKey: string, durable: DurablePeriod) {
|
||||
const sessions = projects.flatMap(p => p.sessions)
|
||||
const { code } = getCurrency()
|
||||
|
||||
|
|
@ -482,67 +482,28 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey:
|
|||
// session files have expired still count), matching the menubar exactly. The
|
||||
// proxied/net split is a surviving-session concept (subscription attribution
|
||||
// isn't stored per day), so it stays live; net is taken off the durable total.
|
||||
const totalCostUSD = durable ? durable.data.cost : projects.reduce((s, p) => s + p.totalCostUSD, 0)
|
||||
const totalSavingsUSD = durable ? durable.data.savingsUSD : projects.reduce((s, p) => s + p.totalSavingsUSD, 0)
|
||||
const totalEstimatedUSD = durable ? (durable.data.estimatedCostUSD ?? 0) : projects.reduce((s, p) => s + (p.totalEstimatedCostUSD ?? 0), 0)
|
||||
const totalCostUSD = durable.data.cost
|
||||
const totalSavingsUSD = durable.data.savingsUSD
|
||||
const totalEstimatedUSD = durable.data.estimatedCostUSD ?? 0
|
||||
// Subscription-covered (proxied) portion of totalCostUSD, and the resulting
|
||||
// out-of-pocket figure. `cost` stays the full billable/would-be amount.
|
||||
const totalProxiedUSD = projects.reduce((s, p) => s + p.totalProxiedCostUSD, 0)
|
||||
const netCostUSD = totalCostUSD - totalProxiedUSD
|
||||
const totalCalls = durable ? durable.data.calls : projects.reduce((s, p) => s + p.totalApiCalls, 0)
|
||||
const totalSessions = durable ? durable.data.sessions : projects.reduce((s, p) => s + p.sessions.length, 0)
|
||||
const totalInput = durable ? durable.data.inputTokens : sessions.reduce((s, sess) => s + sess.totalInputTokens, 0)
|
||||
const totalOutput = durable ? durable.data.outputTokens : sessions.reduce((s, sess) => s + sess.totalOutputTokens, 0)
|
||||
const totalCacheRead = durable ? durable.data.cacheReadTokens : sessions.reduce((s, sess) => s + sess.totalCacheReadTokens, 0)
|
||||
const totalCacheWrite = durable ? durable.data.cacheWriteTokens : sessions.reduce((s, sess) => s + sess.totalCacheWriteTokens, 0)
|
||||
const totalCalls = durable.data.calls
|
||||
const totalSessions = durable.data.sessions
|
||||
const totalInput = durable.data.inputTokens
|
||||
const totalOutput = durable.data.outputTokens
|
||||
const totalCacheRead = durable.data.cacheReadTokens
|
||||
const totalCacheWrite = durable.data.cacheWriteTokens
|
||||
// Match src/menubar-json.ts:cacheHitPercent: reads over reads+fresh-input. cache_write
|
||||
// counts tokens being stored, not served, so it doesn't belong in the denominator.
|
||||
const cacheHitDenom = totalInput + totalCacheRead
|
||||
const cacheHitPercent = cacheHitDenom > 0 ? Math.round((totalCacheRead / cacheHitDenom) * 1000) / 10 : 0
|
||||
|
||||
// Per-day rollup. Mirrors parser.ts categoryBreakdown semantics so a
|
||||
// consumer summing daily[].editTurns over a period gets the same total as
|
||||
// sum(activities[].editTurns) for that period: every turn counts once for
|
||||
// `turns`, edit turns count for `editTurns`, edit turns with zero retries
|
||||
// count for `oneShotTurns`. Issue #279 — daily-resolution efficiency
|
||||
// dashboards need this without re-deriving from activity-level rollups.
|
||||
const dailyMap: Record<string, { cost: number; savings: number; calls: number; turns: number; editTurns: number; oneShotTurns: number }> = {}
|
||||
for (const sess of sessions) {
|
||||
for (const turn of sess.turns) {
|
||||
// Prefer the user-message timestamp on the turn; fall back to the first
|
||||
// assistant-call timestamp when the user line is missing (continuation
|
||||
// sessions where the JSONL begins mid-conversation). Previously these
|
||||
// turns dropped from daily but stayed in activities, breaking the
|
||||
// sum(daily[].editTurns) === sum(activities[].editTurns) invariant.
|
||||
const ts = turn.timestamp || turn.assistantCalls[0]?.timestamp
|
||||
if (!ts) { continue }
|
||||
const day = dateKey(ts)
|
||||
if (!dailyMap[day]) { dailyMap[day] = { cost: 0, savings: 0, calls: 0, turns: 0, editTurns: 0, oneShotTurns: 0 } }
|
||||
dailyMap[day].turns += 1
|
||||
if (turn.hasEdits) {
|
||||
dailyMap[day].editTurns += 1
|
||||
if (turn.retries === 0) dailyMap[day].oneShotTurns += 1
|
||||
}
|
||||
for (const call of turn.assistantCalls) {
|
||||
// Cost/savings/calls bucket under each call's OWN day — the same
|
||||
// per-call rule as the durable day set (day-aggregator.ts), so this
|
||||
// fallback and durable.days never diverge on a midnight-straddling
|
||||
// turn (issue #852). Turn counts/edit stats stay anchored on the
|
||||
// turn's day above. An unparseable call timestamp falls back to the
|
||||
// turn's day rather than producing a garbage date key.
|
||||
const callDay = Number.isNaN(new Date(call.timestamp).getTime()) ? day : dateKey(call.timestamp)
|
||||
if (!dailyMap[callDay]) { dailyMap[callDay] = { cost: 0, savings: 0, calls: 0, turns: 0, editTurns: 0, oneShotTurns: 0 } }
|
||||
dailyMap[callDay].cost += call.costUSD
|
||||
dailyMap[callDay].savings += call.savingsUSD ?? 0
|
||||
dailyMap[callDay].calls += 1
|
||||
}
|
||||
}
|
||||
}
|
||||
// Daily rows come from the same durable day set as the headline so they sum
|
||||
// to it, carried days included. The live per-turn rollup (dailyMap) is only
|
||||
// the fallback for callers that pass no durable period.
|
||||
const daily = durable
|
||||
? durable.days.map(d => {
|
||||
// to it, carried days included. Both JSON call sites always pass durable
|
||||
// (#1067); the live dailyMap fallback was unreachable and is gone.
|
||||
const daily = durable.days.map(d => {
|
||||
const turns = Object.values(d.categories).reduce((s, c) => s + c.turns, 0)
|
||||
return {
|
||||
date: d.date,
|
||||
|
|
@ -557,21 +518,6 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey:
|
|||
: null,
|
||||
}
|
||||
})
|
||||
: Object.entries(dailyMap).sort().map(([date, d]) => ({
|
||||
date,
|
||||
cost: convertCost(d.cost),
|
||||
savings: convertCost(d.savings),
|
||||
calls: d.calls,
|
||||
turns: d.turns,
|
||||
editTurns: d.editTurns,
|
||||
oneShotTurns: d.oneShotTurns,
|
||||
// Pre-computed convenience for dashboards that don't want to do the math.
|
||||
// null when there are no edit turns (the rate is undefined, not zero —
|
||||
// a day where the user only had Q&A turns shouldn't read as 0% one-shot).
|
||||
oneShotRate: d.editTurns > 0
|
||||
? Math.round((d.oneShotTurns / d.editTurns) * 1000) / 10
|
||||
: null,
|
||||
}))
|
||||
|
||||
const projectList = projects.map(p => ({
|
||||
name: p.project,
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ import stripAnsi from 'strip-ansi'
|
|||
|
||||
import { codexCredits } from './codex-credits.js'
|
||||
import { formatCost, formatTokens } from './format.js'
|
||||
import { sanitizeModelForDisplay } from './models.js'
|
||||
import { billableOutputTokens, fallbackRawModelDisplayName, getShortModelName, resolveCanonicalModelId, sanitizeModelForDisplay } from './models.js'
|
||||
import { getProvider } from './providers/index.js'
|
||||
import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
|
||||
|
||||
|
|
@ -28,8 +28,12 @@ export type ModelReportRow = {
|
|||
savingsBaselineModel: string
|
||||
calls: number
|
||||
/// Codex credit consumption (issues #408/#495). null for non-Codex models or
|
||||
/// Codex models without a known credit rate.
|
||||
/// Codex models without a known credit rate. A merged row that mixed rated
|
||||
/// and unrated buckets stores the partial sum of the rated ones.
|
||||
credits: number | null
|
||||
/// True when `credits` is a partial sum because some contributing buckets
|
||||
/// had no known credit rate.
|
||||
creditsIncomplete?: boolean
|
||||
topCategory?: TaskCategory
|
||||
topCategoryCost?: number
|
||||
topCategoryShare?: number
|
||||
|
|
@ -73,9 +77,10 @@ function bucketKey(provider: string, model: string, category: TaskCategory | nul
|
|||
}
|
||||
|
||||
/// Walks every parsed turn, attributes each assistant call to a
|
||||
/// (provider, model, category, agent) bucket, and returns rows keyed by
|
||||
/// (provider, model) by default, (provider, model, category) under `byTask`, or
|
||||
/// (provider, model, agent) under `byAgent`.
|
||||
/// (provider, raw-model, category, agent) bucket, then merges buckets that
|
||||
/// resolve to the same provider + alias-resolved canonical id. Display names
|
||||
/// stay cosmetic. Returned rows are keyed by (provider, canonical id) by
|
||||
/// default, plus category under `byTask` or agent under `byAgent`.
|
||||
///
|
||||
/// Default view: rows sorted by cost descending.
|
||||
/// byTask / byAgent view: rows grouped by (provider, model) so the renderer can
|
||||
|
|
@ -120,7 +125,7 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat
|
|||
buckets.set(key, bucket)
|
||||
}
|
||||
bucket.inputTokens += call.usage.inputTokens
|
||||
bucket.outputTokens += call.usage.outputTokens + call.usage.reasoningTokens
|
||||
bucket.outputTokens += billableOutputTokens(provider, call.usage.outputTokens, call.usage.reasoningTokens)
|
||||
bucket.cacheWriteTokens += call.usage.cacheCreationInputTokens
|
||||
// cacheReadInputTokens (Anthropic vocab) and cachedInputTokens (OpenAI vocab)
|
||||
// are two names for the same thing. Providers populate one or set both to the
|
||||
|
|
@ -155,74 +160,133 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat
|
|||
const entry = {
|
||||
displayName: p?.displayName ?? name,
|
||||
formatModel: p
|
||||
? (m: string) => sanitizeModelForDisplay(p.modelDisplayName(m))
|
||||
: sanitizeModelForDisplay,
|
||||
? (m: string) => sanitizeModelForDisplay(fallbackRawModelDisplayName(p.modelDisplayName(m), m))
|
||||
: (m: string) => sanitizeModelForDisplay(getShortModelName(m)),
|
||||
}
|
||||
providerCache.set(name, entry)
|
||||
return entry
|
||||
}
|
||||
|
||||
const rows: ModelReportRow[] = []
|
||||
const rowsByKey = new Map<string, ModelReportRow>()
|
||||
const foldedCategoryCost = new Map<string, Map<CategoryKey, number>>()
|
||||
const foldedTotalCost = new Map<string, number>()
|
||||
const foldedRawSeen = new Set<string>()
|
||||
// Empty string on the row means both "none seen" and "conflict". Track
|
||||
// distinct non-empty baselines separately so a later bucket cannot
|
||||
// repopulate a conflict that was already detected.
|
||||
const baselinesByKey = new Map<string, Set<string>>()
|
||||
|
||||
for (const bucket of buckets.values()) {
|
||||
const meta = await resolveProvider(bucket.provider)
|
||||
const modelDisplayName = meta.formatModel(bucket.model)
|
||||
const canonicalId = resolveCanonicalModelId(bucket.model)
|
||||
const resolvedKey = bucketKey(bucket.provider, canonicalId, bucket.category, bucket.agentType)
|
||||
const foldKey = `${bucket.provider} ${canonicalId}`
|
||||
const total = bucket.inputTokens + bucket.outputTokens + bucket.cacheWriteTokens + bucket.cacheReadTokens
|
||||
const row: ModelReportRow = {
|
||||
provider: bucket.provider,
|
||||
providerDisplayName: meta.displayName,
|
||||
model: bucket.model,
|
||||
modelDisplayName: meta.formatModel(bucket.model),
|
||||
category: bucket.category,
|
||||
agentType: bucket.agentType,
|
||||
inputTokens: bucket.inputTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
cacheWriteTokens: bucket.cacheWriteTokens,
|
||||
cacheReadTokens: bucket.cacheReadTokens,
|
||||
totalTokens: total,
|
||||
costUSD: bucket.costUSD,
|
||||
savingsUSD: bucket.savingsUSD,
|
||||
savingsBaselineModel: bucket.savingsBaselineModel,
|
||||
calls: bucket.calls,
|
||||
// outputTokens already includes reasoning (folded in above), and for Codex
|
||||
// inputTokens is non-cached with cacheReadTokens holding cached input, which
|
||||
// is exactly what the credit rates expect.
|
||||
credits: bucket.provider === 'codex'
|
||||
? codexCredits(bucket.model, {
|
||||
inputTokens: bucket.inputTokens,
|
||||
cachedReadTokens: bucket.cacheReadTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
})
|
||||
: null,
|
||||
// Credits are per raw id (aliases can have different rates). Sum the
|
||||
// rated buckets and flag the row incomplete when any contributor is
|
||||
// unrated — nulling the whole merge would zero a real menubar total.
|
||||
// outputTokens is already the billable output (for Codex that includes
|
||||
// reasoning, so nothing is added on top), and inputTokens is non-cached
|
||||
// with cacheReadTokens holding cached input - exactly what the credit
|
||||
// rates expect.
|
||||
const bucketCredits = bucket.provider === 'codex'
|
||||
? codexCredits(bucket.model, {
|
||||
inputTokens: bucket.inputTokens,
|
||||
cachedReadTokens: bucket.cacheReadTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
})
|
||||
: null
|
||||
|
||||
const baselines = baselinesByKey.get(resolvedKey) ?? new Set<string>()
|
||||
if (bucket.savingsBaselineModel) baselines.add(bucket.savingsBaselineModel)
|
||||
baselinesByKey.set(resolvedKey, baselines)
|
||||
const resolvedBaseline = baselines.size === 1 ? [...baselines][0]! : ''
|
||||
|
||||
const existing = rowsByKey.get(resolvedKey)
|
||||
if (existing) {
|
||||
existing.inputTokens += bucket.inputTokens
|
||||
existing.outputTokens += bucket.outputTokens
|
||||
existing.cacheWriteTokens += bucket.cacheWriteTokens
|
||||
existing.cacheReadTokens += bucket.cacheReadTokens
|
||||
existing.totalTokens += total
|
||||
existing.costUSD += bucket.costUSD
|
||||
existing.savingsUSD += bucket.savingsUSD
|
||||
existing.calls += bucket.calls
|
||||
existing.savingsBaselineModel = resolvedBaseline
|
||||
const existingRated = existing.credits !== null
|
||||
const incomingRated = bucketCredits !== null
|
||||
if (incomingRated) existing.credits = (existing.credits ?? 0) + bucketCredits
|
||||
if (existingRated !== incomingRated) existing.creditsIncomplete = true
|
||||
} else {
|
||||
rowsByKey.set(resolvedKey, {
|
||||
provider: bucket.provider,
|
||||
providerDisplayName: meta.displayName,
|
||||
model: bucket.model,
|
||||
modelDisplayName,
|
||||
category: bucket.category,
|
||||
agentType: bucket.agentType,
|
||||
inputTokens: bucket.inputTokens,
|
||||
outputTokens: bucket.outputTokens,
|
||||
cacheWriteTokens: bucket.cacheWriteTokens,
|
||||
cacheReadTokens: bucket.cacheReadTokens,
|
||||
totalTokens: total,
|
||||
costUSD: bucket.costUSD,
|
||||
savingsUSD: bucket.savingsUSD,
|
||||
savingsBaselineModel: resolvedBaseline,
|
||||
calls: bucket.calls,
|
||||
credits: bucketCredits,
|
||||
})
|
||||
}
|
||||
|
||||
if (!opts.byTask && !opts.byAgent) {
|
||||
const perCat = perModelCategoryCost.get(`${bucket.provider} ${bucket.model}`)
|
||||
if (perCat && perCat.size > 0) {
|
||||
let topCat: TaskCategory = 'general'
|
||||
let topCost = -1
|
||||
let totalCost = 0
|
||||
for (const [cat, cost] of perCat.entries()) {
|
||||
totalCost += cost
|
||||
if (cost > topCost) {
|
||||
topCost = cost
|
||||
topCat = cat
|
||||
}
|
||||
const rawKey = `${bucket.provider} ${bucket.model}`
|
||||
if (!foldedRawSeen.has(rawKey)) {
|
||||
foldedRawSeen.add(rawKey)
|
||||
const rawCat = perModelCategoryCost.get(rawKey)
|
||||
if (rawCat) {
|
||||
let folded = foldedCategoryCost.get(foldKey)
|
||||
if (!folded) {
|
||||
folded = new Map()
|
||||
foldedCategoryCost.set(foldKey, folded)
|
||||
}
|
||||
row.topCategory = topCat
|
||||
row.topCategoryCost = topCost
|
||||
row.topCategoryShare = totalCost > 0 ? topCost / totalCost : 0
|
||||
for (const [cat, cost] of rawCat) {
|
||||
folded.set(cat, (folded.get(cat) ?? 0) + cost)
|
||||
}
|
||||
}
|
||||
foldedTotalCost.set(
|
||||
foldKey,
|
||||
(foldedTotalCost.get(foldKey) ?? 0) + (perModelTotalCost.get(rawKey) ?? 0),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const rows = [...rowsByKey.values()]
|
||||
for (const row of rows) {
|
||||
if (opts.byTask || opts.byAgent) continue
|
||||
const perCat = foldedCategoryCost.get(`${row.provider} ${resolveCanonicalModelId(row.model)}`)
|
||||
if (!perCat || perCat.size === 0) continue
|
||||
let topCat: TaskCategory = 'general'
|
||||
let topCost = -1
|
||||
let totalCost = 0
|
||||
for (const [cat, cost] of perCat.entries()) {
|
||||
totalCost += cost
|
||||
if (cost > topCost) {
|
||||
topCost = cost
|
||||
topCat = cat
|
||||
}
|
||||
}
|
||||
|
||||
rows.push(row)
|
||||
row.topCategory = topCat
|
||||
row.topCategoryCost = topCost
|
||||
row.topCategoryShare = totalCost > 0 ? topCost / totalCost : 0
|
||||
}
|
||||
|
||||
if (opts.byTask || opts.byAgent) {
|
||||
rows.sort((a, b) => {
|
||||
const aTotal = perModelTotalCost.get(`${a.provider} ${a.model}`) ?? 0
|
||||
const bTotal = perModelTotalCost.get(`${b.provider} ${b.model}`) ?? 0
|
||||
const aTotal = foldedTotalCost.get(`${a.provider} ${resolveCanonicalModelId(a.model)}`) ?? 0
|
||||
const bTotal = foldedTotalCost.get(`${b.provider} ${resolveCanonicalModelId(b.model)}`) ?? 0
|
||||
if (aTotal !== bTotal) return bTotal - aTotal
|
||||
if (a.provider !== b.provider) return a.provider.localeCompare(b.provider)
|
||||
if (a.model !== b.model) return a.model.localeCompare(b.model)
|
||||
if (a.modelDisplayName !== b.modelDisplayName) return a.modelDisplayName.localeCompare(b.modelDisplayName)
|
||||
return (b.costUSD + b.savingsUSD) - (a.costUSD + a.savingsUSD)
|
||||
})
|
||||
} else {
|
||||
|
|
@ -482,7 +546,7 @@ export function renderTable(
|
|||
const rowEntries: RowCells[] = []
|
||||
let prevProviderModel = ''
|
||||
for (const row of rows) {
|
||||
const groupKey = `${row.provider} ${row.model}`
|
||||
const groupKey = `${row.provider} ${row.modelDisplayName}`
|
||||
const isNewGroup = !grouped || groupKey !== prevProviderModel
|
||||
prevProviderModel = groupKey
|
||||
const allCells = defaultColumns(byTask, byAgent, showSaved).map(col => {
|
||||
|
|
@ -604,6 +668,7 @@ export function renderJson(rows: ModelReportRow[]): string {
|
|||
savingsUSD: r.savingsUSD,
|
||||
savingsBaselineModel: r.savingsBaselineModel,
|
||||
credits: r.credits,
|
||||
creditsIncomplete: r.creditsIncomplete === true,
|
||||
})),
|
||||
null,
|
||||
2,
|
||||
|
|
|
|||
|
|
@ -13,6 +13,28 @@ export type ModelCosts = {
|
|||
cacheReadCostPerToken: number
|
||||
webSearchCostPerRequest: number
|
||||
fastMultiplier: number
|
||||
/// True only when the pricing source carried a real cache-write rate. When
|
||||
/// absent/false, `cacheWriteCostPerToken` is the fabricated `1.25 x input`
|
||||
/// default, which is right for Anthropic-style pricing but would invent a
|
||||
/// surcharge on providers that charge nothing extra to write cache. Callers
|
||||
/// that decide WHICH bucket to put tokens in (rather than what to multiply
|
||||
/// them by) must consult this before routing tokens to the cache-write
|
||||
/// bucket. Optional so an incomplete literal defaults to the safe answer.
|
||||
cacheWriteCostIsExplicit?: boolean
|
||||
}
|
||||
|
||||
/// Providers whose reported `reasoningTokens` are a SUBSET of `outputTokens`
|
||||
/// rather than a separate bucket to add on top. OpenAI bills reasoning as part
|
||||
/// of output (every codex `token_count` event satisfies input + output ==
|
||||
/// total), and Anthropic folds thinking into output the same way, so summing
|
||||
/// the two double-counts both the cost and the displayed output tokens.
|
||||
const REASONING_INCLUDED_IN_OUTPUT = new Set(['claude', 'codex'])
|
||||
|
||||
/// Output tokens to bill and display for one call. Single source of truth so
|
||||
/// the pricing sites and the display sums can never disagree about whether a
|
||||
/// provider's reasoning tokens are already inside its output count (#1075).
|
||||
export function billableOutputTokens(provider: string, outputTokens: number, reasoningTokens: number): number {
|
||||
return REASONING_INCLUDED_IN_OUTPUT.has(provider) ? outputTokens : outputTokens + reasoningTokens
|
||||
}
|
||||
|
||||
type PriceOverrideRates = {
|
||||
|
|
@ -37,6 +59,11 @@ type SnapshotEntry = [number, number, number | null, number | null, (number | nu
|
|||
|
||||
const LITELLM_URL = 'https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json'
|
||||
const CACHE_TTL_MS = 24 * 60 * 60 * 1000
|
||||
// Bump whenever a ModelCosts field changes pricing behavior (cacheWriteCostIsExplicit,
|
||||
// added in #1075/#1078). A cache written under an older/missing version is treated as a
|
||||
// miss instead of read verbatim, so a stale on-disk file can't reintroduce a killed bug
|
||||
// for up to CACHE_TTL_MS after an upgrade.
|
||||
const CACHE_SCHEMA_VERSION = 2
|
||||
const WEB_SEARCH_COST = 0.01
|
||||
const ONE_HOUR_CACHE_WRITE_MULTIPLIER_FROM_FIVE_MINUTE_RATE = 1.6
|
||||
|
||||
|
|
@ -71,6 +98,7 @@ function buildCosts(
|
|||
cacheReadCostPerToken: cacheRead ?? input * 0.1,
|
||||
webSearchCostPerRequest: WEB_SEARCH_COST,
|
||||
fastMultiplier: fast ?? 1,
|
||||
cacheWriteCostIsExplicit: cacheWrite !== null && cacheWrite !== undefined,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -200,6 +228,7 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
|
|||
|
||||
await mkdir(getCodeburnCacheDir(), { recursive: true })
|
||||
await writeFile(getCachePath(), JSON.stringify({
|
||||
version: CACHE_SCHEMA_VERSION,
|
||||
timestamp: Date.now(),
|
||||
data: Object.fromEntries(pricing),
|
||||
}))
|
||||
|
|
@ -210,7 +239,8 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
|
|||
async function loadCachedPricing(): Promise<Map<string, ModelCosts> | null> {
|
||||
try {
|
||||
const raw = await readFile(getCachePath(), 'utf-8')
|
||||
const cached = JSON.parse(raw) as { timestamp: number; data: Record<string, ModelCosts> }
|
||||
const cached = JSON.parse(raw) as { version?: number; timestamp: number; data: Record<string, ModelCosts> }
|
||||
if (cached.version !== CACHE_SCHEMA_VERSION) return null
|
||||
if (Date.now() - cached.timestamp > CACHE_TTL_MS) return null
|
||||
return new Map(Object.entries(cached.data))
|
||||
} catch {
|
||||
|
|
@ -282,6 +312,16 @@ const BUILTIN_ALIASES: Record<string, string> = {
|
|||
'openclaw-auto': 'claude-sonnet-4-5',
|
||||
'warp-auto-efficient': 'gpt-5.3-codex',
|
||||
'warp-auto-powerful': 'claude-opus-4-6',
|
||||
// Codex activity ids are product surfaces, not subscription SKUs and not
|
||||
// LiteLLM rows. OpenAI's tracker (openai/codex#32224) says auto review
|
||||
// consumes normal model usage. Public evidence: review_model defaults to
|
||||
// the session model; GPT-5.5 is the currently recommended review model.
|
||||
// Price as that existing bundled row. Do not invent a rate. Do not treat
|
||||
// the id as honestly $0 — it draws from the same credit pool. Display
|
||||
// stays on autoModelNames (same class as cursor-auto / copilot-openai-auto).
|
||||
// Only alias ids observed in Codex source / real rollouts. Do not infer
|
||||
// `codex-code-review` from the activity name "code review".
|
||||
'codex-auto-review': 'gpt-5.5',
|
||||
'grok-build': 'grok-build-0.1',
|
||||
'GPT-5.3 Codex (low reasoning)': 'gpt-5.3-codex',
|
||||
'GPT-5.3 Codex (medium reasoning)': 'gpt-5.3-codex',
|
||||
|
|
@ -736,6 +776,19 @@ function getCanonicalName(model: string): string {
|
|||
)
|
||||
}
|
||||
|
||||
/// Alias-resolved identity for report merge. Display names stay cosmetic —
|
||||
/// prefix matches in SHORT_NAMES must not fold distinct SKUs into one row.
|
||||
/// Path-form ids (`accounts/fireworks/models/<slug>`, `cline-pass/<slug>`)
|
||||
/// peel to the leaf so they share a bucket with the bare slug.
|
||||
export function resolveCanonicalModelId(model: string): string {
|
||||
const viaUser = Object.hasOwn(userAliases, model) ? userAliases[model]! : model
|
||||
const aliased = resolveAlias(getCanonicalName(viaUser))
|
||||
if (!aliased.includes('/')) return aliased
|
||||
const leaf = aliased.slice(aliased.lastIndexOf('/') + 1)
|
||||
if (!leaf) return aliased
|
||||
return resolveAlias(getCanonicalName(leaf))
|
||||
}
|
||||
|
||||
// Namespaces the pricing catalog itself uses, plus the ones below. An unknown
|
||||
// `provider/model` must stay unpriced — do not treat `/` as authority. Derived
|
||||
// rather than hand-listed so a vendor LiteLLM already knows (`x-ai/`, `qwen/`,
|
||||
|
|
@ -1090,6 +1143,7 @@ const autoModelNames: Record<string, string> = {
|
|||
'openclaw-auto': 'OpenClaw (auto)',
|
||||
'qwen-auto': 'Qwen (auto)',
|
||||
'kimi-auto': 'Kimi (auto)',
|
||||
'codex-auto-review': 'Codex Auto Review',
|
||||
}
|
||||
|
||||
const SHORT_NAMES: Record<string, string> = {
|
||||
|
|
@ -1230,6 +1284,14 @@ export function getShortModelName(model: string): string {
|
|||
return shortModelName(model, new Set())
|
||||
}
|
||||
|
||||
/// Provider-first display name. Local labels win (Cursor estimated suffixes,
|
||||
/// provider tables that intentionally override the global map). If the provider
|
||||
/// echoed the raw id, it missed — fall back to the global resolver instead of
|
||||
/// showing `gpt-5.6-sol` / `accounts/fireworks/models/kimi-k2p6`.
|
||||
export function fallbackRawModelDisplayName(localLabel: string, rawModel: string): string {
|
||||
return localLabel === rawModel ? getShortModelName(rawModel) : localLabel
|
||||
}
|
||||
|
||||
function shortModelName(model: string, seen: Set<string>): string {
|
||||
if (autoModelNames[model]) return autoModelNames[model]
|
||||
if (seen.has(model)) {
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { existsSync } from 'fs'
|
|||
import { lstat, readFile, readdir, stat } from 'fs/promises'
|
||||
import { basename, dirname, join, resolve, sep } from 'path'
|
||||
import { readSessionLines } from './fs-utils.js'
|
||||
import { calculateCost, calculateLocalModelSavings, getShortModelName, isProxiedPath, getProxyPathsConfigHash, getModelAliasesConfigHash, getPriceOverridesConfigHash, getLocalModelSavingsConfigHash } from './models.js'
|
||||
import { billableOutputTokens, calculateCost, calculateLocalModelSavings, getShortModelName, isProxiedPath, getProxyPathsConfigHash, getModelAliasesConfigHash, getPriceOverridesConfigHash, getLocalModelSavingsConfigHash } from './models.js'
|
||||
import { resolveSubagentAttribution, sessionIdentity } from './sessions-report.js'
|
||||
import { normalizeContentBlocks, flatSlice, flatString } from './content-utils.js'
|
||||
import { discoverAllSessions, getProvider } from './providers/index.js'
|
||||
|
|
@ -1768,7 +1768,7 @@ function buildSessionSummary(
|
|||
modelBreakdown[modelKey].tokens.reasoningTokens += call.usage.reasoningTokens
|
||||
if (call.activeDurationMs !== undefined) {
|
||||
modelBreakdown[modelKey].activeDurationMs = (modelBreakdown[modelKey].activeDurationMs ?? 0) + call.activeDurationMs
|
||||
modelBreakdown[modelKey].activeGeneratedTokens = (modelBreakdown[modelKey].activeGeneratedTokens ?? 0) + (call.activeGeneratedTokens ?? call.usage.outputTokens + call.usage.reasoningTokens)
|
||||
modelBreakdown[modelKey].activeGeneratedTokens = (modelBreakdown[modelKey].activeGeneratedTokens ?? 0) + (call.activeGeneratedTokens ?? billableOutputTokens(call.provider, call.usage.outputTokens, call.usage.reasoningTokens))
|
||||
modelBreakdown[modelKey].toolWaitMs = (modelBreakdown[modelKey].toolWaitMs ?? 0) + (call.toolWaitMs ?? 0)
|
||||
}
|
||||
|
||||
|
|
@ -2629,9 +2629,11 @@ function providerCallsToCachedTurns(calls: ParsedProviderCall[]): CachedTurn[] {
|
|||
|
||||
function cachedCallToApiCall(call: CachedCall): ParsedApiCall {
|
||||
const u = call.usage
|
||||
const outputForCost = call.provider === 'claude'
|
||||
? u.outputTokens
|
||||
: u.outputTokens + u.reasoningTokens
|
||||
// Cache-rehydration twin of the fresh-parse pricing in
|
||||
// src/providers/codex.ts (and every other provider's parser): both go
|
||||
// through billableOutputTokens so a cached read and a cold parse can never
|
||||
// disagree about whether reasoning is already inside output (#1075).
|
||||
const outputForCost = billableOutputTokens(call.provider, u.outputTokens, u.reasoningTokens)
|
||||
const costUSD = calculateCost(
|
||||
call.model, u.inputTokens, outputForCost,
|
||||
u.cacheCreationInputTokens, u.cacheReadInputTokens,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ import { homedir } from 'os'
|
|||
import { fileURLToPath } from 'url'
|
||||
import https from 'https'
|
||||
|
||||
import { getCodeburnCacheDir } from '../cache-dir.js'
|
||||
import { getCodeburnCacheDir, readExistingTextFile } from '../cache-dir.js'
|
||||
import { calculateCost } from '../models.js'
|
||||
import { isSqliteAvailable, isSqliteBusyError, openDatabase } from '../sqlite.js'
|
||||
import type { ProbeRoot, Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
|
||||
|
|
@ -50,6 +50,11 @@ function conversationRoots(): readonly AntigravityConversationRoot[] {
|
|||
]
|
||||
}
|
||||
const CACHE_VERSION = 5
|
||||
export const ANTIGRAVITY_CACHE_VERSION = CACHE_VERSION
|
||||
export const ANTIGRAVITY_LEGACY_CACHE_FILE = 'antigravity-results.json'
|
||||
export function antigravityCacheFileName(version = CACHE_VERSION): string {
|
||||
return `antigravity-results.v${version}.json`
|
||||
}
|
||||
|
||||
const RPC_TIMEOUT_MS = 5000
|
||||
const MAX_RESPONSE_BYTES = 16 * 1024 * 1024
|
||||
|
|
@ -190,7 +195,15 @@ function currentCacheDir(): string {
|
|||
}
|
||||
|
||||
function getCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, 'antigravity-results.json')
|
||||
return join(cacheDir, antigravityCacheFileName())
|
||||
}
|
||||
|
||||
function getLegacyCachePath(cacheDir: string): string {
|
||||
return join(cacheDir, ANTIGRAVITY_LEGACY_CACHE_FILE)
|
||||
}
|
||||
|
||||
function isCurrentCache(cache: AntigravityCache): boolean {
|
||||
return cache.version === CACHE_VERSION && !!cache.cascades && typeof cache.cascades === 'object'
|
||||
}
|
||||
|
||||
export function getAntigravityStatusLineEventsPath(): string {
|
||||
|
|
@ -336,15 +349,42 @@ export function extractAntigravityGeneratorMetadata(resp: unknown): GeneratorMet
|
|||
async function loadCache(cacheDir: string): Promise<AntigravityCacheState> {
|
||||
const inMemory = cacheStates.get(cacheDir)
|
||||
if (inMemory) return inMemory
|
||||
const versioned = await readExistingTextFile(getCachePath(cacheDir))
|
||||
if (versioned.status === 'ok') {
|
||||
try {
|
||||
const cache = JSON.parse(versioned.text) as AntigravityCache
|
||||
if (isCurrentCache(cache)) {
|
||||
const state = { cache, dirty: false }
|
||||
cacheStates.set(cacheDir, state)
|
||||
return state
|
||||
}
|
||||
} catch { /* present but invalid */ }
|
||||
const invalid: AntigravityCacheState = {
|
||||
cache: { version: CACHE_VERSION, cascades: {} },
|
||||
dirty: false,
|
||||
}
|
||||
cacheStates.set(cacheDir, invalid)
|
||||
return invalid
|
||||
}
|
||||
if (versioned.status === 'unreadable') {
|
||||
const invalid: AntigravityCacheState = {
|
||||
cache: { version: CACHE_VERSION, cascades: {} },
|
||||
dirty: false,
|
||||
}
|
||||
cacheStates.set(cacheDir, invalid)
|
||||
return invalid
|
||||
}
|
||||
// Versioned file is absent (ENOENT). Adopt the unsuffixed file only when its
|
||||
// version matches — old binaries still own that path; we never write or delete it.
|
||||
try {
|
||||
const raw = await readFile(getCachePath(cacheDir), 'utf-8')
|
||||
const raw = await readFile(getLegacyCachePath(cacheDir), 'utf-8')
|
||||
const cache = JSON.parse(raw) as AntigravityCache
|
||||
if (cache.version === CACHE_VERSION && cache.cascades && typeof cache.cascades === 'object') {
|
||||
if (isCurrentCache(cache)) {
|
||||
const state = { cache, dirty: false }
|
||||
cacheStates.set(cacheDir, state)
|
||||
return state
|
||||
}
|
||||
} catch { /* no cache or invalid */ }
|
||||
} catch { /* no legacy cache or invalid */ }
|
||||
const state: AntigravityCacheState = {
|
||||
cache: { version: CACHE_VERSION, cascades: {} },
|
||||
dirty: false,
|
||||
|
|
|
|||
|
|
@ -5,8 +5,9 @@ import { basename, join } from 'path'
|
|||
import { homedir } from 'os'
|
||||
|
||||
import { readSessionLines } from '../fs-utils.js'
|
||||
import { calculateCost } from '../models.js'
|
||||
import { billableOutputTokens, calculateCost, getModelCosts } from '../models.js'
|
||||
import { readCachedCodexResults, writeCachedCodexResults, getCachedCodexProject, fingerprintFile, type CodexFileFingerprint } from '../codex-cache.js'
|
||||
import { mergeToolIntervals } from '../codex-throughput.js'
|
||||
import { normalizeContentBlocks } from '../content-utils.js'
|
||||
import { estimateTokensFromChars } from '../token-estimate.js'
|
||||
import type { ToolCall } from '../types.js'
|
||||
|
|
@ -66,11 +67,7 @@ const toolNameMap: Record<string, string> = {
|
|||
// false-positive, an accepted tradeoff for the common case. \s+ and the token
|
||||
// class don't overlap, so there is no catastrophic backtracking.
|
||||
const MCP_CLI_CALL = /(?<![\w.-])mcp-cli(?:\s+(?!call\b)[^\s;|&]+)*\s+call\s+(\S+)\s+(\S+)/
|
||||
function mcpToolFromShellCommand(command: unknown): string | null {
|
||||
const text = typeof command === 'string'
|
||||
? command
|
||||
: Array.isArray(command) ? command.filter(x => typeof x === 'string').join(' ') : ''
|
||||
if (!text) return null
|
||||
function mcpToolFromShellCommand(text: string): string | null {
|
||||
const m = MCP_CLI_CALL.exec(text)
|
||||
if (!m) return null
|
||||
const server = m[1]!.replace(/['"]/g, '')
|
||||
|
|
@ -79,6 +76,37 @@ function mcpToolFromShellCommand(command: unknown): string | null {
|
|||
return `mcp__${server}__${tool}`
|
||||
}
|
||||
|
||||
// Codex has no dedicated skill tool: loading a skill is an ordinary shell exec
|
||||
// that reads the skill's `SKILL.md`, so the usage landed entirely under Bash and
|
||||
// the Skills dimension stayed empty (#478). Recognize only a *read* of a
|
||||
// SKILL.md, deliberately narrowly:
|
||||
// - the command segment must start with a file-reading binary (cat/sed/head/
|
||||
// tail/less/more/bat) — a `grep`/`rg`/`ls` that merely mentions a SKILL.md
|
||||
// is a search near the file, not a skill load, and stays plain Bash;
|
||||
// - the path it reads must END in `<name>/SKILL.md`, so `<name>` is the skill.
|
||||
// The name is the parent directory, the same key `src/providers/pi.ts` derives
|
||||
// for a native skill read (#588) and the same vocabulary the Claude parser
|
||||
// records from the `Skill` tool, so the Skills breakdown stays cross-provider
|
||||
// coherent. Substring matching, so a command that merely quotes the phrase can
|
||||
// false-positive — the same accepted tradeoff as MCP_CLI_CALL above.
|
||||
const SKILL_MD_READ = /(?<![\w.-])(?:cat|bat|sed|head|tail|less|more)\b[^;|&]*?[\s'"]([^\s;|&'"]*[\\/]([^\\/;|&'"]+)[\\/]SKILL\.md)\b/
|
||||
function skillFromShellCommand(text: string): string | null {
|
||||
const m = SKILL_MD_READ.exec(text)
|
||||
const name = m?.[2]?.trim()
|
||||
return name ? name : null
|
||||
}
|
||||
|
||||
/// Flatten every shape Codex records a shell command in to one string:
|
||||
/// `function_call` arguments carry it as a plain string, the item model's
|
||||
/// `CommandExecution` item as an argv array (`["/bin/zsh","-lc","..."]`), and
|
||||
/// the `exec` custom tool as the JS program that calls `tools.exec_command`.
|
||||
/// One text, one classification pipeline (#478).
|
||||
function shellCommandText(command: unknown): string {
|
||||
if (typeof command === 'string') return command
|
||||
if (Array.isArray(command)) return command.filter(x => typeof x === 'string').join(' ')
|
||||
return ''
|
||||
}
|
||||
|
||||
// Count added/removed lines from a Codex `patch_apply_end` change's
|
||||
// `unified_diff`. A leading '+' is an added line and '-' a removed line; the
|
||||
// '+++'/'---' file headers and '@@' hunk headers are excluded. Numbers only —
|
||||
|
|
@ -126,6 +154,10 @@ type CodexEntry = {
|
|||
type CodexTokenUsage = {
|
||||
input_tokens?: number
|
||||
cached_input_tokens?: number
|
||||
/// Portion of `input_tokens` that was WRITTEN to the prompt cache this call
|
||||
/// (codex PR #33454). Like `cached_input_tokens`, it is carved out of
|
||||
/// `input_tokens`, not added on top.
|
||||
cache_write_input_tokens?: number
|
||||
output_tokens?: number
|
||||
reasoning_output_tokens?: number
|
||||
total_tokens?: number
|
||||
|
|
@ -323,6 +355,7 @@ function getRawTokenUsage(head: string, field: 'last_token_usage' | 'total_token
|
|||
return {
|
||||
input_tokens: getRawJsonNumberField(body, 'input_tokens'),
|
||||
cached_input_tokens: getRawJsonNumberField(body, 'cached_input_tokens'),
|
||||
cache_write_input_tokens: getRawJsonNumberField(body, 'cache_write_input_tokens'),
|
||||
output_tokens: getRawJsonNumberField(body, 'output_tokens'),
|
||||
reasoning_output_tokens: getRawJsonNumberField(body, 'reasoning_output_tokens'),
|
||||
total_tokens: getRawJsonNumberField(body, 'total_tokens'),
|
||||
|
|
@ -593,10 +626,14 @@ type CodexResumeState = {
|
|||
prevCumulativeTotal: number | null
|
||||
prevInput: number
|
||||
prevCached: number
|
||||
prevCacheWrite: number
|
||||
prevOutput: number
|
||||
prevReasoning: number
|
||||
pendingTools: string[]
|
||||
pendingToolSequence: ToolCall[][]
|
||||
/// Optional so a resume state written before skills attribution existed still
|
||||
/// decodes (isResumeState does not require it); absent reads as "no skills".
|
||||
pendingSkills?: string[]
|
||||
pendingUserMessage: string
|
||||
pendingOutputChars: number
|
||||
pendingLocAdded: number
|
||||
|
|
@ -606,6 +643,12 @@ type CodexResumeState = {
|
|||
turnCounter: number
|
||||
currentTurnId: string
|
||||
taskStartedAt?: number
|
||||
// #1088 BUG-1: timestamp of the first request-context event (turn_context,
|
||||
// world_state, event_msg/user_message, or response_item/message) seen since
|
||||
// the last task_started. Codex fires task_started before it assembles the
|
||||
// request, so the gap up to this event is CLI/harness startup, not model
|
||||
// wait -- the active window for Tok/s starts here, not at task_started.
|
||||
taskActiveStartedAt?: number
|
||||
}
|
||||
|
||||
// The state comes back off our own JSON cache; a truncated or hand-edited file
|
||||
|
|
@ -619,6 +662,7 @@ function isResumeState(value: unknown): value is CodexResumeState {
|
|||
&& (v['prevCumulativeTotal'] === null || typeof v['prevCumulativeTotal'] === 'number')
|
||||
&& typeof v['prevInput'] === 'number'
|
||||
&& typeof v['prevCached'] === 'number'
|
||||
&& typeof v['prevCacheWrite'] === 'number'
|
||||
&& typeof v['prevOutput'] === 'number'
|
||||
&& typeof v['prevReasoning'] === 'number'
|
||||
&& Array.isArray(v['pendingTools'])
|
||||
|
|
@ -678,10 +722,51 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
let prevCumulativeTotal: number | null = resume?.state.prevCumulativeTotal ?? null
|
||||
let prevInput = resume?.state.prevInput ?? 0
|
||||
let prevCached = resume?.state.prevCached ?? 0
|
||||
let prevCacheWrite = resume?.state.prevCacheWrite ?? 0
|
||||
let prevOutput = resume?.state.prevOutput ?? 0
|
||||
let prevReasoning = resume?.state.prevReasoning ?? 0
|
||||
let pendingTools: string[] = resume ? [...resume.state.pendingTools] : []
|
||||
let pendingToolSequence: ToolCall[][] = resume ? [...resume.state.pendingToolSequence] : []
|
||||
let pendingSkills: string[] = resume?.state.pendingSkills ? [...resume.state.pendingSkills] : []
|
||||
// Attribution names already emitted from a `response_item` tool call.
|
||||
// Codex's item model repeats a finished shell command as
|
||||
// `item_completed`/`CommandExecution`; a rollout that carries BOTH shapes
|
||||
// must attribute the call once, and one that carries ONLY the item must
|
||||
// still attribute it. Counting per derived name (not per command text,
|
||||
// which the two shapes spell differently - argv array vs plain string)
|
||||
// makes repeats of the same call line up one-for-one. The response item is
|
||||
// always written first (the model requests the call, the item completes
|
||||
// it), so the item side only ever cancels an attribution, never adds a
|
||||
// duplicate.
|
||||
const itemDedup = new Map<string, number>()
|
||||
|
||||
/// The single classification pipeline every Codex shell-command shape
|
||||
/// feeds (#478): `function_call` arguments, the `exec` custom tool's JS
|
||||
/// `input`, and the item model's `CommandExecution` item. It adds MCP and
|
||||
/// Skill attribution only - the exec itself is counted as Bash by its
|
||||
/// response item, and `viaItem` calls never add a tool of their own, so
|
||||
/// call/token/cost totals cannot move.
|
||||
const attributeShellCommand = (text: string, viaItem: boolean): void => {
|
||||
if (!text) return
|
||||
const claim = (name: string): boolean => {
|
||||
const pending = itemDedup.get(name) ?? 0
|
||||
if (!viaItem) { itemDedup.set(name, pending + 1); return true }
|
||||
if (pending === 0) return true
|
||||
itemDedup.set(name, pending - 1)
|
||||
return false
|
||||
}
|
||||
const mcpTool = mcpToolFromShellCommand(text)
|
||||
if (mcpTool && claim(mcpTool)) {
|
||||
pendingTools.push(mcpTool)
|
||||
pendingToolSequence.push([{ tool: mcpTool }])
|
||||
}
|
||||
const skill = skillFromShellCommand(text)
|
||||
if (skill && claim(`skill:${skill}`)) {
|
||||
pendingSkills.push(skill)
|
||||
pendingTools.push('Skill')
|
||||
pendingToolSequence.push([{ tool: 'Skill', file: skill }])
|
||||
}
|
||||
}
|
||||
let pendingUserMessage = resume?.state.pendingUserMessage ?? ''
|
||||
let pendingOutputChars = resume?.state.pendingOutputChars ?? 0
|
||||
// Rich-session-capture: edit LOC deltas and failed-patch count accumulated
|
||||
|
|
@ -712,6 +797,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
let taskGeneratedTokens = 0
|
||||
let taskToolIntervals: Array<[number, number]> = []
|
||||
let taskStartedAt: number | undefined = resume?.state.taskStartedAt
|
||||
let taskActiveStartedAt: number | undefined = resume?.state.taskActiveStartedAt
|
||||
const openToolStarts = new Map<string, number>()
|
||||
|
||||
// Resume point for the NEXT run, refreshed at every task boundary.
|
||||
|
|
@ -755,6 +841,22 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
continue
|
||||
}
|
||||
|
||||
// #1088 BUG-1: the first request-context event since task_started marks
|
||||
// where model-request assembly actually began. Checked before any of
|
||||
// these types `continue` below, and unconditionally (like turn_context's
|
||||
// model capture above) so a forked replay's own request-context events
|
||||
// still mark it -- matching how those events are already read regardless
|
||||
// of isForkReplay, which only filters task boundaries and tool events.
|
||||
if (taskActiveStartedAt === undefined && (
|
||||
entry.type === 'turn_context'
|
||||
|| entry.type === 'world_state'
|
||||
|| (entry.type === 'event_msg' && entry.payload?.type === 'user_message')
|
||||
|| (entry.type === 'response_item' && entry.payload?.type === 'message')
|
||||
)) {
|
||||
const ctxAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
if (Number.isFinite(ctxAt)) taskActiveStartedAt = ctxAt
|
||||
}
|
||||
|
||||
if (entry.type === 'turn_context' && typeof entry.payload?.model === 'string') {
|
||||
sessionModel = entry.payload.model
|
||||
continue
|
||||
|
|
@ -769,6 +871,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
entry.payload?.type === 'custom_tool_call' ||
|
||||
entry.payload?.type === 'custom_tool_call_output' ||
|
||||
entry.payload?.type === 'mcp_tool_call_end' ||
|
||||
entry.payload?.type === 'item_completed' ||
|
||||
entry.payload?.type === 'patch_apply_end'
|
||||
)) continue
|
||||
|
||||
|
|
@ -781,6 +884,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
taskToolIntervals = []
|
||||
const startedAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
taskStartedAt = Number.isFinite(startedAt) ? startedAt : undefined
|
||||
taskActiveStartedAt = undefined
|
||||
openToolStarts.clear()
|
||||
// Everything decoded so far is now in `results` and the per-task
|
||||
// accumulators are empty: a clean restart point for an appended tail.
|
||||
|
|
@ -795,10 +899,12 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
prevCumulativeTotal,
|
||||
prevInput,
|
||||
prevCached,
|
||||
prevCacheWrite,
|
||||
prevOutput,
|
||||
prevReasoning,
|
||||
pendingTools: [...pendingTools],
|
||||
pendingToolSequence: [...pendingToolSequence],
|
||||
pendingSkills: [...pendingSkills],
|
||||
pendingUserMessage,
|
||||
pendingOutputChars,
|
||||
pendingLocAdded,
|
||||
|
|
@ -808,6 +914,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
turnCounter,
|
||||
currentTurnId,
|
||||
...(taskStartedAt !== undefined ? { taskStartedAt } : {}),
|
||||
...(taskActiveStartedAt !== undefined ? { taskActiveStartedAt } : {}),
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
|
@ -826,13 +933,21 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
if (typeof fp === 'string') call.file = fp
|
||||
const cmd = args['command'] ?? args['cmd']
|
||||
if (typeof cmd === 'string') call.command = cmd
|
||||
// Attribute a CLI-wrapped MCP call (e.g. `mcp-cli call server tool`)
|
||||
// to the MCP breakdown too; the exec still counts as Bash above.
|
||||
const mcpTool = mcpToolFromShellCommand(cmd)
|
||||
if (mcpTool) {
|
||||
pendingTools.push(mcpTool)
|
||||
pendingToolSequence.push([{ tool: mcpTool }])
|
||||
}
|
||||
attributeShellCommand(shellCommandText(cmd), false)
|
||||
}
|
||||
// A `custom_tool_call` has no `arguments`: the shell tool's payload is
|
||||
// the `input` program that calls `tools.exec_command({cmd: ...})`. The
|
||||
// MCP/skill matchers never saw it, so a CLI-wrapped MCP call or a
|
||||
// SKILL.md read made through Codex's custom-tool transport counted only
|
||||
// as Bash (#478). Restricted to the shell tool: `apply_patch` is a
|
||||
// custom tool too and its `input` is file content, where a documented
|
||||
// command would read as a real invocation. `call.command` is
|
||||
// deliberately NOT set from it either - that field feeds the retry
|
||||
// heuristic's read-shaped-command test, and a JS program is not a
|
||||
// shell command.
|
||||
if (mapped === 'Bash') {
|
||||
const input = (entry.payload as Record<string, unknown>)['input']
|
||||
if (typeof input === 'string') attributeShellCommand(input, false)
|
||||
}
|
||||
const callId = entry.payload.call_id
|
||||
const started = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
|
|
@ -851,26 +966,33 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
}
|
||||
|
||||
if (entry.type === 'event_msg' && entry.payload?.type === 'task_complete') {
|
||||
const durationMs = entry.payload.duration_ms
|
||||
// #1088 BUG-8: task_complete's duration can arrive as {secs,nanos} or
|
||||
// a string too (mcp_tool_call_end already tolerates both, below, via
|
||||
// this same durationValueMs helper); read it the same permissive way
|
||||
// instead of only the plain-number `duration_ms` field.
|
||||
const durationMs = durationValueMs(entry.payload.duration_ms) ?? durationValueMs(entry.payload.duration)
|
||||
if (typeof durationMs === 'number' && durationMs > 0 && taskGeneratedTokens > 0 && pendingTaskCalls.length > 0) {
|
||||
const completedAt = entry.timestamp ? Date.parse(entry.timestamp) : NaN
|
||||
const windowStart = taskStartedAt ?? (Number.isFinite(completedAt) ? completedAt - durationMs : undefined)
|
||||
const windowEnd = windowStart !== undefined ? windowStart + durationMs : undefined
|
||||
const clipped = taskToolIntervals.map(([start, end]) => [
|
||||
windowStart !== undefined ? Math.max(start, windowStart) : start,
|
||||
windowEnd !== undefined ? Math.min(end, windowEnd) : end,
|
||||
] as [number, number]).filter(([start, end]) => end > start)
|
||||
const merged = clipped.sort((a, b) => a[0] - b[0]).reduce<Array<[number, number]>>((acc, interval) => {
|
||||
const previous = acc.at(-1)
|
||||
if (previous && interval[0] <= previous[1]) previous[1] = Math.max(previous[1], interval[1])
|
||||
else acc.push([...interval])
|
||||
return acc
|
||||
}, [])
|
||||
const toolWaitMs = Math.min(durationMs, merged.reduce((sum, interval) => sum + interval[1] - interval[0], 0))
|
||||
const activeMs = durationMs - toolWaitMs
|
||||
// #1088 BUG-1: Codex fires task_started before it assembles the
|
||||
// request, so the gap up to the first request-context event is
|
||||
// CLI/harness startup, not model wait. The active window starts
|
||||
// there instead of at task_started; task completion (windowEnd,
|
||||
// inside mergeToolIntervals) is unchanged.
|
||||
const activeWindowStart = taskActiveStartedAt ?? taskStartedAt
|
||||
const startupGapMs = activeWindowStart !== undefined && taskStartedAt !== undefined
|
||||
? activeWindowStart - taskStartedAt
|
||||
: 0
|
||||
const effectiveDurationMs = Math.max(0, durationMs - startupGapMs)
|
||||
// #1088 BUG-8: shared with codex-throughput.ts's live estimate
|
||||
// instead of a second inline copy of the same clip/merge/cap.
|
||||
const toolWaitMs = mergeToolIntervals(taskToolIntervals, effectiveDurationMs, activeWindowStart, Number.isFinite(completedAt) ? completedAt : undefined)
|
||||
const activeMs = effectiveDurationMs - toolWaitMs
|
||||
if (activeMs <= 0) continue
|
||||
for (const call of pendingTaskCalls) {
|
||||
const generated = call.outputTokens + call.reasoningTokens
|
||||
// Reasoning is already inside output_tokens (#1075/#1078); the
|
||||
// throughput numerator must agree with the cost numerator or
|
||||
// Tok/s reads high for reasoning-heavy calls (#1079).
|
||||
const generated = billableOutputTokens('codex', call.outputTokens, call.reasoningTokens)
|
||||
if (generated <= 0) continue
|
||||
call.activeGeneratedTokens = generated
|
||||
call.activeDurationMs = activeMs * (generated / taskGeneratedTokens)
|
||||
|
|
@ -923,6 +1045,21 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
continue
|
||||
}
|
||||
|
||||
// Codex's item model records a finished shell command as
|
||||
// `event_msg`/`item_completed` carrying a `CommandExecution` item whose
|
||||
// `command` is the argv array (`["/bin/zsh","-lc","..."]`) - a shape the
|
||||
// `function_call`-only tool path never reached, so a CLI-wrapped MCP call
|
||||
// or a SKILL.md read made under it stayed invisible (#478). Attribution
|
||||
// only: the exec's own Bash count comes from its response item, exactly
|
||||
// as `exec_command_end` is left alone for the classic transport.
|
||||
if (entry.type === 'event_msg' && entry.payload?.type === 'item_completed') {
|
||||
const item = (entry.payload as Record<string, unknown>)['item'] as Record<string, unknown> | undefined
|
||||
if (item && item['type'] === 'CommandExecution') {
|
||||
attributeShellCommand(shellCommandText(item['command']), true)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (entry.type === 'response_item' && entry.payload?.type === 'message' && entry.payload?.role === 'user') {
|
||||
const texts = normalizeContentBlocks(entry.payload.content)
|
||||
.filter(c => c.type === 'input_text')
|
||||
|
|
@ -959,7 +1096,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
const timestamp = entry.timestamp ?? ''
|
||||
const dedupKey = `codex:${sessionId}:${timestamp}:est${estCounter++}`
|
||||
|
||||
if (seenKeys.has(dedupKey)) { pendingTools = []; pendingToolSequence = []; pendingUserMessage = ''; pendingOutputChars = 0; pendingLocAdded = 0; pendingLocRemoved = 0; pendingEditFailed = 0; continue }
|
||||
if (seenKeys.has(dedupKey)) { pendingTools = []; pendingToolSequence = []; pendingSkills = []; pendingUserMessage = ''; pendingOutputChars = 0; pendingLocAdded = 0; pendingLocRemoved = 0; pendingEditFailed = 0; continue }
|
||||
seenKeys.add(dedupKey)
|
||||
|
||||
const costUSD = calculateCost(model, estInput, estOutput, 0, 0, 0)
|
||||
|
|
@ -983,6 +1120,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
deduplicationKey: dedupKey,
|
||||
turnId: currentTurnId,
|
||||
toolSequence: pendingToolSequence.length > 0 ? pendingToolSequence : undefined,
|
||||
...(pendingSkills.length > 0 ? { skills: pendingSkills } : {}),
|
||||
userMessage: pendingUserMessage,
|
||||
sessionId,
|
||||
...(sessionCwd ? { projectPath: sessionCwd, workingDirectory: sessionCwd } : {}),
|
||||
|
|
@ -994,6 +1132,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
|
||||
pendingTools = []
|
||||
pendingToolSequence = []
|
||||
pendingSkills = []
|
||||
pendingUserMessage = ''
|
||||
pendingOutputChars = 0
|
||||
pendingLocAdded = 0
|
||||
|
|
@ -1014,12 +1153,14 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
const last = info.last_token_usage
|
||||
let inputTokens = 0
|
||||
let cachedInputTokens = 0
|
||||
let cacheWriteTokens = 0
|
||||
let outputTokens = 0
|
||||
let reasoningTokens = 0
|
||||
|
||||
if (last) {
|
||||
inputTokens = last.input_tokens ?? 0
|
||||
cachedInputTokens = last.cached_input_tokens ?? 0
|
||||
cacheWriteTokens = last.cache_write_input_tokens ?? 0
|
||||
outputTokens = last.output_tokens ?? 0
|
||||
reasoningTokens = last.reasoning_output_tokens ?? 0
|
||||
} else if (cumulativeTotal > 0) {
|
||||
|
|
@ -1027,6 +1168,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
if (!total) continue
|
||||
inputTokens = (total.input_tokens ?? 0) - prevInput
|
||||
cachedInputTokens = (total.cached_input_tokens ?? 0) - prevCached
|
||||
cacheWriteTokens = (total.cache_write_input_tokens ?? 0) - prevCacheWrite
|
||||
outputTokens = (total.output_tokens ?? 0) - prevOutput
|
||||
reasoningTokens = (total.reasoning_output_tokens ?? 0) - prevReasoning
|
||||
}
|
||||
|
|
@ -1042,6 +1184,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
if (total) {
|
||||
prevInput = total.input_tokens ?? 0
|
||||
prevCached = total.cached_input_tokens ?? 0
|
||||
prevCacheWrite = total.cache_write_input_tokens ?? 0
|
||||
prevOutput = total.output_tokens ?? 0
|
||||
prevReasoning = total.reasoning_output_tokens ?? 0
|
||||
}
|
||||
|
|
@ -1053,7 +1196,22 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
// Normalize to Anthropic semantics: inputTokens = non-cached only.
|
||||
const uncachedInputTokens = Math.max(0, inputTokens - cachedInputTokens)
|
||||
|
||||
// Cache writes are carved out of the uncached input, never added to
|
||||
// it: clamp so a malformed or lagging count can never drive the plain
|
||||
// input bucket negative.
|
||||
const cacheWriteInputTokens = Math.max(0, Math.min(cacheWriteTokens, uncachedInputTokens))
|
||||
|
||||
const model = resolveModel(entry.payload, sessionModel)
|
||||
// Only move tokens into the cache-write bucket when the pricing
|
||||
// source publishes a real cache-write rate for this model (gpt-5.6+
|
||||
// charges 1.25x input; everything before it charges nothing extra).
|
||||
// Otherwise buildCosts' fabricated 1.25x default would invent a
|
||||
// surcharge that OpenAI never billed, so the tokens stay where they
|
||||
// already were -- in plain input, priced exactly as before.
|
||||
const billedCacheWriteTokens = cacheWriteInputTokens > 0 && getModelCosts(model)?.cacheWriteCostIsExplicit
|
||||
? cacheWriteInputTokens
|
||||
: 0
|
||||
const billedInputTokens = uncachedInputTokens - billedCacheWriteTokens
|
||||
const timestamp = entry.timestamp ?? ''
|
||||
// Forked sessions copy the parent's entire token_count history
|
||||
// (re-timestamped), so replays must collide with the parent's events
|
||||
|
|
@ -1071,14 +1229,22 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
// key would spuriously diverge on a replay and double-count it.
|
||||
const dedupKey = `codex:${forkedFromId || sessionId}:${cumulativeTotal}:${total?.input_tokens ?? 0}:${total?.cached_input_tokens ?? 0}:${total?.output_tokens ?? 0}:${total?.reasoning_output_tokens ?? 0}`
|
||||
|
||||
// A drop here can only be a byte-identical replay: the
|
||||
// prevCumulativeTotal guard above already discards a repeated
|
||||
// running total, so nothing reaching this point ever loses real
|
||||
// tokens -- no active-time rescaling needed (#1088 investigation).
|
||||
if (seenKeys.has(dedupKey)) continue
|
||||
seenKeys.add(dedupKey)
|
||||
|
||||
// Reasoning tokens are already inside output_tokens, so they are NOT
|
||||
// added here. The cache-rehydration twin of this line lives in
|
||||
// src/parser.ts (cachedCallToApiCall); both call billableOutputTokens
|
||||
// so a fresh parse and a cache read can never price differently.
|
||||
const costUSD = calculateCost(
|
||||
model,
|
||||
uncachedInputTokens,
|
||||
outputTokens + reasoningTokens,
|
||||
0,
|
||||
billedInputTokens,
|
||||
billableOutputTokens('codex', outputTokens, reasoningTokens),
|
||||
billedCacheWriteTokens,
|
||||
cachedInputTokens,
|
||||
0,
|
||||
)
|
||||
|
|
@ -1086,9 +1252,9 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
pendingTaskCalls.push({
|
||||
provider: 'codex',
|
||||
model,
|
||||
inputTokens: uncachedInputTokens,
|
||||
inputTokens: billedInputTokens,
|
||||
outputTokens,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheCreationInputTokens: billedCacheWriteTokens,
|
||||
cacheReadInputTokens: cachedInputTokens,
|
||||
cachedInputTokens,
|
||||
reasoningTokens,
|
||||
|
|
@ -1101,6 +1267,7 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
deduplicationKey: dedupKey,
|
||||
turnId: currentTurnId,
|
||||
toolSequence: pendingToolSequence.length > 0 ? pendingToolSequence : undefined,
|
||||
...(pendingSkills.length > 0 ? { skills: pendingSkills } : {}),
|
||||
userMessage: pendingUserMessage,
|
||||
sessionId,
|
||||
...(sessionCwd ? { projectPath: sessionCwd, workingDirectory: sessionCwd } : {}),
|
||||
|
|
@ -1108,10 +1275,11 @@ function createParser(source: SessionSource, seenKeys: Set<string>, capture?: {
|
|||
...(pendingLocRemoved ? { locRemoved: pendingLocRemoved } : {}),
|
||||
...(pendingEditFailed ? { editFailed: pendingEditFailed } : {}),
|
||||
})
|
||||
taskGeneratedTokens += outputTokens + reasoningTokens
|
||||
taskGeneratedTokens += billableOutputTokens('codex', outputTokens, reasoningTokens)
|
||||
|
||||
pendingTools = []
|
||||
pendingToolSequence = []
|
||||
pendingSkills = []
|
||||
pendingUserMessage = ''
|
||||
pendingOutputChars = 0
|
||||
pendingLocAdded = 0
|
||||
|
|
|
|||
|
|
@ -759,8 +759,9 @@ function createJsonlParser(
|
|||
// CUMULATIVE per-model totals. Emitting each rollup whole would need the
|
||||
// cache to update a prior call in place — the durable merge is
|
||||
// append-only by dedup key — so we emit per-leg DELTAS keyed by
|
||||
// occurrence instead: re-parses of a growing file append only the new
|
||||
// leg, and each leg lands on its own timestamp.
|
||||
// occurrence (`:n`): re-parses of a growing file append only the new
|
||||
// leg, and each leg lands on its own timestamp. Discovery only yields
|
||||
// `<sid>/events.jsonl`, so two journals cannot share a session id.
|
||||
const prevShutdownUsage = new Map<string, ShutdownModelUsage>()
|
||||
const shutdownCountByModel = new Map<string, number>()
|
||||
|
||||
|
|
@ -841,8 +842,14 @@ function createJsonlParser(
|
|||
const modelMetrics = shutdownData.modelMetrics
|
||||
if (!isRecord(modelMetrics)) continue
|
||||
|
||||
// Prefer lastEventTimestamp over sessionStartTime. sessionStartTime
|
||||
// is identical for every stampless leg of a resumed session, so
|
||||
// using it for the call timestamp (or, previously, the key) collapsed
|
||||
// those legs onto one date. lastEventTimestamp is the last stamped
|
||||
// event in this journal — distinct per leg when intervening events
|
||||
// are stamped, and still a real time when they are not.
|
||||
const shutdownTimestamp =
|
||||
(event.timestamp ?? '') || timestampToISO(shutdownData.sessionStartTime) || lastEventTimestamp
|
||||
(event.timestamp ?? '') || lastEventTimestamp || timestampToISO(shutdownData.sessionStartTime)
|
||||
|
||||
for (const [model, metrics] of Object.entries(modelMetrics)) {
|
||||
if (!model || !isRecord(metrics)) continue
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { readdir, readFile, stat } from 'fs/promises'
|
|||
import { join, basename } from 'path'
|
||||
import { homedir } from 'os'
|
||||
|
||||
import { calculateCost } from '../models.js'
|
||||
import { calculateCost, getShortModelName } from '../models.js'
|
||||
import { openDatabase, type SqliteDatabase } from '../sqlite.js'
|
||||
import { normalizeContentBlocks } from '../content-utils.js'
|
||||
import { estimateTokensFromChars } from '../token-estimate.js'
|
||||
|
|
@ -506,7 +506,7 @@ export function createCursorAgentProvider(baseDirOverride?: string): Provider {
|
|||
|
||||
modelDisplayName(model: string): string {
|
||||
if (model === 'cursor-agent-auto') return 'Cursor (auto)'
|
||||
const label = modelDisplayNames[model] ?? model
|
||||
const label = modelDisplayNames[model] ?? getShortModelName(model)
|
||||
return `${label} (est.)`
|
||||
},
|
||||
|
||||
|
|
|
|||
|
|
@ -405,12 +405,15 @@ function createParser(source: SessionSource, seenKeys: Set<string>): SessionPars
|
|||
inputTokens: parsed.usage.input,
|
||||
// Grok reports reasoning INSIDE outputTokens, but the repo contract is
|
||||
// the opposite: ParsedProviderCall.reasoningTokens is exclusive of
|
||||
// outputTokens, and every consumer sums the two (parser.ts's
|
||||
// cachedCallToApiCall for cost, modelBreakdown for tokens, and the
|
||||
// models/audit reports). tests/providers/kiro.test.ts states it
|
||||
// outright. So split it here rather than special-casing grok in five
|
||||
// downstream places: subtracting reasoning makes `output + reasoning`
|
||||
// reconstruct exactly the number Grok reported.
|
||||
// outputTokens. Downstream consumers reconstitute the billable total
|
||||
// through billableOutputTokens() (models.ts): it adds reasoning back on
|
||||
// top for grok and every other provider, except the
|
||||
// REASONING_INCLUDED_IN_OUTPUT set (claude, codex) whose reasoning is
|
||||
// already inside output_tokens and must not be added again.
|
||||
// tests/providers/kiro.test.ts states the exclusive contract outright.
|
||||
// So split it here rather than special-casing grok in every downstream
|
||||
// site: subtracting reasoning makes `output + reasoning` reconstruct
|
||||
// exactly the number Grok reported.
|
||||
outputTokens: parsed.usage.output - reasoningTokens,
|
||||
cacheCreationInputTokens: parsed.usage.cacheCreation,
|
||||
cacheReadInputTokens: parsed.usage.cacheRead,
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { getShortModelName } from '../models.js'
|
||||
import type { DateRange } from '../types.js'
|
||||
import type { Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
|
||||
import { fetchWithTimeout } from '../fetch-utils.js'
|
||||
|
|
@ -124,7 +125,7 @@ export const vercelGateway: Provider = {
|
|||
|
||||
modelDisplayName(model: string): string {
|
||||
const slash = model.indexOf('/')
|
||||
return slash >= 0 ? model.slice(slash + 1) : model
|
||||
return getShortModelName(slash >= 0 ? model.slice(slash + 1) : model)
|
||||
},
|
||||
|
||||
toolDisplayName(rawTool: string): string {
|
||||
|
|
|
|||
|
|
@ -280,13 +280,34 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
|
|||
// session-meta-model-v1: parse large session_meta records structurally so a
|
||||
// nested base_instructions provenance.model cannot overwrite turn_context.
|
||||
// session-meta-fields-v1: the same depth-1 window for cwd/name/originator/
|
||||
// session_id/forked_from_id/model_provider, not just model.
|
||||
codex: 'mcp-attribution-v5-est-cost-active-timing-mcp-wait-rich-capture-v1-cross-provider-pr-v1-session-meta-model-v1-session-meta-fields-v1',
|
||||
// session_id/forked_from_id/model_provider, not just model. (#1055)
|
||||
// codex-pricing-v1 (#1075): reasoning tokens are no longer added on top of
|
||||
// output, and cache_write_input_tokens moves out of the plain input bucket on
|
||||
// models with an explicit cache-write rate. The bucket move does NOT self-heal
|
||||
// on read (cached entries store the buckets, not the raw event), so cached
|
||||
// sessions must re-parse.
|
||||
// codex-tps-v1 (#1079): activeGeneratedTokens summed output + reasoning, the
|
||||
// same double-count codex-pricing-v1 removed from cost. Cached entries store
|
||||
// activeGeneratedTokens/activeDurationMs/toolWaitMs verbatim (cachedCallToApiCall
|
||||
// passes them through without recomputing), so this does NOT self-heal either.
|
||||
// codex-mcp-skills-v1 (#478): CLI-wrapped MCP calls and SKILL.md reads made
|
||||
// through the `exec` custom tool or the item model's `CommandExecution` item
|
||||
// were counted as Bash only. Cached sessions store tools/toolSequence/skills
|
||||
// verbatim, so they must re-parse to gain the attribution.
|
||||
// activity-price-v1: `codex-auto-review` now prices via the recommended
|
||||
// review model. session-cache.json would otherwise keep the pre-alias $0.
|
||||
// Compose all four — a take-ours merge would drop #1075, #1079, or #1092.
|
||||
codex: 'mcp-attribution-v5-est-cost-active-timing-mcp-wait-rich-capture-v1-cross-provider-pr-v1-session-meta-model-v1-session-meta-fields-v1-codex-pricing-v1-codex-tps-v1-codex-mcp-skills-v1-activity-price-v1',
|
||||
cursor: 'composer-anchored-crediting-v1-est-cost',
|
||||
'cursor-agent': 'workspaceless-transcript-v1',
|
||||
// source-provenance-v1 (#944): CLI sessions were misread as VS Code
|
||||
// transcripts (both carry producer 'copilot-agent'), skipping the shutdown
|
||||
// input/cache rollup; this bump re-parses them so the missing tokens land.
|
||||
// #1051 does NOT bump this again. A fingerprint change drops every present
|
||||
// Copilot source (parser.ts getOrCreateProviderSection) and would erase
|
||||
// conversations already pruned from a still-present OTel DB. Old `:n`
|
||||
// shutdown keys migrate via cachedFileNeedsProviderReparse + a durable
|
||||
// strip of legacy shutdown calls on that JSONL file only.
|
||||
copilot: 'cli-shutdown-cost-v1-skills-source-provenance-v1',
|
||||
// authoritative-usage-v4: persist one Grok session call from top-level
|
||||
// authoritative totals, use modelUsage only for priced attribution, clamp
|
||||
|
|
|
|||
|
|
@ -154,7 +154,7 @@ export const COMMIT_ATTRIBUTION_SPAN_NAME = 'codeburn.commit'
|
|||
* encodes the mutable state (inMain/wasReverted for commits; repo, PR links,
|
||||
* and commit set for sessions), so a state TRANSITION mints a new key and the
|
||||
* updated fact is re-sent on the next push — the receiver upserts by
|
||||
* (repo, sha) / (session). Identical states dedupe via the sent-ledger.
|
||||
* (repo, sha) / traceId. Identical states dedupe via the sent-ledger.
|
||||
*/
|
||||
export type AttributionItem = {
|
||||
kind: 'session' | 'commit'
|
||||
|
|
@ -256,7 +256,6 @@ export function buildAttributionOtlpPayload(items: AttributionItem[]): OtlpPaylo
|
|||
const endNano = (rawEndNano > minEndNano ? rawEndNano : minEndNano).toString()
|
||||
|
||||
const attributes: OtlpAttribute[] = [
|
||||
{ key: 'ai.session_id', value: { stringValue: item.sessionId } },
|
||||
{ key: 'ai.project', value: { stringValue: item.project } },
|
||||
]
|
||||
if (item.repo) {
|
||||
|
|
|
|||
|
|
@ -609,9 +609,9 @@ export async function buildMenubarPayloadForRange(periodInfo: PeriodInfo, opts:
|
|||
}
|
||||
claudeConfigs = claudeConfigs ?? await claudeConfigSelector(scanProjects, null)
|
||||
|
||||
// Codex credits for the period. Reuses the models aggregation (folds reasoning
|
||||
// into output, keeps non-cached input + cached-read separate) so the figure
|
||||
// matches the official credit rates.
|
||||
// Codex credits for the period. Reuses the models aggregation (billable output
|
||||
// already includes reasoning for codex, keeps non-cached input + cached-read
|
||||
// separate) so the figure matches the official credit rates.
|
||||
const modelRows = await aggregateModels(scanProjects)
|
||||
currentData.codexCredits = modelRows.reduce(
|
||||
(sum, r) => sum + (r.provider === 'codex' && r.credits != null ? r.credits : 0),
|
||||
|
|
|
|||
|
|
@ -86,8 +86,11 @@ describe('aggregateAudit', () => {
|
|||
expect(r.raw.reasoningTokens).toBe(10)
|
||||
expect(r.raw.cacheReadInputTokens).toBe(200)
|
||||
expect(r.raw.cachedInputTokens).toBe(300)
|
||||
// reasoning folds into output for pricing
|
||||
expect(r.displayed.outputTokens).toBe(110)
|
||||
// Reasoning does NOT fold into output for claude or codex: both bill it
|
||||
// as part of output_tokens already, so adding it would double-count
|
||||
// (#1075). Providers that report reasoning as a separate bucket still get
|
||||
// the additive treatment - see tests/codex-pricing-1075.test.ts.
|
||||
expect(r.displayed.outputTokens).toBe(100)
|
||||
// cache read is the SUM of per-call max(anthropic, openai), not max of sums
|
||||
expect(r.displayed.cacheReadTokens).toBe(500)
|
||||
// attributed cost is preserved exactly
|
||||
|
|
|
|||
|
|
@ -7,12 +7,14 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
|||
|
||||
import {
|
||||
clearCodexMemCaches,
|
||||
codexCacheFileName,
|
||||
fingerprintFile,
|
||||
flushCodexCache,
|
||||
readCachedCodexResults,
|
||||
writeCachedCodexResults,
|
||||
} from '../src/codex-cache.js'
|
||||
import {
|
||||
antigravityCacheFileName,
|
||||
clearAntigravityCacheStates,
|
||||
createAntigravityProvider,
|
||||
flushAntigravityCache,
|
||||
|
|
@ -53,7 +55,7 @@ async function seedAntigravityCache(
|
|||
): Promise<void> {
|
||||
const sourceStat = await stat(sourcePath)
|
||||
await mkdir(cacheDir, { recursive: true })
|
||||
await writeFile(join(cacheDir, 'antigravity-results.json'), JSON.stringify({
|
||||
await writeFile(join(cacheDir, antigravityCacheFileName()), JSON.stringify({
|
||||
version: 5,
|
||||
cascades: {
|
||||
shared: {
|
||||
|
|
@ -107,7 +109,7 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
await writeCachedCodexResults(sourcePath, 'project-b', [call('codex', 'from-b')], fingerprint!)
|
||||
await flushCodexCache()
|
||||
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, 'codex-results.json'), 'utf8'))
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, codexCacheFileName()), 'utf8'))
|
||||
expect(diskB.files[sourcePath].calls.map((entry: ParsedProviderCall) => entry.model)).toEqual(['from-b'])
|
||||
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheA
|
||||
|
|
@ -129,12 +131,12 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
await writeCachedCodexResults(sourceB, 'project-b', [call('codex', 'dirty-b')], (await fingerprintFile(sourceB))!)
|
||||
await flushCodexCache()
|
||||
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, 'codex-results.json'), 'utf8'))
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, codexCacheFileName()), 'utf8'))
|
||||
expect(Object.keys(diskB.files)).toEqual([sourceB])
|
||||
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheA
|
||||
await flushCodexCache()
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, 'codex-results.json'), 'utf8'))
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, codexCacheFileName()), 'utf8'))
|
||||
expect(Object.keys(diskA.files)).toEqual([sourceA])
|
||||
})
|
||||
|
||||
|
|
@ -200,9 +202,9 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
expect(projects.some(project => project.sessions.some(session =>
|
||||
session.turns.some(turn => turn.assistantCalls.some(entry => entry.provider === 'codex'))
|
||||
))).toBe(true)
|
||||
expect(existsSync(join(cacheA, 'codex-results.json'))).toBe(true)
|
||||
expect(existsSync(join(cacheB, 'codex-results.json'))).toBe(false)
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, 'codex-results.json'), 'utf8'))
|
||||
expect(existsSync(join(cacheA, codexCacheFileName()))).toBe(true)
|
||||
expect(existsSync(join(cacheB, codexCacheFileName()))).toBe(false)
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, codexCacheFileName()), 'utf8'))
|
||||
expect(diskA.files[sourcePath].calls).toHaveLength(1)
|
||||
clearSessionCache()
|
||||
})
|
||||
|
|
@ -238,10 +240,10 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
process.env['CODEBURN_CACHE_DIR'] = cacheB
|
||||
await flushAntigravityCache(new Set(), cacheA)
|
||||
|
||||
expect(existsSync(join(cacheB, 'antigravity-results.json'))).toBe(true)
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, 'antigravity-results.json'), 'utf8'))
|
||||
expect(existsSync(join(cacheB, antigravityCacheFileName()))).toBe(true)
|
||||
const diskB = JSON.parse(await readFile(join(cacheB, antigravityCacheFileName()), 'utf8'))
|
||||
expect(diskB.cascades.shared.calls[0].model).toBe('from-b')
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, 'antigravity-results.json'), 'utf8'))
|
||||
const diskA = JSON.parse(await readFile(join(cacheA, antigravityCacheFileName()), 'utf8'))
|
||||
expect(diskA.cascades).toEqual({})
|
||||
})
|
||||
|
||||
|
|
@ -261,9 +263,9 @@ describe('call-time CODEBURN_CACHE_DIR isolation', () => {
|
|||
// Another process republishes both cache files. Without the clear, the
|
||||
// resident keeps serving its warm copies.
|
||||
await seedAntigravityCache(cacheDir, antigravitySource, 'after')
|
||||
const codexDisk = JSON.parse(await readFile(join(cacheDir, 'codex-results.json'), 'utf8'))
|
||||
const codexDisk = JSON.parse(await readFile(join(cacheDir, codexCacheFileName()), 'utf8'))
|
||||
codexDisk.files[codexSource].calls[0].model = 'after'
|
||||
await writeFile(join(cacheDir, 'codex-results.json'), JSON.stringify(codexDisk))
|
||||
await writeFile(join(cacheDir, codexCacheFileName()), JSON.stringify(codexDisk))
|
||||
|
||||
expect((await readCachedCodexResults(codexSource))?.calls.map(entry => entry.model)).toEqual(['before'])
|
||||
expect(await readAntigravityModel(antigravitySource)).toBe('before')
|
||||
|
|
|
|||
|
|
@ -79,7 +79,12 @@ describe('warm refresh child-process regression', () => {
|
|||
const loserOutcome = await waitForAny(barriers, [
|
||||
`${loser}.timed-out`, `${loser}.parsed`, `${loser}.completed-by-other`, `${loser}.unavailable`,
|
||||
])
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).toBe(`${loser}.timed-out`)
|
||||
// Exactly one owner publishes. The loser of a stale-lock contest is not
|
||||
// the owner: `timed-out` is the usual wait-out, but missing+missing during
|
||||
// the winner's unlink-guard/create-successor gap is honestly
|
||||
// `completed-by-other` (#904). Do not require timed-out.
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).not.toBe(`${loser}.parsed`)
|
||||
expect([`${loser}.timed-out`, `${loser}.completed-by-other`, `${loser}.unavailable`]).toContain(loserOutcome)
|
||||
await writeFile(join(barriers, `${winner}.save`), '')
|
||||
await Promise.all([waitForExit(a), waitForExit(b)])
|
||||
await expect(stat(join(cacheDir, 'session-refresh.lock.takeover'))).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
|
|
@ -113,7 +118,12 @@ describe('warm refresh child-process regression', () => {
|
|||
const loserOutcome = await waitForAny(barriers, [
|
||||
`${loser}.timed-out`, `${loser}.parsed`, `${loser}.completed-by-other`, `${loser}.unavailable`,
|
||||
])
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).toBe(`${loser}.timed-out`)
|
||||
// Exactly one owner publishes. The loser of a stale-lock contest is not
|
||||
// the owner: `timed-out` is the usual wait-out, but missing+missing during
|
||||
// the winner's unlink-guard/create-successor gap is honestly
|
||||
// `completed-by-other` (#904). Do not require timed-out.
|
||||
expect(loserOutcome, (await readdir(barriers)).join(',')).not.toBe(`${loser}.parsed`)
|
||||
expect([`${loser}.timed-out`, `${loser}.completed-by-other`, `${loser}.unavailable`]).toContain(loserOutcome)
|
||||
await writeFile(join(barriers, `${winner}.save`), '')
|
||||
await Promise.all([waitForExit(a), waitForExit(b)])
|
||||
await expect(stat(join(cacheDir, 'session-refresh.lock.takeover'))).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
|
|
|
|||
|
|
@ -88,7 +88,8 @@ describe('codex parser change invalidates stale session-cache (#478/#513)', () =
|
|||
}
|
||||
}
|
||||
await writeCacheOnDisk(cache)
|
||||
const codexCachePath = join(CACHE_DIR, 'codex-results.json')
|
||||
const { codexCacheFileName } = await import('../src/codex-cache.js')
|
||||
const codexCachePath = join(CACHE_DIR, codexCacheFileName())
|
||||
const codexCache = JSON.parse(await readFile(codexCachePath, 'utf8'))
|
||||
codexCache.version = 4
|
||||
for (const f of Object.values(codexCache.files) as any[]) {
|
||||
|
|
|
|||
|
|
@ -17,6 +17,12 @@ describe('codexCreditRate', () => {
|
|||
expect(codexCreditRate('gpt-4o')).toBeNull()
|
||||
expect(codexCreditRate('claude-opus-4-8')).toBeNull()
|
||||
})
|
||||
|
||||
it('resolves the auto-review activity id to the same rate as GPT-5.5', () => {
|
||||
expect(codexCreditRate('codex-auto-review')).toEqual(codexCreditRate('gpt-5.5'))
|
||||
expect(codexCreditRate('codex-auto-review')).not.toBeNull()
|
||||
expect(codexCreditRate('CODEX-AUTO-REVIEW')).toEqual(codexCreditRate('gpt-5.5'))
|
||||
})
|
||||
})
|
||||
|
||||
describe('codexCredits', () => {
|
||||
|
|
@ -32,11 +38,6 @@ describe('codexCredits', () => {
|
|||
expect(codexCredits('gpt-5.5', { inputTokens: 0, cachedReadTokens: 1_000_000, outputTokens: 0 })).toBe(12.5)
|
||||
})
|
||||
|
||||
it('folds reasoning tokens into the output rate', () => {
|
||||
// 500k output + 500k reasoning = 1M output-billed => 750 credits.
|
||||
expect(codexCredits('gpt-5.5', { inputTokens: 0, cachedReadTokens: 0, outputTokens: 500_000, reasoningTokens: 500_000 })).toBe(750)
|
||||
})
|
||||
|
||||
it('sums a mixed record (gpt-5.4)', () => {
|
||||
// 2M input (125) + 1M cached (6.25) + 0.5M output (187.5) = 318.75
|
||||
const credits = codexCredits('gpt-5.4', { inputTokens: 2_000_000, cachedReadTokens: 1_000_000, outputTokens: 500_000 })
|
||||
|
|
@ -50,4 +51,8 @@ describe('codexCredits', () => {
|
|||
it('returns null for an unknown model', () => {
|
||||
expect(codexCredits('gpt-4o', { inputTokens: 1_000_000, cachedReadTokens: 0, outputTokens: 0 })).toBeNull()
|
||||
})
|
||||
|
||||
it('charges auto-review at the GPT-5.5 credit rate, not null', () => {
|
||||
expect(codexCredits('codex-auto-review', { inputTokens: 1_000_000, cachedReadTokens: 0, outputTokens: 0 })).toBe(125)
|
||||
})
|
||||
})
|
||||
|
|
|
|||
69
tests/codex-pricing-1075-rehydrate.test.ts
Normal file
69
tests/codex-pricing-1075-rehydrate.test.ts
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
// #1075, cost site 2 of 2. Codex is NOT on parser.ts's reported-cost
|
||||
// pass-through allowlist, so the session cache stores its calls with
|
||||
// `costUSD: undefined` and every warm run re-prices them from the stored token
|
||||
// buckets in cachedCallToApiCall. That line and the one in the codex provider
|
||||
// are twins: if only one drops the reasoning double-count, a user's number
|
||||
// changes between a cold and a warm run. This drives the full parseAllSessions
|
||||
// pipeline twice against the same file to prove they agree.
|
||||
//
|
||||
// Own file because the codex provider captures CODEX_HOME when its module is
|
||||
// first evaluated, so the env must be set before any import of it.
|
||||
|
||||
import { afterAll, beforeEach, expect, it, vi } from 'vitest'
|
||||
import { mkdir, rm, writeFile } from 'fs/promises'
|
||||
import { join } from 'path'
|
||||
|
||||
const testRoot = vi.hoisted(() => {
|
||||
const root = `${process.env['TMPDIR'] || '/tmp'}/codex-1075-rehydrate-${process.pid}-${Date.now()}`
|
||||
process.env['HOME'] = `${root}/home`
|
||||
process.env['USERPROFILE'] = `${root}/home`
|
||||
process.env['CODEX_HOME'] = `${root}/codex`
|
||||
return root
|
||||
})
|
||||
|
||||
const CODEX_HOME = join(testRoot, 'codex')
|
||||
const CACHE_DIR = join(testRoot, 'cache')
|
||||
|
||||
// gpt-5.5: input 5e-6, output 30e-6, cacheRead 5e-7 (src/data/litellm-snapshot.json).
|
||||
// 800 uncached input + 200 cached + 1000 output, of which 400 are reasoning.
|
||||
const EXPECTED = 800 * 5e-6 + 200 * 5e-7 + 1000 * 30e-6
|
||||
|
||||
beforeEach(() => {
|
||||
process.env['HOME'] = join(testRoot, 'home')
|
||||
process.env['USERPROFILE'] = join(testRoot, 'home')
|
||||
process.env['CODEX_HOME'] = CODEX_HOME
|
||||
process.env['CODEBURN_CACHE_DIR'] = CACHE_DIR
|
||||
})
|
||||
|
||||
afterAll(async () => {
|
||||
await rm(testRoot, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
it('prices a codex call the same on a cold parse and a cache-rehydrated read', async () => {
|
||||
const sessionDir = join(CODEX_HOME, 'sessions', '2026', '08', '16')
|
||||
await mkdir(sessionDir, { recursive: true })
|
||||
await mkdir(CACHE_DIR, { recursive: true })
|
||||
const usage = { input_tokens: 1000, cached_input_tokens: 200, output_tokens: 1000, reasoning_output_tokens: 400, total_tokens: 2000 }
|
||||
await writeFile(join(sessionDir, 'rollout-1075.jsonl'), [
|
||||
JSON.stringify({ type: 'session_meta', timestamp: '2026-08-16T10:00:00Z', payload: { session_id: 's1075', model: 'gpt-5.5', cwd: '/Users/test/proj', originator: 'codex_cli_rs' } }),
|
||||
JSON.stringify({ type: 'response_item', timestamp: '2026-08-16T10:00:10Z', payload: { type: 'message', role: 'user', content: [{ type: 'input_text', text: 'hello' }] } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-08-16T10:01:00Z', payload: { type: 'token_count', info: { model: 'gpt-5.5', last_token_usage: usage, total_token_usage: usage } } }),
|
||||
].join('\n') + '\n')
|
||||
|
||||
const { clearSessionCache, parseAllSessions } = await import('../src/parser.js')
|
||||
|
||||
clearSessionCache()
|
||||
const cold = await parseAllSessions(undefined, 'codex')
|
||||
const coldCost = cold.reduce((sum, p) => sum + p.totalCostUSD, 0)
|
||||
|
||||
// Drop the in-memory cache only: session-cache.json on disk now serves the
|
||||
// unchanged file, so this run's cost comes out of cachedCallToApiCall.
|
||||
clearSessionCache()
|
||||
const warm = await parseAllSessions(undefined, 'codex')
|
||||
const warmCost = warm.reduce((sum, p) => sum + p.totalCostUSD, 0)
|
||||
|
||||
// Revert only src/providers/codex.ts and the cold leg breaks; revert only
|
||||
// src/parser.ts's outputForCost and the warm leg breaks.
|
||||
expect(coldCost).toBeCloseTo(EXPECTED, 12)
|
||||
expect(warmCost).toBeCloseTo(EXPECTED, 12)
|
||||
})
|
||||
353
tests/codex-pricing-1075.test.ts
Normal file
353
tests/codex-pricing-1075.test.ts
Normal file
|
|
@ -0,0 +1,353 @@
|
|||
// Regression suite for #1075 (reported by chr-evensen).
|
||||
//
|
||||
// Two independent codex pricing bugs, each with the site that would silently
|
||||
// drift from its twin if only one half were reverted:
|
||||
//
|
||||
// A. reasoning_output_tokens is a SUBSET of output_tokens (OpenAI bills
|
||||
// reasoning as part of output; every token_count event in a 134k-event
|
||||
// corpus satisfies input + output == total), but codeburn added the two.
|
||||
// Priced in TWO places -- the fresh parse in src/providers/codex.ts and
|
||||
// the cache-rehydration re-price in src/parser.ts -- plus three display
|
||||
// sums. Both cost sites now go through billableOutputTokens(). The
|
||||
// cache-rehydration half lives in codex-pricing-1075-rehydrate.test.ts,
|
||||
// which needs CODEX_HOME set before the provider module is evaluated.
|
||||
//
|
||||
// B. cache_write_input_tokens was never read. It is now carved out of the
|
||||
// uncached-input bucket, but ONLY on models whose pricing source carries
|
||||
// an explicit cache-write rate: buildCosts() fabricates 1.25x input when
|
||||
// the source omits one, which is right for Anthropic but would invent a
|
||||
// surcharge OpenAI never charged on every pre-5.6 model.
|
||||
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises'
|
||||
import { tmpdir } from 'os'
|
||||
import { join } from 'path'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
|
||||
import { aggregateAudit } from '../src/audit-report.js'
|
||||
import { aggregateModels } from '../src/models-report.js'
|
||||
import { clearCodexMemCaches, readCachedCodexResults } from '../src/codex-cache.js'
|
||||
import { currentTzKey, ensureCacheHydrated, toDateString, type DailyEntry } from '../src/daily-cache.js'
|
||||
import { createCodexProvider } from '../src/providers/codex.js'
|
||||
import type { ParsedProviderCall } from '../src/providers/types.js'
|
||||
import type {
|
||||
ClassifiedTurn,
|
||||
ParsedApiCall,
|
||||
ProjectSummary,
|
||||
SessionSummary,
|
||||
TaskCategory,
|
||||
TokenUsage,
|
||||
} from '../src/types.js'
|
||||
|
||||
// Snapshot ground truth (src/data/litellm-snapshot.json), USD per token:
|
||||
// gpt-5.6-terra input 2e-6 output 12e-6 cacheWrite 2.5e-6 (EXPLICIT) cacheRead 2e-7
|
||||
// gpt-5.5 input 5e-6 output 30e-6 cacheWrite null (fabricated) cacheRead 5e-7
|
||||
const TERRA = { input: 2e-6, output: 12e-6, cacheWrite: 2.5e-6, cacheRead: 2e-7 }
|
||||
const GPT55 = { input: 5e-6, output: 30e-6, cacheRead: 5e-7 }
|
||||
|
||||
let tmpDir: string
|
||||
beforeEach(async () => { tmpDir = await mkdtemp(join(tmpdir(), 'codex-1075-')) })
|
||||
afterEach(async () => { await rm(tmpDir, { recursive: true, force: true }) })
|
||||
|
||||
type Usage = {
|
||||
input_tokens: number
|
||||
cached_input_tokens?: number
|
||||
cache_write_input_tokens?: number
|
||||
output_tokens: number
|
||||
reasoning_output_tokens?: number
|
||||
}
|
||||
|
||||
async function parseOneEvent(model: string, usage: Usage): Promise<ParsedProviderCall> {
|
||||
const total = usage.input_tokens + usage.output_tokens
|
||||
const sessionDir = join(tmpDir, 'sessions', '2026', '08', '16')
|
||||
await mkdir(sessionDir, { recursive: true })
|
||||
const filePath = join(sessionDir, `rollout-${model}-${Math.random().toString(36).slice(2)}.jsonl`)
|
||||
await writeFile(filePath, [
|
||||
JSON.stringify({
|
||||
type: 'session_meta',
|
||||
timestamp: '2026-08-16T10:00:00Z',
|
||||
payload: { cwd: '/Users/t/p', originator: 'codex-cli', session_id: 's1075', model },
|
||||
}),
|
||||
JSON.stringify({
|
||||
type: 'event_msg',
|
||||
timestamp: '2026-08-16T10:01:00Z',
|
||||
payload: {
|
||||
type: 'token_count',
|
||||
info: { model, last_token_usage: { ...usage, total_tokens: total }, total_token_usage: { ...usage, total_tokens: total } },
|
||||
},
|
||||
}),
|
||||
].join('\n') + '\n')
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const parser = provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set())
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of parser.parse()) calls.push(call)
|
||||
expect(calls).toHaveLength(1)
|
||||
return calls[0]!
|
||||
}
|
||||
|
||||
// ── Fix A: reasoning is already inside output ─────────────────────────────
|
||||
|
||||
describe('#1075 A - reasoning is not billed on top of output', () => {
|
||||
it('prices a fresh codex parse from output_tokens alone', async () => {
|
||||
const call = await parseOneEvent('gpt-5.5', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
output_tokens: 1000,
|
||||
reasoning_output_tokens: 400,
|
||||
})
|
||||
|
||||
// 800 uncached input + 200 cached + 1000 output. The 400 reasoning tokens
|
||||
// are INSIDE the 1000, so they must not be priced again.
|
||||
const expected = 800 * GPT55.input + 200 * GPT55.cacheRead + 1000 * GPT55.output
|
||||
expect(call.costUSD).toBeCloseTo(expected, 12)
|
||||
// Guard the direction: the pre-fix arithmetic charged 1400 output tokens.
|
||||
const preFix = 800 * GPT55.input + 200 * GPT55.cacheRead + 1400 * GPT55.output
|
||||
expect(call.costUSD).toBeLessThan(preFix)
|
||||
// The raw fields are still reported untouched; only the pricing changed.
|
||||
expect(call.outputTokens).toBe(1000)
|
||||
expect(call.reasoningTokens).toBe(400)
|
||||
})
|
||||
|
||||
it('does not double-count reasoning in the displayed output tokens', async () => {
|
||||
const codex = makeApiCall('codex', 'gpt-5.5', { outputTokens: 1000, reasoningTokens: 400 })
|
||||
// A provider that really does report reasoning as a separate bucket keeps
|
||||
// the additive behaviour, so this is a codex carve-out and not a blanket
|
||||
// change to every display sum. Gemini documents "thoughts" as genuinely
|
||||
// separate from output (src/providers/gemini.ts), unlike codex/claude.
|
||||
const additive = makeApiCall('gemini', 'gemini-2.5-pro', { outputTokens: 1000, reasoningTokens: 400 })
|
||||
const projects = [makeProject([codex, additive])]
|
||||
|
||||
const auditRows = await aggregateAudit(projects)
|
||||
expect(auditRows.find(r => r.provider === 'codex')!.displayed.outputTokens).toBe(1000)
|
||||
expect(auditRows.find(r => r.provider === 'gemini')!.displayed.outputTokens).toBe(1400)
|
||||
|
||||
const modelRows = await aggregateModels(projects)
|
||||
expect(modelRows.find(r => r.provider === 'codex')!.outputTokens).toBe(1000)
|
||||
expect(modelRows.find(r => r.provider === 'gemini')!.outputTokens).toBe(1400)
|
||||
})
|
||||
})
|
||||
|
||||
// ── Fix B: cache_write_input_tokens, guarded ──────────────────────────────
|
||||
|
||||
describe('#1075 B - cache_write_input_tokens', () => {
|
||||
it('prices cache writes at the explicit rate on gpt-5.6-terra', async () => {
|
||||
const call = await parseOneEvent('gpt-5.6-terra', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
cache_write_input_tokens: 300,
|
||||
output_tokens: 100,
|
||||
})
|
||||
|
||||
expect(call.inputTokens).toBe(500)
|
||||
expect(call.cacheCreationInputTokens).toBe(300)
|
||||
expect(call.cacheReadInputTokens).toBe(200)
|
||||
const expected =
|
||||
500 * TERRA.input +
|
||||
300 * TERRA.cacheWrite +
|
||||
200 * TERRA.cacheRead +
|
||||
100 * TERRA.output
|
||||
expect(expected).toBeCloseTo(0.00299, 12)
|
||||
expect(call.costUSD).toBeCloseTo(expected, 12)
|
||||
})
|
||||
|
||||
it('THE GUARD: leaves cache writes in the input bucket when the model has no explicit rate', async () => {
|
||||
// gpt-5.5 carries `null` for cache_creation_input_token_cost, so
|
||||
// buildCosts fabricates 1.25x input for it. OpenAI charges nothing extra
|
||||
// to write cache before gpt-5.6, so routing these tokens through that
|
||||
// fabricated rate would invent a surcharge. Cost must be byte-identical to
|
||||
// the pre-fix number. Delete the guard and this test fails.
|
||||
const withWrite = await parseOneEvent('gpt-5.5', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
cache_write_input_tokens: 300,
|
||||
output_tokens: 100,
|
||||
})
|
||||
const withoutWrite = await parseOneEvent('gpt-5.5', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
output_tokens: 100,
|
||||
})
|
||||
|
||||
expect(withWrite.inputTokens).toBe(800)
|
||||
expect(withWrite.cacheCreationInputTokens).toBe(0)
|
||||
const expected = 800 * GPT55.input + 200 * GPT55.cacheRead + 100 * GPT55.output
|
||||
expect(withWrite.costUSD).toBeCloseTo(expected, 12)
|
||||
expect(withWrite.costUSD).toBeCloseTo(withoutWrite.costUSD, 12)
|
||||
// The fabricated rate is 1.25 x 5e-6; make sure not a cent of it landed.
|
||||
expect(withWrite.costUSD).toBeLessThan(expected + 300 * GPT55.input * 1.25)
|
||||
})
|
||||
|
||||
it('clamps a cache-write count larger than the uncached input', async () => {
|
||||
const call = await parseOneEvent('gpt-5.6-terra', {
|
||||
input_tokens: 1000,
|
||||
cached_input_tokens: 200,
|
||||
cache_write_input_tokens: 5000,
|
||||
output_tokens: 100,
|
||||
})
|
||||
|
||||
expect(call.inputTokens).toBe(0)
|
||||
expect(call.cacheCreationInputTokens).toBe(800)
|
||||
expect(call.costUSD).toBeCloseTo(800 * TERRA.cacheWrite + 200 * TERRA.cacheRead + 100 * TERRA.output, 12)
|
||||
})
|
||||
})
|
||||
|
||||
// ── Cache invalidation: a cost change must not be served from stale bytes ──
|
||||
|
||||
describe('#1075 cache invalidation', () => {
|
||||
it('discards a v10 codex results cache (it stores costUSD verbatim)', async () => {
|
||||
const cacheDir = join(tmpDir, 'cache')
|
||||
await mkdir(cacheDir, { recursive: true })
|
||||
const sessionFile = join(tmpDir, 'rollout-stale.jsonl')
|
||||
await writeFile(sessionFile, '{}\n')
|
||||
|
||||
const { statSync } = await import('fs')
|
||||
const s = statSync(sessionFile)
|
||||
const stale: ParsedProviderCall = {
|
||||
provider: 'codex',
|
||||
model: 'gpt-5.5',
|
||||
inputTokens: 800,
|
||||
outputTokens: 1000,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 200,
|
||||
cachedInputTokens: 200,
|
||||
reasoningTokens: 400,
|
||||
webSearchRequests: 0,
|
||||
costUSD: 0.0445, // the pre-fix, reasoning-double-counted number
|
||||
tools: [],
|
||||
bashCommands: [],
|
||||
timestamp: '2026-08-16T10:01:00Z',
|
||||
speed: 'standard',
|
||||
deduplicationKey: 'codex:stale',
|
||||
}
|
||||
await writeFile(join(cacheDir, 'codex-results.json'), JSON.stringify({
|
||||
version: 10,
|
||||
files: { [sessionFile]: { dev: s.dev, ino: s.ino, mtimeMs: s.mtimeMs, sizeBytes: s.size, project: 'p', calls: [stale] } },
|
||||
}))
|
||||
|
||||
const prevCacheDir = process.env['CODEBURN_CACHE_DIR']
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheDir
|
||||
try {
|
||||
clearCodexMemCaches()
|
||||
// Revert CODEX_CACHE_VERSION to 10 and this returns the stale $0.0445 call.
|
||||
expect(await readCachedCodexResults(sessionFile)).toBeNull()
|
||||
} finally {
|
||||
if (prevCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']; else process.env['CODEBURN_CACHE_DIR'] = prevCacheDir
|
||||
}
|
||||
})
|
||||
|
||||
it('re-derives days finalized at daily-cache v20', async () => {
|
||||
const cacheRoot = join(tmpDir, 'daily')
|
||||
await mkdir(cacheRoot, { recursive: true })
|
||||
const prevCacheDir = process.env['CODEBURN_CACHE_DIR']
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
try {
|
||||
const date = toDateString(new Date(Date.now() - 7 * 24 * 60 * 60 * 1000))
|
||||
const yesterday = toDateString(new Date(Date.now() - 24 * 60 * 60 * 1000))
|
||||
const oldPath = join(cacheRoot, 'daily-cache.v20.json')
|
||||
const oldCache = {
|
||||
version: 20,
|
||||
savingsConfigHash: 'cfg',
|
||||
tzKey: currentTzKey(),
|
||||
lastComputedDate: yesterday,
|
||||
days: [codexDay(date, 99)],
|
||||
complete: true,
|
||||
watermarkTrusted: true,
|
||||
}
|
||||
await writeFile(oldPath, JSON.stringify(oldCache))
|
||||
|
||||
let parseCount = 0
|
||||
const hydrated = await ensureCacheHydrated(
|
||||
async () => { parseCount++; return [] },
|
||||
() => [codexDay(date, 2)],
|
||||
'cfg',
|
||||
() => true,
|
||||
)
|
||||
|
||||
// Drop MIN_SUPPORTED_VERSION back to 20 and the v20 day is trusted as-is,
|
||||
// so parseCount stays 0 and the day keeps its overstated $99.
|
||||
expect(parseCount).toBe(1)
|
||||
expect(hydrated.days.find(d => d.date === date)?.cost).toBe(2)
|
||||
expect(JSON.parse(await readFile(oldPath, 'utf8'))).toEqual(oldCache)
|
||||
} finally {
|
||||
if (prevCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']; else process.env['CODEBURN_CACHE_DIR'] = prevCacheDir
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
// ── fixtures ──────────────────────────────────────────────────────────────
|
||||
|
||||
function makeApiCall(provider: string, model: string, usage: Partial<TokenUsage>): ParsedApiCall {
|
||||
return {
|
||||
provider,
|
||||
model,
|
||||
usage: {
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 0,
|
||||
cachedInputTokens: 0,
|
||||
reasoningTokens: 0,
|
||||
webSearchRequests: 0,
|
||||
...usage,
|
||||
},
|
||||
costUSD: 0,
|
||||
tools: [],
|
||||
mcpTools: [],
|
||||
skills: [],
|
||||
hasAgentSpawn: false,
|
||||
hasPlanMode: false,
|
||||
speed: 'standard',
|
||||
timestamp: '2026-08-16T00:00:00.000Z',
|
||||
bashCommands: [],
|
||||
deduplicationKey: `${provider}-${model}`,
|
||||
}
|
||||
}
|
||||
|
||||
function makeProject(calls: ParsedApiCall[]): ProjectSummary {
|
||||
const turn: ClassifiedTurn = {
|
||||
userMessage: 't',
|
||||
assistantCalls: calls,
|
||||
timestamp: '2026-08-16T00:00:00.000Z',
|
||||
sessionId: 's1',
|
||||
category: 'feature' as TaskCategory,
|
||||
retries: 0,
|
||||
hasEdits: false,
|
||||
}
|
||||
const session: SessionSummary = {
|
||||
sessionId: 's1',
|
||||
project: 'p',
|
||||
firstTimestamp: '2026-08-16T00:00:00.000Z',
|
||||
lastTimestamp: '2026-08-16T00:00:00.000Z',
|
||||
totalCostUSD: 0,
|
||||
totalInputTokens: 0,
|
||||
totalOutputTokens: 0,
|
||||
totalCacheReadTokens: 0,
|
||||
totalCacheWriteTokens: 0,
|
||||
apiCalls: 0,
|
||||
turns: [turn],
|
||||
modelBreakdown: {},
|
||||
toolBreakdown: {},
|
||||
mcpBreakdown: {},
|
||||
bashBreakdown: {},
|
||||
categoryBreakdown: {} as SessionSummary['categoryBreakdown'],
|
||||
skillBreakdown: {},
|
||||
}
|
||||
return { project: 'p', projectPath: 'p', sessions: [session], totalCostUSD: 0, totalApiCalls: 0 }
|
||||
}
|
||||
|
||||
function codexDay(date: string, cost: number): DailyEntry {
|
||||
const tokens = { inputTokens: 100, outputTokens: 20, cacheReadTokens: 30, cacheWriteTokens: 0 }
|
||||
return {
|
||||
date,
|
||||
cost,
|
||||
savingsUSD: 0,
|
||||
calls: 1,
|
||||
sessions: 1,
|
||||
...tokens,
|
||||
editTurns: 0,
|
||||
oneShotTurns: 0,
|
||||
models: { 'GPT-5.5': { calls: 1, cost, savingsUSD: 0, ...tokens } },
|
||||
categories: {},
|
||||
providers: { codex: { calls: 1, cost, savingsUSD: 0, sessions: 1, ...tokens } },
|
||||
}
|
||||
}
|
||||
|
|
@ -21,8 +21,38 @@ describe('Codex throughput prototype', () => {
|
|||
|
||||
const points = await readCodexThroughput(path)
|
||||
expect(points).toHaveLength(2)
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 50, elapsedSeconds: 5, generatedTokensPerSecond: 10, activeDurationSeconds: 7, activeGeneratedTokensPerSecond: 21.428571428571427, toolWaitSeconds: 3, model: 'gpt-5.6-sol' })
|
||||
expect(renderCodexThroughput(points, path)).toContain('21.4 generated tokens/sec')
|
||||
// Reasoning is a subset of output_tokens (#1075/#1078), not additive: the
|
||||
// checkpoints report output 80/40 and reasoning 20/10, so the generated
|
||||
// numerator is output alone (80, then 40), matching the cost path.
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 40, elapsedSeconds: 5, generatedTokensPerSecond: 8, activeDurationSeconds: 7, activeGeneratedTokensPerSecond: (80 + 40) / 7, toolWaitSeconds: 3, model: 'gpt-5.6-sol' })
|
||||
expect(renderCodexThroughput(points, path)).toContain('17.1 generated tokens/sec')
|
||||
})
|
||||
|
||||
it('REGRESSION (#1079): does not add reasoning tokens on top of output for Tok/s', async () => {
|
||||
// Reasoning tokens are a SUBSET of output_tokens (#1075/#1078), not a
|
||||
// separate bucket. A single checkpoint reporting output=60, reasoning=40
|
||||
// must drive Tok/s off 60, not 100 -- summing them would double-count 40
|
||||
// tokens that are already inside the 60. If this ever reverts to
|
||||
// `outputTokens + reasoningTokens`, activeGeneratedTokensPerSecond becomes
|
||||
// 10 (100 tokens / 10s) instead of 6 (60 tokens / 10s).
|
||||
const dir = await mkdtemp(join(tmpdir(), 'codeburn-tps-regression-'))
|
||||
const path = join(dir, 'rollout.jsonl')
|
||||
await writeFile(path, [
|
||||
JSON.stringify({ type: 'session_meta', timestamp: '2026-07-25T00:00:00.000Z', payload: { model: 'gpt-5.5' } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-07-25T00:00:00.000Z', payload: { type: 'task_started' } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-07-25T00:00:10.000Z', payload: { type: 'token_count', info: { last_token_usage: { output_tokens: 60, reasoning_output_tokens: 40 }, total_token_usage: { total_tokens: 100, output_tokens: 60, reasoning_output_tokens: 40 } } } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-07-25T00:00:10.000Z', payload: { type: 'task_complete', duration_ms: 10000 } }),
|
||||
].join('\n'))
|
||||
|
||||
const points = await readCodexThroughput(path)
|
||||
expect(points).toHaveLength(1)
|
||||
expect(points[0]).toMatchObject({
|
||||
outputTokens: 60,
|
||||
reasoningTokens: 40,
|
||||
generatedTokens: 60,
|
||||
taskGeneratedTokens: 60,
|
||||
activeGeneratedTokensPerSecond: 6,
|
||||
})
|
||||
})
|
||||
|
||||
it('parses only appended complete lines while watching a growing rollout', async () => {
|
||||
|
|
@ -36,7 +66,9 @@ describe('Codex throughput prototype', () => {
|
|||
await appendFile(path, first.slice(40) + '\n' + second + '\n')
|
||||
const points = await reader.update(path)
|
||||
expect(points).toHaveLength(2)
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 5, generatedTokensPerSecond: 5 })
|
||||
// second checkpoint: output 4 + reasoning 1 -> billable numerator is 4
|
||||
// (reasoning already inside output_tokens), not the additive 5.
|
||||
expect(points[1]).toMatchObject({ generatedTokens: 4, generatedTokensPerSecond: 4 })
|
||||
})
|
||||
|
||||
it('ignores replayed pre-fork checkpoints before estimating new work', async () => {
|
||||
|
|
|
|||
|
|
@ -463,18 +463,19 @@ describe('buildPeriodDataFromDays', () => {
|
|||
})
|
||||
|
||||
describe('daily-cache ↔ report daily-bucket parity', () => {
|
||||
// The daily cache (history.daily + provider breakdown) and the live report /
|
||||
// headline (main.ts daily rollup) must bucket days by the SAME rule, or their
|
||||
// per-day totals drift and their period sums diverge from current.cost at
|
||||
// window boundaries — the V1 audit's constant -$3.45/-81-calls finding. Both
|
||||
// are now PER-CALL for cost/savings/calls (issue #852) with turn-level stats
|
||||
// still turn-anchored: this asserts per-day equality against a reference
|
||||
// that mirrors main.ts buildJsonReport's dailyMap fallback (each call on its
|
||||
// own date), plus the invariant history.daily Σ == report.daily Σ == total
|
||||
// call cost.
|
||||
// The daily cache (history.daily + provider breakdown) and JSON-report
|
||||
// daily[] rows (durable.days from buildDurablePeriod) must bucket days by the
|
||||
// SAME rule, or their per-day totals drift and their period sums diverge from
|
||||
// current.cost at window boundaries — the V1 audit's constant -$3.45/-81-calls
|
||||
// finding. Both are now PER-CALL for cost/savings/calls (issue #852) with
|
||||
// turn-level stats still turn-anchored: this asserts per-day equality against
|
||||
// an independent per-call oracle for the durable day aggregation used by
|
||||
// durable.days (each call on its own date), plus the invariant
|
||||
// history.daily Σ == report.daily Σ == total call cost.
|
||||
|
||||
// Mirrors the live report/headline daily rollup fallback in src/main.ts
|
||||
// (cost/savings/calls bucket under each call's own date).
|
||||
// Independent per-call reference for durable.days (cost/savings/calls bucket
|
||||
// under each call's own date). Not a live buildJsonReport fallback — that
|
||||
// path was deleted in #1067.
|
||||
function reportDailyByDate(projects: ProjectSummary[]): Record<string, number> {
|
||||
const byDate: Record<string, number> = {}
|
||||
for (const p of projects) {
|
||||
|
|
|
|||
|
|
@ -120,7 +120,7 @@ describe('granular history', () => {
|
|||
])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries.map(series => series.label)).toEqual([
|
||||
'sessio…3456 (claude) · Refactor billing module',
|
||||
'Refactor billing module (claude)',
|
||||
'sessio…3457 (claude) · repos/demo',
|
||||
'sessio…3458 (claude) · repos/demo',
|
||||
'sessio…3459 (claude) · repos/demo',
|
||||
|
|
@ -157,7 +157,7 @@ describe('granular history', () => {
|
|||
// 160px at the chart's 10px font fits roughly 31-32 lowercase glyphs;
|
||||
// compare a conservative prefix that must be visible in that budget.
|
||||
const visibleCharacterBudget = 24
|
||||
const visiblePrefixes = history.sessionSeries.map(series => series.label.slice(0, visibleCharacterBudget))
|
||||
const visiblePrefixes = history.sessionSeries.map(series => Array.from(series.label).slice(0, visibleCharacterBudget).join(''))
|
||||
expect(new Set(visiblePrefixes).size).toBe(2)
|
||||
expect(history.sessionSeries.map(series => series.label)).toEqual(expect.arrayContaining([
|
||||
expect.stringMatching(/^a1b2c3…7f01 \(claude\) · /),
|
||||
|
|
@ -165,6 +165,25 @@ describe('granular history', () => {
|
|||
]))
|
||||
})
|
||||
|
||||
it('counts the visible title-lead window in code points, not UTF-16 units', () => {
|
||||
const timestamp = '2026-07-15T12:05:00.000Z'
|
||||
const start = new Date('2026-07-15T00:00:00.000Z')
|
||||
const end = new Date('2026-07-15T23:59:59.999Z')
|
||||
const emojiPrefix = '😀'.repeat(12)
|
||||
const history = buildGranularHistory([project([
|
||||
{ id: 'emoji-alpha-aaaaaa', title: `${emojiPrefix} alpha work`, calls: [apiCall({ timestamp, cost: 1 })] },
|
||||
{ id: 'emoji-beta-bbbbbb', title: `${emojiPrefix} beta work`, calls: [apiCall({ timestamp, cost: 2 })] },
|
||||
])], { start, end }, end)
|
||||
|
||||
// 12 emoji = 12 glyphs / 24 UTF-16 units. A unit slice collides both
|
||||
// titles on the emoji run and would id-first; a code-point window still
|
||||
// sees " alpha" vs " beta" and can title-lead.
|
||||
expect(history.sessionSeries.map(series => series.label).sort()).toEqual([
|
||||
`${emojiPrefix} alpha work (claude)`,
|
||||
`${emojiPrefix} beta work (claude)`,
|
||||
].sort())
|
||||
})
|
||||
|
||||
it('sanitises control characters and ANSI escapes in session titles', () => {
|
||||
const timestamp = '2026-07-15T12:05:00.000Z'
|
||||
const start = new Date('2026-07-15T00:00:00.000Z')
|
||||
|
|
@ -175,7 +194,7 @@ describe('granular history', () => {
|
|||
calls: [apiCall({ timestamp, cost: 1 })],
|
||||
}])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries[0]?.label).toBe('sessio…3456 (claude) · Refactor billing module')
|
||||
expect(history.sessionSeries[0]?.label).toBe('Refactor billing module (claude)')
|
||||
expect(history.sessionSeries[0]?.label).not.toContain('\x1b')
|
||||
expect(history.sessionSeries[0]?.label).not.toContain('\x00')
|
||||
})
|
||||
|
|
@ -190,7 +209,7 @@ describe('granular history', () => {
|
|||
calls: [apiCall({ timestamp, cost: 1 })],
|
||||
}])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries[0]?.label).toBe('sessio…3456 (claude) · ' + 'x'.repeat(80))
|
||||
expect(history.sessionSeries[0]?.label).toBe('x'.repeat(80) + ' (claude)')
|
||||
})
|
||||
|
||||
it('caps session titles by code point without splitting an emoji', () => {
|
||||
|
|
@ -205,9 +224,8 @@ describe('granular history', () => {
|
|||
}])], { start, end }, end)
|
||||
|
||||
const label = history.sessionSeries[0]?.label ?? ''
|
||||
const titlePart = label.slice(label.indexOf(' · ') + 3)
|
||||
expect(titlePart).toBe('x'.repeat(79) + '😀')
|
||||
expect([...titlePart]).toEqual([...('x'.repeat(79) + '😀')])
|
||||
expect(label).toBe('x'.repeat(79) + '😀' + ' (claude)')
|
||||
expect([...label]).toEqual([...'x'.repeat(79), '😀', ' ', '(', 'c', 'l', 'a', 'u', 'd', 'e', ')'])
|
||||
})
|
||||
|
||||
it('prefers a title from any duplicate session summary sharing a key', () => {
|
||||
|
|
@ -230,7 +248,7 @@ describe('granular history', () => {
|
|||
])], { start, end }, end)
|
||||
|
||||
expect(history.sessionSeries).toHaveLength(1)
|
||||
expect(history.sessionSeries[0]?.label).toBe('sessio…3456 (claude) · Z recovered session title')
|
||||
expect(history.sessionSeries[0]?.label).toBe('Z recovered session title (claude)')
|
||||
})
|
||||
|
||||
it('fills idle buckets and keeps separate model and session lines from real call timestamps', () => {
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import chalk from 'chalk'
|
|||
import stripAnsi from 'strip-ansi'
|
||||
|
||||
import { aggregateModels, renderTable, renderMarkdown, renderJson, renderCsv, type ModelReportRow } from '../src/models-report.js'
|
||||
import { setModelAliases } from '../src/models.js'
|
||||
import type {
|
||||
ProjectSummary,
|
||||
SessionSummary,
|
||||
|
|
@ -37,6 +38,8 @@ function makeCall(opts: {
|
|||
output?: number
|
||||
cacheWrite?: number
|
||||
cacheRead?: number
|
||||
savingsUSD?: number
|
||||
savingsBaselineModel?: string
|
||||
}): ParsedApiCall {
|
||||
return {
|
||||
provider: opts.provider,
|
||||
|
|
@ -58,6 +61,8 @@ function makeCall(opts: {
|
|||
timestamp: '2026-05-09T00:00:00.000Z',
|
||||
bashCommands: [],
|
||||
deduplicationKey: `${opts.provider}-${opts.model}-${opts.costUSD}`,
|
||||
savingsUSD: opts.savingsUSD,
|
||||
savingsBaselineModel: opts.savingsBaselineModel,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -165,6 +170,32 @@ describe('aggregateModels', () => {
|
|||
expect(byKey['codex:gpt-5.5']!.credits).toBeCloseTo(887.5, 6)
|
||||
expect(byKey['codex:gpt-5']!.credits).toBeNull()
|
||||
expect(byKey['claude:claude-sonnet-4-6']!.credits).toBeNull()
|
||||
expect(byKey['codex:gpt-5.5']!.creditsIncomplete).toBeFalsy()
|
||||
})
|
||||
|
||||
it('partial-sums Codex credits when a merge mixes rated and unrated ids', async () => {
|
||||
setModelAliases({ 'codex-house-sku': 'gpt-5.5' })
|
||||
try {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [
|
||||
makeCall({ provider: 'codex', model: 'gpt-5.5', input: 0, output: 1_000_000, costUSD: 9 }),
|
||||
]),
|
||||
makeTurn('feature', [
|
||||
makeCall({ provider: 'codex', model: 'codex-house-sku', input: 0, output: 1_000_000, costUSD: 9 }),
|
||||
]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.model).toBe('gpt-5.5')
|
||||
// gpt-5.5 output is 750 credits/M; the aliased house SKU has no rate.
|
||||
expect(rows[0]!.credits).toBeCloseTo(750, 6)
|
||||
expect(rows[0]!.creditsIncomplete).toBe(true)
|
||||
expect(rows[0]!.calls).toBe(2)
|
||||
const parsed = JSON.parse(renderJson(rows))
|
||||
expect(parsed[0].credits).toBeCloseTo(750, 6)
|
||||
expect(parsed[0].creditsIncomplete).toBe(true)
|
||||
} finally {
|
||||
setModelAliases({})
|
||||
}
|
||||
})
|
||||
|
||||
it('includes credits in the JSON output', async () => {
|
||||
|
|
@ -188,6 +219,110 @@ describe('aggregateModels', () => {
|
|||
expect(rows[0]!.cacheReadTokens).toBe(4000) // not 8000
|
||||
})
|
||||
|
||||
it('falls back from a provider local table miss to the global short name', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'cursor-agent', model: 'gpt-5.6-sol', costUSD: 2.5 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.modelDisplayName).toBe('GPT-5.6 Sol (est.)')
|
||||
expect(rows[0]!.model).toBe('gpt-5.6-sol')
|
||||
})
|
||||
|
||||
it('resolves Fireworks path-form ids through the global table', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'cline', model: 'accounts/fireworks/models/kimi-k2p6', costUSD: 1 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.modelDisplayName).toBe('Kimi K2.6')
|
||||
expect(rows[0]!.model).toBe('accounts/fireworks/models/kimi-k2p6')
|
||||
})
|
||||
|
||||
it('merges two raw ids that resolve to the same alias-resolved canonical id', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('testing', [makeCall({
|
||||
provider: 'cline-cli', model: 'accounts/fireworks/models/glm-5p2',
|
||||
input: 800, output: 67, costUSD: 0.246,
|
||||
})]),
|
||||
makeTurn('conversation', [makeCall({
|
||||
provider: 'cline-cli', model: 'glm-5p2',
|
||||
input: 15, output: 1, costUSD: 0.019,
|
||||
})]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.provider).toBe('cline-cli')
|
||||
expect(rows[0]!.modelDisplayName).toBe('GLM-5.2')
|
||||
// First-seen raw id, not the lexicographically-smallest of the merge.
|
||||
expect(rows[0]!.model).toBe('accounts/fireworks/models/glm-5p2')
|
||||
expect(rows[0]!.inputTokens).toBe(815)
|
||||
expect(rows[0]!.outputTokens).toBe(68)
|
||||
expect(rows[0]!.costUSD).toBeCloseTo(0.265, 6)
|
||||
expect(rows[0]!.calls).toBe(2)
|
||||
expect(rows[0]!.topCategory).toBe('testing')
|
||||
expect(rows[0]!.topCategoryShare).toBeCloseTo(0.246 / 0.265, 3)
|
||||
})
|
||||
|
||||
it('does not pick the lexicographically-smallest raw id after a merge', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'kimi-k3', costUSD: 2 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'k3', costUSD: 1 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.model).toBe('kimi-k3')
|
||||
expect(rows[0]!.modelDisplayName).toBe('Kimi K3')
|
||||
expect(rows[0]!.calls).toBe(2)
|
||||
})
|
||||
|
||||
it('does not merge same-provider ids that only share a display name', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'gpt-5', costUSD: 3 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'codex', model: 'gpt-5-untracked-xyz', costUSD: 1 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'cline-cli', model: 'glm-5p2', costUSD: 1 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'cline-cli', model: 'GLM-5.2', costUSD: 2 })]),
|
||||
])])
|
||||
const codex = rows.filter(r => r.provider === 'codex')
|
||||
expect(codex).toHaveLength(2)
|
||||
expect(codex.every(r => r.modelDisplayName === 'GPT-5')).toBe(true)
|
||||
expect(new Set(codex.map(r => r.model))).toEqual(new Set(['gpt-5', 'gpt-5-untracked-xyz']))
|
||||
|
||||
const cline = rows.filter(r => r.provider === 'cline-cli')
|
||||
expect(cline).toHaveLength(2)
|
||||
expect(cline.every(r => r.modelDisplayName === 'GLM-5.2')).toBe(true)
|
||||
expect(new Set(cline.map(r => r.model))).toEqual(new Set(['glm-5p2', 'GLM-5.2']))
|
||||
})
|
||||
|
||||
it('clears a merged savings baseline when three raw ids disagree', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({
|
||||
provider: 'codex', model: 'k3',
|
||||
costUSD: 1, savingsUSD: 2, savingsBaselineModel: 'gpt-4o',
|
||||
})]),
|
||||
makeTurn('feature', [makeCall({
|
||||
provider: 'codex', model: 'kimi-k3',
|
||||
costUSD: 1, savingsUSD: 2, savingsBaselineModel: 'claude-sonnet-4-6',
|
||||
})]),
|
||||
makeTurn('feature', [makeCall({
|
||||
provider: 'codex', model: 'k3-agent',
|
||||
costUSD: 1, savingsUSD: 2, savingsBaselineModel: 'gpt-5',
|
||||
})]),
|
||||
])])
|
||||
expect(rows).toHaveLength(1)
|
||||
expect(rows[0]!.provider).toBe('codex')
|
||||
expect(rows[0]!.modelDisplayName).toBe('Kimi K3')
|
||||
expect(rows[0]!.savingsUSD).toBe(6)
|
||||
expect(rows[0]!.savingsBaselineModel).toBe('')
|
||||
expect(rows[0]!.calls).toBe(3)
|
||||
})
|
||||
|
||||
it('does not merge the same display name across providers', async () => {
|
||||
const rows = await aggregateModels([makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'cline-cli', model: 'glm-5p2', costUSD: 1 })]),
|
||||
makeTurn('feature', [makeCall({ provider: 'hermes', model: 'glm-5p2', costUSD: 2 })]),
|
||||
])])
|
||||
expect(rows).toHaveLength(2)
|
||||
expect(new Set(rows.map(r => r.provider))).toEqual(new Set(['cline-cli', 'hermes']))
|
||||
expect(rows.every(r => r.modelDisplayName === 'GLM-5.2')).toBe(true)
|
||||
})
|
||||
|
||||
it('reports the dominant task type with its cost share in default mode', async () => {
|
||||
const project = makeProject([
|
||||
makeTurn('feature', [makeCall({ provider: 'claude', model: 'claude-sonnet-4-6', costUSD: 6.0, input: 100, output: 20 })]),
|
||||
|
|
@ -237,12 +372,15 @@ describe('aggregateModels', () => {
|
|||
expect(above.find(r => r.provider === 'cursor')).toBeUndefined()
|
||||
})
|
||||
|
||||
// Providers that report reasoning as a bucket SEPARATE from output still get
|
||||
// it added in. Codex and claude do not - they bill reasoning inside
|
||||
// output_tokens - and that carve-out is covered in codex-pricing-1075.test.ts.
|
||||
it('counts reasoning tokens as output tokens', async () => {
|
||||
const project = makeProject([
|
||||
makeTurn('feature', [
|
||||
{
|
||||
provider: 'codex',
|
||||
model: 'gpt-5',
|
||||
provider: 'gemini',
|
||||
model: 'gemini-2.5-pro',
|
||||
usage: { ...emptyTokens(), inputTokens: 100, outputTokens: 50, reasoningTokens: 200 },
|
||||
costUSD: 1.0,
|
||||
tools: [],
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ import {
|
|||
findUnpricedModels,
|
||||
getModelCosts,
|
||||
getShortModelName,
|
||||
resolveCanonicalModelId,
|
||||
calculateCost,
|
||||
loadPricing,
|
||||
setModelAliases,
|
||||
|
|
@ -24,6 +25,7 @@ import {
|
|||
unpricedModelHint,
|
||||
} from '../src/models.js'
|
||||
import { getDailyCacheConfigHash } from '../src/usage-aggregator.js'
|
||||
import snapshotData from '../src/data/litellm-snapshot.json' with { type: 'json' }
|
||||
|
||||
beforeAll(async () => {
|
||||
await loadPricing()
|
||||
|
|
@ -89,6 +91,32 @@ describe('getModelCosts', () => {
|
|||
expect(getModelCosts('z-ai/glm-5.3')!.inputCostPerToken).toBe(sibling!.inputCostPerToken)
|
||||
})
|
||||
|
||||
it('prices gpt-5.6-codex and gpt-5.6-codex-max, sourced directly from the snapshot (#1077)', () => {
|
||||
// Directly checks the bundled snapshot data (not just the resolved lookup),
|
||||
// so this fails if the litellm-snapshot.json entries are ever reverted even
|
||||
// though getModelCosts would still resolve both ids via the `gpt-5.6` prefix
|
||||
// fallback - explicit rows are still correct and match every other Codex
|
||||
// generation LiteLLM ships (gpt-5-codex, gpt-5.1-codex, gpt-5.1-codex-max,
|
||||
// gpt-5.2-codex, gpt-5.3-codex all carry their base model's exact rate).
|
||||
const snapshot = snapshotData as Record<string, unknown>
|
||||
expect(snapshot['gpt-5.6-codex']).toEqual(snapshot['gpt-5.6'])
|
||||
expect(snapshot['gpt-5.6-codex-max']).toEqual(snapshot['gpt-5.6'])
|
||||
|
||||
const codex = getModelCosts('gpt-5.6-codex')
|
||||
const codexMax = getModelCosts('gpt-5.6-codex-max')
|
||||
expect(codex).not.toBeNull()
|
||||
expect(codexMax).not.toBeNull()
|
||||
expect(codex!.inputCostPerToken).toBe(5e-6)
|
||||
expect(codex!.outputCostPerToken).toBe(3e-5)
|
||||
expect(codex!.cacheWriteCostPerToken).toBe(6.25e-6)
|
||||
expect(codex!.cacheReadCostPerToken).toBe(5e-7)
|
||||
expect(codex!.cacheWriteCostIsExplicit).toBe(true)
|
||||
expect(codexMax).toEqual(codex)
|
||||
|
||||
expect(calculateCost('gpt-5.6-codex', 1_000_000, 1_000_000, 0, 0, 0)).toBeGreaterThan(0)
|
||||
expect(calculateCost('gpt-5.6-codex-max', 1_000_000, 1_000_000, 0, 0, 0)).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
// A price override on a synthetic bare id can only be reached if the leading
|
||||
// segment was stripped, so these assert the namespace allowlist itself without
|
||||
// pinning to any real model's presence in (or absence from) the snapshot.
|
||||
|
|
@ -131,6 +159,22 @@ describe('getModelCosts', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('resolveCanonicalModelId', () => {
|
||||
it('aliases, peels path-form ids, and leaves display-only collisions distinct', () => {
|
||||
expect(resolveCanonicalModelId('k3')).toBe('kimi-k3')
|
||||
expect(resolveCanonicalModelId('k3-agent')).toBe('kimi-k3')
|
||||
expect(resolveCanonicalModelId('kimi-k3')).toBe('kimi-k3')
|
||||
expect(resolveCanonicalModelId('accounts/fireworks/models/glm-5p2')).toBe('glm-5p2')
|
||||
expect(resolveCanonicalModelId('glm-5p2')).toBe('glm-5p2')
|
||||
expect(resolveCanonicalModelId('GLM-5.2')).toBe('glm-5p1')
|
||||
expect(resolveCanonicalModelId('gpt-5-fast')).toBe('gpt-5')
|
||||
expect(resolveCanonicalModelId('gpt-5-untracked-xyz')).toBe('gpt-5-untracked-xyz')
|
||||
expect(resolveCanonicalModelId('claude-opus-4.6')).toBe('claude-opus-4-6')
|
||||
expect(resolveCanonicalModelId('kimi-code')).toBe('kimi-k2-thinking')
|
||||
expect(resolveCanonicalModelId('cline-pass/kimi-k3')).toBe('kimi-k3')
|
||||
})
|
||||
})
|
||||
|
||||
describe('getShortModelName', () => {
|
||||
it('maps gpt-4o-mini correctly (not gpt-4o)', () => {
|
||||
expect(getShortModelName('gpt-4o-mini-2024-07-18')).toBe('GPT-4o Mini')
|
||||
|
|
@ -628,6 +672,8 @@ describe('Cursor model variants resolve to pricing', () => {
|
|||
['claude-4.6-haiku', 'claude-haiku-4-5'],
|
||||
// Cursor auto proxy
|
||||
['cursor-auto', 'claude-sonnet-4-5'],
|
||||
// Codex activity surface (official rate card, observed raw id)
|
||||
['codex-auto-review', 'gpt-5.5'],
|
||||
// OpenAI variants Cursor emits
|
||||
['gpt-5', 'gpt-5'],
|
||||
['gpt-5-fast', 'gpt-5'],
|
||||
|
|
@ -663,6 +709,29 @@ describe('Cursor model variants resolve to pricing', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('Codex activity ids (#1047)', () => {
|
||||
it('keeps the activity label instead of collapsing to the underlying model name', () => {
|
||||
expect(getShortModelName('codex-auto-review')).toBe('Codex Auto Review')
|
||||
})
|
||||
|
||||
it('prices as the exact bundled GPT-5.5 object, not an invented rate', () => {
|
||||
expect(getModelCosts('codex-auto-review')).toBe(getModelCosts('gpt-5.5'))
|
||||
const auto = calculateCost('codex-auto-review', 1_000_000, 1_000_000, 0, 0, 0)
|
||||
const gpt55 = calculateCost('gpt-5.5', 1_000_000, 1_000_000, 0, 0, 0)
|
||||
expect(auto).toBeGreaterThan(0)
|
||||
expect(auto).toBe(gpt55)
|
||||
})
|
||||
|
||||
it('does not invent a family or an unobserved sibling id', () => {
|
||||
expect(getModelCosts('codex-code-review')).toBeNull()
|
||||
expect(getModelCosts('codex-cloud-task')).toBeNull()
|
||||
expect(getModelCosts('codex-automation')).toBeNull()
|
||||
expect(getModelCosts('code-review')).toBeNull()
|
||||
expect(getModelCosts('auto-review')).toBeNull()
|
||||
expect(calculateCost('codex-code-review', 1_000_000, 1_000_000, 0, 0, 0)).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('Cursor house model pricing', () => {
|
||||
const cases: Array<[string, { input: number; output: number; cacheWrite: number; cacheRead: number }]> = [
|
||||
['composer-2.5', { input: 0.5, output: 2.5, cacheWrite: 0.5, cacheRead: 0.2 }],
|
||||
|
|
@ -772,6 +841,7 @@ describe('DeepSeek v4 models resolve to pricing', () => {
|
|||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await mkdir(cacheRoot, { recursive: true })
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
version: 2, // must match models.ts's CACHE_SCHEMA_VERSION or the cache is treated as a miss
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'gpt-4o-mini': {
|
||||
|
|
@ -797,6 +867,42 @@ describe('DeepSeek v4 models resolve to pricing', () => {
|
|||
})
|
||||
})
|
||||
|
||||
describe('pricing cache schema version (#1075/#1078 follow-up)', () => {
|
||||
it('discards a cache written by a pre-#1078 binary instead of reading its missing cacheWriteCostIsExplicit as false', async () => {
|
||||
const cacheRoot = await mkdtemp(join(tmpdir(), 'codeburn-pricing-cache-'))
|
||||
try {
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await mkdir(cacheRoot, { recursive: true })
|
||||
// Shape of a cache file written before #1078 added `version` and
|
||||
// `cacheWriteCostIsExplicit`: no version field, and entries missing the
|
||||
// key despite carrying a real (non-default) cache-write rate.
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'gpt-5.6': {
|
||||
inputCostPerToken: 5e-6,
|
||||
outputCostPerToken: 3e-5,
|
||||
cacheWriteCostPerToken: 6.25e-6,
|
||||
cacheReadCostPerToken: 5e-7,
|
||||
webSearchCostPerRequest: 0.01,
|
||||
fastMultiplier: 1,
|
||||
},
|
||||
},
|
||||
}), 'utf-8')
|
||||
|
||||
await loadPricing()
|
||||
|
||||
// Pre-fix, loadCachedPricing had no version check: it would read this
|
||||
// cache verbatim, and gpt-5.6's missing key would resolve to undefined
|
||||
// (falsy) here instead of the true its LiteLLM entry actually carries.
|
||||
expect(getModelCosts('gpt-5.6')!.cacheWriteCostIsExplicit).toBe(true)
|
||||
} finally {
|
||||
await rm(cacheRoot, { recursive: true, force: true })
|
||||
await loadPricing()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('provider pricing suffix variants', () => {
|
||||
const cases: Array<[string, string]> = [
|
||||
['GLM-4.7-TEE', 'glm-4.7'],
|
||||
|
|
@ -934,6 +1040,7 @@ describe('findUnpricedModels', () => {
|
|||
try {
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
version: 2, // must match models.ts's CACHE_SCHEMA_VERSION or the cache is treated as a miss
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'zz-zero-stub-model': {
|
||||
|
|
|
|||
|
|
@ -236,7 +236,8 @@ async function shardBodies(cacheDir: string): Promise<Record<string, string>> {
|
|||
/// The Codex incremental cache is a single JSON file; both runs read the same
|
||||
/// rollouts, so it must come out identical byte for byte.
|
||||
async function codexResults(cacheDir: string): Promise<string | null> {
|
||||
return readFile(join(cacheDir, 'codex-results.json'), 'utf-8').catch(() => null)
|
||||
const { codexCacheFileName } = await import('../src/codex-cache.js')
|
||||
return readFile(join(cacheDir, codexCacheFileName()), 'utf-8').catch(() => null)
|
||||
}
|
||||
|
||||
function runCli(args: string[], home: string, extraEnv: Record<string, string>) {
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ import { createRequire } from 'node:module'
|
|||
|
||||
import { isSqliteAvailable } from '../src/sqlite.js'
|
||||
import { clearSessionCache, parseAllSessions, setParseReuseValidator } from '../src/parser.js'
|
||||
import { loadCache, saveCache } from '../src/session-cache.js'
|
||||
import { computeEnvFingerprint, loadCache, PROVIDER_PARSE_VERSIONS, saveCache } from '../src/session-cache.js'
|
||||
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
|
||||
import type { SessionSource, SessionParser, ParsedProviderCall } from '../src/providers/types.js'
|
||||
|
||||
|
|
|
|||
|
|
@ -183,7 +183,8 @@ describe('codex incremental resume', () => {
|
|||
sessionPath = await writeRollout([meta(), ...tasks(1, 2)])
|
||||
await parse(cacheDir)
|
||||
|
||||
const cachePath = join(cacheDir, 'codex-results.json')
|
||||
const { codexCacheFileName } = await import('../../src/codex-cache.js')
|
||||
const cachePath = join(cacheDir, codexCacheFileName())
|
||||
const { readFile } = await import('fs/promises')
|
||||
const raw = JSON.parse(await readFile(cachePath, 'utf-8'))
|
||||
raw.files[sessionPath].resumeState = { garbage: true }
|
||||
|
|
|
|||
|
|
@ -1,9 +1,11 @@
|
|||
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'fs/promises'
|
||||
import { mkdtemp, mkdir, writeFile, rm, stat } from 'fs/promises'
|
||||
import { join } from 'path'
|
||||
import { tmpdir } from 'os'
|
||||
|
||||
import { createCodexProvider } from '../../src/providers/codex.js'
|
||||
import { clearCodexMemCaches, CODEX_CACHE_VERSION, codexCacheFileName } from '../../src/codex-cache.js'
|
||||
import { calculateCost } from '../../src/models.js'
|
||||
import type { ParsedProviderCall } from '../../src/providers/types.js'
|
||||
|
||||
let tmpDir: string
|
||||
|
|
@ -722,11 +724,58 @@ describe('codex provider - JSONL parsing', () => {
|
|||
reasoningTokens: 20,
|
||||
tools: ['Bash'],
|
||||
activeDurationMs: 7000,
|
||||
activeGeneratedTokens: 120,
|
||||
// Reasoning (20) is a subset of output_tokens (100), not additive
|
||||
// (#1075/#1078/#1079): the billable/throughput numerator is 100, not 120.
|
||||
activeGeneratedTokens: 100,
|
||||
toolWaitMs: 3000,
|
||||
})
|
||||
})
|
||||
|
||||
it('REGRESSION (#1088 BUG-1): excludes the task_started -> first request-context gap from active time', async () => {
|
||||
// Codex fires task_started before it assembles the request; the 7s gap to
|
||||
// the first request-context event (here, the user message) is CLI/harness
|
||||
// startup, not model wait, and must not count toward active time. If this
|
||||
// ever reverts to windowStart = taskStartedAt, activeDurationMs becomes
|
||||
// 20000 (the full duration_ms) instead of 13000 (20000 - the 7s gap).
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-startup-gap.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-startup-gap', model: 'gpt-5.5' }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:00Z', payload: { type: 'task_started' } }),
|
||||
userMessage('run the tool', '2026-04-14T10:00:07Z'),
|
||||
tokenCount({ timestamp: '2026-04-14T10:00:20Z', last: { output: 100 }, total: { output: 100, total: 100 } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:20Z', payload: { type: 'task_complete', duration_ms: 20_000 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]).toMatchObject({ activeGeneratedTokens: 100, activeDurationMs: 13_000, toolWaitMs: 0 })
|
||||
})
|
||||
|
||||
it('#1088 BUG-8: reads a task_complete duration reported as {secs,nanos}, not only a plain number', async () => {
|
||||
// mcp_tool_call_end already tolerates {secs,nanos} and string durations
|
||||
// (durationValueMs); task_complete only read the plain-number duration_ms
|
||||
// field, so a task_complete reported the object form was silently dropped
|
||||
// (no active timing at all) instead of parsed.
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-object-duration.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-object-duration', model: 'gpt-5.5' }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:00Z', payload: { type: 'task_started' } }),
|
||||
userMessage('run the tool', '2026-04-14T10:00:00Z'),
|
||||
tokenCount({ timestamp: '2026-04-14T10:00:10Z', last: { output: 100 }, total: { output: 100, total: 100 } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:10Z', payload: { type: 'task_complete', duration: { secs: 10, nanos: 0 } } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]).toMatchObject({ activeGeneratedTokens: 100, activeDurationMs: 10_000 })
|
||||
})
|
||||
|
||||
it('keeps estimated output parsing for large token lines without usage info', async () => {
|
||||
// Some rollout variants put token_count metadata beyond the compact head
|
||||
// or omit `info` entirely. The line must still reach the character-based
|
||||
|
|
@ -983,6 +1032,79 @@ describe('codex provider - JSONL parsing', () => {
|
|||
])
|
||||
})
|
||||
|
||||
// #478 follow-up: the shapes the `function_call` path never reached. Fixtures
|
||||
// are synthesized from the real shapes seen in Codex rollouts (a `custom_tool_call`
|
||||
// whose payload is an `input` JS program, and an item-model `item_completed`
|
||||
// carrying a `CommandExecution` item with an argv `command`); no real session
|
||||
// content is used.
|
||||
it('attributes MCP + Skill usage from the exec custom tool and the item model', async () => {
|
||||
const customExec = (input: string, callId: string) => JSON.stringify({
|
||||
type: 'response_item',
|
||||
timestamp: '2026-04-14T10:00:30Z',
|
||||
payload: { type: 'custom_tool_call', call_id: callId, name: 'exec', input },
|
||||
})
|
||||
const commandExecutionItem = (command: string[]) => JSON.stringify({
|
||||
type: 'event_msg',
|
||||
timestamp: '2026-04-14T10:00:40Z',
|
||||
payload: { type: 'item_completed', item: { type: 'CommandExecution', command, exit_code: 0 } },
|
||||
})
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-exec-items.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-exec-items', model: 'gpt-5.5' }),
|
||||
userMessage('use the MCP CLI and load a skill'),
|
||||
// custom-tool transport: MCP call and a skill read, both inside the JS program.
|
||||
customExec('const r = await tools.exec_command({cmd:"mcp-cli call github get_issue \'{}\'"}); text(r.output);', 'c1'),
|
||||
customExec('const r = await tools.exec_command({cmd:"sed -n \'1,200p\' /Users/x/.codex/skills/control-in-app-browser/SKILL.md"}); text(r.output);', 'c2'),
|
||||
// Negatives: a lookup subcommand, and a grep that merely mentions a SKILL.md.
|
||||
customExec('const r = await tools.exec_command({cmd:"mcp-cli info github"}); text(r.output);', 'c3'),
|
||||
customExec('const r = await tools.exec_command({cmd:"grep -rn TODO /Users/x/.codex/skills/deploy/SKILL.md"}); text(r.output);', 'c4'),
|
||||
// item model, no matching response item: must attribute on its own.
|
||||
commandExecutionItem(['/bin/zsh', '-lc', "mcp-cli call optimizely-cms-mcp help '{}'"]),
|
||||
commandExecutionItem(['/bin/zsh', '-lc', 'cat /Users/x/.codex/skills/graphify/SKILL.md']),
|
||||
commandExecutionItem(['/bin/zsh', '-lc', 'ls -la']),
|
||||
tokenCount({ timestamp: '2026-04-14T10:01:00Z', last: { input: 300, output: 100 }, total: { total: 400 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
const tools = calls[0]!.tools
|
||||
// Attribution only: the four custom-tool execs stay Bash, and the item-model
|
||||
// entries add no tool of their own, so the Bash count is unchanged at 4.
|
||||
expect(tools.filter(t => t === 'Bash')).toHaveLength(4)
|
||||
expect(tools.filter(t => t.startsWith('mcp__')).sort()).toEqual([
|
||||
'mcp__github__get_issue',
|
||||
'mcp__optimizely-cms-mcp__help',
|
||||
])
|
||||
expect(calls[0]!.skills?.slice().sort()).toEqual(['control-in-app-browser', 'graphify'])
|
||||
expect(tools.filter(t => t === 'Skill')).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('counts a command carried by BOTH the response item and the item model once', async () => {
|
||||
const cmd = "mcp-cli call github get_issue '{}'"
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-both-shapes.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-both-shapes', model: 'gpt-5.5' }),
|
||||
userMessage('call it twice'),
|
||||
JSON.stringify({ type: 'response_item', timestamp: '2026-04-14T10:00:30Z', payload: { type: 'function_call', name: 'exec_command', arguments: JSON.stringify({ cmd }) } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:31Z', payload: { type: 'item_completed', item: { type: 'CommandExecution', command: ['/bin/zsh', '-lc', cmd] } } }),
|
||||
JSON.stringify({ type: 'response_item', timestamp: '2026-04-14T10:00:32Z', payload: { type: 'function_call', name: 'exec_command', arguments: JSON.stringify({ cmd }) } }),
|
||||
JSON.stringify({ type: 'event_msg', timestamp: '2026-04-14T10:00:33Z', payload: { type: 'item_completed', item: { type: 'CommandExecution', command: ['/bin/zsh', '-lc', cmd] } } }),
|
||||
tokenCount({ timestamp: '2026-04-14T10:01:00Z', last: { input: 300, output: 100 }, total: { total: 400 } }),
|
||||
])
|
||||
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const source = { path: filePath, project: 'test', provider: 'codex' }
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
// Two execs, two MCP attributions - not four.
|
||||
expect(calls[0]!.tools.filter(t => t === 'Bash')).toHaveLength(2)
|
||||
expect(calls[0]!.tools.filter(t => t === 'mcp__github__get_issue')).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('normalizes Codex subagent tool calls to Agent', async () => {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-agent.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-agent', model: 'gpt-5.5' }),
|
||||
|
|
@ -1197,3 +1319,77 @@ describe('codex provider - forked session dedupe', () => {
|
|||
expect(tokens).toBe(300)
|
||||
})
|
||||
})
|
||||
|
||||
describe('codex auto-review pricing (#1047)', () => {
|
||||
it('parses auto-review as itself and prices it as GPT-5.5', async () => {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-auto-review.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-auto', model: 'codex-auto-review' }),
|
||||
userMessage('review the PR'),
|
||||
tokenCount({
|
||||
timestamp: '2026-04-14T10:01:00Z',
|
||||
last: { input: 1_000_000, output: 1_000_000 },
|
||||
total: { total: 2_000_000 },
|
||||
}),
|
||||
])
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set()).parse()) {
|
||||
calls.push(call)
|
||||
}
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]!.model).toBe('codex-auto-review')
|
||||
expect(calls[0]!.costUSD).toBe(calculateCost('gpt-5.5', 1_000_000, 1_000_000, 0, 0, 0))
|
||||
})
|
||||
|
||||
it('discards a warm v11 versioned $0 exact hit so unchanged rollouts reprice', async () => {
|
||||
const cacheDir = join(tmpDir, 'cache')
|
||||
await mkdir(cacheDir, { recursive: true })
|
||||
const prev = process.env['CODEBURN_CACHE_DIR']
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheDir
|
||||
try {
|
||||
const filePath = await writeSession(tmpDir, '2026-04-14', 'rollout-stale-auto.jsonl', [
|
||||
sessionMeta({ session_id: 'sess-stale-auto', model: 'codex-auto-review' }),
|
||||
userMessage('review the PR'),
|
||||
tokenCount({
|
||||
timestamp: '2026-04-14T10:01:00Z',
|
||||
last: { input: 1_000_000, output: 1_000_000 },
|
||||
total: { total: 2_000_000 },
|
||||
}),
|
||||
])
|
||||
const st = await stat(filePath)
|
||||
// Main's #1075 already owns v11. A colliding v11 $0 file must not be
|
||||
// treated as current after this PR takes v12.
|
||||
expect(CODEX_CACHE_VERSION).toBeGreaterThan(11)
|
||||
await writeFile(join(cacheDir, codexCacheFileName(11)), JSON.stringify({
|
||||
version: 11,
|
||||
files: {
|
||||
[filePath]: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
sizeBytes: st.size,
|
||||
project: 'test',
|
||||
calls: [{
|
||||
model: 'codex-auto-review',
|
||||
costUSD: 0,
|
||||
inputTokens: 1_000_000,
|
||||
outputTokens: 1_000_000,
|
||||
deduplicationKey: 'stale',
|
||||
}],
|
||||
},
|
||||
},
|
||||
}))
|
||||
clearCodexMemCaches()
|
||||
const provider = createCodexProvider(tmpDir)
|
||||
const calls: ParsedProviderCall[] = []
|
||||
for await (const call of provider.createSessionParser({ path: filePath, project: 'test', provider: 'codex' }, new Set()).parse()) {
|
||||
calls.push(call)
|
||||
}
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(calls[0]!.costUSD).toBeGreaterThan(0)
|
||||
expect(calls[0]!.costUSD).toBe(calculateCost('gpt-5.5', 1_000_000, 1_000_000, 0, 0, 0))
|
||||
} finally {
|
||||
clearCodexMemCaches()
|
||||
if (prev === undefined) delete process.env['CODEBURN_CACHE_DIR']
|
||||
else process.env['CODEBURN_CACHE_DIR'] = prev
|
||||
}
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -706,9 +706,9 @@ describe('copilot provider - session.shutdown token/cost rollup', () => {
|
|||
it('emits per-leg deltas for a resumed session with cumulative shutdown rollups', async () => {
|
||||
// Numbers from a real resumed CLI 1.0.78 session (3 legs via --resume):
|
||||
// each leg appends a session.shutdown whose modelMetrics are CUMULATIVE.
|
||||
// Emitting deltas keyed by occurrence keeps a growing file append-only
|
||||
// under the durable union-by-key cache merge — re-parsing after each
|
||||
// resume adds only the new leg, never double-counting earlier ones.
|
||||
// Emitting deltas keyed by shutdown timestamp keeps a growing file
|
||||
// append-only under the durable union-by-key cache merge — re-parsing
|
||||
// after each resume adds only the new leg, never double-counting earlier ones.
|
||||
const legs = [
|
||||
{ inputTokens: 24672, outputTokens: 17, cacheReadTokens: 0, cacheWriteTokens: 24670 },
|
||||
{ inputTokens: 74463, outputTokens: 149, cacheReadTokens: 49489, cacheWriteTokens: 24968 },
|
||||
|
|
@ -803,6 +803,54 @@ describe('copilot provider - session.shutdown token/cost rollup', () => {
|
|||
expect(second).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('keeps three stampless shutdown legs as :n keys with lastEventTimestamp, not sessionStartTime', async () => {
|
||||
// sessionStartTime is identical on every leg. Putting it in the key (or
|
||||
// preferring it over lastEventTimestamp for the call timestamp) collapses
|
||||
// a 3-leg journal onto one row. Discovery only yields events.jsonl, so
|
||||
// two-journal fixtures are unreachable; this is the reachable class.
|
||||
const lastEvent = '2026-08-01T10:00:15Z'
|
||||
const sessionStartTime = 1784102040274
|
||||
const stamplessShutdown = (usage: {
|
||||
inputTokens: number
|
||||
outputTokens: number
|
||||
cacheReadTokens: number
|
||||
cacheWriteTokens: number
|
||||
}) => JSON.stringify({
|
||||
type: 'session.shutdown',
|
||||
data: {
|
||||
shutdownType: 'routine',
|
||||
sessionStartTime,
|
||||
modelMetrics: {
|
||||
'claude-sonnet-5': {
|
||||
requests: { count: 1, cost: 1 },
|
||||
usage: { ...usage, reasoningTokens: 0 },
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
const eventsPath = await createSessionDir('sess-stampless', [
|
||||
modelChange('claude-sonnet-5'),
|
||||
assistantMessage({ messageId: 'msg-1', outputTokens: 10, timestamp: lastEvent }),
|
||||
stamplessShutdown({ inputTokens: 3000, outputTokens: 10, cacheReadTokens: 1000, cacheWriteTokens: 500 }),
|
||||
stamplessShutdown({ inputTokens: 7000, outputTokens: 20, cacheReadTokens: 3000, cacheWriteTokens: 1000 }),
|
||||
stamplessShutdown({ inputTokens: 10000, outputTokens: 30, cacheReadTokens: 5000, cacheWriteTokens: 1500 }),
|
||||
])
|
||||
const calls = await collectCalls({ path: eventsPath, project: 'myproject', provider: 'copilot', sourceType: 'jsonl' })
|
||||
const shutdowns = calls.filter(c => c.deduplicationKey.includes(':shutdown:'))
|
||||
expect(shutdowns.map(c => c.deduplicationKey)).toEqual([
|
||||
'copilot:sess-stampless:shutdown:claude-sonnet-5:1',
|
||||
'copilot:sess-stampless:shutdown:claude-sonnet-5:2',
|
||||
'copilot:sess-stampless:shutdown:claude-sonnet-5:3',
|
||||
])
|
||||
expect(shutdowns.map(c => c.timestamp)).toEqual([lastEvent, lastEvent, lastEvent])
|
||||
expect(shutdowns[0]!.inputTokens).toBe(1500)
|
||||
expect(shutdowns[1]!.inputTokens).toBe(1500)
|
||||
expect(shutdowns[2]!.inputTokens).toBe(500)
|
||||
expect(shutdowns.reduce((a, c) => a + c.inputTokens, 0)).toBe(3500)
|
||||
expect(shutdowns.reduce((a, c) => a + c.cacheReadInputTokens, 0)).toBe(5000)
|
||||
expect(shutdowns.reduce((a, c) => a + c.cacheCreationInputTokens, 0)).toBe(1500)
|
||||
})
|
||||
|
||||
it('falls back to the last stamped event when shutdown carries no timestamp at all', async () => {
|
||||
// A shutdown with neither its own timestamp nor sessionStartTime must not
|
||||
// yield an empty-timestamp call: the date-range filters in parser.ts drop
|
||||
|
|
|
|||
|
|
@ -79,6 +79,7 @@ describe('cursor-agent provider', () => {
|
|||
|
||||
expect(provider.modelDisplayName('claude-5-future-model')).toBe('claude-5-future-model (est.)')
|
||||
expect(provider.modelDisplayName('gpt-9')).toBe('gpt-9 (est.)')
|
||||
expect(provider.modelDisplayName('gpt-5.6-sol')).toBe('GPT-5.6 Sol (est.)')
|
||||
})
|
||||
|
||||
it('returns identity for tool display name', () => {
|
||||
|
|
|
|||
|
|
@ -106,8 +106,9 @@ describe('cursor cache', () => {
|
|||
process.env['CODEBURN_CACHE_DIR'] = secondCacheDir
|
||||
await writeCachedResults(dbPath, [], secondFloor)
|
||||
|
||||
const firstPath = join(firstCacheDir, 'cursor-results.json')
|
||||
const secondPath = join(secondCacheDir, 'cursor-results.json')
|
||||
const { cursorCacheFileName } = await import('../../src/cursor-cache.js')
|
||||
const firstPath = join(firstCacheDir, cursorCacheFileName())
|
||||
const secondPath = join(secondCacheDir, cursorCacheFileName())
|
||||
const first = JSON.parse(await readFile(firstPath, 'utf-8')) as { lookbackFloor: string }
|
||||
const second = JSON.parse(await readFile(secondPath, 'utf-8')) as { lookbackFloor: string }
|
||||
expect(first.lookbackFloor).toBe(firstFloor)
|
||||
|
|
|
|||
|
|
@ -66,6 +66,11 @@ describe('vercel-gateway provider', () => {
|
|||
expect(calls[0]?.costUSD).toBe(1.25)
|
||||
expect(calls[0]?.model).toBe('anthropic/claude-sonnet-4.6')
|
||||
})
|
||||
|
||||
it('resolves vendor/slug ids through the global short-name table', () => {
|
||||
expect(vercelGateway.modelDisplayName('openai/gpt-5.6-terra')).toBe('GPT-5.6 Terra')
|
||||
expect(vercelGateway.modelDisplayName('accounts/fireworks/models/kimi-k2p6')).toBe('Kimi K2.6')
|
||||
})
|
||||
})
|
||||
|
||||
describe('vercel-gateway end-to-end (parseAllSessions network path)', () => {
|
||||
|
|
|
|||
317
tests/result-cache-suffix.test.ts
Normal file
317
tests/result-cache-suffix.test.ts
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'fs/promises'
|
||||
import { existsSync } from 'fs'
|
||||
import { tmpdir } from 'os'
|
||||
import { join } from 'path'
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
|
||||
import {
|
||||
CODEX_CACHE_VERSION,
|
||||
CODEX_LEGACY_CACHE_FILE,
|
||||
clearCodexMemCaches,
|
||||
codexCacheFileName,
|
||||
fingerprintFile,
|
||||
flushCodexCache,
|
||||
readCachedCodexResults,
|
||||
writeCachedCodexResults,
|
||||
} from '../src/codex-cache.js'
|
||||
import {
|
||||
CURSOR_CACHE_VERSION,
|
||||
CURSOR_LEGACY_CACHE_FILE,
|
||||
cursorCacheFileName,
|
||||
readCachedResults,
|
||||
writeCachedResults,
|
||||
} from '../src/cursor-cache.js'
|
||||
import {
|
||||
ANTIGRAVITY_CACHE_VERSION,
|
||||
ANTIGRAVITY_LEGACY_CACHE_FILE,
|
||||
antigravityCacheFileName,
|
||||
clearAntigravityCacheStates,
|
||||
createAntigravityProvider,
|
||||
} from '../src/providers/antigravity.js'
|
||||
import type { ParsedProviderCall } from '../src/providers/types.js'
|
||||
|
||||
const originalCacheDir = process.env['CODEBURN_CACHE_DIR']
|
||||
let root: string
|
||||
|
||||
function call(provider: string, marker: string): ParsedProviderCall {
|
||||
return {
|
||||
provider,
|
||||
model: marker,
|
||||
inputTokens: 1,
|
||||
outputTokens: 1,
|
||||
cacheCreationInputTokens: 0,
|
||||
cacheReadInputTokens: 0,
|
||||
cachedInputTokens: 0,
|
||||
reasoningTokens: 0,
|
||||
webSearchRequests: 0,
|
||||
costUSD: 0,
|
||||
tools: [],
|
||||
bashCommands: [],
|
||||
timestamp: '2026-08-22T00:00:00.000Z',
|
||||
speed: 'standard',
|
||||
deduplicationKey: `${provider}:${marker}`,
|
||||
userMessage: '',
|
||||
sessionId: marker,
|
||||
}
|
||||
}
|
||||
|
||||
async function readAntigravityModel(sourcePath: string): Promise<string | undefined> {
|
||||
const parser = createAntigravityProvider().createSessionParser({
|
||||
path: sourcePath,
|
||||
project: 'fixture',
|
||||
provider: 'antigravity',
|
||||
}, new Set())
|
||||
for await (const parsed of parser.parse()) return parsed.model
|
||||
return undefined
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
root = await mkdtemp(join(tmpdir(), 'codeburn-result-suffix-'))
|
||||
process.env['CODEBURN_CACHE_DIR'] = root
|
||||
clearCodexMemCaches()
|
||||
clearAntigravityCacheStates()
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
if (originalCacheDir === undefined) delete process.env['CODEBURN_CACHE_DIR']
|
||||
else process.env['CODEBURN_CACHE_DIR'] = originalCacheDir
|
||||
clearCodexMemCaches()
|
||||
clearAntigravityCacheStates()
|
||||
await rm(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
describe('unsuffixed result caches coexist with mixed-version binaries', () => {
|
||||
it('publishes Codex to the versioned file and leaves an older unsuffixed file intact', async () => {
|
||||
const sourcePath = join(root, 'rollout.jsonl')
|
||||
await writeFile(sourcePath, '{}\n')
|
||||
const fingerprint = await fingerprintFile(sourcePath)
|
||||
expect(fingerprint).not.toBeNull()
|
||||
|
||||
const legacyPath = join(root, CODEX_LEGACY_CACHE_FILE)
|
||||
await writeFile(legacyPath, JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION - 1,
|
||||
files: { [sourcePath]: { mtimeMs: 1, sizeBytes: 1, project: 'old', calls: [call('codex', 'old-binary')] } },
|
||||
}))
|
||||
|
||||
await writeCachedCodexResults(sourcePath, 'project', [call('codex', 'new-binary')], fingerprint!)
|
||||
await flushCodexCache()
|
||||
|
||||
expect(existsSync(join(root, codexCacheFileName()))).toBe(true)
|
||||
expect(existsSync(legacyPath)).toBe(true)
|
||||
const versioned = JSON.parse(await readFile(join(root, codexCacheFileName()), 'utf8'))
|
||||
const legacy = JSON.parse(await readFile(legacyPath, 'utf8'))
|
||||
expect(versioned.version).toBe(CODEX_CACHE_VERSION)
|
||||
expect(versioned.files[sourcePath].calls[0].model).toBe('new-binary')
|
||||
expect(legacy.version).toBe(CODEX_CACHE_VERSION - 1)
|
||||
expect(legacy.files[sourcePath].calls[0].model).toBe('old-binary')
|
||||
})
|
||||
|
||||
it('adopts a matching-version unsuffixed Codex cache and ignores a mismatched one', async () => {
|
||||
const sourcePath = join(root, 'rollout.jsonl')
|
||||
await writeFile(sourcePath, '{}\n')
|
||||
const fingerprint = await fingerprintFile(sourcePath)
|
||||
expect(fingerprint).not.toBeNull()
|
||||
|
||||
await writeFile(join(root, CODEX_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION,
|
||||
files: {
|
||||
[sourcePath]: {
|
||||
dev: fingerprint!.dev,
|
||||
ino: fingerprint!.ino,
|
||||
mtimeMs: fingerprint!.mtimeMs,
|
||||
sizeBytes: fingerprint!.sizeBytes,
|
||||
project: 'adopted',
|
||||
calls: [call('codex', 'adopt-me')],
|
||||
},
|
||||
},
|
||||
}))
|
||||
|
||||
expect((await readCachedCodexResults(sourcePath))?.calls.map(entry => entry.model)).toEqual(['adopt-me'])
|
||||
expect(existsSync(join(root, CODEX_LEGACY_CACHE_FILE))).toBe(true)
|
||||
expect(existsSync(join(root, codexCacheFileName()))).toBe(false)
|
||||
|
||||
clearCodexMemCaches()
|
||||
await writeFile(join(root, CODEX_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION - 1,
|
||||
files: {
|
||||
[sourcePath]: {
|
||||
dev: fingerprint!.dev,
|
||||
ino: fingerprint!.ino,
|
||||
mtimeMs: fingerprint!.mtimeMs,
|
||||
sizeBytes: fingerprint!.sizeBytes,
|
||||
project: 'stale',
|
||||
calls: [call('codex', 'do-not-adopt')],
|
||||
},
|
||||
},
|
||||
}))
|
||||
expect(await readCachedCodexResults(sourcePath)).toBeNull()
|
||||
})
|
||||
|
||||
it('publishes Cursor to the versioned file and leaves an older unsuffixed file intact', async () => {
|
||||
const dbPath = join(root, 'state.vscdb')
|
||||
await writeFile(dbPath, 'cursor-db')
|
||||
const floor = '2026-01-01T00:00:00.000Z'
|
||||
const legacyPath = join(root, CURSOR_LEGACY_CACHE_FILE)
|
||||
await writeFile(legacyPath, JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION - 1,
|
||||
dbMtimeMs: 1,
|
||||
dbSizeBytes: 1,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'old-binary')],
|
||||
}))
|
||||
|
||||
await writeCachedResults(dbPath, [call('cursor', 'new-binary')], floor)
|
||||
|
||||
expect(existsSync(join(root, cursorCacheFileName()))).toBe(true)
|
||||
expect(existsSync(legacyPath)).toBe(true)
|
||||
const versioned = JSON.parse(await readFile(join(root, cursorCacheFileName()), 'utf8'))
|
||||
const legacy = JSON.parse(await readFile(legacyPath, 'utf8'))
|
||||
expect(versioned.version).toBe(CURSOR_CACHE_VERSION)
|
||||
expect(versioned.calls[0].model).toBe('new-binary')
|
||||
expect(legacy.version).toBe(CURSOR_CACHE_VERSION - 1)
|
||||
expect(legacy.calls[0].model).toBe('old-binary')
|
||||
})
|
||||
|
||||
it('adopts a matching-version unsuffixed Cursor cache and ignores a mismatched one', async () => {
|
||||
const dbPath = join(root, 'state.vscdb')
|
||||
await writeFile(dbPath, 'cursor-db')
|
||||
const { stat } = await import('fs/promises')
|
||||
const fp = await stat(dbPath)
|
||||
const floor = '2026-01-01T00:00:00.000Z'
|
||||
|
||||
await writeFile(join(root, CURSOR_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION,
|
||||
dbMtimeMs: fp.mtimeMs,
|
||||
dbSizeBytes: fp.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'adopt-me')],
|
||||
}))
|
||||
expect((await readCachedResults(dbPath, floor))?.map(entry => entry.model)).toEqual(['adopt-me'])
|
||||
expect(existsSync(join(root, cursorCacheFileName()))).toBe(false)
|
||||
|
||||
await writeFile(join(root, CURSOR_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION - 1,
|
||||
dbMtimeMs: fp.mtimeMs,
|
||||
dbSizeBytes: fp.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'do-not-adopt')],
|
||||
}))
|
||||
expect(await readCachedResults(dbPath, floor)).toBeNull()
|
||||
})
|
||||
|
||||
it('publishes Antigravity to the versioned file and leaves an older unsuffixed file intact', async () => {
|
||||
const sourcePath = join(root, 'shared.pb')
|
||||
await writeFile(sourcePath, 'fixture')
|
||||
const { stat } = await import('fs/promises')
|
||||
const sourceStat = await stat(sourcePath)
|
||||
const legacyPath = join(root, ANTIGRAVITY_LEGACY_CACHE_FILE)
|
||||
await mkdir(root, { recursive: true })
|
||||
await writeFile(legacyPath, JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION - 1,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'old-binary')] },
|
||||
},
|
||||
}))
|
||||
|
||||
await writeFile(join(root, antigravityCacheFileName()), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'new-binary')] },
|
||||
},
|
||||
}))
|
||||
|
||||
expect(await readAntigravityModel(sourcePath)).toBe('new-binary')
|
||||
const legacy = JSON.parse(await readFile(legacyPath, 'utf8'))
|
||||
expect(legacy.cascades.shared.calls[0].model).toBe('old-binary')
|
||||
})
|
||||
|
||||
it('adopts a matching-version unsuffixed Antigravity cache and ignores a mismatched one', async () => {
|
||||
const sourcePath = join(root, 'shared.pb')
|
||||
await writeFile(sourcePath, 'fixture')
|
||||
const { stat } = await import('fs/promises')
|
||||
const sourceStat = await stat(sourcePath)
|
||||
|
||||
await writeFile(join(root, ANTIGRAVITY_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'adopt-me')] },
|
||||
},
|
||||
}))
|
||||
expect(await readAntigravityModel(sourcePath)).toBe('adopt-me')
|
||||
expect(existsSync(join(root, antigravityCacheFileName()))).toBe(false)
|
||||
|
||||
clearAntigravityCacheStates()
|
||||
await writeFile(join(root, ANTIGRAVITY_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION - 1,
|
||||
cascades: {
|
||||
shared: { mtimeMs: sourceStat.mtimeMs, sizeBytes: sourceStat.size, calls: [call('antigravity', 'do-not-adopt')] },
|
||||
},
|
||||
}))
|
||||
expect(await readAntigravityModel(sourcePath)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('never serves matching legacy when a versioned file is present but invalid', async () => {
|
||||
const sourcePath = join(root, 'rollout.jsonl')
|
||||
const dbPath = join(root, 'state.vscdb')
|
||||
const pbPath = join(root, 'shared.pb')
|
||||
await writeFile(sourcePath, '{}\n')
|
||||
await writeFile(dbPath, 'cursor-db')
|
||||
await writeFile(pbPath, 'fixture')
|
||||
const fingerprint = await fingerprintFile(sourcePath)
|
||||
const { stat } = await import('fs/promises')
|
||||
const dbStat = await stat(dbPath)
|
||||
const pbStat = await stat(pbPath)
|
||||
const floor = '2026-01-01T00:00:00.000Z'
|
||||
expect(fingerprint).not.toBeNull()
|
||||
|
||||
await writeFile(join(root, CODEX_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CODEX_CACHE_VERSION,
|
||||
files: {
|
||||
[sourcePath]: {
|
||||
dev: fingerprint!.dev,
|
||||
ino: fingerprint!.ino,
|
||||
mtimeMs: fingerprint!.mtimeMs,
|
||||
sizeBytes: fingerprint!.sizeBytes,
|
||||
project: 'legacy',
|
||||
calls: [call('codex', 'legacy-served')],
|
||||
},
|
||||
},
|
||||
}))
|
||||
await writeFile(join(root, CURSOR_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION,
|
||||
dbMtimeMs: dbStat.mtimeMs,
|
||||
dbSizeBytes: dbStat.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'legacy-served')],
|
||||
}))
|
||||
await writeFile(join(root, ANTIGRAVITY_LEGACY_CACHE_FILE), JSON.stringify({
|
||||
version: ANTIGRAVITY_CACHE_VERSION,
|
||||
cascades: {
|
||||
shared: { mtimeMs: pbStat.mtimeMs, sizeBytes: pbStat.size, calls: [call('antigravity', 'legacy-served')] },
|
||||
},
|
||||
}))
|
||||
|
||||
await writeFile(join(root, codexCacheFileName()), '{not-json')
|
||||
await writeFile(join(root, cursorCacheFileName()), '{not-json')
|
||||
await writeFile(join(root, antigravityCacheFileName()), '{not-json')
|
||||
expect(await readCachedCodexResults(sourcePath)).toBeNull()
|
||||
expect(await readCachedResults(dbPath, floor)).toBeNull()
|
||||
expect(await readAntigravityModel(pbPath)).toBeUndefined()
|
||||
|
||||
clearCodexMemCaches()
|
||||
clearAntigravityCacheStates()
|
||||
await writeFile(join(root, codexCacheFileName()), JSON.stringify({ version: CODEX_CACHE_VERSION - 1, files: {} }))
|
||||
await writeFile(join(root, cursorCacheFileName()), JSON.stringify({
|
||||
version: CURSOR_CACHE_VERSION - 1,
|
||||
dbMtimeMs: dbStat.mtimeMs,
|
||||
dbSizeBytes: dbStat.size,
|
||||
lookbackFloor: floor,
|
||||
calls: [call('cursor', 'wrong-version')],
|
||||
}))
|
||||
await writeFile(join(root, antigravityCacheFileName()), JSON.stringify({ version: ANTIGRAVITY_CACHE_VERSION - 1, cascades: {} }))
|
||||
expect(await readCachedCodexResults(sourcePath)).toBeNull()
|
||||
expect(await readCachedResults(dbPath, floor)).toBeNull()
|
||||
expect(await readAntigravityModel(pbPath)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
|
@ -553,7 +553,7 @@ describe('buildAttributionOtlpPayload', () => {
|
|||
expect(sessionSpan.spanId).not.toBe(commitSpan.spanId)
|
||||
|
||||
const sessionAttrs = attrMap(sessionSpan.attributes)
|
||||
expect(sessionAttrs['ai.session_id']).toEqual({ stringValue: 'sess-1' })
|
||||
expect(sessionAttrs['ai.session_id']).toBeUndefined()
|
||||
expect(sessionAttrs['ai.project']).toEqual({ stringValue: 'app' })
|
||||
expect(sessionAttrs['git.repo']).toEqual({ stringValue: 'github.com/acme/widget' })
|
||||
expect(sessionAttrs['git.commit_count']).toEqual({ intValue: '1' })
|
||||
|
|
@ -565,12 +565,24 @@ describe('buildAttributionOtlpPayload', () => {
|
|||
expect(sessionSpan.endTimeUnixNano).toBe((BigInt(new Date('2026-01-01T11:00:00.000Z').getTime()) * 1_000_000n).toString())
|
||||
|
||||
const commitAttrs = attrMap(commitSpan.attributes)
|
||||
expect(commitAttrs['ai.session_id']).toBeUndefined()
|
||||
expect(commitAttrs['git.sha']).toEqual({ stringValue: 'a'.repeat(40) })
|
||||
expect(commitAttrs['git.in_main']).toEqual({ boolValue: true })
|
||||
expect(commitAttrs['git.was_reverted']).toEqual({ boolValue: false })
|
||||
expect(commitAttrs['git.repo']).toEqual({ stringValue: 'github.com/acme/widget' })
|
||||
})
|
||||
|
||||
it('keeps session identity off the wire; join is the shared traceId', () => {
|
||||
const items = flattenAttributionRecords([makeRecord()])
|
||||
const payload = buildAttributionOtlpPayload(items)
|
||||
const spans = payload.resourceSpans[0]!.scopeSpans[0]!.spans
|
||||
expect(spans.length).toBeGreaterThan(0)
|
||||
for (const span of spans) {
|
||||
expect(attrMap(span.attributes)['ai.session_id']).toBeUndefined()
|
||||
expect(span.traceId).toBe(deriveTraceId('sess-1'))
|
||||
}
|
||||
})
|
||||
|
||||
it('omits git.repo when null and pr_links when empty', () => {
|
||||
const items = flattenAttributionRecords([makeRecord({ repo: null, prLinks: [], commits: [] })])
|
||||
const payload = buildAttributionOtlpPayload(items)
|
||||
|
|
|
|||
|
|
@ -156,6 +156,7 @@ describe('buildOtlpPayload', () => {
|
|||
expect(attrMap['ai.cost_usd']).toEqual({ doubleValue: 0.05 })
|
||||
expect(attrMap['ai.project']).toEqual({ stringValue: 'my-project' })
|
||||
expect(attrMap['ai.speed']).toEqual({ stringValue: 'standard' })
|
||||
expect(attrMap['ai.session_id']).toBeUndefined()
|
||||
})
|
||||
|
||||
it('includes tools as array attribute', () => {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue