mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-08-21 22:44:31 +00:00
72aad9e012
Grok CLI writes a turn_completed update carrying a full usage object -- inputTokens, outputTokens, cachedReadTokens, cacheCreationTokens, reasoningTokens -- into the same updates.jsonl the parser already reads. We ignored it and reconstructed an estimate from _meta.totalTokens, a running context-size counter that rides on unrelated events, with a total < prevTotal * 0.5 reset as the turn boundary. On the cache-heavy session reported in #998 that reconstruction captured about 1.4% of the real cache-read volume and roughly 6% of the day's tokens, while over-counting output about fivefold. cacheCreationInputTokens and reasoningTokens were hardcoded to zero regardless of what the session held. The parser now reads turn_completed.usage, keyed by the record's snake_case prompt_id so a re-emitted turn cannot double count, and sums across turns. Two decompositions matter, both derivable from the reported numbers: totalTokens equals inputTokens + outputTokens exactly, so cachedReadTokens and cacheCreationTokens are subsets of input and are subtracted out per record before pricing, matching the cache-exclusive convention codex and copilot already use; and reasoningTokens is a subset of output. That second one needs care, because the repo contract is the opposite of Grok's: ParsedProviderCall.reasoningTokens is exclusive of outputTokens everywhere, and every consumer sums the two -- tests/providers/kiro.test.ts says so outright. So reasoning is clamped to the reported output and output is emitted without it, and the downstream sum reconstructs Grok's number. Without the clamp a record with reasoning > output produced a negative output and left the pipeline pricing reasoning instead. Multi-model attribution is deliberately out of scope. modelUsage only selects a priced attribution id; a session that used two models is priced at one rate. Splitting per model was tried and dropped: chooseAuthoritativeModel's priced-id fallback exists to avoid a truthful-but-$0 row when modelUsage names an id this checkout cannot price, and per-model pricing loses it -- the reporter's own session collapsed from $1.20 to near zero the moment a second id appeared. When no valid completed record exists -- older Grok CLI versions -- the old heuristic still runs, unchanged. The decision is taken from the deduplicated records rather than latched per line, so a superseded or all-zero record cannot flip a session off the heuristic and drop it. A session only partly covered by turn_completed records keeps costIsEstimated: true rather than presenting itself as fully provider-measured. costUsdTicks is deliberately not read. Its scale is undocumented, and guessing it would fabricate spend. Bumps the grok parse version, and DAILY_CACHE_VERSION with MIN_SUPPORTED_VERSION together, since the daily cache serves every day before today and retains ten years. Moving them in lockstep is what keeps the carry-forward lossless: the filename is version-suffixed, so the old file stays on disk and is adopted for days no source can still re-derive. Separately, detectContextBloat divided by outputTokens alone. Reasoning is stored beside output for every reasoning-bearing provider, so the detector saw a fraction of the generated tokens and invented high-impact findings -- a session whose provider-reported ratio is 20:1, below the 25:1 threshold, was reported as 133:1 with 710K tokens of claimed savings. It now uses the same output + reasoning sum the reports use, which fixes codex, kiro, hermes, qwen and cursor-agent too. Reported in #998. |
||
|---|---|---|
| .. | ||
| design | ||
| providers | ||
| sync | ||
| architecture.md | ||