mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-08-26 17:03:31 +00:00
fix(report): join billable output onto existing modelBreakdown keys
Parser sessions key modelBreakdown by getShortModelName. Leftover raw-id buckets (overview fixtures, cached summaries) were minting a second $0 / 0-call short-name row that findUnpricedModels flagged as Unpriced and failed CI on #1116.
This commit is contained in:
parent
ece5548126
commit
80fbae3725
3 changed files with 45 additions and 3 deletions
|
|
@ -19,7 +19,7 @@
|
|||
- **DeepSeek Harness (`dsh`) is now a supported provider.** Reads DeepSeek's open-source agent harness from `~/.dsh/sessions` (`DSH_HOME` relocates the root), both the default zstd logs and the uncompressed `session.jsonl` variant. A `.zstd` log is a concatenation of independent zstd frames, one per write batch, so it is decoded frame by frame behind a structural frame scan and a torn trailing frame from a crashed writer is ignored rather than failing the file (needs Node 22.15+ for `zlib` zstd; below that dsh is skipped with a notice instead of counted as $0). One call per `(turn, step)`, with the step's final `assistant/message` usage superseding the streamed `assistant/chunk` sample of the same call rather than adding to it, the model taken from the message that served the step, and reasoning tokens billed at the output rate. DSH records tokens but no cost, so calls are priced from the shared tables. The events a forked session replays from its parent are skipped, since codeburn already counts the parent's own log. The session format is pinned at version 0 upstream with no compatibility implied, so a log stamped with any other version is skipped with a notice instead of read under today's assumptions.
|
||||
|
||||
### Changed
|
||||
- **Report, sessions, overview, compare, export, report JSON and menubar `localModelSavings` now use `billableOutputTokens` for displayed output.** Cost already did. Exclusive providers (Grok and the rest) were under-reporting output by exactly their reasoning tokens; optimize added reasoning on top of output for the inclusive set `{claude, codex, copilot}` and double-counted it. Displayed aggregates now bill per call while the provider is known, joined on the same `getShortModelName` key the parser uses for `modelBreakdown`. Daily cache v28 re-derives finalized days so `report` matches the live parse. Thanks @saulcanina. (#1115)
|
||||
- **Report, sessions, overview, compare, export, report JSON and menubar `localModelSavings` now use `billableOutputTokens` for displayed output.** Cost already did. Exclusive providers (Grok and the rest) were under-reporting output by exactly their reasoning tokens; optimize added reasoning on top of output for the inclusive set `{claude, codex, copilot}` and double-counted it. Displayed aggregates now bill per call while the provider is known, joined onto an existing `modelBreakdown` key (parser short name first, then the raw id) so a leftover raw-id bucket cannot mint a $0 `Opus 4.8` Unpriced orphan. Daily cache v28 re-derives finalized days so `report` matches the live parse. Thanks @saulcanina. (#1115)
|
||||
- **The discovery sweep issues its metadata syscalls concurrently.** Every dated command re-walks and re-stats every provider tree before it can decide what is already cached, and that sweep was strictly serial: one `readdir`, one `stat`, one `state.json` read at a time, per provider, one provider after another. On a 21k-file / 9-provider corpus it owned most of a warm run's wall clock while the machine sat idle waiting on the kernel. Provider discovery now runs across providers at once, and the four walks that dominate it (claude, codex, kimicode, grok) plus the Claude project-dir walk and both fingerprint passes fan out through a shared bounded-concurrency helper. Order is unchanged everywhere — each level is re-concatenated in registry/`readdir` order before anything reconciles against the cache — so what is discovered, in what sequence, is byte-identical to the serial walk. Two smaller cuts ride along: the Claude walk reads directory entries with their types so a plain file no longer costs a wasted `subagents/` probe, and the Codex result cache (a single file that can reach hundreds of MB) now shares one in-flight load between concurrent readers instead of letting each one re-read and re-parse it. Warm `codeburn today` on that corpus: 5.30s to 2.99s; a cold parse 34.7s to 31.1s. (#1104)
|
||||
- **A kill mid-way through a non-Claude provider phase no longer restarts that whole phase.** `scanProjectDirs` (Claude) has long taken a throttled `saveProgress` callback so a killed cold parse resumes from a warm cache; `parseProviderSources` (every other provider — codex, cursor, gemini, and the rest) did not, and only persisted at the whole-provider boundary. On a large single-provider corpus (a multi-GB codex history is the common case) an app-timeout SIGKILL, crash, or force-quit during that phase discarded everything parsed since the last provider finished, forcing the entire phase to re-parse from zero on the next run. `parseProviderSources` now takes the same callback, invoked once per source right after that source's cache entry lands (mirroring `scanProjectDirs`' placement, outside the per-file try/catch), on the same file-count/wall-clock throttle. A file only ever gets a fingerprint once it has fully parsed, so a mid-file kill can never leave a half-parsed file's entry looking complete on resume.
|
||||
- **Routed model ids price as the model they wrap, and an unknown vendor prefix no longer prices by blind stripping.** Token-plan and gateway spellings of the same model (`omniroute:`, `cp/`, `cline-pass/`, `cline-free/`, `cmd/`, `antigravity/`) are peeled and the remaining id is priced, so a Cline Pass or OmniRoute session shows a `~` estimate instead of $0. In exchange, `provider/model` is no longer treated as authority on its own: the leading segment is stripped only when it is a namespace the bundled pricing catalog itself uses (`anthropic/`, `openai/`, `google/`, `x-ai/`, `qwen/`, `moonshotai/`, `nousresearch/`, `xiaomi/`, `z-ai/`, and every other vendor prefix in the LiteLLM snapshot), one of the routing wrappers above, or one of the client-side spellings `kimi/`, `mimo/`, `zhipu/`, `litellm_proxy/` and `openai_like/`. Anything else stays unpriced and is reported as unpriced rather than inheriting the price of a same-named cloud row, and local-runner prefixes (`ollama/`, `lmstudio/`, `hosted_vllm/`, `local/`) are excluded on purpose so an unlisted local tag can never invent cloud spend. A user price override for the bare id wins over the catalog row a routed spelling would otherwise hit.
|
||||
|
|
|
|||
|
|
@ -19,19 +19,36 @@ export function modelBreakdownKey(call: { provider?: string; model?: string }):
|
|||
}
|
||||
|
||||
/**
|
||||
* Per-model displayed output, keyed like parser `modelBreakdown`.
|
||||
* Prefer a key that already exists on this session's modelBreakdown.
|
||||
* Parser sessions are keyed by getShortModelName. Fixtures and leftover
|
||||
* summaries may still use the raw id. Inventing the other spelling
|
||||
* creates a $0 / 0-call orphan that findUnpricedModels flags as Unpriced.
|
||||
*/
|
||||
export function resolveModelBreakdownKey(
|
||||
call: { provider?: string; model?: string },
|
||||
breakdown: Record<string, unknown> | undefined,
|
||||
): string | undefined {
|
||||
const derived = modelBreakdownKey(call)
|
||||
if (derived && breakdown && Object.hasOwn(breakdown, derived)) return derived
|
||||
if (call.model && breakdown && Object.hasOwn(breakdown, call.model)) return call.model
|
||||
return derived ?? call.model
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-model displayed output, keyed like this session's `modelBreakdown`.
|
||||
* Call usage wins while provider identity is known. Aggregate-only /
|
||||
* stub sessions fall back to each existing bucket so a finite
|
||||
* sessionBillableOutputTokens cannot leave model Output Tokens at 0.
|
||||
*/
|
||||
export function sessionModelBillableOutputTokens(session: SessionSummary): Record<string, number> {
|
||||
const breakdown = session.modelBreakdown ?? {}
|
||||
const out: Record<string, number> = {}
|
||||
let sawUsage = false
|
||||
for (const turn of session.turns ?? []) {
|
||||
for (const call of turn.assistantCalls ?? []) {
|
||||
if (!call.usage) continue
|
||||
sawUsage = true
|
||||
const key = modelBreakdownKey(call)
|
||||
const key = resolveModelBreakdownKey(call, breakdown)
|
||||
if (!key) continue
|
||||
out[key] = (out[key] ?? 0) + callBillableOutputTokens(call)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -251,6 +251,31 @@ describe('#1115 billableOutputTokens on report/optimize totals', () => {
|
|||
expect(out).toMatch(/DeepSeek v4 Pro[\s\S]*\b1\b[\s\S]*\b10\b/)
|
||||
})
|
||||
|
||||
it('does not mint a $0 short-name orphan when modelBreakdown is the raw id', () => {
|
||||
const session = makeSession('claude', 50, 0)
|
||||
session.turns[0]!.assistantCalls[0]!.model = 'claude-opus-4-8'
|
||||
session.modelBreakdown = {
|
||||
'claude-opus-4-8': {
|
||||
calls: 2,
|
||||
costUSD: 5,
|
||||
savingsUSD: 0,
|
||||
tokens: {
|
||||
inputTokens: 100,
|
||||
outputTokens: 50,
|
||||
cacheReadInputTokens: 0,
|
||||
cacheCreationInputTokens: 0,
|
||||
cachedInputTokens: 0,
|
||||
reasoningTokens: 0,
|
||||
webSearchRequests: 0,
|
||||
},
|
||||
},
|
||||
}
|
||||
expect(sessionModelBillableOutputTokens(session)).toEqual({ 'claude-opus-4-8': 50 })
|
||||
const out = renderOverview([makeProject(session)], { label: 'June 2026', color: false })
|
||||
expect(out).not.toContain('Unpriced')
|
||||
expect(out).toContain('$5.00')
|
||||
})
|
||||
|
||||
it('aggregate-only model rows keep finite billable output', async () => {
|
||||
const aggregate = makeSession('grok', 10, 3)
|
||||
aggregate.turns = []
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue