mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-08-16 20:14:27 +00:00
Merge pull request #646 from getagentseal/fix/unpriced-model-visibility
Surface unpriced models instead of silently counting them at $0
This commit is contained in:
Resham Joshi
GitHub
commit
270ee81b45
10 changed files with 354 additions and 7 deletions
88
README.md
88
README.md
|
|
@ -34,8 +34,11 @@ Everything runs locally. No wrapper, no proxy, no API keys, nothing leaves your
|
|||
<p align="center">
|
||||
<a href="#quick-start">Quick start</a> ·
|
||||
<a href="#find-and-fix-waste">Find waste</a> ·
|
||||
<a href="#apply-fixes-undo-anytime">Apply fixes</a> ·
|
||||
<a href="#guard-your-budget">Guard</a> ·
|
||||
<a href="#compare-models">Compare models</a> ·
|
||||
<a href="#track-what-shipped">Track what shipped</a> ·
|
||||
<a href="#codeburn-in-your-agent-mcp">MCP</a> ·
|
||||
<a href="#supported-tools">Supported tools</a> ·
|
||||
<a href="#commands">Commands</a> ·
|
||||
<a href="#features">Features</a> ·
|
||||
|
|
@ -66,6 +69,8 @@ Also runs via `bunx codeburn` or `dx codeburn`, or `brew install codeburn` on ma
|
|||
codeburn menubar
|
||||
```
|
||||
|
||||
On Linux, a GNOME Shell extension gives the same panel view; see [Linux (GNOME)](#linux-gnome).
|
||||
|
||||
Requires **Node.js 22.13+** and at least one supported tool with session data on disk. For Cursor and OpenCode, `better-sqlite3` installs automatically.
|
||||
|
||||
## Your month at a glance
|
||||
|
|
@ -131,6 +136,39 @@ Each finding shows the estimated token and dollar savings plus a ready-to-paste
|
|||
|
||||
You can also open it inline from the dashboard: press `o` when a finding count appears in the status bar, `b` to return.
|
||||
|
||||
## Apply fixes, undo anytime
|
||||
|
||||
```bash
|
||||
codeburn optimize --apply # review and apply fixes interactively
|
||||
codeburn optimize --apply --dry-run # print the plan, change nothing
|
||||
codeburn optimize --apply --yes # apply every appliable fix without prompting
|
||||
codeburn act list # every change CodeBurn has made
|
||||
codeburn act undo --last # roll the most recent change back
|
||||
codeburn act report # realized vs estimated savings
|
||||
```
|
||||
|
||||
`codeburn optimize` finds the waste; `--apply` fixes the config-class findings for you: settings values, environment variables, archiving unused agents and skills. Every change is backed up and journaled before it lands. `codeburn act list` shows the history and `codeburn act undo <id>` restores the original files (it refuses if the files changed since being applied, unless you pass `--force`).
|
||||
|
||||
The loop closes on honesty: once an applied fix is at least 3 days old, `codeburn act report` compares its estimated savings against what your sessions actually did, and later `codeburn optimize` runs show that realized figure in the header. Estimates get checked against reality, not just claimed.
|
||||
|
||||
## Guard your budget
|
||||
|
||||
```bash
|
||||
codeburn guard install # hooks into this project's .claude/settings.json
|
||||
codeburn guard install --global # or into ~/.claude/settings.json
|
||||
codeburn guard status # caps, install locations, flagged projects
|
||||
codeburn guard uninstall # removes cleanly, leaves your own hooks alone
|
||||
```
|
||||
|
||||
Guard installs opt-in hooks into Claude Code that watch session cost while you work:
|
||||
|
||||
- **Soft cap** (default $5): a one-time in-session warning when a session passes it.
|
||||
- **Hard cap** (default $15): stops the session; `codeburn guard allow` lifts it for that session only.
|
||||
- **Checkpoint** (default $3): if a session ends past this with no edits and no commits, a nudge suggests starting fresh with a named deliverable.
|
||||
- **Session openers**: projects where optimize found waste get a one-line flag at session start.
|
||||
|
||||
Caps are edited in `~/.config/codeburn/guard.json` (set a value to `null` to disable it). Add `--statusline` to show session cost in the Claude Code status line. Installs go through the same journal as everything else, so `codeburn act undo` removes them too. Hooks fail open: a broken guard never blocks a session.
|
||||
|
||||
## Compare models
|
||||
|
||||
```bash
|
||||
|
|
@ -237,6 +275,33 @@ defaults write org.agentseal.codeburn-menubar CodeBurnMenubarCompact -bool true
|
|||
|
||||
Relaunch the app to apply. To revert: `defaults delete org.agentseal.codeburn-menubar CodeBurnMenubarCompact`.
|
||||
|
||||
### Linux (GNOME)
|
||||
|
||||
Linux gets the same ambient view through a GNOME Shell extension (GNOME 45+): spend in the top panel, period switcher, compact mode, and daily budget alerts. It lives in [`gnome/`](gnome/):
|
||||
|
||||
```bash
|
||||
git clone https://github.com/getagentseal/codeburn && cd codeburn/gnome
|
||||
./install.sh
|
||||
gnome-extensions enable codeburn@codeburn.dev
|
||||
```
|
||||
|
||||
See [gnome/README.md](gnome/README.md) for settings and development notes. On Windows, `codeburn web` is the always-on view for now.
|
||||
|
||||
## CodeBurn in your agent (MCP)
|
||||
|
||||
```bash
|
||||
claude mcp add codeburn -- npx -y codeburn mcp
|
||||
```
|
||||
|
||||
`codeburn mcp` runs a local MCP server over stdio, so Claude Code, Cursor, or any MCP client can ask "where did my tokens go this week?" or "how do I spend less?" mid-conversation. It exposes two tools:
|
||||
|
||||
| Tool | What it returns |
|
||||
|------|-----------------|
|
||||
| `get_usage` | Spend and usage with breakdowns by tool, model, project, and task (fast) |
|
||||
| `get_savings` | Cost reductions: waste findings, retry tax, routing waste (slower, deeper analysis) |
|
||||
|
||||
Everything is read from local disk, same as the CLI. Project names are pseudonymized by default; the agent only sees real names if it asks with `include_project_names: true`. For other MCP clients, configure a stdio server with command `npx` and args `-y codeburn mcp`.
|
||||
|
||||
## Supported tools
|
||||
|
||||
CodeBurn auto-detects which AI tools you use. Each logo links to its provider doc.
|
||||
|
|
@ -336,6 +401,19 @@ Run `codeburn` for the dashboard, or use a subcommand below. Most commands also
|
|||
| `codeburn yield` | Productive vs reverted/abandoned spend, correlated against git |
|
||||
| `codeburn yield -p 30days` | Yield analysis for the last 30 days |
|
||||
|
||||
**Fix & control**
|
||||
|
||||
| Command | What it does |
|
||||
|---------|--------------|
|
||||
| `codeburn optimize --apply` | Interactively apply config-class fixes (`--yes`, `--dry-run`, `--only <ids>`) |
|
||||
| `codeburn act list` | Every change CodeBurn has applied, newest first |
|
||||
| `codeburn act undo <id>` | Roll a change back (`--last` for the most recent, `--force` if files drifted) |
|
||||
| `codeburn act report` | Realized vs estimated savings for applied fixes |
|
||||
| `codeburn guard install` | Budget-cap hooks for Claude Code (`--global`, `--statusline`) |
|
||||
| `codeburn guard status` | Show caps, install locations, and flagged projects |
|
||||
| `codeburn guard allow` | Lift the hard cap for the current session |
|
||||
| `codeburn mcp` | MCP server (stdio) exposing usage and savings to AI agents |
|
||||
|
||||
**Models**
|
||||
|
||||
| Command | What it does |
|
||||
|
|
@ -428,6 +506,16 @@ codeburn model-alias --remove "my-proxy-model" # remove alias
|
|||
|
||||
Aliases are stored in `~/.config/codeburn/config.json` and applied at runtime before pricing lookup. The target name can be anything in the [LiteLLM model list](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) or a canonical name from the fallback table (e.g. `claude-sonnet-4-6`, `claude-opus-4-5`, `gpt-4o`). Built-in aliases ship for known proxy model name variants. User-configured aliases take precedence over built-ins.
|
||||
|
||||
### Local Models, Custom Prices, and Proxies
|
||||
|
||||
```bash
|
||||
codeburn price-override my-model --input 0.27 --output 1.10 # USD per 1M tokens
|
||||
codeburn model-savings "llama3.1:8b" gpt-4o # local model, counted as savings
|
||||
codeburn proxy-path ~/work/copilot-repo # subscription-covered project
|
||||
```
|
||||
|
||||
`price-override` sets exact rates for any model (input, output, cache read, cache creation), useful for private deployments or models LiteLLM prices wrong. `model-savings` maps a free local model to a paid baseline: the local calls stay $0, and the dashboard shows what the same tokens would have cost on the baseline. `proxy-path` marks a project routed through a subscription-backed proxy (e.g. Claude Code over GitHub Copilot), so its API-rate cost is reported as subscription-covered and your net out-of-pocket stays honest. All three support `--list` and `--remove`.
|
||||
|
||||
### Filtering
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ import { CATEGORY_LABELS, type DateRange, type ProjectSummary, type TaskCategory
|
|||
import { formatCost, formatTokens } from './format.js'
|
||||
import { aggregateModelEfficiency } from './model-efficiency.js'
|
||||
import { parseAllSessions, filterProjectsByName } from './parser.js'
|
||||
import { loadPricing } from './models.js'
|
||||
import { findUnpricedModels, loadPricing } from './models.js'
|
||||
import { getAllProviders } from './providers/index.js'
|
||||
import { scanAndDetect, type WasteFinding, type WasteAction, type OptimizeResult } from './optimize.js'
|
||||
import { estimateContextBudget, type ContextBudget } from './context-budget.js'
|
||||
|
|
@ -358,6 +358,12 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
|
|||
}
|
||||
const sorted = Object.entries(modelTotals).sort(([, a], [, b]) => b.costUSD - a.costUSD)
|
||||
const maxCost = sorted[0]?.[1]?.costUSD ?? 0
|
||||
const unpriced = findUnpricedModels(Object.entries(modelTotals).map(([model, d]) => ({
|
||||
model,
|
||||
calls: d.calls,
|
||||
cost: d.costUSD,
|
||||
tokens: d.freshInput + d.cacheRead + d.cacheWrite,
|
||||
})))
|
||||
|
||||
return (
|
||||
<Panel title="By Model" color={PANEL_COLORS.model} width={pw}>
|
||||
|
|
@ -381,6 +387,11 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
|
|||
</Text>
|
||||
)
|
||||
})}
|
||||
{unpriced.length > 0 && (
|
||||
<Text color="yellow" wrap="truncate-end">
|
||||
{`! ${unpriced.length} model${unpriced.length === 1 ? '' : 's'} unpriced at $0, fix: codeburn model-alias (${unpriced.slice(0, 2).map(u => u.model).join(', ')}${unpriced.length > 2 ? ', ...' : ''})`}
|
||||
</Text>
|
||||
)}
|
||||
</Panel>
|
||||
)
|
||||
}
|
||||
|
|
|
|||
12
src/main.ts
12
src/main.ts
|
|
@ -2,7 +2,7 @@ import { isAbsolute } from 'path'
|
|||
import { Command } from 'commander'
|
||||
import { installMenubarApp } from './menubar-installer.js'
|
||||
import { exportCsv, exportJson, type PeriodExport } from './export.js'
|
||||
import { loadPricing, setModelAliases, setPriceOverrides, setLocalModelSavings, setProxyPaths, normalizeProxyPath } from './models.js'
|
||||
import { findUnpricedModels, loadPricing, setModelAliases, setPriceOverrides, setLocalModelSavings, setProxyPaths, normalizeProxyPath } from './models.js'
|
||||
import { parseAllSessions, filterProjectsByName, filterProjectsByDateRange, clearSessionCache } from './parser.js'
|
||||
import { allProviderNames } from './providers/index.js'
|
||||
import { convertCost } from './currency.js'
|
||||
|
|
@ -460,6 +460,16 @@ function buildJsonReport(projects: ProjectSummary[], period: string, periodKey:
|
|||
daily,
|
||||
projects: projectList,
|
||||
models,
|
||||
// Models with recorded usage that resolve to no pricing data right now
|
||||
// (#638). Their calls contribute $0 to every cost figure above, so
|
||||
// consumers can tell "cheap" from "uncounted". Empty when all models
|
||||
// priced. Fix entries via `codeburn model-alias` or `price-override`.
|
||||
unpricedModels: findUnpricedModels(Object.entries(modelMap).map(([model, d]) => ({
|
||||
model,
|
||||
calls: d.calls,
|
||||
cost: d.cost,
|
||||
tokens: d.inputTokens + d.outputTokens + d.cacheReadTokens + d.cacheWriteTokens,
|
||||
}))),
|
||||
activities,
|
||||
tools: sortedMap(toolMap),
|
||||
mcpServers: sortedMap(mcpMap),
|
||||
|
|
|
|||
|
|
@ -15,9 +15,13 @@ const oneShot = (r: number | null) => (r === null ? 'n/a' : pct(r * 100))
|
|||
|
||||
export function renderSummaryTable(p: MenubarPayload): string {
|
||||
const c = p.current
|
||||
const unpriced = c.unpricedModels ?? []
|
||||
return [
|
||||
`**${c.label}** — ${formatCost(c.cost)} · ${c.calls} calls · ${c.sessions} sessions`,
|
||||
`cache hit ${pct(c.cacheHitPercent)} · one-shot ${oneShot(c.oneShotRate)} · in ${formatTokens(c.inputTokens)} / out ${formatTokens(c.outputTokens)}`,
|
||||
...(unpriced.length > 0
|
||||
? [`⚠ ${unpriced.length} model${unpriced.length === 1 ? '' : 's'} unpriced, counted at $0: ${unpriced.map(u => `${u.model} (${u.calls} calls)`).join(', ')}. Cost above understates real spend; fix with \`codeburn model-alias\` or \`codeburn price-override\`.`]
|
||||
: []),
|
||||
'',
|
||||
'_Top models_',
|
||||
mdTable(['Model', 'Cost', 'Calls'], c.topModels.slice(0, 5).map(m => [m.name, formatCost(m.cost), String(m.calls)])),
|
||||
|
|
|
|||
|
|
@ -21,6 +21,10 @@ export type PeriodData = {
|
|||
codexCredits?: number
|
||||
categories: Array<{ name: string; cost: number; savingsUSD: number; turns: number; editTurns: number; oneShotTurns: number }>
|
||||
models: Array<{ name: string; cost: number; savingsUSD: number; calls: number }>
|
||||
/// Models with usage in the period whose pricing lookup fails against the
|
||||
/// current tables (#638): their calls contribute $0 to `cost`. Optional so
|
||||
/// PeriodData producers that predate the field keep compiling.
|
||||
unpricedModels?: Array<{ model: string; calls: number; tokens: number }>
|
||||
projects?: Array<{ name: string; cost: number; savingsUSD: number; sessions: number; sessionDetails?: Array<{ cost: number; savingsUSD: number; calls: number; inputTokens: number; outputTokens: number; date: string; models: Array<{ name: string; cost: number; savingsUSD: number }> }> }>
|
||||
modelEfficiency?: Array<{ name: string; costPerEdit: number | null; oneShotRate: number | null }>
|
||||
topSessions?: Array<{ project: string; cost: number; savingsUSD: number; calls: number; date: string }>
|
||||
|
|
@ -140,6 +144,10 @@ export type MenubarPayload = {
|
|||
savingsBaselineModel: string
|
||||
calls: number
|
||||
}>
|
||||
/// See PeriodData.unpricedModels: usage priced at $0 for lack of pricing
|
||||
/// data. Empty when every model in the period resolved a price. Optional
|
||||
/// so payload producers that predate the field stay source-compatible.
|
||||
unpricedModels?: Array<{ model: string; calls: number; tokens: number }>
|
||||
/// Local-model savings rollup, distinct from the routing-waste /
|
||||
/// optimize savings concepts which describe hypothetical optimization
|
||||
/// opportunities. This block tracks counterfactual spend that was
|
||||
|
|
@ -368,6 +376,7 @@ export function buildMenubarPayload(
|
|||
codexCredits: current.codexCredits ?? 0,
|
||||
topActivities: buildTopActivities(current.categories),
|
||||
topModels: buildTopModels(current.models),
|
||||
unpricedModels: current.unpricedModels ?? [],
|
||||
localModelSavings: breakdowns?.localModelSavings ?? { totalUSD: 0, calls: 0, byModel: [], byProvider: [] },
|
||||
providers: buildProviders(providers),
|
||||
topProjects: buildTopProjects(current.projects ?? []),
|
||||
|
|
|
|||
|
|
@ -679,6 +679,71 @@ function looksLikeLocalModel(name: string): boolean {
|
|||
return false
|
||||
}
|
||||
|
||||
export interface UnpricedModelUsage {
|
||||
model: string
|
||||
calls: number
|
||||
tokens: number
|
||||
}
|
||||
|
||||
function hasBillableRate(costs: ModelCosts): boolean {
|
||||
return costs.inputCostPerToken > 0
|
||||
|| costs.outputCostPerToken > 0
|
||||
|| costs.cacheWriteCostPerToken > 0
|
||||
|| costs.cacheReadCostPerToken > 0
|
||||
}
|
||||
|
||||
// Exact-override lookup with the same key derivation getModelCosts uses. Lets
|
||||
// the unpriced detector distinguish "explicitly declared free by the user" (a
|
||||
// zero-rate override) from a zero-rate LiteLLM stub, which means "listed but
|
||||
// unknown price" and must still be flagged. Only the EXACT override form is
|
||||
// consulted: getModelCosts checks it before any table hit, so when one exists
|
||||
// it is provably what priced the model. Prefix and case-insensitive overrides
|
||||
// resolve AFTER table hits and so cannot prove the $0 was intentional; a
|
||||
// zero-rate stub shadowed by one still gets flagged (the honest direction).
|
||||
function exactPriceOverrideFor(model: string): ModelCosts | null {
|
||||
const withPrefix = model.replace(/@.*$/, '').replace(/-\d{8}$/, '')
|
||||
const canonicalName = getCanonicalName(model)
|
||||
const canonical = resolveAlias(canonicalName)
|
||||
return getPriceOverrideExact(model, withPrefix, canonicalName, canonical)
|
||||
}
|
||||
|
||||
// Render-time unpriced detection (#638): flag aggregated model rows that carry
|
||||
// usage but $0 cost AND whose pricing lookup yields no billable rate right
|
||||
// now. Cost is computed at parse time and cached, so a parse-time registry
|
||||
// would miss cached sessions; a render-time check covers both and heals the
|
||||
// moment pricing data, an alias, or a price override arrives.
|
||||
//
|
||||
// Rows with cost > 0 are never flagged: aggregation keys rows by DISPLAY name
|
||||
// (parser.ts keys modelBreakdown via getShortModelName), which the pricing
|
||||
// lookup misses, so a priced model like "Opus 4.8" would otherwise false-flag.
|
||||
// $0 display-name rows ARE flagged even when the raw id would price today:
|
||||
// those tokens really did enter the report at $0 (a provider priced a
|
||||
// transformed name, or the session was cached before its model's pricing
|
||||
// landed). Conservative by design: a display key merging priced and unpriced
|
||||
// raw ids carries cost > 0 and is not flagged. Local-looking models and
|
||||
// models with a local-savings mapping are excluded because $0 is their
|
||||
// correct cost, as are zero-rate USER overrides (explicitly declared free).
|
||||
export function findUnpricedModels(
|
||||
rows: Iterable<{ model: string; calls: number; cost: number; tokens?: number }>,
|
||||
): UnpricedModelUsage[] {
|
||||
const out: UnpricedModelUsage[] = []
|
||||
for (const row of rows) {
|
||||
const { model } = row
|
||||
const tokens = row.tokens ?? 0
|
||||
if (!model || model === '<synthetic>') continue
|
||||
if (row.calls <= 0 && tokens <= 0) continue
|
||||
if (row.cost > 0) continue
|
||||
if (looksLikeLocalModel(model)) continue
|
||||
if (getLocalSavingsBaseline(model)) continue
|
||||
const costs = getModelCosts(model)
|
||||
if (costs && hasBillableRate(costs)) continue
|
||||
if (costs && exactPriceOverrideFor(model)) continue
|
||||
out.push({ model, calls: row.calls, tokens })
|
||||
}
|
||||
return out.sort((a, b) => (b.tokens - a.tokens) || (b.calls - a.calls)
|
||||
|| (a.model < b.model ? -1 : a.model > b.model ? 1 : 0))
|
||||
}
|
||||
|
||||
function shouldWarnAboutUnknownModel(name: string): boolean {
|
||||
if (!name || name === '<synthetic>') return false
|
||||
if (warnedUnknownModels.has(name)) return false
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { homedir } from 'os'
|
|||
|
||||
import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
|
||||
import { formatCost as baseCost } from './currency.js'
|
||||
import { getShortModelName } from './models.js'
|
||||
import { findUnpricedModels, getShortModelName } from './models.js'
|
||||
import { dateKey } from './day-aggregator.js'
|
||||
|
||||
// Display-only helpers. The shared formatters omit thousands separators and
|
||||
|
|
@ -160,6 +160,17 @@ export function renderOverview(
|
|||
out.push(kv('Calls', formatCount(calls) + c.dim(' sessions ') + formatCount(sessions)))
|
||||
out.push(kv('Cache hit', `${cacheHit.toFixed(1)}%`))
|
||||
if (savings > 0) out.push(kv('Savings', formatCost(savings) + c.dim(' (local models)')))
|
||||
const unpriced = findUnpricedModels(
|
||||
[...byModel.entries()].map(([model, d]) => ({ model, calls: d.calls, cost: d.cost, tokens: d.tokens })),
|
||||
)
|
||||
if (unpriced.length > 0) {
|
||||
const shown = unpriced.slice(0, 3)
|
||||
.map((u) => `${u.model} (${formatTokens(u.tokens)} tok)`)
|
||||
.join(', ')
|
||||
const more = unpriced.length > 3 ? ` +${unpriced.length - 3} more` : ''
|
||||
out.push(kv('Unpriced', c.yellow(`${unpriced.length} model${unpriced.length === 1 ? '' : 's'} at $0: `) + shown + more))
|
||||
out.push(kv('', c.dim('Fix: codeburn model-alias "<model>" <known-model>')))
|
||||
}
|
||||
out.push('')
|
||||
|
||||
// Tokens breakdown: input / output / cache in (written) / cache out (read)
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { homedir } from 'node:os'
|
|||
import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory, type DateRange } from './types.js'
|
||||
import { type PeriodData, type ProviderCost, type BreakdownArrays, type MenubarPayload, buildMenubarPayload } from './menubar-json.js'
|
||||
import { parseAllSessions, filterProjectsByName, filterProjectsByDays } from './parser.js'
|
||||
import { getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName } from './models.js'
|
||||
import { findUnpricedModels, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName } from './models.js'
|
||||
import { getAllProviders } from './providers/index.js'
|
||||
import { aggregateProjectsIntoDays, buildPeriodDataFromDays } from './day-aggregator.js'
|
||||
import { aggregateModelEfficiency } from './model-efficiency.js'
|
||||
|
|
@ -13,7 +13,7 @@ import { getDaysInRange, ensureCacheHydrated, loadDailyCache, emptyCache, BACKFI
|
|||
export function buildPeriodData(label: string, projects: ProjectSummary[]): PeriodData {
|
||||
const sessions = projects.flatMap(p => p.sessions)
|
||||
const catTotals: Record<string, { turns: number; cost: number; savingsUSD: number; editTurns: number; oneShotTurns: number }> = {}
|
||||
const modelTotals: Record<string, { calls: number; cost: number; savingsUSD: number }> = {}
|
||||
const modelTotals: Record<string, { calls: number; cost: number; savingsUSD: number; tokens: number }> = {}
|
||||
let inputTokens = 0, outputTokens = 0, cacheReadTokens = 0, cacheWriteTokens = 0
|
||||
|
||||
for (const sess of sessions) {
|
||||
|
|
@ -30,10 +30,11 @@ export function buildPeriodData(label: string, projects: ProjectSummary[]): Peri
|
|||
catTotals[cat].oneShotTurns += d.oneShotTurns
|
||||
}
|
||||
for (const [model, d] of Object.entries(sess.modelBreakdown)) {
|
||||
if (!modelTotals[model]) modelTotals[model] = { calls: 0, cost: 0, savingsUSD: 0 }
|
||||
if (!modelTotals[model]) modelTotals[model] = { calls: 0, cost: 0, savingsUSD: 0, tokens: 0 }
|
||||
modelTotals[model].calls += d.calls
|
||||
modelTotals[model].cost += d.costUSD
|
||||
modelTotals[model].savingsUSD += d.savingsUSD
|
||||
modelTotals[model].tokens += d.tokens.inputTokens + d.tokens.outputTokens + d.tokens.cacheReadInputTokens + d.tokens.cacheCreationInputTokens
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -49,7 +50,9 @@ export function buildPeriodData(label: string, projects: ProjectSummary[]): Peri
|
|||
.map(([cat, d]) => ({ name: CATEGORY_LABELS[cat as TaskCategory] ?? cat, ...d })),
|
||||
models: Object.entries(modelTotals)
|
||||
.sort(([, a], [, b]) => b.cost - a.cost)
|
||||
.map(([name, d]) => ({ name, ...d })),
|
||||
.map(([name, d]) => ({ name, calls: d.calls, cost: d.cost, savingsUSD: d.savingsUSD })),
|
||||
unpricedModels: findUnpricedModels(Object.entries(modelTotals)
|
||||
.map(([model, d]) => ({ model, calls: d.calls, cost: d.cost, tokens: d.tokens }))),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import { join } from 'path'
|
|||
import { describe, it, expect, beforeAll, afterEach } from 'vitest'
|
||||
|
||||
import {
|
||||
findUnpricedModels,
|
||||
getModelCosts,
|
||||
getShortModelName,
|
||||
calculateCost,
|
||||
|
|
@ -675,3 +676,115 @@ describe('observed provider model aliases', () => {
|
|||
expect(calculateCost('qwen3-max-2026-01-23', 1_000_000, 1_000_000, 0, 0, 0)).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('findUnpricedModels', () => {
|
||||
it('flags an unknown paid-looking model with $0 cost and skips priced ones', () => {
|
||||
const rows = [
|
||||
{ model: 'claude-opus-4-6', calls: 10, cost: 2.5, tokens: 5000 },
|
||||
{ model: 'zz-mystery-paid-model-999', calls: 3, cost: 0, tokens: 1200 },
|
||||
]
|
||||
const unpriced = findUnpricedModels(rows)
|
||||
expect(unpriced).toEqual([{ model: 'zz-mystery-paid-model-999', calls: 3, tokens: 1200 }])
|
||||
})
|
||||
|
||||
it('never flags a row that carries real cost, even when the lookup misses', () => {
|
||||
// Aggregation keys rows by display name; the lookup misses but the row was
|
||||
// priced at parse time, so it must not be reported as unpriced.
|
||||
const unpriced = findUnpricedModels([
|
||||
{ model: 'Opus 4.8', calls: 100, cost: 42.5, tokens: 1_000_000 },
|
||||
{ model: 'zz-unknown-but-priced-elsewhere', calls: 5, cost: 0.01, tokens: 500 },
|
||||
])
|
||||
expect(unpriced).toEqual([])
|
||||
})
|
||||
|
||||
it('flags $0 display-name rows even when the raw id would price today', () => {
|
||||
// Droid prices the lowercased display name ("claude sonnet 4.6" -> no
|
||||
// pricing -> $0) and the parser keys the row by display name. Those
|
||||
// tokens really entered the report at $0, so the row must be flagged
|
||||
// even though claude-sonnet-4-6 itself is priced.
|
||||
const unpriced = findUnpricedModels([
|
||||
{ model: 'Sonnet 4.6', calls: 12, cost: 0, tokens: 500_000 },
|
||||
])
|
||||
expect(unpriced).toEqual([{ model: 'Sonnet 4.6', calls: 12, tokens: 500_000 }])
|
||||
})
|
||||
|
||||
it('flags zero-rate pricing stubs but not explicit zero-rate user overrides', async () => {
|
||||
// LiteLLM ships [0,0] stubs for models it lists but has no price for;
|
||||
// a stub hit means "unknown price", not "free".
|
||||
const cacheRoot = await mkdtemp(join(tmpdir(), 'codeburn-pricing-cache-'))
|
||||
try {
|
||||
process.env['CODEBURN_CACHE_DIR'] = cacheRoot
|
||||
await writeFile(join(cacheRoot, 'litellm-pricing.json'), JSON.stringify({
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
'zz-zero-stub-model': {
|
||||
inputCostPerToken: 0,
|
||||
outputCostPerToken: 0,
|
||||
cacheWriteCostPerToken: 0,
|
||||
cacheReadCostPerToken: 0,
|
||||
webSearchCostPerRequest: 0,
|
||||
fastMultiplier: 1,
|
||||
},
|
||||
},
|
||||
}), 'utf-8')
|
||||
await loadPricing()
|
||||
|
||||
expect(getModelCosts('zz-zero-stub-model')).not.toBeNull()
|
||||
const rows = [{ model: 'zz-zero-stub-model', calls: 3, cost: 0, tokens: 1100 }]
|
||||
expect(findUnpricedModels(rows)).toHaveLength(1)
|
||||
|
||||
// An explicit user override at zero rates means "this model is free".
|
||||
setPriceOverrides({ 'zz-zero-stub-model': { input: 0, output: 0 } })
|
||||
expect(findUnpricedModels(rows)).toEqual([])
|
||||
|
||||
// A prefix override cannot prove intent: getModelCosts resolves table
|
||||
// hits before prefix overrides, so the $0 came from the stub, not the
|
||||
// user. Still flagged.
|
||||
setPriceOverrides({ 'zz-zero-stub': { input: 0, output: 0 } })
|
||||
expect(findUnpricedModels(rows)).toHaveLength(1)
|
||||
} finally {
|
||||
delete process.env['CODEBURN_CACHE_DIR']
|
||||
await rm(cacheRoot, { recursive: true, force: true })
|
||||
setPriceOverrides({})
|
||||
await loadPricing()
|
||||
}
|
||||
})
|
||||
|
||||
it('skips synthetic, empty, local-looking, and zero-usage rows', () => {
|
||||
const unpriced = findUnpricedModels([
|
||||
{ model: '<synthetic>', calls: 5, cost: 0, tokens: 100 },
|
||||
{ model: '', calls: 5, cost: 0, tokens: 100 },
|
||||
{ model: 'llama3.1:8b', calls: 5, cost: 0, tokens: 100 },
|
||||
{ model: 'zz-quantized-model-bf16', calls: 5, cost: 0, tokens: 100 },
|
||||
{ model: 'zz-no-usage-model', calls: 0, cost: 0, tokens: 0 },
|
||||
])
|
||||
expect(unpriced).toEqual([])
|
||||
})
|
||||
|
||||
it('heals when the user configures an alias or a price override', () => {
|
||||
const model = 'zz-proxy-renamed-model-x1'
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toHaveLength(1)
|
||||
|
||||
setModelAliases({ [model]: 'claude-opus-4-6' })
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
setModelAliases({})
|
||||
|
||||
setPriceOverrides({ [model]: { input: 1, output: 2 } })
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
})
|
||||
|
||||
it('skips models mapped via model-savings (intentionally $0)', () => {
|
||||
const model = 'zz-my-local-runner'
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toHaveLength(1)
|
||||
setLocalModelSavings({ [model]: 'gpt-4o' })
|
||||
expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([])
|
||||
})
|
||||
|
||||
it('sorts by tokens, then calls', () => {
|
||||
const unpriced = findUnpricedModels([
|
||||
{ model: 'zz-small', calls: 9, cost: 0, tokens: 10 },
|
||||
{ model: 'zz-big', calls: 1, cost: 0, tokens: 9999 },
|
||||
])
|
||||
expect(unpriced.map(u => u.model)).toEqual(['zz-big', 'zz-small'])
|
||||
})
|
||||
})
|
||||
|
|
|
|||
|
|
@ -116,3 +116,36 @@ describe('renderOverview', () => {
|
|||
expect(out).not.toContain(' OS ')
|
||||
})
|
||||
})
|
||||
|
||||
describe('renderOverview unpriced models', () => {
|
||||
it('warns when a model with usage has no pricing data', () => {
|
||||
const out = renderOverview([makeProject({
|
||||
project: 'mystery',
|
||||
projectPath: '/Users/test/mystery',
|
||||
cost: 0,
|
||||
calls: 4,
|
||||
model: 'zz-mystery-paid-model-999',
|
||||
provider: 'claude',
|
||||
tokens: { input: 1000, output: 200, cacheR: 0, cacheW: 0 },
|
||||
})], { label: 'June 2026', color: false })
|
||||
|
||||
expect(out).toContain('Unpriced')
|
||||
expect(out).toContain('1 model at $0')
|
||||
expect(out).toContain('zz-mystery-paid-model-999')
|
||||
expect(out).toContain('codeburn model-alias')
|
||||
})
|
||||
|
||||
it('stays silent when every model is priced', () => {
|
||||
const out = renderOverview([makeProject({
|
||||
project: 'priced',
|
||||
projectPath: '/Users/test/priced',
|
||||
cost: 5,
|
||||
calls: 2,
|
||||
model: 'claude-opus-4-8',
|
||||
provider: 'claude',
|
||||
tokens: { input: 100, output: 50, cacheR: 0, cacheW: 0 },
|
||||
})], { label: 'June 2026', color: false })
|
||||
|
||||
expect(out).not.toContain('Unpriced')
|
||||
})
|
||||
})
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue