mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-08-21 06:24:32 +00:00
Merge pull request #1016 from ozymandiashh/fix/969-unpriced-top-and-docs
fix(models): let --unpriced survive --top, and document the flag
This commit is contained in:
commit
8ef3468dca
4 changed files with 69 additions and 3 deletions
|
|
@ -3,6 +3,7 @@
|
|||
## Unreleased
|
||||
|
||||
### Added
|
||||
- **`codeburn models --unpriced`.** The dashboard warns about models that price at $0 and points at `codeburn model-alias`, but the list itself was hard to get out of the TUI. This filters the plain-stdout `models` report to exactly those rows, reusing `findUnpricedModels` so local, free, aliased and price-overridden models are treated the same way the warning treats them, and defaulting that mode's min-cost to 0 so $0 rows are not pre-filtered away. Thanks @kocaemre. (#969)
|
||||
- **`optimize` spots the same long block pasted at the start of many sessions.** The new `recurring-context` detector groups sessions by their opening block — normalized for whitespace and ANSI, hashed over the first 2 KB — and reports a block of at least 1.5 KB that opens 5 or more sessions, with the top three by tokens, their session counts and the project each is confined to. It is a habit, not an apply-able fix: CodeBurn will not move your own text into `CLAUDE.md` for you, so the finding asks Claude to give the block a permanent home (a `CLAUDE.md` rule, or a file read on demand) and hand back a one-line pointer to open sessions with instead. Savings count the repeats only, never the first paste, and are marked `estimated`: provider usage is counted per API call, where the pasted block is mixed in with the system prompt, tool schemas and `CLAUDE.md`, so the block is sized from its own bytes. Injected system reminders and slash-command wrappers are not pastes and are skipped, and neither is a prompt a program wrote — an SDK session or a subagent task — read from the entry's flags, or off the ends of the raw line when the entry is too large for the parser to keep them. The opening block comes from the session scan that already runs, so nothing extra is read from disk.
|
||||
- **Applied fixes get re-measured on every `optimize` run, and told plainly whether they worked.** After `codeburn optimize --apply`, every still-applied fix comes back in an `Applied fixes` section on subsequent `codeburn optimize` runs, carrying the verdict `act report` already computes from the same reconciliation: `worked` (at least 70% of its window-scaled estimate realized), `partial` (something, but under that), `no-effect` (no measured reduction, printed with the exact `codeburn act undo <id>` that puts it back), or `measuring` for anything younger than the 3-day measurement window. The numbers are measured — provider-counted usage over the post-apply window — not re-estimated. `--apply` now says when the re-measure will happen, `--format json` gains `appliedFixes[]` (add-only), and the same section appears in the dashboard TUI and the desktop app. New `codeburn optimize --auto-revert` undoes the fixes that measured no reduction at all through the same code path as `codeburn act undo`; it never touches `partial` or still-measuring fixes, and never auto-reverts a `CLAUDE.md` rule (it prints the undo command instead), matching the `--yes` guardrail.
|
||||
- **Optimize findings say what to do with them and where their number came from.** Every finding now carries a class and a basis, and every surface groups by it: `Fix now (apply-able)` for findings `codeburn optimize --apply` can write itself, `Habits` for the behavioural ones, `FYI` for informational ones whose cost may be justified. A finding only counts as apply-able when a plan can actually be built for that instance, so an `mcp-deferral-off` caused by Vertex policy or a shell-profile override is grouped as a habit rather than promising a fix that does not exist. Alongside it, each finding is marked `measured` (summed from provider-counted usage) or `estimated` (a schema-size or recovery-fraction model), with the split reported in the header as `N measured · M estimated` in place of the blanket "Estimates only." footer. Sessions whose cost the provider never reported are kept out of the `cost-outliers` peer comparison, and a provider that only ever estimates gets the finding marked `estimated` rather than dropped. `--format json` gains `class` and `basis` per finding plus `summary.measuredSavingsUSD` (existing fields unchanged), and the new `docs/optimize.md` covers what is scanned, exactly what `--apply` may write, and how to read the health grade.
|
||||
|
|
@ -28,6 +29,7 @@
|
|||
- **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972)
|
||||
|
||||
### Fixed
|
||||
- **`codeburn models --unpriced --top N` returned nothing for a `--top N` smaller than the number of priced models.** `--top` is applied inside `aggregateModels`, before the unpriced filter, on rows sorted cost-first — and unpriced rows are $0 on both, so they sorted last and the slice removed exactly the rows the flag exists to show. A user with unpriced models was told they had none. The slice now runs after the filter — and after ranking, because unpriced rows tie at $0 on both keys, so slicing them in aggregate order kept whichever models happened to appear earliest in the transcript rather than the largest. The order now matches the one the unpriced-models warning shows. (#969)
|
||||
- **Old durable sources remain visible while they still exist.** The 90-day session-cache age-out now applies only after a durable source disappears from discovery, so an unchanged older Copilot source keeps reporting usage and reuses its persisted fingerprint instead of being reparsed and immediately discarded. (#987) On long-lived machines this makes previously dropped history reappear, so lifetime totals can jump once after upgrading.
|
||||
- **`optimize` no longer treats subagent transcripts as your sessions.** Claude Code writes each subagent's transcript to its own `subagents/agent-*.jsonl` file with `isSidechain: true` on every entry, and optimize counted each one as a user-started session. That inflated the session count in the header and fed the session-level detectors a population that fails their tests by construction: a sidechain is handed a large context and returns a short answer (context-heavy), and it never commits or opens a PR because its parent does (low-worth). Excluded from sidechains now: the header session count, the `low-worth-sessions`, `context-bloat`, `cost-outliers` and `capability-reliability` detectors, the coaching notes, the file-churn table, the median time-to-first-edit, the worst one-shot category, and the model-default recommendation - plus `duplicate-reads`, because a subagent starts on a fresh context and re-reading what its parent read is a necessary read, not a repeat. Everything else keeps the full population: `build-folder-reads` and `read-edit-ratio` still count calls made inside a sidechain, since reading `node_modules` or editing without reading is the same waste whoever does it and the `CLAUDE.md` rule they suggest binds subagents too, and so do the MCP, cache-bloat, ghost-command and configuration-overhead findings. Classification is sticky across the whole file, so calls that appear before the first marked entry are reclassified too, and `isSidechain` now survives the compact parser's 32 KB large-line path and warm-cache range rebuilds. Nothing is deleted from spend: sidechain tokens, calls and cost stay in every total and in `status`, and the optimize result cache keys on sidechain identity so a run cannot be served a pre-fix result. Absent markers still read as user-started, so no cache re-parse is needed. (#974)
|
||||
- **`optimize` no longer offers `claude mcp remove` for claude.ai connectors, and its MCP schema-cost estimate is per session.** A `claude_ai_*` namespace that no readable local MCP config claims is a claude.ai connector, managed through `/mcp` or claude.ai Settings rather than as a local MCP server (a local server that carries the prefix keeps its removal command and gains a same-name connector note); low-coverage findings now render them as a manual follow-up and build `--apply` plans only for exact local server names found in readable MCP config, so mixed findings remove only the local subset and the "apply-able" subtotal counts only that subset. The same change replaces the old global schema-cost cap with per-session, per-server proportional attribution — a more accurate model that lowers `mcp-low-coverage` estimates for everyone, connectors or not (on a large corpus roughly by half). (#975, #991)
|
||||
|
|
|
|||
|
|
@ -516,6 +516,7 @@ Sync sends token counts, costs, models, and projects, never prompts or code. Thi
|
|||
| `codeburn models --by-task` | Break each model into per-task-type rows |
|
||||
| `codeburn models --by-agent` | Break each model into per-agent rows: which agent drove which model's spend (`(main)` covers non-agent sessions; `--min-cost 0` shows sub-cent agents) |
|
||||
| `codeburn models --top 10` | Only the 10 most expensive models |
|
||||
| `codeburn models --unpriced` | Only models with usage that currently price at $0 — the copyable form of the unpriced-models warning |
|
||||
| `codeburn models --format markdown` | Emit a paste-friendly markdown table |
|
||||
| `codeburn models --task feature` | Filter to feature-development work |
|
||||
| `codeburn models --provider claude` | Filter to a single provider |
|
||||
|
|
|
|||
21
src/main.ts
21
src/main.ts
|
|
@ -2110,20 +2110,35 @@ program
|
|||
}
|
||||
|
||||
const projects = await parseAllSessions(range, opts.provider)
|
||||
const topN = typeof opts.top === 'number' && Number.isFinite(opts.top) ? opts.top : undefined
|
||||
let rows = await aggregateModels(projects, {
|
||||
byTask: !!opts.byTask,
|
||||
byAgent: !!opts.byAgent,
|
||||
taskFilter: opts.task,
|
||||
topN: typeof opts.top === 'number' && Number.isFinite(opts.top) ? opts.top : undefined,
|
||||
// `aggregateModels` filters and slices before the unpriced filter. Its
|
||||
// rows are sorted cost-first, so a small --top would remove exactly the
|
||||
// rows `--unpriced` exists to show. Take the whole set here and slice
|
||||
// after filtering and ranking instead.
|
||||
topN: opts.unpriced ? undefined : topN,
|
||||
minCost: typeof opts.minCost === 'number' && Number.isFinite(opts.minCost) ? opts.minCost : (opts.unpriced ? 0 : 0.01),
|
||||
})
|
||||
if (opts.unpriced) {
|
||||
rows = rows.filter(row => findUnpricedModels([{
|
||||
const unpriced = findUnpricedModels(rows.map(row => ({
|
||||
model: row.model,
|
||||
calls: row.calls,
|
||||
cost: row.costUSD,
|
||||
tokens: row.totalTokens,
|
||||
}]).length > 0)
|
||||
})))
|
||||
const unpricedRank = new Map<string, number>()
|
||||
for (const [rank, usage] of unpriced.entries()) {
|
||||
// Breakdown modes can emit several rows for one model. Keep the first
|
||||
// rank so all rows for that model stay together and N still counts rows.
|
||||
if (!unpricedRank.has(usage.model)) unpricedRank.set(usage.model, rank)
|
||||
}
|
||||
rows = rows
|
||||
.filter(row => unpricedRank.has(row.model))
|
||||
.sort((a, b) => (unpricedRank.get(a.model)! - unpricedRank.get(b.model)!))
|
||||
if (topN !== undefined) rows = rows.slice(0, topN)
|
||||
}
|
||||
|
||||
const fmt = (opts.format ?? 'table').toLowerCase()
|
||||
|
|
|
|||
|
|
@ -776,6 +776,54 @@ describe('models CLI breakdown flags', () => {
|
|||
}
|
||||
})
|
||||
|
||||
// Unpriced rows all sort at $0 in aggregateModels, so the old implementation
|
||||
// preserved transcript/Map order instead of findUnpricedModels' token order.
|
||||
it('keeps unpriced rows when --unpriced is combined with --top', async () => {
|
||||
const home = await mkdtemp(join(tmpdir(), 'codeburn-models-unpriced-top-'))
|
||||
try {
|
||||
const projectDir = join(home, '.claude', 'projects', 'models-unpriced-top')
|
||||
await mkdir(projectDir, { recursive: true })
|
||||
const assistant = (id: string, model: string, timestamp: string, input: number) => JSON.stringify({
|
||||
type: 'assistant',
|
||||
sessionId: 'models-unpriced-top-session',
|
||||
timestamp,
|
||||
cwd: '/tmp/models-unpriced-top',
|
||||
message: {
|
||||
id, type: 'message', role: 'assistant', model,
|
||||
content: [{ type: 'text', text: id }],
|
||||
usage: { input_tokens: input, output_tokens: 100, cache_read_input_tokens: 0, cache_creation_input_tokens: 0 },
|
||||
},
|
||||
})
|
||||
await writeFile(join(projectDir, 'session.jsonl'), [
|
||||
JSON.stringify({
|
||||
type: 'user',
|
||||
sessionId: 'models-unpriced-top-session',
|
||||
timestamp: '2026-05-09T00:00:00.000Z',
|
||||
cwd: '/tmp/models-unpriced-top',
|
||||
message: { role: 'user', content: 'Three unpriced models arrive small-first.' },
|
||||
}),
|
||||
// Transcript order is deliberately different from token order:
|
||||
// 1.1k, 9.1k, 5.1k total tokens. The two largest must survive --top 2.
|
||||
assistant('small', 'zz-unpriced-small', '2026-05-09T00:01:00.000Z', 1000),
|
||||
assistant('largest', 'zz-unpriced-largest', '2026-05-09T00:02:00.000Z', 9000),
|
||||
assistant('middle', 'zz-unpriced-middle', '2026-05-09T00:03:00.000Z', 5000),
|
||||
].join('\n') + '\n')
|
||||
|
||||
const res = spawnSync(
|
||||
process.execPath,
|
||||
['--import', 'tsx', 'src/cli.ts', 'models', '--unpriced', '--top', '2', '--from', '2026-05-09', '--to', '2026-05-09', '--provider', 'claude', '--format', 'json'],
|
||||
{ cwd: process.cwd(), env: { ...process.env, HOME: home, CLAUDE_CONFIG_DIR: join(home, '.claude'), CODEBURN_CACHE_DIR: join(home, '.cache', 'codeburn'), TZ: 'UTC' }, encoding: 'utf-8', timeout: 30_000 },
|
||||
)
|
||||
|
||||
expect(res.status, `stdout: ${res.stdout}\nstderr: ${res.stderr}`).toBe(0)
|
||||
const rows = JSON.parse(res.stdout) as Array<{ model: string }>
|
||||
expect(rows).toHaveLength(2)
|
||||
expect(rows.map(row => row.model)).toEqual(['zz-unpriced-largest', 'zz-unpriced-middle'])
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects --by-task and --by-agent together with a clear error and exit 1', () => {
|
||||
const res = spawnSync(
|
||||
process.execPath,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue