From 11173758b5721a9e5dd25af6198076ac2ede3400 Mon Sep 17 00:00:00 2001 From: ozymandiashh <234437643+ozymandiashh@users.noreply.github.com> Date: Tue, 18 Aug 2026 04:40:21 +0300 Subject: [PATCH 1/2] fix(models): let --unpriced survive --top, and document the flag `--top` is applied inside `aggregateModels`, before the unpriced filter runs, on rows sorted by cost + savings descending. Unpriced rows are $0 on both -- `findUnpricedModels` excludes anything carrying a local-savings baseline -- so they always sort last, and any `--top N` smaller than the number of priced models removed exactly the rows `--unpriced` exists to show. In table form the user then read "No model usage found for the selected period", which is the wrong answer twice over: they do have unpriced models, and nothing tells them the two flags fought. The slice now happens after the filter when `--unpriced` is set. Also adds the flag to the README models table and a changelog entry -- it was added for #969, and a filter nobody can discover does not help anyone find their unpriced models. Follow-up to #985. --- CHANGELOG.md | 4 +++ README.md | 1 + src/main.ts | 9 ++++++- tests/models-report.test.ts | 49 +++++++++++++++++++++++++++++++++++++ 4 files changed, 62 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9242c070..7a68d13d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +### Added +- **`codeburn models --unpriced`.** The dashboard warns about models that price at $0 and points at `codeburn model-alias`, but the list itself was hard to get out of the TUI. This filters the plain-stdout `models` report to exactly those rows, reusing `findUnpricedModels` so local, free, aliased and price-overridden models are treated the same way the warning treats them, and defaulting that mode's min-cost to 0 so $0 rows are not pre-filtered away. Thanks @kocaemre. (#969) + ### Changed - **Codex rollouts parse across worker threads too, and the workload gate now takes bytes or files.** Codex is the bigger half of a real cold parse — a 4 GB rollout corpus against 1.8 GB of Claude sessions — and it was still decoding one file at a time. A whole-file rollout decode now runs on the same pool, against an empty dedup set, and comes back with the calls, the dedup keys it claimed, and the codex-cache entry it would have written; the parent installs all three in the serial loop's order, so `codex-results.json` and every payload come out byte-identical to a serial run. Cross-file state stays where it was: a forked rollout replaying its parent's token_count history collides on the parent's keys and is re-parsed in-process, and no worker ever touches the cache module's per-directory state. Files the Codex cache can serve exactly or resume into from a byte offset never reach a worker — they read a few KB and the resume state belongs to the parent. The workload gate is now pending BYTES alone (200 MB), not file count: 250 pending files holding under a megabyte between them spawned threads that made the run ~5% slower, while a few hundred huge rollouts were being turned away. The count takes `max(pendingFiles / 50, pendingBytes / 200 MB)`, and the per-thread memory budget is derived per parse as `clamp(256 MB, 2 × average pending file + 128 MB, 1 GB)` rather than a flat 256 MB — a 260 MB rollout peaks near 430 MB in its worker and scales linearly with the pool, so the flat figure over-subscribed exactly the workload this adds. The decision is per provider, and at most one pool is alive at a time. - **A large cold Claude parse now runs across worker threads.** Reading, decoding and line-parsing a session JSONL is per-file work that never touches anything shared, so it moves onto `worker_threads`; each worker ships its parsed turns back as a JSON string and the parent installs them in the exact order the serial loop would. Everything with cross-file state — the streaming-message dedup, canonical project paths, spawn links, PR correlation, progress saves — stays on the main thread, and a file whose message ids were already claimed by an earlier file (or whose worker failed) is simply re-parsed in-process, so the session cache and every payload are identical either way. On a 6 GB corpus a cold `status` drops from 27.5s to 14.8s with peak RSS up 2.27 GB → 2.52 GB. Threads only engage for a genuinely large cold parse: never with under 200 MB behind the pending whole-file re-parses, 2 or fewer cores, or under 4 GB of available memory — so warm and incremental runs are untouched and spawn nothing. Otherwise the count is `min(cores - 1, min(0.25 × available, 2 GB) / 256 MB, pendingFiles / 50)`, where available is `process.availableMemory()` (cgroup-aware in containers) rather than free memory, which on macOS reports free pages and would switch the feature on and off between runs. `CODEBURN_PARSE_WORKERS=0` forces the serial parse and `CODEBURN_PARSE_WORKERS=N` forces N (capped at the core count), both bypassing every gate; `CODEBURN_VERBOSE=1` prints the resolved count and why. @@ -15,6 +18,7 @@ - **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972) ### Fixed +- **`codeburn models --unpriced --top N` returned nothing.** `--top` is applied inside `aggregateModels`, before the unpriced filter, on rows sorted by cost + savings descending — and unpriced rows are $0 on both, so they sorted last and the slice removed exactly the rows the flag exists to show. A user with unpriced models was told they had none. The slice now runs after the filter. (#969) - **Bash command splitting was quadratic on long whitespace-heavy commands.** The separator regex retried its leading `\s*` from every offset; matching the separator alone and widening over whitespace by hand makes cold parse ~24% and warm ~40% faster on large corpora, output unchanged. - **Cold parse no longer retains full message bodies through cached previews.** `flatSlice` skipped its Buffer round-trip for strings already within the bound, but provider adapters pre-truncate user-message previews with `.slice(0, 500)` before the cache-site call — those pre-sliced views are still V8 SlicedStrings pinning their large parent, so the retention that OOM'd cold parses of large histories survived. The round-trip now always runs. - **Kiro sessions carry the real `projectPath`** (CLI meta.cwd, v2 `workspacePaths[0]`, workspace sessions' `workspaceDirectory`), so git-repo attribution can resolve them; previously they were attribution-blind. Bumps the kiro parse version, so the first run after upgrade re-parses kiro history once, and kiro sessions in linked git worktrees now group under the main repo. diff --git a/README.md b/README.md index 2159e1da..da3bc888 100644 --- a/README.md +++ b/README.md @@ -484,6 +484,7 @@ Sync sends token counts, costs, models, and projects, never prompts or code. Thi | `codeburn models --by-task` | Break each model into per-task-type rows | | `codeburn models --by-agent` | Break each model into per-agent rows: which agent drove which model's spend (`(main)` covers non-agent sessions; `--min-cost 0` shows sub-cent agents) | | `codeburn models --top 10` | Only the 10 most expensive models | +| `codeburn models --unpriced` | Only models with usage that currently price at $0 — the copyable form of the unpriced-models warning | | `codeburn models --format markdown` | Emit a paste-friendly markdown table | | `codeburn models --task feature` | Filter to feature-development work | | `codeburn models --provider claude` | Filter to a single provider | diff --git a/src/main.ts b/src/main.ts index a8af9bba..192cb72b 100644 --- a/src/main.ts +++ b/src/main.ts @@ -2100,11 +2100,17 @@ program } const projects = await parseAllSessions(range, opts.provider) + const topN = typeof opts.top === 'number' && Number.isFinite(opts.top) ? opts.top : undefined let rows = await aggregateModels(projects, { byTask: !!opts.byTask, byAgent: !!opts.byAgent, taskFilter: opts.task, - topN: typeof opts.top === 'number' && Number.isFinite(opts.top) ? opts.top : undefined, + // `aggregateModels` slices to topN on rows sorted by cost + savings + // descending. Unpriced rows are $0 on both (findUnpricedModels excludes + // anything with a local-savings baseline), so they always sort last and + // `--top` would remove exactly the rows `--unpriced` exists to show. + // Take the whole set here and slice after filtering instead. + topN: opts.unpriced ? undefined : topN, minCost: typeof opts.minCost === 'number' && Number.isFinite(opts.minCost) ? opts.minCost : (opts.unpriced ? 0 : 0.01), }) if (opts.unpriced) { @@ -2114,6 +2120,7 @@ program cost: row.costUSD, tokens: row.totalTokens, }]).length > 0) + if (topN !== undefined) rows = rows.slice(0, topN) } const fmt = (opts.format ?? 'table').toLowerCase() diff --git a/tests/models-report.test.ts b/tests/models-report.test.ts index 8808ca58..426f3eaa 100644 --- a/tests/models-report.test.ts +++ b/tests/models-report.test.ts @@ -776,6 +776,55 @@ describe('models CLI breakdown flags', () => { } }) + // `--top` is applied inside aggregateModels, before the unpriced filter runs, + // on rows sorted by cost + savings descending. Unpriced rows are $0 on both, + // so they sort last and a small --top removed exactly the rows --unpriced + // exists to surface: the user was told they had no unpriced models. + it('keeps unpriced rows when --unpriced is combined with --top', async () => { + const home = await mkdtemp(join(tmpdir(), 'codeburn-models-unpriced-top-')) + try { + const projectDir = join(home, '.claude', 'projects', 'models-unpriced-top') + await mkdir(projectDir, { recursive: true }) + const assistant = (id: string, model: string, timestamp: string, input: number) => JSON.stringify({ + type: 'assistant', + sessionId: 'models-unpriced-top-session', + timestamp, + cwd: '/tmp/models-unpriced-top', + message: { + id, type: 'message', role: 'assistant', model, + content: [{ type: 'text', text: id }], + usage: { input_tokens: input, output_tokens: 100, cache_read_input_tokens: 0, cache_creation_input_tokens: 0 }, + }, + }) + await writeFile(join(projectDir, 'session.jsonl'), [ + JSON.stringify({ + type: 'user', + sessionId: 'models-unpriced-top-session', + timestamp: '2026-05-09T00:00:00.000Z', + cwd: '/tmp/models-unpriced-top', + message: { role: 'user', content: 'Two priced models outrank the unpriced one.' }, + }), + // Both priced models cost more than the $0 unpriced row, so they take + // both --top slots unless the filter runs first. + assistant('opus', 'claude-opus-4-6', '2026-05-09T00:01:00.000Z', 5000), + assistant('sonnet', 'claude-sonnet-4-6', '2026-05-09T00:02:00.000Z', 3000), + assistant('unpriced', 'zz-unpriced-frontier-model', '2026-05-09T00:03:00.000Z', 2000), + ].join('\n') + '\n') + + const res = spawnSync( + process.execPath, + ['--import', 'tsx', 'src/cli.ts', 'models', '--unpriced', '--top', '2', '--from', '2026-05-09', '--to', '2026-05-09', '--provider', 'claude', '--format', 'json'], + { cwd: process.cwd(), env: { ...process.env, HOME: home, CLAUDE_CONFIG_DIR: join(home, '.claude'), CODEBURN_CACHE_DIR: join(home, '.cache', 'codeburn'), TZ: 'UTC' }, encoding: 'utf-8', timeout: 30_000 }, + ) + + expect(res.status, `stdout: ${res.stdout}\nstderr: ${res.stderr}`).toBe(0) + const rows = JSON.parse(res.stdout) as Array<{ model: string }> + expect(rows.map(row => row.model)).toEqual(['zz-unpriced-frontier-model']) + } finally { + await rm(home, { recursive: true, force: true }) + } + }) + it('rejects --by-task and --by-agent together with a clear error and exit 1', () => { const res = spawnSync( process.execPath, From d81fca306686cfd64d8dc2f290665720ab1554d5 Mon Sep 17 00:00:00 2001 From: ozymandiashh <234437643+ozymandiashh@users.noreply.github.com> Date: Tue, 18 Aug 2026 05:34:51 +0300 Subject: [PATCH 2/2] fix(models): rank unpriced rows before --top slices them Filtering before the slice was necessary but not sufficient. Every unpriced row is $0 on both cost and savings -- findUnpricedModels excludes anything carrying a local-savings baseline -- so they all tie under aggregateModels' sort key, and Array#sort is stable. The surviving order was Map insertion order: the order each model's first assistant call appears in the transcript. So --unpriced --top N kept the N that showed up earliest, and a model holding almost all of the unpriced volume was dropped if it appeared late. findUnpricedModels already sorts by tokens descending, then calls, then model name, and the dashboard warning renders that order. It is now called once over the whole row set, its order becomes a rank index, and the rows are ranked before the slice -- so the CLI and the warning agree on which N, which is what the README row claims. In the breakdown modes several rows share one model, so they share that model's rank and N still counts rows. The previous test could not catch this: its fixture held one unpriced model, so --top 2 never truncated anything and deleting the slice line left the suite green. It now uses three unpriced models emitted in an order that differs from their size order, and asserts which two survive rather than only how many. --- CHANGELOG.md | 2 +- src/main.ts | 22 +++++++++++++++------- tests/models-report.test.ts | 21 ++++++++++----------- 3 files changed, 26 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a68d13d..3d305d47 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,7 @@ - **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972) ### Fixed -- **`codeburn models --unpriced --top N` returned nothing.** `--top` is applied inside `aggregateModels`, before the unpriced filter, on rows sorted by cost + savings descending — and unpriced rows are $0 on both, so they sorted last and the slice removed exactly the rows the flag exists to show. A user with unpriced models was told they had none. The slice now runs after the filter. (#969) +- **`codeburn models --unpriced --top N` returned nothing for a `--top N` smaller than the number of priced models.** `--top` is applied inside `aggregateModels`, before the unpriced filter, on rows sorted cost-first — and unpriced rows are $0 on both, so they sorted last and the slice removed exactly the rows the flag exists to show. A user with unpriced models was told they had none. The slice now runs after the filter — and after ranking, because unpriced rows tie at $0 on both keys, so slicing them in aggregate order kept whichever models happened to appear earliest in the transcript rather than the largest. The order now matches the one the unpriced-models warning shows. (#969) - **Bash command splitting was quadratic on long whitespace-heavy commands.** The separator regex retried its leading `\s*` from every offset; matching the separator alone and widening over whitespace by hand makes cold parse ~24% and warm ~40% faster on large corpora, output unchanged. - **Cold parse no longer retains full message bodies through cached previews.** `flatSlice` skipped its Buffer round-trip for strings already within the bound, but provider adapters pre-truncate user-message previews with `.slice(0, 500)` before the cache-site call — those pre-sliced views are still V8 SlicedStrings pinning their large parent, so the retention that OOM'd cold parses of large histories survived. The round-trip now always runs. - **Kiro sessions carry the real `projectPath`** (CLI meta.cwd, v2 `workspacePaths[0]`, workspace sessions' `workspaceDirectory`), so git-repo attribution can resolve them; previously they were attribution-blind. Bumps the kiro parse version, so the first run after upgrade re-parses kiro history once, and kiro sessions in linked git worktrees now group under the main repo. diff --git a/src/main.ts b/src/main.ts index 192cb72b..25d8df08 100644 --- a/src/main.ts +++ b/src/main.ts @@ -2105,21 +2105,29 @@ program byTask: !!opts.byTask, byAgent: !!opts.byAgent, taskFilter: opts.task, - // `aggregateModels` slices to topN on rows sorted by cost + savings - // descending. Unpriced rows are $0 on both (findUnpricedModels excludes - // anything with a local-savings baseline), so they always sort last and - // `--top` would remove exactly the rows `--unpriced` exists to show. - // Take the whole set here and slice after filtering instead. + // `aggregateModels` filters and slices before the unpriced filter. Its + // rows are sorted cost-first, so a small --top would remove exactly the + // rows `--unpriced` exists to show. Take the whole set here and slice + // after filtering and ranking instead. topN: opts.unpriced ? undefined : topN, minCost: typeof opts.minCost === 'number' && Number.isFinite(opts.minCost) ? opts.minCost : (opts.unpriced ? 0 : 0.01), }) if (opts.unpriced) { - rows = rows.filter(row => findUnpricedModels([{ + const unpriced = findUnpricedModels(rows.map(row => ({ model: row.model, calls: row.calls, cost: row.costUSD, tokens: row.totalTokens, - }]).length > 0) + }))) + const unpricedRank = new Map() + for (const [rank, usage] of unpriced.entries()) { + // Breakdown modes can emit several rows for one model. Keep the first + // rank so all rows for that model stay together and N still counts rows. + if (!unpricedRank.has(usage.model)) unpricedRank.set(usage.model, rank) + } + rows = rows + .filter(row => unpricedRank.has(row.model)) + .sort((a, b) => (unpricedRank.get(a.model)! - unpricedRank.get(b.model)!)) if (topN !== undefined) rows = rows.slice(0, topN) } diff --git a/tests/models-report.test.ts b/tests/models-report.test.ts index 426f3eaa..4e1dba09 100644 --- a/tests/models-report.test.ts +++ b/tests/models-report.test.ts @@ -776,10 +776,8 @@ describe('models CLI breakdown flags', () => { } }) - // `--top` is applied inside aggregateModels, before the unpriced filter runs, - // on rows sorted by cost + savings descending. Unpriced rows are $0 on both, - // so they sort last and a small --top removed exactly the rows --unpriced - // exists to surface: the user was told they had no unpriced models. + // Unpriced rows all sort at $0 in aggregateModels, so the old implementation + // preserved transcript/Map order instead of findUnpricedModels' token order. it('keeps unpriced rows when --unpriced is combined with --top', async () => { const home = await mkdtemp(join(tmpdir(), 'codeburn-models-unpriced-top-')) try { @@ -802,13 +800,13 @@ describe('models CLI breakdown flags', () => { sessionId: 'models-unpriced-top-session', timestamp: '2026-05-09T00:00:00.000Z', cwd: '/tmp/models-unpriced-top', - message: { role: 'user', content: 'Two priced models outrank the unpriced one.' }, + message: { role: 'user', content: 'Three unpriced models arrive small-first.' }, }), - // Both priced models cost more than the $0 unpriced row, so they take - // both --top slots unless the filter runs first. - assistant('opus', 'claude-opus-4-6', '2026-05-09T00:01:00.000Z', 5000), - assistant('sonnet', 'claude-sonnet-4-6', '2026-05-09T00:02:00.000Z', 3000), - assistant('unpriced', 'zz-unpriced-frontier-model', '2026-05-09T00:03:00.000Z', 2000), + // Transcript order is deliberately different from token order: + // 1.1k, 9.1k, 5.1k total tokens. The two largest must survive --top 2. + assistant('small', 'zz-unpriced-small', '2026-05-09T00:01:00.000Z', 1000), + assistant('largest', 'zz-unpriced-largest', '2026-05-09T00:02:00.000Z', 9000), + assistant('middle', 'zz-unpriced-middle', '2026-05-09T00:03:00.000Z', 5000), ].join('\n') + '\n') const res = spawnSync( @@ -819,7 +817,8 @@ describe('models CLI breakdown flags', () => { expect(res.status, `stdout: ${res.stdout}\nstderr: ${res.stderr}`).toBe(0) const rows = JSON.parse(res.stdout) as Array<{ model: string }> - expect(rows.map(row => row.model)).toEqual(['zz-unpriced-frontier-model']) + expect(rows).toHaveLength(2) + expect(rows.map(row => row.model)).toEqual(['zz-unpriced-largest', 'zz-unpriced-middle']) } finally { await rm(home, { recursive: true, force: true }) }