Commit graph

236 commits

Author SHA1 Message Date
iamtoruk
bd43b15342 feat(compare): model comparison with planning rate fix
5-section compare view: Performance (one-shot, retry, self-correction),
Efficiency (cost/call, cost/edit, output/call, cache hit), Category
Head-to-Head bar charts, Working Style, and Context.

Planning rate now detects TaskCreate/TaskUpdate/TodoWrite instead of
only EnterPlanMode (which was never used, showing 0% for all models).
Validated against raw JSONL with zero false positives.

Responsive side-by-side layout at 90+ cols. Self-correction scanner
with compact file skipping and model+timestamp dedup. 274 tests.
2026-04-19 08:34:49 -07:00
iamtoruk
fb24eea186 fix(compare): refine self-correction patterns, skip compact files, deduplicate
Remove high-false-positive patterns (I'm sorry, I should have, sorry for).
Add precise patterns (you're right I, that was incorrect, let me correct).
Skip compact JSONL files that replay compressed context.
Deduplicate by model+timestamp to prevent double-counting.
Fix test timestamps to work with deduplication.
2026-04-19 07:14:02 -07:00
iamtoruk
d04159a056 fix(compare): remove winner column, green highlight is sufficient 2026-04-19 06:52:50 -07:00
iamtoruk
27a3ddd7f8 fix(compare): strip date suffixes from model names for cleaner display 2026-04-19 06:51:26 -07:00
iamtoruk
2a9ecab05c feat(compare): show self-correction counts in context section 2026-04-19 06:47:43 -07:00
iamtoruk
e0d8ecddd9 fix(compare): show compare-styled loading screen during period switch 2026-04-19 06:45:47 -07:00
iamtoruk
73ae1c3786 feat(compare): period switching in compare view, hide status bar
Period changes (arrows, 1-5) now update comparison results in place
instead of returning to dashboard. Status bar hidden in compare view
to reduce clutter.
2026-04-19 06:43:27 -07:00
iamtoruk
f43ef70922 fix(compare): hide provider indicator and shortcut in compare view 2026-04-19 06:29:58 -07:00
iamtoruk
b285320063 fix(compare): wrap all screens in bordered boxes to match dashboard UI 2026-04-19 06:28:55 -07:00
iamtoruk
d52a55afb4 fix(compare): scan project-level JSONL files, improve results layout
Self-correction scanner was only reading JSONL files inside session
subdirectories, missing the main session transcripts stored at the
project level. Also adds bordered box to results and widens winner
column for readability.
2026-04-19 06:25:47 -07:00
iamtoruk
d3864914a9 fix(compare): extract magic numbers, fix React state mutation 2026-04-19 05:46:11 -07:00
iamtoruk
a303fc7174 feat(compare): integrate into dashboard with c shortcut 2026-04-19 05:41:10 -07:00
iamtoruk
e89706b549 feat(compare): add codeburn compare command 2026-04-19 05:37:34 -07:00
iamtoruk
f67cdd2e45 feat(compare): add ModelSelector, ComparisonResults, and CompareView components 2026-04-19 05:31:44 -07:00
iamtoruk
3cb9a7a7bc feat(compare): add self-correction JSONL scanner
Adds scanSelfCorrections() which reads raw .jsonl session files (including subagent dirs) and counts per-model self-correction patterns for use in the model comparison metrics.
2026-04-19 05:25:31 -07:00
iamtoruk
ac9afffed5 feat(compare): add computeComparison with normalized metrics 2026-04-19 05:22:34 -07:00
iamtoruk
9d119bfe40 feat(compare): add ModelStats type and aggregateModelStats 2026-04-19 05:20:37 -07:00
iamtoruk
7cb1cf58bf Add implementation plan for model comparison feature
9 tasks covering: ModelStats aggregation, comparison metrics,
self-correction scanner, Ink components, CLI command, dashboard integration.
2026-04-19 05:09:11 -07:00
iamtoruk
b69bf39deb Add design spec for model comparison feature
Side-by-side comparison of any two AI models using normalized metrics:
cost/call, one-shot rate, retry rate, self-correction rate, cache hit.
Accessible via codeburn compare and dashboard [c] shortcut.
2026-04-19 04:55:32 -07:00
iamtoruk
e3395d241f Fix daily cache gap fill using UTC instead of local time
The gapStart date was constructed with T00:00:00.000Z (UTC midnight),
causing it to land hours before local midnight. In PDT this meant
the gap fill re-parsed a partial slice of the previous day, and the
upsert replaced the full day with that partial data, losing cost.

Bump DAILY_CACHE_VERSION to 3 to force cache rebuild.
2026-04-19 04:23:17 -07:00
iamtoruk
070a378160 chore: bump to 0.7.4 and update CHANGELOG
Some checks are pending
CI / semgrep (push) Waiting to run
2026-04-19 03:36:27 -07:00
iamtoruk
bc92b49c1b feat(mac): auto-update checker and Plan pane button cleanup
Remove the broken "Connect Claude" / "Reconnect Claude" buttons from
the Plan pane -- they opened a terminal session that did nothing useful
for already-logged-in users. Keep only the "Retry" button.

Add an auto-update checker that queries GitHub releases every 2 days in
the background. When a newer menubar build is available, an "Update"
pill appears in the header. Clicking it runs the existing installer
flow (download, replace, relaunch) with no manual steps.
2026-04-19 03:33:37 -07:00
iamtoruk
72ccf34a5a fix: use local timezone for daily date bucketing instead of UTC
Timestamps in session files are UTC ISO strings. Several code paths
extracted the date via .slice(0, 10) which gives the UTC date, while
date range filtering uses local-time boundaries. This caused turns
between UTC midnight and local midnight to be bucketed under the wrong
day -- the menubar showed lower today cost than the TUI because those
turns were attributed to tomorrow (UTC) but filtered as today (local).

format.ts already had a localDateString fix; this applies the same
pattern everywhere via dateKey() in day-aggregator.ts.
2026-04-19 03:18:38 -07:00
iamtoruk
888030fce3 fix: recompute yesterday in daily cache to prevent stale menubar data
The daily cache never re-processed yesterday once cached, so a mid-day
run would freeze partial cost/call data permanently. The "All" provider
path in menubar-json relied on this cache, causing the menubar to show
wildly incorrect numbers while per-provider views (which parse fresh)
were correct. Now yesterday is evicted and recomputed on every run, and
addNewDays upserts instead of skipping duplicates as defense-in-depth.
2026-04-19 03:07:54 -07:00
Resham Joshi
64aae10175
Merge pull request #104 from aaronflorey/fix/opencode-sqlite-esm-loader
fix(sqlite): load node:sqlite in ESM runtime
2026-04-19 02:11:21 -07:00
AgentSeal
11b3de89e4
fix(sqlite): load node:sqlite in ESM runtime
Replace eval-based require with createRequire(import.meta.url) so the SQLite driver loads correctly when the CLI runs as ESM.

This restores OpenCode and Cursor session discovery instead of returning empty results when require is unavailable.
2026-04-19 05:27:05 +00:00
AgentSeal
82df214958
docs: cover --from/--to, avgCostPerSession, and semgrep guard (#99)
Some checks are pending
CI / semgrep (push) Waiting to run
README gains a --from/--to example in the Usage block, a dedicated
'Date range filtering' subsection, and a note that JSON projects[]
now includes avgCostPerSession.

CHANGELOG opens an Unreleased section crediting @lfl1337 for PRs #78
and #80. Flags the projects.csv column-order shift (Avg/Session now
between Cost and Share) so consumers parsing by position read by
header instead.

Co-authored-by: AgentSeal <hello@agentseal.org>
2026-04-18 15:45:45 -07:00
Ninym
c634b10560
feat(report): add --from/--to date range filtering and avgCostPerSession (#80)
* test(cli): failing tests for parseDateRangeFlags helper

* feat(cli): add parseDateRangeFlags helper with local-time dates

* feat(report): add --from/--to date range filtering

* feat(report): add avgCostPerSession to JSON report and CSV/JSON export
2026-04-18 15:11:33 -07:00
Ninym
5932a273a1
chore(ci): add semgrep guard against prototype pollution regressions in provider hot paths (#78)
* chore(ci): add semgrep rule no-bracket-assign-on-literal-object-map

* chore(ci): add workflow running semgrep bracket-assign guard on push/PR

* fix(parser): use Object.create(null) for categoryBreakdown map

* chore(ci): expand semgrep rule to cover ||, ??=, and if-guard variants

* chore(ci): limit push trigger to main and add semgrep --strict

* chore(ci): use jq to enforce finding count (--error unreliable in semgrep 1.x)
2026-04-18 15:10:24 -07:00
AgentSeal
a031c8d32d
chore: point repo URLs at getagentseal org (#97)
Add package.json repository/bugs/homepage fields. Swap hardcoded
AgentSeal/codeburn URLs to getagentseal/codeburn across README,
mac README, macOS menubar star banner, and the menubar installer's
release-API endpoint. 301 redirects keep old URLs working, but
canonical links now point at the current org.

Co-authored-by: AgentSeal <hello@agentseal.org>
2026-04-18 14:55:44 -07:00
AgentSeal
af3676a2b1
Merge pull request #95 from getagentseal/fix/trend-tooltip-per-provider
fix(mac): show correct cost in trend tooltip for per-provider views
2026-04-18 13:41:49 -07:00
AgentSeal
94240f5341 fix(mac): show correct cost in trend tooltip for per-provider views
The trend chart tooltip always displayed `bar.tokens` in its header,
which is zero for provider-filtered history (the CLI only carries
per-provider cost+calls in the daily cache, not tokens). Result: when
you selected Claude/Codex/Cursor/Pi, hovering a bar showed $0.00 even
on days with real spend.

The trend chart's main metric already falls back to cost when tokens
are zero. Pass that same metric value through to the tooltip so both
stay consistent.

Also removed the misleading "No model breakdown available" fallback
line. For provider-filtered views the per-model breakdown legitimately
doesn't exist in the payload, so the tooltip now just shows date +
cost without the error-sounding message.
2026-04-18 13:18:11 -07:00
AgentSeal
70f47f8d9e
Merge pull request #94 from getagentseal/fix/menubar-today-cache-staleness
fix(mac): keep (today, all) cache fresh for menubar title and tab labels
2026-04-18 13:03:07 -07:00
AgentSeal
7ee8b679f9 fix(mac): keep (today, all) cache fresh for menubar title and tab labels
The refresh loop previously skipped `refreshQuietly(.today)` when the
user was already viewing the Today period. That guard meant while the
user was on (today, claude) or any other non-.all provider, the
(today, all) cache went stale. The menubar title and the agent tab
strip both read from that cache, so they displayed stale costs while
the hero section (which reads the currently-viewed payload) showed
the correct fresh value.

Remove the guard so the (today, all) cache refreshes every cycle
regardless of the currently selected period/provider.

Shipped as mac-v0.7.4.
2026-04-18 12:58:50 -07:00
AgentSeal
8ee1f38f86
Merge pull request #92 from AgentSeal/chore/reset-version-to-0.7.3
chore: reset version to 0.7.3 to match published npm
2026-04-18 09:57:52 -07:00
AgentSeal
c83a12efed chore: reset version to 0.7.3 to match published npm 2026-04-18 09:54:03 -07:00
AgentSeal
476b3c51ee
Merge pull request #91 from AgentSeal/revert/remove-npm-oidc
revert: remove npm OIDC publish workflow
2026-04-18 09:53:53 -07:00
AgentSeal
9ac2144950 revert: remove npm OIDC publish workflow
Three consecutive failed publish attempts on a live repo are not
acceptable. Reverting to manual `npm publish` from the laptop, which
has always worked. OIDC can be revisited later in a staging
environment, not on the production package.
2026-04-18 09:51:58 -07:00
AgentSeal
c62a1cf21f
Merge pull request #90 from AgentSeal/chore/bump-0.7.4-rc.2
chore: bump to 0.7.4-rc.2
2026-04-18 09:47:25 -07:00
AgentSeal
35d4d32955 chore: bump to 0.7.4-rc.2 for Node 24 OIDC retry 2026-04-18 09:47:21 -07:00
AgentSeal
ec130037f5
Merge pull request #89 from AgentSeal/fix/node-24-for-oidc
fix(ci): use Node 24 for npm OIDC trusted publishing
2026-04-18 09:47:18 -07:00
AgentSeal
4fccca47d2 fix(ci): use Node 24 for npm OIDC trusted publishing
Node 22 on GitHub's hosted runners currently pins to a broken npm
10.9.7 whose internal `promise-retry` module is missing from the
toolcache (runner-images#13883, nodejs/node#62430). Self-upgrading
via `npm install -g npm@latest` crashes before the install can run,
because `@npmcli/arborist` cannot start without that module.

Node 24 LTS bundles npm 11.x natively, which supports OIDC trusted
publishing out of the box (minimum is 11.5.1, per npm docs). Bumping
the runtime lets us delete the fragile upgrade step entirely.

Test: tag `v0.7.4-rc.2` after merge to validate the flow publishes
successfully with provenance.
2026-04-18 09:46:13 -07:00
AgentSeal
27af2ef96a
Merge pull request #88 from AgentSeal/chore/bump-0.7.4-rc.1
chore: bump to 0.7.4-rc.1
2026-04-18 09:36:52 -07:00
AgentSeal
e7f1b33196 chore: bump to 0.7.4-rc.1 for OIDC retry after npm upgrade fix 2026-04-18 09:36:38 -07:00
AgentSeal
679363a25c
Merge pull request #87 from AgentSeal/fix/npm-version-for-oidc
fix(ci): upgrade npm to 11.5.1+ for OIDC trusted publishing
2026-04-18 09:36:36 -07:00
AgentSeal
832dd4ada1 fix(ci): upgrade npm to 11.5.1+ for OIDC trusted publishing
Node 22 ships with npm 10.x, which does not know how to exchange the
GitHub OIDC id-token for a short-lived npm token. Without this upgrade,
the publish step silently falls back to the empty NODE_AUTH_TOKEN that
setup-node writes to .npmrc, and the registry returns 404.

First test publish (v0.7.4-rc.0) failed at exactly this point, even
though provenance signing via sigstore succeeded, confirming the OIDC
handshake with GitHub was fine and only the npm-side auth was broken.

Fix: `npm install -g npm@latest` before the publish step. Adds ~5s to
runtime.
2026-04-18 09:33:52 -07:00
AgentSeal
bed772b6a5
Merge pull request #86 from AgentSeal/chore/bump-0.7.4-rc.0
chore: bump to 0.7.4-rc.0 for OIDC test publish
2026-04-18 09:27:05 -07:00
AgentSeal
46f72ba348 chore: bump to 0.7.4-rc.0 for OIDC test publish
Pre-release bump to validate npm OIDC trusted publishing end to end:
workflow trigger, Environment approval gate, Trusted Publisher match,
provenance attestation. Will not be tagged as `latest` on npm (npm
auto-excludes SemVer pre-releases from dist-tags). After this RC
succeeds, cut 0.7.4 proper.
2026-04-18 09:25:59 -07:00
AgentSeal
882deafc2b
Merge pull request #84 from AgentSeal/feat/npm-oidc-publish
CI: npm OIDC trusted publishing workflow
2026-04-18 09:10:56 -07:00
AgentSeal
21a4627780
Merge pull request #85 from AgentSeal/chore/ignore-discord-brand-assets
chore: ignore local Discord brand assets
2026-04-18 09:10:53 -07:00