Commit graph

22 commits

Author SHA1 Message Date
AgentSeal
bf6eee00f3 fix(report): only exact zero-rate overrides suppress the unpriced flag
getModelCosts resolves exact overrides before any table hit, so an exact
zero-rate override provably priced the model and means free. Prefix and
case-insensitive overrides resolve after table hits: a zero-rate stub
shadowed by one still reports $0 from the stub, so it stays flagged.

Refs #638
2026-07-09 21:22:23 +02:00
AgentSeal
e6162f6b50 fix(report): address unpriced-detector review findings
Adversarial review found two false-negative paths and two polish items:

- Drop the reverse-display-name suppression. Droid prices the lowercased
  display name and gets $0, so a $0 row keyed "Sonnet 4.6" is real
  uncounted spend; suppressing it because claude-sonnet-4-6 prices today
  hid exactly the failure this feature exists to expose.
- Treat zero-rate pricing hits as unpriced. LiteLLM ships [0,0] stubs
  for models it lists without a price; a stub hit means unknown price,
  not free. Explicit zero-rate user overrides still count as free.
- Make MenubarPayload.current.unpricedModels optional for source
  compatibility with payload producers that predate the field.
- Replace localeCompare with a byte comparison so tied rows sort
  identically on every host locale.

Refs #638
2026-07-09 21:17:47 +02:00
AgentSeal
231e7cc0d4 feat(report): surface unpriced models across overview, JSON, dashboard, and MCP
Models missing from the pricing tables silently contributed $0 to every
total with no indication anywhere (the warning was gated behind
CODEBURN_VERBOSE). Add render-time detection that flags aggregated model
rows with usage but $0 cost whose pricing lookup fails right now, and
surface it in overview, report/today/month JSON (unpricedModels), the
TUI By Model panel, and the MCP get_usage summary.

Render-time detection covers cached sessions (cost is computed at parse
time) and heals as soon as pricing data, an alias, or a price override
arrives. Rows with real cost are never flagged: aggregation keys rows by
display name, which the pricing lookup misses; $0 display-name rows are
reverse-resolved to their raw id before flagging so sessions cached
before their model's pricing landed don't get a misleading alias hint.
Local models and model-savings mappings stay excluded: $0 is correct
for them.

Fixes #638
2026-07-09 21:04:56 +02:00
AgentSeal
f2fa575a52 Fix model pricing variant resolution 2026-07-08 22:55:44 +02:00
Resham Joshi
f1a4e8cc4f
fix(cursor): use Cursor's real context tokens for input (#574) (#575)
* fix(cursor): use Cursor's real context tokens for input

Current Cursor builds leave the per-bubble tokenCount at {0,0}, so the provider
fell back to estimating input from visible text plus a second agentKv
content-char pass that double-counted the same conversation. Cursor records its
own tokenizer-accurate context size per conversation in
composerData.promptTokenBreakdown (the number behind the in-app context-window
bar); read that and credit it once per conversation for input instead.

Measured on a real local DB: today's Cursor input went 44,873 -> 168,486 tokens,
matching the sum of per-conversation context. The admin portal still counts
cumulative-per-turn plus cache, which are server-side only, so an opt-in Cursor
API stays the path to exact parity.

Output is a reply-text estimate; agentKv is retained for a tools/bash breakdown
in a follow-up.

* feat(cursor): add tools and bash-command breakdown from agentKv

Cursor logs the agent's tool calls (Read, Grep, Glob, Shell, ...) in agentKv
blobs. Join them to conversations via the turn requestId (carried on the bubble's
$.requestId and inherited positionally by the turn's agentKv rows) and attach each
conversation's tool list and Shell commands to the call that carries its input.

Measured on a real DB: Cursor now reports tools {Read, Grep, Glob, Shell,
SemanticSearch} and the executed shell commands, which were previously empty.

Removes the now-superseded parseAgentKv content-char estimate.

* fix(cursor): price composer-2.5 as Sonnet 4.6

composer-2.5 was missing from the built-in Cursor model aliases, so its usage
showed $0. Map it to claude-sonnet-4-6 like composer-2 (per cursor.com/blog).

* fix(cursor): cache version, model attribution, user message join, tool classification

- Bump CURSOR_CACHE_VERSION to 5: parser semantics changed (parseAgentKv
  removed, real context tokens from composerData.promptTokenBreakdown),
  stale v4 caches would show double-counted agentKv calls.
- Fix model attribution: real input tokens are credited on user bubbles
  (type=1) which carry no modelInfo. Add a pre-pass building composerId ->
  model from assistant bubbles so pricing/display uses the conversation's
  actual model instead of the default cursor-auto/sonnet-4.5.
- Fix buildUserMessageMap: was keying by JSON conversationId (empty in
  current Cursor builds). Now extracts composerId from the bubble key,
  matching parseBubbles.
- Add 'Shell' to BASH_TOOLS in classifier: Cursor's agent uses 'Shell'
  as the tool name, but it was missing from the bash tool set so Cursor
  agent turns with shell commands wouldn't classify as bash/build/test.
- Fix null coalescing in loadComposerInputTokens: r.used ?? r.ctx would
  fall through on a valid totalUsedTokens of 0. Use explicit null check.
- Decouple agentTools attachment from input credit: tools/bash were only
  attached on the first credited turn (creditedHere), silently dropping
  tool usage from subsequent turns in multi-turn conversations.
- Update stale comment about parseAgentKv being kept for a follow-up.
- Add tests for real token crediting, once-per-conversation, fallback,
  contextTokensUsed, tool/bash attribution, and model attribution.

* fix(cursor): avoid duplicating aggregated agent tools

* fix(cursor): price house composer models from Cursor's published rates

composer-1/1.5/2/2.5 were proxied to Claude Sonnet, overcounting cost
(~6x for composer-2/2.5). Use Cursor's published per-model rates instead,
and note in the parser why local reads undercount the admin console.

Co-authored-by: AgentSeal <hello@agentseal.org>

* fix(cursor): estimate non-Composer turn input from the agent stream

Non-Composer sessions (e.g. GPT) record no context-window meter and keep
the prompt in the agent stream, so the user bubble's own text is empty.
Those turns hit the 0/0-token fallback with text_length 0 and were dropped
entirely, so that model's traffic never appeared in the report.

loadAgentToolsByComposer now also sums the user-role stream text length per
conversation, and the meterless fallback estimates input from it (chars/4),
credited once per conversation, when the bubble text is empty. Turns with no
stream text are left untouched, so no phantom tokens are invented.

* fix(cursor): stable conversation crediting, restored stream coverage, and cache invalidation

Review fixes for the real-token accounting:

Conversation input now lands on one composer-anchored record
(cursor:composer-input:<id>) timestamped at composerData.createdAt, so
the credited day no longer depends on the parse window or cache floors,
daily-cache gap fills dedupe instead of multiplying, and each
conversation picks exactly one input source (real bubble tokenCounts,
the context meter, the agent stream, or visible text) so sources can
never stack or double count. A zero totalUsedTokens no longer shadows
contextTokensUsed.

The agent stream regained what the parseAgentKv removal dropped: tool
and system rows count as context, stream-only replies count as output,
and sessions with no bubble join are emitted again (DB mtime timestamp,
as before). Block-array content is measured by its text, not its JSON
envelope. Rows written before their requestId appears buffer forward
instead of inheriting the previous conversation, and a system row closes
the boundary. Tool names canonicalize to Bash and commands go through
extractBashCommands so cross-provider breakdowns merge; the classifier
no longer special-cases Shell (which also reclassified Copilot turns).
User bubbles consume their own queue entry so assistant replies pair
with the right question, and every cursor call is flagged
costIsEstimated.

The requestId and model joins ride the existing budgeted bubble scan
instead of two new unbounded full-table decodes, and the composerData
read seeks the key range. SQLITE_BUSY now propagates to the parser's
retry path instead of caching a silently degraded parse.

Upgrades actually take effect: the session cache gets a cursor parse
version, DAILY_CACHE_VERSION bumps to 10 so finalized days re-hydrate
under the new accounting, the cursor results cache bumps to v6, and the
builtin composer rates participate in the price config hash (rates now
cite cursor.com/docs/models).

Verified against a real Cursor store: all metered conversations match
the on-disk meter exactly, narrow and wide parse windows anchor
identically, repeat runs are byte-identical, and agentKv-only sessions
reappear.

---------

Co-authored-by: ozymandiashh <234437643+ozymandiashh@users.noreply.github.com>
2026-07-02 04:53:07 +02:00
ozymandiashh
2a0edd0d68
feat(pricing): add user price overrides for models (#390) (#560) 2026-06-28 19:07:29 +02:00
Resham Joshi
71b1a9ebce
fix: clean model names in reports and re-hydrate daily cache for new providers (#550)
Two post-0.9.12 cleanups.

Report model names: getShortModelName resolves a model's pricing alias before looking up its display name, so models priced via a sibling alias (ZCode/Hermes GLM-5.2 via glm-5p1, Grok Build via grok-build-0.1) leaked the internal pricing key as the model name in report --format json and the menubar model breakdown. Grok Composer was unmapped and showed raw. Add SHORT_NAMES entries so each resolves to its real name (GLM-5.2, Grok Build, Grok Composer 2.5 Fast). The models command was already correct because it uses each provider's own modelDisplayName.

Daily cache: providers added since the v8 rollup (Grok, Hermes, ZCode) parse usage that older binaries skipped, so days cached at v8 omit them and report 0 across history. Bump DAILY_CACHE_VERSION and MIN_SUPPORTED_VERSION to 9 to force a one-time full re-hydration so new providers backfill without a manual cache clear.
2026-06-22 03:38:50 +02:00
ozymandiashh
4dcb7e6c3d
fix(models): price Hermes lowercase glm-5.2 the same as GLM-5.2 (#545) 2026-06-22 01:02:59 +02:00
ozymandiashh
7c2d36f1f0
Distinguish gpt-5.3-codex-spark from base GPT-5.3 Codex label (#539)
gpt-5.3-codex-spark is a distinct model variant, but the longest-first
startsWith(key+'-') matcher (intended for reasoning suffixes -high/-low)
swept it into the base 'GPT-5.3 Codex' label, making a distinct model look
like the retired base in today/models/status output. Add an explicit
display entry in SHORT_NAMES and the codex provider so the longer key wins;
-high/-low still fall through to the base. Pricing is unaffected (LiteLLM
has no spark entry; cost falls back to the base rate as before).

Fixes #461
2026-06-22 00:44:56 +02:00
Tiago Santos
75c32e6d65
fix: fix and improve test isolation and collision with environment (#530)
* fix: fix and improve test isolation and collision with environment

* docs: remove unnecessary comment

* test(env-isolation): clear CODEBURN_FORCE_MACOS_MAJOR and pin TZ

Two env vars read in src/ were not isolated: CODEBURN_FORCE_MACOS_MAJOR
(now cleared so it cannot leak between tests) and TZ (now pinned to UTC,
since clearing it falls back to the OS zone and would shift date buckets
versus a clean CI runner).

---------

Co-authored-by: AgentSeal <hello@agentseal.org>
2026-06-20 13:42:10 +02:00
Resham Joshi
ad251cfa3d
chore(pricing): drop manual Fable/Mythos patch; fable now gap-filled from models.dev/OpenRouter; keep Fable 5 name (#464) 2026-06-09 21:22:38 +02:00
Resham Joshi
a385f65dee
feat(pricing): automatic gap-fill from models.dev and OpenRouter (#457)
Keep model pricing automatic instead of hand-coding new models. The bundler
now layers three sources in priority order: LiteLLM (broad list prices),
hand-curated MANUAL_ENTRIES overrides, then a separate last-resort fallback
file gap-filled from models.dev first-party makers (official direct prices)
and OpenRouter (resale backstop). New models such as MiniMax-M3 ($0.6/$2.4)
now price correctly with no per-model code.

The fallback is written to its own pricing-fallback.json and consulted only
case-insensitively as the final step in getModelCosts, so a reseller variant
name can never shadow a canonical or aliased match.

Fixes surfaced while building and verifying this:
- Alias precedence: LiteLLM ships snowflake/claude-4-opus ($5), which the
  bundler strips to a bare claude-4-opus key that shadowed the curated alias
  to claude-opus-4 ($15 official). An explicit alias for a bare name now wins
  over a coincidental stripped reseller key; the prefixed gateway price is
  still returned for the fully-qualified id.
- Zero-stub guard: LiteLLM [0,0] price stubs (e.g. GigaChat-2-Max) are
  excluded from the case-insensitive index so a case-mismatched query stays
  null and keeps firing the unknown-model warning instead of silently
  reporting $0.
- Negative-sentinel guard: OpenRouter returns -1 for variable/BYOK-priced
  models. The bundler now rejects any non-positive rate pair (and strips the
  sentinel from cache fields) so a negative per-token cost can never ship and
  subtract from spend totals.

Bundler hardening: bareKey strips @pin and date suffixes to match the runtime
canonical form, seen-set dedupes on both full and bare key shapes, and it logs
MANUAL_ENTRIES now covered upstream plus models.dev allowlist drift. Extracted
buildCosts so the cache-cost heuristics live in one place. Added a data-hygiene
test that fails CI if a rebundle reintroduces negative, free, or unreachable
fallback entries.
2026-06-09 21:17:23 +02:00
Resham Joshi
c36f3afa76
chore(pricing): temp Fable 5 + Mythos 5 launch pricing ($10/$50 per M) + names until LiteLLM indexes them (#463) 2026-06-09 20:51:51 +02:00
iamtoruk
aa9bd9f0f1 feat(models): derive Claude names and fast multipliers automatically
New Claude releases no longer need a hand-maintained SHORT_NAMES entry or
FAST_MULTIPLIERS row. Display names are derived from the claude-<family>-<major>-<minor>
id, and the fast-mode multiplier rides along as a 5th element in the LiteLLM
snapshot tuple (provider_specific_entry.fast). Fixes #420: claude-opus-4-8 gets
its own line and correct pricing instead of falling into the Opus 4 bucket.
2026-05-31 05:17:14 -07:00
Tony Amirault
bbbdcd4eb8
fix(models): map Warp Claude variants to canonical pricing IDs (#378)
Co-authored-by: Tony Amirault <tony.amirault@wakam.com>
Co-authored-by: Oz <oz-agent@warp.dev>
2026-05-22 02:05:26 -07:00
ozymandiashh
d0f1f82bf4
Fix Antigravity 2 Gemini 3.5 Flash tracking (#377) 2026-05-22 02:01:20 -07:00
ozymandiashh
17eada2aa1
fix: DeepSeek v4 Claude pricing through stale runtime cache (#367)
Some checks are pending
CI / semgrep (push) Waiting to run
2026-05-21 00:34:53 -07:00
AgentSeal
c85beeaeae
Fix Claude 1-hour cache write pricing (#317)
Co-authored-by: ozymandiashh <234437643+ozymandiashh@users.noreply.github.com>
Co-authored-by: iamtoruk <hello@agentseal.org>
2026-05-11 21:23:04 -07:00
Resham Joshi
cdf7169a89
Cursor model aliases: cover every variant so non-Auto sessions price (#159) (#290)
Cursor emits model names in a `claude-<dot-version>-<tier>` shape
(`claude-4.6-sonnet`, `claude-4.5-opus`, `claude-4.5-opus-high-thinking`,
etc.) plus its own `composer-1` house model. None of these match
the canonical LiteLLM pricing keys (`claude-sonnet-4-6`,
`claude-opus-4-5`).

The alias map in `src/models.ts` filled some of these in v0.9.4
but missed:

- plain no-suffix forms: `claude-4.5-opus`, `claude-4.5-sonnet`,
  `claude-4.6-opus`
- haiku tier: `claude-4.5-haiku`, `claude-4.6-haiku`
- forward-looking: `claude-4.7-opus`
- Cursor's house model: `composer-1`

The dashboard rendered $0 for sessions that used any unaliased
model — visible in the screenshots posted in #159 even after the
v0.9.4 fix that added the `-thinking` variants.

This PR fills the gaps and adds 16 regression tests under
`Cursor model variants resolve to pricing` that assert every
model name in `src/providers/cursor.ts:modelDisplayNames` plus
the additional plain forms resolves to a non-null pricing entry
with `inputCostPerToken > 0` and `outputCostPerToken > 0`. So a
future LiteLLM snapshot bump or a typo in the alias map will fail
the test before users see $0.

Direct hits in the snapshot (no alias needed): `gpt-5`, `gpt-5.2`,
`grok-code-fast-1`, `gemini-3-pro` (already aliased). These are
covered in the test suite as well so a snapshot that drops them
would also be caught.

Tests: 45 files, 617 passing locally (16 new). Closes #159.
2026-05-10 03:27:44 -07:00
iamtoruk
c2ab80d6e2 Merge main into feat/omp-support-model-aliases
Brings the PR branch up to the current main so the OMP provider and the
model-alias command can land cleanly. Resolves six merge conflicts and
applies a handful of small fixups alongside the resolution so the
feature matches the conventions set by the cursor-agent merge earlier
today.

Conflict resolutions:

  README.md               Combine cursor-agent and OMP rows in provider
                          list, Requirements, and data-location table;
                          take main's Node 22+ and node:sqlite text.
  src/cli.ts              Keep both new commands: model-alias and plan.
  src/config.ts           Add modelAliases alongside plan on the config
                          type.
  src/providers/index.ts  Keep the cursor-agent lazy-loader from main
                          and add omp to coreProviders. Fold the two
                          pi-module imports into one statement.
  src/providers/pi.ts     Keep the discovery-cache snapshot path from
                          main and the providerName parameterization
                          from the PR. Propagate providerName through
                          saveDiscoveryCache, loadDiscoveryCache, the
                          parserVersion tag, and the dedup key prefix
                          so OMP sources no longer stamp 'pi:' inside
                          their cache entries or dedup keys.
  tests/models.test.ts    Keep main's pricing-and-short-name tests and
                          add the PR's alias tests alongside, sharing a
                          single loadPricing setup and an afterEach
                          alias reset.

Fixups in the same commit:

  src/models.ts           Replace ?? chain in resolveAlias with
                          Object.hasOwn checks. The previous form
                          returned Object.prototype for a model named
                          '__proto__' and broke downstream
                          canonical.startsWith calls. Caught by the
                          existing prototype-pollution test suite.
  src/providers/pi.ts     Use source.provider in the dedup key prefix
                          and add a trailing newline to the file.
  tests/providers/omp.test.ts  Expect 'omp:' in the dedup key for OMP
                          sources, matching the fix above.

Feature work by @cgrossde.
2026-04-21 03:16:28 -07:00
iamtoruk
a4d261a536 fix: pricing accuracy, stream leak, CSV injection hardening
- Remove bidirectional fuzzy match in getModelCosts that could return
  wrong pricing when a short canonical name prefix-matched a longer key
- Use explicit undefined check in parseLiteLLMEntry so free models with
  zero cost are not silently dropped from the LiteLLM pricing database
- Destroy read stream in finally block of readSessionLines to prevent
  file descriptor leaks when the generator is abandoned early
- Extend CSV injection escaping to cover tab and carriage-return prefixes
- Add optional chaining fallback for empty periods in exportCsv/exportJson
- Add regression tests for all fixes (models, export, fs-utils)
2026-04-20 14:49:32 -07:00
AgentSeal
79e67f0bc9
Add OMP provider support and model alias mapping
- Add OMP provider reading from ~/.omp/agent/sessions (same JSONL
  format as Pi, shared parser)
- Parameterize discoverSessionsInDir with provider name so sessions
  carry correct provider field
- Add BUILTIN_ALIASES for proxy model name variants (anthropic--claude-*
  double-dash format) that don't match LiteLLM keys
- Add model-alias CLI command for user-defined name mappings
- Wire setModelAliases into preAction after config load
- Add modelAliases field to CodeburnConfig
- Update README: OMP in provider table, model-alias section
2026-04-16 23:35:46 +02:00