feat(pricing): automatic gap-fill from models.dev and OpenRouter (#457)

Browse source

Keep model pricing automatic instead of hand-coding new models. The bundler
now layers three sources in priority order: LiteLLM (broad list prices),
hand-curated MANUAL_ENTRIES overrides, then a separate last-resort fallback
file gap-filled from models.dev first-party makers (official direct prices)
and OpenRouter (resale backstop). New models such as MiniMax-M3 ($0.6/$2.4)
now price correctly with no per-model code.

The fallback is written to its own pricing-fallback.json and consulted only
case-insensitively as the final step in getModelCosts, so a reseller variant
name can never shadow a canonical or aliased match.

Fixes surfaced while building and verifying this:
- Alias precedence: LiteLLM ships snowflake/claude-4-opus ($5), which the
  bundler strips to a bare claude-4-opus key that shadowed the curated alias
  to claude-opus-4 ($15 official). An explicit alias for a bare name now wins
  over a coincidental stripped reseller key; the prefixed gateway price is
  still returned for the fully-qualified id.
- Zero-stub guard: LiteLLM [0,0] price stubs (e.g. GigaChat-2-Max) are
  excluded from the case-insensitive index so a case-mismatched query stays
  null and keeps firing the unknown-model warning instead of silently
  reporting $0.
- Negative-sentinel guard: OpenRouter returns -1 for variable/BYOK-priced
  models. The bundler now rejects any non-positive rate pair (and strips the
  sentinel from cache fields) so a negative per-token cost can never ship and
  subtract from spend totals.

Bundler hardening: bareKey strips @pin and date suffixes to match the runtime
canonical form, seen-set dedupes on both full and bare key shapes, and it logs
MANUAL_ENTRIES now covered upstream plus models.dev allowlist drift. Extracted
buildCosts so the cache-cost heuristics live in one place. Added a data-hygiene
test that fails CI if a rebundle reintroduces negative, free, or unreachable
fallback entries.

...

This commit is contained in:

Resham Joshi 2026-06-09 21:17:23 +02:00 • committed by GitHub

parent c36f3afa76

commit a385f65dee

No known key found for this signature in database

GPG key ID: B5690EEEBB952194

7 changed files with 312 additions and 28 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

2

src/data/litellm-snapshot.json

View file

File diff suppressed because one or more lines are too long

1

src/data/pricing-fallback.json Normal file

View file

File diff suppressed because one or more lines are too long