mirror of
https://github.com/AgentSeal/codeburn.git
synced 2026-07-11 18:19:24 +00:00
a385f65dee
Keep model pricing automatic instead of hand-coding new models. The bundler now layers three sources in priority order: LiteLLM (broad list prices), hand-curated MANUAL_ENTRIES overrides, then a separate last-resort fallback file gap-filled from models.dev first-party makers (official direct prices) and OpenRouter (resale backstop). New models such as MiniMax-M3 ($0.6/$2.4) now price correctly with no per-model code. The fallback is written to its own pricing-fallback.json and consulted only case-insensitively as the final step in getModelCosts, so a reseller variant name can never shadow a canonical or aliased match. Fixes surfaced while building and verifying this: - Alias precedence: LiteLLM ships snowflake/claude-4-opus ($5), which the bundler strips to a bare claude-4-opus key that shadowed the curated alias to claude-opus-4 ($15 official). An explicit alias for a bare name now wins over a coincidental stripped reseller key; the prefixed gateway price is still returned for the fully-qualified id. - Zero-stub guard: LiteLLM [0,0] price stubs (e.g. GigaChat-2-Max) are excluded from the case-insensitive index so a case-mismatched query stays null and keeps firing the unknown-model warning instead of silently reporting $0. - Negative-sentinel guard: OpenRouter returns -1 for variable/BYOK-priced models. The bundler now rejects any non-positive rate pair (and strips the sentinel from cache fields) so a negative per-token cost can never ship and subtract from spend totals. Bundler hardening: bareKey strips @pin and date suffixes to match the runtime canonical form, seen-set dedupes on both full and bare key shapes, and it logs MANUAL_ENTRIES now covered upstream plus models.dev allowlist drift. Extracted buildCosts so the cache-cost heuristics live in one place. Added a data-hygiene test that fails CI if a rebundle reintroduces negative, free, or unreachable fallback entries.
35 lines
1.4 KiB
TypeScript
35 lines
1.4 KiB
TypeScript
import { describe, it, expect } from 'vitest'
|
|
|
|
import fallback from '../src/data/pricing-fallback.json' assert { type: 'json' }
|
|
|
|
// The gap-fill fallback is generated from models.dev / OpenRouter. These assert
|
|
// the bundler's hygiene guarantees on the committed artifact, so a future
|
|
// rebundle that regresses them fails CI rather than shipping bad pricing.
|
|
describe('pricing-fallback.json data hygiene', () => {
|
|
const entries = Object.entries(fallback as Record<string, (number | null)[]>)
|
|
|
|
it('is non-empty', () => {
|
|
expect(entries.length).toBeGreaterThan(50)
|
|
})
|
|
|
|
it('has no negative rates (OpenRouter -1 "variable price" sentinels)', () => {
|
|
const bad = entries.filter(([, v]) => (v[0] ?? 0) < 0 || (v[1] ?? 0) < 0 || (v[2] ?? 0) < 0 || (v[3] ?? 0) < 0)
|
|
expect(bad.map(([k]) => k)).toEqual([])
|
|
})
|
|
|
|
it('has no entry that is free on both input and output', () => {
|
|
const bad = entries.filter(([, v]) => v[0] === 0 && v[1] === 0)
|
|
expect(bad.map(([k]) => k)).toEqual([])
|
|
})
|
|
|
|
it('has no unreachable @pin or date-suffixed keys', () => {
|
|
const bad = entries.filter(([k]) => /@/.test(k) || /\d{8}$/.test(k))
|
|
expect(bad.map(([k]) => k)).toEqual([])
|
|
})
|
|
|
|
it('stores per-token rates (no per-million values leaked through)', () => {
|
|
// A per-million value would be >= 1; real per-token rates are tiny.
|
|
const bad = entries.filter(([, v]) => (v[0] ?? 0) >= 1 || (v[1] ?? 0) >= 1)
|
|
expect(bad.map(([k]) => k)).toEqual([])
|
|
})
|
|
})
|