codeburn/tests/daily-cache.test.ts
Resham Joshi 7f2739e130
daily-cache: never lose history the sources can no longer re-derive (v14) (#755)
* daily-cache: never lose history the sources can no longer re-derive (v14)

Session files are ephemeral (Claude Code deletes transcripts after ~30
days), so a cached day whose sources are gone exists nowhere else. Every
invalidation path (schema bump, savings-config change, timezone change,
incomplete-hydration retry) used to discard all cached days and re-derive
from surviving sources, silently truncating history to the source
retention window. Five bumps between June 22 and July 16 erased
everything before April 24 on a machine with usage since March.

Invalidations now re-derive what they can and carry forward every
(day, provider) slice they cannot, marked 'carried'. Loading a missing or
corrupt cache file adopts days from every older daily-cache file in the
cache dir (legacy, versioned, .bak copies, orphaned .tmp) as a union per
(day, provider), higher schema version winning per pair. Provider slices
now store the full per-provider breakdown (tokens, models, categories,
sessions) so carry-forwards stay exact across future rebuilds.

Merge rules hardened by adversarial review: opaque pre-v5 days (totals
without provider slices) merge all-or-nothing so partial parses cannot
double-count into them; zero-data placeholder slices neither block nor
lose carried data (session counts deduplicated by max); a partial parse
never overwrites finalized baseline slices, only fills gaps; adoption
purges today/future entries, applies retention, and clamps a stale
lastComputedDate so a purged day cannot be skipped forever.

Verified end-to-end against a copy of a real cache dir: the rebuilt v14
cache holds every (day, provider) pair present in any older cache file,
including Claude days from early April that the July rebuilds had
dropped.

* daily-cache: per-project daily rollups in the durable record (v15)

Project history previously lived only in the session layer, so it faded
with the source files even though day totals now survive. Days and
provider slices carry a projects breakdown (cost/calls/savings/sessions
per project) filled by the day aggregator and folded through carry
merges, making the By Project dimension as durable as models and
categories. Days recorded before v15 keep their totals with no project
split; nothing can reconstruct one once sources are gone.

This is the first bump to ride the v14 carry-forward: the v14 cache is
adopted losslessly and only source-backed days re-derive (verified on a
real cache dir: identical totals, zero lost pairs, project splits on
every derivable day).

Hardening from adversarial review: placeholder-aware project session
dedup so totals reconcile; migrateDays sanitizes provider slices and
nested projects from foreign caches; all foreign-keyed map access in the
merge path uses hasOwn reads and defineProperty writes, closing a real
prototype-pollution path a regression test caught when a cache key is
named __proto__.

* menubar: serve headline totals and projects from the durable day set

Review finding on this PR: the all-provider headline built cache-backed
totals and then replaced them wholesale with a rebuild from the
surviving-session parse, so current.cost/calls, the models table, and
topProjects stayed truncated to the source-retention window even though
history.daily carried the full record. The replacement existed only to
keep the estimated-cost markers alive.

The cache-backed period data is now the authority; the scan contributes
exactly what day entries lack: estimated-cost markers, unpriced-model
detection, per-session drill-downs, and a fresher project path. Project
totals come from the same day set as the headline, with ProjectDayStats
gaining a path so carried-only projects still display a friendly name.
Sessions merge by max: the cache buckets a session on its start day, the
scan counts it on any active day, and both undercount differently.

End-to-end regression test seeds a cache whose only day is carried (no
session files exist) and asserts the headline, models, and topProjects
all reflect it. Verified on a real cache dir: the 6-month headline now
equals the history sum to the cent.

* daily-cache: close residual corrupt-input gaps in v15 ingestion sanitization

v15 sanitizes provider slices and projects at ingestion and guards merge
lookups with Object.hasOwn/setOwn. Three residual gaps versus the
invariant (no JSON-parseable cache content may throw or produce
NaN/garbage), plus one found in review:

- Day-level models/categories were not sanitized: a day with
  models:'bad' survived load and buildPeriodDataFromDays iterated the
  string per-character, yielding NaN in period model totals. Day-level
  maps now reuse the same sanitizers as slice-level maps.
- Map keys shadowing Object.prototype (constructor, toString, ...) still
  entered the cache from foreign files; modelTotals[name] ?? init in
  buildPeriodDataFromDays resolves such a key to the inherited prototype
  member and produces NaN. All sanitized maps now drop these keys at
  ingestion (deliberate tradeoff: such names are reclassified invalid).
- Top-level lastComputedDate was unvalidated; a non-string later hit
  .slice() in the gap-start parse and threw. Kept only when it is a
  YYYY-MM-DD string, else null (forces a plain re-backfill).
- Stale savingsConfigHash comment still described pre-v14 discard
  behavior; reworded to re-derive + carry forward.

3 regression tests, each verified to fail on the pre-fix code. Relevant
suites 71/71 green, tsc clean. Valid-input behavior unchanged except the
prototype-key reclassification above.

---------

Co-authored-by: ozymandiashh <234437643+ozymandiashh@users.noreply.github.com>
2026-07-20 05:21:22 -07:00

468 lines
18 KiB
TypeScript

import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { readFile, rm } from 'fs/promises'
import { existsSync } from 'fs'
import { tmpdir } from 'os'
import { join } from 'path'
import type { ProjectSummary } from '../src/types.js'
import {
addNewDays,
currentTzKey,
dailyCachePath,
DAILY_CACHE_VERSION,
type DailyCache,
type DailyEntry,
getDaysInRange,
ensureCacheHydrated,
loadDailyCache,
saveDailyCache,
withDailyCacheLock,
} from '../src/daily-cache.js'
function emptyDay(date: string, cost = 0, calls = 0): DailyEntry {
return {
date,
cost,
savingsUSD: 0,
calls,
sessions: 0,
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
editTurns: 0,
oneShotTurns: 0,
models: {},
categories: {},
providers: {},
}
}
const TMP_CACHE_ROOT = join(tmpdir(), `codeburn-cache-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`)
beforeEach(() => {
process.env['CODEBURN_CACHE_DIR'] = TMP_CACHE_ROOT
})
afterEach(async () => {
vi.useRealTimers()
if (existsSync(TMP_CACHE_ROOT)) {
await rm(TMP_CACHE_ROOT, { recursive: true, force: true })
}
})
describe('loadDailyCache', () => {
it('returns an empty cache when the file does not exist', async () => {
const cache = await loadDailyCache()
expect(cache.version).toBe(DAILY_CACHE_VERSION)
expect(cache.lastComputedDate).toBeNull()
expect(cache.days).toEqual([])
})
it('returns an empty cache when the file contains invalid JSON', async () => {
const { writeFile, mkdir } = await import('fs/promises')
await mkdir(TMP_CACHE_ROOT, { recursive: true })
await writeFile(join(TMP_CACHE_ROOT, 'daily-cache.json'), 'not valid json{{', 'utf-8')
const cache = await loadDailyCache()
expect(cache.days).toEqual([])
})
// With carry-forward (v14), a legacy unversioned file whose version is not
// the current one is ADOPTED as a carried baseline — its days survive into
// the new cache, marked `carried` and pending re-derivation. The legacy file
// itself is never rewritten, backed up, or deleted (old binaries still own it).
it('adopts a legacy file too old to trust as a carried baseline, without rewriting it', async () => {
const saved = {
version: 1,
lastComputedDate: '2026-04-10',
days: [{ date: '2026-04-10', cost: 10, calls: 5 }],
}
const { writeFile, mkdir } = await import('fs/promises')
await mkdir(TMP_CACHE_ROOT, { recursive: true })
const legacy = join(TMP_CACHE_ROOT, 'daily-cache.json')
await writeFile(legacy, JSON.stringify(saved), 'utf-8')
const cache = await loadDailyCache()
expect(cache.days).toHaveLength(1)
expect(cache.days[0]).toMatchObject({ date: '2026-04-10', cost: 10, calls: 5, carried: true })
// Adopted days are not yet finalized under current accounting.
expect(cache.complete).not.toBe(true)
// Legacy file untouched (no .bak, contents intact); versioned file persisted.
expect(existsSync(join(TMP_CACHE_ROOT, 'daily-cache.json.v1.bak'))).toBe(false)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(saved)
expect(existsSync(dailyCachePath())).toBe(true)
})
it('adopts a legacy v2 cache as carried days and leaves the file intact', async () => {
const saved = {
version: 2,
lastComputedDate: '2026-04-10',
days: [{
date: '2026-04-10', cost: 10, calls: 5, sessions: 2,
inputTokens: 1000, outputTokens: 500, cacheReadTokens: 200, cacheWriteTokens: 100,
models: { 'claude-opus-4-6': { calls: 5, cost: 10, inputTokens: 1000, outputTokens: 500, cacheReadTokens: 200, cacheWriteTokens: 100 } },
}],
}
const { writeFile, mkdir } = await import('fs/promises')
await mkdir(TMP_CACHE_ROOT, { recursive: true })
const legacy = join(TMP_CACHE_ROOT, 'daily-cache.json')
await writeFile(legacy, JSON.stringify(saved), 'utf-8')
const cache = await loadDailyCache()
expect(cache.version).toBe(DAILY_CACHE_VERSION)
expect(cache.days).toHaveLength(1)
expect(cache.days[0]).toMatchObject({ date: '2026-04-10', cost: 10, calls: 5, sessions: 2, carried: true })
expect(cache.days[0]!.models['claude-opus-4-6']!.cost).toBe(10)
expect(existsSync(join(TMP_CACHE_ROOT, 'daily-cache.json.v2.bak'))).toBe(false)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(saved)
})
it('adopts a legacy v5 cache including its provider slices', async () => {
const saved = {
version: 5,
lastComputedDate: '2026-05-01',
days: [{
date: '2026-05-01',
cost: 0.37575,
calls: 1,
sessions: 1,
inputTokens: 0,
outputTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 60_120,
editTurns: 0,
oneShotTurns: 0,
models: { 'Opus 4.7': { calls: 1, cost: 0.37575, inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 60_120 } },
categories: {},
providers: { claude: { calls: 1, cost: 0.37575 } },
}],
}
const { writeFile, mkdir } = await import('fs/promises')
await mkdir(TMP_CACHE_ROOT, { recursive: true })
const legacy = join(TMP_CACHE_ROOT, 'daily-cache.json')
await writeFile(legacy, JSON.stringify(saved), 'utf-8')
const cache = await loadDailyCache()
expect(cache.version).toBe(DAILY_CACHE_VERSION)
expect(cache.days).toHaveLength(1)
expect(cache.days[0]).toMatchObject({ date: '2026-05-01', cost: 0.37575, calls: 1, carried: true })
expect(cache.days[0]!.providers['claude']).toMatchObject({ calls: 1, cost: 0.37575 })
expect(existsSync(join(TMP_CACHE_ROOT, 'daily-cache.json.v5.bak'))).toBe(false)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(saved)
})
it('adopts a legacy file whose version matches the current one, once, without deleting it', async () => {
const saved = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: 'legacy-hash',
lastComputedDate: '2026-05-01',
days: [emptyDay('2026-05-01', 3.5, 9)],
}
const { writeFile, mkdir } = await import('fs/promises')
await mkdir(TMP_CACHE_ROOT, { recursive: true })
const legacy = join(TMP_CACHE_ROOT, 'daily-cache.json')
await writeFile(legacy, JSON.stringify(saved), 'utf-8')
// First load: versioned file absent → adopt-copy from legacy.
const first = await loadDailyCache()
expect(first.days).toEqual(saved.days)
expect(first.savingsConfigHash).toBe('legacy-hash')
expect(existsSync(dailyCachePath())).toBe(true)
// Legacy file is NOT deleted.
expect(existsSync(legacy)).toBe(true)
// Adoption is one-time: mutate the legacy file, load again — the versioned
// file now wins and the stale legacy edit is never re-adopted.
await writeFile(legacy, JSON.stringify({ ...saved, days: [emptyDay('2000-01-01', 999)] }), 'utf-8')
const second = await loadDailyCache()
expect(second.days).toEqual(saved.days)
})
it('round-trips a valid cache through save and load', async () => {
const saved: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: 'cfg-hash-1',
lastComputedDate: '2026-04-10',
days: [emptyDay('2026-04-09', 12.5, 40), emptyDay('2026-04-10', 7.25, 28)],
complete: true,
}
await saveDailyCache(saved)
const loaded = await loadDailyCache()
expect(loaded).toEqual(saved)
})
})
describe('saveDailyCache', () => {
it('writes atomically so no temp file is left after a successful save', async () => {
const saved: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: 'cfg-hash-1',
lastComputedDate: '2026-04-10',
days: [emptyDay('2026-04-10', 5)],
}
await saveDailyCache(saved)
const { readdir } = await import('fs/promises')
const files = await readdir(TMP_CACHE_ROOT)
const tempLeftovers = files.filter(f => f.endsWith('.tmp'))
expect(tempLeftovers).toEqual([])
const finalFile = await readFile(dailyCachePath(), 'utf-8')
expect(JSON.parse(finalFile)).toEqual(saved)
})
})
describe('addNewDays', () => {
it('returns a new cache with the added days sorted ascending by date', () => {
const base: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: '2026-04-08',
days: [emptyDay('2026-04-07', 3), emptyDay('2026-04-08', 5)],
}
const updated = addNewDays(base, [emptyDay('2026-04-10', 9), emptyDay('2026-04-09', 7)], '2026-04-10')
expect(updated.days.map(d => d.date)).toEqual(['2026-04-07', '2026-04-08', '2026-04-09', '2026-04-10'])
expect(updated.lastComputedDate).toBe('2026-04-10')
})
it('replaces existing days with incoming data (last write wins)', () => {
const base: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: '2026-04-08',
days: [emptyDay('2026-04-08', 5)],
}
const updated = addNewDays(base, [emptyDay('2026-04-08', 99)], '2026-04-08')
const aprilEight = updated.days.find(d => d.date === '2026-04-08')!
expect(aprilEight.cost).toBe(99)
})
it('does not regress lastComputedDate if incoming newestDate is older', () => {
const base: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: '2026-04-10',
days: [emptyDay('2026-04-10', 5)],
}
const updated = addNewDays(base, [emptyDay('2026-04-05', 3)], '2026-04-05')
expect(updated.lastComputedDate).toBe('2026-04-10')
})
it('skips prune when newestDate is malformed (does not silently drop all days)', () => {
// Regression guard: a corrupt newestDate string used to produce a NaN
// cutoff, which made `d.date >= "Invalid Date"` always false and
// wiped every cached day on the next merge. The guard now leaves
// the entries untouched so the next valid run can prune normally.
const base: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: '2026-04-10',
days: [emptyDay('2026-04-08', 1), emptyDay('2026-04-09', 2), emptyDay('2026-04-10', 3)],
}
const updated = addNewDays(base, [], 'not-a-date')
expect(updated.days.map(d => d.date)).toEqual(['2026-04-08', '2026-04-09', '2026-04-10'])
})
it('still prunes when newestDate is valid', () => {
const old = '2020-01-01'
const recent = '2026-04-10'
const base: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: recent,
days: [emptyDay(old, 1), emptyDay(recent, 2)],
}
const updated = addNewDays(base, [], recent)
// 730-day retention from 2026-04-10 → cutoff ~2024-04-11; 2020-01-01 must be gone.
expect(updated.days.find(d => d.date === old)).toBeUndefined()
expect(updated.days.find(d => d.date === recent)).toBeDefined()
})
})
describe('getDaysInRange', () => {
const cache: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: '2026-04-10',
days: [
emptyDay('2026-04-05', 1),
emptyDay('2026-04-06', 2),
emptyDay('2026-04-07', 3),
emptyDay('2026-04-08', 4),
emptyDay('2026-04-09', 5),
emptyDay('2026-04-10', 6),
],
}
it('returns inclusive start and end range', () => {
const days = getDaysInRange(cache, '2026-04-07', '2026-04-09')
expect(days.map(d => d.date)).toEqual(['2026-04-07', '2026-04-08', '2026-04-09'])
})
it('returns empty when range is entirely outside cache', () => {
expect(getDaysInRange(cache, '2026-03-01', '2026-03-10')).toEqual([])
expect(getDaysInRange(cache, '2026-05-01', '2026-05-10')).toEqual([])
})
it('clips to available cache days when range extends beyond', () => {
const days = getDaysInRange(cache, '2026-04-09', '2026-04-20')
expect(days.map(d => d.date)).toEqual(['2026-04-09', '2026-04-10'])
})
})
describe('ensureCacheHydrated', () => {
it('does not recompute yesterday after it has already been cached', async () => {
vi.useFakeTimers()
vi.setSystemTime(new Date('2026-06-12T12:00:00.000Z'))
const saved: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
tzKey: currentTzKey(),
lastComputedDate: '2026-06-11',
days: [emptyDay('2026-06-11', 5, 10)],
complete: true,
}
await saveDailyCache(saved)
let parseCalls = 0
const hydrated = await ensureCacheHydrated(
async () => {
parseCalls += 1
return []
},
() => [],
)
expect(parseCalls).toBe(0)
expect(hydrated).toEqual(saved)
})
it('drops a cached today/future entry so it is recomputed live, keeping yesterday cached', async () => {
vi.useFakeTimers()
vi.setSystemTime(new Date('2026-06-12T12:00:00.000Z'))
// A "today" entry can only exist via a backward clock change or a stale
// cache; it must be purged so today is served live, not from a frozen entry.
const saved: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
lastComputedDate: '2026-06-12',
days: [emptyDay('2026-06-11', 5, 10), emptyDay('2026-06-12', 9, 20)],
complete: true,
}
await saveDailyCache(saved)
let parseCalls = 0
const hydrated = await ensureCacheHydrated(
async () => {
parseCalls += 1
return []
},
() => [],
)
expect(parseCalls).toBe(0)
expect(hydrated.days.map(d => d.date)).toEqual(['2026-06-11'])
expect(hydrated.lastComputedDate).toBe('2026-06-11')
})
})
describe('withDailyCacheLock', () => {
it('serializes concurrent operations', async () => {
const sequence: string[] = []
const op = async (tag: string): Promise<void> => {
await withDailyCacheLock(async () => {
sequence.push(`start-${tag}`)
await new Promise(r => setTimeout(r, 20))
sequence.push(`end-${tag}`)
})
}
await Promise.all([op('a'), op('b'), op('c')])
for (let i = 0; i < sequence.length; i += 2) {
expect(sequence[i]?.startsWith('start-')).toBe(true)
expect(sequence[i + 1]?.startsWith('end-')).toBe(true)
expect(sequence[i]!.slice(6)).toBe(sequence[i + 1]!.slice(4))
}
})
})
describe('ensureCacheHydrated: savings config invalidation', () => {
it('re-derives on savingsConfigHash change but CARRIES days the parse cannot re-derive', async () => {
// Seed a cache with a day OLDER than yesterday so the hydration window
// (which keeps `d.date < yesterdayStr`) actually retains it.
const twoDaysAgo = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000)
const twoDaysAgoStr = `${twoDaysAgo.getFullYear()}-${String(twoDaysAgo.getMonth() + 1).padStart(2, '0')}-${String(twoDaysAgo.getDate()).padStart(2, '0')}`
const seeded: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: 'cfg-A',
lastComputedDate: twoDaysAgoStr,
days: [emptyDay(twoDaysAgoStr, 1.5, 3)],
complete: true,
}
await saveDailyCache(seeded)
// The re-derive parse finds NOTHING (session files already deleted). The
// day must survive as carried — this exact path used to wipe it.
const parseSessions = async (): Promise<ProjectSummary[]> => []
const aggregateDays = (): DailyEntry[] => []
const rehydrated = await ensureCacheHydrated(parseSessions, aggregateDays, 'cfg-B')
expect(rehydrated.savingsConfigHash).toBe('cfg-B')
expect(rehydrated.days).toHaveLength(1)
expect(rehydrated.days[0]).toMatchObject({ date: twoDaysAgoStr, cost: 1.5, calls: 3, carried: true })
expect(rehydrated.complete).toBe(true)
// Same hash → cached days survive untouched (no carried marker).
const seeded2: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: 'cfg-C',
lastComputedDate: twoDaysAgoStr,
days: [emptyDay(twoDaysAgoStr, 1.5, 3)],
complete: true,
}
await saveDailyCache(seeded2)
const preserved = await ensureCacheHydrated(parseSessions, aggregateDays, 'cfg-C')
expect(preserved.days).toHaveLength(1)
expect(preserved.days[0]!.date).toBe(twoDaysAgoStr)
expect(preserved.days[0]!.carried).toBeUndefined()
})
})
describe('ensureCacheHydrated: timezone invalidation', () => {
const twoDaysAgo = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000)
const twoDaysAgoStr = `${twoDaysAgo.getFullYear()}-${String(twoDaysAgo.getMonth() + 1).padStart(2, '0')}-${String(twoDaysAgo.getDate()).padStart(2, '0')}`
const parseSessions = async (): Promise<ProjectSummary[]> => []
const aggregateDays = (): DailyEntry[] => []
it('re-derives on timezone change but keeps days whose sources are gone', async () => {
// Days are bucketed by local midnight, so a cache tagged under a different
// timezone re-derives everything. Days that can no longer be re-derived stay
// (old-tz bucketing beats a silent zero). 'Test/OtherZone' can never equal a
// real IANA zone.
const seeded: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
tzKey: 'Test/OtherZone',
lastComputedDate: twoDaysAgoStr,
days: [emptyDay(twoDaysAgoStr, 1.5, 3)],
complete: true,
}
await saveDailyCache(seeded)
const rehydrated = await ensureCacheHydrated(parseSessions, aggregateDays, '')
expect(rehydrated.tzKey).toBe(currentTzKey())
expect(rehydrated.days).toHaveLength(1)
expect(rehydrated.days[0]).toMatchObject({ date: twoDaysAgoStr, cost: 1.5, carried: true })
})
it('keeps cached days when the tzKey matches the current timezone', async () => {
const seeded: DailyCache = {
version: DAILY_CACHE_VERSION,
savingsConfigHash: '',
tzKey: currentTzKey(),
lastComputedDate: twoDaysAgoStr,
days: [emptyDay(twoDaysAgoStr, 1.5, 3)],
complete: true,
}
await saveDailyCache(seeded)
const preserved = await ensureCacheHydrated(parseSessions, aggregateDays, '')
expect(preserved.days).toHaveLength(1)
expect(preserved.days[0]!.date).toBe(twoDaysAgoStr)
})
})