codeburn/tests/durable-failed-source.test.ts
Resham Joshi 7eaf8967fc
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (22) (push) Waiting to run
Tests / test (22.15.0) (push) Waiting to run
Tests / test (24) (push) Waiting to run
Tests / test (26) (push) Waiting to run
Tests / test-platforms (macos-latest) (push) Waiting to run
Tests / test-platforms (windows-latest) (push) Waiting to run
Tests / desktop (push) Waiting to run
Tests / vscode (push) Waiting to run
fix(copilot): keep chat journals beside a trace database, count each chat once, price Auto-mode requests, make the JetBrains parse linear (#1901)
* fix(copilot): keep chat-session journals of sessions agent-traces.db never recorded

Any agent-traces.db on disk used to hide every VS Code chatSessions journal,
including sessions OTel never saw and their copilotCredits billing. Journals
are now always discovered, and only the journal of a session the trace
database holds is left out at serve time.

* fix(copilot): count a chat once when OTel and its VS Code transcript both record it

A transcript of a conversation agent-traces.db also holds became an extra
$0 estimated call beside every OTel call, inflating call and request counts
and marking the session partly estimated. The transcript of such a session is
now left out at serve time, which also clears transcripts already cached.

* fix(copilot): attribute JetBrains turns to conversations in one pass

Every turn scanned the whole store backwards once per conversation, so the
parse grew with conversations x turns x store size. A synthetic store of 80
conversations x 20 turns (3.7 MB) took 81.7 s and now takes 49 ms. Every GUID
occurrence is collected once and each turn takes the last one at or before
its offset, the same rule as before.

* fix(copilot): keep the cached agent-traces.db entry when a read fails

A corrupt, half-rewritten or otherwise unreadable agent-traces.db raised a
plain error, so the parse recorded a failed entry over the cached one. Once
the extension pruned old spans, that history was gone for good. Every open or
query failure now defers like a locked database, as the session-store parser
already does: the cached spans keep serving and the next refresh retries.

* fix(copilot): keep estimated thinking tokens in the served price of legacy chats

Legacy chat JSON without token counts estimates thinking separately. The
provider priced it as output plus reasoning, but serving re-prices Copilot
from outputTokens alone (reasoning is inside output there), so a request with
100k thinking tokens served at almost $0. The estimate now puts reasoning
inside outputTokens, so cold and cached reads agree. Copilot parse version
bumped so cached calls re-parse once.

* fix(copilot): price Auto-mode journal requests that name no resolved model

A chatSessions journal request in Auto mode with token counts but no
resolvedModel or copilotCredits served as model 'auto' at $0, flagged nowhere
as estimated. It now takes the same inference as the legacy chat JSON path
(tool-call id prefix, else the session's model) and is marked estimated
unless billing credits price it. Copilot parse version bumped so cached calls
re-parse once.

* fix(copilot): report uninferable Auto journal requests as unpriced and keep credit-priced labels

An Auto request with nothing to infer a model from still served as 'auto' at
$0, which no unpriced check flags. It now serves as 'unknown', not marked
estimated, so it reports as unpriced. A request copilotCredits already price
keeps its own label instead of an inferred one.

* fix(copilot): defer only locked agent-traces.db reads, keep cached turns on other failures

Treating every OTel read failure as busy made a permanently corrupt
agent-traces.db retry on every refresh and hold daily-cache finalisation
forever. Only SQLITE_BUSY/LOCKED defer again. Any other parse failure of a
durable provider's source now keeps the cached turns under the failed marker,
so the pruned history survives, the file is not retried until it changes,
finalisation proceeds and doctor lists the failure. A later good parse clears
the marker.

* fix(copilot): match OTel twins by VS Code storage as well as session id

The serve-time dedupe keyed on session id alone, so an OTel session in one
workspace dropped an unrelated journal or transcript with the same id in
another. OTel calls now record the storage of their session's newest journal
or transcript, and only that storage's copy is dropped. OTel calls cached
before this still match by id until they re-parse; the Copilot parse version
is bumped so live spans re-parse once.

* fix(copilot): drop OTel twins only when one VS Code storage holds the session

Picking the newest copy guessed which workspace an OTel session belonged to,
and spans carry nothing that names one. A journal or transcript is now dropped
only when its session id lives in exactly one storage; with copies in several,
every copy is kept, preferring a possible double count over dropping the wrong
workspace. The twin keys are collected in a full pass first, so file order
cannot change the result. The per-call chatStorage field and its parse-version
bump are removed again.

* fix(copilot): match OTel twins per request, not per session

A session OTel recorded had every journal and transcript call dropped,
including requests the trace never saw (before it was enabled, or after the
database was deleted while its cached spans kept the session classified as
traced). Spans carry no VS Code request id, so the match is by time within the
session: a journal request is dropped only when an OTel span starts between
its timestamp and the next request's, a transcript message only when one
starts since the previous message. Anything else is kept.

* fix(copilot): keep the lowest-path copy of a chat session found in two storages

A session VS Code copied into a second storage carries the same dedup keys,
and the copy parsed first kept them, so which workspace it served under
depended on directory listing order. Journals and transcripts are now parsed
in path order, so the copy under the lowest storage path wins on a fresh
parse. A copy already cached keeps its keys.

* fix(cache): retry a failed provider source once a day

A failed-parse marker kept the file's fingerprint, so a repair that left size
and mtime unchanged was never re-read. The marker now records when it failed,
and a provider source whose marker is a day old is parsed again even at the
same fingerprint. Markers written before this carry no stamp and retry once.

* test(cache): a failing durable source keeps its rows, shows in doctor, retries daily

The failed-marker change in this branch applies to every provider with
durableSources, today Copilot and Quick Desktop, not only Copilot: when a
source that already has cached turns fails to parse with an error other than
a locked database, its cached turns are kept under the failed marker instead
of being replaced by an empty one. Doctor lists the entry as failed, daily
finalisation is not held back, and the source is re-read when it changes or
once a day. This covers Quick Desktop end to end.

* fix(copilot): give journal requests stamped alike one shared OTel window

Two journal requests with the same timestamp sorted into a zero-width window
for the first, so a span at that stamp dropped only one of them. Calls stamped
alike now share one window bounded by the nearest distinct stamps.

On same-id twins across workspaces: the journal's session id is the VS Code
chat session UUID (file name and sessionId), and Copilot Chat sets
gen_ai.conversation.id, copilot_chat.session_id and copilot_chat.chat_session_id
on its chat spans from that same id. A pair sharing the id is the same session,
not an unrelated one; two storages holding it are a copy VS Code made, which
is why the multi-storage case keeps both copies instead of guessing.

* fix(copilot): attribute JetBrains turns to uppercase conversation GUIDs

Conversation records and the owner scan matched lowercase GUIDs only, so a
turn under an uppercase GUID fell back to the store id (as it already did
before the one-pass scan). Both now match either case. Java's
UUID.toString() writes lowercase, so no cached store is expected to move;
one written with uppercase ids would re-key its turns from the store id to
the GUID.

On the daily retry of failed sources (previous commits): it applies to every
provider parsed through parseProviderSources, that is every provider except
Claude's transcript scan, durable or not. That is intended: a failed marker
only caches a negative result, so one re-read per broken file per day is
cheap, and it is the only way to pick up a repair that kept size and mtime.

* fix(copilot): match JetBrains conversation GUIDs case-insensitively

Conversation ids were stored as written and looked up case-sensitively, so a
GUID declared in uppercase but referenced in lowercase before a turn handed the
turn to whichever conversation was declared last. Ids are now lowercased on
insert and lookup, which is also the form Java's UUID.toString() writes.

The OTel request window stays strict, with no backward tolerance. This machine
has no agent-traces.db, so no real skew could be measured. VS Code core stamps
a journal request with Date.now() when it creates the request model
(queuePendingRequest), before the extension handler that starts the chat
spans runs, on the same clock. So a request's spans start at or after its
stamp. A queued request is stamped while the previous one is still running,
so that request's late spans can start after the next stamp. A backward
tolerance would give those late spans to the next request.

* fix(copilot): re-parse cached JetBrains stores for the GUID-case fix

Bumps the Copilot parse version so an already-cached JetBrains store is
re-read under case-insensitive conversation matching. Stores with lowercase
GUIDs (Java's UUID.toString() form) attribute exactly as before, so their
dedup keys and totals do not move. A store using uppercase or mixed-case ids
gets new conversation ids in its keys, and the durable union keeps the old
copies beside them, so such a store would count those turns twice.

* fix(copilot): let a re-attributed JetBrains reply replace its cached copy

A parse-version bump does not evict anything for Copilot: the section is
carried forward with a forced re-read (getOrCreateProviderSection), and the
durable merge replaces calls whose key the re-read produces, appends new keys
and keeps every other cached key. A JetBrains reply that the GUID-case fix
files under a different conversation gets a new key, so its old copy stayed
beside it and counted twice. JetBrains keys end in the reply's content hash.
The merge now drops a cached JetBrains call whose hash the re-read produced
under another key. Replies the store no longer holds produce no fresh hash
and stay. No other key changes in this branch: the estimated-reasoning and
Auto-model fixes keep their keys and are replaced in place.

* fix(copilot): drop a cached JetBrains reply only when the re-read restamps the same turn

Matching on the reply hash alone let a live conversation's "OK" remove a
pruned conversation's cached "OK". A cached JetBrains call is now dropped only
when the re-read produced the same reply hash, the same :n index and the same
timestamp under another key. JetBrains calls carry the store's mtime, so this
removes the old copy when a store is re-attributed without changing (the
parse-version bump), and keeps a same-text reply cached at another stamp. A
store that changed between its last parse and the re-read keeps both copies
of a re-attributed reply.

* test(copilot): fix the time base once so stamped-alike requests share a stamp

at() read Date.now() on every call, so two requests meant to share a stamp
could land a millisecond apart and fall into different windows.
2026-11-06 18:02:02 +01:00

72 lines
3.3 KiB
TypeScript

// A durable provider's source that fails to parse keeps the turns already
// cached for it (they may be the only record left), is listed by doctor, does
// not hold back daily finalisation, and is retried once a day even when a
// repair left its fingerprint unchanged. Quick Desktop stands in for any
// durable provider other than Copilot.
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
import { appendFile, mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { clearLoadCacheMemo } from '../src/session-cache.js'
import { clearSessionCache, isSessionHydrationComplete, parseAllSessions } from '../src/parser.js'
import { collectDoctorReport } from '../src/doctor.js'
import { quickdesk } from '../src/providers/quickdesk.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
let tmp: string
let metrics: string
beforeEach(async () => {
tmp = await mkdtemp(join(tmpdir(), 'durable-failed-'))
vi.stubEnv('HOME', tmp)
vi.stubEnv('CODEBURN_CACHE_DIR', join(tmp, 'cache'))
vi.stubEnv('QUICKWORK_HOME', join(tmp, 'quickwork'))
await mkdir(join(tmp, 'quickwork', 'metrics'), { recursive: true })
const day = new Date(Date.now() - 2 * 86400000).toISOString().slice(0, 10)
metrics = join(tmp, 'quickwork', 'metrics', `metrics-${day}.jsonl`)
await writeFile(metrics, JSON.stringify({ Model: 'claude-sonnet-4-5', InputTokens: 12, OutputTokens: 6, CostUSD: 0.002 }) + '\n')
})
afterEach(async () => {
vi.restoreAllMocks()
vi.unstubAllEnvs()
clearSessionCache()
clearLoadCacheMemo()
await rm(tmp, { recursive: true, force: true })
})
async function servedInputs(): Promise<number[]> {
clearSessionCache()
clearLoadCacheMemo()
const projects = await parseAllSessions(undefined, 'quickdesk')
return projects.flatMap(p => p.sessions).flatMap(s => s.turns).flatMap(t => t.assistantCalls).map(c => c.usage.inputTokens).sort()
}
const quickdeskEntry = async () => Object.values((await readCacheOnDisk()).providers['quickdesk']!.files)[0]!
it('keeps the cached rows of a failing durable source, lists it in doctor and retries it a day later', async () => {
expect(await servedInputs()).toEqual([12])
const parser = vi.spyOn(quickdesk, 'createSessionParser').mockImplementation(() => ({
async *parse() { throw new Error('unreadable') },
}))
await appendFile(metrics, JSON.stringify({ Model: 'claude-sonnet-4-5', InputTokens: 20, OutputTokens: 6, CostUSD: 0.003 }) + '\n')
expect(await servedInputs()).toEqual([12])
expect(isSessionHydrationComplete()).toBe(true)
expect((await quickdeskEntry()).failed).toBe(true)
const report = await collectDoctorReport('quickdesk', { cache: await readCacheOnDisk(), dailyCacheDays: [], launchers: [] })
expect(report.cacheHealth?.sessionCacheIssues).toEqual([{ provider: 'quickdesk', path: metrics, reason: 'failed' }])
// Repaired without touching the file: not retried within the day...
parser.mockRestore()
expect(await servedInputs()).toEqual([12])
// ...but retried once the marker is a day old.
const cache = await readCacheOnDisk()
cache.providers['quickdesk']!.files[metrics]!.failedAt = new Date(Date.now() - 2 * 86400000).toISOString()
await writeCacheOnDisk(cache)
expect(await servedInputs()).toEqual([12, 20])
expect((await quickdeskEntry()).failed).toBeUndefined()
})