Bridged via a tagged envelope record across the four parse paths
(jsonl / chatsession / jetbrains / otel): readRecords shapes one envelope
per source kind and a single decodeCopilot dispatches on envelope.kind.
The sqlite driver, the three OTel queries, blob reads, git-root probing,
extractBashCommands / extractOtelBashCommands, and the sanctioned
shutdown-site calculateCost residual all stay host-side. The residual
keeps its exact argument order (model, input, 0, cacheWrite, cacheRead, 0),
emits costUSD + costIsEstimated: false with no costBasis key, and carries
its explanatory comment unchanged.
Validator fixes over the draft:
- collectJetBrainsRepoDirCandidates used a [\|"] character class instead
of the original's (?:\\|") alternation, so a file:// path containing a
pipe was truncated, its dir never resolved, and the JetBrains project
fell back to the generic bucket. Restored the alternation.
- the otel arm dropped the subagentTypes key when undefined; the original
always emits it. Restored so key presence matches byte for byte.
- otel chat-span attributes were looked up with a linear Array.find per
chat span; replaced with a per-span-id map built in the same pass.
Verified by a differential harness running the pre-migration provider
against the migrated one over 27 scenarios (all eight goldens plus
edge cases), comparing calls, seenKeys, and key presence. The harness is
not committed; the scenarios it proved are now goldens G9-G17.
Move the pure record decode for five sqlite-backed providers into
@codeburn/core, keeping discovery, sqlite I/O, pricing, and bash
base-name extraction CLI-side behind the dual-registry bridge.
sqlite variant: the host opens the database, runs the SQL, textualizes
BLOBs (warp stylized_command, quickdesk content/tool_names), probes
sqlite_master table variants (quickdesk), and re-throws SQLITE_BUSY so a
transient lock on a live database retries instead of being cached as an
empty result (hermes). The core decoders receive already-normalized rows.
Validator fixes over the drafted migration (original behavior is the
authority in every case):
- warp: restored the `run_command` tool-name check as an identity
comparison. The draft had rewritten it as a bare Record lookup, so
'constructor'/'toString'/'__proto__' resolved to inherited Object
members instead of passing through. Added a regression test.
- warp: restored `safeNumber` to the host's semantics (positive finite
numbers only). The draft's rewrite accepted numeric strings and
negative values, changing token budgets and dominant-model selection
for untrusted conversation_data.
- warp: `estimateTokensFromChars` now mirrors the host helper exactly.
- devin: model DISPLAY names move back CLI-side. The draft shipped a
7-line regex standing in for the host's `getShortModelName`, which
mis-rendered every non-Claude model id. The decoder now emits the raw
generation_model / model_name ids and the CLI formats them.
- devin: dropped the added `costBasis: 'measured'`; the pre-migration
call carried `costUSD` with no basis marker. Golden corrected.
- quickdesk: fixed an inverted source-variant predicate that routed
`sourceId: 'metrics'` on a sessions.db path (and `sourceId:
'sessions-db'` on a metrics path) to the wrong parser.
- cursor-agent: the "unrecognized transcript format" warning is now
re-emitted from the real decode result via a decode wrapper. The draft
re-derived `recognized` host-side as `hasUser && hasAssistant`, which
silently dropped the warning whenever a jsonl transcript had both roles
but produced no turns.
- cursor-agent: restored the per-conversation summary cache, so the
attribution database is opened once per conversation id, not per source.
- cursor-agent/devin: the session id is derived once, host-side, and
passed through, removing a second derivation that could disagree with
the host's on Windows-style paths.
- hermes: `browser_vision` maps to Vision, matching the original table.
Goldens re-verified by running each bridge fixture through the
pre-migration provider implementation: all five reproduce field-for-field.
Category B (sqlite) variant of the bridge migration: the sqlite driver and every
SQL query stay CLI-side. Each provider's `readRecords` opens the database, runs
the same queries as before, and hands the resulting rows (blob and all) to a
pure core decoder; `toProviderCall` maps the rich, cost-free decode back onto
ParsedProviderCall, where cost re-enters via the parser.ts pricing pass.
Per provider:
- crush: session row + dominant-model query -> one combined record. Crush stores
cost in dollars, so a row with cost > 0 carries `measuredCostUSD` (costBasis
'measured'); a zero-cost row falls back to token estimation, arm order intact.
- zcode: model_usage + tool_usage row sets -> one composite record. Each turn's
tools still attach to the first non-skipped usage row of that turn only.
- zed: threads rows handed over compressed; zstd decompression, JSON parsing and
per-request/cumulative-remainder accounting are pure. The Node >= 22.15 zstd
capability check stays host-side.
- forge: conversation row handed over with `context` still serialized; JSON
parsing and per-message decode are pure. Bash base-name extraction (and its
strip-ansi dependency) stays CLI-side over the decoder's raw command strings.
- goose: session + assistant tool-message + first-user-message rows, BLOB
columns pre-converted to text host-side, bundled into one composite record.
Validator fixes (original behavior is the authority):
- forge: the draft replaced the pre-migration `mapToolName` switch with an
object-literal lookup. Tool names come straight from conversation JSON, so
names colliding with Object.prototype members ("constructor", "toString",
"__proto__", "hasOwnProperty") resolved to inherited Functions / the prototype
object and were pushed into `tools` as non-strings instead of falling through
to the identity default. Restored the switch and pinned the arm in the fixture.
- zed: the draft routed the "skipped N unreadable Zed threads" notice into
record diagnostics, which the bridge discards, silently dropping a warning the
pre-migration decode printed. Re-emitted host-side from the diagnostics count
and pinned with a stderr assertion.
- Fixture coverage extended for the arms that were regression-blind: forge's
prototype-named tool calls, zed's aggregate stderr line, and goose's
single-turn `toolSequence` omission plus the unparseable-timestamp fallback.
Parity was verified independently of the bridge tests with a git-show harness
that runs the same fixtures through the pre-migration provider files and asserts
field-for-field equality, including the extra arms above.
The seeded 'today' session used a fixed noon timestamp; under the suite's
TZ=UTC pin any run between 00:00 and ~12:30 UTC placed it in the future,
so the provider-filtered durable path (today-range ends at now) dropped it
and the parity assertion failed. Seed timestamps proportionally between
start-of-today and now so they are always today and always in the past.
All five providers wired through createBridgedProvider with decode in
packages/core only. Deep validation corrected the initial drafts: zerostack
dedup-key/sessionId/project regressions, droid and mux crude bash-extraction
reimplementations (host now runs the real extractBashCommands on raw
strings), lingtai-tui dedup-key source, open-design dropped project. Goldens
are byte-identical toEqual pins anchored to the pre-migration decode.
Move the pure record decode for grok, kimi, and codewhale into
@codeburn/core and rewire the CLI providers through createBridgedProvider,
keeping discovery, file I/O, config resolution, and pricing host-side.
Validation fixes (legacy-parity, caught arm-by-arm against the pre-lift
decode):
- grok/codewhale: the host toProviderCall wrongly wrapped bashCommands in a
Set. The legacy decode deduped nothing (flat push of extracted base
commands), so per-command breakdown counts drifted. Drop the Set.
- kimi: the core decode dropped the legacy per-turn tool Set, so repeated
tools in a turn no longer collapsed. Restore [...new Set(tools)].
- codewhale: remove an unused codeWhaleToolNameMap import.
Parity fixtures now exercise the dedup arms (repeated tools/commands) and the
goldens are pinned to the ORIGINAL in-CLI decode output, captured by running
the pre-migration providers over the same fixtures.
Redesign the three token-waste detectors as pure @codeburn/core detectors that
consume an ObservationEnvelope of fingerprinted resource refs (never raw paths):
- Envelope schema 0.2.0: CallObservation gains optional resourceReads /
resourceEdits (ResourceRef = {resourceId 16-hex, resourceClass}). claude/codex
toObservations fingerprint toolSequence file paths into these; raw paths never
cross the boundary. Keep observation-0.1.0.json frozen; emit 0.2.0.
- Host privacy key (D1): sync keystore under the codeburn config dir, random
32-byte key generated on first use, stable across runs, never emitted.
- core/detectors: junk-reads, duplicate-reads, context-bloat — pure, zero
fs/env, each emitting Finding[] with confidence(basis), machine-readable
evidence, algorithmVersion.
- optimize.ts delegates the three to core, mapping Finding -> WasteFinding;
display strings, fix payloads and trend stay host-derived.
Numbers parity: all optimize tests pass unchanged; frozen-corpus optimize JSON
(3552 sessions) is byte-identical PRE vs POST.
Carve the Codex decoder into @codeburn/core/providers/codex as a pure function
over supplied rollout records with EXPLICIT, JSON-serializable state
(CodexDecodeState): the running cumulative token counters, fork-replay cutoff,
mid-turn accumulators, id/turn bookkeeping, and the cross-file dedup memory
(seenKeys). Decoding a corpus in one pass equals decoding it in any number of
passes when the state threads between them (the resume invariant).
Cost leaves the decoder: the rich calls are cost-free and the CLI prices them
through the established estimated-cost seam (costBasis: 'estimated'), retiring
both Phase-0 residual calculateCost sites. The CLI codex provider is now
discovery + file streaming + cache I/O + pricing.
codex-results cache bumped to v8 (one deliberate bump): it persists the decoder
end-state blob + a byte offset + host-priced calls, so a grown rollout resumes
from the stored state and decodes only the appended bytes. Lossless — rollout
files are durable, so the one-time re-derive rebuilds identical data.
Adds a codex path to the content-smuggling guardrail and the import-smoke
subpath set. Byte-parity verified against a frozen 826-file corpus (codex-only
and all-providers, rel_tol 1e-9); perf within ±1.5% cold / ±0.5% warm.
Move the pure Claude JSONL decode out of packages/cli/src/parser.ts into
packages/core/src/providers/claude/: the large-line buffer scanner, line
parsing, compaction, per-call/turn extraction, rich-capture meta collectors,
turn grouping, and streaming dedup. These are pure over supplied records/lines
(no fs/env/clock/pricing) and carry content in-memory only at this host-facing
layer.
Add the minimizing transform toObservations(richDecode, {privacyKey,...}) that
maps the rich decode into the strict Phase-2 observation envelope: only
fingerprints, enums, numbers, timestamps, dedup keys, and canonical tool names
cross the boundary. Extend the content-smuggling guardrail with a real claude
decode -> toObservations case planting secrets in every captured free-text
field and asserting the serialized envelope surfaces none.
The CLI parser is now the adapter: it keeps all I/O, discovery, incremental
append, cache shapes, classification, and host-side pricing. parseApiCall /
parseAdvisorCalls / groupIntoTurns become thin wrappers that price the cost-free
core decode (calculateCost + local-model savings) and split raw bash commands
(strip-ansi stays out of the zod-only core), then map into ParsedApiCall /
ParsedTurn. Shared record + tool-vocab definitions move to core; the CLI
re-exports them so every existing import path is unchanged.
No parse-version bump; CachedCall/CachedTurn shapes and values byte-identical.
Core runtime deps stay zod-only.
