JetBrains Copilot plugin ≤1.5.x (e.g. 1.5.59-243) stores all session turns
inside ONE large binary-framed outer Nitrite document, rather than the
per-turn {"__first__":{"type":"Subgraph",...}} blobs introduced in later
plugins (≥1.12.x, e.g. 1.12.1-251).
In the old format each assistant turn is a UUID-keyed Value entry whose
value field contains a JSON-string-escaped AgentRound record:
{"<uuid>":{"type":"Value","value":"{\"type\":\"AgentRound\",
\"data\":\"{...reply...}\"}"}, ...}
The extractResponseText depth-unescape loop already handles this one extra
level of escaping; the only gap was that extractJetBrainsDbTurns never fed
it the outer document — it only scanned for __first__/Subgraph blobs, which
the old plugin never writes.
Add a fallback that activates when the Subgraph scan produces zero turns but
'AgentRound' text is present in the raw file (old-format signal). It locates
the binary-framed outer document (UUID-keyed Value entry, hex matched
case-insensitively so an uppercase UUID does not fall through to $0), extracts
it with matchJsonObject, and passes it to extractResponseText. Because the outer
document holds every turn in one blob, this emits ONE session-level call per
document (all rounds' replies joined): cost/tokens are correct, only the
per-turn call-count granularity is coarser — an accepted tradeoff for legacy
data. MVStore keeps two identical collection copies; seenReplies dedupes them.
The fallback is guarded by turns.length === 0 so new-format sessions (whose
Subgraph scan succeeds) are completely unaffected and never double-counted.
Tests: old-format doc with multiple AgentRound rounds → 1 call whose token
count equals the two non-empty replies joined (the empty tool-call round is
excluded); an uppercase-UUID variant (fails without the case-insensitive
match); and a guard that new-format Subgraph turns are not double-counted.
docs/providers/copilot.md documents the old format and the one-call-per-session
limitation.
JetBrains Copilot has two turn shapes in the Nitrite .db:
- ask mode — the reply is a `Markdown` record's `text`;
- agent / plan mode (e.g. PyCharm agent sessions, `/plan …`) — the reply is the
`reply` field of an `AgentRound` record, and the `Markdown` record instead
holds the USER's prompt.
extractResponseText only read Markdown, so agent-mode turns yielded no reply
text: they were discovered (session/turn counts showed up) but priced at $0
because output tokens came out zero. On this machine that silently
under-counted a PyCharm session ($0 → $0.35) and several IntelliJ agent turns.
Determine the mode by the PRESENCE of an `AgentRound` record and read only that
record's `reply` (collecting every non-empty round in a multi-round blob).
Crucially, an agent blob whose reply is empty — a failed turn or a pure
tool-call round — does NOT fall back to the Markdown record, so a user prompt
is never mistaken for the assistant's output; such turns bill $0 as before.
Ask-mode blobs (no AgentRound) keep reading Markdown. Plan mode's sidecar
records — Thinking, PendingChanges (proposed diff, under `content`), AskQuestion,
Notification, SubTurn, and file-read `text` results — are never read as output.
Verified across all local stores: the two reply shapes never coexist in one
blob, so the split is unambiguous.
Tests: agent-mode reply extraction (ignoring the prompt Markdown), pure
tool-call rounds → $0, multi-round collection, and a failed agent turn → $0.
docs/providers/copilot.md documents both turn shapes and the ignored sidecar
records.
## What & why
The JetBrains Copilot plugin (IntelliJ, PyCharm, RubyMine, …) stores its
chat/agent sessions under `~/.config/github-copilot/<ide>/<kind>/<storeId>/` —
a location none of the existing Copilot sources (CLI JSONL, VS Code chat
sessions/transcripts, OTel SQLite) read. As a result all JetBrains Copilot
usage was silently uncounted in every CodeBurn report. This adds a reader for
that store so those sessions are discovered, priced, and attributed to the
right project.
## How it works
- **Reader.** The store's session content is a Nitrite `.db` — an H2 MVStore of
Java-serialized documents. It is scanned as `latin1` for byte-offset
stability: no Java deserializer, no new dependency, and it is not SQLite so
`node:sqlite` is not involved.
- **Reply text.** Assistant replies live in nested-escaped
`{"__first__":{"type":"Subgraph"…}}` blobs. The text is recovered by
unescaping one level at a time and, at the depth where the Markdown record's
`data` field is a well-formed one-level-escaped JSON document, reading it
structurally — so a reply containing its own quotes is never truncated or
duplicated (which would otherwise inflate the estimate).
- **Tokens/cost.** The store records no token counts, so output tokens are
estimated from the reply text (`CHARS_PER_TOKEN = 4`, re-decoded
latin1→utf8 so multibyte replies count by codepoint) and every call is marked
`costIsEstimated`. Failed generations (error status, no reply) are billed $0.
- **Sessions.** One `.db` holds many chat tabs; turns are grouped back to their
conversation GUID so the UI shows one session per tab, deduped by reply
content per conversation.
- **Project attribution**, most authoritative first:
1. the plugin-recorded `projectName` field (JetBrains Copilot 1.12+), joined
across kind dirs by store id — the billable turns live in
`chat-agent-sessions`, but the label is usually written into the sibling
`chat-sessions`/`chat-edit-sessions` store. Read length-delimited and
re-decoded latin1→utf8 so non-ASCII repo names round-trip.
2. the `.git` repo root of a referenced `file://` path.
3. a generic `copilot-jetbrains` bucket when neither signal exists.
The conversation title is a chat-thread name, not a project, so it is kept
out of the project field and surfaced as the session label instead.
Override the JetBrains github-copilot root with
`CODEBURN_COPILOT_JETBRAINS_DIR`.
## Docs
- `docs/providers/copilot.md` — full JetBrains section (store layout, latin1
scan, reply extraction, projectName precedence + cross-kind join).
- `docs/providers/README.md` — Copilot storage updated to note the Nitrite .db.
## How to verify
- `npm test -- copilot` and `npx tsc --noEmit` (fixtures reproduce the real
nested-escaped .db framing, including quote- and multibyte-bearing replies).
- End to end against a real install:
`CODEBURN_CACHE_DIR=$(mktemp -d) node dist/cli.js status --provider copilot \
--period all --format menubar-json`
— JetBrains sessions appear By-Project under their real repo names.
- Set `CODEBURN_COPILOT_JETBRAINS_DIR` to a fixture root to parse a controlled
store without touching the real config dir.
Bump the version, document the Zed provider (logo, provider doc, data
table row), add the audit and context commands to the README, and
refresh the stale Cursor data-source description to the composer-meter
accounting.
Pi and OMP have no dedicated skill tool: a native skill load is emitted as
an ordinary `read` tool call whose path points at the skill's SKILL.md (or a
`skill://<name>` URI in newer OMP builds). The parser mapped every read to the
Read tool and never populated `skills`, so Pi/OMP sessions over-counted Reads
AND always showed an empty "Skills & Agents" breakdown.
Detect these reads (basename === 'SKILL.md', or a skill:// URI), extract the
skill name (parent directory, or the URI segment), and surface the call as the
`Skill` tool with the name recorded in `skills` -- exactly how the Claude
parser represents a skill invocation. That both removes the Read over-count and
lets the shared classifier tag the turn `general` so the Skills & Agents
breakdown picks it up (populating a field the dashboard never received before).
The path is read from arguments.path with a defensive arguments.file_path
fallback.
Tests cover SKILL.md / skill:// / file_path detection, a non-skill read staying
a Read, and an end-to-end check that a parsed skill load reaches the classifier
subCategory that feeds skillBreakdown.
VS Code GitHub Copilot Chat users with no OTel agent-traces.db and no
~/.copilot/session-state/ saw $0.00 cost: codeburn never read VS Code's
core chat persistence, the only on-disk source carrying their token counts.
Add a fourth Copilot source that reads the VS Code chat delta-journals at
workspaceStorage/<hash>/chatSessions/*.jsonl and
globalStorage/emptyWindowChatSessions/*.jsonl. The files are a kind:0
snapshot / kind:1 path-set / kind:2 array-append journal; we replay it
(prototype-pollution-safe: __proto__/prototype/constructor segments are
rejected and containers use Object.create(null)) and read each request's
result.metadata.promptTokens / outputTokens (falling back to
completionTokens) and resolvedModel for pricing.
Dedup so users with multiple sources are not double-counted: prefer OTel
(skip chatSessions discovery when an OTel source is present), and skip a
workspace's legacy GitHub.copilot-chat/transcripts when that workspace has
chatSessions. New env overrides CODEBURN_COPILOT_GLOBAL_STORAGE_DIR and the
existing CODEBURN_COPILOT_WS_STORAGE_DIR keep discovery testable.
Tests cover the reporter's real request shape (promptTokens 32543 /
outputTokens 60 -> non-zero cost), empty sessions, emptyWindow discovery,
append-then-edit replay, requestId dedup, prototype-pollution paths, the
transcript skip, and the OTel-prefer skip.
* fix: add mssing support for ATIF v1.7
Co-Authored-By: bmcdonough <18721778+bmcdonough@users.noreply.github.com>
* fix(devin): drop unused MCP-coupled types, dead guard, harden metrics fallback
- Remove the @modelcontextprotocol/sdk JsonSchemaType import and the unused
ToolDefinition/FunctionDefinition types it served; type Agent.tool_definitions
as unknown since the parser never reads it (no dependency warranted).
- Remove isImageContentPart: unused, and its `"image" in part` check could never
be true (image parts carry `source`/`type`, not `image`).
- Use the `type` discriminant in isTextContentPart instead of property presence.
- getMetricsFromStep: fall back to legacy metadata.metrics when step.metrics is
present but empty, so a partial metrics object cannot silently zero usage.
- Tests: cover the empty step.metrics fallback and image-only message normalization.
---------
Co-authored-by: bmcdonough <18721778+bmcdonough@users.noreply.github.com>
Co-authored-by: AgentSeal <hello@agentseal.org>
Reads session-level usage from Hermes SQLite state databases
(~/.hermes/state.db and per-profile state.dbs). Tracks input,
output, cache read/write, reasoning tokens, stored costs, tools,
shell commands, and inferred projects.
- Provider reads both root and profile state databases
- Cost fallback: actual_cost_usd > estimated_cost_usd > LiteLLM pricing
- inferProject filters to user/system messages only
- Discovery query capped at LIMIT 10000
- sanitizeProject handles repeated leading separators
- All five review items from PR #386 addressed
- Tested against real Hermes data (557MB state.db with 198 sessions)
Closes#368. Supersedes #386.
Reads ZCode CLI usage from ~/.zcode/cli/db/db.sqlite. The model_usage
table has exact per-request tokens; cost is computed from the pricing
table since ZCode stores none (GLM-5.2 runs on the z.ai start-plan
subscription).
- Split cached tokens out of input_tokens (OpenAI-style) so fresh input
and cache reads price correctly
- Attach each turn's tool calls to one request to avoid double-counting
- Map GLM-5.2 to glm-5p1 (GLM-5.1 rate) until LiteLLM lists it
- Register as a lazy SQLite provider; add test and provider doc
Adds Grok Build (xAI coding CLI) as an eager provider with caching- and compaction-aware token estimation, real tool/shell extraction, Skills & Agents from spawn_subagent, grok-build pricing, and SuperGrok plan presets. Tested against real sessions; full suite green.
* Add zerostack provider scaffold
Register zerostack (gi-dellav/zerostack) as a core provider reading
plain-text sessions from $XDG_DATA_HOME/zerostack/sessions/. Parser is
modeled on pi.ts; on-disk schema is unverified and must be confirmed
against real sessions before a PR (see docs/providers/zerostack.md).
* Rework zerostack provider against real session schema
Replace the initial scaffold (modeled on pi.ts JSONL) with the verified
format from a real local run and the zerostack source:
- Sessions are one JSON file per session under the platform data dir
(~/Library/Application Support/zerostack/sessions on macOS,
$XDG_DATA_HOME/zerostack/sessions on Linux; ZS_DATA_DIR overrides).
- Tokens are cumulative session totals, not per-call, so emit one
ParsedProviderCall per session from total_input/output_tokens.
- Recompute cost via LiteLLM (calculateCost); OpenRouter ids arrive
route-prefixed and resolve through canonical names.
- zerostack persists only final assistant text, so tools/bash are empty.
Verified against a real session (DeepSeek v4 Pro via OpenRouter):
34.1K in / 961 out / $0.016, matching the recorded total_cost. Adds a
fixture-based test and rewrites the provider doc to match.
Maintainer review fixes on top of the OTel cache-token work:
- Remove all DEBUG_OTEL console.warn scaffolding from parser.ts and
copilot.ts (gated but unlike the rest of the codebase).
- Parameterize the spans IN (...) query instead of string-interpolating
trace IDs.
- Fold the per-chat-span metadata query into the trace-span query to drop
the N+1 (one query per chat span -> one per conversation).
- Guard epochToISO against null/NaN/0 so a malformed start_time_ms row no
longer throws on new Date(NaN).toISOString().
- Remove dead code: parseSpanAttributes, OTelSpanRow, and the unreachable
`if (!db) return`; drop unused catch bindings.
- Note in parseProviderSources that the non-durable append path assumes
unique source paths.
- Document the OTel source in docs/providers/copilot.md: Node 22+
requirement, durable-cache monotonic totals, and the one-time
parse-version cache reset on upgrade.
Opt-in Vercel AI Gateway provider (AI_GATEWAY_API_KEY/VERCEL_OIDC_TOKEN), inert without a key. Cursor lookback now period-aligned with a 6-month floor. Gateway fetch hardened on merge with an 8s timeout and try/catch fallback.
* feat(providers): add coder/mux as a datasource
Reads coder/mux session data from ~/.mux/sessions/<workspaceId>/chat.jsonl and normalizes per-assistant-message token usage into ParsedProviderCall. Discovery resolves project names from config.json; the token decomposition matches mux's own inclusive input/output accounting, and cost is recomputed via LiteLLM. Includes unit tests and a provider doc.
Change-Id: Ie6ce9d41254d8bf05ff0965b443328dfa7b598de
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Thomas Kosiewski <tk@coder.com>
* fix(providers): discover mux sub-agent transcripts
discoverSessions only globbed sessions/<id>/chat.jsonl and skipped each
spawned sub-agent's transcript at
sessions/<id>/subagent-transcripts/<childTaskId>/chat.jsonl. Against a real
~/.mux (629 workspaces) those nested files are 5,889 sessions and ~51% of all
assistant messages, so sub-agent spend was silently dropped: total mux usage
went from $17,113 to $21,564 (+26%) once counted.
Walk the subagent-transcripts dir per workspace and attribute each child
session to the parent's project. Dedup is unaffected: the parser keys off the
<childTaskId> dir name, which is disjoint from every workspace id, so each call
is still counted once. Cross-checked parsed per-model token totals against
mux's sibling session-usage.json.
Also corrects the provider doc, which wrongly claimed sub-agents get their own
top-level sessions/<id>/chat.jsonl, and documents the Google reasoning>output
decomposition edge case.
Change-Id: I1d43f26eec7c9c6c523e5ea541e2ff8d0c3aa07e
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds extensionless session index and nested execution file discovery
while preserving legacy .chat support. Modern execution JSON is parsed
for identifiers, timestamps, model IDs, conversation text, structured
tools, and token estimates. Also fixes dedup key poisoning on invalid
timestamps in the legacy parser.
* Capture Antigravity CLI usage via statusline
* Harden statusLine hook: cap stdin at 1MB, fix JSONL file permissions
- readStdin() now rejects input exceeding 1MB to prevent OOM from
a buggy or runaway caller.
- Switch from appendFile (mode only on creation) to open(path, 'a', 0o600)
so the JSONL file is always created with private permissions.
- Set cache directory mode to 0o700.
---------
Co-authored-by: iamtoruk <hello@agentseal.org>
* Add CodeBurn Pro Mac App Store app
SwiftUI MenuBarExtra with litellm-snapshot pricing, Claude/Codex/Copilot
parsers, session discovery, auto-refresh timer, and dashboard UI matching
the real menubar design.
* Add appstore/ to .gitignore
Private Mac App Store build, not for the public repo.
* Fix one-shot rate detection for Gemini, Vibe, Kiro, and Goose
Gemini and Mistral Vibe now emit per-assistant-message calls grouped
by user turn via turnId, so the classifier sees Edit->Bash->Edit as
retries instead of independent one-shot turns. Kiro and Goose record
per-message tool ordering via toolSequence for the same effect on
aggregated sessions.
Vibe now prefers meta.json.stats.session_cost over price-derived
estimates. Session cache bumped to v2. Insight pill switcher scrolls
horizontally instead of wrapping.
Closes#351.
* Remove dead geminiOrdinal variable and add toolSequence cache validation
* Restore geminiOrdinal for idx fallback dedup key
* Add IBM Bob provider
* Add workspace extraction for Cline-family providers
Extract project name from workspace directory in api_conversation_history.json
so sessions show actual folder names instead of the provider display name.
Thread projectPath through ParsedProviderCall to avoid unsanitizePath mangling
hyphenated folder names.
---------
Co-authored-by: ozymandiashh <234437643+ozymandiashh@users.noreply.github.com>
Co-authored-by: iamtoruk <hello@agentseal.org>
Closes#278.
Adds Charmbracelet Crush as a lazy-loaded provider:
- src/providers/crush.ts: walks ~/.local/share/crush/projects.json
(XDG_DATA_HOME and CRUSH_GLOBAL_DATA aware), opens each project's
crush.db read-only, queries root sessions where parent_session_id
IS NULL. Emits one ParsedProviderCall per session with real
prompt_tokens, completion_tokens, cost (dollars), and the
dominant model resolved from messages.model.
- src/providers/index.ts: register crush alongside cursor, goose,
opencode, antigravity, cursor-agent in the lazy import path.
- tests/providers/crush.test.ts: 10 fixture-based tests covering
discovery, parsing, missing-registry, malformed JSON, missing db,
child session exclusion, dominant model selection, dedup, and
array-shaped legacy registry.
Schema source: charmbracelet/crush@v0.66.1
internal/db/migrations/20250424200609_initial.sql, verified by
spawning a research agent against upstream. The schema *comments*
in that migration claim millisecond timestamps but every actual
INSERT/UPDATE uses strftime('%s', 'now') which returns Unix
seconds; the parser treats values as seconds. Tokscale's
parser (junhoyeo/tokscale#346) gets this wrong and is off by
1000x, plus its parser misses the prompt_tokens/completion_tokens
columns that exist in Crush's schema. Our integration uses both,
so Crush sessions get real per-model attribution.
Menubar:
- mac/Sources/CodeBurnMenubar/AppStore.swift: add .crush case to
ProviderFilter and its cliArg switch.
- mac/Sources/CodeBurnMenubar/Views/AgentTabStrip.swift: add
Crush color to the per-tab color extension. The visibleFilters
computed property already filters by detected providers, so the
Crush tab appears automatically when a user has Crush data.
README:
- Replace the provider table with an icon-led layout. Icons live
under assets/providers/<name>.<ext>. 14 icons sourced from
junhoyeo/tokscale (MIT) under nominative fair use, 4 sourced
separately: codex (OpenAI org avatar), cursor-agent (reuses the
Cursor icon), kiro (kiro.dev favicon, ico->png via sips), omp
(can1357/oh-my-pi icon.svg, MIT). Attribution line added.
- Add Crush row.
Docs:
- docs/providers/crush.md: full per-provider doc with verified
schema excerpt, the seconds-vs-milliseconds quirk, and a
"when fixing a bug here" checklist.
- docs/architecture.md: provider count 17 -> 18, test count
41 -> 42, and crush in the lazy list.
- docs/providers/README.md: add Crush row to the lazy index.
- CONTRIBUTING.md: bump test count to 568 (was 558).
All 568 tests pass locally; swift build clean.
Document the contributor onboarding path:
- CONTRIBUTING.md: setup, npm scripts, coding conventions, PR process,
the block-claude-coauthor enforcement, and the five providers without
test coverage today (claude, gemini, goose, qwen, antigravity).
- docs/architecture.md: 12-command CLI surface, parser pipeline, three
cache layers, 14 optimize detectors, and the mac / gnome / build
layouts with cited line numbers.
- docs/providers/: one file per provider (17 providers plus the shared
vscode-cline-parser helper). Each covers data path, storage format,
caching, dedup key, quirks, and a "when fixing a bug here" checklist.
Also fix two pre-existing documentation issues surfaced while writing
the new docs:
- RELEASING.md claimed GitHub Actions auto-publishes the CLI when a
v* tag is pushed. There is no such workflow; CLI publishing is
manual via npm publish. Updated the CLI section to reflect reality
and kept the menubar (mac-v* tag) automation accurate.
- .gitignore had CLAUDE.md unanchored, which on case-insensitive
filesystems also matched docs/providers/claude.md. Anchored to
/CLAUDE.md so the root-level memory file stays ignored without
affecting subdirectory docs.
All cited file paths, line numbers, function names, and test counts
were verified against current code (41 test files, 558 tests passing).
Side-by-side comparison of any two AI models using normalized metrics:
cost/call, one-shot rate, retry rate, self-correction rate, cache hit.
Accessible via codeburn compare and dashboard [c] shortcut.
