Merge pull request #1001 from MiloMMIN/feat/dsh-provider

feat: add DeepSeek Harness (dsh) provider
This commit is contained in:
Resham Joshi 2026-08-18 01:54:48 -07:00 committed by GitHub
commit d5b3720079
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 1400 additions and 9 deletions

View file

@ -9,12 +9,18 @@ jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
# Package floor, and the newest 22.x so paths gated on later node:zlib
# features (zstd, 22.15+) get exercised.
node-version: [22.13.0, 22]
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v4
with:
node-version: 22.13.0
node-version: ${{ matrix.node-version }}
cache: npm
- run: npm ci
- name: Typecheck

View file

@ -2,6 +2,9 @@
## Unreleased
### Added (CLI)
- **DeepSeek Harness (`dsh`) is now a supported provider.** Reads DeepSeek's open-source agent harness from `~/.dsh/sessions` (`DSH_HOME` relocates the root), both the default zstd logs and the uncompressed `session.jsonl` variant. A `.zstd` log is a concatenation of independent zstd frames, one per write batch, so it is decoded frame by frame behind a structural frame scan and a torn trailing frame from a crashed writer is ignored rather than failing the file (needs Node 22.15+ for `zlib` zstd; below that dsh is skipped with a notice instead of counted as $0). One call per `(turn, step)`, with the step's final `assistant/message` usage superseding the streamed `assistant/chunk` sample of the same call rather than adding to it, the model taken from the message that served the step, and reasoning tokens billed at the output rate. DSH records tokens but no cost, so calls are priced from the shared tables. The events a forked session replays from its parent are skipped, since codeburn already counts the parent's own log. The session format is pinned at version 0 upstream with no compatibility implied, so a log stamped with any other version is skipped with a notice instead of read under today's assumptions.
### Changed
- **Codex rollouts parse across worker threads too, and the workload gate now takes bytes or files.** Codex is the bigger half of a real cold parse — a 4 GB rollout corpus against 1.8 GB of Claude sessions — and it was still decoding one file at a time. A whole-file rollout decode now runs on the same pool, against an empty dedup set, and comes back with the calls, the dedup keys it claimed, and the codex-cache entry it would have written; the parent installs all three in the serial loop's order, so `codex-results.json` and every payload come out byte-identical to a serial run. Cross-file state stays where it was: a forked rollout replaying its parent's token_count history collides on the parent's keys and is re-parsed in-process, and no worker ever touches the cache module's per-directory state. Files the Codex cache can serve exactly or resume into from a byte offset never reach a worker — they read a few KB and the resume state belongs to the parent. The workload gate is now pending BYTES alone (200 MB), not file count: 250 pending files holding under a megabyte between them spawned threads that made the run ~5% slower, while a few hundred huge rollouts were being turned away. The count takes `max(pendingFiles / 50, pendingBytes / 200 MB)`, and the per-thread memory budget is derived per parse as `clamp(256 MB, 2 × average pending file + 128 MB, 1 GB)` rather than a flat 256 MB — a 260 MB rollout peaks near 430 MB in its worker and scales linearly with the pool, so the flat figure over-subscribed exactly the workload this adds. The decision is per provider, and at most one pool is alive at a time.
- **A large cold Claude parse now runs across worker threads.** Reading, decoding and line-parsing a session JSONL is per-file work that never touches anything shared, so it moves onto `worker_threads`; each worker ships its parsed turns back as a JSON string and the parent installs them in the exact order the serial loop would. Everything with cross-file state — the streaming-message dedup, canonical project paths, spawn links, PR correlation, progress saves — stays on the main thread, and a file whose message ids were already claimed by an earlier file (or whose worker failed) is simply re-parsed in-process, so the session cache and every payload are identical either way. On a 6 GB corpus a cold `status` drops from 27.5s to 14.8s with peak RSS up 2.27 GB → 2.52 GB. Threads only engage for a genuinely large cold parse: never with under 200 MB behind the pending whole-file re-parses, 2 or fewer cores, or under 4 GB of available memory — so warm and incremental runs are untouched and spawn nothing. Otherwise the count is `min(cores - 1, min(0.25 × available, 2 GB) / 256 MB, pendingFiles / 50)`, where available is `process.availableMemory()` (cgroup-aware in containers) rather than free memory, which on macOS reports free pages and would switch the feature on and off between runs. `CODEBURN_PARSE_WORKERS=0` forces the serial parse and `CODEBURN_PARSE_WORKERS=N` forces N (capped at the core count), both bypassing every gate; `CODEBURN_VERBOSE=1` prints the resolved count and why.

View file

@ -25,7 +25,7 @@
<a href="https://github.com/sponsors/iamtoruk"><img src="https://img.shields.io/badge/sponsor-♥-F97316?logo=github" alt="Sponsor" /></a>
</p>
<p align="center">If CodeBurn shows you something your bill never did, <a href="https://github.com/getagentseal/codeburn/stargazers">star the repo</a> so other developers find it, and consider <a href="https://github.com/sponsors/iamtoruk">sponsoring</a> to keep 40 integrations honest.</p>
<p align="center">If CodeBurn shows you something your bill never did, <a href="https://github.com/getagentseal/codeburn/stargazers">star the repo</a> so other developers find it, and consider <a href="https://github.com/sponsors/iamtoruk">sponsoring</a> to keep 41 integrations honest.</p>
<table align="center">
<tr>
@ -61,11 +61,11 @@
<p align="center"><em>Four surfaces, one source of truth: everything reads the session files already on your disk.</em></p>
**CodeBurn is a free, open-source, local-first tool that tracks AI coding token usage and cost across 40 tools and agents (Claude Code, Cursor, Codex, Gemini, Grok and more), broken down by model, project, and task.**
**CodeBurn is a free, open-source, local-first tool that tracks AI coding token usage and cost across 41 tools and agents (Claude Code, Cursor, Codex, Gemini, Grok and more), broken down by model, project, and task.**
You pay for Claude, Codex, Cursor, and a stack of other AI tools. The bill tells you the total. It never tells you that half of it went to conversation instead of code, or that an expensive model burned your budget on work a cheaper one would have one-shot.
CodeBurn does. It reads the session files your tools already write to disk and breaks down every token and dollar by **task, model, tool, and project**, across **40 AI tools**.
CodeBurn does. It reads the session files your tools already write to disk and breaks down every token and dollar by **task, model, tool, and project**, across **41 AI tools**.
Everything runs locally. No wrapper, no proxy, no API keys, nothing leaves your machine. Pricing comes from [LiteLLM](https://github.com/BerriAI/litellm), refreshed daily.
@ -683,6 +683,7 @@ These are starting points, not verdicts. A 60% cache hit on a single experimenta
| **Cline / Roo Code / KiloCode** | VS Code `globalStorage` across VS Code, VS Code Insiders, and VSCodium (Cline at `saoudrizwan.claude-dev`, plus `~/.cline/data`) | Cline-family agents. CodeBurn reads `ui_messages.json` from each task directory, extracting token counts from `type: "say"` entries with `say: "api_req_started"`. |
| **Cline CLI** | `~/.cline/data/sessions/<session-id>/` (honors `CLINE_SESSION_DATA_DIR`, `CLINE_DATA_DIR`, `CLINE_DIR`) | The Cline command-line agent, whose layout is unrelated to the VS Code extension's. Reads `<session-id>.json` for session metadata and the rolled-up `usage`, and `<session-id>.messages.json` for the per-message `metrics` block (input, output, cacheRead, cacheWrite, cost) that becomes one call each. |
| **CodeWhale** | `~/.codewhale/sessions/*.json` plus unmigrated legacy `~/.deepseek/sessions/*.json`; `$CODEWHALE_HOME/sessions` is an exact override | Emits one cumulative record per saved session. CodeWhale exposes only `total_tokens`, so CodeBurn preserves that aggregate in the input column rather than inventing an input/output split. Cost is the exact stored parent-session plus subagent USD total; model pricing is used only when the cost snapshot is absent. Tool blocks, shell commands, skills, and subagent types are retained. |
| **DeepSeek Harness** (`dsh`) | `~/.dsh/sessions/--<slug>--/<session-id>/session.jsonl.zstd` (or `session.jsonl` when compression is off); `DSH_HOME` relocates the root | DeepSeek's open-source agent harness, unrelated to the CodeWhale desktop app. The `.zstd` log is a concatenation of independent zstd frames (one per write batch), decoded frame by frame; needs Node 22.15+. One call per `(turn, step)`, with usage from the step's `assistant/message` (the streamed `assistant/chunk` sample is a draft of the same call, never a second one). DSH records tokens but no cost, so calls are priced from the shared tables with reasoning billed at the output rate. |
| **IBM Bob** | `User/globalStorage/ibm.bob-code/tasks/<task-id>/` (GA `IBM Bob` and preview `Bob-IDE` app folders) | Reads `ui_messages.json` for API request token/cost records and `api_conversation_history.json` for the selected model. |
| **Kimi Code CLI** | `$KIMI_SHARE_DIR/sessions/<workdir-hash>/<session-id>/` or `~/.kimi/sessions/<workdir-hash>/<session-id>/` | Reads `wire.jsonl` `StatusUpdate.token_usage` records, mapping `input_other`, `input_cache_read`, `input_cache_creation`, and `output` into the standard token columns; includes subagents under each session's `subagents/` folder. |
| **LingTai TUI** | `~/.lingtai/<agent>/logs/token_ledger.jsonl` plus project homes from `~/.lingtai-tui/registry.jsonl` (`<project>/.lingtai/<agent>/logs/token_ledger.jsonl`); honors `LINGTAI_HOME` / `LINGTAI_TUI_HOME` | Reads LingTai's append-only token ledger, mapping `input - cached` to fresh input, `cached` to cache reads, `output` to output, and `thinking` to reasoning. Nested daemon ledgers are skipped because parent ledgers already mirror daemon usage with `source`/`run_id` tags. |
@ -722,12 +723,12 @@ CodeBurn deduplicates messages (by API message ID for Claude, by cumulative toke
CodeBurn is free, runs entirely on your machine, and exists to cut your AI bill. If it has already saved you more than a sponsorship costs, consider sending a little of that back.
Keeping 40 integrations accurate is constant work. The tools underneath change every week: Cursor reshapes its database, Claude moves a config path, new models ship at new prices. Sponsorship keeps CodeBurn current with all of it, so the numbers you see are always the real ones.
Keeping 41 integrations accurate is constant work. The tools underneath change every week: Cursor reshapes its database, Claude moves a config path, new models ship at new prices. Sponsorship keeps CodeBurn current with all of it, so the numbers you see are always the real ones.
Where your sponsorship goes:
- **Honest numbers.** New models and price changes are mapped quickly, so your cost is the real cost, not a guess.
- **More tools.** Every one of the 40 providers started as a single file. Sponsorship funds the next one.
- **More tools.** Every one of the 41 providers started as a single file. Sponsorship funds the next one.
- **Fast fixes.** When a vendor breaks something, paid time is what gets it patched now instead of someday.
Sponsoring as a team or company? Your logo lands right here, in front of every developer who opens the repo. The first sponsor gets it to themselves until the next one shows up.

51
THIRD_PARTY_NOTICES.md Normal file
View file

@ -0,0 +1,51 @@
# Third-party notices
CodeBurn is MIT licensed (see `LICENSE`). It also contains code derived from the
projects below, which carry their own terms. Each notice is reproduced here as
those terms require.
---
## @deepseek-ai/dsh-session-persistence-jsonl
`scanZstdFrames` in `src/providers/dsh.ts` is a transcription of the function of
the same name in this package (`src/zstd.ts`), which is what lets CodeBurn read
a DeepSeek Harness session log without depending on the harness itself. No other
part of the package is used.
Upstream declares two different licenses for this package: the published npm
package (0.0.1-rc.1) ships a BSD 3-Clause `LICENSE` and declares
`"license": "BSD-3-Clause"`, while the monorepo source it is built from
(`deepseek-ai/deepseek-harness`, `packages/session/session-persistence-jsonl`)
declares MIT. The stricter of the two is reproduced below.
```
BSD 3-Clause License
Copyright (c) 2026, DeepSeek
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```

View file

@ -169,6 +169,7 @@
"$HOME/.copilot",
"$HOME/.cursor",
"$HOME/.deepseek",
"$HOME/.dsh/sessions",
"$HOME/.factory",
"$HOME/.forge",
"$HOME/.gemini",

View file

@ -191,7 +191,7 @@ type Provider = {
`src/providers/index.ts` registers providers across two tiers:
- **Eager**: `claude`, `cline`, `codewhale`, `codebuff`, `codex`, `copilot`, `devin`, `droid`, `gemini`, `hermes`, `ibm-bob`, `kilo-code`, `kiro`, `kimi`, `lingtai-tui`, `mistral-vibe`, `mux`, `openclaw`, `open-design`, `pi`, `omp`, `qwen`, `roo-code`, `zerostack`, `grok`. Imported at module load.
- **Eager**: `claude`, `cline`, `codewhale`, `codebuff`, `codex`, `copilot`, `devin`, `droid`, `dsh`, `gemini`, `hermes`, `ibm-bob`, `kilo-code`, `kiro`, `kimi`, `lingtai-tui`, `mistral-vibe`, `mux`, `openclaw`, `open-design`, `pi`, `omp`, `qwen`, `roo-code`, `zerostack`, `grok`. Imported at module load.
- **Lazy**: `antigravity`, `forge`, `goose`, `cursor`, `opencode`, `cursor-agent`, `crush`, `warp`, `vercel-gateway`, `zcode`, `zed`. Imported via dynamic `import()` so the heavy dependencies (SQLite, protobuf, network clients) do not touch users who do not have those tools installed.
Both lists hit the same `getAllProviders()` aggregator. A failed lazy import is silent and excludes that provider from the run.

View file

@ -18,6 +18,7 @@ For the architectural picture, see `../architecture.md`.
| [Copilot](copilot.md) | JSONL + SQLite (OTel) + Nitrite .db (JetBrains) | `src/providers/copilot.ts` | `tests/providers/copilot.test.ts` |
| [Devin](devin.md) | JSON + SQLite enrichment | `src/providers/devin.ts` | `tests/providers/devin.test.ts` |
| [Droid](droid.md) | JSONL | `src/providers/droid.ts` | `tests/providers/droid.test.ts` |
| [DeepSeek Harness](dsh.md) | JSONL (zstd frames) | `src/providers/dsh.ts` | `tests/providers/dsh.test.ts` |
| [Gemini](gemini.md) | JSON / JSONL | `src/providers/gemini.ts` | none |
| [Hermes Agent](hermes.md) | SQLite | `src/providers/hermes.ts` | `tests/providers/hermes.test.ts` |
| [IBM Bob](ibm-bob.md) | JSON | `src/providers/ibm-bob.ts` | `tests/providers/ibm-bob.test.ts` |

71
docs/providers/dsh.md Normal file
View file

@ -0,0 +1,71 @@
# DeepSeek Harness (dsh)
DeepSeek's open-source agent harness (`dsh`, npm `@deepseek-ai/dsh`). Unrelated to the [CodeWhale](codewhale.md) provider, which reads the DeepSeek desktop app.
- **Source:** `src/providers/dsh.ts`
- **Loading:** eager (`src/providers/index.ts`)
- **Test:** `tests/providers/dsh.test.ts`
## Where it reads from
| Level | Env var | Default |
|---|---|---|
| sessions | — | `<root>/sessions` |
| root | `DSH_HOME` | `~/.dsh` |
An empty `DSH_HOME` is treated as unset. `probeRoots()` reports the resolved sessions dir, so `codeburn doctor` distinguishes "dsh not installed" from "`DSH_HOME` pointing somewhere empty".
## Storage format
```
sessions/--<slugified-cwd>--/<session-id>/
session.jsonl.zstd default (compression: zstd)
session.jsonl when compression: none
```
Both variants are read; a session directory never holds both. The log is append-only JSONL whose first line is the session header:
```jsonc
{ "type": "session", "version": 0, "id": "...", "createdAt": 1783352050748,
"cwd": "/home/u/proj", "parentSession": "...", "seedLength": 3, "delegationDepth": 0 }
```
`cwd` becomes `projectPath` / `workingDirectory` (git-repo attribution) and its last segment the project name.
Every later line is one event `{ type, seq, time, data }`. The parser reads:
| Event | Used for |
|---|---|
| `turn/start` | current turn number |
| `user/message` | the turn's preview, when `data.source.kind === 'user'` |
| `request/header` | `data.header.config.model` — the model for steps that follow |
| `assistant/chunk` with `chunk.type === 'usage'` | streamed usage sample for `(turn, step)` |
| `assistant/message` | final usage for `(turn, step)`, plus `data.message.source.model` |
| `tool/call` | tool names, bash commands, skill names |
One parsed call per `(turn, step)` — one model call and the tools it requested. Dedup key: `dsh:<sessionId>:<turn>:<step>`.
`.zstd` logs are a concatenation of **independent** zstd frames, one per write batch, so they are decoded frame by frame behind a structural frame scan ported from `@deepseek-ai/dsh-session-persistence-jsonl`. Needs Node 22.15+ for `zlib.zstdDecompressSync`; below that dsh is skipped with a notice instead of counted as $0.
## Caching
None at the provider level; the log file is the cached source path and the normal parser/cache layers apply. Cache invalidates on `DSH_HOME` (`PROVIDER_ENV_VARS`) and on parser changes (`PROVIDER_PARSE_VERSIONS`).
## Quirks
- **DSH is a developer preview.** `SESSION_FORMAT_VERSION` is pinned at `0` with "no compatibility implied" upstream, and breaking changes are expected. The parser reads version `0` only and skips a log stamped with anything else, with a notice — reading a bumped format under today's assumptions would report confident wrong numbers. **A version bump upstream means this parser needs updating, not just relaxing the check.**
- **The JSONL backend only.** DSH also ships an opt-in SQLite persistence backend (`@deepseek-ai/dsh-session-persistence-sqlite`); it is not the default and is not read.
- **DSH records tokens, never dollars.** `usage` is `{ inputTokens, outputTokens, cacheReadTokens?, cacheWriteTokens?, reasoningTokens? }` with no cost field, so every call is priced from the shared tables. Reasoning bills at the output rate (same as Gemini and Hermes): `outputTokens + reasoningTokens` goes into `calculateCost`, while the two stay separate on the emitted call. Tokens are the provider's own exact counts, so `costIsEstimated` stays false.
- **`assistant/message` usage wins over the `assistant/chunk` sample** for the same `(turn, step)` — the two are adjacent reports of one API call, not two calls. A late chunk never overwrites a final report, so the two are never summed.
- **The model comes from the message, not the request.** `data.message.source.model` is what actually served the step; `request/header` only describes the request DSH was about to make, and is the fallback when a message names no model. The `provider` field there (`deepseek-official`) is the upstream LLM route, not the tool — the codeburn provider name is always `dsh`.
- **A forked session's log replays its parent's events.** The header's `parentSession` + `seedLength` mark that prefix; codeburn parses the parent's own log as its own session, so events with `seq < seedLength` are skipped to avoid billing the same calls twice.
- **`user/message` also carries agent-injected context** (runtime snapshots, skill bodies, file-change notices) under `source.kind: 'plugin'`. Only `kind: 'user'` messages become the preview.
- **Delta chunks are packed.** Runs of streamed deltas are stored as `text-chunks` / `reasoning-chunks` / `tool-call-chunks` storage rows rather than one event per line. They carry no usage and no tool identity the `tool/call` event lacks, so they are ignored — as is any event type the parser does not know.
- **A torn final zstd frame is ignored.** A crashed writer leaves an incomplete trailing frame; the complete frames before it parse normally. A structurally corrupt file is skipped whole with a notice rather than throwing.
## When fixing a bug here
1. Reproduce with a minimal session dir: `sessions/--proj--/<id>/session.jsonl` (uncompressed is easiest to hand-write).
2. `tests/fixtures/dsh/bash-tool-turn.jsonl` is the upstream `examples/acp-agent/tests/snapshots/bash-tool-turn/session.jsonl` snapshot with its template placeholders filled in — refresh it from the DSH repo when the format moves.
3. Run `tests/providers/dsh.test.ts`.
4. `.zstd` fixtures must compress **each batch separately**; one `zstdCompressSync` over the whole file is a single-frame layout DSH never writes.

View file

@ -72,6 +72,8 @@ enum UsageDataChangeGuard {
add(expand(environment["CODEWHALE_HOME"] ?? path(homeDirectory, ".codewhale"), homeDirectory: homeDirectory), scanFirstLevelDirectories: false)
add(path(homeDirectory, ".deepseek", "sessions"), scanFirstLevelDirectories: false)
add(path(homeDirectory, ".cline", "data"), scanFirstLevelDirectories: false)
let dshHome = expand(environment["DSH_HOME"] ?? path(homeDirectory, ".dsh"), homeDirectory: homeDirectory)
add(path(dshHome, "sessions"))
add(expand(environment["CODEBUFF_DATA_DIR"] ?? path(xdgConfig, "manicode"), homeDirectory: homeDirectory), scanFirstLevelDirectories: false)
let factoryHome = expand(environment["FACTORY_DIR"] ?? path(homeDirectory, ".factory"), homeDirectory: homeDirectory)
add(path(factoryHome, "sessions"), scanFirstLevelDirectories: false)

View file

@ -9,6 +9,7 @@
},
"files": [
"dist",
"THIRD_PARTY_NOTICES.md",
"!dist/parse-worker.js.map"
],
"scripts": {
@ -33,6 +34,7 @@
"pi",
"codebuff",
"codewhale",
"dsh",
"ai-coding",
"token-usage",
"cost-tracking",

591
src/providers/dsh.ts Normal file
View file

@ -0,0 +1,591 @@
import { open, readdir, readFile, stat } from 'fs/promises'
import { join } from 'path'
import { homedir } from 'os'
import zlib from 'zlib'
import { MAX_SESSION_FILE_BYTES, readSessionFile } from '../fs-utils.js'
import { calculateCost, getShortModelName } from '../models.js'
import { extractBashCommands } from '../bash-utils.js'
import type { ProbeRoot, Provider, SessionSource, SessionParser, ParsedProviderCall } from './types.js'
// DeepSeek Harness (dsh) stores one session per directory:
// <DSH_HOME|~/.dsh>/sessions/<encoded-cwd>/session-<uuid>/session.jsonl.zstd
// (or an uncompressed session.jsonl when compression=none). The .zstd file is
// a concatenation of INDEPENDENT zstd frames — one per appended event batch —
// so node:zlib's one-shot zstdDecompressSync (which decodes a single frame)
// must be driven frame-by-frame behind a structural frame-boundary scan. The
// scan below is a port of scanZstdFrames from the official
// @deepseek-ai/dsh-session-persistence-jsonl package, which is third-party code
// under its own license - see THIRD_PARTY_NOTICES.md.
// zstd landed in node:zlib in 22.15 / 23.8; the package floor is lower, so the
// provider degrades with a notice instead of assuming the export exists.
const zstdDecompress = (zlib as { zstdDecompressSync?: (buf: Buffer, opts?: { maxOutputLength?: number }) => Buffer }).zstdDecompressSync
const ZSTD_MAGIC = 0xfd2fb528
// SESSION_FORMAT_VERSION in @deepseek-ai/dsh-session. DSH refuses to load a log
// stamped with any other version, and a bump means an event's meaning changed,
// so a foreign version is skipped rather than read with today's assumptions.
// A zstd frame's declared content size is attacker-controlled, so a few KB of
// crafted input can expand to gigabytes. Every decode is capped: no single
// frame may exceed this, and no file may decode to more than it would have been
// allowed to occupy uncompressed (MAX_SESSION_FILE_BYTES). Overflow throws, and
// the caller skips the WHOLE file rather than counting the frames it got to.
const MAX_FRAME_DECODED_BYTES = 64 * 1024 * 1024
const SESSION_FORMAT_VERSION = 0
const MIN_REASONABLE_TIMESTAMP_MS = 1_000_000_000_000
// Discovery walks every session, so a per-file notice would repeat once per
// log; each distinct message is worth saying exactly once.
const noticed = new Set<string>()
function notice(message: string): void {
if (noticed.has(message)) return
noticed.add(message)
process.stderr.write(message)
}
type ZstdFrame = { start: number; end: number }
// Locate complete frames without decompressing their blocks. An EOF inside the
// final frame (a torn append from a crashed writer) returns its start so the
// caller can ignore the tail; invalid complete structure rejects.
function scanZstdFrames(buffer: Buffer, maxFrames = Number.POSITIVE_INFINITY): { frames: ZstdFrame[]; tornStart?: number } {
const frames: ZstdFrame[] = []
let offset = 0
while (offset < buffer.length) {
const start = offset
if (buffer.length - offset < 4) return { frames, tornStart: start }
if (buffer.readUInt32LE(offset) !== ZSTD_MAGIC) {
throw new Error(`invalid zstd frame magic at byte ${offset}`)
}
offset += 4
if (offset === buffer.length) return { frames, tornStart: start }
const descriptor = buffer.readUInt8(offset)!
offset += 1
if ((descriptor & 24) !== 0) throw new Error(`reserved frame-header bit at byte ${offset - 1}`)
const contentSizeFlag = descriptor >>> 6
const singleSegment = (descriptor & 32) !== 0
const checksum = (descriptor & 4) !== 0
const dictionaryFlag = descriptor & 3
const dictionaryBytes = dictionaryFlag === 3 ? 4 : dictionaryFlag
const contentSizeBytes = contentSizeFlag === 0 ? (singleSegment ? 1 : 0) : 1 << contentSizeFlag
const remainingHeaderBytes = (singleSegment ? 0 : 1) + dictionaryBytes + contentSizeBytes
if (buffer.length - offset < remainingHeaderBytes) return { frames, tornStart: start }
offset += remainingHeaderBytes
for (;;) {
if (buffer.length - offset < 3) return { frames, tornStart: start }
const blockHeader = buffer.readUIntLE(offset, 3)
offset += 3
const lastBlock = (blockHeader & 1) !== 0
const blockType = (blockHeader >>> 1) & 3
const blockSize = blockHeader >>> 3
if (blockType === 3) throw new Error(`reserved block type at byte ${offset - 3}`)
const payloadBytes = blockType === 1 ? 1 : blockSize
if (buffer.length - offset < payloadBytes) return { frames, tornStart: start }
offset += payloadBytes
if (lastBlock) break
}
if (checksum) {
if (buffer.length - offset < 4) return { frames, tornStart: start }
offset += 4
}
frames.push({ start, end: offset })
if (frames.length === maxFrames) return { frames }
}
return { frames }
}
type DshUsage = {
inputTokens?: number
outputTokens?: number
cacheReadTokens?: number
cacheWriteTokens?: number
reasoningTokens?: number
}
type DshEvent = {
type?: string
seq?: number
time?: number
// Session header fields live at the top level of the first event.
version?: number
id?: string
cwd?: string
createdAt?: number
parentSession?: string
seedLength?: number
data?: {
turn?: number
step?: number
content?: Array<{ type?: string; text?: string }>
// `user/message` carries the message author: a real prompt is
// `{ kind: 'user' }`, agent-injected context is `{ kind: 'plugin' }`.
source?: { kind?: string }
header?: { config?: { model?: string; provider?: string } }
message?: { source?: { kind?: string; model?: string; provider?: string } }
chunk?: { type?: string; usage?: DshUsage }
usage?: DshUsage
name?: string
arguments?: string
}
}
type StepBucket = {
usage: DshUsage
// A usage report from assistant/message is the final value for its
// (turn, step) and replaces an earlier assistant/chunk sample (the two are
// adjacent reports of the same API call, per dsh-token-meter's usage
// projection). Time follows the winning report.
final: boolean
time?: number
// Model that produced this step: the reporting assistant/message's own
// `message.source` when it names one, else the most recent request/header
// config (a header can change the model mid-turn between steps).
model: string
tools: string[]
skills: string[]
bashCommands: string[]
}
const toolNameMap: Record<string, string> = {
bash: 'Bash',
pwsh: 'Bash',
read: 'Read',
write: 'Write',
edit: 'Edit',
str_replace_editor: 'Edit',
glob: 'Glob',
grep: 'Grep',
todo_write: 'TodoWrite',
todo: 'TodoWrite',
web_search: 'WebSearch',
skill: 'Skill',
agent: 'Agent',
ask_user_question: 'AskUserQuestion',
}
function mapToolName(raw: string): string {
return toolNameMap[raw] ?? raw
}
// Usage fields are whatever the JSON held. A string or array would flow
// straight into the global token totals and the persisted cache, where
// `0 + [1, 2]` silently becomes "01,2". Same semantics as copilot.ts.
function numberOrZero(raw: unknown): number {
return typeof raw === 'number' && Number.isFinite(raw) && raw > 0 ? raw : 0
}
// A log stamped with a version this parser was not written against is skipped
// whole: a bump means an event's meaning changed, so reading it with today's
// assumptions would report confident wrong numbers.
function isReadableVersion(header: DshEvent): boolean {
if (header.version === SESSION_FORMAT_VERSION) return true
// Keyed on the version, not the path: a DSH upgrade makes EVERY session
// unreadable at once, and one line per session log is noise, not a report.
notice(`codeburn: skipping DSH sessions written in session format version ${String(header.version)}; upgrade codeburn.\n`)
return false
}
// DSH writes epoch milliseconds; promote a seconds-resolution value and reject
// what stays implausible, matching the guard cline-cli.ts uses on the hazard.
function isoTimestamp(value: number | undefined, fallback: string): string {
if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) return fallback
const ms = value < MIN_REASONABLE_TIMESTAMP_MS ? value * 1000 : value
const date = new Date(ms)
if (Number.isNaN(date.getTime()) || date.getTime() < MIN_REASONABLE_TIMESTAMP_MS) return fallback
return date.toISOString()
}
function getDshHome(override?: string): string {
// An empty-string DSH_HOME is treated as unset.
return override ?? (process.env['DSH_HOME'] || undefined) ?? join(homedir(), '.dsh')
}
// DSH writes native-platform paths into the header (backslashes on Windows);
// split on both separators so discovery is correct on any host.
function projectFromCwd(cwd: string, fallback: string): string {
const segments = cwd.split(/[\\/]/).filter(Boolean)
return segments[segments.length - 1] ?? fallback
}
// Decode every complete frame and yield its JSONL lines. A torn final frame is
// ignored; a structurally corrupt file, or one that decodes past `budget`,
// throws for the caller to report. Exported for the decode-budget test.
export function* readZstdLines(
buffer: Buffer,
maxFrames = Number.POSITIVE_INFINITY,
budget = MAX_SESSION_FILE_BYTES,
): Generator<string> {
const { frames } = scanZstdFrames(buffer, maxFrames)
let remaining = budget
for (const frame of frames) {
if (remaining <= 0) throw new Error(`decodes past the ${budget}-byte cap`)
// node throws ERR_BUFFER_TOO_LARGE without allocating past the cap, so the
// per-frame limit doubles as the running budget for the frames after it.
const decoded = zstdDecompress!(buffer.subarray(frame.start, frame.end), {
maxOutputLength: Math.min(remaining, MAX_FRAME_DECODED_BYTES),
})
remaining -= decoded.length
for (const line of decoded.toString('utf-8').split('\n')) {
if (line.trim()) yield line
}
}
}
async function readEventLines(filePath: string): Promise<string[] | null> {
if (filePath.endsWith('.zstd')) {
if (!zstdDecompress) {
notice('codeburn: DSH sessions need Node >= 22.15 (zstd support); skipping DSH usage.\n')
return null
}
let buffer: Buffer
try {
// The whole log is buffered to scan its frames, so it needs the same
// oversize guard readSessionFile applies to the uncompressed variant.
const size = (await stat(filePath)).size
if (size > MAX_SESSION_FILE_BYTES) {
notice(`codeburn: skipped oversize DSH session log ${filePath} (${size} bytes)\n`)
return null
}
buffer = await readFile(filePath)
} catch {
return null
}
try {
return [...readZstdLines(buffer)]
} catch (err) {
notice(`codeburn: skipped corrupt DSH session log ${filePath}: ${err instanceof Error ? err.message : err}\n`)
return null
}
}
const content = await readSessionFile(filePath)
if (content === null) return null
return content.split('\n').filter(l => l.trim())
}
// Cheap discovery probe: decompress ONLY the first frame (the session header
// batch) instead of the whole log. The header frame is tiny, so a bounded head
// read almost always contains it; fall back to a full read when it does not.
async function readSessionHeader(filePath: string): Promise<DshEvent | null> {
const firstLine = async (): Promise<string | null> => {
if (filePath.endsWith('.zstd')) {
if (!zstdDecompress) return null
let head: Buffer
try {
const handle = await open(filePath, 'r')
try {
const size = (await handle.stat()).size
const length = Math.min(size, 256 * 1024)
head = Buffer.alloc(length)
await handle.read(head, 0, length, 0)
} finally {
await handle.close()
}
} catch {
return null
}
let { frames } = scanZstdFrames(head, 1)
if (frames.length === 0) {
// Head read did not cover one full frame; take the whole file. A fork's
// first batch carries the whole inherited seed, so this is reachable on
// a real log and needs the same oversize guard as the parse read.
try {
if ((await stat(filePath)).size > MAX_SESSION_FILE_BYTES) return null
const full = await readFile(filePath)
frames = scanZstdFrames(full, 1).frames
if (frames.length === 0) return null
head = full
} catch {
return null
}
}
const text = zstdDecompress(head.subarray(frames[0]!.start, frames[0]!.end), {
maxOutputLength: MAX_FRAME_DECODED_BYTES,
}).toString('utf-8')
return text.split('\n').find(l => l.trim()) ?? null
}
const content = await readSessionFile(filePath)
return content?.split('\n').find(l => l.trim()) ?? null
}
try {
const line = await firstLine()
if (!line) return null
const event = JSON.parse(line) as DshEvent
if (event.type !== 'session') return null
return isReadableVersion(event) ? event : null
} catch {
return null
}
}
async function discoverSessionsInDir(sessionsDir: string): Promise<SessionSource[]> {
const sources: SessionSource[] = []
let projectDirs: string[]
try {
projectDirs = await readdir(sessionsDir)
} catch {
return sources
}
for (const dirName of projectDirs) {
const dirPath = join(sessionsDir, dirName)
const dirStat = await stat(dirPath).catch(() => null)
if (!dirStat?.isDirectory()) continue
let sessionDirs: string[]
try {
sessionDirs = await readdir(dirPath)
} catch {
continue
}
for (const sessionDir of sessionDirs) {
const sessionPath = join(dirPath, sessionDir)
const sessionStat = await stat(sessionPath).catch(() => null)
if (!sessionStat?.isDirectory()) continue
// Compressed log first; the uncompressed variant exists when
// compression=none. Never both for the same session.
let filePath: string | null = null
for (const name of ['session.jsonl.zstd', 'session.jsonl']) {
const candidate = join(sessionPath, name)
const fileStat = await stat(candidate).catch(() => null)
if (fileStat?.isFile()) {
filePath = candidate
break
}
}
if (!filePath) continue
const header = await readSessionHeader(filePath)
if (!header) continue
const cwd = typeof header.cwd === 'string' && header.cwd.trim() ? header.cwd : dirName
sources.push({ path: filePath, project: projectFromCwd(cwd, dirName), provider: 'dsh' })
}
}
return sources
}
function parseToolArguments(raw: string | undefined): Record<string, unknown> | null {
if (!raw) return null
try {
const parsed = JSON.parse(raw) as unknown
return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed as Record<string, unknown> : null
} catch {
return null
}
}
function createParser(source: SessionSource, seenKeys: Set<string>): SessionParser {
return {
async *parse(): AsyncGenerator<ParsedProviderCall> {
const lines = await readEventLines(source.path)
if (!lines) return
let sessionId = ''
let cwd = ''
let model = 'unknown'
let currentTurn = 0
let sessionStart = ''
// Events a forked session inherited from its parent. They are a verbatim
// copy of the parent's log, which codeburn parses as its own session, so
// counting them here would bill the same calls twice.
let seedLength = 0
const userMessageByTurn = new Map<number, string>()
const buckets = new Map<string, StepBucket>()
for (const line of lines) {
let event: DshEvent
try {
event = JSON.parse(line) as DshEvent
} catch {
continue
}
if (event.type === 'session') {
if (!isReadableVersion(event)) return
sessionId = event.id ?? sessionId
cwd = event.cwd ?? cwd
sessionStart = isoTimestamp(event.createdAt, sessionStart)
if (typeof event.parentSession === 'string' && event.parentSession && typeof event.seedLength === 'number') {
seedLength = event.seedLength
}
continue
}
if (typeof event.seq === 'number' && event.seq < seedLength) continue
if (event.type === 'turn/start') {
currentTurn = event.data?.turn ?? currentTurn
continue
}
if (event.type === 'request/header') {
// Emitted at most once per request; steps after the last header
// inherit its config as their model.
const headerModel = event.data?.header?.config?.model
if (typeof headerModel === 'string' && headerModel) model = headerModel
continue
}
if (event.type === 'user/message') {
// Plugin-injected context (runtime snapshots, skill bodies, file-change
// notices) rides the same event type as a typed prompt; only the latter
// is a useful preview.
if (event.data?.source?.kind !== 'user') continue
if (userMessageByTurn.has(currentTurn)) continue
const texts = (event.data?.content ?? [])
.filter(c => c.type === 'text' && typeof c.text === 'string' && c.text)
.map(c => c.text!)
if (texts.length > 0) userMessageByTurn.set(currentTurn, texts.join(' ').slice(0, 500))
continue
}
if (event.type === 'tool/call') {
const turn = event.data?.turn ?? currentTurn
const step = event.data?.step ?? 0
const rawName = event.data?.name
if (!rawName) continue
const key = `${turn}:${step}`
let bucket = buckets.get(key)
if (!bucket) {
bucket = { usage: {}, final: false, model, tools: [], skills: [], bashCommands: [] }
buckets.set(key, bucket)
}
bucket.tools.push(mapToolName(rawName))
const args = parseToolArguments(event.data?.arguments)
if ((rawName === 'bash' || rawName === 'pwsh') && typeof args?.['command'] === 'string') {
bucket.bashCommands.push(...extractBashCommands(args['command']))
}
if (rawName === 'skill' && typeof args?.['name'] === 'string') {
bucket.skills.push(args['name'])
}
continue
}
let usage: DshUsage | undefined
let isFinal = false
// The model that actually served the call, when the message records it.
// request/header only describes the request codeburn is about to see.
let reportedModel = model
if (event.type === 'assistant/chunk' && event.data?.chunk?.type === 'usage') {
usage = event.data.chunk.usage
} else if (event.type === 'assistant/message' && event.data?.usage) {
usage = event.data.usage
isFinal = true
const messageModel = event.data.message?.source?.model
if (typeof messageModel === 'string' && messageModel) reportedModel = messageModel
} else {
continue
}
if (!usage) continue
const turn = event.data?.turn ?? currentTurn
const step = event.data?.step ?? 0
const key = `${turn}:${step}`
let bucket = buckets.get(key)
if (!bucket) {
bucket = { usage: {}, final: false, model, tools: [], skills: [], bashCommands: [] }
buckets.set(key, bucket)
}
// A final report replaces an earlier sample; a late sample never
// overwrites a final one. The model snapshot follows the winning
// report (a header can change the model mid-turn between steps).
if (isFinal || !bucket.final) {
bucket.usage = usage
bucket.final = isFinal
bucket.time = event.time
bucket.model = reportedModel
}
}
const sortedKeys = [...buckets.keys()].sort((a, b) => {
const [ta, sa] = a.split(':').map(Number)
const [tb, sb] = b.split(':').map(Number)
return ta! - tb! || sa! - sb!
})
for (const key of sortedKeys) {
const bucket = buckets.get(key)!
const input = numberOrZero(bucket.usage.inputTokens)
const output = numberOrZero(bucket.usage.outputTokens)
const cacheRead = numberOrZero(bucket.usage.cacheReadTokens)
const cacheWrite = numberOrZero(bucket.usage.cacheWriteTokens)
const reasoning = numberOrZero(bucket.usage.reasoningTokens)
if (input + output + cacheRead + cacheWrite + reasoning === 0) continue
const dedupKey = `dsh:${sessionId || source.path}:${key}`
if (seenKeys.has(dedupKey)) continue
seenKeys.add(dedupKey)
// DSH bills reasoning tokens at the output rate (same as Gemini).
const costUSD = calculateCost(bucket.model, input, output + reasoning, cacheWrite, cacheRead, 0)
const [turn] = key.split(':').map(Number)
yield {
provider: 'dsh',
model: bucket.model,
inputTokens: input,
outputTokens: output,
cacheCreationInputTokens: cacheWrite,
cacheReadInputTokens: cacheRead,
cachedInputTokens: cacheRead,
reasoningTokens: reasoning,
webSearchRequests: 0,
costUSD,
tools: [...new Set(bucket.tools)],
bashCommands: bucket.bashCommands,
skills: bucket.skills.length > 0 ? [...new Set(bucket.skills)] : undefined,
timestamp: isoTimestamp(bucket.time, sessionStart),
speed: 'standard',
deduplicationKey: dedupKey,
userMessage: userMessageByTurn.get(turn!) ?? '',
sessionId: sessionId || source.path,
project: cwd ? projectFromCwd(cwd, source.project) : source.project,
projectPath: cwd || undefined,
workingDirectory: cwd || undefined,
}
}
},
}
}
export function createDshProvider(dshHomeOverride?: string): Provider {
const dshHome = getDshHome(dshHomeOverride)
const sessionsDir = join(dshHome, 'sessions')
return {
name: 'dsh',
displayName: 'DeepSeek Harness',
modelDisplayName(model: string): string {
return getShortModelName(model)
},
toolDisplayName(rawTool: string): string {
return mapToolName(rawTool)
},
async probeRoots(): Promise<ProbeRoot[]> {
return [{ path: sessionsDir, label: 'sessions' }]
},
async discoverSessions(): Promise<SessionSource[]> {
return discoverSessionsInDir(sessionsDir)
},
createSessionParser(source: SessionSource, seenKeys: Set<string>): SessionParser {
return createParser(source, seenKeys)
},
}
}
export const dsh = createDshProvider()

View file

@ -7,6 +7,7 @@ import { codex } from './codex.js'
import { copilot } from './copilot.js'
import { droid } from './droid.js'
import { devin } from './devin.js'
import { dsh } from './dsh.js'
import { gemini } from './gemini.js'
import { hermes } from './hermes.js'
import { ibmBob } from './ibm-bob.js'
@ -192,7 +193,7 @@ async function loadZed(): Promise<Provider | null> {
}
}
const coreProviders: Provider[] = [claude, cline, clineCli, codewhale, codebuff, codex, copilot, devin, droid, gemini, hermes, ibmBob, kiloCode, kiro, kimi, kimicode, lingtaiTui, mistralVibe, mux, openclaw, openclaude, openDesign, pi, omp, qwen, quickdesk, rooCode, zerostack, grok]
const coreProviders: Provider[] = [claude, cline, clineCli, codewhale, codebuff, codex, copilot, devin, droid, dsh, gemini, hermes, ibmBob, kiloCode, kiro, kimi, kimicode, lingtaiTui, mistralVibe, mux, openclaw, openclaude, openDesign, pi, omp, qwen, quickdesk, rooCode, zerostack, grok]
// Lazily loaded providers, listed by name so --provider validation works even
// when an optional module fails to load. Must stay in sync with getAllProviders.

View file

@ -216,6 +216,7 @@ export const PROVIDER_ENV_VARS: Record<string, string[]> = {
hermes: ['HERMES_HOME'],
'lingtai-tui': ['LINGTAI_HOME', 'LINGTAI_TUI_HOME', 'LINGTAI_TUI_GLOBAL_DIR'],
droid: ['FACTORY_DIR'],
dsh: ['DSH_HOME'],
cursor: ['CODEBURN_CURSOR_MAX_BUBBLES'],
// XDG_DATA_HOME is stale here (cursor-agent never reads it) but deliberately
// kept: removing it would force a re-parse to fix nothing.
@ -284,6 +285,10 @@ export const PROVIDER_PARSE_VERSIONS: Record<string, string> = {
// input/cache rollup; this bump re-parses them so the missing tokens land.
copilot: 'cli-shutdown-cost-v1-skills-source-provenance-v1',
grok: 'estimated-cost-v1',
// seed-aware-v1: the parser now skips the parent events a forked session
// replays (double-counted before), takes the model from the reporting
// assistant/message, and keeps agent-injected context out of the preview.
dsh: 'seed-aware-v1',
hermes: 'reasoning-output-accounting-v1-est-cost',
'lingtai-tui': 'token-ledger-registry-activity-v3',
'ibm-bob': 'worktree-project-grouping-v1',

35
tests/fixtures/dsh/bash-tool-turn.jsonl vendored Normal file
View file

@ -0,0 +1,35 @@
{"type":"session","version":0,"id":"e128dda9-ed11-4868-8266-0ef90d03c3d6","createdAt":1783352050748,"cwd":"/home/u/proj","delegationDepth":0}
{"type":"agent/inbox/spliced","seq":0,"time":1785498771334,"data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Use the bash tool to run exactly: echo TERMINAL_OK. Then reply with the single word DONE and stop."}],"source":{"kind":"user"},"role":"user","id":"38694db6-921d-41fd-b1fb-3b0c40caf67c"}]}}
{"type":"turn/start","seq":1,"time":1785821375023,"data":{"turn":1}}
{"type":"agent/inbox/spliced","seq":2,"time":1785821375023,"data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}}
{"type":"step/start","seq":3,"time":1783352050755,"data":{"turn":1,"step":1}}
{"type":"user/message","seq":4,"time":1785498771360,"data":{"content":[{"type":"text","text":"Use the bash tool to run exactly: echo TERMINAL_OK. Then reply with the single word DONE and stop."}],"source":{"kind":"user"},"role":"user","id":"38694db6-921d-41fd-b1fb-3b0c40caf67c"},"surfaceOp":"append"}
{"type":"user/message","seq":5,"time":1785730424635,"data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"80474489-442a-4e98-beef-df6cd1e85870"},"surfaceOp":"append"}
{"type":"session/title","seq":6,"time":1785730424635,"data":{"title":"Use the bash tool to","messageSeqs":[4],"source":{"kind":"fallback"}}}
{"type":"request/header","seq":7,"time":1785498771361,"data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash"},"system":"you are dsh","tools":[]},"reason":"initial"}}
{"type":"request/context","seq":8,"time":1785730424636,"data":{"provider":"deepseek-official","model":"deepseek-v4-flash"}}
{"type":"assistant/chunk","seq":9,"time":1783352051590,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}}
{"type":"reasoning-chunks","seq0":10,"time0":1783352051618,"data":{"turn":1,"step":1,"index":0,"dt":[0,1,0,0,26,30,0,0,1,0,27,1,0,0,0,86,1],"texts":["The"," user"," wants"," me"," to"," run"," a"," simple"," bash"," command"," and"," then"," reply"," with"," \"","D","ONE","\"."]}}
{"type":"assistant/chunk","seq":28,"time":1783352051820,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"tool-call"}}}
{"type":"tool-call-chunks","seq0":29,"time0":1783352051820,"data":{"turn":1,"step":1,"index":1,"dt":[0,0,0,28,0,0,0,29,0,0,28,1,0,29,0,0,0,32,0,0,0,0,0,74,0,0,13,0,63,1],"id":"call_00_fkbBRJsUrGKd1pWVc4Gn8233","name":"bash","args":["","{","\"","command","\"",": ","\"","echo"," TER","MIN","AL","_OK","\"",", ","\"","description","\"",": ","\"","E","cho"," TER","MIN","AL","_OK"," to"," verify"," terminal"," access","\"","}"]}}
{"type":"assistant/chunk","seq":60,"time":1783352052118,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The user wants me to run a simple bash command and then reply with \"DONE\"."}}}}
{"type":"assistant/chunk","seq":61,"time":1783352052118,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_00_fkbBRJsUrGKd1pWVc4Gn8233","name":"bash","arguments":"{\"command\": \"echo TERMINAL_OK\", \"description\": \"Echo TERMINAL_OK to verify terminal access\"}"}}}}
{"type":"assistant/chunk","seq":62,"time":1785498771373,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":2877,"outputTokens":90,"cacheReadTokens":0,"reasoningTokens":18}}}}
{"type":"assistant/chunk","seq":63,"time":1785730424645,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
{"type":"assistant/message","seq":64,"time":1785730424645,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"reasoning","text":"The user wants me to run a simple bash command and then reply with \"DONE\"."},{"type":"tool-call","id":"call_00_fkbBRJsUrGKd1pWVc4Gn8233","name":"bash","arguments":"{\"command\": \"echo TERMINAL_OK\", \"description\": \"Echo TERMINAL_OK to verify terminal access\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"0a855246-fbf6-4f91-87b4-c6f1889effe7"},"usage":{"inputTokens":2877,"outputTokens":90,"cacheReadTokens":0,"reasoningTokens":18}},"sourceEventSeqs":[9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63],"surfaceOp":"append"}
{"type":"tool/call","seq":65,"time":1785730424646,"data":{"turn":1,"step":1,"callId":"call_00_fkbBRJsUrGKd1pWVc4Gn8233","name":"bash","arguments":"{\"command\": \"echo TERMINAL_OK\", \"description\": \"Echo TERMINAL_OK to verify terminal access\"}"}}
{"type":"tool/result","seq":66,"time":1785730424665,"data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"call_00_fkbBRJsUrGKd1pWVc4Gn8233"},"content":[{"type":"tool-result","toolCallId":"call_00_fkbBRJsUrGKd1pWVc4Gn8233","content":[{"type":"text","text":"TERMINAL_OK\n"}],"isError":false}],"role":"user","id":"908ca4f5-efbb-443b-9b07-acbf25edf954"}},"sourceEventSeqs":[65],"surfaceOp":"append"}
{"type":"step/end","seq":67,"time":1785730424665,"data":{"turn":1,"step":1}}
{"type":"step/start","seq":68,"time":1785730424676,"data":{"turn":1,"step":2}}
{"type":"assistant/chunk","seq":69,"time":1783352052780,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}}
{"type":"reasoning-chunks","seq0":70,"time0":1783352052809,"data":{"turn":1,"step":2,"index":0,"dt":[29,0,0,29,0,0,0,0,0,28,1,28,1,0,0,32,0,0,0,0,0],"texts":["The"," command"," ran"," successfully"," and"," output"," \"","TER","MIN","AL","_OK","\"."," I"," should"," now"," reply"," with"," just"," \"","D","ONE","\"."]}}
{"type":"assistant/chunk","seq":92,"time":1783352052986,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":1,"blockType":"text"}}}
{"type":"assistant/chunk","seq":93,"time":1783352052986,"data":{"turn":1,"step":2,"chunk":{"type":"text-delta","index":1,"text":"D"}}}
{"type":"assistant/chunk","seq":94,"time":1783352052986,"data":{"turn":1,"step":2,"chunk":{"type":"text-delta","index":1,"text":"ONE"}}}
{"type":"assistant/chunk","seq":95,"time":1783352052986,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The command ran successfully and output \"TERMINAL_OK\". I should now reply with just \"DONE\"."}}}}
{"type":"assistant/chunk","seq":96,"time":1783352052986,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":1,"block":{"type":"text","text":"DONE"}}}}
{"type":"assistant/chunk","seq":97,"time":1785498771406,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":168,"outputTokens":25,"cacheReadTokens":2816,"reasoningTokens":22}}}}
{"type":"assistant/chunk","seq":98,"time":1785730424681,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
{"type":"assistant/message","seq":99,"time":1785730424681,"data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"reasoning","text":"The command ran successfully and output \"TERMINAL_OK\". I should now reply with just \"DONE\"."},{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"aa705bf0-9b5b-4af3-9763-dbf93c98e4c4"},"usage":{"inputTokens":168,"outputTokens":25,"cacheReadTokens":2816,"reasoningTokens":22}},"sourceEventSeqs":[69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98],"surfaceOp":"append"}
{"type":"step/end","seq":100,"time":1785730424682,"data":{"turn":1,"step":2}}
{"type":"turn/end","seq":101,"time":1785730424682,"data":{"turn":1,"reason":{"kind":"completed"}}}

View file

@ -34,6 +34,7 @@ const FILE_PROVIDERS: Record<string, string[]> = {
'codex.ts': ['codex'],
'copilot.ts': ['copilot'],
'droid.ts': ['droid'],
'dsh.ts': ['dsh'],
'hermes.ts': ['hermes'],
'lingtai-tui.ts': ['lingtai-tui'],
// Its only literal read is CODEBURN_CURSOR_MAX_BUBBLES (cursor.ts:692).

View file

@ -14,7 +14,7 @@ function fakeProvider(name: string, discover: Provider['discoverSessions']): Pro
describe('provider registry', () => {
it('has core providers registered synchronously', () => {
expect(providers.map(p => p.name)).toEqual(['claude', 'cline', 'cline-cli', 'codewhale', 'codebuff', 'codex', 'copilot', 'devin', 'droid', 'gemini', 'hermes', 'ibm-bob', 'kilo-code', 'kiro', 'kimi', 'kimicode', 'lingtai-tui', 'mistral-vibe', 'mux', 'openclaw', 'openclaude', 'open-design', 'pi', 'omp', 'qwen', 'quickdesk', 'roo-code', 'zerostack', 'grok'])
expect(providers.map(p => p.name)).toEqual(['claude', 'cline', 'cline-cli', 'codewhale', 'codebuff', 'codex', 'copilot', 'devin', 'droid', 'dsh', 'gemini', 'hermes', 'ibm-bob', 'kilo-code', 'kiro', 'kimi', 'kimicode', 'lingtai-tui', 'mistral-vibe', 'mux', 'openclaw', 'openclaude', 'open-design', 'pi', 'omp', 'qwen', 'quickdesk', 'roo-code', 'zerostack', 'grok'])
})
it('codebuff tool display names normalize codebuff-native names to canonical set', () => {

620
tests/providers/dsh.test.ts Normal file
View file

@ -0,0 +1,620 @@
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
import { mkdtemp, mkdir, writeFile, readFile, rm, stat } from 'fs/promises'
import { join } from 'path'
import { homedir, tmpdir } from 'os'
import zlib from 'zlib'
import { createDshProvider, readZstdLines } from '../../src/providers/dsh.js'
import { calculateCost } from '../../src/models.js'
import type { ParsedProviderCall } from '../../src/providers/types.js'
// DSH session logs are concatenations of INDEPENDENT zstd frames (one per
// appended event batch), so fixtures must compress each batch separately —
// a single zstdCompressSync over the whole file is a different (single-frame)
// format than what DSH writes.
const zstdCompress = (zlib as { zstdCompressSync?: (buf: Buffer) => Buffer }).zstdCompressSync
// node:zlib gained zstd in 22.15; the package floor (and CI's pinned Node) is
// 22.13. Container-specific tests skip there; the rest fall back to plain jsonl
// so the parsing semantics are still exercised.
const itZstd = zstdCompress ? it : it.skip
let tmpDir: string
beforeEach(async () => {
tmpDir = await mkdtemp(join(tmpdir(), 'dsh-test-'))
})
afterEach(async () => {
await rm(tmpDir, { recursive: true, force: true })
})
function sessionHeader(opts: { id?: string; cwd?: string } = {}) {
return JSON.stringify({
type: 'session',
version: 0,
id: opts.id ?? 'session-00000000-0000-0000-0000-000000000001',
createdAt: 1786707336131,
cwd: opts.cwd ?? 'C:\\Users\\test\\myproject',
delegationDepth: 0,
agentPreset: 'cordis',
})
}
function requestHeader(model: string, time = 1786707337000) {
return JSON.stringify({
type: 'request/header',
seq: 10,
time,
data: { header: { config: { provider: 'deepseek-official', model, reasoningEffort: 'max', maxTokens: 256000 } } },
})
}
function turnStart(turn: number, time: number) {
return JSON.stringify({ type: 'turn/start', seq: 1, time, data: { turn } })
}
function userMessage(text: string, time: number) {
return JSON.stringify({
type: 'user/message',
seq: 2,
time,
data: { content: [{ type: 'text', text }], source: { kind: 'user' }, role: 'user', id: 'msg-1' },
})
}
function chunkUsage(turn: number, step: number, usage: Record<string, number>, time: number) {
return JSON.stringify({
type: 'assistant/chunk',
seq: 3,
time,
data: { turn, step, chunk: { type: 'usage', usage } },
})
}
function assistantMessage(turn: number, step: number, usage: Record<string, number> | undefined, time: number) {
return JSON.stringify({
type: 'assistant/message',
seq: 4,
time,
data: {
turn,
step,
message: { role: 'assistant', content: [{ type: 'text', text: 'done' }] },
...(usage ? { usage } : {}),
},
})
}
function toolCall(turn: number, step: number, name: string, args: Record<string, unknown>, time: number) {
return JSON.stringify({
type: 'tool/call',
seq: 5,
time,
data: { turn, step, callId: `call_${name}`, name, arguments: JSON.stringify(args) },
})
}
// Write one frame per batch of lines, matching DSH's append-per-batch layout.
async function writeZstdSession(projectDirName: string, sessionDirName: string, batches: string[][]) {
const dir = join(tmpDir, 'sessions', projectDirName, sessionDirName)
await mkdir(dir, { recursive: true })
if (!zstdCompress) {
const filePath = join(dir, 'session.jsonl')
await writeFile(filePath, batches.map(lines => lines.join('\n') + '\n').join(''))
return filePath
}
const filePath = join(dir, 'session.jsonl.zstd')
const frames = batches.map(lines => zstdCompress(Buffer.from(lines.join('\n') + '\n', 'utf-8')))
await writeFile(filePath, Buffer.concat(frames))
return filePath
}
async function writePlainSession(projectDirName: string, sessionDirName: string, lines: string[]) {
const dir = join(tmpDir, 'sessions', projectDirName, sessionDirName)
await mkdir(dir, { recursive: true })
const filePath = join(dir, 'session.jsonl')
await writeFile(filePath, lines.join('\n') + '\n')
return filePath
}
async function parseAll(provider: ReturnType<typeof createDshProvider>, filePath: string): Promise<ParsedProviderCall[]> {
const source = { path: filePath, project: 'myproject', provider: 'dsh' }
const calls: ParsedProviderCall[] = []
for await (const call of provider.createSessionParser(source, new Set()).parse()) {
calls.push(call)
}
return calls
}
describe('dsh provider - session discovery', () => {
itZstd('discovers a multi-frame zstd session, project from the header cwd', async () => {
await writeZstdSession('--C-Users-test-myproject--', 'session-abc', [
[sessionHeader({ cwd: 'C:\\Users\\test\\myproject' })],
[assistantMessage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000)],
])
const provider = createDshProvider(tmpDir)
const sessions = await provider.discoverSessions()
expect(sessions).toHaveLength(1)
expect(sessions[0]!.provider).toBe('dsh')
expect(sessions[0]!.project).toBe('myproject')
expect(sessions[0]!.path).toContain('session.jsonl.zstd')
})
it('discovers the uncompressed session.jsonl variant (compression=none)', async () => {
await writePlainSession('--home-u-proj--', 'session-plain', [
sessionHeader({ cwd: '/home/u/proj' }),
assistantMessage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000),
])
const provider = createDshProvider(tmpDir)
const sessions = await provider.discoverSessions()
expect(sessions).toHaveLength(1)
expect(sessions[0]!.path).toContain('session.jsonl')
expect(sessions[0]!.path).not.toContain('zstd')
expect(sessions[0]!.project).toBe('proj')
})
it('returns empty for a non-existent home', async () => {
const provider = createDshProvider('/nonexistent/dsh/home')
expect(await provider.discoverSessions()).toEqual([])
})
it('skips session dirs without a session log', async () => {
await mkdir(join(tmpDir, 'sessions', '--x--', 'session-empty'), { recursive: true })
const provider = createDshProvider(tmpDir)
expect(await provider.discoverSessions()).toEqual([])
})
it('DSH_HOME relocates discovery; an empty string is treated as unset', async () => {
const home = join(tmpDir, 'dsh-home')
await mkdir(join(home, 'sessions', '--x--', 'session-env'), { recursive: true })
await writeFile(
join(home, 'sessions', '--x--', 'session-env', 'session.jsonl'),
sessionHeader({ cwd: '/x' }) + '\n',
)
const saved = process.env['DSH_HOME']
process.env['DSH_HOME'] = home
try {
const sessions = await createDshProvider().discoverSessions()
expect(sessions).toHaveLength(1)
} finally {
if (saved === undefined) delete process.env['DSH_HOME']
else process.env['DSH_HOME'] = saved
}
process.env['DSH_HOME'] = ''
try {
const roots = await createDshProvider().probeRoots!()
expect(roots).toEqual([{ path: join(homedir(), '.dsh', 'sessions'), label: 'sessions' }])
} finally {
if (saved === undefined) delete process.env['DSH_HOME']
else process.env['DSH_HOME'] = saved
}
})
it('probeRoots reports the sessions dir under the factory root', async () => {
expect(await createDshProvider('/tmp/dsh-a').probeRoots!()).toEqual([
{ path: join('/tmp/dsh-a', 'sessions'), label: 'sessions' },
])
})
})
describe('dsh provider - parsing', () => {
itZstd('decodes events spread across multiple independent zstd frames', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-multi', [
[sessionHeader({ id: 'session-multi', cwd: 'C:\\Users\\test\\myproject' })],
[turnStart(1, 1786707339000), userMessage('build the thing', 1786707339100)],
[chunkUsage(1, 1, { inputTokens: 500, outputTokens: 50 }, 1786707340000)],
[chunkUsage(1, 2, { inputTokens: 800, outputTokens: 80 }, 1786707341000)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(2)
expect(calls[0]!.inputTokens).toBe(500)
expect(calls[1]!.inputTokens).toBe(800)
})
it('a final assistant/message usage REPLACES the earlier chunk sample for the same turn/step', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-replace', [
[sessionHeader({ id: 'session-replace' })],
[turnStart(1, 1786707339000)],
// Early sample, then the final report of the SAME API call: the totals
// must come from the final report only, not the sum of both.
[chunkUsage(1, 1, { inputTokens: 14900, outputTokens: 600, reasoningTokens: 500 }, 1786707340000)],
[assistantMessage(1, 1, { inputTokens: 14981, outputTokens: 656, cacheReadTokens: 0, reasoningTokens: 609 }, 1786707340050)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(14981)
expect(calls[0]!.outputTokens).toBe(656)
expect(calls[0]!.reasoningTokens).toBe(609)
expect(calls[0]!.timestamp).toBe(new Date(1786707340050).toISOString())
})
it('a chunk sample arriving after the final report does not overwrite it', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-late', [
[sessionHeader({ id: 'session-late' })],
[assistantMessage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340050)],
[chunkUsage(1, 1, { inputTokens: 999, outputTokens: 99 }, 1786707340100)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(100)
})
it('falls back to the chunk sample when no assistant/message usage arrives', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-sample', [
[sessionHeader({ id: 'session-sample' })],
[chunkUsage(2, 3, { inputTokens: 42, outputTokens: 7 }, 1786707340000)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(42)
expect(calls[0]!.deduplicationKey).toBe('dsh:session-sample:2:3')
})
it('steps inherit the model of the most recent request/header', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-model', [
[sessionHeader({ id: 'session-model' })],
[requestHeader('deepseek-v4-pro', 1786707337000)],
[assistantMessage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000)],
[assistantMessage(1, 2, { inputTokens: 200, outputTokens: 20 }, 1786707341000)],
[requestHeader('deepseek-v4-flash', 1786707342000)],
[assistantMessage(2, 1, { inputTokens: 300, outputTokens: 30 }, 1786707343000)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls.map(c => c.model)).toEqual(['deepseek-v4-pro', 'deepseek-v4-pro', 'deepseek-v4-flash'])
})
it('bills reasoning tokens at the output rate', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-reason', [
[sessionHeader({ id: 'session-reason' })],
[requestHeader('deepseek-v4-pro')],
[assistantMessage(1, 1, { inputTokens: 1000, outputTokens: 100, cacheWriteTokens: 50, cacheReadTokens: 500, reasoningTokens: 400 }, 1786707340000)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.costUSD).toBeCloseTo(calculateCost('deepseek-v4-pro', 1000, 500, 50, 500, 0), 12)
})
it('collects mapped tools, skill names and bash commands from tool/call events', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-tools', [
[sessionHeader({ id: 'session-tools' })],
[
toolCall(1, 1, 'read', { path: '/x/a.ts' }, 1786707339500),
toolCall(1, 1, 'edit', { path: '/x/a.ts' }, 1786707339600),
toolCall(1, 1, 'bash', { command: 'git status && bun test' }, 1786707339700),
toolCall(1, 1, 'skill', { name: 'coding-agent-orchestration' }, 1786707339800),
toolCall(1, 1, 'cordis_run', { id: 'j1' }, 1786707339900),
chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000),
],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.tools).toEqual(['Read', 'Edit', 'Bash', 'Skill', 'cordis_run'])
expect(calls[0]!.bashCommands).toEqual(['git', 'bun'])
expect(calls[0]!.skills).toEqual(['coding-agent-orchestration'])
})
it('pairs the user message of the turn and carries session id and project', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-ctx', [
[sessionHeader({ id: 'session-ctx', cwd: 'C:\\Users\\test\\myproject' })],
[turnStart(1, 1786707339000), userMessage('first question', 1786707339100)],
[chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000)],
[turnStart(2, 1786707350000), userMessage('second question', 1786707350100)],
[chunkUsage(2, 1, { inputTokens: 200, outputTokens: 20 }, 1786707351000)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(2)
expect(calls[0]!.userMessage).toBe('first question')
expect(calls[1]!.userMessage).toBe('second question')
expect(calls[0]!.sessionId).toBe('session-ctx')
expect(calls[0]!.project).toBe('myproject')
expect(calls[0]!.projectPath).toBe('C:\\Users\\test\\myproject')
})
it('parses the uncompressed session.jsonl variant', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-plain', [
sessionHeader({ id: 'session-plain', cwd: '/home/u/proj' }),
turnStart(1, 1786707339000),
userMessage('hello', 1786707339100),
chunkUsage(1, 1, { inputTokens: 123, outputTokens: 45 }, 1786707340000),
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(123)
expect(calls[0]!.outputTokens).toBe(45)
})
it('skips buckets whose usage is all zero', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-zero', [
[sessionHeader({ id: 'session-zero' })],
[assistantMessage(1, 1, { inputTokens: 0, outputTokens: 0 }, 1786707340000)],
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(0)
})
itZstd('ignores a torn final frame appended by a crashed writer', async () => {
const dir = join(tmpDir, 'sessions', '--C-Users-test-myproject--', 'session-torn')
await mkdir(dir, { recursive: true })
const filePath = join(dir, 'session.jsonl.zstd')
const good = zstdCompress!(Buffer.from(
sessionHeader({ id: 'session-torn' }) + '\n' +
chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000) + '\n',
))
const torn = zstdCompress!(Buffer.from(chunkUsage(1, 2, { inputTokens: 1, outputTokens: 1 }, 1786707341000) + '\n'))
await writeFile(filePath, Buffer.concat([good, torn.subarray(0, Math.floor(torn.length / 2))]))
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(100)
})
it('deduplicates (turn, step) calls seen across multiple parses', async () => {
const filePath = await writeZstdSession('--C-Users-test-myproject--', 'session-dedup', [
[sessionHeader({ id: 'session-dedup' })],
[chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000)],
])
const provider = createDshProvider(tmpDir)
const source = { path: filePath, project: 'myproject', provider: 'dsh' }
const seenKeys = new Set<string>()
const firstRun: ParsedProviderCall[] = []
for await (const call of provider.createSessionParser(source, seenKeys).parse()) firstRun.push(call)
const secondRun: ParsedProviderCall[] = []
for await (const call of provider.createSessionParser(source, seenKeys).parse()) secondRun.push(call)
expect(firstRun).toHaveLength(1)
expect(secondRun).toHaveLength(0)
})
it('handles a missing session file gracefully', async () => {
const provider = createDshProvider(tmpDir)
const source = { path: join(tmpDir, 'nope', 'session.jsonl.zstd'), project: 'test', provider: 'dsh' }
const calls: ParsedProviderCall[] = []
for await (const call of provider.createSessionParser(source, new Set()).parse()) calls.push(call)
expect(calls).toHaveLength(0)
})
})
describe('dsh provider - display names', () => {
const provider = createDshProvider('/tmp')
it('has correct name and displayName', () => {
expect(provider.name).toBe('dsh')
expect(provider.displayName).toBe('DeepSeek Harness')
})
it('maps deepseek models to readable names and passes unknown ids through', () => {
expect(provider.modelDisplayName('deepseek-v4-pro')).toBe('DeepSeek v4 Pro')
expect(provider.modelDisplayName('some-future-model')).toBe('some-future-model')
})
it('normalizes tool names, keeping unknown names raw', () => {
expect(provider.toolDisplayName('bash')).toBe('Bash')
expect(provider.toolDisplayName('pwsh')).toBe('Bash')
expect(provider.toolDisplayName('todo_write')).toBe('TodoWrite')
expect(provider.toolDisplayName('cordis_run')).toBe('cordis_run')
})
})
describe('dsh provider - real log fidelity', () => {
// The upstream snapshot from deepseek-ai/deepseek-harness
// (examples/acp-agent/tests/snapshots/bash-tool-turn/session.jsonl), with its
// template placeholders filled in. It is the reference for every shape the
// parser reads: packed `reasoning-chunks`/`tool-call-chunks` storage rows, a
// plugin-injected user/message beside the typed one, and both the streamed
// usage chunk and the final assistant/message usage for the same step.
async function writeRealSession(): Promise<string> {
const lines = (await readFile(join(import.meta.dirname, '../fixtures/dsh/bash-tool-turn.jsonl'), 'utf-8'))
.split('\n').filter(l => l.trim())
return writePlainSession('--home-u-proj--', 'e128dda9-ed11-4868-8266-0ef90d03c3d6', lines)
}
it('parses the upstream snapshot: two steps, exact usage, model from the message source', async () => {
const calls = await parseAll(createDshProvider(tmpDir), await writeRealSession())
expect(calls).toHaveLength(2)
expect(calls.map(c => c.model)).toEqual(['deepseek-v4-flash', 'deepseek-v4-flash'])
expect(calls[0]).toMatchObject({
inputTokens: 2877,
outputTokens: 90,
cacheReadInputTokens: 0,
reasoningTokens: 18,
sessionId: 'e128dda9-ed11-4868-8266-0ef90d03c3d6',
project: 'proj',
projectPath: '/home/u/proj',
workingDirectory: '/home/u/proj',
})
expect(calls[1]).toMatchObject({ inputTokens: 168, outputTokens: 25, cacheReadInputTokens: 2816, reasoningTokens: 22 })
// Reasoning bills at the output rate, so it must not appear as input.
expect(calls[0]!.costUSD).toBe(calculateCost('deepseek-v4-flash', 2877, 90 + 18, 0, 0, 0))
expect(calls[0]!.costUSD).toBeGreaterThan(0)
})
it('takes the typed prompt as the preview, not the plugin-injected context', async () => {
const calls = await parseAll(createDshProvider(tmpDir), await writeRealSession())
expect(calls[0]!.userMessage).toBe('Use the bash tool to run exactly: echo TERMINAL_OK. Then reply with the single word DONE and stop.')
expect(calls[0]!.userMessage).not.toContain('Current runtime context')
})
it('reads the tool call through the packed chunk rows around it', async () => {
const calls = await parseAll(createDshProvider(tmpDir), await writeRealSession())
expect(calls[0]!.tools).toEqual(['Bash'])
expect(calls[0]!.bashCommands).toEqual(['echo'])
})
})
describe('dsh provider - defensive reads', () => {
it('skips a log stamped with an unsupported session format version', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-future', [
JSON.stringify({ type: 'session', version: 1, id: 'session-future', createdAt: 1786707336131, cwd: '/home/u/proj', delegationDepth: 0 }),
chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000),
])
expect(await createDshProvider(tmpDir).discoverSessions()).toEqual([])
expect(await parseAll(createDshProvider(tmpDir), filePath)).toEqual([])
})
it('does not bill a forked session for the events it inherited from its parent', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-fork', [
JSON.stringify({
type: 'session', version: 0, id: 'session-fork', createdAt: 1786707336131,
cwd: '/home/u/proj', parentSession: 'session-parent', seedLength: 3, delegationDepth: 0,
}),
// seq 0..2 are a verbatim copy of the parent's log, which codeburn parses
// as its own session; only seq >= 3 is this session's own work.
JSON.stringify({ type: 'turn/start', seq: 0, time: 1786707337000, data: { turn: 1 } }),
JSON.stringify({ type: 'assistant/message', seq: 1, time: 1786707337100, data: { turn: 1, step: 1, message: { role: 'assistant', content: [] }, usage: { inputTokens: 9999, outputTokens: 999 } } }),
JSON.stringify({ type: 'session/end-seed', seq: 2, time: 1786707337200, data: {} }),
JSON.stringify({ type: 'turn/start', seq: 3, time: 1786707338000, data: { turn: 2 } }),
JSON.stringify({ type: 'assistant/message', seq: 4, time: 1786707338100, data: { turn: 2, step: 1, message: { role: 'assistant', content: [] }, usage: { inputTokens: 100, outputTokens: 10 } } }),
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(100)
})
it('ignores unknown event types, packed chunk rows, and unparsable lines', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-noise', [
sessionHeader({ id: 'session-noise', cwd: '/home/u/proj' }),
JSON.stringify({ type: 'agent/inbox/spliced', seq: 0, time: 1786707337000, data: { target: 'next-turn' } }),
JSON.stringify({ type: 'reasoning-chunks', seq0: 1, time0: 1786707337100, data: { turn: 1, step: 1, index: 0, dt: [0], texts: ['a', 'b'] } }),
'{ not json at all',
' ',
chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000),
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.inputTokens).toBe(100)
})
it('falls back to the header createdAt when a usage event carries no usable time', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-notime', [
JSON.stringify({ type: 'session', version: 0, id: 'session-notime', createdAt: 1786707336131, cwd: '/home/u/proj', delegationDepth: 0 }),
JSON.stringify({ type: 'assistant/message', seq: 1, data: { turn: 1, step: 1, message: { role: 'assistant', content: [] }, usage: { inputTokens: 100, outputTokens: 10 } } }),
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
expect(calls[0]!.timestamp).toBe(new Date(1786707336131).toISOString())
})
})
describe('dsh provider - real log, real container', () => {
itZstd('reads the upstream snapshot out of multi-frame zstd with a torn tail identically to plain jsonl', async () => {
const lines = (await readFile(join(import.meta.dirname, '../fixtures/dsh/bash-tool-turn.jsonl'), 'utf-8'))
.split('\n').filter(l => l.trim())
const plain = await parseAll(
createDshProvider(tmpDir),
await writePlainSession('--home-u-proj--', 'plain', lines),
)
// Header batch, then three append batches — the layout DSH writes.
const dir = join(tmpDir, 'sessions', '--home-u-proj--', 'framed')
await mkdir(dir, { recursive: true })
const filePath = join(dir, 'session.jsonl.zstd')
const frames = [[lines[0]!], lines.slice(1, 10), lines.slice(10, 25), lines.slice(25)]
.map(batch => zstdCompress!(Buffer.from(batch.join('\n') + '\n', 'utf-8')))
// A crashed writer's half-written final batch, carrying usage that must not count.
const torn = zstdCompress!(Buffer.from(assistantMessage(9, 9, { inputTokens: 123456, outputTokens: 1 }, 1785730424999) + '\n', 'utf-8'))
await writeFile(filePath, Buffer.concat([...frames, torn.subarray(0, Math.floor(torn.length / 2))]))
const framed = await parseAll(createDshProvider(tmpDir), filePath)
expect(framed.map(c => [c.inputTokens, c.outputTokens, c.reasoningTokens, c.model]))
.toEqual(plain.map(c => [c.inputTokens, c.outputTokens, c.reasoningTokens, c.model]))
expect(framed).toHaveLength(2)
})
})
describe('dsh provider - hostile input', () => {
itZstd('skips a session whose frames decompress to far more than the file cap', async () => {
const dir = join(tmpDir, 'sessions', '--home-u-proj--', 'session-bomb')
await mkdir(dir, { recursive: true })
const filePath = join(dir, 'session.jsonl.zstd')
// 200 MB of zeros compresses to a few KB. Uncapped this decoded to ~916 MB
// of RSS for a 16 KB file; the per-frame cap now rejects it without
// allocating past the cap.
const bomb = zstdCompress!(Buffer.alloc(200 * 1024 * 1024))
const good = zstdCompress!(Buffer.from(
sessionHeader({ id: 'session-bomb', cwd: '/home/u/proj' }) + '\n'
+ chunkUsage(1, 1, { inputTokens: 100, outputTokens: 10 }, 1786707340000) + '\n',
'utf-8',
))
await writeFile(filePath, Buffer.concat([good, bomb]))
expect((await stat(filePath)).size).toBeLessThan(64 * 1024)
// The whole file is skipped: the frames read before the bomb are not
// counted, so a crafted tail cannot poison a partial total.
expect(await parseAll(createDshProvider(tmpDir), filePath)).toEqual([])
})
itZstd('stops decoding once the frames exceed the running budget', async () => {
const frame = zstdCompress!(Buffer.from('{"type":"turn/start","seq":0,"time":1,"data":{"turn":1}}\n', 'utf-8'))
const buffer = Buffer.concat([frame, frame, frame])
expect([...readZstdLines(buffer, Number.POSITIVE_INFINITY, 4096)]).toHaveLength(3)
// A budget under two frames' plaintext stops at the frame that overruns it.
expect(() => [...readZstdLines(buffer, Number.POSITIVE_INFINITY, 60)]).toThrow()
})
it('coerces non-numeric usage fields instead of poisoning the totals', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-poison', [
sessionHeader({ id: 'session-poison', cwd: '/home/u/proj' }),
JSON.stringify({
type: 'assistant/message', seq: 1, time: 1786707340000,
data: {
turn: 1, step: 1, message: { role: 'assistant', content: [] },
usage: { inputTokens: '999', outputTokens: [1, 2], reasoningTokens: 1e308 * 10, cacheReadTokens: -5, cacheWriteTokens: 7 },
},
}),
])
const calls = await parseAll(createDshProvider(tmpDir), filePath)
expect(calls).toHaveLength(1)
// Only the one genuinely numeric field survives; every other shape is 0.
expect(calls[0]).toMatchObject({
inputTokens: 0,
outputTokens: 0,
reasoningTokens: 0,
cacheReadInputTokens: 0,
cacheCreationInputTokens: 7,
})
for (const value of [calls[0]!.inputTokens, calls[0]!.outputTokens, calls[0]!.costUSD]) {
expect(typeof value).toBe('number')
expect(Number.isFinite(value)).toBe(true)
}
})
it('still skips a call whose usage is all non-numeric', async () => {
const filePath = await writePlainSession('--home-u-proj--', 'session-poison-zero', [
sessionHeader({ id: 'session-poison-zero', cwd: '/home/u/proj' }),
JSON.stringify({
type: 'assistant/message', seq: 1, time: 1786707340000,
data: { turn: 1, step: 1, message: { role: 'assistant', content: [] }, usage: { inputTokens: '999', outputTokens: [1, 2] } },
}),
])
expect(await parseAll(createDshProvider(tmpDir), filePath)).toEqual([])
})
})