* fix(openclaw): read the per-agent sqlite transcript store the 2026-09-01 migration writes OpenClaw 2026.8.1 moved sessions from agents/<agent>/sessions/*.jsonl to agents/<agent>/agent/openclaw-agent.sqlite, so discovery resolved to zero sources and doctor reported the provider empty (#1259). transcript_events keeps each JSONL envelope verbatim in event_json, so the JSONL event reducer is extracted and shared: both eras parse, dedup and cost identically, and migrated sessions keep their original ids. Sessions are listed per store and parsed in keyset batches of 2000 rows so a gigabyte store never loads whole. agent-schema 23 lets a row carry its event as an event_zstd BLOB instead; those decode on Node 22.15+ and skip with one notice below that. Fixes #1259 * test(openclaw): batch fixture inserts in a transaction; hash id-less events before the created_at fallback --------- Co-authored-by: ozymandiashh <ozymandiashh@users.noreply.github.com> Co-authored-by: Resham Joshi <65915470+iamtoruk@users.noreply.github.com>
4 KiB
OpenClaw
OpenClaw, plus the older Clawdbot / Moltbot / Moldbot lineage.
- Source:
src/providers/openclaw.ts - Loading: eager (
src/providers/index.ts:8) - Test:
tests/providers/openclaw.test.ts
Where it reads from
Four directories, all checked on every run (openclaw.ts):
~/.openclaw/agents~/.clawdbot/agents~/.moltbot/agents~/.moldbot/agents
The legacy directories are kept for users who upgraded from older builds.
Storage format
Two eras, discovered side by side under each <agents>/<agent>/ directory:
- SQLite (since the 2026-09-01 migration,
2026.8.1; #1259).agent/openclaw-agent.sqlite, tabletranscript_events(session_id, seq, event_json, created_at). Each row'sevent_jsonis the verbatim JSONL envelope, so both eras run the same event reducer; agent-schema 23 additionally allows a row to carry its event as aevent_zstdBLOB instead (one zstd frame, decoded whennode:zlibhas zstd — Node 22.15+ — and skipped with a notice otherwise). Sessions are listed with aGROUP BY session_idand each parses in keyset batches of 2000 rows so a gigabyte store never loads whole. - JSONL (pre-migration, still read when present).
sessions/sessions.jsonindex plus per-session.jsonlfiles, with a directory-scan fallback when the index is missing or stale. Post-migration the migration tool moves these tosession-sqlite-import-archive/or renames them*.jsonl.deleted.<ts>, which the.jsonlfilter naturally ignores.
The SQLite source path carries the session id after the database path (<db>:<sessionId>, the forge.ts convention), so each session parses — and dedups — on its own.
Caching
None.
Deduplication
Per <sessionId>:<dedupId>, identical keys across both storage eras because the migrated envelopes keep their original session and event ids. An envelope without an event id hashes its payload instead of falling back to its parse position, so the same id-less event hashes identically in either era while distinct ones never collide on an array index.
Store authority
When a session id exists both as a legacy .jsonl file and in the store (a partial migration or a restored backup), discovery keeps only the store's source. Keeping both live would let the legacy file's cached turns suppress the store's first parse — the suppressed (empty) result becomes the store's session-cache entry, and once the legacy file is archived and its entry evicted, the imported history would be gone from every report until the store changed.
Quirks
- Cost is preferred from the provider when reported. OpenClaw emits
costUSDinmessage.usage; the parser uses it directly when present and only computes from tokens when it is missing. - Tokens are reported across
input,output,cacheRead, andcacheWrite. Anthropic semantics throughout, no normalization needed. - Timestamp fallback chain (SQLite): envelope
timestamp→ rowcreated_at(ms) → the store file's mtime. The retry oncreated_atalso covers a present-but-unparseable envelope timestamp, which must not land the call on the store's mtime ("now" on a live gateway). The JSONL era uses envelope → file mtime, as before.
When fixing a bug here
- If the bug is "session not found", check the four legacy dirs first. A user might have a stray
~/.moltbot/that the parser is reading instead of the real~/.openclaw/. - If the bug is "wrong cost", confirm whether
costUSDis present in the source data; the parser trusts it over its own calculation. - The
sessions.jsonindex can drift when the user crashes mid-session. Make sure the directory-scan fallback triggers in those cases. - If a migrated install reports empty, confirm
agent/openclaw-agent.sqliteexists and has atranscript_eventstable; a DB without that table is skipped silently (it is not an agent-schema store). - A live OpenClaw gateway keeps writing to the store; the parser opens it read-only per session and closes it when done, and the sqlite wrapper handles
-walsidecars.