openclaw/scripts/e2e
Josh Lehman 0a8e3604ba
refactor: flip sessions and transcripts to sqlite storage (#98236)
* refactor(sessions): migrate runtime storage to sqlite

* test(sessions): fix sqlite CI regressions

* test(sessions): align remaining sqlite fixtures

* fix(codex): require sqlite trajectory recorder

* test(sessions): align orphan recovery sqlite fixture

* test(sessions): align sqlite rebase fixtures

* fix(sessions): finish current-main integration of the sqlite flip

Resolve the whole-store SDK removal across its owner boundary: drop the
loadSessionStore re-export and the registry whole-store wrappers, wire
hasTrackedActiveSessionRun into gateway chat, complete the
preserveLockedHarnessIds cleanup contract, flip the codex thread-history
import to storePath targets, and port remaining main-side tests from
file-store helpers to session accessor reads.

* chore: drop committed pebbles log, revert plugin-inspector bump, refresh generated docs

Remove the 1.8k-line .pebbles/events.jsonl work log from the branch, restore
the plugin-inspector advisory lane to main's pinned 0.3.10 so the supply-chain
bump gets its own review, and regenerate docs_map, the plugin SDK API baseline,
and the export-surface ratchet for the merged tree.

* feat(sessions): keep archived transcripts by default with zstd cold storage

Codex-style retention: deleting or resetting a session archives its
transcript as a zstd-compressed JSONL artifact (plain when the runtime
lacks node:zlib zstd) and keeps it until the disk budget evicts oldest
first. resetArchiveRetention now governs both deleted and reset archives
and defaults to keep; maxDiskBytes defaults to 2gb so retention stays
bounded, with archives evicted before live sessions. The cron reaper
follows the same knob instead of deleting archives on its own timer.

* fix(state): converge agent DB migration lineages and bound database growth

Merge coherence: run both structure-gated legacy memory-schema repairs
(flip-lineage drop, main-lineage identity rebuild) before the flip
migration so pre-flip v1/v2 and pre-merge flip v1/v4 databases all
converge, and hoist foreign_keys=OFF outside the schema transaction
where the pragma was silently ignored and the v1 sessions rebuild
cascade-deleted session_entries.

Growth guards: fresh agent DBs enable auto_vacuum=INCREMENTAL, WAL
maintenance releases freed pages in bounded passes (never a blocking
full VACUUM), and doctor reports state/agent DB bloat from freelist
stats.

* fix(codex): resolve the store path for thread-history import via the SDK

The supervision catalog passed the legacy sessionFile locator to the
storePath-targeted transcript mirror; resolve the agent store path with
the session-store SDK helper instead of a runtime-object seam so test
fakes and headless callers need no extra surface. Drop the obsolete
missing-session-id preprocessing case: sessions rows are NOT NULL on
session_id and upsert repairs id-less patches at write time.

* fix(sessions): fail safe on malformed disk-budget config and doctor stat errors

A malformed explicit maxDiskBytes disables the budget instead of
falling back to the destructive 2gb default the user never chose, and
the doctor bloat check skips databases whose paths stat-fail instead of
aborting doctor.

* fix(sessions): complete sqlite conflict translations

* test(sqlite): align hardening checks with maintenance

* test(sessions): inspect compressed transcript archives

* fix(tests): await session seeds and drop unused helpers flagged by CI lint

The five unawaited writeSessionStoreSeed calls raced their SQLite seeds
against the assertions, failing compact shards; the bloat probe drops a
useless initializer and the merged tests drop now-unused helpers.

* test(sessions): type legacy proof events directly

* test(sessions): align hardening contracts

* perf(sessions): read usage transcript sizes from SQL aggregates

Usage/cost scans walked every session and materialized every transcript
event just to re-stringify it for a byte estimate — the #86718 stall
class reborn on the DB. readTranscriptStatsSync sums stored JSON bytes
in SQLite without loading a single row.

* fix(sessions): re-root foreign-root transcript paths onto the current sessions dir

Restored backups, moved OPENCLAW_STATE_DIR, and rehearsal copies carry
absolute sessionFile paths from the old root; the containment fallback
kept those foreign paths, so migration read (and would archive) files in
the original root and reported local copies missing. Re-root the
canonical agents/<id>/sessions suffix onto the current dir when the file
exists there; genuine cross-root layouts still fall through unchanged.

* test(agents): seed harness admission through sqlite

* fix(sqlite): close agent db on pragma setup failure

* fix(doctor): compact and retrofit incremental auto-vacuum after session import

The migration is the sanctioned offline window: post-import compact
reclaims import churn and applies auto_vacuum=INCREMENTAL to databases
created before the fresh-DB pragma existed, so runtime maintenance can
release pages in bounded passes on every install.

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-11 14:50:37 -07:00
..
lib refactor: flip sessions and transcripts to sqlite storage (#98236) 2026-07-11 14:50:37 -07:00
parallels chore: migrate active GPT-5.5 references to GPT-5.6 (#104452) 2026-07-11 06:30:57 -07:00
agent-bundle-mcp-tools-docker.sh test: fold remaining Docker lanes into QA (#97708) 2026-06-29 00:39:06 -07:00
agents-delete-shared-workspace-docker.sh
browser-cdp-snapshot-docker.sh fix(e2e): avoid browser cdp doctor race 2026-07-05 14:21:14 +02:00
build-image.sh
bun-global-install-smoke.sh fix(ci): preserve QA package intent boundaries (#104207) 2026-07-11 00:51:57 -07:00
bundled-plugin-install-uninstall-docker.sh
channel-plugin-trust-docker.sh
codex-media-path-docker.sh
codex-npm-plugin-live-docker.sh
codex-on-demand-docker.sh fix(e2e): extend Codex on-demand install timeout 2026-06-25 00:32:29 +02:00
commitments-safety-docker-client.ts test: keep commitments client boundary 2026-06-24 20:52:00 +01:00
commitments-safety-docker.sh fix(qa): bound docker e2e log replay 2026-06-23 13:26:09 +02:00
compose-setup.sh chore(docker): execute Compose and package artifact proofs (#101045) 2026-07-06 13:05:34 -07:00
config-reload-source-docker.sh
crestodian-first-run-docker.sh test: fold remaining Docker lanes into QA (#97708) 2026-06-29 00:39:06 -07:00
crestodian-first-run-spec.json fix(onboarding): skip setup for configured gateways and require inference first (#102883) 2026-07-11 10:25:14 -07:00
crestodian-rescue-docker-client.ts
crestodian-rescue-docker.sh fix(e2e): provide crestodian rescue gateway token 2026-07-05 13:36:27 +02:00
cron-cli-docker.sh fix(cron): use direct lookup instead of paginated search in cron edit (#100836) 2026-07-06 06:58:22 -07:00
cron-mcp-cleanup-docker-client.ts chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475) 2026-07-11 06:15:41 -07:00
cron-mcp-cleanup-docker.sh
cron-mcp-cleanup-seed.ts
docker-openai-seed.ts chore: migrate active GPT-5.5 references to GPT-5.6 (#104452) 2026-07-11 06:30:57 -07:00
docker-package-install.sh chore(docker): execute Compose and package artifact proofs (#101045) 2026-07-06 13:05:34 -07:00
docker-selected-plugins.sh fix(docker): package selected external plugins in source-built images (#103629) 2026-07-10 13:00:50 +01:00
Dockerfile
Dockerfile.qr-import
doctor-install-switch-docker.sh
gateway-network-docker.sh test(gateway): prove suspension lifecycle in Docker (#104203) 2026-07-10 22:53:35 -07:00
kitchen-sink-plugin-docker.sh
kitchen-sink-rpc-docker.sh
kitchen-sink-rpc-walk.d.mts chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475) 2026-07-11 06:15:41 -07:00
kitchen-sink-rpc-walk.mjs refactor(deadcode): localize UI and script symbols (#101858) 2026-07-07 11:55:57 -07:00
live-plugin-tool-docker.sh chore: migrate active GPT-5.5 references to GPT-5.6 (#104452) 2026-07-11 06:30:57 -07:00
mcp-channel-limits.ts refactor(deadcode): localize script helper types (#101917) 2026-07-07 16:59:51 -07:00
mcp-channels-docker.sh test: link mcp gateway transport coverage (#97609) 2026-06-28 19:20:15 -07:00
mcp-channels-seed.ts
mcp-code-mode-gateway-client.ts refactor(deadcode): localize script helper types (#101917) 2026-07-07 16:59:51 -07:00
mcp-code-mode-gateway-docker.sh
mcp-code-mode-gateway-live-docker.sh
mcp-code-mode-gateway-seed.ts
mock-openai-server.mjs chore: migrate active GPT-5.5 references to GPT-5.6 (#104452) 2026-07-11 06:30:57 -07:00
multi-node-update-docker.sh
node-plugin-tools-pond.mjs test(e2e): live-model scenarios for node-hosted MCP tools and skills (#104699) 2026-07-11 14:07:31 -07:00
npm-onboard-channel-agent-docker.sh
npm-telegram-live-docker.sh fix(release): validate exact prepared npm package sets (#102759) 2026-07-09 05:19:24 -07:00
npm-telegram-live-runner.ts chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475) 2026-07-11 06:15:41 -07:00
onboard-docker.sh
openai-chat-tools-docker.sh
openai-image-auth-docker.sh test: fold remaining Docker lanes into QA (#97708) 2026-06-29 00:39:06 -07:00
openai-web-search-minimal-docker.sh fix(qa): allow web search smoke gateway port override 2026-06-23 12:26:07 +02:00
openwebui-docker.sh chore: migrate active GPT-5.5 references to GPT-5.6 (#104452) 2026-07-11 06:30:57 -07:00
openwebui-probe.mjs refactor(scripts): share regexp literal escaping (#99778) 2026-07-03 20:20:28 -07:00
parallels-linux-smoke.sh
parallels-macos-smoke.sh
parallels-npm-update-smoke.sh
parallels-windows-smoke.sh
plugin-binding-command-escape-docker.sh fix(qa): bound docker e2e log replay 2026-06-23 13:26:09 +02:00
plugin-binding-command-escape.Dockerfile
plugin-lifecycle-matrix-docker.sh
plugin-update-unchanged-docker.sh
plugins-docker.sh
qr-import-docker.sh
release-media-memory-docker.sh
release-plugin-marketplace-docker.sh
release-typed-onboarding-docker.sh
release-upgrade-user-journey-docker.sh
release-user-journey-docker.sh
secret-provider-integrations.mjs chore: migrate active GPT-5.5 references to GPT-5.6 (#104452) 2026-07-11 06:30:57 -07:00
session-runtime-context-docker-client.ts test: stabilize runtime context env 2026-06-24 21:04:45 +01:00
session-runtime-context-docker.sh fix(qa): bound docker e2e log replay 2026-06-23 13:26:09 +02:00
skill-install-docker.sh
sqlite-sessions-transcripts-flip-proof.ts refactor: flip sessions and transcripts to sqlite storage (#98236) 2026-07-11 14:50:37 -07:00
status-corrupt-plugin-deps.sh
telegram-bot-api.ts refactor(deadcode): localize script helper types (#101917) 2026-07-07 16:59:51 -07:00
telegram-user-crabbox-proof.ts feat: add metadata-only message audit events (#103903) 2026-07-11 13:14:08 -07:00
telegram-user-credential-io.ts chore(tooling): typecheck root test/** with a dedicated tsgo lane (#104475) 2026-07-11 06:15:41 -07:00
telegram-user-credential-paths.ts
telegram-user-credential.ts fix(e2e): reject flag Telegram credential values 2026-06-21 23:04:51 +02:00
telegram-user-driver.py
update-channel-switch-docker.sh fix(e2e): stage ai runtime in git fixtures 2026-07-05 13:25:36 +02:00
update-corrupt-plugin-docker.sh
upgrade-survivor-docker.sh fix(release): preserve beta validation evidence (#103796) 2026-07-10 09:05:14 -07:00