mirror of
https://github.com/openclaw/openclaw.git
synced 2026-10-03 01:29:56 +00:00
perf(state): retire periodic runtime integrity scans (#162015)
* perf(state): retire periodic runtime integrity scans Remove delayed and daily full-database scans from the Gateway while preserving requested agent quick checks, live-owner confirmation, and quarantine. Keep full verification with admission, migrations, and Doctor maintenance. No config or schema migration is required. * fix(update): restore Windows task autostart after cancellation Carry the existing restoration phase through Windows task recovery so SIGINT fences forward work without rejecting compensation. Preserve executor and native task ownership checks before side effects. The original 40-file CI shard reproduced 821 passes and one SIGINT failure; it passes all 822 tests with this change. Testbox focused tests and changed checks passed, and independent Codex review found no actionable findings. * fix(plugin-sdk): keep updater path context private Pin the legacy home-directory facade to its existing eight exports so new internal updater helpers do not become public SDK contracts. Reduce the wildcard ratchet by one without expanding export or callable budgets. All 11 SDK surface tests passed against the exact failed CI merge plus this fix on Testbox, along with core/script types, targeted lint, export guards, and formatting. Independent Codex review found no actionable findings.
This commit is contained in:
Peter Steinberger
GitHub
parent
a98fb670d3
commit
ba80dfbee5
13 changed files with 206 additions and 272 deletions
|
|
@ -4,4 +4,4 @@
|
|||
# The internal Node-version check no longer contributes a false OPENCLAW_* name.
|
||||
# The next-turn runtime-context preface constant no longer exists as an OPENCLAW_* name.
|
||||
# OPENCLAW_PACKAGE_BUN_LAUNCHER (#159431) is the updater-to-preinstall handoff for Bun-only installs.
|
||||
475
|
||||
473
|
||||
|
|
|
|||
|
|
@ -163,6 +163,11 @@ validation. Other runtime files remain hardlinked when supported.
|
|||
|
||||
These lifecycle and copying changes apply when the installed updater supports
|
||||
them; installing a newer candidate cannot change the updater already running.
|
||||
|
||||
On Windows, interruption before activation still lets the admitted recovery
|
||||
owner restore task autostart after pending task operations settle. Cancellation
|
||||
fences new update work; restoration still requires the original live installation
|
||||
owner and verified task ownership.
|
||||
After that updater exits, run the newer `openclaw doctor --fix` from the original
|
||||
checkout to locate its sibling runtime directories. Doctor also checks known
|
||||
temporary directories, including the managed service's `TMPDIR`, `TMP`, and `TEMP`.
|
||||
|
|
|
|||
|
|
@ -13,6 +13,11 @@ OpenClaw stores control-plane state in the shared state database and agent data
|
|||
|
||||
Schema-version, integrity, canonical-index, and table-existence checks belong to open/admission and the migration owner after migrations; runtime paths must carry admitted schema facts with the handle, never re-query them, and use fresh `PRAGMA data_version` probes to observe foreign commits on the next unpinned read while preserving active SQLite snapshots. Existing per-call checks are legacy and must be migrated when touched.
|
||||
|
||||
The Gateway does not schedule full-database integrity scans after startup or on a
|
||||
daily timer. Use [Doctor maintenance](/reference/database-schemas/integrity-and-recovery#integrity-checks)
|
||||
for operator-requested or scheduled full verification. Admission-requested
|
||||
background `quick_check` work remains limited to the requested agent database.
|
||||
|
||||
Two mechanisms back that contract. CI runs
|
||||
`scripts/check-native-state-schema-version.mjs`, which fails the build when the
|
||||
Swift and TypeScript state-database contracts declare different schema versions.
|
||||
|
|
|
|||
|
|
@ -22,10 +22,20 @@ This checks transcript structure without cross-checking its index
|
|||
entries or uniqueness. Other tables retain table/index cross-checking;
|
||||
`sqlite_schema` includes the freelist check. Partial integrity checks cannot detect
|
||||
pages shared between different tables or unused pages that a full-file check
|
||||
would find. Doctor and the daily verifier retain full-file `integrity_check`;
|
||||
would find. Doctor retains full-file `integrity_check`;
|
||||
pending migrations, repairs, and copied-file verification also retain full checks.
|
||||
No schema, stored data, or configuration changes are required.
|
||||
|
||||
The Gateway does not schedule a delayed full scan after startup or repeat full
|
||||
scans on a daily timer. For operator-requested or scheduled full verification,
|
||||
use `openclaw doctor --fix --non-interactive` during a planned maintenance window.
|
||||
This is repair maintenance: Doctor can apply supported repairs and migrations,
|
||||
and manages the matching Gateway's stop and restart. Externally supervised
|
||||
Gateways must be stopped and restarted through their owning supervisor. See
|
||||
[Run doctor](/cli/doctor/running#postures) for ownership and repair behavior.
|
||||
Removing the runtime schedule changes no configuration, schema, migration,
|
||||
update, or rollback contract.
|
||||
|
||||
Each executed admission gate logs its mode, outcome, ten slowest table checks,
|
||||
total count and duration per check kind, process, thread, and reason. Reasons are
|
||||
`stale-lease` when a previous process left an unreleased lease, `revoked` for other
|
||||
|
|
@ -53,7 +63,6 @@ for the next admission to diagnose.
|
|||
| Same-process agent reopen | Reuse current file-bound runtime proof without another integrity or quick check; recheck owner, version, schema, and canonical indexes |
|
||||
| Clean same-version agent restart | Recheck owner, version, schema, and canonical indexes; queue a child-process `quick_check` and foreign-key check after the Gateway is listening |
|
||||
| Before a pending migration | Run a full integrity, foreign-key, role, schema, and index scan |
|
||||
| Gateway background verifier | Run the full scan about once daily and log results |
|
||||
| Doctor, backup verification, and compaction | Run the full scan before accepting or rewriting the database |
|
||||
|
||||
The existing quarantine store keeps a reconstructible `agent_integrity_verifications`
|
||||
|
|
@ -109,7 +118,7 @@ Background success is logged; only the admission lease owner
|
|||
publishes verification metadata. Confirmed corruption uses the existing quarantine
|
||||
path and prevents the next open. Ordinary writes do not invalidate the file identity. Same-inode damage
|
||||
introduced after a clean close can therefore be detected after readiness by the
|
||||
quick check, SQLite operations, the daily full verifier, or explicit Doctor.
|
||||
quick check, SQLite operations, or explicit Doctor.
|
||||
`openclaw doctor` retains full checks and `doctor --fix` clears verification
|
||||
metadata with quarantine. A failed durable dirty-marker write refuses that open
|
||||
rather than leaving stale clean proof reusable after a crash.
|
||||
|
|
@ -157,7 +166,7 @@ Shared-state integrity, schema, version, and ownership checks remain in place.
|
|||
|
||||
Schema compatibility preflight can read agent schema headers without a full integrity scan. For ordinary rollback-mode agent databases and complete WAL families, a read-only child reads the schema version and optional writer build in one fresh SQLite transaction, including committed WAL changes, without copying unrelated database contents. Its native connection and physical identity remain owned through close; cancellation and timeout wait for child closure. Parent-side diagnostics do not open or close the live agent file, preserving the parent's SQLite locks. As with the previous online-backup reader, native SQLite may update SHM read marks or rebuild existing SHM after a quiescent family reopens; the database and WAL contents remain unchanged. The Gateway carries successful header facts from admission to its later compatibility preflight only while the database, WAL, and rollback-journal files are unchanged. Changed or uncertain files are inspected again. Full readiness and writable admission retain their existing validation and fresh authority checks.
|
||||
|
||||
Private snapshots remain necessary for artifact-preserving inspection, incomplete WAL families whose inspection would create source sidecars, and rollback journals requiring private recovery. Those cases use the existing snapshot owner and deadline; ordinary inspection errors do not trigger a full-copy fallback. Live files use native SQLite reads, never an immutable-file shortcut. Immutable reads are limited to verified private or explicit consolidated copies. `openclaw database preflight` performs the release-local shape comparison for an explicit copied file. The background verifier also scans already-open databases about once daily.
|
||||
Private snapshots remain necessary for artifact-preserving inspection, incomplete WAL families whose inspection would create source sidecars, and rollback journals requiring private recovery. Those cases use the existing snapshot owner and deadline; ordinary inspection errors do not trigger a full-copy fallback. Live files use native SQLite reads, never an immutable-file shortcut. Immutable reads are limited to verified private or explicit consolidated copies. `openclaw database preflight` performs the release-local shape comparison for an explicit copied file.
|
||||
|
||||
Concurrent asynchronous requests for the same physical live database share one
|
||||
snapshot operation. When the canonical runtime already owns an open SQLite
|
||||
|
|
|
|||
|
|
@ -212,7 +212,7 @@ export function readPluginSdkSurfaceBudgets(env: NodeJS.ProcessEnv = process.env
|
|||
),
|
||||
publicWildcardReexports: readPluginSdkSurfaceBudgetEnv(
|
||||
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_WILDCARD_REEXPORTS",
|
||||
47,
|
||||
46,
|
||||
env,
|
||||
),
|
||||
};
|
||||
|
|
|
|||
|
|
@ -580,7 +580,14 @@ it("starts a new topology read after healthy integrity confirmation without revi
|
|||
);
|
||||
await applyOpenClawDatabaseVerificationResults({
|
||||
env: state.env,
|
||||
targets: [{ kind: "state", label: "OpenClaw state database", path: database.path }],
|
||||
targets: [
|
||||
{
|
||||
kind: "state",
|
||||
label: "OpenClaw state database",
|
||||
path: database.path,
|
||||
check: "quick",
|
||||
},
|
||||
],
|
||||
results: [
|
||||
{ path: database.path, ok: false, error: "stale terminal result", terminal: true },
|
||||
],
|
||||
|
|
|
|||
|
|
@ -302,7 +302,16 @@ export * from "../infra/fs-safe.ts";
|
|||
export * from "../infra/heartbeat-events.ts";
|
||||
export * from "../infra/heartbeat-summary.ts";
|
||||
export * from "../infra/heartbeat-visibility.ts";
|
||||
export * from "../infra/home-dir.js";
|
||||
export {
|
||||
expandHomePrefix,
|
||||
resolveEffectiveHomeDir,
|
||||
resolveHomeRelativePath,
|
||||
resolveOsHomeDir,
|
||||
resolveOsHomeRelativePath,
|
||||
resolveRequiredHomeDir,
|
||||
resolveRequiredOsHomeDir,
|
||||
resolveUserPath,
|
||||
} from "../infra/home-dir.js";
|
||||
// Keep this deprecated barrel pinned to its shipped request-body surface; new
|
||||
// response readers belong only to the focused response-limit/media entrypoints.
|
||||
export {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,4 @@
|
|||
import { fork, type ChildProcess } from "node:child_process";
|
||||
import { existsSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { toStructuredErrorObject } from "@openclaw/normalization-core/error-coercion";
|
||||
import { isRecord } from "@openclaw/normalization-core/record-coerce";
|
||||
|
|
@ -10,7 +8,6 @@ import { createSubsystemLogger } from "../logging/subsystem.js";
|
|||
import {
|
||||
closeOpenClawAgentDatabaseByPathAsync,
|
||||
confirmOpenClawAgentDatabaseIntegrity,
|
||||
listOpenClawRegisteredAgentDatabases,
|
||||
recordOpenClawAgentDatabaseOpenFailure,
|
||||
} from "./openclaw-agent-db.js";
|
||||
import type {
|
||||
|
|
@ -22,7 +19,6 @@ import {
|
|||
confirmOpenClawStateDatabaseIntegrity,
|
||||
recordOpenClawStateDatabaseOpenFailure,
|
||||
} from "./openclaw-state-db.js";
|
||||
import { resolveOpenClawStateSqlitePath } from "./openclaw-state-db.paths.js";
|
||||
|
||||
const log = createSubsystemLogger("state/database-verify");
|
||||
const DATABASE_VERIFY_CHILD_ARG = "--openclaw-database-verify-child";
|
||||
|
|
@ -124,8 +120,8 @@ export function runDatabaseVerifyWorker(
|
|||
const execArgv = workerUrl.pathname.endsWith(".ts") ? ["--import", "tsx"] : undefined;
|
||||
let worker: ChildProcess;
|
||||
try {
|
||||
// Snapshot preparation opens and closes raw source descriptors. Isolate it
|
||||
// because POSIX close() can release the Gateway's process-owned SQLite locks.
|
||||
// Closing a source reader can release the Gateway's process-owned SQLite
|
||||
// locks, so quick checks keep their own process.
|
||||
worker = fork(fileURLToPath(workerUrl), [DATABASE_VERIFY_CHILD_ARG], {
|
||||
execArgv,
|
||||
stdio: ["ignore", "ignore", "ignore", "ipc"],
|
||||
|
|
@ -197,37 +193,6 @@ export async function terminateDatabaseVerifyWorker(worker: ChildProcess): Promi
|
|||
await lifecycle.settled;
|
||||
}
|
||||
|
||||
/** Resolve the state database and current registered agent database paths. */
|
||||
export function collectOpenClawDatabaseVerifyTargets(options: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
}): OpenClawDatabaseVerifyTarget[] {
|
||||
const targets = new Map<string, OpenClawDatabaseVerifyTarget>();
|
||||
const statePath = path.resolve(resolveOpenClawStateSqlitePath(options.env));
|
||||
if (existsSync(statePath)) {
|
||||
targets.set(statePath, { kind: "state", label: "OpenClaw state database", path: statePath });
|
||||
}
|
||||
let registeredDatabases: ReturnType<typeof listOpenClawRegisteredAgentDatabases> = [];
|
||||
try {
|
||||
registeredDatabases = listOpenClawRegisteredAgentDatabases({ env: options.env });
|
||||
} catch (error) {
|
||||
log.warn("failed to collect registered agent databases for integrity verification", {
|
||||
error: String(error),
|
||||
});
|
||||
}
|
||||
for (const registered of registeredDatabases) {
|
||||
const agentPath = path.resolve(registered.path);
|
||||
if (!existsSync(agentPath) || targets.has(agentPath)) {
|
||||
continue;
|
||||
}
|
||||
targets.set(agentPath, {
|
||||
kind: "agent",
|
||||
label: `OpenClaw agent database ${registered.agentId}`,
|
||||
path: agentPath,
|
||||
});
|
||||
}
|
||||
return [...targets.values()];
|
||||
}
|
||||
|
||||
/** Reconfirm worker failures on live owners before quarantine and latching. */
|
||||
export async function applyOpenClawDatabaseVerificationResults(options: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
|
|
@ -290,7 +255,6 @@ export async function applyOpenClawDatabaseVerificationResults(options: {
|
|||
reason: confirmation.error.message,
|
||||
});
|
||||
if (!recorded) {
|
||||
// Store unavailable. Daily verification retries persistence.
|
||||
log.error("failed to persist database quarantine; quarantine is process-local", {
|
||||
kind: target.kind,
|
||||
path: result.path,
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
|
|||
import * as nodeSqlite from "../infra/node-sqlite.js";
|
||||
import { runtimeProcessEntrypoints } from "../infra/runtime-process-entrypoints.js";
|
||||
import { resolveRuntimeWorkerArgv, resolveRuntimeWorkerUrl } from "../infra/runtime-worker-url.js";
|
||||
import * as sqliteLocation from "../infra/sqlite-readonly-location.js";
|
||||
import * as sqliteSource from "../infra/sqlite-source-handle.js";
|
||||
import { createDeferredCore } from "../shared/deferred.js";
|
||||
import {
|
||||
runDatabaseVerifyWorker,
|
||||
|
|
@ -68,13 +68,16 @@ describe("database verifier child process entrypoint", () => {
|
|||
fs.writeFileSync(databasePath, source);
|
||||
|
||||
await expect(
|
||||
runDatabaseVerifyWorker([{ path: databasePath, kind: "state", label: "synthetic database" }]),
|
||||
runDatabaseVerifyWorker([
|
||||
{ path: databasePath, kind: "state", label: "synthetic database", check: "quick" },
|
||||
]),
|
||||
).resolves.toEqual([
|
||||
{
|
||||
path: databasePath,
|
||||
ok: false,
|
||||
error: "Error: file is not a database (code=ERR_SQLITE_ERROR, errcode=26)",
|
||||
terminal: false,
|
||||
error:
|
||||
"SqliteIntegrityError: SQLite quick_check failed for synthetic database: file is not a database (code=ERR_SQLITE_ERROR, errcode=26)",
|
||||
terminal: true,
|
||||
},
|
||||
]);
|
||||
expect(fs.readFileSync(databasePath)).toEqual(source);
|
||||
|
|
@ -314,7 +317,12 @@ describe("database verifier worker lifetime", () => {
|
|||
});
|
||||
|
||||
describe("database verifier bounded diagnostics", () => {
|
||||
const target = { path: "synthetic.sqlite", kind: "state", label: "synthetic database" } as const;
|
||||
const target = {
|
||||
path: "synthetic.sqlite",
|
||||
kind: "state",
|
||||
label: "synthetic database",
|
||||
check: "quick",
|
||||
} as const;
|
||||
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
|
||||
|
|
@ -329,7 +337,7 @@ describe("database verifier bounded diagnostics", () => {
|
|||
},
|
||||
{
|
||||
name: "wrapped I/O error without cause prose or metadata",
|
||||
failure: new Error("snapshot failed", {
|
||||
failure: new Error("source read failed", {
|
||||
cause: Object.assign(new Error("private cause prose"), {
|
||||
code: "ERR_SQLITE_ERROR",
|
||||
errcode: 10,
|
||||
|
|
@ -339,18 +347,18 @@ describe("database verifier bounded diagnostics", () => {
|
|||
stack: "private stack",
|
||||
}),
|
||||
}),
|
||||
expected: "Error: snapshot failed (code=ERR_SQLITE_ERROR, errcode=10)",
|
||||
expected: "Error: source read failed (code=ERR_SQLITE_ERROR, errcode=10)",
|
||||
},
|
||||
{
|
||||
name: "distinct extended codes in traversal order with exact duplicates removed",
|
||||
failure: Object.assign(
|
||||
new Error("snapshot failed", {
|
||||
new Error("source read failed", {
|
||||
cause: { code: "EIO", errcode: 778, cause: { code: "ERR_SQLITE_ERROR", errcode: 1034 } },
|
||||
}),
|
||||
{ code: "ERR_SQLITE_ERROR", errcode: 778 },
|
||||
),
|
||||
expected:
|
||||
"Error: snapshot failed (code=ERR_SQLITE_ERROR, errcode=778, code=EIO, errcode=1034)",
|
||||
"Error: source read failed (code=ERR_SQLITE_ERROR, errcode=778, code=EIO, errcode=1034)",
|
||||
},
|
||||
{ name: "non-Error value", failure: "unavailable", expected: "unavailable" },
|
||||
{
|
||||
|
|
@ -372,9 +380,10 @@ describe("database verifier bounded diagnostics", () => {
|
|||
expected: "AggregateError: aggregate failure",
|
||||
},
|
||||
])("preserves $name", async ({ failure, expected }) => {
|
||||
vi.spyOn(sqliteLocation, "prepareSqliteReadOnlyLocationInProcess").mockRejectedValueOnce(
|
||||
failure,
|
||||
);
|
||||
vi.spyOn(sqliteSource, "withSqliteSourceReadDatabase").mockImplementationOnce(() => {
|
||||
// oxlint-disable-next-line typescript/only-throw-error -- Verify diagnostics for non-Error native failures.
|
||||
throw failure;
|
||||
});
|
||||
|
||||
await expect(verifyOpenClawDatabases([target])).resolves.toEqual([
|
||||
{ path: target.path, ok: false, error: expected, terminal: false },
|
||||
|
|
@ -388,9 +397,13 @@ describe("database verifier bounded diagnostics", () => {
|
|||
for (let index = 7; index >= 0; index -= 1) {
|
||||
deep = Object.assign(new Error("deep failure", { cause: deep }), { errcode: index });
|
||||
}
|
||||
vi.spyOn(sqliteLocation, "prepareSqliteReadOnlyLocationInProcess")
|
||||
.mockRejectedValueOnce(cycle)
|
||||
.mockRejectedValueOnce(deep);
|
||||
vi.spyOn(sqliteSource, "withSqliteSourceReadDatabase")
|
||||
.mockImplementationOnce(() => {
|
||||
throw cycle;
|
||||
})
|
||||
.mockImplementationOnce(() => {
|
||||
throw deep;
|
||||
});
|
||||
|
||||
await expect(verifyOpenClawDatabases([target, target])).resolves.toEqual([
|
||||
{
|
||||
|
|
@ -418,39 +431,39 @@ describe("database verifier bounded diagnostics", () => {
|
|||
{ code: { secret: "private metadata" }, errcode: "10" },
|
||||
])("omits invalid code metadata %#", async (metadata) => {
|
||||
const failure = Object.assign(
|
||||
new Error("snapshot failed", {
|
||||
new Error("source read failed", {
|
||||
cause: { code: "EIO", errcode: 10, message: "private cause prose" },
|
||||
}),
|
||||
metadata,
|
||||
);
|
||||
vi.spyOn(sqliteLocation, "prepareSqliteReadOnlyLocationInProcess").mockRejectedValueOnce(
|
||||
failure,
|
||||
);
|
||||
vi.spyOn(sqliteSource, "withSqliteSourceReadDatabase").mockImplementationOnce(() => {
|
||||
throw failure;
|
||||
});
|
||||
|
||||
await expect(verifyOpenClawDatabases([target])).resolves.toEqual([
|
||||
{
|
||||
path: target.path,
|
||||
ok: false,
|
||||
error: "Error: snapshot failed (code=EIO, errcode=10)",
|
||||
error: "Error: source read failed (code=EIO, errcode=10)",
|
||||
terminal: false,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("admits the code length and integer boundaries", async () => {
|
||||
const failure = Object.assign(new Error("snapshot failed", { cause: { errcode: 0 } }), {
|
||||
const failure = Object.assign(new Error("source read failed", { cause: { errcode: 0 } }), {
|
||||
code: "X".repeat(64),
|
||||
errcode: 0x7fff_ffff,
|
||||
});
|
||||
vi.spyOn(sqliteLocation, "prepareSqliteReadOnlyLocationInProcess").mockRejectedValueOnce(
|
||||
failure,
|
||||
);
|
||||
vi.spyOn(sqliteSource, "withSqliteSourceReadDatabase").mockImplementationOnce(() => {
|
||||
throw failure;
|
||||
});
|
||||
|
||||
await expect(verifyOpenClawDatabases([target])).resolves.toEqual([
|
||||
{
|
||||
path: target.path,
|
||||
ok: false,
|
||||
error: `Error: snapshot failed (code=${"X".repeat(64)}, errcode=2147483647, errcode=0)`,
|
||||
error: `Error: source read failed (code=${"X".repeat(64)}, errcode=2147483647, errcode=0)`,
|
||||
terminal: false,
|
||||
},
|
||||
]);
|
||||
|
|
@ -462,12 +475,6 @@ describe("database verifier bounded diagnostics", () => {
|
|||
{ name: "original corruption before close failure", errcode: 779, terminal: true },
|
||||
])("preserves $name and classification", async ({ errcode, terminal }) => {
|
||||
const database = nodeSqlite.openNodeSqliteDatabase(":memory:");
|
||||
const cleanup = vi.fn(() => true);
|
||||
vi.spyOn(sqliteLocation, "prepareSqliteReadOnlyLocationInProcess").mockResolvedValueOnce({
|
||||
location: ":memory:",
|
||||
cleanup,
|
||||
cleanupAsync: async () => cleanup(),
|
||||
});
|
||||
vi.spyOn(nodeSqlite, "openNodeSqliteDatabase").mockReturnValueOnce(database);
|
||||
if (errcode !== undefined) {
|
||||
vi.spyOn(database, "prepare").mockImplementationOnce(() => {
|
||||
|
|
@ -488,11 +495,10 @@ describe("database verifier bounded diagnostics", () => {
|
|||
error:
|
||||
errcode === undefined
|
||||
? "Error: close failed (code=EIO, errcode=10)"
|
||||
: `SqliteIntegrityError: SQLite integrity_check failed for synthetic database: scan failed (code=ERR_SQLITE_ERROR, errcode=${errcode})`,
|
||||
: `SqliteIntegrityError: SQLite quick_check failed for synthetic database: scan failed (code=ERR_SQLITE_ERROR, errcode=${errcode})`,
|
||||
},
|
||||
]);
|
||||
expect(database.isOpen).toBe(false);
|
||||
expect(cleanup).toHaveBeenCalledOnce();
|
||||
} finally {
|
||||
if (database.isOpen) {
|
||||
close();
|
||||
|
|
|
|||
|
|
@ -11,8 +11,6 @@ import {
|
|||
import type { OpenClawDatabaseVerifyResult } from "./openclaw-database-verify.worker.js";
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
collectOpenClawDatabaseVerifyTargets:
|
||||
vi.fn<typeof VerifierImplementation.collectOpenClawDatabaseVerifyTargets>(),
|
||||
runDatabaseVerifyWorker: vi.fn<typeof VerifierImplementation.runDatabaseVerifyWorker>(),
|
||||
terminateDatabaseVerifyWorker:
|
||||
vi.fn<typeof VerifierImplementation.terminateDatabaseVerifyWorker>(),
|
||||
|
|
@ -29,9 +27,6 @@ describe("database verifier shutdown", () => {
|
|||
await import("./openclaw-database-verify.impl.js");
|
||||
vi.useFakeTimers();
|
||||
vi.resetAllMocks();
|
||||
mocks.collectOpenClawDatabaseVerifyTargets.mockReturnValue([
|
||||
{ kind: "state", label: "synthetic state", path: "synthetic.sqlite" },
|
||||
]);
|
||||
mocks.terminateDatabaseVerifyWorker.mockResolvedValue(undefined);
|
||||
mocks.applyOpenClawDatabaseVerificationResults.mockResolvedValue(undefined);
|
||||
});
|
||||
|
|
@ -40,6 +35,19 @@ describe("database verifier shutdown", () => {
|
|||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("starts the Gateway verifier without periodic full database scans", async () => {
|
||||
const env = { OPENCLAW_STATE_DIR: "/synthetic/idle" };
|
||||
mocks.runDatabaseVerifyWorker.mockResolvedValue([]);
|
||||
const verifier = startOpenClawDatabaseIntegrityVerifier({ env });
|
||||
try {
|
||||
await vi.advanceTimersByTimeAsync(5 * 60_000 + 2 * 24 * 60 * 60_000);
|
||||
expect(mocks.runDatabaseVerifyWorker).not.toHaveBeenCalled();
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
await verifier.stop();
|
||||
}
|
||||
});
|
||||
|
||||
it("waits for listening startup, then checks both queued and late cached opens", async () => {
|
||||
const env = { OPENCLAW_STATE_DIR: "/synthetic/queued" };
|
||||
const firstPath = path.resolve("/synthetic/first.sqlite");
|
||||
|
|
@ -61,10 +69,8 @@ describe("database verifier shutdown", () => {
|
|||
expect(mocks.runDatabaseVerifyWorker.mock.calls[1]?.[0]).toEqual([
|
||||
expect.objectContaining({ kind: "agent", path: latePath, check: "quick" }),
|
||||
]);
|
||||
await vi.advanceTimersByTimeAsync(5 * 60_000);
|
||||
expect(mocks.runDatabaseVerifyWorker.mock.calls[2]?.[0]).toEqual([
|
||||
{ kind: "state", label: "synthetic state", path: "synthetic.sqlite" },
|
||||
]);
|
||||
expect(mocks.runDatabaseVerifyWorker).toHaveBeenCalledTimes(2);
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
await verifier.stop();
|
||||
}
|
||||
|
|
@ -162,38 +168,28 @@ describe("database verifier shutdown", () => {
|
|||
},
|
||||
);
|
||||
|
||||
it("shares the daily deadline and includes queued paths absent from the full inventory", async () => {
|
||||
const env = { OPENCLAW_STATE_DIR: "/synthetic/daily-peers" };
|
||||
it("shares requested paths between peers and retains the startup environment", async () => {
|
||||
const env = { OPENCLAW_STATE_DIR: "/synthetic/queued-peers" };
|
||||
const capturedEnv = { ...env };
|
||||
const registeredPath = path.resolve("/synthetic/registered.sqlite");
|
||||
const unregisteredPath = path.resolve("/synthetic/unregistered.sqlite");
|
||||
const registered = {
|
||||
kind: "agent" as const,
|
||||
label: "synthetic registered agent",
|
||||
path: registeredPath,
|
||||
};
|
||||
mocks.collectOpenClawDatabaseVerifyTargets.mockReturnValue([registered]);
|
||||
mocks.runDatabaseVerifyWorker.mockResolvedValue([]);
|
||||
const first = startOpenClawDatabaseIntegrityVerifier({ env });
|
||||
const second = startOpenClawDatabaseIntegrityVerifier({ env });
|
||||
try {
|
||||
vi.setSystemTime(Date.now() + 5 * 60_000);
|
||||
requestOpenClawAgentDatabaseQuickCheck({ env, path: registeredPath });
|
||||
requestOpenClawAgentDatabaseQuickCheck({ env, path: unregisteredPath });
|
||||
env.OPENCLAW_STATE_DIR = "/synthetic/changed-after-start";
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
expect(mocks.runDatabaseVerifyWorker).toHaveBeenCalledOnce();
|
||||
expect(mocks.collectOpenClawDatabaseVerifyTargets).toHaveBeenCalledWith({ env: capturedEnv });
|
||||
expect(mocks.applyOpenClawDatabaseVerificationResults).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ env: capturedEnv }),
|
||||
);
|
||||
expect(mocks.runDatabaseVerifyWorker.mock.calls[0]?.[0]).toEqual([
|
||||
registered,
|
||||
expect.objectContaining({ path: registeredPath, check: "quick" }),
|
||||
expect.objectContaining({ path: unregisteredPath, check: "quick" }),
|
||||
]);
|
||||
await vi.advanceTimersByTimeAsync(24 * 60 * 60_000);
|
||||
expect(mocks.runDatabaseVerifyWorker).toHaveBeenCalledTimes(2);
|
||||
expect(mocks.runDatabaseVerifyWorker.mock.calls[1]?.[0]).toEqual([registered]);
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
await second.stop();
|
||||
await first.stop();
|
||||
|
|
@ -230,7 +226,8 @@ describe("database verifier shutdown", () => {
|
|||
return application.promise;
|
||||
});
|
||||
const verifier = startOpenClawDatabaseIntegrityVerifier({ env: {} });
|
||||
await vi.advanceTimersByTimeAsync(5 * 60_000);
|
||||
requestOpenClawAgentDatabaseQuickCheck({ env: {}, path: "/synthetic/first.sqlite" });
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
await entered.promise;
|
||||
const peer = startOpenClawDatabaseIntegrityVerifier({ env: {} });
|
||||
requestOpenClawAgentDatabaseQuickCheck({ env: {}, path: "/synthetic/late.sqlite" });
|
||||
|
|
@ -270,7 +267,8 @@ describe("database verifier shutdown", () => {
|
|||
return results.promise;
|
||||
});
|
||||
const verifier = startOpenClawDatabaseIntegrityVerifier({ env: {} });
|
||||
await vi.advanceTimersByTimeAsync(5 * 60_000);
|
||||
requestOpenClawAgentDatabaseQuickCheck({ env: {}, path: "/synthetic/first.sqlite" });
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
let stopped = false;
|
||||
const stopping = verifier.stop().then(() => {
|
||||
stopped = true;
|
||||
|
|
|
|||
|
|
@ -151,7 +151,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
"DROP INDEX idx_agent_transcript_context_pending; ALTER TABLE session_transcript_active_events DROP COLUMN context_eligible;",
|
||||
);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "transcript eligibility", path: agent.path },
|
||||
{ kind: "agent", label: "transcript eligibility", path: agent.path, check: "quick" },
|
||||
];
|
||||
await expect(runDatabaseVerifyWorker(targets)).resolves.toEqual([
|
||||
{ path: agent.path, ok: true },
|
||||
|
|
@ -165,9 +165,9 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it.skipIf(process.platform === "win32").each([undefined, "quick"] as const)(
|
||||
"preserves live WAL ownership during an open database check (%s)",
|
||||
async (check) => {
|
||||
it.skipIf(process.platform === "win32")(
|
||||
"preserves live WAL ownership during an open database quick check",
|
||||
async () => {
|
||||
const stateDir = tempDirs.make("openclaw-database-verify-live-locks-");
|
||||
const env = { OPENCLAW_STATE_DIR: stateDir };
|
||||
const agent = openOpenClawAgentDatabase({ agentId: "worker-1", env });
|
||||
|
|
@ -186,7 +186,12 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
]);
|
||||
}
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agent.path, check },
|
||||
{
|
||||
kind: "agent",
|
||||
label: "OpenClaw agent database worker-1",
|
||||
path: agent.path,
|
||||
check: "quick",
|
||||
},
|
||||
];
|
||||
|
||||
await expect(runDatabaseVerifyWorker(targets)).resolves.toEqual([
|
||||
|
|
@ -227,55 +232,6 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
},
|
||||
);
|
||||
|
||||
it("detects corruption off-thread, quarantines it, and latches later opens", async () => {
|
||||
const stateDir = tempDirs.make("openclaw-database-verify-");
|
||||
const env = { OPENCLAW_STATE_DIR: stateDir };
|
||||
const agentPath = openOpenClawAgentDatabase({ agentId: "worker-1", env }).path;
|
||||
createUnsafeIndexDrift(agentPath);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath },
|
||||
];
|
||||
|
||||
const results = await runDatabaseVerifyWorker(targets);
|
||||
expect(results).toEqual([
|
||||
{
|
||||
path: agentPath,
|
||||
ok: false,
|
||||
error: expect.stringMatching(/missing from index unsafe_index_records_value/iu),
|
||||
terminal: true,
|
||||
},
|
||||
]);
|
||||
|
||||
await applyOpenClawDatabaseVerificationResults({
|
||||
env,
|
||||
results,
|
||||
targets,
|
||||
});
|
||||
const quarantine = readPersistedQuarantineRow(agentPath, { env });
|
||||
expect(quarantine).toEqual({
|
||||
kind: "agent",
|
||||
quarantinedAt: expect.any(Number),
|
||||
reason: expect.stringMatching(/missing from index unsafe_index_records_value/iu),
|
||||
});
|
||||
|
||||
expect(() => openOpenClawAgentDatabase({ agentId: "worker-1", env })).toThrow(
|
||||
expect.objectContaining({ name: "SqliteIntegrityError" }),
|
||||
);
|
||||
|
||||
closeOpenClawAgentDatabasesForTest();
|
||||
closeOpenClawStateDatabaseForTest();
|
||||
expect(() => openOpenClawAgentDatabase({ agentId: "worker-1", env })).toThrow(
|
||||
expect.objectContaining({
|
||||
name: "SqliteIntegrityError",
|
||||
message: expect.stringContaining(quarantine?.reason ?? ""),
|
||||
}),
|
||||
);
|
||||
clearOpenClawAgentDatabaseOpenFailure(agentPath, { env });
|
||||
expect(() => openOpenClawAgentDatabase({ agentId: "worker-1", env })).toThrow(
|
||||
expect.objectContaining({ name: "SqliteIntegrityError" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("relays a late restart-receipt Worker open to the parent verifier after native opening settles", async () => {
|
||||
const result = await runNodeScript(
|
||||
resolveRuntimeWorkerArgv(resolveRuntimeWorkerUrl(databaseVerifyHostRuntimeEntrypoint)),
|
||||
|
|
@ -316,7 +272,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
},
|
||||
);
|
||||
|
||||
it("quarantines a background quick-check foreign-key failure and rejects the next open", async () => {
|
||||
it("quarantines a quick-check foreign-key failure across close and reopen", async () => {
|
||||
const stateDir = tempDirs.make("openclaw-database-quick-verify-");
|
||||
const env = { OPENCLAW_STATE_DIR: stateDir };
|
||||
const agent = openOpenClawAgentDatabase({ agentId: "worker-1", env });
|
||||
|
|
@ -355,6 +311,19 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
expect.objectContaining({ name: "SqliteIntegrityError" }),
|
||||
);
|
||||
}
|
||||
|
||||
closeOpenClawAgentDatabasesForTest();
|
||||
closeOpenClawStateDatabaseForTest();
|
||||
expect(() => openOpenClawAgentDatabase({ agentId: "worker-1", env })).toThrow(
|
||||
expect.objectContaining({
|
||||
name: "SqliteIntegrityError",
|
||||
message: expect.stringContaining("foreign_key_check failed"),
|
||||
}),
|
||||
);
|
||||
clearOpenClawAgentDatabaseOpenFailure(agent.path, { env });
|
||||
expect(() => openOpenClawAgentDatabase({ agentId: "worker-1", env })).toThrow(
|
||||
expect.objectContaining({ name: "SqliteIntegrityError" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("does not quarantine a healthy database that replaced the verified file", async () => {
|
||||
|
|
@ -369,7 +338,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
fs.copyFileSync(agentPath, healthyReplacementPath);
|
||||
createUnsafeIndexDrift(agentPath);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath },
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath, check: "quick" },
|
||||
];
|
||||
const results = preparedVerificationResults(targets);
|
||||
|
||||
|
|
@ -392,7 +361,12 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
await copyHealthyDatabase(agent.path, healthyReplacementPath);
|
||||
createUnsafeIndexDrift(agent.path);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agent.path },
|
||||
{
|
||||
kind: "agent",
|
||||
label: "OpenClaw agent database worker-1",
|
||||
path: agent.path,
|
||||
check: "quick",
|
||||
},
|
||||
];
|
||||
const results = preparedVerificationResults(targets);
|
||||
|
||||
|
|
@ -417,7 +391,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
await copyHealthyDatabase(state.path, healthyReplacementPath);
|
||||
createUnsafeIndexDrift(state.path);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "state", label: "OpenClaw state database", path: state.path },
|
||||
{ kind: "state", label: "OpenClaw state database", path: state.path, check: "quick" },
|
||||
];
|
||||
const results = preparedVerificationResults(targets);
|
||||
|
||||
|
|
@ -439,7 +413,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
closeOpenClawStateDatabaseForTest();
|
||||
createUnsafeIndexDrift(agentPath);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath },
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath, check: "quick" },
|
||||
];
|
||||
const results = preparedVerificationResults(targets);
|
||||
|
||||
|
|
@ -493,7 +467,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
closeOpenClawStateDatabaseForTest();
|
||||
createUnsafeIndexDrift(agentPath);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath },
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath, check: "quick" },
|
||||
];
|
||||
const results = preparedVerificationResults(targets);
|
||||
|
||||
|
|
@ -513,8 +487,13 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
const state = openOpenClawStateDatabase({ env });
|
||||
const agent = openOpenClawAgentDatabase({ agentId: "worker-1", env });
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "state", label: "OpenClaw state database", path: state.path },
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agent.path },
|
||||
{ kind: "state", label: "OpenClaw state database", path: state.path, check: "quick" },
|
||||
{
|
||||
kind: "agent",
|
||||
label: "OpenClaw agent database worker-1",
|
||||
path: agent.path,
|
||||
check: "quick",
|
||||
},
|
||||
];
|
||||
|
||||
await applyOpenClawDatabaseVerificationResults({
|
||||
|
|
@ -548,7 +527,12 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
fs.rmSync(agent.path);
|
||||
fs.renameSync(replacementPath, agent.path);
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agent.path },
|
||||
{
|
||||
kind: "agent",
|
||||
label: "OpenClaw agent database worker-1",
|
||||
path: agent.path,
|
||||
check: "quick",
|
||||
},
|
||||
];
|
||||
|
||||
await applyOpenClawDatabaseVerificationResults({
|
||||
|
|
@ -858,7 +842,7 @@ describe("OpenClaw database integrity verifier", () => {
|
|||
closeOpenClawAgentDatabasesForTest();
|
||||
closeOpenClawStateDatabaseForTest();
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = [
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath },
|
||||
{ kind: "agent", label: "OpenClaw agent database worker-1", path: agentPath, check: "quick" },
|
||||
];
|
||||
|
||||
await applyOpenClawDatabaseVerificationResults({
|
||||
|
|
|
|||
|
|
@ -7,13 +7,10 @@ import type { OpenClawDatabaseVerifyTarget } from "./openclaw-database-verify.wo
|
|||
import { resolveOpenClawStateSqlitePath } from "./openclaw-state-db.paths.js";
|
||||
|
||||
const log = createSubsystemLogger("state/database-verify");
|
||||
const OPENCLAW_DATABASE_VERIFY_INITIAL_DELAY_MS = 5 * 60_000;
|
||||
const OPENCLAW_DATABASE_VERIFY_INTERVAL_MS = 24 * 60 * 60_000;
|
||||
type QuickCheckQueue = {
|
||||
paths: Set<string>;
|
||||
subscribers: Set<() => void>;
|
||||
active?: object;
|
||||
nextFullCheckAt: number;
|
||||
};
|
||||
const quickCheckQueues = resolveGlobalSingleton(
|
||||
Symbol.for("openclaw.databaseIntegrityQuickChecks"),
|
||||
|
|
@ -24,7 +21,7 @@ function quickCheckQueue(env: NodeJS.ProcessEnv): QuickCheckQueue {
|
|||
const key = path.resolve(resolveOpenClawStateSqlitePath(env));
|
||||
let queue = quickCheckQueues.get(key);
|
||||
if (!queue) {
|
||||
queue = { paths: new Set(), subscribers: new Set(), nextFullCheckAt: Infinity };
|
||||
queue = { paths: new Set(), subscribers: new Set() };
|
||||
quickCheckQueues.set(key, queue);
|
||||
}
|
||||
return queue;
|
||||
|
|
@ -46,15 +43,12 @@ export function requestOpenClawAgentDatabaseQuickCheck(options: {
|
|||
wakeSubscribers(queue);
|
||||
}
|
||||
|
||||
/** Start the Gateway-owned delayed daily integrity verifier and queued quick checks. */
|
||||
/** Consume requested agent quick checks for the listening Gateway. */
|
||||
export function startOpenClawDatabaseIntegrityVerifier(options: { env: NodeJS.ProcessEnv }): {
|
||||
stop: () => Promise<void>;
|
||||
} {
|
||||
const env = { ...options.env };
|
||||
const queue = quickCheckQueue(env);
|
||||
if (queue.subscribers.size === 0) {
|
||||
queue.nextFullCheckAt = Date.now() + OPENCLAW_DATABASE_VERIFY_INITIAL_DELAY_MS;
|
||||
}
|
||||
const owner = {};
|
||||
const inOwnerContext = AsyncLocalStorage.snapshot();
|
||||
let activeWorker: ChildProcess | undefined;
|
||||
|
|
@ -65,69 +59,49 @@ export function startOpenClawDatabaseIntegrityVerifier(options: { env: NodeJS.Pr
|
|||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
|
||||
const schedule = () => {
|
||||
if (stopped || queue.active) {
|
||||
if (stopped || queue.active || queue.paths.size === 0) {
|
||||
return;
|
||||
}
|
||||
if (timer) {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
timer = setTimeout(
|
||||
() => {
|
||||
timer = undefined;
|
||||
if (stopped || queue.active) {
|
||||
return;
|
||||
}
|
||||
queue.active = owner;
|
||||
activeRun = inOwnerContext(run).finally(() => {
|
||||
activeRun = undefined;
|
||||
queue.active = undefined;
|
||||
wakeSubscribers(queue);
|
||||
});
|
||||
},
|
||||
queue.paths.size > 0 ? 0 : Math.max(0, queue.nextFullCheckAt - Date.now()),
|
||||
);
|
||||
timer = setTimeout(() => {
|
||||
timer = undefined;
|
||||
if (stopped || queue.active || queue.paths.size === 0) {
|
||||
return;
|
||||
}
|
||||
queue.active = owner;
|
||||
activeRun = inOwnerContext(run).finally(() => {
|
||||
activeRun = undefined;
|
||||
queue.active = undefined;
|
||||
wakeSubscribers(queue);
|
||||
});
|
||||
}, 0);
|
||||
timer.unref?.();
|
||||
};
|
||||
const run = async () => {
|
||||
const full = Date.now() >= queue.nextFullCheckAt;
|
||||
const quickPaths = [...queue.paths];
|
||||
claimedQuickPaths = quickPaths;
|
||||
queue.paths.clear();
|
||||
try {
|
||||
const {
|
||||
applyOpenClawDatabaseVerificationResults,
|
||||
collectOpenClawDatabaseVerifyTargets,
|
||||
runDatabaseVerifyWorker,
|
||||
} = await import("./openclaw-database-verify.impl.js");
|
||||
const { applyOpenClawDatabaseVerificationResults, runDatabaseVerifyWorker } =
|
||||
await import("./openclaw-database-verify.impl.js");
|
||||
if (stopped) {
|
||||
return;
|
||||
}
|
||||
const targetsByPath = new Map<string, OpenClawDatabaseVerifyTarget>(
|
||||
(full ? collectOpenClawDatabaseVerifyTargets({ env }) : []).map((target) => [
|
||||
path.resolve(target.path),
|
||||
target,
|
||||
]),
|
||||
);
|
||||
for (const pathname of quickPaths) {
|
||||
if (!targetsByPath.has(pathname)) {
|
||||
targetsByPath.set(pathname, {
|
||||
kind: "agent",
|
||||
label: "OpenClaw agent database",
|
||||
path: pathname,
|
||||
check: "quick",
|
||||
});
|
||||
}
|
||||
}
|
||||
const targets = [...targetsByPath.values()];
|
||||
if (targets.length > 0) {
|
||||
const results = await runDatabaseVerifyWorker(targets, {
|
||||
onWorker: (worker) => {
|
||||
activeWorker = worker;
|
||||
},
|
||||
});
|
||||
if (!stopped) {
|
||||
await applyOpenClawDatabaseVerificationResults({ env, results, targets });
|
||||
}
|
||||
const targets: OpenClawDatabaseVerifyTarget[] = quickPaths.map((pathname) => ({
|
||||
kind: "agent",
|
||||
label: "OpenClaw agent database",
|
||||
path: pathname,
|
||||
check: "quick",
|
||||
}));
|
||||
const results = await runDatabaseVerifyWorker(targets, {
|
||||
onWorker: (worker) => {
|
||||
activeWorker = worker;
|
||||
},
|
||||
});
|
||||
if (!stopped) {
|
||||
await applyOpenClawDatabaseVerificationResults({ env, results, targets });
|
||||
}
|
||||
} catch (error) {
|
||||
if (!stopped) {
|
||||
|
|
@ -136,9 +110,6 @@ export function startOpenClawDatabaseIntegrityVerifier(options: { env: NodeJS.Pr
|
|||
} finally {
|
||||
activeWorker = undefined;
|
||||
claimedQuickPaths = [];
|
||||
if (full && !stopped) {
|
||||
queue.nextFullCheckAt = Date.now() + OPENCLAW_DATABASE_VERIFY_INTERVAL_MS;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
|
|
@ -155,7 +126,6 @@ export function startOpenClawDatabaseIntegrityVerifier(options: { env: NodeJS.Pr
|
|||
queue.subscribers.delete(wake);
|
||||
if (queue.subscribers.size === 0) {
|
||||
queue.paths.clear();
|
||||
queue.nextFullCheckAt = Infinity;
|
||||
} else {
|
||||
// Replay before yielding so a later final stop can still discard this work.
|
||||
for (const pathname of claimedQuickPaths) {
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ export type OpenClawDatabaseVerifyTarget = {
|
|||
path: string;
|
||||
kind: "agent" | "state";
|
||||
label: string;
|
||||
check?: "quick";
|
||||
check: "quick";
|
||||
};
|
||||
|
||||
export type OpenClawDatabaseVerifyResult = {
|
||||
|
|
@ -26,7 +26,7 @@ function isVerifyTarget(value: unknown): value is OpenClawDatabaseVerifyTarget {
|
|||
typeof target.path === "string" &&
|
||||
(target.kind === "agent" || target.kind === "state") &&
|
||||
typeof target.label === "string" &&
|
||||
(target.check === undefined || target.check === "quick")
|
||||
target.check === "quick"
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -38,55 +38,32 @@ function formatVerifyError(error: unknown): string {
|
|||
async function verifyOpenClawDatabase(
|
||||
target: OpenClawDatabaseVerifyTarget,
|
||||
): Promise<OpenClawDatabaseVerifyResult> {
|
||||
const [sqlite, integrity, location, source] = await Promise.all([
|
||||
import("../infra/node-sqlite.js"),
|
||||
const [integrity, source] = await Promise.all([
|
||||
import("../infra/sqlite-integrity.js"),
|
||||
import("../infra/sqlite-readonly-location.js"),
|
||||
import("../infra/sqlite-source-handle.js"),
|
||||
]);
|
||||
let cleanup: (() => Promise<boolean>) | undefined;
|
||||
let database: import("node:sqlite").DatabaseSync | undefined;
|
||||
let result = await (async (): Promise<OpenClawDatabaseVerifyResult> => {
|
||||
try {
|
||||
if (target.check === "quick") {
|
||||
source.withSqliteSourceReadDatabase(target.path, "source", (reader) => {
|
||||
reader.exec(`PRAGMA busy_timeout = ${OPENCLAW_SQLITE_BUSY_TIMEOUT_MS}; BEGIN;`);
|
||||
integrity.assertSqliteIntegrity(reader, target.label, "quick_check");
|
||||
reader.exec("ROLLBACK;");
|
||||
});
|
||||
return { path: target.path, ok: true };
|
||||
}
|
||||
const prepared = await location.prepareSqliteReadOnlyLocationInProcess(target.path);
|
||||
cleanup = prepared.cleanupAsync;
|
||||
database = sqlite.openNodeSqliteDatabase(prepared.location, {
|
||||
readOnly: true,
|
||||
});
|
||||
database.exec(`PRAGMA busy_timeout = ${OPENCLAW_SQLITE_BUSY_TIMEOUT_MS};`);
|
||||
integrity.assertSqliteIntegrity(database, target.label);
|
||||
return { path: target.path, ok: true };
|
||||
} catch (error) {
|
||||
const terminal = error instanceof Error && integrity.isTerminalSqliteIntegrityError(error);
|
||||
return {
|
||||
path: target.path,
|
||||
ok: false,
|
||||
error: formatVerifyError(error),
|
||||
terminal,
|
||||
};
|
||||
}
|
||||
})();
|
||||
const failed = (error: unknown): OpenClawDatabaseVerifyResult => ({
|
||||
path: target.path,
|
||||
ok: false,
|
||||
error: formatVerifyError(error),
|
||||
terminal: error instanceof Error && integrity.isTerminalSqliteIntegrityError(error),
|
||||
});
|
||||
let result: OpenClawDatabaseVerifyResult = { path: target.path, ok: true };
|
||||
try {
|
||||
database?.close();
|
||||
source.withSqliteSourceReadDatabase(target.path, "source", (reader) => {
|
||||
try {
|
||||
reader.exec(`PRAGMA busy_timeout = ${OPENCLAW_SQLITE_BUSY_TIMEOUT_MS}; BEGIN;`);
|
||||
integrity.assertSqliteIntegrity(reader, target.label, "quick_check");
|
||||
reader.exec("ROLLBACK;");
|
||||
} catch (error) {
|
||||
// Preserve the check's classification if the source reader also fails to close.
|
||||
result = failed(error);
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
if (result.ok) {
|
||||
result = {
|
||||
path: target.path,
|
||||
ok: false,
|
||||
error: formatVerifyError(error),
|
||||
terminal: false,
|
||||
};
|
||||
result = failed(error);
|
||||
}
|
||||
} finally {
|
||||
await cleanup?.();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue