mirror of
https://github.com/openclaw/openclaw.git
synced 2026-04-28 06:31:11 +00:00
fix: keep session history redaction forced
This commit is contained in:
parent
5e8cc1d9c2
commit
ccc9dd5eef
9 changed files with 115 additions and 13 deletions
|
|
@ -19,6 +19,7 @@ Docs: https://docs.openclaw.ai
|
|||
|
||||
- Docs/tools: clarify that `tools.profile: "messaging"` is intentionally narrow and that `tools.profile: "full"` is the unrestricted baseline for broader command/control access. Carries forward #39954. Thanks @posigit.
|
||||
- Control UI/Agents: redact tool-call args, partial/final results, derived exec output, and configured custom secret patterns before streaming tool events to the Control UI, so tool output cannot expose provider or channel credentials. Fixes #72283. (#72319) Thanks @volcano303 and @BunsDev.
|
||||
- Agents/sessions: keep `sessions_history` recall redaction enabled even when general log redaction is disabled, and clarify that safety-boundary UI/tool/diagnostic payloads still redact independently of `logging.redactSensitive`. Carries forward #72319. Thanks @volcano303 and @BunsDev.
|
||||
- Providers/Codex: pass agent and workspace directories into provider stream wrappers so Codex native `web_search` activation can evaluate the correct auth context, and smoke-test the built status-message runtime by resolving the emitted bundle name. Carries forward #67843; refs #65909. Thanks @neilofneils404.
|
||||
- Models/fallbacks: treat user-selected session models as exact choices, so `/model ollama/...` and model-picker switches fail visibly when the selected provider is unreachable instead of answering from an unrelated configured fallback. Fixes #73023. Thanks @pavelyortho-cyber.
|
||||
- CLI/model probes: fail local `infer model run` probes when the provider returns no text output, so unreachable local providers and empty completions no longer look like successful smoke tests. Refs #73023. Thanks @pavelyortho-cyber.
|
||||
|
|
|
|||
|
|
@ -863,7 +863,7 @@ Notes:
|
|||
- Set `logging.file` for a stable path.
|
||||
- `consoleLevel` bumps to `debug` when `--verbose`.
|
||||
- `maxFileBytes`: maximum active log file size in bytes before rotation (positive integer; default: `104857600` = 100 MB). OpenClaw keeps up to five numbered archives beside the active file.
|
||||
- `redactSensitive` / `redactPatterns`: best-effort masking for console output, file logs, OTLP log records, and persisted session transcript text.
|
||||
- `redactSensitive` / `redactPatterns`: best-effort masking for console output, file logs, OTLP log records, and persisted session transcript text. `redactSensitive: "off"` only disables this general log/transcript policy; UI/tool/diagnostic safety surfaces still redact secrets before emission.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ You can tune console verbosity independently via:
|
|||
## Redaction
|
||||
|
||||
OpenClaw can mask sensitive tokens before log or transcript output leaves the
|
||||
process. The same redaction policy is applied at console, file-log, OTLP
|
||||
process. This logging redaction policy is applied at console, file-log, OTLP
|
||||
log-record, and session transcript text sinks, so matching secret values are
|
||||
masked before JSONL lines or messages are written to disk.
|
||||
|
||||
|
|
@ -65,6 +65,13 @@ masked before JSONL lines or messages are written to disk.
|
|||
- Matches are masked by keeping the first 6 + last 4 chars (length >= 18), otherwise `***`.
|
||||
- Defaults cover common key assignments, CLI flags, JSON fields, bearer headers, PEM blocks, and popular token prefixes.
|
||||
|
||||
Some safety boundaries always redact regardless of `logging.redactSensitive`.
|
||||
That includes Control UI tool-call events, `sessions_history` tool output,
|
||||
diagnostics support exports, provider error observations, exec approval command
|
||||
display, and Gateway WebSocket protocol logs. These surfaces may still use
|
||||
`logging.redactPatterns` as additional patterns, but `redactSensitive: "off"`
|
||||
does not make them emit raw secrets.
|
||||
|
||||
## Gateway WebSocket logs
|
||||
|
||||
The gateway prints WebSocket protocol logs in two modes:
|
||||
|
|
|
|||
|
|
@ -219,6 +219,14 @@ masked before the line or message is written to disk. Redaction is best-effort:
|
|||
it applies to text-bearing message content and log strings, not every
|
||||
identifier or binary payload field.
|
||||
|
||||
`logging.redactSensitive: "off"` only disables this general log/transcript
|
||||
policy. OpenClaw still redacts safety-boundary payloads that can be shown to UI
|
||||
clients, support bundles, diagnostics observers, approval prompts, or agent
|
||||
tools. Examples include Control UI tool-call events, `sessions_history` output,
|
||||
diagnostics support exports, provider error observations, exec approval command
|
||||
display, and Gateway WebSocket protocol logs. Custom `logging.redactPatterns`
|
||||
can still add project-specific patterns on those surfaces.
|
||||
|
||||
## Diagnostics and OpenTelemetry
|
||||
|
||||
Diagnostics are structured, machine-readable events for model runs and
|
||||
|
|
|
|||
86
src/agents/tools/sessions-history-tool.test.ts
Normal file
86
src/agents/tools/sessions-history-tool.test.ts
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import type { callGateway as gatewayCall } from "../../gateway/call.js";
|
||||
|
||||
type CallGatewayRequest = Parameters<typeof gatewayCall>[0];
|
||||
|
||||
let createSessionsHistoryTool: typeof import("./sessions-history-tool.js").createSessionsHistoryTool;
|
||||
let previousConfigPath: string | undefined;
|
||||
let tempDir: string | undefined;
|
||||
|
||||
function useLoggingConfig(name: string, logging: Record<string, unknown>): void {
|
||||
if (!tempDir) {
|
||||
throw new Error("tempDir not initialized");
|
||||
}
|
||||
const configPath = path.join(tempDir, name);
|
||||
fs.writeFileSync(configPath, `${JSON.stringify({ logging })}\n`, "utf8");
|
||||
process.env.OPENCLAW_CONFIG_PATH = configPath;
|
||||
}
|
||||
|
||||
function createHistoryToolWithMessage(content: string) {
|
||||
return createSessionsHistoryTool({
|
||||
config: {},
|
||||
callGateway: async <T = Record<string, unknown>>(request: CallGatewayRequest): Promise<T> => {
|
||||
if (request.method === "chat.history") {
|
||||
return {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content,
|
||||
},
|
||||
],
|
||||
} as T;
|
||||
}
|
||||
return {} as T;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
describe("sessions_history redaction", () => {
|
||||
beforeAll(async () => {
|
||||
previousConfigPath = process.env.OPENCLAW_CONFIG_PATH;
|
||||
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-sessions-history-redact-"));
|
||||
useLoggingConfig("redaction-off.json", { redactSensitive: "off" });
|
||||
({ createSessionsHistoryTool } = await import("./sessions-history-tool.js"));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
if (previousConfigPath === undefined) {
|
||||
delete process.env.OPENCLAW_CONFIG_PATH;
|
||||
} else {
|
||||
process.env.OPENCLAW_CONFIG_PATH = previousConfigPath;
|
||||
}
|
||||
if (tempDir) {
|
||||
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("redacts recalled session text even when log redaction is disabled", async () => {
|
||||
useLoggingConfig("redaction-off.json", { redactSensitive: "off" });
|
||||
const tool = createHistoryToolWithMessage("OPENROUTER_API_KEY=sk-or-v1-abcdef0123456789");
|
||||
|
||||
const result = await tool.execute("call-1", { sessionKey: "main" });
|
||||
const serialized = JSON.stringify(result.details);
|
||||
|
||||
expect(serialized).not.toContain("sk-or-v1-abcdef0123456789");
|
||||
expect(serialized).toContain("OPENROUTER_API_KEY=");
|
||||
expect(result.details).toMatchObject({ contentRedacted: true });
|
||||
});
|
||||
|
||||
it("applies custom redaction patterns to recalled session text", async () => {
|
||||
useLoggingConfig("custom-patterns.json", {
|
||||
redactSensitive: "off",
|
||||
redactPatterns: [String.raw`\binternal-ticket-[A-Za-z0-9]+\b`],
|
||||
});
|
||||
const tool = createHistoryToolWithMessage("follow up on internal-ticket-AbC12345");
|
||||
|
||||
const result = await tool.execute("call-1", { sessionKey: "main" });
|
||||
const serialized = JSON.stringify(result.details);
|
||||
|
||||
expect(serialized).not.toContain("internal-ticket-AbC12345");
|
||||
expect(serialized).toContain("intern");
|
||||
expect(result.details).toMatchObject({ contentRedacted: true });
|
||||
});
|
||||
});
|
||||
|
|
@ -4,7 +4,7 @@ import type { OpenClawConfig } from "../../config/types.openclaw.js";
|
|||
import { callGateway } from "../../gateway/call.js";
|
||||
import { capArrayByJsonBytes } from "../../gateway/session-utils.fs.js";
|
||||
import { jsonUtf8Bytes } from "../../infra/json-utf8-bytes.js";
|
||||
import { redactSensitiveText } from "../../logging/redact.js";
|
||||
import { redactToolPayloadText } from "../../logging/redact.js";
|
||||
import { readStringValue } from "../../shared/string-coerce.js";
|
||||
import { truncateUtf16Safe } from "../../utils.js";
|
||||
import {
|
||||
|
|
@ -40,9 +40,9 @@ function truncateHistoryText(text: string): {
|
|||
truncated: boolean;
|
||||
redacted: boolean;
|
||||
} {
|
||||
// Redact credentials, API keys, tokens before returning session history.
|
||||
// Prevents sensitive data leakage via sessions_history tool (OC-07).
|
||||
const sanitized = redactSensitiveText(text);
|
||||
// sessions_history is a tool surface, not a log sink. Keep it redacted even
|
||||
// when operators disable general-purpose log redaction.
|
||||
const sanitized = redactToolPayloadText(text);
|
||||
const redacted = sanitized !== text;
|
||||
if (sanitized.length <= SESSIONS_HISTORY_TEXT_MAX_CHARS) {
|
||||
return { text: sanitized, truncated: false, redacted };
|
||||
|
|
|
|||
|
|
@ -466,7 +466,7 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
|
|||
],
|
||||
title: "Sensitive Data Redaction Mode",
|
||||
description:
|
||||
'Sensitive redaction mode: "off" disables built-in masking, while "tools" redacts sensitive tool/config payload fields in log sinks and persisted transcript text. Keep "tools" enabled unless logs and transcripts are isolated.',
|
||||
'Sensitive log/transcript redaction mode: "off" disables general log and transcript masking, while "tools" redacts sensitive tool/config payload fields in those sinks. Safety-boundary UI, tool, and diagnostic payloads may still redact even when this is "off".',
|
||||
},
|
||||
redactPatterns: {
|
||||
type: "array",
|
||||
|
|
@ -475,7 +475,7 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
|
|||
},
|
||||
title: "Custom Redaction Patterns",
|
||||
description:
|
||||
"Additional custom redact regex patterns applied to log output and persisted transcript text before storage. Use this to mask org-specific tokens and identifiers not covered by built-in redaction rules.",
|
||||
"Additional custom redact regex patterns applied to log output, persisted transcript text, and safety-boundary UI/tool/diagnostic payloads before emission. Use this to mask org-specific tokens and identifiers not covered by built-in redaction rules.",
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
|
|
@ -24079,12 +24079,12 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
|
|||
},
|
||||
"logging.redactSensitive": {
|
||||
label: "Sensitive Data Redaction Mode",
|
||||
help: 'Sensitive redaction mode: "off" disables built-in masking, while "tools" redacts sensitive tool/config payload fields in log sinks and persisted transcript text. Keep "tools" enabled unless logs and transcripts are isolated.',
|
||||
help: 'Sensitive log/transcript redaction mode: "off" disables general log and transcript masking, while "tools" redacts sensitive tool/config payload fields in those sinks. Safety-boundary UI, tool, and diagnostic payloads may still redact even when this is "off".',
|
||||
tags: ["privacy", "observability"],
|
||||
},
|
||||
"logging.redactPatterns": {
|
||||
label: "Custom Redaction Patterns",
|
||||
help: "Additional custom redact regex patterns applied to log output and persisted transcript text before storage. Use this to mask org-specific tokens and identifiers not covered by built-in redaction rules.",
|
||||
help: "Additional custom redact regex patterns applied to log output, persisted transcript text, and safety-boundary UI/tool/diagnostic payloads before emission. Use this to mask org-specific tokens and identifiers not covered by built-in redaction rules.",
|
||||
tags: ["privacy", "observability"],
|
||||
},
|
||||
"cli.banner": {
|
||||
|
|
|
|||
|
|
@ -43,9 +43,9 @@ export const FIELD_HELP: Record<string, string> = {
|
|||
"logging.consoleStyle":
|
||||
'Console output format style: "pretty", "compact", or "json" based on operator and ingestion needs. Use json for machine parsing pipelines and pretty/compact for human-first terminal workflows.',
|
||||
"logging.redactSensitive":
|
||||
'Sensitive redaction mode: "off" disables built-in masking, while "tools" redacts sensitive tool/config payload fields in log sinks and persisted transcript text. Keep "tools" enabled unless logs and transcripts are isolated.',
|
||||
'Sensitive log/transcript redaction mode: "off" disables general log and transcript masking, while "tools" redacts sensitive tool/config payload fields in those sinks. Safety-boundary UI, tool, and diagnostic payloads may still redact even when this is "off".',
|
||||
"logging.redactPatterns":
|
||||
"Additional custom redact regex patterns applied to log output and persisted transcript text before storage. Use this to mask org-specific tokens and identifiers not covered by built-in redaction rules.",
|
||||
"Additional custom redact regex patterns applied to log output, persisted transcript text, and safety-boundary UI/tool/diagnostic payloads before emission. Use this to mask org-specific tokens and identifiers not covered by built-in redaction rules.",
|
||||
cli: "CLI presentation controls for local command output behavior such as banner and tagline style. Use this section to keep startup output aligned with operator preference without changing runtime behavior.",
|
||||
"cli.banner":
|
||||
"CLI startup banner controls for title/version line and tagline style behavior. Keep banner enabled for fast version/context checks, then tune tagline mode to your preferred noise level.",
|
||||
|
|
|
|||
|
|
@ -225,7 +225,7 @@ export type LoggingConfig = {
|
|||
maxFileBytes?: number;
|
||||
consoleLevel?: "silent" | "fatal" | "error" | "warn" | "info" | "debug" | "trace";
|
||||
consoleStyle?: "pretty" | "compact" | "json";
|
||||
/** Redact sensitive tokens in log sinks and persisted transcript text. Default: "tools". */
|
||||
/** Redact sensitive tokens in log sinks and persisted transcript text. Default: "tools". Safety-boundary UI/tool/diagnostic payloads may still redact when this is "off". */
|
||||
redactSensitive?: "off" | "tools";
|
||||
/** Regex patterns used to redact sensitive tokens from logs and transcripts. */
|
||||
redactPatterns?: string[];
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue