mirror of https://github.com/openclaw/openclaw.git synced 2026-08-01 19:15:45 +00:00

History

Jesse Merhi d7627d6f4c refactor(prompt): use plain inbound context labels and drop system-tag sanitizer (#112000 ) * refactor(prompt): plain inbound context labels with a provenance marker Replaces trust-worded inbound context labels ("(untrusted metadata)", "(untrusted, for context)") with plain labels plus a fixed provenance marker suffix appended to every OpenClaw-injected context header. Detection keys on the marker, not label text, so strippers stay correct across UI, TUI, replay, /trace segmentation, memory recall, and the Swift chat preprocessor. Drops sanitizeInboundSystemTags in favor of the marker boundary plus trusted system-prompt narration. Renames the untrusted-named plugin SDK context identifiers to channel-provenance names, keeping deprecated aliases registered for removal after 2026-09-08. Adds `openclaw doctor --fix` migrations that rewrite legacy inbound labels in stored SQLite transcripts and purge legacy envelope- contaminated LanceDB recall rows. * fix(ci): resolve gate failures for plain inbound context labels - doctor sqlite readers: open read-only connections via openNodeSqliteDatabase so the Kysely connection-boundary guardrail holds; unexport the now-internal transcript snapshot type (Knip unused-export gate). - compat registry: split the record table into registry-records.ts and plugin-sdk-subpath-records.ts. The new compat record pushed registry.ts past the 700-line oxlint cap; suppressions are disallowed, so follow the existing sibling record-module pattern. Public exports and PluginCompatCode literals unchanged. - acp-runtime test: assert current finalization behavior (newline normalization only). The bracket de-fang and System: rewrite it expected were removed with sanitizeInboundSystemTags; forged system lines are neutralized at the system-event queue, the single chokepoint feeding the System:-per-line render. - regenerate docs_map and the plugin SDK API baseline manifest. * fix(prompt): harden inbound context label migration and drop in-band sanitizer Review follow-ups on the plain-label + provenance-marker change: - Remove src/security/system-tags.ts. Rewriting inbound text to neutralize look-alike `System:`/`[System]` markers corrupted legitimate user text and is not a real injection boundary; role separation plus external-content wrapping is. Explicit product decision, recorded at the system-event queue. - Narrow the LanceDB legacy-row purge so it cannot delete benign memories. It now requires a complete known legacy sentinel line, a legacy label followed by a fenced JSON body, or the complete legacy external-content header. The prior predicates matched ordinary prose such as `Notes (untrusted metadata):`, and deletion is irreversible. - Make explicit-empty canonical ChannelStructuredContext win over the deprecated alias via a present/absent result instead of collapsing `[]` to undefined. - Keep `\r?` in the active-memory doctor rule. It is the only rule spanning the header's line break, migrated assistant rows skip newline normalization, and without it the marked-header replace wins and the body strips to empty. Added a CRLF regression test. - Fix stale comments that described removed behavior, and cover the Swift prose-block strip path. Claude-Session: https://claude.ai/code/session_01WNzsPddQmxy9Y7jKD4wAxH		2026-07-27 11:27:53 +10:00
..
rules/openclaw-policy
check-rule-metadata.mjs
compile-rules.mjs	chore(lint): enable stricter error rules	2026-06-01 01:12:21 +01:00
precise.yml	refactor(prompt): use plain inbound context labels and drop system-tag sanitizer (#112000 )	2026-07-27 11:27:53 +10:00
README.md

README.md

Compiled OpenGrep super-configs

precise.yml is OpenClaw's shipped precise OpenGrep rulepack. Each rule is tied to a source advisory, vulnerability report, or review identifier through metadata and is intended to have concrete coverage of the original vulnerable behavior or a verified variant.

Rule provenance lives in each compiled rule's metadata; no separate manifest is committed or generated by default.

Noisy exploratory rules are intentionally kept out of the tracked repo. Anything appended to precise.yml must be low-noise enough to run as a blocking PR-diff check and as a manual full-repository audit.

Editing rules

precise.yml is the checked-in compiled rulepack. Prefer changing source rule YAML and rerunning security/opengrep/compile-rules.mjs instead of hand-editing compiled rules. The compiler appends new rule IDs by default; use --replace-precise only when intentionally rebuilding the rulepack from a complete source folder. Direct edits are discouraged because they can bypass ID, metadata, duplicate, and OpenGrep validation.

Rule naming and metadata

Every rule's id is rewritten to <source-id>.<original-id>. Every rule's metadata block is augmented with source fields enforced by pnpm check:opengrep-rule-metadata:

Key	Value
`ghsa`	`GHSA-xxxx-xxxx-xxxx` for GHSA-backed rules
`advisory-id`	non-GHSA source identifier, or the GHSA ID normalized by the compiler
`advisory-url`	durable URL to the advisory, report, review record, or source context
`detector-bucket`	`precise`
`source-rule-id`	the original source rule id
`source-file`	optional source YAML file used during compilation

Recompiling

# from the openclaw repo root
node security/opengrep/compile-rules.mjs \
  --rules-dir <folder-with-source-rule-yaml>

The script:

Recursively walks every .yml / .yaml file under --rules-dir
Reads top-level rules arrays from those source files
Requires each source rule to provide metadata.ghsa or metadata.advisory-id
Requires metadata.advisory-url for non-GHSA source identifiers
Rewrites ids and injects metadata as above
Appends only new precise rule ids to the existing precise.yml by default; pass --replace-precise to rebuild it from just the supplied source folder
Runs opengrep scan --no-strict against an empty target to identify schema-invalid or parser-invalid rules and drops mapped bad rules so the published super-config loads cleanly
Writes precise.yml

Skipped, duplicate, or invalid rules are summarized on stdout/stderr for follow-up.

Validating locally

pnpm check:opengrep-rule-metadata
opengrep validate security/opengrep/precise.yml

The metadata check must pass before rules are committed. OpenGrep validation must exit zero. Warnings about unknown fields are acceptable only when OpenGrep still reports Configuration is valid and a non-zero rule count. The compile script drops mapped schema/parser-invalid rules and fails closed when OpenGrep validation itself cannot be completed.

Running locally

scripts/run-opengrep.sh

For SARIF output matching the PR workflow's diff-scoped scan:

scripts/run-opengrep.sh --changed --sarif

For SARIF output matching the manual full-repository workflow:

scripts/run-opengrep.sh --sarif

Why `--no-strict`?

Some generated rules trigger non-fatal opengrep warnings (for example, unknown-field warnings on compatibility-only keys). --no-strict keeps opengrep's exit code clean for those warnings. Parser-invalid rules are still dropped during compilation so the checked-in super-config validates before CI uses it.

Why `--no-git-ignore`?

Some OpenClaw paths are excluded by .gitignore for build reasons even though they contain meaningful source code we want scanned. --no-git-ignore keeps opengrep from skipping them.