mirror of
https://github.com/openclaw/openclaw.git
synced 2026-07-09 15:59:30 +00:00
feat: render inline web chat widgets via capability-gated show_widget tool (#101840)
Adds client-capability-gated tool availability: gateway clients declare capabilities at connect (new inline-widgets cap), chat.send stamps them into the run context, and every tool assembly path (embedded runner, queued followups, Codex app-server harness, plugin-only construction plans) drops tools whose requiredClientCaps the originating client did not declare. The Canvas plugin ships the first such tool, show_widget: agents pass SVG or an HTML fragment plus a title; the plugin hosts it as a bounded, retention-scoped Canvas document and returns the existing canvas preview handle, which web chat renders as a sandboxed iframe fitted to the widget's reported content height. Widget frames never get allow-same-origin (per-preview sandbox ceiling, including the sidebar path) and the Canvas host serves widget documents with a CSP sandbox header so direct navigation runs in an opaque origin. Verified live end-to-end on a Testbox with gpt-5.5 (screenshots on the PR). CLI-backed model backends do not carry client caps yet and stay fail-closed (#102577). Closes #101790
This commit is contained in:
parent
de2af01c9c
commit
3ccf2a0739
63 changed files with 1038 additions and 32 deletions
|
|
@ -1330,6 +1330,7 @@
|
|||
"tools/btw",
|
||||
"tools/code-execution",
|
||||
"tools/diffs",
|
||||
"tools/show-widget",
|
||||
"tools/elevated",
|
||||
"tools/permission-modes",
|
||||
"tools/exec-approvals",
|
||||
|
|
|
|||
|
|
@ -3606,6 +3606,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
|
|||
- Headings:
|
||||
- H2: Transport and framing
|
||||
- H2: Handshake
|
||||
- H3: Client capabilities
|
||||
- H3: Node connect example
|
||||
- H2: Roles and scopes
|
||||
- H2: Presence
|
||||
|
|
@ -9623,6 +9624,14 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
|
|||
- H2: Notes
|
||||
- H2: Related
|
||||
|
||||
## tools/show-widget.md
|
||||
|
||||
- Route: /tools/show-widget
|
||||
- Headings:
|
||||
- H2: Use the tool
|
||||
- H2: Security and storage
|
||||
- H2: Related
|
||||
|
||||
## tools/skill-workshop.md
|
||||
|
||||
- Route: /tools/skill-workshop
|
||||
|
|
|
|||
|
|
@ -167,6 +167,15 @@ stale CLI/device pairing baselines blocking local backend work. Remote,
|
|||
browser-origin, node, and explicit device-token/device-identity clients still
|
||||
go through normal pairing and scope-upgrade checks.
|
||||
|
||||
### Client capabilities
|
||||
|
||||
Operator clients may advertise optional capabilities in `connect.params.caps`:
|
||||
|
||||
- `tool-events`: accepts structured tool lifecycle events.
|
||||
- `inline-widgets`: can render hosted inline widget tool results.
|
||||
|
||||
Client capabilities describe the connected client, not authorization. Agent tools may declare required capabilities; the Gateway omits those tools unless every requirement appears in the originating client's `caps`. Channel-originated runs have no Gateway client capabilities, so capability-gated tools are unavailable even when tool policy explicitly allows them.
|
||||
|
||||
### Node connect example
|
||||
|
||||
```json
|
||||
|
|
|
|||
|
|
@ -109,6 +109,7 @@ for contract details.
|
|||
Common plugin-provided tools include:
|
||||
|
||||
- [Diffs](/tools/diffs) for rendering file and markdown diffs
|
||||
- [Show widget](/tools/show-widget) for self-contained inline SVG and HTML in web chat
|
||||
- [LLM Task](/tools/llm-task) for JSON-only workflow steps
|
||||
- [Lobster](/tools/lobster) for typed workflows with resumable approvals
|
||||
- [Tokenjuice](/tools/tokenjuice) for compacting noisy `exec` and `bash` tool
|
||||
|
|
|
|||
44
docs/tools/show-widget.md
Normal file
44
docs/tools/show-widget.md
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
summary: "Render self-contained SVG or HTML widgets inline in web chat"
|
||||
title: "Show widget"
|
||||
sidebarTitle: "Show widget"
|
||||
read_when:
|
||||
- You want an agent to render an interactive result inside web chat
|
||||
- You need the show_widget input, security, or retention contract
|
||||
---
|
||||
|
||||
`show_widget` renders a self-contained SVG or HTML fragment inline in the Control UI chat transcript. The bundled Canvas plugin owns the tool and hosts each result as a same-origin Canvas document.
|
||||
|
||||
The tool is available only when the originating Gateway client declares the `inline-widgets` capability. The Control UI declares this capability automatically. Channel runs such as Telegram and WhatsApp do not receive `show_widget`.
|
||||
|
||||
Capability transport currently covers the embedded runner and the Codex app-server backend. CLI-backed model backends do not yet carry client capabilities, so capability-gated tools stay unavailable (fail closed) on those backends.
|
||||
|
||||
## Use the tool
|
||||
|
||||
The agent supplies two required strings:
|
||||
|
||||
<ParamField path="title" type="string" required>
|
||||
Short title shown with the inline preview and in the hosted document title.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="widget_code" type="string" required>
|
||||
Self-contained SVG or HTML fragment. Input beginning with `<svg` after trimming is rendered in SVG mode; all other input is treated as an HTML fragment. Maximum length: 262,144 characters.
|
||||
</ParamField>
|
||||
|
||||
The tool result includes a Canvas preview handle, so web chat renders the widget directly from the tool call and restores it after history reload. Transcripts that do not render previews still show the hosted Canvas path.
|
||||
|
||||
## Security and storage
|
||||
|
||||
Widget documents use a restrictive Content Security Policy: inline style and script are allowed, images may use `data:` URLs, and external fetches and resource loads are blocked. Keep all markup, styles, scripts, and image data inside `widget_code`.
|
||||
|
||||
The iframe always omits `allow-same-origin`, even when the Control UI's global embed mode is `trusted`, so widget scripts cannot read the parent application origin. The Canvas host also serves widget documents with a `Content-Security-Policy: sandbox allow-scripts` response header, so opening the hosted URL directly still runs the widget in an opaque origin instead of the Control UI origin. Browser sandboxing does not prevent a script from navigating its own iframe; only render widget code you are willing to execute in that isolated frame.
|
||||
|
||||
The iframe also follows [`gateway.controlUi.embedSandbox`](/web/control-ui#hosted-embeds). The default `scripts` tier supports interactive widgets while preserving origin isolation.
|
||||
|
||||
Canvas retains at most 32 widgets per session (or per agent when no session is available). Creating another widget removes the oldest document in that scope.
|
||||
|
||||
## Related
|
||||
|
||||
- [Control UI hosted embeds](/web/control-ui#hosted-embeds)
|
||||
- [Canvas plugin](/plugins/reference/canvas)
|
||||
- [Gateway protocol client capabilities](/gateway/protocol#client-capabilities)
|
||||
|
|
@ -328,6 +328,8 @@ Web Push is independent of the iOS APNS relay path (see [Configuration](/gateway
|
|||
|
||||
Assistant messages can render hosted web content inline with the `[embed ...]` shortcode. The iframe sandbox policy is controlled by `gateway.controlUi.embedSandbox`:
|
||||
|
||||
The bundled Canvas plugin also provides [`show_widget`](/tools/show-widget) to render self-contained SVG or HTML directly from a tool call. The browser advertises the `inline-widgets` Gateway capability, and the resulting Canvas document remains available when chat history reloads. Channel-originated runs do not receive this tool.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="strict">
|
||||
Disables script execution inside hosted embeds.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue