feat: render inline web chat widgets via capability-gated show_widget tool (#101840)

Adds client-capability-gated tool availability: gateway clients declare
capabilities at connect (new inline-widgets cap), chat.send stamps them into
the run context, and every tool assembly path (embedded runner, queued
followups, Codex app-server harness, plugin-only construction plans) drops
tools whose requiredClientCaps the originating client did not declare. The
Canvas plugin ships the first such tool, show_widget: agents pass SVG or an
HTML fragment plus a title; the plugin hosts it as a bounded, retention-scoped
Canvas document and returns the existing canvas preview handle, which web chat
renders as a sandboxed iframe fitted to the widget's reported content height.
Widget frames never get allow-same-origin (per-preview sandbox ceiling,
including the sidebar path) and the Canvas host serves widget documents with a
CSP sandbox header so direct navigation runs in an opaque origin. Verified
live end-to-end on a Testbox with gpt-5.5 (screenshots on the PR). CLI-backed
model backends do not carry client caps yet and stay fail-closed (#102577).

Closes #101790
This commit is contained in:
Peter Steinberger 2026-07-09 12:29:50 +01:00 committed by GitHub
parent de2af01c9c
commit 3ccf2a0739
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
63 changed files with 1038 additions and 32 deletions

View file

@ -1330,6 +1330,7 @@
"tools/btw",
"tools/code-execution",
"tools/diffs",
"tools/show-widget",
"tools/elevated",
"tools/permission-modes",
"tools/exec-approvals",

View file

@ -3606,6 +3606,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
- Headings:
- H2: Transport and framing
- H2: Handshake
- H3: Client capabilities
- H3: Node connect example
- H2: Roles and scopes
- H2: Presence
@ -9623,6 +9624,14 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
- H2: Notes
- H2: Related
## tools/show-widget.md
- Route: /tools/show-widget
- Headings:
- H2: Use the tool
- H2: Security and storage
- H2: Related
## tools/skill-workshop.md
- Route: /tools/skill-workshop

View file

@ -167,6 +167,15 @@ stale CLI/device pairing baselines blocking local backend work. Remote,
browser-origin, node, and explicit device-token/device-identity clients still
go through normal pairing and scope-upgrade checks.
### Client capabilities
Operator clients may advertise optional capabilities in `connect.params.caps`:
- `tool-events`: accepts structured tool lifecycle events.
- `inline-widgets`: can render hosted inline widget tool results.
Client capabilities describe the connected client, not authorization. Agent tools may declare required capabilities; the Gateway omits those tools unless every requirement appears in the originating client's `caps`. Channel-originated runs have no Gateway client capabilities, so capability-gated tools are unavailable even when tool policy explicitly allows them.
### Node connect example
```json

View file

@ -109,6 +109,7 @@ for contract details.
Common plugin-provided tools include:
- [Diffs](/tools/diffs) for rendering file and markdown diffs
- [Show widget](/tools/show-widget) for self-contained inline SVG and HTML in web chat
- [LLM Task](/tools/llm-task) for JSON-only workflow steps
- [Lobster](/tools/lobster) for typed workflows with resumable approvals
- [Tokenjuice](/tools/tokenjuice) for compacting noisy `exec` and `bash` tool

44
docs/tools/show-widget.md Normal file
View file

@ -0,0 +1,44 @@
---
summary: "Render self-contained SVG or HTML widgets inline in web chat"
title: "Show widget"
sidebarTitle: "Show widget"
read_when:
- You want an agent to render an interactive result inside web chat
- You need the show_widget input, security, or retention contract
---
`show_widget` renders a self-contained SVG or HTML fragment inline in the Control UI chat transcript. The bundled Canvas plugin owns the tool and hosts each result as a same-origin Canvas document.
The tool is available only when the originating Gateway client declares the `inline-widgets` capability. The Control UI declares this capability automatically. Channel runs such as Telegram and WhatsApp do not receive `show_widget`.
Capability transport currently covers the embedded runner and the Codex app-server backend. CLI-backed model backends do not yet carry client capabilities, so capability-gated tools stay unavailable (fail closed) on those backends.
## Use the tool
The agent supplies two required strings:
<ParamField path="title" type="string" required>
Short title shown with the inline preview and in the hosted document title.
</ParamField>
<ParamField path="widget_code" type="string" required>
Self-contained SVG or HTML fragment. Input beginning with `<svg` after trimming is rendered in SVG mode; all other input is treated as an HTML fragment. Maximum length: 262,144 characters.
</ParamField>
The tool result includes a Canvas preview handle, so web chat renders the widget directly from the tool call and restores it after history reload. Transcripts that do not render previews still show the hosted Canvas path.
## Security and storage
Widget documents use a restrictive Content Security Policy: inline style and script are allowed, images may use `data:` URLs, and external fetches and resource loads are blocked. Keep all markup, styles, scripts, and image data inside `widget_code`.
The iframe always omits `allow-same-origin`, even when the Control UI's global embed mode is `trusted`, so widget scripts cannot read the parent application origin. The Canvas host also serves widget documents with a `Content-Security-Policy: sandbox allow-scripts` response header, so opening the hosted URL directly still runs the widget in an opaque origin instead of the Control UI origin. Browser sandboxing does not prevent a script from navigating its own iframe; only render widget code you are willing to execute in that isolated frame.
The iframe also follows [`gateway.controlUi.embedSandbox`](/web/control-ui#hosted-embeds). The default `scripts` tier supports interactive widgets while preserving origin isolation.
Canvas retains at most 32 widgets per session (or per agent when no session is available). Creating another widget removes the oldest document in that scope.
## Related
- [Control UI hosted embeds](/web/control-ui#hosted-embeds)
- [Canvas plugin](/plugins/reference/canvas)
- [Gateway protocol client capabilities](/gateway/protocol#client-capabilities)

View file

@ -328,6 +328,8 @@ Web Push is independent of the iOS APNS relay path (see [Configuration](/gateway
Assistant messages can render hosted web content inline with the `[embed ...]` shortcode. The iframe sandbox policy is controlled by `gateway.controlUi.embedSandbox`:
The bundled Canvas plugin also provides [`show_widget`](/tools/show-widget) to render self-contained SVG or HTML directly from a tool call. The browser advertises the `inline-widgets` Gateway capability, and the resulting Canvas document remains available when chat history reloads. Channel-originated runs do not receive this tool.
<Tabs>
<Tab title="strict">
Disables script execution inside hosted embeds.