openclaw/docs/plugins
Peter Steinberger f78ba09120
feat(talk): support GPT-Live realtime voice over ChatGPT OAuth (#115226)
* feat(gateway): share the canonical browser-origin policy with plugins

Export resolveAcceptedBrowserOrigin through openclaw/plugin-sdk/webhook-request-guards
so browser-facing plugin routes reuse the Gateway's real origin contract instead of a
narrow allowedOrigins array check. Private LAN/Tailnet Control UI loads and the
Host-header fallback were previously rejected with 403 by plugin offer routes while the
Control UI itself worked.

Moves the loopback/forwarded-header helpers to net.ts (re-exported from auth.ts) so the
guard can delegate without importing gateway auth, and migrates the Codex realtime
broker onto the shared seam.

* feat(talk): let providers own agent delegation for realtime voice

Adds an optional runAgentConsult callback to the browser-session create request and
injects the existing embedded consult runtime from talk-client, bound to the same agent
and session key the GA tool path uses. Providers whose realtime protocol delegates work
through their own control channel (rather than GA function calls) can now reach the
OpenClaw agent without a client round-trip.

Threads the effective per-session model into browser-session capability resolution so a
request-level model override selects the right capability set, and propagates a caller
abort signal into consultRealtimeVoiceAgent so a superseded delegation stops its run.

* feat(openai): support GPT-Live realtime voice over ChatGPT OAuth

Implements OpenAI's quicksilver/frameless session natively for Talk browser sessions.
The Gateway creates the WebRTC call (multipart sdp+session to https://api.openai.com/v1/live)
and owns the sideband control socket, so the browser never holds upstream credentials;
delegation.created events run through the OpenClaw agent and stream back as speakable
context appends.

Verified end-to-end on 2026-07-28 against a ChatGPT Pro OAuth profile: call create 201
with an rtc_* id and answer SDP, sideband session.started, session.close teardown.
ChatGPT OAuth is preferred over a Platform API key because /v1/live access for platform
keys is waitlist-gated; the legacy chatgpt.com backend route returns 403 for every model
and protocol version and is not used.

Accepted models are gpt-live-1-codex and gpt-live-1-boulder-alpha; the voice allowlist is
the ten values the route actually accepts, since an invalid voice is rejected at call
creation and cannot be repaired afterwards (session.update reports immutable_field_update).

* docs: document GPT-Live Talk support and its route gotchas

Records the working route and auth, the accepted models and voices, the browser-only
scope, and the two traps that cost the most time: the chatgpt.com backend route returns
403 Voice session access denied for every model, and that same 403 is also what an
invalid voice returns, so it must not be read as an account entitlement block.

* fix(openai): resolve GPT-Live CI failures

* refactor(openai): own zod runtime dependency

* fix(openai): satisfy lint and live-shard gates for GPT-Live

Types the retry-delay finish callback as Error so the rejection reason is provably an
Error at the call site; the abort path already normalized a non-Error AbortSignal reason,
but the unknown parameter type hid that from static analysis.

Registers the new GPT-Live live test in the native-live-extensions-openai shard
expectation. The shard selector already picked the file up from the real tree; only the
hardcoded list in the tooling test lagged.

* fix(openai): clean up post-rebase capability resolver

* fix(openai): preserve GPT-Live delegation fragments

* fix(openai): close GPT-Live sideband handoff race

* fix(openai): accept UUID GPT-Live call ids

* style: apply oxfmt to GPT-Live sources and Talk docs

* style: format Talk docs after rebase

* fix(openai): keep GPT-Live transcript context across ignored delegations
2026-07-28 15:05:55 -04:00
..
reference docs: add Buzz to generated plugin inventory (#114875) 2026-07-27 22:59:32 -04:00
adding-capabilities.md refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527) 2026-07-21 20:28:43 -07:00
admin-http-rpc.md fix(gateway): admit plugin HTTP work during suspension (#104112) 2026-07-10 22:34:57 -07:00
agent-tools.md
architecture-internals.md refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527) 2026-07-21 20:28:43 -07:00
architecture.md feat: auto-collect durable meeting transcripts (#113122) 2026-07-23 11:41:08 -07:00
beam.md feat(plugins): mirror local coding sessions to a remote Beam receiver (#114735) 2026-07-27 17:31:34 -04:00
building-extensions.md docs: rewrite published docs grounded in current source (#100142) 2026-07-05 00:32:47 -04:00
building-plugins.md docs(plugins): link setup-input deprecations to the migration pattern and document reader sweeps (#112692) 2026-07-22 11:04:19 -07:00
bundles.md docs: rewrite published docs grounded in current source (#100142) 2026-07-05 00:32:47 -04:00
cli-backend-plugins.md fix(cron): preserve scheduled caps across CLI and worker upgrades 2026-07-24 01:24:03 -05:00
codex-computer-use.md fix(codex): stop Computer Use readiness stalls across fallbacks (#113393) 2026-07-24 14:27:53 -06:00
codex-harness-reference.md refactor(agents): retire TOOLS.md into an AGENTS.md section with a doctor migration (#113966) 2026-07-27 03:56:30 -04:00
codex-harness-runtime.md fix(gateway): stop showing retired HEARTBEAT.md in the agent files editor (#113621) 2026-07-25 04:58:31 -07:00
codex-harness.md fix(codex): refuse Platform routes on a subscription-signed native Codex home (#114719) 2026-07-27 22:42:18 -04:00
codex-native-plugins.md feat(codex): support workspace-directory plugins (#104188) 2026-07-11 09:04:55 -07:00
codex-supervision.md improve: stream native sessions as hosts finish (#110211) 2026-07-17 23:59:57 +01:00
community.md docs: rewrite published docs grounded in current source (#100142) 2026-07-05 00:32:47 -04:00
compatibility.md refactor(prompt): use plain inbound context labels and drop system-tag sanitizer (#112000) 2026-07-27 11:27:53 +10:00
copilot.md refactor: remove expired plugin compatibility surfaces (#111451) 2026-07-19 11:04:48 -07:00
dependency-resolution.md build(deps): remove npm shrinkwrap; mirror pnpm lock into transient package locks (#114006) 2026-07-26 01:29:55 -04:00
google-meet.md refactor(meetings): close manual action state (#114247) 2026-07-26 23:25:19 -04:00
hooks.md feat(sdk): always persist media facts and ship facts-first replacements for legacy Media* surfaces (#113355) 2026-07-24 10:42:17 -07:00
install-overrides.md docs: rewrite published docs grounded in current source (#100142) 2026-07-05 00:32:47 -04:00
llama-cpp.md refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527) 2026-07-21 20:28:43 -07:00
logbook.md refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527) 2026-07-21 20:28:43 -07:00
manage-plugins.md fix(plugins): serialize lifecycle mutations and preserve setup-required installs (#112763) 2026-07-22 23:25:48 -04:00
manifest.md feat(plugins): match shared models across vendor-namespaced catalog ids (#115210) 2026-07-28 10:22:23 -04:00
meeting-plugins.md refactor(meetings): close manual action state (#114247) 2026-07-26 23:25:19 -04:00
memory-lancedb.md refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527) 2026-07-21 20:28:43 -07:00
memory-wiki.md fix(cli): correct optional plugin guidance (#112019) 2026-07-20 21:31:30 -07:00
message-presentation.md refactor: remove dead tooling and deprecated exports (#112915) 2026-07-23 01:28:11 -04:00
oc-path.md docs: rewrite published docs grounded in current source (#100142) 2026-07-05 00:32:47 -04:00
onepassword.md fix(onepassword): make SecretRef setup production-safe 2026-07-25 06:03:30 -05:00
plugin-inventory.md docs: add Buzz to generated plugin inventory (#114875) 2026-07-27 22:59:32 -04:00
plugin-permission-requests.md feat(gateway): land remaining durable-approvals stack and repair main gates (#104837) 2026-07-12 10:51:38 +01:00
reference.md docs: add Buzz to generated plugin inventory (#114875) 2026-07-27 22:59:32 -04:00
sdk-agent-harness.md refactor(agents): resolve code-mode vs tool-search once for every harness (#115189) 2026-07-28 09:16:35 -04:00
sdk-channel-inbound.md fix(channels): harden record session override 2026-07-26 23:15:06 -05:00
sdk-channel-ingress.md feat(channels): add unified implicit mention policy (#108829) 2026-07-16 17:16:01 -07:00
sdk-channel-message.md refactor: remove expired plugin compatibility surfaces (#111451) 2026-07-19 11:04:48 -07:00
sdk-channel-outbound.md fix(nostr): report unavailable ingress when queue open fails (#115313) 2026-07-28 13:34:23 -04:00
sdk-channel-plugins.md refactor(model-picker): centralize session apply contracts (#114739) 2026-07-27 16:58:23 -04:00
sdk-channel-turn.md
sdk-entrypoints.md refactor(fs): adopt fs-safe 0.5 core primitives (#113705) 2026-07-28 03:41:47 -04:00
sdk-migration.md refactor(fs): adopt fs-safe 0.5 core primitives (#113705) 2026-07-28 03:41:47 -04:00
sdk-overview.md fix(cli): preserve machine-readable stdout (#113654) 2026-07-27 05:44:16 -04:00
sdk-provider-plugins.md refactor(providers): collapse live model discovery onto shared projection hook (#113903) 2026-07-25 16:17:27 -07:00
sdk-runtime.md fix(channels): fail channel start when the durable ingress queue is denied (#114998) 2026-07-28 03:17:09 -04:00
sdk-setup.md feat(channels): add channel-owned setup contracts (#112176) 2026-07-22 19:57:42 -04:00
sdk-subpaths.md feat(talk): support GPT-Live realtime voice over ChatGPT OAuth (#115226) 2026-07-28 15:05:55 -04:00
sdk-testing.md test(telegram): consolidate bot test harness (#113571) 2026-07-25 05:28:26 -07:00
teams-meetings.md feat: auto-collect durable meeting transcripts (#113122) 2026-07-23 11:41:08 -07:00
tool-plugins.md docs(plugins): link setup-input deprecations to the migration pattern and document reader sweeps (#112692) 2026-07-22 11:04:19 -07:00
vault.md Add Vault SecretRef plugin (#89255) 2026-07-09 05:30:12 -05:00
voice-call.md refactor(config): config-surface reduction tranche 3 — product consolidations (review request) (#111527) 2026-07-21 20:28:43 -07:00
webhooks.md fix(webhooks): keep routes cold when SecretRefs are unavailable (#109715) 2026-07-16 23:56:59 -07:00
workboard.md feat(dashboard): workboard:mini aggregates all boards when boardId is unset (#112484) 2026-07-21 20:32:33 -07:00
zalouser.md docs: rewrite published docs grounded in current source (#100142) 2026-07-05 00:32:47 -04:00
zoom-meetings.md feat: auto-collect durable meeting transcripts (#113122) 2026-07-23 11:41:08 -07:00