openclaw/docs/plugins/google-meet.md
Peter Steinberger e7b868bbab
feat(voice): share GPT Live across meetings and calls (#146546)
* feat(voice): unify Live sessions across calls and meetings

Resolve provider capabilities and interruption policy through the shared realtime voice owner. Keep meeting input isolated from virtual-microphone output, reuse native delegation for meetings and Voice Call, and preserve explicit Stop across browser and Apple relay clients.\n\nValidated with real Live API and synthetic Chromium/WebRTC proof, focused regressions, changed-file checks, and independent review. Related to #146289.

* test(voice): align capture fixtures and validation gates

Model browser audio capture in the shared meeting RPC fixtures so startup
failure tests reach the provider and verify capture cleanup. Preserve the
same relay startup behavior while simplifying duplicate lifecycle branches.

Rebalance the pre-existing 702-root platform test graph by moving security
tests beside sandbox/tool tests; keep coverage, graph counts, and limits.

* fix(meetings): preserve configured input commands

Keep explicitly configured capture/filter/mixer output as provider input on
local Chrome and paired nodes, preserving the v2026.9.4 contract. Generated
input and output-only overrides continue to use managed browser capture.

Retain Live's isolation guard and explain how to remove an input override
when selecting Live. Prove the actual PCM paths through both meeting engines
and transports, and document the preserved configuration behavior.
2026-09-12 17:06:51 -07:00

18 KiB

summary read_when title
Google Meet plugin: join explicit Meet URLs through Chrome or Twilio with agent talk-back defaults
You want an OpenClaw agent to join a Google Meet call
You want an OpenClaw agent to create a new Google Meet call
You are configuring Chrome, Chrome node, or Twilio as a Google Meet transport
Google Meet plugin

The google-meet plugin joins explicit Meet URLs on behalf of an OpenClaw agent. It is deliberately narrow:

  • It only joins https://meet.google.com/... URLs; it never dials into a meeting from a phone number it discovers itself.
  • googlemeet create can mint a new Meet URL through the Google Meet API (or a browser fallback) and join it by default.
  • Chrome participation uses a signed-in Chrome profile, optionally on a paired node. Twilio participation dials a phone number plus PIN/DTMF through the Voice call plugin; it cannot dial a Meet URL directly.
  • mode: "agent" (default) transcribes participant speech with a realtime provider, routes it to the configured OpenClaw agent, and speaks the answer with regular OpenClaw TTS. mode: "bidi" uses realtime voice, including GPT-Live with native OpenClaw agent delegation. mode: "transcribe" joins observe-only with no talk-back.
  • There is no automatic consent announcement when the plugin joins a call.
  • The CLI command is googlemeet; meet is reserved for broader agent teleconference workflows.

Quick start

Install the plugin and the native audio dependencies for the Chrome host, then configure provider authentication. OpenAI is the default transcription provider for agent mode; bidi mode supports Google Gemini Live and OpenAI realtime voice, including GPT-Live. For GPT-Live with Cove, use the explicit Live configuration. The default agent path on macOS:

openclaw plugins install @openclaw/google-meet
brew install blackhole-2ch sox
export OPENAI_API_KEY=sk-...
# only needed when realtime.voiceProvider is "google" for bidi mode
export GEMINI_API_KEY=...

blackhole-2ch installs the BlackHole 2ch virtual audio device Chrome routes through. Homebrew's installer requires a reboot before macOS exposes the device:

sudo reboot

After reboot, verify both pieces:

system_profiler SPAudioDataType | grep -i BlackHole
command -v sox

On a Linux desktop with PipeWire-Pulse:

sudo apt install pipewire-audio pulseaudio-utils # Debian/Ubuntu
systemctl --user --now enable pipewire pipewire-pulse wireplumber
pactl info
command -v pactl pacat parec

OpenClaw provisions an OpenClaw Meeting Audio null sink and matching source in that desktop user's audio session. Run the Gateway or paired node as the same user that runs Chrome.

The plugin is enabled by default after installation. Add an entry only to customize it:

{
  plugins: {
    entries: {
      "google-meet": {
        config: {},
      },
    },
  },
}

Run openclaw plugins disable google-meet if you do not want the plugin active.

Check setup, then join:

openclaw googlemeet setup
openclaw googlemeet join https://meet.google.com/abc-defg-hij

setup output is agent-readable and mode/transport-aware: it reports Chrome profile, node pinning, and, for realtime Chrome joins, the native virtual-audio backend and delayed-intro check. Observe-only joins skip realtime prerequisites:

openclaw googlemeet setup --transport chrome-node --mode transcribe

When Twilio delegation is configured, setup also reports whether voice-call, Twilio credentials, and public webhook exposure are ready. Treat any ok: false check as a blocker for that transport/mode before an agent joins. Use --json for machine-readable output, and --transport chrome|chrome-node|twilio to preflight a specific transport ahead of time:

openclaw googlemeet setup --transport twilio

Or let an agent join through the google_meet tool:

{
  "action": "join",
  "url": "https://meet.google.com/abc-defg-hij",
  "transport": "chrome-node",
  "mode": "agent"
}

Local Chrome talk-back supports macOS with BlackHole 2ch and SoX, or Linux with PipeWire-Pulse and pactl/pacat/parec. On other operating systems, use mode: "transcribe", Twilio dial-in, or a supported macOS/Linux chrome-node host.

Create a meeting

openclaw googlemeet create --transport chrome-node --mode agent
openclaw googlemeet create --no-join

create has two paths, reported in the result's source field:

  • api: used when Google Meet OAuth credentials are configured. Deterministic; does not depend on browser UI state.
  • browser: used without OAuth credentials. OpenClaw opens https://meet.google.com/new on the pinned Chrome node and waits for Google to redirect to a real meeting-code URL; the OpenClaw Chrome profile on that node must already be signed in to Google. Join and create both reuse an existing Meet tab (or an in-progress .../new / Google account prompt tab) before opening a new one; tab matching ignores harmless query strings like authuser.

create joins by default and returns joined: true plus the join session. Pass --no-join (CLI) or "join": false (tool) to mint the URL only.

For API-created rooms, set an explicit access policy instead of inheriting the Google account default:

openclaw googlemeet create --access-type OPEN --transport chrome-node --mode agent
--access-type Who can join without knocking
OPEN Anyone with the Meet URL
TRUSTED Host org's trusted users, invited external users, and dial-in users
RESTRICTED Invitees only

This only applies to API-created rooms, so OAuth must be configured. If you authorized OpenClaw before access-type control shipped in 2026.5.2, rerun openclaw googlemeet auth login --json after adding the meetings.space.settings scope to your OAuth consent screen.

If the browser fallback hits a Google login or Meet permission blocker, the tool returns manualAction: { reason, message } with the browser.nodeId/browser.targetId/browserUrl. Report that message and stop opening new Meet tabs until the operator finishes the browser step.

Observe-only join

Set "mode": "transcribe" to skip the duplex realtime bridge (no virtual-audio requirement, no talk-back). Transcribe-mode Chrome joins also skip OpenClaw's microphone/camera permission grant and the Meet Use microphone path; if Meet shows the audio-choice interstitial, automation tries Continue without microphone first. Managed Chrome transports install a best-effort Meet caption observer in every mode so durable notes are available without changing the live agent-consult path. googlemeet status --json and googlemeet doctor report captioning, captionsEnabledAttempted, transcriptLines, lastCaptionAt, lastCaptionSpeaker, lastCaptionText, and a recentTranscript tail.

For the bounded session transcript, read the exact tracked Meet tab:

openclaw googlemeet transcript <session-id>
openclaw googlemeet transcript <session-id> --since <next-index> --json

The observer keeps at most 2,000 completed caption lines in the Meet page. Visible progressive text stays in the status health tail until the caption row completes, so saving nextIndex cannot skip a later text expansion; leaving finalizes visible rows before the snapshot. droppedLines reports lines lost from the head when the cap is exceeded. The bounded googlemeet transcript tail still keeps only the four most recently ended sessions and resets with the Gateway. Separately, OpenClaw appends completed caption rows to the shared state database throughout the meeting and writes a derived summary on leave. Use openclaw transcripts to inspect or export those durable notes.

Automatic notes are enabled by default. Set transcripts.enabled: false to disable durable notes globally; explicit transcribe mode still exposes only its bounded live tail. Twilio joins do not have the browser caption stream and are not captured by this path.

For a yes/no listen probe:

openclaw googlemeet test-listen <meet-url> --transport chrome-node

It joins in transcribe mode, waits for fresh caption/transcript movement, and returns listenVerified, listenTimedOut, manual-action fields, and current caption health.

Audio bridge architecture

Google Meet's official media API is receive-oriented, so speaking into a call still needs a participant path. This plugin keeps that boundary visible: Chrome handles browser participation and local audio routing; Twilio handles phone dial-in participation.

Chrome talk-back modes need a supported native virtual-audio backend plus either:

  • chrome.audioInputCommand plus chrome.audioOutputCommand: with generated native commands, OpenClaw captures participant audio from browser playback, injects assistant audio into the virtual microphone, and uses native input to verify that injection. An explicitly configured input command keeps supplying participant audio to the provider, preserving custom capture/filter/mixer setups. agent mode uses realtime transcription plus regular TTS; bidi mode uses the realtime voice provider. The default path is 24 kHz PCM16 with chrome.audioBufferBytes: 4096; 8 kHz G.711 mu-law remains available for legacy command pairs.
  • chrome.audioBridgeCommand: an external bridge command owns the whole local audio path and must exit after starting or validating its daemon. Valid only for bidi, because agent mode needs direct command-pair access for TTS.

The default browser bridge keeps participant playback off the virtual microphone, so received meeting audio is not sent back into the call. Isolated participant input remains available during assistant speech. Google Meet, Microsoft Teams, and Zoom use this same meeting engine; provider selection, delegation, and interruption policy also share the realtime voice runtime used by Discord and Talk.

GPT-Live owns interruptions and plays continuous audio without waiting for response-completion events. It requires the managed isolated browser input: remove an explicit chrome.audioInputCommand when selecting Live. Custom input commands cannot establish that isolation and are rejected for Live instead of silently replaced. Live does not use chrome.bargeInInputCommand. That optional separate local-microphone command remains available for providers that support host-driven interruption on the Gateway-hosted command-pair bridge. Like the other audio commands, it is an operator-configured local command: use an explicit trusted command path or argument list.

googlemeet speak triggers the active talk-back audio bridge for a Chrome session; googlemeet leave stops it (and, for Twilio sessions delegated through Voice Call, hangs up the underlying call). Use googlemeet end-active-conference to also close the active Google Meet conference for an API-managed space.

Realtime session health

During talk-back sessions, google_meet status reports Chrome/audio bridge health: inCall, manualAction, providerConnected, realtimeReady, audioInputActive, audioOutputActive, last input/output timestamps, byte counters, and bridge-closed state. Managed Chrome sessions only speak the intro/test phrase after health reports inCall: true; otherwise speechReady: false and the speech attempt is blocked rather than silently no-opping.

Local Chrome and paired Chrome nodes use the same input-source selection. Generated commands use isolated browser capture and native injection; explicit input commands keep their configured capture path and existing echo protection. If managed browser playback cannot be captured, the bridge fails clearly instead of sending mixed loopback audio to Live. A room with no participant audio yet is idle, not a capture failure.

Where each section moved

Every section of the single-page version now lives on this page or on one of the five child pages below. The anchors from the single-page version still resolve here.

Google Meet transports and hosts

Google Meet transports and hosts — Chrome, Chrome node, and Twilio transports, the Parallels macOS VM topology, and the host audio tools they need.

Google Meet OAuth and artifacts

Google Meet OAuth and artifacts — Google Cloud credentials, the refresh token, OAuth verification, and reading Meet artifacts, attendance, and exports.

Google Meet configuration

Google Meet configuration — Plugin config defaults, optional overrides, and the ElevenLabs and Twilio config examples.

Google Meet tool and modes

Google Meet tool and modes — Tool actions for agents, session status fields, and the agent and bidi talk-back modes.

Google Meet troubleshooting

Google Meet troubleshooting — The pre-flight live test checklist and fixes for join, speech, creation, and Twilio failures.