vrr/agent-zero
2
0
Fork 0
mirror of https://github.com/agent0ai/agent-zero.git synced 2026-07-10 01:18:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
agent-zero/plugins/_browser/AGENTS.md
Alessandro afa5231a5a Refine BYOB host browser endpoint selection 
Reduce Browser Settings host-browser choices to automatic and advertised debug endpoints, with a validated Custom endpoint input and faster inventory refresh.

Update Browser docs and regressions to document the custom endpoint workflow and CLI websocket selection.
2026-07-05 19:59:30 +02:00

4.3 KiB
Raw Permalink Blame History

Browser Plugin DOX

Purpose

  • Own the built-in Playwright browser tool and WebUI browser viewer.
  • Bridge browser automation, page inspection helpers, and browser panel UI.

Ownership

  • plugin.yaml and default_config.yaml own metadata and browser settings defaults.
  • tools/browser.py owns the agent-facing browser tool.
  • helpers/ owns Playwright runtime, selectors, URL helpers, extension management, and connector runtime logic.
  • api/ owns status, extension, and browser WebSocket handlers.
  • assets/, prompts/, skills/, extensions/, and webui/ own browser scripts, prompts, skill guidance, hook contributions, and UI.

Local Contracts

  • Keep browser actions safe around external pages, credentials, and user data.
  • Preserve Playwright lifecycle cleanup and WebSocket viewer compatibility across regular host browsers and Electron WebContentsView embedding.
  • Keep the WebUI Browser inside its own modal/canvas affordance; do not replace it with page-level navigation.
  • Default the visible WebUI Browser to live CDP screencast for responsiveness. Keep lightweight CDP/DOM state snapshots as the fallback transport.
  • Paint live screencast frames through the Browser panel canvas/ImageBitmap path when available; keep the <img>/data URL path for snapshots and fallback rendering.
  • Push internal screencast frames from the runtime to the WebSocket consumer after subscription; keep read/pop_screencast_frame as fallback/tooling APIs, not the WebUI hot path.
  • Keep Browser viewer frame transport capability-negotiated: updated clients may request binary/slim screencast frames, while older clients must keep the base64/full-metadata fallback. Do not let the WebUI advertise binary frames unless its Socket.IO client reconstructs attachments as real Blob, ArrayBuffer, or typed-array values.
  • Keep WebUI Browser tabs scoped to the active chat context by default; aggregate tabs from other AgentContext runtimes only when the Browser settings tab scope is shared.
  • Keep narrow WebUI Browser controls usable by grouping navigation with Annotate/settings above a full-width address bar.
  • For Bring Your Own Browser with an existing host profile, host_browser_selection may target automatic CLI selection, a browser family/id, or an explicit CDP endpoint and must be forwarded to the connector runtime as browser_selection.
  • Browser Settings must refresh connected A0 CLI host-browser inventory while the settings view is open so newly authorized endpoints appear without saving or reopening.
  • Browser Settings keeps the Host browser dropdown focused on automatic selection, advertised debug endpoints, and a validated Custom endpoint field instead of listing every installed local profile.
  • Browser URL-intent handling must only claim web URL schemes and leave custom Agent Zero schemes to their owning surfaces.
  • Prefer DOM/CDP browser actions with refs, selectors, frame-chain refs, and screenshots over viewport coordinate input. Coordinates remain a visual fallback.
  • Do not hardcode user-specific browser paths or secrets.

Work Guidance

  • Coordinate tool, helper, and panel changes so browser state shown in the UI matches tool behavior.
  • Do not depend on nested Electron <webview> support or launcher-specific preload bridges unless the launcher exposes that bridge as an explicit contract.
  • Keep prompts/agent.system.tool.browser.md as a compact callable contract; move detailed browser workflows into skills/browser-automation/SKILL.md.
  • Keep skills/browser-automation/SKILL.md frontmatter triggers current with rendered browsing, host-browser, screenshot, and web-interaction user phrasing so relevant-skill recall can surface the skill before the full browser workflow is needed.
  • Keep fragile form guidance progressively disclosed through skills/browser-form-workflows/SKILL.md, linked from the browser prompt through browser-automation.

Verification

  • Smoke-test browser launch, navigation, DOM capture, and WebUI viewer after runtime changes.
  • For viewer render-path changes, verify the live Browser panel paints a screencast frame on canvas with frameSrc empty and snapshots still falling back to the image path.
  • Run browser prompt/skill regression tests after changing browser prompt or Browser plugin skills.

Child DOX Index

No child DOX files.