Compare commits

..

60 commits
v2.2 ... main

Author SHA1 Message Date
Alessandro
fddcc3deea Update default utility model
Some checks are pending
Build And Publish Docker Images / plan (push) Waiting to run
Build And Publish Docker Images / build (push) Blocked by required conditions
Use OpenRouter's google/gemini-3.1-flash-lite model without the preview suffix for the global default, Balance preset, and onboarding defaults.
2026-07-10 19:41:40 +02:00
Alessandro
3ac63c6166 Add responsive interface control visibility
Add an Interface settings section with independent mobile and desktop toggles for the project selector, clock, connection status, and right canvas rail.

Persist the visibility map in instance settings, apply it at the shared WebUI surfaces, preserve existing defaults, and cover the contract with focused regression tests and DOX updates.
2026-07-10 19:38:19 +02:00
Alessandro
948409c244 Clear provider-specific kwargs on provider change
Reset additional parameters together with the custom API base when a model provider changes, preventing stale provider-specific options from leaking into the new provider.

Update the model-config contract and README, and extend the focused frontend regression check.
2026-07-10 19:12:02 +02:00
Alessandro
4e8301899e Extend self-update health timeout
Raise the default update and rollback health-check window from two to three minutes so slower Docker and virtiofs installations can finish booting. Preserve the existing environment override and document the runtime default.
2026-07-10 19:07:14 +02:00
Alessandro
705a400fe7 Add NVIDIA NIM provider
Register NVIDIA NIM for chat and embedding models through LiteLLM, including hosted model discovery.\n\nExpose NIM in onboarding with API key and documentation links, and cover the provider contract with a focused regression test.
2026-07-10 19:04:27 +02:00
Alessandro
25d906edb7 Harden Gemini CLI container setup
Separate Gemini CLI installation from probing so long installs remain observable. Use Agent Zero secret aliases for GEMINI_API_KEY instead of assuming saved secrets are exported from /a0/usr/.env, with contract coverage.
2026-07-10 18:53:28 +02:00
Alessandro
9837ff3432 Group multiline terminal commands
Execute multiline terminal input as one current-shell compound so intermediate prompts cannot complete queued work early. Preserve shell state and cover Bash and PowerShell grouping with regression tests.
2026-07-10 18:53:19 +02:00
Alessandro
8aad52784b Add Gemini CLI support to Orchestrator
Register Gemini CLI with auth detection and a verified headless workflow. Expose its settings and status metadata, update the plugin contracts and tests, bump the plugin version, and replace the thumbnail.
2026-07-10 18:41:02 +02:00
Alessandro
82ae8d50e0 Subdivide root DOX guidance
Trim the root AGENTS.md to project-wide contracts and the top-level child index. Move Docker runtime and GitHub release details into their owning child DOX files.
2026-07-10 18:38:07 +02:00
Alessandro
5660848940 Improve built-in command management
List bundled slash commands in their own section and create same-name project or global overrides when users edit them.

Remove the Commands sidebar shortcut and redundant scope banner, then simplify command cards by dropping badges and argument hints while tightening description typography. Add regression coverage for built-in listing and override precedence.
2026-07-10 17:50:33 +02:00
Alessandro
877d063d3c Polish OAuth model selection
Simplify provider cards and model-slot copy, align model pickers with the main model configuration UI, and infer a sole connected OAuth provider only when it does not silently replace a different saved provider.\n\nAdd focused static coverage for the updated picker and provider-default behavior.
2026-07-10 17:29:10 +02:00
Alessandro
1410bdcf6c Refresh Codex CLI during self-update
Refresh an existing Codex CLI after successful or already-current self-updates without making npm failures fatal. Bootstrap the new behavior when startup synchronizes a stale runtime updater, and cover both paths with focused regressions.
2026-07-10 16:53:04 +02:00
Alessandro
d33cac3bf3 Stop runaway unusable response loops
Add a configurable circuit breaker for consecutive malformed or repeated model outputs. Expose the limit in Agent Settings and render the stop notice from a framework prompt.
2026-07-10 16:36:30 +02:00
Alessandro
db7b90823e Move preset editor behavior into Alpine store
Create local preset drafts and handle add, remove, and save actions through modelConfig.createPresetEditor so the HTML remains declarative.

Document the project-wide Alpine expression boundary and keep the plugin contract specific to preset editor ownership.
2026-07-10 14:19:23 +02:00
Alessandro
529dc8c465 Keep model presets bound after deletion
Give each preset editor row a stable UI-only key so Alpine does not reuse nested model state after a middle row is removed.

Add a regression guard and document the row identity invariant in the model-config plugin contract.
2026-07-10 14:18:49 +02:00
Alessandro
811556d24e Make chat persistence crash-safe
Write serialized chats to a same-directory temporary file, fsync it, and atomically replace chat.json so interrupted saves preserve the previous chat data.

Add regression coverage for interrupted replacement and document the persistence guarantee.
2026-07-10 13:08:09 +02:00
Alessandro
3bb40576af Improve README conversion flow
Some checks failed
Build And Publish Docker Images / plan (push) Has been cancelled
Build And Publish Docker Images / build (push) Has been cancelled
Lead with a sharper Linux-computer positioning, desktop hero, and scannable feature table.

Put the Launcher first, collapse alternate install paths, add basic troubleshooting, and move starter prompts above the deep dives.

Trim the mid-README Space Agent detour to a footer note while keeping the safety and documentation sections.
2026-07-09 17:59:27 +02:00
Alessandro
c3f44b2c88 Merge branch 'main' into ready 2026-07-09 17:49:56 +02:00
Alessandro
fa05710da7 Reuse native parallel child logs
Pass each parent-visible child log into direct parallel worker execution so tool before_execution reuses it instead of creating a second generic tool log.

This keeps native badge types such as wait/progress (HDL) intact through execution, updates the parallel helper DOX contract, and adds a regression test for log reuse.
2026-07-09 17:42:41 +02:00
Alessandro
8d79f556c6 Fix parallel child log typing
Resolve tool-specific log objects for all parallel child jobs, removing the code_execution_tool-only special case and using native get_log_object() when available, with generic fallback.

Update parallel_tools docs to reflect the shared logging contract and add coverage for wait/ fallback behavior in parallel-tool tests.
2026-07-09 17:34:48 +02:00
Alessandro
1a6d0eb614 Ignore removed Time Travel workspaces
Drop queued debounced snapshots when the workspace directory is gone before the timer flushes.

This keeps transient delete/rename races from surfacing as Time Travel snapshot errors while preserving normal snapshot failures for existing workspaces.
2026-07-09 17:26:39 +02:00
Alessandro
1bd741ff7f Migrate saved Venice model config
Repair saved _model_config user config and presets so Venice model slots use chat completions with the Venice system prompt disabled. Leave a0_venice slots untouched and add regression coverage for parsing and slot preservation.
2026-07-09 17:19:33 +02:00
Alessandro
84c13dab01 Add built-in goal management plugin
Adds the always-enabled _goal plugin with per-chat goal storage, WebUI goal strip, /goal slash command, and agent-facing goal tools.

Includes command-picker integration, focused plugin tests, and the generated 256x256 thumbnail asset.
2026-07-09 17:09:08 +02:00
Alessandro
9c9a4e00ca Add built-in slash commands plugin
Introduce the _commands plugin with command storage, picker UI, bundled canonical command pack, and legacy migration into the built-in namespace.

Polish the Web UI picker behavior, hide WebUI-only-inappropriate commands from the popover, make /models always open model configuration, and add regression coverage for command CRUD, plugin discovery, migration, canonical names, hidden commands, and picker flows.
2026-07-09 16:00:30 +02:00
Alessandro
3d87998821 Exclude Time Travel history from backups
Skip usr/.time_travel during self-update user-data backups and add the same Time Travel shadow-history exclusion to default BackupService metadata.\n\nAdd focused regressions proving regular user files stay in backup coverage while Time Travel shadow repositories are omitted.
2026-07-09 13:27:03 +02:00
Alessandro
bc96386260 Add External Services plugin settings section
Expose plugins that declare the external settings section from the External Services settings page.

Wire the sidebar item, subsection mount, and settings DOX contract so plugin sections use matching tab IDs.
2026-07-09 13:20:48 +02:00
Alessandro
f64b6490c5 Add bundled Orchestrator plugin
Promote the terminal agent work into the built-in plugins tree as _orchestrator with Orchestrator branding and the orchestrator skill.

Include adapter status APIs, settings UI, per-agent skill references, thumbnail asset, tests, and DOX coverage for the bundled plugin.
2026-07-09 13:18:44 +02:00
Alessandro
f1787b65f5 Fix code execution PTY reset hangs
Start local PTY shells in a new process session and make close() escalate from SIGTERM to SIGKILL when a foreground command refuses to exit. This keeps code_execution_tool reset from blocking indefinitely on stuck terminal processes.\n\nAdd regression coverage for closing a TERM-resistant shell and document the reset lifecycle contract.
2026-07-09 03:29:52 +02:00
Alessandro
5f14435093 Normalize Codex Responses input shape
Ensure the Codex OAuth Responses proxy forwards list-shaped input for empty continuation turns while preserving string prompts as user input items. Add regression coverage for the gpt-5.5 continuation case and document the proxy contract.
2026-07-09 02:41:03 +02:00
Alessandro
a7255d9773 Await cached component module imports
Ensure x-component loading awaits cached dynamic import promises before appending deferred markup, matching first-load behavior and preventing Alpine bindings from running before imported stores register.

Document the loader contract in the WebUI JS DOX notes.
2026-07-08 20:53:56 +02:00
Alessandro
4f80ded531 Fix transient UI layering over modals
Raise notification toasts and Bootstrap tooltips above normal and legacy modal overlays while keeping confirmation dialogs on top.

Remove the duplicate tooltip CSS block and add a focused regression for the layer ordering contract.
2026-07-08 15:37:18 +02:00
Alessandro
6dfe33c493 Fix Codex OAuth request metadata
Send current Codex client metadata and compatibility headers on proxied Responses requests so ChatGPT OAuth calls pass upstream call checks.

Return Codex model slugs as the primary list while preserving model metadata for descriptions in the OAuth settings UI.
2026-07-08 15:08:44 +02:00
Alessandro
eb1326ccd3 Document Agent Zero runtime environment
Some checks are pending
Build And Publish Docker Images / plan (push) Waiting to run
Build And Publish Docker Images / build (push) Blocked by required conditions
Teach the agent prompt about the split framework and task Python runtimes so backend checks use /opt/venv-a0 while user-code tasks use /opt/venv.

Document the WebUI JSON API and CSRF token flow so terminal/API work can use the same session cookies and X-CSRF-Token correctly.
2026-07-08 14:38:43 +02:00
Alessandro
bcf2634000 Add AGENTS.md protocol guidance
Load active-project AGENTS.md path-chain guidance into the protocol area without duplicating the project root instructions.

Move the AGENTS.md protocol wording into a prompt template, reuse existing file/path helpers, and cover direct-path discovery plus prompt assembly with focused tests.
2026-07-08 14:29:15 +02:00
Alessandro
f8b06e0b12 Remove legacy skill prompt cleanup
Delete the empty active-skills prompt extension and its unused prompt template now that selected skills are loaded through chat history instead of protocol. Stop clearing legacy loaded_skills keys from protocol/extras in the loaded-skills hook while keeping history reattachment for compacted skill bodies.
2026-07-08 12:42:43 +02:00
Alessandro
0de0fcec0e Simplify relevant skill recall
Search the raw user message when recalling relevant skills instead of the rendered history wrapper.

Replace the stopword catalog with structural matching: names use normal terms, tags/triggers use long terms or phrase matches, and descriptions require phrase matches.
2026-07-08 12:32:19 +02:00
Alessandro
dea64ddad0 Load skills into chat history
Switch the Skills catalog UI/API from scope-wide prompt pins to current-chat history loading, and remove user-facing removal of loaded skills.

Disable legacy active-skill prompt injection so forgotten scope config no longer inflates new-chat prompts.
2026-07-08 01:52:22 +02:00
Alessandro
5ac4f75a47 Stop WebUI startup scripts from blocking render
Load startup vendor scripts with defer and serve Bootstrap locally so the first WebUI mount no longer depends on a parser-blocking CDN request.

Add a focused regression that rejects undeferred classic startup scripts in index.html and documents the vendor/startup contract.
2026-07-08 01:28:36 +02:00
Alessandro
7aba42d46e Default local providers to chat completions
Force chat-completions transport by default for local and broad OpenAI-compatible chat providers whose Responses API implementations are often missing or unstable.\n\nCovers LM Studio, llama.cpp, Ollama, Ollama Cloud, oMLX, vLLM, and Other OpenAI compatible while leaving embedding provider defaults untouched. Updates provider-default regression coverage for the new chat-only defaults.
2026-07-07 17:01:46 +02:00
Alessandro
ad148f24cc Fix Responses state after history compaction
Clear the active Responses provider continuation when automatic history compression or manual chat compaction rewrites local history, while preserving stored response IDs for cleanup.

Add focused regressions for both compression paths so compacted chats do not keep stale provider-side context.
2026-07-07 16:59:49 +02:00
Alessandro
67895f7b44 Default direct Venice to chat completions
Set the shipped direct Venice chat provider default to use Agent Zero's chat-completions transport while preserving the existing Venice API base and disabled Venice system prompt injection.\n\nAdd a focused regression check that direct venice gets the chat default, a0_venice and embeddings stay untouched, and explicit model kwargs can still override the provider default.
2026-07-07 16:52:27 +02:00
Alessandro
96fd5fc2b2 Ignore browser key handling for local editable targets
Prevent Browser keyboard capture from preventing chat composer typing when focus is in editable fields (including contenteditable).
2026-07-07 15:15:36 +02:00
Alessandro
9bb3cb9469 Render parallel code execution as EXE
Use the code_exe log type for code_execution_tool children launched through parallel so the WebUI reuses the existing code execution renderer.

Add a focused regression test and document the parallel child-log contract.
2026-07-07 13:56:38 +02:00
Alessandro
101d84551d Update launcher README links to v1.2
Some checks failed
Build And Publish Docker Images / plan (push) Has been cancelled
Build And Publish Docker Images / build (push) Has been cancelled
Point the root README's A0 Launcher downloads and release note link at the published v1.2 assets without moving the Agent Zero v2.2 tag.
2026-07-06 15:52:19 +02:00
Alessandro
8871bf5001 Update launcher README links to v1.2
Some checks failed
Build And Publish Docker Images / plan (push) Has been cancelled
Build And Publish Docker Images / build (push) Has been cancelled
Point the root README's A0 Launcher downloads and release note link at the published v1.2 assets without moving the Agent Zero v2.2 tag.
2026-07-06 15:28:57 +02:00
Alessandro
6998789ed3 Normalize MCP stdio command lines
Accept shell-style local MCP command values from the manager before spawning stdio clients, so configs like uvx workspace-mcp resolve to the correct executable and args.

Split collapsed option/value argument lines in the MCP manager, document the input contract, and add a regression for the google_workspace MCP shape.
2026-07-05 20:11:19 +02:00
Alessandro
afa5231a5a Refine BYOB host browser endpoint selection
Reduce Browser Settings host-browser choices to automatic and advertised debug endpoints, with a validated Custom endpoint input and faster inventory refresh.

Update Browser docs and regressions to document the custom endpoint workflow and CLI websocket selection.
2026-07-05 19:59:30 +02:00
Alessandro
c719134537 Document host browser debugging setup
Add Browser Settings guidance for supported Chromium-family remote debugging pages, including Opera and explicit remote-debugging endpoint fallback.

Update BYOB docs and troubleshooting to clarify that browser inventory comes from the connected A0 CLI and requires reconnecting the CLI after authorizing a browser.
2026-07-05 18:59:23 +02:00
Alessandro
7168f0984e Refresh BYOB browser inventory
Some checks are pending
Build And Publish Docker Images / plan (push) Waiting to run
Build And Publish Docker Images / build (push) Blocked by required conditions
Poll connected A0 CLI host-browser metadata while Browser Settings is open so newly authorized endpoints appear without saving or reopening.

Extend Browser Settings labels and remote-debugging guidance for Brave, Opera, Vivaldi, and Chromium-family host browsers with regression coverage.
2026-07-04 19:22:57 +02:00
Alessandro
7298a88fda Add BYOB host browser selection
Add host_browser_selection config normalization, Browser Settings UI choices from CLI-advertised inventory, and browser_selection forwarding through connector browser operations.

Expose host browser ids, labels, and available_browsers through A0 connector metadata and the browser_runtime API, with regression coverage for selection normalization.
2026-07-04 18:33:58 +02:00
Alessandro
d018177927 Add configurable Browser tab scope
Expose Browser settings for separate per-chat tabs versus a shared tab strip, and surface the existing maximum-tabs-per-chat cap. Make viewer WebSocket payloads carry the selected tab scope so the Browser panel can render the right tab list without guessing.

Keep the default scoped per chat, preserve the shared mode as an opt-in fallback, and cover both paths with Browser regression tests.
2026-07-04 16:52:39 +02:00
Alessandro
9bfa3b5c81 Fix Browser viewer input after stream restarts
Notify screencast consumers when a stream is stopped so the WebSocket frame supervisor restarts after viewport-driven screencast resets.

Keep the active canvas as the frame/input surface and avoid replacing live screencasts with click snapshots, preserving hover, click, and wheel feedback.

Tested: conda run -n a0 pytest tests/test_browser_agent_regressions.py -q
2026-07-04 16:21:46 +02:00
Alessandro
153ffcc221 Harden browser frame push completion paths
Wake the WebSocket screencast supervisor when the runtime frame task stops before delivering a consumer frame, and keep fatal frame exits eligible for normal CDP teardown.

Avoid resizing the Browser canvas backing store on every frame when bitmap dimensions are unchanged, preventing unnecessary clears and reallocations.
2026-07-03 11:22:23 +02:00
Alessandro
69fb0fca2a Smooth browser viewer rendering and frame push
Render live browser screencast frames through a canvas/ImageBitmap path while keeping the existing image/data-url path for snapshots and fallback rendering.

Attach WebSocket streams as runtime screencast consumers so delivered frames are pushed to the server loop and acknowledged after emit, removing the per-frame read_screencast_frame runtime round trip.
2026-07-03 11:06:14 +02:00
Alessandro
277bcd0783 Prevent browser frame stretch on resize
Keep the live browser frame proportional while the viewer is being resized by using contain sizing instead of stretching the stale frame to the transient surface shape.

Update the regression guard so the viewer contract pins the proportional render path.
2026-07-03 03:16:11 +02:00
Alessandro
727a840e10 Keep WebUI browser frames on base64 fallback
Disable WebUI binary-frame negotiation until its Socket.IO client reconstructs frame attachments as real Blob, ArrayBuffer, or typed-array values.

Reject placeholder binary payloads instead of wrapping them into broken image blobs, preserving the slim-frame path through base64 frames.

Document the transport constraint and pin the fallback behavior in browser regression coverage.
2026-07-03 02:15:54 +02:00
Alessandro
91b3bc9613 Slim browser viewer frame transport
Negotiate binary and slim screencast frames so updated clients avoid base64-in-JSON and repeated viewer metadata on every frame.

Clamp screencast capture to the subscriber viewport and DPR, lower streaming JPEG quality while preserving snapshot quality, and teach the client to manage binary frame URLs safely.

Document the browser viewer performance plan and update regression coverage for the new transport contract.
2026-07-03 01:49:07 +02:00
Alessandro
686e039d1b Wait for Browser screencast frames
Replace the WebSocket viewer frame pump's idle poll with read_screencast_frame so live screencast delivery follows the producer cadence instead of a 50 ms sleep cycle.

Keep idle-page timeouts inside the stream loop to avoid screencast restart flashes, and update the Browser regression test to pin the blocking read path and timeout handling.
2026-07-03 01:25:56 +02:00
Alessandro
306012eb00 Default desktop text files to Writer
Set LibreOffice Writer as the Desktop MIME default for Markdown and plain text while keeping Agent Zero Editor as the secondary Open With association.

Update the Desktop DOX contract and extend the routing regression test to pin the Writer-first association order.
2026-07-03 01:08:02 +02:00
Alessandro
05481c222a Fix browser URL intent routing
Limit the Browser URL-intent handler to web URL schemes so custom Agent Zero schemes fall through to their owning surfaces.

Document the scheme-ownership contract and cover the a0-editor misroute regression in the desktop/editor routing test.
2026-07-03 01:07:49 +02:00
257 changed files with 13800 additions and 1150 deletions

7
.github/AGENTS.md vendored
View file

@ -9,13 +9,16 @@
- `workflows/` contains GitHub Actions workflow definitions.
- `scripts/` contains Python helpers called by workflows.
- Root-level release rules remain in the root `AGENTS.md`; this file owns automation-specific details.
- This file owns release automation rules; user-facing release documentation belongs under `docs/`.
## Local Contracts
- Docker publishing lives in `workflows/docker-publish.yml` and delegates planning to `scripts/docker_release_plan.py`.
- Releasable tags are `vX.Y` tags at or above `v1.0`, matching the workflow environment.
- On `main`, the newest eligible tag publishes both the version tag and `latest`, then creates or updates its GitHub release after the image push succeeds; other allowed branches publish only their branch tag.
- Manual dispatch without a tag backfills missing Docker Hub tags. Manual dispatch with a tag rebuilds that target and refreshes `latest` and the GitHub release only when it remains the newest eligible tag on `main`.
- Release-note generation reads `scripts/openrouter_release_notes_system_prompt.md` from the repository root and requires OpenRouter credentials from workflow environment variables.
- Release notes compare against the previous published GitHub release tag and fall back to `No release notes.` when no meaningful summary is generated.
- Keep workflow secrets in GitHub Actions secrets or environment variables. Do not commit credentials, tokens, or generated release bodies containing private data.
- Workflow scripts must fail loudly with actionable messages when required environment variables or git refs are missing.
@ -23,7 +26,7 @@
- Prefer deterministic, testable Python for workflow planning logic instead of complex inline shell in YAML.
- Preserve manual dispatch behavior when changing Docker publishing.
- Keep branch, tag, and release behavior synchronized between workflow YAML, release scripts, tests, and root documentation.
- Keep branch, tag, and release behavior synchronized between workflow YAML, release scripts, tests, and user-facing release documentation.
## Verification

399
AGENTS.md
View file

@ -1,364 +1,83 @@
# Agent Zero - AGENTS.md
# Agent Zero DOX
[Generated using reconnaissance on 2026-02-22]
## Purpose
## Quick Reference
Tech Stack: Framework Python 3.12+ | Agent execution Python 3.13 in Docker | Flask | Alpine.js | LiteLLM | WebSocket (Socket.io)
Dev Server: python run_ui.py (discover host/port from startup output or runtime configuration; do not assume a default port)
Run Tests: pytest (standard) or pytest tests/test_name.py (file-scoped)
Documentation: README.md | docs/
Frontend & Plugin DOX: [WebUI](webui/AGENTS.md) | [Components](webui/components/AGENTS.md) | [Frontend JS](webui/js/AGENTS.md) | [Plugins](plugins/AGENTS.md)
- Own project-wide engineering rules and the top-level DOX index.
- Keep detailed contracts in the closest applicable child `AGENTS.md`.
---
## Project
## Table of Contents
1. [Project Overview](#project-overview)
2. [Core Commands](#core-commands)
3. [Docker Environment](#docker-environment)
4. [Project Structure](#project-structure)
5. [Development Patterns & Conventions](#development-patterns--conventions)
6. [Safety and Permissions](#safety-and-permissions)
7. [Code Examples](#code-examples)
8. [Git Workflow](#git-workflow)
9. [Release Notes](#release-notes)
10. [Troubleshooting](#troubleshooting)
- Stack: Python 3.12+ framework, Python 3.13 agent execution runtime, Flask, Alpine.js, LiteLLM, and Socket.IO.
- Start the WebUI with `python run_ui.py`; discover its URL from startup output, Docker mappings, or explicit configuration rather than assuming a port.
- Run the full test suite with `pytest` or a focused file with `pytest tests/test_name.py`.
- Human-facing documentation lives in `README.md` and `docs/`.
---
## Root Ownership
## Project Overview
- `agent.py` owns `Agent`, `AgentContext`, and loop data.
- `initialize.py` owns framework initialization.
- `models.py` owns model-provider configuration and LiteLLM integration.
- `run_ui.py` is the WebUI entry point.
- `DockerfileLocal` must remain compatible with the contracts under `docker/`.
- Runtime or user state under `usr/` and `tmp/` is intentionally outside tracked DOX unless the user explicitly asks otherwise.
Agent Zero is a dynamic, organic agentic framework designed to grow and learn. It uses the operating system as a tool, featuring a multi-agent cooperation model where every agent can create subordinates to break down tasks.
## Project-Wide Contracts
Type: Full-Stack Agentic Framework (Python Backend + Alpine.js Frontend)
Status: Active Development
Primary Language(s): Python, JavaScript (ES Modules)
- Import `AgentContext` and `AgentContextType` from `agent`, not `helpers.context`.
- Never commit secrets, `.env` files, API keys, tokens, or private user data.
- Preserve authentication and CSRF protections.
- Use Linux paths and commands in examples.
- Treat the Docker container exposed at `localhost:32080` as the live plugin/backend runtime when that target is named.
- Copy live core-plugin changes back into tracked source under `plugins/`.
- Develop new custom plugins under ignored `usr/plugins/`; tracked bundled plugins live under `plugins/`.
- Use the framework runtime for backend and plugin-hook verification, not the separate agent execution runtime.
---
## Permissions
## Core Commands
Allowed without asking:
### Setup
Do not combine these commands; run them individually:
```bash
pip install -r requirements.txt
```
- Start WebUI: python run_ui.py
- Discover the WebUI URL from startup output, launcher/Docker port mappings, or explicit `--host`/`--port`/`WEB_UI_PORT` configuration; do not hardcode a default port.
- Read repository files.
- Update files under `usr/`.
---
Ask before:
## Docker Environment
- Installing dependencies.
- Deleting core files outside `usr/` or `tmp/`.
- Modifying `agent.py` or `initialize.py`.
- Creating commits or pushing branches.
When running in Docker, Agent Zero uses two distinct Python runtimes to isolate the framework itself from code executed on behalf of the agent:
## DOX Workflow
### 1. Framework Runtime (/opt/venv-a0)
- Version: Python 3.12.4
- Purpose: Runs the Agent Zero framework itself: WebUI backend, API, core loop, scheduler, framework imports, and plugin hooks/tools that execute inside the framework process.
- Packages: Contains framework dependencies from requirements.txt.
- Verification: Use this runtime for framework/backend import checks, WebUI startup checks, and plugin hook behavior unless the code explicitly switches environments.
### 2. Agent Execution Runtime (/opt/venv)
- Version: Python 3.13
- Purpose: Default Python environment for the agent's terminal/code-execution tasks and user code run by the agent.
- Behavior: Packages installed by the agent during a task belong here so task dependencies do not pollute or prove the framework runtime. Do not use this runtime as evidence that framework imports, WebUI startup, or plugin hooks work.
---
## Project Structure
```
/
├── agent.py # Core Agent and AgentContext definitions
├── initialize.py # Framework initialization logic
├── models.py # LLM provider configurations
├── run_ui.py # WebUI server entry point
├── api/ # API Handlers (ApiHandler subclasses) + WsHandler subclasses (ws_*.py)
├── extensions/ # Backend lifecycle extensions
├── helpers/ # Shared Python utilities (plugins, files, etc.)
├── tools/ # Agent tools (Tool subclasses)
├── webui/
│ ├── components/ # Alpine.js components
│ ├── js/ # Core frontend logic (modals, stores, etc.)
│ └── index.html # Main UI shell
├── usr/ # User data directory (isolated from core)
│ ├── plugins/ # Custom user plugins
│ ├── settings.json # User-specific configuration
│ └── workdir/ # Default agent workspace
├── plugins/ # Core system plugins
├── agents/ # Agent profiles (prompts and config)
├── prompts/ # System and message prompt templates
├── knowledge/
│ └── main/about/ # Agent self-knowledge reference material
│ ├── identity.md # Philosophy, principles, project context
│ ├── architecture.md # Agent loop, multi-agent coordination, extensions
│ ├── capabilities.md # Detailed capabilities and limitations
│ ├── configuration.md # LLM roles, providers, profiles, plugins, settings
│ └── setup-and-deployment.md # Docker deployment, updates, troubleshooting
└── tests/ # Pytest suite
```
Key Files:
- agent.py: Defines AgentContext, LoopData virtual prompt areas (Protocol before history and Extras after history), and the main Agent class.
- helpers/plugins.py: Plugin discovery and configuration logic.
- webui/js/AlpineStore.js: Store factory for reactive frontend state.
- helpers/api.py: Base class for all API endpoints.
- models.py: LLM provider configuration and LiteLLM wrappers; framework LiteLLM defaults such as `drop_params=True` are merged with `litellm_global_kwargs`, configured values override framework defaults, documented module-level switches such as `drop_params` are applied to LiteLLM, and merged kwargs are passed per call.
- scripts/openrouter_release_notes_system_prompt.md: Editable system prompt used to generate GitHub release notes during Docker publishing.
- knowledge/main/about/: Agent self-knowledge files. Not user-facing docs - written for the agent's internal reference.
- webui/components/AGENTS.md: DOX contract for Alpine component architecture.
- webui/js/AGENTS.md: DOX contract for frontend infrastructure, modal stack, API helpers, and extension loading.
- plugins/AGENTS.md: DOX contract for bundled and custom plugin architecture; `usr/plugins/` remains ignored user state.
---
## Development Patterns & Conventions
### Backend (Python)
- Context Access: Use from agent import AgentContext, AgentContextType (not helpers.context).
- Communication: Use mq from helpers.messages to log proactive UI messages:
mq.log_user_message(context.id, "Message", source="Plugin")
- API Handlers: Derive from ApiHandler in helpers/api.py.
- Extensions: Use the extension framework in helpers/extension.py for lifecycle hooks.
- Error Handling: Use RepairableException for errors the LLM might be able to fix.
### Frontend (Alpine.js)
- Store Gating: Always wrap store-dependent content in a template:
```html
<div x-data>
<template x-if="$store.myStore">
<div x-init="$store.myStore.onOpen()">...</div>
</template>
</div>
```
- Store Registration: Use createStore from /js/AlpineStore.js.
- Modals: Use openModal(path) and closeModal() from /js/modals.js.
### Plugin Architecture
- Location: Always develop new plugins in usr/plugins/.
- Manifest: Every plugin requires a plugin.yaml with name, description, version, and optionally settings_sections, per_project_config, per_agent_config, and always_enabled.
- Discovery: Conventions based on folder names (api/, tools/, webui/, extensions/).
- Plugin-local Python imports: Prefer `usr.plugins.<plugin_name>...` for code that lives under `usr/plugins/`. Avoid `sys.path` hacks and avoid symlink-dependent `plugins.<plugin_name>...` imports for community plugins.
- Runtime hooks: Plugins may also expose hooks in hooks.py, callable by the framework through helpers.plugins.call_plugin_hook(...).
- Hook runtime: hooks.py executes inside the Agent Zero framework Python environment, so sys.executable -m pip installs dependencies into that same framework runtime.
- Environment targeting: If a plugin needs packages or binaries for the separate agent execution runtime or system environment, it must explicitly switch environments in a subprocess by targeting the correct interpreter, virtualenv, or package manager.
- Settings: Use get_plugin_config(plugin_name, agent=agent) to retrieve settings. Plugins can expose a UI for settings via webui/config.html. Plugin settings modals instantiate a local context from $store.pluginSettingsPrototype; bind plugin fields to config.* and use context.* for modal-level state and actions.
- Activation: Global and scoped activation rules are stored as .toggle-1 (ON) and .toggle-0 (OFF). Scoped rules are handled via the plugin "Switch" modal.
- Cleanup rule: Plugins should not permanently modify the system in ways that outlive the plugin. Deleting a plugin should not leave behind symlinks, unmanaged services, or stray files outside plugin-owned paths unless the user explicitly requested that behavior.
### Releases
- Docker publishing automation lives in `.github/workflows/docker-publish.yml`.
- Releasable tags follow `v{X}.{Y}` and only tags `>= v1.0` are considered by the workflow.
- The latest eligible tag on `main` also creates or updates a GitHub release after the Docker image push succeeds.
- GitHub release notes are generated on the fly in `.github/scripts/docker_release_plan.py` by comparing the new tag against the previous published GitHub release tag, collecting commit subjects and descriptions in that range, and sending them to OpenRouter.
- The OpenRouter call uses `OPENROUTER_API_KEY` and `OPENROUTER_MODEL_NAME` from the workflow environment, with the system prompt stored in `scripts/openrouter_release_notes_system_prompt.md`.
- Prioritize user-visible features, important fixes, infra or packaging changes, and breaking notes. Skip low-signal churn.
- If the generated summary has no meaningful content, the release body falls back to `No release notes.`
### Lifecycle Synchronization
| Action | Backend Extension | Frontend Lifecycle |
|---|---|---|
| Initialization | agent_init | init() in Store |
| Mounting | N/A | x-create directive |
| Processing | monologue_start/end | UI loading state |
| Cleanup | context_deleted | x-destroy directive |
---
## Safety and Permissions
### Allowed Without Asking
- Read any file in the repository.
- Update code files in usr/.
### Ask Before Executing
- pip install (new dependencies).
- Deleting core files outside of usr/ or tmp/.
- Modifying agent.py or initialize.py.
- Making git commits or pushes.
### Never Do
- Commit, hardcode or leak secrets or .env files.
- Bypass CSRF or authentication checks.
- Hardcode API keys.
---
## Code Examples
### API Handler (Good)
```python
from helpers.api import ApiHandler, Request, Response
class MyHandler(ApiHandler):
async def process(self, input: dict, request: Request) -> dict | Response:
# Business logic here
return {"ok": True, "data": "result"}
```
### Alpine Store (Good)
```javascript
import { createStore } from "/js/AlpineStore.js";
export const store = createStore("myStore", {
items: [],
init() { /* global setup */ },
onOpen() { /* mount setup */ },
cleanup() { /* unmount cleanup */ }
});
```
### Tool Definition (Good)
```python
from helpers.tool import Tool, Response
class MyTool(Tool):
async def execute(self, **kwargs):
# Tool logic
return Response(message="Success", break_loop=False)
```
---
## Git Workflow
- Docker publish automation lives in `.github/workflows/docker-publish.yml`.
- Release tags handled by automation must match `vX.Y` and be `>= v1.0`.
- Allowed release branches are configured at the top of the workflow. `main` publishes `<tag>` and `latest`; other allowed branches publish only the branch tag.
- Manual dispatch accepts an optional tag. Without a tag it backfills missing Docker Hub tags. With a tag it rebuilds that exact target and only refreshes `latest` and the GitHub release when that tag is still the newest eligible tag on `main`.
---
## Release Notes
- The latest eligible `main` tag generates its GitHub release notes during Docker publish instead of reading committed Markdown files.
- The release-note prompt is editable in `scripts/openrouter_release_notes_system_prompt.md`.
- The commit range starts at the previous published GitHub release tag, not merely the previous semantic tag in the repository.
## Troubleshooting
### Dependency Conflicts
If pip install fails, try running in a clean virtual environment:
```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
### WebSocket Connection Failures
- Check if X-CSRF-Token is being sent.
- Ensure the runtime ID in the session matches the current server instance.
---
*Last updated: 2026-06-01*
*Maintained by: Agent Zero Core Team*
# DOX framework
- DOX is highly performant AGENTS.md hierarchy installed here
- Agent must follow DOX instructions across any edits
## Core Contract
- AGENTS.md files are binding work contracts for their subtrees
- Work products, source materials, instructions, records, assets, and durable docs must stay understandable from the nearest applicable AGENTS.md plus every parent AGENTS.md above it
## Read Before Editing
1. Read the root AGENTS.md
2. Identify every file or folder you expect to touch
3. Walk from the repository root to each target path
4. Read every AGENTS.md found along each route
5. If a parent AGENTS.md lists a child AGENTS.md whose scope contains the path, read that child and continue from there
6. Use the nearest AGENTS.md as the local contract and parent docs for repo-wide rules
7. If docs conflict, the closer doc controls local work details, but no child doc may weaken DOX
Do not rely on prior context. Re-read the applicable DOX chain in the current session before editing.
## Update After Editing
Every meaningful change requires a DOX pass before the task is done.
Update the closest owning AGENTS.md when a change affects:
- purpose, scope, ownership, or responsibilities
- durable structure, contracts, workflows, or operating rules
- required inputs, outputs, permissions, constraints, side effects, or artifacts
- user preferences about behavior, communication, process, organization, or quality
- AGENTS.md creation, deletion, move, rename, or index contents
Update parent docs when parent-level structure, ownership, workflow, or child index changes. Update child docs when parent changes alter local rules. Remove stale or contradictory text immediately. Small edits that do not change behavior or contracts may leave docs unchanged, but the DOX pass still must happen.
Do not create or update DOX docs for changes confined to ignored runtime or user-state folders under `usr/` or `tmp/` unless the user explicitly asks for those folders to be documented.
## Hierarchy
- Root AGENTS.md is the DOX rail: project-wide instructions, global preferences, durable workflow rules, and the top-level Child DOX Index
- Child AGENTS.md files own domain-specific instructions and their own Child DOX Index
- Each parent explains what its direct children cover and what stays owned by the parent
- The closer a doc is to the work, the more specific and practical it must be
## Child Doc Shape
- Create a child AGENTS.md when a folder becomes a durable boundary with its own purpose, rules, responsibilities, workflow, materials, or quality standards
- Work Guidance must reflect the current standards of the project or user instructions; if there are no specific standards or instructions yet, leave it empty
- Verification must reflect an existing check; if no verification framework exists yet, leave it empty and update it when one exists
Default section order:
- Purpose
- Ownership
- Local Contracts
- Work Guidance
- Verification
- Child DOX Index
## Style
- Keep docs concise, current, and operational
- Document stable contracts, not diary entries
- Put broad rules in parent docs and concrete details in child docs
- Prefer direct bullets with explicit names
- Do not duplicate rules across many files unless each scope needs a local version
- Delete stale notes instead of explaining history
- Trim obvious statements, repeated rules, misplaced detail, and warnings for risks that no longer exist
## Closeout
1. Re-check changed paths against the DOX chain
2. Update nearest owning docs and any affected parents or children
3. Refresh every affected Child DOX Index
4. Remove stale or contradictory text
5. Run existing verification when relevant
6. Report any docs intentionally left unchanged and why
## User Preferences
- Do not document changes in `usr/` or `tmp/`; treat both as ignored runtime/user-state folders unless explicitly requested otherwise.
- `AGENTS.md` files are binding contracts for their subtrees.
- Before editing, read this file and every `AGENTS.md` on the path to each target; the closest contract controls local details without weakening parent rules.
- Keep work understandable from the applicable DOX chain. Put project-wide rules here and concrete ownership, workflows, inputs, outputs, side effects, and verification in child docs.
- Create a child `AGENTS.md` only for a durable boundary with distinct ownership or workflow.
- Child docs should use: Purpose, Ownership, Local Contracts, Work Guidance, Verification, and Child DOX Index.
- After every meaningful change, re-check the affected paths, update the closest owning docs and indexes, remove stale guidance, and run relevant verification.
- Do not document ignored `usr/` or `tmp/` changes unless explicitly requested.
- Keep DOX concise, current, operational, and free of diary entries or duplicated parent guidance.
## Child DOX Index
Direct child DOX files:
| Child | Scope |
| --- | --- |
| [.github/AGENTS.md](.github/AGENTS.md) | GitHub Actions workflows and release automation scripts. |
| [agents/AGENTS.md](agents/AGENTS.md) | Bundled agent profiles, profile-local prompts, and profile-local tools. |
| [api/AGENTS.md](api/AGENTS.md) | HTTP API handlers and WebSocket handler entry points. |
| [agents/AGENTS.md](agents/AGENTS.md) | Bundled agent profiles, profile-local prompts, and tools. |
| [api/AGENTS.md](api/AGENTS.md) | HTTP API and WebSocket handler entry points. |
| [conf/AGENTS.md](conf/AGENTS.md) | Repository-shipped configuration defaults and templates. |
| [docker/AGENTS.md](docker/AGENTS.md) | Docker build contexts, image definitions, and runtime compose files. |
| [docs/AGENTS.md](docs/AGENTS.md) | Human-facing documentation, developer guides, screenshots, and agent deep dives. |
| [extensions/AGENTS.md](extensions/AGENTS.md) | Core lifecycle extension hook implementations for backend and WebUI surfaces. |
| [helpers/AGENTS.md](helpers/AGENTS.md) | Shared backend framework utilities and cross-cutting runtime services. |
| [knowledge/AGENTS.md](knowledge/AGENTS.md) | Built-in agent self-knowledge and indexed reference material. |
| [lib/AGENTS.md](lib/AGENTS.md) | Lightweight browser-side helper scripts outside the main WebUI bundle. |
| [plugins/AGENTS.md](plugins/AGENTS.md) | Bundled system plugins shipped with the framework. |
| [prompts/AGENTS.md](prompts/AGENTS.md) | Core prompt templates loaded by agents and framework workflows. |
| [scripts/AGENTS.md](scripts/AGENTS.md) | Repository maintenance scripts invoked by automation or maintainers. |
| [skills/AGENTS.md](skills/AGENTS.md) | Bundled Agent Zero skills and their agent-facing instructions. |
| [docker/AGENTS.md](docker/AGENTS.md) | Docker build contexts, images, compose files, and runtime layout. |
| [docs/AGENTS.md](docs/AGENTS.md) | Human-facing documentation and screenshots. |
| [extensions/AGENTS.md](extensions/AGENTS.md) | Backend and WebUI lifecycle extensions. |
| [helpers/AGENTS.md](helpers/AGENTS.md) | Shared backend utilities and runtime services. |
| [knowledge/AGENTS.md](knowledge/AGENTS.md) | Built-in agent self-knowledge. |
| [lib/AGENTS.md](lib/AGENTS.md) | Lightweight browser-side helpers outside the WebUI bundle. |
| [plugins/AGENTS.md](plugins/AGENTS.md) | Bundled system plugins and custom-plugin architecture. |
| [prompts/AGENTS.md](prompts/AGENTS.md) | Core prompt templates. |
| [scripts/AGENTS.md](scripts/AGENTS.md) | Repository maintenance scripts and automation inputs. |
| [skills/AGENTS.md](skills/AGENTS.md) | Bundled Agent Zero skills. |
| [tests/AGENTS.md](tests/AGENTS.md) | Pytest regression and contract tests. |
| [tools/AGENTS.md](tools/AGENTS.md) | Core agent tool implementations. |
| [webui/AGENTS.md](webui/AGENTS.md) | Flask-served Alpine.js WebUI shell, frontend modules, components, CSS, assets, and vendor libraries. |
| [webui/AGENTS.md](webui/AGENTS.md) | Alpine.js WebUI shell, components, JavaScript, CSS, and assets. |
Intentionally unindexed local or generated roots:
@ -368,6 +87,6 @@ Intentionally unindexed local or generated roots:
| `.pytest_cache/`, `__pycache__/` | Generated test and bytecode caches. |
| `.vscode/`, `.windsurf/` | Editor-local configuration and assistant metadata. |
| `logs/` | Runtime output. |
| `tmp/` | Ignored runtime caches, uploads, and generated working files; do not document changes here unless explicitly requested. |
| `usr/` | Ignored local user data, settings, plugins, uploads, chats, and workdirs; do not document changes here unless explicitly requested. |
| `python/` | Generated or legacy runtime cache mirror; current source lives in root-level `api/`, `helpers/`, `tools/`, and `extensions/`. |
| `tmp/` | Ignored runtime caches, uploads, and generated work. |
| `usr/` | Ignored local user data, settings, plugins, chats, and workdirs. |
| `python/` | Generated or legacy runtime mirror; current source is in root modules and tracked source directories. |

131
README.md
View file

@ -3,56 +3,58 @@
<img src="docs/res/a0-vector-graphics/horizontal_banner.svg" alt="Agent Zero Banner" width="100%"/>
# Agent Zero
### A full Linux system for your AI agent.
### Give your agent a full Linux computer.
Agent Zero is an open, dynamic, organic agentic framework. One Docker container ships a full Linux system with a desktop and a plugin hub that the agent can extend using Skills.
Agent Zero is an open agent framework for work that needs more than chat: a Dockerized Linux desktop, a browser with DOM annotation, live document cowork, projects, skills, plugins, and a bridge back to your host machine.
[![Website](https://img.shields.io/badge/Website-agent--zero.ai-0A192F?style=for-the-badge&logo=vercel&logoColor=white)](https://agent-zero.ai)
[![Docs](https://img.shields.io/badge/Docs-Read%20the%20guides-1F6FEB?style=for-the-badge&logo=readthedocs&logoColor=white)](./docs/)
[![Discord](https://img.shields.io/badge/Discord-Join%20us-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/B8KZKNsPpj)
[![GitHub Sponsors](https://img.shields.io/badge/Sponsors-Thank%20you-FF69B4?style=for-the-badge&logo=githubsponsors&logoColor=white)](https://github.com/sponsors/agent0ai)
[Install](#how-to-install) |
[Launcher](#a0-launcher) |
[What's Different](#what-makes-agent-zero-different) |
[A0 CLI](#a0-cli-connector-extend-onto-your-host-machine) |
[Docs](#documentation)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/agent0ai/agent-zero)
[Ask ChatGPT](https://chatgpt.com/?q=Analyze%20this%3A%20https%3A%2F%2Fgithub.com%2Fagent0ai%2Fagent-zero) |
[Ask Claude](https://claude.ai/new?q=Analyze%20this%3A%20https%3A%2F%2Fgithub.com%2Fagent0ai%2Fagent-zero)
[Quick Start](#quick-start) |
[Why Agent Zero](#why-agent-zero) |
[Try These First](#try-these-first) |
[Deep Dives](#deep-dives) |
[Docs](#documentation)
</div>
<div align="center">
<a href="https://www.youtube.com/watch?v=k78HX_RA9Q0&t=19s">
<img src="docs/res/thumbnail-install.webp" alt="Agent Zero Installation Guide" width="100%"/>
</a>
<img alt="Agent Zero driving Blender in its built-in XFCE desktop" src="docs/res/usage/webui/agentzero-xfce-computer.gif" width="100%" />
</div>
# How To Install
# Why Agent Zero
Choose the install path that matches your machine.
| Feature | Why it matters |
| --- | --- |
| **Full Linux desktop** | The agent can use real GUI software, terminals, files, and desktop apps inside the Canvas. |
| **Browser DOM annotation** | Click page elements and turn them into inspect, change, lift, or review instructions. |
| **Live document cowork** | Edit Markdown, Writer, Spreadsheet, and Presentation files together instead of losing work in chat. |
| **Plugin Hub** | Install 100+ community plugins or publish your own extension points. |
| **Projects and memory** | Keep files, instructions, secrets, memories, repositories, and model presets isolated per project. |
| **Host-machine bridge** | Connect with the A0 CLI so the same agent can work in your real local repositories. |
| **Multi-agent cooperation** | Let agents delegate research, coding, analysis, or review tasks to focused subagents. |
| **Transparent internals** | Prompts, tools, plugins, skills, and settings are inspectable and editable. |
| Path | Best for | What it does |
| --- | --- | --- |
| **A0 Launcher** | Desktop users who want the guided path | Downloads Agent Zero, creates and manages local Instances, and helps set up the container runtime when needed. |
| **A0 Install** | Terminals, SSH sessions, servers, and scripted setup | Installs Agent Zero from the command line, reuses an existing Docker-compatible runtime first, and can run headlessly. |
| **Docker** | Machines that already have Docker ready | Runs the Agent Zero container directly. |
# Quick Start
## A0 Launcher
## Recommended: A0 Launcher
The desktop **A0 Launcher** is the recommended way to install Agent Zero on a personal machine. Download the Launcher, open it, and let it check your local runtime. If Docker is missing or stopped, the Launcher offers a setup path before it downloads Agent Zero. If you already host Agent Zero elsewhere, add it as a remote Instance and use the Launcher without local Docker setup.
The desktop **A0 Launcher** is the fastest guided path on a personal machine. Download it, open it, and let it check Docker, create Instances, manage ports, and connect to local or remote Agent Zero installs.
### Downloads
Agent Zero runs wherever Docker runs, from a $6 VPS or Raspberry Pi to a local workstation or GPU server.
| Architecture | macOS | Linux | Windows |
| --- | --- | --- | --- |
| x86 | [Mac Intel](https://github.com/agent0ai/a0-launcher/releases/download/v1.1/a0-launcher-1.1-macos-x64.dmg) | [Linux x86](https://github.com/agent0ai/a0-launcher/releases/download/v1.1/a0-launcher-1.1-linux-x64.AppImage) | [Windows x86](https://github.com/agent0ai/a0-launcher/releases/download/v1.1/a0-launcher-1.1-windows-x64.exe) |
| ARM64 | [Mac Apple Silicon](https://github.com/agent0ai/a0-launcher/releases/download/v1.1/a0-launcher-1.1-macos-arm64.dmg) | [Linux ARM64](https://github.com/agent0ai/a0-launcher/releases/download/v1.1/a0-launcher-1.1-linux-arm64.AppImage) | [Windows ARM64](https://github.com/agent0ai/a0-launcher/releases/download/v1.1/a0-launcher-1.1-windows-arm64.exe) |
| x86 | [Mac Intel](https://github.com/agent0ai/a0-launcher/releases/download/v1.2/a0-launcher-1.2-macos-x64.dmg) | [Linux x86](https://github.com/agent0ai/a0-launcher/releases/download/v1.2/a0-launcher-1.2-linux-x64.AppImage) | [Windows x86](https://github.com/agent0ai/a0-launcher/releases/download/v1.2/a0-launcher-1.2-windows-x64.exe) |
| ARM64 | [Mac Apple Silicon](https://github.com/agent0ai/a0-launcher/releases/download/v1.2/a0-launcher-1.2-macos-arm64.dmg) | [Linux ARM64](https://github.com/agent0ai/a0-launcher/releases/download/v1.2/a0-launcher-1.2-linux-arm64.AppImage) | [Windows ARM64](https://github.com/agent0ai/a0-launcher/releases/download/v1.2/a0-launcher-1.2-windows-arm64.exe) |
See the [A0 Launcher v1.1 release](https://github.com/agent0ai/a0-launcher/releases/tag/v1.1) for release notes and updater metadata. See the [Launcher guide](./docs/guides/launcher.md) for the first-run walkthrough.
See the [A0 Launcher v1.2 release](https://github.com/agent0ai/a0-launcher/releases/tag/v1.2) for release notes and updater metadata. See the [Launcher guide](./docs/guides/launcher.md) for the first-run walkthrough.
<details>
<summary><strong>Other install paths</strong></summary>
## A0 Install
@ -92,13 +94,28 @@ docker run -p 80:80 -v a0_usr:/a0/usr agent0ai/agent-zero
Open the Web UI, configure your LLM provider, and start with a concrete task. For the full setup and onboarding experience, see the [Installation guide](./docs/setup/installation.md).
# What Makes Agent Zero Different
</details>
## Troubleshooting
- **Docker is not running:** start Docker Desktop or your Docker service, then reopen the Launcher or rerun the install command.
- **Port 80 is already in use:** use the Launcher to pick another port, or run Docker directly with `-p 5080:80` and open `http://localhost:5080`.
- **Installing on a server:** use the A0 Install Quick Start command with `--quick-start --name agent-zero --port 5080`.
- **Still blocked:** see the [Troubleshooting guide](./docs/guides/troubleshooting.md).
# Try These First
- **Annotate a design you like:** "Open this template site in the Browser. I'll annotate the hero section - re-implement it in my project's React + Tailwind stack."
- **Cowork on a spreadsheet:** "Create an editable ODS budget model with assumptions and monthly projections."
- **Drive a desktop app:** "Use the Linux Desktop to open Blender and create a simple 3D logo for me."
- **Review a web UI:** "Open my local app in the Browser. I will annotate the page with comments; then implement the requested UI fixes."
- **Create a specialist:** "Create an Agent Profile for financial analysis with cautious reasoning, clear assumptions, and spreadsheet-first deliverables."
- **Recover a workspace:** "Show me recent Time Travel snapshots and explain what changed before I revert anything."
# Deep Dives
## A Real Linux Desktop in the Canvas
<img alt="Agent Zero driving Blender in its built-in XFCE desktop" src="docs/res/usage/webui/agentzero-xfce-computer.gif" />
<br>
Agent Zero opens its own Linux desktop inside the right-side Canvas. Not a remote VM, not a shared clipboard, but a real XFCE desktop session running in the container.
That means the agent can drive *real desktop software*: open Blender to model a 3D object, jump into a terminal window, manage files visually, run a GUI tool that has no API.
@ -121,7 +138,7 @@ Annotate mode turns any webpage into an interactive directive surface. Click an
- **Lift it** - see a card, hero, or component on someone else's site that you like? Capture it and have the agent re-implement it in your own project's stack.
- **Comment it** - leave actionable notes pinned to elements during a UI review; the agent reads the comments and ships the fixes.
The Docker browser is the default live Browser surface. Browser history keeps screenshots of important steps, so older chats can still show what the agent saw. The Browser also supports Chrome extensions inside the Docker browser, and **Bring Your Own Browser** through the A0 CLI Connector lets the agent drive Chrome/Edge/Chromium on your own machine.
The Docker browser is the default live Browser surface. Browser history keeps screenshots of important steps, so older chats can still show what the agent saw. The Browser also supports Chrome extensions inside the Docker browser, and **Bring Your Own Browser** through the A0 CLI Connector lets the agent drive Chrome, Edge, Brave, Opera, Vivaldi, or Chromium on your own machine.
See the [Browser guide](./docs/guides/browser.md) for screenshots, settings, host-browser setup, and troubleshooting.
@ -226,31 +243,9 @@ Almost nothing is hidden. Prompts live in `prompts/`, tools live in `tools/` or
Agent Zero supports plugins, MCP, A2A, custom tools, custom prompts, project-scoped configuration, environment-based deployment settings, and a Web UI designed to keep the agent's work readable in real time.
## Try These First
## Time Travel
- **Annotate a design you like:** "Open this template site in the Browser. I'll annotate the hero section - re-implement it in my project's React + Tailwind stack."
- **Cowork on a spreadsheet:** "Create an editable ODS budget model with assumptions and monthly projections."
- **Drive a desktop app:** "Use the Linux Desktop to open Blender and create a simple 3D logo for me."
- **Review a web UI:** "Open my local app in the Browser. I will annotate the page with comments; then implement the requested UI fixes."
- **Create a specialist:** "Create an Agent Profile for financial analysis with cautious reasoning, clear assumptions, and spreadsheet-first deliverables."
- **Recover a workspace:** "Show me recent Time Travel snapshots and explain what changed before I revert anything."
## Agent Zero and Space Agent
Agent Zero is the open framework and Linux-powered agent workbench.
[Space Agent](https://github.com/agent0ai/space-agent) is our newer product direction for the agent-shaped workspace: a Space the agent can reshape from inside your browser, with live demos, a desktop app, and a path to running a real server for yourself or your team.
<p align="left">
<a href="https://www.youtube.com/watch?v=CNRHxEZ8yqs"><img src="https://github.com/agent0ai/space-agent/raw/main/.github/thumbnail.webp" alt="Watch Space Agent on YouTube" width="560" /></a>
</p>
If you want the raw power and deep customizability of an agent with a full Linux system, start here with Agent Zero. If you want the polished Space experience for easier personal, team, desktop, or self-hosted use, explore [Space Agent](https://github.com/agent0ai/space-agent).
## Time Travel (powered by Space Agent)
Time Travel gives Agent Zero-owned `/a0/usr` workspaces snapshot history, diff inspection, travel, and revert. It is designed for recoverable agent work: see what changed, compare files, inspect a past state, and roll back when needed. Try it in Space Agent as well (link above).
Time Travel gives Agent Zero-owned `/a0/usr` workspaces snapshot history, diff inspection, travel, and revert. It is designed for recoverable agent work: see what changed, compare files, inspect a past state, and roll back when needed.
<img alt="Time Travel" src="docs/res/time-travel.png" />
@ -268,17 +263,6 @@ It is not a replacement for Git or backups. It is a practical safety layer for t
- **Client/project isolation:** keep memory, secrets, instructions, files, and model choices separated by project.
- **Scheduled operations:** run recurring checks and monitoring tasks with project-scoped context and credentials.
## Safety Model
Agent Zero is powerful because it can use a real environment.
- Keep it running inside Docker or another isolated environment.
- Do not mount your entire home directory unless you understand the risk.
- Grant A0 CLI Read+Write access and remote code execution only for machines and workspaces you trust.
- Store credentials in project secrets or settings, not in prompts or public files.
- Review actions that touch accounts, money, production systems, or private data.
- Keep backups for important workspaces.
## Documentation
| I want to... | Start here |
@ -314,3 +298,16 @@ You can help by improving docs, creating skills, publishing plugins, testing mod
- [YouTube](https://www.youtube.com/@AgentZeroFW) for demos and tutorials.
- [X](https://x.com/Agent0ai), [LinkedIn](https://www.linkedin.com/company/109758317), and [Warpcast](https://warpcast.com/agent-zero) for updates.
- [GitHub Issues](https://github.com/agent0ai/agent-zero/issues) for bugs and feature requests.
[Space Agent](https://github.com/agent0ai/space-agent) is the related, more polished product direction for the agent-shaped workspace. Agent Zero remains the open framework and Linux-powered workbench.
## Safety Model
Agent Zero is powerful because it can use a real environment.
- Keep it running inside Docker or another isolated environment.
- Do not mount your entire home directory unless you understand the risk.
- Grant A0 CLI Read+Write access and remote code execution only for machines and workspaces you trust.
- Store credentials in project secrets or settings, not in prompts or public files.
- Review actions that touch accounts, money, production systems, or private data.
- Keep backups for important workspaces.

View file

@ -28,6 +28,7 @@
## Key Concepts
- Important called helpers/classes observed in the source: `settings.get_settings`, `settings.convert_out`.
- The returned settings map includes normalized `ui_control_visibility` mobile and desktop flags for the configurable WebUI controls.
- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
## Work Guidance

View file

@ -26,6 +26,7 @@
## Key Concepts
- Important called helpers/classes observed in the source: `settings.convert_in`, `settings.set_settings`, `settings.convert_out`, `settings.Settings`.
- The settings payload accepts normalized `ui_control_visibility` mobile and desktop flags and returns the persisted map with the rest of the settings.
- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
## Work Guidance

View file

@ -84,6 +84,7 @@ chat:
endpoint_url: "/v1/models"
default_base: "http://host.docker.internal:1234"
kwargs:
a0_api_mode: chat
api_base: "http://host.docker.internal:1234/v1"
api_key: "lm-studio"
llama_cpp:
@ -93,6 +94,7 @@ chat:
endpoint_url: "/v1/models"
default_base: "http://host.docker.internal:8080"
kwargs:
a0_api_mode: chat
api_base: "http://host.docker.internal:8080/v1"
api_key: "llama-cpp"
mistral:
@ -112,6 +114,11 @@ chat:
endpoint_url: "/models"
kwargs:
api_base: https://api.tokenfactory.nebius.com/v1
nvidia_nim:
name: NVIDIA NIM
litellm_provider: nvidia_nim
models_list:
endpoint_url: "https://integrate.api.nvidia.com/v1/models"
ollama:
name: Ollama
litellm_provider: ollama
@ -120,6 +127,7 @@ chat:
format: "ollama"
default_base: "http://host.docker.internal:11434"
kwargs:
a0_api_mode: chat
api_base: "http://host.docker.internal:11434"
omlx:
name: oMLX
@ -128,6 +136,7 @@ chat:
endpoint_url: "/v1/models"
default_base: "http://host.docker.internal:8000"
kwargs:
a0_api_mode: chat
api_base: "http://host.docker.internal:8000/v1"
api_key: "omlx"
ollama_cloud:
@ -136,6 +145,7 @@ chat:
models_list:
endpoint_url: "/models"
kwargs:
a0_api_mode: chat
api_base: https://ollama.com/v1
openai:
name: OpenAI
@ -173,6 +183,7 @@ chat:
models_list:
endpoint_url: "https://api.venice.ai/api/v1/models"
kwargs:
a0_api_mode: chat
api_base: https://api.venice.ai/api/v1
venice_parameters:
include_venice_system_prompt: false
@ -183,6 +194,7 @@ chat:
endpoint_url: "/v1/models"
default_base: "http://host.docker.internal:8000"
kwargs:
a0_api_mode: chat
api_base: "http://host.docker.internal:8000/v1"
api_key: "vllm"
xai:
@ -207,6 +219,8 @@ chat:
other:
name: Other OpenAI compatible
litellm_provider: openai
kwargs:
a0_api_mode: chat
embedding:
huggingface:
@ -230,6 +244,11 @@ embedding:
mistral:
name: Mistral AI
litellm_provider: mistral
nvidia_nim:
name: NVIDIA NIM
litellm_provider: nvidia_nim
models_list:
endpoint_url: "https://integrate.api.nvidia.com/v1/models"
ollama:
name: Ollama
litellm_provider: ollama

View file

@ -13,7 +13,8 @@
## Local Contracts
- Preserve the two-runtime model documented in the root contract: framework runtime under `/opt/venv-a0` and agent execution runtime under `/opt/venv`.
- Preserve the two-runtime model: the Python 3.12 framework runtime under `/opt/venv-a0` runs the WebUI, APIs, scheduler, framework imports, and plugin hooks; the Python 3.13 agent execution runtime under `/opt/venv` runs agent terminal tasks and user code.
- Verify backend imports and plugin hooks with `/opt/venv-a0`; packages installed into `/opt/venv` do not prove framework compatibility.
- Do not bake secrets, local `.env` values, or user data into images.
- Keep compose mounts aligned with `usr/`, `logs/`, and other runtime-state expectations.
- Image changes that affect GitHub publishing must stay synchronized with `.github/workflows/docker-publish.yml`.

View file

@ -22,6 +22,9 @@
- Do not bake secrets, local `.env` values, or user data into the image.
- Runtime startup must ensure `/a0/usr/uploads` exists before supervised services start.
- Runtime startup raises the soft open-file limit toward `A0_NOFILE_LIMIT` (default `65535`) before supervisord starts, bounded by the container hard limit.
- Self-update user-data backups skip Time Travel shadow history under `usr/.time_travel/` and transient Desktop agent state.
- Self-update waits up to 180 seconds for the updated or restored WebUI health check by default; `A0_SELF_UPDATE_HEALTH_TIMEOUT_SECONDS` may override it.
- Successful or already-current self-updates refresh an installed Codex CLI with npm on a best-effort basis; missing CLIs and registry failures must not block Agent Zero startup.
## Work Guidance

View file

@ -36,7 +36,7 @@ DEFAULT_HEALTH_URL = os.environ.get(
"http://127.0.0.1:80/api/health",
)
DEFAULT_HEALTH_TIMEOUT_SECONDS = int(
os.environ.get("A0_SELF_UPDATE_HEALTH_TIMEOUT_SECONDS", "120")
os.environ.get("A0_SELF_UPDATE_HEALTH_TIMEOUT_SECONDS", "180")
)
DEFAULT_HEALTH_POLL_INTERVAL_SECONDS = float(
os.environ.get("A0_SELF_UPDATE_HEALTH_POLL_INTERVAL_SECONDS", "2")
@ -405,6 +405,11 @@ def should_exclude_from_usr_backup(
logger: AttemptLogger,
) -> bool:
parts = relative_dir.parts
if parts and parts[0] == ".time_travel":
logger.log(
f"Skipping Time Travel history during usr backup: {Path('usr') / relative_dir}"
)
return True
if (
len(parts) >= 6
and parts[0] == "plugins"
@ -630,6 +635,29 @@ def clean_uv_cache(logger: AttemptLogger) -> None:
logger.log(f"uv cache clean skipped after error: {exc}")
def refresh_codex_cli(logger: AttemptLogger) -> None:
codex_path = shutil.which("codex")
if not codex_path:
logger.log("Codex CLI not installed, skipping Codex refresh.")
return
npm_path = shutil.which("npm")
if not npm_path:
logger.log("npm executable not found, skipping Codex refresh.")
return
logger.log("Refreshing the installed Codex CLI after self-update.")
try:
run_command(
[npm_path, "install", "--global", "@openai/codex@latest"],
cwd=None,
logger=logger,
error_message="Failed to refresh the installed Codex CLI.",
)
except Exception as exc:
logger.log(f"Codex CLI refresh skipped after error: {exc}")
def has_local_rollback_changes(repo_dir: Path) -> bool:
status = git_output(repo_dir, "status", "--porcelain=v1", "--untracked-files=all")
return bool(status.strip())
@ -1143,6 +1171,7 @@ def execute_pending_update(
logger=logger,
)
if healthy:
refresh_codex_cli(logger)
record_result(
status="success",
message=f"Updated Agent Zero to branch {branch}, {resolved_target['target_description']}.",
@ -1420,6 +1449,7 @@ def docker_run_ui() -> int:
logger.log(
"Requested tag already matches the installed version, skipping file replacement."
)
refresh_codex_cli(logger)
record_result(
status="skipped",
message="Requested tag already matches the installed version.",
@ -1463,8 +1493,11 @@ def main(argv: list[str] | None = None) -> int:
return docker_run_ui()
if args[0] == "trigger-update":
return trigger_update_command(args[1:])
if args[0] == "refresh-codex":
refresh_codex_cli(AttemptLogger(LOG_FILE))
return 0
if args[0] in {"-h", "--help"}:
print("Usage: self_update_manager.py [docker-run-ui | trigger-update ...]")
print("Usage: self_update_manager.py [docker-run-ui | trigger-update ... | refresh-codex]")
return 0
print(f"Unknown command: {args[0]}", file=sys.stderr)
return 1

View file

@ -9,6 +9,7 @@
- `README.md`, `quickstart.md`, `guides/`, and `setup/` cover user-facing setup and workflows.
- `developer/` covers compact developer references and source handoffs.
- `plans/` covers implementation plans, migration notes, and staged technical roadmaps.
- `res/` contains documentation images and other documentation assets.
## Local Contracts

View file

@ -132,15 +132,22 @@ machine.
- [ ] Keep A0 CLI connected to the Agent Zero chat.
- [ ] In Agent Zero Web UI, open Browser plugin settings and choose **Bring Your
Own Browser**.
- [ ] If you want Agent Zero to use an already-open personal Chrome window, open
that browser first.
- [ ] In that browser, go to `chrome://inspect/#remote-debugging`.
- [ ] If you want Agent Zero to use an already-open personal browser window,
open that browser first.
- [ ] In that browser, open its remote debugging page.
- [ ] Enable **Allow remote debugging for this browser instance**.
Remote debugging pages:
| Browser | Page |
| --- | --- |
| Chrome, Edge, Brave, Vivaldi, Chromium | `chrome://inspect/#remote-debugging` |
| Opera | `opera://inspect/#remote-debugging` |
![Chrome remote debugging setting](../res/usage/browser/host-browser-remote-debugging-setting.png)
When Agent Zero performs its first Browser action against that host browser,
Chrome asks for confirmation. Click **Allow** if you trust this Agent Zero
the browser asks for confirmation. Click **Allow** if you trust this Agent Zero
instance and A0 CLI connection.
![Chrome remote debugging allow prompt](../res/usage/browser/host-browser-remote-debugging-allow.png)
@ -149,11 +156,30 @@ A0 CLI does not take over the browser while it is only checking status. Browser
control starts when Agent Zero actually needs to use the browser.
> [!IMPORTANT]
> Remote debugging gives the connected app full control of that Chrome session,
> Remote debugging gives the connected app full control of that browser session,
> including access to saved data, cookies, site data, and navigation. Use it only
> with trusted Agent Zero instances and browser windows you intend the agent to
> control.
The **Host browser** list in Browser settings comes from the connected local A0
CLI, not from the Agent Zero Web UI server. It shows Automatic, currently
advertised debug endpoints, and **Custom endpoint**. If a newly authorized
browser does not appear, restart or reconnect A0 CLI.
If the inspect checkbox is not enough for your browser build, launch it with an
explicit remote debugging port and a separate profile:
```bash
opera --remote-debugging-port=9222 --user-data-dir="$HOME/.config/a0-opera-debug"
```
Then choose **Custom endpoint** in Browser settings, run `/browser ws://...` in
A0 CLI, or pass the full DevTools websocket endpoint to A0 CLI:
```bash
export A0_HOST_BROWSER_REMOTE_DEBUGGING_ENDPOINTS="ws://127.0.0.1:9222/devtools/browser/..."
```
### Browser Profiles
```bash

View file

@ -137,8 +137,8 @@ development, Agent Zero can install it the first time it is needed.
## Bring Your Own Browser
Bring Your Own Browser lets Agent Zero use Chrome, Edge, or Chromium on your own
computer through A0 CLI.
Bring Your Own Browser lets Agent Zero use Chrome, Edge, Brave, Opera, Vivaldi,
or Chromium on your own computer through A0 CLI.
Use it when the page, login, or browser profile should stay on your machine.
@ -146,8 +146,35 @@ Requirements:
- [ ] Keep A0 CLI connected to the Agent Zero chat.
- [ ] Choose **Bring Your Own Browser** in Browser settings.
- [ ] Use Chrome, Edge, or Chromium on the host.
- [ ] For personal Chrome remote debugging, open the host browser first, go to `chrome://inspect/#remote-debugging`, and enable **Allow remote debugging for this browser instance**.
- [ ] Use a Chromium-family browser on the host: Chrome, Edge, Brave, Opera, Vivaldi, or Chromium.
- [ ] For an already-open browser, open its remote debugging page and enable **Allow remote debugging for this browser instance**.
Remote debugging pages:
| Browser | Page |
| --- | --- |
| Chrome, Edge, Brave, Vivaldi, Chromium | `chrome://inspect/#remote-debugging` |
| Opera | `opera://inspect/#remote-debugging` |
The **Host browser** list shows Automatic, currently advertised debug endpoints,
and **Custom endpoint**. If a browser does not appear after enabling remote
debugging, restart or reconnect the local A0 CLI. Restarting only the Agent Zero
Web UI server does not refresh the browser inventory; the list comes from the
connected CLI.
As a fallback, launch the browser with an explicit debugging port and profile
directory:
```bash
opera --remote-debugging-port=9222 --user-data-dir="$HOME/.config/a0-opera-debug"
```
Then choose **Custom endpoint** in Browser settings, or pass the full DevTools
websocket endpoint to the CLI:
```bash
export A0_HOST_BROWSER_REMOTE_DEBUGGING_ENDPOINTS="ws://127.0.0.1:9222/devtools/browser/..."
```
![Host browser remote debugging setting](../res/usage/browser/host-browser-remote-debugging-setting.png)

View file

@ -41,6 +41,7 @@ PLAYWRIGHT_BROWSERS_PATH=tmp/playwright playwright install chromium
If **Bring Your Own Browser** mode fails:
- keep A0 CLI connected to the chat;
- restart or reconnect A0 CLI after enabling remote debugging in a browser;
- run `/browser status` in A0 CLI;
- check that Browser settings still say **Bring Your Own Browser**;
- check **Page content access** if page text or screenshots are blocked.

View file

@ -16,6 +16,7 @@
- Extension functions must match the implicit hook's supplied arguments.
- Preserve ordering prefixes where exception handling, watchdog registration, or cleanup depends on them.
- Hooks that mirror persisted AI responses into UI logs must reuse existing stream log items and avoid duplicating live response-tool logs.
- Recovery-loop circuit breakers must stop at the General Settings limit and render their user-visible cost warning from a core framework prompt.
## Work Guidance

View file

@ -0,0 +1,55 @@
from helpers.errors import HandledException
from helpers.extension import Extension
from helpers.settings import get_settings
STATE_KEY = "_unusable_response_failures"
class StopUnusableResponseLoop(Extension):
def execute(self, data: dict | None = None, **kwargs):
if not self.agent or not isinstance(data, dict):
return
call_kwargs = data.get("kwargs")
message = call_kwargs.get("message") if isinstance(call_kwargs, dict) else None
call_args = data.get("args")
if message is None and isinstance(call_args, tuple) and len(call_args) > 1:
message = call_args[1]
if not isinstance(message, str):
return
if message not in {
self.agent.read_prompt("fw.msg_misformat.md"),
self.agent.read_prompt("fw.msg_repeat.md"),
}:
return
loop_data = getattr(self.agent, "loop_data", None)
state = getattr(loop_data, "params_persistent", None)
iteration = getattr(loop_data, "iteration", None)
if not isinstance(state, dict) or not isinstance(iteration, int):
return
previous = state.get(STATE_KEY, {})
if not isinstance(previous, dict):
previous = {}
previous_iteration = previous.get("iteration")
if previous_iteration == iteration:
return
count = (
previous.get("count", 0) + 1
if previous_iteration == iteration - 1
else 1
)
state[STATE_KEY] = {"iteration": iteration, "count": count}
limit = get_settings()["max_consecutive_unusable_responses"]
if count < limit:
return
stop_message = self.agent.read_prompt(
"fw.msg_unusable_response_limit.md", limit=limit
)
self.agent.context.log.log(type="warning", content=stop_message)
data["exception"] = HandledException(stop_message)

View file

@ -12,6 +12,7 @@
- Preserve history consistency before saving chats.
- Do not skip persistence for successful loops unless the hook contract explicitly permits it.
- History compression that rewrites local history must clear the active Responses provider continuation while preserving stored response IDs for cleanup.
## Work Guidance

View file

@ -1,10 +1,18 @@
from helpers.extension import Extension
from agent import LoopData
from helpers.defer import DeferredTask, THREAD_BACKGROUND
from helpers.history import clear_responses_provider_state
DATA_NAME_TASK = "_organize_history_task"
async def compress_history(agent) -> bool:
compressed = bool(await agent.history.compress())
if compressed:
clear_responses_provider_state(agent)
return compressed
class OrganizeHistory(Extension):
async def execute(self, loop_data: LoopData = LoopData(), **kwargs):
if not self.agent:
@ -17,6 +25,6 @@ class OrganizeHistory(Extension):
# start task
task = DeferredTask(thread_name=THREAD_BACKGROUND)
task.start_task(self.agent.history.compress)
task.start_task(compress_history, self.agent)
# set to agent to be able to wait for it
self.agent.set_data(DATA_NAME_TASK, task)

View file

@ -2,20 +2,21 @@
## Purpose
- Own prompt protocol and extras appended around primary message-loop prompt construction.
- Own prompt protocol, prompt extras, and history reattachment around primary message-loop prompt construction.
## Ownership
- Ordered Python files own current datetime, skill recall/load context, agent info, parallel job status, and workdir extras injection.
- Active skill instructions belong in prompt protocol.
- Ordered Python files own current datetime, relevant-skill hints, loaded-skill history reattachment, agent info, parallel job status, and workdir extras injection.
- Explicitly loaded skill bodies belong in tool-result history with metadata so they can survive persistence and be reattached after compaction.
- Explicitly loaded skill IDs are chat-wide context data, not agent-local state.
- Skills must not write selected or loaded skill bodies into protocol or extras.
## Local Contracts
- Keep injected content bounded and clearly attributed.
- Preserve ordering where later prompt extras depend on earlier recall or load results.
- Do not expose secrets or private files from workdir extras.
- Relevant-skill recall should search the raw user message when available, not the rendered history wrapper.
## Work Guidance

View file

@ -8,9 +8,13 @@ class RecallRelevantSkills(Extension):
if not self.agent or loop_data.iteration != 0:
return
user_instruction = (
loop_data.user_message.output_text() if loop_data.user_message else ""
).strip()
content = loop_data.user_message.content if loop_data.user_message else ""
if isinstance(content, dict):
user_instruction = str(content.get("user_message") or "").strip()
else:
user_instruction = (
loop_data.user_message.output_text() if loop_data.user_message else ""
).strip()
if len(user_instruction) < 8:
return

View file

@ -14,9 +14,6 @@ class IncludeLoadedSkills(Extension):
if not self.agent:
return
loop_data.protocol_persistent.pop("loaded_skills", None)
loop_data.extras_persistent.pop("loaded_skills", None)
skill_names = skills.get_loaded_skill_names(self.agent)
if not skill_names:
return

View file

@ -1,6 +1,9 @@
from helpers.extension import Extension
from agent import LoopData
from extensions.python.message_loop_end._10_organize_history import DATA_NAME_TASK
from extensions.python.message_loop_end._10_organize_history import (
DATA_NAME_TASK,
compress_history,
)
from helpers.defer import DeferredTask, THREAD_BACKGROUND
MAX_SYNC_COMPRESSION_PASSES = 64
@ -33,7 +36,7 @@ class OrganizeHistoryWait(Extension):
else:
# no task was running, start and wait
self.agent.context.log.set_progress("Compressing history...")
compressed = await self.agent.history.compress()
compressed = await compress_history(self.agent)
after_tokens = self.agent.history.get_tokens()
if not compressed or after_tokens >= before_tokens:

View file

@ -14,6 +14,7 @@
- Preserve user data and create backups or reversible paths when changing durable state.
- Keep long-running work bounded and observable.
- `_10_self_update_manager.py` may replace `/exe/self_update_manager.py` from the repository copy when the installed runtime updater is stale; it must validate required safety markers and keep a backup before replacement.
- After synchronizing a stale self-update manager, `_10_self_update_manager.py` starts that manager's best-effort Codex CLI refresh in the background so the update that introduces the hook does not need a second restart.
## Work Guidance

View file

@ -3,6 +3,8 @@ from __future__ import annotations
import os
import shutil
import stat
import subprocess
import sys
from pathlib import Path
from typing import Any
@ -25,6 +27,8 @@ REQUIRED_RUNTIME_MARKERS = (
"Skipping non-regular usr backup entry",
"def clean_transient_desktop_agent_state(",
"clean_transient_desktop_agent_state(REPO_DIR, logger)",
"def refresh_codex_cli(",
"refresh_codex_cli(logger)",
)
@ -33,6 +37,9 @@ class SelfUpdateManagerRuntimeSync(Extension):
result = ensure_self_update_manager_runtime_current()
if result.get("updated"):
PrintStyle.info("Self-update manager runtime synchronized:", result["target"])
warning = start_codex_cli_refresh(result["target"])
if warning:
PrintStyle.warning("Codex CLI refresh could not be started:", warning)
elif result.get("warning"):
PrintStyle.warning("Self-update manager runtime sync skipped:", result["warning"])
@ -84,6 +91,20 @@ def ensure_self_update_manager_runtime_current(
}
def start_codex_cli_refresh(manager_path: Path | str) -> str:
try:
subprocess.Popen(
[sys.executable, str(manager_path), "refresh-codex"],
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
)
except OSError as exc:
return str(exc)
return ""
def _missing_required_markers(text: str) -> list[str]:
return [marker for marker in REQUIRED_RUNTIME_MARKERS if marker not in text]

View file

@ -7,7 +7,7 @@
## Ownership
- Ordered Python files own main, tools, MCP, secrets, skills, and project prompt sections.
- Active project instruction bodies are moved into prompt protocol; the system prompt keeps project metadata and stable project rules.
- Active project instruction bodies and active-project AGENTS.md path-chain guidance are moved into prompt protocol; the system prompt keeps project metadata and stable project rules.
## Local Contracts

View file

@ -25,9 +25,16 @@ async def build_prompt(agent: Agent, loop_data: LoopData | None = None) -> str:
result = agent.read_prompt("agent.system.projects.main.md")
project_name = agent.context.get_data(projects.CONTEXT_DATA_KEY_PROJECT)
if loop_data:
loop_data.protocol_persistent.pop("agents_md_instructions", None)
loop_data.protocol_persistent.pop("project_instructions", None)
if project_name:
project_vars = projects.build_system_prompt_vars(project_name)
if loop_data and project_vars.get("include_agents_md", True):
agents_md_protocol = projects.build_agents_md_protocol(project_name)
if agents_md_protocol:
loop_data.protocol_persistent["agents_md_instructions"] = (
agents_md_protocol
)
if loop_data and project_vars.get("project_instructions"):
loop_data.protocol_persistent["project_instructions"] = agent.read_prompt(
"agent.protocol.projects.instructions.md",

View file

@ -15,7 +15,7 @@
- Preserve public helper APIs used by core code and plugins unless all callers, docs, and tests are updated.
- Use structured parsers and serializers for YAML, JSON, paths, and URLs instead of ad hoc string handling.
- Keep path handling constrained to intended roots for user files, uploads, downloads, projects, and workdirs.
- Project metadata defaults must remain backwards-compatible; missing `include_agents_md` is treated as enabled and project instruction file content is injected with an explicit source path.
- Project metadata defaults must remain backwards-compatible; missing `include_agents_md` is treated as enabled, project instruction file content is injected with an explicit source path, and active-project AGENTS.md path-chain guidance is assembled into prompt protocol without duplicating the project root AGENTS.md.
- Do not hardcode secrets, provider keys, local absolute paths, or environment-specific values.
- Use `RepairableException` for errors an agent may be able to fix.
- This directory is a file-documented DOX profile: every direct `*.py` helper module must have a same-directory `*.py.dox.md` file named by appending `.dox.md` to the full Python filename.

View file

@ -63,6 +63,7 @@ class BackupService:
return f"""# User data
# All persistent user data is now centralized in /usr for easier backup and restore
{agent_root}/usr/**
!{agent_root}/usr/.time_travel/**
"""
def _get_agent_zero_version(self) -> str:

View file

@ -26,6 +26,7 @@
- Observed side-effect areas: filesystem reads, filesystem writes, filesystem deletion, settings/state persistence, secret handling.
- Imported dependency areas include: `datetime`, `helpers`, `helpers.localization`, `helpers.print_style`, `json`, `os`, `pathspec`, `platform`, `tempfile`, `typing`, `zipfile`.
- `test_patterns(..., max_files=None)` is the unlimited scan mode. UI preview and dry-run callers may pass bounded limits, but real backup creation and restore clean-before-restore must use unlimited matching so archives and cleanup are not silently truncated.
- Default backup metadata includes persistent `/usr` data but excludes Time Travel shadow history under `usr/.time_travel/**`.
## Key Concepts

View file

@ -723,6 +723,28 @@ def output_text(messages: list[OutputMessage], ai_label="ai", human_label="human
return "\n".join(_stringify_output(o, ai_label, human_label) for o in messages)
def clear_responses_provider_state(agent) -> None:
key = getattr(agent, "DATA_NAME_RESPONSES_STATE", "responses_state")
get_data = getattr(agent, "get_data", None)
set_data = getattr(agent, "set_data", None)
if not callable(get_data) or not callable(set_data):
return
state = get_data(key)
if not isinstance(state, dict):
return
state = dict(state)
removed = False
for field in ("response_id", "previous_response_id"):
if field in state:
state.pop(field, None)
removed = True
if removed:
set_data(key, state)
def _merge_outputs(a: MessageContent, b: MessageContent) -> MessageContent:
if isinstance(a, str) and isinstance(b, str):
return a + "\n" + b

View file

@ -65,6 +65,7 @@
- `group_messages_abab(messages: list[BaseMessage]) -> list[BaseMessage]`
- `output_langchain(messages: list[OutputMessage])`
- `output_text(messages: list[OutputMessage], ai_label=..., human_label=...)`
- `clear_responses_provider_state(agent) -> None`
- `_merge_outputs(a: MessageContent, b: MessageContent) -> MessageContent`
- `_merge_properties(a: Dict[str, MessageContent], b: Dict[str, MessageContent]) -> Dict[str, MessageContent]`
- `_is_raw_message(obj: object) -> bool`
@ -77,6 +78,7 @@
- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
- `clear_responses_provider_state(agent)` removes the active provider continuation IDs after local history rewrites while preserving stored response ID lists for later cleanup.
- Observed side-effect areas: filesystem writes, filesystem deletion, model calls, plugin state, settings/state persistence, secret handling.
- Imported dependency areas include: `abc`, `asyncio`, `collections`, `collections.abc`, `enum`, `helpers`, `json`, `langchain_core.messages`, `math`, `plugins._model_config.helpers.model_config`, `typing`, `uuid`.

View file

@ -21,6 +21,7 @@ from contextlib import AsyncExitStack
from shutil import which
from datetime import timedelta
import json
import shlex
import uuid
from helpers import errors
from helpers import settings
@ -112,6 +113,44 @@ def _normalize_disabled_tools(value: Any) -> list[str]:
return [str(item).strip() for item in value if str(item).strip()]
def _split_stdio_command(command: Any) -> tuple[str, list[str]]:
text = str(command or "").strip()
if not text:
return "", []
try:
parts = shlex.split(text)
except ValueError:
return text, []
if not parts:
return "", []
return parts[0], parts[1:]
def _split_stdio_arg_fragment(arg: str) -> list[str]:
try:
parts = shlex.split(arg)
except ValueError:
return [arg]
if len(parts) <= 1:
return parts or []
if parts[0].startswith("-") and "=" not in parts[0]:
return parts
if any(part.startswith("-") for part in parts[1:]):
return parts
return [arg]
def _normalize_stdio_args(value: Any) -> list[str]:
if not isinstance(value, list):
return []
args: list[str] = []
for item in value:
text = str(item).strip()
if text:
args.extend(_split_stdio_arg_fragment(text))
return args
def initialize_mcp(mcp_servers_config: str):
if not MCPConfig.get_instance().is_initialized():
try:
@ -649,6 +688,15 @@ class MCPServerLocal(BaseModel):
def update(self, config: dict[str, Any]) -> "MCPServerLocal":
with self.__lock:
command = self.command
command_args: list[str] = []
args = list(self.args)
if "command" in config:
command, command_args = _split_stdio_command(config.get("command"))
args = [*command_args, *args]
if "args" in config:
args = [*command_args, *_normalize_stdio_args(config.get("args"))]
for key, value in config.items():
if key in [
"name",
@ -665,11 +713,15 @@ class MCPServerLocal(BaseModel):
"disabled_tools",
"scope",
]:
if key in ["command", "args"]:
continue
if key == "name":
value = normalize_name(value)
if key == "disabled_tools":
value = _normalize_disabled_tools(value)
setattr(self, key, value)
self.command = command
self.args = args
return self
async def initialize(self) -> "MCPServerLocal":

View file

@ -66,6 +66,9 @@
- `_is_streaming_http_type(server_type: str) -> bool`: Check if the server type is a streaming HTTP variant.
- `_split_qualified_tool_name(tool_name: str) -> tuple[str, str]`: Split `server.tool` names while preserving dots inside MCP tool names.
- `_normalize_disabled_tools(value: Any) -> list[str]`: Normalize the optional per-server disabled tool list.
- `_split_stdio_command(command: Any) -> tuple[str, list[str]]`: Split shell-style local MCP command lines into an executable plus leading arguments.
- `_split_stdio_arg_fragment(arg: str) -> list[str]`: Split collapsed option/value argument fragments while preserving obvious single values with spaces.
- `_normalize_stdio_args(value: Any) -> list[str]`: Normalize local MCP argument lists after manager/raw JSON parsing.
- `initialize_mcp(mcp_servers_config: str)`
- Notable constants/configuration names: `DEFAULT_MCP_SERVERS_CONFIG`, `MCP_MEDIA_TOKENS_ESTIMATE`, `MAX_MCP_RESOURCE_TEXT_CHARS`, `MCP_SESSION_CLEANUP_TIMEOUT_SECONDS`, `MCP_OPERATION_TIMEOUT_GRACE_SECONDS`, `T`.
@ -81,12 +84,13 @@
- MCP tool names are qualified as `server_name.tool_name`; server names are normalized without dots, and the tool portion may contain dots.
- Servers may define `disabled_tools` as a list of MCP tool names. Disabled tools are omitted from agent-facing prompts, status counts, `has_tool`, and calls, while detail views can still retrieve them through `get_all_tools()` with a `disabled` flag so users can re-enable them.
- Server-specific `init_timeout` and `tool_timeout` override global MCP client timeout settings for list-tools and call-tool operations.
- Local stdio server configs accept either strict MCP JSON (`command: "uvx", args: [...]`) or manager-style command lines (`command: "uvx package"`) and normalize them before spawning the process.
- MCP image and image-resource content is materialized to scoped artifact files and returned both as model-visible image attachments and as path metadata (`attachments`/`media_paths`) for downstream delivery.
- MCP config locks must not be held across awaited server initialization or tool-call operations. Slow or wedged MCP servers must not block status reads, prompt construction, unrelated MCP servers, or later tool calls through the shared config lock.
- MCP client session work runs inside disposable isolated `DeferredTask` workers with an outer timeout. Normal `AsyncExitStack` cleanup is also bounded; if cleanup or transport shutdown does not finish, the operation reports failure or warning while Agent Zero keeps control of the agent loop.
- Server status marks initialized server objects with cached initialization errors as disconnected, even if the config object exists.
- Observed side-effect areas: filesystem writes, network calls, WebSocket state, settings/state persistence, secret handling.
- Imported dependency areas include: `abc`, `anyio.streams.memory`, `asyncio`, `contextlib`, `datetime`, `helpers`, `helpers.defer`, `helpers.log`, `helpers.print_style`, `helpers.tool`, `httpx`, `json`, `mcp`, `mcp.client.sse`, `mcp.client.stdio`, `mcp.client.streamable_http`, `mcp.shared.message`.
- Imported dependency areas include: `abc`, `anyio.streams.memory`, `asyncio`, `contextlib`, `datetime`, `helpers`, `helpers.defer`, `helpers.log`, `helpers.print_style`, `helpers.tool`, `httpx`, `json`, `mcp`, `mcp.client.sse`, `mcp.client.stdio`, `mcp.client.streamable_http`, `mcp.shared.message`, `shlex`.
## Key Concepts

View file

@ -487,37 +487,36 @@ async def _run_direct_tool_job(parent_context_id: str, job: ParallelJob) -> str:
worker_agent = worker_context.agent0
worker_agent.loop_data = LoopData()
return await execute_tool_call(worker_agent, job.tool_name, job.tool_args)
return await execute_tool_call(
worker_agent,
job.tool_name,
job.tool_args,
log_item=job.log_item,
)
finally:
if worker_context:
await _remove_context(worker_context.id)
async def execute_tool_call(agent: "Agent", tool_name: str, tool_args: dict[str, Any]) -> str:
async def execute_tool_call(
agent: "Agent",
tool_name: str,
tool_args: dict[str, Any],
*,
log_item: "LogItem | None" = None,
) -> str:
if tool_name == "parallel":
raise ValueError("`parallel` cannot be nested inside a parallel worker.")
tool = None
try:
import helpers.mcp_handler as mcp_helper
tool = mcp_helper.MCPConfig.get_instance().get_tool(agent, tool_name)
except ImportError:
tool = None
except Exception as exc:
PrintStyle.warning(f"Failed to initialize MCP tool '{tool_name}' for parallel job: {exc}")
if not tool:
tool = agent.get_tool(
name=tool_name,
method=None,
args=tool_args,
message=json.dumps({"tool_name": tool_name, "tool_args": tool_args}),
loop_data=agent.loop_data,
)
tool = _resolve_parallel_tool(agent, tool_name, tool_args, strict=True)
if not tool:
raise ValueError(f"Tool '{tool_name}' not found or could not be initialized.")
original_get_log_object = None
if log_item is not None:
original_get_log_object = tool.get_log_object
tool.get_log_object = lambda: log_item
agent.loop_data.current_tool = tool
try:
await agent.handle_intervention()
@ -541,9 +540,63 @@ async def execute_tool_call(agent: "Agent", tool_name: str, tool_args: dict[str,
await agent.handle_intervention()
return response.message
finally:
if original_get_log_object is not None:
tool.get_log_object = original_get_log_object
agent.loop_data.current_tool = None
def _resolve_parallel_tool(
agent: "Agent",
tool_name: str,
tool_args: dict[str, Any],
*,
strict: bool = False,
):
message = json.dumps({"tool_name": tool_name, "tool_args": tool_args})
tool = None
try:
import helpers.mcp_handler as mcp_helper
tool = mcp_helper.MCPConfig.get_instance().get_tool(agent, tool_name)
except ImportError:
tool = None
except Exception as exc:
if strict:
raise
PrintStyle.warning(f"Failed to initialize MCP tool '{tool_name}' for parallel job: {exc}")
if not tool:
get_tool = getattr(agent, "get_tool", None)
if not callable(get_tool):
if strict:
raise ValueError(f"Tool '{tool_name}' not found or could not be initialized.")
return None
try:
tool = get_tool(
name=tool_name,
method=None,
args=tool_args,
message=message,
loop_data=getattr(agent, "loop_data", None),
)
except Exception as exc:
if strict:
raise
PrintStyle.warning(f"Failed to initialize tool '{tool_name}' for parallel job: {exc}")
tool = None
if not tool:
return None
try:
tool.args = dict(tool_args)
except Exception:
pass
return tool
async def _cancel_job(
job: ParallelJob,
*,
@ -600,6 +653,17 @@ def _log_parallel_child_started(agent: "Agent", job: ParallelJob) -> None:
)
return
tool = _resolve_parallel_tool(agent, job.tool_name, job.tool_args)
if tool is not None:
try:
job.log_item = tool.get_log_object()
if job.log_item is not None:
return
except Exception as exc:
PrintStyle.warning(
f"Failed to derive parallel child log for {job.tool_name}: {exc}"
)
heading = f"icon://construction {agent.agent_name}: Using tool '{job.tool_name}'"
job.log_item = agent.context.log.log(
type="tool",

View file

@ -31,6 +31,8 @@
- Direct tool background context cleanup removes both the in-memory context and any transient chat folder left on disk.
- Parent-visible child log items are created for each wrapped call so the WebUI can inspect concurrent children separately while the wrapper result remains model-history-only.
- Child tool logs mirror normal tool-call visible args; job ids remain available through wrapper results and prompt extras rather than visible process-step args.
- Wrapped tool child logs use each tool's native `get_log_object()` output when available, preserving special log rendering (for example: `code_execution_tool` uses `code_exe`, `wait` uses `progress`, MCP tools use `mcp`, and regular tools use `tool`).
- Direct parallel worker execution reuses the parent-visible child log item so tool `before_execution()` cannot create a second generic worker log or lose the native badge type.
- Job IDs are stable handles for later await, collect, or cancel operations.
- Prompt extras must stay bounded and expose only job IDs, tool names, status, and compact result/error summaries.

View file

@ -1,12 +1,15 @@
import json
import os
import tempfile
import uuid
from collections import OrderedDict
from datetime import datetime
from typing import Any
import uuid
from agent import Agent, AgentConfig, AgentContext, AgentContextType
from helpers import files, history
from helpers.litellm_transport import delete_stored_response_ids
from helpers.localization import Localization
import json
from initialize import initialize_agent
from helpers.log import Log, LogItem
@ -51,10 +54,9 @@ def save_tmp_chat(context: AgentContext):
return
path = _get_chat_file_path(context.id)
files.make_dirs(path)
data = _serialize_context(context)
js = _safe_json_serialize(data, ensure_ascii=False)
files.write_file(path, js)
_write_atomic(path, js)
mark_chat_saved(context)
@ -94,6 +96,28 @@ def _get_chat_file_path(ctxid: str):
return files.get_abs_path(CHATS_FOLDER, ctxid, CHAT_FILE_NAME)
def _write_atomic(path: str, content: str) -> None:
directory = os.path.dirname(path)
os.makedirs(directory, exist_ok=True)
fd, tmp_path = tempfile.mkstemp(
prefix=f".{os.path.basename(path)}.", suffix=".tmp", dir=directory
)
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
handle.write(content.encode("utf-8", "replace").decode("utf-8"))
handle.flush()
os.fsync(handle.fileno())
os.replace(tmp_path, path)
directory_fd = os.open(directory, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
if os.path.exists(tmp_path):
os.unlink(tmp_path)
def mark_chat_saved(context: AgentContext) -> None:
context.data[SAVED_CHAT_CONTEXT_DATA_KEY] = True

View file

@ -45,6 +45,7 @@
- Serialized chats store `agent_profile` both at the context level for the main chat and on each serialized agent so subordinate profiles survive server restart.
- Deserialization must rebuild each agent with its serialized profile when present, falling back to the context profile for older chat files.
- Chat loading skips directories that do not contain `chat.json`; malformed existing chat files still report load errors.
- Chat saves write and fsync a same-directory temporary file, atomically replace `chat.json`, and fsync the directory so an interrupted save cannot truncate the previous chat.
- Contexts are marked with private `SAVED_CHAT_CONTEXT_DATA_KEY` only after a successful save or load from disk so snapshot code can detect deleted chat files without hiding fresh unsaved chats.
## Key Concepts

View file

@ -1,5 +1,5 @@
import os
from typing import Literal, NotRequired, TypedDict, TYPE_CHECKING, cast
from typing import NotRequired, TypedDict, TYPE_CHECKING, cast
from helpers import files, dirty_json, persist_chat, file_tree, extension
from helpers.print_style import PrintStyle
@ -15,7 +15,13 @@ PROJECT_KNOWLEDGE_DIR = "knowledge"
PROJECT_SKILLS_DIR = "skills"
PROJECT_HEADER_FILE = "project.json"
PROJECT_MCP_SERVERS_FILE = "mcp_servers.json"
PROJECT_AGENTS_MD_FILES = ("AGENTS.md", "Agents.md", "agents.md")
PROJECT_AGENTS_MD_FILES = (
"AGENTS.override.md",
"AGENTS.Override.md",
"AGENTS.md",
"Agents.md",
"agents.md",
)
DEFAULT_MCP_SERVERS_CONFIG = '{\n "mcpServers": {}\n}'
CONTEXT_DATA_KEY_PROJECT = "project"
@ -466,9 +472,10 @@ def deactivate_project_in_chats(name: str):
def build_system_prompt_vars(name: str):
project_data = load_basic_project_data(name)
main_instructions = project_data.get("instructions", "") or ""
include_agents_md = project_data.get("include_agents_md", True)
instruction_files = get_project_instruction_files(
name,
include_agents_md=project_data.get("include_agents_md", True),
include_agents_md=include_agents_md,
)
instruction_parts = [
main_instructions,
@ -481,11 +488,75 @@ def build_system_prompt_vars(name: str):
"project_name": project_data.get("title", ""),
"project_description": project_data.get("description", ""),
"project_instructions": complete_instructions or "",
"include_agents_md": include_agents_md,
"project_path": files.normalize_a0_path(get_project_folder(name)),
"project_git_url": project_data.get("git_url", ""),
}
def get_agents_md_chain(root: str, target: str) -> list[tuple[str, str]]:
root_real = os.path.realpath(files.fix_dev_path(root))
target_real = os.path.realpath(files.fix_dev_path(target))
if os.path.isfile(target_real):
target_real = os.path.dirname(target_real)
if files.is_in_dir(target_real, root_real):
dirs = []
cursor = target_real
while True:
dirs.append(cursor)
if cursor == root_real:
break
parent = os.path.dirname(cursor)
if parent == cursor:
break
cursor = parent
dirs.reverse()
else:
dirs = [root_real]
chain = []
for dir_path in dirs:
for filename in PROJECT_AGENTS_MD_FILES:
matches = files.read_text_files_in_dir(dir_path, pattern=filename)
if filename not in matches:
continue
chain.append((files.get_abs_path(dir_path, filename), matches[filename]))
break
return chain
def build_agents_md_protocol(name: str, target: str | None = None) -> str:
project_folder = get_project_folder(name)
project_agents_md = get_project_agents_md_instruction_file(name)
project_agents_md_path = (
os.path.realpath(files.fix_dev_path(project_agents_md[0]))
if project_agents_md
else ""
)
entries = [
(path, content)
for path, content in get_agents_md_chain(
files.get_abs_path(""),
target or project_folder,
)
if os.path.realpath(path) != project_agents_md_path
]
if not entries:
return ""
instructions = []
for path, content in entries:
instructions.append(
f"### path: {files.normalize_a0_path(path)}\n\n{content.strip()}"
)
return files.read_prompt_file(
"agent.protocol.projects.agents_md.md",
_directories=["prompts"],
agents_md_instructions="\n\n".join(instructions),
).strip()
def get_additional_instructions_files(name: str):
instructions_folder = files.get_abs_path(
get_project_folder(name), PROJECT_META_DIR, PROJECT_INSTRUCTIONS_DIR
@ -522,11 +593,8 @@ def get_project_instruction_files(
def get_project_agents_md_instruction_file(name: str) -> tuple[str, str] | None:
project_folder = get_project_folder(name)
for filename in PROJECT_AGENTS_MD_FILES:
matches = files.read_text_files_in_dir(project_folder, pattern=filename)
if filename in matches:
path = files.get_abs_path(project_folder, filename)
return (files.normalize_a0_path(path), matches[filename])
for path, content in get_agents_md_chain(project_folder, project_folder):
return (files.normalize_a0_path(path), content)
return None

View file

@ -47,6 +47,8 @@
- `reactivate_project_in_chats(name: str)`
- `deactivate_project_in_chats(name: str)`
- `build_system_prompt_vars(name: str)`
- `get_agents_md_chain(root: str, target: str) -> list[tuple[str, str]]`
- `build_agents_md_protocol(name: str, target: str | None=...) -> str`
- `get_additional_instructions_files(name: str)`
- `get_project_instruction_files(name: str, include_agents_md: bool=...) -> list[tuple[str, str]]`
- `get_project_agents_md_instruction_file(name: str) -> tuple[str, str] | None`
@ -63,6 +65,8 @@
- Project extension data may add named top-level sections such as `llm`, but it must not overwrite core project fields owned by `EditProjectData`.
- Project extension save payloads exclude core project fields and transient inputs such as `git_token`; plugins needing core metadata should load it by project name.
- Project metadata setup creates and repairs `.a0proj/instructions`, `.a0proj/knowledge`, and `.a0proj/skills` so settings surfaces can open those folders consistently.
- AGENTS.md discovery is a linear root-to-target chain walk with `AGENTS.override.md` precedence; sibling directories are not scanned.
- Active-project AGENTS.md protocol guidance excludes the exact project root AGENTS.md because `build_system_prompt_vars(...)` already loads it into project instructions; prose for that protocol block lives in `prompts/agent.protocol.projects.agents_md.md`.
- Project MCP config uses the same JSON string shape as global MCP settings: an object with `mcpServers`.
- Project MCP load/save paths validate project names as simple folder basenames before touching `.a0proj/mcp_servers.json`.
- Observed side-effect areas: filesystem reads, filesystem writes, filesystem deletion, plugin state, settings/state persistence, secret handling.

View file

@ -56,8 +56,10 @@ class Settings(TypedDict):
agent_profile: str
agent_knowledge_subdir: str
max_consecutive_unusable_responses: int
timezone: str
time_format: str
ui_control_visibility: dict[str, dict[str, bool]]
workdir_path: str
workdir_show: bool
@ -163,6 +165,12 @@ API_KEY_PLACEHOLDER = "************"
TIMEZONE_AUTO = "auto"
TIME_FORMAT_12H = "12h"
TIME_FORMAT_24H = "24h"
UI_CONTROL_VISIBILITY_DEFAULTS = {
"projectSelector": {"mobile": True, "desktop": True},
"time": {"mobile": False, "desktop": True},
"connectionStatus": {"mobile": True, "desktop": True},
"rightCanvasRail": {"mobile": True, "desktop": True},
}
SETTINGS_FILE = files.get_abs_path("usr/settings.json")
_settings: Settings | None = None
@ -209,6 +217,22 @@ def _normalize_time_format(value: Any, default: str = TIME_FORMAT_12H) -> str:
return default if default in {TIME_FORMAT_12H, TIME_FORMAT_24H} else TIME_FORMAT_12H
def _normalize_ui_control_visibility(value: Any) -> dict[str, dict[str, bool]]:
submitted = value if isinstance(value, dict) else {}
normalized = {}
for control, devices in UI_CONTROL_VISIBILITY_DEFAULTS.items():
submitted_devices = submitted.get(control, {})
if not isinstance(submitted_devices, dict):
submitted_devices = {}
normalized[control] = {
device: submitted_devices.get(device)
if isinstance(submitted_devices.get(device), bool)
else default
for device, default in devices.items()
}
return normalized
def _resolve_runtime_timezone(setting_value: str, browser_timezone: str | None = None) -> str:
if setting_value == TIMEZONE_AUTO:
candidate = str(browser_timezone or "").strip()
@ -401,8 +425,12 @@ def normalize_settings(settings: Settings) -> Settings:
# mcp server token is set automatically
copy["mcp_server_token"] = create_auth_token()
copy["max_consecutive_unusable_responses"] = max(
1, copy["max_consecutive_unusable_responses"]
)
copy["timezone"] = _normalize_timezone_setting(copy.get("timezone"), default["timezone"])
copy["time_format"] = _normalize_time_format(copy.get("time_format"), default["time_format"])
copy["ui_control_visibility"] = _normalize_ui_control_visibility(copy.get("ui_control_visibility"))
return copy
@ -498,8 +526,14 @@ def get_default_settings() -> Settings:
root_password="",
agent_profile=get_default_value("agent_profile", "agent0"),
agent_knowledge_subdir=get_default_value("agent_knowledge_subdir", "custom"),
max_consecutive_unusable_responses=get_default_value(
"max_consecutive_unusable_responses", 2
),
timezone=_normalize_timezone_setting(get_default_value("timezone", TIMEZONE_AUTO)),
time_format=_normalize_time_format(get_default_value("time_format", TIME_FORMAT_12H)),
ui_control_visibility=_normalize_ui_control_visibility(
get_default_value("ui_control_visibility", UI_CONTROL_VISIBILITY_DEFAULTS)
),
workdir_path=get_default_value("workdir_path", files.get_abs_path_dockerized("usr/workdir")),
workdir_show=get_default_value("workdir_show", True),
workdir_max_depth=get_default_value("workdir_max_depth", 5),

View file

@ -25,6 +25,7 @@
- `_is_valid_timezone(value: str) -> bool`
- `_normalize_timezone_setting(value: Any, default: str=...) -> str`
- `_normalize_time_format(value: Any, default: str=...) -> str`
- `_normalize_ui_control_visibility(value: Any) -> dict[str, dict[str, bool]]`
- `_resolve_runtime_timezone(setting_value: str, browser_timezone: str | None=...) -> str`
- `_timezone_options() -> list[FieldOption]`
- `convert_out(settings: Settings) -> SettingsOutput`
@ -50,7 +51,7 @@
- `_dict_to_env(data_dict)`
- `set_root_password(password: str)`
- `get_runtime_config(set: Settings)`
- Notable constants/configuration names: `T`, `PASSWORD_PLACEHOLDER`, `API_KEY_PLACEHOLDER`, `TIMEZONE_AUTO`, `TIME_FORMAT_12H`, `TIME_FORMAT_24H`, `SETTINGS_FILE`.
- Notable constants/configuration names: `T`, `PASSWORD_PLACEHOLDER`, `API_KEY_PLACEHOLDER`, `TIMEZONE_AUTO`, `TIME_FORMAT_12H`, `TIME_FORMAT_24H`, `UI_CONTROL_VISIBILITY_DEFAULTS`, `SETTINGS_FILE`.
## Runtime Contracts
@ -64,6 +65,8 @@
- Important called helpers/classes observed in the source: `TypeVar`, `files.get_abs_path`, `dotenv.get_dotenv_value`, `opts.insert`, `str.strip`, `_is_valid_timezone`, `str.strip.lower`, `_normalize_timezone_setting`, `SettingsOutput`, `get_default_settings`, `_ensure_option_present`, `_resolve_runtime_timezone`, `get_default_secrets_manager`, `get_settings`, `normalize_settings`, `_load_sensitive_settings`, `settings.copy`, `_write_settings_file`, `reload_settings`, `set_settings`, `initialize_agent`.
- Applying settings refreshes active context configs while preserving each subordinate agent's own profile.
- Applying settings starts a deferred `MCPConfig.update(...)` with the current `mcp_servers` string when global MCP server settings change.
- `max_consecutive_unusable_responses` defaults to `2` and controls the cost circuit breaker for malformed or repeated main-model outputs.
- `ui_control_visibility` stores validated mobile and desktop visibility flags for the project selector, clock, connection status, and right canvas rail; missing or malformed values fall back per device.
- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
## Work Guidance

View file

@ -543,11 +543,15 @@ def search_skills(
if not q:
return []
raw_terms = [t for t in re.split(r"\s+", q) if t]
raw_terms = re.findall(r"[a-z0-9][a-z0-9_-]*", q)
terms = [
t for t in raw_terms
if len(t) >= 3 or any(ch.isdigit() for ch in t)
] or raw_terms
if len(t) >= 4 or any(ch.isdigit() for ch in t)
]
long_terms = [
t for t in raw_terms
if len(t) >= 6 or any(ch.isdigit() for ch in t)
]
candidates = list_skills(agent, include_hidden=include_hidden)
scored: List[Tuple[int, Skill]] = []
@ -568,14 +572,13 @@ def search_skills(
score += 4
if any(q in tag for tag in tags):
score += 3
if any(q in trigger for trigger in triggers):
if any(q in trigger or trigger in q for trigger in triggers):
score += 8
for term in terms:
if term in name:
score += 3
if term in desc:
score += 2
for term in long_terms:
if any(term in tag for tag in tags):
score += 1
if any(term in trigger for trigger in triggers):
@ -1132,7 +1135,6 @@ def hide_chat_skill(agent: Agent, entry: Any) -> list[ActiveSkillEntry]:
CONTEXT_DATA_NAME_CHAT_VISIBLE_SKILLS,
visible_entries,
)
unload_agent_skill(agent, normalized)
return get_hidden_skills(agent)
@ -1183,8 +1185,7 @@ def clear_chat_skill_overrides(agent: Agent) -> list[ActiveSkillEntry]:
def build_active_skills_prompt(agent: Agent | None) -> str:
items = _resolve_active_skill_entries(agent, get_active_skills(agent))
return "\n\n".join(item["content"] for item in items if item.get("content")).strip()
return ""
def _format_skill_prompt(skill: Skill) -> str:

View file

@ -56,6 +56,9 @@
- Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together.
- Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change.
- Loaded skill names are chat-wide context data under `CONTEXT_DATA_NAME_LOADED_SKILLS`; legacy agent-local `loaded_skills` lists are migrated into context data and cleared when read.
- Loaded skill bodies live in chat history; hiding a skill changes catalog visibility but does not remove the loaded-skill ledger.
- `build_active_skills_prompt()` returns empty because selected skills are loaded through history, not prompt protocol.
- `search_skills()` normalizes query words, scores normal terms against skill names, and scores only long terms against tags/triggers; descriptions match only full query phrases so generic prose does not produce irrelevant suggestions.
- Invalid `SKILL.md` frontmatter emits a once-per-path scan warning with the skipped skill path/name and a line number when the parser can identify one directly.
- Observed side-effect areas: filesystem reads, filesystem deletion, plugin state, settings/state persistence, context data, secret handling.
- Imported dependency areas include: `__future__`, `dataclasses`, `helpers`, `os`, `pathlib`, `re`, `typing`.

View file

@ -1,6 +1,7 @@
from dataclasses import dataclass, field
from datetime import timedelta
import asyncio
import json
import logging
import os
import secrets
@ -262,6 +263,13 @@ class UiRouteHandlers:
user_time_format_setting = str(settings_helper.get_settings().get("time_format", "12h"))
except Exception:
user_time_format_setting = "12h"
try:
user_ui_control_visibility = json.dumps(
settings_helper.get_settings()["ui_control_visibility"],
separators=(",", ":"),
)
except Exception:
user_ui_control_visibility = json.dumps(settings_helper.UI_CONTROL_VISIBILITY_DEFAULTS)
index = files.read_file("webui/index.html")
return files.replace_placeholders_text(
@ -273,6 +281,7 @@ class UiRouteHandlers:
logged_in=("true" if login.get_credentials_hash() else "false"),
user_timezone_setting=user_timezone_setting,
user_time_format_setting=user_time_format_setting,
user_ui_control_visibility=user_ui_control_visibility,
)
@requires_auth

View file

@ -41,6 +41,7 @@
## Key Concepts
- Important called helpers/classes observed in the source: `logging.getLogger.setLevel`, `Localization.get.apply_process_timezone`, `_positive_int_env`, `field`, `Flask`, `threading.RLock`, `socketio.AsyncServer`, `WsManager`, `set_shared_ws_manager`, `cls`, `server_runtime.refresh_runtime_settings`, `settings_helper.get_settings`, `settings_helper.set_runtime_settings_snapshot`, `self.ws_manager.set_server_restart_broadcast`, `UiRouteHandlers`, `self.webapp.add_url_rule`, `register_api_route`, `register_ws_namespace`, `files.read_file`, `render_template_string`, `session.pop`.
- `serve_index()` bootstraps the normalized UI control visibility map alongside timezone and time-format preferences so controls render correctly before Settings is opened.
- Keep request/response, tool, or helper semantics documented here at the same time as source changes.
## Work Guidance

View file

@ -70,6 +70,7 @@ Direct child DOX files:
| [_browser/AGENTS.md](_browser/AGENTS.md) | Playwright browser tool, helpers, viewer, and browser panel UI. |
| [_chat_branching/AGENTS.md](_chat_branching/AGENTS.md) | Chat branching from an existing message. |
| [_chat_compaction/AGENTS.md](_chat_compaction/AGENTS.md) | Full-chat compaction into a summary message. |
| [_commands/AGENTS.md](_commands/AGENTS.md) | Built-in slash command manager, command file discovery, and chat composer slash picker. |
| [_code_execution/AGENTS.md](_code_execution/AGENTS.md) | Terminal, Python, and Node.js execution tools and shell runtimes. |
| [_desktop/AGENTS.md](_desktop/AGENTS.md) | Linux desktop runtime, sessions, and desktop surface. |
| [_discovery/AGENTS.md](_discovery/AGENTS.md) | Welcome-screen plugin discovery cards and promotions. |
@ -77,6 +78,7 @@ Direct child DOX files:
| [_editor/AGENTS.md](_editor/AGENTS.md) | Native Markdown editor surface and sessions. |
| [_email_integration/AGENTS.md](_email_integration/AGENTS.md) | IMAP/Exchange polling and SMTP reply integration. |
| [_error_retry/AGENTS.md](_error_retry/AGENTS.md) | Critical exception retry lifecycle hooks. |
| [_goal/AGENTS.md](_goal/AGENTS.md) | Built-in chat goal strip, `/goal` slash command, and agent-facing goal tools. |
| [_infection_check/AGENTS.md](_infection_check/AGENTS.md) | Prompt-injection safety analysis before tool execution. |
| [_kokoro_tts/AGENTS.md](_kokoro_tts/AGENTS.md) | Kokoro text-to-speech integration. |
| [_memory/AGENTS.md](_memory/AGENTS.md) | Optional persistent recall plugin, knowledge import, tools, and dashboard; do not assume it is enabled outside this plugin. |
@ -84,6 +86,7 @@ Direct child DOX files:
| [_oauth/AGENTS.md](_oauth/AGENTS.md) | OAuth-backed model-provider connections and local proxy routes. |
| [_office/AGENTS.md](_office/AGENTS.md) | LibreOffice office artifacts and office canvas sessions. |
| [_onboarding/AGENTS.md](_onboarding/AGENTS.md) | First-time model onboarding wizard. |
| [_orchestrator/AGENTS.md](_orchestrator/AGENTS.md) | External terminal coding-agent orchestration skill, adapter status, and settings UI. |
| [_plugin_installer/AGENTS.md](_plugin_installer/AGENTS.md) | Plugin install and update flows from ZIP, Git, and Plugin Index. |
| [_plugin_scan/AGENTS.md](_plugin_scan/AGENTS.md) | LLM-guided security scanner for third-party plugins. |
| [_plugin_validator/AGENTS.md](_plugin_validator/AGENTS.md) | Plugin manifest, structure, convention, and security validator. |

View file

@ -24,6 +24,7 @@
- File operation results may arrive as chunked JSON/base64
`connector_file_op_result` frames; resolve the pending file operation only
after all chunks for the `op_id` are assembled.
- Host browser status metadata may advertise `available_browsers` entries with browser ids, labels, CDP endpoints, status, and enabled state; keep older CLI payloads without those fields compatible.
## Work Guidance

View file

@ -24,6 +24,11 @@ def _normalize_requested_backend(value: object) -> str:
return ""
def _normalize_host_browser_selection(value: object) -> str:
raw = _string(value).lower().replace(" ", "_")
return "".join(ch for ch in raw if ch.isalnum() or ch in {"_", "-", ":", ".", "/"})[:200]
def _normalize_profile_mode(value: object) -> str:
normalized = _string(value).lower().replace("-", "_").replace(" ", "_")
if normalized in {"agent", "clean", "clean_agent", "a0", "dedicated"}:
@ -68,6 +73,10 @@ class BrowserRuntime(connector_base.ProtectedConnectorApiHandler):
mimetype="application/json",
)
settings["runtime_backend"] = runtime_backend
if "host_browser_selection" in input or "browser_selection" in input:
settings["host_browser_selection"] = _normalize_host_browser_selection(
input.get("host_browser_selection", input.get("browser_selection"))
)
if "host_browser_profile_mode" in input or "profile_mode" in input:
profile_mode = _normalize_profile_mode(
input.get("host_browser_profile_mode", input.get("profile_mode"))
@ -86,11 +95,13 @@ class BrowserRuntime(connector_base.ProtectedConnectorApiHandler):
runtime_backend = settings.get("runtime_backend") or "container"
profile_mode = _normalize_profile_mode(settings.get("host_browser_profile_mode")) or "existing"
browser_selection = _normalize_host_browser_selection(settings.get("host_browser_selection"))
return {
"ok": True,
"runtime_backend": runtime_backend,
"host_browser_profile_mode": profile_mode,
"host_browser_selection": browser_selection,
"label": _runtime_label(runtime_backend),
"project_name": project_name,
"agent_profile": "",

View file

@ -80,6 +80,9 @@ class HostBrowserMetadata:
profile_label: str
profile_path: str
cdp_endpoint: str
browser_id: str
browser_label: str
available_browsers: tuple[dict[str, Any], ...]
content_helper_sha256: str
features: tuple[str, ...]
support_reason: str
@ -416,6 +419,9 @@ def store_sid_host_browser_metadata(sid: str, payload: dict[str, Any]) -> HostBr
profile_label=str(payload.get("profile_label", "") or "").strip(),
profile_path=str(payload.get("profile_path", "") or "").strip(),
cdp_endpoint=str(payload.get("cdp_endpoint", "") or "").strip(),
browser_id=str(payload.get("browser_id", payload.get("browser_selection", "")) or "").strip(),
browser_label=str(payload.get("browser_label", "") or "").strip(),
available_browsers=_normalize_available_host_browsers(payload.get("available_browsers")),
content_helper_sha256=str(payload.get("content_helper_sha256", "") or "").strip().lower(),
features=features,
support_reason=support_reason,
@ -445,6 +451,32 @@ def _host_browser_can_prepare(
)
def _normalize_available_host_browsers(value: Any) -> tuple[dict[str, Any], ...]:
if not isinstance(value, (list, tuple)):
return ()
browsers: list[dict[str, Any]] = []
for item in value:
if not isinstance(item, dict):
continue
browser_id = str(item.get("id", item.get("browser_id", item.get("selection", ""))) or "").strip()
family = str(item.get("family", item.get("browser_family", "")) or "").strip()
label = str(item.get("label", item.get("name", "")) or "").strip()
cdp_endpoint = str(item.get("cdp_endpoint", "") or "").strip()
status = str(item.get("status", "") or "").strip()
enabled = bool(item.get("enabled", True))
if not any((browser_id, family, label, cdp_endpoint)):
continue
browsers.append({
"id": browser_id or family or cdp_endpoint,
"family": family,
"label": label or family or browser_id or cdp_endpoint,
"cdp_endpoint": cdp_endpoint,
"status": status,
"enabled": enabled,
})
return tuple(browsers)
def clear_sid_host_browser_metadata(sid: str) -> None:
with _state_lock:
_sid_host_browser_metadata.pop(sid, None)
@ -464,6 +496,9 @@ def host_browser_metadata_for_sid(sid: str) -> dict[str, Any] | None:
"profile_label": metadata.profile_label,
"profile_path": metadata.profile_path,
"cdp_endpoint": metadata.cdp_endpoint,
"browser_id": metadata.browser_id,
"browser_label": metadata.browser_label,
"available_browsers": copy.deepcopy(list(metadata.available_browsers)),
"content_helper_sha256": metadata.content_helper_sha256,
"features": list(metadata.features),
"support_reason": metadata.support_reason,
@ -529,6 +564,10 @@ def all_host_browser_metadata() -> list[dict[str, Any]]:
"browser_family": metadata.browser_family,
"profile_label": metadata.profile_label,
"profile_path": metadata.profile_path,
"cdp_endpoint": metadata.cdp_endpoint,
"browser_id": metadata.browser_id,
"browser_label": metadata.browser_label,
"available_browsers": copy.deepcopy(list(metadata.available_browsers)),
"content_helper_sha256": metadata.content_helper_sha256,
"features": list(metadata.features),
"support_reason": metadata.support_reason,

View file

@ -19,7 +19,15 @@
- Preserve Playwright lifecycle cleanup and WebSocket viewer compatibility across regular host browsers and Electron WebContentsView embedding.
- Keep the WebUI Browser inside its own modal/canvas affordance; do not replace it with page-level navigation.
- Default the visible WebUI Browser to live CDP screencast for responsiveness. Keep lightweight CDP/DOM state snapshots as the fallback transport.
- Paint live screencast frames through the Browser panel canvas/ImageBitmap path when available; keep the `<img>`/data URL path for snapshots and fallback rendering.
- Push internal screencast frames from the runtime to the WebSocket consumer after subscription; keep `read/pop_screencast_frame` as fallback/tooling APIs, not the WebUI hot path.
- Keep Browser viewer frame transport capability-negotiated: updated clients may request binary/slim screencast frames, while older clients must keep the base64/full-metadata fallback. Do not let the WebUI advertise binary frames unless its Socket.IO client reconstructs attachments as real `Blob`, `ArrayBuffer`, or typed-array values.
- Keep WebUI Browser tabs scoped to the active chat context by default; aggregate tabs from other AgentContext runtimes only when the Browser settings tab scope is `shared`.
- Keep narrow WebUI Browser controls usable by grouping navigation with Annotate/settings above a full-width address bar.
- For Bring Your Own Browser with an existing host profile, `host_browser_selection` may target automatic CLI selection, a browser family/id, or an explicit CDP endpoint and must be forwarded to the connector runtime as `browser_selection`.
- Browser Settings must refresh connected A0 CLI host-browser inventory while the settings view is open so newly authorized endpoints appear without saving or reopening.
- Browser Settings keeps the Host browser dropdown focused on automatic selection, advertised debug endpoints, and a validated Custom endpoint field instead of listing every installed local profile.
- Browser URL-intent handling must only claim web URL schemes and leave custom Agent Zero schemes to their owning surfaces.
- Prefer DOM/CDP browser actions with refs, selectors, frame-chain refs, and screenshots over viewport coordinate input. Coordinates remain a visual fallback.
- Do not hardcode user-specific browser paths or secrets.
@ -34,6 +42,7 @@
## Verification
- Smoke-test browser launch, navigation, DOM capture, and WebUI viewer after runtime changes.
- For viewer render-path changes, verify the live Browser panel paints a screencast frame on canvas with `frameSrc` empty and snapshots still falling back to the image path.
- Run browser prompt/skill regression tests after changing browser prompt or Browser plugin skills.
## Child DOX Index

View file

@ -1,6 +1,7 @@
from __future__ import annotations
import asyncio
import base64
import contextlib
import time
from typing import Any, ClassVar
@ -8,14 +9,20 @@ from typing import Any, ClassVar
from agent import AgentContext
from helpers.ws import WsHandler
from helpers.ws_manager import WsResult
from plugins._browser.helpers.config import (
DEFAULT_BROWSER_TAB_SCOPE,
TAB_SCOPE_KEY,
get_browser_config,
)
from plugins._browser.helpers.runtime import get_runtime, list_runtime_sessions
FRAME_IDLE_POLL_SECONDS = 0.05
FRAME_READ_TIMEOUT_SECONDS = 0.5
FRAME_RETRY_DELAY_SECONDS = 0.5
FRAME_STATE_REFRESH_SECONDS = 0.75
SNAPSHOT_STATE_POLL_SECONDS = 0.75
SCREENCAST_QUALITY = 92
SCREENCAST_STREAM_QUALITY = 80
SCREENSHOT_QUALITY = 92
VIEWER_TRANSPORT_SCREENCAST = "screencast"
VIEWER_TRANSPORT_SNAPSHOT = "snapshot"
VIEWER_TRANSPORTS = {VIEWER_TRANSPORT_SCREENCAST, VIEWER_TRANSPORT_SNAPSHOT}
@ -97,24 +104,40 @@ class WsBrowser(WsHandler):
existing.cancel()
viewer_id = str(data.get("viewer_id") or "")
viewer_transport = self._viewer_transport(data)
binary_frames = self._bool(data.get("binary_frames", data.get("binaryFrames")))
slim_frames = self._bool(data.get("slim_frames", data.get("slimFrames", binary_frames)))
capture_scale = self._capture_scale_from_data(data)
snapshot = None
if runtime:
if viewer_transport == VIEWER_TRANSPORT_SCREENCAST:
stream_task = self._stream_frames(sid, context_id, active_id, viewer_id)
stream_task = self._stream_frames(
sid,
context_id,
active_id,
viewer_id,
binary_frames=binary_frames,
slim_frames=slim_frames,
capture_scale=capture_scale,
)
else:
stream_task = self._stream_state(sid, context_id, active_id, viewer_id)
self._streams[stream_key] = asyncio.create_task(stream_task)
snapshot = await self._snapshot_for_browser(runtime, active_id)
browsers, all_browsers, tab_scope = await self._tabs_for_scope(context_id, browsers)
return {
"context_id": context_id,
"active_browser_context_id": context_id,
"active_browser_id": active_id,
"snapshot": snapshot,
"browsers": await self._all_browser_tabs(),
"all_browsers": True,
"browsers": browsers,
"all_browsers": all_browsers,
"tab_scope": tab_scope,
"viewer_id": viewer_id,
"viewer_transport": viewer_transport,
"binary_frames": binary_frames,
"slim_frames": slim_frames,
}
def _unsubscribe(self, data: dict[str, Any], sid: str) -> dict[str, Any] | WsResult:
@ -127,10 +150,23 @@ class WsBrowser(WsHandler):
return {"context_id": context_id, "unsubscribed": True}
async def _sessions(self, data: dict[str, Any]) -> dict[str, Any]:
context_id = self._context_id(data)
tab_scope = self._tab_scope()
if tab_scope == "shared":
return {
"context_id": context_id,
"browsers": await self._all_browser_tabs(),
"all_browsers": True,
"tab_scope": tab_scope,
}
runtime = await get_runtime(context_id, create=False) if context_id else None
listing = await runtime.call("list") if runtime else {}
return {
"context_id": self._context_id(data),
"browsers": await self._all_browser_tabs(),
"all_browsers": True,
"context_id": context_id,
"browsers": listing.get("browsers") or [],
"all_browsers": False,
"tab_scope": tab_scope,
}
async def _snapshot(self, data: dict[str, Any]) -> dict[str, Any] | WsResult:
@ -142,13 +178,15 @@ class WsBrowser(WsHandler):
runtime = await get_runtime(context_id, create=False)
if not runtime:
browsers, all_browsers, tab_scope = await self._tabs_for_scope(context_id, [])
return {
"context_id": context_id,
"active_browser_context_id": context_id,
"active_browser_id": None,
"snapshot": None,
"browsers": await self._all_browser_tabs(),
"all_browsers": True,
"browsers": browsers,
"all_browsers": all_browsers,
"tab_scope": tab_scope,
}
listing = await runtime.call("list")
@ -157,9 +195,9 @@ class WsBrowser(WsHandler):
snapshot = None
if active_id:
try:
quality = int(data.get("quality") or SCREENCAST_QUALITY)
quality = int(data.get("quality") or SCREENSHOT_QUALITY)
except (TypeError, ValueError):
quality = SCREENCAST_QUALITY
quality = SCREENSHOT_QUALITY
with contextlib.suppress(Exception):
snapshot = await runtime.call(
"screenshot",
@ -167,13 +205,16 @@ class WsBrowser(WsHandler):
quality=quality,
)
browsers, all_browsers, tab_scope = await self._tabs_for_scope(context_id, browsers)
return {
"context_id": context_id,
"active_browser_context_id": context_id,
"active_browser_id": active_id,
"snapshot": snapshot,
"browsers": await self._all_browser_tabs(),
"all_browsers": True,
"browsers": browsers,
"all_browsers": all_browsers,
"tab_scope": tab_scope,
}
async def _command(self, data: dict[str, Any], sid: str) -> dict[str, Any] | WsResult:
@ -208,7 +249,10 @@ class WsBrowser(WsHandler):
listing = await runtime.call("list")
last_interacted_browser_id = listing.get("last_interacted_browser_id")
snapshot = await self._snapshot_for_result(runtime, result)
all_browsers = await self._all_browser_tabs()
browsers, all_browsers, tab_scope = await self._tabs_for_scope(
context_id,
listing.get("browsers") or [],
)
await self.emit_to(
sid,
"browser_viewer_state",
@ -220,8 +264,9 @@ class WsBrowser(WsHandler):
"browser_id": browser_id,
"result": result,
"snapshot": snapshot,
"browsers": all_browsers,
"all_browsers": True,
"browsers": browsers,
"all_browsers": all_browsers,
"tab_scope": tab_scope,
"last_interacted_browser_id": last_interacted_browser_id,
"viewer_transport": self._viewer_transport(data),
},
@ -230,8 +275,9 @@ class WsBrowser(WsHandler):
return {
"result": result,
"snapshot": snapshot,
"browsers": all_browsers,
"all_browsers": True,
"browsers": browsers,
"all_browsers": all_browsers,
"tab_scope": tab_scope,
"active_browser_context_id": context_id,
"last_interacted_browser_id": last_interacted_browser_id,
"command": command,
@ -347,7 +393,7 @@ class WsBrowser(WsHandler):
if not browser_id:
return None
with contextlib.suppress(Exception):
return await runtime.call("screenshot", browser_id, quality=SCREENCAST_QUALITY)
return await runtime.call("screenshot", browser_id, quality=SCREENSHOT_QUALITY)
return None
async def _snapshot_for_browser(
@ -358,9 +404,27 @@ class WsBrowser(WsHandler):
if not browser_id:
return None
with contextlib.suppress(Exception):
return await runtime.call("screenshot", browser_id, quality=SCREENCAST_QUALITY)
return await runtime.call("screenshot", browser_id, quality=SCREENSHOT_QUALITY)
return None
@staticmethod
def _tab_scope() -> str:
scope = str(
(get_browser_config() or {}).get(TAB_SCOPE_KEY, DEFAULT_BROWSER_TAB_SCOPE)
or DEFAULT_BROWSER_TAB_SCOPE
).strip().lower().replace("-", "_")
return "shared" if scope == "shared" else DEFAULT_BROWSER_TAB_SCOPE
async def _tabs_for_scope(
self,
context_id: str,
browsers: list[dict[str, Any]] | None,
) -> tuple[list[dict[str, Any]], bool, str]:
tab_scope = self._tab_scope()
if tab_scope == "shared":
return await self._all_browser_tabs(), True, tab_scope
return browsers or [], False, tab_scope
async def _all_browser_tabs(self) -> list[dict[str, Any]]:
browsers: list[dict[str, Any]] = []
for session in await list_runtime_sessions():
@ -377,6 +441,10 @@ class WsBrowser(WsHandler):
context_id: str,
browser_id: int | str | None,
viewer_id: str = "",
*,
binary_frames: bool = False,
slim_frames: bool = False,
capture_scale: float = 1.0,
) -> None:
runtime = None
stream_id = None
@ -397,12 +465,14 @@ class WsBrowser(WsHandler):
browsers = listing.get("browsers") or []
active_id = self._active_browser_id(listing, browser_id)
if not active_id:
await self._emit_empty_frame(
await self._emit_viewer_state(
sid,
context_id,
active_id,
browsers=browsers,
viewer_id=viewer_id,
frame_source=VIEWER_TRANSPORT_SCREENCAST,
state=None,
viewer_transport=VIEWER_TRANSPORT_SCREENCAST,
)
await asyncio.sleep(FRAME_RETRY_DELAY_SECONDS)
continue
@ -410,30 +480,60 @@ class WsBrowser(WsHandler):
screencast = await runtime.call(
"start_screencast",
active_id,
quality=SCREENCAST_QUALITY,
quality=SCREENCAST_STREAM_QUALITY,
every_nth_frame=1,
capture_scale=capture_scale,
)
stream_id = screencast["stream_id"]
active_id = screencast["browser_id"]
state = screencast.get("state")
await self.emit_to(
await self._emit_viewer_state(
sid,
"browser_viewer_frame",
{
"context_id": context_id,
"viewer_id": viewer_id,
"browser_id": active_id,
"browsers": browsers,
"image": "",
"mime": "",
"state": state,
"frame_source": "state",
"viewer_transport": VIEWER_TRANSPORT_SCREENCAST,
},
context_id,
active_id,
browsers=browsers,
viewer_id=viewer_id,
state=state,
viewer_transport=VIEWER_TRANSPORT_SCREENCAST,
)
last_state_refresh = 0.0
last_state_signature = self._state_signature(active_id, browsers)
frame_sequence = 0
server_loop = asyncio.get_running_loop()
stop_event = asyncio.Event()
async def emit_frame(frame: dict[str, Any]) -> None:
nonlocal frame_sequence
try:
frame_sequence += 1
payload = self._frame_payload(
frame,
context_id=context_id,
viewer_id=viewer_id,
browser_id=active_id,
sequence=frame_sequence,
binary_frames=binary_frames,
)
if not slim_frames:
payload["browsers"] = browsers
payload["state"] = state
await self._emit_to_connected_viewer(sid, "browser_viewer_frame", payload)
except BaseException:
stop_event.set()
raise
def frame_consumer(frame: dict[str, Any]):
return asyncio.run_coroutine_threadsafe(emit_frame(frame), server_loop)
def stop_consumer() -> None:
server_loop.call_soon_threadsafe(stop_event.set)
await runtime.call("attach_screencast_consumer", stream_id, frame_consumer, stop_consumer)
while True:
if stop_event.is_set():
break
now = time.monotonic()
if now - last_state_refresh >= FRAME_STATE_REFRESH_SECONDS:
listing = await runtime.call("list")
@ -442,24 +542,25 @@ class WsBrowser(WsHandler):
if str(active_id) not in browser_ids:
break
state = self._state_for_browser(browsers, active_id, state)
state_signature = self._state_signature(active_id, browsers)
if state_signature != last_state_signature:
await self._emit_viewer_state(
sid,
context_id,
active_id,
browsers=browsers,
viewer_id=viewer_id,
state=state,
viewer_transport=VIEWER_TRANSPORT_SCREENCAST,
)
last_state_signature = state_signature
last_state_refresh = now
try:
frame = await runtime.call("pop_screencast_frame", stream_id)
except KeyError:
await asyncio.wait_for(stop_event.wait(), timeout=FRAME_READ_TIMEOUT_SECONDS)
break
if frame is None:
await asyncio.sleep(FRAME_IDLE_POLL_SECONDS)
except TimeoutError:
continue
frame["context_id"] = context_id
frame["viewer_id"] = viewer_id
frame["browser_id"] = active_id
frame["browsers"] = browsers
frame["state"] = state
frame["frame_source"] = VIEWER_TRANSPORT_SCREENCAST
frame["viewer_transport"] = VIEWER_TRANSPORT_SCREENCAST
await self.emit_to(sid, "browser_viewer_frame", frame)
except asyncio.CancelledError:
raise
except Exception:
@ -512,20 +613,14 @@ class WsBrowser(WsHandler):
),
)
if signature != last_signature:
await self.emit_to(
await self._emit_viewer_state(
sid,
"browser_viewer_frame",
{
"context_id": context_id,
"viewer_id": viewer_id,
"browser_id": active_id,
"browsers": browsers,
"image": "",
"mime": "",
"state": state,
"frame_source": VIEWER_TRANSPORT_SNAPSHOT,
"viewer_transport": VIEWER_TRANSPORT_SNAPSHOT,
},
context_id,
active_id,
browsers=browsers,
viewer_id=viewer_id,
state=state,
viewer_transport=VIEWER_TRANSPORT_SNAPSHOT,
)
last_signature = signature
await asyncio.sleep(SNAPSHOT_STATE_POLL_SECONDS)
@ -564,6 +659,107 @@ class WsBrowser(WsHandler):
return browser
return current_state
@staticmethod
def _state_signature(
active_id: int | str | None,
browsers: list[dict[str, Any]],
) -> tuple[str, tuple[tuple[str, str, str, str, bool], ...]]:
return (
str(active_id or ""),
tuple(
(
str(browser.get("context_id") or ""),
str(browser.get("id") or ""),
str(browser.get("currentUrl") or ""),
str(browser.get("title") or ""),
bool(browser.get("loading")),
)
for browser in browsers
),
)
@staticmethod
def _frame_payload(
frame: dict[str, Any],
*,
context_id: str,
viewer_id: str,
browser_id: int | str,
sequence: int,
binary_frames: bool,
) -> dict[str, Any]:
image = str(frame.get("image") or "")
payload: dict[str, Any] = {
"context_id": context_id,
"viewer_id": viewer_id,
"browser_id": browser_id,
"seq": sequence,
"mime": frame.get("mime") or "image/jpeg",
"frame_source": VIEWER_TRANSPORT_SCREENCAST,
"viewer_transport": VIEWER_TRANSPORT_SCREENCAST,
}
dimensions = WsBrowser._frame_dimensions(frame.get("metadata"))
if dimensions:
payload.update(dimensions)
if binary_frames:
try:
payload["image"] = base64.b64decode(image, validate=False)
payload["encoding"] = "binary"
except Exception:
payload["image"] = image
payload["encoding"] = "base64"
else:
payload["image"] = image
payload["encoding"] = "base64"
return payload
@staticmethod
def _frame_dimensions(metadata: Any) -> dict[str, int]:
if not isinstance(metadata, dict):
return {}
for width_key, height_key in (
("expectedWidth", "expectedHeight"),
("deviceWidth", "deviceHeight"),
("jpegWidth", "jpegHeight"),
):
try:
width = int(metadata.get(width_key) or 0)
height = int(metadata.get(height_key) or 0)
except (TypeError, ValueError):
continue
if width > 0 and height > 0:
return {"width": width, "height": height}
return {}
async def _emit_viewer_state(
self,
sid: str,
context_id: str,
browser_id: int | str | None,
*,
browsers: list[dict[str, Any]] | None = None,
viewer_id: str = "",
state: dict[str, Any] | None = None,
viewer_transport: str = VIEWER_TRANSPORT_SNAPSHOT,
) -> None:
browsers, all_browsers, tab_scope = await self._tabs_for_scope(context_id, browsers or [])
await self._emit_to_connected_viewer(
sid,
"browser_viewer_state",
{
"context_id": context_id,
"active_browser_context_id": context_id,
"viewer_id": viewer_id,
"browser_id": browser_id,
"active_browser_id": browser_id,
"browsers": browsers or [],
"state": state,
"all_browsers": all_browsers,
"tab_scope": tab_scope,
"viewer_transport": viewer_transport,
},
)
async def _emit_empty_frame(
self,
sid: str,
@ -573,7 +769,8 @@ class WsBrowser(WsHandler):
viewer_id: str = "",
frame_source: str = "",
) -> None:
await self.emit_to(
browsers, all_browsers, tab_scope = await self._tabs_for_scope(context_id, browsers or [])
await self._emit_to_connected_viewer(
sid,
"browser_viewer_frame",
{
@ -581,6 +778,8 @@ class WsBrowser(WsHandler):
"viewer_id": viewer_id,
"browser_id": None,
"browsers": browsers or [],
"all_browsers": all_browsers,
"tab_scope": tab_scope,
"image": "",
"mime": "",
"state": None,
@ -589,6 +788,20 @@ class WsBrowser(WsHandler):
},
)
async def _emit_to_connected_viewer(
self,
sid: str,
event: str,
data: dict[str, Any],
) -> None:
manager = getattr(self, "_manager", None)
if manager is not None:
with manager.lock:
connected = (getattr(self, "namespace", "/ws"), sid) in manager.connections
if not connected:
raise asyncio.CancelledError()
await self.emit_to(sid, event, data)
@staticmethod
def _viewer_transport(data: dict[str, Any]) -> str:
raw = (
@ -620,6 +833,20 @@ class WsBrowser(WsHandler):
"height": max(200, min(4096, height)),
}
@staticmethod
def _capture_scale_from_data(data: dict[str, Any]) -> float:
try:
scale = float(
data.get("device_pixel_ratio")
or data.get("devicePixelRatio")
or data.get("pixel_ratio")
or data.get("pixelRatio")
or 1
)
except (TypeError, ValueError):
return 1.0
return max(1.0, min(2.0, scale))
@staticmethod
def _context_id(data: dict[str, Any]) -> str:
return str(data.get("context_id") or data.get("context") or "").strip()

View file

@ -8,6 +8,11 @@ default_homepage: "about:blank"
# When the Browser surface is already open, keep it synced to agent Browser tool results.
autofocus_active_page: true
# Browser tab visibility in the WebUI:
# - per_context: each chat shows only its own Browser tabs.
# - shared: show Browser tabs from all active chats.
browser_tab_scope: "per_context"
# Maximum number of Browser tabs/pages a single chat context may keep open.
# Raise this only for deliberate parallel browsing workflows.
max_open_tabs: 32
@ -28,6 +33,11 @@ host_browser_privacy_policy: "allow"
# - agent: use a clean A0-controlled browser profile on the host.
host_browser_profile_mode: "existing"
# Optional host browser target when using an existing browser.
# Empty means A0 CLI chooses the first supported/active browser.
# Values may be browser family ids (chrome, edge, chromium) or CLI-advertised ids/endpoints.
host_browser_selection: ""
# Optional _model_config preset used by Browser-owned model helpers.
# Empty uses the effective Main Model.
model_preset: ""

View file

@ -11,13 +11,17 @@ PLUGIN_NAME = "_browser"
MODEL_PRESET_KEY = "model_preset"
DEFAULT_HOMEPAGE_KEY = "default_homepage"
AUTOFOCUS_ACTIVE_PAGE_KEY = "autofocus_active_page"
TAB_SCOPE_KEY = "browser_tab_scope"
MAX_OPEN_TABS_KEY = "max_open_tabs"
RUNTIME_BACKEND_KEY = "runtime_backend"
HOST_BROWSER_PRIVACY_POLICY_KEY = "host_browser_privacy_policy"
HOST_BROWSER_PROFILE_MODE_KEY = "host_browser_profile_mode"
HOST_BROWSER_SELECTION_KEY = "host_browser_selection"
RUNTIME_BACKENDS = {"container", "host_required"}
BROWSER_TAB_SCOPES = {"per_context", "shared"}
HOST_BROWSER_PRIVACY_POLICIES = {"enforce_local", "warn", "allow"}
HOST_BROWSER_PROFILE_MODES = {"existing", "agent"}
DEFAULT_BROWSER_TAB_SCOPE = "per_context"
DEFAULT_MAX_OPEN_TABS = 32
MIN_MAX_OPEN_TABS = 1
HARD_MAX_OPEN_TABS = 50
@ -55,6 +59,15 @@ def _normalize_model_preset(value: Any) -> str:
return str(value or "").strip()
def _normalize_host_browser_selection(value: Any) -> str:
raw = str(value or "").strip()
if not raw:
return ""
normalized = raw.lower().replace(" ", "_")
# Keep explicit CLI ids/ports/endpoints usable while avoiding control characters.
return "".join(ch for ch in normalized if ch.isalnum() or ch in {"_", "-", ":", ".", "/"})[:200]
def _normalize_default_homepage(value: Any) -> str:
homepage = str(value or "").strip()
return homepage or "about:blank"
@ -117,6 +130,11 @@ def normalize_browser_config(settings: dict[str, Any] | None) -> dict[str, Any]:
raw.get(AUTOFOCUS_ACTIVE_PAGE_KEY, True),
default=True,
),
TAB_SCOPE_KEY: _normalize_choice(
raw.get(TAB_SCOPE_KEY, DEFAULT_BROWSER_TAB_SCOPE),
allowed=BROWSER_TAB_SCOPES,
default=DEFAULT_BROWSER_TAB_SCOPE,
),
MAX_OPEN_TABS_KEY: _normalize_int(
raw.get(MAX_OPEN_TABS_KEY, DEFAULT_MAX_OPEN_TABS),
default=DEFAULT_MAX_OPEN_TABS,
@ -136,6 +154,9 @@ def normalize_browser_config(settings: dict[str, Any] | None) -> dict[str, Any]:
allowed=HOST_BROWSER_PROFILE_MODES,
default="existing",
),
HOST_BROWSER_SELECTION_KEY: _normalize_host_browser_selection(
raw.get(HOST_BROWSER_SELECTION_KEY, raw.get("host_browser_choice", ""))
),
MODEL_PRESET_KEY: _normalize_model_preset(raw.get(MODEL_PRESET_KEY, "")),
}

View file

@ -57,6 +57,11 @@ HOST_BROWSER_PROFILE_MODE_KEY = getattr(
"HOST_BROWSER_PROFILE_MODE_KEY",
"host_browser_profile_mode",
)
HOST_BROWSER_SELECTION_KEY = getattr(
browser_config,
"HOST_BROWSER_SELECTION_KEY",
"host_browser_selection",
)
get_browser_config = browser_config.get_browser_config
_LOCAL_PROVIDERS = {"ollama", "lm_studio", "llama_cpp", "omlx", "vllm"}
_LOCAL_HOSTS = {"localhost", "127.0.0.1", "::1", "host.docker.internal"}
@ -78,7 +83,8 @@ _REQUIRED_API_NAMES_RE = re.compile(
re.S,
)
_HOST_BROWSER_REMOTE_DEBUGGING_HELP = (
'For an already-open Chrome-family browser, open `chrome://inspect/#remote-debugging`, '
"For an already-open Chromium-family browser, open its inspect page, such as "
"`chrome://inspect/#remote-debugging` or `opera://inspect/#remote-debugging`, "
'enable "Allow remote debugging for this browser instance", run `/browser host on`, '
"and retry."
)
@ -123,6 +129,7 @@ class ConnectorBrowserRuntime:
"context_id": self.context_id,
"action": action,
"profile_mode": self._host_browser_profile_mode(),
"browser_selection": self._host_browser_selection(),
}
if action == "open":
@ -283,6 +290,7 @@ class ConnectorBrowserRuntime:
"context_id": self.context_id,
"action": "ensure",
"profile_mode": self._host_browser_profile_mode(),
"browser_selection": self._host_browser_selection(),
},
),
)
@ -295,6 +303,10 @@ class ConnectorBrowserRuntime:
mode = str(config.get(HOST_BROWSER_PROFILE_MODE_KEY) or "existing").strip().lower()
return "agent" if mode == "agent" else "existing"
def _host_browser_selection(self) -> str:
config = get_browser_config(self.agent)
return str(config.get(HOST_BROWSER_SELECTION_KEY) or "").strip()
def _with_content_helper(self, sid: str, payload: dict[str, Any]) -> dict[str, Any]:
return self._with_browser_helpers(sid, payload)
@ -508,7 +520,10 @@ class ConnectorBrowserRuntime:
if not message:
message = "Host browser operation failed"
normalized = message.lower()
if "chrome://inspect/#remote-debugging" in normalized:
if (
"chrome://inspect/#remote-debugging" in normalized
or "opera://inspect/#remote-debugging" in normalized
):
return _append_docker_browser_recovery(message)
if any(token in normalized for token in _REMOTE_DEBUGGING_ERROR_TOKENS):
return _append_docker_browser_recovery(

View file

@ -317,8 +317,11 @@ class _BrowserScreencast:
self.browser_id = browser_id
self.session = session
self.mime = mime
self.frame_consumer: Any | None = None
self.stop_callback: Any | None = None
self.queue = asyncio.Queue(maxsize=1)
self.stopped = False
self._closed = False
self._ack_tasks: set[asyncio.Task] = set()
self._expected_width = 0
self._expected_height = 0
@ -329,10 +332,14 @@ class _BrowserScreencast:
quality: int,
every_nth_frame: int,
viewport: dict[str, int],
capture_scale: float = 1.0,
) -> None:
self.session.on("Page.screencastFrame", self._on_frame)
width = max(320, min(4096, int(viewport.get("width") or DEFAULT_VIEWPORT["width"])))
height = max(200, min(4096, int(viewport.get("height") or DEFAULT_VIEWPORT["height"])))
scale = max(1.0, min(2.0, float(capture_scale or 1.0)))
max_width = max(320, min(SCREENCAST_MAX_WIDTH, int(round(width * scale))))
max_height = max(200, min(SCREENCAST_MAX_HEIGHT, int(round(height * scale))))
self._expected_width = width
self._expected_height = height
with contextlib.suppress(Exception):
@ -343,8 +350,8 @@ class _BrowserScreencast:
{
"format": "jpeg",
"quality": max(20, min(95, int(quality))),
"maxWidth": SCREENCAST_MAX_WIDTH,
"maxHeight": SCREENCAST_MAX_HEIGHT,
"maxWidth": max_width,
"maxHeight": max_height,
"everyNthFrame": max(1, int(every_nth_frame)),
},
)
@ -394,10 +401,21 @@ class _BrowserScreencast:
raise RuntimeError("Browser screencast stopped.")
return frame
async def attach_consumer(self, frame_consumer: Any, stop_callback: Any | None = None) -> None:
self.frame_consumer = frame_consumer
self.stop_callback = stop_callback
frame = await self.pop_frame()
if frame:
await self._deliver_frame(frame)
async def stop(self) -> None:
if self.stopped:
if self._closed:
return
was_stopped = self.stopped
self._closed = True
self.stopped = True
if not was_stopped:
self._notify_stopped()
self._drop_queued_frames()
with contextlib.suppress(asyncio.QueueFull):
self.queue.put_nowait(None)
@ -419,6 +437,8 @@ class _BrowserScreencast:
task.add_done_callback(self._ack_tasks.discard)
async def _handle_frame(self, params: dict[str, Any]) -> None:
stop_after_ack = False
notify_stop = False
try:
data = params.get("data") or ""
if data:
@ -428,7 +448,7 @@ class _BrowserScreencast:
metadata["jpegWidth"], metadata["jpegHeight"] = size
metadata["expectedWidth"] = self._expected_width
metadata["expectedHeight"] = self._expected_height
self._queue_latest(
await self._deliver_frame(
{
"browser_id": self.browser_id,
"mime": self.mime,
@ -436,6 +456,14 @@ class _BrowserScreencast:
"metadata": metadata,
}
)
except asyncio.CancelledError:
stop_after_ack = True
except Exception:
if self.frame_consumer:
stop_after_ack = True
notify_stop = True
else:
raise
finally:
session_id = params.get("sessionId")
if session_id is not None and not self.stopped:
@ -444,6 +472,24 @@ class _BrowserScreencast:
"Page.screencastFrameAck",
{"sessionId": int(session_id)},
)
if stop_after_ack:
self.stopped = True
if notify_stop:
self._notify_stopped()
def _notify_stopped(self) -> None:
if not self.stop_callback:
return
with contextlib.suppress(Exception):
self.stop_callback()
async def _deliver_frame(self, frame: dict[str, Any]) -> None:
if not self.frame_consumer:
self._queue_latest(frame)
return
future = self.frame_consumer(frame)
if future is not None:
await asyncio.wrap_future(future)
def _queue_latest(self, frame: dict[str, Any]) -> None:
self._drop_queued_frames()
@ -1616,6 +1662,7 @@ class _BrowserRuntimeCore:
*,
quality: int = 78,
every_nth_frame: int = 1,
capture_scale: float = 1.0,
) -> dict[str, Any]:
await self.ensure_started()
resolved_id = self._resolve_browser_id(browser_id)
@ -1634,6 +1681,7 @@ class _BrowserRuntimeCore:
quality=quality,
every_nth_frame=every_nth_frame,
viewport=page.viewport_size or DEFAULT_VIEWPORT,
capture_scale=capture_scale,
)
except Exception:
self.screencasts.pop(stream_id, None)
@ -1663,6 +1711,17 @@ class _BrowserRuntimeCore:
raise KeyError("Browser screencast is not active.")
return await screencast.pop_frame()
async def attach_screencast_consumer(
self,
stream_id: str,
frame_consumer: Any,
stop_callback: Any | None = None,
) -> None:
screencast = self.screencasts.get(str(stream_id or ""))
if not screencast:
raise KeyError("Browser screencast is not active.")
await screencast.attach_consumer(frame_consumer, stop_callback)
async def stop_screencast(self, stream_id: str) -> None:
screencast = self.screencasts.pop(str(stream_id or ""), None)
if screencast:

View file

@ -3,7 +3,7 @@ Rendered browser automation for pages that need interaction, JavaScript, forms,
Prefer `search_engine` or `document_query` for plain text research. The tool must not open a Browser surface automatically. Use the tool headlessly unless the user opens the Browser surface or asks for the optional visible WebUI viewer.
When the user asks for "my browser", "host browser", "local browser", local Chrome, or opening a URL in their host browser, use this `browser` tool. Do not substitute `computer_use_remote`, `code_execution_remote`, `xdg-open`, `sensible-browser`, or Python `webbrowser.open`. If setup fails and mentions remote debugging, tell the user to open `chrome://inspect/#remote-debugging`, enable "Allow remote debugging for this browser instance", run `/browser host on`, and retry.
When the user asks for "my browser", "host browser", "local browser", a local Chromium browser, or opening a URL in their host browser, use this `browser` tool. Do not substitute `computer_use_remote`, `code_execution_remote`, `xdg-open`, `sensible-browser`, or Python `webbrowser.open`. If setup fails and mentions remote debugging, tell the user to open the browser inspect page, such as `chrome://inspect/#remote-debugging` or `opera://inspect/#remote-debugging`, enable "Allow remote debugging for this browser instance", run `/browser host on`, and retry.
For rendered browsing workflows, multi-step interaction, screenshots, downloads, uploads, forms, or host/container mode decisions, first load `browser-automation` with `skills_tool:load`, then call this tool using the loaded instructions. For fragile forms, `browser-automation` links to `browser-form-workflows`; load it when selects, checkboxes, radios, uploads, contenteditable fields, validation, or submission state are central.

View file

@ -4,8 +4,14 @@ import { callJsonApi } from "/js/api.js";
const BROWSER_EXTENSIONS_API = "/plugins/_browser/extensions";
const BROWSER_STATUS_API = "/plugins/_browser/status";
const RUNTIME_BACKENDS = new Set(["container", "host_required"]);
const BROWSER_TAB_SCOPES = new Set(["per_context", "shared"]);
const HOST_PRIVACY_POLICIES = new Set(["enforce_local", "warn", "allow"]);
const HOST_PROFILE_MODES = new Set(["existing", "agent"]);
const DEFAULT_MAX_OPEN_TABS = 32;
const MIN_MAX_OPEN_TABS = 1;
const HARD_MAX_OPEN_TABS = 50;
const HOST_BROWSER_STATUS_REFRESH_MS = 1000;
const CUSTOM_HOST_BROWSER_SELECTION = "__custom_endpoint__";
function normalizePathList(value) {
const source = Array.isArray(value)
@ -27,6 +33,8 @@ function ensureConfig(config) {
config.extension_paths = normalizePathList(config.extension_paths);
config.default_homepage = String(config.default_homepage || "about:blank").trim() || "about:blank";
config.autofocus_active_page = normalizeBoolean(config.autofocus_active_page, true);
config.browser_tab_scope = normalizeChoice(config.browser_tab_scope, BROWSER_TAB_SCOPES, "per_context");
config.max_open_tabs = normalizeInt(config.max_open_tabs, DEFAULT_MAX_OPEN_TABS, MIN_MAX_OPEN_TABS, HARD_MAX_OPEN_TABS);
config.runtime_backend = normalizeRuntimeBackend(config.runtime_backend);
config.host_browser_privacy_policy = normalizeChoice(
config.host_browser_privacy_policy,
@ -38,6 +46,7 @@ function ensureConfig(config) {
HOST_PROFILE_MODES,
"existing",
);
config.host_browser_selection = normalizeHostBrowserSelection(config.host_browser_selection);
config.model_preset = String(config.model_preset || "").trim();
delete config.model;
return config;
@ -48,12 +57,55 @@ function normalizeChoice(value, allowed, fallback) {
return allowed.has(normalized) ? normalized : fallback;
}
function normalizeInt(value, fallback, minimum, maximum) {
const number = Number.parseInt(value, 10);
if (!Number.isFinite(number)) return fallback;
return Math.max(minimum, Math.min(maximum, number));
}
function normalizeRuntimeBackend(value) {
const normalized = String(value || "").trim().toLowerCase().replace(/-/g, "_");
if (normalized === "host_when_available") return "host_required";
return RUNTIME_BACKENDS.has(normalized) ? normalized : "container";
}
function normalizeHostBrowserSelection(value) {
return String(value || "").trim().toLowerCase().replace(/\s+/g, "_").slice(0, 200);
}
function normalizeCustomHostBrowserEndpoint(value) {
const raw = String(value || "").trim();
if (!raw) return "";
const candidate = raw.includes("://") ? raw : `ws://${raw}`;
try {
const url = new URL(candidate);
if (!["ws:", "wss:"].includes(url.protocol) || !url.host || !url.pathname.startsWith("/devtools/browser/")) {
return "";
}
return normalizeHostBrowserSelection(`${url.protocol}//${url.host}${url.pathname}${url.search || ""}`);
} catch (_error) {
return "";
}
}
function isCustomHostBrowserEndpoint(value) {
return Boolean(normalizeCustomHostBrowserEndpoint(value));
}
function debugPortVersionUrl(value) {
const raw = String(value || "").trim();
if (!raw) return "";
const candidate = raw.includes("://") ? raw : `ws://${raw}`;
try {
const url = new URL(candidate);
if (!["ws:", "wss:"].includes(url.protocol) || !url.host) return "";
if (url.pathname && url.pathname !== "/") return "";
return `http://${url.host}/json/version`;
} catch (_error) {
return "";
}
}
function normalizeBoolean(value, fallback = true) {
if (value === undefined || value === null || value === "") return fallback;
if (typeof value === "boolean") return value;
@ -74,6 +126,9 @@ function hostBrowserFamilyLabel(value) {
chromium: "Chromium",
edge: "Edge",
"edge-dev": "Edge Dev",
brave: "Brave",
opera: "Opera",
vivaldi: "Vivaldi",
};
const label = labels[base] || "Host browser";
if (remoteDebugging) return `${label} (allowed)`;
@ -99,13 +154,18 @@ export const store = createStore("browserConfig", {
extensionDeleteLoadingPath: "",
hostBrowserStatus: null,
hostBrowserStatusLoading: false,
hostBrowserStatusRefreshTimer: null,
hostBrowserCustomEndpoint: "",
hostBrowserCustomMode: false,
async init(config) {
this.bindConfig(config);
await Promise.all([this.loadExtensionsList(), this.loadHostBrowserStatus()]);
this.startHostBrowserStatusRefresh();
},
cleanup() {
this.stopHostBrowserStatusRefresh();
this.config = null;
this.extensionsList = [];
this.extensionsError = "";
@ -113,6 +173,22 @@ export const store = createStore("browserConfig", {
this.extensionDeleteLoadingPath = "";
this.hostBrowserStatus = null;
this.hostBrowserStatusLoading = false;
this.hostBrowserCustomEndpoint = "";
this.hostBrowserCustomMode = false;
},
startHostBrowserStatusRefresh() {
this.stopHostBrowserStatusRefresh();
this.hostBrowserStatusRefreshTimer = window.setInterval(
() => this.loadHostBrowserStatus(),
HOST_BROWSER_STATUS_REFRESH_MS,
);
},
stopHostBrowserStatusRefresh() {
if (!this.hostBrowserStatusRefreshTimer) return;
window.clearInterval(this.hostBrowserStatusRefreshTimer);
this.hostBrowserStatusRefreshTimer = null;
},
bindConfig(config) {
@ -120,6 +196,9 @@ export const store = createStore("browserConfig", {
if (!safeConfig) return;
if (this.config === safeConfig) return;
this.config = safeConfig;
if (isCustomHostBrowserEndpoint(safeConfig.host_browser_selection)) {
this.hostBrowserCustomEndpoint = safeConfig.host_browser_selection;
}
},
setAutofocusActivePage(enabled) {
@ -132,6 +211,27 @@ export const store = createStore("browserConfig", {
return this.config?.autofocus_active_page === false ? "Off" : "On";
},
setBrowserTabScope(value) {
const safeConfig = ensureConfig(this.config);
if (!safeConfig) return;
safeConfig.browser_tab_scope = normalizeChoice(value, BROWSER_TAB_SCOPES, "per_context");
},
browserTabScopeLabel() {
return this.config?.browser_tab_scope === "shared" ? "Shared" : "Per chat";
},
normalizeMaxOpenTabs() {
const safeConfig = ensureConfig(this.config);
if (!safeConfig) return;
safeConfig.max_open_tabs = normalizeInt(
safeConfig.max_open_tabs,
DEFAULT_MAX_OPEN_TABS,
MIN_MAX_OPEN_TABS,
HARD_MAX_OPEN_TABS,
);
},
runtimeBackendLabel() {
const value = this.config?.runtime_backend || "container";
if (value === "host_required") return "Bring Your Own Browser";
@ -145,6 +245,93 @@ export const store = createStore("browserConfig", {
return "Local Models Only";
},
hostBrowserOptions() {
const connectors = Array.isArray(this.hostBrowserStatus?.connectors)
? this.hostBrowserStatus.connectors
: [];
const options = [{ value: "", label: "Automatic (A0 CLI chooses)" }];
const seen = new Set([""]);
for (const connector of connectors) {
const advertised = Array.isArray(connector?.available_browsers)
? connector.available_browsers
: [];
for (const browser of advertised) {
const value = normalizeCustomHostBrowserEndpoint(browser?.cdp_endpoint || browser?.id);
if (!value || seen.has(value)) continue;
seen.add(value);
const label = browser?.label || hostBrowserFamilyLabel(browser?.family || value);
const status = browser?.status ? ` - ${hostBrowserStatusLabel(browser.status)}` : "";
options.push({ value, label: `${label}${status}` });
}
const fallbackValue = normalizeCustomHostBrowserEndpoint(connector?.cdp_endpoint || connector?.browser_id);
if (fallbackValue && !seen.has(fallbackValue)) {
seen.add(fallbackValue);
const label = connector?.browser_label || hostBrowserFamilyLabel(connector?.browser_family || fallbackValue);
options.push({ value: fallbackValue, label });
}
}
const selected = normalizeHostBrowserSelection(this.config?.host_browser_selection);
if (selected && !seen.has(selected) && !isCustomHostBrowserEndpoint(selected)) {
seen.add(selected);
options.push({ value: selected, label: `Saved: ${selected}` });
}
options.push({ value: CUSTOM_HOST_BROWSER_SELECTION, label: "Custom endpoint" });
return options;
},
hostBrowserSelectValue() {
if (this.hostBrowserCustomMode) return CUSTOM_HOST_BROWSER_SELECTION;
const selected = normalizeHostBrowserSelection(this.config?.host_browser_selection);
if (!selected) return "";
if (this.hostBrowserOptions().some((option) => option.value === selected)) return selected;
if (isCustomHostBrowserEndpoint(selected)) return CUSTOM_HOST_BROWSER_SELECTION;
return selected;
},
setHostBrowserSelection(value) {
const safeConfig = ensureConfig(this.config);
if (!safeConfig) return;
if (value === CUSTOM_HOST_BROWSER_SELECTION) {
this.hostBrowserCustomMode = true;
if (isCustomHostBrowserEndpoint(safeConfig.host_browser_selection)) {
this.hostBrowserCustomEndpoint = safeConfig.host_browser_selection;
} else {
safeConfig.host_browser_selection = "";
}
return;
}
this.hostBrowserCustomMode = false;
safeConfig.host_browser_selection = normalizeHostBrowserSelection(value);
},
showCustomHostBrowserEndpoint() {
return this.hostBrowserSelectValue() === CUSTOM_HOST_BROWSER_SELECTION;
},
setCustomHostBrowserEndpoint(value) {
this.hostBrowserCustomMode = true;
this.hostBrowserCustomEndpoint = String(value || "").trim();
const safeConfig = ensureConfig(this.config);
if (!safeConfig) return;
const endpoint = normalizeCustomHostBrowserEndpoint(this.hostBrowserCustomEndpoint);
if (endpoint || !this.hostBrowserCustomEndpoint) {
safeConfig.host_browser_selection = endpoint;
}
},
customHostBrowserEndpointDiagnostic() {
if (!this.hostBrowserCustomEndpoint) {
return "Paste a ws://.../devtools/browser/... endpoint from the browser inspect page.";
}
const endpoint = normalizeCustomHostBrowserEndpoint(this.hostBrowserCustomEndpoint);
if (endpoint) return `Using ${endpoint}`;
const versionUrl = debugPortVersionUrl(this.hostBrowserCustomEndpoint);
if (versionUrl) {
return `This looks like a debug port. Open ${versionUrl} and copy webSocketDebuggerUrl.`;
}
return "Endpoint must be a ws:// or wss:// URL ending in /devtools/browser/...";
},
hostBrowserProfileModeLabel() {
const value = this.config?.host_browser_profile_mode || "existing";
if (value === "agent") return "Clean Agent Profile";

View file

@ -16,7 +16,7 @@
<div class="browser-meta">
<div class="browser-meta-top">
<div class="browser-session-tabs" role="tablist" aria-label="Browser sessions">
<template x-for="browser in $store.browserPage.browsers" :key="$store.browserPage.browserTabKey(browser)">
<template x-for="browser in $store.browserPage.visibleBrowsers()" :key="$store.browserPage.browserTabKey(browser)">
<div class="browser-tab-shell" :class="{ 'is-active': $store.browserPage.isActiveBrowser(browser) }">
<button type="button" class="browser-tab" role="tab"
:aria-selected="$store.browserPage.isActiveBrowser(browser).toString()"
@ -187,12 +187,17 @@
<div class="browser-stage" tabindex="0" @click="$el.focus()"
:class="{ 'is-annotating': $store.browserPage.annotating }"
@wheel.prevent="$store.browserPage.handleStageWheel($event)">
<canvas class="browser-frame browser-frame-canvas"
x-show="$store.browserPage.frameCanvasReady"
x-init="$store.browserPage.attachFrameCanvas($el)"
@click="$store.browserPage.sendMouse('click', $event)"
@mousemove.throttle.250ms="$store.browserPage.sendMouse('move', $event)"></canvas>
<template x-if="$store.browserPage.frameSrc">
<img class="browser-frame" :src="$store.browserPage.frameSrc"
<img class="browser-frame browser-frame-image" :src="$store.browserPage.frameSrc"
@click="$store.browserPage.sendMouse('click', $event)"
@mousemove.throttle.250ms="$store.browserPage.sendMouse('move', $event)" draggable="false" />
</template>
<template x-if="$store.browserPage.annotating && $store.browserPage.frameSrc">
<template x-if="$store.browserPage.annotating && $store.browserPage.hasFrame()">
<div class="browser-annotation-layer"
:class="{ 'is-busy': $store.browserPage.annotationBusy }"
@pointerdown.stop.prevent="$store.browserPage.startAnnotationSelection($event)"
@ -267,7 +272,7 @@
@click="$store.browserPage.sendAnnotationsToChat()">Send now</button>
</div>
</div>
<template x-if="!$store.browserPage.frameSrc && !$store.browserPage.isBusy()">
<template x-if="!$store.browserPage.hasFrame() && !$store.browserPage.isBusy()">
<div class="browser-empty">
<span class="material-symbols-outlined">captive_portal</span>
<button class="btn btn-field" @click="$store.browserPage.command('open')">Open
@ -999,7 +1004,7 @@
height: 100%;
min-width: 0;
min-height: 0;
object-fit: fill;
object-fit: contain;
image-rendering: auto;
user-select: none;
background: #fff;

View file

@ -34,6 +34,10 @@ const ANNOTATION_DOM_LIMIT = 1200;
const ANNOTATION_TRAY_MARGIN = 10;
const BROWSER_VISUAL_SHORTCUT_KEYS = new Set(["a", "c", "insert", "v", "x", "y", "z"]);
const LOCAL_EDITABLE_SELECTOR = "input, textarea, select, [contenteditable]";
const BROWSER_BINARY_FRAME_REQUESTS_ENABLED = false;
const BROWSER_BINARY_PAYLOADS_SUPPORTED = typeof Blob === "function"
&& typeof globalThis.URL?.createObjectURL === "function";
const BROWSER_CANVAS_FRAMES_SUPPORTED = typeof globalThis.createImageBitmap === "function";
function makeViewerToken() {
return globalThis.crypto?.randomUUID?.()
@ -114,6 +118,43 @@ function loadFrameDimensions(src) {
});
}
function frameImageSource(data = {}) {
const image = data?.image;
if (!image) return null;
const mime = data.mime || "image/jpeg";
const isArrayBuffer = typeof ArrayBuffer !== "undefined" && image instanceof ArrayBuffer;
const isView = typeof ArrayBuffer !== "undefined" && ArrayBuffer.isView?.(image);
const isBlob = typeof Blob !== "undefined" && image instanceof Blob;
if (isArrayBuffer || isView || isBlob) {
if (!BROWSER_BINARY_PAYLOADS_SUPPORTED) return null;
const blob = isBlob ? image : new Blob([image], { type: mime });
const src = globalThis.URL.createObjectURL(blob);
return {
src,
blob,
objectUrl: src,
cleanup: () => globalThis.URL.revokeObjectURL(src),
};
}
if (data.encoding === "binary") return null;
if (typeof image !== "string") return null;
return {
src: `data:${mime};base64,${image}`,
objectUrl: "",
cleanup: null,
};
}
async function loadFrameBitmap(src, options = {}) {
if (!BROWSER_CANVAS_FRAMES_SUPPORTED || !src) return null;
try {
const blob = options.blob || await fetch(src).then((response) => response.blob());
return await globalThis.createImageBitmap(blob);
} catch {
return null;
}
}
const model = {
loading: true,
error: "",
@ -124,8 +165,10 @@ const model = {
activeBrowserContextId: "",
address: "",
frameSrc: "",
frameCanvasReady: false,
frameState: null,
viewerTransport: BROWSER_VIEWER_TRANSPORT_SCREENCAST,
tabScope: "per_context",
liveScreencastEnabled: true,
annotating: false,
annotationComments: [],
@ -146,9 +189,11 @@ const model = {
_lastFrameDimensions: null,
_pendingFrameSrc: "",
_pendingFrameOptions: null,
_frameObjectUrl: "",
_frameRenderHandle: null,
_frameRenderCancel: null,
_frameRenderSequence: 0,
_frameCanvas: null,
_floatingCleanup: null,
_stageElement: null,
_stageResizeObserver: null,
@ -295,8 +340,10 @@ const model = {
{ timeoutMs: 10000 },
);
const data = firstOk(response);
this.applyTabScope(data);
this.applyBrowserListing(data.browsers || [], data.context_id || "", {
replaceAll: Boolean(data.all_browsers),
replaceContext: !data.all_browsers,
});
})();
try {
@ -442,7 +489,8 @@ const model = {
this.setActiveBrowserId(null);
this.address = "";
this.frameState = null;
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
if (this.contextId) {
await this.connectViewer();
}
@ -754,11 +802,13 @@ const model = {
},
releaseSurfaceBindings() {
this.freezeCanvasFrameToImage();
this._floatingCleanup?.();
this._floatingCleanup = null;
this._stageResizeObserver?.disconnect?.();
this._stageResizeObserver = null;
this._stageElement = null;
this._frameCanvas = null;
},
isCanvasSurfaceVisible(element = null) {
@ -813,7 +863,7 @@ const model = {
this._surfaceMounted = true;
this._surfaceOpenedAt = Date.now();
this._lastViewportKey = "";
if (this.frameSrc && !targetChanged) {
if (this.hasFrame() && !targetChanged) {
this._surfaceSwitching = false;
this.switchingBrowserId = null;
return;
@ -833,13 +883,14 @@ const model = {
resetRenderedFrame() {
this.cancelFrameRender();
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
this._lastFrameDimensions = null;
this._lastFrameAt = 0;
},
resetRenderedFrameIfViewportChanged(viewport = null, requestedBrowserId = null, requestedContextId = "") {
if (!viewport || !this.frameSrc || !this._lastViewport) return;
if (!viewport || !this.hasFrame() || !this._lastViewport) return;
const targetBrowserId = requestedBrowserId || this.activeBrowserId || this.firstBrowserId();
const targetContextId = this.normalizeContextId(requestedContextId || this.contextIdForBrowserId(targetBrowserId) || this.activeBrowserContextId);
if (!this.sameBrowserTab(this._lastViewport.browserId, this._lastViewport.contextId, targetBrowserId, targetContextId)) return;
@ -911,10 +962,41 @@ const model = {
return BROWSER_VIEWER_TRANSPORT_SNAPSHOT;
},
normalizeTabScope(value = "") {
return String(value || "").trim().toLowerCase().replace("-", "_") === "shared"
? "shared"
: "per_context";
},
applyTabScope(data = {}) {
if (!data || typeof data !== "object") return;
if (!Object.prototype.hasOwnProperty.call(data, "tab_scope")) return;
this.tabScope = this.normalizeTabScope(data.tab_scope);
},
usesScreencastTransport() {
return this.viewerTransport === BROWSER_VIEWER_TRANSPORT_SCREENCAST;
},
supportsBinaryFrames() {
return BROWSER_BINARY_FRAME_REQUESTS_ENABLED && BROWSER_BINARY_PAYLOADS_SUPPORTED;
},
captureDevicePixelRatio() {
const value = Number(globalThis.devicePixelRatio || 1);
if (!Number.isFinite(value) || value <= 1) return 1;
return Math.min(2, value);
},
frameDimensionsFromData(data = null) {
const width = Number(data?.width || 0);
const height = Number(data?.height || 0);
if (Number.isFinite(width) && Number.isFinite(height) && width > 0 && height > 0) {
return { width, height };
}
return this.frameDimensionsFromMetadata(data?.metadata);
},
frameDimensionsFromMetadata(metadata = null) {
if (!metadata || typeof metadata !== "object") return null;
const width = Number(metadata.expectedWidth || metadata.deviceWidth || metadata.jpegWidth || 0);
@ -986,6 +1068,9 @@ const model = {
viewer_id: viewerToken,
create_browser: Boolean(options.createBrowser || options.create_browser),
viewer_transport: this.requestedViewerTransport(),
binary_frames: this.supportsBinaryFrames(),
slim_frames: true,
device_pixel_ratio: this.captureDevicePixelRatio(),
viewport_width: initialViewport?.width,
viewport_height: initialViewport?.height,
},
@ -1007,16 +1092,20 @@ const model = {
return;
}
const data = firstOk(response);
this.applyBrowserListing(data.browsers || [], contextId, { replaceAll: Boolean(data.all_browsers) });
this.applyTabScope(data);
this.applyBrowserListing(data.browsers || [], contextId, {
replaceAll: Boolean(data.all_browsers),
replaceContext: !data.all_browsers,
});
this.viewerTransport = this.normalizeViewerTransport(data.viewer_transport);
this.setActiveBrowserId(
data.active_browser_id || requestedBrowserId || this.activeBrowserId || null,
data.active_browser_context_id || contextId,
);
this.applySnapshot(data.snapshot);
this.connected = true;
this.browserInstallExpected = false;
},
this.connected = true;
this.browserInstallExpected = false;
},
async _bindSocketEvents() {
if (!this._frameOff) {
@ -1026,9 +1115,15 @@ const model = {
if (data?.viewer_transport) {
this.viewerTransport = this.normalizeViewerTransport(data.viewer_transport);
}
this.applyTabScope(data);
const incomingContextId = this.normalizeContextId(data.context_id || this.contextId);
const incomingBrowserId = this.normalizeBrowserId(data.browser_id || data.state?.id);
this.applyBrowserListing(data.browsers || [], incomingContextId, { replaceContext: true });
if (Array.isArray(data.browsers)) {
this.applyBrowserListing(data.browsers, incomingContextId, {
replaceAll: Boolean(data.all_browsers),
replaceContext: !data.all_browsers,
});
}
if (incomingBrowserId && !this.activeBrowserId) {
this.setActiveBrowserId(incomingBrowserId, incomingContextId);
}
@ -1046,11 +1141,17 @@ const model = {
this.address = data.state.currentUrl;
}
if (data.image) {
const frameImage = frameImageSource(data);
if (!frameImage?.src) return;
const frameBrowserId = incomingBrowserId || this.activeBrowserId;
this.queueFrameRender(`data:${data.mime || "image/jpeg"};base64,${data.image}`, {
this.queueFrameRender(frameImage.src, {
browserId: frameBrowserId,
contextId: incomingContextId,
dimensions: this.frameDimensionsFromMetadata(data.metadata),
dimensions: this.frameDimensionsFromData(data),
blob: frameImage.blob,
objectUrl: frameImage.objectUrl,
useCanvas: true,
cleanup: frameImage.cleanup,
onAccepted: () => {
if (
this.sameBrowserId(this.switchingBrowserId, frameBrowserId)
@ -1063,13 +1164,15 @@ const model = {
});
} else if (!data.state) {
this.cancelFrameRender();
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
}
if (!data.image && !data.state) {
if (!this.activeBrowserId) {
this.setActiveBrowserId(null, "");
this.frameState = null;
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
}
}
this._lastFrameAt = Date.now();
@ -1077,15 +1180,21 @@ const model = {
await websocket.on("browser_viewer_frame", frameHandler);
this._frameOff = () => websocket.off("browser_viewer_frame", frameHandler);
}
if (!this._stateOff) {
const stateHandler = ({ data }) => {
if (data?.context_id !== this.contextId) return;
if (data?.viewer_id && data.viewer_id !== this._viewerToken) return;
if (data?.viewer_transport) {
this.viewerTransport = this.normalizeViewerTransport(data.viewer_transport);
}
if (!this._stateOff) {
const stateHandler = ({ data }) => {
if (data?.context_id !== this.contextId) return;
if (data?.viewer_id && data.viewer_id !== this._viewerToken) return;
if (data?.viewer_transport) {
this.viewerTransport = this.normalizeViewerTransport(data.viewer_transport);
}
this.applyTabScope(data);
const commandContextId = this.normalizeContextId(data.active_browser_context_id || data.context_id || this.contextId);
this.applyBrowserListing(data.browsers || [], commandContextId, { replaceAll: Boolean(data.all_browsers) });
if (Array.isArray(data.browsers)) {
this.applyBrowserListing(data.browsers, commandContextId, {
replaceAll: Boolean(data.all_browsers),
replaceContext: !data.all_browsers,
});
}
const command = String(data.command || "").toLowerCase();
const commandBrowserId = this.normalizeBrowserId(data.browser_id);
const result = data.result || {};
@ -1102,23 +1211,40 @@ const model = {
|| this.activeBrowserId
|| this.firstBrowserId(resultContextId)
);
if (
!this.activeBrowserId
|| command === "open"
|| command === "close"
|| this.sameBrowserTab(commandBrowserId, commandContextId, this.activeBrowserId, this.activeBrowserContextId)
) {
this.setActiveBrowserId(preferredBrowserId, resultContextId);
}
this.applyActiveFrameState(resultState || this.browserById(this.activeBrowserId, this.activeBrowserContextId));
this.applySnapshot(data.snapshot);
};
const stateBrowserId = this.normalizeBrowserId(data.active_browser_id || data.browser_id || data.state?.id);
if (
stateBrowserId
&& (
!this.activeBrowserId
|| this.sameBrowserTab(stateBrowserId, commandContextId, this.activeBrowserId, this.activeBrowserContextId)
)
) {
this.setActiveBrowserId(stateBrowserId, commandContextId);
}
if (
!this.activeBrowserId
|| command === "open"
|| command === "close"
|| this.sameBrowserTab(commandBrowserId, commandContextId, this.activeBrowserId, this.activeBrowserContextId)
) {
this.setActiveBrowserId(preferredBrowserId, resultContextId);
}
this.applyActiveFrameState(
resultState
|| data.state
|| this.browserById(this.activeBrowserId, this.activeBrowserContextId)
);
this.applySnapshot(data.snapshot);
};
await websocket.on("browser_viewer_state", stateHandler);
this._stateOff = () => websocket.off("browser_viewer_state", stateHandler);
}
},
queueFrameRender(frameSrc, options = {}) {
if (this._pendingFrameSrc) {
this.releasePendingFrame();
}
this._pendingFrameSrc = frameSrc;
this._pendingFrameOptions = options || null;
if (this._frameRenderHandle) return;
@ -1148,22 +1274,44 @@ const model = {
async renderDecodedFrame(frameSrc, options = {}, sequence = 0, surfaceSequence = this._surfaceOpenSequence) {
if (!frameSrc) {
if (sequence === this._frameRenderSequence) {
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
}
return;
}
const dimensions = options?.dimensions || await loadFrameDimensions(frameSrc);
let bitmap = null;
let dimensions = options?.dimensions || null;
if (options?.useCanvas && this.canUseCanvasFrames()) {
bitmap = await loadFrameBitmap(frameSrc, options);
if (bitmap) {
dimensions ||= { width: bitmap.width || 0, height: bitmap.height || 0 };
}
}
dimensions ||= await loadFrameDimensions(frameSrc);
if (sequence !== this._frameRenderSequence || surfaceSequence !== this._surfaceOpenSequence) {
bitmap?.close?.();
options?.cleanup?.();
return;
}
const viewport = this.currentViewportSize() || this._lastViewport;
if (!this.frameMatchesViewport(dimensions, viewport)) {
this.requestViewportSyncAfterRejectedFrame();
if (!this.shouldAcceptMismatchedFrame(dimensions)) {
bitmap?.close?.();
options?.cleanup?.();
return;
}
}
this.frameSrc = frameSrc;
if (bitmap && this.paintFrameBitmap(bitmap)) {
this.clearFrameSrc();
options?.cleanup?.();
} else {
this.clearFrameCanvas();
this.releaseRenderedFrameUrl(frameSrc);
this.frameSrc = frameSrc;
this._frameObjectUrl = options?.objectUrl || "";
}
bitmap?.close?.();
this._lastFrameDimensions = dimensions;
this._lastFrameAt = Date.now();
options?.onAccepted?.();
@ -1175,7 +1323,7 @@ const model = {
return Boolean(
dimensions?.width
&& dimensions?.height
&& (!this.frameSrc || this._surfaceSwitching || this.isSwitchingBrowser())
&& (!this.hasFrame() || this._surfaceSwitching || this.isSwitchingBrowser())
);
},
@ -1246,7 +1394,7 @@ const model = {
clearRenderedFrameIfViewportChanged() {
const viewport = this.currentViewportSize();
if (!this.frameSrc || !this._lastFrameDimensions || !viewport) return;
if (!this.hasFrame() || !this._lastFrameDimensions || !viewport) return;
if (this.frameMatchesViewport(this._lastFrameDimensions, viewport)) return;
this.cancelFrameRender();
this.resetViewportTracking();
@ -1262,9 +1410,84 @@ const model = {
}
this._frameRenderHandle = null;
this._frameRenderCancel = null;
this.releasePendingFrame();
this._frameRenderSequence += 1;
},
releasePendingFrame() {
this._pendingFrameOptions?.cleanup?.();
this._pendingFrameSrc = "";
this._pendingFrameOptions = null;
this._frameRenderSequence += 1;
},
releaseRenderedFrameUrl(nextSrc = "") {
if (this._frameObjectUrl && this._frameObjectUrl !== nextSrc) {
globalThis.URL?.revokeObjectURL?.(this._frameObjectUrl);
this._frameObjectUrl = "";
}
},
clearFrameSrc() {
this.releaseRenderedFrameUrl("");
this.frameSrc = "";
},
attachFrameCanvas(canvas = null) {
this._frameCanvas = canvas || null;
},
currentFrameCanvas() {
const stageCanvas = this._stageElement?.querySelector?.(".browser-frame-canvas");
if (stageCanvas?.isConnected) return stageCanvas;
if (this._frameCanvas?.isConnected) return this._frameCanvas;
return null;
},
canUseCanvasFrames() {
return Boolean(BROWSER_CANVAS_FRAMES_SUPPORTED && this.currentFrameCanvas()?.getContext);
},
hasFrame() {
return Boolean(this.frameSrc || this.frameCanvasReady);
},
paintFrameBitmap(bitmap) {
const canvas = this.currentFrameCanvas();
if (!canvas || !bitmap?.width || !bitmap?.height) return false;
if (canvas.width !== bitmap.width) canvas.width = bitmap.width;
if (canvas.height !== bitmap.height) canvas.height = bitmap.height;
const context = canvas.getContext("2d");
if (!context) return false;
context.drawImage(bitmap, 0, 0);
this.frameCanvasReady = true;
return true;
},
clearFrameCanvas() {
const canvas = this.currentFrameCanvas();
if (canvas?.width && canvas?.height) {
canvas.getContext("2d")?.clearRect(0, 0, canvas.width, canvas.height);
}
this.frameCanvasReady = false;
},
freezeCanvasFrameToImage() {
const canvas = this.currentFrameCanvas();
if (!this.frameCanvasReady || !canvas) return;
try {
this.frameSrc = canvas.toDataURL("image/jpeg", 0.86);
} catch {
this.frameSrc = "";
}
this.clearFrameCanvas();
},
frameElement() {
if (this.frameCanvasReady) {
const canvas = this.currentFrameCanvas();
if (canvas) return canvas;
}
return this._stageElement?.querySelector?.(".browser-frame-image") || null;
},
beginCommand() {
@ -1303,7 +1526,11 @@ const model = {
{ timeoutMs: 20000 },
);
const data = firstOk(response);
this.applyBrowserListing(data.browsers || [], targetContextId, { replaceAll: Boolean(data.all_browsers) });
this.applyTabScope(data);
this.applyBrowserListing(data.browsers || [], targetContextId, {
replaceAll: Boolean(data.all_browsers),
replaceContext: !data.all_browsers,
});
this.viewerTransport = this.normalizeViewerTransport(data.viewer_transport);
const result = data.result || {};
const resultContextId = this.normalizeContextId(
@ -1329,7 +1556,8 @@ const model = {
);
if (!this.activeBrowserId) {
this.frameState = null;
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
}
if (result.state?.currentUrl || result.currentUrl) {
this.address = result.state?.currentUrl || result.currentUrl;
@ -1438,7 +1666,8 @@ const model = {
this.error = "";
this.switchingBrowserId = targetId;
this.cancelFrameRender();
this.frameSrc = "";
this.clearFrameSrc();
this.clearFrameCanvas();
this.frameState = browser || null;
if (!this.addressFocused && browser?.currentUrl) {
this.address = browser.currentUrl;
@ -1546,6 +1775,15 @@ const model = {
return browsers[0] || null;
},
visibleBrowsers() {
const browsers = Array.isArray(this.browsers) ? this.browsers : [];
if (this.tabScope === "shared") return browsers;
const contextId = this.normalizeContextId(this.activeBrowserContextId || this.contextId || this.resolveContextId());
return contextId
? browsers.filter((browser) => this.normalizeContextId(browser?.context_id) === contextId)
: browsers;
},
firstBrowserInContext(contextId = "") {
const normalizedContextId = this.normalizeContextId(contextId);
if (!normalizedContextId || !Array.isArray(this.browsers)) return null;
@ -1643,35 +1881,35 @@ const model = {
}
},
applySnapshot(snapshot = null) {
if (!snapshot?.image) return;
const snapshotId = this.normalizeBrowserId(snapshot.browser_id || snapshot.state?.id);
applySnapshot(snapshot = null) {
if (!snapshot?.image) return;
const snapshotId = this.normalizeBrowserId(snapshot.browser_id || snapshot.state?.id);
const snapshotContextId = this.normalizeContextId(snapshot.context_id || snapshot.state?.context_id || this.activeBrowserContextId);
if (
if (
snapshotId
&& this.activeBrowserId
&& !this.sameBrowserTab(snapshotId, snapshotContextId, this.activeBrowserId, this.activeBrowserContextId)
) {
return;
}
if (snapshot.state) {
this.applyActiveFrameState(snapshot.state);
}
const frameBrowserId = snapshotId || this.activeBrowserId;
this.queueFrameRender(`data:${snapshot.mime || "image/jpeg"};base64,${snapshot.image}`, {
browserId: frameBrowserId,
return;
}
if (snapshot.state) {
this.applyActiveFrameState(snapshot.state);
}
const frameBrowserId = snapshotId || this.activeBrowserId;
this.queueFrameRender(`data:${snapshot.mime || "image/jpeg"};base64,${snapshot.image}`, {
browserId: frameBrowserId,
contextId: snapshotContextId,
onAccepted: () => {
if (
onAccepted: () => {
if (
this.sameBrowserId(this.switchingBrowserId, frameBrowserId)
&& this.normalizeContextId(this.activeBrowserContextId) === snapshotContextId
) {
this.switchingBrowserId = null;
}
this._surfaceSwitching = false;
},
});
},
this.switchingBrowserId = null;
}
this._surfaceSwitching = false;
},
});
},
isSwitchingBrowser() {
return Boolean(
@ -1710,8 +1948,8 @@ const model = {
const target = element || event?.currentTarget;
if (!target) return null;
const rect = target.getBoundingClientRect();
const naturalWidth = target.naturalWidth || rect.width;
const naturalHeight = target.naturalHeight || rect.height;
const naturalWidth = target.naturalWidth || target.width || rect.width;
const naturalHeight = target.naturalHeight || target.height || rect.height;
let contentLeft = rect.left;
let contentTop = rect.top;
let contentWidth = rect.width;
@ -1748,6 +1986,7 @@ const model = {
},
handleKeydown(event) {
if (isLocalEditableTarget(event?.target)) return;
const annotateShortcut = event?.key === "." && (event.metaKey || event.ctrlKey) && !event.altKey;
if (annotateShortcut && this._surfaceMounted) {
event.preventDefault();
@ -1862,7 +2101,7 @@ const model = {
},
canAnnotate() {
return Boolean(this.activeBrowserId && this.frameSrc && !this.isBusy());
return Boolean(this.activeBrowserId && this.hasFrame() && !this.isBusy());
},
activeAnnotationUrl() {
@ -2023,8 +2262,7 @@ const model = {
},
stagePointForEvent(event) {
const image = this._stageElement?.querySelector?.(".browser-frame") || null;
return this.pointerCoordinatesFor(event, image);
return this.pointerCoordinatesFor(event, this.frameElement());
},
normalizeAnnotationRect(start = {}, end = {}) {
@ -2425,7 +2663,9 @@ const model = {
const response = await websocket.request("browser_viewer_input", payload, { timeoutMs: 10000 });
const data = firstOk(response);
this.applyActiveFrameState(data.state);
this.applySnapshot(data.snapshot);
if (!this.frameCanvasReady || !this.usesScreencastTransport()) {
this.applySnapshot(data.snapshot);
}
} catch (error) {
this.error = error instanceof Error ? error.message : String(error);
}
@ -2437,8 +2677,7 @@ const model = {
async sendWheel(event) {
const contextId = this.normalizeContextId(this.activeBrowserContextId || this.contextId);
if (!contextId || !this.activeBrowserId || !event) return;
const image = event.currentTarget?.querySelector?.(".browser-frame") || event.target?.closest?.(".browser-frame");
const pointer = this.pointerCoordinatesFor(event, image);
const pointer = this.pointerCoordinatesFor(event, this.frameElement());
if (!pointer) return;
const payload = {
context_id: contextId,
@ -2462,8 +2701,7 @@ const model = {
const contextId = this.normalizeContextId(this.activeBrowserContextId || this.contextId);
if (!contextId || !this.activeBrowserId) return;
if (event.ctrlKey || event.metaKey || event.altKey) return;
const editable = ["INPUT", "TEXTAREA", "SELECT"].includes(event.target?.tagName);
if (editable) return;
if (isLocalEditableTarget(event?.target)) return;
event.preventDefault();
const printable = event.key && event.key.length === 1;
await websocket.emit("browser_viewer_input", {
@ -2565,6 +2803,7 @@ const model = {
this._viewerToken = "";
this.switchingBrowserId = null;
this.viewerTransport = this.requestedViewerTransport();
this.tabScope = "per_context";
this._surfaceMounted = false;
this._surfaceSwitching = false;
this.commandInFlight = false;
@ -2846,8 +3085,21 @@ const model = {
export const store = createStore("browserPage", model);
const WEB_INTENT_SCHEMES = new Set(["http", "https", "file", "about"]);
function isWebUrlIntent(url = "") {
const value = String(url || "").trim();
if (!value) return true;
const scheme = /^([a-z][a-z0-9+.-]*):/i.exec(value);
if (!scheme) return true;
return WEB_INTENT_SCHEMES.has(scheme[1].toLowerCase());
}
registerUrlHandler(async (intent = {}) => {
const url = String(intent.url || "").trim();
// Custom schemes such as a0-editor: belong to other surfaces; claiming them
// here would navigate the browser to an unloadable URL.
if (!isWebUrlIntent(url)) return false;
const payload = { url, source: intent.source || "surface-url-intent" };
await openLatestSurface("browser", payload);
return await store.openUrlIntent(url, { source: payload.source });

View file

@ -37,7 +37,7 @@
class="browser-config-field-help"
x-show="$store.browserConfig.config.runtime_backend === 'host_required'"
>
Uses Chrome, Edge, or Chromium on the A0 CLI host. Keep A0 CLI connected; the browser opens when the agent first needs it. For an already-open browser, use chrome://inspect/#remote-debugging and enable "Allow remote debugging for this browser instance".
Uses Chrome, Edge, Brave, Opera, Vivaldi, or Chromium on the A0 CLI host. Keep A0 CLI connected; the browser opens when the agent first needs it. For an already-open browser, use its inspect page, such as chrome://inspect/#remote-debugging or opera://inspect/#remote-debugging, and enable "Allow remote debugging for this browser instance".
</span>
</label>
@ -66,6 +66,50 @@
</span>
</label>
<label
class="browser-config-field"
x-show="$store.browserConfig.config.runtime_backend === 'host_required' && $store.browserConfig.config.host_browser_profile_mode !== 'agent'"
>
<span class="browser-config-field-label">Host browser</span>
<select
:value="$store.browserConfig.hostBrowserSelectValue()"
@focus="$store.browserConfig.loadHostBrowserStatus()"
@click="$store.browserConfig.loadHostBrowserStatus()"
@change="$store.browserConfig.setHostBrowserSelection($event.target.value)"
>
<template x-for="option in $store.browserConfig.hostBrowserOptions()" :key="option.value">
<option :value="option.value" x-text="option.label"></option>
</template>
</select>
<span class="browser-config-field-help">Choose an allowed debug endpoint, or leave Automatic so A0 CLI picks.</span>
</label>
<label
class="browser-config-field"
x-show="$store.browserConfig.config.runtime_backend === 'host_required' && $store.browserConfig.config.host_browser_profile_mode !== 'agent' && $store.browserConfig.showCustomHostBrowserEndpoint()"
>
<span class="browser-config-field-label">Custom endpoint</span>
<input
type="text"
:value="$store.browserConfig.hostBrowserCustomEndpoint"
@focus="$store.browserConfig.loadHostBrowserStatus()"
@input="$store.browserConfig.setCustomHostBrowserEndpoint($event.target.value)"
placeholder="ws://127.0.0.1:9222/devtools/browser/..."
autocomplete="off"
/>
<span class="browser-config-field-help" x-text="$store.browserConfig.customHostBrowserEndpointDiagnostic()"></span>
</label>
<div
class="browser-config-note"
x-show="$store.browserConfig.config.runtime_backend === 'host_required' && $store.browserConfig.config.host_browser_profile_mode !== 'agent'"
>
<span class="material-symbols-outlined">info</span>
<span>
For an already-open browser, Chrome, Edge, Brave, Vivaldi, and Chromium use chrome://inspect/#remote-debugging; Opera uses opera://inspect/#remote-debugging. Enable "Allow remote debugging for this browser instance", then restart or reconnect A0 CLI if the browser does not appear. If needed, launch the browser with --remote-debugging-port=9222 and --user-data-dir=&lt;profile-dir&gt;, or set A0_HOST_BROWSER_REMOTE_DEBUGGING_ENDPOINTS to the full ws://.../devtools/browser/... endpoint.
</span>
</div>
<div
class="browser-config-warning"
x-show="$store.browserConfig.config.runtime_backend === 'host_required' && $store.browserConfig.config.host_browser_profile_mode !== 'agent'"
@ -112,6 +156,44 @@
/>
</label>
<label class="browser-config-field">
<span class="browser-config-field-label">Browser tabs</span>
<select
x-model="$store.browserConfig.config.browser_tab_scope"
@change="$store.browserConfig.setBrowserTabScope($event.target.value)"
>
<option value="per_context">Separate per chat</option>
<option value="shared">Shared across chats</option>
</select>
<span
class="browser-config-field-help"
x-show="$store.browserConfig.config.browser_tab_scope !== 'shared'"
>
Each chat shows only its own Browser tabs.
</span>
<span
class="browser-config-field-help"
x-show="$store.browserConfig.config.browser_tab_scope === 'shared'"
>
The Browser tab strip shows tabs from every active chat.
</span>
</label>
<label class="browser-config-field">
<span class="browser-config-field-label">Maximum tabs per chat</span>
<input
type="number"
min="1"
max="50"
step="1"
x-model.number="$store.browserConfig.config.max_open_tabs"
@change="$store.browserConfig.normalizeMaxOpenTabs()"
/>
<span class="browser-config-field-help">
New Browser tabs stop opening in a chat after this limit.
</span>
</label>
<label class="browser-config-switch-row">
<span class="browser-config-switch-copy">
<span class="browser-config-field-label">Autofocus active page</span>
@ -242,6 +324,7 @@
}
.browser-config-field input[type="text"],
.browser-config-field input[type="number"],
.browser-config-field select {
width: 100%;
min-height: 36px;

View file

@ -19,6 +19,7 @@
- Backup JSON and transcript artifacts must remain UTF-8 writable when chat content contains malformed Unicode such as lone surrogates.
- Keep generated summaries bounded by configured model and token limits.
- Preserve loaded skill names from `skill_instructions` metadata without copying full skill bodies into compacted summaries.
- After replacing local history, clear the active Responses provider continuation while preserving stored response IDs for cleanup.
- Do not discard original context data unless the compaction flow explicitly owns that behavior.
## Work Guidance

View file

@ -5,7 +5,7 @@ from collections import deque
import models as models_module
from agent import Agent
from helpers import files, tokens
from helpers.history import History, output_text
from helpers.history import History, clear_responses_provider_state, output_text
from helpers.persist_chat import (
export_json_chat,
get_chat_folder_path,
@ -145,6 +145,7 @@ async def run_compaction(
agent.history = History(agent=agent)
agent.history.add_message(ai=True, content=compacted_content)
clear_responses_provider_state(agent)
# Clear subordinate chain
agent.data.pop(Agent.DATA_NAME_SUBORDINATE, None)

View file

@ -14,6 +14,8 @@
## Local Contracts
- Keep session concurrency, timeout, streaming, and reset behavior predictable.
- Execute multi-line terminal input as one current-shell compound so intermediate prompts cannot mark queued work complete; preserve `cd`, exports, and other shell state.
- Terminal reset/close must not hang on foreground commands or shells that ignore SIGTERM.
- Explicitly target local versus SSH execution runtimes.
- Do not hardcode secrets, SSH credentials, or local user paths.

View file

@ -17,6 +17,7 @@ This plugin provides the code execution tool used by agents for development task
- **Persistent shells**
- Maintains per-session shell state in agent data so subsequent calls can reuse the same terminal session.
- Groups multi-line terminal input in the current shell so only the final prompt marks the command complete.
- **Multiple runtimes**
- Dispatches requests based on `runtime`: `terminal`, `python`, `nodejs`, `output`, or `reset`.
- **Remote execution support**

View file

@ -1,10 +1,12 @@
import asyncio, os, sys, platform, errno
import asyncio, os, sys, platform, errno, signal
_IS_WIN = platform.system() == "Windows"
if _IS_WIN:
import winpty # pip install pywinpty # type: ignore
import msvcrt
_CLOSE_TIMEOUT_SECONDS = 2
def _reconfigure_stream_errors(stream) -> None:
reconfigure = getattr(stream, "reconfigure", None)
@ -77,15 +79,16 @@ class TTYSession:
# Terminate the process if it exists
if self._proc:
if getattr(self._proc, "returncode", None) is None:
self._signal_process(signal.SIGTERM)
try:
if getattr(self._proc, "returncode", None) is None:
self._proc.terminate()
except ProcessLookupError:
pass
except Exception:
pass
try:
await self._proc.wait()
await asyncio.wait_for(self._proc.wait(), _CLOSE_TIMEOUT_SECONDS)
except asyncio.TimeoutError:
self._signal_process(signal.SIGKILL)
try:
await asyncio.wait_for(self._proc.wait(), _CLOSE_TIMEOUT_SECONDS)
except Exception:
pass
except Exception:
pass
@ -93,6 +96,28 @@ class TTYSession:
self._proc = None
self._pump_task = None
def _signal_process(self, sig):
if self._proc is None:
return
try:
if _IS_WIN:
if sig == signal.SIGKILL:
self._proc.kill()
else:
self._proc.terminate()
return
os.killpg(self._proc.pid, sig)
except ProcessLookupError:
pass
except Exception:
try:
if sig == signal.SIGKILL:
self._proc.kill()
else:
self._proc.terminate()
except Exception:
pass
def _release_pty_master(self):
"""Release the POSIX PTY master exactly once.
@ -166,11 +191,7 @@ class TTYSession:
# Only attempt to kill if the process is still running
if getattr(self._proc, "returncode", None) is None:
try:
self._proc.kill()
except ProcessLookupError:
# Child already gone treat as successfully killed
pass
self._signal_process(signal.SIGKILL)
self._release_pty_master()
async def read(self, timeout=None):
@ -241,6 +262,7 @@ async def _spawn_posix_pty(cmd, cwd, env, echo):
cwd=cwd,
env=env,
close_fds=True,
start_new_session=True,
)
os.close(slave)

View file

@ -27,6 +27,14 @@ def _is_closed_pty_error(exc: BaseException) -> bool:
return False
def _group_multiline_command(command: str, powershell: bool = False) -> str:
body = command.rstrip("\n")
if "\n" not in body:
return body
opener = ". {" if powershell else "{"
return f"{opener}\n{body}\n}}"
@dataclass
class ShellWrap:
id: int
@ -165,6 +173,9 @@ class CodeExecution(Tool):
+ self.format_command_for_output(command)
+ "\n\n"
)
command = _group_multiline_command(
command, powershell=runtime.is_windows() and not cfg["ssh_enabled"]
)
return await self.terminal_session(cfg, session, command, reset, prefix)
async def terminal_session(

View file

@ -0,0 +1,47 @@
# Commands Plugin DOX
## Purpose
- Own the built-in slash command manager and chat composer slash picker.
- Keep file-backed `/command` discovery consistent across project, global, and plugin-provided scopes.
## Ownership
- `plugin.yaml` owns the built-in `_commands` plugin metadata.
- `helpers/commands.py` owns command name sanitization, argument parsing, scope resolution, file persistence, plugin command discovery, and command invocation resolution.
- `api/commands.py` owns the Commands API actions used by the WebUI.
- `webui/` owns the manager/editor modal stores, HTML surfaces, and thumbnail asset.
- `commands/` owns bundled read-only slash command definitions shipped by `_commands`.
- `extensions/` owns the chat composer slash picker.
- `extensions/python/startup_migration/` owns one-time migration from the legacy community `commands` plugin namespace.
- `skills/commands-create-slash-command/` owns the agent-facing authoring workflow for reusable slash commands.
- `tests/` owns regression coverage for parsing, CRUD, scope precedence, plugin-distributed commands, legacy migration, and skill discovery.
## Local Contracts
- The plugin identity is `_commands`; user-created command files live under `usr/plugins/_commands/commands/` or `usr/projects/<project>/.a0proj/plugins/_commands/commands/`.
- Each command is one `.command.yaml` config plus one same-directory `.txt` text template or `.py` script hook.
- Project commands override global commands, global commands override bundled `_commands/commands/` defaults, and bundled defaults override other plugin-distributed commands with the same name.
- Bundled `_commands/commands/` definitions and commands contributed by other plugins are read-only from this manager.
- The manager lists bundled commands separately; editing one copies it unchanged into the selected project or global scope under the same name, then edits that higher-precedence override.
- Bundled command files use canonical command names only; do not ship alias-only built-ins such as `/img` for `/attach`.
- Command configs may set `webui_hidden: true` to stay resolvable but be omitted from the chat composer picker.
- Commands contributed by enabled plugins live in their `commands/` directory and must not be rediscovered through the generic plugin-distributed path from `_commands` itself.
- On startup, `_commands` copies legacy `usr/plugins/commands` command and skill files into `usr/plugins/_commands` without overwriting existing files, copies scoped legacy command folders to `_commands`, and disables the legacy `commands` plugin roots to prevent duplicate WebUI popovers.
- Script commands must expose `run(payload)` and return a string or a dict with `text` and optional `effects`; `show_markdown` effects render as auto-dismissing toast notifications.
- Script commands may emit `send_message` with `text` to submit the rendered composer text immediately after command resolution.
## Work Guidance
- Keep the command storage and route namespace aligned with `_commands`.
- Preserve unknown command config keys when editing commands.
- Keep built-in source files immutable; user edits must be same-name scope overrides.
- Keep WebUI paths pointed at `/plugins/_commands/...`.
## Verification
- Run `conda run -n a0 pytest plugins/_commands/tests` after backend or command contract changes.
## Child DOX Index
No child DOX files.

21
plugins/_commands/LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Commands plugin contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

145
plugins/_commands/README.md Normal file
View file

@ -0,0 +1,145 @@
# Commands
YAML-configured slash commands for Agent Zero.
This plugin lets you define reusable `/commands` as `.command.yaml` files with either:
- a `.txt` template body
- a `.py` script hook
Commands are managed from the plugin modal and can be inserted directly from the chat composer when the first token starts with `/`.
## Features
- `.command.yaml` config files with command metadata
- Text template commands with `{}` placeholders and parsed args
- Python hook commands with parsed args and optional chat history payload
- Unified parser for positional args, free-form tail, and flags
- Scope-aware command resolution across project and global scopes
- Built-in A0 CLI connector command pack for common session, queue, model, project, browser, and connector status commands
- Slash picker in the chat composer with keyboard navigation and create-on-empty flow
## Command File Model
Each command is defined by one config file plus one content file in the same scope directory.
Set `webui_hidden: true` to keep a command resolvable while omitting it from the chat composer picker.
Example text command:
`scan.command.yaml`
```yaml
name: scan
description: Scan a Git repository.
argument_hint: /scan --git-url https://github.com/org/repo
type: text
template_path: scan.txt
```
`scan.txt`
```txt
Please scan repository: {args.flags.git_url}
Raw input:
{raw}
```
Example python hook command:
`optimize.command.yaml`
```yaml
name: optimize
description: Optimize the current request.
argument_hint: /optimize 30%
type: script
script_path: optimize.py
include_history: true
```
`optimize.py`
```python
def run(payload):
args = payload["arguments"]
pct = args["positional"][0] if args["positional"] else "10%"
return {
"text": f"Optimize this response by {pct}.",
"effects": [],
}
```
## Argument Parsing
The parser supports:
- Positional input: `/scan https://github.com/org/repo`
- Long flags: `/scan --git-url https://github.com/org/repo`
- Long flags with equals: `/scan --git-url=https://github.com/org/repo`
- Short flags and bundles: `/scan -v -q` or `/scan -vq`
Parsed data is available to:
- Text templates via `{}` placeholders:
- `{raw}`
- `{args.positional.0}`
- `{args.flags.git_url}`
- Python scripts via `payload["arguments"]`
## Script Hook Contract
Python hook file must expose:
```python
def run(payload): ...
```
It can return:
- `str` (used as replacement text)
- `dict` with:
- `text: str` (replacement text)
- `effects: list[dict]`
Supported frontend effects:
- `{"type": "replace_input", "text": "..."}`
- `{"type": "append_input", "text": "..."}`
- `{"type": "toast", "level": "info|error|success", "message": "..."}`
- Built-in UI effects for existing WebUI actions such as chat switching, modals, attachments, compaction, queue actions, transcript copy, and toast output
## Scope Resolution
Commands are discovered from these scope folders:
- Project: `usr/projects/<project>/.a0proj/plugins/_commands/commands/`
- Global fallback: `usr/plugins/_commands/commands/`
- Built-in defaults: `plugins/_commands/commands/`
- Other enabled plugins: `plugins/<plugin>/commands/` or `usr/plugins/<plugin>/commands/`
Precedence in the chat picker:
1. Project
2. Global
3. Built-in `_commands`
4. Other plugin-distributed commands
## Legacy Community Plugin Migration
When the built-in `_commands` plugin starts, it migrates files from the older community `commands` plugin namespace:
- Copies `usr/plugins/commands/commands/` into `usr/plugins/_commands/commands/`
- Copies `usr/plugins/commands/skills/` into `usr/plugins/_commands/skills/`
- Copies project and agent scoped `plugins/commands/commands/` folders to matching `plugins/_commands/commands/` folders
- Skips existing destination files
- Disables the legacy `commands` plugin roots so the WebUI does not load two slash-command popovers
## UI Surfaces
- Plugin modal: manage project/global commands and create editable same-name overrides of bundled commands
- Chat composer: type `/` at the start of the inline input to browse commands
## Agent Skill
The plugin ships with `commands-create-slash-command`, a plugin-scoped skill that helps Agent Zero create or update command files.

View file

@ -0,0 +1 @@
"""Commands plugin package."""

View file

@ -0,0 +1,166 @@
from __future__ import annotations
from helpers.api import ApiHandler, Request, Response
from plugins._commands.helpers import commands as commands_helper
class Commands(ApiHandler):
async def process(self, input: dict, request: Request) -> dict | Response:
action = str(input.get("action", "") or "").strip()
if action == "list_effective":
return self._list_effective(input)
if action == "list_scope":
return self._list_scope(input)
if action == "get":
return self._get(input)
if action == "save":
return self._save(input)
if action == "delete":
return self._delete(input)
if action == "duplicate":
return self._duplicate(input)
if action == "scope_info":
return self._scope_info(input)
if action == "resolve":
return await self._resolve(input)
return Response(status=400, response=f"Unknown action: {action}")
def _list_effective(self, input: dict) -> dict | Response:
context_scope = commands_helper.get_context_scope(str(input.get("context_id", "") or ""))
commands, scope = commands_helper.list_effective_commands(
project_name=context_scope["project_name"],
)
commands = [
command
for command in commands
if not command.get("frontmatter_extra", {}).get("webui_hidden")
]
return {
"ok": True,
"commands": commands,
"scope": scope,
}
def _list_scope(self, input: dict) -> dict | Response:
commands, scope = commands_helper.list_scope_commands(
project_name=str(input.get("project_name", "") or ""),
)
return {
"ok": True,
"commands": commands,
"builtin_commands": commands_helper.list_builtin_commands(),
"scope": scope,
}
def _get(self, input: dict) -> dict | Response:
path = str(input.get("path", "") or "")
if not path:
return Response(status=400, response="Missing path")
try:
command = commands_helper.get_command(
path,
project_name=str(input.get("project_name", "") or ""),
)
except FileNotFoundError:
return Response(status=404, response="Command not found")
except ValueError as error:
return Response(status=400, response=str(error))
return {"ok": True, "command": command}
def _save(self, input: dict) -> dict | Response:
try:
command = commands_helper.save_command(
project_name=str(input.get("project_name", "") or ""),
existing_path=str(input.get("existing_path", "") or ""),
name=str(input.get("name", "") or ""),
description=str(input.get("description", "") or ""),
argument_hint=str(input.get("argument_hint", "") or ""),
command_type=str(input.get("command_type", "text") or "text"),
body=str(input.get("body", "") or ""),
include_history=bool(input.get("include_history", False)),
extra_frontmatter=input.get("extra_frontmatter", {}) or {},
)
except FileExistsError as error:
return Response(status=409, response=str(error))
except ValueError as error:
return Response(status=400, response=str(error))
return {"ok": True, "command": command}
def _delete(self, input: dict) -> dict | Response:
path = str(input.get("path", "") or "")
if not path:
return Response(status=400, response="Missing path")
try:
commands_helper.delete_command(
path,
project_name=str(input.get("project_name", "") or ""),
)
except FileNotFoundError:
return Response(status=404, response="Command not found")
except ValueError as error:
return Response(status=400, response=str(error))
return {"ok": True}
def _duplicate(self, input: dict) -> dict | Response:
path = str(input.get("path", "") or "")
if not path:
return Response(status=400, response="Missing path")
try:
command = commands_helper.duplicate_command(
path,
project_name=str(input.get("project_name", "") or ""),
)
except FileNotFoundError:
return Response(status=404, response="Command not found")
except ValueError as error:
return Response(status=400, response=str(error))
return {"ok": True, "command": command}
def _scope_info(self, input: dict) -> dict | Response:
explicit_project = str(input.get("project_name", "") or "")
context_scope = commands_helper.get_context_scope(str(input.get("context_id", "") or ""))
project_name = explicit_project if "project_name" in input else context_scope["project_name"]
scope = commands_helper.get_scope_payload(
project_name=project_name,
ensure_directory=bool(input.get("ensure_directory", False)),
)
return {
"ok": True,
"scope": commands_helper.strip_private_scope(scope),
"context_scope": context_scope,
}
async def _resolve(self, input: dict) -> dict | Response:
path = str(input.get("path", "") or "")
if not path:
return Response(status=400, response="Missing path")
slash_text = str(input.get("slash_text", "") or "")
if not slash_text:
return Response(status=400, response="Missing slash_text")
try:
resolution = await commands_helper.resolve_command_invocation(
path=path,
slash_text=slash_text,
project_name=str(input.get("project_name", "") or ""),
context_id=str(input.get("context_id", "") or ""),
)
except FileNotFoundError:
return Response(status=404, response="Command not found")
except ValueError as error:
return Response(status=400, response=str(error))
return {"ok": True, "resolution": resolution}

View file

@ -0,0 +1,5 @@
name: attach
description: Attach local image file(s) to the next message.
argument_hint: Choose local file(s) from the WebUI picker.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,5 @@
name: browser
description: Choose Browser host/container mode and manage host-browser control.
argument_hint: "[host|container|status]"
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,5 @@
name: chat
description: Switch to a chat context by id.
argument_hint: "<context_id>"
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,6 @@
name: chats
description: List previous chats (default sorted by last updated). Use --project to filter by active project.
argument_hint: "[--project|--all-projects] [--sort=updated|created|name]"
type: script
script_path: connector_commands.py
webui_hidden: true

View file

@ -0,0 +1,4 @@
name: clear
description: Clear the visible chat log.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: compact
description: Open the connector-backed compaction confirmation flow.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,5 @@
name: computer-use
description: Turn local Computer Use on or off.
argument_hint: "[on|off|status]"
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,305 @@
from __future__ import annotations
from typing import Any
from agent import AgentContext
from helpers import message_queue as mq
from helpers import plugins, projects
from helpers.integration_commands import try_handle_command
from helpers.state_monitor_integration import mark_dirty_for_context
CLI_ONLY = {
"quit": "Quit is an A0 CLI shell command. Close this browser tab or stop the WebUI session when you are done.",
}
def run(payload: dict[str, Any]) -> dict[str, Any]:
invocation = payload.get("invocation") or {}
raw_name = str(invocation.get("command_name") or "").strip().lower()
command = raw_name
raw_args = str(invocation.get("raw_arguments") or "").strip()
arguments = invocation.get("arguments") if isinstance(invocation.get("arguments"), dict) else {}
context_id = str((payload.get("context") or {}).get("context_id") or "").strip()
context = _context(context_id)
if command == "new":
return _effects(_toast("Created a new chat."), {"type": "new_chat"})
if command == "chat":
return _handle_chat(arguments)
if command == "chats":
return _show_markdown("Chats", _chat_list(context, arguments))
if command == "clear":
return _effects(_toast("Visible transcript cleared."), {"type": "clear_transcript"})
if command == "project":
return _handle_project(context, raw_args)
if command == "profile":
return _handle_profile(context, raw_args)
if command == "plugins":
return _effects({"type": "open_modal", "path": "/components/plugins/list/plugin-list.html"})
if command == "compact":
return _effects({"type": "compact_chat"})
if command == "pause":
return _effects(_toast("Pause requested."), {"type": "pause_agent", "paused": True})
if command == "resume":
return _effects(_toast("Resume requested."), {"type": "pause_agent", "paused": False})
if command == "nudge":
return _effects(_toast("Nudge sent."), {"type": "nudge_agent"})
if command == "send":
return _handle_queue(context, ["send"])
if command == "queue":
return _handle_queue(context, list(arguments.get("tokens") or []))
if command == "presets":
return _effects({"type": "open_modal", "path": "/plugins/_model_config/webui/main.html"})
if command == "models":
return _handle_models(context, raw_args)
if command == "browser":
return _handle_browser(context, raw_args)
if command == "attach":
return _effects({"type": "attach_files"})
if command == "computer-use":
return _show_markdown("Computer Use", _computer_use_status(context_id, raw_args))
if command == "copy":
return _effects({"type": "copy_transcript"})
if command == "status":
return _show_markdown("Status", _status(context))
if command in CLI_ONLY:
return _effects(_toast(CLI_ONLY[command], level="info"))
return _effects(_toast(f"Unknown command: /{raw_name or command}", level="error"))
def _context(context_id: str) -> AgentContext | None:
if context_id:
return AgentContext.get(context_id)
return AgentContext.current() or AgentContext.first()
def _require_context(context: AgentContext | None) -> str | None:
if context:
return None
return "Open or create a chat context first."
def _effects(*effects: dict[str, Any]) -> dict[str, Any]:
return {"text": "", "effects": [effect for effect in effects if effect]}
def _toast(message: str, *, level: str = "success") -> dict[str, Any]:
return {"type": "toast", "message": message, "level": level}
def _show_markdown(title: str, content: str) -> dict[str, Any]:
return _effects({"type": "show_markdown", "title": title, "content": content})
def _handle_chat(arguments: dict[str, Any]) -> dict[str, Any]:
selector = str((arguments.get("positional") or [""])[0] or "").strip()
if not selector:
return _effects(_toast("Usage: /chat <context_id>", level="error"))
if not AgentContext.get(selector):
return _effects(_toast(f"Chat context '{selector}' was not found.", level="error"))
return _effects(_toast(f"Switched to {selector}."), {"type": "select_chat", "context_id": selector})
def _handle_project(context: AgentContext | None, raw_args: str) -> dict[str, Any]:
if not raw_args:
return _effects({"type": "open_modal", "path": "/components/projects/project-list.html"})
error = _require_context(context)
if error:
return _effects(_toast(error, level="error"))
return _show_markdown("Project", try_handle_command(context, f"/project {raw_args}") or "")
def _handle_profile(context: AgentContext | None, raw_args: str) -> dict[str, Any]:
if not raw_args:
return _effects({"type": "open_modal", "path": "/components/settings/settings.html"})
error = _require_context(context)
if error:
return _effects(_toast(error, level="error"))
return _show_markdown("Agent Profile", try_handle_command(context, f"/agent {raw_args}") or "")
def _handle_models(context: AgentContext | None, raw_args: str) -> dict[str, Any]:
return _effects({"type": "open_plugin_config", "plugin": "_model_config"})
def _handle_browser(context: AgentContext | None, raw_args: str) -> dict[str, Any]:
args = raw_args.strip().lower().replace("-", "_").split()
action = args[0] if args else ""
if not action:
return _effects({"type": "open_modal", "path": "/plugins/_browser/webui/main.html"})
if action in {"status", "state"}:
return _show_markdown("Browser", _browser_status(context))
if action not in {"host", "container", "docker"}:
return _effects(_toast("Usage: /browser [host|container|status]", level="error"))
project_name = projects.get_context_project_name(context) if context else ""
settings = plugins.get_plugin_config("_browser", project_name=project_name or "", agent_profile="") or {}
settings["runtime_backend"] = "host_required" if action == "host" else "container"
plugins.save_plugin_config("_browser", project_name or "", "", settings)
if context:
mark_dirty_for_context(context.id, reason="plugins._commands.browser_runtime")
label = "Host browser through A0 CLI" if settings["runtime_backend"] == "host_required" else "Internal Docker browser"
return _effects(_toast(f"Browser runtime set to {label}."))
def _browser_status(context: AgentContext | None) -> str:
project_name = projects.get_context_project_name(context) if context else ""
settings = plugins.get_plugin_config("_browser", project_name=project_name or "", agent_profile="") or {}
runtime = str(settings.get("runtime_backend") or "container")
label = "Host browser through A0 CLI" if runtime == "host_required" else "Internal Docker browser"
return f"Browser runtime: {label}\n\nUse `/browser host` or `/browser container` to switch."
def _handle_queue(context: AgentContext | None, tokens: list[str]) -> dict[str, Any]:
error = _require_context(context)
if error:
return _effects(_toast(error, level="error"))
queue = mq.get_queue(context)
if not tokens:
return _show_markdown("Queue", _queue_summary(queue))
action = str(tokens[0] or "").lower()
if action in {"send", "all", "flush"}:
if not queue:
return _effects(_toast("No queued messages."))
sent_count = mq.send_all_aggregated(context)
mark_dirty_for_context(context.id, reason="plugins._commands.queue_send")
noun = "message" if sent_count == 1 else "messages"
return _effects(_toast(f"Sent {sent_count} queued {noun}."))
if action in {"clear", "delete"} and len(tokens) == 1:
mq.remove(context)
mark_dirty_for_context(context.id, reason="plugins._commands.queue_clear")
return _effects(_toast("Queue cleared."))
if action in {"remove", "rm", "delete"}:
if len(tokens) < 2:
return _effects(_toast("Usage: /queue remove <number|id>", level="error"))
item_id = _queue_selector_to_id(queue, str(tokens[1]))
if not item_id:
return _effects(_toast(f"No queued message matches '{tokens[1]}'.", level="error"))
mq.remove(context, item_id)
mark_dirty_for_context(context.id, reason="plugins._commands.queue_remove")
return _effects(_toast("Queued message removed."))
return _effects(_toast("Usage: /queue [send|clear|remove <number|id>]", level="error"))
def _queue_summary(queue: list[dict[str, Any]]) -> str:
if not queue:
return "No queued messages."
lines = [f"Queued messages ({len(queue)}):"]
for index, item in enumerate(queue, start=1):
text = str(item.get("text") or "").strip() or "(attachment only)"
if len(text) > 100:
text = text[:97].rstrip() + "..."
attachments = item.get("attachments") or []
suffix = f" [{len(attachments)} files]" if attachments else ""
lines.append(f"{index}. {text}{suffix}")
return "\n".join(lines)
def _queue_selector_to_id(queue: list[dict[str, Any]], selector: str) -> str:
value = selector.strip()
if value.isdigit():
index = int(value) - 1
if 0 <= index < len(queue):
return str(queue[index].get("id") or "")
return ""
return value
def _chat_list(context: AgentContext | None, arguments: dict[str, Any]) -> str:
items = list(AgentContext.all())
flags = arguments.get("flags") or {}
active_project_only = bool(flags.get("project") or flags.get("active_project") or flags.get("p"))
sort_by = str(flags.get("sort") or "").lower()
positional = [str(item).lower() for item in (arguments.get("positional") or [])]
if not sort_by:
sort_by = next((item for item in positional if item in {"updated", "created", "name"}), "updated")
if sort_by not in {"updated", "created", "name"}:
return "Usage: /chats [--project|--all-projects] [--sort=updated|created|name]"
if active_project_only and context:
project_name = projects.get_context_project_name(context) or ""
items = [item for item in items if (projects.get_context_project_name(item) or "") == project_name]
def sort_key(item: AgentContext) -> Any:
output = item.output()
if sort_by == "name":
return (item.name or item.id).casefold()
if sort_by == "created":
return str(output.get("created_at") or "")
return str(output.get("last_message") or output.get("created_at") or "")
items = sorted(items, key=sort_key, reverse=sort_by != "name")
if not items:
return "No chats found."
lines = ["| Chat | Context | State |", "| --- | --- | --- |"]
for item in items[:30]:
marker = "current" if context and item.id == context.id else ("running" if item.is_running() else "idle")
lines.append(f"| {_escape_cell(item.name or item.id)} | `{item.id}` | {marker} |")
if len(items) > 30:
lines.append(f"\nShowing 30 of {len(items)} chats.")
return "\n".join(lines)
def _status(context: AgentContext | None) -> str:
error = _require_context(context)
if error:
return error
project_name = projects.get_context_project_name(context) or "none"
profile = getattr(context.agent0.config, "profile", "default") if context.agent0 else "default"
running = "running" if context.is_running() else "idle"
if getattr(context, "paused", False):
running = "paused"
return "\n".join(
[
f"Context: `{context.id}`",
f"State: {running}",
f"Project: {project_name}",
f"Agent profile: {profile}",
f"Queued messages: {len(mq.get_queue(context))}",
]
)
def _computer_use_status(context_id: str, raw_args: str) -> str:
from plugins._a0_connector.helpers import ws_runtime
action = "-".join(part.strip().lower().replace("_", "-") for part in raw_args.split()) or "status"
sids = ws_runtime.remote_tool_sids_for_context(context_id) if context_id else sorted(ws_runtime.connected_sids())
if not sids:
return (
"No A0 CLI is connected to this WebUI session.\n\n"
"Computer Use requires the CLI because the desktop permission prompt and native backend live on the CLI host. "
"Start A0 CLI, connect it to this Agent Zero instance, then run `/computer-use on` in the CLI."
)
if action in {"on", "off", "enable", "disable", "enabled", "disabled", "true", "false", "yes", "no", "1", "0"}:
return (
"Computer Use must be armed from the connected A0 CLI because it controls local desktop permissions.\n\n"
"Run `/computer-use on` or `/computer-use off` in the CLI terminal."
)
lines = ["Connected A0 CLI sessions:"]
for sid in sids:
metadata = ws_runtime.computer_use_metadata_for_sid(sid) or {}
if not metadata:
lines.append(f"- `{sid}`: connected, but not advertising Computer Use metadata.")
continue
state = "enabled" if metadata.get("enabled") else "disabled"
supported = "supported" if metadata.get("supported") else "unsupported"
status = str(metadata.get("status") or "unknown")
detail = str(metadata.get("last_error") or metadata.get("support_reason") or "").strip()
suffix = f" ({detail})" if detail else ""
lines.append(f"- `{sid}`: {state}, {supported}, status: {status}{suffix}")
lines.append("\nUse `/computer-use on|off|status` in the CLI to change local Computer Use.")
return "\n".join(lines)
def _escape_cell(value: str) -> str:
return str(value).replace("|", "\\|")

View file

@ -0,0 +1,4 @@
name: copy
description: Copy the currently visible transcript text to the clipboard.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: models
description: Open Main/Utility model runtime editor.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: new
description: Create a brand-new empty chat context.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: nudge
description: Nudge the current agent run.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: pause
description: Pause the active agent run.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: plugins
description: Open the installed-only Agent Zero plugin toggle view.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: presets
description: Open preset picker with Main/Utility model details.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,5 @@
name: profile
description: Pick or set the active Agent Zero Core profile.
argument_hint: "[profile]"
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,5 @@
name: project
description: Open the project menu, or switch directly with /project <name>.
argument_hint: "[name]"
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,5 @@
name: queue
description: Show, send, clear, or remove queued messages.
argument_hint: "[send|clear|remove <number|id>]"
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: quit
description: Disconnect and exit the CLI.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: resume
description: Resume a paused agent run.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: send
description: Send all queued messages now.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,4 @@
name: status
description: Show this chat's project, model, agent, and queue state.
type: script
script_path: connector_commands.py

View file

@ -0,0 +1,124 @@
from __future__ import annotations
import shutil
from pathlib import Path
from typing import Any
from helpers import cache, files, plugins
from helpers.extension import Extension
from helpers.print_style import PrintStyle
LEGACY_PLUGIN_NAME = "commands"
PLUGIN_NAME = "_commands"
COMMANDS_DIR = "commands"
SKILLS_DIR = "skills"
class LegacyCommandsMigration(Extension):
def execute(self, **kwargs):
result = migrate_legacy_commands()
if result["copied_commands"] or result["copied_skills"] or result["disabled_roots"]:
PrintStyle.info("Migrated legacy commands plugin data:", result)
def migrate_legacy_commands(base_dir: str | Path | None = None) -> dict[str, Any]:
root = Path(base_dir or files.get_abs_path("")).resolve()
result: dict[str, Any] = {
"copied_commands": 0,
"copied_skills": 0,
"disabled_roots": 0,
}
legacy_plugin_dir = root / "usr" / "plugins" / LEGACY_PLUGIN_NAME
if not legacy_plugin_dir.exists() and not _legacy_scoped_plugin_dirs(root):
return result
for legacy_commands_dir in _legacy_command_dirs(root):
target = _replace_plugin_segment(legacy_commands_dir, PLUGIN_NAME)
result["copied_commands"] += _copy_tree_files(legacy_commands_dir, target)
result["copied_skills"] += _copy_tree_files(
legacy_plugin_dir / SKILLS_DIR,
root / "usr" / "plugins" / PLUGIN_NAME / SKILLS_DIR,
)
for legacy_plugin_root in _legacy_plugin_roots(root):
if _disable_legacy_plugin_root(legacy_plugin_root):
result["disabled_roots"] += 1
if result["disabled_roots"]:
_clear_runtime_caches()
return result
def _legacy_command_dirs(root: Path) -> list[Path]:
return [
plugin_root / COMMANDS_DIR
for plugin_root in _legacy_plugin_roots(root)
if (plugin_root / COMMANDS_DIR).is_dir()
]
def _legacy_plugin_roots(root: Path) -> list[Path]:
roots = []
for candidate in [
root / "usr" / "plugins" / LEGACY_PLUGIN_NAME,
*root.glob(f"usr/projects/*/.a0proj/plugins/{LEGACY_PLUGIN_NAME}"),
*root.glob(f"usr/projects/*/.a0proj/agents/*/plugins/{LEGACY_PLUGIN_NAME}"),
*root.glob(f"usr/agents/*/plugins/{LEGACY_PLUGIN_NAME}"),
]:
if candidate.exists() and candidate not in roots:
roots.append(candidate)
return roots
def _legacy_scoped_plugin_dirs(root: Path) -> list[Path]:
return [
*root.glob(f"usr/projects/*/.a0proj/plugins/{LEGACY_PLUGIN_NAME}"),
*root.glob(f"usr/projects/*/.a0proj/agents/*/plugins/{LEGACY_PLUGIN_NAME}"),
*root.glob(f"usr/agents/*/plugins/{LEGACY_PLUGIN_NAME}"),
]
def _replace_plugin_segment(path: Path, plugin_name: str) -> Path:
parts = list(path.parts)
for index in range(len(parts) - 1):
if parts[index] == "plugins" and parts[index + 1] == LEGACY_PLUGIN_NAME:
parts[index + 1] = plugin_name
return Path(*parts)
return path
def _copy_tree_files(source: Path, target: Path) -> int:
if not source.is_dir():
return 0
copied = 0
for source_file in source.rglob("*"):
if not source_file.is_file():
continue
relative_path = source_file.relative_to(source)
target_file = target / relative_path
if target_file.exists():
continue
target_file.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(source_file, target_file)
copied += 1
return copied
def _disable_legacy_plugin_root(plugin_root: Path) -> bool:
plugin_root.mkdir(parents=True, exist_ok=True)
enabled_file = plugin_root / plugins.ENABLED_FILE_NAME
disabled_file = plugin_root / plugins.DISABLED_FILE_NAME
changed = enabled_file.exists() or not disabled_file.exists()
enabled_file.unlink(missing_ok=True)
disabled_file.write_text("", encoding="utf-8")
return changed
def _clear_runtime_caches() -> None:
cache.clear("*(plugins)*")
cache.clear("*(extensions)*")

View file

@ -0,0 +1,192 @@
<html>
<head>
<script type="module">
import { store } from "/plugins/_commands/webui/commands-slash-store.js";
</script>
</head>
<body>
<div x-data>
<template x-if="$store.commandsSlash">
<div class="commands-slash-menu-root" x-create="$store.commandsSlash.onMount()" x-destroy="$store.commandsSlash.cleanup()">
<div class="commands-slash-menu" x-show="$store.commandsSlash.menuVisible" x-transition.opacity.duration.120ms>
<template x-if="$store.commandsSlash.loading">
<div class="commands-slash-loading">
<span class="material-symbols-outlined spinning">progress_activity</span>
<span>Loading slash commands...</span>
</div>
</template>
<template x-if="!$store.commandsSlash.loading && $store.commandsSlash.filteredCommands.length > 0">
<div class="commands-slash-results">
<template x-for="(command, index) in $store.commandsSlash.filteredCommands" :key="command.path">
<button type="button"
class="commands-slash-item"
:class="{ active: index === $store.commandsSlash.selectedIndex }"
@mouseenter="$store.commandsSlash.selectedIndex = index"
@mousedown.prevent
@click.prevent="$store.commandsSlash.applySelection(command)">
<div class="commands-slash-item-header">
<div class="commands-slash-item-name">
<span class="commands-slash-prefix">/</span><span x-text="command.name"></span>
</div>
<span class="commands-slash-scope" x-text="command.source_scope_label"></span>
</div>
<div class="commands-slash-item-description" x-text="command.description"></div>
<template x-if="command.argument_hint">
<div class="commands-slash-item-hint" x-text="command.argument_hint"></div>
</template>
</button>
</template>
</div>
</template>
<template x-if="!$store.commandsSlash.loading && $store.commandsSlash.filteredCommands.length === 0">
<div class="commands-slash-empty">
<div class="commands-slash-empty-copy">
No matching slash commands.
</div>
<button type="button"
class="commands-slash-create"
@mousedown.prevent
@click.prevent="$store.commandsSlash.openCreateCommand()">
<span class="material-symbols-outlined">add</span>
<span x-text="$store.commandsSlash.emptyStateLabel"></span>
</button>
</div>
</template>
</div>
</div>
</template>
</div>
<style>
.commands-slash-menu-root {
position: relative;
z-index: 30;
}
.commands-slash-menu {
margin-bottom: 0.55rem;
border: 1px solid var(--color-border);
border-radius: 8px;
background: color-mix(in srgb, var(--color-panel) 96%, var(--color-background));
box-shadow: 0 14px 32px rgba(0, 0, 0, 0.14);
overflow: hidden;
}
.commands-slash-results {
max-height: 18rem;
overflow-y: auto;
}
.commands-slash-item {
display: flex;
flex-direction: column;
gap: 0.32rem;
width: 100%;
padding: var(--spacing-sm) 0.8rem;
border: 0;
border-bottom: 1px solid color-mix(in srgb, var(--color-border) 60%, transparent);
background: transparent;
color: inherit;
text-align: left;
cursor: pointer;
}
.commands-slash-item:last-child {
border-bottom: 0;
}
.commands-slash-item.active,
.commands-slash-item:hover {
background: color-mix(in srgb, var(--color-highlight) 10%, transparent);
}
.commands-slash-item-header {
display: flex;
justify-content: space-between;
gap: 0.75rem;
align-items: center;
}
.commands-slash-item-name {
font-weight: 600;
}
.commands-slash-prefix {
color: var(--color-highlight);
}
.commands-slash-scope {
padding: 0.18rem 0.45rem;
border-radius: 999px;
background: color-mix(in srgb, var(--color-highlight) 12%, transparent);
font-size: 0.72rem;
font-weight: 600;
white-space: nowrap;
}
.commands-slash-item-description {
color: var(--color-text-secondary);
font-size: 0.86rem;
line-height: 1.4;
}
.commands-slash-item-hint {
color: var(--color-text-secondary);
font-family: "Roboto Mono", monospace;
font-size: 0.77rem;
opacity: 0.85;
}
.commands-slash-empty,
.commands-slash-loading {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.75rem;
padding: 0.9rem 1rem;
}
.commands-slash-empty-copy,
.commands-slash-loading {
color: var(--color-text-secondary);
}
.commands-slash-create {
display: inline-flex;
align-items: center;
gap: 0.35rem;
padding: 0.5rem 0.75rem;
border: 1px solid var(--color-border);
border-radius: 999px;
background: transparent;
color: var(--color-text);
cursor: pointer;
font-weight: 600;
white-space: nowrap;
}
.spinning {
animation: commands-slash-spin 1s linear infinite;
}
@keyframes commands-slash-spin {
from { transform: rotate(0deg); }
to { transform: rotate(360deg); }
}
@media (max-width: 640px) {
.commands-slash-empty,
.commands-slash-loading {
flex-direction: column;
align-items: stretch;
}
.commands-slash-create {
justify-content: center;
}
}
</style>
</body>
</html>

View file

@ -0,0 +1 @@
"""Helpers for the commands plugin."""

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,8 @@
name: _commands
title: Commands
description: YAML-configured slash commands with text templates or Python hooks.
version: 0.5.0
settings_sections: []
per_project_config: false
per_agent_config: false
always_enabled: false

View file

@ -0,0 +1,72 @@
---
name: commands-create-slash-command
description: Create or update Agent Zero slash commands for the built-in Commands plugin. Use when the user asks to add, edit, duplicate, or refine a reusable /command backed by YAML config plus text/python content files.
version: 1.0.0
tags: ["commands", "slash-commands", "plugin", "yaml", "python", "templates"]
triggers:
- create slash command
- add slash command
- update slash command
- edit slash command
- commands plugin
---
# Commands Plugin Slash Command Authoring
Use this skill when the user wants a reusable `/command` for Agent Zero's built-in `_commands` plugin.
## Source Of Truth
- Slash commands are file-backed, not database rows.
- Each command uses:
- one config file: `<slug>.command.yaml`
- one content file:
- text template: `<slug>.txt`, or
- python hook: `<slug>.py`
- Required config keys:
- `name`
- `description`
- `type` (`text` or `script`)
- Optional config keys:
- `argument_hint`
- `include_history` (script commands)
- Preserve unknown config keys when editing existing commands.
## Scope Resolution
Choose the target folder from the requested scope:
- Project: `usr/projects/<project>/.a0proj/plugins/_commands/commands/`
- Global fallback: `usr/plugins/_commands/commands/`
If the user does not specify a scope, prefer the active chat scope when it is clear. Otherwise use the global scope.
## File Rules
- Config file format: `<slug>.command.yaml`
- Slash command name should be lowercase and hyphenated, for example `explain-code`
- For text commands, keep the `.txt` template concise and directly reusable
- For script commands, implement `run(payload)` in the `.py` file
- If the command expects trailing input, use `{raw}`, `{args.positional.0}`, or `{args.flags.some_flag}`
Use the bundled templates in `template.command.yaml` and `template.command.txt` when creating a new text command from scratch.
## Editing Workflow
1. Determine scope and final slash command name.
2. Check whether a command file already exists in that scope.
3. If it exists, load the file first and preserve unknown frontmatter keys.
4. Update YAML config and template/script content.
5. Save the file in the correct scope folder.
6. Report:
- the saved config path
- the saved content path
- the slash command name in `/name` form
## Output Contract
After saving, explicitly state the final file path and the exact slash command invocation, for example:
- `Saved config: /a0/usr/plugins/_commands/commands/explain-code.command.yaml`
- `Saved content: /a0/usr/plugins/_commands/commands/explain-code.txt`
- `Invoke with: /explain-code`

View file

@ -0,0 +1,3 @@
Describe the work to perform here.
{raw}

View file

@ -0,0 +1,5 @@
name: example-command
description: Briefly describe what this slash command does.
argument_hint: Optional free-form text after /example-command
type: text
template_path: example-command.txt

Some files were not shown because too many files have changed in this diff Show more