mirror of
https://github.com/agent0ai/agent-zero.git
synced 2026-10-19 07:23:51 +00:00
fix(computer-use): start host sessions on the first action
Start a missing chat session and resume the requested desktop action within the same tool call. Limit recovery to one session-required response on the same host and context, preserving approval checks and explicit session IDs. Update host prompts to begin with an observation and add Linux X11/XWayland guidance for verified focus, viewer scaling, popup handling, slow input, and visual verification. Keep the internal Docker/Xpra desktop separate. Validation: 26 Core prompt/metadata checks passed locally and in the live framework environment; 42 focused connector tool tests passed. Native GUI acceptance was not rerun because host Computer Use was disabled.
This commit is contained in:
Alessandro
parent
60c940fd03
commit
4dceb49211
9 changed files with 105 additions and 16 deletions
|
|
@ -28,6 +28,8 @@
|
|||
access, timeout, transfer, and disconnect handling; ordinary model args cannot
|
||||
enable `allow_running`. There is no `runtime=input` or `code` input alias.
|
||||
- Computer Use prompts keep `element_index` top-level and semantic `target` syntax in backend skills. macOS `app-scoped-semantic-targeting` enables explicit app/window scope; older clients use the indexed window workflow. Native `get_window_state.mode` only labels output and stays out of model-facing guidance.
|
||||
- `computer_use_remote` starts a missing chat session inside the first requested action only after the host returns `COMPUTER_USE_SESSION_REQUIRED` before dispatch. Start once on the same socket/context and resume with the returned session ID and a fresh operation ID. Never auto-replace an explicit session binding, retry permission/ambiguous failures, or start from `status`/`stop_session`; keep existing host enablement, trust, and platform approval checks. Attach the requested action's capture, not an extra startup capture.
|
||||
- Host Computer Use skills begin with `list_windows` or `capture`. Linux X11/XWayland viewer guidance lives in `skills/host-computer-use-linux/references/x11-viewers.md` and requires independently enabled host code execution for shell techniques. Permission denials remain stop conditions; the internal Docker/Xpra desktop and standalone X11 backend selection are unchanged.
|
||||
- Agent-scoped skill discovery includes this plugin's `skills/` root only when a routed connected socket has connector capability metadata. WebUI-only sockets do not qualify; disabled capabilities still count as a connected CLI. Filter roots on each discovery without mutating cached paths or persisting visibility settings. `setup-a0-cli` lives in root `skills/` and stays discoverable while disconnected.
|
||||
- Never re-add a connector prompt that the effective project/profile tool policy
|
||||
blocks.
|
||||
|
|
@ -104,6 +106,7 @@
|
|||
## Verification
|
||||
|
||||
- Run connector-specific tests or smoke-test HTTP and `/ws` integration when changing runtime behavior.
|
||||
- Cross-platform first-call session startup and no-retry boundaries are covered in the sibling connector's `tests/test_plugin_backend.py`; Core prompt gating remains covered by `tests/test_a0_connector_prompt_gating.py`.
|
||||
- Launcher gateway regression coverage lives in
|
||||
`tests/test_a0_connector_launcher_gateway.py`.
|
||||
|
||||
|
|
|
|||
|
|
@ -5,15 +5,15 @@ Shown when a connected A0 CLI or Launcher host gateway advertises enabled Comput
|
|||
|
||||
Use this for native host desktop UI inspection, screenshots, background-safe window/element actions when supported, clicking, scrolling, typing, key presses, and status checks. Do not use it for ordinary web-page navigation or host-browser control; use the browser tool for web pages unless browser automation cannot express the task. For complex desktop workflows, load and follow skill `host-computer-use` before proceeding.
|
||||
|
||||
This is the only desktop-control path for the user's connected host/local computer. Do not substitute the `linux-desktop` skill, the Agent Zero Desktop/Xpra surface, `desktopctl.sh`, `code_execution_tool`, or Docker/server shell commands for host screen actions; those target the internal Agent Zero runtime and cannot see or control the user's host screen.
|
||||
This is the primary desktop-control path for the user's connected host/local computer. Do not substitute the `linux-desktop` skill, the Agent Zero Desktop/Xpra surface, `desktopctl.sh`, `code_execution_tool`, or Docker/server shell commands for host screen actions; those target the internal Agent Zero runtime and cannot see or control the user's host screen. For a Linux X11/XWayland app limitation during an authorized session, the Linux host skill documents a narrow host-shell workflow that also requires enabled host code execution; it never bypasses a disabled or denied Computer Use grant.
|
||||
|
||||
If the tool reports no CLI/Launcher host bridge, disabled computer use, or `COMPUTER_USE_REARM_REQUIRED`, stop and tell the user to run `/computer-use on` in the A0 Launcher chat when using Launcher Host access, or in A0 CLI otherwise, and approve any host permission prompt.
|
||||
|
||||
If a Computer Use call returns an error, do not repeat the same action with identical arguments. Report the error, or use a materially different safe recovery only when the task still requires it.
|
||||
|
||||
Call `start_session` before screen-driven tasks. Use `status` for state only, `capture` for screenshots without an action, and `stop_session` when the desktop task is complete. Read `backend_id`, `backend_family`, `features`, and the structured `capabilities` object in status/session results. When capabilities report native windows, window state, element indexes, and background dispatch, prefer `list_windows` -> `get_window_state` -> `element_action` with `dispatch: "background"` before using global coordinates. Interactive coordinate actions should use normalized global-screen coordinates from the most recent capture.
|
||||
Start with the needed observation: `list_windows` for native apps or `capture` for a screen. A missing chat session starts automatically within that call using the host's existing enablement and normal permission checks; do not make a separate `start_session` call for every chat. Existing sessions are reused. `start_session` remains available explicitly; `status` only reports state and `stop_session` stops the current chat's session. Read `backend_id`, `backend_family`, `features`, and the structured `capabilities` object in results. When capabilities report native windows, window state, element indexes, and background dispatch, prefer `list_windows` -> `get_window_state` -> `element_action` with `dispatch: "background"` before using global coordinates. Interactive coordinate actions should use normalized global-screen coordinates from the most recent capture.
|
||||
|
||||
Some actions are backend-specific and intentionally documented only in backend skills. If `status` or `start_session` reports backend-specific features or tells you to load a backend skill, load and follow that skill before using those backend-only actions. For structural targeting details, load and follow the backend-specific skill such as `host-computer-use-macos` or `host-computer-use-windows`; do not apply one backend's guidance to another backend.
|
||||
Some actions are backend-specific and intentionally documented only in backend skills. If a result reports backend-specific features or tells you to load a backend skill, load and follow that skill before using those backend-only actions. For structural targeting details, load and follow the backend-specific skill such as `host-computer-use-macos` or `host-computer-use-windows`; do not apply one backend's guidance to another backend.
|
||||
|
||||
State-changing actions automatically attach a fresh screen after they run unless the backend returns a definitive structural background result. Treat key presses, clicks, scrolling, and typing as attempts, not success; treat foreground fallbacks the same way. Inspect the latest attached screen, or one explicit `capture` if it is unclear or unchanged, before saying the requested outcome happened. If the tool says a screen was attached but you cannot actually inspect the image, stop and report that visual verification is unavailable; do not continue by assuming the host state. A `type` result proves only that keyboard events were sent unless it explicitly reports both `focus_verified=true` and the target `window_id`.
|
||||
|
||||
|
|
@ -23,7 +23,7 @@ When Linux advertises `verified-window-focus` and `target-verified-keyboard-inpu
|
|||
{
|
||||
"tool_name": "computer_use_remote",
|
||||
"tool_args": {
|
||||
"action": "status"
|
||||
"action": "list_windows"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
|
@ -32,7 +32,7 @@ Required argument:
|
|||
- `action`: one of `start_session`, `status`, `capture`, `list_windows`, `get_window_state`, `element_action`, `move`, `click`, `scroll`, `key`, `type`, `stop_session`; backend skills may document additional backend-only action values
|
||||
|
||||
Optional arguments by action:
|
||||
- `session_id`: session returned by `start_session`
|
||||
- `session_id`: optional explicit session binding; normally omit it to reuse this chat's session. An explicit stale/mismatched ID fails without automatic replacement.
|
||||
- `pid`, `window_id`: target a native window for `get_window_state`, `element_action`, scoped backend snapshots, and guarded keyboard input when supported
|
||||
- `element_index`: top-level index from the latest `get_window_state`
|
||||
- `target`: backend-specific semantic fields for `element_action`
|
||||
|
|
|
|||
|
|
@ -1,14 +1,16 @@
|
|||
---
|
||||
name: host-computer-use-linux
|
||||
description: Backend-specific Linux guidance for `computer_use_remote`. Load after `status` or `start_session` reports backend_family `linux`, backend_id `wayland`, or AT-SPI features. Covers AT-SPI structural targeting, Wayland portal caveats, and screenshot verification.
|
||||
description: Host Linux guidance for `computer_use_remote` when results report Linux, Wayland, X11, or AT-SPI features. Covers verified focus, AT-SPI targeting, XWayland app limitations, remote desktop viewers, and screenshot verification; excludes the internal Docker/Xpra desktop.
|
||||
---
|
||||
|
||||
# Host Computer Use - Linux
|
||||
|
||||
Use this after `host-computer-use` when the connected A0 CLI reports the Linux/Wayland computer-use backend.
|
||||
Use this after `host-computer-use` when the connected A0 CLI or Launcher gateway reports a Linux computer-use backend. Start with the needed `list_windows` or `capture` observation; the shared tool starts a missing chat session automatically. Use `status` only when you need to inspect authorization or capabilities.
|
||||
|
||||
Do not use this skill for macOS, Windows, Xpra, Docker, browser-only tasks, or the internal Agent Zero Desktop. If the backend is not Linux or does not advertise AT-SPI support, skip Linux structural actions and follow the generic host computer-use rules.
|
||||
|
||||
For an X11/XWayland app or remote desktop viewer with an incomplete accessibility tree, read [X11 and viewer workflows](references/x11-viewers.md). Use its host-shell techniques only while Computer Use is enabled and authorized and host code execution is independently enabled. A backend limitation is not a permission denial.
|
||||
|
||||
## Linux AT-SPI Targeting
|
||||
|
||||
Linux backends can advertise structural AT-SPI features:
|
||||
|
|
@ -67,7 +69,7 @@ Targeting options:
|
|||
- If an action reports ambiguity, take a fresh snapshot and narrow the target with role plus title/name/description.
|
||||
- If an action reports a missing target, take a fresh snapshot before trying coordinates.
|
||||
|
||||
## Wayland Notes
|
||||
## Focus and Observation
|
||||
|
||||
Use screenshots for proof after every state-changing action. AT-SPI actions and keyboard events are attempts, not proof, and Wayland focus can reject or redirect input when the active window changes.
|
||||
|
||||
|
|
@ -86,10 +88,16 @@ On GNOME/Wayland, useful shortcuts include:
|
|||
|
||||
Treat every shortcut as an attempt. Inspect the fresh screenshot before saying it worked. If text lands in the wrong app, stop and reassess from `capture` or `ax_snapshot`; do not continue typing from assumed focus.
|
||||
|
||||
A remote desktop viewer exposes the local viewer's controls, not necessarily the remote application's accessibility tree. Keep local window identity, viewer scaling, and the remote desktop separate. Use remote pixels only after inspecting the rendered viewer. Recalculate coordinates after resizing or changing scaling; local tool coordinates remain normalized to the host screen.
|
||||
|
||||
When a popup is already open, do not refocus its parent window; this may dismiss the popup. Inspect a fresh frame and select the actual control, such as a scaling radio button, rather than assuming a click on nearby text selected it.
|
||||
|
||||
Slow VNC viewers may drop fast typing and clicks. Wait for the resulting frame, verify field focus and masked character count, and use slower input if needed. Click confirmation fields explicitly when Tab does not move focus. A partial redraw or unchanged screenshot warrants one fresh capture before another action, not repeated credential submission.
|
||||
|
||||
Some apps expose shallow AT-SPI trees unless their own accessibility support is enabled. If the AT-SPI tree is too shallow for a task, fall back in this order: app-native/browser tooling, reliable keyboard paths, then normalized coordinate clicks from a fresh screenshot.
|
||||
|
||||
## Permissions
|
||||
|
||||
If `computer_use_remote` returns `COMPUTER_USE_AX_UNAVAILABLE`, `COMPUTER_USE_REARM_REQUIRED`, `COMPUTER_USE_APPROVAL_REQUIRED`, or `status=rearm required`, stop immediately and ask the user to re-arm or fix the Linux desktop accessibility/session state.
|
||||
If `computer_use_remote` returns `COMPUTER_USE_REARM_REQUIRED`, `COMPUTER_USE_APPROVAL_REQUIRED`, or `status=rearm required`, stop immediately and ask the user to re-arm or fix the Linux desktop session. If only `COMPUTER_USE_AX_UNAVAILABLE` is returned, stop structural actions and check the reported cause: use an already-authorized capture/input capability only for an accessibility availability limitation, never to evade an access denial.
|
||||
|
||||
Do not bypass a permission or host-visibility failure with server screenshots, Docker commands, the built-in Linux Desktop/Xpra skill, or `code_execution_tool`.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,47 @@
|
|||
# X11, XWayland, and remote desktop viewers
|
||||
|
||||
Use this for a native Linux app limitation during an already-authorized host Computer Use task. It does not enable the standalone X11 connector backend, switch the user's desktop session, or authorize another machine.
|
||||
|
||||
## Choose the right surface
|
||||
|
||||
- Prefer `computer_use_remote` and its advertised window/element capabilities. A Remmina or RustDesk accessibility tree may describe only the viewer chrome; the remote desktop inside it is a separate pixel surface.
|
||||
- If a necessary viewer operation is unsupported, existing X11 utilities may help through `code_execution_remote`. Load `host-code-execution`, confirm both host Computer Use and host code execution remain enabled, and use only the connected host shell. Do not use `code_execution_tool`, Docker, Xpra, or `desktopctl.sh`.
|
||||
- A disabled grant, missing bridge, re-arm requirement, screen-capture denial, or unverifiable target is a stop condition. Shell access does not override it. Do not install desktop-control dependencies or change session configuration just to escape such a failure.
|
||||
|
||||
## Discover, target, verify
|
||||
|
||||
Check available tools and actual desktop/window state through the host shell:
|
||||
|
||||
```bash
|
||||
printf 'Session: %s\n' "$XDG_SESSION_TYPE"
|
||||
command -v wmctrl xdotool xwininfo import
|
||||
wmctrl -lpGx
|
||||
```
|
||||
|
||||
On Wayland these X11 tools see XWayland clients, not every native Wayland window. Match PID, app class, and title to the intended app; never copy a window ID from an earlier task. If a new GTK viewer must expose an X11 window, `GDK_BACKEND=x11 remmina` or `GDK_BACKEND=x11 rustdesk` can select XWayland for that process. An existing singleton may receive the request instead. Inspect the resulting window; do not kill an unrelated instance or force the entire desktop to X11.
|
||||
|
||||
Use a verified ID for activation/resizing and coordinates relative to that window. In the following examples, `$window_id` must be the ID just discovered:
|
||||
|
||||
```bash
|
||||
xdotool windowactivate --sync "$window_id"
|
||||
xdotool getactivewindow
|
||||
xwininfo -id "$window_id"
|
||||
```
|
||||
|
||||
Compare the active ID numerically: `wmctrl` commonly prints hexadecimal, while `xdotool` prints decimal. Missing `_NET_ACTIVE_WINDOW` on Wayland is not confirmation of focus. Do not type into an unverified window. After activation, inspect the actual field or remote login dialog as well: correct viewer focus does not prove correct remote field focus.
|
||||
|
||||
Before ordinary typing, verify the active window again. For secrets, prefer the app's credential dialog and a supported secret input; if host-side Python supplies text to `xdotool`, pass it through subprocess stdin with `type --file -`, never as a command argument, literal in a saved helper, or shell-history entry. Do not print secrets, capture revealed passwords, or infer the destination from a successful process exit.
|
||||
|
||||
## Observe through the host tool
|
||||
|
||||
Use `computer_use_remote` `capture` for a viewable image before and after shell-driven input. Shell actions do not receive the tool's automatic screenshot. An app-only `import -window "$window_id" ...` capture is useful only when an authorized artifact-transfer or shared-path mechanism can actually deliver it for inspection. A host filename printed in terminal output is not visual evidence, and a host path must not be passed to a server-side image reader as though it were local.
|
||||
|
||||
Do not refocus a parent window while a native popup is open. That may dismiss the popup and make the next click hit the underlying content. Inspect the popup and click the actual radio button/checkbox, or use a verified keyboard path. Select adaptive scaling in a viewer when needed, then inspect the changed view before using new coordinates. Window pixels, scaled remote pixels, and normalized host-screen coordinates are different coordinate systems.
|
||||
|
||||
## Slow viewer input and completion
|
||||
|
||||
- Add a short pause after moving the pointer; if fast clicks are lost, hold the press briefly before release. If text is dropped, around 500 ms between characters worked in a slow VNC session. Use this as a recovery setting, not a global delay for every app.
|
||||
- Inspect masked field length before submission. Explicitly click a confirmation field if Tab does not reach it. Never fix an uncertain password attempt by repeatedly submitting the same input.
|
||||
- If wheel events barely move a pane, inspect its scrollbar and use a deliberate drag with pauses between press, movement, and release, or a reliable keyboard path. Do not diagnose a hung app from one stale frame.
|
||||
- Verify both pointer and keyboard with a harmless task, such as clicking `2` in Calculator, typing `+2`, and observing `4`. Close the test window afterward. Tool receipt success alone does not prove that the remote desktop received input.
|
||||
- Close only setup viewers/tabs created for this task; preserve the user's working session and other windows. Report what was tested without claiming improved latency or reboot persistence unless those were measured.
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
name: host-computer-use-macos
|
||||
description: Backend-specific macOS guidance for `computer_use_remote`. Load after `status` or `start_session` reports backend_family/backend_id `macos` or Accessibility-tree features. Covers AX structural targeting, macOS window actions, and screenshot verification.
|
||||
description: Backend-specific macOS guidance for `computer_use_remote`. Load when a tool result reports backend_family/backend_id `macos` or Accessibility-tree features. Covers AX structural targeting, macOS window actions, and screenshot verification.
|
||||
---
|
||||
|
||||
# Host Computer Use - macOS
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
name: host-computer-use-windows
|
||||
description: Backend-specific Windows guidance for `computer_use_remote`. Load after `status` or `start_session` reports backend_family/backend_id `windows` or Windows UI Automation features. Covers UIA structural targeting, Windows session caveats, and screenshot verification.
|
||||
description: Backend-specific Windows guidance for `computer_use_remote`. Load when a tool result reports backend_family/backend_id `windows` or Windows UI Automation features. Covers UIA structural targeting, Windows session caveats, and screenshot verification.
|
||||
---
|
||||
|
||||
# Host Computer Use - Windows
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ Use:
|
|||
{
|
||||
"tool_name": "computer_use_remote",
|
||||
"tool_args": {
|
||||
"action": "start_session"
|
||||
"action": "list_windows"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
|
@ -56,7 +56,7 @@ Use:
|
|||
Arguments:
|
||||
|
||||
- `action`: `start_session`, `status`, `capture`, `list_windows`, `get_window_state`, `element_action`, `move`, `click`, `scroll`, `key`, `type`, `stop_session`
|
||||
- `session_id`: optional after `start_session`
|
||||
- `session_id`: optional explicit binding; normally omit it to reuse the current chat's session. An explicit stale/mismatched ID is not automatically replaced.
|
||||
- backend skills may document additional backend-only action values; use them only when backend metadata advertises matching support and after loading the backend-specific skill
|
||||
- `list_windows`: returns native top-level window records when the backend supports them
|
||||
- `get_window_state`: pass `pid` and/or `window_id`; returns a target-window accessibility tree with stable `element_index` values for the current state
|
||||
|
|
@ -74,7 +74,7 @@ If any tool result contains `COMPUTER_USE_REARM_REQUIRED` or `status=rearm requi
|
|||
|
||||
## Core Loop
|
||||
|
||||
1. Call `start_session` first.
|
||||
1. Start with `list_windows` for a native app or `capture` for a screen. When this chat has no session, the tool starts it once and continues the request within the same call. Existing host enablement, trust mode, and platform approval checks still apply; no separate `start_session` step is needed per chat.
|
||||
2. Read the returned `backend_id`, `backend_family`, `features`, `contract_version`, and `capabilities`; load a backend-specific Computer Use skill when the task needs backend-only affordances.
|
||||
3. Prefer the structured `capabilities` object over guessing from OS names. Use `capabilities.identity.pid`, `capabilities.identity.window_id`, `capabilities.identity.element_index`, and `capabilities.dispatch.background` as the portable contract for the native background loop.
|
||||
4. If the backend advertises native window listing through capabilities or `native-window-list`, call `list_windows` before using coordinates.
|
||||
|
|
@ -89,7 +89,7 @@ If any tool result contains `COMPUTER_USE_REARM_REQUIRED` or `status=rearm requi
|
|||
|
||||
## Backend Skills
|
||||
|
||||
- If the backend is Linux/Wayland or features include `atspi-tree-snapshot` / `atspi-structural-targeting`, load `host-computer-use-linux` before using Linux AT-SPI structural actions.
|
||||
- If the backend is Linux/Wayland/X11 or features include `atspi-tree-snapshot` / `atspi-structural-targeting`, load `host-computer-use-linux` for Linux targeting and X11/XWayland app workflows. Its host-shell workflow additionally requires enabled host code execution; it cannot replace denied Computer Use access.
|
||||
- If the backend is macOS or features include `accessibility-tree-snapshot` / `accessibility-structural-targeting`, load `host-computer-use-macos` before using macOS structural Accessibility actions.
|
||||
- If the backend is Windows or features include `uia-tree-snapshot` / `uia-structural-targeting`, load `host-computer-use-windows` before using Windows UI Automation structural actions.
|
||||
- Do not use backend-specific actions just because their argument names exist in the generic contract. Treat them as unavailable unless the connected CLI advertises the matching feature.
|
||||
|
|
@ -105,6 +105,7 @@ If any tool result contains `COMPUTER_USE_REARM_REQUIRED` or `status=rearm requi
|
|||
- Prefer accessibility and semantic UI paths first: shortcuts, command palettes, menu accelerators, address/search bars, focus traversal, and other keyboard-accessible controls.
|
||||
- Prefer `key` and `type` over pointer actions whenever a reliable keyboard path exists.
|
||||
- When a menu or popup is open, treat it as the active UI and prefer keyboard navigation over clicking small transient rows by coordinate.
|
||||
- Do not reactivate the parent window while its popup is open: activation can dismiss the popup and send the next click to the underlying content. Inspect the popup and its actual selected control, including radio buttons, before acting.
|
||||
- If a click dismisses a menu or popup without producing the expected next UI, treat that attempt as failed.
|
||||
- If the same approach has already failed twice without visible progress, switch strategy instead of repeating it.
|
||||
- Do not infer focus or task completion from chat logs, sidebars, tool summaries, or status text.
|
||||
|
|
|
|||
|
|
@ -119,6 +119,34 @@ class ComputerUseRemote(Tool):
|
|||
try:
|
||||
payload = self._build_payload(op_id=str(uuid.uuid4()), context_id=context_id, action=action)
|
||||
result = await self._dispatch_payload(sid=sid, payload=payload)
|
||||
session_note = ""
|
||||
if (
|
||||
not result.get("ok")
|
||||
and result.get("code") == "COMPUTER_USE_SESSION_REQUIRED"
|
||||
and action not in {"start_session", "status", "stop_session"}
|
||||
and not payload.get("session_id")
|
||||
):
|
||||
# This rejection happens before input dispatch. Start once on the
|
||||
# same host/context; never retry a permission or ambiguous failure.
|
||||
started = await self._dispatch_payload(
|
||||
sid=sid,
|
||||
payload={
|
||||
"op_id": str(uuid.uuid4()),
|
||||
"context_id": context_id,
|
||||
"action": "start_session",
|
||||
},
|
||||
)
|
||||
if not started.get("ok"):
|
||||
return self._response(self._format_error(started))
|
||||
data = started.get("result")
|
||||
session_id = str(data.get("session_id") or "").strip() if isinstance(data, dict) else ""
|
||||
if not session_id:
|
||||
raise ValueError("Host started computer use without returning a session_id")
|
||||
session_note = self._extract_result("start_session", started)
|
||||
result = await self._dispatch_payload(
|
||||
sid=sid,
|
||||
payload={**payload, "op_id": str(uuid.uuid4()), "session_id": session_id},
|
||||
)
|
||||
capture_note = await self._maybe_attach_latest_capture(
|
||||
action=action,
|
||||
sid=sid,
|
||||
|
|
@ -126,6 +154,8 @@ class ComputerUseRemote(Tool):
|
|||
result=result,
|
||||
)
|
||||
message = self._extract_result(action, result)
|
||||
if session_note:
|
||||
message = f"{session_note}\n{message}"
|
||||
except ValueError as exc:
|
||||
return Response(
|
||||
message=f"computer_use_remote: {exc}",
|
||||
|
|
|
|||
|
|
@ -733,7 +733,7 @@ def test_host_computer_use_does_not_fall_back_to_linux_desktop_skill():
|
|||
linux_skill = linux_skill_path.read_text(encoding="utf-8")
|
||||
linux_frontmatter = _parse_skill_frontmatter(linux_skill_path)
|
||||
|
||||
assert "only desktop-control path for the user's connected host/local computer" in computer_stub
|
||||
assert "primary desktop-control path for the user's connected host/local computer" in computer_stub
|
||||
assert "Do not substitute the `linux-desktop` skill" in computer_stub
|
||||
assert "Never switch to `linux-desktop`" in host_skill
|
||||
assert "Those paths only see the internal Agent Zero runtime" in host_skill
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue