mirror of
https://github.com/wirenboard/agent-vm.git
synced 2026-07-09 16:00:54 +00:00
4eafe4bbac
Two coordinated changes that together raise the per-sandbox virtio
device cap on x86_64 by ~10x:
1. **Enable msb_krun's userspace split irqchip** in the runtime
(vendor/microsandbox submodule bump). KVM's in-kernel IOAPIC is
hardcoded to 24 pins, of which libkrun's allocator hands out only
IRQs 5..15 — saturated by a typical agent-vm config (rootfs, upper,
runtime fs, network, vsock, console, balloon, rng) plus a couple of
`--mount`s, so an extra mount has historically tripped
`RegisterNetDevice(IrqsExhausted)` at boot. The userspace IOAPIC
exposes 256 pins, lifting the libkrun allocator ceiling to
IRQ_MAX_SPLIT = 223. Requires a libkrun fork (pinned in
Cargo.toml) with three correctness fixes on top of upstream 0.1.13;
without them, enabling split_irqchip crashes the VMM during boot or
silently truncates the kernel cmdline.
2. **Patch libkrunfw to bump x86 COMMAND_LINE_SIZE 2048 → 16384**.
Each virtio_mmio device adds ~36 bytes of cmdline; past ~10 user
mounts the assembled cmdline crosses the stock 2048 cap, the kernel
silently truncates the tail (which includes virtio-console), and
the guest hangs in early boot with `kernel.log` stuck at 0 bytes.
New `libkrunfw-overrides/cmdline-size_x86_64.patch` lifts the cap;
CI workflow picks it up via libkrunfw's `patches/0*.patch` glob
(the `0999-overrides-` prefix sorts strictly after libkrunfw's own
numbered patches).
CI workflow hardening that landed alongside:
- cache key now keyed per-arch via `*_${ARCH}.patch` glob (an
aarch64 patch edit no longer invalidates the x86_64 cache);
- `config-libkrunfw*` skip pattern uses a bare suffix so future
`config-libkrunfw-tdx_${ARCH}.patch` / `-sev_` variants are also
excluded from the kernel source-patch pipeline.
User-facing knobs (`--mount` doc in run.rs, README troubleshooting
guide, ARCHITECTURE/PLAN narrative) updated to reflect the new
practical ceiling.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
173 lines
6.8 KiB
Markdown
173 lines
6.8 KiB
Markdown
# agent-vm
|
|
|
|
Run Claude Code / Codex / OpenCode inside a per-project libkrun microVM,
|
|
booting in ~2 seconds, with:
|
|
|
|
- **Host OAuth tokens never enter the VM.** The TLS-intercept proxy in
|
|
[microsandbox](https://github.com/wirenboard/microsandbox) substitutes
|
|
the real bearer for a placeholder on the way out. OAuth refresh is
|
|
MITM'd so multi-hour sessions survive token rotation.
|
|
- **Per-launch GitHub repo allow-list.** Auto-detected from
|
|
`git remote -v`; extend with `--repo OWNER/NAME`. `gh pr create`,
|
|
`git push` etc. are filtered at the proxy — off-list calls get a 403
|
|
before they reach GitHub.
|
|
- **Sandbox is the boundary.** Root inside the VM, project bind-mounted
|
|
at its host path, `--dangerously-skip-permissions` set by default
|
|
(the microVM is the only thing actually keeping the agent on rails).
|
|
|
|
This is the Rust rewrite of the original Bash
|
|
[`wirenboard/agent-vm`](https://github.com/wirenboard/agent-vm) on
|
|
top of microsandbox. Living on `rewrite-microsandbox` until v1.
|
|
|
|
## Requirements
|
|
|
|
- Linux with `/dev/kvm` (rw) — your user must be in the `kvm`
|
|
group: `sudo usermod -aG kvm $USER` and re-login.
|
|
- Node.js 18+ (already there if you use Claude Code / Codex CLI /
|
|
OpenCode — they're all npm-distributed).
|
|
|
|
`~/.microsandbox/lib/libkrunfw.so.5.x` auto-installs on first
|
|
launch.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
npm install -g @wirenboard/agent-vm # or: npx @wirenboard/agent-vm <cmd>
|
|
|
|
agent-vm setup # pulls the latest image from ghcr.io and verifies it boots
|
|
|
|
cd ~/your-project
|
|
agent-vm claude # or codex / opencode / shell
|
|
```
|
|
|
|
The npm package bundles a prebuilt `agent-vm` binary, the patched
|
|
`msb`, and libkrunfw. agent-vm finds them via
|
|
`current_exe()`-relative paths, so a user's separate
|
|
`~/.microsandbox/bin/msb` (if any) never shadows the patched build.
|
|
|
|
## Build from source
|
|
|
|
```bash
|
|
git clone -b rewrite-microsandbox https://github.com/wirenboard/agent-vm
|
|
cd agent-vm
|
|
git submodule update --init vendor/microsandbox
|
|
sudo apt-get install -y libcap-ng-dev libdbus-1-dev pkg-config
|
|
cargo build --release -p agent-vm
|
|
cargo build --release --manifest-path vendor/microsandbox/Cargo.toml \
|
|
-p microsandbox-cli --bin msb
|
|
./target/release/agent-vm setup # uses the locally-built msb sibling
|
|
```
|
|
|
|
`agent-vm setup` pulls
|
|
`ghcr.io/wirenboard/agent-vm-template:latest` by default; pass
|
|
`--image localhost:5000/agent-vm-template:latest` to use a local image
|
|
you've built via `images/build.sh`.
|
|
|
|
## Subcommands
|
|
|
|
```
|
|
claude | codex | opencode | shell launch an agent in a per-project sandbox
|
|
pull refresh the cached image
|
|
setup pull base image + verify boot
|
|
clipboard {get,put} [--sys] exchange a string with the project sandbox
|
|
```
|
|
|
|
## Image release cadence
|
|
|
|
The base OCI image (`ghcr.io/wirenboard/agent-vm-template:latest`) is
|
|
rebuilt hourly by CI, picking up the latest Claude Code, Codex CLI,
|
|
and OpenCode releases automatically. Pin a specific build with
|
|
`--image ghcr.io/wirenboard/agent-vm-template:YYYY-MM-DDTHH` (date tags are
|
|
immutable; the last 14 days are retained).
|
|
|
|
The agent-vm binary and the image are version-locked through an
|
|
**image-API-version** integer
|
|
(`/etc/agent-vm-image-version` inside the image). Mismatch → clean
|
|
error at launch instead of mysterious in-VM failures.
|
|
|
|
Each launcher accepts:
|
|
|
|
| flag | what |
|
|
|---|---|
|
|
| `--memory N` | VM memory GiB (default 2) |
|
|
| `--cpus N` | vCPUs (default 2) |
|
|
| `--image REF` | override the OCI image |
|
|
| `--no-update-check` | skip the registry HEAD on launch |
|
|
| `--no-git` | skip gh/git auth injection (still respects `--repo`) |
|
|
| `--repo OWNER/NAME` | add to the GitHub allow-list (repeatable) |
|
|
| `--mount HOST[:GUEST]` | extra bind mount (one virtio-fs each, ~210 mount headroom) |
|
|
|
|
Trailing args go to the agent: `agent-vm claude -p "say hi"`,
|
|
`agent-vm shell -- -c 'cargo test'`.
|
|
|
|
Env-var knobs (all opt-in; set to *any* value, empty included):
|
|
|
|
| var | what |
|
|
|---|---|
|
|
| `RUST_LOG` | tracing filter; default `warn`. e.g. `RUST_LOG=agent_vm=debug` |
|
|
| `AGENT_VM_PROFILE` | print per-phase wall-time (create/run/stop/remove) |
|
|
| `AGENT_VM_DEBUG_CONFIG` | dump the SandboxConfig JSON before boot |
|
|
| `AGENT_VM_NO_CHROME_MCP` | skip the Chrome DevTools MCP entirely (no entry in claude.json, no chrome-user setup at boot) |
|
|
| `AGENT_VM_IMAGE_TAG` | override the OCI image (same as `--image`) |
|
|
| `AGENT_VM_MEMORY_GIB` / `AGENT_VM_CPUS` | same as `--memory` / `--cpus` |
|
|
|
|
## Chrome DevTools MCP
|
|
|
|
The image ships chromium and a `chrome-devtools` MCP entry pinned to
|
|
`chrome-devtools-mcp@1.0.1`. To keep chromium's nested user-namespace
|
|
sandbox active (we'd rather not pass `--no-sandbox`) the MCP runs as a
|
|
dedicated `chrome` user via a sudo wrapper at
|
|
`/usr/local/bin/agent-vm-chrome-mcp`. The launcher installs the
|
|
per-boot microsandbox MITM CA into chrome's NSS DB at startup so
|
|
chromium accepts the intercepted TLS chain without
|
|
`--acceptInsecureCerts` (which would trust *any* untrusted cert).
|
|
|
|
If the CA install fails (e.g. someone broke the in-image sudoers rule)
|
|
the launcher prints a warning naming the symptom — without it, every
|
|
HTTPS navigate would silently return `ERR_CERT_AUTHORITY_INVALID`.
|
|
Set `AGENT_VM_NO_CHROME_MCP=1` to skip the whole setup.
|
|
|
|
## Credentials
|
|
|
|
Reads from the host:
|
|
|
|
- `~/.claude/.credentials.json` (Claude)
|
|
- `~/.codex/auth.json` (Codex, OpenCode)
|
|
- `gh auth token` (git/gh)
|
|
|
|
The guest gets placeholder strings; the proxy substitutes on the wire.
|
|
Real tokens live in `${XDG_STATE_HOME}/agent-vm/<hash>.secrets/` (0700)
|
|
on the host, **outside** the bind mount the guest sees. A SHA-256
|
|
snapshot of the three credential files is taken at launch and
|
|
re-checked on exit; unexpected mutations print a warning.
|
|
|
|
For Claude/Codex, when the in-VM agent's bearer expires the
|
|
hook MITMs the OAuth refresh, runs `claude -p`/`codex exec` on the
|
|
host to rotate, and feeds the new placeholder back to the guest — no
|
|
re-attach required.
|
|
|
|
## Project hook
|
|
|
|
If the project root contains an executable `.agent-vm.runtime.sh`,
|
|
the launcher sources it inside the guest before exec'ing the agent.
|
|
Use for `npm install`, env exports, dev-server startup. Non-zero
|
|
exit aborts the launch.
|
|
|
|
## Troubleshooting
|
|
|
|
- **`RegisterNetDevice(IrqsExhausted)` at boot** — the userspace split
|
|
irqchip raises the cap to ~219 IRQs, so this should only happen with
|
|
hundreds of `--mount`s or on a host whose KVM lacks
|
|
`KVM_CAP_SPLIT_IRQCHIP` (pre-Linux 4.7 or some nested-virt /
|
|
seccomp-restricted setups). Drop a `--mount` to recover.
|
|
- **`handshake read id_offset: timed out`** — `free -h`; the VM needs
|
|
more memory than is available. Try `--memory 1`.
|
|
- **GitHub 403 from the proxy** — repo isn't in the allow-list.
|
|
Pass `--repo OWNER/NAME` or run from a project with the right
|
|
remote.
|
|
|
|
## See also
|
|
|
|
- [PLAN.md](PLAN.md) — phased roadmap, what's done, what's deferred.
|
|
- [ARCHITECTURE.md](ARCHITECTURE.md) — design notes; why things look
|
|
the way they do.
|