Commit graph

287 commits

Author SHA1 Message Date
Evgeny Boger
23cba5663b distribution: ship via npm + ghcr.io image, drop local-build flow
Replaces the build-from-source quickstart (Rust toolchain + docker +
local registry container + manual `images/build.sh`) with a single
`npm install -g @wirenboard/agent-vm` and a `ghcr.io/wirenboard/
agent-vm:latest` image pulled from CI.

## What ships via npm

`npm-dist/` scaffolds the established per-platform-subpackage
pattern (esbuild / ruff / biome):

- `@wirenboard/agent-vm` — JS launcher that detects
  `${platform}-${arch}` at runtime and execs the right native binary
  from the matching subpackage. `optionalDependencies` lists each
  platform.
- `@wirenboard/agent-vm-linux-x64` — bundles a prebuilt `agent-vm`,
  the patched `msb`, and `lib/libkrunfw.so.5.2.1`.

Smoke-tested locally via `npm link`: launcher resolves the subpackage
correctly and execs the bundled binary; `MSB_PATH=/usr/bin/cat
agent-vm pull` correctly rejects an unpatched msb with the
actionable "reinstall agent-vm" hint.

## msb discovery (msb_install.rs rewrite)

`point_at_workspace_msb` → `point_at_msb` with a fallback chain:
1. `MSB_PATH` env (explicit override).
2. `<exe-dir>/msb` (npm bundle layout — sibling of agent-vm).
3. `<workspace>/vendor/microsandbox/target/release/msb` (dev mode).

`msb --version` is verified to contain `+agent-vm` — refuses to
launch if a user's separate microsandbox install has somehow ended
up on our discovered path. Catches the bug where shadow installs
would silently lose SecretValue::File and the request-interceptor
hook, producing inscrutable in-VM failures instead of one clear
"reinstall" message.

## Image flow (separate cadence)

- Default image ref: `ghcr.io/wirenboard/agent-vm:latest`.
- `images/build.sh` and local-registry flow stay as the
  source-checkout dev path (pass `--image localhost:5000/...`).
- `agent-vm setup` becomes "pull + verify"; no more docker/registry
  dependency for end users.
- New `image_api_version` module reads `/etc/agent-vm-image-version`
  inside the booted sandbox and requires it within
  `MIN_SUPPORTED_IMAGE_API..=MAX_SUPPORTED_IMAGE_API` (both 1
  initially). Out-of-range → actionable error pointing at the right
  side (binary vs image) to update.
- Image registry hostname heuristic (`is_plain_http_registry`)
  keeps `.insecure()` for localhost-style refs but drops it for
  ghcr.io / docker.io / public registries that require HTTPS.

## CI workflows

- `.github/workflows/release-npm.yml` — on `v*.*.*` tag, cross-
  compile binaries per platform (linux-x64 only initially; arm64 /
  darwin commented out, ready to enable when microsandbox grows
  those backends), bundle into subpackages, publish all packages
  to npm with provenance. Subpackages publish before the main
  package so optionalDependencies resolve.
- `.github/workflows/build-image.yml` — hourly cron + push-to-
  images/ trigger. Builds Dockerfile, tags `latest` + timestamped
  `YYYY-MM-DDTHH` + `sha-<sha>`, pushes to ghcr.io. Retention
  policy prunes hourly tags older than 14 days.

## README

Quickstart leads with `npm install -g`. Source-build instructions
moved to a dedicated section. Image-cadence section explains the
hourly rebuild + pinnable date tags + image-API contract.

## Tests (84/84 pass)

- msb_install: 4 (version-marker accept/reject + env-override behaviour).
- pull: 2 (local-registry detection: localhost/127.0.0.1/0.0.0.0/
  *.local/*.localhost vs ghcr.io/docker.io/example.com).
- image_api_version: 5 (parse trim, parse rejects garbage,
  in-range accept, too-new with actionable hint, too-old with hint).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 21:02:56 +03:00
Evgeny Boger
f40e6df2bc fix(phase7): run chrome-devtools MCP under a dedicated user with scoped CA trust
The Phase 7 chrome-devtools MCP entry (npx ... --headless --isolated)
reported `✓ Connected` to `claude mcp list` but every tool call
returned either `Protocol error (Target.setDiscoverTargets): Target
closed` or `net::ERR_CERT_AUTHORITY_INVALID`. Two distinct causes:

1. Chromium's user-namespace sandbox refuses to initialize as root
   (the guest's only user), so the browser dies before the CDP pipe
   is read.
2. Chromium on Linux ignores `/etc/ssl/certs/ca-certificates.crt`
   and only honours its built-in root store + the per-user NSS DB,
   so the microsandbox MITM CA isn't trusted by chromium even though
   curl/openssl trust it via the bundle.

Both of these had naïve fixes (`--no-sandbox` and `--acceptInsecureCerts`)
but we'd rather keep chromium's nested sandbox active (defence in depth
against untrusted content the agent navigates to) and only trust *our*
one CA (not every untrusted cert).

Approach:

- **Dedicated `chrome` user (UID 9999)** baked into the image via
  direct /etc/passwd + /etc/shadow + /etc/group edits (debian-slim
  has no `useradd`; the shadow entry is needed because `su` calls
  PAM and rejects users without one). Sudoers rule grants password-
  less `root -> chrome`.

- **Wrapper `/usr/local/bin/agent-vm-chrome-mcp`** that re-execs the
  MCP under chrome via `sudo -u chrome -H -n`. The env allow-list
  preserves agentd-set NODE_EXTRA_CA_CERTS / SSL_CERT_FILE /
  CURL_CA_BUNDLE / REQUESTS_CA_BUNDLE / PATH; without these, sudo's
  default env_reset strips them and node fails `npx -y` with
  SELF_SIGNED_CERT_IN_CHAIN fetching from registry.npmjs.org.

- **Pre-initialized empty NSS DB** at /home/chrome/.pki/nssdb (built
  at image time with `certutil -N --empty-password`) plus
  `libnss3-tools` package so the launcher's runtime `certutil -A`
  has something to add to.

- **Launcher prelude (run.rs)** runs `certutil -A -t TC -n
  microsandbox -i /usr/local/share/ca-certificates/microsandbox-ca.crt`
  on every boot. Can't bake the CA into the image — agentd writes it
  into the guest at boot time, and the cert is per-boot. Add-by-name
  is idempotent so re-running on every launch is fine.

- **MCP config (secrets.rs)** swaps `command: npx` for `command:
  /usr/local/bin/agent-vm-chrome-mcp` with the original npx + args
  passed through. No more `--no-sandbox`, `--disable-dev-shm-usage`,
  or `--acceptInsecureCerts`.

Verified e2e inside a fresh sandbox: `take_snapshot` after
`navigate_page https://example.com` returns the real "Example
Domain" heading text; `ps -ef | grep chrome` shows chromium running
as `chrome`, not root; NSS DB lists only `microsandbox` with `CT,,`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 20:39:55 +03:00
Evgeny Boger
daa5922f36 observability: default network proxy logs to info + post-mortem hints on attach failure
Three small improvements that arose from debugging chrome-MCP and
sandbox-startup issues in the dev VM:

- `init_tracing` now defaults `microsandbox_network::proxy` and
  `microsandbox_network::tls::proxy` to `info` even when `RUST_LOG`
  isn't set, so the per-connection lifecycle records (duration,
  bytes, close reason) land in `msb.stderr.log` out of the box.
  These are the only breadcrumbs that distinguish "Claude API
  hung" from "Claude's pooled HTTP/2 conn died after hours of
  idle"; debugging without them is guesswork.

- `launch` now wraps `sandbox.attach(...)` and on Err prints a
  pointer to the three log files the runtime now writes:
  `runtime.log` (msb tracing / panic), `msb.stderr.log`
  (libkrun / kernel printk / OOM), `msb-exit.log` (per-boot exit
  status), plus a `dmesg` hint for host-side OOM-kills. Helper
  `msb_sandbox_log_dir` mirrors microsandbox's resolve_home logic
  so the hint paths are usable rather than placeholder.

- Renames the placeholder secret strings to look obviously
  non-token-shaped (e.g. `msb-anthropic-placeholder-a-v2`
  instead of `MSB_PLACEHOLDER_ANTHROPIC_ACCESS_TOKEN_v1`).
  The previous values matched enough token heuristics to get
  some clients to reset/refuse requests when the strings showed
  up in shell transcripts copied between sessions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 20:38:57 +03:00
Evgeny Boger
920f6c155c vendor: bump msb to surface boot-time panics in agent-vm errors
Picks up wirenboard/microsandbox@4ad3a67 'sandbox: surface boot-time
panics to the user'. Replaces the opaque
`sandbox process exited (signal: 6 (SIGABRT)) before agent relay
became available` error with a runtime.log tail that includes the
actual Rust panic message — so users hitting kvm-permission
issues, missing kernel features, libkrun init errors etc. see
the actionable line instead of just a signal number.

The improvement was a direct outcome of debugging this in the dev
VM: the user wasn't in the `kvm` group, msb's stderr capture lost
the panic on abort, and the only way to find the cause was reading
`~/.microsandbox/sandboxes/<name>/logs/runtime.log` by hand.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 19:29:23 +03:00
Evgeny Boger
b7829bf81b fix(phase6): close keep-alive bypass in github smart-HTTP hook
msb's Interceptor handles one request per TLS/TCP connection. After
the first dispatch on a streaming rule the state goes to
State::Disabled and every subsequent chunk on the same connection
returns Verdict::Forward — which means the secret-substitution layer's
output (real GitHub token already swapped into the Authorization
header) flows STRAIGHT to upstream, bypassing the hook entirely.

Real-world impact: gh repo clone of a non-allow-listed private repo
SUCCEEDED. libcurl (git's HTTP backend) reuses TLS connections
aggressively. The first GET /info/refs request gets stripped by the
hook (no auth → 401 from GitHub), but libcurl's follow-up retry /
the POST /git-upload-pack request rides the same connection — the
hook is now Disabled, the real token reaches GitHub, clone proceeds.

The fix: inject `Connection: close` into every request the hook
emits, both Authenticated (allow-listed) and Anonymous (third-party)
paths. The server closes the TCP after responding; libcurl opens a
fresh connection for the next request; the fresh connection creates
a fresh Interceptor that re-evaluates the policy.

Also strips hop-by-hop headers (Connection / Keep-Alive /
Proxy-Connection) so we emit exactly one Connection: close header
regardless of what the guest sent.

Tests added:
- connection_close_injected_when_header_absent
- connection_close_replaces_existing_keep_alive
- connection_close_preserves_body_verbatim
- anonymous_passthrough_forces_connection_close (e2e: secrets →
  hook simulation; real token must not reach wire AND Connection:
  close must be present)
- authenticated_passthrough_forces_connection_close (e2e: same for
  the allow-listed path — token allowed through, but connection
  still torn down)
- keep_alive_second_request_bytes_contain_real_token_pre_hook
  (hypothesis-confirmation test: documents the lower-level leak in
  the secrets layer alone — the Connection: close fix is what makes
  it unexploitable)
- private_repo_clone_does_not_leak_real_token_through_pipeline
  (full end-to-end: build the exact bytes a git clone sends, run
  through real SecretsHandler + real hook helpers, assert no token
  in the wire bytes)
- allowlisted_repo_clone_does_pass_real_token_through_pipeline
  (counterpart: token must reach upstream when the repo IS in the
  allow-list)

agent-vm/Cargo.toml gains a dev-dep on microsandbox-network so the
e2e tests can drive the real SecretsHandler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 18:30:41 +03:00
Evgeny Boger
c95135765f fix(intercept-hook): strip-authorization preserves header/body separator
When Authorization was the LAST header line, the previous
implementation emitted the preceding header WITH its trailing
\r\n, then dropped the Authorization line, then appended `rest`
which itself starts with \r\n\r\n. Result: three consecutive
\r\n between headers and body. Servers see two \r\n\r\n: the
first ends the headers, the second's \r\n becomes the start of
the body, shifting body bytes by 2. For Content-Length-tagged
bodies this corrupts the payload; for keep-alive connections it
can poison the next pipelined request.

Existing test `strip_auth_preserves_request_line_and_other_colons`
exercised this exact shape but only asserted on prefixes, so the
trailing-CRLF bug was invisible.

Rewrite: collect kept lines first, then join with \r\n and append
`rest`. Dropping any line — first, middle, or last — produces a
single \r\n\r\n between headers and body. Bumps vendored msb to
pick up the partial-headers fast-path gate.

Regression test:
strip_auth_when_authorization_is_last_header_no_extra_crlf —
asserts no triple-CRLF and that body bytes follow exactly one
\r\n\r\n.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 17:33:20 +03:00
Evgeny Boger
4d21b82d71 cargo: relax release profile, add dist profile for distribution builds
release: lto="thin" + codegen-units=16. Cuts iteration build
time roughly in half on this workstation. Loses ~marginal
runtime perf vs fat-LTO; not measurable in agent-vm's
network-bound workload.

dist: inherits release with lto="fat" + codegen-units=1. For
release artifacts where binary size and steady-state perf
matter most. Use: `cargo build --profile dist`.

Also bumps the vendor/microsandbox submodule to pick up the
body-only chunk fast path fix (secrets/handler: skip lossy
UTF-8 round-trip on body-only chunks).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 17:00:53 +03:00
Evgeny Boger
9de2d98bb1 fix(phase6): scope github search & notifications to third-party access
Per the access policy (allow-listed = my access; else = third
party), `/search/*` and `/notifications[/*]` must NOT carry the
user's GH token. Authenticated search would leak private repo
inventory and personal state to the agent; authenticated
notifications would expose the user's inbox.

Strip auth on these paths → agent sees exactly what a third
party sees: public-only search results, 401 on notifications.
Agents needing private-repo search can `git grep` inside the
project bind-mount (works directly on disk).

`/graphql`, `/rate_limit`, `/meta`, `/markdown` stay Authenticated
— they're not user-scoped and gh tooling depends on them.

Test added: gh_access_search_and_notifications_are_anonymous.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 17:00:44 +03:00
Evgeny Boger
939367f0c5 feat(phase6): allow-listed = my access; else = third-party access
Revises the GitHub policy per the user's intent: rather than a list of
allowed methods/paths, defer to GitHub itself. For allow-listed repos
the request goes out with the real bearer (the user's normal access);
for any other repo or surface, the Authorization header is stripped
and the request reaches GitHub as a third-party / anonymous caller —
GitHub then enforces public-vs-private as it does for any unauth
visitor.

Concretely:

  - allow-listed repo READ  → my access (private content visible)
  - allow-listed repo WRITE → my access (PR creation, push, etc.)
  - public non-listed READ  → succeeds (third party sees public)
  - private non-listed READ → 404 (GitHub hides it from third party)
  - any non-listed WRITE    → 401 (third party can't write)

**`intercept_hook::github_access(method, path, allowed)`** for
api.github.com:
  - `/repos/<o>/<r>/...` → Authenticated if allow-listed, else
    Anonymous.
  - `/user`, `/user/orgs[/...]` → Authenticated (gh auth status,
    gh repo view org/x).
  - other `/user/*` (repos, keys, emails, gpg_keys, …) → Anonymous
    (third party can't see; GitHub returns 401).
  - `/graphql`, `/search/*`, `/rate_limit`, `/meta`, `/markdown`,
    `/notifications` → Authenticated (utility, gh-tooling-friendly).
  - everything else (/users/<x>, /orgs/<x>, /licenses, …) →
    Anonymous (third-party-visible).
  - `..` traversal → Deny.

The launcher's `forward_github_api` (reqwest-based) now consults
this and conditionally injects/strips Authorization before
forwarding.

**`intercept_hook::github_smart_decision`** for github.com /
codeload / raw / objects:
  - allow-listed repo → Authenticated (empty stdout → proxy
    passthroughs verbatim, secret layer substitutes).
  - any other repo → Anonymous. Hook now returns the buffered
    request with the `Authorization` header line removed, via the
    new upstream "passthrough-with-modified-bytes" verdict (see
    `vendor/microsandbox` commit). The placeholder isn't in the
    forwarded bytes, so the secret-substitution layer is a no-op
    and GitHub sees a third-party request.
  - `..` traversal → Deny.
  - Malformed → 400.

**`strip_authorization_from_request(bytes)`** byte-precise helper
that removes the Authorization header line (case-insensitive)
while preserving the request line, every other header, and the
body verbatim. Correctly handles the LAST header (whose trailing
CRLF lives in the `\r\n\r\n` separator, not inside the header
block).

Test coverage rewritten around the new enums (62 tests pass before
e2e):
- `gh_access_*` (10 tests): allow-listed all-methods, third-party
  all-methods, case-insensitivity, user/identity vs PII paths,
  utility allow list, public lookup endpoints, traversal, query
  string, malformed.
- `smart_*` (7 tests): allow-listed authenticated for clone+push,
  any-other-repo anonymous, single-`.git` strip, traversal,
  case-insensitivity, malformed, malformed-owner.
- `strip_auth_*` (5 tests): basic auth strip + body preserve,
  case-insensitive header name, no-op when no auth present,
  malformed input pass-through, request line + Cookie URL with
  `:` not confused with header separator.

E2E verified on a real host:

  $ gh api repos/octocat/Hello-World    # public, not allow-listed
  → "octocat/Hello-World / public"      (third-party read works)

  $ git ls-remote https://github.com/octocat/Hello-World
  → 7fd1a60b...  HEAD                   (anon clone works)

  $ git ls-remote …/octocat/Hello-World.git/info/refs?service=git-receive-pack
  → remote: Not Found                   (anon push correctly denied
                                         by GitHub)

  $ gh api 'repos/wirenboard/.../../octocat/Hello-World'
  → agent-vm 403 traversal              (our hook synthesises)

63/63 tests green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 16:15:15 +03:00
Evgeny Boger
b58165ffe1 test+feat: expand test coverage; per-project .bash_history persistence
Tests (+30, total 59):

**run::tests** (+16) — `parse_github_slug` matrix (https/http/scp/ssh
URL/ssh-with-port forms, extra path components, non-github URLs,
`.git` suffix stripped exactly once, empty owner/repo, double `.git`);
`parse_extra_mounts` (mirror, remap, relative-path rejection,
empty-side, nonexistent host); `mkdir_chain` (root, single segment,
prefix walk); `guest_path_is_safe` (normal paths, tmpfs prefixes,
lookalike non-prefix matches like /tmpfoo).

**secrets::tests** (+7) — `host_secret_dir` safe across trailing-
slash state_dirs (closes a code-review concern); placeholder-pairwise-
distinctness sanity (catches a future placeholder that's a substring
of another — would silently swap the wrong token);
`hash_file` deterministic + None on missing; `decode_id_token_account`
URL-safe + standard-alphabet fallback + missing-field paths;
`write_default_opencode_config` first-write defaults + merge-preserves-
user-`model`.

**intercept_hook::tests** (+6) — `parse_http_request`: basic GET, POST
with body, header value with embedded colons, missing-separator error,
empty-request-line error, whitespace-trimmed header values.

Also fixes a `.git`-suffix greediness bug in `parse_github_slug`
(same shape as the one already fixed in `github_smart_decision`):
`trim_end_matches(".git")` was greedy, so a `repo.git.git` URL
yielded slug `o/repo` instead of `o/repo.git`. Use `strip_suffix`
to strip exactly one suffix; the new test
`parse_github_slug_handles_dot_git_only_once` would have caught a
regression.

Feature: persistent per-project bash history.

`agent-vm shell` now preserves bash history across launches, scoped
to the project. `secrets::refresh` touches
`<state>/bash_history` if missing; the launcher patch builder
symlinks `/root/.bash_history -> /agent-vm-state/bash_history` so
bash's default HISTFILE points at the per-project state. Subsequent
launches inherit whatever `bash` appended on exit (clean `exit` or
Ctrl-D — Ctrl-C of the launcher won't save).

Verified end-to-end:
- 1st session writes `echo foo >> /root/.bash_history` → host file
  contains foo;
- 2nd session reads it + appends another entry → host file
  accumulates;
- Symlink target matches: `lrwxrwxrwx ... /root/.bash_history ->
  /agent-vm-state/bash_history`.

59/59 tests green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 15:58:07 +03:00
Evgeny Boger
3bdfbd4a53 test(phase6): unit tests for the github allow-list policy + auth helpers
The github allow-list is the security surface — getting it wrong
silently lets an in-VM agent push to / mutate repos the user didn't
list. Cover the policy matrix in unit tests so a future refactor
can't regress it without a test failure.

22 new tests in `intercept_hook::tests`:

**github_path_allowed (api.github.com):**
- Reads (GET/HEAD) unrestricted across /repos/*, /user/*, /search,
  /orgs/*, /notifications — including endpoints the previous policy
  narrowed (/user/repos, /user/keys, /user/emails).
- Writes (POST/PATCH/PUT/DELETE) on /repos/<owner>/<repo>/* must
  match the allow-list (case-insensitive on both owner and repo).
- Writes on non-allow-listed repos are denied for every write method.
- POST /graphql is allowed unconditionally (v1 gap).
- POST /markdown and POST /user/repos are allowed (utility / own-repo
  creation).
- Writes to other /user/* (POST /user/keys, DELETE /user/keys/N) and
  /admin/* are denied.
- `..` path segments rejected for ANY method, including reads
  (GitHub server-side normalisation would otherwise route to a
  different repo than we checked).
- Query strings are stripped before matching.
- Malformed /repos/ paths (empty owner or repo) deny on writes.

**github_smart_decision (github.com / codeload / raw / objects):**
- Clone/fetch (info/refs?service=git-upload-pack, git-upload-pack POST,
  codeload zip, raw blob) is Passthrough against any repo.
- Push handshake (info/refs?service=git-receive-pack GET) and push
  data (git-receive-pack POST) are checked against the allow-list.
- Multi-param query strings — `?foo=bar&service=git-receive-pack&baz=`
  — still parsed correctly.
- `.git` suffix on the repo segment is stripped exactly once
  (strip_suffix not trim_end_matches).
- `..` traversal in push paths is rejected.
- Case-insensitive owner/repo match.
- Malformed input (no CRLF, empty, single-token request line) returns
  Malformed.

**substitute_authorization_header:**
- `Bearer <PLACEHOLDER>` / `token <PLACEHOLDER>` → literal replace.
- `Basic base64(x-access-token:<PLACEHOLDER>)` → decode, substitute,
  re-encode (round-trip verified). Placeholder isn't in the output
  at any layer.
- `Basic <something-else>` (no placeholder) is preserved verbatim —
  we don't silently re-encode other people's basic-auth values.

`GithubSmartDecision` gets `#[cfg_attr(test, derive(Debug))]` so the
`assert!(matches!(...))` failure messages are readable.

28/28 tests now green (6 pre-existing + 22 new).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 15:35:03 +03:00
Evgeny Boger
3a11466d84 fix(phase6): allow-list scopes WRITES only; reads unrestricted
The previous policy denied any path not in a short allow-list of
read-only surfaces (/user, /user/orgs, /search, /rate_limit, /meta,
/markdown, plus /repos/<allow-listed>/...). That broke common agent
flows: `gh repo view octocat/Hello-World`, `git clone
https://github.com/<any>/<any>`, `gh issue list <any>/<any>` all
403'd.

Re-spec to what the user actually wants: read-only access is
unrestricted (the agent often needs to browse other repos —
look up an API, fetch a README, check who reviewed a PR); only
write / management ops are scoped to the per-launch allow-list.

**`github_path_allowed(method, path, allowed)`** (api.github.com):
- `GET` / `HEAD` → allow (read).
- `POST /graphql` → allow (no body-level mutation filter in v1;
  documented gap).
- `POST /markdown`, `POST /user/repos` → allow (utility / creates a
  new repo owned by the user).
- `POST/PATCH/PUT/DELETE` under `/repos/<owner>/<repo>/*` → allowed
  iff `<owner>/<repo>` in the allow-list. Catches PR creation, issue
  creation, merges, comments, deletes, branch protection edits,
  releases.
- Anything else (e.g. `POST /user/keys`, `DELETE /orgs/<x>/...`) →
  deny. These aren't routine gh CLI flows.
- `..` path segments still rejected up front (GitHub server-side
  normalisation would otherwise resolve to a different repo).

**`github_smart_decision(request, allowed)`** (github.com / codeload
/ raw / objects smart-HTTP):
- `git-receive-pack` POST (push data) → owner/repo must be in allow-
  list.
- `info/refs?service=git-receive-pack` GET (push handshake) → same.
- Everything else (`git-upload-pack`, codeload archives, raw blobs,
  any other path) → passthrough. clone, fetch, browse all work
  against any repo.

Verified inside the guest:
- `gh api repos/octocat/Hello-World` (not allow-listed) — reaches
  GitHub (no 403).
- `git ls-remote https://github.com/octocat/Hello-World` — real refs
  returned via streaming passthrough.
- `gh api -X POST repos/octocat/Hello-World/issues` — clean 403 from
  our hook naming the read/write distinction and the allow-list.
- `gh api -X POST repos/wirenboard/agent-vm/issues` (allow-listed) —
  reaches GitHub.
- `gh api graphql -f query=...` — reaches GitHub.

6/6 cargo tests still green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 15:31:25 +03:00
Evgeny Boger
546d24750f docs(readme): rewrite — tighten, drop status/why, add subcommand table
The old README spent two-thirds of its lines on a phase-by-phase
status list (which belongs in PLAN.md) and an architecture-style
'Why' explainer that's not useful for someone trying to install
and use the tool. Cut ~80 lines, restructured around:

- One paragraph elevator pitch with the 3 differentiators
- Requirements + quick-start (copy-paste)
- Subcommand reference with a per-flag table
- Credentials in two paragraphs
- Project hook + troubleshooting in 3 bullets each
- Links to PLAN.md / ARCHITECTURE.md for depth

170 → ~110 lines, no information loss for a new user.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:52:10 +03:00
Evgeny Boger
9a45806b2c fix(opencode): pin default model so opencode run works without --model
Before this commit, `agent-vm opencode run "..."` failed with
  "model 'gpt-5.5-pro' is not supported when using Codex with a
  ChatGPT account"
because OpenCode's built-in default is gpt-5.5-pro and the OpenAI API
rejects it on ChatGPT-OAuth (the auth flow we wire up). The user had
to pass `--model openai/gpt-5.5` on every invocation.

Two changes:

1. **secrets::write_default_opencode_config** writes
   `<state>/opencode-config/opencode.json` with `model =
   openai/gpt-5.5`, merge-on-existing so user overrides survive.
   File schema is OpenCode's `https://opencode.ai/config.json`.

2. **run.rs** symlinks `/root/.config/opencode →
   /agent-vm-state/opencode-config` so the file lands at the XDG
   config path opencode actually reads — distinct from the data
   dir at `~/.local/share/opencode` we were already mounting.
   The previous attempt wrote `config.json` into the data dir,
   which opencode ignored.

Verified inside the guest: `opencode run "Reply: OPENCODE_DEFAULT_OK"`
(no --model flag) now shows `> build · gpt-5.5` and returns the real
response. 6/6 cargo tests still green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:50:04 +03:00
Evgeny Boger
b985bebfa9 docs(plan): record Phase 6 follow-up commits (gh scope, sweep, github.com streaming)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:21:20 +03:00
Evgeny Boger
4e6ea00efb docs(run): update gh-secret comment now that the github.com gap is closed
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:20:36 +03:00
Evgeny Boger
62c9eb6ff8 fix(phase6): close github.com smart-HTTP bypass via streaming intercept
Builds on the upstream microsandbox change (vendor bumped to the new
dispatch_on_headers + empty-stdout passthrough primitive — see
`vendor/microsandbox` commit message).

Closes review finding #1 — the per-launch GitHub-repo allow-list now
binds on git smart-HTTP too, not just api.github.com.

**Launcher (`run.rs`).**
- After the existing api.github.com rules, add `rule_streaming(host,
  method, "/")` for github.com, codeload.github.com,
  raw.githubusercontent.com, objects.githubusercontent.com on GET +
  POST. The streaming form fires the hook on headers only — the
  multi-MB pack-file body from `git push` never gets buffered.

**Hook (`intercept_hook.rs`).**
- New `github_smart_decision(request, allowed_repos)` parses the
  request line, extracts owner/repo (strips a single `.git` suffix),
  rejects any `..` traversal, and either returns Passthrough (empty
  stdout — proxy continues with secret substitution) or Deny(reason)
  (synthesized 403 to the guest).
- Dispatch path: if the SNI matches one of the four smart-HTTP
  hosts, run github_smart_decision; map Passthrough → empty
  write_response (the proxy interprets this as ForwardBuffered),
  Deny → error_response(403, ...), Malformed → error_response(400,
  ...).

End-to-end verified on a real host with a real `gh auth token`:

  $ git ls-remote https://github.com/wirenboard/agent-vm  # allowed
  ae6d450f17…  HEAD
  b181d33245…  refs/heads/...
  …
  $ git ls-remote https://github.com/octocat/Hello-World  # denied
  fatal: unable to access 'https://github.com/octocat/Hello-World/':
    The requested URL returned error: 403

The allowed call streamed all the way to GitHub via the proxy
substitution layer (the placeholder bearer got swapped to the real
gh token); the denied call never touched GitHub — the hook
synthesized the 403 from the request line alone and returned it to
the guest.

The earlier doc-comment in run.rs noting the "smart-HTTP gap" still
stands as background but no longer describes a real gap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 14:20:06 +03:00
Evgeny Boger
29c0ccc6c7 fix: address remaining code-review findings (sweep)
Catches the rest of the xhigh review's verified findings, focused on
security + reliability + correctness. Each bullet references the
finding from the JSON output in this conversation.

**Security**
- intercept_hook: strip Set-Cookie / Set-Cookie2 / WWW-Authenticate /
  Proxy-Authenticate from upstream responses. Prevents in-VM agent
  from harvesting GitHub session cookies and replaying them on the
  unfiltered github.com path to authenticate as the host user
  (review #3).
- intercept_hook: substitute_authorization_header handles both Bearer
  (literal substring) AND Basic auth (decode → replace → re-encode
  base64). Closes the "git Basic-auth always 401s" path while keeping
  the gh CLI Bearer path working. Also injects Bearer with the real
  token if the guest sent no Authorization header at all, so the
  forwarder never sends anonymous-but-substituted-looking requests
  (review #12).
- run.rs detect_github_repos_in_cwd hardens the git invocation against
  repo-local config that can execute code on the host (core.fsmonitor,
  core.pager, core.editor, alias.*). Pass `safe.directory=*` so a
  foreign-UID checkout doesn't fail closed, `protocol.allow=never` so
  the call can't fetch anything, and force GIT_CONFIG_SYSTEM /
  GIT_CONFIG_GLOBAL to /dev/null so user-global git config can't
  inject an attacker-controlled include path. Closes the "host RCE
  before sandbox boot via untrusted .git/config" finding (review #6).
- run.rs adds a long inline note above the gh secret entry calling
  out the github.com smart-HTTP gap (review #1). Filtering git
  smart-HTTP requires a streaming intercept primitive microsandbox
  doesn't expose (current intercept buffers full request, capped at
  64 KiB; git push pack data exceeds that). Documented; the threat
  model is now explicit.

**Reliability**
- intercept_hook forward_github_api: reqwest::Client gets a 60 s
  timeout and `Policy::none()` redirect policy — hung api.github.com
  no longer freezes the sandbox; 3xx responses are surfaced to the
  guest verbatim instead of silently following (review #7).
- intercept_hook trigger_host_refresh: 90 s wait-with-timeout on the
  host `claude -p` / `codex exec` so a stuck CLI doesn't hold the
  intercept hook indefinitely (review #8).
- run.rs streaming exec: distinguish "stream closed without Exited
  event" (infra disconnect) from "agent exited with code 1" (real
  failure). The former bails with a clear error instead of returning
  exit 1 (review #9).

**Correctness**
- run.rs adds a SnapshotGuard with `impl Drop` so the Phase-5 host-
  cred mutation check runs on every exit path from launch(),
  including `?` propagation from attach/exec_stream errors. Previously
  the safety net only fired on the happy path (review #10).
- run.rs: --no-git no longer silently discards explicit --repo
  arguments. The flag suppresses cwd auto-detection but explicit
  --repo entries are kept; the launcher warns when --no-git +
  --repo are combined (review #11).
- secrets.rs: `Ok(_)` → `Ok(Some(()))` in the opencode token-file
  arm. `Ok(None)` (host file missing) no longer registers the
  placeholder pointing at a token file that wasn't written (review
  #13).
- secrets.rs decode_id_token_account: try URL_SAFE_NO_PAD first
  (conformant JWT base64url) then fall back to STANDARD with manual
  '+'/'_' alphabet conversion. Closes the silent zero-UUID fallback
  when an upstream library emits standard-alphabet JWT payloads
  (review #14).
- bin/agent-vm-ccusage: switch from `paste -sd,` to a
  null-delimited find→read loop, and skip any path containing `,`
  with a clear warning. CLAUDE_CONFIG_DIR's comma-separator has no
  escape mechanism so any path with a comma silently corrupts the
  union (review #15).

Verified end-to-end:
- `gh api graphql ...` reaches GitHub (allow-list extended in the
  previous commit; this commit didn't touch path filtering, but the
  pipeline still works: clean 403 for denied repos, real responses
  for allowed paths).
- `git status` works on the bind-mounted project — no dubious-
  ownership (safe.directory) and the hardened cwd-detection git
  call still finds the remote.
- gh hosts.yml is written, gh auth status shows the placeholder
  account is active (Bad credentials when reaching GitHub is the
  same documented outer-bridge nested-host limit).
- 6/6 cargo tests still green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 13:40:20 +03:00
Evgeny Boger
4479b1f2d1 fix(phase6): make gh CLI actually usable inside the VM (graphql, utility paths, safe.directory)
Two related fixes from real end-to-end testing of Phase 6:

**Fix 1: gh CLI was effectively broken inside the VM.**
The original github_path_allowed accepted only /user/* and
/repos/<owner>/<repo>/..., which blocked the GraphQL endpoint gh CLI
needs for the most common workflows:

- `gh pr create` → POST /graphql (rejected with 403)
- `gh issue create` → POST /graphql (rejected)
- `gh repo view --json` → POST /graphql (rejected)
- `gh issue list --search ...` → /search/issues (rejected)
- `gh repo view org/x` → /orgs/<org> (rejected)
- gh ambient probes → /rate_limit, /meta, /markdown (rejected)

Extend the allow-list:

- /graphql is allowed unconditionally. Per-repo scoping doesn't reach
  into GraphQL bodies in v1; this is a documented trade-off.
- /search/*, /rate_limit, /meta, /markdown are allowed (read-only
  utility, no per-repo state exposed).
- /orgs/<org> (bare) is allowed for `gh repo view org/...` resolution;
  /orgs by itself (listing) is denied to avoid leaking org membership.
- /user surface narrowed: only /user (auth probe) and /user/orgs are
  allowed. /user/repos, /user/keys, /user/emails, /user/gpg_keys are
  now BLOCKED — the previous catch-all leaked the full inventory of
  private repos and SSH keys / verified emails (see code-review
  finding #4 from this session).
- Reject any path segment equal to ".." anywhere to close the
  /repos/<allowed>/<repo>/../../<victim> path-traversal that GitHub
  would otherwise normalise upstream (code-review finding #2).

**Fix 2: `fatal: detected dubious ownership in repository`.**
The project is bind-mounted into the guest as the host user (UID 1000)
but the in-guest agent runs as root (UID 0). git refuses operations on
repos whose .git is owned by a different UID. Symptom: every `git
status` / `git log` / `git show` etc. inside the guest aborts.

write_guest_gh_config now ALWAYS writes a base /root/.gitconfig with
`[safe] directory = *` regardless of whether gh auth was captured.
The credential helper + url.insteadOf stanzas are only included
when has_gh_token=true; the gh hosts.yml is still gated on having
a token. Launcher always calls write_guest_gh_config (was gated on
gh_token_file before) so safe.directory is in place even with
--no-git or no host gh auth.

Verified inside the guest:
- `git status` returns "On branch master / nothing to commit" — no
  dubious-ownership abort.
- `git log` reads the bind-mounted history.
- `gh api graphql -f query="query { viewer { login } }"` reaches
  GitHub (response is Bad credentials, same nested-host limit as
  documented for the other providers — not our allow-list).
- `gh api /rate_limit` reaches GitHub.
- `gh api repos/octocat/Hello-World` still returns the clean 403
  naming the allow-listed repos.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 13:11:27 +03:00
Evgeny Boger
68b293a364 docs(plan): mark Phase 5/6/7/9 status after the implementation pass
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 06:09:24 +03:00
Evgeny Boger
f6020f1f88 feat(phase9): auto-install runtime, --image / --no-update-check / runtime hook, README
The "ready to share with a teammate" pass — the things a first-time
user shouldn't have to figure out by reading source.

**Auto-install of the microsandbox runtime.**
- `msb_install::ensure_runtime_installed()` wraps the SDK's
  `microsandbox::setup::{is_installed, install}`. Called once from
  `main` (skipped for the `_intercept-hook` and `clipboard`
  subcommands — neither needs the runtime).
- First-launch UX is now: clone, `cargo build`, `agent-vm setup`,
  done. No more manual tarball download into ~/.microsandbox.

**CLI flag promotion** (Phase 9 line item):
- `--image REF` overrides the OCI reference for one launch. Layered
  on top of the existing `AGENT_VM_IMAGE_TAG` env var: flag wins,
  then env, then `localhost:5000/agent-vm:latest`.
- `--no-update-check` skips the registry HEAD that drives the
  "newer image available" banner. Useful in CI and on flaky links.

**`.agent-vm.runtime.sh` project hook.**
- The bash prelude that already strips IPv6 nameservers + closes
  stdin now also `source`s `<project>/.agent-vm.runtime.sh` if it
  exists. Runs inside the guest at the project's bind path with
  `cd <project>`; non-zero exit aborts the launch with the same
  code, so a broken `npm install` doesn't drop the user into a
  half-set-up shell.
- Use cases: `npm install`, project env exports, dev-server
  startup, anything that has to happen *inside* the sandbox before
  the agent runs.

**README rewrite.**
- Drop the "rewrite-in-progress" framing for an "actually
  publishable" one. Order: why → status (phases done/deferred) →
  requirements → build → use → credentials → project hook →
  troubleshooting → docs links.
- Status list updated to reflect Phases 5/6/7 done (this session)
  plus the new auto-install behavior. Real-world failure modes
  surfaced from this conversation get a short Troubleshooting
  section (IRQ exhausted, handshake timeout, stdin hang,
  doubly-nested 401).

Verified:
- `agent-vm shell --no-update-check --image localhost:5000/agent-vm:latest`
  works (`==>` banner shows correct image, no update-check noise).
- A test `.agent-vm.runtime.sh` was sourced before the agent ran,
  with `$PWD` correctly at the project's guest bind path.
- 6/6 cargo tests still green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 06:08:31 +03:00
Evgeny Boger
5c5bf22c1e feat(phase7): --mount, clipboard bridge, ccusage wrapper, Chrome MCP
DX additions from the original Bash agent-vm that the user wants in
v1. Four independent pieces; one commit because they all touch the
same layer (launcher + image + a new subcommand).

**`--mount HOST[:GUEST]`** (repeatable).
- New `Args::mount: Vec<String>` parsed via `parse_extra_mounts` into
  `(host, guest)` pairs. `HOST` alone defaults `GUEST` to mirror.
  Both must be absolute; the host path is canonicalized (follows
  symlinks) and checked to exist before the launcher proceeds.
- For each mount we call `.volume(guest, |m| m.bind(host))` and
  `mkdir_chain(guest)` in the patch builder, mirroring what the
  project bind already does. The launcher prints a per-mount line.
- Note: libkrun's virtio IRQ pool is *tight* on this build —
  empirically even one extra `--mount` on top of project + state +
  network + agentd trips `RegisterNetDevice(IrqsExhausted)` at boot,
  with or without secrets registered. So the feature ships with a
  warning eprintln rather than a client-side cap (libkrun configs
  vary) and the error message is clear when the cap is hit. See
  discovered upstream issue #3 in PLAN — the right long-term fix
  is to raise libkrun's IRQ budget upstream.

**Clipboard bridge.**
- New `agent-vm clipboard {get,put}` subcommand (`clipboard.rs`).
  `put` reads stdin (or `VALUE` arg) into `<state>/clipboard.txt`;
  `get` prints the file content to stdout. `--sys` on either side
  also exchanges with the host system clipboard via
  `wl-copy/wl-paste`, `xclip`, or `pbcopy/pbpaste` — first one on
  PATH wins.
- The launcher's existing `/agent-vm-state` bind exposes the file at
  `/agent-vm-state/clipboard.txt` inside the guest. The in-VM agent
  reads/writes that path directly; no in-guest helper script
  needed.

**`agent-vm-ccusage` wrapper.**
- Standalone shell script at `bin/agent-vm-ccusage` that resolves
  the state root via the same precedence the Rust launcher uses
  (`$AGENT_VM_STATE_DIR` → `$XDG_STATE_HOME/agent-vm` →
  `$HOME/.local/state/agent-vm`), enumerates per-project
  `<hash>/claude` session dirs, and runs
  `npx ccusage@latest "$@"` with `CLAUDE_CONFIG_DIR` set to the
  comma-joined union of host + every project. Ported 1:1 from the
  original.

**Chrome DevTools MCP.**
- `images/Dockerfile` adds `chromium` + `fonts-liberation` from the
  Debian repo and the usual `google-chrome` / `google-chrome-stable`
  / `/opt/google/chrome/chrome` symlink dance so tools that hard-
  code those paths find the binary.
- `secrets::write_default_claude_root_state` force-sets the
  `mcpServers["chrome-devtools"]` entry in the guest's
  `~/.claude.json` (`{"command": "npx", "args": ["-y",
  "chrome-devtools-mcp@latest", "--headless=true",
  "--isolated=true"]}`). User-set MCP servers are preserved
  (merge semantics on the surrounding map). Opt out with
  `AGENT_VM_NO_CHROME_MCP=1`.

Verified:
- `agent-vm clipboard put` then `agent-vm clipboard get` round-trips
  on the host.
- Inside the guest with the rebuilt image: `which chromium` →
  `/usr/bin/chromium`, `which google-chrome` →
  `/usr/bin/google-chrome`, `chromium --version` → Chromium 148.
- `~/.claude.json` in the guest contains the chrome-devtools MCP
  entry.
- `agent-vm-ccusage --help` exits cleanly, ccusage CLI loads.
- `--mount` parsed correctly; print + fail-on-missing work; boot
  fails with the libkrun IRQ error as documented.
- 6/6 cargo tests still green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 06:04:08 +03:00
Evgeny Boger
396011beb5 feat(phase6): gh/git credential injection + per-launch repo allow-list
In-VM agents need to push commits and create PRs to be useful for
actual development. Original Bash agent-vm did this with the host's
gh auth and a credential proxy. The rewrite has been silent on it.
This wires it end-to-end with a per-launch GitHub repo allow-list
enforced at the proxy hook.

**Credential injection.**
- `secrets::refresh_gh` shells out to `gh auth token` and stashes the
  result in the host-only `<state>.secrets/gh` (mode 0600). Returns
  None gracefully if `gh` isn't installed or the user isn't logged in
  — no warning, just no gh substitution.
- New constant `GH_TOKEN_PLACEHOLDER` ("MSB_PLACEHOLDER_GH_TOKEN_v1")
  is what the guest's gh / git tools see; the proxy substitutes it
  for the real bearer on outbound traffic.
- `secrets::write_guest_gh_config` writes `<state>/gitconfig` (with a
  credential helper that echoes the placeholder so `git push` sends
  it as Basic auth) and `<state>/gh-config/hosts.yml` (for the gh
  CLI). The launcher's patch builder symlinks both into the standard
  guest paths (`/root/.gitconfig`, `/root/.config/gh`).

**Allow-list.**
- `run::detect_github_repos_in_cwd` parses `git remote -v` in the
  project dir and pulls out `owner/repo` slugs from any
  github.com remote (https, ssh, ssh:// — `parse_github_slug`
  covers each form).
- New CLI flags on `claude/codex/opencode/shell`: `--no-git` (opt
  out entirely) and `--repo OWNER/NAME` (repeatable, widens the
  list beyond the cwd remote).
- The launcher gates `use_github` on having at least one allowed
  repo + `--no-git` not set; passes that into
  `secrets::refresh(state_dir, project_guest_path, use_github)`
  (now takes a 3rd arg).

**Proxy enforcement (the new bit).**
- New secret entry in `run.rs` for the gh token, with
  `inject_basic_auth(true)` so git's Basic-auth wire format
  (base64(x-access-token:placeholder)) is substituted correctly.
  Allowed hosts: api.github.com, github.com, codeload.github.com,
  raw.githubusercontent.com, objects.githubusercontent.com.
- New intercept rules for api.github.com on every method gh CLI
  uses (GET/POST/PATCH/PUT/DELETE), all with path "/" (prefix
  match — catches everything). Only added when a gh token was
  captured; otherwise the hook has nothing to forward with.
- New hook path in `intercept_hook::forward_github_api` (hook is
  now `async fn`): parses the buffered HTTP request, calls
  `github_path_allowed(path, allowed_repos)`, and either
  synthesizes a 403 (denied) or forwards via `reqwest` after
  replacing `GH_TOKEN_PLACEHOLDER` in Authorization with the real
  bearer from `<state>.secrets/gh`. Response is rebuilt as
  HTTP/1.1 bytes for the proxy to encrypt back to the guest.
- `github_path_allowed` covers `/repos/<owner>/<repo>/...` and
  `/user[/...]` (gh CLI needs the latter for auth-status and
  `gh repo list`; doesn't expose specific-repo state).

**Image (`images/Dockerfile`).**
- Adds gh from the official apt repo (gh.io packages signed
  by /etc/apt/keyrings/githubcli-archive-keyring.gpg).

**Verified end-to-end on this host:**
- `agent-vm shell` from a project with `origin =
  github.com/wirenboard/agent-vm`:
  - `gh api repos/wirenboard/agent-vm` reaches GitHub (Bad
    credentials — outer-bridge nested limit; placeholder is
    actually reaching GitHub via our forwarder).
  - `gh api repos/octocat/Hello-World` returns our clean 403 with
    the allow-list named in the message.
  - `gh api user` allowed (gh auth-status etc. work).
- With `--repo octocat/Hello-World`, the allow-list grows; the
  octocat path now reaches GitHub, anthropics/claude-code still
  returns 403 listing both allowed repos.
- No host token in the guest mount (`grep -ras "<real prefix>"
  /agent-vm-state /root` empty).

**Out of scope (per user direction):** GitHub App device flow,
per-repo scoped tokens, Copilot subcommand. `github.com` smart-
protocol git push is NOT path-filtered — token is substituted but
agents could in principle push to other repos via the git wire
protocol. Filtering that requires stream-aware intercept hooks
upstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 05:46:15 +03:00
Evgeny Boger
f66a0b6e75 feat(phase5): OpenCode auth + host-cred security snapshot
OpenCode is the third v1 agent. Phase 4 covered Claude + Codex; this
finishes the auth/secret story.

**OpenCode auth.**
- `secrets::refresh_opencode` derives an OpenCode-shaped auth.json
  from the host's `~/.codex/auth.json` (same OpenAI account, different
  on-disk format). It writes `<state>/opencode/auth.json` with
  `{openai:{type:oauth, refresh:<placeholder>, access:<synthetic JWT>,
  expires:<far-future>, accountId:<from id_token>}}`.
- The `access` JWT is a *placeholder* — OpenCode parses it
  client-side (only `chatgpt_account_id` and `exp`) and sends it as
  `Authorization: Bearer <jwt>` on api.openai.com / chatgpt.com
  requests. The proxy substitutes the placeholder JWT for the real
  OpenAI access token (same on-disk file as Codex uses).
- `accountId` is pulled from the host Codex JWT's id_token payload via
  `decode_id_token_account` (adds `base64 = "0.22"`).
- New constants: `OPENCODE_OPENAI_ACCESS_PLACEHOLDER` (a synthetic
  alg-none JWT, deliberately minimal — see upstream issue #8 in PLAN
  about long-placeholder boot failures) and
  `OPENCODE_OPENAI_REFRESH_PLACEHOLDER`.
- `CredsState` grows `opencode_openai_access_token_file: Option<PathBuf>`
  pointing at the same host-only file as `openai_token_file`. The
  launcher uses this to register a third proxy-substitution entry
  in `run.rs` keyed off the OpenCode placeholder.
- `host_paths::host_opencode_auth_path` (`$HOME/.local/share/opencode/
  auth.json`) added for symmetry with the Claude/Codex helpers and
  consumed by the snapshot below.

**Security snapshot.**
- `snapshot_host_creds()` SHA-256s the three host credential files at
  launcher start. `verify_snapshot(&snap)` re-hashes after the sandbox
  exits and prints a one-line warning naming each file that changed.
- The Phase 4 OAuth refresh hook *legitimately* rewrites these mid-
  session; the warning surfaces anything else that mutates them — a
  bug to investigate.
- Wired into `run.rs` after the `Sandbox::remove` call, non-fatal.
- `CredsState::snapshot: Option<HostCredsSnapshot>` carries the
  pre-run state between the two calls.

Verified end-to-end on this host:
- `agent-vm opencode run "..."` reaches api.openai.com — the real
  OpenAI returns model-level errors (auth was accepted), proving the
  synthetic-JWT placeholder substitution works on the wire.
- `agent-vm claude -p "Say PHASE5_OK"` still returns a real response;
  no false-positive snapshot warning.
- 6/6 cargo tests still green.

PLAN.md: new upstream issue #8 documenting the long-placeholder
sandbox-boot failure that constrained the JWT design choice.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 05:23:28 +03:00
Evgeny Boger
8926abb308 docs(plan): refine v1 scope; add Phase 5/6/7; renumber deferred phases
This conversation's verification work surfaced both new in-scope items
that need plans and stale doc references after renumbering. Captures
all of it in one pass.

PLAN.md:

- v1 scope (in) expanded to call out: OpenCode auth, security snapshot
  of host creds, gh/git injection with per-launch repo allow-list,
  --mount, clipboard bridge, agent-vm-ccusage wrapper, Chrome DevTools
  MCP. Each had been ambiguous or implicit before.
- v1 scope (out) trimmed: items moved to in are gone from the out
  list; added explicit notes for GitHub App device flow (we reuse
  host gh instead), GitHub Copilot CLI, and setup-time
  --minimal/--disk.
- Phase 4 status bumped to include this session's commits (...85ffd34)
  and Codex+Claude end-to-end verification against real credentials.
- New Phase 5/6/7 sections with design notes, references to the
  original Bash agent-vm functions they port, and explicit "what we
  don't do" carve-outs.
- Phase 5 verification entry (2026-05-21) for the earlier session
  carried forward; Phase 4 verification entry (2026-05-24) for this
  session is new.
- "Still untested before Phase 4 can claim its 'Done when'" honest
  note: mid-session token rotation hasn't actually been observed in
  the wild; single-flight refresh isn't implemented yet.
- Three new entries in "Discovered upstream issues" (#5 IPv6 gateway
  DNS unresponsive, #6 StdinMode::Null doesn't behave as /dev/null
  for codex, #7 exec_with is buffer-until-exit only).
- Phase 9 (was Phase 6) gets a new bullet about upstream-fixing or
  formalizing the IPv6 DNS workaround.

Renumbered the deferred phases: old "Phase 5 — Fast-launch (deferred)"
is now Phase 8; old "Phase 6 — Distribution + polish + docs" is now
Phase 9. Fixed the cross-references in README.md (phase status list)
and ARCHITECTURE.md (three mentions of Phase 5/5+ polish — corrected
to Phase 9 or to the 2026-05-24 verification session where the
streaming-output change actually landed).

README.md status list expanded to name the three new phases and the
Codex E2E verification result.

ARCHITECTURE.md gets a new "Token files live outside the guest bind
mount" subsection from the earlier doc-update pass (already in HEAD
via db50cc2) and the renumbering touch-ups.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 04:59:46 +03:00
Evgeny Boger
85ffd34cea fix(run): unblock codex e2e — resolv.conf prelude, stdin null, streaming
Three changes that together get `agent-vm codex exec` working end-to-
end (verified: returns a real gpt-5.5 response in ~7s):

1. **Strip IPv6 nameservers from /etc/resolv.conf before the agent
   runs.** microsandbox's agentd writes both v4 and v6 gateway DNS at
   boot, but the v6 gateway was observed unresponsive in at least one
   libkrun configuration. glibc's getaddrinfo skips a broken v6
   resolver and uses v4; codex's Rust async resolver returns EAI_AGAIN
   instead. Result: codex hung at "failed to lookup address
   information" for chatgpt.com even though `getent hosts chatgpt.com`
   returned immediately. A two-line bash prelude in front of the agent
   command sed's out lines whose nameserver value contains a colon
   (v4 addresses never do; v6 always do).

2. **Force stdin to /dev/null in non-TTY mode.** exec_with's default
   StdinMode::Null didn't satisfy codex 0.133's `exec` subcommand: it
   blocked indefinitely waiting for what it thought was unbounded
   interactive input. Backgrounding (`&` in bash) fixed it because
   that auto-redirects stdin to /dev/null. The wrapper now does the
   redirect explicitly with `[ -t 0 ] || exec < /dev/null` so a real
   TTY launch is unaffected.

3. **Stream stdout/stderr live in non-TTY mode** via exec_stream_with
   instead of buffer-until-exit via exec_with. Long-running agent
   commands (codex exec can take 30+s) used to look completely silent,
   then dump everything on exit — making "is it hung or just slow?"
   indistinguishable. Now output appears as it's produced and partial
   output survives Ctrl-C / timeout.

shell_escape() is a small new helper to forward the user's agent_args
through the prelude correctly. Unit-tested.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 04:13:56 +03:00
Evgeny Boger
523353637c fix(secrets): synthetic JWT for OpenAI id_token placeholder
The host's OpenAI `tokens.id_token` JWT was being copied verbatim into
the guest's auth.json — codex's id_token decodes to user email,
chatgpt_account_id, plan type, org list, user_id. That's a PII leak
on top of the access_token issue: any process in the guest could
`cat /agent-vm-state/codex/auth.json | jq .tokens.id_token` and read
the user's identity.

The constant OPENAI_ID_PLACEHOLDER already existed and was used by the
refresh hook's synthesized response; the launcher's initial snapshot
just forgot to apply it. Adding the substitution exposed a second
problem: codex parses tokens.id_token client-side at startup and the
static string "MSB_PLACEHOLDER_OPENAI_ID_TOKEN_v1" isn't a JWT, so
codex refused to load ("invalid ID token format at line 1 column N").

Make the placeholder a syntactically valid alg-none JWT whose payload
carries only obvious-placeholder fields (email=placeholder@msb.local,
chatgpt_account_id=0000..., chatgpt_user_id=user-placeholder, etc.).
The signature segment is kept as the literal "MSB_PLACEHOLDER..._v1"
marker so a leak-grep for the marker still flags any place this
escapes.

Verified end-to-end on a real host Codex auth: guest auth.json's
id_token decodes to placeholder PII, leak grep for any real-token
prefix or the user's email returns absent, and codex 0.133 loads
and reaches chatgpt.com successfully.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 04:13:39 +03:00
Evgeny Boger
db50cc26ab fix(secrets): keep real tokens out of the guest bind mount
The launcher bind-mounts the per-project state_dir into the guest at
/agent-vm-state (a single mount, to stay under libkrun's virtio-IRQ
cap). secrets.rs wrote the host's real access token to
state_dir/tokens/{anthropic,openai} — i.e. INSIDE that mount — so the
in-VM agent could `cat /agent-vm-state/tokens/anthropic` and read the
host bearer, silently defeating the whole Phase 3/4 "real tokens never
enter the VM" guarantee.

The nested test host masked this: there the "real" token is itself the
outer agent-vm bridge's placeholder, so a leak isn't a real leak. It
only surfaced when grepping the guest filesystem during end-to-end
verification on a host with a genuine credential.

Move the token files to a host-only sibling dir <hash>.secrets/ (0700),
derived from state_dir by secrets::{anthropic,openai}_token_path so the
launcher and the refresh hook agree on the path. The microsandbox proxy
reads them host-side via SecretValue::File, so they never need to be
mounted. Best-effort remove of the legacy state_dir/tokens on launch so
an upgrade doesn't leave a real token lingering in the mount.

Adds token_files_live_outside_the_guest_mount as a regression guard.
Verified in-guest: no host token anywhere under /agent-vm-state;
credentials.json and PID1 environ show only placeholders;
api.anthropic.com server cert is issued by CN=microsandbox CA.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:33:44 +03:00
Evgeny Boger
2e5b5c546e fix(setup): robust registry-state detection on Docker 29.x
`docker inspect` of a missing container emits a stray blank line to
stdout before exiting non-zero on Docker 29.x, so
`state=$(... || echo missing)` captured "\nmissing" and never matched
the `missing` case — ensure_registry then tried to `docker start` a
container that doesn't exist and aborted setup.

Strip whitespace and treat empty as missing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:33:32 +03:00
Evgeny Boger
c7f7386b7c secrets: inject_basic_auth=false on agent-vm secrets so the fast path fires
Lets the upstream-side "no eligible placeholder in chunk → forward
verbatim" optimization actually engage. Without this, the upstream
fast path bails because basic_auth wants to decode base64 even when
no Basic header is present.

We only ever substitute Bearer headers; inject_basic_auth=true was
dead weight from the SecretInjection::default().

Plus: bump vendor/microsandbox to dda4a87 for the matching fast-path
fix on the proxy side.
2026-05-18 02:14:49 +03:00
Evgeny Boger
4786c2631d chore: bump vendor/microsandbox to pick up ALPN + body-bytes fixes
Pulls in two correctness fixes likely affecting codex traffic against
chatgpt.com: HTTP/1.1 ALPN pinning so a server preferring h2 can't
force a protocol the proxy can't parse; and body bytes are now
forwarded verbatim unless body substitution is actually configured.
2026-05-18 01:15:27 +03:00
Evgeny Boger
18173911b8 feat(launcher): promote --memory + --cpus to CLI flags
Add `--memory <gib>` and `--cpus <n>` on every launcher subcommand
(claude/codex/opencode/shell). Both have env-var counterparts via
clap's `#[arg(env = ...)]`:

  --memory   ↔ AGENT_VM_MEMORY_GIB   (was AGENT_VM_MEMORY_MIB)
  --cpus     ↔ AGENT_VM_CPUS         (unchanged)

Memory takes GiB (matching the original Bash agent-vm UX: `--memory 8`
= 8 GiB). The internal MiB conversion (×1024) is fully checked.

Drops the parse_env_u32 helper since clap handles the validation now,
and removes the now-unused `bail!` import.

Default stays 2 GiB / 2 vCPU.
2026-05-18 01:03:55 +03:00
Evgeny Boger
d280d78ba2 fix(launcher): pre-approve project folder + always pass --dangerously-skip-permissions to claude
Two more bypasses the in-VM Claude was tripping on:

- "Do you trust the files in this folder?" prompt. Lives in
  ~/.claude.json under `projects.<absolute-path>.hasTrustDialogAccepted`.
  secrets::refresh now takes the in-guest project path (the same one
  bind-mounted at the cwd's host path) and pre-writes the trust flag
  + project onboarding flag for that path. Re-uses the existing
  ~/.claude.json symlink wiring from the previous commit.

- `--dangerously-skip-permissions` was not being passed to the in-VM
  claude. New Agent::default_args() returns the per-agent default
  argv prefix; for Claude that's the dangerous-skip flag. The launcher
  prepends it before the user's own args, deduping if the user already
  passed it explicitly. Same flag the original Bash agent-vm used; the
  microVM is the sandbox so the in-VM "are you sure?" prompt adds no
  protection and breaks agent-mode flows.

Codex / OpenCode get an empty default_args slice; the dangerous-mode
equivalents for those agents already land via Codex's config.toml
written in the previous commit (sandbox_mode = "danger-full-access",
approval_policy = "never").
2026-05-18 01:00:53 +03:00
Evgeny Boger
f1632b8e9e fix(secrets): bypass in-VM agent onboarding wizards
Claude Code and Codex CLI both show a first-run wizard (theme picker
for Claude, "trust this folder?" + approval-policy prompts for Codex)
that blocks the agent from doing useful work until a human types
something interactively. The original Bash agent-vm pre-wrote three
files to suppress them; we never ported that.

Adds `write_agent_config_defaults` to secrets.rs, called from
`refresh()` regardless of whether the host has credentials for the
provider. Three files get the bypass treatment:

- <state>/claude/settings.json — theme, hasCompletedOnboarding,
  skipDangerousModePermissionPrompt, effortLevel ("xhigh"). Merge
  semantics: existing user settings preserved, our flags forced.
- <state>/claude.json — the per-user onboarding-state file Claude
  Code actually checks for the wizard trigger. Lives at $HOME root,
  outside our existing /agent-vm-state/claude symlink. run.rs now
  also symlinks /root/.claude.json → /agent-vm-state/claude.json
  so user-side state persists across launches.
- <state>/codex/config.toml — sandbox_mode = "danger-full-access"
  and approval_policy = "never". The microVM is the sandbox, so
  Codex's redundant approval prompts add no security and break
  agent-mode flows.

All three writes use atomic write-and-rename; the JSON ones merge
into an existing file so the bypass is force-set every launch
without trampling user tweaks.

Verified inside the guest after a clean state: `cat /root/.claude/
settings.json` and `cat /root/.claude.json` both show the bypass
flags; `cat /agent-vm-state/codex/config.toml` is the expected TOML.
2026-05-18 00:55:48 +03:00
Evgeny Boger
8e03a835bd chore: bump vendor/microsandbox to pick up patched msb version tag
`msb --version` now reports `0.4.6+agent-vm.phase4` so a host running
the upstream prebuilt instead of our patched binary is identifiable
without comparing hashes.
2026-05-18 00:38:59 +03:00
Evgeny Boger
bc523b84d9 simplify: shared host_paths utility; placeholder + host consts; drop TOCTOU
Code-review pass after Phase 4 surfaced four classes of duplication
and one efficiency regression. None of it functional, all of it
cleanup.

- New crates/agent-vm/src/host_paths.rs centralizes state_root,
  host_claude_creds_path, host_codex_auth_path, and atomic_write.
  secrets.rs, intercept_hook.rs, and pulled_marker.rs all stop
  rolling their own.
- All Phase 3/4 placeholder strings are now consts in secrets.rs:
  ANTHROPIC_{ACCESS,REFRESH}_PLACEHOLDER, OPENAI_{ACCESS,REFRESH,ID}_
  PLACEHOLDER. Hardcoded literals in intercept_hook.rs and the
  refreshToken/refresh_token writes in secrets.rs use them.
- Host + path constants too: ANTHROPIC_{API,OAUTH}_HOST, OPENAI_
  {API,CHATGPT,OAUTH}_HOST, ANTHROPIC_OAUTH_TOKEN_PATH, OPENAI_
  OAUTH_TOKEN_PATH. run.rs's allow_host calls + intercept_hook.rs's
  SNI dispatch share them, no more typo-one-character-from-silent-
  breakage.
- intercept_hook.rs gets MSB_INTERCEPT_SNI via a typed clap arg
  (#[arg(env = ...)]) instead of std::env::var, so --help shows it.
- TOCTOU fix in secrets::refresh_*: drop the `host_path.exists()`
  pre-check; let read_to_string return ErrorKind::NotFound which we
  map to Ok(None). One less race, one less syscall.
- intercept_hook.rs's http_200_json + error_response now share a
  build_response(code, reason, body) helper.
- Tightened doc comments in secrets.rs and removed phase-marker
  comments ("Phase 3:", "Phase 4:") in run.rs / setup.rs / main.rs
  — they tell *when* not *why* and won't age well.
- pulled_marker.rs's marker write switches from raw write+rename to
  the shared atomic_write helper (mode 0644 since the marker isn't
  sensitive).
2026-05-17 23:51:27 +03:00
Evgeny Boger
8e262c03fb Phase 4: file-backed secrets + OAuth refresh interception
In-VM agent's OAuth refresh attempts now trigger a host-side rotation
via a microsandbox TLS-intercept request-interceptor hook. The hook
spawns the host claude/codex CLI to do the real refresh, then
synthesizes a placeholder response. Combined with switching from
SecretValue::Static to SecretValue::File (which the patched msb
re-reads on every connection), long sessions survive token rotation
without manual intervention.

Pieces:

- vendor/microsandbox submodule pointer bumped to include the
  interceptor extension (commit cf00a11 on agent-vm-secret-file).
- msb_install.rs: cargo build --release -p microsandbox-cli --bin msb
  from vendor/, point MSB_PATH at the result so the patched binary is
  what runs the VM. The user's ~/.microsandbox/bin/msb is untouched.
- intercept_hook.rs: hidden `agent-vm _intercept-hook` subcommand.
  Reads the OAuth refresh request from stdin, spawns the host CLI to
  rotate the host credential file the normal way, re-reads the host
  file, rewrites <state>/tokens/<provider> so the next non-refresh
  request gets the new bearer via SecretValue::File, synthesizes an
  OAuth response with placeholder tokens, writes to stdout.
- secrets.rs: switched from Static(token) to File(<state>/tokens/…).
- run.rs: registers the interceptor with two rules
  (platform.claude.com/v1/oauth/token and auth.openai.com/oauth/token).
- ARCHITECTURE.md gets a Phase 4 section covering the two-layer
  refresh flow, the msb/cli build-target gotcha (~30 min debugging
  lesson: vendor/microsandbox/target/release/microsandbox is a
  5-line shim, not the real binary), and the deliberate
  non-features (no proactive expiry timer, no global msb install,
  no concurrent-refresh single-flight).

Smoke verified end-to-end on the nested-VM test host: in-VM
`curl POST https://platform.claude.com/v1/oauth/token` returns
HTTP 200 with a fresh OAuth JSON response containing the
placeholder tokens, expires_in derived from the just-rotated host
expiresAt. The host claude -p ran and updated the host file as a
side effect.
2026-05-17 23:38:40 +03:00
Evgeny Boger
273b0261b8 docs: drop proactive expiry watcher from Phase 4
Per discussion: once SecretValue::File re-reads the host token file on
every connection and the OAuth refresh endpoint is MITM'd (spawning a
host-side claude -p / codex exec on demand to trigger the host CLI's
own refresh), the proactive "token nearing expiry" timer is just
belt-and-suspenders.

External rotations (user runs claude on the host) are picked up
naturally because the file is re-read per connection. Internal
expiries (no host activity) are caught when the in-VM agent itself
hits a 401 and attempts refresh, which triggers the MITM flow. No
need to time-bomb anything ahead of time.

Smaller phase, fewer moving parts to maintain.
2026-05-17 21:33:27 +03:00
Evgeny Boger
8cc036bb96 Phase 3: host-rooted Claude/Codex credentials never enter the VM
The big architectural payoff of microsandbox. Real OAuth access tokens
stay on the host; the in-VM agent only ever sees placeholder strings
in its credentials.json files, env vars, and outgoing request headers.
The substitution into the real token happens at the network layer in
microsandbox's TLS-intercept proxy.

Two-layer placeholder dance per provider:

1. agent-vm reads host ~/.claude/.credentials.json and
   ~/.codex/auth.json at every launch, extracts the access token, and
   passes it to microsandbox as a SecretValue::Static(...) on the
   sandbox builder with allow_host for the API + OAuth endpoints.
2. agent-vm writes a "placeholder credentials" JSON into the per-project
   state dir using a stable placeholder string and the host file's
   non-secret fields (expiresAt, scopes, account_id, etc.) so the
   in-VM agent sees a plausible shape.
3. microsandbox's TLS intercept splices the real token in on the way
   out — the guest never holds the real value.

What landed:

- vendor/microsandbox submodule bumped to agent-vm-secret-file branch
  (commits b87d6bb + b95614c): adds SecretValue { Static, File } enum
  with bare-string + NUL-sentinel wire format so existing prebuilt msb
  daemons keep working with Static values. File variant is forward-
  looking infrastructure for Phase 4.
- crates/agent-vm/src/secrets.rs: snapshots host creds, writes
  placeholder credentials files atomically (0600), redacts in Debug.
- crates/agent-vm/src/run.rs: wires .network(|n| n.tls(...).secret(...))
  per provider with the right allow-host lists, and sets IS_SANDBOX=1
  so Claude Code's "don't run as root" guard yields.
- AGENT_VM_DEBUG_CONFIG=1 dumps the effective SandboxConfig JSON for
  debugging.
- serde_json bumped to "preserve_order" so the dumped config is
  diffable in a sensible order.

Smoke verified end-to-end up to the network layer (TLS cert chain
shows CN=microsandbox CA on api.anthropic.com requests, debug config
dump confirms the secret value reaches msb, the substitute path is
exercised). The final "Anthropic returns a real response" leg can't
be verified on this nested test host because of an outer credential
bridge; structurally equivalent to the Bash agent-vm's
credential-proxy flow.

Phase 4 will rebuild ~/.microsandbox/bin/msb from our fork to
activate SecretValue::File, add the inotify/proactive refresh loop,
and short-circuit the platform.claude.com/v1/oauth/token endpoint
so long sessions survive token rotation.
2026-05-17 21:21:28 +03:00
Evgeny Boger
0f9c1d08d2 docs: defer Phase 5 (snapshots don't speed up launches)
Reading vendor/microsandbox/crates/microsandbox/lib/snapshot/mod.rs
confirms that microsandbox snapshots are filesystem-only: they capture
the stopped sandbox's writable upper layer plus metadata pinning the
immutable lower. Booting from a snapshot still goes through the full
libkrun kernel boot (~1 s) — the only thing snapshots actually save is
the EROFS materialize step on a re-pull, which we already pay rarely
because pulls are explicit.

So filesystem snapshots are the wrong instrument for the "<500 ms
launch" goal. The real lever is detached mode (create_detached + Sandbox::
get-style attach) which microsandbox already exposes, but it's a real
shape change (lifecycle subcommands, sandbox reuse strategy, idle-timeout
policy, persistent in-VM state) so it's deferred pending a clearer
product call. Phase 3 (host-rooted secrets) is the next phase to start.
2026-05-17 20:49:00 +03:00
Evgeny Boger
d979c31028 docs: refresh PLAN.md after Phase 2.x and re-plan remaining phases
- Add a "Phase 2.x — Post-MVP polish" section listing the work that
  landed between Phase 2 and Phase 3 (tracing, registry auto-recover,
  host-path cwd, profile flag, pull progress bar, explicit pull +
  update banner). Each entry references its commit so the next reader
  doesn't redo any of it.
- Expand Phase 3 with the things we learned: SecretEntry.value is
  captured at builder time so we need a File variant upstream; TLS
  intercept must be enabled for substitution to fire; only access
  tokens cross into the marker file, never refresh tokens.
- Add Phase 5 (snapshot-based fast launch) as a real phase rather than
  a vague polish bullet. Surfaced naturally from the "why is pull so
  slow" profiling — the EROFS materialize cost is the dominant pull
  cost, and snapshot/restore sidesteps both that and the ~1s libkrun
  boot. DoD: `agent-vm shell -- -c true` well under 500ms.
- Rename Phase 5→6 ("Distribution + polish + docs"). Add concrete
  items: microsandbox runtime auto-install, CLI flag promotion,
  .agent-vm.runtime.sh hook, CI smoke, macOS/aarch64 builds.
- Add "Discovered upstream issues" section recording the four
  microsandbox bugs/quirks we worked around (PullPolicy::Always not
  refreshing manifest digest; LayerDownloadProgress elided on fast
  registries; libkrun IRQ cap; no SDK-level "resolve manifest" helper).
  These should eventually be filed upstream.
- README status list extended with Phase 2.x done and 5/6 split.
2026-05-17 20:36:11 +03:00
Evgeny Boger
d3914b990b fix(pull): never invalidate the cache before the new image is ready
Previous design called Image::remove(force=true) at the start of every
`agent-vm pull` (and therefore of `agent-vm setup`, which chains into
it) so subsequent Image::get would return the new digest. That worked
around microsandbox's "Always doesn't refresh the cached manifest
record" bug, but at a real cost: between the remove and the next
successful pull, the cache was empty. ^C, network failure, or a
registry hiccup left the user with no cached image and the next
`agent-vm shell` triggering a multi-minute IfMissing re-pull.

New design: track the last successfully-pulled per-platform digest in
our own marker file under XDG_STATE_HOME (pulled_marker module),
written atomically (write-and-rename) only after a pull completes.
image_check::check_for_update now compares the registry against this
marker instead of asking microsandbox for the cached digest. pull.rs
drops Image::remove entirely; the marker is updated only on success.

Result: microsandbox's cache is untouched during a pull and remains
fully usable if the pull is interrupted. The only thing that can be
briefly wrong is the update-available banner — and it self-corrects
on the next pull.

Verified: (1) fresh state → no spurious banner, (2) pull → marker
written, (3) launch → banner clears, (4) new push → banner returns
with correct registry digest.
2026-05-17 20:24:51 +03:00
Evgeny Boger
bfab9d3daf feat: explicit agent-vm pull + per-launch update-available banner
User feedback: re-checking the registry on every launch (PullPolicy::
Always) made fast launches feel surprising — when there *was* a new
image, the next innocuous `agent-vm shell` blew up into a multi-minute
pull. Move pulls onto an explicit `agent-vm pull` verb and just notify
on launch when the registry has something newer.

- pull.rs: new `agent-vm pull` subcommand. Image::remove(force=true)
  the cached entry first (PullPolicy::Always empirically re-fetches
  layer blobs but does NOT update the cached manifest digest record,
  so subsequent Image::get returns the stale digest forever) then
  boot a throwaway sandbox to trigger a fresh pull, then stop+remove.
- image_check.rs: cheap manifest probe. Compare microsandbox's cached
  per-platform manifest digest against the registry's. Critical
  subtlety: microsandbox stores the per-platform (linux/amd64) digest
  while a HEAD on a tag returns the multi-arch *index* digest. A
  `docker tag X Y; docker push Y` churns the index digest but reuses
  the per-platform manifest, so naively comparing the two would warn
  forever. We GET the index, find the linux/amd64 entry, take its
  digest, compare that.
- run.rs: switch PullPolicy back to IfMissing, call
  notify_if_update_available before create. Failures (registry down,
  etc.) are silently swallowed so offline launches still work.
- setup.rs: chain an explicit pull_image() call after build.sh so
  setup itself updates the cache.
- Cargo.toml: add reqwest + serde_json.

Verified end-to-end: (1) churn list digest only -> no banner.
(2) Real Dockerfile change -> "A newer image is available (cached X,
registry Y)" banner. (3) After `agent-vm pull` -> banner gone.
2026-05-17 20:09:47 +03:00
Evgeny Boger
66ed8f3234 fix(pull): split download and materialize into separate UI phases
Two related bugs the user spotted: the bar jumped from 0 to ~350 MiB
instantly (microsandbox elides LayerDownloadProgress on a fast registry
and only fires LayerDownloadComplete per layer, so we inc'd a layer's
full compressed size in one tick), and after that the MiB/s never
stopped climbing (the burst stayed in indicatif's average for the rest
of the pull, while the slow materialize bytes only added small samples).

A reset_eta() after the burst wasn't enough on its own because the
estimator still measured (current_pos - 0) / elapsed_since_reset and
the current_pos was sitting at 350 MiB.

Real fix: don't mix the two workloads on one bar. Phase 1 (resolve +
download) is now a spinner with text — no bar, no rate, since the
numbers from a localhost burst aren't meaningful anyway. On the first
materialize event we finish_and_clear() the spinner and replace it
with a fresh ProgressBar sized in materialize bytes only. Indicatif's
estimator starts from a clean slate and only sees materialize samples,
so the rate reflects actual throughput.

After warmup over a full pull: min 1.0, max 5.5, mean 2.4 MiB/s
(vs. min 1.5, max 99.2, mean 11.5 with the previous design).
2026-05-17 19:55:59 +03:00
Evgeny Boger
3ffd6b6513 feat(pull): byte-weighted bar with honest ETA; drop misleading detach hint
The "steps" bar gave indicatif a moving window of mixed step durations
(a download-complete is ~10ms; a materialize-complete is anywhere from
100ms to 30s for the Node.js layer) so its ETA bounced from 17s to 29
minutes between samples.

New design: bar length and position are in *bytes*. Length starts as
total_download_bytes + 3*total_download_bytes (heuristic 3:1
decompression ratio for Debian-based images) and is corrected to the
real number as each layer's actual materialize total comes in from
LayerMaterializeProgress.total_bytes. Bar position is the sum of bytes
downloaded + bytes materialized, with deltas tracked per layer so
microsandbox's "no progress events, only Complete" fast-path still
credits the right amount.

Observed in a fresh-cache pull: 1.37 GiB initial estimate → 1.22 GiB
once the real materialize totals were known, byte-per-sec rate stable
within each phase, ETA converged from "5m" to "2m" to "30s" as the
pull progressed — no more multi-minute oscillation.

Also drop the "Ctrl-P Ctrl-Q to detach" hint on attach: our launcher
unconditionally stops + removes the sandbox after attach() returns
regardless of whether the user exited or detached, so the hint
misrepresents the behavior. Detach-keeping-VM-alive is Phase 5 work.
2026-05-17 19:39:08 +03:00
Evgeny Boger
984680ab27 polish(pull): add a steady-tick spinner to the bar template
The bar position only advances on LayerDownloadComplete or
LayerMaterializeComplete. For the materialize step that fires roughly
once every 15s, so the bar visually freezes between layers even while
microsandbox is working hard. Embed a {spinner} in the bar template and
leave the steady_tick enabled past the Resolved transition so the line
always shows activity.
2026-05-17 19:30:16 +03:00
Evgeny Boger
e1b1568779 fix(image): restore the apt cache cleanup lines that the last commit dropped
`rm -rf /var/lib/apt/lists/*` got removed from two RUN lines as collateral
damage from the local marker-test loop in the previous test session
(backup-and-restore via cp). Bring them back so the image stays small.
2026-05-17 19:26:03 +03:00
Evgeny Boger
388d18e599 fix(pull): one bar instead of two, no stuck-at-zero, clean finish
User feedback on the two-bar layout:
- "Two lines instead of one"
- "Doesn't work" — the download bar sat at 0 B for a long time on a
  local-registry pull because microsandbox usually elides
  LayerDownloadProgress events when the source is fast, leaving the
  old code with nothing to inc(). The bar looked frozen even while
  layers were actually being downloaded.

New design: a single bar of length 2*layer_count. Each layer ticks once
when its download finishes (LayerDownloadComplete now always credits the
bar, even with no prior progress events) and once when its materialize
finishes. The bar's message text reflects the current phase:
"Resolving... -> Downloading layer N/M -> Materializing layer N/M ->
Stitching -> Writing fsmeta -> Writing VMDK -> Rootfs ready". On
completion the bar is wiped (finish_and_clear) so subsequent output
starts on a clean line — no leftover "Manifest resolved" leak from the
old finish_with_message path.

Verified end-to-end under a pty (script -qc): single line that advances
0/14 -> 14/14 over ~2 min, never stuck at zero, no second row.
2026-05-17 19:25:18 +03:00
Evgeny Boger
2489168def feat(run): show pull progress bars during image fetch + materialize
A cold-cache or new-tag launch downloads ~350 MiB of compressed layers
plus a 1-2 minute EROFS materialize step, and until now produced no
output between "Booting sandbox..." and the agent actually starting.
That looked indistinguishable from a hang for the first ~2 minutes.

Wire microsandbox's create_with_pull_progress into both the launcher
(run.rs) and the setup verify path (setup.rs). A small renderer
(pull_progress.rs) collapses microsandbox's per-layer event stream into:

- a manifest-resolve spinner
- one overall byte-progress bar (sized from Resolved.total_download_bytes)
- one layer-count bar for the materialize step
- a spinner for the stitching tail (fsmeta + VMDK writes)

Per-layer bars are deliberately omitted; the OCI image has 7+ small
layers and a wall of per-layer bars is louder than useful.

indicatif's default ProgressDrawTarget::stderr() correctly detects when
stderr isn't a TTY (CI, pipes, smoke tests under `sg`/`sudo -c`) and
silently no-ops there, so this never produces noisy output for scripts.
Verified rendering under a pty (script -qc) and silence under a pipe.
2026-05-17 19:15:52 +03:00
Evgeny Boger
55d52385bb fix(image): re-check manifest on every boot so rebuilds aren't stale
microsandbox's default PullPolicy::IfMissing keys its layer cache by
*reference*. When we rebuild and re-push under the same :latest tag
(the entire `agent-vm setup` flow), the reference is unchanged, so
microsandbox happily boots the previous manifest and never sees the
new image.

Set PullPolicy::Always on both the launcher path (run.rs) and the
setup verify path (setup.rs). Always means "re-check the manifest at
the registry; reuse cached layers whose digests still match" — so the
cost when nothing changed is one HTTP round-trip + a few bytes.

Verified end-to-end: dropped a /etc/agent-vm-marker file into the
Dockerfile, pushed, observed sandbox sees the marker; rebuilt without
the marker, observed the next sandbox no longer sees it. Both
transitions required PullPolicy::Always — IfMissing kept showing the
old version in both directions.
2026-05-17 18:59:53 +03:00