diff --git a/PLAN.md b/PLAN.md index 3f26d90..4c32d67 100644 --- a/PLAN.md +++ b/PLAN.md @@ -111,44 +111,158 @@ CODEX_HOME redirect, env propagation). The live API smoke was deferred — see ARCHITECTURE.md "What Phase 2 deliberately doesn't do". Phase 3's host-OAuth work closes the gap naturally. -### Phase 3 — Static host-rooted secrets [pending] +### Phase 2.x — Post-MVP polish [done — commits `7608f27`..`d3914b9`] + +A series of small fixes landed between Phase 2 and Phase 3, all triggered by +real testing on the user's laptop. Listed here so the next reader knows +they're in already and doesn't redo the work. + +- **`RUST_LOG` wiring** (`7608f27`). `tracing-subscriber` initialized in + `main`, defaults to `warn`. The microsandbox stack is silent otherwise. +- **Auto-recover the local registry container** (`a57ed6d`). `build.sh`'s + `ensure_registry` was rewritten as a state machine that handles every + `docker ps` state, polls `/v2/` after start, and recreates from scratch + if a stale container is running with no port mapping. Plus per-phase + banners so long waits don't look like a hang. +- **Mirror the host project path inside the guest** (`92ff582`). `cwd` is + bind-mounted at the same absolute path, so anything the agent emits + (compiler errors, stack traces, file:line references) is interpretable on + the host. Paths under tmpfs mount points (`/tmp`, `/run`, `/dev/shm`, + `/var/run`) fall back to `/workspace` with a warning, because the guest + tmpfs-mounts them at boot and wipes any patch-created mount point. +- **`AGENT_VM_PROFILE=1`** (`127f6b3`). Prints per-phase wall-time + (create / run / stop / remove) for the launcher. Confirmed total is + ~1.5 s, dominated by VM boot (~1.0 s of libkrun kernel boot). +- **Pull progress bar** (`2489168`..`66ed8f3`..`3ffd6b6`..`984680a`). + Two-phase indicatif renderer: spinner with text during download, real + byte-weighted bar during materialize. Single line, single spinner, ETA + based on materialize-only rate (no more "29 minute" → "17 second" jumps). +- **`agent-vm pull` + per-launch update-available banner** + (`bfab9d3`..`d3914b9`). Pulls are explicit; `agent-vm shell` only does a + cheap manifest-digest HEAD against the registry and prints a banner when + the per-platform digest differs from what we last pulled. The "what we + last pulled" digest is tracked in our own marker file + (`~/.local/state/agent-vm/pulled-digests/`), atomically written + only after a successful pull, so an interrupted pull never leaves the + microsandbox cache in an empty or stale state. + +### Phase 3 — Static host-rooted secrets [next] + +The big architectural payoff of moving to microsandbox: real tokens never +enter the VM. - Branch `vendor/microsandbox` to add a `SecretValue::File(PathBuf)` variant - alongside the existing `String` value. The TLS-intercept proxy re-reads the - file on every substitution. + alongside the existing `String` value. The TLS-intercept proxy re-reads + the file on every substitution. Required because microsandbox's + `SecretEntry.value: String` is captured at sandbox-builder time, with no + runtime update path. +- Enable TLS interception on the sandbox (`network(|n| n.tls(...))`) — + microsandbox's secrets are only substituted on TLS-intercepted + connections to allowed hosts, and the guest needs the sandbox CA in its + trust store. - `agent-vm` on startup snapshots host `~/.claude/.credentials.json` and - `~/.codex/auth.json` into `/tokens/{anthropic,openai}.token` files. + `~/.codex/auth.json` into `/tokens/{anthropic,openai}.token` + files (just the `access_token`, never the refresh token). - Register file-backed secret entries for `api.anthropic.com`, - `api.openai.com`, `chatgpt.com`, `platform.claude.com`, etc. -- Cross-instance lock so two `agent-vm` processes don't fight over the token - files. + `api.openai.com`, `chatgpt.com`, `platform.claude.com`, + `auth.openai.com`, etc. +- Cross-instance lockfile under `/tokens/` so two `agent-vm` + processes don't race when both want to refresh. -**Done when:** inside the guest, `cat /proc/$$/environ | tr '\0' '\n' | grep -ANTHROPIC` shows only placeholders, while a real Claude request succeeds. +**Done when:** inside the guest, `cat /proc/$$/environ | tr '\0' '\n' | +grep -i token` shows only placeholders, while a real Claude (host OAuth) +request succeeds end-to-end and writes a turn to disk. ### Phase 4 — Refresh semantics [pending] +Tokens rotate; long-running sandbox sessions must survive that without +re-attaching. Phase 3 makes the access token swappable; this phase teaches +agent-vm to actually do the swap. + - inotify (Linux) / kqueue (macOS) watcher on host creds files: when host - Claude/Codex rotates tokens, we re-snapshot to the file microsandbox watches. -- Proactive expiry watch: when the access token is < 5 minutes from expiry - and no host activity has refreshed it, spawn `claude -p "ping" --model - sonnet` / `codex exec --skip-git-repo-check "Reply OK"` on the host. + Claude/Codex rotates tokens externally (the user runs `claude` on the + host, etc.), we re-snapshot to the file microsandbox watches. +- Proactive expiry watch: when the access token is < 5 minutes from + expiry and no host activity has refreshed it, spawn `claude -p "ping" + --model sonnet` / `codex exec --skip-git-repo-check "Reply OK"` on the + host so the host-side CLI rotates the file. - Single-flight per credential; cross-instance lockfile so concurrent `agent-vm` instances don't all kick off a refresh at once. -**Done when:** a multi-hour session crosses a token rotation without the agent -seeing an auth error. +**Done when:** a multi-hour session crosses a token rotation without the +agent seeing an auth error and without manual intervention. -### Phase 5 — Polish & docs [pending] +### Phase 5 — Snapshot-based fast launch [pending — perf win] -- `.agent-vm.runtime.sh` project hook. -- `--memory N` flag (passed through to microsandbox builder). -- Full arg passthrough to the agent command. -- README rewrite: install, setup, usage, troubleshooting. -- Smoke tests: at least one end-to-end test that boots the image and runs a - trivial agent command. +Surfaced during Phase 2.x profiling and discussed with the user when they +asked why `agent-vm pull` was so slow. Every `agent-vm shell` currently +spends ~1.0 s booting the libkrun kernel + initing the guest agent; +microsandbox has a snapshot/restore API that should drop that to ~100 ms +*and* skip the EROFS materialize step on a re-pull. -**Done when:** README is publishable; one CI smoke test green. +- One-shot snapshot generation: `agent-vm setup` (or a new `agent-vm + snapshot`) boots the image once, lets it reach a quiet steady state, + takes a snapshot, and stashes it under `/snapshots/`. +- Launcher: if a snapshot exists, use `Sandbox::builder(...).from_snapshot + (snapshot_path)` instead of `.image(...)`. Falls back to cold boot if + the snapshot is missing or wrong-digest. +- Re-snapshot whenever `agent-vm pull` lands a new manifest (snapshots are + pinned to a specific image digest by microsandbox). +- Bake the per-agent symlinks + state mount point into the snapshot so + the launcher doesn't need to re-apply patches each invocation. + +**Done when:** `time agent-vm shell -- -c true` is well under 500 ms in +the steady state. + +### Phase 6 — Distribution + polish + docs [pending] + +The "ready to share with a teammate" phase. + +- **Auto-install of the microsandbox runtime.** The agent-vm binary ships + on its own; `~/.microsandbox/{bin/msb, lib/libkrunfw.so.5.2.1}` are + needed but not bundled. Wrap `microsandbox::setup::install()` so first + run downloads them automatically if missing. Verify version matches the + prebuilt the binary was built against. +- **CLI flag promotion.** `--memory N`, `--cpus N`, `--image REF`, + `--no-update-check`. Today these are env-var only (`AGENT_VM_*`). +- **`.agent-vm.runtime.sh` project hook.** Script executed in the guest + immediately before the agent starts, for project-local setup + (`npm install`, `docker compose up`, etc.). +- **README rewrite.** Install, prereqs, setup, usage, troubleshooting, + the registry/marker/snapshot internals at a high level. +- **CI smoke test.** GitHub Actions workflow that builds the image, runs + `agent-vm setup --no-verify`, and `agent-vm shell -- -c 'echo ok'`. +- **macOS/aarch64 binary.** Cross-compile or native-build on each + platform microsandbox supports. + +**Done when:** README is publishable, CI smoke green on at least linux-amd64, +binary works from a fresh checkout on a host where microsandbox runtime is +not pre-installed. + +## Discovered upstream issues + +Things we worked around during Phase 2.x that should eventually be filed +or fixed in `wirenboard/microsandbox`: + +1. **`PullPolicy::Always` doesn't refresh the cached manifest digest.** + It re-fetches layer blobs correctly, but `Image::persist`'s fast-path + detection skips the DB update under the same reference even when the + per-platform manifest digest changed. We work around it with our own + marker file rather than `Image::remove` (because remove + re-pull + opens an empty-cache window). +2. **`LayerDownloadProgress` events are often elided** for fast registries + (we never see them with localhost). Only `LayerDownloadComplete` fires. + Not exactly a bug, but undocumented and bit us when we tried to drive a + download-bytes bar. +3. **libkrun virtio IRQ cap is low** (~6 devices). We're constrained to + 2 bind mounts on top of the OCI overlay's 2-device cost. Bigger fan-out + needs upstream tuning of the libkrun build. +4. **Manifest media-type assumptions.** Microsandbox stores the + per-platform manifest digest; a registry HEAD on a tag returns the + multi-arch index digest by default. Either would be fine to use, but + the SDK doesn't expose either as "ask the registry what's there now" + so we end up doing raw HTTP. A `Image::resolve(reference) -> RemoteRef` + helper would clean this up. ## Working agreements diff --git a/README.md b/README.md index 323d810..1340a7b 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,13 @@ until v1 of the rewrite ships. - Phase 1 (base OCI image + `agent-vm setup`): done. - Phase 2 (launcher MVP — `claude`/`codex`/`opencode`/`shell`): done; live API smoke deferred to Phase 3. -- Phase 3 (host-rooted secrets via microsandbox TLS intercept): pending. +- Phase 2.x (RUST_LOG, host-path mounting, pull progress bar, + `agent-vm pull` + update-available banner, registry auto-recovery): + done. +- Phase 3 (host-rooted secrets via microsandbox TLS intercept): **next**. +- Phase 4 (token-refresh semantics): pending. +- Phase 5 (snapshot-based fast launch): pending. +- Phase 6 (distribution + polish + docs): pending. ## Building