mirror of
https://github.com/wirenboard/agent-vm.git
synced 2026-07-09 16:00:54 +00:00
docs: refresh PLAN.md after Phase 2.x and re-plan remaining phases
- Add a "Phase 2.x — Post-MVP polish" section listing the work that
landed between Phase 2 and Phase 3 (tracing, registry auto-recover,
host-path cwd, profile flag, pull progress bar, explicit pull +
update banner). Each entry references its commit so the next reader
doesn't redo any of it.
- Expand Phase 3 with the things we learned: SecretEntry.value is
captured at builder time so we need a File variant upstream; TLS
intercept must be enabled for substitution to fire; only access
tokens cross into the marker file, never refresh tokens.
- Add Phase 5 (snapshot-based fast launch) as a real phase rather than
a vague polish bullet. Surfaced naturally from the "why is pull so
slow" profiling — the EROFS materialize cost is the dominant pull
cost, and snapshot/restore sidesteps both that and the ~1s libkrun
boot. DoD: `agent-vm shell -- -c true` well under 500ms.
- Rename Phase 5→6 ("Distribution + polish + docs"). Add concrete
items: microsandbox runtime auto-install, CLI flag promotion,
.agent-vm.runtime.sh hook, CI smoke, macOS/aarch64 builds.
- Add "Discovered upstream issues" section recording the four
microsandbox bugs/quirks we worked around (PullPolicy::Always not
refreshing manifest digest; LayerDownloadProgress elided on fast
registries; libkrun IRQ cap; no SDK-level "resolve manifest" helper).
These should eventually be filed upstream.
- README status list extended with Phase 2.x done and 5/6 split.
This commit is contained in:
parent
d3914b990b
commit
d979c31028
2 changed files with 144 additions and 24 deletions
160
PLAN.md
160
PLAN.md
|
|
@ -111,44 +111,158 @@ CODEX_HOME redirect, env propagation). The live API smoke was deferred — see
|
|||
ARCHITECTURE.md "What Phase 2 deliberately doesn't do". Phase 3's host-OAuth
|
||||
work closes the gap naturally.
|
||||
|
||||
### Phase 3 — Static host-rooted secrets [pending]
|
||||
### Phase 2.x — Post-MVP polish [done — commits `7608f27`..`d3914b9`]
|
||||
|
||||
A series of small fixes landed between Phase 2 and Phase 3, all triggered by
|
||||
real testing on the user's laptop. Listed here so the next reader knows
|
||||
they're in already and doesn't redo the work.
|
||||
|
||||
- **`RUST_LOG` wiring** (`7608f27`). `tracing-subscriber` initialized in
|
||||
`main`, defaults to `warn`. The microsandbox stack is silent otherwise.
|
||||
- **Auto-recover the local registry container** (`a57ed6d`). `build.sh`'s
|
||||
`ensure_registry` was rewritten as a state machine that handles every
|
||||
`docker ps` state, polls `/v2/` after start, and recreates from scratch
|
||||
if a stale container is running with no port mapping. Plus per-phase
|
||||
banners so long waits don't look like a hang.
|
||||
- **Mirror the host project path inside the guest** (`92ff582`). `cwd` is
|
||||
bind-mounted at the same absolute path, so anything the agent emits
|
||||
(compiler errors, stack traces, file:line references) is interpretable on
|
||||
the host. Paths under tmpfs mount points (`/tmp`, `/run`, `/dev/shm`,
|
||||
`/var/run`) fall back to `/workspace` with a warning, because the guest
|
||||
tmpfs-mounts them at boot and wipes any patch-created mount point.
|
||||
- **`AGENT_VM_PROFILE=1`** (`127f6b3`). Prints per-phase wall-time
|
||||
(create / run / stop / remove) for the launcher. Confirmed total is
|
||||
~1.5 s, dominated by VM boot (~1.0 s of libkrun kernel boot).
|
||||
- **Pull progress bar** (`2489168`..`66ed8f3`..`3ffd6b6`..`984680a`).
|
||||
Two-phase indicatif renderer: spinner with text during download, real
|
||||
byte-weighted bar during materialize. Single line, single spinner, ETA
|
||||
based on materialize-only rate (no more "29 minute" → "17 second" jumps).
|
||||
- **`agent-vm pull` + per-launch update-available banner**
|
||||
(`bfab9d3`..`d3914b9`). Pulls are explicit; `agent-vm shell` only does a
|
||||
cheap manifest-digest HEAD against the registry and prints a banner when
|
||||
the per-platform digest differs from what we last pulled. The "what we
|
||||
last pulled" digest is tracked in our own marker file
|
||||
(`~/.local/state/agent-vm/pulled-digests/<hash>`), atomically written
|
||||
only after a successful pull, so an interrupted pull never leaves the
|
||||
microsandbox cache in an empty or stale state.
|
||||
|
||||
### Phase 3 — Static host-rooted secrets [next]
|
||||
|
||||
The big architectural payoff of moving to microsandbox: real tokens never
|
||||
enter the VM.
|
||||
|
||||
- Branch `vendor/microsandbox` to add a `SecretValue::File(PathBuf)` variant
|
||||
alongside the existing `String` value. The TLS-intercept proxy re-reads the
|
||||
file on every substitution.
|
||||
alongside the existing `String` value. The TLS-intercept proxy re-reads
|
||||
the file on every substitution. Required because microsandbox's
|
||||
`SecretEntry.value: String` is captured at sandbox-builder time, with no
|
||||
runtime update path.
|
||||
- Enable TLS interception on the sandbox (`network(|n| n.tls(...))`) —
|
||||
microsandbox's secrets are only substituted on TLS-intercepted
|
||||
connections to allowed hosts, and the guest needs the sandbox CA in its
|
||||
trust store.
|
||||
- `agent-vm` on startup snapshots host `~/.claude/.credentials.json` and
|
||||
`~/.codex/auth.json` into `<state>/tokens/{anthropic,openai}.token` files.
|
||||
`~/.codex/auth.json` into `<state>/tokens/{anthropic,openai}.token`
|
||||
files (just the `access_token`, never the refresh token).
|
||||
- Register file-backed secret entries for `api.anthropic.com`,
|
||||
`api.openai.com`, `chatgpt.com`, `platform.claude.com`, etc.
|
||||
- Cross-instance lock so two `agent-vm` processes don't fight over the token
|
||||
files.
|
||||
`api.openai.com`, `chatgpt.com`, `platform.claude.com`,
|
||||
`auth.openai.com`, etc.
|
||||
- Cross-instance lockfile under `<state>/tokens/` so two `agent-vm`
|
||||
processes don't race when both want to refresh.
|
||||
|
||||
**Done when:** inside the guest, `cat /proc/$$/environ | tr '\0' '\n' | grep
|
||||
ANTHROPIC` shows only placeholders, while a real Claude request succeeds.
|
||||
**Done when:** inside the guest, `cat /proc/$$/environ | tr '\0' '\n' |
|
||||
grep -i token` shows only placeholders, while a real Claude (host OAuth)
|
||||
request succeeds end-to-end and writes a turn to disk.
|
||||
|
||||
### Phase 4 — Refresh semantics [pending]
|
||||
|
||||
Tokens rotate; long-running sandbox sessions must survive that without
|
||||
re-attaching. Phase 3 makes the access token swappable; this phase teaches
|
||||
agent-vm to actually do the swap.
|
||||
|
||||
- inotify (Linux) / kqueue (macOS) watcher on host creds files: when host
|
||||
Claude/Codex rotates tokens, we re-snapshot to the file microsandbox watches.
|
||||
- Proactive expiry watch: when the access token is < 5 minutes from expiry
|
||||
and no host activity has refreshed it, spawn `claude -p "ping" --model
|
||||
sonnet` / `codex exec --skip-git-repo-check "Reply OK"` on the host.
|
||||
Claude/Codex rotates tokens externally (the user runs `claude` on the
|
||||
host, etc.), we re-snapshot to the file microsandbox watches.
|
||||
- Proactive expiry watch: when the access token is < 5 minutes from
|
||||
expiry and no host activity has refreshed it, spawn `claude -p "ping"
|
||||
--model sonnet` / `codex exec --skip-git-repo-check "Reply OK"` on the
|
||||
host so the host-side CLI rotates the file.
|
||||
- Single-flight per credential; cross-instance lockfile so concurrent
|
||||
`agent-vm` instances don't all kick off a refresh at once.
|
||||
|
||||
**Done when:** a multi-hour session crosses a token rotation without the agent
|
||||
seeing an auth error.
|
||||
**Done when:** a multi-hour session crosses a token rotation without the
|
||||
agent seeing an auth error and without manual intervention.
|
||||
|
||||
### Phase 5 — Polish & docs [pending]
|
||||
### Phase 5 — Snapshot-based fast launch [pending — perf win]
|
||||
|
||||
- `.agent-vm.runtime.sh` project hook.
|
||||
- `--memory N` flag (passed through to microsandbox builder).
|
||||
- Full arg passthrough to the agent command.
|
||||
- README rewrite: install, setup, usage, troubleshooting.
|
||||
- Smoke tests: at least one end-to-end test that boots the image and runs a
|
||||
trivial agent command.
|
||||
Surfaced during Phase 2.x profiling and discussed with the user when they
|
||||
asked why `agent-vm pull` was so slow. Every `agent-vm shell` currently
|
||||
spends ~1.0 s booting the libkrun kernel + initing the guest agent;
|
||||
microsandbox has a snapshot/restore API that should drop that to ~100 ms
|
||||
*and* skip the EROFS materialize step on a re-pull.
|
||||
|
||||
**Done when:** README is publishable; one CI smoke test green.
|
||||
- One-shot snapshot generation: `agent-vm setup` (or a new `agent-vm
|
||||
snapshot`) boots the image once, lets it reach a quiet steady state,
|
||||
takes a snapshot, and stashes it under `<microsandbox-home>/snapshots/`.
|
||||
- Launcher: if a snapshot exists, use `Sandbox::builder(...).from_snapshot
|
||||
(snapshot_path)` instead of `.image(...)`. Falls back to cold boot if
|
||||
the snapshot is missing or wrong-digest.
|
||||
- Re-snapshot whenever `agent-vm pull` lands a new manifest (snapshots are
|
||||
pinned to a specific image digest by microsandbox).
|
||||
- Bake the per-agent symlinks + state mount point into the snapshot so
|
||||
the launcher doesn't need to re-apply patches each invocation.
|
||||
|
||||
**Done when:** `time agent-vm shell -- -c true` is well under 500 ms in
|
||||
the steady state.
|
||||
|
||||
### Phase 6 — Distribution + polish + docs [pending]
|
||||
|
||||
The "ready to share with a teammate" phase.
|
||||
|
||||
- **Auto-install of the microsandbox runtime.** The agent-vm binary ships
|
||||
on its own; `~/.microsandbox/{bin/msb, lib/libkrunfw.so.5.2.1}` are
|
||||
needed but not bundled. Wrap `microsandbox::setup::install()` so first
|
||||
run downloads them automatically if missing. Verify version matches the
|
||||
prebuilt the binary was built against.
|
||||
- **CLI flag promotion.** `--memory N`, `--cpus N`, `--image REF`,
|
||||
`--no-update-check`. Today these are env-var only (`AGENT_VM_*`).
|
||||
- **`.agent-vm.runtime.sh` project hook.** Script executed in the guest
|
||||
immediately before the agent starts, for project-local setup
|
||||
(`npm install`, `docker compose up`, etc.).
|
||||
- **README rewrite.** Install, prereqs, setup, usage, troubleshooting,
|
||||
the registry/marker/snapshot internals at a high level.
|
||||
- **CI smoke test.** GitHub Actions workflow that builds the image, runs
|
||||
`agent-vm setup --no-verify`, and `agent-vm shell -- -c 'echo ok'`.
|
||||
- **macOS/aarch64 binary.** Cross-compile or native-build on each
|
||||
platform microsandbox supports.
|
||||
|
||||
**Done when:** README is publishable, CI smoke green on at least linux-amd64,
|
||||
binary works from a fresh checkout on a host where microsandbox runtime is
|
||||
not pre-installed.
|
||||
|
||||
## Discovered upstream issues
|
||||
|
||||
Things we worked around during Phase 2.x that should eventually be filed
|
||||
or fixed in `wirenboard/microsandbox`:
|
||||
|
||||
1. **`PullPolicy::Always` doesn't refresh the cached manifest digest.**
|
||||
It re-fetches layer blobs correctly, but `Image::persist`'s fast-path
|
||||
detection skips the DB update under the same reference even when the
|
||||
per-platform manifest digest changed. We work around it with our own
|
||||
marker file rather than `Image::remove` (because remove + re-pull
|
||||
opens an empty-cache window).
|
||||
2. **`LayerDownloadProgress` events are often elided** for fast registries
|
||||
(we never see them with localhost). Only `LayerDownloadComplete` fires.
|
||||
Not exactly a bug, but undocumented and bit us when we tried to drive a
|
||||
download-bytes bar.
|
||||
3. **libkrun virtio IRQ cap is low** (~6 devices). We're constrained to
|
||||
2 bind mounts on top of the OCI overlay's 2-device cost. Bigger fan-out
|
||||
needs upstream tuning of the libkrun build.
|
||||
4. **Manifest media-type assumptions.** Microsandbox stores the
|
||||
per-platform manifest digest; a registry HEAD on a tag returns the
|
||||
multi-arch index digest by default. Either would be fine to use, but
|
||||
the SDK doesn't expose either as "ask the registry what's there now"
|
||||
so we end up doing raw HTTP. A `Image::resolve(reference) -> RemoteRef`
|
||||
helper would clean this up.
|
||||
|
||||
## Working agreements
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,13 @@ until v1 of the rewrite ships.
|
|||
- Phase 1 (base OCI image + `agent-vm setup`): done.
|
||||
- Phase 2 (launcher MVP — `claude`/`codex`/`opencode`/`shell`): done; live
|
||||
API smoke deferred to Phase 3.
|
||||
- Phase 3 (host-rooted secrets via microsandbox TLS intercept): pending.
|
||||
- Phase 2.x (RUST_LOG, host-path mounting, pull progress bar,
|
||||
`agent-vm pull` + update-available banner, registry auto-recovery):
|
||||
done.
|
||||
- Phase 3 (host-rooted secrets via microsandbox TLS intercept): **next**.
|
||||
- Phase 4 (token-refresh semantics): pending.
|
||||
- Phase 5 (snapshot-based fast launch): pending.
|
||||
- Phase 6 (distribution + polish + docs): pending.
|
||||
|
||||
## Building
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue