mirror of
https://github.com/wirenboard/agent-vm.git
synced 2026-07-11 00:41:25 +00:00
fix(secrets): keep real tokens out of the guest bind mount
The launcher bind-mounts the per-project state_dir into the guest at
/agent-vm-state (a single mount, to stay under libkrun's virtio-IRQ
cap). secrets.rs wrote the host's real access token to
state_dir/tokens/{anthropic,openai} — i.e. INSIDE that mount — so the
in-VM agent could `cat /agent-vm-state/tokens/anthropic` and read the
host bearer, silently defeating the whole Phase 3/4 "real tokens never
enter the VM" guarantee.
The nested test host masked this: there the "real" token is itself the
outer agent-vm bridge's placeholder, so a leak isn't a real leak. It
only surfaced when grepping the guest filesystem during end-to-end
verification on a host with a genuine credential.
Move the token files to a host-only sibling dir <hash>.secrets/ (0700),
derived from state_dir by secrets::{anthropic,openai}_token_path so the
launcher and the refresh hook agree on the path. The microsandbox proxy
reads them host-side via SecretValue::File, so they never need to be
mounted. Best-effort remove of the legacy state_dir/tokens on launch so
an upgrade doesn't leave a real token lingering in the mount.
Adds token_files_live_outside_the_guest_mount as a regression guard.
Verified in-guest: no host token anywhere under /agent-vm-state;
credentials.json and PID1 environ show only placeholders;
api.anthropic.com server cert is issued by CN=microsandbox CA.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
2e5b5c546e
commit
db50cc26ab
3 changed files with 141 additions and 13 deletions
|
|
@ -459,7 +459,7 @@ vendor/microsandbox/ (branch: agent-vm-secret-file)
|
|||
crates/agent-vm/src/
|
||||
├── msb_install.rs # new: build patched msb from vendor; point MSB_PATH at it
|
||||
├── intercept_hook.rs # new: `agent-vm _intercept-hook` subprocess
|
||||
├── secrets.rs # switched from Static(token) to File(<state>/tokens/{anthropic,openai})
|
||||
├── secrets.rs # switched from Static(token) to File(<state>.secrets/{anthropic,openai})
|
||||
└── run.rs # registers the interceptor with two rules
|
||||
```
|
||||
|
||||
|
|
@ -486,9 +486,10 @@ person doesn't redo it.
|
|||
### `SecretValue::File`
|
||||
|
||||
Phase 3's per-launch snapshot becomes a per-launch *file write*. We
|
||||
write the host's `accessToken` to `<state>/tokens/anthropic` (and
|
||||
`.../openai`) with 0600 perms via atomic-write-then-rename. The
|
||||
launcher passes the file path to microsandbox as a `SecretValue::File`.
|
||||
write the host's `accessToken` to a host-only secret file (see next
|
||||
subsection for *where*) with 0600 perms via atomic-write-then-rename.
|
||||
The launcher passes the file path to microsandbox as a
|
||||
`SecretValue::File`.
|
||||
|
||||
The patched msb's TLS-intercept proxy calls `SecretValue::resolve()`
|
||||
at *connection-setup* time — every new TCP connection re-reads the
|
||||
|
|
@ -496,6 +497,33 @@ file. So any host-side rotation (whether triggered by the user's
|
|||
external `claude` use or by our interceptor hook below) is visible
|
||||
to the very next request, without rebuilding the sandbox.
|
||||
|
||||
### Token files live *outside* the guest bind mount
|
||||
|
||||
The launcher bind-mounts the per-project `state_dir` into the guest at
|
||||
`/agent-vm-state` as a *single* mount (libkrun caps virtio IRQs, so we
|
||||
deliberately use one bind for all per-agent state instead of one per
|
||||
agent). That makes mount placement security-critical: **anything under
|
||||
`state_dir` is readable from inside the VM.**
|
||||
|
||||
The real access-token files therefore must *not* live under
|
||||
`state_dir`. They sit in a sibling host-only directory
|
||||
`${state_root}/<hash>.secrets/` (mode 0700), derived from `state_dir`
|
||||
by `secrets::{anthropic,openai}_token_path` so the launcher and the
|
||||
refresh hook agree on the path without passing it explicitly. The
|
||||
microsandbox proxy reads these files on the *host* side via
|
||||
`SecretValue::File`, so they never need to be mounted into the guest at
|
||||
all.
|
||||
|
||||
This was a real leak found during Phase 4 end-to-end verification: the
|
||||
first cut wrote the tokens to `<state>/tokens/{anthropic,openai}`, i.e.
|
||||
*inside* the mount, so `cat /agent-vm-state/tokens/anthropic` in the
|
||||
guest returned the host's real bearer — silently defeating the entire
|
||||
"real tokens never enter the VM" guarantee. The nested test host masked
|
||||
it (there the "real" token is itself the outer bridge's placeholder), so
|
||||
it only surfaced once we grepped the guest filesystem for the token
|
||||
during verification. A `token_files_live_outside_the_guest_mount` unit
|
||||
test now guards the invariant.
|
||||
|
||||
### Request-interceptor hook (the OAuth refresh MITM)
|
||||
|
||||
`microsandbox-network` gained an `InterceptConfig` that the launcher
|
||||
|
|
@ -525,8 +553,8 @@ clap subcommand mode:
|
|||
2. Spawns `claude -p hi --model sonnet` (or `codex exec --skip-git-
|
||||
repo-check 'Reply OK'`) on the **host** so the host CLI rotates
|
||||
`~/.claude/.credentials.json` / `~/.codex/auth.json` the normal way.
|
||||
3. Re-reads the rotated host file, rewrites
|
||||
`<state>/tokens/{anthropic,openai}` so the next non-refresh
|
||||
3. Re-reads the rotated host file, rewrites the host-only token file
|
||||
(`<state>.secrets/{anthropic,openai}`) so the next non-refresh
|
||||
request from the guest gets the new bearer via `SecretValue::File`.
|
||||
4. Synthesizes an OAuth refresh-response JSON shaped like what the
|
||||
upstream server would return, but with **placeholder** strings in
|
||||
|
|
@ -536,6 +564,8 @@ clap subcommand mode:
|
|||
|
||||
The guest never holds a real token at any layer:
|
||||
- `~/.claude/.credentials.json` always contains placeholders (Phase 3).
|
||||
- The real-token file is on the host *outside* the guest bind mount
|
||||
(see "Token files live outside the guest bind mount").
|
||||
- The proxy substitutes real-for-placeholder on the way *out* (Phase 3).
|
||||
- The OAuth refresh response also returns placeholders (Phase 4).
|
||||
- The host CLI on the host is the only thing that ever touches real
|
||||
|
|
@ -573,7 +603,7 @@ Response:
|
|||
|
||||
Confirmed on the same nested-VM test host as Phase 3. The hook ran,
|
||||
host `claude -p` rotated the host file, the new bearer landed in
|
||||
`<state>/tokens/anthropic`, and the synthesized response reached the
|
||||
`<state>.secrets/anthropic`, and the synthesized response reached the
|
||||
guest. `expires_in: 3499` is the freshly-derived seconds-until-expiry
|
||||
of the just-rotated token.
|
||||
|
||||
|
|
|
|||
36
PLAN.md
36
PLAN.md
|
|
@ -146,7 +146,7 @@ they're in already and doesn't redo the work.
|
|||
only after a successful pull, so an interrupted pull never leaves the
|
||||
microsandbox cache in an empty or stale state.
|
||||
|
||||
### Phase 3 — Static host-rooted secrets [done — submodule branch `agent-vm-secret-file`, agent-vm commit pending]
|
||||
### Phase 3 — Static host-rooted secrets [done — submodule branch `agent-vm-secret-file`, committed `8cc036b`]
|
||||
|
||||
The big architectural payoff of moving to microsandbox: real tokens
|
||||
never enter the VM.
|
||||
|
|
@ -191,7 +191,7 @@ verified on the nested test host because of an outer credential bridge
|
|||
that itself substitutes placeholders. Structurally equivalent to the
|
||||
original Bash agent-vm's credential-proxy flow.
|
||||
|
||||
### Phase 4 — Refresh semantics [done — submodule branch `agent-vm-secret-file`, agent-vm commit pending]
|
||||
### Phase 4 — Refresh semantics [done — committed `8e262c0`..`c7f7386`; end-to-end verified + leak fixed this session]
|
||||
|
||||
Tokens rotate; long-running sandbox sessions must survive that without
|
||||
re-attaching. Phase 3 makes the access token swappable in principle;
|
||||
|
|
@ -236,6 +236,38 @@ externally-rotated case.
|
|||
end-to-end without the agent seeing an auth error and without manual
|
||||
intervention.
|
||||
|
||||
**Verification session (2026-05-21):** stood the runtime back up on a
|
||||
fresh host (installed `libkrunfw` bundle + the patched
|
||||
`0.4.6+agent-vm.phase4` msb), rebuilt the image, and ran the launcher
|
||||
against a real host Claude credential. Findings:
|
||||
|
||||
- **`images/build.sh` registry bug fixed.** Docker 29.x emits a stray
|
||||
blank line to *stdout* on `docker inspect` of a missing container, so
|
||||
`state=$(... || echo missing)` became `"\nmissing"` and never matched
|
||||
the `missing` case — the script tried to `docker start` a nonexistent
|
||||
container. Now whitespace-stripped with an empty→missing fallback.
|
||||
- **Real-token leak into the guest found + fixed.** The token files
|
||||
were written under `state_dir`, which is bind-mounted into the guest
|
||||
at `/agent-vm-state`, so `cat /agent-vm-state/tokens/anthropic`
|
||||
returned the host bearer. Moved to a host-only sibling
|
||||
`<hash>.secrets/` (0700), never mounted; added a guard test. See
|
||||
ARCHITECTURE.md "Token files live outside the guest bind mount".
|
||||
- **Network layer verified.** Inside the guest, `credentials.json` and
|
||||
PID1 environ show only placeholders; `api.anthropic.com`'s server
|
||||
cert is issued by `CN=microsandbox CA` (traffic goes through the
|
||||
intercept proxy). The final real-response leg still can't be checked
|
||||
here — this is a doubly-nested host whose *outer* agent-vm bridge
|
||||
already replaced the host token with its own placeholder
|
||||
(`sk-ant-oat01-placeholder-proxy-managed`) and doesn't re-intercept
|
||||
the nested VM's egress, so Anthropic returns 401. Same documented
|
||||
limitation as Phase 3.
|
||||
- **Codex path not exercisable here:** no `~/.codex/auth.json` on this
|
||||
host, so the OpenAI/Codex websocket flow (the original stop point)
|
||||
can't be authenticated. The `chatgpt.com` WebSocket support
|
||||
(`inject_basic_auth(false)` + zero-copy fast path) is in place and
|
||||
the codex CLI (now 0.133.0) is present in the verified image, but a
|
||||
host with real Codex credentials is needed to confirm it end-to-end.
|
||||
|
||||
### Phase 5 — Fast-launch (deferred — wrong instrument for the job)
|
||||
|
||||
Originally framed around `Sandbox::from_snapshot(...)` on the assumption
|
||||
|
|
|
|||
|
|
@ -52,13 +52,35 @@ pub struct CredsState {
|
|||
pub openai_token_file: Option<PathBuf>,
|
||||
}
|
||||
|
||||
/// Per-project location of the token file the proxy re-reads.
|
||||
/// Host-only directory holding the real access-token files the proxy
|
||||
/// re-reads via `SecretValue::File`.
|
||||
///
|
||||
/// **Must live outside `state_dir`.** The launcher bind-mounts
|
||||
/// `state_dir` into the guest at `/agent-vm-state` (a single mount, to
|
||||
/// stay under libkrun's virtio-IRQ cap), so anything written *under*
|
||||
/// `state_dir` is readable from inside the VM — a `cat
|
||||
/// /agent-vm-state/tokens/anthropic` would hand the in-VM agent the
|
||||
/// host's real token and defeat the entire point of Phase 3/4. The
|
||||
/// microsandbox proxy reads these files on the *host* side, so they
|
||||
/// never need to be mounted; we keep them in a sibling `<hash>.secrets/`
|
||||
/// directory that is never bind-mounted anywhere.
|
||||
fn host_secret_dir(state_dir: &Path) -> PathBuf {
|
||||
let name = state_dir
|
||||
.file_name()
|
||||
.and_then(|n| n.to_str())
|
||||
.unwrap_or("default");
|
||||
let parent = state_dir.parent().unwrap_or(state_dir);
|
||||
parent.join(format!("{name}.secrets"))
|
||||
}
|
||||
|
||||
/// Per-project location of the token file the proxy re-reads. Lives in
|
||||
/// the host-only [`host_secret_dir`], never inside the guest mount.
|
||||
pub fn anthropic_token_path(state_dir: &Path) -> PathBuf {
|
||||
state_dir.join("tokens/anthropic")
|
||||
host_secret_dir(state_dir).join("anthropic")
|
||||
}
|
||||
|
||||
pub fn openai_token_path(state_dir: &Path) -> PathBuf {
|
||||
state_dir.join("tokens/openai")
|
||||
host_secret_dir(state_dir).join("openai")
|
||||
}
|
||||
|
||||
/// Read host credentials, write the token file (atomically, 0600) and
|
||||
|
|
@ -66,7 +88,22 @@ pub fn openai_token_path(state_dir: &Path) -> PathBuf {
|
|||
/// the written token files so the launcher can plumb them into
|
||||
/// microsandbox's SecretValue::File config.
|
||||
pub fn refresh(state_dir: &Path, project_guest_path: &str) -> Result<CredsState> {
|
||||
std::fs::create_dir_all(state_dir.join("tokens"))?;
|
||||
// The token files hold the host's *real* access tokens, so their
|
||||
// directory must never be bind-mounted into the guest. Create it
|
||||
// 0700 in the host-only sibling location (see `host_secret_dir`).
|
||||
let secret_dir = host_secret_dir(state_dir);
|
||||
std::fs::create_dir_all(&secret_dir)?;
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(&secret_dir, std::fs::Permissions::from_mode(0o700))
|
||||
.with_context(|| format!("chmod 700 {}", secret_dir.display()))?;
|
||||
}
|
||||
// Best-effort: an older agent-vm wrote token files to
|
||||
// `state_dir/tokens/`, which is inside the guest bind mount. If a
|
||||
// user upgrades over such a state dir, remove the stale dir so a real
|
||||
// token doesn't linger where the guest can read it.
|
||||
let _ = std::fs::remove_dir_all(state_dir.join("tokens"));
|
||||
|
||||
// First-run bypasses, run regardless of whether the user has host
|
||||
// credentials for the provider. Without these the in-VM agent
|
||||
|
|
@ -289,3 +326,32 @@ fn write_default_codex_config(path: &Path) -> Result<()> {
|
|||
f.write_all(body.as_bytes())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Security invariant: the real-token files must never live under
|
||||
/// `state_dir`, because the launcher bind-mounts `state_dir` into the
|
||||
/// guest at `/agent-vm-state`. A token file under that path would be
|
||||
/// readable by the in-VM agent (`cat /agent-vm-state/tokens/...`),
|
||||
/// defeating the whole "real tokens never enter the VM" design.
|
||||
#[test]
|
||||
fn token_files_live_outside_the_guest_mount() {
|
||||
let state_dir = Path::new("/home/u/.local/state/agent-vm/abc123");
|
||||
for token in [
|
||||
anthropic_token_path(state_dir),
|
||||
openai_token_path(state_dir),
|
||||
] {
|
||||
assert!(
|
||||
!token.starts_with(state_dir),
|
||||
"{} must not be under the bind-mounted state dir {}",
|
||||
token.display(),
|
||||
state_dir.display(),
|
||||
);
|
||||
// ...but still derivable from it (same parent) so the launcher
|
||||
// and the refresh hook agree on the path.
|
||||
assert_eq!(token.parent().unwrap().parent(), state_dir.parent());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue