mirror of
https://github.com/wirenboard/agent-vm.git
synced 2026-07-09 16:00:54 +00:00
merge docker-in-vm-explore: in-VM docker engine support
Three libkrunfw kernel-config additions and one image change so agent-vm can host a working dockerd inside the guest with default `--network=bridge` networking:75297b6CONFIG_POSIX_MQUEUE=y — runc/crun mqueue mounteadd712+netfilter + bridge family — docker0 + iptables NAT93a2aab+nf_tables (NFT_*, NFT_COMPAT) — iptables-nft backendff4f3aeimage: install docker engine — dockerd + cli + containerd + runc Verified end-to-end with real dockerd inside a nested microsandbox guest on the rebuilt libkrunfw: port-publish DNAT, SNAT MASQUERADE, user-defined networks, embedded DNS, cross-network isolation all work. Cost summary: - libkrunfw.so: +128 KB compressed (+0.6%) - boot time: within noise (~10 ms / +1%) - guest image: +87 MB compressed pull (+15.4%) / +281 MB on-disk
This commit is contained in:
commit
0d60870acd
3 changed files with 129 additions and 11 deletions
|
|
@ -4,9 +4,10 @@
|
|||
# @wirenboard/agent-vm on npm).
|
||||
#
|
||||
# Contents: Debian 13 slim + the three AI coding agents (Claude
|
||||
# Code, OpenCode, Codex) and the minimum dev tooling they need to be
|
||||
# useful. Heavier extras (Docker-in-VM, LSPs, additional MCP
|
||||
# servers) are deliberately deferred.
|
||||
# Code, OpenCode, Codex), minimum dev tooling, and the docker engine
|
||||
# (docker.io + containerd + runc + fuse-overlayfs). Other heavy
|
||||
# extras (LSPs, additional MCP servers, docker-buildx) are
|
||||
# deliberately deferred.
|
||||
#
|
||||
# Published to ghcr.io/wirenboard/agent-vm-template:latest by the
|
||||
# hourly CI workflow (.github/workflows/build-image.yml). Source-
|
||||
|
|
@ -179,6 +180,45 @@ RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
|
|||
RUN sudo -u chrome -H npx -y chrome-devtools-mcp@1.0.1 --help >/dev/null 2>&1 \
|
||||
|| echo "==> pre-warm skipped (will lazy-fetch chrome-devtools-mcp at first MCP call)"
|
||||
|
||||
# Docker engine — dockerd + cli + containerd + runc.
|
||||
#
|
||||
# Why each piece:
|
||||
# - docker.io / docker-cli: the daemon and the CLI a user actually
|
||||
# invokes; no buildx (legacy builder is fine for the small builds
|
||||
# agents do in-VM; +20 MB compressed isn't worth the polish).
|
||||
# - containerd / runc: the OCI runtime stack the daemon delegates to.
|
||||
# Both are kernel-required (runc=container init, containerd=image
|
||||
# management); without them dockerd refuses to start.
|
||||
# - fuse-overlayfs: the storage driver we pin in daemon.json below.
|
||||
# Native overlay2 doesn't work because the VM rootfs is already
|
||||
# overlay (libkrun rootfs design), and overlay-on-overlay returns
|
||||
# EINVAL inside dockerd's snapshotter. fuse-overlayfs sidesteps
|
||||
# that at userspace level.
|
||||
# - iptables: dockerd installs NAT rules on every container start;
|
||||
# missing binary makes the daemon log a warning and disable bridge
|
||||
# networking. The kernel-side iptables/nf_tables support comes
|
||||
# from the libkrunfw-overrides patch (see libkrunfw-overrides/
|
||||
# README.md).
|
||||
#
|
||||
# Storage-driver choice (daemon.json): see comment above. iptables
|
||||
# alternative left at debian's default (iptables-nft) — both nft and
|
||||
# legacy backends are functional in the patched kernel.
|
||||
#
|
||||
# Auto-start: not done in the image. The launcher (or a per-project
|
||||
# `.agent-vm.runtime.sh`) is the right place to `dockerd &` if the
|
||||
# session needs it — most sessions won't, so paying ~50 MB RSS at
|
||||
# every boot would be wasteful.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
docker.io docker-cli \
|
||||
containerd runc \
|
||||
fuse-overlayfs \
|
||||
iptables \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& mkdir -p /etc/docker \
|
||||
&& printf '%s\n' '{' ' "storage-driver": "fuse-overlayfs"' '}' \
|
||||
> /etc/docker/daemon.json
|
||||
|
||||
# Claude Code official installer.
|
||||
RUN curl -fsSL https://claude.ai/install.sh | bash
|
||||
|
||||
|
|
@ -191,7 +231,8 @@ RUN curl -fsSL https://opencode.ai/install | bash \
|
|||
RUN curl -fsSL https://github.com/openai/codex/releases/latest/download/install.sh | sh
|
||||
|
||||
# Sanity check at build time so a broken installer surfaces before we push.
|
||||
RUN claude --version && opencode --version && codex --version
|
||||
RUN claude --version && opencode --version && codex --version \
|
||||
&& dockerd --version && docker --version && containerd --version && runc --version
|
||||
|
||||
# Smoke entrypoint; the launcher overrides this in Phase 2.
|
||||
CMD ["/bin/bash"]
|
||||
|
|
|
|||
|
|
@ -19,14 +19,34 @@ Stock libkrunfw ships only the paravirt-guest helpers
|
|||
can't host its own. `agent-vm` needs the in-kernel KVM hypervisor for
|
||||
Docker-in-Docker / nested KVM use cases, so we flip:
|
||||
|
||||
| symbol | reason |
|
||||
| symbol(s) | reason |
|
||||
|---|---|
|
||||
| `CONFIG_KVM=y` | enable the in-kernel KVM hypervisor |
|
||||
| `CONFIG_KVM_INTEL=y` | Intel VMX backend |
|
||||
| `CONFIG_KVM_AMD=y` | AMD SVM backend |
|
||||
| `CONFIG_KVM=y` + `CONFIG_KVM_INTEL=y` + `CONFIG_KVM_AMD=y` | in-kernel KVM hypervisor for nested KVM use cases |
|
||||
| `CONFIG_POSIX_MQUEUE=y` | runc/crun default OCI spec always mounts `/dev/mqueue`; without it every container start fails with `mount mqueue: no such device` |
|
||||
| netfilter + bridge family (`NETFILTER`, `NF_CONNTRACK`, `NF_NAT`, `IP_NF_*`, `IP6_NF_*`, `BRIDGE`, `BRIDGE_NETFILTER`, `VLAN_8021Q`, …) | docker/podman `--network=bridge` (the default) needs iptables and the bridge driver to set up `docker0` + SNAT. Without these the daemon starts but every container start fails on `Table does not exist` |
|
||||
| `nf_tables` family (`NF_TABLES`, `NFT_NAT`, `NFT_MASQ`, `NFT_CT`, `NFT_COMPAT`, …) | debian's default `iptables` alternative is `iptables-nft`, which talks to the kernel via nf_tables instead of ip_tables; without these the user has to flip `update-alternatives --set iptables iptables-legacy` first |
|
||||
|
||||
`olddefconfig` pulls in the rest (`KVM_X86`, `KVM_SMM`, `KVM_HYPERV`,
|
||||
`KVM_VFIO`, …) automatically.
|
||||
`olddefconfig` pulls in the rest of each cascade (`KVM_X86`, `KVM_SMM`,
|
||||
`KVM_HYPERV`, `KVM_VFIO`, `POSIX_MQUEUE_SYSCTL`, `NETFILTER_NETLINK`,
|
||||
`NF_DEFRAG_IPV4`/`IPV6`, `BRIDGE_IGMP_SNOOPING`, …) automatically.
|
||||
|
||||
`devtmpfs` and `DEVTMPFS_MOUNT` are already enabled upstream so
|
||||
`/dev/kvm` materializes at boot without further config changes.
|
||||
|
||||
## Cost of each override
|
||||
|
||||
Measured by rebuilding the `.so` from a pristine v5.2.1 checkout with each
|
||||
hunk added incrementally (Debian 13 toolchain, gcc 14.2.0). Boot time is
|
||||
15-run mean of a nested `msb run alpine -- /bin/true`, sd ≈ 17–20 ms:
|
||||
|
||||
| cumulative change | `bzImage` | `libkrunfw.so` (shipped) | nested-boot mean |
|
||||
|---|---|---|---|
|
||||
| baseline (just `+KVM`) | 7,431 KB | 21,300 KB | 1.015 s |
|
||||
| `+POSIX_MQUEUE` | +12 KB | +64 KB (page padding) | +2 ms |
|
||||
| `+netfilter + bridge` | +348 KB | +0 (still fits in mqueue's page) | +5 ms |
|
||||
| `+nf_tables / NFT_*` | +8 KB | +64 KB (next page) | +3 ms |
|
||||
| **all overrides** | **+368 KB (+5%)** | **+128 KB (+0.6%)** | **+10 ms (~1%)** |
|
||||
|
||||
The shipped `.so` is what affects npm-package size and `agent-vm setup` cold-pull
|
||||
time; the `bzImage` figure is informational. Boot-time deltas were all within
|
||||
1σ of baseline noise.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,14 @@
|
|||
--- a/config-libkrunfw_x86_64
|
||||
+++ b/config-libkrunfw_x86_64
|
||||
@@ -51,7 +51,7 @@
|
||||
CONFIG_DEFAULT_HOSTNAME="(none)"
|
||||
CONFIG_SYSVIPC=y
|
||||
CONFIG_SYSVIPC_SYSCTL=y
|
||||
-# CONFIG_POSIX_MQUEUE is not set
|
||||
+CONFIG_POSIX_MQUEUE=y
|
||||
# CONFIG_WATCH_QUEUE is not set
|
||||
CONFIG_CROSS_MEMORY_ATTACH=y
|
||||
# CONFIG_USELIB is not set
|
||||
@@ -485,7 +485,7 @@
|
||||
|
||||
CONFIG_HAVE_KVM=y
|
||||
|
|
@ -9,9 +18,57 @@
|
|||
CONFIG_AS_AVX512=y
|
||||
CONFIG_AS_SHA1_NI=y
|
||||
CONFIG_AS_SHA256_NI=y
|
||||
@@ -2389,3 +2389,5 @@
|
||||
@@ -2389,3 +2389,53 @@
|
||||
#
|
||||
# end of Rust hacking
|
||||
# end of Kernel hacking
|
||||
+CONFIG_KVM_INTEL=y
|
||||
+CONFIG_KVM_AMD=y
|
||||
+
|
||||
+# Netfilter + bridge for docker/podman `--network=bridge` (the
|
||||
+# default). Covers the iptables-legacy backend; the nf_tables
|
||||
+# backend's `NF_TABLES` / `NFT_*` are in a follow-up patch hunk.
|
||||
+# `olddefconfig` resolves the rest of each dependency cascade
|
||||
+# (`NETFILTER_NETLINK`, `NF_DEFRAG_IPV4`/`IPV6`, `BRIDGE_IGMP_SNOOPING`, …).
|
||||
+CONFIG_NETFILTER=y
|
||||
+CONFIG_NETFILTER_ADVANCED=y
|
||||
+CONFIG_NETFILTER_XTABLES=y
|
||||
+CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
|
||||
+CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
|
||||
+CONFIG_NETFILTER_XT_NAT=y
|
||||
+CONFIG_NETFILTER_XT_TARGET_MASQUERADE=y
|
||||
+CONFIG_NF_CONNTRACK=y
|
||||
+CONFIG_NF_NAT=y
|
||||
+CONFIG_NF_NAT_MASQUERADE=y
|
||||
+CONFIG_NF_NAT_REDIRECT=y
|
||||
+CONFIG_IP_NF_IPTABLES=y
|
||||
+CONFIG_IP_NF_FILTER=y
|
||||
+CONFIG_IP_NF_NAT=y
|
||||
+CONFIG_IP_NF_TARGET_MASQUERADE=y
|
||||
+CONFIG_IP_NF_TARGET_REJECT=y
|
||||
+CONFIG_IP6_NF_IPTABLES=y
|
||||
+CONFIG_IP6_NF_FILTER=y
|
||||
+CONFIG_IP6_NF_NAT=y
|
||||
+CONFIG_IP6_NF_TARGET_MASQUERADE=y
|
||||
+CONFIG_BRIDGE=y
|
||||
+CONFIG_BRIDGE_NETFILTER=y
|
||||
+CONFIG_LLC=y
|
||||
+CONFIG_STP=y
|
||||
+CONFIG_VLAN_8021Q=y
|
||||
+
|
||||
+# nf_tables backend — debian's default `iptables` alternative is
|
||||
+# `iptables-nft`, so without these the user has to manually
|
||||
+# `update-alternatives --set iptables iptables-legacy` before
|
||||
+# dockerd's iptables rules will apply. NFT_COMPAT lets the
|
||||
+# xtables-nft userspace translate `-j MASQUERADE` into native nft.
|
||||
+CONFIG_NF_TABLES=y
|
||||
+CONFIG_NF_TABLES_INET=y
|
||||
+CONFIG_NF_TABLES_IPV4=y
|
||||
+CONFIG_NF_TABLES_IPV6=y
|
||||
+CONFIG_NFT_NAT=y
|
||||
+CONFIG_NFT_MASQ=y
|
||||
+CONFIG_NFT_REDIR=y
|
||||
+CONFIG_NFT_CT=y
|
||||
+CONFIG_NFT_LOG=y
|
||||
+CONFIG_NFT_LIMIT=y
|
||||
+CONFIG_NFT_COMPAT=y
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue