vrr/wirenboard-agent-vm

Fork 0

mirror of https://github.com/wirenboard/agent-vm.git synced 2026-07-09 16:00:54 +00:00

Commit graph

Author	SHA1	Message	Date
Evgeny Boger	0468a2649a	B2: ship a native aarch64-linux release binary via npm Why --- PLAN.md item "B2 — Cross-arch binaries" (section B, Distribution / release) calls for cross-arch builds and per-platform npm packaging, noting that "the package currently bundles a linux-x86_64 binary". As a result Apple Silicon and ARM Linux users get no native artifact and fall back to slow x86 emulation (Rosetta / qemu) — a poor fit for a tool whose job is to launch microVMs quickly. This change delivers the linux/aarch64 half of B2: a genuine aarch64 binary, built, packaged, and selected at install time, so ARM hosts run native. How --- Release workflow (.github/workflows/release-npm.yml): every build job (build-agent-vm, build-msb, build-libkrunfw) and the package job gain a linux-arm64 matrix leg. Because GitHub's hosted arm64 Linux runners aren't on the free tier, the arm64 leg cross-compiles on the x64 runner: a `cross: true` matrix flag drives a `CROSS` env switch that (a) enables the arm64 multiarch apt repo and installs the cross linker (gcc-aarch64-linux-gnu) plus the :arm64 dev libs agent-vm / msb link against (libcap-ng, libdbus, libsqlite3), and (b) exports PKG_CONFIG_ALLOW_CROSS / PKG_CONFIG_PATH / PKG_CONFIG_SYSROOT_DIR so the `pkg-config` crate resolves the target libs. The msb job also sets CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER directly, since it builds from vendor/microsandbox (a separate workspace not on the superproject's .cargo config search path). All of these guards are no-ops on the native x64 leg, so the existing x86_64 path is unchanged. The libkrunfw arm64 leg cross-builds the guest kernel with ARCH=arm64 / CROSS_COMPILE and fails fast with a clear message if its arm64 kbuild .config seed (libkrunfw-overrides/config-libkrunfw_aarch64.patch) hasn't been ported yet, rather than silently shipping a mis-configured kernel. The publish job downloads the new agent-vm-linux-arm64 artifact alongside the x64 one. Cross toolchain config (.cargo/config.toml, new): pins the aarch64-unknown-linux-gnu linker to aarch64-linux-gnu-gcc so a local cross build reproduces CI; the file documents the matching apt packages. npm launcher (npm-dist/agent-vm/bin/agent-vm.js, npm-dist/agent-vm/package.json): un-comment the linux-arm64 entry in the launcher's PLATFORM_PACKAGES map and add @wirenboard/agent-vm-linux-arm64 to the main package's optionalDependencies, so `npm install` pulls the arm64 subpackage on ARM hosts and the launcher dispatches to it. arm64 subpackage scaffold (npm-dist/agent-vm-linux-arm64/, new): mirrors the x64 subpackage layout — package.json (os linux / cpu arm64), README.md, and bin/.gitkeep + lib/.gitkeep placeholders — so the directory exists in the tree for the release workflow to drop the cross-built binary, msb, and libkrunfw into. Dispatch test (npm-dist/agent-vm/bin/agent-vm.dispatch.test.js, new): arm64 dispatch can't be exercised on an x86_64 CI host because node reports the host's real process.arch. The test re-derives the launcher's PLATFORM_PACKAGES map + bin-path logic from the source and asserts the linux-arm64 key resolves to the arm64 subpackage with the same bin/ layout as x64, so a future edit that forgets arm64 fails here instead of silently falling through to "no prebuilt binary" on ARM hardware. README (npm-dist/README.md): document that the package now ships both the linux-x64 and linux-arm64 per-platform subpackages. macOS / darwin and win32 cross builds remain out of scope for this change (still commented placeholders in the launcher) and are tracked under the rest of B2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-05-31 18:57:07 +00:00
Evgeny Boger	494b8dd0aa	distribution: address code-review findings before v0.1.0 Top fixes from the max-effort review: Correctness: 1. setup.rs::verify_image now calls image_api_version::check() and runs each agent --version separately. Old code did `cat ... && claude --version && ...` so a non-zero exit could mean any of four causes; error message blamed only the API file. New code names which check failed. 2. main.rs runs `point_at_msb` BEFORE constructing the tokio runtime. setenv() is not thread-safe; the previous code ran `unsafe { set_var }` after `#[tokio::main]` had already spawned workers, technically UB if a worker read env concurrently (reqwest / sea-orm do this on first use). Switched to plain `fn main` + manual Runtime::new() so the env mutation happens single-threaded. 3. release-npm.yml cache key now includes a hash of `rustc -V`. Without this, an ubuntu-latest rustc rollover (we hit this exact bug locally this session) reused cached rlibs from the prior toolchain → undefined-symbol link errors. 4. release-npm.yml's subpackage publish loop is now idempotent: skips already-published versions instead of erroring. Avoids the split-state where a transient failure leaves subpackages at v but the main package never published. 5. build-image.yml's retention switched from `actions/delete-package-versions@v5`'s `min-versions-to-keep: 350` (which breaks if cron is paused for days) to an explicit age-based prune via actions/github-script — deletes versions older than 14 days regardless of build cadence. UX: 6. pull.rs adds `AGENT_VM_INSECURE_REGISTRY=1` env-var escape hatch so airgapped/intranet plain-HTTP registries (`registry.corp.example:5000`) can opt in. The hostname heuristic stays narrow (localhost/127.0.0.1/0.0.0.0/.local/ .localhost) for safety. 7. msb_install.rs: MSB_PATH set-but-file-missing now falls back to the sibling-of-current_exe() if one exists (with a warn) and gives a much clearer error otherwise. The vanilla-msb rejection message now detects MSB_PATH presence and tells the user to unset it instead of generically suggesting "set MSB_PATH explicitly" (the LAST thing they need to hear when they just set it). 8. setup.rs warns when running from a source checkout (manifest_dir has images/Dockerfile) but the submodule isn't initialised — instead of silently pulling ghcr.io and confusing the dev who just edited images/Dockerfile. 9. npm-dist/agent-vm/bin/agent-vm.js' missing-optional error lists network-flake as the top cause (it is) plus --no-optional and inconsistent lockfile pinning. Previously blamed --no-optional as the most likely cause, which it usually isn't. 10. README files for both npm packages so npmjs.com renders something useful. 11. release-npm.yml's libkrunfw symlink step replaced the `( cd && ln && ln )` subshell with absolute-path `ln -sf`s. Subshell `set -e` propagation is bash-version-sensitive; the new form is unconditional. 84/84 → 85/85 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-25 11:43:08 +03:00
Evgeny Boger	23cba5663b	distribution: ship via npm + ghcr.io image, drop local-build flow Replaces the build-from-source quickstart (Rust toolchain + docker + local registry container + manual `images/build.sh`) with a single `npm install -g @wirenboard/agent-vm` and a `ghcr.io/wirenboard/ agent-vm:latest` image pulled from CI. ## What ships via npm `npm-dist/` scaffolds the established per-platform-subpackage pattern (esbuild / ruff / biome): - `@wirenboard/agent-vm` — JS launcher that detects `${platform}-${arch}` at runtime and execs the right native binary from the matching subpackage. `optionalDependencies` lists each platform. - `@wirenboard/agent-vm-linux-x64` — bundles a prebuilt `agent-vm`, the patched `msb`, and `lib/libkrunfw.so.5.2.1`. Smoke-tested locally via `npm link`: launcher resolves the subpackage correctly and execs the bundled binary; `MSB_PATH=/usr/bin/cat agent-vm pull` correctly rejects an unpatched msb with the actionable "reinstall agent-vm" hint. ## msb discovery (msb_install.rs rewrite) `point_at_workspace_msb` → `point_at_msb` with a fallback chain: 1. `MSB_PATH` env (explicit override). 2. `<exe-dir>/msb` (npm bundle layout — sibling of agent-vm). 3. `<workspace>/vendor/microsandbox/target/release/msb` (dev mode). `msb --version` is verified to contain `+agent-vm` — refuses to launch if a user's separate microsandbox install has somehow ended up on our discovered path. Catches the bug where shadow installs would silently lose SecretValue::File and the request-interceptor hook, producing inscrutable in-VM failures instead of one clear "reinstall" message. ## Image flow (separate cadence) - Default image ref: `ghcr.io/wirenboard/agent-vm:latest`. - `images/build.sh` and local-registry flow stay as the source-checkout dev path (pass `--image localhost:5000/...`). - `agent-vm setup` becomes "pull + verify"; no more docker/registry dependency for end users. - New `image_api_version` module reads `/etc/agent-vm-image-version` inside the booted sandbox and requires it within `MIN_SUPPORTED_IMAGE_API..=MAX_SUPPORTED_IMAGE_API` (both 1 initially). Out-of-range → actionable error pointing at the right side (binary vs image) to update. - Image registry hostname heuristic (`is_plain_http_registry`) keeps `.insecure()` for localhost-style refs but drops it for ghcr.io / docker.io / public registries that require HTTPS. ## CI workflows - `.github/workflows/release-npm.yml` — on `v..` tag, cross- compile binaries per platform (linux-x64 only initially; arm64 / darwin commented out, ready to enable when microsandbox grows those backends), bundle into subpackages, publish all packages to npm with provenance. Subpackages publish before the main package so optionalDependencies resolve. - `.github/workflows/build-image.yml` — hourly cron + push-to- images/ trigger. Builds Dockerfile, tags `latest` + timestamped `YYYY-MM-DDTHH` + `sha-<sha>`, pushes to ghcr.io. Retention policy prunes hourly tags older than 14 days. ## README Quickstart leads with `npm install -g`. Source-build instructions moved to a dedicated section. Image-cadence section explains the hourly rebuild + pinnable date tags + image-API contract. ## Tests (84/84 pass) - msb_install: 4 (version-marker accept/reject + env-override behaviour). - pull: 2 (local-registry detection: localhost/127.0.0.1/0.0.0.0/ .local/*.localhost vs ghcr.io/docker.io/example.com). - image_api_version: 5 (parse trim, parse rejects garbage, in-range accept, too-new with actionable hint, too-old with hint). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-24 21:02:56 +03:00

Author

SHA1

Message

Date

Evgeny Boger

0468a2649a

B2: ship a native aarch64-linux release binary via npm

Why
---
PLAN.md item "B2 — Cross-arch binaries" (section B, Distribution /
release) calls for cross-arch builds and per-platform npm packaging,
noting that "the package currently bundles a linux-x86_64 binary". As a
result Apple Silicon and ARM Linux users get no native artifact and fall
back to slow x86 emulation (Rosetta / qemu) — a poor fit for a tool whose
job is to launch microVMs quickly. This change delivers the linux/aarch64
half of B2: a genuine aarch64 binary, built, packaged, and selected at
install time, so ARM hosts run native.

How
---
Release workflow (.github/workflows/release-npm.yml): every build job
(build-agent-vm, build-msb, build-libkrunfw) and the package job gain a
linux-arm64 matrix leg. Because GitHub's hosted arm64 Linux runners
aren't on the free tier, the arm64 leg cross-compiles on the x64 runner:
a `cross: true` matrix flag drives a `CROSS` env switch that (a) enables
the arm64 multiarch apt repo and installs the cross linker
(gcc-aarch64-linux-gnu) plus the :arm64 dev libs agent-vm / msb link
against (libcap-ng, libdbus, libsqlite3), and (b) exports
PKG_CONFIG_ALLOW_CROSS / PKG_CONFIG_PATH / PKG_CONFIG_SYSROOT_DIR so the
`pkg-config` crate resolves the target libs. The msb job also sets
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER directly, since it builds
from vendor/microsandbox (a separate workspace not on the superproject's
.cargo config search path). All of these guards are no-ops on the native
x64 leg, so the existing x86_64 path is unchanged. The libkrunfw arm64
leg cross-builds the guest kernel with ARCH=arm64 / CROSS_COMPILE and
fails fast with a clear message if its arm64 kbuild .config seed
(libkrunfw-overrides/config-libkrunfw_aarch64.patch) hasn't been ported
yet, rather than silently shipping a mis-configured kernel. The publish
job downloads the new agent-vm-linux-arm64 artifact alongside the x64 one.

Cross toolchain config (.cargo/config.toml, new): pins the
aarch64-unknown-linux-gnu linker to aarch64-linux-gnu-gcc so a local
cross build reproduces CI; the file documents the matching apt packages.

npm launcher (npm-dist/agent-vm/bin/agent-vm.js,
npm-dist/agent-vm/package.json): un-comment the linux-arm64 entry in the
launcher's PLATFORM_PACKAGES map and add @wirenboard/agent-vm-linux-arm64
to the main package's optionalDependencies, so `npm install` pulls the
arm64 subpackage on ARM hosts and the launcher dispatches to it.

arm64 subpackage scaffold (npm-dist/agent-vm-linux-arm64/, new): mirrors
the x64 subpackage layout — package.json (os linux / cpu arm64),
README.md, and bin/.gitkeep + lib/.gitkeep placeholders — so the
directory exists in the tree for the release workflow to drop the
cross-built binary, msb, and libkrunfw into.

Dispatch test (npm-dist/agent-vm/bin/agent-vm.dispatch.test.js, new):
arm64 dispatch can't be exercised on an x86_64 CI host because node
reports the host's real process.arch. The test re-derives the launcher's
PLATFORM_PACKAGES map + bin-path logic from the source and asserts the
linux-arm64 key resolves to the arm64 subpackage with the same bin/
layout as x64, so a future edit that forgets arm64 fails here instead of
silently falling through to "no prebuilt binary" on ARM hardware.

README (npm-dist/README.md): document that the package now ships both the
linux-x64 and linux-arm64 per-platform subpackages.

macOS / darwin and win32 cross builds remain out of scope for this change
(still commented placeholders in the launcher) and are tracked under the
rest of B2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-31 18:57:07 +00:00

Evgeny Boger

494b8dd0aa

distribution: address code-review findings before v0.1.0

Top fixes from the max-effort review:

**Correctness:**

1. setup.rs::verify_image now calls image_api_version::check() and
   runs each agent --version separately. Old code did
   `cat ... && claude --version && ...` so a non-zero exit could
   mean any of four causes; error message blamed only the API file.
   New code names which check failed.

2. main.rs runs `point_at_msb` BEFORE constructing the tokio
   runtime. setenv() is not thread-safe; the previous code ran
   `unsafe { set_var }` after `#[tokio::main]` had already spawned
   workers, technically UB if a worker read env concurrently
   (reqwest / sea-orm do this on first use). Switched to plain
   `fn main` + manual Runtime::new() so the env mutation happens
   single-threaded.

3. release-npm.yml cache key now includes a hash of `rustc -V`.
   Without this, an ubuntu-latest rustc rollover (we hit this
   exact bug locally this session) reused cached rlibs from the
   prior toolchain → undefined-symbol link errors.

4. release-npm.yml's subpackage publish loop is now idempotent:
   skips already-published versions instead of erroring. Avoids
   the split-state where a transient failure leaves subpackages
   at v but the main package never published.

5. build-image.yml's retention switched from
   `actions/delete-package-versions@v5`'s `min-versions-to-keep:
   350` (which breaks if cron is paused for days) to an explicit
   age-based prune via actions/github-script — deletes versions
   older than 14 days regardless of build cadence.

**UX:**

6. pull.rs adds `AGENT_VM_INSECURE_REGISTRY=1` env-var escape hatch
   so airgapped/intranet plain-HTTP registries
   (`registry.corp.example:5000`) can opt in. The hostname
   heuristic stays narrow (localhost/127.0.0.1/0.0.0.0/*.local/
   *.localhost) for safety.

7. msb_install.rs: MSB_PATH set-but-file-missing now falls back to
   the sibling-of-current_exe() if one exists (with a warn) and
   gives a much clearer error otherwise. The vanilla-msb rejection
   message now detects MSB_PATH presence and tells the user to
   unset it instead of generically suggesting "set MSB_PATH
   explicitly" (the LAST thing they need to hear when they just set
   it).

8. setup.rs warns when running from a source checkout (manifest_dir
   has images/Dockerfile) but the submodule isn't initialised —
   instead of silently pulling ghcr.io and confusing the dev who
   just edited images/Dockerfile.

9. npm-dist/agent-vm/bin/agent-vm.js' missing-optional error lists
   network-flake as the top cause (it is) plus --no-optional and
   inconsistent lockfile pinning. Previously blamed --no-optional
   as the most likely cause, which it usually isn't.

10. README files for both npm packages so npmjs.com renders
    something useful.

11. release-npm.yml's libkrunfw symlink step replaced the
    `( cd && ln && ln )` subshell with absolute-path `ln -sf`s.
    Subshell `set -e` propagation is bash-version-sensitive; the
    new form is unconditional.

84/84 → 85/85 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-25 11:43:08 +03:00

Evgeny Boger

23cba5663b

distribution: ship via npm + ghcr.io image, drop local-build flow

Replaces the build-from-source quickstart (Rust toolchain + docker +
local registry container + manual `images/build.sh`) with a single
`npm install -g @wirenboard/agent-vm` and a `ghcr.io/wirenboard/
agent-vm:latest` image pulled from CI.

## What ships via npm

`npm-dist/` scaffolds the established per-platform-subpackage
pattern (esbuild / ruff / biome):

- `@wirenboard/agent-vm` — JS launcher that detects
  `${platform}-${arch}` at runtime and execs the right native binary
  from the matching subpackage. `optionalDependencies` lists each
  platform.
- `@wirenboard/agent-vm-linux-x64` — bundles a prebuilt `agent-vm`,
  the patched `msb`, and `lib/libkrunfw.so.5.2.1`.

Smoke-tested locally via `npm link`: launcher resolves the subpackage
correctly and execs the bundled binary; `MSB_PATH=/usr/bin/cat
agent-vm pull` correctly rejects an unpatched msb with the
actionable "reinstall agent-vm" hint.

## msb discovery (msb_install.rs rewrite)

`point_at_workspace_msb` → `point_at_msb` with a fallback chain:
1. `MSB_PATH` env (explicit override).
2. `<exe-dir>/msb` (npm bundle layout — sibling of agent-vm).
3. `<workspace>/vendor/microsandbox/target/release/msb` (dev mode).

`msb --version` is verified to contain `+agent-vm` — refuses to
launch if a user's separate microsandbox install has somehow ended
up on our discovered path. Catches the bug where shadow installs
would silently lose SecretValue::File and the request-interceptor
hook, producing inscrutable in-VM failures instead of one clear
"reinstall" message.

## Image flow (separate cadence)

- Default image ref: `ghcr.io/wirenboard/agent-vm:latest`.
- `images/build.sh` and local-registry flow stay as the
  source-checkout dev path (pass `--image localhost:5000/...`).
- `agent-vm setup` becomes "pull + verify"; no more docker/registry
  dependency for end users.
- New `image_api_version` module reads `/etc/agent-vm-image-version`
  inside the booted sandbox and requires it within
  `MIN_SUPPORTED_IMAGE_API..=MAX_SUPPORTED_IMAGE_API` (both 1
  initially). Out-of-range → actionable error pointing at the right
  side (binary vs image) to update.
- Image registry hostname heuristic (`is_plain_http_registry`)
  keeps `.insecure()` for localhost-style refs but drops it for
  ghcr.io / docker.io / public registries that require HTTPS.

## CI workflows

- `.github/workflows/release-npm.yml` — on `v*.*.*` tag, cross-
  compile binaries per platform (linux-x64 only initially; arm64 /
  darwin commented out, ready to enable when microsandbox grows
  those backends), bundle into subpackages, publish all packages
  to npm with provenance. Subpackages publish before the main
  package so optionalDependencies resolve.
- `.github/workflows/build-image.yml` — hourly cron + push-to-
  images/ trigger. Builds Dockerfile, tags `latest` + timestamped
  `YYYY-MM-DDTHH` + `sha-<sha>`, pushes to ghcr.io. Retention
  policy prunes hourly tags older than 14 days.

## README

Quickstart leads with `npm install -g`. Source-build instructions
moved to a dedicated section. Image-cadence section explains the
hourly rebuild + pinnable date tags + image-API contract.

## Tests (84/84 pass)

- msb_install: 4 (version-marker accept/reject + env-override behaviour).
- pull: 2 (local-registry detection: localhost/127.0.0.1/0.0.0.0/
  *.local/*.localhost vs ghcr.io/docker.io/example.com).
- image_api_version: 5 (parse trim, parse rejects garbage,
  in-range accept, too-new with actionable hint, too-old with hint).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-24 21:02:56 +03:00

3 commits