mirror of
https://github.com/wirenboard/agent-vm.git
synced 2026-07-09 16:00:54 +00:00
aba431cfc8
Code-review follow-up to the deterministic-manifest change. SOURCE_DATE_EPOCH only clamps the image CONFIG (created + history); layer file mtimes still defaulted to build wall-clock, so the manifest digest was reproducible only while the gha layer cache held. On a cache miss (eviction / size cap / base-layer invalidation) the apt/curl RUN steps re-ran and emitted layers with fresh mtimes → new diff_ids → new manifest digest for an unchanged commit, re-introducing the ~700 MiB consumer re-pull. - outputs: add `rewrite-timestamp=true` so buildkit clamps every in-layer file mtime to SOURCE_DATE_EPOCH (the commit time). Layer digests — and thus the manifest digest — are now reproducible independent of cache state; the digest moves only on a real content change. - sbom: false — pin it (defaults off today) so a future buildx default flip can't re-wrap the single-platform build in an attestation index whose build-time digest would churn :latest hourly. Complements provenance:false. - "Show pushed digest" now asserts the pushed `:sha-…` is a single image manifest, not an index — fails the build if an attestation ever leaks back in and silently restores the churn. Follow-up to #15 / part of #17. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016gtxsKVWPrR8K1M1Te57MG
308 lines
16 KiB
YAML
308 lines
16 KiB
YAML
name: build-image
|
||
|
||
# Build the agent-vm OCI image and push to GHCR on:
|
||
# - hourly schedule (picks up new claude-code/codex/opencode releases),
|
||
# - any push that touches images/ (Dockerfile or related),
|
||
# - manual dispatch (force a rebuild).
|
||
#
|
||
# Tags pushed:
|
||
# - `latest` — moving tag, used by agent-vm by default.
|
||
# - `YYYY-MM-DDTHH` — immutable, for users who need to pin.
|
||
# - `<short-sha>` — also immutable, for traceability to source.
|
||
#
|
||
# The binary release pipeline (release-npm.yml) runs on a separate
|
||
# cadence; image-API-version contract (images/Dockerfile + defaults.rs)
|
||
# keeps them compatible.
|
||
|
||
on:
|
||
schedule:
|
||
- cron: '0 * * * *' # every hour, on the hour, UTC
|
||
push:
|
||
branches:
|
||
- main
|
||
paths:
|
||
- 'images/**'
|
||
- '.github/workflows/build-image.yml'
|
||
workflow_dispatch:
|
||
|
||
permissions:
|
||
contents: read
|
||
packages: write # for ghcr.io push
|
||
|
||
concurrency:
|
||
# One image build at a time. Newer runs cancel queued older ones —
|
||
# but never interrupt a build in progress (cancel-in-progress: false).
|
||
group: build-image
|
||
cancel-in-progress: false
|
||
|
||
env:
|
||
IMAGE: ghcr.io/${{ github.repository_owner }}/agent-vm-template
|
||
|
||
jobs:
|
||
build:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v5
|
||
|
||
- name: Compute timestamp tag + source epoch
|
||
id: ts
|
||
run: |
|
||
# Immutable hourly pin (:YYYY-MM-DDTHH).
|
||
echo "tag=$(date -u +%Y-%m-%dT%H)" >> "$GITHUB_OUTPUT"
|
||
# SOURCE_DATE_EPOCH = HEAD commit time. Fed to buildkit (see
|
||
# the build step), it clamps the image config's `created`
|
||
# field and every history timestamp to a value that is
|
||
# CONSTANT across the hourly cron builds of a given commit.
|
||
# So two builds with the same commit + same agent versions +
|
||
# same layers produce a byte-identical config → an identical
|
||
# linux/amd64 manifest digest. microsandbox keys its
|
||
# fsmeta/VMDK materialization on THAT digest, so an unchanged
|
||
# `:latest` becomes a true no-op pull (0 bytes) instead of
|
||
# re-downloading the whole image. A new commit, or any changed
|
||
# agent layer, moves the digest as it should. Without this the
|
||
# config carried a fresh build timestamp every hour, so every
|
||
# cron `:latest` had a new manifest digest and consumers
|
||
# re-pulled ~700 MiB of byte-identical layers hourly.
|
||
echo "epoch=$(git log -1 --pretty=%ct)" >> "$GITHUB_OUTPUT"
|
||
|
||
- name: Set up Docker Buildx
|
||
uses: docker/setup-buildx-action@v4
|
||
|
||
- name: Log in to GHCR
|
||
uses: docker/login-action@v4
|
||
with:
|
||
registry: ghcr.io
|
||
username: ${{ github.actor }}
|
||
password: ${{ secrets.GITHUB_TOKEN }}
|
||
|
||
- name: Resolve agent versions
|
||
id: ver
|
||
env:
|
||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||
run: |
|
||
set -euo pipefail
|
||
fail() { echo "::error::$1"; exit 1; }
|
||
# Resolve each agent's current upstream version and feed it to
|
||
# the Dockerfile as a per-agent cache key (AGENT_VERSION_*), so
|
||
# an installer layer is rebuilt only when its agent actually
|
||
# released — not on every hourly cron run. Each source is the
|
||
# EXACT one the matching installer reads, so the key tracks
|
||
# what actually gets installed:
|
||
# - codex: openai/codex releases/latest tag — install.sh's
|
||
# own resolver hits the same endpoint.
|
||
# - opencode: anomalyco/opencode releases/latest tag — the
|
||
# repo opencode.ai/install downloads from (it was
|
||
# renamed from sst/opencode; sst/* only resolves
|
||
# today via GitHub's rename redirect, so pin the
|
||
# real repo).
|
||
# - claude: downloads.claude.ai native channel `/latest`
|
||
# (a plain version string) — what claude.ai/
|
||
# install.sh installs. NOT npm: the npm dist-tags
|
||
# (@anthropic-ai/claude-code) move on a separate
|
||
# cadence (e.g. stable != latest), so keying on
|
||
# npm would miss native releases / rebuild for
|
||
# npm-only bumps.
|
||
# Any lookup failure fails the build (vs shipping a stale key
|
||
# that would skip a real agent update).
|
||
codex=$(gh api repos/openai/codex/releases/latest --jq .tag_name) \
|
||
|| fail "codex version lookup failed (openai/codex releases/latest)"
|
||
opencode=$(gh api repos/anomalyco/opencode/releases/latest --jq .tag_name) \
|
||
|| fail "opencode version lookup failed (anomalyco/opencode releases/latest)"
|
||
claude=$(curl -fsSL https://downloads.claude.ai/claude-code-releases/latest) \
|
||
|| fail "claude version lookup failed (downloads.claude.ai/claude-code-releases/latest)"
|
||
# Guard against an HTTP-200-but-garbage response (empty body,
|
||
# jq 'null', or an HTML error page) silently becoming a key.
|
||
[ -n "$codex" ] && [ "$codex" != null ] || fail "codex version empty/null"
|
||
[ -n "$opencode" ] && [ "$opencode" != null ] || fail "opencode version empty/null"
|
||
case "$claude" in [0-9]*.[0-9]*.[0-9]*) : ;; *) fail "claude version implausible: '$claude'" ;; esac
|
||
{
|
||
echo "codex=$codex"
|
||
echo "opencode=$opencode"
|
||
echo "claude=$claude"
|
||
} >> "$GITHUB_OUTPUT"
|
||
echo "resolved agent versions: codex=$codex opencode=$opencode claude=$claude"
|
||
|
||
- name: Build and push
|
||
id: build
|
||
env:
|
||
# Clamps buildkit's image-config `created` + history
|
||
# timestamps to the commit time (see "Compute timestamp tag +
|
||
# source epoch") so an unchanged build yields an identical
|
||
# manifest digest. docker/build-push-action forwards
|
||
# SOURCE_DATE_EPOCH from the environment to buildkit.
|
||
SOURCE_DATE_EPOCH: ${{ steps.ts.outputs.epoch }}
|
||
uses: docker/build-push-action@v7
|
||
with:
|
||
context: images
|
||
file: images/Dockerfile
|
||
# `outputs: type=registry,push=true` replaces a bare
|
||
# `push: true` so we can carry compression directives on the
|
||
# same step. The benchmark in crates/agent-vm/examples/
|
||
# erofs_bench.rs showed gzip → zstd cuts the per-layer
|
||
# OCI→EROFS conversion cost by ~24× (143s → 5s on 856 MB
|
||
# of /usr/lib). `force-compression=true` re-emits even
|
||
# already-compressed base-image layers as zstd; without it
|
||
# we'd only re-compress layers we add and the base layers
|
||
# would stay gzip, halving the win. `compression-level=3`
|
||
# is zstd's default — the bench shows diminishing returns
|
||
# past that for binary-heavy content.
|
||
# microsandbox's `tar_ingest.rs:427` already accepts the
|
||
# `application/vnd.oci.image.layer.v1.tar+zstd` media type.
|
||
#
|
||
# `rewrite-timestamp=true` clamps the mtime of every file
|
||
# INSIDE each layer tar to SOURCE_DATE_EPOCH (the commit time).
|
||
# SOURCE_DATE_EPOCH alone only fixes the image config; layer
|
||
# file mtimes default to build wall-clock, so a `cache-from`
|
||
# MISS (gha eviction / size cap / a base-layer invalidation)
|
||
# would re-run the apt/curl RUN steps and emit layers with
|
||
# fresh mtimes → new diff_ids → new manifest digest for an
|
||
# unchanged commit, re-introducing the ~700 MiB consumer
|
||
# re-pull. Clamping mtimes makes the layer digests — and thus
|
||
# the manifest digest — reproducible independent of cache
|
||
# state, so the digest only moves on a real content change.
|
||
outputs: type=registry,push=true,compression=zstd,compression-level=3,force-compression=true,rewrite-timestamp=true
|
||
# Per-agent version cache keys. `Resolve agent versions`
|
||
# above looks up each agent's current upstream version; the
|
||
# Dockerfile references these in the matching installer RUN,
|
||
# so a layer is rebuilt ONLY when that agent actually
|
||
# released — not on every hourly cron run. This keeps an
|
||
# unchanged hourly build a pure cache hit (identical
|
||
# agent-layer digests → `:latest` consumers re-pull nothing)
|
||
# instead of re-emitting ~200 MiB of fresh-but-identical
|
||
# agent layers every hour. The heavy apt/chromium/docker/
|
||
# node layers below the args stay cached regardless.
|
||
build-args: |
|
||
AGENT_VERSION_CODEX=${{ steps.ver.outputs.codex }}
|
||
AGENT_VERSION_OPENCODE=${{ steps.ver.outputs.opencode }}
|
||
AGENT_VERSION_CLAUDE=${{ steps.ver.outputs.claude }}
|
||
# `cache-from`/`cache-to` keep layer rebuilds fast when only
|
||
# the agent-version layers at the top of the Dockerfile
|
||
# change (the common hourly case).
|
||
cache-from: type=gha
|
||
cache-to: type=gha,mode=max
|
||
# No provenance/SBOM attestation. Either one makes buildx wrap
|
||
# even a single-platform build in an OCI index whose extra
|
||
# attestation manifest embeds the build time, so the index
|
||
# digest changes every build even when the image content is
|
||
# identical — which would keep `:latest`'s digest churning
|
||
# hourly (and nag the update banner) despite a stable
|
||
# linux/amd64 manifest. We don't consume either anywhere.
|
||
# Both are pinned off (provenance defaults to mode=min on push;
|
||
# sbom defaults off but is pinned so a future default flip
|
||
# can't silently re-introduce the index). The "Show pushed
|
||
# digest" step asserts the push stayed a single manifest.
|
||
provenance: false
|
||
sbom: false
|
||
# Moving tags (`:latest`, `:YYYY-MM-DDTHH`) are gated to
|
||
# the integration branches (`main`, `rewrite-microsandbox`)
|
||
# so workflow_dispatch on a feature branch can verify the
|
||
# build without clobbering production. The immutable
|
||
# `:sha-…` tag is always pushed and is what we inspect
|
||
# for branch-level verification.
|
||
# rewrite-microsandbox is included because it's the de-facto
|
||
# trunk for the microsandbox-rewrite work; release-npm
|
||
# already documents that workflow_dispatch happens from
|
||
# there too (release-npm.yml:96).
|
||
tags: |
|
||
${{ (github.ref_name == 'main' || github.ref_name == 'rewrite-microsandbox') && format('{0}:latest', env.IMAGE) || '' }}
|
||
${{ (github.ref_name == 'main' || github.ref_name == 'rewrite-microsandbox') && format('{0}:{1}', env.IMAGE, steps.ts.outputs.tag) || '' }}
|
||
${{ env.IMAGE }}:sha-${{ github.sha }}
|
||
# Tag with the short SHA too so we can trace any image
|
||
# back to the exact Dockerfile commit.
|
||
# NOTE: no `org.opencontainers.image.created` label. It was
|
||
# set to the per-hour timestamp and lived in the image config,
|
||
# so it changed the manifest digest every cron build even with
|
||
# identical content (the root cause of the hourly re-pull). The
|
||
# config still carries a `created` field, but clamped to the
|
||
# commit time via SOURCE_DATE_EPOCH, so it's stable per commit.
|
||
# `revision` (the commit sha) is stable across a commit's
|
||
# hourly builds, so it doesn't reintroduce churn.
|
||
labels: |
|
||
org.opencontainers.image.source=https://github.com/${{ github.repository }}
|
||
org.opencontainers.image.revision=${{ github.sha }}
|
||
|
||
- name: Show pushed digest + assert single manifest
|
||
run: |
|
||
echo "Pushed ${{ env.IMAGE }}:${{ steps.ts.outputs.tag }}"
|
||
echo " digest: ${{ steps.build.outputs.digest }}"
|
||
# Guard the digest-stability invariant: the push must be a
|
||
# single image manifest, NOT a multi-arch index. An index
|
||
# (from a re-enabled provenance/sbom attestation, or a buildx
|
||
# default flip) carries a build-time-varying attestation digest
|
||
# and would silently restore the hourly `:latest` churn this
|
||
# workflow exists to prevent. The immutable `:sha-…` tag is
|
||
# always pushed (on every branch), so inspect that.
|
||
ref="${IMAGE}:sha-${{ github.sha }}"
|
||
if mt=$(docker buildx imagetools inspect --raw "$ref" 2>/dev/null | jq -r '.mediaType'); then
|
||
echo " pushed mediaType: $mt"
|
||
case "$mt" in
|
||
*manifest.list*|*image.index*)
|
||
echo "::error::pushed an index ($mt) — an attestation leaked back in; :latest digest will churn hourly. Ensure provenance:false + sbom:false."
|
||
exit 1 ;;
|
||
esac
|
||
else
|
||
echo "::warning::could not inspect $ref to assert single-manifest; skipping the check"
|
||
fi
|
||
|
||
retain:
|
||
# Delete date-tagged image versions older than 14 days so the
|
||
# registry doesn't grow unbounded. `latest` and `sha-...` tags
|
||
# are immune via the ignore-versions regex.
|
||
#
|
||
# Why a custom script and not actions/delete-package-versions's
|
||
# `min-versions-to-keep`: that knob only fires once you HAVE at
|
||
# least N versions, so a paused/disabled cron can leave stale
|
||
# versions outliving the 14-day window unbounded. The age-based
|
||
# approach prunes correctly regardless of build cadence.
|
||
needs: build
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- name: Prune image versions older than 14 days
|
||
uses: actions/github-script@v9
|
||
with:
|
||
script: |
|
||
const owner = context.repo.owner;
|
||
// GHCR package name — `agent-vm-template`, not the repo's
|
||
// short name. The repo was renamed in ca6a2a3 but this
|
||
// string was left stale, so every `retain` run since then
|
||
// has failed with HttpError 404 and pruned nothing.
|
||
const pkg = 'agent-vm-template';
|
||
const horizonMs = 14 * 24 * 60 * 60 * 1000;
|
||
const now = Date.now();
|
||
// Patterns we never delete, regardless of age.
|
||
const keepTagRe = /^(latest|sha-[0-9a-f]+)$/i;
|
||
|
||
// List all versions (paginated).
|
||
const versions = await github.paginate(
|
||
github.rest.packages.getAllPackageVersionsForPackageOwnedByOrg,
|
||
{
|
||
package_type: 'container',
|
||
package_name: pkg,
|
||
org: owner,
|
||
per_page: 100,
|
||
}
|
||
);
|
||
core.info(`found ${versions.length} versions of ${owner}/${pkg}`);
|
||
|
||
let deleted = 0;
|
||
for (const v of versions) {
|
||
const tags = (v.metadata?.container?.tags) ?? [];
|
||
// Skip anything carrying a protected tag.
|
||
if (tags.some((t) => keepTagRe.test(t))) continue;
|
||
// Untagged manifest lists / blob orphans can be GC'd too.
|
||
const age = now - new Date(v.created_at).getTime();
|
||
if (age <= horizonMs) continue;
|
||
core.info(`deleting ${pkg}@${v.id} tags=${JSON.stringify(tags)} age_days=${(age/86400000).toFixed(1)}`);
|
||
try {
|
||
await github.rest.packages.deletePackageVersionForOrg({
|
||
package_type: 'container',
|
||
package_name: pkg,
|
||
org: owner,
|
||
package_version_id: v.id,
|
||
});
|
||
deleted++;
|
||
} catch (e) {
|
||
core.warning(`delete ${v.id} failed: ${e.message}`);
|
||
}
|
||
}
|
||
core.info(`deleted ${deleted} version(s) older than 14 days`);
|