Sprite has a bun shim at /.sprite/bin/bun that delegates to
$HOME/.bun/bin/bun, but that binary doesn't exist on fresh VMs.
`command -v bun` returns true (finds the shim) so the install script
skips bun installation, then bun fails when actually invoked.
Fixed in two places:
- installSpawnCli: source shell profiles, test `bun --version` (not
just existence), and install bun fresh if it doesn't work
- install.sh: replace `command -v bun` with `bun --version` to detect
broken shims
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: spawn step skipped when no explicit --steps passed
The spawn skill injection condition used `enabledSteps?.has("spawn")`
which is falsy when enabledSteps is undefined (no --steps flag). Now
checks the recursive beta flag directly and falls through when no
explicit steps are selected, matching how auto-update works.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: embed skill content in spawn-skill.ts instead of reading from disk
The skills/ directory exists in the repo but isn't bundled when the CLI
is installed via npm. readSkillContent() couldn't find the files at
runtime, causing "No spawn skill file for agent" on every deploy.
Fixed by embedding all skill content directly as string constants in the
module. Removed fs-based getSkillsDir/readSkillContent/getSpawnSkillSourceFile
in favor of a single AGENT_SKILLS config map with inline content.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When `--beta recursive` is active, a new "Spawn CLI" setup step injects
agent-native instruction files teaching each agent how to use the `spawn`
CLI to create child VMs. Skill files live in `skills/` at the repo root
and use each agent's native format (YAML frontmatter for Claude/Codex/
OpenClaw, plain markdown for others, append mode for Hermes).
- Add `skills/` directory with 8 agent-specific skill files
- Add `spawn-skill.ts` module with path mapping, file reading, and injection
- Register "spawn" as a conditional setup step gated by `--beta recursive`
- Wire `injectSpawnSkill()` into orchestrate.ts postInstall flow
- Add 52 tests covering path mapping, append mode, file existence, injection
- Bump CLI version to 0.26.0 (minor: new feature)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds non-empty guard to makeDockerExec to make the security boundary
explicit and prevent silent misuse with empty commands.
Fixes#2985
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously only `settingsB64` had a validation check. Added the same
`/^[A-Za-z0-9+/=]+$/` guard for wrapperB64, unitB64, and timerB64
before they are interpolated into shell commands, closing the consistency gap.
Fixes#2986
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The `skips when no credential files exist` test in recursive-spawn.test.ts
was failing in the full suite (1911 pass, 1 fail) because other test files
(oauth-cov.test.ts, cmd-uninstall-cov.test.ts) write openrouter.json and
hetzner.json to $HOME/.config/spawn/ without cleanup, contaminating the
shared sandbox HOME used by bun's test runner. The test passed in isolation
but failed 100% of the time in the full suite.
Fix: add a beforeEach inside the delegateCloudCredentials describe block
that removes $HOME/.config/spawn/ before each test, making the test
self-contained and immune to cross-file pollution.
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add recursive spawn (--beta recursive)
Enables VMs to spawn child VMs. When --beta recursive is active:
- Injects SPAWN_PARENT_ID, SPAWN_DEPTH, SPAWN_BETA=recursive into .spawnrc
- Installs spawn CLI on the VM via install.sh
- Delegates cloud + OpenRouter credentials to the VM
- Tracks parent_id and depth on SpawnRecord for tree relationships
- Adds `spawn tree` command for full recursive tree view
- Adds `spawn history export` for pulling child history via SSH
- Adds `spawn list --json` and `spawn list --flat` flags
- Adds tree rendering in `spawn list` when parent-child relationships exist
- Adds cascade delete support in delete.ts
- Adds mergeChildHistory() for backward-pass history sync
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add recursive spawn to README
Add --beta recursive to beta features table, new commands
(spawn tree, spawn history export, spawn list --flat/--json)
to commands table, and a dedicated Recursive Spawn section
with usage examples for tree view and cascade delete.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add cmdTree coverage tests to fix mock test CI
The CI coverage threshold (90% functions, 80% lines) was failing
because tree.ts had 0% coverage. Added tests that exercise cmdTree
with empty history, tree rendering, JSON output, flat records,
and deleted/depth labels. tree.ts now has 100% coverage.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(security): validate cloudName and use valibot in pullChildHistory
- Add cloudName validation against ^[a-z0-9-]+$ to prevent
command injection in delegateCloudCredentials
- Export SpawnRecordSchema from history.ts and replace loose
type guard with valibot schema validation in pullChildHistory
- Resolve merge conflicts with main (include both docker and
recursive beta features)
Agent: pr-maintainer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* test: add installSpawnCli and delegateCloudCredentials coverage
Export and test installSpawnCli (success + timeout failure paths)
and delegateCloudCredentials (no creds, with creds, write failure,
mkdir failure paths) to improve orchestrate.ts function coverage.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: gritQL rule false positives and delete.ts coverage
- use TsAsExpression() AST node instead of backtick pattern to avoid
matching import aliases as type assertions
- export and test findDescendants() and pullChildHistory() to bring
delete.ts line coverage above the 35% threshold
- add 8 new tests for descendant finding and history pull edge cases
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: A <258483684+la14-1@users.noreply.github.com>
* fix: pin all GitHub Actions to commit SHAs and version-lock tools
Addresses supply chain hardening findings from issue #2982:
- Pin all 6 GitHub Actions to full commit SHAs with version comments:
- actions/checkout@v4 → SHA 34e1148...
- oven-sh/setup-bun@v2 → SHA 0c5077e...
- actions/github-script@v7 → SHA f28e40c...
- docker/login-action@v3 → SHA c94ce9f...
- docker/build-push-action@v6 → SHA 10e90e3...
- hashicorp/setup-packer@main → SHA c3d53c5... (v3.2.0)
- Pin Packer version: latest → 1.15.0 (in packer-snapshots.yml)
- Pin bun version: latest → 1.3.11 (in agent-tarballs.yml)
- Pin shellcheck: replace apt-get (no version) with pinned download
of v0.10.0 from GitHub releases with SHA256 integrity check
These changes eliminate the primary LiteLLM-style attack vector:
a compromised action maintainer can no longer force-push malicious
code to an existing tag and have it run in CI.
Fixes#2982
Agent: issue-fixer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: exclude import aliases from no-type-assertion lint rule
The `JsNamedImportSpecifier` exclusion prevents `import { foo as bar }`
patterns from being flagged as type assertions. Previously, any `as`
keyword in import/export statements triggered the ban because the GritQL
pattern `$value as $type` matched import specifiers as well as actual
TypeScript type assertions.
This also removes the `as _foo` import aliases in the script-failure-guidance
test file (replaced with direct imports + distinctly-named wrapper functions)
which were the original manifestation of this bug.
All 1944 tests pass. Biome check clean across 169 files.
Agent: issue-fixer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove weaker duplicates found during QA quality sweep:
orchestrate-cov.test.ts: remove "orchestrate restart loop" describe block
(2 tests) — duplicates tests already in orchestrate.test.ts with fewer
assertions (missing "my-agent --run" and "Restarting in 5s" checks).
cmd-delete-cov.test.ts: remove theatrical "intercepts stderr writes to
update spinner" test — handler was a no-op mock, only asserted return
value, never verified actual stderr interception. Duplicate of
"calls custom deleteHandler and reports success" in the same file.
Real stderr/spinner behavior is covered by delete-spinner.test.ts.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- Suppress remote command output in Hetzner runServer() by piping
stdout/stderr instead of inheriting. This prevents raw ANSI escape
sequences from remote install commands (spinners, progress bars)
from leaking into the local terminal as garbled characters, and
eliminates duplicate status messages that were repeated 15+ times.
Captured stderr is logged via logDebug on failure for debugging.
- Add LC_ALL=C.UTF-8 to both the interactive SSH session and the
.spawnrc env config to ensure consistent UTF-8 locale across all
locale categories, preventing garbled Unicode rendering in Claude
Code's TUI welcome interface.
Agent: ux-engineer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: remove docker from --fast and fix docker cp into container
Two fixes for --beta docker:
1. Remove "docker" from --fast beta features — --fast was auto-enabling
--beta docker, pulling ghcr images that hang the session.
Users must now opt in explicitly with --beta docker.
2. Fix uploadFile in docker mode — .spawnrc was uploaded to the host
but never copied into the container. Add docker cp after SCP upload
so env vars and configs reach the agent inside the container.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: keep docker in --fast beta features
The docker cp fix resolves the hang — no need to remove docker from
--fast. The issue was missing file copy into the container, not the
docker mode itself.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: extract makeDockerRunner helper, fix uploadFile into container
Add makeDockerRunner() that wraps a CloudRunner so all commands and
file uploads target the Docker container. Replaces inline lambdas in
hetzner/main.ts and gcp/main.ts with a clean one-liner.
The key fix: uploadFile now docker cp's files into the container after
SCP — previously .spawnrc (API keys, env vars) only landed on the host,
so the agent inside the container had no config and hung.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(security): shellQuote remotePath in docker cp command
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The "falls back to most recent record with connection when no spawnId"
test in history-spawn-id.test.ts duplicates the same-named test in
history-cov.test.ts. The history-cov version is more thorough: it uses
two records where the first lacks a connection, exercising the
"skip records without connection" logic. The history-spawn-id version
only had one record, providing no additional signal.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: remove local tarball download, use remote-only tarball install
The local-download-then-SCP-upload path was unnecessary complexity —
downloading a tarball to the user's machine just to re-upload it to the
VM is wasteful. The VM downloads directly from GitHub instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: force zeroclaw native runtime to prevent Docker container hang
ZeroClaw auto-detects Docker and launches in a container (pulling
ghcr.io/openrouterteam/spawn-zeroclaw), which hangs the interactive
session. Force native mode via ZEROCLAW_RUNTIME=native env var and
adapter = "native" in config.toml.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: disable openclaw Docker sandbox to prevent container hang
Same issue as zeroclaw — openclaw auto-detects Docker and runs agents
in containers, hanging the interactive session. Disable via
agents.defaults.sandbox.mode = off in config and fallback JSON.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: disable codex Docker sandbox to prevent container hang
Codex CLI also auto-detects Docker for sandboxing. Set
sandbox_mode = "danger-full-access" in config.toml — the VM itself
provides isolation, Docker sandboxing just causes hangs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 'junie agent envVars include JUNIE_OPENROUTER_API_KEY' test in
agent-setup-cov.test.ts was a weaker duplicate of the more precise
coverage in junie-agent.test.ts, which verifies the exact env var value.
1890 → 1889 tests (1 duplicate removed, 0 regressions).
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Extract duplicate dockerExec helper from gcp/main.ts and hetzner/main.ts
into shared makeDockerExec() in orchestrate.ts. Both local functions were
identical — wrapping commands with docker exec using DOCKER_CONTAINER_NAME
and shellQuote.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove 5 duplicate test cases from orchestrate-cov.test.ts that were
already covered by orchestrate.test.ts with stronger assertions:
- orchestrate checkAccountReady throws (duplicate, weaker version)
- orchestrate preProvision throws (duplicate, weaker version)
- tarball falls back to install when tarball returns false (exact duplicate)
- tarball skips for local cloud (exact duplicate)
- skipTarball agent flag (exact duplicate)
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 180s timeout to uploadFileSprite to prevent indefinite hangs during
tarball uploads. Without a timeout, large tarballs or stalled Sprite
connections block the entire provisioning pipeline past the 720s E2E
provision timeout, causing agent binary not-found failures for openclaw,
zeroclaw, and codex.
Also skip the redundant remote tarball download fallback when a local
tarball was already downloaded but its upload/extract failed -- the
remote download would face the same extraction issues. This saves ~150s
in the fallback chain, leaving enough time for the live install to
complete within the provision timeout.
Fixes#2960
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The hermes install script's mini-swe-agent pip dependency uses
git+ssh:// URLs that timeout on fresh cloud VMs (hetzner/gcp/digitalocean)
where outbound SSH to GitHub is blocked or slow.
Add `git config --global url.https://github.com/.insteadOf` rules
before the hermes install and update commands to force git to use
HTTPS instead of SSH for all GitHub URLs. This eliminates the SSH
connection timeout that was causing install failures.
Fixes#2955
Agent: ux-engineer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
safe_substitute: Switch sed delimiter from | to \x01 (SOH control char) across
qa.sh, refactor.sh, security.sh, and discovery.sh. This eliminates delimiter
injection regardless of value content, since \x01 cannot appear in normal input.
Values containing \x01 are explicitly rejected as defense-in-depth.
SPAWN_ISSUE: Fix qa.sh validation from ^[0-9]+$ to ^[1-9][0-9]*$ to reject
leading zeros and zero itself. Add 32-bit signed integer range check
(max 2147483647) to all three scripts (qa.sh, refactor.sh, security.sh)
to prevent integer overflow in downstream consumers.
Fixes#2961Fixes#2962
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Merged "createCloudAgents" and "createCloudAgents detailed" into a single
describe block. Both blocks tested the same function with no structural
distinction, causing duplicate organization without value.
Eliminated 26 repetitive inline runner object constructions by moving
runner and result setup into beforeEach. This removes ~115 lines of
boilerplate while keeping all 21 tests and their assertions intact.
1895 tests still pass.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
On interactive provision failure, save the harness log to a persistent
path (/tmp/spawn-interactive-harness-last.log) for post-mortem inspection,
and filter output to only show [harness] prefixed lines (30 lines) instead
of dumping 50 raw lines of mixed output.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Ahmed Abushagur <ahmed@abushagur.com>
Extracts the inline docker-mode condition from hetzner/main.ts and
gcp/main.ts into a testable exported function in shared/cloud-init.ts,
then adds real unit tests that import from the source. Fixes#2952.
Agent: test-engineer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Hermes installs a Python virtualenv which takes 20+ min on fresh VMs.
The previous 300s install timeout caused the CLI to give up before
writing .spawnrc, leading to 30-min E2E timeouts on Hetzner, DigitalOcean,
and GCP (but not Sprite, which has a manual .spawnrc fallback).
Changes:
- agent-setup.ts: hermes installAgent timeout 300s → 600s
- common.sh: add hermes per-agent overrides (_PROVISION_TIMEOUT_hermes=720,
_AGENT_TIMEOUT_hermes=3600) to give the install enough headroom
- package.json: bump CLI version 0.25.26 → 0.25.27
-- qa/e2e-tester
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
docker-cloudinit-skip.test.ts was reading source file contents with readFileSync
and checking for the presence of specific string literals — a source-grep
anti-pattern that tests the text exists, not that the behavior works.
The waitForReady() closure in hetzner/main.ts and gcp/main.ts cannot be directly
unit tested without refactoring (tracked in #2952). The source-grep tests are
removed to avoid false confidence.
Filed https://github.com/OpenRouterTeam/spawn/issues/2952 to track proper
behavioral testing via extracting the skip-cloud-init condition into a testable
exported helper.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
`buildFixScript()` was missing `export LANG='C.UTF-8'` that was added to
the canonical `generateEnvConfig()` in commit f93c799d. Users running
`spawn fix` would get a `.spawnrc` without the UTF-8 locale export,
causing garbled Unicode in agent TUIs — the same regression that f93c799d
fixed for fresh provisioning.
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
do-min-size.test.ts was reading source file contents with readFileSync
and checking for the presence of specific strings (bash-grep anti-pattern).
Fixes:
- Export slugRamGb and AGENT_MIN_SIZE from digitalocean.ts
- Import them in main.ts instead of re-defining
- Rewrite do-min-size tests to call functions with inputs and assert outputs
(3 source-grep tests → 6 behavior tests)
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1. Suppress Claude Code curl installer stdout — the remote installer
prints its own "Installation complete!" which duplicated the local
"Claude Code agent installed successfully" message.
2. Export LANG=C.UTF-8 in both the interactive SSH session command and
the .spawnrc env config. Fresh cloud VMs often default to the C
locale which cannot render Unicode properly, causing garbled ANSI
output in agent TUIs (e.g. "⏵⏵bypasspermissionson" instead of
properly spaced text).
Fixes#2946
Agent: ux-engineer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Two test files (do-min-size.test.ts, docker-cloudinit-skip.test.ts) existed
on disk but were not documented in the README. Add entries for both.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
When the quality cycle e2e-tester re-runs only failed agents
(e.g. `e2e.sh --cloud hetzner zeroclaw codex`), e2e.sh was firing
a matrix email showing only those 2 agents — both PASS if the retry
succeeded. This looked like "2 tests ran, all passed" when in reality
32 tests ran with 2 failures.
- Add SPAWN_E2E_SKIP_EMAIL=1 env var check at the top of send_matrix_email
- Update qa-quality-prompt.md to set SPAWN_E2E_SKIP_EMAIL=1 on re-runs
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The QA account's primary IP limit is ~3, so running 5 agents in parallel
exhausted the quota, causing codex and zeroclaw to fail with
resource_limit_exceeded. Reducing _hetzner_max_parallel to 3 keeps
provisioning within quota while still running agents concurrently.
Verified: zeroclaw and codex both PASS on Hetzner after this fix.
-- qa/e2e-tester
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
- hetzner.sh: Pipe base64-encoded command via stdin to SSH instead of
embedding it in the SSH command string via variable expansion. The
remote bash reads stdin, base64-decodes, and executes.
- verify.sh: Add remote-side re-validation of base64 and timeout values
in _stage_prompt_remotely and _stage_timeout_remotely. Values are
assigned to remote shell variables and validated before writing to
temp files, providing defense-in-depth against injection.
- provision.sh: Add explicit early rejection of dangerous shell chars
($, `, \) in env var values from cloud_headless_env, and add
remote-side re-validation of base64 payload before writing.
Fixes#2937Fixes#2938Fixes#2939
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
- Stash uncommitted changes before git pull --rebase so the pull
never aborts with "You have unstaged changes"
- Pull --rebase before pushing star count commit to avoid
non-fast-forward rejection (was failing every single cycle)
- Remove --yes flag from claude update (flag was removed upstream)
- Fix interactive harness AI prompt: update success marker text from
"is ready" or "Starting agent" to match code check
("Starting agent..." or "setup completed successfully")
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- fix misplaced interactive_provision comment block in interactive.sh:
the comment was positioned before _report_ux_issues but described the
interactive_provision function; moved it to be adjacent to its function
- apply interactive E2E improvements already in main working tree:
e2e.sh: add verify_agent call after interactive_provision to wait for
.spawnrc before running input tests (aligns interactive with headless flow)
-- qa/code-quality
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Three fixes for Sprite E2E failures in long-running batches (73+ min):
1. Retry `_sprite_provision_verify`: list failures now retry 3x with
exponential backoff (5s, 10s, 20s) instead of failing immediately.
Fixes kilocode batch 6 "Could not list Sprite instances" errors.
2. Increase `CREATE_TIMEOUT_SECS` default from 300s to 600s and add
`Client.Timeout`, `request canceled`, and `authentication failed`
to the transient error retry pattern in `spriteRetry`. Also uses
linear backoff (3s * attempt) instead of fixed 3s delay.
Fixes hermes batch 7 HTTP timeout errors.
3. Add `_sprite_refresh_auth` + `cloud_refresh_auth` interface. The
E2E orchestrator calls `cloud_refresh_auth` before each provisioning
batch. For Sprite, this re-validates the token via `sprite org list`
and attempts `sprite auth refresh` if expired.
Fixes junie batch 8 "authentication failed" errors.
Fixes#2934
Agent: ux-engineer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Hetzner E2E runs fail with `resource_limit_exceeded` when stale primary
IPs from previous test runs consume the account quota. This adds proactive
cleanup at two levels:
1. E2E shell driver: `_hetzner_cleanup_orphaned_ips()` deletes unattached
primary IPs during pre-batch stale cleanup, freeing quota before any
new servers are provisioned.
2. TypeScript CLI: `hetzner/main.ts` calls `cleanupOrphanedPrimaryIps()`
before `createServer()` in headless/non-interactive mode, ensuring
each agent provisioning attempt starts with a clean IP quota.
The existing reactive cleanup (retry after failure) in `hetzner.ts`
remains as a fallback.
Fixes#2933
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Hetzner's waitForReady() was missing the useDocker check that GCP
already has. Non-minimal agents (openclaw, codex) with --beta docker
waited 5 minutes for a cloud-init marker that never appears on Docker
CE app images.
Adds useDocker to the condition and a source-level regression test
verifying both Hetzner and GCP include the check.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
node:path.normalize() is platform-dependent — on Windows it converts
forward slashes to backslashes, which then fail the character allowlist
regex. Remote paths are always Linux paths regardless of the client OS.
Switch to node:path/posix so normalization always uses forward slashes.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the guard returns false, both functions re-threw the raw caught
value (e) instead of the normalized Error (err). If a non-Error value
was thrown (string, number), downstream handlers received inconsistent
types instead of always getting Error instances.
Changed throw e → throw err in both functions.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When history metadata lacks a project ID, spawn delete silently fell
back to the gcloud default project, attempting deletion in the wrong
project (404) while the instance kept running and billing.
Now fails fast with a clear error and link to GCP Console. Also adds
a defensive check in destroyInstance() to reject empty project.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
The CI biome check only covered packages/cli/src/, .claude/scripts/,
and .claude/skills/setup-spa/ — packages/shared/src/ was unchecked,
allowing 7 lint/format violations to accumulate in its test files.
- Auto-fix import ordering, formatting, and useNumberNamespace lint
across 3 test files in packages/shared/src/__tests__/
- Add packages/shared/src/ to the biome check in lint.yml so future
violations are caught in CI
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The min-size check only triggered when the exact default slug was
selected (s-2vcpu-2gb). Users who chose s-1vcpu-1gb or s-1vcpu-2gb
bypassed the check and got OOM crashes on openclaw.
Now parses RAM from the DO slug and compares GB values, so any size
below the agent's minimum gets upgraded.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
If the end marker (# <<< spawn <<<) is missing from .bashrc/.zshrc,
cleanRcFile dropped all content after the start marker. Now detects
unclosed blocks and skips the file with a warning instead of writing
a truncated version.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
getSshFingerprint called Bun.spawnSync without error handling, crashing
the CLI if ssh-keygen is not in PATH. Wrapped with unwrapOr(tryCatch())
to return empty string on failure, matching getKeyType's pattern.
Also added empty fingerprint handling to Hetzner SSH key registration
(matching DigitalOcean's existing pattern) to skip keys that can't be
fingerprinted instead of attempting re-registration.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
* fix: validate manifest fields are plain objects, not just truthy
isValidManifest used !!data.agents/clouds/matrix which accepts strings,
numbers, and arrays. Downstream Object.keys() then silently returns
character indices or array indices instead of real agent/cloud names.
Replace with isPlainObject() checks to reject non-object values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add validation tests for non-object manifest fields
Tests that loadManifest rejects manifests where agents/clouds/matrix
are strings, arrays, or numbers instead of plain objects.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Two bugs in acquireLock:
1. PID write failure was ignored — process returned success but left a
lock dir without a PID file. If it crashed, no other process could
detect the lock as stale, making it permanent.
2. Lock dirs without PID files were not treated as stale — other
processes waited until timeout instead of cleaning up immediately.
Fix: retry on PID write failure (clean up dir first), and treat
lock dirs without PID files as broken/stale (force remove).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add sudo to tarball mirror commands for non-root SSH users
The mirror step copies files from /root/ to $HOME/ for non-root users
(e.g. ubuntu on AWS Lightsail), but cp and chown ran without sudo.
A non-root user can't read /root/ or chown root-owned files, so the
mirror silently failed (errors suppressed by 2>/dev/null || true).
Adds sudo to cp/chown in both mirror blocks (tryTarballInstall and
uploadAndExtractTarball) and removes error suppression so failures
propagate to the caller.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: verify sudo in tarball mirror commands for both install paths
Adds tests for tryTarballInstall and uploadAndExtractTarball that assert:
- cp and chown use sudo (needed to read /root/ as non-root user)
- error suppression (2>/dev/null || true) is not present
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Signed-off-by: Ahmed Abushagur <ahmed@abushagur.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update agent GitHub star counts
* chore: update agent GitHub star counts
* chore: update agent GitHub star counts
* chore: update agent GitHub star counts
* chore: update agent GitHub star counts
* fix(install): force IPv4 DNS for npm installs and add junie binary verify
On Sprite VMs (and potentially other clouds with flaky IPv6 routing), npm
install of packages with native-binary postinstall scripts (kilocode, junie)
fails with i/o timeout when connecting to the npm registry over IPv6.
Changes:
- Add NODE_OPTIONS=--dns-result-order=ipv4first to NPM_PREFIX_SETUP so all
npm installs prefer IPv4, preventing the IPv6 timeout on first attempt
- Add cd ~ before postinstall re-run in KILOCODE_BINARY_VERIFY to avoid
"current working directory was deleted" errors in bun/node on retry
- Add JUNIE_BINARY_VERIFY snippet (analogous to kilocode) that detects and
recovers from a failed junie postinstall by re-running it from $HOME
- Apply JUNIE_BINARY_VERIFY to the junie install command
Fixes sprite kilocode and junie failures seen in E2E run 2026-03-23.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
When --output json is requested, the auto-update install script was
running with stdio: "inherit", causing [spawn] install messages to
pollute stdout before the JSON result, breaking JSON consumers.
Fix:
- Pre-scan process.argv for --output json before checkForUpdates()
is called in index.ts (formal flag parsing happens later at line 944)
- Pass jsonOutput flag through checkForUpdates() -> performAutoUpdate()
- When jsonOutput=true, use stdio: ["pipe", stderr, stderr] for the
install script execution so all output goes to stderr only
- Set SPAWN_CLI_UPDATED=1 env var on re-exec so JSON consumers can
detect the update via cli_updated: true in SpawnResult
- Add cli_updated?: boolean to SpawnResult interface in commands/run.ts
- Add tests covering both json and non-json stdio behavior
Fixes#2918
Agent: issue-fixer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
