- Replace `-H "Authorization: Bearer ..."` curl args with temp curl config
files (`-K`) in digitalocean.sh and hetzner.sh e2e drivers, keeping API
tokens out of `ps` output
- Replace dangerous-var blocklist in provision.sh with a positive whitelist
of allowed cloud_headless_env variable names
Agent: complexity-hunter
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
All four SSH-based cloud drivers (aws, digitalocean, gcp, hetzner)
passed the command string directly as an SSH argument, which gets
interpreted by the remote shell. While current callers pass trusted
E2E test code, this creates a security footgun for future changes.
Fix: base64-encode the command locally and decode it on the remote
side before piping to bash. The encoded string contains only safe
characters [A-Za-z0-9+/=], eliminating any injection vector. Stdin
is preserved for callers that pipe data into cloud_exec.
Closes#2432, closes#2433, closes#2434, closes#2435
Agent: complexity-hunter
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
- Replace word-split _sprite_org_flags() call sites with _sprite_cmd()
helper that uses a proper bash array for the -o flag, eliminating
injection risk from org names with spaces or shell metacharacters
- Validate _SPRITE_ORG against [A-Za-z0-9_-]+ in _sprite_validate_env
- Use grep -qF (fixed-string) instead of grep -q for app name matching
to prevent regex metacharacters in names from causing false matches
- Use mktemp for _stderr_tmp in _sprite_exec instead of predictable
PID-based path (/tmp/sprite-exec-err.$$) to prevent symlink attacks
Closes#2436
Agent: complexity-hunter
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
- Validate app_name at function entry (alphanumeric, dots, hyphens, underscores
only) before it's used in file paths or passed to cloud_exec
- Add trap-based cleanup for the temp file used during .spawnrc fallback creation
- Add security comments documenting the three-layer defense model: printf %q
quoting, base64 encoding, and stdin piping (no interpolation into command
strings)
The core vulnerability (env_b64 interpolated into the cloud_exec command string)
was already fixed in a prior commit that switched to stdin piping. This change
adds defense-in-depth and documentation.
Fixes#2437, #2441
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
install.sh: Replace color variable interpolation in printf format strings
with %b arguments to prevent format string injection (fixes#2443).
common.sh: Use %b for color escapes in logging functions. Document that
BASH_SOURCE and source usage in load_cloud_driver is intentional since
e2e scripts are filesystem-only, not curl|bash (fixes#2438).
Agent: ux-engineer
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Add defense-in-depth validation across all e2e cloud driver scripts:
- Validate IP addresses match IPv4 format before use in SSH commands
(aws, digitalocean, gcp, hetzner)
- Validate SSH username contains only safe characters (gcp)
- Validate resource IDs are numeric before interpolating into API URLs
(digitalocean droplet IDs, hetzner server IDs)
- URL-encode app name in Hetzner API query parameter to prevent
query parameter injection
- Validate numeric env vars (INPUT_TEST_TIMEOUT, PROVISION_TIMEOUT,
INSTALL_WAIT) that get interpolated into remote command strings
Fixes#2432, #2433, #2434, #2435, #2442
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* security: escape pkill regex metacharacters in app_name
Fixes#2409 - escape regex metacharacters (., [, \, *, ^, $) in
app_name before using in pkill -f pattern to prevent unintended
process termination. Even though app_name is validated against a
safe character whitelist, . and - are regex metacharacters that
could match broader patterns than intended.
Note: #2410 (unquoted regex in bash conditional) was already fixed
by a prior commit that refactored the code to use sed instead of
[[ =~ BASH_REMATCH ]].
Agent: security-auditor
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: remove dead exec_long functions reintroduced from pre-#2407 code
Remove cloud_exec_long dispatcher and all _*_exec_long() functions
from common.sh and cloud driver files (aws, digitalocean, gcp,
hetzner, sprite). These were explicitly removed as dead code in
PR #2407 (commit c4ae1684) and must not be reintroduced.
Issue #2410 (unquoted regex in bash conditional) is already resolved:
the [[ =~ ]] pattern was previously replaced with case/sed parsing.
Fixes#2409Fixes#2410
Agent: pr-maintainer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
The cloud_exec_long dispatcher in common.sh and all five cloud-specific
_exec_long implementations (aws, digitalocean, gcp, hetzner, sprite)
were defined but never called by any code in the e2e test suite.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
After SSH reconnect, agent commands (openclaw, codex, kilocode, junie) were
not found because PATH was only written to ~/.bashrc, which is not sourced
by login shells. Login shells (used by SSH) source ~/.profile or
~/.bash_profile instead.
Changes:
- Write .spawnrc sourcing to ~/.profile and ~/.bash_profile in addition
to ~/.bashrc and ~/.zshrc (orchestrate.ts)
- Write npm-global PATH export to ~/.profile and ~/.bash_profile for all
npm-installed agents: OpenClaw, Codex, Kilo Code, Junie (agent-setup.ts)
- Write Claude Code PATH to ~/.profile and ~/.bash_profile (agent-setup.ts)
- Write OpenCode PATH to ~/.profile and ~/.bash_profile (agent-setup.ts)
- Extract NPM_GLOBAL_PATH_PERSIST constant to DRY up repeated shell snippets
- Fix e2e provision.sh to also write .spawnrc sourcing to login shell configs
- Bump CLI version to 0.15.32
Fixes#2394
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Replaces ${cfg}.fix$$ temp pattern with mktemp for guaranteed uniqueness.
Both temp file usages in the function are updated.
Fixes#2354
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds defense-in-depth check to reject malformed base64 output
before it is embedded in the cloud_exec remote command.
Fixes#2353
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove intermediate $env_b64 shell variable that stored base64-encoded
credentials. Pipe directly from base64 to cloud_exec, preventing any
credential data from appearing in process listings or shell traces.
Fixes#2333
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
- Block dangerous system env vars (PATH, LD_PRELOAD, etc.) before export
- Add explicit alphanumeric validation on env var names
- Validate app_name is non-empty and safe before pkill -f
- Tighten pkill regex from "sprite.*exec.*" to "sprite exec.*"
Fixes#2330#2332
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Pipe the command via stdin to bash instead of embedding it in a bash -c
string. This eliminates shell injection risk from unquoted cmd parameter,
consistent with _sprite_exec_long in the same file and other cloud drivers.
Fixes#2327
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
The junie agent was added in #2300 but the E2E test scripts were not
updated. This adds junie to ALL_AGENTS, verify dispatch, input test
dispatch, and the provision.sh fallback env configuration.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Three distinct E2E bugs fixed:
1. SSH key generation race condition: When multiple agents provision in
parallel, concurrent processes all call generateSshKey() and race to
create ~/.ssh/id_ed25519. ssh-keygen won't overwrite an existing file
(prompts on stdin which is "ignore"), causing zeroclaw/codex to fail
with "SSH key generation failed". Fix: check if key already exists
before generating, and re-check after a failed generation attempt.
2. Hetzner SSH key 409 uniqueness_error: The Hetzner API returns HTTP 409
with "SSH key not unique" when the same key content is registered under
a different name. The hetznerApi() function throws on non-2xx before
the error-parsing code runs, and the regex /already/ didn't match
"not unique". Fix: catch 409 in ensureSshKey() and match against
uniqueness_error/not unique/already patterns.
3. Hermes binary not found: The hermes install script (uv tool) creates
the actual binary + venv at ~/.hermes/hermes-agent/venv/ with a symlink
at ~/.local/bin/hermes. The tarball capture script only captured the
symlink + ~/.local/share/, leaving a dangling symlink. Fix: include
~/.hermes/ in capture paths, add venv/bin to verify.sh PATH check,
and update hermes launchCmd to include the venv PATH.
Fixes#2304
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
After every e2e run, send an HTML matrix report to KEY_REQUEST_EMAIL
via Resend showing pass/fail/skip per agent x cloud combination.
- e2e.sh: add send_matrix_email() — builds result table from LOG_DIR
result files, writes temp TS, calls bun run to POST to Resend API.
Called just before exit so LOG_DIR is still available.
- qa.sh (e2e mode): load RESEND_API_KEY + KEY_REQUEST_EMAIL from
/etc/spawn-key-server-auth.env before launching Claude so the creds
are inherited by the e2e.sh subprocess.
Both changes are no-ops when credentials are absent (silent skip).
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes#2292
Unanchored grep -q would match the marker anywhere in output, including
error messages like "Expected SPAWN_E2E_OK but got...". Using grep -qx
requires the marker to appear as a complete line, preventing false passes.
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace unsafe pattern where base64-encoded commands were interpolated
into remote command strings with secure stdin piping — command data now
travels as stdin rather than as part of the command string, eliminating
injection risk from shell metacharacter interpretation.
Affected functions across all 5 cloud drivers:
- _hetzner_exec_long
- _aws_exec_long
- _gcp_exec_long
- _digitalocean_exec_long
- _sprite_exec_long
Fixes#2286Fixes#2287
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: replace base64 interpolation with stdin piping in verify.sh (Fixes#2283)
Replace unsafe pattern where encoded prompt was interpolated into remote
command strings with secure stdin piping — prompt data now travels as stdin
rather than as part of the command string, eliminating injection risk.
Affected functions: input_test_claude, input_test_codex, input_test_openclaw,
input_test_zeroclaw.
Agent: security-auditor
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: use cloud_exec (not cloud_exec_long) for stdin piping
cloud_exec_long ignores stdin - remote base64 -d would hang.
cloud_exec passes cmd to bash -c, which preserves stdin piping.
Agent: pr-maintainer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: restore timeout protection for input tests using cloud_exec
Wraps each agent command in `timeout ${INPUT_TEST_TIMEOUT}` on the remote
side so tests cannot hang indefinitely after switching from cloud_exec_long
to cloud_exec. Updates stale comment referencing cloud_exec_long.
Agent: pr-maintainer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Simplify the cloud matrix by removing Daytona. All Daytona-specific code,
scripts, tests, and configuration have been removed. Daytona has been moved
to "Previously Considered" in the Cloud Provider Wishlist (#1183) and can
be revived on community demand.
Closes#2260
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: tarball workflow failures (root ownership, swapfile, hermes TTY)
- Use sudo mv + chown for tarball in release step (root-owned from capture)
- Skip swapfile creation if /swapfile already exists (GitHub Actions runners)
- Tolerate hermes setup wizard failure when /dev/tty unavailable in CI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: capture claude symlink target in tarball + fix verify PATH
The claude installer creates a symlink at ~/.local/bin/claude pointing
to ~/.local/share/claude/versions/X.Y.Z. The capture script was missing
~/.local/share/claude/, causing a broken symlink in the tarball.
Also add ~/.npm-global/bin to the verify PATH check for claude (npm
fallback install path).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
- Remove sh/e2e/aws-e2e.sh: dead backwards-compat wrapper with no
references (superseded by unified e2e.sh --cloud aws)
- Remove getStatusDescription from commands/shared.ts: defined and
tested but never called in production code
- Remove parseJsonRaw from packages/cli/src/shared/parse.ts: zero
production usages (still available in packages/shared if needed)
- Update corresponding test files to remove dead code tests
- Bump CLI version to 0.14.4
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- manifest.json: change aws auth to AWS_ACCESS_KEY_ID+AWS_SECRET_ACCESS_KEY
so the key-request system includes AWS in its missing-key emails
- sh/e2e/e2e.sh: clouds missing credentials now SKIP (not FAIL), so
running --cloud all is safe and only tests what's configured
- qa.sh: include e2e mode in cloud credential loading (was fixtures+quality only)
- qa-quality-prompt.md: e2e-tester now runs e2e.sh --cloud all --parallel 6 --skip-input-test
- qa-e2e-prompt.md: standalone e2e bot now runs e2e.sh --cloud all --parallel 6
Also wires KEY_SERVER_URL + KEY_SERVER_SECRET into /etc/spawn-qa-auth.env
(system change, not in this commit) so missing-key emails are actually sent.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Ahmed Abushagur <ahmed@abushagur.com>
* test(e2e): add openclaw gateway kill/restart resilience test
Verifies that the openclaw gateway auto-restarts after being killed
with SIGKILL, validating the systemd Restart=always supervision.
The test runs as part of verify_openclaw:
1. Confirms gateway is listening on :18789
2. Kills it with SIGKILL (simulates a hard crash)
3. Waits up to 30s for systemd to auto-restart it
4. Verifies port 18789 comes back online
If the gateway isn't running (e.g. non-systemd env), the test is
skipped gracefully. On failure, dumps systemd status and gateway
logs for diagnostics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "test(e2e): add openclaw gateway kill/restart resilience test"
This reverts commit 39b79d5c12.
* test: add unit tests for openclaw gateway resilience config
Verifies that startGateway() produces correct systemd and cron
configuration for auto-restart after a gateway crash:
- Restart=always and RestartSec=5 in the systemd unit
- Cron heartbeat checks port 18789 and restarts if dead
- Wrapper script sources .spawnrc and execs openclaw gateway
- Multiple port-check fallbacks (ss, /dev/tcp, nc)
- Non-systemd fallback to setsid/nohup
- 300s startup timeout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(e2e): add openclaw gateway kill/restart resilience test
Kills the gateway with SIGKILL during verify_openclaw and verifies
systemd Restart=always brings it back within 30s. Skips gracefully
on non-systemd environments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
1. promptSpawnName() now checks DO_DROPLET_NAME before generating a
random name, matching getServerName() behavior. This fixes the e2e
harness creating droplets as spawn-XXXX when it expects
e2e-digitalocean-AGENT-TIMESTAMP.
2. Replace BASH_REMATCH with sed-based parsing in provision.sh for
macOS bash 3.2 compatibility. BASH_REMATCH was returning empty
values, causing `export: '=': not a valid identifier`.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
* fix(security): validate env values in cloud_headless_env parser
Reject values containing shell metacharacters ($, backtick, ;, &, |, <, >)
to prevent potential command injection if a cloud driver returns malicious output.
Fixes#2139
Agent: security-auditor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): replace env value blacklist with whitelist regex
The blacklist approach missed dangerous characters like (), quotes,
backslash, newlines, {}, and !. Switch to a whitelist that only allows
[A-Za-z0-9@%+=:,./_-] — a strict safe set sufficient for env values.
Agent: security-auditor
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The fallback .spawnrc construction (used when provision times out before
.spawnrc is written) had two bugs:
1. zeroclaw case wrongly included OPENAI_API_KEY and OPENAI_BASE_URL —
these are hermes env vars, not zeroclaw's. zeroclaw only needs
ZEROCLAW_PROVIDER=openrouter (plus the base OPENROUTER_API_KEY).
2. hermes and kilocode were missing from the case statement entirely.
- hermes needs OPENAI_BASE_URL and OPENAI_API_KEY (verify_hermes
checks for OPENAI_BASE_URL in .spawnrc)
- kilocode needs KILO_PROVIDER_TYPE=openrouter and
KILO_OPEN_ROUTER_API_KEY (verify_kilocode checks KILO_PROVIDER_TYPE)
Without these fixes, hermes and kilocode would fail verification whenever
provisioning timed out before the normal .spawnrc was written.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove `cleanup_stale_apps()` in `sh/e2e/lib/cleanup.sh` which was dead
code — defined but never called. The E2E orchestrator (`e2e.sh`) invokes
`cloud_cleanup_stale` directly on the active cloud driver; the wrapper
function and its file served no purpose.
Also remove the corresponding `source` call in `e2e.sh`.
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
* fix: address 4 reliability issues across codebase
1. sprite.ts: add --force to destroy command (stdin is "ignore" so
interactive prompts would hang until 60s timeout)
2. verify.sh: replace /dev/tcp port checks with ss -tln primary
(Debian/Ubuntu bash compiled without /dev/tcp support)
3. verify.sh: make _openclaw_restart_gateway a hard failure instead
of log_warn (matching _openclaw_ensure_gateway behavior)
4. agent-setup.ts: add ss -tln port check + "already running" early
exit + increase timeout from 120s to 300s (gateway takes ~3min
to initialize on AWS medium instances)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: biome format - use consistent double quotes in portCheck
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
* fix(e2e): improve openclaw reliability on AWS and other clouds
Three changes to make openclaw e2e tests more robust:
1. Increase PROVISION_TIMEOUT from 480s to 720s — AWS cloud-init
for "full" tier (Node.js + Bun + build-essential) can exceed 480s,
causing the CLI to be killed before .spawnrc is written.
2. Add .spawnrc manual fallback in provision.sh — if the CLI is killed
before writing .spawnrc, construct it via SSH using OPENROUTER_API_KEY
with agent-specific env vars (openclaw, zeroclaw).
3. Add retry logic to openclaw gateway input test — the gateway can
crash with 1006 websocket closure on resource-constrained instances.
Now retries once after killing and restarting the gateway process.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): fix command injection in e2e provision scripts
- Use printf %q and temp file for api_key handling in provision.sh to
prevent shell metachar injection (single quotes, backticks, $)
- Double-quote env_b64 interpolation in cloud_exec call to prevent
word splitting
- Replace echo with printf in bashrc append to avoid portability issues
- Replace overbroad pkill -f 'openclaw gateway' in verify.sh with
PID-targeted kill via lsof/fuser on port 18789
Agent: pr-maintainer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Removes eval-based function creation pattern in e2e/lib/common.sh.
Uses variable indirection (ACTIVE_CLOUD global + wrapper functions)
instead of eval to reduce attack surface.
Fixes#2118
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes unquoted ${timeout} in _sprite_exec_long that could allow
command injection if timeout contained shell metacharacters.
Adds numeric validation before use.
Fixes#2117
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
The outer cloud_exec_long already enforces a timeout via
INPUT_TEST_TIMEOUT. The inner --timeout 60 was redundant and could
cause premature kills before the outer timeout expired.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: A <258483684+la14-1@users.noreply.github.com>
The gateway startup was silently swallowed with log_warn, masking
real failures. Now tracks whether the port came up and fails the
test with the gateway log contents if it didn't.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Without --force, sprite destroy prompts for confirmation in
non-interactive E2E mode and silently fails ("Ok, come back later!"),
leaving stale instances running indefinitely.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Add hermes shim scripts for GCP, Hetzner, DigitalOcean, and Daytona
- Update manifest.json matrix entries from "missing" to "implemented"
- Bump default INSTALL_WAIT from 300s to 600s to fix zeroclaw timeout
on small VMs where Rust compilation takes 8-12 minutes
- Update cloud READMEs with hermes usage docs
- Bump CLI version to 0.11.18
Co-authored-by: Ahmed Abushagur <ahmed@abushagur.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
PR #2064 fixed _exec_long shell injection for DigitalOcean and Sprite
but missed the same bash -c '${cmd}' pattern in Hetzner, GCP, AWS, and
Daytona. Apply the same base64-encoding fix to all four.
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Base64-encode the command before embedding it in bash -c to prevent
single-quote breakout in _sprite_exec_long and _digitalocean_exec_long.
Fixes#2063
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(sprite): fix all 6 Sprite agent installs for E2E
- Use `npm install -g --prefix` instead of `npm config set prefix` to
avoid creating .npmrc that conflicts with nvm on Sprite VMs
- Fix shell environment setup to only modify .bash_profile (not .bashrc)
so non-interactive bash -c commands retain PATH config
- Add $HOME/.cargo/bin to PATH for zeroclaw (Sprite has no ~/.cargo/env)
- Add $HOME/.local/bin to PATH config for Sprite shell environment
- Add sprite E2E cloud driver with org detection, config corruption fix,
direct command embedding (not $1 positional), and retry logic
- Fix provision.sh to kill full process tree after timeout (prevents
orphaned sprite exec sessions from corrupting config)
- Fix verify.sh zeroclaw check to not rely on ~/.cargo/env existing
Tested: 6/6 Sprite agents pass E2E (claude, codex, openclaw, zeroclaw,
opencode, kilocode). Hermes is not in the Sprite manifest.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: biome format - collapse runSprite call to single line
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Non-interactive SSH sessions don't source .bashrc or .zshrc, so binaries
installed to ~/.local/bin (hermes via uv) or ~/.npm-global/bin (codex,
kilocode via npm) were not found during verification.
Fix all three verify functions and the codex input test to use explicit
PATH with the known install directories, matching the pattern already
used by openclaw and claude.
Verified: AWS 7/7, Hetzner 6/6 implemented, GCP 6/6 implemented.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The teardown was doing a single DELETE without --max-time, so connection
timeouts caused HTTP 000 and the droplet was never deleted. When running
6 agents in batches of 3, batch 1's stale droplet caused batch 2 to fail
with "will exceed your droplet limit."
Fix:
- Add --max-time 30 to prevent curl hangs
- Retry DELETE up to 3 times on failure
- Poll the API after DELETE to confirm the droplet is actually gone (up to 60s)
- Remove -f flag from curl so %{http_code} is always captured
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The openclaw e2e input test was failing for three independent reasons:
1. PATH missing ~/.npm-global/bin — openclaw installs via npm with a
custom prefix, but verify_openclaw and input_test_openclaw didn't
include that directory in PATH
2. Wrong CLI invocation — used `openclaw -p` which doesn't exist.
The correct command is `openclaw agent --message "..." --session-id`
3. Gateway not running — the openclaw gateway (port 18789) can die
between provisioning and verification. Now the input test ensures
the gateway is running before sending the prompt.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
- Add `hermes` to ALL_AGENTS in sh/e2e/lib/common.sh (stale: hermes added to
manifest.json in #2023 but never added to the e2e agent list)
- Add verify_hermes() and input_test_hermes() to sh/e2e/lib/verify.sh and
wire them into verify_agent/run_input_test dispatch tables
- Remove dead log_warn() from sh/shared/github-auth.sh (defined but never called)
- Remove dead get_cloud_env_vars() from sh/shared/key-request.sh (no callers outside file)
- Remove dead invalidate_cloud_key() from sh/shared/key-request.sh (no callers anywhere)
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
Replace eval "${cloud_env}" with a while-read loop that only
processes lines matching ^export VAR="VALUE" — arbitrary shell
commands in cloud driver output are silently ignored.
Also removes the now-unused cloud_env local variable since the
while-read loop calls cloud_headless_env directly.
Fixes#2019
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace bash -c "${cmd}" with bash -c '$1' _ "${cmd}" so the
command is passed as a positional argument, not interpolated into
the shell string. Same pattern applied to the timeout wrapper.
Fixes#2018
Agent: security-auditor
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: SPAWN_CLI_DIR env var to force local source in e2e and shell scripts
When SPAWN_CLI_DIR is set, the entire toolchain uses local TypeScript
source instead of downloading pre-bundled scripts from GitHub releases:
- e2e.sh: auto-sets SPAWN_CLI_DIR to repo root when running locally
- provision.sh: exports SPAWN_CLI_DIR into the headless subshell
- commands.ts: reads local shell scripts instead of fetching from CDN
- All 36 cloud/agent shell scripts: exec local main.ts when set
This enables e2e tests to validate local changes before they're released.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): add path traversal defense to SPAWN_CLI_DIR script loading
Canonicalize the path via realpathSync and verify it stays inside the
resolved CLI directory before reading. Prevents SPAWN_CLI_DIR from
being used to read arbitrary files via ../ traversal.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): harden SPAWN_CLI_DIR path traversal defense
- Validate cloud/agent names don't contain '..', '/' or '\' before
constructing file paths
- Fix root-directory edge case in prefix check by handling trailing
separator correctly
Agent: pr-maintainer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
E2E tests were failing because agent installs didn't complete within
the default 120s timeout, and small VMs ran out of memory during builds.
- INSTALL_WAIT: 120s → 300s (with per-cloud override via cloud_install_wait)
- AWS: nano_3_0 → medium_3_0 (all agents need 4GB for reliable installs)
- DigitalOcean: s-1vcpu-512mb-10gb → s-2vcpu-2gb, cap at 3 parallel
- GCP: e2-medium → e2-standard-2
- Hetzner: cap at 5 parallel (primary IP limit)
- Sprite: 300s install wait (slower exec than SSH)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
When multiple clouds run in parallel, they generate the same app name
(e.g. e2e-claude-TIMESTAMP) and write to the same temp files
(.exit/.stdout/.stderr), causing data corruption.
- Include ACTIVE_CLOUD in make_app_name: e2e-gcp-claude-TIMESTAMP
- Use ${app_name} instead of ${agent} for provision temp files
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
