- Add `hermes` to ALL_AGENTS in sh/e2e/lib/common.sh (stale: hermes added to
manifest.json in #2023 but never added to the e2e agent list)
- Add verify_hermes() and input_test_hermes() to sh/e2e/lib/verify.sh and
wire them into verify_agent/run_input_test dispatch tables
- Remove dead log_warn() from sh/shared/github-auth.sh (defined but never called)
- Remove dead get_cloud_env_vars() from sh/shared/key-request.sh (no callers outside file)
- Remove dead invalidate_cloud_key() from sh/shared/key-request.sh (no callers anywhere)
Co-authored-by: spawn-qa-bot <qa@openrouter.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
E2E tests were failing because agent installs didn't complete within
the default 120s timeout, and small VMs ran out of memory during builds.
- INSTALL_WAIT: 120s → 300s (with per-cloud override via cloud_install_wait)
- AWS: nano_3_0 → medium_3_0 (all agents need 4GB for reliable installs)
- DigitalOcean: s-1vcpu-512mb-10gb → s-2vcpu-2gb, cap at 3 parallel
- GCP: e2-medium → e2-standard-2
- Hetzner: cap at 5 parallel (primary IP limit)
- Sprite: 300s install wait (slower exec than SSH)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
When multiple clouds run in parallel, they generate the same app name
(e.g. e2e-claude-TIMESTAMP) and write to the same temp files
(.exit/.stdout/.stderr), causing data corruption.
- Include ACTIVE_CLOUD in make_app_name: e2e-gcp-claude-TIMESTAMP
- Use ${app_name} instead of ${agent} for provision temp files
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
* feat(e2e): multi-cloud test suite with cloud driver pattern
Scale the E2E test suite from AWS-only to all 6 infrastructure clouds
(aws, hetzner, digitalocean, gcp, daytona, sprite) with parallel
execution support.
Architecture:
- Cloud driver pattern: each cloud implements _cloudname_func() functions
- load_cloud_driver() wires cloud-specific functions to generic names
(cloud_exec, cloud_teardown, etc.)
- Shared orchestration stays in one place, cloud details are isolated
New files:
- sh/e2e/e2e.sh — unified entry point with --cloud flag
- sh/e2e/lib/clouds/{aws,hetzner,digitalocean,gcp,daytona,sprite}.sh
Refactored:
- common.sh — removed AWS constants, added load_cloud_driver()
- provision.sh — cloud-agnostic via cloud_headless_env/cloud_provision_verify
- verify.sh — replaced aws_ssh with cloud_exec/cloud_exec_long
- teardown.sh/cleanup.sh — delegate to cloud driver functions
- aws-e2e.sh — thin wrapper: exec e2e.sh --cloud aws
Usage:
e2e.sh --cloud aws # Single cloud
e2e.sh --cloud aws --cloud hetzner # Multiple clouds in parallel
e2e.sh --cloud all --parallel 3 # All clouds, 3 agents parallel
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(e2e): prevent subshell EXIT trap inheritance and single-cloud early exit
- Reset EXIT trap in multi-cloud subshells to prevent LOG_DIR deletion
before the main process reads log files
- Use `|| true` for single-cloud run_agents_for_cloud to prevent set -e
from skipping the summary on env validation failure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: default to parallel agent provisioning in e2e tests
All agents within a cloud now run in parallel by default instead of
sequentially. Use --sequential to restore the old behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: cap sprite parallelism, 4GB for openclaw, remove stderr suppression
- Sprite: add _sprite_max_parallel (cap 2 concurrent agents) to avoid
CLI rate limiting that caused all 6 agents to fail
- AWS: use medium_3_0 (4GB) bundle for openclaw which needs more RAM
- Input tests: remove 2>/dev/null from agent commands so failures
produce visible error output instead of empty responses
- Add cloud_max_parallel to driver interface, respected by e2e.sh
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use bash instead of sh for exec_long across all cloud drivers
Ubuntu's /bin/sh is dash, which doesn't support bash-specific PATH
sourcing from .spawnrc/.cargo/env. This caused codex and zeroclaw
input tests to fail with "command not found" even though verify passed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: codex input test uses positional prompt, not -q flag
codex CLI takes prompt as positional arg: `codex "PROMPT"`.
The -q flag doesn't exist, causing "Usage:" error output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use codex exec -q for non-interactive input test
codex requires `exec` subcommand for non-interactive mode.
Plain `codex PROMPT` expects a TTY (stdin is not a terminal).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: codex exec takes no -q flag, just positional prompt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use cx23 instead of deprecated cx22 for Hetzner e2e tests
Hetzner deprecated server type cx22 (ID 104). The default now uses cx23.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
* feat!: remove Fly.io cloud provider support
Drop Fly.io as a supported cloud provider. Sprite (which uses Fly.io
infrastructure internally) is retained.
- Delete packages/cli/src/fly/ module, sh/fly/ scripts, fixtures/fly/
- Remove fly cloud entry and 6 fly matrix entries from manifest.json
- Remove fly imports, destroy cases, and connection handlers from commands.ts
- Remove fly-ssh sentinel from security.ts
- Port E2E test suite from Fly.io to AWS Lightsail (fly-e2e.sh → aws-e2e.sh)
- Update README (7 clouds, 42 combinations), CLAUDE.md, and skill prompts
- Clean up fly references in build config, gitignore, icon sources
- Bump CLI version to 0.11.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: restore Docker image build under sh/docker/
Move openclaw Dockerfile from sh/fly/docker/ to sh/docker/ and rename
workflow from fly-docker.yml to docker.yml with updated paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: fix extra blank lines in commands.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: spawn-bot <spawn-bot@openrouter.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: L <6723574+louisgv@users.noreply.github.com>
- MEDIUM: Validate flyctl auth status before empty FLY_API_TOKEN fallback
in provision.sh (fail fast instead of silent failure)
- LOW: Fix sed -i portability in qa.sh (use sed -i.bak for macOS compat)
- LOW: Increase FLY_API_TOKEN expiry from 2h to 8h in common.sh
- LOW: Add --proto '=https' to all curl -L calls in digitalocean scripts
(6 files) to prevent HTTP downgrade on redirects
Fixes#1913
Agent: code-health
Co-authored-by: B <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat: add live input/output E2E verification for agents
The E2E suite previously only verified static artifacts (binaries, config
files, env vars). An agent with a broken API key or crash-on-launch bug
would pass all checks. This adds an input test phase that sends a real
prompt to each agent and verifies the response contains a marker string.
- Add fly_ssh_long() with configurable timeout for long-running commands
- Add per-agent input test functions (claude -p, codex -q, openclaw -p,
zeroclaw agent -p; opencode/kilocode skip as TUI-only)
- Add run_input_test() dispatcher with SKIP_INPUT_TEST env var support
- Add --skip-input-test CLI flag to fly-e2e.sh
- Chain input test after verify in run_single_agent() pipeline
- Add INPUT_TEST_TIMEOUT constant (default 120s, env-overridable)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: format p.text({ message }) to multi-line for biome
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: spawn-bot <spawn-bot@openrouter.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Reorganizes the project so all shell scripts live under a dedicated
/sh directory, enabling the OpenRouter rewrite URL to point at /sh/
instead of the repository root.
Moves:
- cli/install.sh → sh/cli/install.sh
- shared/*.sh → sh/shared/*.sh
- {cloud}/{agent}.sh → sh/{cloud}/{agent}.sh (48 scripts)
- {cloud}/README.md → sh/{cloud}/README.md
- e2e/*.sh → sh/e2e/*.sh
- test/macos-compat.sh → sh/test/macos-compat.sh
- test/fixtures/**/*.sh → sh/test/fixtures/**/*.sh
Updates all references:
- RAW_BASE path construction in commands.ts, update-check.ts
- GitHub auth URL in agent-setup.ts
- Self-referencing URLs in install.sh, github-auth.sh
- CI workflow paths in lint.yml, cli-release.yml
- Test file paths in install-script-validation, manifest-integrity
- Documentation in README.md, cli/README.md, CLAUDE.md
- QA scripts in .claude/skills/
Co-authored-by: lab <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
