refactor: move all shell scripts to /sh directory (#1843)

Reorganizes the project so all shell scripts live under a dedicated
/sh directory, enabling the OpenRouter rewrite URL to point at /sh/
instead of the repository root.

Moves:
- cli/install.sh → sh/cli/install.sh
- shared/*.sh → sh/shared/*.sh
- {cloud}/{agent}.sh → sh/{cloud}/{agent}.sh (48 scripts)
- {cloud}/README.md → sh/{cloud}/README.md
- e2e/*.sh → sh/e2e/*.sh
- test/macos-compat.sh → sh/test/macos-compat.sh
- test/fixtures/**/*.sh → sh/test/fixtures/**/*.sh

Updates all references:
- RAW_BASE path construction in commands.ts, update-check.ts
- GitHub auth URL in agent-setup.ts
- Self-referencing URLs in install.sh, github-auth.sh
- CI workflow paths in lint.yml, cli-release.yml
- Test file paths in install-script-validation, manifest-integrity
- Documentation in README.md, cli/README.md, CLAUDE.md
- QA scripts in .claude/skills/

Co-authored-by: lab <6723574+louisgv@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
A 2026-02-23 21:14:54 -08:00 committed by GitHub
parent 8812f693c0
commit b84adfb74e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
88 changed files with 76 additions and 68 deletions

View file

@ -21,8 +21,8 @@ cd WORKTREE_BASE_PLACEHOLDER
```bash
cd REPO_ROOT_PLACEHOLDER
chmod +x e2e/fly-e2e.sh
./e2e/fly-e2e.sh --parallel 6
chmod +x sh/e2e/fly-e2e.sh
./sh/e2e/fly-e2e.sh --parallel 6
```
Capture the full output. Note which agents passed and which failed.
@ -43,7 +43,7 @@ For each failed agent, investigate the root cause. The failure categories are:
- Fly.io API auth issues
- Agent-specific install script changed upstream
3. Read the agent's provisioning code: `cli/src/fly/agents.ts` and `cli/src/shared/agent-setup.ts`
4. Read the E2E provision script: `e2e/lib/provision.sh`
4. Read the E2E provision script: `sh/e2e/lib/provision.sh`
### Verification failure (app exists but checks fail)
@ -54,7 +54,7 @@ For each failed agent, investigate the root cause. The failure categories are:
```
2. Check if the binary path changed — read the agent's install script in `cli/src/shared/agent-setup.ts`
3. Check if the env var names changed — read the agent's config in `manifest.json`
4. Update the verification checks in `e2e/lib/verify.sh` if they are stale
4. Update the verification checks in `sh/e2e/lib/verify.sh` if they are stale
### Timeout (provision took too long)
@ -65,17 +65,17 @@ For each failed agent, investigate the root cause. The failure categories are:
Make fixes in the worktree at WORKTREE_BASE_PLACEHOLDER. Fixes may be in:
- `e2e/lib/provision.sh` — env vars, timeouts, headless flags
- `e2e/lib/verify.sh` — binary paths, config file locations, env var checks
- `e2e/lib/common.sh` — API helpers, constants
- `e2e/lib/teardown.sh` — cleanup logic
- `e2e/lib/cleanup.sh` — stale app detection
- `sh/e2e/lib/provision.sh` — env vars, timeouts, headless flags
- `sh/e2e/lib/verify.sh` — binary paths, config file locations, env var checks
- `sh/e2e/lib/common.sh` — API helpers, constants
- `sh/e2e/lib/teardown.sh` — cleanup logic
- `sh/e2e/lib/cleanup.sh` — stale app detection
After fixing:
1. Run `bash -n` on every modified `.sh` file
2. Re-run the E2E suite for the failed agent(s) only to verify the fix:
```bash
./e2e/fly-e2e.sh AGENT_NAME
./sh/e2e/fly-e2e.sh AGENT_NAME
```
## Step 5 — Commit and PR

View file

@ -159,8 +159,8 @@ log "Pre-cycle cleanup done."
# --- Fixtures mode: load cloud credentials ---
if [[ "${RUN_MODE}" == "fixtures" ]]; then
if [[ -f "${REPO_ROOT}/shared/key-request.sh" ]]; then
source "${REPO_ROOT}/shared/key-request.sh"
if [[ -f "${REPO_ROOT}/sh/shared/key-request.sh" ]]; then
source "${REPO_ROOT}/sh/shared/key-request.sh"
load_cloud_keys_from_config
if [[ -n "${MISSING_KEY_PROVIDERS:-}" ]]; then
log "Missing keys for: ${MISSING_KEY_PROVIDERS}"
@ -172,7 +172,7 @@ if [[ "${RUN_MODE}" == "fixtures" ]]; then
log "All cloud keys available"
fi
else
log "shared/key-request.sh not found, skipping key preflight"
log "sh/shared/key-request.sh not found, skipping key preflight"
fi
fi

View file

@ -79,7 +79,7 @@ jobs:
--title "${name} bundle v${{ steps.version.outputs.version }}" \
--notes "Pre-built ${name} cloud provider bundle.
Downloaded by \`${name}/*.sh\` shims for \`bash <(curl ...)\` execution.
Downloaded by \`sh/${name}/*.sh\` shims for \`bash <(curl ...)\` execution.
**Built:** $(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--prerelease \

View file

@ -68,4 +68,4 @@ jobs:
# Warn-only for initial rollout — switch to blocking after burn-in
- name: Run macOS compat linter
run: bash test/macos-compat.sh --warn-only
run: bash sh/test/macos-compat.sh --warn-only

View file

@ -30,7 +30,7 @@ Look at `manifest.json` → `matrix` for any `"missing"` entry. To implement it:
- Find the **agent's** existing script on another cloud — it shows the install steps, config files, env vars, and launch command
- The agent scripts are thin bash wrappers that bootstrap bun and run the TypeScript CLI
- The script goes at `{cloud}/{agent}.sh`
- The script goes at `sh/{cloud}/{agent}.sh`
**OpenRouter injection is mandatory.** Every agent script MUST:
- Set `OPENROUTER_API_KEY` in the shell environment
@ -63,7 +63,7 @@ Steps to add one:
2. Add an entry to `manifest.json``clouds`
3. Add `"missing"` entries to the matrix for every existing agent
4. Implement at least 2-3 agent scripts to prove the lib works
5. Update the cloud's `README.md`
5. Update the cloud's `sh/{cloud}/README.md`
6. **Add test coverage** (mandatory) — add unit tests in `cli/src/__tests__/`
**DO NOT add GPU clouds** (CoreWeave, RunPod, etc.). Spawn agents call remote LLM APIs for inference — they need cheap CPU instances with SSH, not expensive GPU VMs.
@ -108,12 +108,20 @@ spawn/
src/commands.ts # All subcommands (interactive, list, run, etc.)
src/version.ts # Version constant
package.json # npm package (@openrouter/spawn)
install.sh # One-liner installer (bun → npm → auto-install bun)
shared/
github-auth.sh # Standalone GitHub CLI auth helper
key-request.sh # API key provisioning helpers (used by QA)
{cloud}/
{agent}.sh # Agent deployment scripts (thin bash → bun wrappers)
sh/
cli/
install.sh # One-liner installer (bun → npm → auto-install bun)
shared/
github-auth.sh # Standalone GitHub CLI auth helper
key-request.sh # API key provisioning helpers (used by QA)
e2e/
fly-e2e.sh # Fly.io E2E test suite
lib/*.sh # E2E helper libraries
test/
macos-compat.sh # macOS compatibility test script
{cloud}/
{agent}.sh # Agent deployment scripts (thin bash → bun wrappers)
README.md # Cloud-specific usage docs
.claude/skills/setup-agent-team/
trigger-server.ts # HTTP trigger server (concurrent runs, dedup)
discovery.sh # Discovery cycle script (fill gaps, scout new clouds/agents)
@ -144,17 +152,17 @@ Examples of files that should NOT be committed:
The only documentation files allowed in the repository are:
- `README.md` (user-facing)
- `CLAUDE.md` (contributor guide)
- Cloud-specific `README.md` files in `{cloud}/README.md`
- Cloud-specific `README.md` files in `sh/{cloud}/README.md`
If you need to create documentation during development, write it to `.docs/` and add `.docs/` to `.gitignore`.
### Architecture
All cloud provisioning and agent setup logic lives in TypeScript under `cli/src/`. Agent scripts (`{cloud}/{agent}.sh`) are thin bash wrappers that bootstrap bun and invoke the CLI.
All cloud provisioning and agent setup logic lives in TypeScript under `cli/src/`. Agent scripts (`sh/{cloud}/{agent}.sh`) are thin bash wrappers that bootstrap bun and invoke the CLI.
**`shared/github-auth.sh`** — Standalone GitHub CLI installer + OAuth login helper. Used by `cli/src/shared/agent-setup.ts` to set up `gh` on remote VMs.
**`sh/shared/github-auth.sh`** — Standalone GitHub CLI installer + OAuth login helper. Used by `cli/src/shared/agent-setup.ts` to set up `gh` on remote VMs.
**`shared/key-request.sh`** — API key provisioning helpers sourced by the QA harness (`qa.sh`) for loading cloud credentials from `~/.config/spawn/{cloud}.json`.
**`sh/shared/key-request.sh`** — API key provisioning helpers sourced by the QA harness (`qa.sh`) for loading cloud credentials from `~/.config/spawn/{cloud}.json`.
## Shell Script Rules
@ -176,8 +184,8 @@ macOS ships bash 3.2. All scripts MUST work on it:
### Conventions
- `#!/bin/bash` + `set -eo pipefail` (no `u` flag)
- Use `${VAR:-}` for all optional env var checks (`OPENROUTER_API_KEY`, cloud tokens, etc.)
- Remote fallback URL: `https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/{path}`
- All env vars documented in the cloud's README.md
- Remote fallback URL: `https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/{path}` (shell scripts are under `sh/`, e.g., `sh/{cloud}/{agent}.sh`)
- All env vars documented in the cloud's `sh/{cloud}/README.md`
### Use Bun + TypeScript for Inline Scripting — NEVER python/python3
When shell scripts need JSON processing, HTTP calls, crypto, or any non-trivial logic:
@ -395,7 +403,7 @@ Draft PRs that go stale (no updates for 1 week) will be auto-closed.
1. `bash -n {file}` syntax check on all modified scripts
2. `cd cli && bunx @biomejs/biome lint src/`**must pass with zero errors** on all modified TypeScript
3. Update `manifest.json` matrix status to `"implemented"`
4. Update the cloud's `README.md` with usage instructions
4. Update the cloud's `sh/{cloud}/README.md` with usage instructions
5. Commit with a descriptive message
## Filing Issues for Discovered Problems

View file

@ -8,7 +8,7 @@ Launch any AI agent on any cloud with a single command. Coding agents, research
**macOS / Linux — and Windows users inside a WSL2 terminal (Ubuntu, Debian, etc.):**
```bash
curl -fsSL https://openrouter.ai/labs/spawn/cli/install.sh | bash
curl -fsSL https://openrouter.ai/labs/spawn/install.sh | bash
```
**Windows PowerShell (outside WSL):**
@ -116,7 +116,7 @@ If spawn fails to install, try these steps:
```bash
curl -fsSL https://bun.sh/install | bash
source ~/.bashrc # or ~/.zshrc for zsh
curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/cli/install.sh | bash
curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/sh/cli/install.sh | bash
```
3. **PATH issues**: If `spawn` command not found after install
@ -160,7 +160,7 @@ If an agent fails to install or launch on a cloud:
## Matrix
| | [Local Machine](local/) | [Hetzner Cloud](hetzner/) | [Fly.io](fly/) | [AWS Lightsail](aws/) | [Daytona](daytona/) | [DigitalOcean](digitalocean/) | [GCP Compute Engine](gcp/) | [Sprite](sprite/) |
| | [Local Machine](sh/local/) | [Hetzner Cloud](sh/hetzner/) | [Fly.io](sh/fly/) | [AWS Lightsail](sh/aws/) | [Daytona](sh/daytona/) | [DigitalOcean](sh/digitalocean/) | [GCP Compute Engine](sh/gcp/) | [Sprite](sh/sprite/) |
|---|---|---|---|---|---|---|---|---|
| [**Claude Code**](https://claude.ai) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| [**OpenClaw**](https://github.com/openclaw/openclaw) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
@ -191,7 +191,7 @@ git config core.hooksPath .githooks
### Structure
```
{cloud}/{agent}.sh # Agent deployment script (thin bash → bun wrapper)
sh/{cloud}/{agent}.sh # Agent deployment script (thin bash → bun wrapper)
cli/ # TypeScript CLI — all provisioning logic (bun)
manifest.json # Source of truth for the matrix
```

View file

@ -33,7 +33,7 @@ cli/
│ ├── update-check.ts # Auto-update check (once per day)
│ ├── version.ts # Version constant
│ └── __tests__/ # Test suite (Bun test runner)
├── install.sh # Installer (auto-installs bun if needed)
├── ../sh/cli/install.sh # Installer (auto-installs bun if needed, lives in sh/cli/)
├── package.json # Package metadata and dependencies
└── tsconfig.json # TypeScript configuration
```
@ -57,7 +57,7 @@ The TypeScript CLI (`src/*.ts`) provides:
### Quick Install
```bash
curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/cli/install.sh | bash
curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/sh/cli/install.sh | bash
```
The installer will:

View file

@ -1,6 +1,6 @@
{
"name": "@openrouter/spawn",
"version": "0.8.5",
"version": "0.9.0",
"type": "module",
"bin": {
"spawn": "cli.js"

View file

@ -4,7 +4,7 @@ import { resolve, join } from "node:path";
import { isString } from "../shared/type-guards";
/**
* Validation tests for cli/install.sh.
* Validation tests for sh/cli/install.sh.
*
* install.sh is the critical entry point for all new users
* (curl -fsSL ... | bash). It has been modified in multiple recent PRs
@ -15,7 +15,7 @@ import { isString } from "../shared/type-guards";
*/
const REPO_ROOT = resolve(import.meta.dir, "../../..");
const INSTALL_SH = join(REPO_ROOT, "cli", "install.sh");
const INSTALL_SH = join(REPO_ROOT, "sh", "cli", "install.sh");
const content = readFileSync(INSTALL_SH, "utf-8");
const lines = content.split("\n");
@ -450,7 +450,7 @@ describe("install.sh validation", () => {
describe("error handling", () => {
it("should show re-run instructions on bun install failure", () => {
expect(content).toContain("curl -fsSL ${SPAWN_RAW_BASE}/cli/install.sh | bash");
expect(content).toContain("curl -fsSL ${SPAWN_RAW_BASE}/sh/cli/install.sh | bash");
});
it("should show manual bun install instructions on failure", () => {

View file

@ -190,7 +190,7 @@ describe("Manifest Integrity", () => {
const missing: string[] = [];
for (const [key] of implemented) {
const scriptPath = join(REPO_ROOT, key + ".sh");
const scriptPath = join(REPO_ROOT, "sh", key + ".sh");
if (!existsSync(scriptPath)) {
missing.push(key + ".sh");
}
@ -216,7 +216,7 @@ describe("Manifest Integrity", () => {
const badScripts: string[] = [];
for (const [key] of sample) {
const scriptPath = join(REPO_ROOT, key + ".sh");
const scriptPath = join(REPO_ROOT, "sh", key + ".sh");
if (existsSync(scriptPath)) {
const content = readFileSync(scriptPath, "utf-8");
if (!content.trimStart().startsWith("#!")) {
@ -234,7 +234,7 @@ describe("Manifest Integrity", () => {
const badScripts: string[] = [];
for (const [key] of sample) {
const scriptPath = join(REPO_ROOT, key + ".sh");
const scriptPath = join(REPO_ROOT, "sh", key + ".sh");
if (existsSync(scriptPath)) {
const content = readFileSync(scriptPath, "utf-8");
if (!content.includes("set -eo pipefail")) {
@ -257,7 +257,7 @@ describe("Manifest Integrity", () => {
const orphaned: string[] = [];
for (const [key] of missingEntries) {
const scriptPath = join(REPO_ROOT, key + ".sh");
const scriptPath = join(REPO_ROOT, "sh", key + ".sh");
if (existsSync(scriptPath)) {
orphaned.push(key + ".sh");
}

View file

@ -834,7 +834,7 @@ function showDryRunPreview(manifest: Manifest, agent: string, cloud: string, pro
printDryRunSection("Agent", buildAgentLines(manifest.agents[agent]));
printDryRunSection("Cloud", buildCloudLines(manifest.clouds[cloud]));
printDryRunSection("Script", [
` URL: ${RAW_BASE}/${cloud}/${agent}.sh`,
` URL: ${RAW_BASE}/sh/${cloud}/${agent}.sh`,
]);
const envLines = buildEnvironmentLines(manifest, agent);
@ -1186,7 +1186,7 @@ export async function cmdRunHeadless(agent: string, cloud: string, opts: Headles
// Phase 2: Download script (exit code 2)
const url = `https://openrouter.ai/labs/spawn/${resolvedCloud}/${resolvedAgent}.sh`;
const ghUrl = `${RAW_BASE}/${resolvedCloud}/${resolvedAgent}.sh`;
const ghUrl = `${RAW_BASE}/sh/${resolvedCloud}/${resolvedAgent}.sh`;
let scriptContent: string;
try {
@ -1697,7 +1697,7 @@ async function execScript(
spawnName?: string,
): Promise<void> {
const url = `https://openrouter.ai/labs/spawn/${cloud}/${agent}.sh`;
const ghUrl = `${RAW_BASE}/${cloud}/${agent}.sh`;
const ghUrl = `${RAW_BASE}/sh/${cloud}/${agent}.sh`;
let scriptContent: string;
try {
@ -3241,7 +3241,7 @@ async function fetchRemoteVersion(): Promise<string> {
return data.version;
}
const INSTALL_CMD = `curl -fsSL ${RAW_BASE}/cli/install.sh | bash`;
const INSTALL_CMD = `curl -fsSL ${RAW_BASE}/sh/cli/install.sh | bash`;
async function performUpdate(_remoteVersion: string): Promise<void> {
const { execSync } = await import("node:child_process");
@ -3355,7 +3355,7 @@ function getHelpAuthSection(): string {
function getHelpInstallSection(): string {
return `${pc.bold("INSTALL")}
curl -fsSL ${RAW_BASE}/cli/install.sh | bash`;
curl -fsSL ${RAW_BASE}/sh/cli/install.sh | bash`;
}
function getHelpTroubleshootingSection(): string {

View file

@ -207,7 +207,7 @@ export async function offerGithubAuth(runner: CloudRunner): Promise<void> {
return;
}
let ghCmd = "curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/shared/github-auth.sh | bash";
let ghCmd = "curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/sh/shared/github-auth.sh | bash";
let localTmpFile = "";
if (githubToken) {
const escaped = githubToken.replace(/'/g, "'\\''");

View file

@ -202,7 +202,7 @@ function performAutoUpdate(latestVersion: string): void {
}
try {
executor.execSync(`curl -fsSL ${RAW_BASE}/cli/install.sh | bash`, {
executor.execSync(`curl -fsSL ${RAW_BASE}/sh/cli/install.sh | bash`, {
stdio: "inherit",
shell: "/bin/bash",
});
@ -217,7 +217,7 @@ function performAutoUpdate(latestVersion: string): void {
console.error(pc.red(pc.bold(`${CROSS_MARK} Auto-update failed`)));
console.error(pc.dim(" Please update manually:"));
console.error();
console.error(pc.cyan(` curl -fsSL ${RAW_BASE}/cli/install.sh | bash`));
console.error(pc.cyan(` curl -fsSL ${RAW_BASE}/sh/cli/install.sh | bash`));
console.error();
// Continue with original command despite update failure
}

View file

@ -2,7 +2,7 @@
# Installer for the spawn CLI
#
# Usage:
# curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/cli/install.sh | bash
# curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/sh/cli/install.sh | bash
#
# This installs spawn via bun. If bun is not available, it auto-installs it first.
#
@ -62,7 +62,7 @@ ensure_min_bun_version() {
echo " bun upgrade"
echo ""
echo "Then re-run:"
echo " curl -fsSL ${SPAWN_RAW_BASE}/cli/install.sh | bash"
echo " curl -fsSL ${SPAWN_RAW_BASE}/sh/cli/install.sh | bash"
exit 1
fi
log_info "bun upgraded to ${current}"
@ -276,7 +276,7 @@ if ! command -v bun &>/dev/null; then
echo " curl -fsSL https://bun.sh/install | bash"
echo ""
echo "Then reopen your terminal and re-run:"
echo " curl -fsSL ${SPAWN_RAW_BASE}/cli/install.sh | bash"
echo " curl -fsSL ${SPAWN_RAW_BASE}/sh/cli/install.sh | bash"
exit 1
fi

View file

@ -1,12 +1,12 @@
#!/bin/bash
# e2e/fly-e2e.sh — Main E2E test orchestrator for Spawn on Fly.io
# sh/e2e/fly-e2e.sh — Main E2E test orchestrator for Spawn on Fly.io
#
# Usage:
# ./e2e/fly-e2e.sh # All agents, sequential
# ./e2e/fly-e2e.sh claude # Single agent
# ./e2e/fly-e2e.sh claude codex opencode # Specific agents
# ./e2e/fly-e2e.sh --parallel 2 # Parallel (2 at a time)
# ./e2e/fly-e2e.sh --skip-cleanup # Skip stale app cleanup
# ./sh/e2e/fly-e2e.sh # All agents, sequential
# ./sh/e2e/fly-e2e.sh claude # Single agent
# ./sh/e2e/fly-e2e.sh claude codex opencode # Specific agents
# ./sh/e2e/fly-e2e.sh --parallel 2 # Parallel (2 at a time)
# ./sh/e2e/fly-e2e.sh --skip-cleanup # Skip stale app cleanup
set -eo pipefail
# ---------------------------------------------------------------------------

View file

@ -3,12 +3,12 @@
# Sourceable by any agent script, or executable directly via curl|bash
#
# Usage (sourced):
# source shared/github-auth.sh
# source sh/shared/github-auth.sh
# ensure_github_auth
#
# Usage (direct):
# bash shared/github-auth.sh
# curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/shared/github-auth.sh | bash
# bash sh/shared/github-auth.sh
# curl -fsSL https://raw.githubusercontent.com/OpenRouterTeam/spawn/main/sh/shared/github-auth.sh | bash
# ============================================================
# Logging helpers

View file

@ -6,11 +6,11 @@ set -eo pipefail
# This script itself is bash 3.2 compatible.
#
# Usage:
# bash test/macos-compat.sh # Scan all .sh files
# bash test/macos-compat.sh --warn-only # Always exit 0
# bash test/macos-compat.sh path/to/file.sh # Scan specific file(s)
# bash sh/test/macos-compat.sh # Scan all .sh files
# bash sh/test/macos-compat.sh --warn-only # Always exit 0
# bash sh/test/macos-compat.sh path/to/file.sh # Scan specific file(s)
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
WARN_ONLY=false
FILES_CHECKED=0