diff --git a/packages/cli/src/__tests__/README.md b/packages/cli/src/__tests__/README.md index 9e3815e4..dc3ae29b 100644 --- a/packages/cli/src/__tests__/README.md +++ b/packages/cli/src/__tests__/README.md @@ -1,59 +1,81 @@ # CLI Tests -This directory contains comprehensive tests for the Spawn CLI TypeScript implementation. +This directory contains unit tests for the Spawn CLI TypeScript implementation. + +## Test Runner + +Tests use **Bun's built-in test runner** (`bun:test`). Do NOT use vitest. + +```bash +# Run all tests +bun test + +# Run a specific file +bun test src/__tests__/manifest.test.ts +``` ## Test Files -### `manifest.test.ts` -Tests for manifest loading, caching, and parsing: -- Network fetching and fallback behavior -- Disk cache TTL and invalidation -- Offline mode with stale cache -- Agent/cloud key extraction -- Matrix status checking -- Implemented combination counting +### Core manifest +- `manifest.test.ts` — `agentKeys`, `cloudKeys`, `matrixStatus`, `countImplemented`, `loadManifest` (cache/network) +- `manifest-integrity.test.ts` — Structural validation: script files exist for implemented entries, no orphans +- `manifest-type-contracts.test.ts` — Field type precision for every agent/cloud in the real manifest +- `manifest-cache-lifecycle.test.ts` — Cache TTL, expiry, forced refresh -### `commands.test.ts` -Tests for CLI command handlers: -- `cmdHelp` - Help text display -- `cmdList` - Matrix table rendering -- `cmdAgents` - Agent listing -- `cmdClouds` - Cloud provider listing -- `cmdAgentInfo` - Agent details with available clouds -- `cmdRun` - Script execution with validation and fallback +### Commands: happy paths +- `cmdrun-happy-path.test.ts` — Successful download, history recording, env var passing +- `cmd-interactive.test.ts` — Interactive agent/cloud selection flow +- `cmd-listing-output.test.ts` — `cmdMatrix`, `cmdAgents`, `cmdClouds` output formatting +- `cmdlast.test.ts` — `cmdLast`: history display and resumption +- `cmdlist-integration.test.ts` — `cmdList` with real history records +- `commands-display.test.ts` — `cmdAgentInfo` (happy path), `cmdHelp` +- `commands-cloud-info.test.ts` — `cmdCloudInfo` display +- `commands-update-download.test.ts` — `cmdUpdate`, script download and execution -### `integration.test.ts` -Integration tests for end-to-end workflows: -- Version command -- Manifest caching across loads -- Offline scenarios -- Agent/cloud key extraction -- Matrix validation -- Implementation counting +### Commands: error paths +- `commands-error-paths.test.ts` — Validation failures, unknown agents/clouds, prompt rejection +- `commands-name-suggestions.test.ts` — Display name typo suggestions in errors +- `commands-swap-resolve.test.ts` — `detectAndFixSwappedArgs`, `resolveAndLog` +- `commands-resolve-run.test.ts` — Display name resolution in `cmdRun` +- `cmdrun-duplicate-detection.test.ts` — `--name` collision detection -## Running Tests +### Commands: utilities +- `commands-exported-utils.test.ts` — `parseAuthEnvVars`, `getImplementedAgents`, `getMissingClouds`, `getErrorMessage`, etc. +- `script-failure-guidance.test.ts` — `getScriptFailureGuidance`, `getSignalGuidance`, `buildRetryCommand` +- `download-and-failure.test.ts` — Download fallback pipeline, failure reporting +- `run-path-credential-display.test.ts` — `prioritizeCloudsByCredentials`, run-path validation -```bash -# Run all tests once -npm test +### Security +- `security.test.ts` — `validateIdentifier`, `validateScriptContent`, `validatePrompt` (core cases) +- `security-edge-cases.test.ts` — Boundary conditions and character-level edge cases +- `security-encoding.test.ts` — Encoding edge cases, `stripDangerousKeys` +- `security-connection-validation.test.ts` — `validateConnectionIP`, `validateUsername`, `validateServerIdentifier`, `validateLaunchCmd` +- `prompt-file-security.test.ts` — `validatePromptFilePath`, `validatePromptFileStats` -# Run tests in watch mode -npm run test:watch +### Infrastructure +- `manifest-cache-lifecycle.test.ts` — Cache lifecycle: write, read, expiry, forced refresh +- `history.test.ts` — History read/write +- `history-trimming.test.ts` — History trimming at size limits +- `clear-history.test.ts` — `clearHistory`, `cmdListClear` +- `ssh-keys.test.ts` — SSH key discovery, generation, fingerprinting +- `update-check.test.ts` — Auto-update check logic +- `with-retry-result.test.ts` — `withRetry`, `wrapSshCall`, Result constructors +- `orchestrate.test.ts` — `runOrchestration` -# Run with coverage (requires coverage provider) -npm test -- --coverage -``` +### Parsing and type utilities +- `parse.test.ts` — `parseJsonWith`, `parseJsonRaw` +- `fuzzy-key-matching.test.ts` — `findClosestKeyByNameOrKey`, `levenshtein`, `findClosestMatch`, `resolveAgentKey`, `resolveCloudKey` +- `unknown-flags.test.ts` — Unknown flag detection, `KNOWN_FLAGS`, `expandEqualsFlags` +- `custom-flag.test.ts` — `--custom` flag for AWS, GCP, Hetzner, DigitalOcean +- `credential-hints.test.ts` — `credentialHints` +- `cloud-credentials.test.ts` — `hasCloudCredentials` +- `preflight-credentials.test.ts` — `preflightCredentialCheck` -## Test Coverage +### Cloud-specific +- `aws.test.ts` — AWS credential cache, SigV4 signing helpers +- `cloud-init.test.ts` — `getPackagesForTier`, `needsNode`, `needsBun`, `NODE_INSTALL_CMD` +- `check-entity.test.ts` / `check-entity-messages.test.ts` — Entity validation -Current coverage targets critical paths: -- **manifest.ts**: ~80% coverage of caching, fetching, and parsing logic -- **commands.ts**: ~70% coverage of command handlers and validation -- **integration**: Basic end-to-end scenarios - -## Notes - -- Tests use vitest for fast execution with Bun/Node compatibility -- Mock manifest data is used to avoid network dependencies -- Cache directory is isolated per test to prevent interference -- Some tests account for local `manifest.json` fallback in project directory +### Support files (not test files) +- `test-helpers.ts` — Shared fixtures: `createMockManifest`, `mockClackPrompts`, `setupTestEnvironment`, etc. +- `preload.ts` — Global test setup (temp dir isolation, env sandboxing)