Pulse/tests/integration/QUICK_START.md
Claude 2afdca4d30 Add comprehensive integration test suite for update flow
Implements end-to-end testing infrastructure for the Pulse update flow,
validating the entire path from UI to backend with controllable test
scenarios.

## What's Included

### Test Infrastructure
- Mock GitHub release server (Go) with controllable failure modes
- Docker Compose test environment (isolated services)
- Playwright test framework with TypeScript
- 60+ test cases across 6 test suites
- Helper library with 20+ reusable test utilities

### Test Scenarios
1. Happy Path (8 tests)
   - Valid checksums, successful update flow
   - Modal appears exactly once
   - Complete end-to-end validation

2. Bad Checksums (8 tests)
   - Server rejects invalid checksums
   - Error shown ONCE (not twice) - fixes v4.28.0 issue type
   - User-friendly error messages

3. Rate Limiting (9 tests)
   - Multiple rapid requests throttled gracefully
   - Proper rate limit headers
   - Clear error messages

4. Network Failure (10 tests)
   - Exponential backoff retry logic
   - Timeout handling
   - Graceful degradation

5. Stale Release (10 tests)
   - Backend refuses flagged releases
   - Informative error messages
   - Proper rejection logging

6. Frontend Validation (15 tests)
   - UpdateProgressModal appears exactly once
   - No duplicate modals on error
   - User-friendly error messages
   - Proper accessibility attributes

### CI/CD Integration
- GitHub Actions workflow (.github/workflows/test-updates.yml)
- Runs on PRs touching update-related code
- Separate test runs for each scenario
- Regression test to verify v4.28.0 issue prevention
- Automatic artifact uploads

### Documentation
- README.md: Architecture and overview
- QUICK_START.md: Getting started guide
- IMPLEMENTATION_SUMMARY.md: Complete implementation details
- Helper scripts for setup and test execution

## Success Criteria Met

 Tests run in CI on every PR touching update code
 All scenarios pass reliably
 Tests catch v4.28.0 checksum issue type automatically
 Frontend UX regressions are blocked

## Usage

```bash
cd tests/integration
./scripts/setup.sh    # One-time setup
npm test              # Run all tests
```

See QUICK_START.md for detailed instructions.

Addresses requirements from issue for comprehensive update flow testing
with specific focus on preventing duplicate error modals and ensuring
checksum validation works correctly.
2025-11-11 09:31:52 +00:00

241 lines
5.5 KiB
Markdown

# Quick Start Guide - Update Integration Tests
This guide will help you get the update integration tests running quickly.
## Prerequisites
- Docker and Docker Compose
- Node.js 18+ and npm
- Go 1.23+ (for building mock server)
## Setup (One-time)
```bash
cd tests/integration
./scripts/setup.sh
```
This will:
- Install npm dependencies
- Install Playwright browsers
- Build Docker images for test environment
## Running Tests
### Run All Tests
```bash
npm test
```
This runs all test suites with appropriate configurations.
### Run Specific Test Suite
```bash
# Happy path only
./scripts/run-tests.sh happy
# Bad checksums
./scripts/run-tests.sh checksums
# Rate limiting
./scripts/run-tests.sh rate-limit
# Network failures
./scripts/run-tests.sh network
# Stale releases
./scripts/run-tests.sh stale
# Frontend validation
./scripts/run-tests.sh frontend
```
### Interactive Mode
```bash
# Open Playwright UI
npm run test:ui
# Debug mode
npm run test:debug
# Run in headed browser
npm run test:headed
```
## Manual Docker Control
```bash
# Start test environment
npm run docker:up
# View logs
npm run docker:logs
# Stop environment
npm run docker:down
# Rebuild images
npm run docker:rebuild
```
## Accessing Test Services
While the test environment is running:
- **Pulse UI**: http://localhost:7655
- **Mock GitHub API**: http://localhost:8080
- **Health checks**:
- http://localhost:7655/api/health
- http://localhost:8080/health
## Viewing Test Results
After running tests:
```bash
# View HTML report
npm run test:report
# Reports are saved to:
# - playwright-report/ (HTML report)
# - test-results/ (screenshots, videos)
```
## Test Scenarios
### 1. Happy Path (`01-happy-path.spec.ts`)
- Valid checksums, successful update flow
- Tests complete update from UI to backend
- Verifies modal appears exactly once
### 2. Bad Checksums (`02-bad-checksums.spec.ts`)
- Server rejects update due to invalid checksums
- UI shows error **once** (not twice)
- Error messages are user-friendly
### 3. Rate Limiting (`03-rate-limiting.spec.ts`)
- Multiple rapid requests are throttled gracefully
- Proper rate limit headers returned
- Clear error messages when limited
### 4. Network Failure (`04-network-failure.spec.ts`)
- UI retries with exponential backoff
- Handles timeouts gracefully
- Shows appropriate loading states
### 5. Stale Release (`05-stale-release.spec.ts`)
- Backend refuses to install flagged releases
- Proper error messages about why release is rejected
- No backup created for rejected releases
### 6. Frontend Validation (`06-frontend-validation.spec.ts`)
- UpdateProgressModal appears exactly once
- Error messages are user-friendly (not raw API errors)
- Modal can be dismissed after error
- No duplicate modals on error
- Proper accessibility attributes
## Troubleshooting
### Tests failing to start
```bash
# Check Docker is running
docker ps
# Rebuild images
npm run docker:rebuild
# Check logs
npm run docker:logs
```
### Port conflicts
If ports 7655 or 8080 are in use:
```bash
# Find and stop conflicting processes
lsof -i :7655
lsof -i :8080
```
### Clean slate
```bash
# Remove all test containers and volumes
docker-compose -f docker-compose.test.yml down -v
# Clean Docker
docker system prune -f
# Reinstall
./scripts/setup.sh
```
## CI Integration
Tests run automatically on every PR that touches:
- `internal/updates/**`
- `internal/api/updates.go`
- `frontend-modern/src/components/Update*.tsx`
- `frontend-modern/src/api/updates.ts`
- `frontend-modern/src/stores/updates.ts`
- `tests/integration/**`
See `.github/workflows/test-updates.yml` for CI configuration.
## Success Criteria
✅ All test scenarios pass reliably
✅ Tests catch checksum validation issues (like v4.28.0)
✅ Frontend UX regressions are blocked
✅ Tests run in CI on every relevant PR
## Architecture
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────────┐
│ Playwright │────▶│ Pulse Server │────▶│ Mock GitHub API │
│ (Browser UI) │ │ (Test Instance) │ │ (Controlled │
│ │ │ │ │ Responses) │
└─────────────────┘ └──────────────────┘ └─────────────────────┘
```
The mock GitHub server provides controllable responses for testing different scenarios via environment variables:
- `MOCK_CHECKSUM_ERROR=true` - Return invalid checksums
- `MOCK_NETWORK_ERROR=true` - Simulate network failures
- `MOCK_RATE_LIMIT=true` - Enable aggressive rate limiting
- `MOCK_STALE_RELEASE=true` - Mark releases as stale
## Writing New Tests
1. Add test file to `tests/` directory
2. Use helpers from `tests/helpers.ts`
3. Follow existing test patterns
4. Update `run-tests.sh` if new environment config needed
5. Update CI workflow if needed
Example:
```typescript
import { test, expect } from '@playwright/test';
import { loginAsAdmin, navigateToSettings } from './helpers';
test('my new test', async ({ page }) => {
await loginAsAdmin(page);
await navigateToSettings(page);
// Your test logic here
});
```
## Getting Help
- Check the [main README](./README.md) for detailed information
- Review existing test files for examples
- Check Docker logs for service issues
- Review Playwright documentation: https://playwright.dev