mirror of
https://github.com/Alishahryar1/free-claude-code.git
synced 2026-04-28 11:30:03 +00:00
Add local live smoke test suite (#148)
## Summary - add an opt-in local `smoke/` pytest suite for API, auth, providers, CLI, IDE-shaped requests, messaging, voice, tools, and thinking stream contracts - keep smoke tests out of normal CI collection with `testpaths = ["tests"]` - write sanitized smoke artifacts under `.smoke-results/` ## Verification - `uv run ruff format` - `uv run ruff check` - `uv run ty check` - `uv run ty check smoke` - `FCC_LIVE_SMOKE=1 FCC_SMOKE_TARGETS=all FCC_SMOKE_RUN_VOICE=1 uv run pytest smoke -n 0 -m live -s --tb=short` -> 17 passed, 9 skipped - `uv run pytest` -> 904 passed ## Notes - Skipped live checks require local credentials/tools/services, such as provider models, Telegram/Discord targets, voice backend, or Claude CLI. - `claude-pick` smoke was intentionally removed.
This commit is contained in:
parent
e8e13b9fea
commit
462a9430bb
24 changed files with 1789 additions and 2 deletions
54
smoke/README.md
Normal file
54
smoke/README.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
# Local Live Smoke Tests
|
||||
|
||||
These tests are for maintainers running against their own `.env`. They are not
|
||||
part of CI and are not collected by plain `uv run pytest`.
|
||||
|
||||
## Safe Default Run
|
||||
|
||||
```powershell
|
||||
$env:FCC_LIVE_SMOKE = "1"
|
||||
uv run pytest smoke -n 0 -m live -s --tb=short
|
||||
```
|
||||
|
||||
`-n 0` is recommended because the normal project pytest config enables xdist.
|
||||
The smoke suite can run with workers, but one process gives clearer logs when a
|
||||
real provider or bot fails.
|
||||
|
||||
## Targeted Runs
|
||||
|
||||
```powershell
|
||||
$env:FCC_LIVE_SMOKE = "1"
|
||||
$env:FCC_SMOKE_TARGETS = "api,providers,thinking,tools"
|
||||
uv run pytest smoke -n 0 -m live -s --tb=short
|
||||
```
|
||||
|
||||
Use `FCC_SMOKE_TARGETS=all` to include Telegram, Discord, and voice checks.
|
||||
The default target set intentionally excludes those side-effectful integrations.
|
||||
|
||||
## Environment
|
||||
|
||||
- `FCC_ENV_FILE`: optional explicit dotenv path. The app still uses its normal
|
||||
env-file precedence.
|
||||
- `FCC_SMOKE_PROVIDER_MATRIX`: comma-separated provider prefixes to test.
|
||||
- `FCC_SMOKE_TIMEOUT_S`: per-request/subprocess timeout, default `45`.
|
||||
- `FCC_SMOKE_CLAUDE_BIN`: Claude CLI executable name, default `claude`.
|
||||
- `FCC_SMOKE_TELEGRAM_CHAT_ID`: Telegram chat/user ID for send/edit/delete.
|
||||
- `FCC_SMOKE_DISCORD_CHANNEL_ID`: Discord channel ID for send/edit/delete.
|
||||
- `FCC_SMOKE_INTERACTIVE=1`: enables manual inbound messaging checks.
|
||||
- `FCC_SMOKE_RUN_VOICE=1`: allows the voice transcription backend to load/run.
|
||||
|
||||
## Results
|
||||
|
||||
Smoke artifacts are written to `.smoke-results/` and ignored by git. Reports and
|
||||
logs redact env values whose names contain `KEY`, `TOKEN`, `SECRET`, `WEBHOOK`,
|
||||
or `AUTH`.
|
||||
|
||||
## How To Read Failures
|
||||
|
||||
- `missing_env`: configure the required key, token, channel, or local base URL.
|
||||
- `upstream_unavailable`: the provider/local model/bot API is not reachable.
|
||||
- `product_failure`: the app returned the wrong shape or crashed.
|
||||
- `harness_bug`: the smoke test itself made an invalid assumption.
|
||||
|
||||
The first real run is expected to find product failures. Fix those separately
|
||||
from harness problems so the suite becomes a reliable regression signal.
|
||||
Loading…
Add table
Add a link
Reference in a new issue