Contribution rules: require real-data testing for new providers, one PR at a time

This commit is contained in:
iamtoruk 2026-05-11 20:03:27 -07:00
parent cd8c646818
commit 4737bfb1fa
2 changed files with 35 additions and 0 deletions

18
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View file

@ -0,0 +1,18 @@
## Summary
<!-- What does this PR do? 1-3 bullet points. -->
## Testing
- [ ] I have tested this locally against real data (not just unit tests)
- [ ] `npm test` passes
- [ ] `npm run build` succeeds
### For new providers only:
- [ ] I installed the tool and generated real sessions by using it
- [ ] `npm run dev -- today` shows correct costs and session counts for this provider
- [ ] `npm run dev -- models --provider <name>` shows correct model names and pricing
- [ ] Screenshot or terminal output attached below proving it works with real data
<!-- Paste screenshot / terminal output here -->

View file

@ -84,6 +84,23 @@ The `.github/workflows/block-claude-coauthor.yml` workflow rejects any PR whose
If a flagged PR rejects on this check, the workflow prints the exact rebase command to fix it.
## Before You Start
**Comment on the issue first.** Before writing code for a feature or new provider, leave a comment on the relevant issue saying what you plan to do. Wait for a maintainer to confirm the approach. Unsolicited PRs that duplicate work already in progress or take an incompatible approach will be closed.
**One PR at a time.** We will not review a second PR from you until the first is merged or closed. This keeps the review queue manageable and ensures each contribution gets proper attention.
## Adding a New Provider
New providers have the highest bar because broken parsing silently produces wrong data for users. Before opening a PR:
1. **Install the tool and use it.** Generate real sessions by actually coding with the provider. We do this ourselves for every provider we ship.
2. **Test against real data.** Run `npm run dev -- today` and `npm run dev -- models` with your real sessions and confirm the output looks correct — costs are non-zero, model names resolve, session counts match what you see in the tool.
3. **Include proof in the PR.** Attach a screenshot or terminal output showing codeburn correctly parsing your real sessions. PRs for new providers without evidence of local testing will not be reviewed.
4. **Do not rely on AI-generated guesses about storage paths or schemas.** Tools change their data formats between versions. The only way to know the current schema is to install the tool and inspect the actual files on disk.
PRs that add a provider based solely on online documentation or AI-generated code, without evidence of testing against real data, will be closed.
## Pull Requests
1. Fork or branch from `main`.