diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0078e90..cc5dde3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -76,6 +76,13 @@ SurfSense consists of three main components: ## ๐Ÿงช Development Guidelines +### Code Quality & Pre-commit Hooks +We use pre-commit hooks to maintain code quality, security, and consistency across the codebase. Before you start developing: + +1. **Install and set up pre-commit hooks** - See our detailed [Pre-commit Guide](./PRE_COMMIT.md) +2. **Understand the automated checks** that will run on your code +3. **Learn about bypassing hooks** when necessary (use sparingly!) + ### Code Style - **Backend**: Follow Python PEP 8 style guidelines - **Frontend**: Use TypeScript and follow the existing code patterns diff --git a/PRE_COMMIT.md b/PRE_COMMIT.md new file mode 100644 index 0000000..96d123d --- /dev/null +++ b/PRE_COMMIT.md @@ -0,0 +1,237 @@ +# Pre-commit Hooks for SurfSense Contributors + +Welcome to SurfSense! As an open-source project, we use pre-commit hooks to maintain code quality, security, and consistency across our multi-component codebase. This guide will help you set up and work with our pre-commit configuration. + +## ๐Ÿš€ What is Pre-commit? + +Pre-commit is a framework for managing multi-language pre-commit hooks. It runs automatically before each commit to catch issues early, ensuring high code quality and consistency across the project. + +## ๐Ÿ“ Project Structure + +SurfSense consists of three main components: +- **`surfsense_backend/`** - Python backend API +- **`surfsense_web/`** - Next.js web application +- **`surfsense_browser_extension/`** - TypeScript browser extension + +## ๐Ÿ›  Installation + +### Prerequisites +- Python 3.8 or higher +- Node.js 18+ and pnpm (for frontend components) +- Git + +### Install Pre-commit + +```bash +# Install pre-commit globally +pip install pre-commit + +# Or using your preferred package manager +# pipx install pre-commit # Recommended for isolation +``` + +### Setup Pre-commit Hooks + +1. **Clone the repository**: + ```bash + git clone https://github.com/masabinhok/SurfSense.git + cd SurfSense + ``` + +2. **Install the pre-commit hooks**: + ```bash + pre-commit install + ``` + +3. **Install commit message hooks** (optional, for conventional commits): + ```bash + pre-commit install --hook-type commit-msg + ``` + +## ๐Ÿ”ง Configuration Files Added + +When you install pre-commit, the following files are part of the setup: + +- **`.pre-commit-config.yaml`** - Main pre-commit configuration +- **`.secrets.baseline`** - Baseline file for secret detection (prevents false positives) +- **`.github/workflows/pre-commit.yml`** - CI workflow that runs pre-commit on PRs + +## ๐ŸŽฏ What Gets Checked + +### All Files +- โœ… Trailing whitespace removal +- โœ… YAML, JSON, and TOML validation +- โœ… Large file detection (>10MB) +- โœ… Merge conflict markers +- ๐Ÿ”’ **Secret detection** using detect-secrets + +### Python Backend (`surfsense_backend/`) +- ๐Ÿ **Black** - Code formatting +- ๐Ÿ“ฆ **isort** - Import sorting +- โšก **Ruff** - Fast linting and formatting +- ๐Ÿ” **MyPy** - Static type checking +- ๐Ÿ›ก๏ธ **Bandit** - Security vulnerability scanning + +### Frontend (`surfsense_web/` & `surfsense_browser_extension/`) +- ๐Ÿ’… **Prettier** - Code formatting +- ๐Ÿ” **ESLint** - Linting (Next.js config) +- ๐Ÿ“ **TypeScript** - Compilation checks + +### Commit Messages +- ๐Ÿ“ **Commitizen** - Conventional commit format validation + +## ๐Ÿš€ Usage + +### Normal Workflow +Pre-commit will run automatically when you commit: + +```bash +git add . +git commit -m "feat: add new feature" +# Pre-commit hooks will run automatically +``` + +### Manual Execution + +Run on staged files only: +```bash +pre-commit run +``` + +Run on specific files: +```bash +pre-commit run --files path/to/file.py path/to/file.ts +``` + +Run all hooks on all files: +```bash +pre-commit run --all-files +``` + +โš ๏ธ **Warning**: Running `--all-files` may generate numerous errors as this codebase has existing linting and type issues that are being gradually resolved. + +### Advanced Commands + +Update all hooks to latest versions: +```bash +pre-commit autoupdate +``` + +Run only specific hooks: +```bash +pre-commit run black # Run only black +pre-commit run --all-files prettier # Run prettier on all files +``` + +Clean pre-commit cache: +```bash +pre-commit clean +``` + +## ๐Ÿ†˜ Bypassing Pre-commit (When Necessary) + +Sometimes you might need to bypass pre-commit hooks (use sparingly!): + +### Skip all hooks for one commit: +```bash +git commit -m "fix: urgent hotfix" --no-verify +``` + +### Skip specific hooks: +```bash +SKIP=mypy,black git commit -m "feat: work in progress" +``` + +Available hook IDs to skip: +- `trailing-whitespace`, `check-yaml`, `check-json` +- `detect-secrets` +- `black`, `isort`, `ruff`, `ruff-format`, `mypy`, `bandit` +- `prettier`, `eslint` +- `typescript-check-web`, `typescript-check-extension` +- `commitizen` + +## ๐Ÿ› Common Issues & Solutions + +### Secret Detection False Positives + +If detect-secrets flags legitimate content as secrets: + +1. **Review the detection** - Ensure it's not actually a secret +2. **Update baseline**: + ```bash + detect-secrets scan --baseline .secrets.baseline --update + git add .secrets.baseline + ``` + +### TypeScript/Node.js Issues + +Ensure dependencies are installed: +```bash +cd surfsense_web && pnpm install +cd surfsense_browser_extension && pnpm install +``` + +### Python Environment Issues + +For Python hooks, ensure you're in the correct environment: +```bash +cd surfsense_backend +# If using uv +uv sync +# Or traditional pip +pip install -r requirements.txt +``` + +### Hook Installation Issues + +If hooks aren't running: +```bash +pre-commit uninstall +pre-commit install --install-hooks +``` + +## ๐Ÿ“Š Performance Tips + +- **Incremental runs**: Pre-commit only runs on changed files by default +- **Parallel execution**: Many hooks run in parallel for speed +- **Caching**: Pre-commit caches environments to speed up subsequent runs + +## ๐Ÿ”„ CI Integration + +Pre-commit also runs in our GitHub Actions CI pipeline on every PR to `main`. The CI: +- Runs only on changed files for efficiency +- Provides the same feedback as local pre-commit +- Prevents merging code that doesn't pass quality checks + +## ๐Ÿ“‹ Best Practices + +1. **Install pre-commit early** in your development setup +2. **Fix issues incrementally** rather than bypassing hooks +3. **Update your branch regularly** to avoid conflicts with formatting changes +4. **Run `--all-files` periodically** on feature branches (in small chunks) +5. **Keep the `.secrets.baseline` updated** when legitimate secrets-like strings are added + +## ๐Ÿ’ก Contributing to Pre-commit Config + +To modify the pre-commit configuration: + +1. Edit `.pre-commit-config.yaml` +2. Test your changes: + ```bash + pre-commit run --all-files # Test with caution! + ``` +3. Update the baseline if needed: + ```bash + detect-secrets scan --baseline .secrets.baseline --update + ``` +4. Submit a PR with your changes + +## ๐Ÿ†˜ Getting Help + +- **Pre-commit docs**: https://pre-commit.com/ +- **Project issues**: Open an issue on GitHub +- **Hook-specific help**: Check individual tool documentation (Black, Ruff, ESLint, etc.) + +--- + +Thank you for contributing to SurfSense! ๐Ÿ„โ€โ™€๏ธ Quality code makes everyone's surfing experience smoother. \ No newline at end of file