mirror of
https://github.com/lfnovo/open-notebook.git
synced 2026-05-02 21:30:38 +00:00
docs: generate comprehensive CLAUDE.md reference documentation across codebase
Create a hierarchical CLAUDE.md documentation system for the entire Open Notebook codebase with focus on concise, pattern-driven reference cards rather than comprehensive tutorials. ## Changes ### Core Documentation System - Updated `.claude/commands/build-claude-md.md` to distinguish between leaf and parent modules, with special handling for prompt/template modules - Established clear patterns: * Leaf modules (40-70 lines): Components, hooks, API clients * Parent modules (50-150 lines): Architecture, cross-layer patterns, data flows * Template modules: Pattern focus, not catalog listings ### Generated Documentation Created 15 CLAUDE.md reference files across the project: **Frontend (React/Next.js)** - frontend/src/CLAUDE.md: Architecture overview, data flow, three-tier design - frontend/src/lib/hooks/CLAUDE.md: React Query patterns, state management - frontend/src/lib/api/CLAUDE.md: Axios client, FormData handling, interceptors - frontend/src/lib/stores/CLAUDE.md: Zustand state persistence, auth patterns - frontend/src/components/ui/CLAUDE.md: Radix UI primitives, CVA styling **Backend (Python/FastAPI)** - open_notebook/CLAUDE.md: System architecture, layer interactions - open_notebook/ai/CLAUDE.md: Model provisioning, Esperanto integration - open_notebook/domain/CLAUDE.md: Data models, ObjectModel/RecordModel patterns - open_notebook/database/CLAUDE.md: Repository pattern, async migrations - open_notebook/graphs/CLAUDE.md: LangGraph workflows, async orchestration - open_notebook/utils/CLAUDE.md: Cross-cutting utilities, context building - open_notebook/podcasts/CLAUDE.md: Episode/speaker profiles, job tracking **API & Other** - api/CLAUDE.md: REST layer, service architecture - commands/CLAUDE.md: Async command handlers, job queue patterns - prompts/CLAUDE.md: Jinja2 templates, prompt engineering patterns (refactored) **Project Root** - CLAUDE.md: Project overview, three-tier architecture, tech stack, getting started ### Key Features - Zero duplication: Parent modules reference child CLAUDE.md files, don't repeat them - Pattern-focused: Emphasizes how components work together, not component catalogs - Scannable: Short bullets, code examples only when necessary (1-2 per file) - Practical: "How to extend" guides, quirks/gotchas for each module - Navigation: Root CLAUDE.md acts as hub pointing to specialized documentation ### Cleanup - Removed unused `batch_fix_services.py` - Removed deprecated `open_notebook/plugins/podcasts.py` - Updated .gitignore for documentation consistency ## Impact New contributors can now: 1. Read root CLAUDE.md for system architecture (5 min) 2. Jump to specific layer documentation (frontend, api, open_notebook) 3. Dive into module-specific patterns in child CLAUDE.md files (1 min per module) All documentation is lean, reference-focused, and avoids duplication.
This commit is contained in:
parent
ab5560c9a2
commit
71b8d13b24
19 changed files with 1949 additions and 372 deletions
117
api/CLAUDE.md
Normal file
117
api/CLAUDE.md
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
# API Module
|
||||
|
||||
FastAPI-based REST backend exposing services for notebooks, sources, notes, chat, podcasts, and AI model management.
|
||||
|
||||
## Purpose
|
||||
|
||||
FastAPI application serving three architectural layers: routes (HTTP endpoints), services (business logic), and models (request/response schemas). Integrates LangGraph workflows (chat, ask, source_chat), SurrealDB persistence, and AI providers via Esperanto.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
**Three layers**:
|
||||
1. **Routes** (`routers/*`): HTTP endpoints mapping to services
|
||||
2. **Services** (`*_service.py`): Business logic orchestrating domain models, database, graphs, AI providers
|
||||
3. **Models** (`models.py`): Pydantic request/response schemas with validation
|
||||
|
||||
**Startup flow**:
|
||||
- Load .env environment variables
|
||||
- Initialize CORS middleware + password auth middleware
|
||||
- Run database migrations via AsyncMigrationManager on lifespan startup
|
||||
- Register all routers
|
||||
|
||||
**Key services**:
|
||||
- `chat_service.py`: Invokes chat graph with messages, context
|
||||
- `podcast_service.py`: Orchestrates outline + transcript generation
|
||||
- `sources_service.py`: Content ingestion, vectorization, metadata
|
||||
- `notes_service.py`: Note creation, linking to sources/insights
|
||||
- `transformations_service.py`: Applies transformations to content
|
||||
- `models_service.py`: Manages AI provider/model configuration
|
||||
- `episode_profiles_service.py`: Manages podcast speaker/episode profiles
|
||||
|
||||
## Component Catalog
|
||||
|
||||
### Main Application
|
||||
- **main.py**: FastAPI app initialization, CORS setup, auth middleware, lifespan event, router registration
|
||||
- **Lifespan handler**: Runs AsyncMigrationManager on startup (database schema migration)
|
||||
- **Auth middleware**: PasswordAuthMiddleware protects endpoints (password-based access control)
|
||||
|
||||
### Services (Business Logic)
|
||||
- **chat_service.py**: Invokes chat.py graph; handles message history via SqliteSaver
|
||||
- **podcast_service.py**: Generates outline (outline.jinja), then transcript (transcript.jinja) for episodes
|
||||
- **sources_service.py**: Ingests files/URLs (content_core), extracts text, vectorizes, saves to SurrealDB
|
||||
- **transformations_service.py**: Applies transformations via transformation.py graph
|
||||
- **models_service.py**: Manages ModelManager config (AI provider overrides)
|
||||
- **episode_profiles_service.py**: CRUD for EpisodeProfile and SpeakerProfile models
|
||||
- **insights_service.py**: Generates and retrieves source insights
|
||||
- **notes_service.py**: Creates notes linked to sources/insights
|
||||
|
||||
### Models (Schemas)
|
||||
- **models.py**: Pydantic schemas for request/response validation
|
||||
- Request bodies: ChatRequest, CreateNoteRequest, PodcastGenerationRequest, etc.
|
||||
- Response bodies: ChatResponse, NoteResponse, PodcastResponse, etc.
|
||||
- Custom validators for enum fields, file paths, model references
|
||||
|
||||
### Routers
|
||||
- **routers/chat.py**: POST /chat
|
||||
- **routers/source_chat.py**: POST /source/{source_id}/chat
|
||||
- **routers/podcasts.py**: POST /podcasts, GET /podcasts/{id}, etc.
|
||||
- **routers/notes.py**: POST /notes, GET /notes/{id}
|
||||
- **routers/sources.py**: POST /sources, GET /sources/{id}, DELETE /sources/{id}
|
||||
- **routers/models.py**: GET /models, POST /models/config
|
||||
- **routers/transformations.py**: POST /transformations
|
||||
- **routers/insights.py**: GET /sources/{source_id}/insights
|
||||
- **routers/auth.py**: POST /auth/password (password-based auth)
|
||||
- **routers/commands.py**: GET /commands/{command_id} (job status tracking)
|
||||
|
||||
## Common Patterns
|
||||
|
||||
- **Service injection via FastAPI**: Routers import services directly; no DI framework
|
||||
- **Async/await throughout**: All DB queries, graph invocations, AI calls are async
|
||||
- **SurrealDB transactions**: Services use repo_query, repo_create, repo_upsert from database layer
|
||||
- **Config override pattern**: Models/config override via models_service passed to graph.ainvoke(config=...)
|
||||
- **Error handling**: Services catch exceptions and return HTTP status codes (400 Bad Request, 404 Not Found, 500 Internal Server Error)
|
||||
- **Logging**: loguru logger in main.py; services expected to log key operations
|
||||
- **Response normalization**: All responses follow standard schema (data + metadata structure)
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
- `fastapi`: FastAPI app, routers, HTTPException
|
||||
- `pydantic`: Validation models with Field, field_validator
|
||||
- `open_notebook.graphs`: chat, ask, source_chat, source, transformation graphs
|
||||
- `open_notebook.database`: SurrealDB repository functions (repo_query, repo_create, repo_upsert)
|
||||
- `open_notebook.domain`: Notebook, Source, Note, SourceInsight models
|
||||
- `open_notebook.ai.provision`: provision_langchain_model() factory
|
||||
- `ai_prompter`: Prompter for template rendering
|
||||
- `content_core`: extract_content() for file/URL processing
|
||||
- `esperanto`: AI provider client library (LLM, embeddings, TTS)
|
||||
- `surreal_commands`: Job queue for async operations (podcast generation)
|
||||
- `loguru`: Structured logging
|
||||
|
||||
## Important Quirks & Gotchas
|
||||
|
||||
- **Migration auto-run**: Database schema migrations run on every API startup (via lifespan); no manual migration steps
|
||||
- **PasswordAuthMiddleware is basic**: Uses simple password check; production deployments should replace with OAuth/JWT
|
||||
- **No request rate limiting**: No built-in rate limiting; deployment must add via proxy/middleware
|
||||
- **Service state is stateless**: Services don't cache results; each request re-queries database/AI models
|
||||
- **Graph invocation is blocking**: chat/podcast workflows may take minutes; no timeout handling in services
|
||||
- **Command job fire-and-forget**: podcast_service.py submits jobs but doesn't wait (async job queue pattern)
|
||||
- **Model override scoping**: Model config override via RunnableConfig is per-request only (not persistent)
|
||||
- **CORS open by default**: main.py CORS settings allow all origins (restrict before production)
|
||||
- **No OpenAPI security scheme**: API docs available without auth (disable before production)
|
||||
- **Services don't validate user permission**: All endpoints trust authentication layer; no per-notebook permission checks
|
||||
|
||||
## How to Add New Endpoint
|
||||
|
||||
1. Create router file in `routers/` (e.g., `routers/new_feature.py`)
|
||||
2. Import router into `main.py` and register: `app.include_router(new_feature.router, tags=["new_feature"])`
|
||||
3. Create service in `new_feature_service.py` with business logic
|
||||
4. Define request/response schemas in `models.py` (or create `new_feature_models.py`)
|
||||
5. Implement router functions calling service methods
|
||||
6. Test with `uv run uvicorn api.main:app --host 0.0.0.0 --port 5055`
|
||||
|
||||
## Testing Patterns
|
||||
|
||||
- **Interactive docs**: http://localhost:5055/docs (Swagger UI)
|
||||
- **Direct service tests**: Import service, call methods directly with test data
|
||||
- **Mock graphs**: Replace graph.ainvoke() with mock for testing service logic
|
||||
- **Database: Use test database** (separate SurrealDB instance or mock repo_query)
|
||||
Loading…
Add table
Add a link
Reference in a new issue