Commit graph

70 commits

Author SHA1 Message Date
LUIS NOVO
71b8d13b24 docs: generate comprehensive CLAUDE.md reference documentation across codebase
Create a hierarchical CLAUDE.md documentation system for the entire Open Notebook
codebase with focus on concise, pattern-driven reference cards rather than
comprehensive tutorials.

## Changes

### Core Documentation System
- Updated `.claude/commands/build-claude-md.md` to distinguish between leaf and
  parent modules, with special handling for prompt/template modules
- Established clear patterns:
  * Leaf modules (40-70 lines): Components, hooks, API clients
  * Parent modules (50-150 lines): Architecture, cross-layer patterns, data flows
  * Template modules: Pattern focus, not catalog listings

### Generated Documentation
Created 15 CLAUDE.md reference files across the project:

**Frontend (React/Next.js)**
- frontend/src/CLAUDE.md: Architecture overview, data flow, three-tier design
- frontend/src/lib/hooks/CLAUDE.md: React Query patterns, state management
- frontend/src/lib/api/CLAUDE.md: Axios client, FormData handling, interceptors
- frontend/src/lib/stores/CLAUDE.md: Zustand state persistence, auth patterns
- frontend/src/components/ui/CLAUDE.md: Radix UI primitives, CVA styling

**Backend (Python/FastAPI)**
- open_notebook/CLAUDE.md: System architecture, layer interactions
- open_notebook/ai/CLAUDE.md: Model provisioning, Esperanto integration
- open_notebook/domain/CLAUDE.md: Data models, ObjectModel/RecordModel patterns
- open_notebook/database/CLAUDE.md: Repository pattern, async migrations
- open_notebook/graphs/CLAUDE.md: LangGraph workflows, async orchestration
- open_notebook/utils/CLAUDE.md: Cross-cutting utilities, context building
- open_notebook/podcasts/CLAUDE.md: Episode/speaker profiles, job tracking

**API & Other**
- api/CLAUDE.md: REST layer, service architecture
- commands/CLAUDE.md: Async command handlers, job queue patterns
- prompts/CLAUDE.md: Jinja2 templates, prompt engineering patterns (refactored)

**Project Root**
- CLAUDE.md: Project overview, three-tier architecture, tech stack, getting started

### Key Features
- Zero duplication: Parent modules reference child CLAUDE.md files, don't repeat them
- Pattern-focused: Emphasizes how components work together, not component catalogs
- Scannable: Short bullets, code examples only when necessary (1-2 per file)
- Practical: "How to extend" guides, quirks/gotchas for each module
- Navigation: Root CLAUDE.md acts as hub pointing to specialized documentation

### Cleanup
- Removed unused `batch_fix_services.py`
- Removed deprecated `open_notebook/plugins/podcasts.py`
- Updated .gitignore for documentation consistency

## Impact
New contributors can now:
1. Read root CLAUDE.md for system architecture (5 min)
2. Jump to specific layer documentation (frontend, api, open_notebook)
3. Dive into module-specific patterns in child CLAUDE.md files (1 min per module)
All documentation is lean, reference-focused, and avoids duplication.
2026-01-03 16:27:52 -03:00
LUIS NOVO
ab5560c9a2 refactor: reorganize folder structure for better maintainability
Changes:
- Move migrations/ under open_notebook/database/migrations/
- Extract AI models to open_notebook/ai/ (Model, ModelManager, provision)
- Extract podcasts to open_notebook/podcasts/ (EpisodeProfile, SpeakerProfile, PodcastEpisode)
- Reorganize prompts to mirror graphs structure (chat/, source_chat/)

This improves code organization by:
- Consolidating database concerns (migrations now with database code)
- Separating AI infrastructure from domain entities
- Isolating podcast feature into its own module
- Creating consistent prompt/graph naming conventions

All 52 tests pass.
2026-01-03 14:04:27 -03:00
Justin Florentine
855e730577
fix: preserve AIMessage metadata when cleaning thinking content
Use model_copy() instead of creating new AIMessage to preserve
response_metadata, id, usage_metadata, etc. Also adds test coverage
for malformed thinking tags pattern.

Addresses PR #333 feedback from lfnovo and cubic-dev-ai.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 20:08:12 -05:00
Justin Florentine
869664a10b
fix: strip <think> tags from chat responses
Add thinking content cleaning to notebook and source chat graphs.
Previously, models that output <think>...</think> tags (like DeepSeek)
or malformed variants without opening tags (like Nemotron) would leak
reasoning content into user-visible responses.

Changes:
- chat.py: Clean AI response content before returning messages
- source_chat.py: Same fix for source-specific chat
- text_utils.py: Handle malformed output where opening <think> tag
  is missing but </think> is present

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 16:31:23 -05:00
Luis Novo
f79a9040ae
Release 1.2 (#242)
* chore: improve podcast transcripts

* fix: remove date from insight - fixes #241

* fix: improve scrolling on source and insights - fixes #237

* chore: update esperanto to fix: #234

* chore: update esperanto to fix #226

* fix: process vectorization as subcommands to handle larger documents more gracefully - fix: #229

* feat: enable background job retry capabilities

* feat: reenable content types that were disabled during alpha version

* fix: remove unnecessary model caching causing many issues.

* feat: support multiple azure endpoints and keys just like openai compatible. Fixes #215

* docs: update azure variables

* chore: bump and update dependencies
2025-11-01 14:40:00 -03:00
LUIS NOVO
a51bb9d792 fix: missing parenthesis 2025-10-18 13:22:39 -03:00
LUIS NOVO
8b5daa86bc fix: max tokens max is 8192 now 2025-10-18 13:21:53 -03:00
Luis Novo
b7e656a319
Version 1 (#160)
New front-end
Launch Chat API
Manage Sources
Enable re-embedding of all contents
Sources can be added without a notebook now
Improved settings
Enable model selector on all chats
Background processing for better experience
Dark mode
Improved Notes

Improved Docs: 
- Remove all Streamlit references from documentation
- Update deployment guides with React frontend setup
- Fix Docker environment variables format (SURREAL_URL, SURREAL_PASSWORD)
- Update docker image tag from :latest to :v1-latest
- Change navigation references (Settings → Models to just Models)
- Update development setup to include frontend npm commands
- Add MIGRATION.md guide for users upgrading from Streamlit
- Update quick-start guide with correct environment variables
- Add port 5055 documentation for API access
- Update project structure to reflect frontend/ directory
- Remove outdated source-chat documentation files
2025-10-18 12:46:22 -03:00
Luis Novo
d7b0fff954
Api podcast migration (#93)
Creates the API layer for Open Notebook
Creates a services API gateway for the Streamlit front-end
Migrates the SurrealDB SDK to the official one
Change all database calls to async
New podcast framework supporting multiple speaker configurations
Implement the surreal-commands library for async processing
Improve docker image and docker-compose configurations
2025-07-17 08:36:11 -03:00
LUIS NOVO
7eee271232 feat: extract think tags from reasoning models 2025-06-26 11:41:15 -03:00
LUIS NOVO
bea43f3ce7 feat: implement the new model management based on esperanto framework 2025-06-08 19:38:43 -03:00
LUIS NOVO
2afbd36cb4 refactor: implement ai_prompter library 2025-06-01 08:09:33 -03:00
LUIS NOVO
1afb5d81e8 feat: implement new content settings page and remove options from the source panel 2025-05-30 15:25:39 -03:00
LUIS NOVO
36e928eb75 feat: replace content processing engine with content-core 2025-05-30 13:35:46 -03:00
LUIS NOVO
c297dcb809 refactor objectmodel 2024-11-19 19:03:32 -03:00
LUIS NOVO
4a5d47d934 refactor transformation, add graph and admin 2024-11-18 22:01:11 -03:00
LUIS NOVO
066c7a06e2 improve search functions 2024-11-13 15:52:44 -03:00
LUIS NOVO
80353a97c9 make model rag work with vector only 2024-11-13 12:18:26 -03:00
LUIS NOVO
e4b8fa8cc7 cleanup logging 2024-11-13 12:17:57 -03:00
LUIS NOVO
281abdf01b improve the accuracy of ids in the citations 2024-11-13 11:55:38 -03:00
LUIS NOVO
a33228de5a split system and user message in patterns 2024-11-12 12:56:03 -03:00
LUIS NOVO
8cb6d835fe add ui improvements to embed and transformation dialogs 2024-11-11 18:17:08 -03:00
LUIS NOVO
817b1bc7f9 add initial embedding to the content graph 2024-11-11 17:47:50 -03:00
LUIS NOVO
01cf15e7d1 add check 2024-11-11 17:33:28 -03:00
LUIS NOVO
00f070a644 add async content processing 2024-11-11 17:32:35 -03:00
LUIS NOVO
2e2a4947b3 separate source and content graph 2024-11-10 13:30:03 -03:00
LUIS NOVO
d5be2b0d5b make rag async 2024-11-09 16:03:41 -03:00
LUIS NOVO
183149014e change model provisioning parameters 2024-11-08 16:08:54 -03:00
LUIS NOVO
99b8ada280 new ask model strategy 2024-11-08 16:08:13 -03:00
LUIS NOVO
418c67f69f add search and rag functions in beta 2024-11-04 09:53:49 -03:00
LUIS NOVO
b4ba3ef4c8 change model provisioning strategy 2024-11-04 09:49:11 -03:00
LUIS NOVO
d9c0c93deb improved typing 2024-11-01 22:50:27 -03:00
LUIS NOVO
7dc37a3ac7 model fixes 2024-11-01 22:43:33 -03:00
LUIS NOVO
223f1bdaf5 improve default_models 2024-11-01 22:38:21 -03:00
LUIS NOVO
212d3a33b0 improve object typing 2024-11-01 22:29:59 -03:00
LUIS NOVO
15048b0839 simplify model provisioning 2024-11-01 21:32:40 -03:00
LUIS NOVO
3b262a63f4 better model mgmt 2024-11-01 21:11:23 -03:00
LUIS NOVO
a9ac4a6dc8 model manager 2024-11-01 20:37:23 -03:00
LUIS NOVO
feabfaed01 remove defaultmodel from config file 2024-11-01 19:56:27 -03:00
LUIS NOVO
edf839cd1b unused graphs 2024-11-01 19:08:33 -03:00
LUIS NOVO
0d4d9473b2 add transformation playground with model selection 2024-11-01 17:17:19 -03:00
LUIS NOVO
b89250d3ca temporary fix to config cache 2024-11-01 17:06:10 -03:00
LUIS NOVO
796012f716 rename to patterns 2024-11-01 13:08:18 -03:00
LUIS NOVO
0876e94658 transformation folder change 2024-11-01 12:36:59 -03:00
LUIS NOVO
c65bf8ba12 rename model_name to model_id 2024-11-01 12:07:00 -03:00
LUIS NOVO
2aa453f73f enable support for large context fallback 2024-10-30 15:39:45 -03:00
LUIS NOVO
859b7f6e7e simplify the model selector 2024-10-30 14:30:29 -03:00
LUIS NOVO
8bb5db158f implement model config 2024-10-30 14:09:24 -03:00
LUIS NOVO
ab9824ebed default to bs4 for url processing 2024-10-28 16:32:38 -03:00
LUIS NOVO
ba4b8ad0f7 reorg content 2024-10-28 16:32:27 -03:00