open-notebook/README.md
Luis Novo 9bdfd99f1b
Some checks are pending
Development Build / extract-version (push) Waiting to run
Development Build / test-build-regular (push) Blocked by required conditions
Development Build / test-build-single (push) Blocked by required conditions
Development Build / summary (push) Blocked by required conditions
feat: simplify reverse proxy configuration with Next.js rewrites (#213)
* feat: simplify reverse proxy configuration with Next.js rewrites

Add Next.js API rewrites to proxy /api/* requests internally from port 8502
to the FastAPI backend on port 5055. This eliminates the need for complex
reverse proxy configurations with multiple upstreams and location blocks.

Changes:
- Add rewrites to next.config.ts proxying /api/* to INTERNAL_API_URL
- Introduce INTERNAL_API_URL env var (defaults to http://localhost:5055)
- Update supervisord configs to pass INTERNAL_API_URL to Next.js
- Document INTERNAL_API_URL in .env.example with usage examples
- Add simplified reverse proxy examples for nginx, Traefik, Caddy, Coolify
- Update README architecture diagram to show internal proxying
- Add explanatory comments to _config route handler

Benefits:
- Reduces reverse proxy config from 12 lines to 3 (75% reduction)
- Single-port deployment (8502 only) for 95% of use cases
- Zero breaking changes - backward compatible with existing setups
- Zero performance overhead (validated through testing)
- Preserves proxy headers (X-Forwarded-*) for rate limiting/SSL

Resolves: #179
Related: OSS-321

* fix: rename _config to config to fix production routing

CRITICAL BUG FIX: The /_config endpoint has never worked in production builds
because Next.js treats folders starting with underscore as "private folders"
and excludes them from routing entirely.

This endpoint is critical for:
- Providing API_URL to the browser at runtime
- Enabling zero-config deployments with auto-detection
- Supporting reverse proxy scenarios where API URL differs from frontend URL

Changes:
- Rename frontend/src/app/_config/ → frontend/src/app/config/
- Update client code references (/_config → /config)
- Update documentation with correct endpoint path
- Bump version to 1.1.0 (minor version for new rewrites feature + bug fix)

Impact:
- Runtime configuration now works in production builds
- /config returns {"apiUrl":"http://localhost:5055"} correctly
- Auto-detection for reverse proxy deployments now functional

Related: #179, OSS-321

* fix: resolve React hook exhaustive-deps warning in AddExistingSourceDialog

Wrap performSearch function in useCallback to properly memoize it and satisfy
React Hook exhaustive-deps rule. This prevents unnecessary re-renders and
ensures the useEffect dependency array is correctly specified.

Changes:
- Import useCallback from React
- Wrap performSearch with useCallback([debouncedSearchQuery, allSources])
- Add performSearch to useEffect dependency array

* final fixes
2025-10-24 11:24:14 -03:00

22 KiB

Forks Stargazers Issues MIT License


Logo

Open Notebook

An open source, privacy-focused alternative to Google's Notebook LM!
Join our Discord server for help, to share workflow ideas, and suggest features!
Checkout our website »

📚 Get Started · 📖 User Guide · Features · 🚀 Deploy

New Notebook

In a world dominated by Artificial Intelligence, having the ability to think 🧠 and acquire new knowledge 💡, is a skill that should not be a privilege for a few, nor restricted to a single provider.

Open Notebook empowers you to:

  • 🔒 Control your data - Keep your research private and secure
  • 🤖 Choose your AI models - Support for 16+ providers including OpenAI, Anthropic, Ollama, LM Studio, and more
  • 📚 Organize multi-modal content - PDFs, videos, audio, web pages, and more
  • 🎙️ Generate professional podcasts - Advanced multi-speaker podcast generation
  • 🔍 Search intelligently - Full-text and vector search across all your content
  • 💬 Chat with context - AI conversations powered by your research

Learn more about our project at https://www.open-notebook.ai


⚠️ IMPORTANT: v1.0 Breaking Changes

If you're upgrading from a previous version, please note:

  • 🏷️ Docker tags have changed: The latest tag is now frozen at the last Streamlit version
  • 🆕 Use v1-latest tag for the new React/Next.js version (recommended)
  • 🔌 Port 5055 required: You must expose port 5055 for the API to work
  • 📖 Read the migration guide: See MIGRATION.md for detailed upgrade instructions

New users: You can ignore this notice and proceed with the Quick Start below using the v1-latest-single tag.


🆚 Open Notebook vs Google Notebook LM

Feature Open Notebook Google Notebook LM Advantage
Privacy & Control Self-hosted, your data Google cloud only Complete data sovereignty
AI Provider Choice 16+ providers (OpenAI, Anthropic, Ollama, LM Studio, etc.) Google models only Flexibility and cost optimization
Podcast Speakers 1-4 speakers with custom profiles 2 speakers only Extreme flexibility
Context Control 3 granular levels All-or-nothing Privacy and performance tuning
Content Transformations Custom and built-in Limited options Unlimited processing power
API Access Full REST API No API Complete automation
Deployment Docker, cloud, or local Google hosted only Deploy anywhere
Citations Comprehensive with sources Basic references Research integrity
Customization Open source, fully customizable Closed system Unlimited extensibility
Cost Pay only for AI usage Monthly subscription + usage Transparent and controllable

Why Choose Open Notebook?

  • 🔒 Privacy First: Your sensitive research stays completely private
  • 💰 Cost Control: Choose cheaper AI providers or run locally with Ollama
  • 🎙️ Better Podcasts: Full script control and multi-speaker flexibility vs limited 2-speaker deep-dive format
  • 🔧 Unlimited Customization: Modify, extend, and integrate as needed
  • 🌐 No Vendor Lock-in: Switch providers, deploy anywhere, own your data

Built With

Python Next.js React SurrealDB LangChain

🚀 Quick Start

Docker Images Available:

  • Docker Hub: lfnovo/open_notebook:v1-latest-single
  • GitHub Container Registry: ghcr.io/lfnovo/open-notebook:v1-latest-single

Both registries contain identical images - choose whichever you prefer!

Choose Your Setup:

🏠 Local Machine Setup

Perfect if Docker runs on the same computer where you'll access Open Notebook.

mkdir open-notebook && cd open-notebook

docker run -d \
  --name open-notebook \
  -p 8502:8502 -p 5055:5055 \
  -v ./notebook_data:/app/data \
  -v ./surreal_data:/mydata \
  -e OPENAI_API_KEY=your_key_here \
  -e SURREAL_URL="ws://localhost:8000/rpc" \
  -e SURREAL_USER="root" \
  -e SURREAL_PASSWORD="root" \
  -e SURREAL_NAMESPACE="open_notebook" \
  -e SURREAL_DATABASE="production" \
  lfnovo/open_notebook:v1-latest-single

Access at: http://localhost:8502

🌐 Remote Server Setup

Use this for servers, Raspberry Pi, NAS, Proxmox, or any remote machine.

mkdir open-notebook && cd open-notebook

docker run -d \
  --name open-notebook \
  -p 8502:8502 -p 5055:5055 \
  -v ./notebook_data:/app/data \
  -v ./surreal_data:/mydata \
  -e OPENAI_API_KEY=your_key_here \
  -e API_URL=http://YOUR_SERVER_IP:5055 \
  -e SURREAL_URL="ws://localhost:8000/rpc" \
  -e SURREAL_USER="root" \
  -e SURREAL_PASSWORD="root" \
  -e SURREAL_NAMESPACE="open_notebook" \
  -e SURREAL_DATABASE="production" \
  lfnovo/open_notebook:v1-latest-single

Replace YOUR_SERVER_IP with your server's IP (e.g., 192.168.1.100) or domain

Access at: http://YOUR_SERVER_IP:8502

⚠️ Critical Setup Notes:

Both ports are required:

  • Port 8502: Web interface (what you see in your browser)
  • Port 5055: API backend (required for the app to function)

API_URL must match how YOU access the server:

  • Access via http://192.168.1.100:8502 → set API_URL=http://192.168.1.100:5055
  • Access via http://myserver.local:8502 → set API_URL=http://myserver.local:5055
  • Don't use localhost for remote servers - it won't work from other devices!

Create a docker-compose.yml file:

services:
  open_notebook:
    image: lfnovo/open_notebook:v1-latest-single
    # Or use: ghcr.io/lfnovo/open-notebook:v1-latest-single
    ports:
      - "8502:8502"  # Web UI
      - "5055:5055"  # API (required!)
    environment:
      - OPENAI_API_KEY=your_key_here
      # For remote access, uncomment and set your server IP/domain:
      # - API_URL=http://192.168.1.100:5055
      # Database connection (required for single-container)
      - SURREAL_URL=ws://localhost:8000/rpc
      - SURREAL_USER=root
      - SURREAL_PASSWORD=root
      - SURREAL_NAMESPACE=open_notebook
      - SURREAL_DATABASE=production
    volumes:
      - ./notebook_data:/app/data
      - ./surreal_data:/mydata
    restart: always

Start with: docker compose up -d

What gets created:

open-notebook/
├── docker-compose.yml # Your configuration
├── notebook_data/     # Your notebooks and research content
└── surreal_data/      # Database files

🆘 Quick Troubleshooting

Problem Solution
"Unable to connect to server" Set API_URL environment variable to match how you access the server (see remote setup above)
Blank page or errors Ensure BOTH ports (8502 and 5055) are exposed in your docker command
Works on server but not from other computers Don't use localhost in API_URL - use your server's actual IP address
"404" or "config endpoint" errors Don't add /api to API_URL - use just http://your-ip:5055
Still having issues? Check our 5-minute troubleshooting guide or join Discord

How Open Notebook Works

┌─────────────────────────────────────────────────────────┐
│  Your Browser                                           │
│  Access: http://your-server-ip:8502                     │
└────────────────┬────────────────────────────────────────┘
                 │
                 ▼
         ┌───────────────┐
         │   Port 8502   │  ← Next.js Frontend (what you see)
         │   Frontend    │    Also proxies API requests internally!
         └───────┬───────┘
                 │ proxies /api/* requests ↓
                 ▼
         ┌───────────────┐
         │   Port 5055   │  ← FastAPI Backend (handles requests)
         │     API       │
         └───────┬───────┘
                 │
                 ▼
         ┌───────────────┐
         │   SurrealDB   │  ← Database (internal, auto-configured)
         │   (Port 8000) │
         └───────────────┘

Key Points:

  • v1.1+: Next.js automatically proxies /api/* requests to the backend, simplifying reverse proxy setup
  • Your browser loads the frontend from port 8502
  • The frontend needs to know where to find the API - when accessing remotely, set: API_URL=http://your-server-ip:5055
  • Behind reverse proxy? You only need to proxy to port 8502 now! See Reverse Proxy Guide

Star History

Star History Chart

🛠️ Full Installation

For development or customization:

git clone https://github.com/lfnovo/open-notebook
cd open-notebook
make start-all

📖 Need Help?

Provider Support Matrix

Thanks to the Esperanto library, we support this providers out of the box!

Provider LLM Support Embedding Support Speech-to-Text Text-to-Speech
OpenAI
Anthropic
Groq
Google (GenAI)
Vertex AI
Ollama
Perplexity
ElevenLabs
Azure OpenAI
Mistral
DeepSeek
Voyage
xAI
OpenRouter
OpenAI Compatible*

*Supports LM Studio and any OpenAI-compatible endpoint

Key Features

Core Capabilities

  • 🔒 Privacy-First: Your data stays under your control - no cloud dependencies
  • 🎯 Multi-Notebook Organization: Manage multiple research projects seamlessly
  • 📚 Universal Content Support: PDFs, videos, audio, web pages, Office docs, and more
  • 🤖 Multi-Model AI Support: 16+ providers including OpenAI, Anthropic, Ollama, Google, LM Studio, and more
  • 🎙️ Professional Podcast Generation: Advanced multi-speaker podcasts with Episode Profiles
  • 🔍 Intelligent Search: Full-text and vector search across all your content
  • 💬 Context-Aware Chat: AI conversations powered by your research materials
  • 📝 AI-Assisted Notes: Generate insights or write notes manually

Advanced Features

  • Reasoning Model Support: Full support for thinking models like DeepSeek-R1 and Qwen3
  • 🔧 Content Transformations: Powerful customizable actions to summarize and extract insights
  • 🌐 Comprehensive REST API: Full programmatic access for custom integrations API Docs
  • 🔐 Optional Password Protection: Secure public deployments with authentication
  • 📊 Fine-Grained Context Control: Choose exactly what to share with AI models
  • 📎 Citations: Get answers with proper source citations

Three-Column Interface

  1. Sources: Manage all your research materials
  2. Notes: Create manual or AI-generated notes
  3. Chat: Converse with AI using your content as context

Check out our podcast sample

📚 Documentation

Getting Started

User Guide

Advanced Topics

(back to top)

🗺️ Roadmap

Upcoming Features

  • Live Front-End Updates: Real-time UI updates for smoother experience
  • Async Processing: Faster UI through asynchronous content processing
  • Cross-Notebook Sources: Reuse research materials across projects
  • Bookmark Integration: Connect with your favorite bookmarking apps

Recently Completed

  • Next.js Frontend: Modern React-based frontend with improved performance
  • Comprehensive REST API: Full programmatic access to all functionality
  • Multi-Model Support: 16+ AI providers including OpenAI, Anthropic, Ollama, LM Studio
  • Advanced Podcast Generator: Professional multi-speaker podcasts with Episode Profiles
  • Content Transformations: Powerful customizable actions for content processing
  • Enhanced Citations: Improved layout and finer control for source citations
  • Multiple Chat Sessions: Manage different conversations within notebooks

See the open issues for a full list of proposed features and known issues.

(back to top)

🤝 Community & Contributing

Join the Community

  • 💬 Discord Server - Get help, share ideas, and connect with other users
  • 🐛 GitHub Issues - Report bugs and request features
  • Star this repo - Show your support and help others discover Open Notebook

Contributing

We welcome contributions! We're especially looking for help with:

  • Frontend Development: Help improve our modern Next.js/React UI
  • Testing & Bug Fixes: Make Open Notebook more robust
  • Feature Development: Build the coolest research tool together
  • Documentation: Improve guides and tutorials

Current Tech Stack: Python, FastAPI, Next.js, React, SurrealDB Future Roadmap: Real-time updates, enhanced async processing

See our Contributing Guide for detailed information on how to get started.

(back to top)

📄 License

Open Notebook is MIT licensed. See the LICENSE file for details.

📞 Contact

Luis Novo - @lfnovo

Community Support:

🙏 Acknowledgments

Open Notebook is built on the shoulders of amazing open-source projects:

(back to top)