Docling and local Crawl4AI are now opt-in, installed on first container startup
via OPEN_NOTEBOOK_ENABLE_DOCLING / OPEN_NOTEBOOK_ENABLE_CRAWL4AI, keeping the
default image lean (Crawl4AI un-bundled). Downloads cache on the /app/data
volume; installs are blocking with loud logs and degrade-don't-die on failure.
A GET /api/capabilities probe reports actual availability and the Settings UI
gates the Docling/Crawl4AI engines and OCR toggle (with env-var hints) until the
runtime is present. Docs folded from #1121 and rewritten to opt-in; ADR-007 added.
Closes#1122. Closes#1105.
* feat(sources): add Crawl4AI URL engine and honor persisted engine settings (#432)
- Add "crawl4ai" as a selectable URL processing engine (domain Literal,
settings API validation, SettingsForm select, and label across all 14
locales; urlHelp updated in en-US to describe the new fallback chain).
- The source graph now loads the persisted ContentSettings and passes the
document/URL engine choices to ContentCoreConfig. Previously it built a
hard-coded ContentSettings with "auto" engines, so a user's selection in
Settings never took effect. Falls back to defaults if settings can't load.
- Crawl4AI Docker mode is driven by content-core's native CRAWL4AI_API_URL
env var (documented separately under #1105).
Part of #939.
* fix(432): bundle Crawl4AI runtime + address review
- Bundle the Crawl4AI runtime so its local, no-API-key mode works out of
the box: depend on content-core[crawl4ai] and install the Chromium
browser via playwright in the Docker runtime-base (both image variants).
Footprint is modest (no torch/transformers/CUDA); image grows ~300 MB
from Chromium + system libs.
- Preserve the server-side traceback when persisted content settings fail
to load (logger.opt(exception=True)) instead of only the message.
- Reset the ContentSettings singleton between domain tests (clear_instance)
so a non-default value can't leak into neighboring tests.
Addresses review on #432.
* i18n(432): translate urlHelp Crawl4AI description across all 13 non-en locales
The Crawl4AI engine label was already localized; this brings the URL-engine
'help me choose' text in line with en-US in every locale — describing
Crawl4AI (local JS rendering, no API key) and its place in the auto
fallback chain (Firecrawl -> Jina -> Crawl4AI -> simple).
Container runtimes inject HOSTNAME=<container/pod hostname> at runtime; under
Podman pods it resolves to 127.0.1.1, so Next.js standalone bound to the wrong
address and the UI became unreachable. Setting HOSTNAME explicitly in the
supervisord command (with a FRONTEND_BIND_HOST override knob) beats any
injected value. Drops the now-dead ENV HOSTNAME from the Dockerfile and
updates the environment reference and reverse-proxy docs.
Closes#994
Both published image variants are now built from one Dockerfile with
shared stages, so deploy fixes no longer have to be applied twice:
- regular (multi-container): default build / --target runtime
- single-container (app + SurrealDB): --target single
Dockerfile.single and supervisord.single.conf are removed. The single
image appends supervisord.surrealdb.conf (the SurrealDB program block)
to the shared supervisord.conf at build time. CI workflows, Makefile
targets and the single-container compose example now build via
--target; published image names and tags are unchanged.
The multi-arch build runs `npm ci` for the arm64 image under QEMU emulation,
which is slow and prone to transient `ECONNRESET` errors from the npm registry
(exit code 152) — this broke the Development Build after a recent merge while
the amd64 leg succeeded.
Add npm fetch-retry settings and wrap `npm ci` in a retry loop (up to 5
attempts) in both Dockerfile and Dockerfile.single so a single network blip no
longer fails the whole multi-arch build (incl. the release build).
In air-gapped / offline Docker deployments, tiktoken.get_encoding() tries
to download the encoding file from openaipublic.blob.core.windows.net.
When that request fails it raises a URLError / OSError — not an ImportError
— so the previous except clause silently missed it and the crash surfaced in
the UI.
Widened `except ImportError` to `except Exception` so all failures —
"not installed" and "network unreachable" — fall through to the word-count
fallback (words × 1.3). Added a loguru WARNING so operators can see when
the fallback is active.
TIKTOKEN_CACHE_DIR now reads from the environment with a blank-safe
fallback (`or` guard prevents os.makedirs("") on empty env var). This lets
Docker images redirect the cache to a path outside /app/data/ so user-data
volume mounts cannot shadow the pre-baked encoding.
Both images now pre-download the o200k_base encoding during the builder
stage (internet is available at build time) and copy it into the runtime
image at /app/tiktoken-cache. ENV TIKTOKEN_CACHE_DIR=/app/tiktoken-cache
is set in the runtime stage so no network call is ever needed at runtime.
Added test_token_count_network_error_fallback in tests/test_utils.py:
patches tiktoken.get_encoding with a URLError and asserts token_count()
returns a positive int instead of raising.
Fixes#264
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: force env file on api
* fix: Docker networking and permissions documentation
- Add HOSTNAME=0.0.0.0 as default in Dockerfiles for reverse proxy compatibility
- Document Linux extra_hosts requirement for host.docker.internal
- Add user: root to SurrealDB examples for bind mount permissions on Linux
- Update environment reference and reverse proxy docs
Fixes#483, fixes#485, fixes#409
Replace cross-env with a simple Node.js wrapper script that:
- Respects externally supplied PORT environment variable
- Falls back to 8502 if PORT is not set
- Works cross-platform without extra dependencies
- No runtime dependencies beyond Node.js itself
Changes:
- Add start-server.js wrapper script
- Update package.json to use wrapper
- Remove cross-env dependency
- Copy start-server.js in Dockerfile
This fixes both issues:
1. Preserves PORT fallback behavior (PORT can be overridden)
2. No extra runtime dependencies needed
Users reported "Unable to Connect to API Server" errors on startup
because the frontend started before the API finished initialization
(database migrations, etc.).
- Add wait-for-api.sh script that polls /health endpoint
- Update supervisord configs to use wait script instead of sleep 5
- Waits up to 5 minutes for API to be ready before starting frontend
- Applies to both single-container and multi-container deployments
Fixes#315
New front-end
Launch Chat API
Manage Sources
Enable re-embedding of all contents
Sources can be added without a notebook now
Improved settings
Enable model selector on all chats
Background processing for better experience
Dark mode
Improved Notes
Improved Docs:
- Remove all Streamlit references from documentation
- Update deployment guides with React frontend setup
- Fix Docker environment variables format (SURREAL_URL, SURREAL_PASSWORD)
- Update docker image tag from :latest to :v1-latest
- Change navigation references (Settings → Models to just Models)
- Update development setup to include frontend npm commands
- Add MIGRATION.md guide for users upgrading from Streamlit
- Update quick-start guide with correct environment variables
- Add port 5055 documentation for API access
- Update project structure to reflect frontend/ directory
- Remove outdated source-chat documentation files
Creates the API layer for Open Notebook
Creates a services API gateway for the Streamlit front-end
Migrates the SurrealDB SDK to the official one
Change all database calls to async
New podcast framework supporting multiple speaker configurations
Implement the surreal-commands library for async processing
Improve docker image and docker-compose configurations
