- New user-guide page "Content Processing Engines": document vs URL engines
(auto/docling/simple and auto/firecrawl/jina/crawl4ai/simple), the auto
fallback chain, the OCR toggle, and a troubleshooting quick reference.
- adding-sources: note image/OCR support, Reddit URLs, YouTube live/shorts,
JavaScript-heavy-site guidance, and the new immediate "unsupported file
type" rejection (no long hang); link the engines page.
- environment-reference + advanced: document FIRECRAWL_API_URL,
CCORE_FIRECRAWL_PROXY, CCORE_FIRECRAWL_WAIT_FOR and CRAWL4AI_API_URL
(closes#602 — these pass straight through to content-core via CCORE_*).
- ADR-002: addendum on the content-core 2.x upgrade and its AGPL->MIT
licensing improvement.
Closes#1105Closes#602
* feat(sources): add Crawl4AI URL engine and honor persisted engine settings (#432)
- Add "crawl4ai" as a selectable URL processing engine (domain Literal,
settings API validation, SettingsForm select, and label across all 14
locales; urlHelp updated in en-US to describe the new fallback chain).
- The source graph now loads the persisted ContentSettings and passes the
document/URL engine choices to ContentCoreConfig. Previously it built a
hard-coded ContentSettings with "auto" engines, so a user's selection in
Settings never took effect. Falls back to defaults if settings can't load.
- Crawl4AI Docker mode is driven by content-core's native CRAWL4AI_API_URL
env var (documented separately under #1105).
Part of #939.
* fix(432): bundle Crawl4AI runtime + address review
- Bundle the Crawl4AI runtime so its local, no-API-key mode works out of
the box: depend on content-core[crawl4ai] and install the Chromium
browser via playwright in the Docker runtime-base (both image variants).
Footprint is modest (no torch/transformers/CUDA); image grows ~300 MB
from Chromium + system libs.
- Preserve the server-side traceback when persisted content settings fail
to load (logger.opt(exception=True)) instead of only the message.
- Reset the ContentSettings singleton between domain tests (clear_instance)
so a non-default value can't leak into neighboring tests.
Addresses review on #432.
* i18n(432): translate urlHelp Crawl4AI description across all 13 non-en locales
The Crawl4AI engine label was already localized; this brings the URL-engine
'help me choose' text in line with en-US in every locale — describing
Crawl4AI (local JS rendering, no API key) and its place in the auto
fallback chain (Firecrawl -> Jina -> Crawl4AI -> simple).
Unsupported files used to enqueue a background job that failed and then
burned the full 15-attempt retry budget (~1h) before showing a generic
"Failed" with no actionable detail.
- Add a pre-flight `_assert_file_supported()` using content-core 2.x's
header-only `check_file_support()` (same routing as real extraction) in
the upload branch of `_build_content_state`, before any job is enqueued
and before the source record is created.
- Guard the source-retry endpoint the same way. Unexpected check errors
(e.g. file removed before a retry) fall through to normal extraction.
- Map `UnsupportedTypeException` to `415 Unsupported Media Type` via a
dedicated global handler (previously fell through to the base
OpenNotebookError handler's 500). The message names the detected type.
Part of #939.
* chore(sources): upgrade to content-core 2.x
Bump content-core 1.14.x -> 2.0.4 and adapt the source graph to the new
keyword-only extract_content API.
- extract_content is now keyword-only; engine/model overrides move to
ContentCoreConfig (audio model still sourced from Default Models).
- ProcessSourceState was removed; the graph now consumes ExtractionOutput,
which no longer echoes url/file_path back — carry those from the input
state into the saved Asset.
- content-core 2.x no longer deletes the uploaded source file, so honor
delete_source on our side after a successful extraction.
- drop the obsolete output_format param (markdown is the default).
pymupdf (AGPL) is replaced transitively by pdfplumber (MIT); moviepy is
gone (direct ffmpeg), which fixes MP3-with-chapters audio.
Part of #939.
* docs(changelog): note content-core 2.x upgrade (#1103)
* fix(sources): wire YouTube transcript language preferences into ContentCoreConfig
content-core's default youtube_languages is only en/es/pt. Pass the broader
list Open Notebook has always intended so non-English videos still resolve a
transcript. (cubic P2 on #1116)
Since #1112 dropped the legacy provider/model strings, EpisodeCard's outline,
transcript and speaker rows showed "— / —" for new episodes. The API now
resolves the snapshot's model references (outline_llm/transcript_llm/voice_model)
to provider/name display fields at serialization time, batched into a single
query per request via Model.get_display_info_for_ids so listing episodes never
does a per-row model lookup. The card falls back to the legacy snapshot strings
for old episodes and degrades to a dash when a referenced model was deleted.
Closes#1114
Migration 22 best-effort maps profiles whose outline_llm/transcript_llm/
voice_model references are still empty to existing model records
(provider + name + type, no auto-create), clears the legacy values and
drops the 6 columns. The startup data migration that retried this
mapping on every boot (open_notebook/podcasts/migration.py) is deleted
along with its api/main.py lifespan hook; the legacy fields are removed
from the Pydantic models, API schemas and frontend types/panels.
Accepted trade-off: profiles whose mapping never converged lose the
legacy strings and stay unresolved - they were already non-functional
and the UI already flags them; the user re-picks models once.
* refactor(podcasts): store audio paths relative to PODCASTS_FOLDER
audio_file previously stored the absolute path returned by
podcast-creator (sometimes as a file:// URI), which kept path traversal
representable in the DB and broke playback when DATA_FOLDER moved.
- New single choke point open_notebook/podcasts/audio_paths.py:
to_relative_audio_path() validates at write time (the DB can never
hold an absolute or root-escaping value) and
resolve_contained_audio_path() joins + resolves + contains at read
time, replacing the per-router guards from #1018. Absolute/file://
legacy values are treated as invalid (same 403/404 as before).
- Generation command stores the relative form; in-band "ERROR: ..."
values from podcast-creator now fail the job with the real error.
- build_episode_output_dir() builds from PODCASTS_FOLDER so the write
root and the validation root cannot drift.
- Migration 21 converts legacy rows under the known roots (plain
file:/// URIs, /app/data/podcasts/, /data/podcasts/,
./data/podcasts/, data/podcasts/) via a single-strip IF/ELSE chain;
rows under other roots stay untouched by design.
Closes#1030
* test: use the tolerant migration-count assertion pattern
Per review: assert >= 21 with up/down parity and a distinctive-SQL check on
index 20, matching the convention from the migration-19/20 tests, so the next
migration doesn't trip an unrelated test file.
* fix(podcasts): reference speaker profiles by record ID instead of name
episode_profile.speaker_config now stores a record<speaker_profile>
reference (migration 20 converts existing rows; orphaned names become
null). The generate API keeps accepting the speaker profile by name and
resolves it to a record ID at the boundary.
Closes#630
* fix(podcasts): survive orphaned speaker references in podcast-creator config
Review fixes: rewrite record IDs back to speaker names (and drop orphaned
profiles) when building podcast-creator's episode config so one orphaned
profile can't fail validation for every generation; clearer error for
dangling references; resolve a single speaker name on single-record
endpoints instead of fetching the whole speaker table; accurate wording
about record links not preventing dangling references.
* fix(podcasts): submit the speaker profile record ID from the generate dialog
Submitting the cached display name made generation fail if the speaker
profile was renamed while the dialog was open - the record ID is stable
and the API resolves either form.
* refactor(frontend): migrate SourceDetailContent to the useSource hook
Fetch the source through the shared useSource React Query hook instead of
an imperative fetchSource with local loading/loadError state, matching the
insight/note dialogs (404 -> ContentUnavailable not-found, other errors ->
error, never-retry-404 from the global query client).
- Title edits and deletes now go through useUpdateSource/useDeleteSource,
so source lists are invalidated and a deleted source can't be served
from the query cache when reopening the dialog.
- A failed background refetch (window focus) no longer replaces a rendered
source: the unavailable state only shows when there is no data.
- The key={sourceId} remounts on SourceDialog and sources/[id]/page.tsx
are removed; the component keys an inner subcomponent by sourceId itself
so all per-source UI state (tab, transient flags, insight selection)
still resets on navigation without a parent contract.
Closes#1106
* fix(frontend): show not-found over stale cached data when a source refetch 404s
React Query retains the previous data when a refetch fails, so gating the
unavailable state on !source alone let a deleted-but-cached source render
after a background 404 (dangling citation reopened within the cache
window). A definitive 404 now always yields ContentUnavailable, while a
transient refetch error over good cached data still keeps the source
rendered. Adds a regression test seeding a stale cache entry whose refetch
rejects with 404.
The settings frontend now fetches the provider list from the backend
registry endpoint (session-cached react-query hook useProviders())
instead of keeping its own hardcoded copies of provider names, display
names, modalities and docs URLs in lib/providers.tsx.
- New api module (lib/api/providers.ts) + hook (lib/hooks/use-providers.ts)
with staleTime: Infinity — the list only changes on deploy.
- lib/providers.tsx reduced to modality presentation (icon/color/label)
behind fallback-safe helpers, so an unknown modality from a future
provider still renders instead of breaking.
- The backend registry declaration order is the display order (verified
identical to the old curated ALL_PROVIDERS order; endpoint test now
pins order, not just set-equality).
- api-keys page gains loading/error states for the provider fetch; new
i18n keys added to all 14 locales.
- Deleted the regex-based frontend/backend sync test; the
SupportedProvider Literal test remains the backend guarantee.
- Renamed the dead useProviders() in use-models.ts (availability
endpoint) to useProviderAvailability() to avoid a name collision.
- Updated stale docs/comments (AGENTS.md, credentials.md, cubic.yaml,
provider_registry.py, api/models.py) that still described the frontend
table as a manual sync point.
Closes#1082
Clicking a chat/Ask citation that references a deleted source, insight,
or note previously behaved inconsistently: a generic red error for
sources, a silently blank dialog for insights, and an empty ghost editor
(inviting edits of a deleted note) for notes.
- Add a shared ContentUnavailable component (built on EmptyState) with
two variants: 'not-found' (404 — deleted / no longer exists) and
'error' (transient load failure), used by all three dialogs
- SourceInsightDialog and NoteEditorDialog now consume isError from
their hooks instead of falling back to the blank stub; the note
editor (and its save path) is fully gated behind the fetch result
- SourceDetailContent distinguishes 404 from other errors and renders
the shared state; keyed by sourceId so state resets per source
- Global react-query retry now skips 404s (retrying cannot resurrect a
deleted item); added isNotFoundError helper and reused it for the
file-download 404 check
- New common.contentUnavailable.* strings in all 14 locales; removed
now-unused sources.notFound and sources.loadFailed keys
- Component tests for the three dialogs' not-found/error states,
including the note-editor gate
Closes#455
source_insight was the only content table without created/updated field
definitions. Since the table is SCHEMAFULL and insights are created via a
raw CREATE (create_insight command), SurrealDB silently dropped any
timestamps - rows genuinely held NONE - and the API then wrapped them in
str(), so clients received the literal string "None".
- Migration 19 defines created/updated on source_insight with the same
time::now() defaults used by source, note and notebook, so new insights
are stamped at creation. Existing rows are left untouched (no backfill).
- SourceInsightResponse.created/updated are now Optional[str]; both
routers emit an ISO 8601 string when the timestamp is present and null
when it is absent (legacy rows), never "None".
- Frontend types updated to string | null accordingly.
- New tests cover the migration definition/registration and the API
serialization (absent -> null, present -> ISO string).
Note: Python-side stamping alone was not viable - the table is SCHEMAFULL,
so undefined fields written by the client are silently dropped on
SurrealDB 2.x (verified against a live instance), hence the schema
migration mirroring the other tables.
Closes#1045
The dev toolchain was split across two drifted lists: the legacy
[project.optional-dependencies].dev (mypy, notebook packages, old pins) and
[dependency-groups].dev (pytest plugins, current pins). Plain 'uv sync'
installs only the group, so the documented 'uv run python -m mypy .' failed
on a fresh clone while CI's typecheck job papered over it with --extra dev.
Merge everything into [dependency-groups].dev, move Jupyter-only packages
(ipykernel, ipywidgets) to a non-default 'notebooks' group, drop the legacy
extras list, and switch the CI typecheck job to plain 'uv sync'.
Documents the policy decided while reviewing PR #1085: one migration per PR
that needs one, numbers allocated in merge order, never consolidate after a
migration lands on main (v1-dev images apply migrations the moment they hit
main, so post-hoc squashing would desync _sbl_migrations for dev users).
Cross-referenced from the Database Migration playbook; also adds the missing
ADR-005 row to the decisions index.
PUT /api/models/defaults used 'is not None' guards, so an explicit null
sent to clear a default was silently ignored — the old value survived
while the client saw success (same anti-pattern fixed for credentials
in #1046). The handler is now keyed on field presence (model_fields_set):
absent keeps the current value, explicit null clears it. The required
defaults (chat, embedding) reject null with a 400.
In the UI, the optional default-model selects (transformation, tools,
large context, TTS, STT) now offer a clear option — labeled 'Use
fallback (chat default)' for transformation/tools, which fall back to
the chat default when unset, and 'None' otherwise — and display that
state when no model is assigned. The transformation default is no
longer marked required, matching the backend fallback behavior.
Closes#1091
Compose interpolates ${SURREAL_PASSWORD} into the scalar command string
before splitting it into arguments, so a password containing spaces became
multiple argv entries and SurrealDB failed to start. The list (exec) form
keeps each interpolated value in exactly one argument slot.
Also resyncs the mirrored compose snippets in
docs/1-INSTALLATION/docker-compose.md and README.md with the shipped file
(both had drifted: no credential interpolation, SurrealDB port published on
all interfaces) and documents the optional .env credential override for
readers who create the file manually.
Closes#1093
Container runtimes inject HOSTNAME=<container/pod hostname> at runtime; under
Podman pods it resolves to 127.0.1.1, so Next.js standalone bound to the wrong
address and the UI became unreachable. Setting HOSTNAME explicitly in the
supervisord command (with a FRONTEND_BIND_HOST override knob) beats any
injected value. Drops the now-dead ENV HOSTNAME from the Dockerfile and
updates the environment reference and reverse-proxy docs.
Closes#994
* docs: stop teaching 0.0.0.0 SurrealDB port exposure in setup snippets
Bind port 8000 to 127.0.0.1 in every compose and docker run snippet
(README, quick starts, installation, configuration and development docs,
and the examples/docker-compose-*.yml files), matching the shipped
docker-compose.yml from #1025. Drop the redundant --bind 0.0.0.0:8000
from containerized surreal start commands (it is the in-container
default) and add !override to docker-compose.override.yml.example so the
opt-in re-publish actually replaces the base port binding instead of
colliding with it. Docs that discuss reaching the database from another
machine now point at the override example plus a firewall/SSH-tunnel
note.
Closes#1034
* docs: parameterize creds in manual compose example, note Compose version for !override
The command always re-derived the speaker from
episode_profile.speaker_config, silently ignoring the speaker_profile
value accepted by POST /api/podcasts/generate. Resolve the explicitly
requested profile when provided and fall back to the episode profile's
speaker_config otherwise. speaker_profile is now Optional on
PodcastGenerationInput so the fallback is a real contract.
Closes#1044
- Known Gotchas: dev-machine port ownership before starting/killing
services, validating error-path checklist items against fallback code,
and the dev-DB record-count diff that catches test write leaks
- Note that the released label must be recreated if dropped from the
taxonomy while the process still references it
TransformationsList only rendered TransformationEditorDialog in the
non-empty branch, so after deleting every transformation the empty
state's 'New Transformation' button set editorOpen=true but there was
no dialog mounted to open — the click appeared to do nothing.
Render the dialog in both branches so the create flow works again from
the empty state.
The valid-provider gate test posted a real credential through the route;
with a developer .env loaded (SURREAL_URL + encryption key set) every
pytest run persisted a 'Test' openai credential to the live dev database.
Force the encryption-key gate to fail so the request proves validation
clearance without ever reaching persistence.
Consolidate the three copies of context assembly into
open_notebook/utils/context_builder.py:
- POST /api/chat/context now delegates to build_notebook_context()
(same request/response shapes, same string-matching config semantics)
- The source-chat graph now calls build_source_context() instead of the
495-line generalized ContextBuilder class, which had exactly one
caller and whose notebook/notes/priority-config flexibility was dead
- POST /api/notebooks/{notebook_id}/context removed: it duplicated
/api/chat/context with a slightly different envelope and had zero
callers (frontend, docs, tests)
Behavior is pinned by new characterization tests written before the
refactor (tests/test_context_endpoint_characterization.py) plus unit
tests for build_source_context.
The routers wrapped endpoint bodies in a broad 'except Exception' that
re-raised everything as HTTPException(500), intercepting the typed
open_notebook.exceptions hierarchy before the global handlers in
api/main.py could map it to its documented status code (NotFoundError
-> 404, InvalidInputError -> 400, ConfigurationError -> 422,
RateLimitError -> 429, NetworkError/ExternalServiceError -> 502).
Every endpoint-level handler chain across the 18 affected routers now
re-raises HTTPException and OpenNotebookError before the final generic
arm, which keeps catching untyped exceptions and returning a sanitized
500 (the no-raw-exception-text guarantee is unchanged and covered by
tests/test_error_message_sanitization.py). Nested best-effort blocks
(fallbacks, SSE error events, per-item loops) are deliberately left
untouched. In create_source and _create_source_async_path the new
re-raise arms preserve the uploaded-file / orphaned-source cleanup.
Characterization tests that pinned the old wrapped-500 behavior for
source-chat sessions with a missing refers_to relation are updated to
the correct 404, and a new test module asserts per router that a
domain-layer ConfigurationError maps to 422 instead of 500.
- Rename useNotebookChat.ts/useSourceChat.ts to kebab-case, matching the
other 19 hook files
- Merge src/components/source/ into src/components/sources/ (no collisions)
- Extract the copy-pasted auth-storage localStorage parsing into a single
getAuthToken() helper (src/lib/auth-token.ts) used by the apiClient
interceptor and the SSE fetch paths
- Route non-streaming raw fetch calls through apiClient: podcast audio
blob download (EpisodeCard) and the auth-status check (auth-store).
SSE/streaming paths, config bootstrap and the login/checkAuth credential
probes deliberately keep raw fetch
- Fix 8 podcast toast descriptions that rendered a literal {name}
placeholder: normalize single-brace placeholders to i18next {{name}}
in all locales and pass the profile name from the mutation
response/variables
* ci: gate PRs on mypy, start ignore_errors burn-down
Add a backend-typecheck CI job running uv run python -m mypy . and bring
the repo-wide baseline from 197 errors to 0 so the gate blocks new type
errors from now on:
- enable the pydantic mypy plugin (resolves 138 false positives on
models whose fields have Field(None, ...) defaults)
- fix the remaining errors with real annotations; the only new type:
ignore comments cover a genuine langgraph typing limitation (partial
state dicts are valid at runtime but the overloads require the full
state type) and tests that intentionally pass invalid input
- start the ignore_errors burn-down: open_notebook.graphs.transformation,
open_notebook.graphs.ask and api.routers.models are now type-checked;
stale blocks for the deleted api.client and api.podcast_api_service
modules removed. Only open_notebook.domain.notebook remains exempt
(DB layer is migrating to surreal-basics)
- fix the mypy.ini header comment to describe what the config does
* test: use typing.get_args on the registry literal check
The registry test landed in parallel using Literal.__args__, which the
mypy gate in this branch rejects; align it with the get_args() idiom
used by the rest of the file.
* refactor(ai): single provider registry as the backend source of truth
Provider metadata (env vars, modalities, connection-test models,
OpenAI-compatible discovery URLs, display names, docs links) is now
defined once in open_notebook/ai/provider_registry.py. The existing
surfaces are derived from it, keeping every import and call-site shape
unchanged:
- api/credentials_service.py: PROVIDER_ENV_CONFIG, PROVIDER_MODALITIES
and the discovery url_map are built from the registry
- open_notebook/ai/connection_tester.py: TEST_MODELS derived
- open_notebook/ai/model_discovery.py: OPENAI_COMPAT_PROVIDERS built
from registry entries with a discovery URL (quirk hooks stay local)
The SupportedProvider Literal (typing, can't be built at runtime) and
the frontend provider tables remain manual copies; the cross-check
tests now assert registry keys == Literal == frontend list, plus
registry internal consistency and discovery-table coverage.
New GET /api/providers endpoint exposes the registry (name, display
name, modalities, docs_url, env-configured status) so clients can stop
hardcoding provider lists (frontend adoption is a follow-up).
Docs updated: open_notebook/AGENTS.md and docs/7-DEVELOPMENT/credentials.md
now describe the registry instead of the four-place sync rule.
* refactor(ai): address review findings on the provider registry
- Build PROVIDERS via _build_registry(), which raises on a duplicate
provider name at import time instead of silently dropping the earlier
spec (dict-comprehension behavior); regression test added
- Pin the exact OpenAI-compatible provider -> discovery URL mapping in
a test so a registry edit can't silently drop or misassign a URL
- Give TEST_MODELS a real type annotation
(Dict[str, Tuple[Optional[str], str]]) instead of bare dict
Convert every t('key').replace('{ph}', value) call site (77 t() calls
across 37 files) to native i18next interpolation t('key', { ph: value }),
and switch the corresponding locale placeholders from {ph} to {{ph}} in
all 14 locales (842 strings).
- count params are passed as numbers, enabling i18next plural resolution
(podcasts.usedByCount now uses the existing _one/_other forms instead
of a manual ternary)
- podcasts.tokens/chars keep their pre-formatted display value
(formatNumber) under a renamed {{value}} placeholder, since i18next
types reserve count for numbers
- escapeValue: false was already set in src/lib/i18n.ts (React escapes
at render), so rendered output is unchanged
- unused-key test now strips plural suffixes before matching, and a new
interpolation test covers variables, plurals and non-escaping
- Extract _source_to_response() shared builder (SourceResponse was
hand-rolled 5x with the same nested AssetModel ternary)
- Extract _cleanup_uploaded_file() (the unlink ritual was pasted 6x
inside create_source); inner cleanups were redundant with the outer
exception handlers, which always run for the same failures
- Split create_source into _build_content_state() (type validation +
SSRF/LFI guards) and _create_source_async_path() /
_create_source_sync_path()
- Unify the paginated list query (with/without notebook filter differed
only in the FROM clause and bound params)
Pure structural refactor: same status codes, error messages and
response shapes. Security checks (atomic filename claim via
touch(exist_ok=False), path-traversal containment, SSRF/LFI guards)
preserved verbatim, with their comments.
Remove langchain-community and langchain-deepseek (zero imports; DeepSeek
and xAI go through esperanto's OpenAI-compatible path, which uses
langchain-openai). Declare langchain-core and langchain-text-splitters
explicitly (directly imported, previously only transitive — the
text-splitters import actually broke once langchain-community left the
tree). Add upper bounds to the whole langchain/langgraph family and
document why the provider packages must stay: esperanto's to_langchain()
imports them dynamically and only declares them as optional extras.
* test(api): characterize shared chat/source-chat router behaviors
* refactor(api): extract shared session and message helpers for chat routers
* refactor(api): mark intentionally unused source id unpacks
The session handlers only need the verified session; the source-level
verification happens inside get_verified_source_session. Underscore the
unused binding in get/update/delete to make that explicit.
- embed_note / embed_insight / embed_source now share a single
load->embed->write runner (_embed_record) with one common
ValueError/Exception epilogue; note and insight additionally share
_embed_markdown_record for the identical load/validate/embed/UPSERT body
- rebuild_embeddings submits jobs for sources/notes/insights through one
_submit_embedding_jobs helper instead of three copy-pasted loops
- the four embed-family commands reuse one EMBED_RETRY_CONFIG dict, with a
NOTE marking that stop_on can never trigger today (commands catch
ValueError internally) - preserved as-is for a future error-handling PR
- full_model_dump() was copy-pasted in three command files but only used by
podcast_commands: moved to open_notebook/utils/model_utils.py, imported
where used, dead copies in embedding/source commands removed
Behavior-identical: same outputs, success=False paths, log messages and
retry configuration. 403 tests pass; ruff and mypy clean (no new errors).
Collapse the eight structurally identical OpenAI-compatible provider
discovery functions (openai, groq, mistral, deepseek, xai, openrouter,
dashscope, minimax) into one generic discover_openai_compatible_provider()
driven by an OPENAI_COMPAT_PROVIDERS spec table. Per-provider quirks
(Mistral capability flags, OpenRouter fixed type + description) are kept
as optional hooks on the spec. Module-level function names and
PROVIDER_DISCOVERY_FUNCTIONS keys are preserved.
Replace the stale hardcoded Anthropic model list (claude-3 era) with
real discovery via GET https://api.anthropic.com/v1/models (paginated,
x-api-key + anthropic-version headers) — the comment claiming Anthropic
has no listing API was wrong. Keep a refreshed static fallback
(ANTHROPIC_FALLBACK_MODELS, current Claude 4.x/5 aliases) for when the
API call fails, and reuse the same fetch-with-fallback in the
credential-based discovery path in api/credentials_service.py, which
carried a second copy of the stale list.
Add tests covering the table-driven path, provider quirks, Anthropic
pagination/headers, and the fallback behavior (all HTTP mocked).
Extract ContentSelectionPanel and shared selection helpers into their
own modules, deduplicate the sources/notes context-config reduction
into selectionsToContextConfigs, delete the obsolete translation
cache (plain react-i18next t() now), debounce the token/char counter
with a stale-response guard, and close the dialog when the episode
refetch resolves instead of after a fixed 500ms timer.
Both published image variants are now built from one Dockerfile with
shared stages, so deploy fixes no longer have to be applied twice:
- regular (multi-container): default build / --target runtime
- single-container (app + SurrealDB): --target single
Dockerfile.single and supervisord.single.conf are removed. The single
image appends supervisord.surrealdb.conf (the SurrealDB program block)
to the shared supervisord.conf at build time. CI workflows, Makefile
targets and the single-container compose example now build via
--target; published image names and tags are unchanged.
- Derive TranslationShape from the en-US locale and add 'satisfies
TranslationShape' to the other 13 locales so missing/extra i18n keys
fail tsc at compile time (previously only the runtime parity test)
- Remove unused dependencies next-themes and @monaco-editor/react
(zero imports in src/; theming is the hand-rolled zustand theme-store)
- Fix frontend/AGENTS.md drift: 14 locales (not 7), dark mode mechanism
is the zustand theme-store setting the 'dark' class (not next-themes)
Remove F401 (unused imports), F841 (unused local variables) and E722
(bare except) from the ruff ignore list; the legacy Streamlit-era code
that motivated ignoring them is gone.
Fallout fixed: 10 unused imports removed (none were load-bearing
re-exports or side-effect imports), 2 unused mock bindings in tests
dropped, and a now-empty TYPE_CHECKING block in
open_notebook/utils/embedding.py cleaned up. No bare excepts remained
in the codebase.
Add a math formatting hint to the chat, source chat, Ask and
transformation prompts: emit display math as $$...$$ and inline math
as $...$ so formulas render via KaTeX, reserving fenced latex code
blocks for when the user explicitly asks for the LaTeX source itself.
Closes#1051
The embed_single_item, embed_chunk and vectorize_source command handlers
existed only so jobs queued by a pre-1.6 version could drain after an
upgrade; any worker restarted on 1.6+ has no such jobs. Remove them, their
input/output models and their tests.
Also drop dead tooling config from pyproject.toml: the [tool.mypy] block
(mypy.ini takes precedence and is the real config) and the Streamlit-era
ruff per-file-ignores for app_home.py and pages/**, which no longer exist.
Delete api/client.py (synchronous httpx client calling the app's own
FastAPI server, a leftover from the removed Streamlit UI) and the 13
api/*_service.py wrappers around it. None of these files had any
importer in routers, commands or tests, verified by grepping the whole
repo for each module name.
Also remove commands/example_commands.py (process_text/analyze_data
demo commands from the surreal-commands README) and its export from
commands/__init__.py, and update remaining docstring examples to
reference the real generate_podcast command.
The real services (command_service, credentials_service,
podcast_service) are untouched.
These 12 QA screenshots and a local debug query log were accidentally
committed to the repo root in PR #978. Adds .gitignore entries for
screenshot-*.png and history.txt to prevent recurrence.
Captures the process designed and executed for v1.11.0 so every future
release reproduces it:
- .github/RELEASE_PROCESS.md v2: changelog audit, risk-based test
matrix (buckets A/B/C), the Docker image gate, fix-loop re-test
policy, CI-based publishing path, communication structure with a
mandatory credits section, retro, and the gotchas that cost
iterations this cycle
- ADR-005: why releases now pass a risk-based confidence process gated
on the real image, with the v1.11.0 evidence (bugs the unit suite
could not catch: SEARCH-index ORDER BY 500, credential clear no-op)
- scripts/release-test/: the harness built during v1.11.0 —
fresh-install + upgrade gate (release-image-test.sh) and the
browsable RC stack with optional dev-data copy (rc-stack.sh), plus
compose/nginx encoding the API_URL, host.docker.internal and
SurrealDB import learnings
- make release-test / release-stack / release-stack-down targets
- Bump version to 1.11.0
- Date the Unreleased changelog section as [1.11.0] - 2026-07-11
- Add the two entries from release testing: Pillow 12.3.0 security
bump (#1041) and the credential field-clearing fix (#1046)
