Captures the process designed and executed for v1.11.0 so every future
release reproduces it:
- .github/RELEASE_PROCESS.md v2: changelog audit, risk-based test
matrix (buckets A/B/C), the Docker image gate, fix-loop re-test
policy, CI-based publishing path, communication structure with a
mandatory credits section, retro, and the gotchas that cost
iterations this cycle
- ADR-005: why releases now pass a risk-based confidence process gated
on the real image, with the v1.11.0 evidence (bugs the unit suite
could not catch: SEARCH-index ORDER BY 500, credential clear no-op)
- scripts/release-test/: the harness built during v1.11.0 —
fresh-install + upgrade gate (release-image-test.sh) and the
browsable RC stack with optional dev-data copy (rc-stack.sh), plus
compose/nginx encoding the API_URL, host.docker.internal and
SurrealDB import learnings
- make release-test / release-stack / release-stack-down targets
- Bump version to 1.11.0
- Date the Unreleased changelog section as [1.11.0] - 2026-07-11
- Add the two entries from release testing: Pillow 12.3.0 security
bump (#1041) and the credential field-clearing fix (#1046)
* fix(frontend): actually clear credential fields when emptied in the edit dialog
Clearing base_url (or the Vertex project/location/credentials_path)
in the credential edit dialog silently did nothing: the submit handler
mapped an emptied field to `undefined`, JSON.stringify dropped the key
from the PUT body, and the backend's partial-update semantics kept the
old value — while the UI reported success.
Emptied fields are now sent as an explicit `null`, which the API
already accepts and persists as a cleared value (verified live).
UpdateCredentialRequest's nullable fields are typed accordingly.
Found in v1.11 release testing (pre-existing, not a release regression):
an Ollama credential with a stale IP in base_url could not be cleared
from the UI.
* fix(api): clear credential fields on explicit null, not just empty string
The update handler guarded every field with 'is not None', so a JSON
null sent to clear base_url (or endpoint/api_version/endpoint_*/
project/location/credentials_path/num_ctx) was silently skipped — the
old value survived while the client saw a 200. Combined with the
frontend bug fixed in the previous commit (emptied fields dropped from
the payload entirely), clearing a credential field was impossible from
the UI: during release testing an Ollama credential kept pointing at a
stale IP after being 'cleared', and only an empty-string PUT crafted by
hand actually cleared it.
Field updates are now keyed on presence in model_fields_set: absent
keeps the old value, explicit null or "" clears. name/modalities/
api_key keep their non-null guards (clearing those is not meaningful).
Regression tests cover null-clears, empty-string-clears, absent-keeps
and the Vertex credentials_path case.
* test(frontend): extract credential update payload builder and cover clear-on-empty
Addresses the cubic review: the frontend half of the fix had no
reproducing test. The edit dialog's payload construction now lives in
a pure buildCredentialUpdatePayload() (frontend/src/lib/
credential-update-payload.ts) used by the component, with tests that
fail on the original bug: an emptied base_url (and Vertex project/
location/credentials_path) must survive JSON.stringify as an explicit
null, unchanged fields must be omitted, and emptied num_ctx clears
via 0.
* fix(frontend): keep the note editor dialog inside the viewport with long content
Regression from #932: removing the editor's max-h-[500px] cap relied on
the flex-1/h-full parent chain, but DialogContent has no definite height
(only max-h-[90vh]) and isn't a flex container — so the form's h-full
resolved to auto, the flex-1 wrapper (min-height:auto) never shrank
below its content, and the inner overflow-y-auto never engaged. With
enough content the dialog grew past the viewport with no way to scroll
or reach the Save button.
DialogContent is now flex flex-col, the form takes flex-1 min-h-0, and
the editor wrapper gets min-h-0 — the standard flex pattern for
scroll-inside-dialog. Long notes now scroll inside the editor pane and
the dialog stays within 90vh.
Verified in-browser (Playwright, 200-line note): dialog height 1089px
within a 1210px viewport, footer visible, internal scroll active.
* fix(frontend): give the note dialog a definite height so the editor fills it when empty
With only max-h-[90vh], the dialog's height stayed content-based, so
the percentage chain (h-full -> .w-md-editor !h-full) resolved to auto
on an empty note and the typing area collapsed to ~90px inside a
420px-floored border box. h-[90vh] makes every level of the chain
definite: empty notes get a full-height editor (895px content area
verified via Playwright), long notes still scroll internally without
pushing past the viewport.
Two findings from v1.11 release testing:
- GET /api/sources?sort_by=title returned a 500. source.title carries a
SEARCH (BM25) index (idx_source_title, migration 1) and SurrealDB's
planner fails ORDER BY on such a column with 'No iterator has been
found'. The query now sorts by a computed alias
(string::lowercase(title OR '') AS title_sort), which sidesteps the
index and makes the sort case-insensitive as a bonus.
- POST /api/sources with an over-limit notebooks/transformations array
(or invalid JSON in either field) returned a raw 500.
parse_source_form_data() builds SourceCreate manually, so pydantic's
ValidationError never reached FastAPI's request-validation handler.
Both cases now surface as a clean 422 with a descriptive message.
Verified against a live SurrealDB v2 instance; regression tests added
for the ORDER BY alias, all six sort fields, and the 422 paths.
All 6 open Dependabot alerts (3 high, 3 moderate) are Pillow advisories
fixed in 12.1.1/12.2.0: PSD out-of-bounds writes, a FITS GZIP
decompression bomb, a PDF trailer parsing DoS, a font integer overflow
and a heap buffer overflow.
The only constraint holding Pillow at 11.x is moviepy 2.2.1's
`pillow<12` cap, pulled in via podcast-creator 0.12.0. moviepy only
uses PIL in its video modules, which the audio-only podcast pipeline
never imports, so a uv override-dependencies entry forces Pillow to
12.3.0. The override is documented inline and should be dropped once
podcast-creator publishes a release without moviepy (already removed
on its main branch).
Verified: podcast_creator/moviepy/PIL import cleanly, full test suite
passes (397 passed).
* fix: make the provider connection test resilient to model retirement
Two independent hardenings against the #970 class of breakage (a hard-coded
Gemini test model getting shut down by Google, which made testing a valid
key fail with a 404):
- Use Google's floating alias gemini-flash-latest for the Google/Vertex
test model instead of a dated id, so a retirement repoints it for us.
- Reframe the provider connection test around what an error actually
proves: only a rejected key (401), missing permissions (403), or an
unreachable endpoint are failures. Anything the provider returns after
authenticating - a rate limit, or a missing/retired/unsupported model -
still proves the credentials work, so it reports success. Previously this
relied on matching the literal phrase 'not found' + 'model', which a
differently-worded retirement/deprecation error slipped past.
Unifies the auth/network/rate-limit classification (previously duplicated
and divergent between connection_tester and credentials_service) into
shared helpers. The individual-model test keeps model-not-found as a
failure, since there a specific registered model really is broken.
Adds classification tests with realistic provider error strings.
* docs: add CHANGELOG entry for connection-test resilience fix
* fix: update Google model version in connection tester and tests
* fix: updated the gemini model lists that references deprecated models
* fix: update also documentation. This is more prone to not follow the maintainer's directives, so PTADL
* fix: forgot two references to gemini deprecated versions
* fix: use valid, longer-lived gemini model IDs
Several IDs the PR introduced don't exist or are near shutdown, verified
against Google's official model/deprecation pages (2026-07):
- connection test model gemini-2.5-flash -> gemini-3.5-flash (2.5-flash
retires 2026-10-16; 3.5-flash is the current stable GA and Google's
named replacement, so the #970 fix doesn't re-break in ~3 months)
- gemini-3.5-pro (does not exist) -> gemini-2.5-pro in docs
- plain gemini-3.1-flash (not a GA Gemini-API id) dropped from the Vertex
discovery list / preferences / docs; use gemini-3.5-flash or -flash-lite
- dead gemini-pro dropped from preferences and docs
Ported #996's #970 regression test, pinned to gemini-3.5-flash.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
check_api_password() (an unused HTTPBearer-based dependency, superseded
by PasswordAuthMiddleware) and its now-unused imports are dead code -
nothing calls it. Removed.
Docs across api/CLAUDE.md, docs/3-USER-GUIDE/api-configuration.md,
docs/5-CONFIGURATION/security.md, docs/7-DEVELOPMENT/security.md, and
docs/SECURITY_REVIEW.md still described a hardcoded default password
("open-notebook-change-me") that PasswordAuthMiddleware doesn't actually
have - if OPEN_NOTEBOOK_PASSWORD is unset, auth is fully disabled
instead. Updated to match actual behavior.
* fix: bind SurrealDB's docker-compose port to localhost only
The surrealdb service published 8000:8000, exposing it on 0.0.0.0 -
reachable from the network with the default root:root credentials. The
open_notebook service reaches it over the internal compose network
regardless, so the host port mapping is purely for local debugging
(Surrealist, `surreal sql`). Bind it to 127.0.0.1 instead.
* chore: add gitignored compose override for host-specific tweaks
The port bind is now localhost-only by default; ship a
docker-compose.override.yml.example (and gitignore the real override,
which docker compose auto-merges) so users who genuinely need remote DB
access have a sanctioned, secure-by-default way to re-expose it instead of
editing the tracked compose file.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* fix: validate Host/X-Forwarded-Proto before trusting them for API URL
The runtime-config endpoint's auto-detection built the browser-facing
API URL directly from the request's Host header and X-Forwarded-Proto,
with no validation. Behind a reverse proxy that forwards these headers
untrusted, a spoofed or malformed Host could redirect the browser's
subsequent API traffic - including the auth bearer token
(lib/api/client.ts) - to an attacker-chosen host.
Reject anything that isn't syntactically a hostname/IP (or a bracketed
IPv6 literal) before using it to build the URL, falling back to
localhost on rejection. X-Forwarded-Proto is now restricted to a literal
"http"/"https" instead of passing through whatever value a
misconfigured or spoofed proxy header supplies.
* fix: make Host-header port stripping IPv6-aware
The split(':')[0] port strip mangled bracketed IPv6 literals ([::1]:5055
-> '['), so IPv6-literal access always fell back to localhost and the
IPV6_LITERAL_PATTERN was unreachable dead code. Strip the port bracket-
aware instead, so the pattern is now actually exercised. Deliberately not
new URL(), which would extract the host from userinfo/path payloads
(legit@evil.com -> evil.com) and defeat the strict validation - added
regression tests covering IPv6 accept/reject and confirming the existing
userinfo/path-traversal cases still fall back to localhost.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* docs: restructure documentation around AGENTS.md, VISION.md and decision records
- Consolidate 17 CLAUDE.md files into 3 AGENTS.md (root, backend, frontend);
CLAUDE.md files become @AGENTS.md pointers
- Add VISION.md: product identity + current posture with horizon clusters
- Add docs/7-DEVELOPMENT/decisions/ with 4 retroactive ADRs and 2 PDRs
- Add 5 new engineering docs pages (credentials, content-processing,
podcasts, prompts, frontend) absorbing knowledge from removed CLAUDE.md
- Dismember TRIAGE.md: label taxonomy into maintainer-guide.md, product
jurisprudence into VISION.md, operator heuristics stay local (gitignored)
- Add AI-assisted/agent-generated PR guidelines to contributing.md
- Convert README.dev.md into a pointer after migrating its unique content
(make workflow matrix, Docker publishing, add-a-language playbook)
- Fix stale docs: migration path/format, provider count, locale list;
fix broken links (docs/index.md, PR template, CONFIGURATION.md)
* docs: fix README doc links and add markdown link check to CI
- Repoint 9 README links to pages that actually exist in docs/
- Replace literal (link) placeholder in maintainer-guide templates
- Add scripts/check_md_links.py validating relative links in tracked
markdown (skips URLs, anchors and code spans)
- Add docs-links workflow running the check on PRs that touch markdown
* docs: add documentation restructure to changelog
* feat: add cubic.yaml with project-aware AI review agents
Three custom review agents (vision & principles alignment backed by
VISION.md, known mechanical caveats, security & testability), PR-contract
review instructions, and automatic ultrareviews for auth, credential,
encryption and migration changes.
* docs: graduate issue-first policy by change size
Small obvious fixes (typos, docs, tiny bugs, i18n completions) no longer
require an issue; features and architecture changes still do. Sizeable
PRs opened without an issue convert to draft while the issue goes
through triage (1-2 days). Applied consistently across contributing
guide, root CONTRIBUTING pointer, PR template, maintainer guide red
flags and cubic review instructions.
* docs: align PR template Related Issue section with graduated issue-first policy
* docs: address review — generalize ADR-002/004, unwrap hard-wrapped lines
- ADR-002 now records the general delegation rule (platform/media support
that needs heavy coding lives in focused external libraries) covering
Esperanto, Content Core and podcast-creator
- ADR-004 now records the durable decision (long-running work runs on
background workers — heavy content, varied machine sizes, never lock
usage) with the queue technology as a swappable implementation detail
pending #381
- Remove mid-paragraph hard line wrapping from authored docs to match
repo convention (one line per paragraph)
* fix: address cubic review — stale doc facts, make dev/full targets, link checker query strings
- credentials.md: only PROVIDER_CONFIG exists as a map; Vertex/Azure/
OpenAI-compatible provisioning is inline in _provision_*() functions
- content-processing.md: correct ContextConfig priority weights
(source 100 > insight 75 > note 50)
- development-setup.md + Makefile: make dev/full pointed at root compose
files that don't exist; targets now use examples/docker-compose-dev.yml
and examples/docker-compose-full-local.yml with --project-directory .
- check_md_links.py: strip query strings before file-existence checks
* fix: prevent SurrealQL injection in Credential.get_all()'s order_by
ObjectModel.get_all() already validates order_by against an allowlist
before interpolating it into the query string - but Credential overrides
get_all() (to handle per-row api_key decryption) and its override built
the ORDER BY clause with a raw, unvalidated f-string, bypassing that
protection entirely.
Extract the base class's validation logic into a reusable
_validate_order_by() classmethod and route Credential.get_all() through
it. Not reachable from any current API surface today (the only caller
passes a hardcoded "provider, created", not user input), but any
subclass that builds its own query around order_by instead of
delegating to the base get_all() needs this so the allowlist can't
silently drift between call sites.
No test currently exercises this path - worth adding one (e.g.
Credential.get_all(order_by="field; DROP TABLE credential") raises
InvalidInputError) before merging.
* test: add order_by injection regression tests
Covers _validate_order_by() normalization/rejection and the Credential
.get_all() path, which raises on an injected clause before any query
reaches the database.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* fix: surface silent command-submission failures where they matter
Source.add_insight() caught submission failures and returned None
instead of raising - callers (transformation.py, source.py) run inside
surreal-commands jobs whose outer exception handling already
retries/fails on this, so a swallowed submission failure meant a
transformation could report success while the insight was silently
never persisted. Now raises DatabaseOperationError, matching
vectorize()'s existing contract.
Note.save()'s auto-embed needs the opposite treatment: it's an implicit
side effect of save() (not an explicit dedicated call), and the note
itself is already durably saved by the time it runs - so a submission
hiccup there shouldn't turn an otherwise-successful save into a 500.
Wrapped in try/except, logs and returns None. api/routers/embedding.py's
explicit POST /embed (item_type=note) is the one caller for whom
submission success genuinely is the point of the call, so it separately
checks for a missing command_id and surfaces that as a failure.
* fix: collapse duplicate error logging in add_insight
logger.error + logger.exception logged the same failure twice; a single
logger.exception call carries both the message and the traceback (same
form vectorize() already uses).
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
Two small fixes to the podcast episode-listing path, plus a doc note:
- audio_file is only ever set server-side today from a UUID-named
directory under PODCASTS_FOLDER, so this can't currently be tripped -
but the stream/retry/delete endpoints didn't verify the resolved path
actually stayed within PODCASTS_FOLDER before following it. Add
_is_audio_path_contained() as defense in depth against a future code
path (e.g. importing external audio) setting audio_file to something
else.
- Listing episodes called get_job_detail() -> get_command_status() once
per episode, each its own round trip (no connection pooling). Add
PodcastEpisode.get_job_details_for_commands() to batch-fetch status
for every episode's command in one query instead.
Also documents (docs/7-DEVELOPMENT/security.md) that podcast_creator's
configure("templates", {...}) compiles strings as Jinja2 template
source - the same shape as the SSTI vulnerability fixed in
transformation.py (GHSA-f35w-wx37-26q7). Confirmed dormant: no code path
in this repo calls it today. commands/podcast_commands.py gets a
matching code comment warning against wiring user text into it if a
"custom podcast template" feature is ever added.
api/routers/sources.py and api/podcast_service.py interpolated the raw
exception (detail=f"...: {str(e)}") into client-facing error responses -
inconsistent with the safer pattern already used elsewhere in the same
files (e.g. the download handlers), which log the raw exception
server-side but return a fixed generic message. Internal details (DB
hostnames, connection errors, stack-trace fragments) could leak to any
API caller through a 500 response.
Every occurrence already had a matching logger.error() call, so this is
a client-facing message change only, not a logging change. Deliberate
app-authored messages (InvalidInputError text, "Notebook X not found",
result.error_message) are untouched - only raw f"{str(e)}" interpolation
is affected.
tests/test_config_endpoint_no_leak.py is a regression lock for
api/routers/config.py (already correct, not touched by this diff) rather
than a fix - added since it shares the same "unauthenticated endpoint,
don't leak exception text" concern.
* fix: restrict CreateCredentialRequest.provider to a known allowlist
provider was a bare `str`, so any string (typo'd or bogus) flowed
through validation to the domain layer and failed later with a less
clear error, instead of a clean 422 at the API boundary.
Add a SupportedProvider Literal covering the 17 providers already
handled elsewhere - kept in sync with the frontend's ALL_PROVIDERS,
connection_tester.py's TEST_MODELS, and credentials_service.py's
PROVIDER_ENV_CONFIG (a test asserts all three agree on the same set).
* test: lock the frontend provider list into the sync test
The frontend's ALL_PROVIDERS copy was only cross-checked in a docstring;
extract its string literals from the source so all four provider lists
(Literal, TEST_MODELS, PROVIDER_ENV_CONFIG, frontend) are enforced by CI.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
Three small, independent hardening fixes to source ingestion:
- generate_unique_filename() checked `if not resolved.exists()` then let
a separate write happen later - two concurrent uploads landing on the
same candidate name could both pass the check and clobber each other.
Now atomically claims the name via Path.touch(exist_ok=False) (O_EXCL)
as part of the search loop itself.
- _resolve_source_file() and _is_source_file_available() compared
`resolved_path.startswith(safe_root)` without a trailing separator - a
sibling directory that merely starts with the same string (e.g.
"uploads_evil/") would incorrectly be treated as contained, unlike this
file's other two path checks which already guard with `+ os.sep`. Not
reachable today (source.asset.file_path is only ever set server-side),
but this closes the gap and matches the existing pattern.
- SourceCreate.notebooks/transformations had no length limit; both are
iterated with a per-item DB lookup in create_source(), so an unbounded
array let a single request trigger an unbounded number of sequential DB
round trips. Capped at 50.
tests/test_upload_type_mitigations.py adds no code change - it documents
why the adjacent "no file type allowlist on uploads" finding was
investigated and judged low-risk without one (downloads are already
served as application/octet-stream regardless of actual file type).
* fix: reject oversized request bodies before auth/routing
No limit existed on request body size, so a single upload could exhaust
memory/disk before any validation ran. MaxBodySizeMiddleware
(api/middleware.py) rejects requests over OPEN_NOTEBOOK_MAX_UPLOAD_SIZE_MB
(default 100MB) via both a Content-Length pre-check and by counting bytes
as the body streams in, so it also catches chunked requests with no
Content-Length header.
Registered after PasswordAuthMiddleware (wrapping it) so oversized
requests are rejected before spending any work on credential checks, and
inside CORSMiddleware so a rejected upload still gets CORS headers.
* fix: clamp non-positive size limits and log rejected requests
OPEN_NOTEBOOK_MAX_UPLOAD_SIZE_MB=0 (or negative) made every request with a
body 413; fall back to the default instead, with a warning. Also log each
413 rejection so operators can diagnose failed uploads.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* fix: don't combine wildcard CORS origins with allow_credentials
Combining allow_origins=["*"] with allow_credentials=True makes
Starlette's CORSMiddleware reflect the request's Origin header verbatim
instead of returning a literal "*" (browsers reject a literal wildcard
alongside credentials) - defeating the origin allowlist for any
credentialed request.
allow_credentials is now tied to whether CORS_ORIGINS was explicitly
scoped: False for the default wildcard, True once an operator opts into
specific origins. _cors_headers() (the manual CORS builder for error
responses raised before CORSMiddleware runs) is updated to match, so it
can't grant credentials the real middleware wouldn't.
Not independently exploitable today (the frontend never sends
credentialed requests, and auth is a Bearer header, not a cookie), but
there's no reason to allow it for the default wildcard case.
* fix: key allow_credentials on the parsed origins list, not the env var
An operator who explicitly sets CORS_ORIGINS=* got allow_credentials=True
with a wildcard origin list - the exact reflect-any-Origin behavior this
change exists to prevent. Introduce CORS_ALLOW_CREDENTIALS keyed on the
parsed list containing '*' and use it at both the middleware registration
and the manual error-response headers; replace the tautological formula
tests with ones exercising the real parser.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* fix: don't leak filesystem info via Vertex credential test errors
Vertex's credentials_path is free text with no path validation, and
Google's auth library raises distinguishable exceptions for "file
missing" (OSError), "not valid JSON" (json.JSONDecodeError), and "valid
JSON but wrong shape" (GoogleAuthError). Echoing any of these back to an
API caller turns credential/model testing into a filesystem oracle - an
attacker who can create/test a Vertex credential could probe for the
existence and contents-shape of arbitrary files on the server.
_is_vertex_credentials_file_error() catches all three and both call
sites (api/credentials_service.py's test_credential, connection_tester's
test_individual_model) return one generic message instead.
* fix: don't classify network failures as credentials-file errors
ConnectionError/TimeoutError are OSError subclasses and TransportError is
a GoogleAuthError subclass, so the oracle guard also swallowed genuine
network failures - a user with a blocked network would be told to go debug
their credentials file. Excluding them reveals only the error's category,
which keeps the filesystem oracle closed.
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
The sanitize fix itself landed via #1011's squash; this restores the code-block copy button that the default sanitize schema stripped, with a regression test.
* fix: sanitize raw HTML in the note/transformation markdown preview
MarkdownEditor's live preview renders through @uiw/react-markdown-
preview, which parses literal HTML in the markdown source into real
elements (its `raw` default) - including a live <iframe>. Notes can
hold AI-generated content that echoes an indirect prompt injection
from an ingested document, so this was reachable without the user
writing any HTML themselves.
Add rehype-sanitize (default schema) ahead of rehype-katex in the
preview pipeline. Ordering matters: sanitizing after katex strips
katex's own generated markup (not in the default allowlist), while
sanitizing before it only touches the raw-HTML-derived tree and
leaves not-yet-rendered math nodes alone. Verified against the real
preview component that this strips <iframe>/<script>/<style>/
javascript: URLs while fully preserving math, syntax highlighting,
and GFM tables/task-lists.
* fix: make validate_url() async so DNS resolution doesn't block the event loop
socket.getaddrinfo() in validate_url()'s hostname-resolution branch ran
synchronously on every call - and this is called on the hot path of model
provisioning, potentially once per chat message/transformation. A slow or
hanging DNS lookup stalled every other concurrent request, not just the
one that triggered it.
Run the resolution via asyncio.to_thread() instead, and thread `await`
through every call site: credentials_service.py's discover functions,
routers/credentials.py's create/update, routers/sources.py's source-URL
ingestion, and connection_tester.py's three provider test functions.
save_uploaded_file() did a plain synchronous open()/write() directly
in the async create_source handler, blocking the event loop - and
every other concurrent request - for the duration of a large upload.
Same bug class the recent chat-graph fix (#971) addressed, just not
applied here.
Move the filesystem work (filename resolution + write) into a sync
helper run via asyncio.to_thread(), matching the pattern already used
for execute_command_sync elsewhere in this file. Confirmed the event
loop stays responsive (ticking normally) during a simulated slow
write, and that errors/cleanup still propagate correctly through the
thread.
Building default chat context, the notebook context endpoint, and
podcast generation all looped over every source in a notebook calling
get_context() -> get_insights(), each a separate query that also pays
its own connection setup (no pooling). A notebook with hundreds of
sources meant hundreds of serialized round trips before a chat
message even reached the LLM.
Add SourceInsight.get_for_sources() to fetch insights for every
source in one query, and thread an optional pre-fetched insights list
through Source.get_context() so callers can opt in without changing
its behavior for anyone who doesn't. Measured against a real
(embedded) SurrealDB instance: 14 queries down to 3 for a 12-source
notebook, with correctness verified. The two router call sites treat
a batch-fetch failure the same way the old per-source loop treated a
single failure - falls back to empty insights rather than failing the
whole request.
validate_url() only ran when a credential was created or updated. The
actual HTTP requests (connection testing, model discovery, and real
inference through Esperanto) re-resolve DNS fresh on every call, so a
hostname that resolved to a public IP at save time can later be
repointed to an internal or cloud-metadata address - a classic
DNS-rebinding TOCTOU that a one-time check can't catch.
Move validate_url() to open_notebook/utils so it can be re-run from
the AI layer without an api-depends-on-open_notebook layering
violation, and re-check immediately before every outbound request:
connection_tester's three providers, credential model discovery, and
ModelManager.get_model() on the real-inference path. Also close a
second gap in the same validator: it missed AWS's IPv6 metadata
address (fd00:ec2::254), which isn't link-local so the existing check
never caught it.
POST /sources with type=link copied the user-supplied URL straight
into content-core's fetch with zero validation - unlike the
credential-URL path, which already blocks internal/metadata
addresses. Any user could make the server fetch cloud metadata
endpoints or scan the internal network via "add a web source".
Reuse the same validate_url() guard at the point the URL is first
accepted, before it's ever handed to content-core.
transformation.prompt and the generic pattern-chain's prompt were
passed to Prompter(template_text=...), compiling attacker-influenced
text directly as Jinja2 template source. ai-prompter's sandboxed
environment blocks the classic __globals__/__subclasses__ RCE
gadgets, but not an unbounded {% for %} loop - a trivial DoS for any
authenticated user, and one instance of the same "user text becomes
template source" pattern behind a previously-disclosed critical CVE
(GHSA-f35w-wx37-26q7), whose fix only added sandboxing without
removing the pattern itself.
Render through fixed, developer-authored templates instead, with the
user's text passed in as a plain variable. Verified byte-identical
output for legitimate prompts and confirmed the same payload that
used to be a DoS vector now renders as inert text in under a
millisecond.
PasswordAuthMiddleware and check_api_password compared the bearer
token to the configured password with `!=`, a timing side-channel on
the single secret gating the whole API. There's no rate limiting to
blunt repeated probing, so switch both to secrets.compare_digest().
repo_relate() interpolated the relate target directly into the query
string, reachable via an unvalidated notebook_id on the save-insight-
as-note flow - a crafted ID could inject and execute arbitrary
SurrealQL (confirmed against a live embedded instance: a single
crafted RELATE call wiped an entire table). Bind record identifiers
as query parameters instead of building them into the query text, and
validate notebook_id exists before relating, matching the pattern
already used by every other caller of add_to_notebook().
* fix: make API startup resilient to a not-yet-ready database
Fixes#708
* fix: bound the database readiness probe with a per-attempt timeout
_wait_for_database awaited migration_manager.ping() with no timeout, so a
hung connection probe could bypass the retry budget and block startup
indefinitely. Wrap the probe in asyncio.wait_for with a per-attempt ceiling;
a timed-out probe is treated as a transient failure and retried like any
other unreachable-database attempt, keeping the retry budget bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: add CHANGELOG entry for startup database readiness retry (#708)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* feat: add markdown syntax highlighter
Also fixes the markdown rendering, migrating old tailwind config to v4
* perf: use PrismLight with a bundled language set, add renderer tests and CHANGELOG
---------
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* feat: Recently Viewed section for notebooks and sources
Fixes#850
* fix: export RecentlyViewedResponse type from api types
The RecentlyViewed component and notebooks API client import
RecentlyViewedResponse from @/lib/types/api but the interface was missing,
failing the type-check build. Add it to match the backend api/models.py model.
* fix: harden recently-viewed read path and index the recency query
Three robustness fixes on the recently-viewed feature:
- Make the last_viewed_at write-on-read stamping best-effort. _stamp_source_view
and _stamp_notebook_view now swallow and log their own errors, so a failed
stamp update can no longer turn a successful GET /sources/{id} or
GET /notebooks/{id} into a 500.
- Stop leaking internal details from GET /recently-viewed: log the full
exception server-side and return a generic error message to the client.
- Add indexes on last_viewed_at for the notebook and source tables in
migration 16 so the ORDER BY last_viewed_at DESC LIMIT recently-viewed
query does not degrade into a full table scan as data grows. The down
migration removes the indexes before dropping the field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat: register migration 18, add recently-viewed i18n keys and CHANGELOG entry
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* feat: per-transformation custom model selection
Fixes#776
* fix: validate model_id exists when creating or updating a transformation
The create and update endpoints persisted model_id without checking the
referenced model exists, so an invalid reference was stored silently and only
surfaced later as a 404 at execution time. Add the same existence check
execute_transformation already performs (Model.get -> 404 if missing) to both
the create and update paths, so a bad model_id is rejected up front with a
clear 404. Update still allows clearing model_id to None.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: cover model_id existence validation on transformation create/update
The create test now mocks Model.get so the new existence check passes and
asserts it was awaited with the supplied model_id. The update test expects
Model.get to be awaited twice (once validating the update, once at execute)
rather than once.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: add CHANGELOG entry for per-transformation model selection (#776)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Luis Novo <lfnovo@gmail.com>
* fix: pass max_tokens through to podcast_creator for outline/transcript generation
* fix: persist and expose episode_profile max_tokens (migration + API)
* docs: add CHANGELOG entry for episode profile max_tokens pass-through (#639)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Luis Novo <lfnovo@gmail.com>
Add type annotations to PasswordAuthMiddleware.__init__ and dispatch (plus
excluded_paths) and drop the [mypy-api.auth] ignore_errors stanza so the
module is type-checked. Annotations only; no behavior change.
Refs #945
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Add backend (pytest-cov) and frontend (Vitest v8) coverage to CI: pytest runs with --cov + uploads coverage.xml, vitest runs with --coverage + uploads the report. Lockfiles (uv.lock, package-lock.json) regenerated for the new deps; verified both coverage runs pass in CI.
Closes#942
Remove the mypy ignore_errors override for open_notebook.domain.base and fix the surfaced type errors by adding typed DB config helpers (namespace/database default to open_notebook, password to root — matching the documented defaults). Adds tests/test_repository_config.py.
One slice of the incremental mypy cleanup. Refs #945
Add .github/RELEASE_PROCESS.md documenting the flow-driven release model (ready → PR → main → cut a version when mature), dev/stable image flow, and a maintainer verification checklist. The other #938 cleanup tasks (deprecated RC label, RC scripts, release-please) were already done or moot.
Fixes#938
Expand /api/sources sorting to type, title, created, updated, insights_count, and embedded (whitelisted, with a computed type alias so SurrealDB can order by it). Frontend adds sortable headers for every column plus an Updated column.
Fixes#895
Notebook chat (execute_chat) and source-chat SSE ran LangGraph's synchronous invoke() on the event loop, freezing the whole API while the LLM responded. Both now run via asyncio.to_thread(), matching the existing get_state() calls.
Fixes#704
Add a 'Refresh content' menu item for completed web-link sources that re-fetches the URL and re-embeds via process_source (no transformations, so no duplicate insights). Translated across all 14 locales.
Fixes#259
Rewrite the Windows native Quick Start to start the four services manually via uv run, and add an optional sample launcher users can save themselves, instead of pointing at a start-open-notebook.bat that the repo never shipped.
Fixes#846
Extend KaTeX rendering (remark-math + rehype-katex) from chat-only to source content, source insights, Ask answers, transformation output, and the note editor live preview.
Fixes#269
Lay the model-discovery dialog out as grid-rows-[auto_1fr_auto] (fixed header/footer, scrollable body) so the Add button stays visible with long model lists. Fixes the OpenRouter-models registration blocker.
Fixes#816
