* ci: gate PRs on mypy, start ignore_errors burn-down
Add a backend-typecheck CI job running uv run python -m mypy . and bring
the repo-wide baseline from 197 errors to 0 so the gate blocks new type
errors from now on:
- enable the pydantic mypy plugin (resolves 138 false positives on
models whose fields have Field(None, ...) defaults)
- fix the remaining errors with real annotations; the only new type:
ignore comments cover a genuine langgraph typing limitation (partial
state dicts are valid at runtime but the overloads require the full
state type) and tests that intentionally pass invalid input
- start the ignore_errors burn-down: open_notebook.graphs.transformation,
open_notebook.graphs.ask and api.routers.models are now type-checked;
stale blocks for the deleted api.client and api.podcast_api_service
modules removed. Only open_notebook.domain.notebook remains exempt
(DB layer is migrating to surreal-basics)
- fix the mypy.ini header comment to describe what the config does
* test: use typing.get_args on the registry literal check
The registry test landed in parallel using Literal.__args__, which the
mypy gate in this branch rejects; align it with the get_args() idiom
used by the rest of the file.
Two findings from v1.11 release testing:
- GET /api/sources?sort_by=title returned a 500. source.title carries a
SEARCH (BM25) index (idx_source_title, migration 1) and SurrealDB's
planner fails ORDER BY on such a column with 'No iterator has been
found'. The query now sorts by a computed alias
(string::lowercase(title OR '') AS title_sort), which sidesteps the
index and makes the sort case-insensitive as a bonus.
- POST /api/sources with an over-limit notebooks/transformations array
(or invalid JSON in either field) returned a raw 500.
parse_source_form_data() builds SourceCreate manually, so pydantic's
ValidationError never reached FastAPI's request-validation handler.
Both cases now surface as a clean 422 with a descriptive message.
Verified against a live SurrealDB v2 instance; regression tests added
for the ORDER BY alias, all six sort fields, and the 422 paths.
Three small, independent hardening fixes to source ingestion:
- generate_unique_filename() checked `if not resolved.exists()` then let
a separate write happen later - two concurrent uploads landing on the
same candidate name could both pass the check and clobber each other.
Now atomically claims the name via Path.touch(exist_ok=False) (O_EXCL)
as part of the search loop itself.
- _resolve_source_file() and _is_source_file_available() compared
`resolved_path.startswith(safe_root)` without a trailing separator - a
sibling directory that merely starts with the same string (e.g.
"uploads_evil/") would incorrectly be treated as contained, unlike this
file's other two path checks which already guard with `+ os.sep`. Not
reachable today (source.asset.file_path is only ever set server-side),
but this closes the gap and matches the existing pattern.
- SourceCreate.notebooks/transformations had no length limit; both are
iterated with a per-item DB lookup in create_source(), so an unbounded
array let a single request trigger an unbounded number of sequential DB
round trips. Capped at 50.
tests/test_upload_type_mitigations.py adds no code change - it documents
why the adjacent "no file type allowlist on uploads" finding was
investigated and judged low-risk without one (downloads are already
served as application/octet-stream regardless of actual file type).
