perf: improve source listing speed by 20-30x (#436)

* fix(i18n): resolve podcast dialog translation infinite loop and profile issues

- Remove incorrect translation keys for user-defined episode profiles
- Cache translation strings in ContentSelectionPanel to avoid repeated
  Proxy accesses that triggered infinite loop detection
- Stabilize useEffect dependencies with dataKey pattern to prevent
  re-initialization on every keystroke
- Replace unstable sourcesQueries prop with stable fetchingNotebookIds set
- Clean up unused getSourceModes function and TranslationKeys import

* chore: bump lock

* chore: bump version to 1.5.1 and update CHANGELOG

* fix: guard .join() call in dataKey when query data is undefined

* fix(api): use FETCH command instead of async status lookups for sources list

Replace N async calls to surreal-commands with SurrealDB FETCH clause
to resolve command status in a single query. This eliminates the
command status cascade bottleneck.

* perf(db): add indexes on source field for insights and embeddings

Add migration #10 that creates indexes on the `source` field of
`source_insight` and `source_embedding` tables. These indexes
dramatically improve the performance of source listing queries
that use subqueries to count insights and check embedding existence.

Performance improvement: ~8.5s -> ~0.3s for 30 sources (28x faster)

* perf(db): make index concurrent

* fix: add IF NOT EXISTS to index definitions for idempotency

* fix: address code review feedback

- Add IF EXISTS to rollback migration for safer rollbacks
- Add fallback for unresolved command references (status = "unknown")
This commit is contained in:
Luis Novo 2026-01-15 22:29:20 -03:00 committed by GitHub
parent c6ec1fcddf
commit 4dc1539838
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 41 additions and 81 deletions

View file

@ -105,6 +105,7 @@ class AsyncMigrationManager:
AsyncMigration.from_file("open_notebook/database/migrations/7.surrealql"),
AsyncMigration.from_file("open_notebook/database/migrations/8.surrealql"),
AsyncMigration.from_file("open_notebook/database/migrations/9.surrealql"),
AsyncMigration.from_file("open_notebook/database/migrations/10.surrealql"),
]
self.down_migrations = [
AsyncMigration.from_file(
@ -134,6 +135,9 @@ class AsyncMigrationManager:
AsyncMigration.from_file(
"open_notebook/database/migrations/9_down.surrealql"
),
AsyncMigration.from_file(
"open_notebook/database/migrations/10_down.surrealql"
),
]
self.runner = AsyncMigrationRunner(
up_migrations=self.up_migrations,