fix: async insight creation to prevent transaction conflicts (#512)

Migrate insight creation to the command system with automatic retry logic
to prevent SurrealDB transaction conflicts during batch imports.

Changes:
- Add create_insight_command with retry logic for transaction conflicts
- Add run_transformation_command for async transformation execution
- Make Source.add_insight() fire-and-forget (returns command_id)
- Update POST /sources/{id}/insights to return 202 Accepted immediately
- Frontend polls command status until complete, then refreshes
- Auto-update notebook page icon when source gains insights
- Add i18n keys for insight generation feedback

Related to #489
This commit is contained in:
Luis Novo 2026-01-31 15:51:27 -03:00 committed by GitHub
parent 303ffacfa5
commit 301dd4e20a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 429 additions and 68 deletions

View file

@ -32,7 +32,7 @@ Two base classes support different persistence patterns: **ObjectModel** (mutabl
- `vectorize()`: Submit async embedding job (returns command_id, fire-and-forget)
- `get_status()`, `get_processing_progress()`: Track job via surreal_commands
- `get_context()`: Returns summary for LLM context
- `add_insight()`: Generate and store insights with embeddings
- `add_insight()`: Submit async insight creation via `create_insight_command` (fire-and-forget, returns command_id)
- **Note**: Standalone or linked notes
- `save()`: Submits `embed_note` command after save (fire-and-forget)
@ -80,7 +80,7 @@ Two base classes support different persistence patterns: **ObjectModel** (mutabl
- **Auto-embedding behavior**:
- `Note.save()` → auto-submits `embed_note` command
- `Source.save()` → does NOT auto-submit (must call `vectorize()` explicitly)
- `Source.add_insight()`auto-submits `embed_insight` command
- `Source.add_insight()` → submits `create_insight_command` which handles DB insert + `embed_insight` command (all fire-and-forget)
- **Relationship strings**: Must match SurrealDB schema (reference, artifact, refers_to)
## How to Add New Model