vrr/open-notebook
2
0
Fork 0
mirror of https://github.com/lfnovo/open-notebook.git synced 2026-04-29 03:50:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: embedding batch sizing and 413 error classification (1.7.4)

Browse source 
- Add batching to generate_embeddings() (50 texts per batch with per-batch retry)
  to prevent 413 Payload Too Large errors on large documents
- Add 413 error classification rule for user-friendly error messages
- Fix misleading "Created 0 embedded chunks" log in process_source_command
  by removing premature get_embedded_chunks() call (embedding is fire-and-forget)

Closes #594
This commit is contained in:
Luis Novo 2026-02-18 11:39:47 -03:00
parent 924cd88494
commit 5d84ab0768
12 changed files with 190 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

2
open_notebook/domain/notebook.py
View file

@ -416,7 +416,7 @@ class Source(ObjectModel):
pool exhaustion when processing large documents. The embed_source command:
1. Detects content type from file path
2. Chunks text using content-type aware splitter
3. Generates all embeddings in a single API call
3. Generates all embeddings in batches
4. Bulk inserts source_embedding records
Returns: