SurfSense/docs/architecture/documents.md

Documents and ingest

A workspace holds a library of sources: uploaded files, notes written in the app, and the bodies of Studio artifacts. Every source ends up the same way, as a markdown body on its documents row cut into passages that are embedded and indexed, so chat and search treat them alike. The API only stores bytes and enqueues a job; parsing, chunking and embedding happen in the ingest worker, one document at a time, and a document is ready only once it is searchable. Ingest also keeps a file's pictures beside its original, as figures the agent can place in a document. Studio indexes an artifact's body itself, in its own worker, with the same chunking and embedding code.

Code: modules/workspaces/, modules/documents/, worker/ingestion/, worker/jobs.py, shared/queue.py, frontend/src/features/sources/ Decisions: ADR 0003, ADR 0007, ADR 0036, ADR 0008, ADR 0009

Workspaces

Method	Path	Does
POST	/workspaces	create; 201
GET	/workspaces	list, oldest first
GET	/workspaces/{workspace_id}	read one
PATCH	/workspaces/{workspace_id}	rename
DELETE	/workspaces/{workspace_id}	delete it and everything in it; 204

• The API seeds "My Workspace" at startup when no workspace exists (seed.py), so a first launch always has one to open into.
• Every workspace has a Library: a managed source root and its root folder, made when the workspace is created, by 0025 for older ones, and on first use by any folder route (managed_root.py).
• A name is trimmed and must be 1 to 200 characters, so a name of spaces fails instead of rendering blank.
• Delete cascades the workspace's documents (with their chunks and index entries), threads, messages and artifacts, then removes data/workspaces/<id>/ after the commit, which a rollback would undo.
• Every route with a workspace in its path resolves it first and answers 404 for an unknown id. A document is looked up within the workspace in its path, so one workspace's id cannot reach another's document.

Document routes

Method	Path	Does
GET	/workspaces/{workspace_id}/documents	list (below)
POST	/workspaces/{workspace_id}/documents	write a note, with an optional document_metadata object and folder_id; 201 with its body, 422 for a field it does not know
POST	/workspaces/{workspace_id}/documents/upload	upload files, with optional folder_id and relative_paths form fields; 201 with created, duplicates and rejected
POST	/workspaces/{workspace_id}/documents/move	{document_ids ≤ 1000, folder_id}: file sources in a folder; 200 with moved and skipped (Folders)
GET	/workspaces/{workspace_id}/documents/{document_id}	one document with its content and document_metadata; content is null for a FILE the worker has not written yet
PATCH	/workspaces/{workspace_id}/documents/{document_id}	rename; edit a note's content
DELETE	/workspaces/{workspace_id}/documents/{document_id}	delete; 409 while processing
POST	/workspaces/{workspace_id}/documents/{document_id}/retry	requeue a failed or cancelled document
POST	/workspaces/{workspace_id}/documents/{document_id}/cancel	stop a pending or processing ingest
GET	/workspaces/{workspace_id}/documents/{document_id}/original	the uploaded file, as an attachment
GET	/workspaces/{workspace_id}/documents/by-chunk/{chunk_id}	a cited chunk and its neighbours, for the citation panel (chat.md)

Chunks have no router of their own.

Listing

• Filters are ?document_type= and ?status=, both repeatable, and ?folder_id=. Paging is ?limit= (default 50, at most 200) and ?offset=; the sources panel reads pages of 200 until one comes short, so it holds every source, not the newest 50. Rows come newest first, by created_at and then id, since one upload batch shares a second, so a fresh upload leads the sources list while it is still indexing.
• ARTIFACT rows are included and the caller filters them out. The sources panel asks for document_type=FILE&document_type=NOTE, so Studio output does not look like something the user uploaded.
• A row carries id, title, document_type, status, error_message, folder_id and timestamps, but no content. The list is polled while ingest runs, and a body per row would ride along on every poll; GET .../documents/{document_id} returns one row with it.

What is editable

• Title, always: 1 to 500 characters after trimming.
• Content, only for a NOTE. Anything else was extracted from bytes or generated, and an edit would vanish at the next ingest, so PATCH answers 409. This is also the server-side read-only guard ADR 0003 asks for on artifact bodies; renaming and deleting an artifact stay allowed.
• Editing a note's content puts it back to pending and enqueues ingest, because until the worker rebuilds the index, search would keep returning the old text.
• In the sources panel, every row's menu has Rename, which sends title alone so a file is never sent a content it would refuse, and a note's row also has Edit note. Both forms check the title's 1 to 500 characters before sending; the server's answer stays authoritative.

Notes

A note is a document the user writes, with no file behind it. Creating one commits the row as pending and enqueues ingest: nothing needs parsing, but the note is not ready until it is chunked and indexed. A note never touches the filesystem and carries no dedup key, so two notes with the same text are two documents. + › New note in the sources panel header writes one; Edit note reopens it with GET /workspaces/{id}/documents/{doc} and saves with PATCH, and the edited row comes back pending, so the panel's ingestion poll carries it to ready like any upload.

A note may carry document_metadata, stored as given. A plugin names itself there on the notes it adds, with plugin_id, plugin_version, action and run_id; a note the user writes has none, and editing a note leaves it alone. Writing a note with a field the API does not know is refused rather than dropped, so a client that names a field differently finds out at once.

The sources tree

The sources panel shows the Library as a tree (features/sources/tree/). It reads every folder with GET /folders and every file and note page by page, 200 at a time, and builds the tree on the client; the Library's own folder is never a row, so a workspace with no folders is the flat list it always was.

• Rows. Folders come first, by name, then sources newest first. The list is an ARIA tree with one treeitem per row: arrow keys move and open, Space ticks, Enter opens, F2 renames and Delete asks to delete. A name filter, opened from the search button in the header, shows matches with the folders holding them opened; Escape or closing it clears it. A name the column cuts off shows whole in a tooltip on hover, or while its row has keyboard focus (overflow-tooltip.tsx). Rows are memoized, and a re-read of the list keeps the object of every unchanged source (reuse-unchanged.ts), so a poll or an event renders only the rows whose source changed.
• Actions. New folder (from the header's + menu, or inside a folder from its menu), Rename, Move to… and Delete. Move to… is a picker of folders and the keyboard's path; a row dragged onto a folder, or onto a source inside one, moves there too. A folder cannot be moved inside itself. Deleting a folder names how many sources, and filed Studio outputs, go with it, as GET /folders/{folder_id}/summary counts them, and deletes them for good.
• Ticks. Each row's tick is tri-state. Ticking a folder ticks everything under it, now and added later; the nearest tick up the tree decides (scope-state.ts). Chat and Studio send the ticks as source_scope, so every ticked source counts, not only the ones loaded.

Upload

• The request is multipart, one files part per file. Each file streams to a temporary file inside the workspace directory in 1 MB reads, hashed with SHA-256 on the way past and never held whole in memory. A file over 500 MB is refused with 413. The temporary file sits on the same filesystem as its destination, because a rename is only atomic within one filesystem.
• Only a plain extension, a dot and 1 to 16 letters or digits, is validated from the client's filename. The file is stored alone in a directory built from row ids, data/workspaces/<workspace>/documents/<document>/, under the client's filename made safe for every platform (original_file.py): the last path part only, with pathvalidate removing characters no filesystem accepts and suffixing Windows reserved names, leading dots dropped so the file is never hidden, cut before the extension so the whole name fits in 120 bytes, which keeps a typical Windows path under its 260-character limit, and ending in the validated extension lowercased. A name with nothing left becomes untitled, and one that would be extracted.md becomes extracted (1).md (see The original file). The name reaches the disk; the path never does.
• Accepted extensions are .pdf, .docx, .pptx, .xlsx, .html, .htm, .csv, .md, .markdown, .txt, .text, .png, .jpg, .jpeg, .tif, .tiff, .bmp and .webp. The bytes must match the extension: the PDF signature; an OOXML zip holding its main part, with at most 10,000 entries and 2 GB unpacked; UTF-8 text with no NUL and few control bytes; or an image's magic number. The MIME type stored is the server's, never the client's. The frontend mirrors the list to filter its file picker and to skip unsupported files, dropped ones included, before uploading.
• The sources panel takes files two ways: its + menu, and files dropped onto the panel, which shows "Drop files to add them as sources" while files are dragged over it (use-sources-drop.ts). Both go through the same upload. Dropped files and rows dragged between folders go through Pragmatic drag and drop, which also scrolls the list while a drag nears its top or bottom, and draws a dragged row as a small chip. The panel takes no drop while an upload runs, as uploading is disabled then, and a drag carrying no files, such as selected text, does not light it up. A file dropped anywhere else in the window is refused (file-drop-guard.ts): unhandled, Chromium opens it in the window, and in Electron that replaces the app, since the main process lets file: navigation through.
• A whole folder comes in two ways: + › Upload folder…, an <input webkitdirectory> whose files carry webkitRelativePath, and a dropped folder, which the panel walks with webkitGetAsEntry() (dropped-files.ts). Either sends each file's path under the folder as relative_paths, and a drop on a folder row, or on a source inside one, adds folder_id. Before sending, the client skips paths through .git, node_modules, __pycache__, .DS_Store, .obsidian and .trash, sets aside unsupported files and any file over 500 MB, which it lists as rejected rather than letting one fail its request with 413, and sends the rest in requests of at most 50 files or 200 MB (upload-plan.ts).
• Files go into the folder_id form field's folder, else the top of the Library. relative_paths, a JSON list index for index with files, carries an added folder's tree: each file lands in the folders its path names below the target (Research/2024/a.pdf makes Research and 2024), made as needed with ensure_folder_path. Levels past the depth cap are joined with " / " into the deepest level there is room for, so a deep tree arrives whole. A list of the wrong length is 422; a folder from another workspace is 404.
• dedup_key is the SHA-256 of the bytes and is unique per folder. Keying on the bytes rather than the filename means the same report saved twice in one folder is one document, while two different files both called report.pdf are two. A file whose bytes sit in another folder is created in the new one too, so a copied folder has no holes, and copying it again adds only what is new.
• A batch is split, not rejected, except that a file over 500 MB fails the whole request with 413 and nothing in it is created. The response lists created, duplicates (each with the existing document's id and its folder_id) and rejected (each with a reason), so a dropped folder holding one known file keeps the rest.
• An upload of more than 20 files enqueues at PRIORITY_BULK (10); every other ingest runs at PRIORITY_INTERACTIVE (100), so a note written during a long folder copy is read first (tasks.py).
• Accepted files become FILE rows in pending, with mime_type, size_bytes and suffix in document_metadata. The files move into place, the transaction commits, and only then is one ingest job enqueued per file, because the worker is another process and would look for rows this request had not yet written.

Folders

Sources sit in folders of the workspace's Library (data model, proposal). Routes are under /workspaces/{workspace_id} (modules/folders/):

Method	Path	Does
GET	/folders	every live folder, flat, the Library's own folder (no parent) first, then by folded name
POST	/folders	{parent_id?, name, role?}; no parent means the top of the Library; 201
PATCH	/folders/{folder_id}	{name?, parent_id?, role?}: rename, move, set or clear the role
GET	/folders/{folder_id}/summary	{folders, sources, artifacts} below it, for the delete dialog
DELETE	/folders/{folder_id}	delete it, its subfolders and every source in them, for good; 204
POST	/folders/{folder_id}/cancel	stop every queued or running read below it; {cancelled}

• A name is trimmed, 1 to 255 characters, and unique among its siblings by name_key, the name in NFC and case-folded in Python, since SQLite's NOCASE folds ASCII only: "Été" beside "ÉTÉ" is 409.
• Library folders go at most 8 levels below the Library's own folder; a create or move past that is 400, and so is a move into the folder itself or below it.
• The Library's own folder cannot be renamed, moved or deleted (409). A folder of another workspace, or one being deleted, is 404.
• Delete answers 409 while any source below is processing. Otherwise it marks the subtree deleting, which takes it out of every scope at once, and cancels its queued reads in the same transaction, so none starts and outlives its batch. It then removes the documents in batches of 500, each its own transaction with its bytes removed after the commit, then the folders. A delete the app stopped partway through is finished on a thread when the API next starts, so start-up does not wait on it; a source still processing there keeps its subtree deleting, out of every scope, until a later start. There is no Trash yet.
• POST /documents/move refuses an id from another workspace (422) and skips a source whose bytes already sit in the target folder, reported as {document_id, reason: "duplicate", duplicate_of}.
• Every change sends a folders event with the folder ids, an upload that made folders from its paths included; a move, a cancel and a delete also send a documents event with the document ids.
• Every constructor files a source: a note and an upload go to the folder named or the top of the Library; an import rebuilds the bundle's folder path under the Library; an artifact starts unfiled.

Retry, cancel and delete

• Retry accepts a failed or cancelled document: it sets pending, clears error_message and enqueues. Without it those states would be terminal, since re-uploading the same bytes is a duplicate.
• Cancel accepts a pending or processing document and answers 409 otherwise. It marks the row cancelled, clears error_message and revokes any queued copy of the job (revoke_pending). A running job is not killed: the pipeline checks between steps and unwinds at the next one, so a long Docling parse finishes first.
• Delete refuses a processing document with 409. Otherwise the row goes, its chunks cascade, the triggers clear both index tables, and the document's directory is removed after the commit; removing it first would leave a row describing a missing file if the transaction rolled back.
• Delete also accepts an ARTIFACT document, and removes the artifact's files under artifacts/<id>/ as well as documents/<id>/, since a Studio output keeps its rendered blobs under the artifact's own id rather than the document's. DELETE /artifacts/{id} removes the same two (studio.md).

The original file

No filename is stored in a row: the document's directory is the record. The original is the one file in it, so not the figures/ folder, that is not hidden, not Thumbs.db or desktop.ini, and not extracted.md. Any other count, an empty or absent directory included, means the file is missing. Directories written before uploads kept their names hold original.<ext> beside an extracted.md nothing reads. They are read as they are and deliberately never rewritten: nothing in the index depends on a file's name, so renaming would only change what the file manager shows, and a bulk rename can fail part way on a file that is open or already deleted. Because extracted.md is never taken for the original, a legacy directory whose original was deleted reads as missing rather than serving Docling's text as the user's file.

GET .../original serves the stored bytes with the document's title as the filename and the stored MIME type, always as an attachment: an HTML or SVG file rendered inline would run its script against the app's origin. It answers 404 when there is no file behind the document, as for a note.

The desktop app previews an original in place of the left sidebar when its MIME type has a source viewer, starting with PDF. The sidebar widens to the detail width while it shows, or stays at the user's own width if that is wider, as the right panel does for an inspected artifact, and the right panel stays as the user left it. Dragging the preview's edge resizes the preview alone, and that width is remembered apart from the sidebar's (column-widths.ts). Other file types still open natively: through the preload bridge, Electron applies the same rule to the directory under the data directory for the given ids, since the main process never calls the backend, and hands it to the operating system, or reveals it in the file manager (document-files.ts). The DocumentRead row carries mime_type from stored metadata only for FILE; notes always report null, even if their user-provided metadata contains a mime_type. The preview fetches the existing /original route, which remains an attachment so directly navigating to HTML or SVG cannot execute it on the app's origin.

Ingest pipeline

ingest_document(document_id) is declared in modules/documents/tasks.py, because Huey binds a task to its queue when it is decorated and the API is what enqueues it. The task body imports the pipeline lazily and Docling is imported only when the converter is first built, so the API never loads Docling or torch. Each job opens its own database engine.

1. Start. begin_job marks the row processing unless it was cancelled, and the worker notifies the API. The job then reads the active embedding index; with none, because onboarding has not chosen an embedder, it fails with no embedding model has been chosen yet.
2. Parse (parsing.py). A NOTE or ARTIFACT already has its markdown in content. A FILE with a text extension (.md, .markdown, .txt, .text) is read as UTF-8; anything else goes to Docling. A .pptx with no slides, such as a template, skips Docling, which refuses a deck with no pages: its text is PowerPoint template with no slides., its slide size and its slide layout names in order, one line each, read with python-pptx (slideless_deck.py), so search finds it and the agent can start a deck from it as a template (studio). A blank Word or Excel file goes to Docling, which reads it as empty, and is ready with no text. OCR and table structure are switched on, so a scanned PDF is readable and a table survives as a table. RapidOCR's per-line orientation classifier is off: on noisy scans it turned upright lines 180 degrees and they read back as garbage, and dropping it cut the character error rate on generated scans about eightfold. An image is first turned upright by its EXIF orientation, which Docling ignores, and given a DPI that reads it as one A4 page, never finer than the 216 dpi OCR renders at nor coarser than its own DPI (image_page.py); with none, Docling takes it as 72 dpi and OCR upsampled a photo threefold. Images and PDFs share one set of options, so they share one pipeline and its models. Inference gets one thread per physical core, at most 8, as audio.cpp does, instead of Docling's 4: on an 8-core CPU that parsed generated scans, photos and PDFs 1.4 times faster. The converter is built once per process and its models load on the first conversion, so the first document a worker parses pays for the import and the load. HF_HOME defaults to the models directory, because Docling would otherwise write weights into site-packages, which is read-only in a frozen bundle. When the bundled parser pack is present Docling uses it with RapidOCR, and HF_HUB_OFFLINE=1 is set before Docling is imported, because huggingface_hub reads it only then, so a PDF or an image converts with networking off. The markdown is kept only in content. Beside the original, ingest writes only the pictures Docling found, as figures.
3. Chunk (chunking.py). Chonkie's RecursiveChunker splits at the coarsest boundary that fits: a heading, a paragraph, a line, a sentence, a word, then a bare split so an over-long line still ends. Passages are at most 480 tokens by bge-small's tokenizer, under its 512-token limit with room for the two tokens it adds. That tokenizer is pinned to the chunker rather than taken from the active embedder, so changing the embedder would not re-cut documents and move every chunk id. Pieces under 24 characters are merged into a neighbour while splitting, so only a document shorter than that yields a shorter passage. The tokenizer is passed as an object, because one named by a string would reach the network even offline. Each passage records the lines its first and last characters fall on.
4. Embed (encoder.py). The active index's model, as its spec says, on onnxruntime's CPU provider: the document prefix, tokenize (truncated at the spec's max_tokens), pool, normalise, in batches of 32. For bge-small that is no prefix, 512 tokens, CLS pooling and L2 normalisation. No network and no model server. A vector whose width is not the spec's is refused rather than stored.
5. Index (indexing.py). The document's old chunks are deleted, which clears both index tables through the delete trigger, and the new chunks are inserted with their float32 embeddings, followed by one row per chunk in the active index's vector table, and the document records that index in embedding_index_id. The keyword index follows by trigger; the vector cannot, because only ingest holds it. Editing a note therefore stops its old text being findable.
6. Finish. finish_job writes ready unless a cancel won the race, and beside it only what the job produced: a file's extracted markdown on documents.content. A note's content is the user's and is only read, and ingest never writes title, so a rename or a note edit committed while the job ran stands, and the ingest that edit enqueued indexes the new text.

Cancellation is checked after parsing and after embedding. On any other failure the worker rolls back and re-reads the row. If the document was deleted or cancelled meanwhile it stops; otherwise it writes failed with error_message set to the exception type and message, cut to 500 characters, notifies, and re-raises so Huey runs the job again, up to two more times. A file whose bytes Docling cannot load (a ConversionError caused by a DocumentLoadError, such as a damaged Office file) is not run again, because the same bytes are refused the same way: parsing raises UnreadableFileError and the task turns it into Huey's CancelExecution(retry=False). Docling raises ConversionError too for a file it cannot open yet, such as one another program holds, and for a model that fails on every page, such as running out of memory; those are tried again. A later success clears the message. The sources panel shows it and offers Retry.

Figures

Ingest keeps the pictures in each source, so the agent can place a source's chart or logo in a document it makes (agent). Chat and search do not use them.

• Where. Beside the original, in the document's own folder: figures/<n>.png, numbered from 1 in the order the source shows them, and figures/figures.json, written last and replaced whole (source_figures/layout.py). Deleting the source removes its folder and so its figures, and no table or migration is needed.

  {"version": 1, "figures": [{"n": 1, "caption": "Figure 1: Projected yearly costs …", "page": 2, "width": 1104, "height": 620}]}
  

• Which pictures (figures/pictures.py), from the DoclingDocument the parse step already made, before it is exported to Markdown:

  • PDF: each picture Docling's layout model finds, drawn charts and pictures in scans included, cropped from its page with pypdfium2 at 2x (144 dpi) after the conversion, with its caption and page. Docling's own generate_picture_images stays off, because it keeps every page's render until the conversion ends: about 7 MiB a page, 574 MiB for 80 A4 pages (figures/pdf_crops.py).
  • Word and PowerPoint: the pixels the file embeds. Docling links no caption for them; PowerPoint gives the slide as the page, Word gives none.
  • An image source is its own one figure, turned upright by its EXIF orientation.
  • Pictures in page headers and footers count, since a letterhead logo is the picture a user asks for. A picture repeated with the same pixels is kept once. A picture that refers to a path or a URL rather than carrying its pixels is skipped, so an uploaded page cannot pull an image from the machine into the library. Every figure is a PNG in RGB, or RGBA when it has transparency, which python-docx and ReportLab both read.
  • A text upload (.md, .txt, .csv, .html) keeps none.

• Never fails an ingest. A failure while keeping figures is logged and recorded as figures.json with no figures and an error, and the text is indexed as usual.

• Names. A figure is <document id>-<n>, built from the document when it is listed, never stored (source_figures/). list_figures() answers a source's figures from its figures.json; figure_file() resolves a name to its PNG only for a source in the caller's workspace.

• The figures-only pass. A source ingested before figures were kept has no figures.json. The first list_figures() on it, when it is a ready FILE, queues extract_figures(document_id) on the ingest queue, once, and answers that its figures are being extracted; one still pending or processing gets them from its ingest. The pass reads the original with Docling again, without holding a transaction, and writes the figures, leaving the text and chunks alone (figures/backfill.py). It is not retried, since its failure is recorded in the index.

• Cost. scripts/measure_figure_ingest.py converts a file without and with figures, each in a fresh process. On generated files: an 80-page A4 PDF peaked at 2,519 MiB with figures and 2,517 MiB without, and cropping 20 vector charts from a 20-page PDF took 0.1 seconds.

Embedding index. Which model embeds is a property of the library, not a preference: a row in embedding_indexes holds the model's spec and names its vector table, which is built at the spec's width when onboarding locks the choice (data-model.md). upgrade_to_head() refuses to start when the table's width differs from the spec's. The per-batch check in embed() is the second line of defence. Studio indexes an artifact's body through the same index.

Before the choice. Onboarding gates the UI but not the API, so the routes that queue work which ends up embedded (writing, uploading, editing or retrying a document, a cloud import, and creating or regenerating a Studio artifact) answer 409 with the code embedding_not_chosen until an index exists.

Models on disk. The bge-small files (model_optimized.onnx and tokenizer.json, from Qdrant/bge-small-en-v1.5-onnx-Q) live in bge-small-en-v1.5/ under the models directory. They are pinned to one revision and checked against their hashes (bundled.py): every existing library was built with that exact file, so a different build would be a different vector space. Packaged builds ship them; a development checkout fetches them with scripts/fetch_embedding_model.py, which refuses a file that does not match. Any other curated embedder downloads into embeddings/<id>/ under the data directory, since the models pack is read-only in a packaged app. Chat answers 503 while the active model's files are missing.

The queue

• ingest is a SqliteHuey queue in huey.db, a file of its own so the consumer's constant polling never contends for the write lock on surfsense.db.
• worker-ingest drains it with one thread. Ingest saturates a CPU and writes to the database the API is serving from, so a second thread would spend its time behind the first one's lock. Studio has its own queue so an import never sits in front of a summary (ADR 0008).
• The queue survives restarts. Electron starts the workers beside the API and only the API migrates, so each consumer waits until the database is at the latest revision before taking a job (wait_for_schema.py); a job queued before an update would otherwise run against the old schema and fail. A job carries only its task's name, so each consumer imports every task before it starts (import_tasks()), and tests/integration/test_registration.py fails when that list falls behind the task modules on disk.
• Jobs are enqueued by upload, note creation, a note content edit and retry, and by import (import.md). The queue takes the highest priority first: interactive work at 100, a bulk copy and an import at 10. The figures-only pass, extract_figures, shares the queue (Figures).
• A source the app quit in the middle of ingesting is failed with interrupted when the app closed when the ingest worker next starts, before it takes a job (interrupted_documents.py). Huey drops a job as it starts it, so nothing else would end it, and Retry accepts only a failed or cancelled document. A source still pending is still queued and is left alone.
• Each transition the ingest worker makes, to processing, ready or failed, sends a documents event keyed by document id, and so does each change the API makes itself: a note written, an upload, a rename or edit, a retry, a cancel, and a delete, whose status is deleted. The sources panel reloads its list on each event and whenever a dropped stream is back, so a note a plugin writes shows as it is written; it still refetches every 10 seconds while any row is pending or processing, in case a notice was lost (overview.md).

Known gaps

• Deleting a folder is permanent; the Trash is designed, not built (proposal).
• Folders on disk (linked roots) are designed, not built.
• The sources tree renders every visible row of the open folders, with no windowing, so a very large open folder costs render time. Which folders are open is not kept across a reload.
• The tree's name filter runs on the client over the full listing, folding case with toLocaleLowerCase, which is close to the server's name_key but not identical; there is no GET /folders/search.
• There is no multi-select with Shift or Ctrl, so Move to… and Delete act on one row at a time.
• Parsing runs on the CPU. Windows and Linux ship CPU-only torch and no CUDA payload (ADR 0012).
• A picture in a Word or PowerPoint file is kept with no caption, even under a paragraph in Word's Caption style, because Docling links none; only a PDF's figures carry theirs.
• OCR reads one script family per converter: PP-OCRv6 covers Latin, Chinese and Japanese, so scanned Korean, Cyrillic, Devanagari or Arabic text is misread.