Git owns the body, Postgres owns metadata, on-disk front-matter deferred — the
settled part, already the running behavior (C1/C5). The clean switch and the
record_*->intent rename are the Phase 5 cut's re-org, not a now-task: the cut
deletes the legacy arm, and the dual-run behind flags is correct meanwhile.
The MCP surfsense_update_document tool was the only live caller of
PUT /documents/{id}; it wrote the deprecated content column with no reindex,
and 409'd on git-backed workspaces. Repoint it at the wired editor save path
(source_markdown -> facade -> git + projection + reindex), which works on
flagged and unflagged workspaces alike, then delete the endpoint, its unused
DocumentUpdate schema, and its orphaned frontend (updateDocument service and
updateDocumentMutationAtom). One edit path, one source of truth.
Every path helper the agent runtime exposed already lives in
app.knowledge_store.paths, so path_resolver was a second, drifting copy of
the store's path law. Point all 28 importers at the package and delete the
425-line duplicate.
This also closes a layering inversion: index/{converge,rows,project} were
importing up into the agent runtime for path helpers that belong to their
own module.
The one behavioural difference was parse_documents_path, which strips the
.xml extension and " (<id>)" suffix to form a title — the contract the
indexer and its tests depend on. It moves to paths/legacy.py (it is xml
aware, so legacy) and becomes the exported one; store_path's unused raw
splitter is dropped.
virtual_path_to_doc is now the healed, column-first resolver everywhere,
a strict superset of the old hash/suffix/title lookup.
Folder create, rename, move and delete now record to git after the row
op, through thin module verbs (record_created_folder, record_moved_folder,
record_removed_folder, folder_virtual_path). Routes never spell a path or
bind a workspace; every verb self-guards, so an unflipped workspace is
untouched.
Rename and move capture the old path before mutating the row, then record
the move: the row is already at its new name, so the in-place reparent
no-ops and git still follows, id kept.
Delete drops only the folder's .keep markers (remove_folder_markers), never
its files. The incremental indexer prunes a row the moment its file leaves
the tree, so removing documents here would race the purge task that owns
their chunks and blobs. The markers are the resurrection gap; the purge
owns the documents.
Two blockers on routing folder CRUD through the facade are cleared.
move_folder renames the folder row in place (reparent_folder) before
reconcile, so a rename or reparent keeps the folder id instead of
prune-then-create churning it; child rows follow on parent_id, and
reconcile then finds the row already at the live chain and leaves it be.
The seed materializes each empty leaf folder as a .keep, since git holds
no empty directory. Only the leaf of an empty chain carries the marker;
its ancestors ride along on that path, and a folder a document already
lives in needs none.
The three facade writers that choose a name — save_document,
ingest_documents, move_documents — now derive through the naming law
(allocate_path / normalize_filename, "(2)" collisions) rather than the
legacy .xml derivation. A flipped workspace stops creating the very .xml
debt the seed heals; without this, every new note or rename re-stamped
.xml, and healing only ever touches files that already exist.
Occupancy comes from the git tree, the one authority on which files
exist, and a row's own file is excluded so a re-derivation after a lost
marker cannot collide the document with itself. The three sites switch
together: a partial swap forks a document between .md and .xml and breaks
the did-not-move check. doc_to_virtual_path / virtual_path_of stay for
the resolver and unflipped kb_postgres; only the flipped facade authors
.md.
The document-move handlers now record the move on the store after the
folder_id change, so a flipped workspace's move reaches git and a rebuild
finds the file at its new folder instead of resurrecting the old path. A
no-op on an unflipped workspace, since the verb self-guards per workspace;
bulk move groups by workspace because a move to root can span several and
each store binds to one.
_relocation_of reparents through the same derivation as the rest of the
live write path, so a move never forks the spelling. Folder CRUD stays on
folder_service for now: routing it onto the facade verbs waits on the seed
materializing .keep for existing empty folders (whole-workspace reconcile
would prune them) and on an id-preserving folder rename.
An import-boundary test pins that nothing under app/ outside the module
reaches the transaction, the engines, or the path submodules, so a second
writer or a second path spelling cannot creep back past the facade. The
package root stops re-exporting Transaction, which was the one way to reach
it without tripping the guard.
A symmetry test asserts the load-bearing invariant: a path authored into
git projects to a row whose path is the same string git stores, and that
path resolves back to the one row that authored it.
Folders now start in the store facade, empty ones included. StorePath
reserves .keep so a folder marker can never be authored as a document,
and three facade verbs each record one revision: create_folder writes
path/.keep, remove_folder removes the whole subtree, move_folder moves
every descendant so document ids survive the rename.
index/folders.py derives the folders rows from the union of document
parents and .keep folders and prunes the rows no path needs. It runs on
every folder verb (immediate) and on the full rebuild, gated on a clean
run so a mid-rollback session is never touched. A pruned row replicates
to the UI via Zero, closing the Phase-6 gap where an emptied folder
lingered.
Point the seed at the path law it applies, amend C1's .xml rule (the
unique_identifier_hash demotion lifts it), and note the folder law
closes the Phase-6 emptied-folder gap.
Locks one naming/resolution law for both the git tree and the Postgres
rows: id is identity, the path is an authored-once label. Folders start
in the store and materialize empty ones with a .keep, so an empty folder
survives a rebuild instead of leaving an orphan row. The migration seed
is the per-workspace vehicle that heals the old .xml/id-suffix debt.
The delete gap was not one endpoint. Six HTTP writers reach git; about
twenty do not -- every delete, every move, and the creates that skip the
ingestion pipeline.
Twenty handlers each remembering to call the recorder is how six got
wired and twenty did not, so the fix is a verb on the adapter and callers
that hand it documents rather than paths. Deletes and moves have to
record before the Postgres commit: the path is read from the row that is
about to stop existing.
The canary turned up four files in the store with no row behind them.
DELETE /documents/{id} hands off to a task that never learned about the
store, so the row goes and the file stays -- the deferred REST adapter's
missing half.
That inverts the drift monitor: index_tree treats the surviving file as
truth, re-creates the document, and then reports ok, having just brought
back content a user deleted. Flipping real workspaces on that is how
deletions come back.
Phase 6 planned to fold the projection into the indexer and Phase 4 did; the
canary showed why that couples UI freshness to embedding latency. Marks the
phase shipped with the reversal, and notes the one gap left behind: an emptied
folders row is never pruned, because no diff announces a folder emptying.
Restores the work from #1649, which was reverted on main in #1650 to keep
it out of the last release. Content is identical to that merge.
This is a revert of the revert (a8292f5dc) rather than a merge of
kb_git_mvp, deliberately. Merging the branch would make its commits an
ancestor of both dev and main; main's side deleted those files, so the
next merge between the two branches would silently delete them again. A
revert carries the content without the history, so dev and main share no
ancestor that knows about these files, and the eventual dev -> main merge
sees them as added on one side only and keeps them.
Git becomes the source of truth for knowledge base content; Postgres and
pgvector become a derived, rebuildable index. Both switches guarding the
new path default to off: the KNOWLEDGE_STORE_ENABLED env var and the
per-workspace knowledge_store_enabled column, so merging this changes no
runtime behaviour.
Migrations 175 and 176 only add columns and use ADD COLUMN IF NOT EXISTS.
They are already applied on production, where alembic_version was moved
back to 174 during the revert, so they will re-run harmlessly.
Verified on this branch: 0 conflicts against dev, every app.* import in
the restored files resolves, 2251 unit tests pass. The one collection
error (platforms/google_maps) is missing a fixture that is untracked on
both dev and main, and predates this change.
