An opened Hugging Face embedder repo that cannot run says why in a sentence
the backend words, so it was English in every language. Four of its five
refusals now carry a NotAnEmbedderCode in the row's not_runnable_code:
gated, no ONNX build, no tokenizer.json, flagged by Hugging Face's scan.
The interface has a line for each, in ten languages. The fifth names a
file and keeps travelling as its English sentence.
Refs #1967
An accepted ADR takes only Status-line and link edits, so the Correction
section goes and its fact moves to the Status line. ADR 0036 linked to the
Known gaps line this branch deletes; it now links to where the note lives.
The cosine against the int8 export was a mean over passages paired with a
minimum over queries: state both figures over the 66 queries, and the
passages' own beside them.
Move the model step's files into kinds/ (slot, step kind, step models, and the per-kind hooks), local/ (Hugging Face search, local choices and list) and server/ (server option and path). Move the per-step copy out of model-step.tsx into step-copy.ts. Rename use-slot-delete.ts to step-deletes.ts, since it also handles deletes on the embedding step.
Reword the embedding copy to say plainly that the choice is permanent:
- the step body says what an embedding model is for
- the notice says it can't be changed later, and that the default works well for English
- the search note no longer repeats that the choice is permanent
- Settings says the model was chosen in onboarding and can't be changed
Update all ten translations, the tests and the doc links to match.
The embedding model is now chosen in the last onboarding step and fixed
for the whole library. Skipping keeps bge-small, which ships with the
app and can't be deleted. Changing the model later is not built.
- An embedding_indexes row records the model's spec and its vector
table, and each document records which index embedded it. Migration
0021 stamps existing libraries with bge-small. Routes that queue
embedding work return 409 embedding_not_chosen until a model is
chosen.
- Ingest, Studio and search embed through the active index; the old
worker embedding module is removed. Workers wait for the schema, and
documents interrupted when the app closed are marked failed.
- Curated embedders (bge-small, granite-97m, granite-311m) live in the
local manifest, run by a new onnxruntime engine. EMBEDDING is a
catalog type, never a selection slot.
- Hugging Face search finds ONNX embedders. Each download is checked: a
probe sets its width, and a ten-question search check must pass.
Small tokenizer.json files that Hugging Face lists without a checksum
are hashed at the pinned commit. Results use the GGUF search's shapes.
- Onboarding renders the embedding step with the same component and
shared list as the other steps; only its choice hook differs. Use
marks the choice and Finish sends it. Bundled models have no Delete.
- Settings has a read-only Embedding section.
- ADRs 0036 and 0037, a new embedding architecture doc, and updates to
the docs it touches. New strings are translated into every language.