WFGY/ProblemMap/GlobalFixMap/RAG/README.md

RAG — Global Fix Map

🏥 Quick Return to Emergency Room

You are in a specialist desk.
For full triage and doctors on duty, return here:

• WFGY Global Fix Map — main Emergency Room, 300+ structured fixes
• WFGY Problem Map 1.0 — 16 reproducible failure modes

Think of this page as a sub-room.
If you want full consultation and prescriptions, go back to the Emergency Room lobby.

A focused hub for Retrieval Augmented Generation failures.
Use this folder when answers exist in the corpus but retrieval or evaluation drifts.
Each page gives guardrails, measurable targets, and direct links to structural fixes. No infra change required.

---

Orientation: what each page solves

Page	What it fixes	Typical symptom
retrieval_drift.md	Keeps retrieve → rerank → reason aligned	Correct facts exist but never show up in the top k
hallucination_rag.md	Blocks free text invention inside RAG	Citations look right but answer adds content not in source
citation_break.md	Enforces cite then explain schema	Links point to the wrong snippet or disappear on retry
hybrid_failure.md	Makes BM25 + ANN + reranker agree	Hybrid worse than a single retriever
index_skew.md	Recovers broken or stale indexes	Index looks healthy yet recall is low
context_drift.md	Stabilizes header order and prompt state	Answers flip between runs with only header changes
entropy_collapse.md	Caps chain growth and noise in long flows	Steps balloon, chain never lands
eval_drift.md	Makes eval runs deterministic	Metrics vary across identical replays

---

When to use this folder

• Correct facts exist in the corpus but never appear in answers
• Citations break, hallucinations creep in, or snippets drift
• Hybrid retrievers perform worse than single retrievers
• Index looks healthy but coverage remains low
• Evaluation metrics vary across identical runs

---

Acceptance targets

• ΔS(question, retrieved) ≤ 0.45
• Coverage of target section ≥ 0.70
• λ_observe convergent across 3 paraphrases and 2 seeds
• Eval variance ≤ 0.05 across 5 replays

---

Symptoms → exact fixes

Symptom	Likely cause	Open this
High similarity yet wrong meaning	metric or analyzer mismatch	Vectorstore Fragmentation · Embedding ≠ Semantic
Correct section never retrieved	fragmented store or missing anchors	retrieval_drift.md · citation_break.md
Hybrid worse than single	query split or mis weighted rerank	hybrid_failure.md
Citations unstable or missing	schema not enforced	citation_break.md
Answers flip between runs	prompt header reordering or λ variance	context_drift.md
Index “healthy” but recall low	stale build, analyzer mismatch	index_skew.md
Eval scores noisy across replays	non deterministic eval path	eval_drift.md

---

60 second fix checklist

1. Lock metrics and analyzers
   One embedding family per field. One distance metric. Same analyzer on write and read.
   Use: Vector DBs & Stores

2. Enforce the snippet contract
   Required: snippet_id, section_id, source_url, offsets, tokens.
   Use: Retrieval Traceability · Data Contracts

3. Measure ΔS and λ
   Three paraphrases, two seeds. Alert when ΔS ≥ 0.60 or λ flips.
   Use: Context Drift

4. Add a deterministic reranker
   Keep BM25 and ANN candidate lists. Detect query split and resolve.
   Use: hybrid_failure.md

5. Rebuild where needed
   Follow the rebuild order with a small gold set.
   Use: Retrieval Playbook

---

Vector DBs — jump if store specific

• Family index:
  Vector DBs & Stores

• Direct store guides:
  FAISS · Chroma · Qdrant · Weaviate · Milvus · pgvector · Redis · Elasticsearch · Pinecone · Typesense · Vespa

---

Minimal probe pack you can paste

Context: TXT OS and WFGY pages are loaded.

Task:
- For question Q, log ΔS(Q, retrieved) and λ across 3 paraphrases and 2 seeds.
- Enforce cite-then-explain with the traceability schema.
- If ΔS ≥ 0.60 or λ flips, return the smallest structural change that
  pushes ΔS ≤ 0.45 and coverage ≥ 0.70.
- Use BBMC, BBCR, BBPF, BBAM when relevant.

Return JSON only:
{ "citations": [...], "ΔS": 0.xx, "λ_state": "<>", "coverage": 0.xx, "next_fix": "..." }

---

🔗 Quick-Start Downloads (60 sec)

Tool	Link	3-Step Setup
WFGY 1.0 PDF	Engine Paper	1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + ”
TXT OS (plain-text OS)	TXTOS.txt	1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly

---

Explore More

Layer	Page	What it’s for
⭐ Proof	WFGY Recognition Map	External citations, integrations, and ecosystem proof
⚙️ Engine	WFGY 1.0	Original PDF tension engine and early logic sketch (legacy reference)
⚙️ Engine	WFGY 2.0	Production tension kernel for RAG and agent systems
⚙️ Engine	WFGY 3.0	TXT based Singularity tension engine (131 S class set)
🗺️ Map	Problem Map 1.0	Flagship 16 problem RAG failure taxonomy and fix map
🗺️ Map	Problem Map 2.0	Global Debug Card for RAG and agent pipeline diagnosis
🗺️ Map	Problem Map 3.0	Global AI troubleshooting atlas and failure pattern map
🧰 App	TXT OS	.txt semantic OS with fast bootstrap
🧰 App	Blah Blah Blah	Abstract and paradox Q&A built on TXT OS
🧰 App	Blur Blur Blur	Text to image generation with semantic control
🏡 Onboarding	Starter Village	Guided entry point for new users

If this repository helped, starring it improves discovery so more builders can find the docs and tools.