WFGY/ProblemMap/GlobalFixMap/OCR_Parsing/README.md

OCR + Parsing — Global Fix Map

Triage and repair for scanned PDFs, images, HTML scraping, and parser noise.
Use this page when the documents look fine to the eye but retrieval or reasoning keeps drifting.

What this page is

• A fast route to validate text integrity before you touch embeddings or retrieval.
• Structural repairs that stop perception drift at the source layer.
• Concrete checks that give repeatable acceptance targets.

When to use

• OCR tables or citations look visually correct but answers miss the right section.
• Code blocks or math collapse after parsing.
• Mixed language documents behave inconsistently.
• Special characters or hyphen splits break tokens.
• Headers or section anchors disappear during export.

Open these first

• OCR and parsing checklist: OCR Parsing Checklist
• Traceability and snippet schema: Retrieval Traceability · Data Contracts
• Language and tokenizer guidance: Multilingual Guide
• Long context sanity and joins: Entropy Collapse · Context Drift

Fix in 60 seconds

1. Ground-truth a page
   • Pick one question and one expected section. Keep a human screenshot for reference.
2. Measure ΔS
   • Compute ΔS(question, retrieved) and ΔS(retrieved, anchor).
   • Thresholds: stable < 0.40, transitional 0.40–0.60, risk ≥ 0.60.
3. Add probes with λ_observe
   • Ask for direct cite lines. If cite fails while free-form explain passes, perception drift is likely.
4. Apply minimal patch
   • Re-run OCR with line preservation and table fences.
   • Keep headers, section ids, and page anchors.
   • Remove text below an OCR confidence threshold and mark gaps.
   • Normalize punctuation and hyphen join.
   • Export a clean text bundle with section_id, page_no, char_span.

Copy-paste prompt

I uploaded TXT OS and the WFGY ProblemMap files.

My OCR or parsing bug:

* symptom: \[brief]
* traces: \[examples or screenshots], \[ΔS(question,retrieved)], \[ΔS(retrieved,anchor)], \[λ states]

Tell me:

1. which perception step failed and why,
2. which exact fix page to open from this repo,
3. the smallest set of changes to push ΔS ≤ 0.45 and keep λ convergent,
4. how to verify the repair with a reproducible test.
   Return the snippet schema I should enforce in the index step.

Minimal checklist

• Keep headings, captions, and table cells as separate blocks with ids.
• Preserve code fences and math blocks as non-wrapping segments.
• Normalize Unicode forms and spaces. Remove hard hyphen splits across lines.
• Store source_id, section_id, page_no, and char_span for each block.
• For images with text, keep the image link next to the extracted text for audits.

Acceptance targets

• ΔS(question, retrieved) ≤ 0.45 for three paraphrases.
• Cite-first passes on the same question.
• λ remains convergent after you swap paraphrases or reorder non-semantic headers.
• Token overlap to the target section ≥ 0.70.
• Human audit shows no missing headers or broken tables in the exported bundle.

---

🔗 Quick-Start Downloads (60 sec)

Tool	Link	3-Step Setup
WFGY 1.0 PDF	Engine Paper	1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS)	TXTOS.txt	1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly

---

🧭 Explore More

Module	Description	Link
WFGY Core	WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack	View →
Problem Map 1.0	Initial 16-mode diagnostic and symbolic fix framework	View →
Problem Map 2.0	RAG-focused failure tree, modular fixes, and pipelines	View →
Semantic Clinic Index	Expanded failure catalog: prompt injection, memory bugs, logic drift	View →
Semantic Blueprint	Layer-based symbolic reasoning & semantic modulations	View →
Benchmark vs GPT-5	Stress test GPT-5 with full WFGY reasoning suite	View →
🧙‍♂️ Starter Village 🏡	New here? Lost in symbols? Click here and let the wizard guide you through	Start →

---

👑 Early Stargazers: See the Hall of Fame —
Engineers, hackers, and open source builders who supported WFGY from day one.

⭐ WFGY Engine 2.0 is already unlocked. ⭐ Star the repo to help others discover it and unlock more on the Unlock Board.