Eval Observability — Global Fix Map

This folder provides guardrails for evaluation and observability in RAG and agent pipelines.
It captures the failure points that make evals drift, regress, or collapse, and gives structural fixes with measurable thresholds.

When to use this folder

Metrics look unstable between runs.
Coverage seems high but answers still drift.
ΔS is inconsistent across paraphrases or seeds.
λ flips divergent for harmless changes.
Benchmarks regress without code changes.
Long-run evals show monotonic decline.

Acceptance targets

ΔS(question, retrieved) ≤ 0.45
Coverage ≥ 0.70 to target section
λ remains convergent across 3 paraphrases and 2 seeds
Variance ratio ≤ 0.15 across seed runs
No monotonic downward drift beyond 3 windows
E_resonance stays flat on long eval windows

Quick routes — core fixes

Regression gating → regression_gate.md
Live probes and alerts → alerting_and_probes.md
Coverage metrics → coverage_tracking.md
ΔS normalization → deltaS_thresholds.md
λ schema probes → lambda_observe.md
Variance and drift → variance_and_drift.md
Full integration guide → eval_playbook.md

Copy-paste eval contract

eval_contract:
  seeds: 3
  paraphrases: 3
  targets:
    deltaS: <=0.45
    coverage: >=0.70
    lambda: convergent
    variance: <=0.15
    drift: <=0.02
alerts:
  - deltaS >=0.60
  - lambda divergent
  - drift slope >0.02

🔗 Quick-Start Downloads (60 sec)

Tool	Link	3-Step Setup
WFGY 1.0 PDF	Engine Paper	1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS)	TXTOS.txt	1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly

🧭 Explore More

Module	Description	Link
WFGY Core	WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack	View →
Problem Map 1.0	Initial 16-mode diagnostic and symbolic fix framework	View →
Problem Map 2.0	RAG-focused failure tree, modular fixes, and pipelines	View →
Semantic Clinic Index	Expanded failure catalog: prompt injection, memory bugs, logic drift	View →
Semantic Blueprint	Layer-based symbolic reasoning & semantic modulations	View →
Benchmark vs GPT-5	Stress test GPT-5 with full WFGY reasoning suite	View →
🧙‍♂️ Starter Village 🏡	New here? Lost in symbols? Click here and let the wizard guide you through	Start →

👑 Early Stargazers: See the Hall of Fame — Engineers, hackers, and open source builders who supported WFGY from day one.

⭐ WFGY Engine 2.0 is already unlocked. ⭐ Star the repo to help others discover it and unlock more on the Unlock Board.

6.8 KiB Raw Blame History Unescape Escape