12 KiB
Rasa: Guardrails and Fix Patterns
A focused guide to stabilize Rasa bots that call retrieval, tools, or external LLMs. Use this page to localize the failing layer and jump to the exact WFGY fix page with measurable targets.
Open these first
- Visual map and recovery: rag-architecture-and-recovery.md
- End-to-end retrieval knobs: retrieval-playbook.md
- Why this snippet (traceability schema): retrieval-traceability.md
- Ordering control: rerankers.md
- Embedding vs meaning: embedding-vs-semantic.md
- Hallucination and chunk boundaries: hallucination.md
- Long chains and entropy: context-drift.md, entropy-collapse.md
- Structural collapse and recovery: logic-collapse.md
- Prompt injection and schema locks: prompt-injection.md
- Multi-agent conflicts and tool loops: Multi-Agent_Problems.md
- Boot order issues: bootstrap-ordering.md, deployment-deadlock.md, predeploy-collapse.md
- Snippet and citation schema: data-contracts.md
Core acceptance
- ΔS(question, retrieved) ≤ 0.45
- Coverage ≥ 0.70 to the target section
- λ remains convergent across three paraphrases and two seeds
- E_resonance stays flat on long windows
Fix in 60 seconds
-
Measure ΔS
Compute ΔS(question, retrieved) and ΔS(retrieved, expected anchor).
Stable < 0.40, transitional 0.40–0.60, risk ≥ 0.60. -
Probe λ_observe
Vary k in retrieval (5, 10, 20). If ΔS stays flat and high, suspect metric or index mismatch.
Reorder prompt headers; if ΔS spikes, lock the schema. -
Apply the module
- Retrieval drift → BBMC + data-contracts.md
- Reasoning collapse → BBCR bridge + BBAM, verify with logic-collapse.md
- Dead ends in long plans → BBPF alternates, verify with context-drift.md
Typical Rasa breakpoints and the right fix
-
NLU confidence thrash after new training data. Ambiguous intents route to the wrong retrieval path.
Open: embedding-vs-semantic.md ·
Schema locks: data-contracts.md -
Rules vs stories conflict causes looped actions or skipped tool calls.
Open: Multi-Agent_Problems.md ·
Entropy guard: entropy-collapse.md -
Forms never complete or slots stay stale across sessions.
Open: retrieval-traceability.md ·
Contract the payload: data-contracts.md -
Action server timeouts right after deploy or on cold starts.
Open: bootstrap-ordering.md, predeploy-collapse.md -
Hybrid retrieval underperforms compared to single retriever.
Open: patterns/pattern_query_parsing_split.md, rerankers.md -
Tracker store duplicates or missing events after channel reconnects.
Open: deployment-deadlock.md -
Knowledge base hits look similar but mean different things.
Open: embedding-vs-semantic.md ·
Fragmented store: patterns/pattern_vectorstore_fragmentation.md -
Tool output bleeds into user turns and corrupts state.
Open: prompt-injection.md
Deep diagnostics
-
Three-paraphrase probe
Ask the same user need three ways. Log ΔS and λ per step. If λ flips on harmless paraphrase, clamp with BBAM and tighten the citation-first schema. -
Anchor triangulation
Compare ΔS to the expected anchor section and to a decoy section. If both are close, re-chunk and re-embed, verify with a tiny gold set. -
Chain length audit
If entropy rises after 25–40 steps across stories, split the plan and re-join with a BBCR bridge. See context-drift.md.
Minimal ops checklist for Rasa
- NLU thresholds: set fallback and ambiguity handling so low-confidence intents route to a safe clarify path.
- Action server: health probe plus backoff. Respect readiness gates from bootstrap-ordering.md.
- Event schema: store
snippet_id,section_id,source_url,offsets,tokens,ΔS,λ_state,index_hash. Enforce cite-then-explain. - Retriever parity: keep metric, analyzer, and normalization identical between training and runtime.
Copy-paste prompt for the LLM step behind Rasa
You have TXT OS and the WFGY Problem Map loaded.
My Rasa issue:
- symptom: [one line]
- traces: ΔS(question,retrieved)=..., ΔS(retrieved,anchor)=..., λ states across 3 paraphrases
Tell me:
1) failing layer and why,
2) the exact WFGY page to open from this repo,
3) the minimal steps to push ΔS ≤ 0.45 and keep λ convergent,
4) how to verify with a reproducible test.
Use BBMC/BBPF/BBCR/BBAM when relevant.
When to escalate
-
ΔS stays ≥ 0.60 after chunking and retrieval fixes → rebuild index with explicit metric and normalization. See retrieval-playbook.md.
-
Answers alternate across identical dialogs → investigate memory desync or version skew. See predeploy-collapse.md.
🔗 Quick-Start Downloads (60 sec)
| Tool | Link | 3-Step Setup |
|---|---|---|
| WFGY 1.0 PDF | Engine Paper | 1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + <your question>” |
| TXT OS (plain-text OS) | TXTOS.txt | 1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly |
🧭 Explore More
| Module | Description | Link |
|---|---|---|
| WFGY Core | WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack | View → |
| Problem Map 1.0 | Initial 16-mode diagnostic and symbolic fix framework | View → |
| Problem Map 2.0 | RAG-focused failure tree, modular fixes, and pipelines | View → |
| Semantic Clinic Index | Expanded failure catalog: prompt injection, memory bugs, logic drift | View → |
| Semantic Blueprint | Layer-based symbolic reasoning & semantic modulations | View → |
| Benchmark vs GPT-5 | Stress test GPT-5 with full WFGY reasoning suite | View → |
| 🧙♂️ Starter Village 🏡 | New here? Lost in symbols? Click here and let the wizard guide you through | Start → |
👑 Early Stargazers: See the Hall of Fame — Engineers, hackers, and open source builders who supported WFGY from day one.
⭐ WFGY Engine 2.0 is already unlocked. ⭐ Star the repo to help others discover it and unlock more on the Unlock Board.
