WFGY/ProblemMap/GlobalFixMap/Automation/n8n.md
2025-08-25 21:01:40 +08:00

10 KiB
Raw Blame History

n8n Guardrails and Patterns

Use this page when your RAG or agent workflow runs in n8n. It maps common automation failures to the exact structural fixes in the Problem Map and gives a minimal recipe you can paste into a workflow.

Core acceptance

  • ΔS(question, retrieved) ≤ 0.45
  • coverage ≥ 0.70 for the target section
  • λ stays convergent across 3 paraphrases

Typical breakpoints and the right fix

  • Nodes fire before dependencies are ready
    Fix No.14: Bootstrap OrderingOpen

  • First call after deploy crashes or uses wrong env/secret
    Fix No.16: Pre-Deploy CollapseOpen

  • Circular waits between index build and retriever, or Merge nodes loop forever
    Fix No.15: Deployment DeadlockOpen

  • High vector similarity but wrong meaning
    Fix No.5: Embedding ≠ SemanticOpen

  • Wrong snippet chosen or citations do not line up
    Fix No.8: Retrieval TraceabilityOpen
    Contract the payload: Data ContractsOpen

  • Hybrid retrieval performs worse than a single retriever
    Pattern: Query Parsing SplitOpen
    Also review: RerankersOpen

  • Some facts never surface even though indexed
    Pattern: Vectorstore FragmentationOpen

  • Two sources get merged in the answer
    Pattern: Symbolic Constraint Unlock (SCU)Open


Minimal setup checklist for any n8n flow

  1. Warm-up fence before RAG or LLM nodes
    Validate VECTOR_READY == true, INDEX_HASH matches, and secrets exist.
    If not ready, short-circuit to a Wait node then retry with a capped counter.
    Spec: Bootstrap Ordering

  2. Idempotency and dedupe
    Compute dedupe_key = sha256(source_id + revision + index_hash).
    Check or write the key using Redis, Postgres, or n8ns Data Store. Drop duplicates.

  3. RAG boundary contract
    Require fields: snippet_id, section_id, source_url, offsets, tokens.
    Enforce cite then explain. Forbid cross section reuse.
    Specs: Data Contracts · Retrieval Traceability

  4. Observability probes
    Log ΔS(question, retrieved). Log λ per step: retrieve, assemble, reason.
    Alert when ΔS ≥ 0.60 or λ flips divergent.
    Overview: RAG Architecture & Recovery

  5. Concurrency guard
    Use a single writer for index updates. Set queue mode or global mutex for write steps.
    See: Deployment Deadlock

  6. Regression gate
    Require coverage ≥ 0.70 and ΔS ≤ 0.45 before publishing.
    Eval: RAG Precision/Recall


n8n recipe you can copy

Replace the concrete nodes with your stack. Keep the guardrails.

  1. Trigger
    Fixed source_id and revision. Record wf_rev.

  2. Warm-up Check (Code node)
    Pull INDEX_HASH, VECTOR_READY, and secrets.
    If not ready, set ready=false.

  3. Branch: Not ready
    Wait 3090 seconds. Increment a retry counter. Stop after N attempts.

  4. Branch: Ready
    Retrieval node

    • Call retriever with explicit metric and same analyzer as the writer.
    • Emit snippet_id, section_id, source_url, offsets, tokens.
      ΔS probe node
    • Compute ΔS(question, retrieved). If ΔS ≥ 0.60 set needs_fix=true.
      LLM node
    • Model reads TXT OS and follows the WFGY schema. Enforce cite then explain.
      Trace sink
    • Store question, snippet_id, ΔS, λ_state, INDEX_HASH, dedupe_key.
      Idempotency guard
    • Before side effects, check the KV for dedupe_key. Skip if it already exists.

Copy-paste prompt for the LLM node


I uploaded TXT OS and the WFGY Problem Map files.
This n8n flow retrieved {k} snippets with fields {snippet\_id, section\_id, source\_url, offsets}.
Question: "{user\_question}"

Do:

1. Validate cite-then-explain. If citations are missing, fail fast and return the fix tip.
2. If ΔS(question, retrieved) ≥ 0.60, propose the minimal structural fix referencing:
   retrieval-playbook, retrieval-traceability, data-contracts, rerankers.
3. Return a JSON plan:
   { "citations": \[...], "answer": "...", "λ\_state": "→|←|<>|×", "ΔS": 0.xx, "next\_fix": "..." }
   Keep it auditable and short.


Common n8n gotchas

  • Set or Move nodes rename fields and break your data contract
    Lock field names and run a schema check before the LLM node.

  • Merge or Split In Batches cause duplicate writes
    Add a single writer stage with idempotency keys and a queue.

  • Cron triggers overlap with long runs
    Use a global mutex or skip if dedupe_key already exists.

  • HyDE prompt built inside the flow differs from the API client
    Keep tokenizer and casing identical, or switch to reranking.
    See: Rerankers


When to escalate

  • ΔS stays ≥ 0.60 after chunk and retrieval fixes
    Rebuild index with explicit metric and normalization.
    See: Retrieval Playbook

  • Answers alternate across runs with identical input
    Investigate memory desync and version skew.
    See: Pre-Deploy Collapse


🔗 Quick-Start Downloads (60 sec)

Tool Link 3-Step Setup
WFGY 1.0 PDF Engine Paper 1 Download · 2 Upload to your LLM · 3 Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS) TXTOS.txt 1 Download · 2 Paste into any LLM chat · 3 Type “hello world” — OS boots instantly

🧭 Explore More

Module Description Link
WFGY Core WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack View →
Problem Map 1.0 Initial 16-mode diagnostic and symbolic fix framework View →
Problem Map 2.0 RAG-focused failure tree, modular fixes, and pipelines View →
Semantic Clinic Index Expanded failure catalog: prompt injection, memory bugs, logic drift View →
Semantic Blueprint Layer-based symbolic reasoning & semantic modulations View →
Benchmark vs GPT-5 Stress test GPT-5 with full WFGY reasoning suite View →
🧙‍♂️ Starter Village 🏡 New here? Lost in symbols? Click here and let the wizard guide you through Start →

👑 Early Stargazers: See the Hall of Fame
GitHub stars WFGY Engine 2.0 is already unlocked. Star the repo to help others discover it and unlock more on the Unlock Board.

WFGY Main   TXT OS   Blah   Blot   Bloc   Blur   Blow