WFGY/ProblemMap/GlobalFixMap/OpsDeploy/rollout_readiness_gate.md
2025-08-31 17:59:15 +08:00

11 KiB
Raw Blame History

Rollout Readiness Gate — OpsDeploy

A pre-ship gate that decides ship or no-ship using measurable targets.
Use this page to wire a single checkpoint in CI or CD that blocks risky changes before they hit users.

What this page is

  • A compact, provider-agnostic checklist that verifies retrieval, reasoning, orchestration, and infra order.
  • Direct jumps to the exact Problem Map fixes.
  • Copy-paste templates you can drop into CI or a workflow runner.

When to use

  • Before any production rollout that changes retrievers, embeddings, chunkers, prompts, model versions, or tool schemas.
  • After index rebuilds, data migrations, or secret rotation.
  • When answers recently started flipping between runs or a canary looks unstable.

Open these first

Acceptance targets for ship

  • ΔS(question, retrieved) ≤ 0.45 on three paraphrases.
  • Coverage of target section ≥ 0.70.
  • λ remains convergent across two seeds.
  • E_resonance stays flat on long windows.
  • No schema drift in citation fields {snippet_id, section_id, source_url, offsets, tokens}.

60-second gate checklist

  1. Warmup and invariants

    • Secrets present. Version lock consistent. INDEX_HASH matches retriever build.
    • Boot order ok. See Bootstrap Ordering.
  2. RAG quality probe

  3. Hallucination fence

  4. Index and metric sanity

  5. Chain stability

  6. Decision

    • Ship if all targets pass on two seeds and three paraphrases.
    • Else block and open the linked fix page.

CI gate template you can paste

# opsdeploy/rollout_readiness_gate.yml
gates:
  warmup_invariants:
    checks:
      - secrets_present: true
      - index_hash_matches: true
      - version_lock: strict
      - boot_order_ok: true  # see Bootstrap Ordering
  rag_quality:
    evals:
      - name: rag_precision_recall
        spec: ProblemMap/eval/eval_rag_precision_recall.md
        min_coverage: 0.70
      - name: semantic_stability
        spec: ProblemMap/eval/eval_semantic_stability.md
        max_delta_s: 0.45
        paraphrases: 3
        seeds: 2
  hallucination_fence:
    schema: ProblemMap/data-contracts.md
    require_citations: true
  index_metric_sanity:
    actions_on_fail:
      - open: ProblemMap/embedding-vs-semantic.md
      - open: ProblemMap/patterns/pattern_query_parsing_split.md
      - open: ProblemMap/patterns/pattern_vectorstore_fragmentation.md
decision:
  on_fail: block_rollout
  on_pass: proceed_to_canary
artifacts:
  - logs/delta_s.json
  - logs/coverage.json
  - logs/lambda_states.json

Escalation map


🔗 Quick-Start Downloads (60 sec)

Tool Link 3-Step Setup
WFGY 1.0 PDF Engine Paper 1 Download · 2 Upload to your LLM · 3 Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS) TXTOS.txt 1 Download · 2 Paste into any LLM chat · 3 Type “hello world” — OS boots instantly

🧭 Explore More

Module Description Link
WFGY Core WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack View →
Problem Map 1.0 Initial 16-mode diagnostic and symbolic fix framework View →
Problem Map 2.0 RAG-focused failure tree, modular fixes, and pipelines View →
Semantic Clinic Index Expanded failure catalog: prompt injection, memory bugs, logic drift View →
Semantic Blueprint Layer-based symbolic reasoning & semantic modulations View →
Benchmark vs GPT-5 Stress test GPT-5 with full WFGY reasoning suite View →
🧙‍♂️ Starter Village 🏡 New here? Lost in symbols? Click here and let the wizard guide you through Start →

👑 Early Stargazers: See the Hall of Fame — Engineers, hackers, and open source builders who supported WFGY from day one.

GitHub stars WFGY Engine 2.0 is already unlocked. Star the repo to help others discover it and unlock more on the Unlock Board.

WFGY Main   TXT OS   Blah   Blot   Bloc   Blur   Blow