WFGY/ProblemMap/GlobalFixMap/Cloud_Serverless/blue_green_switchovers.md

12 KiB
Raw Blame History

Blue-Green Switchovers for Serverless and Edge

🧭 Quick Return to Map

You are in a sub-page of Cloud_Serverless.
To reorient, go back here:

Think of this page as a desk within a ward.
If you need the full triage and all prescriptions, return to the Emergency Room lobby.

A safe pattern to shift 100 percent of live traffic from blue to green without downtime, cache poison, or state skew. Works for API Gateway + Lambda, Cloudflare Workers, Vercel Edge, Fastly Compute, and similar stacks.

Open these first


Core acceptance to flip colors

  • ΔS(question, retrieved) median ≤ 0.45 on gold probes.
  • Coverage ≥ 0.70 to correct section.
  • λ convergent across three paraphrases and two seeds.
  • p95 warm latency within 25 percent of blue.
  • Zero new error class at headers or body read.
  • Cache hit rate within five points once namespaces are split.
  • INDEX_HASH, metric, analyzer match between blue and green or are intentionally versioned.

Topologies that work

Edge weight switch

  • One CDN route, two origins. Blue and green behind a weight.
  • Keep sticky hashing by stable_user_id so dialogs do not jump during ramp.

Header based switch

  • Default to blue. Send x-exp: green for selected cohorts.
  • Propagate header to core services and logs.

DNS label switch

  • api-blue.example.com and api-green.example.com.
  • Edge forwards based on label. Use short TTL only after green passes probes.

Pre-flight checklist

  • Env parity proved with the parity sheet. Open: Runtime Env Parity
  • Secret bundles rotated with overlap. Open: Secrets Rotation
  • Index family built and labeled: docs-v3-green, analyzer and metric recorded.
  • Cache namespaces isolated: blue: vs green: prefixes at edge and core.
  • Queue topics duplicated: jobs.blue and jobs.green. Producers switch at flip time with idempotent keys. Open: Stateless KV & Queues
  • Probe set ready: 50 to 200 questions, three paraphrases, two seeds.

Switch playbook

  1. Shadow validate Replay a sample to green. Block promotion if ΔS ≥ 0.60, coverage < 0.70, or JSON schema violations.

  2. Canary ramp Move one percent of sticky users to green at the edge. Keep cache namespaces split. Open: Canary Release

  3. Green warmup Ensure cold start panels are flat. If p95 warm is worse only due to cold, fix memory or reserve concurrency. Open: Cold Start & Concurrency

  4. Cutover window

    • Freeze new blue writes that create cross request state.
    • Drain blue job queues with a backfill guard.
    • Switch producers to jobs.green.
    • Flip traffic weight to green. Keep blue read only for a short fence window.
  5. Cache migration

  6. Post cutover audit

    • Compare ΔS, coverage, λ, p95 warm to pre-flip baselines.
    • Verify INDEX_HASH and analyzer match.
    • Confirm egress success rate and dedupe guard. Open: Egress & Webhooks
  7. Decommission blue

    • Stop accepting writes.
    • Keep blue for rollback until probe board stays green for one peak hour.

Data and state guidance

  • Databases Prefer live backward compatible migrations. If you need forward only, pause blue writes to the affected tables during the flip.

  • Vector indexes Version by family. Never point green to blue vectors by accident. Verify metric and analyzer, not only the hash.

  • Queues and schedulers Idempotent keys: sha256(source_id + revision + index_hash). Keep retry policies aligned across colors.

  • Caches Always split prefixes. Never reuse the blue prefix for green.


Observability fields to join on

{
  "ts": "2025-08-28T04:00:00Z",
  "route": "chat.rag.answer",
  "color": "green",
  "sticky": "h21",
  "region": "eu-west",
  "edge_pop": "cdg",
  "INDEX_HASH": "0x9a77",
  "retrieval": { "metric": "cosine", "analyzer": "bilstem", "ΔS_q_r": 0.36, "coverage": 0.74, "λ_state": "<>" },
  "cache": { "prefix": "green:", "hit": true },
  "status": 200,
  "latency_ms": { "tffb": 140, "tusable": 320, "tfinal": 1180 }
}

Rollback cookbook


Common traps

  • Mixing cache prefixes so users see stale blue answers after the flip.
  • DNS TTL set short before green is warm.
  • Single region green while blue is multi region.
  • Reranker versions differ between colors.
  • Secrets rotated on edge but not core.

🔗 Quick-Start Downloads (60 sec)

Tool Link 3-Step Setup
WFGY 1.0 PDF Engine Paper 1 Download · 2 Upload to your LLM · 3 Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS) TXTOS.txt 1 Download · 2 Paste into any LLM chat · 3 Type “hello world” — OS boots instantly

Explore More

Layer Page What its for
Proof WFGY Recognition Map External citations, integrations, and ecosystem proof
Engine WFGY 1.0 Original PDF based tension engine
Engine WFGY 2.0 Production tension kernel and math engine for RAG and agents
Engine WFGY 3.0 TXT based Singularity tension engine, 131 S class set
Map Problem Map 1.0 Flagship 16 problem RAG failure checklist and fix map
Map Problem Map 2.0 RAG focused recovery pipeline
Map Problem Map 3.0 Global Debug Card, image as a debug protocol layer
Map Semantic Clinic Symptom to family to exact fix
Map Grandmas Clinic Plain language stories mapped to Problem Map 1.0
Onboarding Starter Village Guided tour for newcomers
App TXT OS TXT semantic OS, fast boot
App Blah Blah Blah Abstract and paradox Q and A built on TXT OS
App Blur Blur Blur Text to image with semantic control
App Blow Blow Blow Reasoning game engine and memory demo

If this repository helped, starring it improves discovery so more builders can find the docs and tools. GitHub Repo stars