WFGY/ProblemMap/GlobalFixMap/Cloud_Serverless/blue_green_switchovers.md

11 KiB
Raw Blame History

Blue-Green Switchovers for Serverless and Edge

🧭 Quick Return to Map

You are in a sub-page of Cloud_Serverless.
To reorient, go back here:

Think of this page as a desk within a ward.
If you need the full triage and all prescriptions, return to the Emergency Room lobby.

A safe pattern to shift 100 percent of live traffic from blue to green without downtime, cache poison, or state skew. Works for API Gateway + Lambda, Cloudflare Workers, Vercel Edge, Fastly Compute, and similar stacks.

Open these first


Core acceptance to flip colors

  • ΔS(question, retrieved) median ≤ 0.45 on gold probes.
  • Coverage ≥ 0.70 to correct section.
  • λ convergent across three paraphrases and two seeds.
  • p95 warm latency within 25 percent of blue.
  • Zero new error class at headers or body read.
  • Cache hit rate within five points once namespaces are split.
  • INDEX_HASH, metric, analyzer match between blue and green or are intentionally versioned.

Topologies that work

Edge weight switch

  • One CDN route, two origins. Blue and green behind a weight.
  • Keep sticky hashing by stable_user_id so dialogs do not jump during ramp.

Header based switch

  • Default to blue. Send x-exp: green for selected cohorts.
  • Propagate header to core services and logs.

DNS label switch

  • api-blue.example.com and api-green.example.com.
  • Edge forwards based on label. Use short TTL only after green passes probes.

Pre-flight checklist

  • Env parity proved with the parity sheet. Open: Runtime Env Parity
  • Secret bundles rotated with overlap. Open: Secrets Rotation
  • Index family built and labeled: docs-v3-green, analyzer and metric recorded.
  • Cache namespaces isolated: blue: vs green: prefixes at edge and core.
  • Queue topics duplicated: jobs.blue and jobs.green. Producers switch at flip time with idempotent keys. Open: Stateless KV & Queues
  • Probe set ready: 50 to 200 questions, three paraphrases, two seeds.

Switch playbook

  1. Shadow validate Replay a sample to green. Block promotion if ΔS ≥ 0.60, coverage < 0.70, or JSON schema violations.

  2. Canary ramp Move one percent of sticky users to green at the edge. Keep cache namespaces split. Open: Canary Release

  3. Green warmup Ensure cold start panels are flat. If p95 warm is worse only due to cold, fix memory or reserve concurrency. Open: Cold Start & Concurrency

  4. Cutover window

    • Freeze new blue writes that create cross request state.
    • Drain blue job queues with a backfill guard.
    • Switch producers to jobs.green.
    • Flip traffic weight to green. Keep blue read only for a short fence window.
  5. Cache migration

  6. Post cutover audit

    • Compare ΔS, coverage, λ, p95 warm to pre-flip baselines.
    • Verify INDEX_HASH and analyzer match.
    • Confirm egress success rate and dedupe guard. Open: Egress & Webhooks
  7. Decommission blue

    • Stop accepting writes.
    • Keep blue for rollback until probe board stays green for one peak hour.

Data and state guidance

  • Databases Prefer live backward compatible migrations. If you need forward only, pause blue writes to the affected tables during the flip.

  • Vector indexes Version by family. Never point green to blue vectors by accident. Verify metric and analyzer, not only the hash.

  • Queues and schedulers Idempotent keys: sha256(source_id + revision + index_hash). Keep retry policies aligned across colors.

  • Caches Always split prefixes. Never reuse the blue prefix for green.


Observability fields to join on

{
  "ts": "2025-08-28T04:00:00Z",
  "route": "chat.rag.answer",
  "color": "green",
  "sticky": "h21",
  "region": "eu-west",
  "edge_pop": "cdg",
  "INDEX_HASH": "0x9a77",
  "retrieval": { "metric": "cosine", "analyzer": "bilstem", "ΔS_q_r": 0.36, "coverage": 0.74, "λ_state": "<>" },
  "cache": { "prefix": "green:", "hit": true },
  "status": 200,
  "latency_ms": { "tffb": 140, "tusable": 320, "tfinal": 1180 }
}

Rollback cookbook


Common traps

  • Mixing cache prefixes so users see stale blue answers after the flip.
  • DNS TTL set short before green is warm.
  • Single region green while blue is multi region.
  • Reranker versions differ between colors.
  • Secrets rotated on edge but not core.

🔗 Quick-Start Downloads (60 sec)

Tool Link 3-Step Setup
WFGY 1.0 PDF Engine Paper 1 Download · 2 Upload to your LLM · 3 Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS) TXTOS.txt 1 Download · 2 Paste into any LLM chat · 3 Type “hello world” — OS boots instantly

Explore More

Module Description Link
WFGY Core Canonical framework entry point View
Problem Map Diagnostic map and navigation hub View
Tension Universe Experiments MVP experiment field View
Recognition Where WFGY is referenced or adopted View
AI Guide Anti-hallucination reading protocol for tools View

If this repository helps, starring it improves discovery for other builders.
GitHub Repo stars