Observability and SLO — Serverless and Edge

🧭 Quick Return to Map

You are in a sub-page of Cloud_Serverless.
To reorient, go back here:

Cloud_Serverless — scalable functions and event-driven pipelines

WFGY Global Fix Map — main Emergency Room, 300+ structured fixes

WFGY Problem Map 1.0 — 16 reproducible failure modes

Think of this page as a desk within a ward.
If you need the full triage and all prescriptions, return to the Emergency Room lobby.

Make failures visible before users feel them. This page defines the SLIs, SLOs, probes, and alerts you need for serverless and edge stacks that run RAG, agents, and tool calls.

When to use this page

Latency or error patterns vary by region or edge POP.
Cold starts and concurrency caps spike at random times.
RAG answers flip on harmless paraphrases and you cannot see why.
Webhooks and egress succeed sometimes and duplicate other times.

Open these first

Visual map and recovery: RAG Architecture & Recovery
Retrieval knobs and ordering: Retrieval Playbook · Rerankers
Trace and prove snippets: Retrieval Traceability · Contract payloads: Data Contracts
Meaning vs distance: Embedding ≠ Semantic
Boot and deploy hazards: Bootstrap Ordering · Deployment Deadlock · Pre-Deploy Collapse
Cloud and edge companions: Cold Start & Concurrency · Timeouts & Streaming · Stateless KV & Queues · Edge Cache Invalidation · Runtime Env Parity · Egress & Webhooks · Pricing vs Latency · Secrets Rotation · Multi-Region Routing · Region Failover Drills · Live ops guides: Live Monitoring for RAG · Debug Playbook

Core SLIs

Availability

Request success rate per region and per edge POP.
Distinguish user cancellations, timeouts at connect, TLS, headers, body, tool call.

Latency

End to end p50 p95 p99 by verb and route. Separate cold vs warm.
Streaming time to first byte, and time to usable first chunk.

RAG quality

ΔS(question, retrieved) and ΔS(retrieved, anchor section).
Coverage to the target section.
λ state across three paraphrases and two seeds.

Cold start pressure

Cold start rate per minute. Concurrency utilization per function.

Queues and webhooks

Backlog length, backlog age p50 p95.
Outbound webhook success, retries, dedupe drops.

Edge and cache

Cache hit ratio for hot prefixes. Invalidation lag.

Index integrity

{INDEX_HASH, METRIC, ANALYZER, BUILD_TS} parity across regions.

SLO examples you can adopt

Availability SLO 99.5 percent monthly per region. Two burn windows: 2 hours and 30 days.
Latency SLO p95 within 25 percent of baseline for served geography. Track separate SLOs for cold and warm.
RAG SLO ΔS ≤ 0.45 and coverage ≥ 0.70 on a fixed probe set against production indices. λ convergent across three paraphrases.
Webhook SLO 99 percent delivery within 90 seconds with no duplicates for a given dedupe_key.
Queue SLO backlog age p95 ≤ 60 seconds during peak.

Event schema to log on every request

{
  "ts": "2025-08-27T06:21:45Z",
  "region": "eu-west",
  "edge_pop": "cdg",
  "route": "chat.rag.answer",
  "cold_start": false,
  "concurrency_util": 0.62,
  "latency_ms": { "tffb": 180, "tusable": 400, "tfinal": 1450 },
  "status": 200,
  "timeout_stage": null,
  "retrieval": {
    "k": 10,
    "metric": "cosine",
    "analyzer": "bilstem",
    "INDEX_HASH": "a9c1…",
    "ΔS_q_r": 0.31,
    "ΔS_r_anchor": 0.28,
    "coverage": 0.78,
    "λ_state": "<>"
  },
  "webhook": { "emitted": true, "tries": 1, "dedupe_key": "sha256(...)" },
  "queue": { "enq": 12, "deq": 12, "backlog": 3, "age_p95_ms": 4200 }
}

Keep PII out. Redact secrets at source.

Dashboards that catch real incidents

Availability and error breakdown by stage: connect, TLS, headers, body read, tool call.
p50 p95 p99 latency split by cold and warm.
Cold start rate and concurrency utilization.
RAG probe board: ΔS histogram, coverage violin, λ flip count.
Queue backlog and age percentiles with alarm lines.
Webhook success and dedupe drops.
Edge cache hit and purge counts per prefix.
Index parity table per region showing INDEX_HASH.

Burn rate alerts you can paste

Availability budget

Error budget over 30 days: EB = 1 - 0.995 = 0.005
High burn alert: 2 hour window where burn = errors / requests > 14 * EB / 720
Slow burn alert: 24 hour window where burn > 6 * EB / 720

Latency budget

Define baseline p95_ref. Alert when p95_now > 1.25 * p95_ref for 10 minutes and cold rate not the cause.

RAG quality

Alert when ΔS median on probe set ≥ 0.60 or λ flips on more than 1 of 3 paraphrases.
Gate deploy if coverage on probes < 0.70.

Probes to run continuously

Three paraphrases per gold question. Log ΔS, coverage, λ.
Two-seed repeat for the same inputs. If answers flip, clamp variance using BBAM and lock snippet schema.
Anchor triangulation against a decoy section to detect bad chunking. Open: Retrieval Traceability · Embedding ≠ Semantic

Instrumentation tips per layer

Cold vs warm separation Tag every invocation with cold_start. Keep distinct latency panels. Open: Cold Start & Concurrency
Timeout anatomy Record stage where it failed. Add separate counters for connect, TLS, headers, body, tool. Open: Timeouts & Streaming
Queues Always report backlog and age percentiles. Alert on age, not only length. Open: Stateless KV & Queues
Edge caches Expose purge counts and hit ratio by prefix and tenant. Open: Edge Cache Invalidation
Region and routing Add stickiness headers and show which region served. Log weights and hysteresis values. Open: Multi-Region Routing
Egress and webhooks Include dedupe_key, tries, final status, and emitter region. Open: Egress & Webhooks

Release gates that prevent regressions

Block deploy if probe set fails any of: ΔS ≤ 0.45, coverage ≥ 0.70, λ convergent.
Block if p95 increased more than 25 percent at warm for 10 minutes.
Block if cold start rate doubles against last week baseline.
Block if INDEX_HASH changed but analyzer or metric changed silently.

Escalate with the right fix

Answers drift across regions → verify INDEX_HASH, rebuild, purge caches. Open: Region Failover Drills
High similarity but wrong meaning → fix chunking and metrics. Open: Embedding ≠ Semantic
Flip states on long chains → clamp with BBAM and shorten plans. Open: Context Drift · Entropy Collapse · Logic Collapse

🔗 Quick-Start Downloads (60 sec)

Tool	Link	3-Step Setup
WFGY 1.0 PDF	Engine Paper	1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS)	TXTOS.txt	1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly

🧭 Explore More

Module	Description	Link
WFGY Core	WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack	View →
Problem Map 1.0	Initial 16-mode diagnostic and symbolic fix framework	View →
Problem Map 2.0	RAG-focused failure tree, modular fixes, and pipelines	View →
Semantic Clinic Index	Expanded failure catalog: prompt injection, memory bugs, logic drift	View →
Semantic Blueprint	Layer-based symbolic reasoning & semantic modulations	View →
Benchmark vs GPT-5	Stress test GPT-5 with full WFGY reasoning suite	View →
🧙‍♂️ Starter Village 🏡	New here? Lost in symbols? Click here and let the wizard guide you through	Start →