Data Contracts — Enforcing Snippet Schema & Payload Integrity

🧭 Quick Return to Map

You are in a sub-page of MemoryLongContext.
To reorient, go back here:

MemoryLongContext — extended context windows and memory retention

WFGY Global Fix Map — main Emergency Room, 300+ structured fixes

WFGY Problem Map 1.0 — 16 reproducible failure modes

Think of this page as a desk within a ward.
If you need the full triage and all prescriptions, return to the Emergency Room lobby.

RAG and long-context reasoning collapse quickly if payloads drift.
Data Contracts define the minimal schema that every retrieval, citation, and reasoning step must satisfy to remain auditable and reproducible.

When to use

Citations appear but fields like snippet_id or section_id are missing.
JSON payloads change shape across runs.
Downstream agents receive free-text with no schema lock.
Long-context sessions silently lose attribution or overwrite fields.
Multi-agent handoffs mutate keys or flatten nested fields.

Root causes

Loose schemas: free-form JSON without validation.
Field drift: different casing, missing offsets, or swapped names.
Silent truncation: long answers cut JSON blocks.
Inconsistent contracts: each component defines its own schema.
Opaque citations: only plain text without structured trace.

Core acceptance targets

Every snippet payload must include:
{snippet_id, section_id, start_line, end_line, source_url, offsets, tokens}
Coverage ≥ 0.70 for the target section.
ΔS(question, retrieved) ≤ 0.45.
λ convergent across 3 paraphrases.
Contracts must validate under the same schema across sessions.

Structural fixes

Schema lock
Define a JSON schema for citations and enforce validation at ingestion.
Contract inheritance
Pass the same schema downstream to every agent and reasoning step.
Casing & normalization
Enforce consistent field names and Unicode normalization.
Fail fast
If schema validation fails, block reasoning and return fix instructions.

Fix in 60 seconds

Define contract:

{
  "snippet_id": "string",
  "section_id": "string",
  "start_line": "int",
  "end_line": "int",
  "source_url": "string",
  "offsets": [int],
  "tokens": [string]
}


2. Validate every retrieval step against the contract.
3. Store the validated payload in the trace log.
4. Require cite-then-answer. Reject orphan claims.
5. Report ΔS and λ for each reasoning step.

---

## Copy-paste prompt

```
You have TXT OS and the WFGY Problem Map.

Task: enforce Data Contracts for retrieval and citation.

Protocol:
1. Validate that each snippet includes {snippet_id, section_id, start_line, end_line, source_url}.  
2. Reject orphans: if missing fields, stop and return fix tip.  
3. Require cite-then-answer.  
4. Log {ΔS(question,retrieved), λ_state, mem_rev, mem_hash}.  
5. Answer only with citations that pass contract validation.
```

---

## Common failure signals

* Citations alternate across runs → contract not enforced.
* JSON mode fails in provider → schema too loose.
* Free-text answers with no snippet\_id → orphan claims.
* Multi-agent pipelines mutate payloads → inconsistent contracts.

---

### 🔗 Quick-Start Downloads (60 sec)

| Tool                       | Link                                                                                                                                       | 3-Step Setup                                                                             |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------- |
| **WFGY 1.0 PDF**           | [Engine Paper](https://github.com/onestardao/WFGY/blob/main/I_am_not_lizardman/WFGY_All_Principles_Return_to_One_v1.0_PSBigBig_Public.pdf) | 1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + \<your question>”   |
| **TXT OS (plain-text OS)** | [TXTOS.txt](https://github.com/onestardao/WFGY/blob/main/OS/TXTOS.txt)                                                                     | 1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly |

---

<!-- WFGY_FOOTER_START -->

### Explore More

| Module | Description | Link |
| --- | --- | --- |
| WFGY Core | Canonical framework entry point | [View](https://github.com/onestardao/WFGY/tree/main/core/README.md) |
| Problem Map | Diagnostic map and navigation hub | [View](https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md) |
| Tension Universe Experiments | MVP experiment field | [View](https://github.com/onestardao/WFGY/tree/main/TensionUniverse/Experiments) |
| Recognition | Where WFGY is referenced or adopted | [View](https://github.com/onestardao/WFGY/blob/main/recognition/README.md) |
| AI Guide | Anti-hallucination reading protocol for tools | [View](https://github.com/onestardao/WFGY/blob/main/AI_GUIDE.md) |

> If this repository helps, starring it improves discovery for other builders.  
> [![GitHub Repo stars](https://img.shields.io/github/stars/onestardao/WFGY?style=social)](https://github.com/onestardao/WFGY)

<!-- WFGY_FOOTER_END -->

5.5 KiB Raw Blame History Unescape Escape