mirror of
https://github.com/onestardao/WFGY.git
synced 2026-04-28 11:40:07 +00:00
130 lines
3.7 KiB
Text
130 lines
3.7 KiB
Text
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 🧪 WFGY Diagnostic Notebook (v0.5 — widget-free)\n",
|
||
"Measure **ΔS** and classify common reasoning failures with one click.\n",
|
||
"No API key • No widgets • Works in any Colab session."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 📐 Core formula\n",
|
||
"$$\\Delta S = 1 - \\cos\\theta\\bigl(I, G\\bigr)$$ \n",
|
||
"*$I$ = current-step embedding, $G$ = ground-truth / prompt embedding.*"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"### 🔎 What does ΔS mean?\n",
|
||
"\n",
|
||
"ΔS = 1 − cosθ(I, G) \n",
|
||
"*I = Prompt intent, G = Generated output*\n",
|
||
"\n",
|
||
"📉 **Low ΔS** → Stable \n",
|
||
"🔺 **Medium ΔS** → Interpretation Collapse \n",
|
||
"🚨 **High ΔS** → Hallucination or Chunk Drift"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": { "id": "install" },
|
||
"source": [
|
||
"!pip -q install sentence-transformers --upgrade"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": { "id": "imports" },
|
||
"source": [
|
||
"from sentence_transformers import SentenceTransformer, util"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": { "id": "model" },
|
||
"source": [
|
||
"model = SentenceTransformer('all-MiniLM-L6-v2')"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": { "id": "functions" },
|
||
"source": [
|
||
"def delta_s(a_vec, b_vec):\n",
|
||
" return 1 - util.cos_sim(a_vec, b_vec).item()\n",
|
||
"\n",
|
||
"def classify_failure(dS):\n",
|
||
" if dS > 0.80:\n",
|
||
" return ('No.1 Hallucination / Chunk Drift', 'Apply BBMC + boundary check')\n",
|
||
" if dS > 0.60:\n",
|
||
" return ('No.2 Interpretation Collapse', 'Insert λ_observe checkpoint')\n",
|
||
" if dS > 0.40:\n",
|
||
" return ('No.6 Logic Collapse', 'Trigger BBCR fallback')\n",
|
||
" if dS > 0.25:\n",
|
||
" return ('No.9 Entropy Collapse', 'Use BBAM attention modulation')\n",
|
||
" return ('Stable (ΔS ≤ 0.25)', 'No action — pass')"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## ✏️ Edit & run\n",
|
||
"Replace the text in **Prompt** / **Answer**, press ▶️ to diagnose."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"metadata": { "id": "user_input" },
|
||
"source": [
|
||
"prompt = \"### Product Review\\nThe new phone has a crystal-clear display and fast charging…\"\n",
|
||
"answer = \"The capital of France is Paris. Also, your phone seems great!\"\n",
|
||
"\n",
|
||
"# --- diagnostic ---\n",
|
||
"dS = delta_s(model.encode(prompt, convert_to_tensor=True),\n",
|
||
" model.encode(answer, convert_to_tensor=True))\n",
|
||
"\n",
|
||
"failure, fix = classify_failure(dS)\n",
|
||
"print(\"ΔS :\", f\"{dS:.3f}\")\n",
|
||
"print(\"Failure detected :\", failure)\n",
|
||
"print(\"Suggested fix :\", fix)"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"---\n",
|
||
"### Next Steps \n",
|
||
"* Explore **Problem Map 1.0** & **2.0** for full 16 failure modes. \n",
|
||
"* Read about **λ_observe, E_resonance, BBCR** in the WFGY 1.0 paper. \n",
|
||
"* Fork & extend this notebook — MIT-licensed. \n",
|
||
"\n",
|
||
"> Repo → https://github.com/onestardao/WFGY"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" },
|
||
"language_info": { "name": "python" }
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|