vrr/WFGY
2
0
Fork 0
mirror of https://github.com/onestardao/WFGY.git synced 2026-07-09 15:58:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
WFGY/Polaris/experiments/downloads/PP02B.ipynb
PSBigBig + MiniPS c5652e7302
Add files via upload
2026-05-06 17:23:22 +08:00

354 lines
13 KiB
Text
Raw Permalink Blame History

{
"cells": [
{
"cell_type": "markdown",
"id": "e3f5929e",
"metadata": {},
"source": [
"# PP02B — Math Stress Results Review\n",
"\n",
"**Belongs to:** WFGY 5.0 Polaris Protocol \n",
"**Repository:** https://github.com/onestardao/WFGY \n",
"**Polaris Protocol path:** https://github.com/onestardao/WFGY/tree/main/Polaris \n",
"**Experiment page:** https://github.com/onestardao/WFGY/blob/main/Polaris/experiments/README.md\n",
"\n",
"## Experiment spirit\n",
"\n",
"Math stress evidence. The notebook is a review / report Colab over the published results package, not a new universal math proof.\n",
"\n",
"## What this Colab tests\n",
"\n",
"Reviews the PP02B math-stress results package, separating deterministic compiler certificate evidence from model stress evidence.\n",
"\n",
"## Published result summary\n",
"\n",
"- 120 cases; 720 expected / actual total outputs.\n",
"- 600 model API calls in the published run.\n",
"- Group C parse pass rate 1.0; Group C contract pass rate 1.0.\n",
"- Main certificate result: T4_MAIN_CERTIFICATE_PASS.\n",
"\n",
"## Expected usage / token-cost note\n",
"\n",
"Review Colab does not need to call a model API if using the packaged result files. Token/API usage is negligible unless adapted to rerun model stress.\n",
"\n",
"## Scientific boundary\n",
"\n",
"Math stress evidence only. Not universal math proof. Compiler certificate and model stress are separate evidence lanes.\n",
"\n",
"This Colab is provided for **reproduction, inspection, and adaptation**. You do not have to rerun it to read the published result summary. If you rerun it, model behavior, token usage, cost, and output details may vary.\n",
"\n",
"---\n",
"\n",
"\n",
"## How this review Colab works\n",
"\n",
"The original PP02B package in the uploaded archive contains result files but no notebook. \n",
"This Colab is therefore a **results-review Colab**:\n",
"\n",
"1. It looks for the PP02B evidence zip in `evidence_packages/`.\n",
"2. If present, it extracts and lists key files.\n",
"3. It generates a compact report from the published metrics.\n",
"4. It does **not** call a model API by default.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fe81cc6a",
"metadata": {},
"outputs": [],
"source": [
"# Polaris Protocol experiment identity\n",
"POLARIS_PROTOCOL_REPO = \"https://github.com/onestardao/WFGY\"\n",
"POLARIS_EXPERIMENT_ID = \"PP02B\"\n",
"POLARIS_EXPERIMENT_TITLE = \"PP02B — Math Stress Results Review\"\n",
"POLARIS_EXPERIMENT_ROLE = \"Math Stress Results Review\"\n",
"POLARIS_CLAIM_BOUNDARY = \"Math stress evidence only. Not universal math proof. Compiler certificate and model stress are separate evidence lanes.\"\n",
"print(f\"{POLARIS_EXPERIMENT_ID} — {POLARIS_EXPERIMENT_TITLE}\")\n",
"print(f\"Repo: {POLARIS_PROTOCOL_REPO}\")\n"
]
},
{
"cell_type": "markdown",
"id": "352ce80e",
"metadata": {},
"source": [
"## Locate or upload the PP02B evidence package"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68a7e496",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"import zipfile, os, json, csv, pandas as pd\n",
"\n",
"EXPECTED_ZIP_NAME = \"PP02B_SP_MATH_COLAB_RESULTS_v4_5_120CASE_UNIVERSE_REAL_RUN.zip\"\n",
"candidate_paths = [\n",
" Path(\"evidence_packages\") / EXPECTED_ZIP_NAME,\n",
" Path(\"../evidence_packages\") / EXPECTED_ZIP_NAME,\n",
" Path(EXPECTED_ZIP_NAME),\n",
" Path(\"/content\") / EXPECTED_ZIP_NAME,\n",
"]\n",
"\n",
"evidence_zip = next((p for p in candidate_paths if p.exists()), None)\n",
"\n",
"if evidence_zip is None:\n",
" print(\"PP02B evidence zip was not found automatically.\")\n",
" print(\"Upload it manually in Colab if you want to inspect raw package files.\")\n",
" try:\n",
" from google.colab import files\n",
" uploaded = files.upload()\n",
" for name in uploaded:\n",
" if name.endswith(\".zip\"):\n",
" evidence_zip = Path(name)\n",
" break\n",
" except Exception:\n",
" pass\n",
"\n",
"if evidence_zip is not None and evidence_zip.exists():\n",
" print(f\"Using evidence package: {evidence_zip}\")\n",
"else:\n",
" print(\"No evidence package loaded. The final report will use the published metric fallback.\")\n"
]
},
{
"cell_type": "markdown",
"id": "560c165d",
"metadata": {},
"source": [
"## Inspect package structure, if the zip is available"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61bd8f1b",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"import zipfile, pandas as pd, json, os\n",
"\n",
"if 'evidence_zip' in globals() and evidence_zip is not None and evidence_zip.exists():\n",
" extract_dir = Path(\"pp02b_evidence_extracted\")\n",
" extract_dir.mkdir(exist_ok=True)\n",
" with zipfile.ZipFile(evidence_zip) as z:\n",
" z.extractall(extract_dir)\n",
" package_files = z.namelist()\n",
" print(f\"Extracted {len(package_files)} files to {extract_dir}\")\n",
" display(pd.DataFrame({\"file\": package_files[:80]}))\n",
"else:\n",
" package_files = []\n",
" print(\"Skipping package inspection because no zip was loaded.\")\n"
]
},
{
"cell_type": "markdown",
"id": "05be6e71",
"metadata": {},
"source": [
"## Final report"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2ad96299",
"metadata": {},
"outputs": [],
"source": [
"# Final Polaris Protocol report generator\n",
"# This cell uses the published result summary as a stable reporting fallback.\n",
"# If you adapt the notebook and compute fresh metrics, update POLARIS_PUBLISHED_METRICS before running this cell.\n",
"\n",
"from pathlib import Path\n",
"import json, csv, math, re\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"\n",
"POLARIS_REPORT_EXPERIMENT_ID = \"PP02B\"\n",
"POLARIS_REPORT_TITLE = \"PP02B — Math Stress Results Review\"\n",
"POLARIS_REPORT_REPO = \"https://github.com/onestardao/WFGY\"\n",
"POLARIS_REPORT_PURPOSE = \"Reviews the PP02B math-stress results package, separating deterministic compiler certificate evidence from model stress evidence.\"\n",
"POLARIS_REPORT_SPIRIT = \"Math stress evidence. The notebook is a review / report Colab over the published results package, not a new universal math proof.\"\n",
"POLARIS_REPORT_CLAIM_BOUNDARY = \"Math stress evidence only. Not universal math proof. Compiler certificate and model stress are separate evidence lanes.\"\n",
"POLARIS_PUBLISHED_SUMMARY = [\n",
" \"120 cases; 720 expected / actual total outputs.\",\n",
" \"600 model API calls in the published run.\",\n",
" \"Group C parse pass rate 1.0; Group C contract pass rate 1.0.\",\n",
" \"Main certificate result: T4_MAIN_CERTIFICATE_PASS.\"\n",
"]\n",
"POLARIS_PUBLISHED_METRICS = {\n",
" \"Cases\": 120,\n",
" \"Expected total outputs\": 720,\n",
" \"Actual total outputs\": 720,\n",
" \"Model API calls\": 600,\n",
" \"Group C parse pass rate\": 1.0,\n",
" \"Group C contract pass rate\": 1.0,\n",
" \"Family red count\": 0,\n",
" \"Compiler verifier red count\": 0,\n",
" \"Model stress warnings\": 0\n",
"}\n",
"POLARIS_CHART_SPECS = [\n",
" {\n",
" \"title\": \"Output coverage\",\n",
" \"ylabel\": \"Outputs\",\n",
" \"labels\": [\n",
" \"Expected total\",\n",
" \"Actual total\",\n",
" \"Model stress\"\n",
" ],\n",
" \"values\": [\n",
" 720,\n",
" 720,\n",
" 600\n",
" ]\n",
" },\n",
" {\n",
" \"title\": \"Compiler lane pass rates\",\n",
" \"ylabel\": \"Rate\",\n",
" \"labels\": [\n",
" \"Parse pass\",\n",
" \"Contract pass\"\n",
" ],\n",
" \"values\": [\n",
" 1.0,\n",
" 1.0\n",
" ]\n",
" },\n",
" {\n",
" \"title\": \"Red / warning counts\",\n",
" \"ylabel\": \"Count\",\n",
" \"labels\": [\n",
" \"Family red\",\n",
" \"Compiler red\",\n",
" \"Stress warnings\"\n",
" ],\n",
" \"values\": [\n",
" 0,\n",
" 0,\n",
" 0\n",
" ]\n",
" }\n",
"]\n",
"\n",
"report_dir = Path(f\"polaris_report_{POLARIS_REPORT_EXPERIMENT_ID}\")\n",
"report_dir.mkdir(exist_ok=True)\n",
"\n",
"# Save metrics table.\n",
"metrics_df = pd.DataFrame([\n",
" {\"metric\": key, \"value\": value}\n",
" for key, value in POLARIS_PUBLISHED_METRICS.items()\n",
"])\n",
"metrics_path = report_dir / \"metrics_summary.csv\"\n",
"metrics_df.to_csv(metrics_path, index=False)\n",
"\n",
"# Generate compact charts.\n",
"chart_paths = []\n",
"def _safe_chart_name(title):\n",
" return re.sub(r\"[^a-z0-9_\\\\-]+\", \"_\", title.lower().replace(\" \", \"_\")).strip(\"_\")\n",
"\n",
"for idx, spec in enumerate(POLARIS_CHART_SPECS, start=1):\n",
" labels = spec[\"labels\"]\n",
" values = spec[\"values\"]\n",
" plt.figure(figsize=(max(6, len(labels) * 1.2), 4))\n",
" bars = plt.bar(labels, values)\n",
" plt.title(spec[\"title\"])\n",
" plt.ylabel(spec.get(\"ylabel\", \"Value\"))\n",
" plt.xticks(rotation=25, ha=\"right\")\n",
"\n",
" numeric_values = [float(v) for v in values if isinstance(v, (int, float))]\n",
" max_value = max(numeric_values) if numeric_values else 0.0\n",
" min_value = min(numeric_values) if numeric_values else 0.0\n",
"\n",
" if max_value == 0 and min_value == 0:\n",
" # Make zero-count success charts visually readable instead of looking blank.\n",
" plt.ylim(0, 1)\n",
" for bar, value in zip(bars, values):\n",
" x = bar.get_x() + bar.get_width() / 2\n",
" plt.text(x, 0.05, str(value), ha=\"center\", va=\"bottom\", fontsize=10)\n",
" plt.text(\n",
" 0.5, 0.88,\n",
" \"All tracked counts are 0\",\n",
" transform=plt.gca().transAxes,\n",
" ha=\"center\",\n",
" va=\"center\",\n",
" fontsize=11,\n",
" )\n",
" else:\n",
" upper = max_value * 1.18 if max_value > 0 else 1\n",
" if min_value >= 0:\n",
" plt.ylim(0, upper)\n",
" for bar, value in zip(bars, values):\n",
" x = bar.get_x() + bar.get_width() / 2\n",
" y = bar.get_height()\n",
" label = f\"{value:.4g}\" if isinstance(value, float) else str(value)\n",
" plt.text(x, y + (upper * 0.02 if max_value > 0 else 0.03), label, ha=\"center\", va=\"bottom\", fontsize=9)\n",
"\n",
" plt.tight_layout()\n",
" chart_path = report_dir / f\"chart_{idx:02d}_{_safe_chart_name(spec['title'])}.png\"\n",
" plt.savefig(chart_path, dpi=180, bbox_inches=\"tight\")\n",
" plt.show()\n",
" chart_paths.append(chart_path)\n",
"\n",
"# Save executive markdown report.\n",
"summary_lines = \"\\n\".join([f\"- {item}\" for item in POLARIS_PUBLISHED_SUMMARY])\n",
"chart_lines = \"\\n\".join([f\"- {p.name}\" for p in chart_paths])\n",
"\n",
"report_md = f\"\"\"# {POLARIS_REPORT_TITLE}\n",
"\n",
"**Repository:** {POLARIS_REPORT_REPO} \n",
"**Experiment ID:** {POLARIS_REPORT_EXPERIMENT_ID}\n",
"\n",
"## Experiment spirit\n",
"\n",
"{POLARIS_REPORT_SPIRIT}\n",
"\n",
"## What this tests\n",
"\n",
"{POLARIS_REPORT_PURPOSE}\n",
"\n",
"## Published result summary\n",
"\n",
"{summary_lines}\n",
"\n",
"## Generated files\n",
"\n",
"- metrics_summary.csv\n",
"{chart_lines}\n",
"\n",
"## Claim boundary\n",
"\n",
"{POLARIS_REPORT_CLAIM_BOUNDARY}\n",
"\n",
"This report is designed for reproduction / inspection. It does not convert scoped evidence into universal proof.\n",
"\"\"\"\n",
"\n",
"report_path = report_dir / \"executive_report.md\"\n",
"report_path.write_text(report_md, encoding=\"utf-8\")\n",
"\n",
"print(\"Polaris Protocol report generated:\")\n",
"print(f\"- {report_path}\")\n",
"print(f\"- {metrics_path}\")\n",
"for path in chart_paths:\n",
" print(f\"- {path}\")\n"
]
}
],
"metadata": {
"polaris_protocol": {
"claim_boundary": "Math stress evidence only. Not universal math proof. Compiler certificate and model stress are separate evidence lanes.",
"experiment_id": "PP02B",
"experiment_title": "PP02B — Math Stress Results Review",
"generated_review_notebook": true,
"mvp_official_colab": true,
"repo": "https://github.com/onestardao/WFGY"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in a new issue View git blame Copy permalink