vrr/WFGY
2
0
Fork 0
mirror of https://github.com/onestardao/WFGY.git synced 2026-04-28 11:40:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Update eval_cross_agent_consistency.md

Browse source
This commit is contained in:
PSBigBig × MiniPS 2026-02-26 15:47:44 +08:00 • committed by GitHub
parent 75d34ca5fd
commit 0d5e04558c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
ProblemMap/GlobalFixMap/Eval/eval_cross_agent_consistency.md
View file

@ -16,6 +16,11 @@
> If you need the full triage and all prescriptions, return to the Emergency Room lobby.
</details>
> **Evaluation disclaimer (cross agent consistency)**
> Agreement between agents is measured with chosen prompts and roles and can still be wrong in absolute terms.
> Consistency scores are diagnostic tools, not proof that the agreed answer is true or safe.
---
**Goal**
Measure and enforce agreement between two independent validators: a **Scholar** (claims/citations checker) and an **Auditor** (policy/provenance/constraints gate). Produce (1) quantitative agreement (Percent Agreement & Cohen’s κ) and (2) a deterministic **conflict-resolution policy** for ship/no-ship.