fix(mcp-gate): close the calibration guard, and correct the overhead claim

Audit of the escalation ladder confirmed the mandatory-inspection
guarantee under a brute-force sweep -- 24,300 constructible configs across
five mandatory operation types, 121,500 checks, zero bypasses -- but found
the calibration guard that is supposed to reject a never-escalate
configuration was defeatable by a single field, and found two published
numbers that did not mean what they appeared to.

`expected_uncertainty` was operator-declared, used only in the
construction-time ceiling, never checked against what the detector
actually reports, and bounded only as finite and positive. Setting it to
100 made the guard accept a ladder that then purchased nothing at the
escalation threshold -- the never-escalate switch the guard exists to
prevent. It now comes from `TinyDetector::expected_uncertainty()`, so
there is no declared figure left to diverge from, and it is additionally
bounded to (0,1] as defence in depth. Bounding alone would have left an
operator free to declare 1.0 against a detector reporting 0.05.

The ceiling also used sigma where it had to use the predictive standard
deviation. What a rung can reveal is sigma/sqrt(1+(tau/sigma)^2), strictly
smaller whenever the rung is noisy -- and the cheapest rung is
deliberately the noisiest -- so the guard was loose by up to 5x and
accepted ladders that purchased nothing at maximum VoI. It now calls
`voi_upper_bound`, the same primitive `rung_is_purchasable` uses;
`INV_SQRT_2PI` became dead code and was removed, which is itself evidence
the guard no longer recomputes by hand.

THE OVERHEAD CLAIM IS CORRECTED, INCLUDING A CORRECTION TO THE FIRST FIX.
Charging each rung its declared latency was expected to hold the budget.
Measured, it does not:

  investigator resolves the ambiguity (0.2)      2.6%   20 rungs
  investigator leaves it ambiguous (0.5), priced  108%   80 rungs
  same, unpriced                                  205%   80 rungs

The limiter is the belief update, not the price. When the first rung
settles the question VoI collapses and the ladder stops, comfortably under
target. When it does not, the belief sits at the threshold, every further
rung still looks worth buying, and the ladder spends `max_rounds` on every
operation. Pricing latency halves overhead by shifting the mix toward
cheaper rungs; it does not reduce the count. `max_rounds` is therefore the
only hard bound on wall-clock cost -- four rounds of a 200ms rung against a
20ms workload exceeds the budget by 40x on its own. That table is now in
the module docs, replacing the claim it disproves.

Also: `Classification` no longer derives `Default`, which had made the
documented unforgeability claim false -- `Classification::default()`
compiled and reported "in no mandatory class". Coverage now has a
denominator (`mandatory_operations_seen`) so it is a measurable ratio
rather than an uncountable assertion. `record_rung` moved ahead of the
observation validity check so a paid investigator call that returns
garbage is still counted. Six plain-ASCII destructive verbs -- unlink,
shred, rmdir, wipefs, mkfs, dd -- were missing from the classifier.
`rung_is_purchasable` no longer skips the validation choke point.

The docs now state plainly that this module has no call site in the gate's
request path and protects no production traffic, and that the recognition
gap lives in the classifier's marker list rather than in the detector, so
replacing the detector does not close it.

All seven negative checks bit before their fix.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_016QSCkKnxDjqU49NVVpWMK5
This commit is contained in:
ruv 2026-08-23 10:30:58 -04:00
parent 10c15bedaa
commit 5d81fb1cd1
7 changed files with 516 additions and 52 deletions

View file

@ -14,19 +14,29 @@
//!
//! - [`MandatoryClass`] is a closed enum. There is no "custom class" arm and
//! no registry, so the set cannot be narrowed by adding configuration.
//! - [`Classification`] has a private field and **no public constructor**.
//! The only way to obtain one is [`classify`], which is `pub(crate)`. A
//! caller cannot hand the ladder a forged empty classification, and the
//! ladder does not accept one as a parameter — it calls [`classify`]
//! - [`Classification`] has a private field, **no public constructor**, and
//! deliberately does **not** derive `Default` — a derived `Default` would
//! hand any external crate a `Classification` reporting "in no mandatory
//! class", which is precisely the forgery this is meant to prevent. The
//! only way to obtain one is [`classify`], which is `pub(crate)`. The
//! ladder also does not accept one as a parameter: it calls [`classify`]
//! itself on the subject it was given.
//! - No type in [`crate::monitor`] carries a per-class enable/disable flag,
//! an allowlist, or a bypass token. There is nothing to set.
//!
//! The remaining honest gap is *recognition*, not *enforcement*: an operation
//! this module fails to recognise as belonging to a class is not inspected as
//! one. That is a detector-quality problem (see [`crate::monitor::detector`]),
//! and it is why the matchers below are deliberately broad — a false positive
//! costs one inspection, a false negative costs the guarantee.
//! one. That gap is **here**, in the marker lists below — not in the tiny
//! detector. The ladder consults [`classify`] directly and it never consults
//! a detector, so replacing [`crate::monitor::detector::KeywordDetector`]
//! with a real classifier does nothing for it. Closing it means improving
//! these markers, or replacing substring matching with something that
//! understands the operation.
//!
//! The matchers are therefore deliberately broad — a false positive costs one
//! inspection, a false negative costs the guarantee. They still only match
//! literal substrings: an operation that spells its verb differently, encodes
//! it, or expresses it purely through an opaque identifier is not recognised.
use super::detector::InspectionSubject;
@ -147,7 +157,7 @@ impl MandatoryClass {
/// Deliberately opaque: the private field and absent public constructor mean
/// this can only be produced by [`classify`], so no caller can assert "this
/// operation is in no mandatory class".
#[derive(Debug, Clone, PartialEq, Eq, Default)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Classification {
matched: Vec<MandatoryClass>,
}

View file

@ -54,41 +54,50 @@ pub struct MonitorConfig {
pub halt_risk: f64,
/// Hard cap on investigator purchases per operation. Bounds the loop
/// independently of the economics.
///
/// With [`VoiConfig::latency_price`] at zero this is the **only** thing
/// bounding an operation's wall-clock monitoring cost, because a rung
/// declaring ten seconds of latency then costs exactly what an instant
/// rung costs. Price latency if the overhead budget is meant to be
/// enforced economically rather than merely capped.
pub max_rounds: usize,
/// Uncertainty the detector typically reports. Used **only** to verify at
/// construction that the cheapest rung is purchasable in principle.
pub expected_uncertainty: f64,
}
impl Default for MonitorConfig {
/// Defaults calibrated against the reference two-rung ladder (a $0.01
/// verifier at σ=0.15 and a $0.50 investigator at σ=0.05) so that both
/// failure modes are avoided.
/// Defaults calibrated against the reference two-rung ladder (a $0.01 /
/// 5 ms verifier at σ=0.15 and a $0.50 / 200 ms investigator at σ=0.05)
/// so that both calibration failure modes are avoided.
///
/// `value_of_success` is bounded on *both* sides, and the window is
/// narrower than it looks. Too low and nothing is ever purchasable
/// ([`MonitorError::Miscalibrated`] catches that at construction). Too
/// high and every operation is worth investigating, including zero-risk
/// ones — which silently blows the overhead budget while looking
/// perfectly configured, and which construction *cannot* catch because
/// such a ladder is well-formed. For this ladder the usable window is
/// roughly `(0.16, 263)`: at 100 a zero-risk operation is not worth
/// investigating (`100 × 3.8e-5 < $0.01`) while an operation sitting at
/// the escalation threshold is (`100 × 0.064 > $0.01`).
/// perfectly configured, and which construction *cannot* catch, because
/// such a ladder is well-formed. At 100 a zero-risk operation is not
/// worth investigating while one sitting at the escalation threshold is.
///
/// Re-derive this when the rungs or the detector's uncertainty change;
/// [`rung_is_purchasable`] answers it directly for a given belief.
/// `latency_price` defaults to `1e-6`, i.e. **one dollar per second** of
/// added latency. A zero default would make the wall-clock cost of a rung
/// economically invisible: a ten-second investigator would price
/// identically to an instant one, and only [`Self::max_rounds`] would
/// bound the damage. Pricing latency is what lets the ladder trade
/// against an overhead budget at all.
///
/// Re-derive these when the rungs or the detector's uncertainty change;
/// [`rung_is_purchasable`](crate::monitor::rung_is_purchasable) answers
/// the question directly for a given belief.
fn default() -> Self {
Self {
voi: VoiConfig {
// Currency value of catching one violation.
value_of_success: 100.0,
latency_price: 0.0,
// $1 per second of added latency.
latency_price: 1e-6,
},
escalation_threshold: 0.5,
halt_risk: 0.95,
max_rounds: 4,
expected_uncertainty: 0.2,
}
}
}

View file

@ -128,6 +128,24 @@ pub trait TinyDetector {
/// Score an operation. Returning `Err` is treated as *fail closed* by the
/// ladder: an unscoreable operation is escalated or halted, never allowed.
fn score(&self, subject: &InspectionSubject) -> Result<RiskSignal, MonitorError>;
/// The uncertainty this detector actually reports from [`Self::score`],
/// in `(0, 1]`.
///
/// The ladder's construction-time calibration check needs to know how
/// much uncertainty there is for an investigator to resolve, because
/// that bounds what any rung can be worth. This is a trait method rather
/// than a configuration field on purpose: an operator-declared figure is
/// free to diverge from what the detector really produces, and a
/// *larger* declared value inflates the ceiling and makes the guard more
/// permissive — so the one field that exists to catch a never-escalate
/// ladder could be used to wave one through. Deriving it from the
/// detector removes the divergence rather than bounding it.
///
/// A detector whose uncertainty varies per input should report the
/// **largest** value it can return, since that is the most permissive
/// ceiling it could honestly justify.
fn expected_uncertainty(&self) -> f64;
}
/// A trivial substring-matching detector.
@ -227,6 +245,12 @@ impl TinyDetector for KeywordDetector {
let risk = (hits as f64 / self.saturation as f64).min(1.0);
RiskSignal::new(risk, self.uncertainty)
}
/// This detector reports one fixed uncertainty from every `score`, so the
/// value it declares here is exactly the value it produces.
fn expected_uncertainty(&self) -> f64 {
self.uncertainty
}
}
#[cfg(test)]

View file

@ -35,6 +35,17 @@ pub enum MonitorError {
/// The offending uncertainty.
uncertainty: f64,
},
/// A detector reported an uncertainty above 1 on a unit-scale risk.
///
/// Refused because it is meaningless there, and because the only thing
/// an inflated uncertainty does is raise the calibration ceiling — the
/// guard that exists to catch a never-escalate ladder would otherwise be
/// the lever for waving one through.
#[error("detector uncertainty must be within (0, 1] on a unit-scale risk, got {uncertainty}")]
UncertaintyOutOfRange {
/// The offending uncertainty.
uncertainty: f64,
},
/// A detector saturation of zero would divide by zero on every score.
#[error("detector saturation must be > 0")]
ZeroSaturation,

View file

@ -35,12 +35,46 @@
//! site, and a monitoring gate that fails open under error is worse than no
//! gate, because it looks like one.
//!
//! # Design target
//! # Design target, and what is actually enforced
//!
//! Under 5% average monitoring overhead with 100% mandatory-class coverage.
//! [`OverheadAccount`] measures the first and counts the second separately,
//! so the claim is checkable rather than asserted, and so the ratio cannot be
//! flattered by narrowing the mandatory set.
//! The target is under 5% average monitoring overhead with 100%
//! mandatory-class coverage. [`OverheadAccount`] measures the first and
//! [`OverheadAccount::mandatory_coverage`] reports the second against a
//! denominator, so both are checkable rather than asserted.
//!
//! Only the coverage half is *enforced*. Overhead is bounded by
//! [`MonitorConfig::max_rounds`] and priced by [`VoiConfig::latency_price`],
//! but nothing refuses a ladder whose rungs are simply slow. Measured on the
//! reference ladder (5 ms / 200 ms rungs, 200 operations, one in ten
//! mandatory, 20 ms attributed per operation):
//!
//! | Case | Overhead | Rungs |
//! |---|---|---|
//! | Investigator resolves the ambiguity (reports 0.2) | **2.6%** | 20 |
//! | Investigator leaves it ambiguous (reports 0.5), latency priced | **108%** | 80 |
//! | Same, latency unpriced | **205%** | 80 |
//!
//! So the target holds only when the first investigator actually settles the
//! question. When it does not, the belief stays near the escalation
//! threshold, every further rung still looks worth buying, and the ladder
//! spends [`MonitorConfig::max_rounds`] rungs on every operation — four
//! rounds of a 200 ms investigator against a 20 ms workload is 40× the
//! budget on its own.
//!
//! Pricing latency halves that (205% → 108%) by shifting the *mix* toward
//! cheaper rungs, but it does not reduce the *count*: at
//! `value_of_success = 100` even a priced 200 ms rung is worth buying at
//! maximum ambiguity. **`max_rounds` is the only hard bound on wall-clock
//! cost.** Treat 5% as a budget to configure toward — chiefly by keeping
//! `max_rounds` small and the top rung fast — not as a property this module
//! guarantees.
//!
//! # Not wired into the request path
//!
//! This is a library module with **no call site** in the gate's request
//! handling: `server.rs`, `tools.rs`, `types.rs`, and `main.rs` do not
//! reference it. Nothing here is protecting production traffic yet. Wiring
//! it into [`crate::tools::McpGateTools`] is separate work.
pub mod classes;
pub mod config;
@ -60,10 +94,6 @@ pub use detector::{InspectionSubject, KeywordDetector, RiskSignal, TinyDetector}
pub use error::{HaltReason, MonitorError};
pub use overhead::OverheadAccount;
/// 1/√(2π): the value-of-information ceiling is `s·φ(0) = s/√(2π)` and
/// `s ≤ σ`, so `σ/√(2π)` bounds what any rung can ever be worth.
const INV_SQRT_2PI: f64 = 0.398_942_280_401_432_7;
/// Result of monitoring one operation.
///
/// Two variants only, and no `Default`: there is no neutral outcome to fall
@ -162,9 +192,17 @@ impl<D: TinyDetector, I: Investigator> EscalationLadder<D, I> {
}
}
config.voi.validate().map_err(voi_err)?;
// Round-trips the uncertainty through the same validation the
// detector's signals face.
RiskSignal::new(0.0, config.expected_uncertainty)?;
// The uncertainty comes from the detector rather than configuration,
// so it cannot diverge from what `score` actually reports. Bounded to
// (0, 1] like risk itself: an uncertainty above 1 is meaningless on a
// unit-scale risk, and would only serve to inflate the ceiling below.
let detector_uncertainty = detector.expected_uncertainty();
RiskSignal::new(0.0, detector_uncertainty)?;
if detector_uncertainty > 1.0 {
return Err(MonitorError::UncertaintyOutOfRange {
uncertainty: detector_uncertainty,
});
}
let mut previous = f64::NEG_INFINITY;
for (index, rung) in rungs.iter().enumerate() {
@ -187,17 +225,30 @@ impl<D: TinyDetector, I: Investigator> EscalationLadder<D, I> {
previous = cost;
}
// Calibration: no purchase can be worth more than
// value_of_success × σ/√(2π). If the cheapest rung exceeds that, the
// ladder can never escalate on economics alone.
// Calibration: if the cheapest rung costs more than the most any
// purchase could ever be worth, the ladder can never escalate on
// economics alone.
//
// The ceiling is computed from the *predictive* standard deviation
// `s = σ/√(1+(τ/σ)²)`, not from σ itself. A noisy rung reveals
// strictly less than the full prior uncertainty, and this design
// deliberately makes the cheapest rung the noisiest — so using σ
// would overstate the ceiling by up to ~5× at realistic noise levels
// and wave through ladders that then purchase nothing. `voi_upper_bound`
// is exactly this quantity, and is the same call `rung_is_purchasable`
// makes.
let cheapest_cost = monetized_cost(&rungs[0].spec, &config.voi);
let ceiling = config.voi.value_of_success * config.expected_uncertainty * INV_SQRT_2PI;
let calibration_belief =
Belief::new(config.escalation_threshold, detector_uncertainty).map_err(voi_err)?;
let bound =
voi_upper_bound(calibration_belief, rungs[0].spec.noise_std).map_err(voi_err)?;
let ceiling = config.voi.value_of_success * bound;
if cheapest_cost > ceiling {
return Err(MonitorError::Miscalibrated {
cheapest_cost,
ceiling,
value_of_success: config.voi.value_of_success,
uncertainty: config.expected_uncertainty,
uncertainty: detector_uncertainty,
});
}
@ -256,6 +307,11 @@ impl<D: TinyDetector, I: Investigator> EscalationLadder<D, I> {
let classification = classes::classify(subject);
let mandatory: Vec<MandatoryClass> = classification.matched().to_vec();
let is_mandatory = classification.is_mandatory();
if is_mandatory {
// Recorded here, before any decision can divert the operation, so
// `mandatory_coverage` has an honest denominator.
self.account.record_mandatory_seen();
}
let signal = match self.detector.score(subject) {
Ok(s) => s,
@ -307,6 +363,21 @@ impl<D: TinyDetector, I: Investigator> EscalationLadder<D, I> {
)
}
};
// The rung ran and was paid for, whatever it returned. Count it
// before validating the observation, or an investigator that
// fails validation is billed but never recorded.
rungs_purchased += 1;
inspected = true;
self.account.record_rung();
// Charge the rung's *declared* latency, not the wall-clock of
// whatever ran. A fixture investigator returns instantly, so
// wall-clock alone would report an overhead figure that says more
// about the test double than about the ladder as configured.
// Declared latency is also what the economics priced, so the
// accounting and the purchase decision agree on what a rung costs.
self.account
.record_monitoring(rung.spec.latency_us.max(0.0) as u128);
// Validate before any comparison: a non-finite observation would
// otherwise pass the halt check silently.
if !observation.is_finite() || !(0.0..=1.0).contains(&observation) {
@ -317,10 +388,6 @@ impl<D: TinyDetector, I: Investigator> EscalationLadder<D, I> {
);
}
rungs_purchased += 1;
inspected = true;
self.account.record_rung();
if rung.is_oracle() {
// Ground truth: take it and exit without calling `observe`,
// which refuses noiseless specs.
@ -384,11 +451,18 @@ fn voi_err(e: impl std::fmt::Display) -> MonitorError {
///
/// Exposed so an operator can answer "is this ladder actually able to
/// escalate?" without running traffic through it.
///
/// Validates its inputs through the same choke point the ladder uses. Without
/// that, a `value_of_success` of infinity reports `true` — a diagnostic that
/// answers "yes, escalation is possible" for a configuration the ladder
/// itself would refuse is worse than no diagnostic.
pub fn rung_is_purchasable(
belief: Belief,
rung: &LadderRung,
config: &MonitorConfig,
) -> Result<bool, MonitorError> {
config.voi.validate().map_err(voi_err)?;
rung.spec.validate().map_err(voi_err)?;
let bound = voi_upper_bound(belief, rung.spec.noise_std).map_err(voi_err)?;
Ok(config.voi.value_of_success * bound > monetized_cost(&rung.spec, &config.voi))
}

View file

@ -21,6 +21,21 @@
//! overhead figure cannot be improved by quietly narrowing the mandatory set:
//! declassifying a class shows up immediately as `mandatory_inspections`
//! falling while throughput is unchanged. The ratio alone would hide that.
//!
//! A count with no denominator cannot support a coverage claim, though.
//! `mandatory_inspections = 0` is what a ladder reports when it saw no
//! mandatory operations *and* what it reports when it stopped classifying
//! them — indistinguishable without knowing how many it saw. So
//! `mandatory_operations_seen` is recorded at classification time, before any
//! decision can divert the operation, and
//! [`OverheadAccount::mandatory_coverage`] is the ratio that actually means
//! "100% of mandatory operations were inspected".
//!
//! Note the two are legitimately unequal when an operation halts *before*
//! reaching an investigator — a critical detector score, or a failure on any
//! fail-closed path. Those operations were stopped, not skipped, which is the
//! stronger outcome; coverage below 1.0 therefore wants reading alongside
//! [`OverheadAccount::halts`] rather than alone.
/// Running totals for one ladder's monitoring activity.
///
@ -33,6 +48,7 @@ pub struct OverheadAccount {
monitoring_us: u128,
operations: u64,
inspections: u64,
mandatory_operations_seen: u64,
mandatory_inspections: u64,
rungs_purchased: u64,
halts: u64,
@ -55,6 +71,14 @@ impl OverheadAccount {
self.monitoring_us = self.monitoring_us.saturating_add(micros);
}
/// Record that an operation was classified into at least one mandatory
/// class. Called immediately after classification, before any decision
/// can divert the operation, so it is the honest denominator for
/// [`Self::mandatory_coverage`].
pub(crate) fn record_mandatory_seen(&mut self) {
self.mandatory_operations_seen = self.mandatory_operations_seen.saturating_add(1);
}
/// Record that an operation was inspected, and whether it was inspected
/// because a mandatory class required it.
pub(crate) fn record_inspection(&mut self, mandatory: bool) {
@ -109,12 +133,34 @@ impl OverheadAccount {
/// Inspections that a mandatory class compelled.
///
/// Read alongside [`Self::inspections`]: a drop here without a
/// corresponding drop in workload means the mandatory set was narrowed.
/// Read alongside [`Self::mandatory_operations_seen`]: this count alone
/// cannot distinguish "no mandatory operations arrived" from "mandatory
/// operations stopped being recognised".
pub fn mandatory_inspections(&self) -> u64 {
self.mandatory_inspections
}
/// Operations that classification placed in at least one mandatory class.
pub fn mandatory_operations_seen(&self) -> u64 {
self.mandatory_operations_seen
}
/// Fraction of mandatory operations that reached an investigator.
///
/// `None` when none were seen — not `1.0`, which would report perfect
/// coverage for a ladder that has never classified anything.
///
/// Below 1.0 does not necessarily mean an operation slipped through: an
/// operation halted before reaching a rung (critical detector score, or
/// any fail-closed path) counts in the denominator but not the numerator.
/// Read with [`Self::halts`].
pub fn mandatory_coverage(&self) -> Option<f64> {
if self.mandatory_operations_seen == 0 {
return None;
}
Some(self.mandatory_inspections as f64 / self.mandatory_operations_seen as f64)
}
/// Investigator rungs purchased across all operations.
pub fn rungs_purchased(&self) -> u64 {
self.rungs_purchased
@ -164,4 +210,31 @@ mod tests {
assert_eq!(a.inspections(), 3);
assert_eq!(a.mandatory_inspections(), 1);
}
#[test]
fn coverage_needs_a_denominator_and_reports_none_without_one() {
let mut a = OverheadAccount::new();
// Zero inspections and zero seen is not 100% coverage.
assert_eq!(a.mandatory_coverage(), None);
a.record_mandatory_seen();
a.record_mandatory_seen();
a.record_inspection(true);
assert_eq!(a.mandatory_coverage(), Some(0.5));
a.record_inspection(true);
assert_eq!(a.mandatory_coverage(), Some(1.0));
}
#[test]
fn a_ladder_that_stopped_classifying_is_distinguishable_from_a_quiet_one() {
// The failure MEDIUM-5 identified: both of these report
// mandatory_inspections == 0, and only the denominator tells them
// apart.
let quiet = OverheadAccount::new();
let mut stopped_classifying = OverheadAccount::new();
stopped_classifying.record_mandatory_seen();
assert_eq!(quiet.mandatory_inspections(), 0);
assert_eq!(stopped_classifying.mandatory_inspections(), 0);
assert_eq!(quiet.mandatory_coverage(), None);
assert_eq!(stopped_classifying.mandatory_coverage(), Some(0.0));
}
}

View file

@ -73,6 +73,9 @@ impl TinyDetector for FailingDetector {
fn score(&self, _s: &InspectionSubject) -> Result<RiskSignal, MonitorError> {
Err(MonitorError::ZeroSaturation)
}
fn expected_uncertainty(&self) -> f64 {
0.2
}
}
/// Detector returning a fixed signal, so tests control the risk exactly.
@ -81,12 +84,33 @@ impl TinyDetector for FixedDetector {
fn score(&self, _s: &InspectionSubject) -> Result<RiskSignal, MonitorError> {
RiskSignal::new(self.0, self.1)
}
fn expected_uncertainty(&self) -> f64 {
self.1
}
}
/// Declares an uncertainty that differs from what it scores — the divergence
/// the calibration guard used to be blind to when the figure was configured
/// rather than derived.
struct LyingDetector {
scored: f64,
declared: f64,
}
impl TinyDetector for LyingDetector {
fn score(&self, _s: &InspectionSubject) -> Result<RiskSignal, MonitorError> {
RiskSignal::new(0.5, self.scored)
}
fn expected_uncertainty(&self) -> f64 {
self.declared
}
}
/// Reference ladder. Latencies are declared honestly (5 ms, 200 ms) — modest
/// for real verifiers — so that pricing them has something to price.
fn rungs() -> Vec<LadderRung> {
vec![
LadderRung::new("cheap-verifier", 0.01, 0.0, 0.15),
LadderRung::new("strong-investigator", 0.50, 0.0, 0.05),
LadderRung::new("cheap-verifier", 0.01, 5_000.0, 0.15),
LadderRung::new("strong-investigator", 0.50, 200_000.0, 0.05),
]
}
@ -160,6 +184,64 @@ fn a_never_escalating_value_of_success_is_refused_loudly() {
assert!(matches!(err, MonitorError::Miscalibrated { .. }), "{err}");
}
#[test]
fn a_detector_cannot_inflate_the_ceiling_to_wave_through_a_dead_ladder() {
// MEDIUM-1. When the uncertainty was an operator-declared config field it
// was used ONLY in the construction ceiling and never cross-checked
// against what the detector reported, so declaring a large value bought
// past the very guard that exists to catch a never-escalate ladder.
// It is now taken from the detector and bounded to (0, 1].
for absurd in [100.0_f64, 1.5, 1e12] {
let err = EscalationLadder::new(
vec![LadderRung::new("verifier", 0.50, 0.0, 0.1)],
MonitorConfig {
voi: VoiConfig {
value_of_success: 1.0,
latency_price: 0.0,
},
..MonitorConfig::default()
},
LyingDetector {
scored: 0.2,
declared: absurd,
},
RecordingInvestigator::new(0.1),
)
.expect_err("an uncertainty above 1 must be refused");
assert!(
matches!(err, MonitorError::UncertaintyOutOfRange { .. }),
"declared {absurd} gave {err}"
);
}
}
#[test]
fn the_calibration_ceiling_uses_the_predictive_std_not_sigma() {
// MEDIUM-2. What a rung can reveal is s = σ/√(1+(τ/σ)²), strictly less
// than σ whenever the rung is noisy — and the cheapest rung is
// deliberately the noisiest. A σ-based ceiling therefore overstates what
// is purchasable and admits ladders that then buy nothing.
//
// For the reference ladder (σ=0.2, τ=0.15, cheapest monetized cost
// $0.015) the σ-based ceiling accepts value_of_success ≥ 0.188 while
// real escalation needs ≥ 0.235. This value sits in that gap.
let in_the_gap = MonitorConfig {
voi: VoiConfig {
value_of_success: 0.20,
latency_price: 1e-6,
},
..MonitorConfig::default()
};
let err = EscalationLadder::new(
rungs(),
in_the_gap,
KeywordDetector::new(),
RecordingInvestigator::new(0.5),
)
.expect_err("a ladder inside the σ/s gap must be refused");
assert!(matches!(err, MonitorError::Miscalibrated { .. }), "{err}");
}
#[test]
fn zero_round_cap_is_refused() {
let config = MonitorConfig {
@ -218,7 +300,6 @@ fn no_valid_config_can_skip_a_mandatory_class() {
escalation_threshold: 0.99,
halt_risk: 1.0,
max_rounds: 4,
expected_uncertainty: 0.2,
};
let cases = [
@ -550,6 +631,184 @@ fn measured_overhead_on_a_synthetic_workload_is_under_five_percent() {
);
}
#[test]
fn overhead_charges_declared_rung_latency_not_fixture_speed() {
// MEDIUM-4. With `latency_price` at zero a ten-second rung priced
// identically to an instant one, and the measured figure reflected only
// how fast the test double returned. The account now charges what the
// rung declares, so the number means something about the configuration.
let mut ladder = EscalationLadder::new(
rungs(),
MonitorConfig::default(),
FixedDetector(0.0, 0.2),
RecordingInvestigator::new(0.1),
)
.expect("ladder builds");
let outcome = ladder.inspect(&InspectionSubject::new(
"op",
"delete the bucket",
json!({}),
));
assert!(outcome.permits_execution(), "benign verdict should allow");
let purchased = ladder.account().rungs_purchased();
assert!(purchased >= 1);
// The cheap rung alone declares 5 ms; a fixture returning instantly
// cannot account for less than that.
assert!(
ladder.account().monitoring_us() >= 5_000,
"declared latency was not charged: {} µs for {} rungs",
ladder.account().monitoring_us(),
purchased
);
}
#[test]
fn measured_overhead_with_honest_latency_pricing() {
// The number that actually bears on the 5% design target: rungs declare
// realistic latency (5 ms / 200 ms) and the account charges it. Reported
// rather than asserted against 5%, because whether the target is met is a
// property of the configuration, not of this module.
const OPS: usize = 200;
const WORKLOAD_US_PER_OP: u128 = 20_000;
let mut ladder = EscalationLadder::new(
rungs(),
MonitorConfig::default(),
KeywordDetector::new(),
RecordingInvestigator::new(0.2),
)
.expect("ladder builds");
for i in 0..OPS {
let subject = if i % 10 == 0 {
InspectionSubject::new("deploy", "delete the old bucket", json!({"i": i}))
} else {
InspectionSubject::new("sum", "add two integers", json!({"i": i}))
};
let outcome = ladder.inspect(&subject);
let _ = outcome.permits_execution();
ladder.record_workload_us(WORKLOAD_US_PER_OP);
}
let a = ladder.account();
let fraction = a.fraction().expect("workload recorded");
// Coverage is the enforced half and IS asserted.
assert_eq!(
a.mandatory_coverage(),
Some(1.0),
"mandatory coverage regressed"
);
eprintln!(
"HONEST OVERHEAD: {:.4} ({} monitoring µs over {} workload µs) | mandatory {}/{} = {:?} | {} rungs purchased | {} halts",
fraction,
a.monitoring_us(),
a.workload_us(),
a.mandatory_inspections(),
a.mandatory_operations_seen(),
a.mandatory_coverage(),
a.rungs_purchased(),
a.halts()
);
}
#[test]
fn latency_pricing_binds_only_when_the_belief_stays_ambiguous() {
// What actually limits purchases at the reference configuration is the
// belief update, not the price of latency: an investigator reporting a
// clearly-benign 0.2 moves the belief away from the 0.5 threshold, value
// of information collapses, and the ladder declines the second rung
// whether or not latency costs anything.
//
// Latency pricing binds in the case that update does NOT resolve — an
// investigator that keeps reporting exactly the threshold, so the
// operation stays maximally ambiguous and every further rung still looks
// worth buying. This measures both, so the difference is evidence rather
// than assertion.
fn run(latency_price: f64, verdict: f64) -> (f64, u64) {
let config = MonitorConfig {
voi: VoiConfig {
value_of_success: 100.0,
latency_price,
},
..MonitorConfig::default()
};
let mut ladder = EscalationLadder::new(
rungs(),
config,
KeywordDetector::new(),
RecordingInvestigator::new(verdict),
)
.expect("ladder builds");
for i in 0..200 {
let subject = if i % 10 == 0 {
InspectionSubject::new("deploy", "delete the old bucket", json!({"i": i}))
} else {
InspectionSubject::new("sum", "add two integers", json!({"i": i}))
};
let outcome = ladder.inspect(&subject);
let _ = outcome.permits_execution();
ladder.record_workload_us(20_000);
}
let a = ladder.account();
(
a.fraction().expect("workload recorded"),
a.rungs_purchased(),
)
}
let (resolved_priced, resolved_rungs) = run(1e-6, 0.2);
let (ambiguous_priced, ambiguous_priced_rungs) = run(1e-6, 0.5);
let (ambiguous_free, ambiguous_free_rungs) = run(0.0, 0.5);
eprintln!(
"OVERHEAD resolved/priced {:.4} ({} rungs)\n\
OVERHEAD ambiguous/priced {:.4} ({} rungs)\n\
OVERHEAD ambiguous/unpriced {:.4} ({} rungs)",
resolved_priced,
resolved_rungs,
ambiguous_priced,
ambiguous_priced_rungs,
ambiguous_free,
ambiguous_free_rungs
);
// Pricing latency can only reduce or leave unchanged what is purchased.
assert!(
ambiguous_priced_rungs <= ambiguous_free_rungs,
"pricing latency increased purchases: {ambiguous_priced_rungs} vs {ambiguous_free_rungs}"
);
}
#[test]
fn the_purchasability_diagnostic_validates_its_own_inputs() {
// LOW-8. This is the one public fn that answers "can this ladder ever
// escalate?", so it must not answer "yes" for a configuration the ladder
// itself would refuse. Without validation, an infinite value_of_success
// returned Ok(true).
use mcp_gate::monitor::rung_is_purchasable;
use ruvector_tiny_dancer_core::voi::Belief;
let belief = Belief::new(0.5, 0.2).unwrap();
let rung = LadderRung::new("verifier", 0.01, 0.0, 0.15);
for bad in [f64::INFINITY, f64::NAN, 0.0, -1.0] {
let config = MonitorConfig {
voi: VoiConfig {
value_of_success: bad,
latency_price: 0.0,
},
..MonitorConfig::default()
};
assert!(
rung_is_purchasable(belief, &rung, &config).is_err(),
"value_of_success {bad} produced an answer instead of an error"
);
}
// A well-formed configuration still answers.
assert!(rung_is_purchasable(belief, &rung, &MonitorConfig::default()).is_ok());
}
#[test]
fn an_empty_account_reports_no_overhead_figure() {
let ladder = EscalationLadder::new(
@ -573,17 +832,21 @@ fn the_investigator_actually_saw_the_mandatory_operations() {
investigator,
)
.unwrap();
// `MonitorOutcome` is `#[must_use]`: an unexamined outcome permits an
// unmonitored operation, so even a test must consume it deliberately.
let outcome = ladder.inspect(&InspectionSubject::new(
"op",
"delete the bucket",
json!({}),
));
assert!(outcome.permits_execution() || !outcome.permits_execution());
assert!(
outcome
.mandatory()
.contains(&MandatoryClass::DestructiveOperation),
"operation was not classified as destructive"
);
// Reach back through the ladder to confirm a rung genuinely ran, rather
// than inferring it from counters alone.
assert!(ladder.account().rungs_purchased() >= 1);
assert_eq!(ladder.account().mandatory_coverage(), Some(1.0));
}
#[test]