mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-07-09 17:28:42 +00:00
Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547)
* Add ruvector-graph-condense: structure-preserving graph condensation
New crate implementing training-free, structure-preserving graph
condensation built on the dynamic min-cut engine (ruvector-mincut).
Collapses a feature graph into a small synthetic graph of super-nodes
(regions) while preserving cut structure and node provenance.
Positioning vs. SOTA (GCond/SFGC/GEOM/SGDD): those synthesise a fake
graph via bi-level gradient/distribution/trajectory matching and discard
the node->original mapping. This is the complementary, training-free
route the 2024-2026 surveys flag as under-explored: min-cut community
structure as the condensation prior, cuts preserved by construction
(boundary edges become weighted super-edges), and members retained per
super-node for audit/explainability. Closest published analogs are CGC
(clustering, 2025) and GCTD (tensor decomposition, 2025).
Components:
- NodeFeatures: validated per-vertex embeddings + optional labels
- CondensedNode/Edge/Graph: centroid, weight, class histogram, coherence,
medoid representative, member provenance; round-trips to DynamicGraph
- GraphCondenser with 4 region methods:
- WeakBoundary (default): single-pass union-find over weak-edge removal,
linear-time, recovers planted structure
- MinCutCommunity / Partition: delegate to the min-cut engine
(CommunityDetector / GraphPartitioner); best-effort, documented as
super-linear and prone to singleton-peeling on graphs without
sharp bottlenecks
- ConnectedComponents baseline
- metrics: retrain-free proxies (reduction ratios, intra-weight ratio,
coherence, label purity) + opt-in cut_inflation via exact MinCutBuilder
- StreamingCondenser: lazy re-condensation for growing graphs
- PlantedPartition synthetic generator; criterion benchmarks
Benchmarks (this machine): WeakBoundary scales linearly (~4ms @ 2048
nodes); the recursive min-cut engine methods are super-linear (~24s @ 96
nodes), which is why WeakBoundary is the default.
33 unit tests + 1 doctest pass; clippy clean.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add differentiable min-cut loss (diffcut) to graph condensation
Implements the open research gap flagged by the SOTA review: a
differentiable min-cut / normalized-cut objective used as the
condensation mechanism. The 2024-2026 surveys note that only spectral
terms (SGDD's Laplacian Energy Distribution, GDEM's eigenbasis) exist;
an explicit relaxed-min-cut loss in the condensation objective does not.
New `diffcut` module (after Bianchi et al., MinCutPool 2020):
- Relaxed normalized-cut loss L_cut = -Tr(SᵀAS)/Tr(SᵀDS) plus an
orthogonality/anti-collapse term L_ortho, over a row-softmax soft
assignment S (N×K) of learned logits.
- Analytic gradients (cut, ortho, and softmax backprop), all maths in
f64, no autodiff dependency. Verified against central finite
differences (gradient_matches_finite_differences passes to 1e-5).
- DiffCutCondenser: gradient-descent training -> DiffCutResult with
soft_assignment() and hard_regions() (argmax grouping).
- Public min_cut_loss() for evaluating any soft assignment.
Wired in as CondenseMethod::DiffMinCut(DiffCutConfig): trains the soft
assignment, hardens to regions, then flows through the existing
provenance-preserving super-node/super-edge construction. The only
region method whose structure is *trained* to preserve the cut.
Tests: 36 unit (incl. gradient check + uniform-assignment behaviour) +
6 integration (recovery, determinism, errors) + doctest. clippy clean;
all source files <500 lines. Benchmarks add a diffcut training group.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* docs(adr): ADR-196 + ADR-197 for graph condensation
ADR-196: Structure-preserving graph condensation (ruvector-graph-condense)
— context (SOTA gap + RuView/WorldGraph substrate), decision (training-
free coarsening-condensation with min-cut prior, provenance retained),
the CondenseMethod taxonomy with honest tradeoffs (WeakBoundary default;
engine methods peel + are super-linear), metrics, streaming, alternatives.
ADR-197: Differentiable min-cut condensation loss (diffcut) — the relaxed
normalized-cut + orthogonality objective (MinCutPool-style), analytic
gradients verified by finite differences, DiffCutCondenser + DiffMinCut
integration, and the novelty framing (differentiable min-cut term in the
condensation loss is unpublished as of 2026).
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add WorldGraph example + momentum optimizer; harden diffcut for K>2
- examples/worldgraph.rs: RuView WorldGraph -> condense -> OccWorld demo.
WeakBoundary condenses 600 observations into 12 event summaries (50x,
100% activity purity, cut preserved 1.000); a smaller dense scene shows
the trained DiffMinCut recovering ~86% activity purity.
- diffcut: add heavy-ball `momentum` to DiffCutConfig (default 0.0, all
existing behaviour/tests/benchmarks unchanged) and unit-scale logit init
for stronger symmetry-breaking at K>2.
- Extend the gradient check to K = 2, 3, 4 (proves the K-general gradient
formulas; max abs error < 1e-5).
- Honest finding documented in ADR-197: DiffMinCut (MinCutPool-style) is
K-sensitive — reliable at small/moderate K, underperforms WeakBoundary at
large K, reinforcing WeakBoundary as the default (ADR-196).
- Workspace manifest validated (member resolves; crate is additive so it
cannot break other crates).
43 tests pass (36 unit + 6 integration + 1 doctest); clippy clean; all
source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Optimize trained min-cut for large K: Adam + warm-start + restarts
Plain/momentum GD from random init stalled the differentiable min-cut at
large K (12-event WorldGraph: ~30% purity, ~24s @ 96 nodes). Rebuilt the
optimizer so the trained method is viable at scale:
- Split loss math into cutloss.rs (CompactGraph + softmax + cut/ortho +
analytic gradients, gradient-checked K=2,3,4); diffcut.rs now owns the
optimizer/orchestration. Both files <500 lines.
- Optimizer enum: Adam (default; adaptive moments) and Sgd { momentum }.
- InitStrategy enum: WarmStart (default) seeds logits from the WeakBoundary
structural prior and refines (coreset/K-Center idea), or Random.
- restarts: keep the lowest-loss run. Deterministic region ordering in
warm-start so same seed => identical result.
Result on the 12-event WorldGraph example: DiffMinCut now reaches 100%
activity purity, cut preserved (inflation 1.000) — matching WeakBoundary —
in milliseconds (bench condense_diffcut: ~0.96ms @64, ~6.4ms @192 nodes;
was ~24s @96 under plain GD).
New tests: warm_start_recovers_many_clusters (K=8, purity>0.85),
warm_start_beats_random_at_large_k, warm_start_seeds_a_good_partition,
adam_refines_to_low_cut. Config call sites use ..Default::default().
ADR-197 updated. 47 tests pass (38 unit + 8 integration + 1 doctest);
clippy clean.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* diffcut scale levers: early-stop, Rayon parallelism, edge-minibatching
Three further optimizations for large/million-node graphs (off by default):
- Early-stopping (tolerance, default 1e-6): warm-start lands near the
optimum, so stop when the loss plateaus. iterations_run() reports actual.
- Parallelism (parallel, Rayon): CSR row-parallel A·S plus parallel O(N·K²)
SᵀS + ortho-gradient loops. Deterministic / bit-identical to sequential
(same chunked partial-sum ordering), proven by a test.
- Edge-minibatching (minibatch_edges): stochastic gradient from a sampled
edge subset, O(batch·K)/step; final loss still full-batch exact.
Refactor: cutloss.rs gains CSR adjacency + as_matrix (parallel) +
as_matrix_minibatch + a chunked gram(); loss_and_grad split so the optimizer
supplies A·S. New tests: parallel_matches_sequential_exactly,
minibatch_recovers_structure, early_stopping_cuts_iterations. New bench group
condense_diffcut_levers (1024 nodes, 4 cores: seq ~95ms, parallel ~83ms,
minibatch ~77ms). ADR-197 updated.
50 tests pass (38 unit + 11 integration + 1 doctest); clippy clean; all
source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add GNN accuracy-retention harness (closes the no-accuracy-validation gap)
Implements the graph-condensation field's core success metric: train a GNN
on the condensed graph, test on the ORIGINAL graph's held-out nodes, report
accuracy(condensed)/accuracy(full).
- gnn_eval.rs: self-contained, dependency-free 2-layer GCN (symmetric-
normalised CSR propagation, ReLU, softmax-CE, Adam, analytic backprop).
Gradient-checked against finite differences (<1e-6) and verified to learn a
separable task.
- examples/accuracy_eval.rs + tests/accuracy.rs: the full protocol on a
controlled synthetic node-classification task (planted communities as
classes, noisy features so the graph carries real signal).
Measured: baseline (full-graph GNN) 100%. On an UNWEIGHTED graph (the SOTA
benchmark setting), DiffMinCut condensing 360 nodes -> 18 super-nodes (20x)
yields **100% retention** (GNN trained on 18 nodes matches the full-graph GNN
on held-out test nodes).
Also fixes a real failure the harness surfaced: on uniform-weight graphs
WeakBoundary collapses to one component; DiffMinCut's warm-start inherited
that collapse. Warm-start now falls back to random init when the structural
prior finds <2 regions, letting the min-cut objective do the partitioning
(retention 14.9% -> 66% at K=classes, 100% at K=3*classes).
Honest scope: controlled synthetic data, not Cora/Citeseer; WeakBoundary
still needs weight contrast (documented). 53 tests pass; clippy clean.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add WASM bindings + gate Rayon behind a feature for wasm builds
- crates/ruvector-graph-condense-wasm: wasm-bindgen bindings exposing
condense_weak / condense_diffmincut / version to JS. Graphs in as flat
typed arrays, CondensedGraph out as JSON. Builds for
wasm32-unknown-unknown (667 KB release, pre wasm-opt), so the condenser
(including the trained DiffMinCut) runs in the browser / on the edge —
the deployable-artifact goal from the original brief.
- ruvector-graph-condense: Rayon is now an optional `parallel` feature
(default on for native, off for wasm — no threads on
wasm32-unknown-unknown). cutloss.rs cfg-gates every Rayon path with a
sequential fallback; no-default-features builds clean.
- getrandom `js` backend is wasm-target-gated so native feature
unification is unaffected; ruvector-mincut built with its `wasm` feature.
- ADR-196 updated with the WASM deployment + accuracy-validation notes.
53 tests pass; clippy clean (both crates); native + wasm32 both build.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add ruvector-perception: the layer under classification (delta->proof->action)
Beyond-SOTA wedge: instead of a better CSI classifier, build the substrate
underneath one. Pipeline: delta -> boundary -> coherence -> proof -> action.
Emits a structured DeltaWitness, not a class label, and requires evidence
(not confidence) before exercising bounded authority.
- modality.rs: physically-typed modalities (RF/vibration/acoustic/thermal/
chemical/optical) with latency/decay/spoof-resistance — typed graph edges.
- state.rs: rolling per-(zone,modality) baselines + learned responsiveness.
- coherence.rs: zones as a coherence graph; dynamic min-cut isolates the moved
boundary (reuses ruvector-mincut). Coherence = separation cleanliness.
- witness.rs: ProofGate (Ignore/Observe/Alert/Mutate) + SHA-256 evidence
chain. Contradicted evidence is capped at Observe (no escalation on
confidence alone). Contradiction = a modality that usually reacts here but
stayed silent, weighted by spoof-resistance.
- engine.rs: orchestrates delta -> boundary -> contradiction -> novelty
(nearest-prior) -> proof gate -> chained witness.
- absence.rs: missing expected continuation (bed_exit->bathroom->return) as a
structural safety signal, not a threshold.
Flagship test reproduces the brief exactly: an inert object move yields
changed_boundary=table_left_zone, supporting={rf,vibration,acoustic},
contradicting={thermal}, novelty=high, action=observe. ADR-198 documents the
architecture and honest scope (mechanism on synthetic deltas, not validated on
real CSI).
11 tests pass; clippy clean; all files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Perception: 5 beyond-classification capability modules (swarm-built)
Built via a 5-agent parallel swarm, then integrated and validated. Each
emits structure, not a class label:
- captcha: Physical CAPTCHA — learned per-stimulus multi-modal challenge-
response profiles; verifies a fresh response (delay/magnitude tolerance,
spoof-resistance weighted) -> RealityProof. Detects replay/spoof.
- predict: Boundary-first world model — forecasts where coherence breaks next
(instability = coherence*(1+contradiction), level + least-squares trend).
- identity: Resonant identity / continuity — per-object EWMA signature, cosine
drift detection ("is this still the same physical thing?").
- hypothesis: Multi-modal disagreement engine — contradictions produce ranked
hypotheses (RealEvent/SensorDrift/SensorRelocation/AdversarialReplay/
EnvironmentalArtifact), not forced agreement.
- topology: Self-healing sensor topology — EWMA agreement graph; roles
Critical/Redundant/Noisy/Normal. Critical = articulation point (removal
fragments the graph) — replaced the agent's unreliable min-cut-partition
rule with robust articulation detection so triangle/star outliers keep their
real roles.
lib.rs re-exports all five. ADR-198 updated. 42 tests pass (38 unit + 2
integration + 2 doctest); clippy clean; all source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Perception: complete the substrate — custody, swarm, reality-graph, node
Final beyond-classification pieces (custody + swarm built by a 2-agent swarm;
reality + node integration built directly):
- custody: tamper-evident, replayable chain-of-custody ledger over witness
evidence hashes (chain-linkage verification; honest scope: link integrity,
not raw-signal re-hash).
- swarm: facility/swarm-scale fragility — coupling graph + global min-cut
answers "where is the system closest to breaking?". Bottlenecks derived from
the weakest link (edge weights), since the engine's min-cut value is reliable
but its partition is not (same quirk handled in topology).
- reality: reality-graph agent grounding — an agent queries physical state
(presence / changed-since / which-untrusted / action-allowed) and gets
answers backed by witness evidence hashes, not prompt inference.
- node: NervousSystemNode appliance facade wiring engine + reality + custody +
boundary forecaster; emits deltas/boundaries/witnesses/forecasts (no raw
signal) and answers grounded queries.
Fixes during integration: swarm bottleneck now uses the weakest edge (engine
partition is unreliable); node test uses 3 zones (2-zone min-cut boundary is
ambiguous — a real limitation now documented). ADR-198 updated.
59 tests pass (54 unit + 2 integration + 3 doctest), deterministic; clippy
clean; all source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* chore(ci): wire condense+perception crates into publish + regression guard (#547)
Aligns the new ruvector-graph-condense, ruvector-graph-condense-wasm, and
ruvector-perception crates with the workspace release plumbing.
- Bump their ruvector-mincut (and graph-condense) dep pins from "2.0.1" to
"2.2.3" to match the workspace version they are built and tested against.
The old "^2.0.1" pin would resolve a crates.io publish against the stale
published mincut 2.0.6, risking a crate that fails to compile downstream.
- publish-all.yml: publish the three crates (plus mincut as substrate) to
crates.io in dependency order with index-settle waits, matching the
existing --allow-dirty / continue-on-error style.
- regression-guard.yml: run the new crates' tests (they were build-checked
but never tested in CI) and forbid regressing the mincut pin back to 2.0.x.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(graph-condense): rustfmt, clippy -D warnings, and cargo-deny advisory (#547)
CI green-up for the new condense/perception crates:
- rustfmt: format all source/bench/example/test files in the new crates
(the PR was committed unformatted; CI Rustfmt flagged all 29 files).
- clippy -D warnings: condense.rs used `sort_by(|a,b| key.cmp(&key))` which
trips clippy::unnecessary_sort_by under `-D warnings`; switch to
`sort_by_key`. (Earlier local clippy didn't deny warnings, so it slipped.)
- cargo-deny: ignore RUSTSEC-2026-0173 (proc-macro-error2 unmaintained).
Pre-existing transitive dep (validator_derive -> validator, via the
ruvector-scipix example), same crate family as the already-ignored
RUSTSEC-2024-0370. Not introduced by this PR. Re-review 2026-07-01.
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs(graph-condense): add crate READMEs for crates.io publish (#547)
The new graph-condense crates were wired to publish without a README (101/136
workspace crates have one; every published crate does). Add READMEs matching
the repo's badge-header convention and the `readme = "README.md"` field so the
crates.io pages render properly on first publish.
- ruvector-graph-condense: overview, SOTA positioning, quick-start (using the
real NodeFeatures::new/set + DynamicGraph::insert_edge API), region-method
table, and the honest ADR-196/197 limitations.
- ruvector-graph-condense-wasm: short binding README pointing at the core crate.
Perception crate intentionally left as-is (out of scope for this request).
Co-Authored-By: claude-flow <ruv@ruv.net>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: ruvnet <ruvnet@gmail.com>
This commit is contained in:
rUv
GitHub
parent
edf6b04219
commit
22689a7511
48 changed files with 9108 additions and 0 deletions
41
.github/workflows/publish-all.yml
vendored
41
.github/workflows/publish-all.yml
vendored
|
|
@ -223,6 +223,43 @@ jobs:
|
|||
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
||||
continue-on-error: true
|
||||
|
||||
# Graph condensation + perception stack (PR #547). Published in dependency
|
||||
# order: mincut (substrate) -> graph-condense / perception -> wasm bindings.
|
||||
# Each new crate pins ruvector-mincut = "2.2.3", so mincut 2.2.3 must reach
|
||||
# the index first — hence the publish + settle step below.
|
||||
- name: Wait for crates.io index
|
||||
run: sleep 30
|
||||
|
||||
- name: Publish ruvector-mincut
|
||||
run: cargo publish -p ruvector-mincut --allow-dirty
|
||||
env:
|
||||
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
||||
continue-on-error: true
|
||||
|
||||
- name: Wait for crates.io index
|
||||
run: sleep 30
|
||||
|
||||
- name: Publish ruvector-graph-condense
|
||||
run: cargo publish -p ruvector-graph-condense --allow-dirty
|
||||
env:
|
||||
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
||||
continue-on-error: true
|
||||
|
||||
- name: Publish ruvector-perception
|
||||
run: cargo publish -p ruvector-perception --allow-dirty
|
||||
env:
|
||||
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
||||
continue-on-error: true
|
||||
|
||||
- name: Wait for crates.io index
|
||||
run: sleep 30
|
||||
|
||||
- name: Publish ruvector-graph-condense-wasm
|
||||
run: cargo publish -p ruvector-graph-condense-wasm --allow-dirty
|
||||
env:
|
||||
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
||||
continue-on-error: true
|
||||
|
||||
- name: Summary
|
||||
run: |
|
||||
echo "## crates.io Publishing" >> $GITHUB_STEP_SUMMARY
|
||||
|
|
@ -230,6 +267,10 @@ jobs:
|
|||
echo "✅ ruvector-attention" >> $GITHUB_STEP_SUMMARY
|
||||
echo "✅ ruvector-math-wasm" >> $GITHUB_STEP_SUMMARY
|
||||
echo "✅ ruvector-attention-wasm" >> $GITHUB_STEP_SUMMARY
|
||||
echo "✅ ruvector-mincut" >> $GITHUB_STEP_SUMMARY
|
||||
echo "✅ ruvector-graph-condense" >> $GITHUB_STEP_SUMMARY
|
||||
echo "✅ ruvector-perception" >> $GITHUB_STEP_SUMMARY
|
||||
echo "✅ ruvector-graph-condense-wasm" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
# ============================================================================
|
||||
# Phase 5: Publish to npm
|
||||
|
|
|
|||
37
.github/workflows/regression-guard.yml
vendored
37
.github/workflows/regression-guard.yml
vendored
|
|
@ -392,3 +392,40 @@ jobs:
|
|||
done <<< "$entries"
|
||||
done < <(find npm/packages -name package.json -not -path '*/node_modules/*')
|
||||
exit $fail
|
||||
|
||||
# PR #547: the graph-condense + perception crates are workspace members, so
|
||||
# `cargo check --workspace` builds them — but no job runs their unit tests,
|
||||
# integration tests, or benches, so they can silently rot. Run them explicitly.
|
||||
graph-condense-perception-tests:
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
- name: cargo test graph-condense + perception
|
||||
run: |
|
||||
set -e
|
||||
cargo test -p ruvector-graph-condense --all-features
|
||||
cargo test -p ruvector-perception --all-features
|
||||
# wasm crate: rlib path type-checks on host; the cdylib/getrandom-js
|
||||
# path is exercised on wasm32 by the no-systemtime-in-wasm-crates job.
|
||||
cargo check -p ruvector-graph-condense-wasm
|
||||
|
||||
# PR #547: these crates were built and tested against the local workspace
|
||||
# ruvector-mincut (2.2.3, path dep). A `version = "2.0.x"` pin would resolve
|
||||
# downstream to the stale published mincut 2.0.6 and could ship a crate that
|
||||
# fails to compile for crates.io consumers. Forbid regressing the pin to 2.0.x.
|
||||
mincut-pin-tracks-workspace-version:
|
||||
runs-on: ubuntu-22.04
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: ruvector-mincut pin must not regress to 2.0.x
|
||||
run: |
|
||||
set -e
|
||||
if grep -rnE 'ruvector-mincut[^#]*version *= *"[~^]?2\.0\.' \
|
||||
crates/ruvector-graph-condense/Cargo.toml \
|
||||
crates/ruvector-graph-condense-wasm/Cargo.toml \
|
||||
crates/ruvector-perception/Cargo.toml ; then
|
||||
echo "::error::ruvector-mincut pinned to 2.0.x but the workspace is 2.2.3 (regression of PR #547). A crates.io publish would resolve mincut to the stale published 2.0.6 and may not compile downstream. Keep the pin aligned with the workspace version."
|
||||
exit 1
|
||||
fi
|
||||
|
|
|
|||
36
Cargo.lock
generated
36
Cargo.lock
generated
|
|
@ -9458,6 +9458,31 @@ dependencies = [
|
|||
"zstd",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruvector-graph-condense"
|
||||
version = "2.2.3"
|
||||
dependencies = [
|
||||
"criterion 0.5.1",
|
||||
"rand 0.8.5",
|
||||
"rayon",
|
||||
"ruvector-mincut 2.2.3",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"thiserror 2.0.18",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruvector-graph-condense-wasm"
|
||||
version = "2.2.3"
|
||||
dependencies = [
|
||||
"getrandom 0.2.17",
|
||||
"ruvector-graph-condense",
|
||||
"ruvector-mincut 2.2.3",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruvector-graph-node"
|
||||
version = "2.2.3"
|
||||
|
|
@ -9870,6 +9895,17 @@ dependencies = [
|
|||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruvector-perception"
|
||||
version = "2.2.3"
|
||||
dependencies = [
|
||||
"ruvector-mincut 2.2.3",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2 0.10.9",
|
||||
"thiserror 2.0.18",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruvector-profiler"
|
||||
version = "2.2.3"
|
||||
|
|
|
|||
|
|
@ -233,6 +233,11 @@ members = [
|
|||
"crates/ruvllm_retrieval_diffusion",
|
||||
# RAIRS IVF: Redundant Assignment + Amplified Inverse Residual (ADR-193)
|
||||
"crates/ruvector-rairs",
|
||||
# Structure-preserving graph condensation via dynamic min-cut communities
|
||||
"crates/ruvector-graph-condense",
|
||||
"crates/ruvector-graph-condense-wasm",
|
||||
# Perception substrate: delta -> boundary -> coherence -> proof -> action
|
||||
"crates/ruvector-perception",
|
||||
]
|
||||
resolver = "2"
|
||||
|
||||
|
|
|
|||
29
crates/ruvector-graph-condense-wasm/Cargo.toml
Normal file
29
crates/ruvector-graph-condense-wasm/Cargo.toml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
[package]
|
||||
name = "ruvector-graph-condense-wasm"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
authors.workspace = true
|
||||
repository.workspace = true
|
||||
readme = "README.md"
|
||||
description = "WASM bindings for ruvector-graph-condense: structure-preserving + differentiable-min-cut graph condensation in the browser / on the edge"
|
||||
|
||||
[lib]
|
||||
crate-type = ["cdylib", "rlib"]
|
||||
|
||||
[dependencies]
|
||||
# Core condenser without the Rayon parallel feature (wasm32 has no threads).
|
||||
ruvector-graph-condense = { version = "2.2.3", path = "../ruvector-graph-condense", default-features = false }
|
||||
ruvector-mincut = { version = "2.2.3", path = "../ruvector-mincut", default-features = false, features = ["wasm"] }
|
||||
wasm-bindgen = { workspace = true }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
|
||||
# Force the JS RNG backend only on wasm (rand 0.8 -> getrandom 0.2); keeps the
|
||||
# `js` feature out of native builds / feature unification.
|
||||
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
||||
getrandom = { version = "0.2", features = ["js"] }
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
30
crates/ruvector-graph-condense-wasm/README.md
Normal file
30
crates/ruvector-graph-condense-wasm/README.md
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
# RuVector Graph Condense — WASM
|
||||
|
||||
[](https://crates.io/crates/ruvector-graph-condense-wasm)
|
||||
[](https://docs.rs/ruvector-graph-condense-wasm)
|
||||
[](LICENSE)
|
||||
[](https://github.com/ruvnet/ruvector)
|
||||
[](https://ruv.io)
|
||||
|
||||
**WASM bindings for [`ruvector-graph-condense`](https://crates.io/crates/ruvector-graph-condense).**
|
||||
|
||||
*Structure-preserving + differentiable-min-cut graph condensation in the browser or on the edge.*
|
||||
|
||||
---
|
||||
|
||||
Thin `wasm-bindgen` wrapper over the core condenser, built without the Rayon
|
||||
`parallel` feature (wasm32 has no threads) and with the JS `getrandom` backend
|
||||
gated to `cfg(target_arch = "wasm32")` so native builds are unaffected.
|
||||
|
||||
For the algorithm, region methods, and limitations, see the core crate's
|
||||
[README](https://crates.io/crates/ruvector-graph-condense) and **ADR-196 / ADR-197**.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
wasm-pack build crates/ruvector-graph-condense-wasm --target web
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT © [ruv.io](https://ruv.io)
|
||||
136
crates/ruvector-graph-condense-wasm/src/lib.rs
Normal file
136
crates/ruvector-graph-condense-wasm/src/lib.rs
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
//! WASM bindings for `ruvector-graph-condense`.
|
||||
//!
|
||||
//! Exposes the structure-preserving condenser and the trained differentiable
|
||||
//! min-cut condenser to JavaScript / the browser / edge runtimes, so a graph can
|
||||
//! be condensed into a small deployable artifact client-side. Built without the
|
||||
//! `parallel` (Rayon) feature, since `wasm32-unknown-unknown` has no threads.
|
||||
//!
|
||||
//! Graphs are passed as flat typed arrays from JS (`src`, `dst`, `w`: parallel
|
||||
//! arrays, one entry per undirected edge; `features`: row-major `n × dim` `f32`
|
||||
//! embeddings). Results are returned as JSON (a serialised `CondensedGraph`).
|
||||
|
||||
use ruvector_graph_condense::{
|
||||
CondenseConfig, CondenseMethod, DiffCutConfig, GraphCondenser, NodeFeatures,
|
||||
};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
use wasm_bindgen::prelude::*;
|
||||
|
||||
fn build(
|
||||
n: u32,
|
||||
src: &[u32],
|
||||
dst: &[u32],
|
||||
w: &[f32],
|
||||
features: &[f32],
|
||||
dim: u32,
|
||||
) -> Result<(DynamicGraph, NodeFeatures), String> {
|
||||
let n = n as usize;
|
||||
let dim = dim as usize;
|
||||
if src.len() != dst.len() || src.len() != w.len() {
|
||||
return Err("src/dst/w length mismatch".into());
|
||||
}
|
||||
if features.len() != n * dim {
|
||||
return Err(format!(
|
||||
"features length {} != n*dim {}",
|
||||
features.len(),
|
||||
n * dim
|
||||
));
|
||||
}
|
||||
let g = DynamicGraph::new();
|
||||
let mut f = NodeFeatures::new(dim, 0);
|
||||
for v in 0..n {
|
||||
f.set_embedding(v as u64, features[v * dim..(v + 1) * dim].to_vec())
|
||||
.map_err(|e| e.to_string())?;
|
||||
g.add_vertex(v as u64);
|
||||
}
|
||||
for i in 0..src.len() {
|
||||
let _ = g.insert_edge(src[i] as u64, dst[i] as u64, w[i] as f64);
|
||||
}
|
||||
Ok((g, f))
|
||||
}
|
||||
|
||||
fn run(config: CondenseConfig, args: BuildArgs) -> String {
|
||||
match build(args.n, args.src, args.dst, args.w, args.features, args.dim) {
|
||||
Ok((g, f)) => match GraphCondenser::new(config).condense(&g, &f) {
|
||||
Ok(c) => serde_json::to_string(&c).unwrap_or_else(|e| err_json(&e.to_string())),
|
||||
Err(e) => err_json(&e.to_string()),
|
||||
},
|
||||
Err(e) => err_json(&e),
|
||||
}
|
||||
}
|
||||
|
||||
struct BuildArgs<'a> {
|
||||
n: u32,
|
||||
src: &'a [u32],
|
||||
dst: &'a [u32],
|
||||
w: &'a [f32],
|
||||
features: &'a [f32],
|
||||
dim: u32,
|
||||
}
|
||||
|
||||
fn err_json(msg: &str) -> String {
|
||||
format!(
|
||||
"{{\"error\":{}}}",
|
||||
serde_json::to_string(msg).unwrap_or_default()
|
||||
)
|
||||
}
|
||||
|
||||
/// Condense with the default structure-preserving `WeakBoundary` method.
|
||||
/// Returns a JSON `CondensedGraph` (or `{"error": "..."}`).
|
||||
#[wasm_bindgen]
|
||||
pub fn condense_weak(
|
||||
n: u32,
|
||||
src: &[u32],
|
||||
dst: &[u32],
|
||||
w: &[f32],
|
||||
features: &[f32],
|
||||
dim: u32,
|
||||
) -> String {
|
||||
run(
|
||||
CondenseConfig::default(),
|
||||
BuildArgs {
|
||||
n,
|
||||
src,
|
||||
dst,
|
||||
w,
|
||||
features,
|
||||
dim,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Condense with the trained differentiable min-cut method (Adam + warm-start).
|
||||
#[wasm_bindgen]
|
||||
pub fn condense_diffmincut(
|
||||
n: u32,
|
||||
src: &[u32],
|
||||
dst: &[u32],
|
||||
w: &[f32],
|
||||
features: &[f32],
|
||||
dim: u32,
|
||||
num_clusters: u32,
|
||||
) -> String {
|
||||
let cfg = CondenseConfig {
|
||||
method: CondenseMethod::DiffMinCut(DiffCutConfig {
|
||||
num_clusters: num_clusters.max(1) as usize,
|
||||
..Default::default()
|
||||
}),
|
||||
normalize_centroids: false,
|
||||
};
|
||||
run(
|
||||
cfg,
|
||||
BuildArgs {
|
||||
n,
|
||||
src,
|
||||
dst,
|
||||
w,
|
||||
features,
|
||||
dim,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Crate version (handy for cache-busting a deployed bundle).
|
||||
#[wasm_bindgen]
|
||||
pub fn version() -> String {
|
||||
env!("CARGO_PKG_VERSION").to_string()
|
||||
}
|
||||
42
crates/ruvector-graph-condense/Cargo.toml
Normal file
42
crates/ruvector-graph-condense/Cargo.toml
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
[package]
|
||||
name = "ruvector-graph-condense"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
authors.workspace = true
|
||||
repository.workspace = true
|
||||
readme = "README.md"
|
||||
description = "Structure-preserving graph condensation: collapse large feature graphs into small synthetic graphs using dynamic min-cut community boundaries"
|
||||
keywords = ["graph", "condensation", "distillation", "min-cut", "gnn"]
|
||||
categories = ["algorithms", "science", "mathematics"]
|
||||
|
||||
[lib]
|
||||
crate-type = ["rlib"]
|
||||
|
||||
[dependencies]
|
||||
# Substrate: dynamic min-cut engine (DynamicGraph, CommunityDetector, GraphPartitioner).
|
||||
# Default features only (exact + approximate); integration re-exports are unconditional.
|
||||
ruvector-mincut = { version = "2.2.3", path = "../ruvector-mincut", default-features = false }
|
||||
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
thiserror = { workspace = true }
|
||||
rand = { workspace = true }
|
||||
rayon = { workspace = true, optional = true }
|
||||
|
||||
[features]
|
||||
default = ["parallel"]
|
||||
# Rayon-based parallelism for the differentiable min-cut optimiser. Off for
|
||||
# targets without threads (e.g. wasm32-unknown-unknown).
|
||||
parallel = ["dep:rayon"]
|
||||
|
||||
[dev-dependencies]
|
||||
criterion = { workspace = true }
|
||||
|
||||
[[bench]]
|
||||
name = "condense_bench"
|
||||
harness = false
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
74
crates/ruvector-graph-condense/README.md
Normal file
74
crates/ruvector-graph-condense/README.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# RuVector Graph Condense
|
||||
|
||||
[](https://crates.io/crates/ruvector-graph-condense)
|
||||
[](https://docs.rs/ruvector-graph-condense)
|
||||
[](LICENSE)
|
||||
[](https://github.com/ruvnet/ruvector)
|
||||
[](https://ruv.io)
|
||||
|
||||
**Training-free, structure-preserving, provenance-retaining graph condensation.**
|
||||
|
||||
*Collapse a large feature graph into a small synthetic graph of super-nodes while preserving its cut structure — plus a differentiable min-cut loss.*
|
||||
|
||||
---
|
||||
|
||||
## Why This Matters
|
||||
|
||||
Graph condensation shrinks a graph + per-node embeddings (+ optional labels) into a much smaller graph that downstream tasks can still reason over. The published SOTA — GCond, SFGC, GEOM, SGDD — *synthesises* a fake graph via expensive, supervised bi-level gradient/distribution/trajectory matching, and **discards the node→original mapping**.
|
||||
|
||||
`ruvector-graph-condense` takes the complementary, training-free route the 2024–2026 condensation surveys flag as under-explored:
|
||||
|
||||
- **Min-cut community structure as the condensation prior** (not k-means).
|
||||
- **Cuts preserved by construction** — boundary edges become weighted super-edges; `metrics::cut_inflation` quantifies fidelity.
|
||||
- **Provenance retained** — every `CondensedNode` keeps its `members`, so each super-node is auditable and explainable.
|
||||
- **A differentiable min-cut *loss*** (`diffcut`, MinCutPool-style relaxed normalized cut + orthogonality) — analytic gradients, gradient-checked across K=2,3,4 to <1e-5.
|
||||
|
||||
Built on the dynamic min-cut engine [`ruvector-mincut`](https://crates.io/crates/ruvector-mincut).
|
||||
|
||||
## Quick Start
|
||||
|
||||
```rust
|
||||
use ruvector_graph_condense::{CondenseConfig, GraphCondenser, NodeFeatures};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
// Build a graph (insert_edge returns a Result; &self — it is concurrent).
|
||||
let graph = DynamicGraph::new();
|
||||
let _ = graph.insert_edge(0, 1, 1.0);
|
||||
let _ = graph.insert_edge(1, 2, 1.0);
|
||||
let _ = graph.insert_edge(2, 3, 0.1); // weak boundary edge
|
||||
|
||||
// Per-vertex embeddings (+ optional labels): NodeFeatures::new(dim, num_classes).
|
||||
let mut features = NodeFeatures::new(2, 1);
|
||||
for v in 0..4u64 {
|
||||
features.set(v, vec![v as f32, 0.0], 0).unwrap();
|
||||
}
|
||||
|
||||
let condenser = GraphCondenser::new(CondenseConfig::default()); // WeakBoundary, 0.5
|
||||
let condensed = condenser.condense(&graph, &features).unwrap();
|
||||
|
||||
for node in &condensed.nodes {
|
||||
// Each super-node keeps the original vertices it came from (provenance).
|
||||
println!("super-node {:?} <- members {:?}", node.representative, node.members);
|
||||
}
|
||||
```
|
||||
|
||||
## Region Methods (`CondenseMethod`)
|
||||
|
||||
| Method | Notes |
|
||||
|--------|-------|
|
||||
| `WeakBoundary` (default) | Linear-time union-find on weak edges. ~4 ms @ 2048 nodes. |
|
||||
| `MinCutCommunity` / `Partition` | Delegate to the min-cut engine. Structure-aware on graphs with sharp bottlenecks; documented best-effort otherwise. |
|
||||
| `ConnectedComponents` | Cheap baseline — one region per component. |
|
||||
| `DiffMinCut` | Differentiable, *trained* assignment (opt-in). |
|
||||
|
||||
## Honest Limitations
|
||||
|
||||
- The recursive **global min-cut engine methods degenerate to singleton-peeling** on graphs without sharp bottlenecks and are super-linear (~24 s @ 96 nodes) — which is why the linear-time `WeakBoundary` is the default.
|
||||
- **`DiffMinCut` is K-sensitive** (known MinCutPool finickiness): it recovers small/dense graphs but underperforms `WeakBoundary` at large K. Momentum + unit-scale init help, but there is no convergence guarantee.
|
||||
- This is structure-preserving **coarsening-condensation** (keeps provenance) — **not** accuracy-matched GCond-style condensation; no GNN-retrain accuracy numbers are claimed.
|
||||
|
||||
See **ADR-196** (structure-preserving condensation) and **ADR-197** (differentiable min-cut loss) for the full design and findings.
|
||||
|
||||
## License
|
||||
|
||||
MIT © [ruv.io](https://ruv.io)
|
||||
210
crates/ruvector-graph-condense/benches/condense_bench.rs
Normal file
210
crates/ruvector-graph-condense/benches/condense_bench.rs
Normal file
|
|
@ -0,0 +1,210 @@
|
|||
//! Condensation throughput benchmarks.
|
||||
//!
|
||||
//! Run with: `cargo bench -p ruvector-graph-condense --bench condense_bench`
|
||||
//!
|
||||
//! Two groups, because the methods differ by orders of magnitude:
|
||||
//!
|
||||
//! * **scalable** — `WeakBoundary` (default) and `ConnectedComponents` are
|
||||
//! single-pass + union-find, ~microseconds even at thousands of nodes.
|
||||
//! * **engine** — `MinCutCommunity` and `Partition` delegate to the recursive
|
||||
//! dynamic-min-cut engine, which copies the graph per split; they are
|
||||
//! super-linear and benchmarked only at small sizes to document the cost.
|
||||
|
||||
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
|
||||
use ruvector_graph_condense::condense::{CondenseConfig, CondenseMethod, GraphCondenser};
|
||||
use ruvector_graph_condense::diffcut::{DiffCutCondenser, DiffCutConfig};
|
||||
use ruvector_graph_condense::metrics::evaluate_full;
|
||||
use ruvector_graph_condense::synthetic::PlantedPartition;
|
||||
|
||||
fn planted(communities: usize, size: usize, seed: u64) -> PlantedPartition {
|
||||
PlantedPartition {
|
||||
num_communities: communities,
|
||||
community_size: size,
|
||||
dim: 16,
|
||||
p_intra: 0.4,
|
||||
p_inter: 0.002,
|
||||
seed,
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
/// Fast methods, swept to larger graphs.
|
||||
fn bench_scalable(c: &mut Criterion) {
|
||||
let mut group = c.benchmark_group("condense_scalable");
|
||||
for &(communities, size) in &[(8usize, 32usize), (16, 64), (32, 64)] {
|
||||
let pp = planted(communities, size, 1);
|
||||
let (graph, features) = pp.generate();
|
||||
let n = pp.total_vertices();
|
||||
group.throughput(Throughput::Elements(n as u64));
|
||||
|
||||
for (name, method) in [
|
||||
(
|
||||
"weak_boundary",
|
||||
CondenseMethod::WeakBoundary {
|
||||
relative_threshold: 0.5,
|
||||
},
|
||||
),
|
||||
("connected_components", CondenseMethod::ConnectedComponents),
|
||||
] {
|
||||
let condenser = GraphCondenser::new(CondenseConfig {
|
||||
method,
|
||||
normalize_centroids: false,
|
||||
});
|
||||
group.bench_with_input(
|
||||
BenchmarkId::new(name, n),
|
||||
&(condenser, &graph, &features),
|
||||
|b, (condenser, graph, features)| {
|
||||
b.iter(|| {
|
||||
let c = condenser.condense(graph, features).unwrap();
|
||||
criterion::black_box(c.node_count())
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
group.finish();
|
||||
}
|
||||
|
||||
/// Engine-backed methods, small sizes only (super-linear cost).
|
||||
fn bench_engine(c: &mut Criterion) {
|
||||
let mut group = c.benchmark_group("condense_engine");
|
||||
group.sample_size(10);
|
||||
// Capped small: recursive global min-cut is super-linear (e.g. ~24s at 96
|
||||
// nodes), so larger sizes would make the suite intractable. The point is to
|
||||
// document the cost gap vs. the scalable group, not to sweep.
|
||||
for &(communities, size) in &[(3usize, 10usize), (4, 12)] {
|
||||
let pp = planted(communities, size, 2);
|
||||
let (graph, features) = pp.generate();
|
||||
let n = pp.total_vertices();
|
||||
group.throughput(Throughput::Elements(n as u64));
|
||||
|
||||
for (name, method) in [
|
||||
(
|
||||
"mincut_community",
|
||||
CondenseMethod::MinCutCommunity { min_region_size: 2 },
|
||||
),
|
||||
(
|
||||
"partition",
|
||||
CondenseMethod::Partition {
|
||||
num_regions: communities,
|
||||
},
|
||||
),
|
||||
] {
|
||||
let condenser = GraphCondenser::new(CondenseConfig {
|
||||
method,
|
||||
normalize_centroids: false,
|
||||
});
|
||||
group.bench_with_input(
|
||||
BenchmarkId::new(name, n),
|
||||
&(condenser, &graph, &features),
|
||||
|b, (condenser, graph, features)| {
|
||||
b.iter(|| {
|
||||
let c = condenser.condense(graph, features).unwrap();
|
||||
criterion::black_box(c.node_count())
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
group.finish();
|
||||
}
|
||||
|
||||
/// Cost of the full metric bundle (includes two exact min-cut solves).
|
||||
fn bench_metrics(c: &mut Criterion) {
|
||||
let pp = planted(8, 24, 3);
|
||||
let (graph, features) = pp.generate();
|
||||
let condenser = GraphCondenser::new(CondenseConfig::default());
|
||||
let condensed = condenser.condense(&graph, &features).unwrap();
|
||||
|
||||
c.bench_function("evaluate_full_with_cut", |b| {
|
||||
b.iter(|| {
|
||||
let m = evaluate_full(&graph, &condensed);
|
||||
criterion::black_box(m.node_reduction_ratio)
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/// Differentiable min-cut training cost (gradient descent over the assignment).
|
||||
fn bench_diffcut(c: &mut Criterion) {
|
||||
let mut group = c.benchmark_group("condense_diffcut");
|
||||
group.sample_size(10);
|
||||
for &(communities, size) in &[(4usize, 16usize), (8, 24)] {
|
||||
let pp = planted(communities, size, 4);
|
||||
let (graph, _features) = pp.generate();
|
||||
let n = pp.total_vertices();
|
||||
group.throughput(Throughput::Elements(n as u64));
|
||||
let condenser = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: communities,
|
||||
iterations: 100,
|
||||
seed: 1,
|
||||
..Default::default()
|
||||
});
|
||||
group.bench_with_input(
|
||||
BenchmarkId::new("train", n),
|
||||
&(condenser, &graph),
|
||||
|b, (condenser, graph)| {
|
||||
b.iter(|| {
|
||||
let r = condenser.train(graph).unwrap();
|
||||
criterion::black_box(r.loss().total)
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
group.finish();
|
||||
}
|
||||
|
||||
/// DiffMinCut optimisation levers on a larger graph: full-sequential vs
|
||||
/// full-parallel vs edge-minibatch (fixed 100 iterations, early-stop off).
|
||||
fn bench_diffcut_levers(c: &mut Criterion) {
|
||||
let mut group = c.benchmark_group("condense_diffcut_levers");
|
||||
group.sample_size(10);
|
||||
let pp = planted(16, 64, 5); // 1024 nodes
|
||||
let (graph, _f) = pp.generate();
|
||||
let n = pp.total_vertices();
|
||||
group.throughput(Throughput::Elements(n as u64));
|
||||
let base = DiffCutConfig {
|
||||
num_clusters: 16,
|
||||
iterations: 100,
|
||||
tolerance: 0.0,
|
||||
seed: 1,
|
||||
..Default::default()
|
||||
};
|
||||
let variants = [
|
||||
("full_sequential", DiffCutConfig { ..base.clone() }),
|
||||
(
|
||||
"full_parallel",
|
||||
DiffCutConfig {
|
||||
parallel: true,
|
||||
..base.clone()
|
||||
},
|
||||
),
|
||||
(
|
||||
"minibatch_2048",
|
||||
DiffCutConfig {
|
||||
minibatch_edges: Some(2048),
|
||||
..base.clone()
|
||||
},
|
||||
),
|
||||
];
|
||||
for (name, cfg) in variants {
|
||||
let condenser = DiffCutCondenser::new(cfg);
|
||||
group.bench_with_input(
|
||||
BenchmarkId::new(name, n),
|
||||
&(condenser, &graph),
|
||||
|b, (c, g)| {
|
||||
b.iter(|| criterion::black_box(c.train(g).unwrap().loss().total));
|
||||
},
|
||||
);
|
||||
}
|
||||
group.finish();
|
||||
}
|
||||
|
||||
criterion_group!(
|
||||
benches,
|
||||
bench_scalable,
|
||||
bench_engine,
|
||||
bench_diffcut,
|
||||
bench_diffcut_levers,
|
||||
bench_metrics
|
||||
);
|
||||
criterion_main!(benches);
|
||||
225
crates/ruvector-graph-condense/examples/accuracy_eval.rs
Normal file
225
crates/ruvector-graph-condense/examples/accuracy_eval.rs
Normal file
|
|
@ -0,0 +1,225 @@
|
|||
//! Accuracy-retention evaluation — the graph-condensation field's standard
|
||||
//! protocol: train a GNN on the **condensed** graph, test it on the **original**
|
||||
//! graph's held-out nodes, and report `accuracy(condensed) / accuracy(full)`.
|
||||
//!
|
||||
//! Run: `cargo run --release -p ruvector-graph-condense --example accuracy_eval`
|
||||
//!
|
||||
//! Honest scope: this runs on a *controlled synthetic* node-classification task
|
||||
//! (planted communities as classes, noisy features so the graph actually
|
||||
//! matters), not the canonical Cora/Citeseer benchmarks — so it is a
|
||||
//! substantiated *retention* measurement, not a literal "beats GCond on Cora"
|
||||
//! claim. It closes the gap of having no learning-accuracy validation at all.
|
||||
|
||||
#![allow(clippy::needless_range_loop)] // index-heavy numeric example code
|
||||
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
use ruvector_graph_condense::gnn_eval::{accuracy, Gcn, GcnConfig, GcnGraph};
|
||||
use ruvector_graph_condense::{
|
||||
CondenseConfig, CondenseMethod, CondensedGraph, DiffCutConfig, GraphCondenser, NodeFeatures,
|
||||
};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
struct Task {
|
||||
classes: usize,
|
||||
per_class: usize, // nodes per class
|
||||
dim: usize,
|
||||
p_intra: f64,
|
||||
p_inter: f64,
|
||||
noise: f64,
|
||||
seed: u64,
|
||||
}
|
||||
|
||||
impl Task {
|
||||
fn n(&self) -> usize {
|
||||
self.classes * self.per_class
|
||||
}
|
||||
|
||||
/// Build graph + features + per-node class labels. Node `i` has class
|
||||
/// `i / per_class`; features are a class centroid + Gaussian-ish noise so
|
||||
/// raw features overlap and the graph carries real signal.
|
||||
fn generate(&self) -> (DynamicGraph, NodeFeatures, Vec<usize>) {
|
||||
let mut rng = StdRng::seed_from_u64(self.seed);
|
||||
let g = DynamicGraph::new();
|
||||
let mut f = NodeFeatures::new(self.dim, self.classes);
|
||||
let mut labels = vec![0usize; self.n()];
|
||||
|
||||
let centroids: Vec<Vec<f64>> = (0..self.classes)
|
||||
.map(|c| {
|
||||
(0..self.dim)
|
||||
.map(|d| if d % self.classes == c { 1.5 } else { 0.0 })
|
||||
.collect()
|
||||
})
|
||||
.collect();
|
||||
|
||||
for i in 0..self.n() {
|
||||
let cls = i / self.per_class;
|
||||
labels[i] = cls;
|
||||
let emb: Vec<f32> = (0..self.dim)
|
||||
.map(|d| (centroids[cls][d] + self.noise * rng.gen_range(-1.0..1.0)) as f32)
|
||||
.collect();
|
||||
f.set(i as u64, emb, cls).unwrap();
|
||||
g.add_vertex(i as u64);
|
||||
}
|
||||
for a in 0..self.n() {
|
||||
for b in (a + 1)..self.n() {
|
||||
let same = a / self.per_class == b / self.per_class;
|
||||
let p = if same { self.p_intra } else { self.p_inter };
|
||||
if rng.gen_bool(p) {
|
||||
let _ = g.insert_edge(a as u64, b as u64, 1.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
(g, f, labels)
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract contiguous `0..n` edge list / feature matrix from the graph.
|
||||
fn full_arrays(
|
||||
g: &DynamicGraph,
|
||||
f: &NodeFeatures,
|
||||
n: usize,
|
||||
) -> (Vec<(usize, usize, f64)>, Vec<f64>) {
|
||||
let edges = g
|
||||
.edges()
|
||||
.iter()
|
||||
.map(|e| (e.source as usize, e.target as usize, e.weight))
|
||||
.collect();
|
||||
let dim = f.dim();
|
||||
let mut x = vec![0f64; n * dim];
|
||||
for i in 0..n {
|
||||
if let Some(emb) = f.embedding(i as u64) {
|
||||
for d in 0..dim {
|
||||
x[i * dim + d] = emb[d] as f64;
|
||||
}
|
||||
}
|
||||
}
|
||||
(edges, x)
|
||||
}
|
||||
|
||||
/// Build the GCN training arrays for a condensed graph: centroids as features,
|
||||
/// dominant class as label, super-edges as adjacency.
|
||||
fn condensed_arrays(c: &CondensedGraph) -> (GcnGraph, Vec<f64>, Vec<usize>) {
|
||||
let cn = c.node_count();
|
||||
let dim = c.dim;
|
||||
let mut x = vec![0f64; cn * dim];
|
||||
let mut labels = vec![0usize; cn];
|
||||
for (i, node) in c.nodes.iter().enumerate() {
|
||||
for d in 0..dim {
|
||||
x[i * dim + d] = node.centroid[d] as f64;
|
||||
}
|
||||
labels[i] = node.dominant_class().unwrap_or(0);
|
||||
}
|
||||
let edges: Vec<(usize, usize, f64)> = c
|
||||
.edges
|
||||
.iter()
|
||||
.map(|e| (e.source as usize, e.target as usize, e.weight))
|
||||
.collect();
|
||||
(GcnGraph::from_edges(cn, &edges), x, labels)
|
||||
}
|
||||
|
||||
fn split(n: usize, train_frac: f64, seed: u64) -> (Vec<usize>, Vec<usize>) {
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
let (mut tr, mut te) = (Vec::new(), Vec::new());
|
||||
for i in 0..n {
|
||||
if rng.gen_bool(train_frac) {
|
||||
tr.push(i);
|
||||
} else {
|
||||
te.push(i);
|
||||
}
|
||||
}
|
||||
(tr, te)
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let task = Task {
|
||||
classes: 6,
|
||||
per_class: 60,
|
||||
dim: 24,
|
||||
p_intra: 0.12,
|
||||
p_inter: 0.004,
|
||||
noise: 1.4,
|
||||
seed: 2026,
|
||||
};
|
||||
let n = task.n();
|
||||
let (graph, feats, labels) = task.generate();
|
||||
let (full_edges, x_full) = full_arrays(&graph, &feats, n);
|
||||
let full_graph = GcnGraph::from_edges(n, &full_edges);
|
||||
let (train, test) = split(n, 0.6, 7);
|
||||
let cfg = GcnConfig::default();
|
||||
|
||||
println!(
|
||||
"Task: {} nodes, {} classes, {} edges, dim {}, noise {} (features overlap; graph matters)",
|
||||
n,
|
||||
task.classes,
|
||||
graph.num_edges(),
|
||||
task.dim,
|
||||
task.noise
|
||||
);
|
||||
println!("Protocol: train GNN on condensed graph -> test on original held-out nodes.\n");
|
||||
|
||||
// Baseline: train on the FULL graph's train split.
|
||||
let base = Gcn::train(
|
||||
&cfg,
|
||||
&full_graph,
|
||||
&x_full,
|
||||
task.dim,
|
||||
&labels,
|
||||
task.classes,
|
||||
&train,
|
||||
);
|
||||
let acc_full = accuracy(&base.predict(&full_graph, &x_full), &labels, &test);
|
||||
println!(
|
||||
"Baseline GNN (trained on full graph): test accuracy {:.1}%\n",
|
||||
acc_full * 100.0
|
||||
);
|
||||
|
||||
for (name, method) in [
|
||||
(
|
||||
"WeakBoundary",
|
||||
CondenseMethod::WeakBoundary {
|
||||
relative_threshold: 0.5,
|
||||
},
|
||||
),
|
||||
(
|
||||
"DiffMinCut",
|
||||
CondenseMethod::DiffMinCut(DiffCutConfig {
|
||||
num_clusters: task.classes * 3, // a few super-nodes per class -> more GNN training signal
|
||||
restarts: 3,
|
||||
iterations: 500,
|
||||
..Default::default()
|
||||
}),
|
||||
),
|
||||
] {
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method,
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&graph, &feats)
|
||||
.unwrap();
|
||||
let (cg, x_cond, lab_cond) = condensed_arrays(&c);
|
||||
let all: Vec<usize> = (0..c.node_count()).collect();
|
||||
// Train on condensed, test on the ORIGINAL graph's test split.
|
||||
let model = Gcn::train(&cfg, &cg, &x_cond, task.dim, &lab_cond, task.classes, &all);
|
||||
let acc_cond = accuracy(&model.predict(&full_graph, &x_full), &labels, &test);
|
||||
let retention = if acc_full > 0.0 {
|
||||
acc_cond / acc_full
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
println!(
|
||||
"{name:>12}: {} -> {} super-nodes ({:.0}x) | test acc {:.1}% | retention {:.1}%",
|
||||
n,
|
||||
c.node_count(),
|
||||
c.node_reduction_ratio(),
|
||||
acc_cond * 100.0,
|
||||
retention * 100.0,
|
||||
);
|
||||
}
|
||||
|
||||
println!(
|
||||
"\nRetention near 100% means a GNN trained on the tiny condensed graph classifies the\n\
|
||||
original's held-out nodes about as well as one trained on the full graph — the field's\n\
|
||||
core success criterion, here measured on controlled synthetic data."
|
||||
);
|
||||
}
|
||||
173
crates/ruvector-graph-condense/examples/worldgraph.rs
Normal file
173
crates/ruvector-graph-condense/examples/worldgraph.rs
Normal file
|
|
@ -0,0 +1,173 @@
|
|||
//! WorldGraph condensation demo — RuView `WorldGraph -> condense -> OccWorld`.
|
||||
//!
|
||||
//! Run: `cargo run -p ruvector-graph-condense --example worldgraph`
|
||||
//!
|
||||
//! RuView (github.com/ruvnet/RuView) records `WorldGraph` snapshots — a stream
|
||||
//! of spatial-occupancy observations from WiFi CSI sensing — and feeds them to
|
||||
//! an OccWorld world-model retrainer. A day of sensing is millions of
|
||||
//! observations; training on all of them on an edge device is impractical.
|
||||
//!
|
||||
//! This example simulates a small "day" of WorldGraph observations as a feature
|
||||
//! graph (observations = nodes with occupancy embeddings + an activity label;
|
||||
//! edges = spatial-temporal adjacency, heavy inside an activity, light across
|
||||
//! transitions) and condenses it into a handful of **event summaries** — exactly
|
||||
//! the `EventSummary { embedding, confidence, ... }` shape from the design brief,
|
||||
//! realised as [`CondensedNode`]s with provenance.
|
||||
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
use ruvector_graph_condense::{
|
||||
condense, evaluate_full, CondenseConfig, CondenseMethod, DiffCutConfig, GraphCondenser,
|
||||
NodeFeatures,
|
||||
};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
/// A simulated "day": `num_events` activities, each spanning `obs_per_event`
|
||||
/// consecutive observations, joined by light transition edges.
|
||||
struct DaySim {
|
||||
num_events: usize,
|
||||
obs_per_event: usize,
|
||||
num_activities: usize,
|
||||
dim: usize,
|
||||
seed: u64,
|
||||
}
|
||||
|
||||
impl DaySim {
|
||||
#[allow(clippy::needless_range_loop)] // `e` is the event index, used widely
|
||||
fn generate(&self) -> (DynamicGraph, NodeFeatures, Vec<usize>) {
|
||||
let mut rng = StdRng::seed_from_u64(self.seed);
|
||||
let g = DynamicGraph::new();
|
||||
let mut feats = NodeFeatures::new(self.dim, self.num_activities);
|
||||
let mut true_event = Vec::new(); // ground-truth event id per observation
|
||||
|
||||
// Each event gets a distinct occupancy centroid and an activity label.
|
||||
let centroids: Vec<Vec<f32>> = (0..self.num_events)
|
||||
.map(|e| {
|
||||
let mut c = vec![0f32; self.dim];
|
||||
c[e % self.dim] = 5.0 + (e / self.dim) as f32 * 5.0;
|
||||
c
|
||||
})
|
||||
.collect();
|
||||
let activity_of = |e: usize| e % self.num_activities;
|
||||
|
||||
let mut id = 0u64;
|
||||
let mut first_of_event = Vec::new();
|
||||
for e in 0..self.num_events {
|
||||
first_of_event.push(id);
|
||||
let prev_first = id;
|
||||
for i in 0..self.obs_per_event {
|
||||
let mut emb = centroids[e].clone();
|
||||
for x in &mut emb {
|
||||
*x += rng.gen_range(-0.4..0.4);
|
||||
}
|
||||
feats.set(id, emb, activity_of(e)).unwrap();
|
||||
true_event.push(e);
|
||||
// Temporal chain inside the event (heavy edges).
|
||||
if i > 0 {
|
||||
let _ = g.insert_edge(id - 1, id, 1.0);
|
||||
}
|
||||
// Dense intra-event co-occurrence: link to a few random earlier
|
||||
// observations of the same event, so each event is a coherent
|
||||
// community (not a thin chain).
|
||||
let links = i.min(3);
|
||||
for _ in 0..links {
|
||||
let other = prev_first + rng.gen_range(0..i as u64);
|
||||
let _ = g.insert_edge(other, id, 1.0);
|
||||
}
|
||||
id += 1;
|
||||
}
|
||||
}
|
||||
// Light transition edges between consecutive events (person moves zones).
|
||||
for e in 1..self.num_events {
|
||||
let a = first_of_event[e] - 1; // last obs of previous event
|
||||
let b = first_of_event[e]; // first obs of this event
|
||||
let _ = g.insert_edge(a, b, 0.1);
|
||||
}
|
||||
(g, feats, true_event)
|
||||
}
|
||||
}
|
||||
|
||||
fn report(title: &str, g: &DynamicGraph, condensed: &ruvector_graph_condense::CondensedGraph) {
|
||||
let m = evaluate_full(g, condensed);
|
||||
println!("\n=== {title} ===");
|
||||
println!(
|
||||
" observations (nodes): {} -> condensed events: {} ({:.1}x reduction)",
|
||||
m.source_nodes, m.condensed_nodes, m.node_reduction_ratio
|
||||
);
|
||||
println!(
|
||||
" edges: {} -> {} ({:.1}x) intra-weight kept: {:.1}% mean coherence: {:.2}",
|
||||
m.source_edges,
|
||||
m.condensed_edges,
|
||||
m.edge_reduction_ratio,
|
||||
m.intra_weight_ratio * 100.0,
|
||||
m.mean_coherence
|
||||
);
|
||||
println!(
|
||||
" activity purity: {:.1}% cut inflation: {}",
|
||||
m.label_purity * 100.0,
|
||||
m.cut_inflation
|
||||
.map(|c| format!("{c:.3} (1.0 = global cut preserved)"))
|
||||
.unwrap_or_else(|| "n/a".into())
|
||||
);
|
||||
println!(" event summaries (CondensedNode == EventSummary):");
|
||||
for n in condensed.nodes.iter().take(6) {
|
||||
println!(
|
||||
" event {:>2}: {:>3} obs | representative=obs#{:<3} | activity={:?} | confidence(coherence)={:.2}",
|
||||
n.id,
|
||||
n.weight,
|
||||
n.representative,
|
||||
n.dominant_class(),
|
||||
n.coherence
|
||||
);
|
||||
}
|
||||
if condensed.nodes.len() > 6 {
|
||||
println!(" ... ({} more)", condensed.nodes.len() - 6);
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let day = DaySim {
|
||||
num_events: 12,
|
||||
obs_per_event: 50,
|
||||
num_activities: 4,
|
||||
dim: 8,
|
||||
seed: 2026,
|
||||
};
|
||||
let (graph, features, _truth) = day.generate();
|
||||
println!(
|
||||
"Simulated WorldGraph: {} observations across {} events, {} edges.",
|
||||
graph.num_vertices(),
|
||||
day.num_events,
|
||||
graph.num_edges()
|
||||
);
|
||||
|
||||
// 1) Default structure-preserving condensation (weak-boundary) — the
|
||||
// recommended pipeline for a full-day, many-event WorldGraph.
|
||||
let weak = condense(&graph, &features).expect("condense");
|
||||
report("WeakBoundary (default)", &graph, &weak);
|
||||
println!(
|
||||
" -> a day of {} observations becomes {} deployable event summaries \
|
||||
(the artifact OccWorld would retrain on).",
|
||||
graph.num_vertices(),
|
||||
weak.node_count()
|
||||
);
|
||||
|
||||
// 2) Trained differentiable min-cut on the SAME large-K WorldGraph. With
|
||||
// Adam + warm-start init (default) it now recovers all 12 events — the
|
||||
// optimisation work that made the trained method viable at scale.
|
||||
let diff = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::DiffMinCut(DiffCutConfig {
|
||||
num_clusters: day.num_events,
|
||||
..Default::default()
|
||||
}),
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&graph, &features)
|
||||
.expect("diff condense");
|
||||
report("DiffMinCut (trained, Adam + warm-start)", &graph, &diff);
|
||||
println!(
|
||||
"\nBoth methods recover the day's events; DiffMinCut now scales to large K \
|
||||
via Adam + warm-start (it refines the WeakBoundary prior with the \
|
||||
differentiable normalized-cut objective)."
|
||||
);
|
||||
}
|
||||
500
crates/ruvector-graph-condense/src/condense.rs
Normal file
500
crates/ruvector-graph-condense/src/condense.rs
Normal file
|
|
@ -0,0 +1,500 @@
|
|||
//! The condensation engine: partition a feature graph into structural regions
|
||||
//! and collapse each region into a representative super-node.
|
||||
//!
|
||||
//! Unlike gradient-/distribution-matching condensation (GCond, GCDM, SFGC),
|
||||
//! which *synthesise* a small graph by optimising a learning objective, this is
|
||||
//! a **structure-preserving** condenser: regions come from the dynamic min-cut
|
||||
//! community structure, so the condensed topology mirrors the real cut
|
||||
//! structure of the source graph. Boundary edges survive as weighted
|
||||
//! super-edges; cuts are preserved by construction rather than by training.
|
||||
|
||||
use crate::diffcut::{DiffCutCondenser, DiffCutConfig};
|
||||
use crate::error::{CondenseError, Result};
|
||||
use crate::features::NodeFeatures;
|
||||
use crate::node::{CondensedEdge, CondensedGraph, CondensedNode};
|
||||
use crate::regions::{
|
||||
centroid_and_medoid, class_distribution, ensure_coverage, l2_normalize, weak_boundary_regions,
|
||||
};
|
||||
use ruvector_mincut::{CommunityDetector, DynamicGraph, GraphPartitioner, VertexId};
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
/// How the source graph is partitioned into regions before collapsing.
|
||||
///
|
||||
/// Note: region detection only decides *membership*. Super-edges are always
|
||||
/// rebuilt from the original graph's edges, so structure preservation does not
|
||||
/// depend on the method chosen here.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum CondenseMethod {
|
||||
/// **Default.** Cut every edge lighter than `relative_threshold * mean
|
||||
/// edge weight`, then take the connected components of what remains. This
|
||||
/// is a one-shot approximation to removing the light min-cut boundaries:
|
||||
/// robust, deterministic, and effective whenever intra-community edges are
|
||||
/// heavier than inter ones. With near-uniform weights it degrades gracefully
|
||||
/// to [`CondenseMethod::ConnectedComponents`].
|
||||
WeakBoundary {
|
||||
/// Fraction of the mean edge weight below which an edge is treated as a
|
||||
/// boundary and removed. `0.5` is a sensible default.
|
||||
relative_threshold: f64,
|
||||
},
|
||||
/// Recursive min-cut community detection via
|
||||
/// [`ruvector_mincut::CommunityDetector`]. Structure-aware for graphs with
|
||||
/// clear bottlenecks, but recursive *global* min cut tends to peel off
|
||||
/// single low-degree vertices otherwise (many tiny regions); prefer
|
||||
/// [`CondenseMethod::WeakBoundary`]. `min_region_size` bounds recursion.
|
||||
MinCutCommunity {
|
||||
/// Recursion stops splitting regions at or below this size.
|
||||
min_region_size: usize,
|
||||
},
|
||||
/// Recursive bisection into up to `num_regions` regions via
|
||||
/// [`ruvector_mincut::GraphPartitioner`]. Effective on clustered graphs;
|
||||
/// reduction is graph-dependent (the bisection can peel single vertices,
|
||||
/// which become singleton regions). Prefer [`CondenseMethod::WeakBoundary`].
|
||||
Partition {
|
||||
/// Target number of regions.
|
||||
num_regions: usize,
|
||||
},
|
||||
/// **Differentiable min-cut** (relaxed normalized cut, MinCutPool-style):
|
||||
/// learns a soft `N×K` assignment by gradient descent on a cut +
|
||||
/// orthogonality loss, then hardens it (argmax) into regions. The only
|
||||
/// method whose regions are *trained* to preserve the cut — see
|
||||
/// [`crate::diffcut`]. `K` upper-bounds the super-node count.
|
||||
DiffMinCut(DiffCutConfig),
|
||||
/// Cheap baseline: one region per connected component.
|
||||
ConnectedComponents,
|
||||
}
|
||||
|
||||
impl Default for CondenseMethod {
|
||||
fn default() -> Self {
|
||||
CondenseMethod::WeakBoundary {
|
||||
relative_threshold: 0.5,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Configuration for [`GraphCondenser`].
|
||||
///
|
||||
/// `Default` yields [`CondenseMethod::WeakBoundary`] with a `0.5` threshold and
|
||||
/// no centroid normalisation.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct CondenseConfig {
|
||||
/// Region partitioning strategy.
|
||||
pub method: CondenseMethod,
|
||||
/// L2-normalise centroids after averaging (useful for cosine-space
|
||||
/// embeddings such as HNSW vectors).
|
||||
pub normalize_centroids: bool,
|
||||
}
|
||||
|
||||
/// Stateless condenser. Construct once with a [`CondenseConfig`] and reuse
|
||||
/// across graphs.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct GraphCondenser {
|
||||
config: CondenseConfig,
|
||||
}
|
||||
|
||||
impl GraphCondenser {
|
||||
/// Create a condenser with the given configuration.
|
||||
pub fn new(config: CondenseConfig) -> Self {
|
||||
Self { config }
|
||||
}
|
||||
|
||||
/// Borrow the active configuration.
|
||||
pub fn config(&self) -> &CondenseConfig {
|
||||
&self.config
|
||||
}
|
||||
|
||||
/// Condense `graph` using the per-vertex `features`.
|
||||
///
|
||||
/// Every vertex in `graph` must have an embedding in `features` (a vertex
|
||||
/// with no incident edges is still condensed, as a singleton region).
|
||||
///
|
||||
/// # Errors
|
||||
/// - [`CondenseError::EmptyGraph`] if the graph has no vertices.
|
||||
/// - [`CondenseError::MissingFeature`] if a vertex lacks an embedding.
|
||||
/// - [`CondenseError::InvalidConfig`] for a degenerate configuration.
|
||||
pub fn condense(
|
||||
&self,
|
||||
graph: &DynamicGraph,
|
||||
features: &NodeFeatures,
|
||||
) -> Result<CondensedGraph> {
|
||||
let vertices = graph.vertices();
|
||||
if vertices.is_empty() {
|
||||
return Err(CondenseError::EmptyGraph);
|
||||
}
|
||||
let dim = features.dim();
|
||||
let num_classes = features.num_classes();
|
||||
|
||||
// 1. Partition into structural regions, then guarantee full coverage
|
||||
// and a deterministic ordering (region id == position).
|
||||
let mut regions = self.partition_regions(graph)?;
|
||||
ensure_coverage(&mut regions, &vertices);
|
||||
for r in &mut regions {
|
||||
r.sort_unstable();
|
||||
}
|
||||
regions.retain(|r| !r.is_empty());
|
||||
regions.sort_by(|a, b| a[0].cmp(&b[0]));
|
||||
|
||||
// 2. Vertex -> region index.
|
||||
let mut region_of: HashMap<VertexId, usize> = HashMap::with_capacity(vertices.len());
|
||||
for (ri, members) in regions.iter().enumerate() {
|
||||
for &v in members {
|
||||
region_of.insert(v, ri);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Single edge pass: internal vs boundary weight (for coherence) and
|
||||
// super-edge accumulation.
|
||||
let n = regions.len();
|
||||
let mut internal_w = vec![0f64; n];
|
||||
let mut boundary_w = vec![0f64; n];
|
||||
let mut super_edges: HashMap<(usize, usize), (f64, u32)> = HashMap::new();
|
||||
|
||||
for e in graph.edges() {
|
||||
// region_of is total over graph vertices after ensure_coverage.
|
||||
let rs = region_of[&e.source];
|
||||
let rt = region_of[&e.target];
|
||||
if rs == rt {
|
||||
internal_w[rs] += e.weight;
|
||||
} else {
|
||||
boundary_w[rs] += e.weight;
|
||||
boundary_w[rt] += e.weight;
|
||||
let key = if rs < rt { (rs, rt) } else { (rt, rs) };
|
||||
let slot = super_edges.entry(key).or_insert((0.0, 0));
|
||||
slot.0 += e.weight;
|
||||
slot.1 += 1;
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Build super-nodes.
|
||||
let mut nodes = Vec::with_capacity(n);
|
||||
for (ri, members) in regions.iter().enumerate() {
|
||||
let (mut centroid, representative) = centroid_and_medoid(members, features, dim)?;
|
||||
if self.config.normalize_centroids {
|
||||
l2_normalize(&mut centroid);
|
||||
}
|
||||
let class_distribution = class_distribution(members, features, num_classes);
|
||||
let iw = internal_w[ri];
|
||||
let bw = boundary_w[ri];
|
||||
let coherence = if iw + bw <= 0.0 {
|
||||
1.0
|
||||
} else {
|
||||
(iw / (iw + bw)) as f32
|
||||
};
|
||||
nodes.push(CondensedNode {
|
||||
id: ri as u64,
|
||||
centroid,
|
||||
weight: members.len() as u32,
|
||||
class_distribution,
|
||||
coherence,
|
||||
representative,
|
||||
members: members.clone(),
|
||||
});
|
||||
}
|
||||
|
||||
// 5. Build super-edges (region index == id), canonical & sorted.
|
||||
let mut edges: Vec<CondensedEdge> = super_edges
|
||||
.into_iter()
|
||||
.map(|((s, t), (w, c))| CondensedEdge {
|
||||
source: s as u64,
|
||||
target: t as u64,
|
||||
weight: w,
|
||||
crossings: c,
|
||||
})
|
||||
.collect();
|
||||
edges.sort_by_key(|e| (e.source, e.target));
|
||||
|
||||
Ok(CondensedGraph {
|
||||
nodes,
|
||||
edges,
|
||||
source_nodes: vertices.len(),
|
||||
source_edges: graph.num_edges(),
|
||||
dim,
|
||||
num_classes,
|
||||
})
|
||||
}
|
||||
|
||||
fn partition_regions(&self, graph: &DynamicGraph) -> Result<Vec<Vec<VertexId>>> {
|
||||
match &self.config.method {
|
||||
CondenseMethod::ConnectedComponents => Ok(graph.connected_components()),
|
||||
CondenseMethod::WeakBoundary { relative_threshold } => {
|
||||
Ok(weak_boundary_regions(graph, *relative_threshold))
|
||||
}
|
||||
CondenseMethod::MinCutCommunity { min_region_size } => {
|
||||
let arc = Arc::new(graph.clone());
|
||||
let mut detector = CommunityDetector::new(arc);
|
||||
Ok(detector.detect(*min_region_size).to_vec())
|
||||
}
|
||||
CondenseMethod::Partition { num_regions } => {
|
||||
if *num_regions == 0 {
|
||||
return Err(CondenseError::InvalidConfig(
|
||||
"num_regions must be > 0".to_string(),
|
||||
));
|
||||
}
|
||||
let arc = Arc::new(graph.clone());
|
||||
let partitioner = GraphPartitioner::new(arc, *num_regions);
|
||||
Ok(partitioner.partition())
|
||||
}
|
||||
CondenseMethod::DiffMinCut(cfg) => {
|
||||
let result = DiffCutCondenser::new(cfg.clone()).train(graph)?;
|
||||
Ok(result.hard_regions())
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Convenience wrapper: condense with default ([`CondenseMethod::WeakBoundary`])
|
||||
/// settings.
|
||||
pub fn condense(graph: &DynamicGraph, features: &NodeFeatures) -> Result<CondensedGraph> {
|
||||
GraphCondenser::default().condense(graph, features)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn two_triangles() -> (DynamicGraph, NodeFeatures) {
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 1.0),
|
||||
(1, 2, 1.0),
|
||||
(2, 0, 1.0),
|
||||
(3, 4, 1.0),
|
||||
(4, 5, 1.0),
|
||||
(5, 3, 1.0),
|
||||
(2, 3, 0.05),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
let mut f = NodeFeatures::new(2, 2);
|
||||
// Cluster A near (0,0) labelled 0, cluster B near (10,10) labelled 1.
|
||||
for v in 0..3u64 {
|
||||
f.set(v, vec![v as f32 * 0.01, 0.0], 0).unwrap();
|
||||
}
|
||||
for v in 3..6u64 {
|
||||
f.set(v, vec![10.0 + v as f32 * 0.01, 10.0], 1).unwrap();
|
||||
}
|
||||
(g, f)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_graph_errors() {
|
||||
let g = DynamicGraph::new();
|
||||
let f = NodeFeatures::new(2, 0);
|
||||
assert!(matches!(
|
||||
condense(&g, &f).unwrap_err(),
|
||||
CondenseError::EmptyGraph
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_feature_errors() {
|
||||
let g = DynamicGraph::new();
|
||||
g.insert_edge(0, 1, 1.0).unwrap();
|
||||
let mut f = NodeFeatures::new(2, 0);
|
||||
f.set_embedding(0, vec![0.0, 0.0]).unwrap();
|
||||
// vertex 1 has no feature
|
||||
assert!(matches!(
|
||||
condense(&g, &f).unwrap_err(),
|
||||
CondenseError::MissingFeature(1)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn condenses_two_communities() {
|
||||
let (g, f) = two_triangles();
|
||||
let c = condense(&g, &f).unwrap();
|
||||
// Should collapse 6 nodes into 2 communities.
|
||||
assert_eq!(c.source_nodes, 6);
|
||||
assert_eq!(c.node_count(), 2);
|
||||
assert_eq!(c.total_weight(), 6);
|
||||
// Exactly one super-edge across the bridge.
|
||||
assert_eq!(c.edge_count(), 1);
|
||||
let e = c.edges[0];
|
||||
assert_eq!((e.source, e.target), (0, 1));
|
||||
assert_eq!(e.crossings, 1);
|
||||
assert!((e.weight - 0.05).abs() < 1e-9);
|
||||
// Region ids are deterministic & sorted; first region holds {0,1,2}.
|
||||
assert_eq!(c.nodes[0].members, vec![0, 1, 2]);
|
||||
assert_eq!(c.nodes[1].members, vec![3, 4, 5]);
|
||||
// Pure, well-formed class distributions.
|
||||
assert_eq!(c.nodes[0].dominant_class(), Some(0));
|
||||
assert_eq!(c.nodes[1].dominant_class(), Some(1));
|
||||
assert!(c.nodes[0].purity() > 0.99);
|
||||
// High internal cohesion (3 internal edges vs 0.05 boundary).
|
||||
assert!(c.nodes[0].coherence > 0.9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mincut_community_recovers_clear_bottleneck() {
|
||||
// Dense triangles (weight 5) joined by a single light bridge (weight 1).
|
||||
// This is the regime where recursive min-cut community detection works:
|
||||
// a sharp bottleneck and no low-degree vertices to peel.
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 5.0),
|
||||
(1, 2, 5.0),
|
||||
(2, 0, 5.0),
|
||||
(3, 4, 5.0),
|
||||
(4, 5, 5.0),
|
||||
(5, 3, 5.0),
|
||||
(2, 3, 1.0),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
for v in 0..6u64 {
|
||||
f.set_embedding(v, vec![v as f32]).unwrap();
|
||||
}
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::MinCutCommunity { min_region_size: 2 },
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
// The min-cut engine reduces and fully covers the graph. It may split
|
||||
// more finely than the planted 2 communities (recursive global min cut
|
||||
// is aggressive); we assert reduction + coverage, not exact recovery —
|
||||
// exact community recovery is the default WeakBoundary method's job.
|
||||
assert_eq!(c.total_weight(), 6); // full coverage
|
||||
assert!(c.node_count() >= 2 && c.node_count() < 6); // runs + reduces
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diff_mincut_condenses_via_trained_assignment() {
|
||||
use crate::diffcut::DiffCutConfig;
|
||||
let (g, f) = two_triangles();
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::DiffMinCut(DiffCutConfig {
|
||||
num_clusters: 2,
|
||||
..Default::default()
|
||||
}),
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
assert_eq!(c.node_count(), 2);
|
||||
assert_eq!(c.total_weight(), 6);
|
||||
assert_eq!(c.nodes[0].members, vec![0, 1, 2]);
|
||||
assert_eq!(c.nodes[1].members, vec![3, 4, 5]);
|
||||
assert_eq!(c.edge_count(), 1); // the bridge -> one super-edge
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn weak_boundary_falls_back_to_components_without_contrast() {
|
||||
// Uniform weights -> no edge is below 0.5*mean -> nothing cut ->
|
||||
// regions equal connected components (here, one).
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v) in &[(0, 1), (1, 2), (2, 0)] {
|
||||
g.insert_edge(u, v, 1.0).unwrap();
|
||||
}
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
for v in 0..3u64 {
|
||||
f.set_embedding(v, vec![v as f32]).unwrap();
|
||||
}
|
||||
let c = condense(&g, &f).unwrap(); // default WeakBoundary
|
||||
assert_eq!(c.node_count(), 1);
|
||||
assert_eq!(c.total_weight(), 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partition_runs_and_covers() {
|
||||
// GraphPartitioner is best-effort; we assert it runs and covers every
|
||||
// vertex exactly once (reduction is graph-dependent, not guaranteed).
|
||||
let g = DynamicGraph::new();
|
||||
for i in 0..15u64 {
|
||||
g.insert_edge(i, i + 1, 1.0).unwrap();
|
||||
}
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
for v in 0..16u64 {
|
||||
f.set_embedding(v, vec![v as f32]).unwrap();
|
||||
}
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::Partition { num_regions: 4 },
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
assert_eq!(c.total_weight(), 16); // full, non-overlapping coverage
|
||||
assert!(c.node_count() <= 16 && c.node_count() >= 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partition_zero_regions_errors() {
|
||||
let g = DynamicGraph::new();
|
||||
g.insert_edge(0, 1, 1.0).unwrap();
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
f.set_embedding(0, vec![0.0]).unwrap();
|
||||
f.set_embedding(1, vec![1.0]).unwrap();
|
||||
let err = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::Partition { num_regions: 0 },
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, CondenseError::InvalidConfig(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn coverage_includes_isolated_vertex() {
|
||||
let g = DynamicGraph::new();
|
||||
g.insert_edge(0, 1, 1.0).unwrap();
|
||||
g.add_vertex(99); // isolated
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
for v in [0u64, 1, 99] {
|
||||
f.set_embedding(v, vec![v as f32]).unwrap();
|
||||
}
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::ConnectedComponents,
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
// {0,1} component + {99} singleton = 2 regions covering all vertices.
|
||||
assert_eq!(c.total_weight(), 3);
|
||||
assert!(c.nodes.iter().any(|n| n.members == vec![99]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn centroid_is_member_mean_and_medoid_valid() {
|
||||
let g = DynamicGraph::new();
|
||||
g.insert_edge(0, 1, 1.0).unwrap();
|
||||
g.insert_edge(1, 2, 1.0).unwrap();
|
||||
g.insert_edge(2, 0, 1.0).unwrap();
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
f.set_embedding(0, vec![0.0]).unwrap();
|
||||
f.set_embedding(1, vec![2.0]).unwrap();
|
||||
f.set_embedding(2, vec![4.0]).unwrap();
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::ConnectedComponents,
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
assert_eq!(c.node_count(), 1);
|
||||
assert!((c.nodes[0].centroid[0] - 2.0).abs() < 1e-6);
|
||||
// Medoid is the member nearest the mean (2.0) -> vertex 1.
|
||||
assert_eq!(c.nodes[0].representative, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_centroids_unit_length() {
|
||||
let g = DynamicGraph::new();
|
||||
g.insert_edge(0, 1, 1.0).unwrap();
|
||||
let mut f = NodeFeatures::new(2, 0);
|
||||
f.set_embedding(0, vec![3.0, 0.0]).unwrap();
|
||||
f.set_embedding(1, vec![3.0, 0.0]).unwrap();
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::ConnectedComponents,
|
||||
normalize_centroids: true,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
let norm: f32 = c.nodes[0]
|
||||
.centroid
|
||||
.iter()
|
||||
.map(|x| x * x)
|
||||
.sum::<f32>()
|
||||
.sqrt();
|
||||
assert!((norm - 1.0).abs() < 1e-6);
|
||||
}
|
||||
}
|
||||
477
crates/ruvector-graph-condense/src/cutloss.rs
Normal file
477
crates/ruvector-graph-condense/src/cutloss.rs
Normal file
|
|
@ -0,0 +1,477 @@
|
|||
//! Differentiable relaxed-min-cut loss core (all maths in `f64`).
|
||||
//!
|
||||
//! Pure functions shared by [`crate::diffcut`]: the compact graph view, the
|
||||
//! row-softmax, the loss (`L_cut + λ·L_ortho`) and its analytic gradients. Kept
|
||||
//! separate from the optimiser/orchestration so each file stays small and the
|
||||
//! gradient-checked maths is isolated.
|
||||
|
||||
#[cfg(feature = "parallel")]
|
||||
use rayon::prelude::*;
|
||||
use ruvector_mincut::{DynamicGraph, VertexId};
|
||||
use std::collections::HashMap;
|
||||
|
||||
pub(crate) const EPS: f64 = 1e-12;
|
||||
|
||||
/// The three components of the loss at a point.
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
pub struct MinCutLoss {
|
||||
/// Relaxed normalized-cut term in `[-1, 0]` (lower is better).
|
||||
pub cut: f64,
|
||||
/// Orthogonality / balance term in `[0, 2]` (lower is better).
|
||||
pub ortho: f64,
|
||||
/// `cut + λ·ortho`.
|
||||
pub total: f64,
|
||||
}
|
||||
|
||||
/// Contiguous, index-mapped view of a graph for the loss maths.
|
||||
///
|
||||
/// Carries both an edge list (for minibatch scatter) and a CSR adjacency (for
|
||||
/// conflict-free, row-parallel `A·S`).
|
||||
pub(crate) struct CompactGraph {
|
||||
pub(crate) n: usize,
|
||||
pub(crate) degree: Vec<f64>,
|
||||
pub(crate) edges: Vec<(usize, usize, f64)>,
|
||||
pub(crate) vertices: Vec<VertexId>,
|
||||
/// CSR row offsets, length `n + 1`.
|
||||
nbr_off: Vec<usize>,
|
||||
/// CSR neighbours `(col, weight)`, length `2 * num_edges`.
|
||||
nbr: Vec<(usize, f64)>,
|
||||
}
|
||||
|
||||
impl CompactGraph {
|
||||
pub(crate) fn from_graph(graph: &DynamicGraph) -> Self {
|
||||
let mut vertices = graph.vertices();
|
||||
vertices.sort_unstable(); // deterministic row order
|
||||
let mut index: HashMap<VertexId, usize> = HashMap::with_capacity(vertices.len());
|
||||
for (i, &v) in vertices.iter().enumerate() {
|
||||
index.insert(v, i);
|
||||
}
|
||||
let n = vertices.len();
|
||||
let mut degree = vec![0f64; n];
|
||||
let mut edges = Vec::with_capacity(graph.num_edges());
|
||||
let mut deg_count = vec![0usize; n];
|
||||
for e in graph.edges() {
|
||||
let i = index[&e.source];
|
||||
let j = index[&e.target];
|
||||
edges.push((i, j, e.weight));
|
||||
degree[i] += e.weight;
|
||||
degree[j] += e.weight;
|
||||
deg_count[i] += 1;
|
||||
deg_count[j] += 1;
|
||||
}
|
||||
// Build CSR (both directions) from the edge list.
|
||||
let mut nbr_off = vec![0usize; n + 1];
|
||||
for i in 0..n {
|
||||
nbr_off[i + 1] = nbr_off[i] + deg_count[i];
|
||||
}
|
||||
let mut cursor = nbr_off[..n].to_vec();
|
||||
let mut nbr = vec![(0usize, 0f64); edges.len() * 2];
|
||||
for &(i, j, w) in &edges {
|
||||
nbr[cursor[i]] = (j, w);
|
||||
cursor[i] += 1;
|
||||
nbr[cursor[j]] = (i, w);
|
||||
cursor[j] += 1;
|
||||
}
|
||||
Self {
|
||||
n,
|
||||
degree,
|
||||
edges,
|
||||
vertices,
|
||||
nbr_off,
|
||||
nbr,
|
||||
}
|
||||
}
|
||||
|
||||
/// Vertex-id → row-index map (rows are sorted-ascending vertices).
|
||||
pub(crate) fn index_map(&self) -> HashMap<VertexId, usize> {
|
||||
self.vertices
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, &v)| (v, i))
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn softmax_rows(logits: &[f64], n: usize, k: usize) -> Vec<f64> {
|
||||
let mut s = vec![0f64; n * k];
|
||||
for i in 0..n {
|
||||
let row = &logits[i * k..(i + 1) * k];
|
||||
let max = row.iter().copied().fold(f64::NEG_INFINITY, f64::max);
|
||||
let mut sum = 0f64;
|
||||
for c in 0..k {
|
||||
let e = (row[c] - max).exp();
|
||||
s[i * k + c] = e;
|
||||
sum += e;
|
||||
}
|
||||
let inv = 1.0 / sum;
|
||||
for c in 0..k {
|
||||
s[i * k + c] *= inv;
|
||||
}
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
/// `A · S` (`N×K`) via CSR — each output row depends only on its node's
|
||||
/// neighbours, so it is conflict-free and row-parallel. Deterministic
|
||||
/// regardless of thread count (fixed row + neighbour order).
|
||||
pub(crate) fn as_matrix(
|
||||
g: &CompactGraph,
|
||||
s: &[f64],
|
||||
n: usize,
|
||||
k: usize,
|
||||
parallel: bool,
|
||||
) -> Vec<f64> {
|
||||
let mut as_mat = vec![0f64; n * k];
|
||||
let row_fn = |i: usize, row: &mut [f64]| {
|
||||
for idx in g.nbr_off[i]..g.nbr_off[i + 1] {
|
||||
let (j, w) = g.nbr[idx];
|
||||
let sj = &s[j * k..(j + 1) * k];
|
||||
for c in 0..k {
|
||||
row[c] += w * sj[c];
|
||||
}
|
||||
}
|
||||
};
|
||||
#[cfg(feature = "parallel")]
|
||||
if parallel {
|
||||
as_mat
|
||||
.par_chunks_mut(k)
|
||||
.enumerate()
|
||||
.for_each(|(i, row)| row_fn(i, row));
|
||||
return as_mat;
|
||||
}
|
||||
let _ = parallel;
|
||||
as_mat
|
||||
.chunks_mut(k)
|
||||
.enumerate()
|
||||
.for_each(|(i, row)| row_fn(i, row));
|
||||
as_mat
|
||||
}
|
||||
|
||||
/// Stochastic `A · S` estimate from a sampled subset of edges, scaled by
|
||||
/// `|E| / |sample|`. O(|sample|·K) per call — the lever for million-edge graphs.
|
||||
pub(crate) fn as_matrix_minibatch(
|
||||
g: &CompactGraph,
|
||||
s: &[f64],
|
||||
n: usize,
|
||||
k: usize,
|
||||
sample: &[usize],
|
||||
) -> Vec<f64> {
|
||||
let mut as_mat = vec![0f64; n * k];
|
||||
if sample.is_empty() {
|
||||
return as_mat;
|
||||
}
|
||||
let scale = g.edges.len() as f64 / sample.len() as f64;
|
||||
for &e in sample {
|
||||
let (i, j, w) = g.edges[e];
|
||||
let ws = w * scale;
|
||||
for c in 0..k {
|
||||
as_mat[i * k + c] += ws * s[j * k + c];
|
||||
as_mat[j * k + c] += ws * s[i * k + c];
|
||||
}
|
||||
}
|
||||
as_mat
|
||||
}
|
||||
|
||||
/// Forward-only loss (full-batch, sequential).
|
||||
pub(crate) fn forward(g: &CompactGraph, s: &[f64], k: usize, lambda: f64) -> MinCutLoss {
|
||||
let as_mat = as_matrix(g, s, g.n, k, false);
|
||||
let (cut, _, ortho, _) = cut_and_ortho(g, s, &as_mat, k, false, false);
|
||||
MinCutLoss {
|
||||
cut,
|
||||
ortho,
|
||||
total: cut + lambda * ortho,
|
||||
}
|
||||
}
|
||||
|
||||
/// Loss and gradient w.r.t. `S` (full-batch, sequential) — convenience used by
|
||||
/// the gradient-check test.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn loss_and_grad_wrt_soft(
|
||||
g: &CompactGraph,
|
||||
s: &[f64],
|
||||
k: usize,
|
||||
lambda: f64,
|
||||
) -> (MinCutLoss, Vec<f64>) {
|
||||
let as_mat = as_matrix(g, s, g.n, k, false);
|
||||
loss_and_grad_with_as(g, s, &as_mat, k, lambda, false)
|
||||
}
|
||||
|
||||
/// Loss and gradient given a precomputed `A·S`. `parallel` parallelises the
|
||||
/// heavy `O(N·K²)` loops (SᵀS build, ortho gradient) deterministically.
|
||||
pub(crate) fn loss_and_grad_with_as(
|
||||
g: &CompactGraph,
|
||||
s: &[f64],
|
||||
as_mat: &[f64],
|
||||
k: usize,
|
||||
lambda: f64,
|
||||
parallel: bool,
|
||||
) -> (MinCutLoss, Vec<f64>) {
|
||||
let (cut, grad_cut, ortho, grad_ortho) = cut_and_ortho(g, s, as_mat, k, true, parallel);
|
||||
let n = g.n;
|
||||
let mut grad = grad_cut;
|
||||
for idx in 0..n * k {
|
||||
grad[idx] += lambda * grad_ortho[idx];
|
||||
}
|
||||
(
|
||||
MinCutLoss {
|
||||
cut,
|
||||
ortho,
|
||||
total: cut + lambda * ortho,
|
||||
},
|
||||
grad,
|
||||
)
|
||||
}
|
||||
|
||||
/// Rows per Rayon task — coarse enough to amortise dispatch overhead.
|
||||
fn rows_per_task(n: usize) -> usize {
|
||||
#[cfg(feature = "parallel")]
|
||||
let threads = rayon::current_num_threads();
|
||||
#[cfg(not(feature = "parallel"))]
|
||||
let threads = 1usize;
|
||||
(n / (threads * 4)).max(1)
|
||||
}
|
||||
|
||||
/// `P = SᵀS` (`K×K`). Both paths use the *same* chunked partial-sum ordering
|
||||
/// (parallel only changes who computes each chunk), so parallel is bit-identical
|
||||
/// to sequential — no float-reordering surprises.
|
||||
fn gram(s: &[f64], n: usize, k: usize, parallel: bool) -> Vec<f64> {
|
||||
let chunk = rows_per_task(n) * k;
|
||||
let acc_block = |block: &[f64]| -> Vec<f64> {
|
||||
let mut local = vec![0f64; k * k];
|
||||
for row in block.chunks(k) {
|
||||
for a in 0..k {
|
||||
let sa = row[a];
|
||||
if sa != 0.0 {
|
||||
for b in 0..k {
|
||||
local[a * k + b] += sa * row[b];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
local
|
||||
};
|
||||
#[cfg(feature = "parallel")]
|
||||
let partials: Vec<Vec<f64>> = if parallel {
|
||||
s.par_chunks(chunk).map(acc_block).collect()
|
||||
} else {
|
||||
s.chunks(chunk).map(acc_block).collect()
|
||||
};
|
||||
#[cfg(not(feature = "parallel"))]
|
||||
let partials: Vec<Vec<f64>> = {
|
||||
let _ = parallel;
|
||||
s.chunks(chunk).map(acc_block).collect()
|
||||
};
|
||||
let mut p = vec![0f64; k * k];
|
||||
for part in partials {
|
||||
for i in 0..k * k {
|
||||
p[i] += part[i];
|
||||
}
|
||||
}
|
||||
p
|
||||
}
|
||||
|
||||
/// Shared core given a precomputed `A·S`: (cut, grad_cut, ortho, grad_ortho).
|
||||
/// The gradient vectors are empty when `want_grad` is false.
|
||||
fn cut_and_ortho(
|
||||
g: &CompactGraph,
|
||||
s: &[f64],
|
||||
as_mat: &[f64],
|
||||
k: usize,
|
||||
want_grad: bool,
|
||||
parallel: bool,
|
||||
) -> (f64, Vec<f64>, f64, Vec<f64>) {
|
||||
let n = g.n;
|
||||
|
||||
// numer = Tr(SᵀAS), denom = Tr(SᵀDS) (O(N·K), kept sequential).
|
||||
let mut numer = 0f64;
|
||||
for idx in 0..n * k {
|
||||
numer += s[idx] * as_mat[idx];
|
||||
}
|
||||
let mut denom = 0f64;
|
||||
for i in 0..n {
|
||||
let mut s2 = 0f64;
|
||||
for c in 0..k {
|
||||
let v = s[i * k + c];
|
||||
s2 += v * v;
|
||||
}
|
||||
denom += g.degree[i] * s2;
|
||||
}
|
||||
let cut = if denom > EPS { -numer / denom } else { 0.0 };
|
||||
|
||||
let mut grad_cut = Vec::new();
|
||||
if want_grad {
|
||||
grad_cut = vec![0f64; n * k];
|
||||
if denom > EPS {
|
||||
// ∂L_cut/∂S = -2/denom · (AS + L_cut·DS); rows are independent.
|
||||
let coef = -2.0 / denom;
|
||||
let row = |i: usize, gc: &mut [f64]| {
|
||||
let di = g.degree[i];
|
||||
for c in 0..k {
|
||||
gc[c] = coef * (as_mat[i * k + c] + cut * di * s[i * k + c]);
|
||||
}
|
||||
};
|
||||
#[cfg(feature = "parallel")]
|
||||
if parallel {
|
||||
grad_cut
|
||||
.par_chunks_mut(k)
|
||||
.enumerate()
|
||||
.for_each(|(i, gc)| row(i, gc));
|
||||
} else {
|
||||
grad_cut
|
||||
.chunks_mut(k)
|
||||
.enumerate()
|
||||
.for_each(|(i, gc)| row(i, gc));
|
||||
}
|
||||
#[cfg(not(feature = "parallel"))]
|
||||
grad_cut
|
||||
.chunks_mut(k)
|
||||
.enumerate()
|
||||
.for_each(|(i, gc)| row(i, gc));
|
||||
}
|
||||
}
|
||||
|
||||
let p = gram(s, n, k, parallel);
|
||||
let np = p.iter().map(|x| x * x).sum::<f64>().sqrt();
|
||||
let inv_sqrt_k = 1.0 / (k as f64).sqrt();
|
||||
|
||||
let mut ortho = 0f64;
|
||||
let mut q = vec![0f64; k * k];
|
||||
if np > EPS {
|
||||
let mut sq = 0f64;
|
||||
for a in 0..k {
|
||||
for b in 0..k {
|
||||
let target = if a == b { inv_sqrt_k } else { 0.0 };
|
||||
let qv = p[a * k + b] / np - target;
|
||||
q[a * k + b] = qv;
|
||||
sq += qv * qv;
|
||||
}
|
||||
}
|
||||
ortho = sq.sqrt();
|
||||
}
|
||||
|
||||
let mut grad_ortho = Vec::new();
|
||||
if want_grad {
|
||||
grad_ortho = vec![0f64; n * k];
|
||||
if np > EPS && ortho > EPS {
|
||||
// Gf = Q/ortho ; G_P = Gf/np − (⟨Gf,P⟩/np³)·P ; ∂L/∂S = 2·S·G_P
|
||||
let mut dot = 0f64;
|
||||
for idx in 0..k * k {
|
||||
dot += (q[idx] / ortho) * p[idx];
|
||||
}
|
||||
let np3 = np * np * np;
|
||||
let mut gp = vec![0f64; k * k];
|
||||
for idx in 0..k * k {
|
||||
gp[idx] = (q[idx] / ortho) / np - (dot / np3) * p[idx];
|
||||
}
|
||||
// ∂L/∂S row i = 2 · S[i] · G_P; rows independent.
|
||||
let row = |s_row: &[f64], go: &mut [f64]| {
|
||||
for kk in 0..k {
|
||||
let mut acc = 0f64;
|
||||
for b in 0..k {
|
||||
acc += s_row[b] * gp[b * k + kk];
|
||||
}
|
||||
go[kk] = 2.0 * acc;
|
||||
}
|
||||
};
|
||||
#[cfg(feature = "parallel")]
|
||||
if parallel {
|
||||
grad_ortho
|
||||
.par_chunks_mut(k)
|
||||
.zip(s.par_chunks(k))
|
||||
.for_each(|(go, s_row)| row(s_row, go));
|
||||
} else {
|
||||
grad_ortho
|
||||
.chunks_mut(k)
|
||||
.zip(s.chunks(k))
|
||||
.for_each(|(go, s_row)| row(s_row, go));
|
||||
}
|
||||
#[cfg(not(feature = "parallel"))]
|
||||
grad_ortho
|
||||
.chunks_mut(k)
|
||||
.zip(s.chunks(k))
|
||||
.for_each(|(go, s_row)| row(s_row, go));
|
||||
}
|
||||
}
|
||||
|
||||
(cut, grad_cut, ortho, grad_ortho)
|
||||
}
|
||||
|
||||
/// Backprop a gradient w.r.t. `S` through the row-softmax to the logits `Θ`.
|
||||
pub(crate) fn softmax_backprop(s: &[f64], grad_s: &[f64], n: usize, k: usize) -> Vec<f64> {
|
||||
let mut grad = vec![0f64; n * k];
|
||||
for i in 0..n {
|
||||
let mut dot = 0f64;
|
||||
for c in 0..k {
|
||||
dot += grad_s[i * k + c] * s[i * k + c];
|
||||
}
|
||||
for c in 0..k {
|
||||
grad[i * k + c] = s[i * k + c] * (grad_s[i * k + c] - dot);
|
||||
}
|
||||
}
|
||||
grad
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn barbell() -> DynamicGraph {
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 1.0),
|
||||
(1, 2, 1.0),
|
||||
(2, 0, 1.0),
|
||||
(3, 4, 1.0),
|
||||
(4, 5, 1.0),
|
||||
(5, 3, 1.0),
|
||||
(2, 3, 0.05),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
g
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gradient_matches_finite_differences() {
|
||||
// Decisive correctness test: analytic ∂L/∂Θ vs finite differences across
|
||||
// several K (proves the K-general gradient formulas, not just K=2).
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
let g = CompactGraph::from_graph(&barbell());
|
||||
let n = g.n;
|
||||
let lambda = 1.0;
|
||||
let h = 1e-6;
|
||||
for k in [2usize, 3, 4] {
|
||||
let mut rng = StdRng::seed_from_u64(99 + k as u64);
|
||||
let mut theta = vec![0f64; n * k];
|
||||
for t in &mut theta {
|
||||
*t = rng.gen_range(-0.5..0.5);
|
||||
}
|
||||
let s = softmax_rows(&theta, n, k);
|
||||
let (_, grad_s) = loss_and_grad_wrt_soft(&g, &s, k, lambda);
|
||||
let analytic = softmax_backprop(&s, &grad_s, n, k);
|
||||
let mut max_abs_err = 0f64;
|
||||
for idx in 0..n * k {
|
||||
let mut tp = theta.clone();
|
||||
tp[idx] += h;
|
||||
let lp = forward(&g, &softmax_rows(&tp, n, k), k, lambda).total;
|
||||
let mut tm = theta.clone();
|
||||
tm[idx] -= h;
|
||||
let lm = forward(&g, &softmax_rows(&tm, n, k), k, lambda).total;
|
||||
let num = (lp - lm) / (2.0 * h);
|
||||
max_abs_err = max_abs_err.max((num - analytic[idx]).abs());
|
||||
}
|
||||
assert!(max_abs_err < 1e-5, "k={k}: grad mismatch {max_abs_err}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn uniform_assignment_fools_cut_but_not_ortho() {
|
||||
let g = CompactGraph::from_graph(&barbell());
|
||||
let soft = vec![0.5f64; g.n * 2];
|
||||
let l = forward(&g, &soft, 2, 1.0);
|
||||
// numer==denom -> cut "fooled" to -1; ortho catches the collapse.
|
||||
assert!((l.cut + 1.0).abs() < 1e-9, "cut {}", l.cut);
|
||||
assert!(l.ortho > 0.5, "ortho {}", l.ortho);
|
||||
}
|
||||
}
|
||||
449
crates/ruvector-graph-condense/src/diffcut.rs
Normal file
449
crates/ruvector-graph-condense/src/diffcut.rs
Normal file
|
|
@ -0,0 +1,449 @@
|
|||
//! Trainable differentiable min-cut condenser — the relaxed normalized-cut
|
||||
//! objective (MinCutPool-style; loss + analytic gradients live in
|
||||
//! [`crate::cutloss`]) optimised into a cluster assignment.
|
||||
//!
|
||||
//! The 2024–2026 surveys flag a differentiable min-cut term in the condensation
|
||||
//! loss as unpublished. This module makes that objective practical **on large-K
|
||||
//! problems** with three standard-but-essential ingredients:
|
||||
//!
|
||||
//! - **Adam** (default) instead of plain GD — adaptive, robust on the
|
||||
//! ill-conditioned, non-convex cut objective.
|
||||
//! - **Warm-start init** (default) — seed the logits from the cheap
|
||||
//! [`crate::CondenseMethod::WeakBoundary`] structural prior and *refine* with
|
||||
//! the differentiable objective, rather than descending from random noise.
|
||||
//! This is the same coreset/K-Center idea GCond/SFGC use, and it is what makes
|
||||
//! K ≫ 2 converge.
|
||||
//! - **Restarts** — keep the lowest-loss run.
|
||||
//!
|
||||
//! Hardening the trained assignment (argmax) yields the regions consumed by
|
||||
//! [`crate::condense`] via [`crate::CondenseMethod::DiffMinCut`].
|
||||
|
||||
use crate::cutloss::{
|
||||
as_matrix, as_matrix_minibatch, forward, loss_and_grad_with_as, softmax_backprop, softmax_rows,
|
||||
CompactGraph,
|
||||
};
|
||||
use crate::error::{CondenseError, Result};
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
use ruvector_mincut::{DynamicGraph, VertexId};
|
||||
use std::collections::HashMap;
|
||||
|
||||
pub use crate::cutloss::MinCutLoss;
|
||||
|
||||
/// First-order optimiser used to minimise the loss.
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
pub enum Optimizer {
|
||||
/// (Heavy-ball) stochastic gradient descent. `momentum = 0` is plain GD.
|
||||
Sgd {
|
||||
/// Momentum coefficient in `[0, 1)`.
|
||||
momentum: f64,
|
||||
},
|
||||
/// Adam — adaptive moments; far more robust for large `K`.
|
||||
Adam {
|
||||
/// First-moment decay (typ. 0.9).
|
||||
beta1: f64,
|
||||
/// Second-moment decay (typ. 0.999).
|
||||
beta2: f64,
|
||||
/// Numerical-stability epsilon (typ. 1e-8).
|
||||
epsilon: f64,
|
||||
},
|
||||
}
|
||||
|
||||
impl Default for Optimizer {
|
||||
fn default() -> Self {
|
||||
Optimizer::Adam {
|
||||
beta1: 0.9,
|
||||
beta2: 0.999,
|
||||
epsilon: 1e-8,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// How the cluster logits are initialised before optimisation.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum InitStrategy {
|
||||
/// Unit-scale random logits.
|
||||
Random,
|
||||
/// **Default.** Seed from the [`crate::CondenseMethod::WeakBoundary`]
|
||||
/// structural prior, then refine — the key to large-K convergence.
|
||||
#[default]
|
||||
WarmStart,
|
||||
}
|
||||
|
||||
/// Configuration for the differentiable min-cut condenser. `Default` is a
|
||||
/// large-K-ready setup: Adam + warm-start.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct DiffCutConfig {
|
||||
/// Number of clusters `K` (upper bound on condensed super-nodes).
|
||||
pub num_clusters: usize,
|
||||
/// Weight `λ` on the orthogonality (anti-collapse) term.
|
||||
pub ortho_weight: f64,
|
||||
/// Optimiser step size (Adam likes ~0.05; SGD ~0.3).
|
||||
pub learning_rate: f64,
|
||||
/// Number of optimisation iterations per restart.
|
||||
pub iterations: usize,
|
||||
/// Optimiser.
|
||||
pub optimizer: Optimizer,
|
||||
/// Logit initialisation strategy.
|
||||
pub init: InitStrategy,
|
||||
/// Number of independent restarts; the lowest-loss run wins (min 1).
|
||||
pub restarts: usize,
|
||||
/// Early-stop when the loss improves by less than this between iterations
|
||||
/// (`0.0` disables). Warm-start starts near the optimum, so this typically
|
||||
/// cuts most of `iterations`.
|
||||
pub tolerance: f64,
|
||||
/// Use Rayon to parallelise the per-iteration `A·S` and parameter update.
|
||||
/// Deterministic (row-parallel); pays off on large graphs, adds overhead on
|
||||
/// tiny ones, so it defaults to `false`.
|
||||
pub parallel: bool,
|
||||
/// If `Some(b)`, estimate the gradient from `b` randomly sampled edges per
|
||||
/// iteration (stochastic) instead of the full edge set — the lever for
|
||||
/// million-edge graphs. `None` = full batch (exact).
|
||||
pub minibatch_edges: Option<usize>,
|
||||
/// RNG seed (determinism).
|
||||
pub seed: u64,
|
||||
}
|
||||
|
||||
impl Default for DiffCutConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
num_clusters: 8,
|
||||
ortho_weight: 1.0,
|
||||
learning_rate: 0.05,
|
||||
iterations: 300,
|
||||
optimizer: Optimizer::default(),
|
||||
init: InitStrategy::default(),
|
||||
restarts: 1,
|
||||
tolerance: 1e-6,
|
||||
parallel: false,
|
||||
minibatch_edges: None,
|
||||
seed: 0x0D1F_FC07,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl DiffCutConfig {
|
||||
fn validate(&self) -> Result<()> {
|
||||
if self.num_clusters == 0 {
|
||||
return Err(CondenseError::InvalidConfig(
|
||||
"num_clusters must be > 0".to_string(),
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of training: the learned assignment plus provenance.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DiffCutResult {
|
||||
soft: Vec<f64>,
|
||||
vertices: Vec<VertexId>,
|
||||
k: usize,
|
||||
loss: MinCutLoss,
|
||||
iterations_run: usize,
|
||||
}
|
||||
|
||||
impl DiffCutResult {
|
||||
/// Number of clusters `K`.
|
||||
pub fn num_clusters(&self) -> usize {
|
||||
self.k
|
||||
}
|
||||
|
||||
/// Final (best-restart) loss.
|
||||
pub fn loss(&self) -> MinCutLoss {
|
||||
self.loss
|
||||
}
|
||||
|
||||
/// Iterations actually run in the best restart (≤ `iterations`; lower when
|
||||
/// early-stopping triggered).
|
||||
pub fn iterations_run(&self) -> usize {
|
||||
self.iterations_run
|
||||
}
|
||||
|
||||
/// Borrow the soft assignment matrix (row-major `N×K`).
|
||||
pub fn soft_assignment(&self) -> &[f64] {
|
||||
&self.soft
|
||||
}
|
||||
|
||||
/// Hard regions: group vertices by argmax cluster. Empty clusters are
|
||||
/// dropped; every vertex is assigned exactly once.
|
||||
pub fn hard_regions(&self) -> Vec<Vec<VertexId>> {
|
||||
let mut buckets: HashMap<usize, Vec<VertexId>> = HashMap::new();
|
||||
for i in 0..self.vertices.len() {
|
||||
let row = &self.soft[i * self.k..(i + 1) * self.k];
|
||||
let mut best = 0usize;
|
||||
let mut best_v = row[0];
|
||||
for (c, &v) in row.iter().enumerate().skip(1) {
|
||||
if v > best_v {
|
||||
best_v = v;
|
||||
best = c;
|
||||
}
|
||||
}
|
||||
buckets.entry(best).or_default().push(self.vertices[i]);
|
||||
}
|
||||
buckets.into_values().collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// Trainable differentiable min-cut condenser.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DiffCutCondenser {
|
||||
config: DiffCutConfig,
|
||||
}
|
||||
|
||||
impl DiffCutCondenser {
|
||||
/// Create a condenser with the given configuration.
|
||||
pub fn new(config: DiffCutConfig) -> Self {
|
||||
Self { config }
|
||||
}
|
||||
|
||||
/// Borrow the configuration.
|
||||
pub fn config(&self) -> &DiffCutConfig {
|
||||
&self.config
|
||||
}
|
||||
|
||||
/// Train the soft assignment by minimising the min-cut loss.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`CondenseError::EmptyGraph`] for a graph with no vertices, or
|
||||
/// [`CondenseError::InvalidConfig`] for `num_clusters == 0`.
|
||||
pub fn train(&self, graph: &DynamicGraph) -> Result<DiffCutResult> {
|
||||
self.config.validate()?;
|
||||
let g = CompactGraph::from_graph(graph);
|
||||
if g.n == 0 {
|
||||
return Err(CondenseError::EmptyGraph);
|
||||
}
|
||||
let (n, k) = (g.n, self.config.num_clusters);
|
||||
let restarts = self.config.restarts.max(1);
|
||||
|
||||
let mut best: Option<(Vec<f64>, MinCutLoss, usize)> = None;
|
||||
for r in 0..restarts {
|
||||
let seed = self
|
||||
.config
|
||||
.seed
|
||||
.wrapping_add((r as u64).wrapping_mul(0x9E37_79B9_7F4A_7C15));
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
let mut theta = match self.config.init {
|
||||
InitStrategy::Random => random_logits(n, k, &mut rng),
|
||||
InitStrategy::WarmStart => warm_start_logits(&g, graph, k, &mut rng),
|
||||
};
|
||||
let iters = self.optimize(&g, &mut theta, n, k, &mut rng);
|
||||
let soft = softmax_rows(&theta, n, k);
|
||||
let loss = forward(&g, &soft, k, self.config.ortho_weight);
|
||||
if best.as_ref().map_or(true, |(_, b, _)| loss.total < b.total) {
|
||||
best = Some((soft, loss, iters));
|
||||
}
|
||||
}
|
||||
|
||||
let (soft, loss, iterations_run) = best.expect("restarts >= 1");
|
||||
Ok(DiffCutResult {
|
||||
soft,
|
||||
vertices: g.vertices,
|
||||
k,
|
||||
loss,
|
||||
iterations_run,
|
||||
})
|
||||
}
|
||||
|
||||
/// Run the configured optimiser in place on `theta`; returns the number of
|
||||
/// iterations actually performed (early-stops on loss convergence). `rng`
|
||||
/// drives edge-minibatch sampling when enabled.
|
||||
fn optimize(
|
||||
&self,
|
||||
g: &CompactGraph,
|
||||
theta: &mut [f64],
|
||||
n: usize,
|
||||
k: usize,
|
||||
rng: &mut StdRng,
|
||||
) -> usize {
|
||||
let lr = self.config.learning_rate;
|
||||
let lambda = self.config.ortho_weight;
|
||||
let tol = self.config.tolerance;
|
||||
let parallel = self.config.parallel;
|
||||
let nnz = g.edges.len();
|
||||
let minibatch = self.config.minibatch_edges.filter(|_| nnz > 0);
|
||||
let mut prev = f64::INFINITY;
|
||||
let mut vel = vec![0f64; n * k];
|
||||
let mut m = vec![0f64; n * k];
|
||||
let mut v = vec![0f64; n * k];
|
||||
let mut iters_run = 0;
|
||||
|
||||
for t in 1..=self.config.iterations {
|
||||
let soft = softmax_rows(theta, n, k);
|
||||
// A·S: full (parallel optional) or a stochastic edge minibatch.
|
||||
let as_mat = match minibatch {
|
||||
Some(b) => {
|
||||
let b = b.min(nnz);
|
||||
let sample: Vec<usize> = (0..b).map(|_| rng.gen_range(0..nnz)).collect();
|
||||
as_matrix_minibatch(g, &soft, n, k, &sample)
|
||||
}
|
||||
None => as_matrix(g, &soft, n, k, parallel),
|
||||
};
|
||||
// loss_and_grad gives the loss at the *current* theta for free.
|
||||
let (loss, grad_s) = loss_and_grad_with_as(g, &soft, &as_mat, k, lambda, parallel);
|
||||
let gt = softmax_backprop(&soft, &grad_s, n, k);
|
||||
|
||||
match self.config.optimizer {
|
||||
Optimizer::Sgd { momentum } => {
|
||||
for idx in 0..n * k {
|
||||
vel[idx] = momentum * vel[idx] - lr * gt[idx];
|
||||
theta[idx] += vel[idx];
|
||||
}
|
||||
}
|
||||
Optimizer::Adam {
|
||||
beta1,
|
||||
beta2,
|
||||
epsilon,
|
||||
} => {
|
||||
let bc1 = 1.0 - beta1.powi(t as i32);
|
||||
let bc2 = 1.0 - beta2.powi(t as i32);
|
||||
for idx in 0..n * k {
|
||||
m[idx] = beta1 * m[idx] + (1.0 - beta1) * gt[idx];
|
||||
v[idx] = beta2 * v[idx] + (1.0 - beta2) * gt[idx] * gt[idx];
|
||||
let mhat = m[idx] / bc1;
|
||||
let vhat = v[idx] / bc2;
|
||||
theta[idx] -= lr * mhat / (vhat.sqrt() + epsilon);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
iters_run = t;
|
||||
if tol > 0.0 && (prev - loss.total).abs() < tol {
|
||||
break;
|
||||
}
|
||||
prev = loss.total;
|
||||
}
|
||||
iters_run
|
||||
}
|
||||
}
|
||||
|
||||
/// Unit-scale random logits.
|
||||
fn random_logits(n: usize, k: usize, rng: &mut StdRng) -> Vec<f64> {
|
||||
let mut theta = vec![0f64; n * k];
|
||||
for t in &mut theta {
|
||||
*t = rng.gen_range(-1.0..1.0);
|
||||
}
|
||||
theta
|
||||
}
|
||||
|
||||
/// Warm-start logits from the WeakBoundary structural prior: each detected
|
||||
/// region is mapped to a cluster (largest regions get their own; overflow is
|
||||
/// distributed round-robin) and biased into the logits, plus small noise.
|
||||
fn warm_start_logits(
|
||||
g: &CompactGraph,
|
||||
graph: &DynamicGraph,
|
||||
k: usize,
|
||||
rng: &mut StdRng,
|
||||
) -> Vec<f64> {
|
||||
const BIAS: f64 = 4.0; // softmax(4 vs 0) ~ 0.98 mass on the seeded cluster
|
||||
let index = g.index_map();
|
||||
|
||||
let mut regions = crate::regions::weak_boundary_regions(graph, 0.5);
|
||||
// If the structural prior found no usable split (e.g. an unweighted graph,
|
||||
// where WeakBoundary collapses to one component), warm-start would seed every
|
||||
// node into one cluster and the optimiser would stay collapsed. Fall back to
|
||||
// random init and let the min-cut objective do the partitioning.
|
||||
if regions.len() < 2 {
|
||||
return random_logits(g.n, k, rng);
|
||||
}
|
||||
// Deterministic order (weak_boundary_regions yields HashMap order): largest
|
||||
// first, ties broken by smallest member id.
|
||||
regions.sort_by(|a, b| {
|
||||
b.len()
|
||||
.cmp(&a.len())
|
||||
.then_with(|| a.iter().min().cmp(&b.iter().min()))
|
||||
});
|
||||
|
||||
let mut cluster_of = vec![0usize; g.n];
|
||||
for (ri, region) in regions.iter().enumerate() {
|
||||
let cluster = if ri < k { ri } else { ri % k };
|
||||
for v in region {
|
||||
if let Some(&row) = index.get(v) {
|
||||
cluster_of[row] = cluster;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut theta = vec![0f64; g.n * k];
|
||||
for row in 0..g.n {
|
||||
for c in 0..k {
|
||||
theta[row * k + c] = rng.gen_range(-0.1..0.1);
|
||||
}
|
||||
theta[row * k + cluster_of[row]] += BIAS;
|
||||
}
|
||||
theta
|
||||
}
|
||||
|
||||
/// Evaluate the min-cut loss for an arbitrary soft assignment (row-major `N×K`,
|
||||
/// rows in ascending-vertex order). Useful as a quality metric for any
|
||||
/// assignment, learned or hand-built.
|
||||
///
|
||||
/// # Errors
|
||||
/// [`CondenseError::DimensionMismatch`] if `soft.len() != N*k`.
|
||||
pub fn min_cut_loss(
|
||||
graph: &DynamicGraph,
|
||||
soft: &[f64],
|
||||
k: usize,
|
||||
ortho_weight: f64,
|
||||
) -> Result<MinCutLoss> {
|
||||
let g = CompactGraph::from_graph(graph);
|
||||
if soft.len() != g.n * k {
|
||||
return Err(CondenseError::DimensionMismatch {
|
||||
expected: g.n * k,
|
||||
got: soft.len(),
|
||||
});
|
||||
}
|
||||
Ok(forward(&g, soft, k, ortho_weight))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn barbell() -> DynamicGraph {
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 1.0),
|
||||
(1, 2, 1.0),
|
||||
(2, 0, 1.0),
|
||||
(3, 4, 1.0),
|
||||
(4, 5, 1.0),
|
||||
(5, 3, 1.0),
|
||||
(2, 3, 0.05),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
g
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_seeds_a_good_partition() {
|
||||
// Warm start alone (0 iterations) should already encode the 2 triangles.
|
||||
let g = barbell();
|
||||
let res = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 2,
|
||||
iterations: 0,
|
||||
..Default::default()
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
let mut regions = res.hard_regions();
|
||||
for r in &mut regions {
|
||||
r.sort_unstable();
|
||||
}
|
||||
regions.sort_by_key(|r| r[0]);
|
||||
assert_eq!(regions, vec![vec![0, 1, 2], vec![3, 4, 5]]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adam_refines_to_low_cut() {
|
||||
let g = barbell();
|
||||
let res = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 2,
|
||||
..Default::default()
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
assert!(res.loss().cut < -0.9, "cut {}", res.loss().cut);
|
||||
}
|
||||
}
|
||||
35
crates/ruvector-graph-condense/src/error.rs
Normal file
35
crates/ruvector-graph-condense/src/error.rs
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
//! Error types for graph condensation.
|
||||
|
||||
use thiserror::Error;
|
||||
|
||||
/// Errors that can occur during graph condensation.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum CondenseError {
|
||||
/// The input graph has no vertices, so there is nothing to condense.
|
||||
#[error("empty graph: nothing to condense")]
|
||||
EmptyGraph,
|
||||
|
||||
/// A feature vector did not match the configured embedding dimension.
|
||||
#[error("feature dimension mismatch: expected {expected}, got {got}")]
|
||||
DimensionMismatch {
|
||||
/// The dimension the [`crate::NodeFeatures`] container was created with.
|
||||
expected: usize,
|
||||
/// The dimension of the offending vector.
|
||||
got: usize,
|
||||
},
|
||||
|
||||
/// A vertex present in the graph had no associated feature vector.
|
||||
#[error("vertex {0} has no feature vector")]
|
||||
MissingFeature(u64),
|
||||
|
||||
/// The configuration was internally inconsistent.
|
||||
#[error("invalid config: {0}")]
|
||||
InvalidConfig(String),
|
||||
|
||||
/// An error bubbled up from the underlying min-cut engine.
|
||||
#[error("min-cut engine error: {0}")]
|
||||
MinCut(#[from] ruvector_mincut::MinCutError),
|
||||
}
|
||||
|
||||
/// Convenience result alias for condensation operations.
|
||||
pub type Result<T> = std::result::Result<T, CondenseError>;
|
||||
142
crates/ruvector-graph-condense/src/features.rs
Normal file
142
crates/ruvector-graph-condense/src/features.rs
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
//! Per-vertex feature storage (embeddings + optional class labels).
|
||||
//!
|
||||
//! Graph condensation needs more than topology: each original vertex carries a
|
||||
//! feature vector (e.g. a node embedding) and, for supervised settings, a class
|
||||
//! label. [`NodeFeatures`] is a thin, validated container keyed by the same
|
||||
//! [`VertexId`](ruvector_mincut::VertexId) used by the min-cut engine's
|
||||
//! [`DynamicGraph`](ruvector_mincut::DynamicGraph).
|
||||
|
||||
use crate::error::{CondenseError, Result};
|
||||
use ruvector_mincut::VertexId;
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Feature store mapping graph vertices to embeddings and optional labels.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NodeFeatures {
|
||||
dim: usize,
|
||||
num_classes: usize,
|
||||
embeddings: HashMap<VertexId, Vec<f32>>,
|
||||
labels: HashMap<VertexId, usize>,
|
||||
}
|
||||
|
||||
impl NodeFeatures {
|
||||
/// Create an empty feature store for `dim`-dimensional embeddings.
|
||||
///
|
||||
/// `num_classes` may be `0` for the unsupervised case (no class
|
||||
/// distributions are produced during condensation).
|
||||
pub fn new(dim: usize, num_classes: usize) -> Self {
|
||||
Self {
|
||||
dim,
|
||||
num_classes,
|
||||
embeddings: HashMap::new(),
|
||||
labels: HashMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Embedding dimension.
|
||||
pub fn dim(&self) -> usize {
|
||||
self.dim
|
||||
}
|
||||
|
||||
/// Number of distinct classes (`0` if unsupervised).
|
||||
pub fn num_classes(&self) -> usize {
|
||||
self.num_classes
|
||||
}
|
||||
|
||||
/// Number of vertices with a stored embedding.
|
||||
pub fn len(&self) -> usize {
|
||||
self.embeddings.len()
|
||||
}
|
||||
|
||||
/// Whether any embeddings are stored.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.embeddings.is_empty()
|
||||
}
|
||||
|
||||
/// Insert or replace the embedding for `vertex`.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns [`CondenseError::DimensionMismatch`] if `embedding.len() != dim`.
|
||||
pub fn set_embedding(&mut self, vertex: VertexId, embedding: Vec<f32>) -> Result<()> {
|
||||
if embedding.len() != self.dim {
|
||||
return Err(CondenseError::DimensionMismatch {
|
||||
expected: self.dim,
|
||||
got: embedding.len(),
|
||||
});
|
||||
}
|
||||
self.embeddings.insert(vertex, embedding);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Attach a class label to `vertex`. Labels at or above `num_classes` are
|
||||
/// accepted but will be ignored when building class distributions.
|
||||
pub fn set_label(&mut self, vertex: VertexId, label: usize) {
|
||||
self.labels.insert(vertex, label);
|
||||
}
|
||||
|
||||
/// Insert an embedding and label together.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns [`CondenseError::DimensionMismatch`] if the embedding dimension
|
||||
/// is wrong.
|
||||
pub fn set(&mut self, vertex: VertexId, embedding: Vec<f32>, label: usize) -> Result<()> {
|
||||
self.set_embedding(vertex, embedding)?;
|
||||
self.set_label(vertex, label);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Borrow the embedding for `vertex`, if present.
|
||||
pub fn embedding(&self, vertex: VertexId) -> Option<&[f32]> {
|
||||
self.embeddings.get(&vertex).map(Vec::as_slice)
|
||||
}
|
||||
|
||||
/// Get the label for `vertex`, if present.
|
||||
pub fn label(&self, vertex: VertexId) -> Option<usize> {
|
||||
self.labels.get(&vertex).copied()
|
||||
}
|
||||
|
||||
/// Borrow the embedding for `vertex` or fail with
|
||||
/// [`CondenseError::MissingFeature`].
|
||||
pub(crate) fn require(&self, vertex: VertexId) -> Result<&[f32]> {
|
||||
self.embedding(vertex)
|
||||
.ok_or(CondenseError::MissingFeature(vertex))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn rejects_wrong_dimension() {
|
||||
let mut f = NodeFeatures::new(3, 2);
|
||||
assert!(f.set_embedding(1, vec![0.0, 1.0, 2.0]).is_ok());
|
||||
let err = f.set_embedding(2, vec![0.0, 1.0]).unwrap_err();
|
||||
assert!(matches!(
|
||||
err,
|
||||
CondenseError::DimensionMismatch {
|
||||
expected: 3,
|
||||
got: 2
|
||||
}
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stores_and_reads_back() {
|
||||
let mut f = NodeFeatures::new(2, 3);
|
||||
f.set(7, vec![1.0, 2.0], 1).unwrap();
|
||||
assert_eq!(f.embedding(7), Some(&[1.0f32, 2.0][..]));
|
||||
assert_eq!(f.label(7), Some(1));
|
||||
assert_eq!(f.len(), 1);
|
||||
assert_eq!(f.num_classes(), 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn require_reports_missing() {
|
||||
let f = NodeFeatures::new(2, 0);
|
||||
assert!(matches!(
|
||||
f.require(42).unwrap_err(),
|
||||
CondenseError::MissingFeature(42)
|
||||
));
|
||||
}
|
||||
}
|
||||
426
crates/ruvector-graph-condense/src/gnn_eval.rs
Normal file
426
crates/ruvector-graph-condense/src/gnn_eval.rs
Normal file
|
|
@ -0,0 +1,426 @@
|
|||
//! Minimal 2-layer GCN — **for evaluating condensation quality only**.
|
||||
//!
|
||||
//! The graph-condensation literature is benchmarked by one protocol: train a GNN
|
||||
//! on the condensed graph, then test it on the *original* graph's held-out nodes,
|
||||
//! and report `accuracy(condensed) / accuracy(full)` ("retention"). Structural
|
||||
//! proxies (cut preservation, purity) do not substitute for it. This module is a
|
||||
//! self-contained, dependency-free (plain `f64`) reference GCN so the crate can
|
||||
//! report that number honestly.
|
||||
//!
|
||||
//! It is deliberately small: symmetric-normalised propagation `Â = D̃^{-1/2}
|
||||
//! (A+I) D̃^{-1/2}`, two graph-conv layers with ReLU, softmax cross-entropy,
|
||||
//! Adam, **analytic backprop** (gradient-checked in tests). Weights are
|
||||
//! graph-agnostic, so a GCN trained on the small condensed graph can be applied
|
||||
//! to the full graph at test time — exactly the condensation eval protocol.
|
||||
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
|
||||
/// Symmetric-normalised adjacency (with self-loops) in CSR form.
|
||||
pub struct GcnGraph {
|
||||
n: usize,
|
||||
off: Vec<usize>,
|
||||
nbr: Vec<(usize, f64)>,
|
||||
}
|
||||
|
||||
impl GcnGraph {
|
||||
/// Build `Â` from an undirected edge list `(i, j, w)` over `n` nodes.
|
||||
pub fn from_edges(n: usize, edges: &[(usize, usize, f64)]) -> Self {
|
||||
// Degrees including the self-loop (A + I).
|
||||
let mut deg = vec![1f64; n];
|
||||
for &(i, j, w) in edges {
|
||||
deg[i] += w;
|
||||
deg[j] += w;
|
||||
}
|
||||
let inv_sqrt: Vec<f64> = deg.iter().map(|d| 1.0 / d.sqrt()).collect();
|
||||
|
||||
// Count entries per row (neighbours both directions + self).
|
||||
let mut cnt = vec![1usize; n];
|
||||
for &(i, j, _) in edges {
|
||||
cnt[i] += 1;
|
||||
cnt[j] += 1;
|
||||
}
|
||||
let mut off = vec![0usize; n + 1];
|
||||
for i in 0..n {
|
||||
off[i + 1] = off[i] + cnt[i];
|
||||
}
|
||||
let mut cursor = off[..n].to_vec();
|
||||
let mut nbr = vec![(0usize, 0f64); off[n]];
|
||||
// Self-loops first.
|
||||
for i in 0..n {
|
||||
nbr[cursor[i]] = (i, inv_sqrt[i] * inv_sqrt[i]);
|
||||
cursor[i] += 1;
|
||||
}
|
||||
for &(i, j, w) in edges {
|
||||
let a = w * inv_sqrt[i] * inv_sqrt[j];
|
||||
nbr[cursor[i]] = (j, a);
|
||||
cursor[i] += 1;
|
||||
nbr[cursor[j]] = (i, a);
|
||||
cursor[j] += 1;
|
||||
}
|
||||
Self { n, off, nbr }
|
||||
}
|
||||
|
||||
/// `Â · M` where `M` is row-major `n × d`.
|
||||
fn spmm(&self, m: &[f64], d: usize) -> Vec<f64> {
|
||||
let mut out = vec![0f64; self.n * d];
|
||||
for i in 0..self.n {
|
||||
let orow = &mut out[i * d..(i + 1) * d];
|
||||
for e in self.off[i]..self.off[i + 1] {
|
||||
let (j, w) = self.nbr[e];
|
||||
let mrow = &m[j * d..(j + 1) * d];
|
||||
for c in 0..d {
|
||||
orow[c] += w * mrow[c];
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
}
|
||||
|
||||
/// A trained 2-layer GCN classifier.
|
||||
pub struct Gcn {
|
||||
w1: Vec<f64>, // f x h
|
||||
w2: Vec<f64>, // h x c
|
||||
f: usize,
|
||||
h: usize,
|
||||
c: usize,
|
||||
}
|
||||
|
||||
/// Training hyper-parameters.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct GcnConfig {
|
||||
/// Hidden width.
|
||||
pub hidden: usize,
|
||||
/// Adam learning rate.
|
||||
pub learning_rate: f64,
|
||||
/// Training epochs.
|
||||
pub epochs: usize,
|
||||
/// L2 weight decay.
|
||||
pub weight_decay: f64,
|
||||
/// Seed for weight init.
|
||||
pub seed: u64,
|
||||
}
|
||||
|
||||
impl Default for GcnConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
hidden: 16,
|
||||
learning_rate: 0.01,
|
||||
epochs: 200,
|
||||
weight_decay: 5e-4,
|
||||
seed: 0x6CD,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn relu(x: f64) -> f64 {
|
||||
x.max(0.0)
|
||||
}
|
||||
|
||||
fn mm(a: &[f64], b: &[f64], n: usize, p: usize, q: usize) -> Vec<f64> {
|
||||
// (n×p) · (p×q)
|
||||
let mut out = vec![0f64; n * q];
|
||||
for i in 0..n {
|
||||
for k in 0..p {
|
||||
let aik = a[i * p + k];
|
||||
if aik != 0.0 {
|
||||
for j in 0..q {
|
||||
out[i * q + j] += aik * b[k * q + j];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn mm_at(a: &[f64], b: &[f64], n: usize, p: usize, q: usize) -> Vec<f64> {
|
||||
// (n×p)ᵀ · (n×q) = (p×q)
|
||||
let mut out = vec![0f64; p * q];
|
||||
for i in 0..n {
|
||||
for k in 0..p {
|
||||
let aik = a[i * p + k];
|
||||
if aik != 0.0 {
|
||||
for j in 0..q {
|
||||
out[k * q + j] += aik * b[i * q + j];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn mm_bt(a: &[f64], b: &[f64], n: usize, q: usize, p: usize) -> Vec<f64> {
|
||||
// (n×q) · (p×q)ᵀ = (n×p)
|
||||
let mut out = vec![0f64; n * p];
|
||||
for i in 0..n {
|
||||
for k in 0..p {
|
||||
let mut acc = 0f64;
|
||||
for j in 0..q {
|
||||
acc += a[i * q + j] * b[k * q + j];
|
||||
}
|
||||
out[i * p + k] = acc;
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Forward intermediates kept for backprop.
|
||||
struct Fwd {
|
||||
ax: Vec<f64>,
|
||||
h1: Vec<f64>,
|
||||
ar: Vec<f64>,
|
||||
probs: Vec<f64>,
|
||||
}
|
||||
|
||||
impl Gcn {
|
||||
fn forward(&self, g: &GcnGraph, x: &[f64]) -> Fwd {
|
||||
let n = g.n;
|
||||
let ax = g.spmm(x, self.f);
|
||||
let h1 = mm(&ax, &self.w1, n, self.f, self.h);
|
||||
let r: Vec<f64> = h1.iter().map(|&v| relu(v)).collect();
|
||||
let ar = g.spmm(&r, self.h);
|
||||
let o = mm(&ar, &self.w2, n, self.h, self.c);
|
||||
let probs = softmax_rows(&o, n, self.c);
|
||||
Fwd { ax, h1, ar, probs }
|
||||
}
|
||||
|
||||
/// Predicted class per node.
|
||||
pub fn predict(&self, g: &GcnGraph, x: &[f64]) -> Vec<usize> {
|
||||
let fwd = self.forward(g, x);
|
||||
(0..g.n)
|
||||
.map(|i| argmax(&fwd.probs[i * self.c..(i + 1) * self.c]))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Train on `(g, x, labels)` over the nodes in `mask`. Returns the trained
|
||||
/// classifier.
|
||||
pub fn train(
|
||||
cfg: &GcnConfig,
|
||||
g: &GcnGraph,
|
||||
x: &[f64],
|
||||
f: usize,
|
||||
labels: &[usize],
|
||||
c: usize,
|
||||
mask: &[usize],
|
||||
) -> Gcn {
|
||||
let h = cfg.hidden;
|
||||
let mut rng = StdRng::seed_from_u64(cfg.seed);
|
||||
// Xavier-ish init.
|
||||
let s1 = (6.0 / (f + h) as f64).sqrt();
|
||||
let s2 = (6.0 / (h + c) as f64).sqrt();
|
||||
let mut model = Gcn {
|
||||
w1: (0..f * h).map(|_| rng.gen_range(-s1..s1)).collect(),
|
||||
w2: (0..h * c).map(|_| rng.gen_range(-s2..s2)).collect(),
|
||||
f,
|
||||
h,
|
||||
c,
|
||||
};
|
||||
// Adam state.
|
||||
let (mut m1, mut v1) = (vec![0f64; f * h], vec![0f64; f * h]);
|
||||
let (mut m2, mut v2) = (vec![0f64; h * c], vec![0f64; h * c]);
|
||||
for t in 1..=cfg.epochs {
|
||||
let (dw1, dw2) = model.grads(g, x, labels, mask);
|
||||
adam_step(&mut model.w1, &dw1, &mut m1, &mut v1, cfg, t);
|
||||
adam_step(&mut model.w2, &dw2, &mut m2, &mut v2, cfg, t);
|
||||
}
|
||||
model
|
||||
}
|
||||
|
||||
/// Analytic gradients of masked softmax-CE (+ L2) w.r.t. `w1`, `w2`.
|
||||
fn grads(
|
||||
&self,
|
||||
g: &GcnGraph,
|
||||
x: &[f64],
|
||||
labels: &[usize],
|
||||
mask: &[usize],
|
||||
) -> (Vec<f64>, Vec<f64>) {
|
||||
let n = g.n;
|
||||
let fwd = self.forward(g, x);
|
||||
let inv = 1.0 / mask.len().max(1) as f64;
|
||||
// dO = (P - onehot)/|mask| on masked rows, else 0.
|
||||
let mut d_o = vec![0f64; n * self.c];
|
||||
for &i in mask {
|
||||
for cc in 0..self.c {
|
||||
d_o[i * self.c + cc] = fwd.probs[i * self.c + cc] * inv;
|
||||
}
|
||||
d_o[i * self.c + labels[i]] -= inv;
|
||||
}
|
||||
let dw2 = mm_at(&fwd.ar, &d_o, n, self.h, self.c);
|
||||
let d_ar = mm_bt(&d_o, &self.w2, n, self.c, self.h);
|
||||
let d_r = g.spmm(&d_ar, self.h);
|
||||
let mut d_h1 = vec![0f64; n * self.h];
|
||||
for idx in 0..n * self.h {
|
||||
d_h1[idx] = if fwd.h1[idx] > 0.0 { d_r[idx] } else { 0.0 };
|
||||
}
|
||||
let dw1 = mm_at(&fwd.ax, &d_h1, n, self.f, self.h);
|
||||
// L2 weight decay is applied in `adam_step`, so gradients here are the
|
||||
// pure cross-entropy gradients (which the gradient check verifies).
|
||||
(dw1, dw2)
|
||||
}
|
||||
}
|
||||
|
||||
fn adam_step(w: &mut [f64], grad: &[f64], m: &mut [f64], v: &mut [f64], cfg: &GcnConfig, t: usize) {
|
||||
let (b1, b2, eps): (f64, f64, f64) = (0.9, 0.999, 1e-8);
|
||||
let bc1 = 1.0 - b1.powi(t as i32);
|
||||
let bc2 = 1.0 - b2.powi(t as i32);
|
||||
for i in 0..w.len() {
|
||||
let g = grad[i] + cfg.weight_decay * w[i];
|
||||
m[i] = b1 * m[i] + (1.0 - b1) * g;
|
||||
v[i] = b2 * v[i] + (1.0 - b2) * g * g;
|
||||
w[i] -= cfg.learning_rate * (m[i] / bc1) / ((v[i] / bc2).sqrt() + eps);
|
||||
}
|
||||
}
|
||||
|
||||
fn softmax_rows(o: &[f64], n: usize, c: usize) -> Vec<f64> {
|
||||
let mut p = vec![0f64; n * c];
|
||||
for i in 0..n {
|
||||
let row = &o[i * c..(i + 1) * c];
|
||||
let mx = row.iter().copied().fold(f64::NEG_INFINITY, f64::max);
|
||||
let mut s = 0f64;
|
||||
for j in 0..c {
|
||||
let e = (row[j] - mx).exp();
|
||||
p[i * c + j] = e;
|
||||
s += e;
|
||||
}
|
||||
for j in 0..c {
|
||||
p[i * c + j] /= s;
|
||||
}
|
||||
}
|
||||
p
|
||||
}
|
||||
|
||||
fn argmax(row: &[f64]) -> usize {
|
||||
let mut best = 0;
|
||||
for (i, &v) in row.iter().enumerate() {
|
||||
if v > row[best] {
|
||||
best = i;
|
||||
}
|
||||
}
|
||||
best
|
||||
}
|
||||
|
||||
/// Accuracy over the nodes in `mask`.
|
||||
pub fn accuracy(preds: &[usize], labels: &[usize], mask: &[usize]) -> f64 {
|
||||
if mask.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
let correct = mask.iter().filter(|&&i| preds[i] == labels[i]).count();
|
||||
correct as f64 / mask.len() as f64
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[allow(clippy::needless_range_loop)] // index-heavy numeric test code
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn ring(n: usize) -> (GcnGraph, Vec<(usize, usize, f64)>) {
|
||||
let edges: Vec<(usize, usize, f64)> = (0..n).map(|i| (i, (i + 1) % n, 1.0)).collect();
|
||||
(GcnGraph::from_edges(n, &edges), edges)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gradient_matches_finite_differences() {
|
||||
// The decisive correctness test for the GCN backprop.
|
||||
let n = 6;
|
||||
let (g, _e) = ring(n);
|
||||
let f = 3;
|
||||
let c = 2;
|
||||
let h = 4;
|
||||
let mut rng = StdRng::seed_from_u64(1);
|
||||
let x: Vec<f64> = (0..n * f).map(|_| rng.gen_range(-1.0..1.0)).collect();
|
||||
let labels: Vec<usize> = (0..n).map(|i| i % c).collect();
|
||||
let mask: Vec<usize> = (0..n).collect();
|
||||
let model = Gcn {
|
||||
w1: (0..f * h).map(|_| rng.gen_range(-0.5..0.5)).collect(),
|
||||
w2: (0..h * c).map(|_| rng.gen_range(-0.5..0.5)).collect(),
|
||||
f,
|
||||
h,
|
||||
c,
|
||||
};
|
||||
let loss = |m: &Gcn| -> f64 {
|
||||
let fwd = m.forward(&g, &x);
|
||||
let inv = 1.0 / mask.len() as f64;
|
||||
let mut l = 0f64;
|
||||
for &i in &mask {
|
||||
l -= (fwd.probs[i * c + labels[i]].max(1e-12)).ln() * inv;
|
||||
}
|
||||
l
|
||||
};
|
||||
let (dw1, dw2) = model.grads(&g, &x, &labels, &mask);
|
||||
let hh = 1e-6;
|
||||
let mut max_err = 0f64;
|
||||
for idx in 0..f * h {
|
||||
let mut mp = Gcn {
|
||||
w1: model.w1.clone(),
|
||||
w2: model.w2.clone(),
|
||||
f,
|
||||
h,
|
||||
c,
|
||||
};
|
||||
mp.w1[idx] += hh;
|
||||
let mut mm_ = Gcn {
|
||||
w1: model.w1.clone(),
|
||||
w2: model.w2.clone(),
|
||||
f,
|
||||
h,
|
||||
c,
|
||||
};
|
||||
mm_.w1[idx] -= hh;
|
||||
let num = (loss(&mp) - loss(&mm_)) / (2.0 * hh);
|
||||
max_err = max_err.max((num - dw1[idx]).abs());
|
||||
}
|
||||
for idx in 0..h * c {
|
||||
let mut mp = Gcn {
|
||||
w1: model.w1.clone(),
|
||||
w2: model.w2.clone(),
|
||||
f,
|
||||
h,
|
||||
c,
|
||||
};
|
||||
mp.w2[idx] += hh;
|
||||
let mut mm_ = Gcn {
|
||||
w1: model.w1.clone(),
|
||||
w2: model.w2.clone(),
|
||||
f,
|
||||
h,
|
||||
c,
|
||||
};
|
||||
mm_.w2[idx] -= hh;
|
||||
let num = (loss(&mp) - loss(&mm_)) / (2.0 * hh);
|
||||
max_err = max_err.max((num - dw2[idx]).abs());
|
||||
}
|
||||
assert!(max_err < 1e-6, "GCN grad mismatch: {max_err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn learns_a_separable_task() {
|
||||
// Two cliques, distinct features per class -> GCN should fit train set.
|
||||
let n = 20;
|
||||
let mut edges = Vec::new();
|
||||
for i in 0..10 {
|
||||
for j in (i + 1)..10 {
|
||||
edges.push((i, j, 1.0));
|
||||
}
|
||||
}
|
||||
for i in 10..20 {
|
||||
for j in (i + 1)..20 {
|
||||
edges.push((i, j, 1.0));
|
||||
}
|
||||
}
|
||||
let g = GcnGraph::from_edges(n, &edges);
|
||||
let f = 2;
|
||||
let c = 2;
|
||||
let mut x = vec![0f64; n * f];
|
||||
let mut labels = vec![0usize; n];
|
||||
for i in 0..n {
|
||||
let cls = i / 10;
|
||||
labels[i] = cls;
|
||||
x[i * f + cls] = 1.0;
|
||||
}
|
||||
let mask: Vec<usize> = (0..n).collect();
|
||||
let model = Gcn::train(&GcnConfig::default(), &g, &x, f, &labels, c, &mask);
|
||||
let preds = model.predict(&g, &x);
|
||||
assert!(accuracy(&preds, &labels, &mask) > 0.95);
|
||||
}
|
||||
}
|
||||
145
crates/ruvector-graph-condense/src/lib.rs
Normal file
145
crates/ruvector-graph-condense/src/lib.rs
Normal file
|
|
@ -0,0 +1,145 @@
|
|||
//! # ruvector-graph-condense
|
||||
//!
|
||||
//! Structure-preserving **graph condensation** built on RuVector's dynamic
|
||||
//! min-cut engine ([`ruvector_mincut`]).
|
||||
//!
|
||||
//! ## What this is (and isn't)
|
||||
//!
|
||||
//! The graph-condensation literature (GCond, SFGC, GEOM, SGDD, …) defines
|
||||
//! *condensation* as **synthesising a small fake graph** by optimising a
|
||||
//! learning objective (gradient/distribution/trajectory matching) so that a GNN
|
||||
//! trained on the synthetic graph matches one trained on the original. That is
|
||||
//! powerful but expensive (bi-level optimisation), supervised, and — by design
|
||||
//! — **destroys the mapping back to real nodes**.
|
||||
//!
|
||||
//! This crate takes the complementary, **training-free** route that the 2024–
|
||||
//! 2026 surveys flag as under-explored:
|
||||
//!
|
||||
//! - **Min-cut community structure as the condensation prior.** Regions come
|
||||
//! from recursive dynamic min-cut ([`ruvector_mincut::CommunityDetector`]),
|
||||
//! not k-means. No published method (as of 2026) uses graph-cut community
|
||||
//! detection as the core condensation mechanism — the closest analogs are
|
||||
//! CGC (generic clustering, 2025) and GCTD (tensor decomposition, 2025).
|
||||
//! - **A differentiable min-cut *loss*** ([`diffcut`], [`CondenseMethod::DiffMinCut`]).
|
||||
//! A relaxed normalized-cut + orthogonality objective (MinCutPool-style) whose
|
||||
//! region structure is *trained* by gradient descent to preserve the cut.
|
||||
//! The surveys flag an explicit differentiable min-cut term in the
|
||||
//! condensation loss as unpublished; only spectral terms (SGDD's LED, GDEM's
|
||||
//! eigenbasis) exist. Gradients are analytic (no autodiff dependency) and
|
||||
//! gradient-checked.
|
||||
//! - **Cuts preserved by construction.** Every original edge that crosses a
|
||||
//! region boundary survives as a weighted super-edge, so the condensed graph
|
||||
//! reproduces the source's cut structure instead of having to learn it. The
|
||||
//! [`metrics::cut_inflation`] proxy quantifies exactly this.
|
||||
//! - **Provenance retained.** Each [`CondensedNode`] keeps its `members`, so
|
||||
//! the original↔condensed mapping is intact (useful for audit / explainability
|
||||
//! — the thing learned condensation throws away).
|
||||
//!
|
||||
//! In the field's taxonomy this is closer to **structure-preserving coarsening
|
||||
//! with synthetic representatives** than to GCond-style condensation: it trades
|
||||
//! peak downstream accuracy for being fast, label-optional, deterministic,
|
||||
//! streaming-friendly, and interpretable.
|
||||
//!
|
||||
//! ## Pipeline
|
||||
//!
|
||||
//! ```text
|
||||
//! DynamicGraph + NodeFeatures
|
||||
//! │ recursive dynamic min-cut
|
||||
//! ▼
|
||||
//! Regions (communities)
|
||||
//! │ per region: centroid · weight · class histogram · coherence · medoid
|
||||
//! ▼
|
||||
//! CondensedGraph (super-nodes + boundary-weighted super-edges)
|
||||
//! ```
|
||||
//!
|
||||
//! ## Quick start
|
||||
//!
|
||||
//! ```
|
||||
//! use ruvector_graph_condense::{condense, NodeFeatures};
|
||||
//! use ruvector_mincut::DynamicGraph;
|
||||
//!
|
||||
//! // Two triangles joined by a weak bridge.
|
||||
//! let g = DynamicGraph::new();
|
||||
//! for &(u, v, w) in &[(0,1,1.0),(1,2,1.0),(2,0,1.0),
|
||||
//! (3,4,1.0),(4,5,1.0),(5,3,1.0),
|
||||
//! (2,3,0.05)] {
|
||||
//! g.insert_edge(u, v, w).unwrap();
|
||||
//! }
|
||||
//! let mut f = NodeFeatures::new(1, 0);
|
||||
//! for v in 0..6u64 { f.set_embedding(v, vec![v as f32]).unwrap(); }
|
||||
//!
|
||||
//! let condensed = condense(&g, &f).unwrap();
|
||||
//! assert_eq!(condensed.node_count(), 2); // recovered both communities
|
||||
//! assert_eq!(condensed.edge_count(), 1); // the bridge -> one super-edge
|
||||
//! assert!(condensed.node_reduction_ratio() == 3.0);
|
||||
//! ```
|
||||
|
||||
#![forbid(unsafe_code)]
|
||||
|
||||
pub mod condense;
|
||||
mod cutloss;
|
||||
pub mod diffcut;
|
||||
pub mod error;
|
||||
pub mod features;
|
||||
pub mod gnn_eval;
|
||||
pub mod metrics;
|
||||
pub mod node;
|
||||
mod regions;
|
||||
pub mod stream;
|
||||
pub mod synthetic;
|
||||
|
||||
pub use condense::{condense, CondenseConfig, CondenseMethod, GraphCondenser};
|
||||
pub use diffcut::{
|
||||
min_cut_loss, DiffCutCondenser, DiffCutConfig, DiffCutResult, InitStrategy, MinCutLoss,
|
||||
Optimizer,
|
||||
};
|
||||
pub use error::{CondenseError, Result};
|
||||
pub use features::NodeFeatures;
|
||||
pub use metrics::{cut_inflation, evaluate, evaluate_full, CondensationMetrics};
|
||||
pub use node::{CondensedEdge, CondensedGraph, CondensedNode};
|
||||
pub use stream::StreamingCondenser;
|
||||
pub use synthetic::PlantedPartition;
|
||||
|
||||
/// Crate version string.
|
||||
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
#[test]
|
||||
fn end_to_end_condense_and_evaluate() {
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 5,
|
||||
community_size: 16,
|
||||
dim: 8,
|
||||
p_intra: 0.5,
|
||||
p_inter: 0.001,
|
||||
seed: 42,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, f) = pp.generate();
|
||||
let condensed = condense(&g, &f).unwrap();
|
||||
let m = evaluate(&g, &condensed);
|
||||
|
||||
assert_eq!(m.source_nodes, 80);
|
||||
assert!(m.condensed_nodes >= 5);
|
||||
assert!(m.node_reduction_ratio > 1.0);
|
||||
assert!(m.intra_weight_ratio > 0.8);
|
||||
assert!(m.label_purity > 0.8);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn public_api_is_reachable() {
|
||||
let _ = VERSION;
|
||||
let g = DynamicGraph::new();
|
||||
g.insert_edge(0, 1, 1.0).unwrap();
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
f.set_embedding(0, vec![0.0]).unwrap();
|
||||
f.set_embedding(1, vec![1.0]).unwrap();
|
||||
let condenser = GraphCondenser::new(CondenseConfig::default());
|
||||
let c = condenser.condense(&g, &f).unwrap();
|
||||
assert_eq!(c.total_weight(), 2);
|
||||
}
|
||||
}
|
||||
214
crates/ruvector-graph-condense/src/metrics.rs
Normal file
214
crates/ruvector-graph-condense/src/metrics.rs
Normal file
|
|
@ -0,0 +1,214 @@
|
|||
//! Quality metrics for a condensation result.
|
||||
//!
|
||||
//! Accuracy-retention (retrain-a-GNN) evaluation is out of scope for this crate
|
||||
//! — the 2024–2026 literature explicitly calls for cheap *proxy* metrics that
|
||||
//! avoid retraining many GNNs. These are structural proxies computable directly
|
||||
//! from the source and condensed graphs.
|
||||
|
||||
use crate::node::CondensedGraph;
|
||||
use ruvector_mincut::{DynamicGraph, MinCutBuilder};
|
||||
|
||||
/// A bundle of cheap, retrain-free quality proxies.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct CondensationMetrics {
|
||||
/// Original vertex count.
|
||||
pub source_nodes: usize,
|
||||
/// Condensed super-node count.
|
||||
pub condensed_nodes: usize,
|
||||
/// `source_nodes / condensed_nodes`.
|
||||
pub node_reduction_ratio: f64,
|
||||
/// Original edge count.
|
||||
pub source_edges: usize,
|
||||
/// Condensed super-edge count.
|
||||
pub condensed_edges: usize,
|
||||
/// `source_edges / condensed_edges`.
|
||||
pub edge_reduction_ratio: f64,
|
||||
/// Fraction of total edge weight that stayed *inside* a region. Higher is
|
||||
/// better: it means the partition cut few/light edges (good community
|
||||
/// structure was found).
|
||||
pub intra_weight_ratio: f64,
|
||||
/// Mean per-region coherence in `[0, 1]`.
|
||||
pub mean_coherence: f32,
|
||||
/// Weight-averaged region purity (dominant-class share); `1.0` when
|
||||
/// unsupervised.
|
||||
pub label_purity: f32,
|
||||
/// Global-min-cut inflation: `mincut(condensed) / mincut(source)`.
|
||||
/// `Some(1.0)` means the source's global min cut survives coarsening
|
||||
/// exactly; `> 1.0` means the true cut got hidden inside a region. `None`
|
||||
/// when undefined (disconnected source, or condensed graph too small).
|
||||
pub cut_inflation: Option<f64>,
|
||||
}
|
||||
|
||||
/// Compute the cheap structural proxies (no min-cut solve).
|
||||
pub fn evaluate(graph: &DynamicGraph, condensed: &CondensedGraph) -> CondensationMetrics {
|
||||
let total_weight: f64 = graph.edges().iter().map(|e| e.weight).sum();
|
||||
let inter_weight: f64 = condensed.edges.iter().map(|e| e.weight).sum();
|
||||
let intra_weight_ratio = if total_weight > 0.0 {
|
||||
((total_weight - inter_weight) / total_weight).clamp(0.0, 1.0)
|
||||
} else {
|
||||
1.0
|
||||
};
|
||||
|
||||
let (mean_coherence, label_purity) = aggregate_node_quality(condensed);
|
||||
|
||||
CondensationMetrics {
|
||||
source_nodes: condensed.source_nodes,
|
||||
condensed_nodes: condensed.node_count(),
|
||||
node_reduction_ratio: condensed.node_reduction_ratio(),
|
||||
source_edges: condensed.source_edges,
|
||||
condensed_edges: condensed.edge_count(),
|
||||
edge_reduction_ratio: condensed.edge_reduction_ratio(),
|
||||
intra_weight_ratio,
|
||||
mean_coherence,
|
||||
label_purity,
|
||||
cut_inflation: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Like [`evaluate`], but also solves the global min cut on both graphs to fill
|
||||
/// in [`CondensationMetrics::cut_inflation`]. This is **O(min-cut)** on the full
|
||||
/// source graph and is therefore opt-in.
|
||||
pub fn evaluate_full(graph: &DynamicGraph, condensed: &CondensedGraph) -> CondensationMetrics {
|
||||
let mut m = evaluate(graph, condensed);
|
||||
m.cut_inflation = cut_inflation(graph, condensed);
|
||||
m
|
||||
}
|
||||
|
||||
/// Ratio of the condensed graph's global min cut to the source's. See
|
||||
/// [`CondensationMetrics::cut_inflation`] for interpretation.
|
||||
pub fn cut_inflation(graph: &DynamicGraph, condensed: &CondensedGraph) -> Option<f64> {
|
||||
// Need a meaningful cut on both sides.
|
||||
if graph.num_vertices() < 2 || condensed.node_count() < 2 {
|
||||
return None;
|
||||
}
|
||||
|
||||
let source_cut = global_min_cut(graph.edges().iter().map(|e| (e.source, e.target, e.weight)))?;
|
||||
if source_cut <= 0.0 {
|
||||
// Disconnected source: ratio undefined.
|
||||
return None;
|
||||
}
|
||||
let condensed_cut = global_min_cut(
|
||||
condensed
|
||||
.edges
|
||||
.iter()
|
||||
.map(|e| (e.source, e.target, e.weight)),
|
||||
)?;
|
||||
|
||||
Some(condensed_cut / source_cut)
|
||||
}
|
||||
|
||||
/// Solve an exact global min cut over an edge iterator, returning `None` if the
|
||||
/// result is non-finite (e.g. fewer than 2 connected vertices).
|
||||
fn global_min_cut<I>(edges: I) -> Option<f64>
|
||||
where
|
||||
I: IntoIterator<Item = (u64, u64, f64)>,
|
||||
{
|
||||
let edge_vec: Vec<(u64, u64, f64)> = edges.into_iter().collect();
|
||||
if edge_vec.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let mincut = MinCutBuilder::new()
|
||||
.exact()
|
||||
.with_edges(edge_vec)
|
||||
.build()
|
||||
.ok()?;
|
||||
let v = mincut.min_cut_value();
|
||||
if v.is_finite() {
|
||||
Some(v)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
fn aggregate_node_quality(condensed: &CondensedGraph) -> (f32, f32) {
|
||||
if condensed.nodes.is_empty() {
|
||||
return (0.0, 1.0);
|
||||
}
|
||||
let mut coherence_sum = 0.0f32;
|
||||
let mut purity_weighted = 0.0f32;
|
||||
let mut weight_total = 0.0f32;
|
||||
for n in &condensed.nodes {
|
||||
coherence_sum += n.coherence;
|
||||
let w = n.weight as f32;
|
||||
purity_weighted += n.purity() * w;
|
||||
weight_total += w;
|
||||
}
|
||||
let mean_coherence = coherence_sum / condensed.nodes.len() as f32;
|
||||
let label_purity = if weight_total > 0.0 {
|
||||
purity_weighted / weight_total
|
||||
} else {
|
||||
1.0
|
||||
};
|
||||
(mean_coherence, label_purity)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::condense::{condense, CondenseConfig, CondenseMethod, GraphCondenser};
|
||||
use crate::features::NodeFeatures;
|
||||
|
||||
fn barbell() -> (DynamicGraph, NodeFeatures) {
|
||||
// Two K3 cliques joined by a single weak bridge.
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 1.0),
|
||||
(1, 2, 1.0),
|
||||
(2, 0, 1.0),
|
||||
(3, 4, 1.0),
|
||||
(4, 5, 1.0),
|
||||
(5, 3, 1.0),
|
||||
(2, 3, 0.1),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
let mut f = NodeFeatures::new(1, 2);
|
||||
for v in 0..3u64 {
|
||||
f.set(v, vec![0.0], 0).unwrap();
|
||||
}
|
||||
for v in 3..6u64 {
|
||||
f.set(v, vec![1.0], 1).unwrap();
|
||||
}
|
||||
(g, f)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reports_reduction_and_quality() {
|
||||
let (g, f) = barbell();
|
||||
let c = condense(&g, &f).unwrap();
|
||||
let m = evaluate(&g, &c);
|
||||
assert_eq!(m.source_nodes, 6);
|
||||
assert_eq!(m.condensed_nodes, 2);
|
||||
assert_eq!(m.node_reduction_ratio, 3.0);
|
||||
// Only the 0.1 bridge crosses regions; 6 unit edges stay internal.
|
||||
assert!(m.intra_weight_ratio > 0.95);
|
||||
assert!(m.mean_coherence > 0.9);
|
||||
assert!(m.label_purity > 0.99);
|
||||
assert_eq!(m.cut_inflation, None); // evaluate() doesn't solve cuts
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cut_inflation_preserved_for_clean_partition() {
|
||||
let (g, f) = barbell();
|
||||
let c = condense(&g, &f).unwrap();
|
||||
// Source global min cut = 0.1 (the bridge). Condensed graph is a single
|
||||
// super-edge of weight 0.1, so its min cut is also 0.1 -> ratio 1.0.
|
||||
let infl = cut_inflation(&g, &c).expect("defined for connected barbell");
|
||||
assert!((infl - 1.0).abs() < 1e-9, "got {infl}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn evaluate_full_fills_cut() {
|
||||
let (g, f) = barbell();
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::ConnectedComponents,
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
// Connected barbell -> single component -> single super-node -> cut None.
|
||||
let m = evaluate_full(&g, &c);
|
||||
assert_eq!(m.condensed_nodes, 1);
|
||||
assert_eq!(m.cut_inflation, None);
|
||||
}
|
||||
}
|
||||
237
crates/ruvector-graph-condense/src/node.rs
Normal file
237
crates/ruvector-graph-condense/src/node.rs
Normal file
|
|
@ -0,0 +1,237 @@
|
|||
//! Condensed graph data model: super-nodes (regions) and super-edges.
|
||||
|
||||
use ruvector_mincut::{DynamicGraph, VertexId};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A single super-node in a condensed graph: one structural region of the
|
||||
/// original graph collapsed to a representative summary.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct CondensedNode {
|
||||
/// Stable region id (assigned deterministically by the condenser).
|
||||
pub id: u64,
|
||||
/// Mean embedding of the region's members.
|
||||
pub centroid: Vec<f32>,
|
||||
/// Number of original vertices collapsed into this region.
|
||||
pub weight: u32,
|
||||
/// Normalised class histogram over `num_classes` (empty when unsupervised).
|
||||
pub class_distribution: Vec<f32>,
|
||||
/// Internal cohesion in `[0, 1]`: fraction of incident edge weight that
|
||||
/// stays inside the region (1.0 = fully self-contained).
|
||||
pub coherence: f32,
|
||||
/// Member closest to the centroid (the region's medoid).
|
||||
pub representative: VertexId,
|
||||
/// The original vertices that belong to this region (sorted ascending).
|
||||
pub members: Vec<VertexId>,
|
||||
}
|
||||
|
||||
impl CondensedNode {
|
||||
/// The dominant class of this region, if a class distribution is present.
|
||||
pub fn dominant_class(&self) -> Option<usize> {
|
||||
if self.class_distribution.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let mut best = 0usize;
|
||||
let mut best_p = self.class_distribution[0];
|
||||
for (i, &p) in self.class_distribution.iter().enumerate().skip(1) {
|
||||
if p > best_p {
|
||||
best_p = p;
|
||||
best = i;
|
||||
}
|
||||
}
|
||||
Some(best)
|
||||
}
|
||||
|
||||
/// Purity of the dominant class (its share of the region), or `1.0` when
|
||||
/// unsupervised (empty distribution).
|
||||
pub fn purity(&self) -> f32 {
|
||||
if self.class_distribution.is_empty() {
|
||||
return 1.0;
|
||||
}
|
||||
self.class_distribution
|
||||
.iter()
|
||||
.copied()
|
||||
.fold(0.0_f32, f32::max)
|
||||
}
|
||||
}
|
||||
|
||||
/// A weighted super-edge between two regions, aggregating every original edge
|
||||
/// that crosses the corresponding region boundary.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize)]
|
||||
pub struct CondensedEdge {
|
||||
/// Source region id (always `< target` for canonical undirected storage).
|
||||
pub source: u64,
|
||||
/// Target region id.
|
||||
pub target: u64,
|
||||
/// Sum of crossing original edge weights.
|
||||
pub weight: f64,
|
||||
/// Number of original edges merged into this super-edge.
|
||||
pub crossings: u32,
|
||||
}
|
||||
|
||||
/// The result of condensing a graph: a small set of super-nodes and the
|
||||
/// weighted super-edges connecting them, plus provenance for computing
|
||||
/// reduction ratios.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct CondensedGraph {
|
||||
/// Super-nodes, ordered by ascending `id`.
|
||||
pub nodes: Vec<CondensedNode>,
|
||||
/// Super-edges (canonical, deduplicated).
|
||||
pub edges: Vec<CondensedEdge>,
|
||||
/// Original vertex count (provenance).
|
||||
pub source_nodes: usize,
|
||||
/// Original edge count (provenance).
|
||||
pub source_edges: usize,
|
||||
/// Embedding dimension.
|
||||
pub dim: usize,
|
||||
/// Class count (`0` if unsupervised).
|
||||
pub num_classes: usize,
|
||||
}
|
||||
|
||||
impl CondensedGraph {
|
||||
/// Number of super-nodes.
|
||||
pub fn node_count(&self) -> usize {
|
||||
self.nodes.len()
|
||||
}
|
||||
|
||||
/// Number of super-edges.
|
||||
pub fn edge_count(&self) -> usize {
|
||||
self.edges.len()
|
||||
}
|
||||
|
||||
/// Node reduction factor (`source_nodes / condensed_nodes`).
|
||||
pub fn node_reduction_ratio(&self) -> f64 {
|
||||
if self.nodes.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
self.source_nodes as f64 / self.nodes.len() as f64
|
||||
}
|
||||
|
||||
/// Edge reduction factor (`source_edges / condensed_edges`).
|
||||
pub fn edge_reduction_ratio(&self) -> f64 {
|
||||
if self.edges.is_empty() {
|
||||
return if self.source_edges == 0 {
|
||||
1.0
|
||||
} else {
|
||||
self.source_edges as f64
|
||||
};
|
||||
}
|
||||
self.source_edges as f64 / self.edges.len() as f64
|
||||
}
|
||||
|
||||
/// Look up a super-node by region id (binary search; nodes are id-sorted).
|
||||
pub fn get_node(&self, id: u64) -> Option<&CondensedNode> {
|
||||
self.nodes
|
||||
.binary_search_by_key(&id, |n| n.id)
|
||||
.ok()
|
||||
.map(|i| &self.nodes[i])
|
||||
}
|
||||
|
||||
/// Total weight (member count) across all super-nodes — equals
|
||||
/// `source_nodes` for a complete partition.
|
||||
pub fn total_weight(&self) -> u64 {
|
||||
self.nodes.iter().map(|n| n.weight as u64).sum()
|
||||
}
|
||||
|
||||
/// Rebuild the condensed graph as a [`DynamicGraph`] (region id → vertex
|
||||
/// id). Enables hierarchical / iterated condensation and feeding the
|
||||
/// condensed structure back into the min-cut engine.
|
||||
pub fn to_dynamic_graph(&self) -> DynamicGraph {
|
||||
let g = DynamicGraph::with_capacity(self.nodes.len(), self.edges.len());
|
||||
for n in &self.nodes {
|
||||
g.add_vertex(n.id);
|
||||
}
|
||||
for e in &self.edges {
|
||||
// Super-edges are canonical and unique, so insert cannot collide.
|
||||
let _ = g.insert_edge(e.source, e.target, e.weight);
|
||||
}
|
||||
g
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn node(id: u64, dist: Vec<f32>) -> CondensedNode {
|
||||
CondensedNode {
|
||||
id,
|
||||
centroid: vec![0.0],
|
||||
weight: 1,
|
||||
class_distribution: dist,
|
||||
coherence: 1.0,
|
||||
representative: id,
|
||||
members: vec![id],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dominant_class_picks_argmax() {
|
||||
let n = node(0, vec![0.1, 0.7, 0.2]);
|
||||
assert_eq!(n.dominant_class(), Some(1));
|
||||
let unsup = node(1, vec![]);
|
||||
assert_eq!(unsup.dominant_class(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reduction_ratios() {
|
||||
let g = CondensedGraph {
|
||||
nodes: vec![node(0, vec![]), node(1, vec![])],
|
||||
edges: vec![CondensedEdge {
|
||||
source: 0,
|
||||
target: 1,
|
||||
weight: 1.0,
|
||||
crossings: 3,
|
||||
}],
|
||||
source_nodes: 100,
|
||||
source_edges: 400,
|
||||
dim: 1,
|
||||
num_classes: 0,
|
||||
};
|
||||
assert_eq!(g.node_reduction_ratio(), 50.0);
|
||||
assert_eq!(g.edge_reduction_ratio(), 400.0);
|
||||
assert_eq!(g.total_weight(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn get_node_binary_search() {
|
||||
let g = CondensedGraph {
|
||||
nodes: vec![node(0, vec![]), node(5, vec![]), node(9, vec![])],
|
||||
edges: vec![],
|
||||
source_nodes: 3,
|
||||
source_edges: 0,
|
||||
dim: 1,
|
||||
num_classes: 0,
|
||||
};
|
||||
assert_eq!(g.get_node(5).map(|n| n.id), Some(5));
|
||||
assert!(g.get_node(7).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn round_trips_to_dynamic_graph() {
|
||||
let g = CondensedGraph {
|
||||
nodes: vec![node(0, vec![]), node(1, vec![]), node(2, vec![])],
|
||||
edges: vec![
|
||||
CondensedEdge {
|
||||
source: 0,
|
||||
target: 1,
|
||||
weight: 2.0,
|
||||
crossings: 1,
|
||||
},
|
||||
CondensedEdge {
|
||||
source: 1,
|
||||
target: 2,
|
||||
weight: 3.0,
|
||||
crossings: 1,
|
||||
},
|
||||
],
|
||||
source_nodes: 3,
|
||||
source_edges: 2,
|
||||
dim: 1,
|
||||
num_classes: 0,
|
||||
};
|
||||
let dg = g.to_dynamic_graph();
|
||||
assert_eq!(dg.num_vertices(), 3);
|
||||
assert_eq!(dg.num_edges(), 2);
|
||||
assert_eq!(dg.edge_weight(0, 1), Some(2.0));
|
||||
}
|
||||
}
|
||||
253
crates/ruvector-graph-condense/src/regions.rs
Normal file
253
crates/ruvector-graph-condense/src/regions.rs
Normal file
|
|
@ -0,0 +1,253 @@
|
|||
//! Region formation and per-region summarisation helpers.
|
||||
//!
|
||||
//! Split out from [`crate::condense`] to keep the orchestration small: this
|
||||
//! module owns *how a region is detected and summarised* (weak-boundary
|
||||
//! components, coverage, centroid/medoid, class histograms), while `condense`
|
||||
//! owns the pipeline that wires them together.
|
||||
|
||||
use crate::error::Result;
|
||||
use crate::features::NodeFeatures;
|
||||
use ruvector_mincut::{DynamicGraph, VertexId};
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Regions = connected components of the graph after removing edges lighter than
|
||||
/// `relative_threshold * mean_weight`. Isolated / weak-only vertices fall out as
|
||||
/// singletons. Deterministic for a fixed graph. Single edge pass + union-find,
|
||||
/// so it scales near-linearly.
|
||||
pub(crate) fn weak_boundary_regions(
|
||||
graph: &DynamicGraph,
|
||||
relative_threshold: f64,
|
||||
) -> Vec<Vec<VertexId>> {
|
||||
let vertices = graph.vertices();
|
||||
let edges = graph.edges();
|
||||
|
||||
// Index vertices contiguously for the union-find.
|
||||
let mut index: HashMap<VertexId, usize> = HashMap::with_capacity(vertices.len());
|
||||
for (i, &v) in vertices.iter().enumerate() {
|
||||
index.insert(v, i);
|
||||
}
|
||||
let mut uf = UnionFind::new(vertices.len());
|
||||
|
||||
let threshold = if edges.is_empty() {
|
||||
0.0
|
||||
} else {
|
||||
let mean = edges.iter().map(|e| e.weight).sum::<f64>() / edges.len() as f64;
|
||||
relative_threshold * mean
|
||||
};
|
||||
|
||||
for e in &edges {
|
||||
if e.weight >= threshold {
|
||||
uf.union(index[&e.source], index[&e.target]);
|
||||
}
|
||||
}
|
||||
|
||||
// Group vertices by their union-find root.
|
||||
let mut groups: HashMap<usize, Vec<VertexId>> = HashMap::new();
|
||||
for (i, &v) in vertices.iter().enumerate() {
|
||||
groups.entry(uf.find(i)).or_default().push(v);
|
||||
}
|
||||
groups.into_values().collect()
|
||||
}
|
||||
|
||||
/// Append singleton regions for any graph vertex not already covered by the
|
||||
/// partitioner (some partitioners drop isolated or unsplittable vertices).
|
||||
pub(crate) fn ensure_coverage(regions: &mut Vec<Vec<VertexId>>, vertices: &[VertexId]) {
|
||||
let mut seen: std::collections::HashSet<VertexId> =
|
||||
std::collections::HashSet::with_capacity(vertices.len());
|
||||
for r in regions.iter() {
|
||||
for &v in r {
|
||||
seen.insert(v);
|
||||
}
|
||||
}
|
||||
for &v in vertices {
|
||||
if seen.insert(v) {
|
||||
regions.push(vec![v]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Mean embedding and medoid (member closest to the mean) of a region.
|
||||
/// `members` must be non-empty.
|
||||
pub(crate) fn centroid_and_medoid(
|
||||
members: &[VertexId],
|
||||
features: &NodeFeatures,
|
||||
dim: usize,
|
||||
) -> Result<(Vec<f32>, VertexId)> {
|
||||
let mut centroid = vec![0f32; dim];
|
||||
for &v in members {
|
||||
let emb = features.require(v)?;
|
||||
for (c, &x) in centroid.iter_mut().zip(emb.iter()) {
|
||||
*c += x;
|
||||
}
|
||||
}
|
||||
let inv = 1.0 / members.len() as f32;
|
||||
for c in &mut centroid {
|
||||
*c *= inv;
|
||||
}
|
||||
|
||||
let mut best = members[0];
|
||||
let mut best_dist = f32::INFINITY;
|
||||
for &v in members {
|
||||
let emb = features.require(v)?;
|
||||
let d = l2_sq(¢roid, emb);
|
||||
if d < best_dist {
|
||||
best_dist = d;
|
||||
best = v;
|
||||
}
|
||||
}
|
||||
Ok((centroid, best))
|
||||
}
|
||||
|
||||
/// Normalised class histogram over `num_classes`, or empty when unsupervised.
|
||||
pub(crate) fn class_distribution(
|
||||
members: &[VertexId],
|
||||
features: &NodeFeatures,
|
||||
num_classes: usize,
|
||||
) -> Vec<f32> {
|
||||
if num_classes == 0 {
|
||||
return Vec::new();
|
||||
}
|
||||
let mut hist = vec![0f32; num_classes];
|
||||
let mut counted = 0f32;
|
||||
for &v in members {
|
||||
if let Some(label) = features.label(v) {
|
||||
if label < num_classes {
|
||||
hist[label] += 1.0;
|
||||
counted += 1.0;
|
||||
}
|
||||
}
|
||||
}
|
||||
if counted > 0.0 {
|
||||
let inv = 1.0 / counted;
|
||||
for h in &mut hist {
|
||||
*h *= inv;
|
||||
}
|
||||
}
|
||||
hist
|
||||
}
|
||||
|
||||
/// Squared Euclidean distance.
|
||||
pub(crate) fn l2_sq(a: &[f32], b: &[f32]) -> f32 {
|
||||
a.iter()
|
||||
.zip(b.iter())
|
||||
.map(|(x, y)| {
|
||||
let d = x - y;
|
||||
d * d
|
||||
})
|
||||
.sum()
|
||||
}
|
||||
|
||||
/// L2-normalise in place (no-op for a zero vector).
|
||||
pub(crate) fn l2_normalize(v: &mut [f32]) {
|
||||
let norm: f32 = v.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
if norm > 0.0 {
|
||||
let inv = 1.0 / norm;
|
||||
for x in v {
|
||||
*x *= inv;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Minimal union-find with path compression and union by size.
|
||||
struct UnionFind {
|
||||
parent: Vec<usize>,
|
||||
size: Vec<usize>,
|
||||
}
|
||||
|
||||
impl UnionFind {
|
||||
fn new(n: usize) -> Self {
|
||||
Self {
|
||||
parent: (0..n).collect(),
|
||||
size: vec![1; n],
|
||||
}
|
||||
}
|
||||
|
||||
fn find(&mut self, mut x: usize) -> usize {
|
||||
while self.parent[x] != x {
|
||||
self.parent[x] = self.parent[self.parent[x]];
|
||||
x = self.parent[x];
|
||||
}
|
||||
x
|
||||
}
|
||||
|
||||
fn union(&mut self, a: usize, b: usize) {
|
||||
let (ra, rb) = (self.find(a), self.find(b));
|
||||
if ra == rb {
|
||||
return;
|
||||
}
|
||||
let (big, small) = if self.size[ra] >= self.size[rb] {
|
||||
(ra, rb)
|
||||
} else {
|
||||
(rb, ra)
|
||||
};
|
||||
self.parent[small] = big;
|
||||
self.size[big] += self.size[small];
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn weak_boundary_splits_on_light_edges() {
|
||||
let g = DynamicGraph::new();
|
||||
// Heavy clique {0,1,2}, heavy clique {3,4,5}, light bridge 2-3.
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 1.0),
|
||||
(1, 2, 1.0),
|
||||
(2, 0, 1.0),
|
||||
(3, 4, 1.0),
|
||||
(4, 5, 1.0),
|
||||
(5, 3, 1.0),
|
||||
(2, 3, 0.05),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
let mut regions = weak_boundary_regions(&g, 0.5);
|
||||
for r in &mut regions {
|
||||
r.sort_unstable();
|
||||
}
|
||||
regions.sort_by_key(|r| r[0]);
|
||||
assert_eq!(regions, vec![vec![0, 1, 2], vec![3, 4, 5]]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ensure_coverage_adds_missing() {
|
||||
let mut regions = vec![vec![0u64, 1]];
|
||||
ensure_coverage(&mut regions, &[0, 1, 2, 3]);
|
||||
let singletons: usize = regions.iter().filter(|r| r.len() == 1).count();
|
||||
assert_eq!(singletons, 2); // 2 and 3 added
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn centroid_mean_and_medoid() {
|
||||
let mut f = NodeFeatures::new(1, 0);
|
||||
f.set_embedding(0, vec![0.0]).unwrap();
|
||||
f.set_embedding(1, vec![2.0]).unwrap();
|
||||
f.set_embedding(2, vec![4.0]).unwrap();
|
||||
let (centroid, medoid) = centroid_and_medoid(&[0, 1, 2], &f, 1).unwrap();
|
||||
assert!((centroid[0] - 2.0).abs() < 1e-6);
|
||||
assert_eq!(medoid, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn class_dist_normalises() {
|
||||
let mut f = NodeFeatures::new(1, 3);
|
||||
f.set(0, vec![0.0], 0).unwrap();
|
||||
f.set(1, vec![0.0], 0).unwrap();
|
||||
f.set(2, vec![0.0], 2).unwrap();
|
||||
let d = class_distribution(&[0, 1, 2], &f, 3);
|
||||
assert!((d[0] - 2.0 / 3.0).abs() < 1e-6);
|
||||
assert_eq!(d[1], 0.0);
|
||||
assert!((d[2] - 1.0 / 3.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_unit_length() {
|
||||
let mut v = vec![3.0f32, 4.0];
|
||||
l2_normalize(&mut v);
|
||||
let n: f32 = v.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||
assert!((n - 1.0).abs() < 1e-6);
|
||||
}
|
||||
}
|
||||
215
crates/ruvector-graph-condense/src/stream.rs
Normal file
215
crates/ruvector-graph-condense/src/stream.rs
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
//! Streaming condensation: maintain a growing graph + features and re-condense
|
||||
//! on demand.
|
||||
//!
|
||||
//! The 2024–2026 literature treats streaming/temporal condensation as an open
|
||||
//! problem (only OpenGC and CaT/PUMA touch it, in restricted settings). This
|
||||
//! crate takes a deliberately honest stance: it does **lazy re-condensation**,
|
||||
//! not true incremental region surgery. Edges/features are buffered into a
|
||||
//! [`DynamicGraph`]; the condensed view is rebuilt when it is read while dirty,
|
||||
//! or every `rebuild_interval` mutations. The win is amortisation and a stable
|
||||
//! API for edge pipelines (e.g. condensing a RuView WorldGraph as it grows),
|
||||
//! not sublinear updates — that remains future work.
|
||||
|
||||
use crate::condense::{CondenseConfig, GraphCondenser};
|
||||
use crate::error::Result;
|
||||
use crate::features::NodeFeatures;
|
||||
use crate::node::CondensedGraph;
|
||||
use ruvector_mincut::{DynamicGraph, VertexId, Weight};
|
||||
|
||||
/// A mutable graph + feature store that condenses lazily.
|
||||
pub struct StreamingCondenser {
|
||||
graph: DynamicGraph,
|
||||
features: NodeFeatures,
|
||||
condenser: GraphCondenser,
|
||||
cached: Option<CondensedGraph>,
|
||||
dirty: bool,
|
||||
ops_since_rebuild: usize,
|
||||
rebuild_interval: usize,
|
||||
}
|
||||
|
||||
impl StreamingCondenser {
|
||||
/// Create a streaming condenser.
|
||||
///
|
||||
/// `rebuild_interval` is the maximum number of mutations tolerated before
|
||||
/// [`StreamingCondenser::condensed`] forces a rebuild even if not otherwise
|
||||
/// read. Use `0` to rebuild only on explicit reads of a dirty state.
|
||||
pub fn new(
|
||||
config: CondenseConfig,
|
||||
dim: usize,
|
||||
num_classes: usize,
|
||||
rebuild_interval: usize,
|
||||
) -> Self {
|
||||
Self {
|
||||
graph: DynamicGraph::new(),
|
||||
features: NodeFeatures::new(dim, num_classes),
|
||||
condenser: GraphCondenser::new(config),
|
||||
cached: None,
|
||||
dirty: true,
|
||||
ops_since_rebuild: 0,
|
||||
rebuild_interval,
|
||||
}
|
||||
}
|
||||
|
||||
/// Number of vertices currently buffered.
|
||||
pub fn num_vertices(&self) -> usize {
|
||||
self.graph.num_vertices()
|
||||
}
|
||||
|
||||
/// Number of edges currently buffered.
|
||||
pub fn num_edges(&self) -> usize {
|
||||
self.graph.num_edges()
|
||||
}
|
||||
|
||||
/// Borrow the underlying graph (read-only).
|
||||
pub fn graph(&self) -> &DynamicGraph {
|
||||
&self.graph
|
||||
}
|
||||
|
||||
/// Set/replace the embedding (and optional label) for a vertex. Marks the
|
||||
/// condensed view dirty.
|
||||
///
|
||||
/// # Errors
|
||||
/// Propagates dimension validation from [`NodeFeatures`].
|
||||
pub fn upsert_feature(
|
||||
&mut self,
|
||||
vertex: VertexId,
|
||||
embedding: Vec<f32>,
|
||||
label: Option<usize>,
|
||||
) -> Result<()> {
|
||||
self.features.set_embedding(vertex, embedding)?;
|
||||
if let Some(l) = label {
|
||||
self.features.set_label(vertex, l);
|
||||
}
|
||||
self.touch();
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Insert an edge. Both endpoints must already have features (call
|
||||
/// [`StreamingCondenser::upsert_feature`] first) for a later condense to
|
||||
/// succeed. Duplicate edges are ignored (idempotent).
|
||||
pub fn insert_edge(&mut self, u: VertexId, v: VertexId, weight: Weight) {
|
||||
if self.graph.insert_edge(u, v, weight).is_ok() {
|
||||
self.touch();
|
||||
}
|
||||
}
|
||||
|
||||
/// Update an existing edge's weight (no-op if the edge is absent).
|
||||
pub fn update_edge(&mut self, u: VertexId, v: VertexId, weight: Weight) {
|
||||
if self.graph.update_edge_weight(u, v, weight).is_ok() {
|
||||
self.touch();
|
||||
}
|
||||
}
|
||||
|
||||
/// Delete an edge (no-op if absent).
|
||||
pub fn delete_edge(&mut self, u: VertexId, v: VertexId) {
|
||||
if self.graph.delete_edge(u, v).is_ok() {
|
||||
self.touch();
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the cached condensed view is stale.
|
||||
pub fn is_dirty(&self) -> bool {
|
||||
self.dirty
|
||||
}
|
||||
|
||||
/// Get the current condensed view, rebuilding if dirty (or if the rebuild
|
||||
/// interval has elapsed). Returns `None` only when the graph is empty.
|
||||
///
|
||||
/// # Errors
|
||||
/// Propagates condensation errors (e.g. a vertex missing its feature).
|
||||
pub fn condensed(&mut self) -> Result<Option<&CondensedGraph>> {
|
||||
if self.graph.num_vertices() == 0 {
|
||||
self.cached = None;
|
||||
self.dirty = false;
|
||||
return Ok(None);
|
||||
}
|
||||
let interval_elapsed =
|
||||
self.rebuild_interval > 0 && self.ops_since_rebuild >= self.rebuild_interval;
|
||||
if self.dirty || interval_elapsed || self.cached.is_none() {
|
||||
self.rebuild()?;
|
||||
}
|
||||
Ok(self.cached.as_ref())
|
||||
}
|
||||
|
||||
/// Force an immediate re-condensation regardless of dirty state.
|
||||
///
|
||||
/// # Errors
|
||||
/// Propagates condensation errors.
|
||||
pub fn rebuild(&mut self) -> Result<()> {
|
||||
let condensed = self.condenser.condense(&self.graph, &self.features)?;
|
||||
self.cached = Some(condensed);
|
||||
self.dirty = false;
|
||||
self.ops_since_rebuild = 0;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn touch(&mut self) {
|
||||
self.dirty = true;
|
||||
self.ops_since_rebuild += 1;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::condense::{CondenseConfig, CondenseMethod};
|
||||
|
||||
fn cfg() -> CondenseConfig {
|
||||
CondenseConfig {
|
||||
method: CondenseMethod::ConnectedComponents,
|
||||
normalize_centroids: false,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_returns_none() {
|
||||
let mut s = StreamingCondenser::new(cfg(), 1, 0, 0);
|
||||
assert!(s.condensed().unwrap().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn condenses_after_growth() {
|
||||
let mut s = StreamingCondenser::new(cfg(), 1, 0, 0);
|
||||
for v in 0..4u64 {
|
||||
s.upsert_feature(v, vec![v as f32], None).unwrap();
|
||||
}
|
||||
s.insert_edge(0, 1, 1.0);
|
||||
s.insert_edge(2, 3, 1.0);
|
||||
assert!(s.is_dirty());
|
||||
let c = s.condensed().unwrap().unwrap();
|
||||
// Two components -> two super-nodes.
|
||||
assert_eq!(c.node_count(), 2);
|
||||
assert!(!s.is_dirty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn caches_until_mutated() {
|
||||
let mut s = StreamingCondenser::new(cfg(), 1, 0, 0);
|
||||
s.upsert_feature(0, vec![0.0], None).unwrap();
|
||||
s.upsert_feature(1, vec![1.0], None).unwrap();
|
||||
s.insert_edge(0, 1, 1.0);
|
||||
let n1 = s.condensed().unwrap().unwrap().node_count();
|
||||
assert_eq!(n1, 1);
|
||||
assert!(!s.is_dirty());
|
||||
// Reading again without mutation does not re-dirty.
|
||||
let _ = s.condensed().unwrap();
|
||||
assert!(!s.is_dirty());
|
||||
|
||||
// A new disconnected vertex+edge splits into a second component.
|
||||
s.upsert_feature(2, vec![2.0], None).unwrap();
|
||||
s.upsert_feature(3, vec![3.0], None).unwrap();
|
||||
s.insert_edge(2, 3, 1.0);
|
||||
assert!(s.is_dirty());
|
||||
assert_eq!(s.condensed().unwrap().unwrap().node_count(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interval_forces_rebuild_path() {
|
||||
// rebuild_interval=1 exercises the interval branch; result stays correct.
|
||||
let mut s = StreamingCondenser::new(cfg(), 1, 0, 1);
|
||||
s.upsert_feature(0, vec![0.0], None).unwrap();
|
||||
s.upsert_feature(1, vec![1.0], None).unwrap();
|
||||
s.insert_edge(0, 1, 1.0);
|
||||
assert_eq!(s.condensed().unwrap().unwrap().node_count(), 1);
|
||||
}
|
||||
}
|
||||
155
crates/ruvector-graph-condense/src/synthetic.rs
Normal file
155
crates/ruvector-graph-condense/src/synthetic.rs
Normal file
|
|
@ -0,0 +1,155 @@
|
|||
//! Synthetic planted-partition graphs for testing, benchmarking, and demos.
|
||||
//!
|
||||
//! Produces a graph with `num_communities` ground-truth communities: dense,
|
||||
//! heavy intra-community edges and sparse, light inter-community edges, with
|
||||
//! each community's embeddings drawn around a distinct centroid and sharing a
|
||||
//! class label. This is the canonical stress test for a structure-preserving
|
||||
//! condenser — a good condenser should recover the planted communities.
|
||||
|
||||
use crate::features::NodeFeatures;
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
/// Parameters for a planted-partition (stochastic-block-model-style) graph.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PlantedPartition {
|
||||
/// Number of ground-truth communities.
|
||||
pub num_communities: usize,
|
||||
/// Vertices per community.
|
||||
pub community_size: usize,
|
||||
/// Embedding dimension.
|
||||
pub dim: usize,
|
||||
/// Probability of an edge between two vertices in the same community.
|
||||
pub p_intra: f64,
|
||||
/// Probability of an edge between two vertices in different communities.
|
||||
pub p_inter: f64,
|
||||
/// Weight assigned to intra-community edges.
|
||||
pub w_intra: f64,
|
||||
/// Weight assigned to inter-community edges.
|
||||
pub w_inter: f64,
|
||||
/// RNG seed for reproducibility.
|
||||
pub seed: u64,
|
||||
}
|
||||
|
||||
impl Default for PlantedPartition {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
num_communities: 8,
|
||||
community_size: 32,
|
||||
dim: 16,
|
||||
p_intra: 0.4,
|
||||
p_inter: 0.002,
|
||||
w_intra: 1.0,
|
||||
w_inter: 0.1,
|
||||
seed: 0xC0FFEE,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl PlantedPartition {
|
||||
/// Total vertex count.
|
||||
pub fn total_vertices(&self) -> usize {
|
||||
self.num_communities * self.community_size
|
||||
}
|
||||
|
||||
/// Generate the graph and matching [`NodeFeatures`].
|
||||
///
|
||||
/// Vertices are numbered `0..total_vertices`; community `c` owns the
|
||||
/// contiguous block `[c*size, (c+1)*size)`. Every vertex receives an
|
||||
/// embedding (so condensation never hits a missing feature) clustered
|
||||
/// around its community centroid, plus that community's class label.
|
||||
pub fn generate(&self) -> (DynamicGraph, NodeFeatures) {
|
||||
let mut rng = StdRng::seed_from_u64(self.seed);
|
||||
let n = self.total_vertices();
|
||||
let graph = DynamicGraph::with_capacity(n, n * 4);
|
||||
let mut features = NodeFeatures::new(self.dim, self.num_communities);
|
||||
|
||||
// Community centroids spaced far apart so feature space mirrors topology.
|
||||
let centroids: Vec<Vec<f32>> = (0..self.num_communities)
|
||||
.map(|c| {
|
||||
let mut v = vec![0f32; self.dim];
|
||||
v[c % self.dim] = 10.0 * (c / self.dim + 1) as f32;
|
||||
v
|
||||
})
|
||||
.collect();
|
||||
|
||||
for (c, centroid) in centroids.iter().enumerate() {
|
||||
for i in 0..self.community_size {
|
||||
let vid = (c * self.community_size + i) as u64;
|
||||
let mut emb = centroid.clone();
|
||||
for x in &mut emb {
|
||||
*x += rng.gen_range(-1.0..1.0);
|
||||
}
|
||||
// set() only fails on dimension mismatch, which cannot happen here.
|
||||
let _ = features.set(vid, emb, c);
|
||||
graph.add_vertex(vid);
|
||||
}
|
||||
}
|
||||
|
||||
// Edges. insert_edge dedups and rejects self-loops, so collisions are
|
||||
// simply skipped.
|
||||
for a in 0..n {
|
||||
for b in (a + 1)..n {
|
||||
let same = a / self.community_size == b / self.community_size;
|
||||
let (p, w) = if same {
|
||||
(self.p_intra, self.w_intra)
|
||||
} else {
|
||||
(self.p_inter, self.w_inter)
|
||||
};
|
||||
if rng.gen_bool(p) {
|
||||
let _ = graph.insert_edge(a as u64, b as u64, w);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
(graph, features)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::condense::condense;
|
||||
|
||||
#[test]
|
||||
fn generates_requested_size() {
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 4,
|
||||
community_size: 10,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, f) = pp.generate();
|
||||
assert_eq!(g.num_vertices(), 40);
|
||||
assert_eq!(f.len(), 40);
|
||||
assert_eq!(f.num_classes(), 4);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn condenser_recovers_planted_structure() {
|
||||
// Strong planted structure should condense to roughly the planted count
|
||||
// and keep most weight intra-region.
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 4,
|
||||
community_size: 24,
|
||||
dim: 8,
|
||||
p_intra: 0.6,
|
||||
p_inter: 0.001,
|
||||
seed: 7,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, f) = pp.generate();
|
||||
let c = condense(&g, &f).unwrap();
|
||||
assert_eq!(c.source_nodes, 96);
|
||||
// Recursive min-cut can over-split; expect at least the planted count
|
||||
// and a strong reduction.
|
||||
assert!(c.node_count() >= 4);
|
||||
assert!(c.node_reduction_ratio() > 2.0);
|
||||
let m = crate::metrics::evaluate(&g, &c);
|
||||
assert!(
|
||||
m.intra_weight_ratio > 0.8,
|
||||
"intra ratio {}",
|
||||
m.intra_weight_ratio
|
||||
);
|
||||
}
|
||||
}
|
||||
141
crates/ruvector-graph-condense/tests/accuracy.rs
Normal file
141
crates/ruvector-graph-condense/tests/accuracy.rs
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
//! End-to-end accuracy-retention test: a GNN trained on the condensed graph must
|
||||
//! classify the original graph's held-out nodes nearly as well as one trained on
|
||||
//! the full graph. This is the graph-condensation field's core success metric.
|
||||
#![allow(clippy::needless_range_loop)] // index-heavy numeric test code
|
||||
|
||||
use rand::rngs::StdRng;
|
||||
use rand::{Rng, SeedableRng};
|
||||
use ruvector_graph_condense::gnn_eval::{accuracy, Gcn, GcnConfig, GcnGraph};
|
||||
use ruvector_graph_condense::{
|
||||
CondenseConfig, CondenseMethod, CondensedGraph, DiffCutConfig, GraphCondenser, NodeFeatures,
|
||||
};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
fn gen(
|
||||
classes: usize,
|
||||
per_class: usize,
|
||||
dim: usize,
|
||||
noise: f64,
|
||||
seed: u64,
|
||||
) -> (DynamicGraph, NodeFeatures, Vec<usize>, usize) {
|
||||
let n = classes * per_class;
|
||||
let mut rng = StdRng::seed_from_u64(seed);
|
||||
let g = DynamicGraph::new();
|
||||
let mut f = NodeFeatures::new(dim, classes);
|
||||
let mut labels = vec![0usize; n];
|
||||
for i in 0..n {
|
||||
let cls = i / per_class;
|
||||
labels[i] = cls;
|
||||
let emb: Vec<f32> = (0..dim)
|
||||
.map(|d| {
|
||||
let base = if d % classes == cls { 1.5 } else { 0.0 };
|
||||
(base + noise * rng.gen_range(-1.0..1.0)) as f32
|
||||
})
|
||||
.collect();
|
||||
f.set(i as u64, emb, cls).unwrap();
|
||||
g.add_vertex(i as u64);
|
||||
}
|
||||
for a in 0..n {
|
||||
for b in (a + 1)..n {
|
||||
let same = a / per_class == b / per_class;
|
||||
let p = if same { 0.15 } else { 0.005 };
|
||||
if rng.gen_bool(p) {
|
||||
let _ = g.insert_edge(a as u64, b as u64, 1.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
(g, f, labels, n)
|
||||
}
|
||||
|
||||
fn full_arrays(g: &DynamicGraph, f: &NodeFeatures, n: usize) -> (GcnGraph, Vec<f64>) {
|
||||
let edges: Vec<(usize, usize, f64)> = g
|
||||
.edges()
|
||||
.iter()
|
||||
.map(|e| (e.source as usize, e.target as usize, e.weight))
|
||||
.collect();
|
||||
let dim = f.dim();
|
||||
let mut x = vec![0f64; n * dim];
|
||||
for i in 0..n {
|
||||
if let Some(emb) = f.embedding(i as u64) {
|
||||
for d in 0..dim {
|
||||
x[i * dim + d] = emb[d] as f64;
|
||||
}
|
||||
}
|
||||
}
|
||||
(GcnGraph::from_edges(n, &edges), x)
|
||||
}
|
||||
|
||||
fn condensed_arrays(c: &CondensedGraph) -> (GcnGraph, Vec<f64>, Vec<usize>) {
|
||||
let (cn, dim) = (c.node_count(), c.dim);
|
||||
let mut x = vec![0f64; cn * dim];
|
||||
let mut labels = vec![0usize; cn];
|
||||
for (i, node) in c.nodes.iter().enumerate() {
|
||||
for d in 0..dim {
|
||||
x[i * dim + d] = node.centroid[d] as f64;
|
||||
}
|
||||
labels[i] = node.dominant_class().unwrap_or(0);
|
||||
}
|
||||
let edges: Vec<(usize, usize, f64)> = c
|
||||
.edges
|
||||
.iter()
|
||||
.map(|e| (e.source as usize, e.target as usize, e.weight))
|
||||
.collect();
|
||||
(GcnGraph::from_edges(cn, &edges), x, labels)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn condensed_graph_trains_a_usable_classifier() {
|
||||
let classes = 3;
|
||||
let (g, f, labels, n) = gen(classes, 24, 12, 1.2, 2026);
|
||||
let (full, x_full) = full_arrays(&g, &f, n);
|
||||
|
||||
// Train/test split.
|
||||
let mut rng = StdRng::seed_from_u64(7);
|
||||
let (mut train, mut test) = (Vec::new(), Vec::new());
|
||||
for i in 0..n {
|
||||
if rng.gen_bool(0.6) {
|
||||
train.push(i);
|
||||
} else {
|
||||
test.push(i);
|
||||
}
|
||||
}
|
||||
|
||||
let cfg = GcnConfig {
|
||||
epochs: 150,
|
||||
..Default::default()
|
||||
};
|
||||
let base = Gcn::train(&cfg, &full, &x_full, f.dim(), &labels, classes, &train);
|
||||
let acc_full = accuracy(&base.predict(&full, &x_full), &labels, &test);
|
||||
assert!(
|
||||
acc_full > 0.7,
|
||||
"baseline too weak to be a fair test: {acc_full}"
|
||||
);
|
||||
|
||||
// Condense (DiffMinCut, a few super-nodes per class) and train on it.
|
||||
let c = GraphCondenser::new(CondenseConfig {
|
||||
method: CondenseMethod::DiffMinCut(DiffCutConfig {
|
||||
num_clusters: classes * 3,
|
||||
restarts: 2,
|
||||
iterations: 300,
|
||||
..Default::default()
|
||||
}),
|
||||
normalize_centroids: false,
|
||||
})
|
||||
.condense(&g, &f)
|
||||
.unwrap();
|
||||
let (cg, x_cond, lab_cond) = condensed_arrays(&c);
|
||||
let all: Vec<usize> = (0..c.node_count()).collect();
|
||||
let model = Gcn::train(&cfg, &cg, &x_cond, f.dim(), &lab_cond, classes, &all);
|
||||
let acc_cond = accuracy(&model.predict(&full, &x_full), &labels, &test);
|
||||
|
||||
let retention = acc_cond / acc_full;
|
||||
assert!(
|
||||
c.node_count() < n / 4,
|
||||
"expected real reduction, got {} of {n}",
|
||||
c.node_count()
|
||||
);
|
||||
assert!(
|
||||
retention > 0.8,
|
||||
"retention too low: cond {acc_cond:.3} / full {acc_full:.3} = {retention:.3}"
|
||||
);
|
||||
}
|
||||
285
crates/ruvector-graph-condense/tests/diffcut.rs
Normal file
285
crates/ruvector-graph-condense/tests/diffcut.rs
Normal file
|
|
@ -0,0 +1,285 @@
|
|||
//! Public-API integration tests for the differentiable min-cut condenser.
|
||||
//! (Internal gradient-check / maths tests live in the `diffcut` module itself.)
|
||||
|
||||
use ruvector_graph_condense::{
|
||||
CondenseError, DiffCutCondenser, DiffCutConfig, InitStrategy, Optimizer, PlantedPartition,
|
||||
};
|
||||
use ruvector_mincut::DynamicGraph;
|
||||
|
||||
fn barbell() -> DynamicGraph {
|
||||
let g = DynamicGraph::new();
|
||||
for &(u, v, w) in &[
|
||||
(0, 1, 1.0),
|
||||
(1, 2, 1.0),
|
||||
(2, 0, 1.0),
|
||||
(3, 4, 1.0),
|
||||
(4, 5, 1.0),
|
||||
(5, 3, 1.0),
|
||||
(2, 3, 0.05),
|
||||
] {
|
||||
g.insert_edge(u, v, w).unwrap();
|
||||
}
|
||||
g
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn loss_decreases_during_training() {
|
||||
// From a *random* start with SGD, training must reduce the loss (a clean
|
||||
// descent test, independent of the warm-start prior).
|
||||
let g = barbell();
|
||||
let base = DiffCutConfig {
|
||||
num_clusters: 2,
|
||||
learning_rate: 0.3,
|
||||
init: InitStrategy::Random,
|
||||
optimizer: Optimizer::Sgd { momentum: 0.0 },
|
||||
iterations: 1,
|
||||
seed: 7,
|
||||
..Default::default()
|
||||
};
|
||||
let early = DiffCutCondenser::new(base.clone())
|
||||
.train(&g)
|
||||
.unwrap()
|
||||
.loss();
|
||||
let late = DiffCutCondenser::new(DiffCutConfig {
|
||||
iterations: 300,
|
||||
..base
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap()
|
||||
.loss();
|
||||
assert!(
|
||||
late.total < early.total,
|
||||
"training did not reduce loss: {} -> {}",
|
||||
early.total,
|
||||
late.total
|
||||
);
|
||||
assert!(late.cut < -0.7, "cut term {} not minimised", late.cut);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recovers_barbell_partition() {
|
||||
let g = barbell();
|
||||
let res = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 2,
|
||||
..Default::default()
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
let mut regions = res.hard_regions();
|
||||
for r in &mut regions {
|
||||
r.sort_unstable();
|
||||
}
|
||||
regions.sort_by_key(|r| r[0]);
|
||||
assert_eq!(regions, vec![vec![0, 1, 2], vec![3, 4, 5]]);
|
||||
}
|
||||
|
||||
/// Weighted dominant-class purity of a hard assignment vs. ground-truth
|
||||
/// communities (vertex `v` belongs to community `v / community_size`).
|
||||
fn purity(regions: &[Vec<u64>], community_size: u64) -> f64 {
|
||||
let mut correct = 0u64;
|
||||
let mut total = 0u64;
|
||||
for r in regions {
|
||||
let mut counts: std::collections::HashMap<u64, u64> = std::collections::HashMap::new();
|
||||
for &v in r {
|
||||
*counts.entry(v / community_size).or_default() += 1;
|
||||
}
|
||||
correct += counts.values().copied().max().unwrap_or(0);
|
||||
total += r.len() as u64;
|
||||
}
|
||||
correct as f64 / total.max(1) as f64
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_recovers_many_clusters() {
|
||||
// The headline "works on big problems" test: K = 8 on 8 planted communities.
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 8,
|
||||
community_size: 24,
|
||||
dim: 8,
|
||||
p_intra: 0.5,
|
||||
p_inter: 0.002,
|
||||
seed: 3,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, _f) = pp.generate();
|
||||
let res = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 8,
|
||||
..Default::default() // Adam + warm-start
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
let pur = purity(&res.hard_regions(), pp.community_size as u64);
|
||||
assert!(pur > 0.85, "warm-start purity at K=8 too low: {pur}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn warm_start_beats_random_at_large_k() {
|
||||
// Same graph, same budget: warm-start should reach a lower (better) loss
|
||||
// than random init at large K — the whole point of the optimisation work.
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 8,
|
||||
community_size: 20,
|
||||
dim: 8,
|
||||
p_intra: 0.5,
|
||||
p_inter: 0.002,
|
||||
seed: 11,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, _f) = pp.generate();
|
||||
let common = DiffCutConfig {
|
||||
num_clusters: 8,
|
||||
iterations: 200,
|
||||
seed: 1,
|
||||
..Default::default()
|
||||
};
|
||||
let warm = DiffCutCondenser::new(common.clone()).train(&g).unwrap();
|
||||
let rand = DiffCutCondenser::new(DiffCutConfig {
|
||||
init: InitStrategy::Random,
|
||||
..common
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
assert!(
|
||||
warm.loss().total <= rand.loss().total,
|
||||
"warm-start ({}) not better than random ({})",
|
||||
warm.loss().total,
|
||||
rand.loss().total
|
||||
);
|
||||
let pur_warm = purity(&warm.hard_regions(), pp.community_size as u64);
|
||||
let pur_rand = purity(&rand.hard_regions(), pp.community_size as u64);
|
||||
assert!(
|
||||
pur_warm >= pur_rand,
|
||||
"warm purity {pur_warm} < random purity {pur_rand}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn determinism_same_seed_same_result() {
|
||||
let g = barbell();
|
||||
let cfg = DiffCutConfig {
|
||||
num_clusters: 2,
|
||||
iterations: 200,
|
||||
seed: 5,
|
||||
..Default::default()
|
||||
};
|
||||
let a = DiffCutCondenser::new(cfg.clone()).train(&g).unwrap();
|
||||
let b = DiffCutCondenser::new(cfg).train(&g).unwrap();
|
||||
assert_eq!(a.soft_assignment(), b.soft_assignment());
|
||||
assert_eq!(a.loss(), b.loss());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parallel_matches_sequential_exactly() {
|
||||
// Row-parallel A·S is deterministic, so parallel must equal sequential
|
||||
// bit-for-bit (same seed, same config otherwise).
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 6,
|
||||
community_size: 24,
|
||||
dim: 8,
|
||||
seed: 4,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, _f) = pp.generate();
|
||||
let base = DiffCutConfig {
|
||||
num_clusters: 6,
|
||||
iterations: 120,
|
||||
seed: 2,
|
||||
tolerance: 0.0, // disable early-stop so both run identical iterations
|
||||
..Default::default()
|
||||
};
|
||||
let seq = DiffCutCondenser::new(base.clone()).train(&g).unwrap();
|
||||
let par = DiffCutCondenser::new(DiffCutConfig {
|
||||
parallel: true,
|
||||
..base
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
assert_eq!(seq.soft_assignment(), par.soft_assignment());
|
||||
assert_eq!(seq.loss(), par.loss());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn minibatch_recovers_structure() {
|
||||
// Stochastic edge-minibatch should still recover the planted communities
|
||||
// (warm-start prior + refinement), at a fraction of the per-step edge cost.
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 6,
|
||||
community_size: 24,
|
||||
dim: 8,
|
||||
p_intra: 0.5,
|
||||
p_inter: 0.002,
|
||||
seed: 9,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, _f) = pp.generate();
|
||||
let res = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 6,
|
||||
minibatch_edges: Some(256),
|
||||
iterations: 150,
|
||||
seed: 1,
|
||||
..Default::default()
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
let pur = purity(&res.hard_regions(), pp.community_size as u64);
|
||||
assert!(pur > 0.8, "minibatch purity too low: {pur}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn early_stopping_cuts_iterations() {
|
||||
// Warm-start lands near the optimum, so early-stop should finish well under
|
||||
// the iteration cap.
|
||||
let pp = PlantedPartition {
|
||||
num_communities: 6,
|
||||
community_size: 20,
|
||||
dim: 8,
|
||||
seed: 6,
|
||||
..Default::default()
|
||||
};
|
||||
let (g, _f) = pp.generate();
|
||||
let res = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 6,
|
||||
iterations: 1000,
|
||||
tolerance: 1e-4,
|
||||
seed: 1,
|
||||
..Default::default()
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap();
|
||||
assert!(
|
||||
res.iterations_run() < 1000,
|
||||
"early-stop did not trigger: {}",
|
||||
res.iterations_run()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_graph_errors() {
|
||||
let g = DynamicGraph::new();
|
||||
assert!(matches!(
|
||||
DiffCutCondenser::new(DiffCutConfig::default())
|
||||
.train(&g)
|
||||
.unwrap_err(),
|
||||
CondenseError::EmptyGraph
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn zero_clusters_errors() {
|
||||
let g = barbell();
|
||||
let err = DiffCutCondenser::new(DiffCutConfig {
|
||||
num_clusters: 0,
|
||||
..Default::default()
|
||||
})
|
||||
.train(&g)
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, CondenseError::InvalidConfig(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn public_min_cut_loss_dimension_check() {
|
||||
use ruvector_graph_condense::min_cut_loss;
|
||||
let g = barbell();
|
||||
let err = min_cut_loss(&g, &[0.5; 3], 2, 1.0).unwrap_err();
|
||||
assert!(matches!(err, CondenseError::DimensionMismatch { .. }));
|
||||
}
|
||||
27
crates/ruvector-perception/Cargo.toml
Normal file
27
crates/ruvector-perception/Cargo.toml
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
[package]
|
||||
name = "ruvector-perception"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
authors.workspace = true
|
||||
repository.workspace = true
|
||||
description = "The layer under classification: physical delta -> boundary -> coherence -> proof -> action. A trusted-physical-memory engine that emits structured delta witnesses, not class labels."
|
||||
keywords = ["sensing", "coherence", "min-cut", "edge-ai", "anomaly"]
|
||||
categories = ["algorithms", "science"]
|
||||
|
||||
[lib]
|
||||
crate-type = ["rlib"]
|
||||
|
||||
[dependencies]
|
||||
# Boundary detection reuses the dynamic min-cut engine.
|
||||
ruvector-mincut = { version = "2.2.3", path = "../ruvector-mincut", default-features = false }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
thiserror = { workspace = true }
|
||||
sha2 = "0.10"
|
||||
|
||||
[dev-dependencies]
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
129
crates/ruvector-perception/src/absence.rs
Normal file
129
crates/ruvector-perception/src/absence.rs
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
//! Contrastive absence sensing: detect a *missing* expected continuation as a
|
||||
//! structured signal, not a threshold alert. The expected temporal pattern is a
|
||||
//! sequence of zone events (e.g. `bed_exit → bathroom_path → return_path`); when
|
||||
//! a continuation edge never arrives within its deadline, the sequence graph is
|
||||
//! left incomplete — that incompleteness is the signal.
|
||||
|
||||
/// A structured absence: an expected next step that did not occur in time.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Absence {
|
||||
/// The step that was expected but never arrived.
|
||||
pub missing_step: String,
|
||||
/// The last step that *did* occur.
|
||||
pub after: String,
|
||||
/// How long we have waited past the last observed step.
|
||||
pub elapsed: u64,
|
||||
}
|
||||
|
||||
/// Monitors progress through an expected sequence and flags missing
|
||||
/// continuations.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SequenceMonitor {
|
||||
steps: Vec<String>,
|
||||
deadline: u64,
|
||||
pos: usize,
|
||||
last_t: Option<u64>,
|
||||
started: bool,
|
||||
}
|
||||
|
||||
impl SequenceMonitor {
|
||||
/// New monitor for an ordered list of expected zone events, with a
|
||||
/// per-step deadline (in the same time units as observations).
|
||||
pub fn new(steps: Vec<String>, deadline: u64) -> Self {
|
||||
Self {
|
||||
steps,
|
||||
deadline,
|
||||
pos: 0,
|
||||
last_t: None,
|
||||
started: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the full sequence has completed.
|
||||
pub fn complete(&self) -> bool {
|
||||
self.pos >= self.steps.len()
|
||||
}
|
||||
|
||||
/// Record that an event happened in `zone` at time `t`. Advances the
|
||||
/// sequence if it matches the next expected step.
|
||||
pub fn observe_zone(&mut self, zone: &str, t: u64) {
|
||||
if self.complete() {
|
||||
return;
|
||||
}
|
||||
if self.steps[self.pos] == zone {
|
||||
self.pos += 1;
|
||||
self.last_t = Some(t);
|
||||
self.started = true;
|
||||
}
|
||||
}
|
||||
|
||||
/// Check for a missing continuation as of `now`. Returns an [`Absence`] if
|
||||
/// the sequence has started, is not complete, and the next step is overdue.
|
||||
pub fn check(&self, now: u64) -> Option<Absence> {
|
||||
if !self.started || self.complete() {
|
||||
return None;
|
||||
}
|
||||
let last = self.last_t?;
|
||||
let elapsed = now.saturating_sub(last);
|
||||
if elapsed > self.deadline {
|
||||
Some(Absence {
|
||||
missing_step: self.steps[self.pos].clone(),
|
||||
after: self.steps[self.pos - 1].clone(),
|
||||
elapsed,
|
||||
})
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Reset to the start (e.g. for a new day/cycle).
|
||||
pub fn reset(&mut self) {
|
||||
self.pos = 0;
|
||||
self.last_t = None;
|
||||
self.started = false;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn routine() -> SequenceMonitor {
|
||||
SequenceMonitor::new(
|
||||
vec![
|
||||
"bed_exit".to_string(),
|
||||
"bathroom_path".to_string(),
|
||||
"return_path".to_string(),
|
||||
],
|
||||
100,
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_return_is_flagged() {
|
||||
let mut m = routine();
|
||||
m.observe_zone("bed_exit", 0);
|
||||
m.observe_zone("bathroom_path", 10);
|
||||
assert!(m.check(50).is_none()); // still within deadline
|
||||
let a = m.check(200).expect("overdue return");
|
||||
assert_eq!(a.missing_step, "return_path");
|
||||
assert_eq!(a.after, "bathroom_path");
|
||||
assert!(a.elapsed > 100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn completed_routine_is_silent() {
|
||||
let mut m = routine();
|
||||
m.observe_zone("bed_exit", 0);
|
||||
m.observe_zone("bathroom_path", 10);
|
||||
m.observe_zone("return_path", 20);
|
||||
assert!(m.complete());
|
||||
assert!(m.check(10_000).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unstarted_routine_is_silent() {
|
||||
let m = routine();
|
||||
assert!(m.check(10_000).is_none());
|
||||
}
|
||||
}
|
||||
400
crates/ruvector-perception/src/captcha.rs
Normal file
400
crates/ruvector-perception/src/captcha.rs
Normal file
|
|
@ -0,0 +1,400 @@
|
|||
//! Physical CAPTCHA — proof-of-reality via active challenge–response.
|
||||
//!
|
||||
//! A replayed or statically-spoofed sensor stream can mimic *passive* readings,
|
||||
//! but it cannot answer a *fresh* physical challenge. This module models that
|
||||
//! interaction: the device emits a [`Stimulus`] (a chirp, an RF pulse, a tap,
|
||||
//! …) and expects a multi-modal [`ObservedResponse`] with characteristic
|
||||
//! per-modality delays and magnitudes.
|
||||
//!
|
||||
//! [`CaptchaVerifier`] learns the expected response *profile* for each stimulus
|
||||
//! from known-good challenges (an EWMA over delay and magnitude per modality),
|
||||
//! then [`CaptchaVerifier::verify`] scores a fresh observation against it. The
|
||||
//! score is weighted by each modality's [`spoof_resistance`]: a missing
|
||||
//! hard-to-fake modality (e.g. vibration or thermal) costs far more than a
|
||||
//! missing easy-to-fake one (e.g. RF or optical).
|
||||
//!
|
||||
//! [`spoof_resistance`]: crate::modality::Physics::spoof_resistance
|
||||
|
||||
use crate::modality::Modality;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// EWMA smoothing factor for profile learning. Newer observations get weight
|
||||
/// `ALPHA`, the running estimate keeps `1 - ALPHA`.
|
||||
const ALPHA: f32 = 0.3;
|
||||
|
||||
/// Minimum fraction of the expected magnitude an observed response must reach to
|
||||
/// count as a valid (non-spoofed, non-attenuated) reply.
|
||||
const MAGNITUDE_FLOOR_FRACTION: f32 = 0.5;
|
||||
|
||||
/// An active physical challenge emitted by the device.
|
||||
///
|
||||
/// Each variant maps to a distinct emission whose echoes/responses propagate
|
||||
/// across several [`Modality`]s with modality-specific delays.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
pub enum Stimulus {
|
||||
/// A swept-frequency acoustic tone ("chirp").
|
||||
AcousticChirp,
|
||||
/// A short radio-frequency burst.
|
||||
RfPulse,
|
||||
/// A mechanical tap exciting structural vibration.
|
||||
VibrationTap,
|
||||
/// A modulated-light flash.
|
||||
LightModulation,
|
||||
/// A brief thermal/IR pulse.
|
||||
ThermalPulse,
|
||||
}
|
||||
|
||||
impl Stimulus {
|
||||
/// Stable short name (useful for logs and witnesses).
|
||||
pub fn name(self) -> &'static str {
|
||||
match self {
|
||||
Stimulus::AcousticChirp => "acoustic-chirp",
|
||||
Stimulus::RfPulse => "rf-pulse",
|
||||
Stimulus::VibrationTap => "vibration-tap",
|
||||
Stimulus::LightModulation => "light-modulation",
|
||||
Stimulus::ThermalPulse => "thermal-pulse",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One observed response on a single modality to an emitted [`Stimulus`].
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct ObservedResponse {
|
||||
/// Which modality this reading came from.
|
||||
pub modality: Modality,
|
||||
/// Time from stimulus emission to observed response (seconds).
|
||||
pub delay: f32,
|
||||
/// Response magnitude in arbitrary, modality-normalised units (`>= 0`).
|
||||
pub magnitude: f32,
|
||||
}
|
||||
|
||||
/// A complete challenge–response record: one stimulus, many modality responses.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ChallengeResponse {
|
||||
/// The stimulus that was emitted.
|
||||
pub stimulus: Stimulus,
|
||||
/// All observed responses (at most one expected per modality).
|
||||
pub responses: Vec<ObservedResponse>,
|
||||
}
|
||||
|
||||
/// The verdict produced by [`CaptchaVerifier::verify`].
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct RealityProof {
|
||||
/// `true` when the weighted score met the verifier's `min_score`.
|
||||
pub trusted: bool,
|
||||
/// Weighted fraction of expected modalities that responded correctly, in `[0, 1]`.
|
||||
pub score: f32,
|
||||
/// Expected modalities that were missing or out of tolerance.
|
||||
pub missing: Vec<Modality>,
|
||||
/// Human-readable explanation of the verdict.
|
||||
pub reason: String,
|
||||
}
|
||||
|
||||
/// Expected per-modality response, maintained as a running EWMA.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
struct Expected {
|
||||
delay: f32,
|
||||
magnitude: f32,
|
||||
}
|
||||
|
||||
/// Learns expected challenge-response profiles and verifies fresh observations
|
||||
/// against them.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CaptchaVerifier {
|
||||
/// Per `(Stimulus, Modality)` expected response, learned via EWMA.
|
||||
profiles: HashMap<(Stimulus, Modality), Expected>,
|
||||
/// Allowed absolute deviation in delay (seconds) for a response to count.
|
||||
delay_tolerance: f32,
|
||||
/// Minimum weighted score to mark a proof as `trusted`.
|
||||
min_score: f32,
|
||||
}
|
||||
|
||||
impl CaptchaVerifier {
|
||||
/// Create a verifier.
|
||||
///
|
||||
/// * `delay_tolerance` — max absolute delay error (seconds) tolerated per modality.
|
||||
/// * `min_score` — weighted score threshold (`[0, 1]`) for `trusted`.
|
||||
pub fn new(delay_tolerance: f32, min_score: f32) -> Self {
|
||||
Self {
|
||||
profiles: HashMap::new(),
|
||||
delay_tolerance,
|
||||
min_score,
|
||||
}
|
||||
}
|
||||
|
||||
/// Learn (or update) the expected response profile for a stimulus from a
|
||||
/// known-good challenge response.
|
||||
///
|
||||
/// For every observed modality the expected `{delay, magnitude}` is folded
|
||||
/// in with an EWMA (`ALPHA`). The first time a `(stimulus, modality)` pair
|
||||
/// is seen it is initialised directly to the observed values.
|
||||
pub fn learn(&mut self, cr: &ChallengeResponse) {
|
||||
for r in &cr.responses {
|
||||
let key = (cr.stimulus, r.modality);
|
||||
self.profiles
|
||||
.entry(key)
|
||||
.and_modify(|e| {
|
||||
e.delay = ewma(e.delay, r.delay);
|
||||
e.magnitude = ewma(e.magnitude, r.magnitude);
|
||||
})
|
||||
.or_insert(Expected {
|
||||
delay: r.delay,
|
||||
magnitude: r.magnitude,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Verify an observed challenge response against the learned profile.
|
||||
///
|
||||
/// Each expected modality contributes weight equal to its
|
||||
/// [`spoof_resistance`]; a modality passes only when an observed response
|
||||
/// exists whose delay is within `delay_tolerance` and whose magnitude is at
|
||||
/// least [`MAGNITUDE_FLOOR_FRACTION`] of the expected magnitude. The
|
||||
/// returned `score` is `passed_weight / total_weight`.
|
||||
///
|
||||
/// [`spoof_resistance`]: crate::modality::Physics::spoof_resistance
|
||||
pub fn verify(&self, cr: &ChallengeResponse) -> RealityProof {
|
||||
// Gather everything we expect for this stimulus.
|
||||
let expected: Vec<(Modality, Expected)> = Modality::ALL
|
||||
.iter()
|
||||
.filter_map(|&m| self.profiles.get(&(cr.stimulus, m)).map(|e| (m, *e)))
|
||||
.collect();
|
||||
|
||||
if expected.is_empty() {
|
||||
return RealityProof {
|
||||
trusted: false,
|
||||
score: 0.0,
|
||||
missing: Vec::new(),
|
||||
reason: "unknown stimulus profile".to_string(),
|
||||
};
|
||||
}
|
||||
|
||||
let mut total_weight = 0.0_f32;
|
||||
let mut passed_weight = 0.0_f32;
|
||||
let mut missing: Vec<Modality> = Vec::new();
|
||||
|
||||
for (modality, exp) in &expected {
|
||||
let weight = modality.physics().spoof_resistance;
|
||||
total_weight += weight;
|
||||
|
||||
if self.responded_correctly(cr, *modality, exp) {
|
||||
passed_weight += weight;
|
||||
} else {
|
||||
missing.push(*modality);
|
||||
}
|
||||
}
|
||||
|
||||
// `total_weight` is > 0 because `expected` is non-empty and every
|
||||
// modality has a positive spoof_resistance.
|
||||
let score = passed_weight / total_weight;
|
||||
let trusted = score >= self.min_score;
|
||||
let reason = self.explain(trusted, &missing);
|
||||
|
||||
RealityProof {
|
||||
trusted,
|
||||
score,
|
||||
missing,
|
||||
reason,
|
||||
}
|
||||
}
|
||||
|
||||
/// Does `cr` contain a correct response for `modality` matching `exp`?
|
||||
fn responded_correctly(
|
||||
&self,
|
||||
cr: &ChallengeResponse,
|
||||
modality: Modality,
|
||||
exp: &Expected,
|
||||
) -> bool {
|
||||
let floor = exp.magnitude * MAGNITUDE_FLOOR_FRACTION;
|
||||
cr.responses.iter().any(|r| {
|
||||
r.modality == modality
|
||||
&& (r.delay - exp.delay).abs() <= self.delay_tolerance
|
||||
&& r.magnitude >= floor
|
||||
})
|
||||
}
|
||||
|
||||
/// Build a human-readable reason for the verdict.
|
||||
fn explain(&self, trusted: bool, missing: &[Modality]) -> String {
|
||||
if missing.is_empty() {
|
||||
return "all expected modalities responded within tolerance".to_string();
|
||||
}
|
||||
|
||||
// Surface the hardest-to-spoof missing modality first — that is the one
|
||||
// an attacker is least likely to fake.
|
||||
let worst = missing
|
||||
.iter()
|
||||
.copied()
|
||||
.max_by(|a, b| {
|
||||
a.physics()
|
||||
.spoof_resistance
|
||||
.partial_cmp(&b.physics().spoof_resistance)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
})
|
||||
.expect("missing is non-empty here");
|
||||
|
||||
let names: Vec<&str> = missing.iter().map(|m| m.name()).collect();
|
||||
let prefix = if trusted {
|
||||
"degraded but trusted"
|
||||
} else {
|
||||
"rejected"
|
||||
};
|
||||
format!(
|
||||
"{prefix}: missing high-spoof-resistance modality: {} (absent/out-of-tolerance: {})",
|
||||
worst.name(),
|
||||
names.join(", ")
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Exponentially-weighted moving-average update.
|
||||
fn ewma(prev: f32, sample: f32) -> f32 {
|
||||
(1.0 - ALPHA) * prev + ALPHA * sample
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A representative known-good response for an acoustic chirp across the
|
||||
/// fast modalities plus vibration (the hard-to-spoof one).
|
||||
fn good_chirp() -> ChallengeResponse {
|
||||
ChallengeResponse {
|
||||
stimulus: Stimulus::AcousticChirp,
|
||||
responses: vec![
|
||||
ObservedResponse {
|
||||
modality: Modality::Acoustic,
|
||||
delay: 0.030,
|
||||
magnitude: 1.0,
|
||||
},
|
||||
ObservedResponse {
|
||||
modality: Modality::Vibration,
|
||||
delay: 0.050,
|
||||
magnitude: 0.8,
|
||||
},
|
||||
ObservedResponse {
|
||||
modality: Modality::Rf,
|
||||
delay: 0.010,
|
||||
magnitude: 0.5,
|
||||
},
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn matching_response_is_trusted() {
|
||||
let mut v = CaptchaVerifier::new(0.01, 0.8);
|
||||
// Learn the profile a few times so the EWMA settles.
|
||||
for _ in 0..5 {
|
||||
v.learn(&good_chirp());
|
||||
}
|
||||
|
||||
let proof = v.verify(&good_chirp());
|
||||
assert!(
|
||||
proof.trusted,
|
||||
"matching response should be trusted: {proof:?}"
|
||||
);
|
||||
assert!(
|
||||
proof.score > 0.99,
|
||||
"score should be near 1.0: {}",
|
||||
proof.score
|
||||
);
|
||||
assert!(proof.missing.is_empty());
|
||||
assert_eq!(
|
||||
proof.reason,
|
||||
"all expected modalities responded within tolerance"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replayed_missing_vibration_is_rejected() {
|
||||
let mut v = CaptchaVerifier::new(0.01, 0.8);
|
||||
for _ in 0..5 {
|
||||
v.learn(&good_chirp());
|
||||
}
|
||||
|
||||
// A replay that drops vibration entirely and zeroes the rest's delays.
|
||||
let replay = ChallengeResponse {
|
||||
stimulus: Stimulus::AcousticChirp,
|
||||
responses: vec![
|
||||
ObservedResponse {
|
||||
modality: Modality::Acoustic,
|
||||
delay: 0.0,
|
||||
magnitude: 1.0,
|
||||
},
|
||||
ObservedResponse {
|
||||
modality: Modality::Rf,
|
||||
delay: 0.0,
|
||||
magnitude: 0.5,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
let proof = v.verify(&replay);
|
||||
assert!(!proof.trusted, "replay should be rejected: {proof:?}");
|
||||
assert!(
|
||||
proof.missing.contains(&Modality::Vibration),
|
||||
"vibration must be flagged missing: {:?}",
|
||||
proof.missing
|
||||
);
|
||||
// Acoustic delay (0.0 vs ~0.03) is out of the 0.01 tolerance too.
|
||||
assert!(proof.missing.contains(&Modality::Acoustic));
|
||||
assert!(proof.score < 0.8);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_stimulus_is_rejected() {
|
||||
let v = CaptchaVerifier::new(0.01, 0.8);
|
||||
let probe = ChallengeResponse {
|
||||
stimulus: Stimulus::ThermalPulse,
|
||||
responses: vec![ObservedResponse {
|
||||
modality: Modality::Thermal,
|
||||
delay: 2.0,
|
||||
magnitude: 1.0,
|
||||
}],
|
||||
};
|
||||
|
||||
let proof = v.verify(&probe);
|
||||
assert!(!proof.trusted);
|
||||
assert_eq!(proof.score, 0.0);
|
||||
assert_eq!(proof.reason, "unknown stimulus profile");
|
||||
assert!(proof.missing.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn weak_magnitude_fails_tolerance() {
|
||||
let mut v = CaptchaVerifier::new(0.01, 0.9);
|
||||
for _ in 0..5 {
|
||||
v.learn(&good_chirp());
|
||||
}
|
||||
|
||||
// Vibration arrives on time but with collapsed magnitude (< 50%).
|
||||
let attenuated = ChallengeResponse {
|
||||
stimulus: Stimulus::AcousticChirp,
|
||||
responses: vec![
|
||||
ObservedResponse {
|
||||
modality: Modality::Acoustic,
|
||||
delay: 0.030,
|
||||
magnitude: 1.0,
|
||||
},
|
||||
ObservedResponse {
|
||||
modality: Modality::Vibration,
|
||||
delay: 0.050,
|
||||
magnitude: 0.1,
|
||||
},
|
||||
ObservedResponse {
|
||||
modality: Modality::Rf,
|
||||
delay: 0.010,
|
||||
magnitude: 0.5,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
let proof = v.verify(&attenuated);
|
||||
assert!(
|
||||
!proof.trusted,
|
||||
"weak vibration should fail high threshold: {proof:?}"
|
||||
);
|
||||
assert!(proof.missing.contains(&Modality::Vibration));
|
||||
}
|
||||
}
|
||||
151
crates/ruvector-perception/src/coherence.rs
Normal file
151
crates/ruvector-perception/src/coherence.rs
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
//! Coherence field + boundary detection.
|
||||
//!
|
||||
//! Sensors/zones do not vote on an answer; they contribute to the *stability* of
|
||||
//! a physical graph. Zones are nodes; edge weight is delta-pattern coherence
|
||||
//! (quiet zones agree strongly; a zone whose physical state moved disagrees).
|
||||
//! Dynamic min-cut then isolates the side that broke away — that is the moved
|
||||
//! boundary, not a class label.
|
||||
|
||||
use ruvector_mincut::MinCutBuilder;
|
||||
|
||||
/// Where coherence broke this window.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Boundary {
|
||||
/// The single most-changed zone (the headline `changed_boundary`).
|
||||
pub zone: String,
|
||||
/// Every zone on the changed side of the cut.
|
||||
pub side: Vec<String>,
|
||||
/// Cleanliness of the separation in `[0, 1]`: high = the changed side is
|
||||
/// weakly coupled to the rest (a sharp, coherent boundary).
|
||||
pub coherence: f32,
|
||||
}
|
||||
|
||||
/// Detect the moved boundary from per-zone delta vectors (each vector is the
|
||||
/// per-modality |delta| for that zone, in a fixed modality order).
|
||||
pub fn detect_boundary(deltas: &[(String, Vec<f64>)]) -> Option<Boundary> {
|
||||
let k = deltas.len();
|
||||
if k == 0 {
|
||||
return None;
|
||||
}
|
||||
let norm = |v: &[f64]| -> f64 { v.iter().map(|x| x * x).sum::<f64>().sqrt() };
|
||||
if k == 1 {
|
||||
return Some(Boundary {
|
||||
zone: deltas[0].0.clone(),
|
||||
side: vec![deltas[0].0.clone()],
|
||||
coherence: 0.0,
|
||||
});
|
||||
}
|
||||
|
||||
// Pairwise distances and the scale that maps distance -> coherence weight.
|
||||
let dist = |a: &[f64], b: &[f64]| -> f64 {
|
||||
a.iter()
|
||||
.zip(b)
|
||||
.map(|(x, y)| (x - y) * (x - y))
|
||||
.sum::<f64>()
|
||||
.sqrt()
|
||||
};
|
||||
let mut max_d = 0.0f64;
|
||||
for i in 0..k {
|
||||
for j in (i + 1)..k {
|
||||
max_d = max_d.max(dist(&deltas[i].1, &deltas[j].1));
|
||||
}
|
||||
}
|
||||
let scale = if max_d > 1e-9 { max_d } else { 1.0 };
|
||||
const EPS: f64 = 1e-3;
|
||||
let weight = |a: &[f64], b: &[f64]| -> f64 {
|
||||
(1.0 - dist(a, b) / scale).max(EPS) // quiet-quiet ~1, outlier ~EPS
|
||||
};
|
||||
|
||||
// Complete weighted graph over zones; global min cut isolates the outlier.
|
||||
let mut edges = Vec::with_capacity(k * (k - 1) / 2);
|
||||
for i in 0..k {
|
||||
for j in (i + 1)..k {
|
||||
edges.push((i as u64, j as u64, weight(&deltas[i].1, &deltas[j].1)));
|
||||
}
|
||||
}
|
||||
let mincut = MinCutBuilder::new()
|
||||
.exact()
|
||||
.with_edges(edges)
|
||||
.build()
|
||||
.ok()?;
|
||||
let result = mincut.min_cut();
|
||||
let (a, b) = result.partition?;
|
||||
// Changed side = the smaller partition (the part that broke away).
|
||||
let (changed, _rest) = if a.len() <= b.len() { (a, b) } else { (b, a) };
|
||||
if changed.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let side: Vec<String> = changed
|
||||
.iter()
|
||||
.map(|&i| deltas[i as usize].0.clone())
|
||||
.collect();
|
||||
// Headline zone = largest-magnitude delta on the changed side.
|
||||
let zone = changed
|
||||
.iter()
|
||||
.max_by(|&&i, &&j| {
|
||||
norm(&deltas[i as usize].1)
|
||||
.partial_cmp(&norm(&deltas[j as usize].1))
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
})
|
||||
.map(|&i| deltas[i as usize].0.clone())
|
||||
.unwrap_or_else(|| side[0].clone());
|
||||
|
||||
// Coherence = how weakly the changed side couples to the rest.
|
||||
let changed_set: std::collections::HashSet<usize> =
|
||||
changed.iter().map(|&i| i as usize).collect();
|
||||
let mut cross_sum = 0.0;
|
||||
let mut cross_n = 0;
|
||||
for i in 0..k {
|
||||
for j in (i + 1)..k {
|
||||
if changed_set.contains(&i) != changed_set.contains(&j) {
|
||||
cross_sum += weight(&deltas[i].1, &deltas[j].1);
|
||||
cross_n += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
let mean_cross = if cross_n > 0 {
|
||||
cross_sum / cross_n as f64
|
||||
} else {
|
||||
1.0
|
||||
};
|
||||
let coherence = (1.0 - mean_cross).clamp(0.0, 1.0) as f32;
|
||||
|
||||
Some(Boundary {
|
||||
zone,
|
||||
side,
|
||||
coherence,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn isolates_the_changed_zone() {
|
||||
// Three quiet zones, one (B) moved.
|
||||
let deltas = vec![
|
||||
("A".to_string(), vec![0.0, 0.0, 0.0]),
|
||||
("B".to_string(), vec![3.0, 2.0, 2.5]),
|
||||
("C".to_string(), vec![0.0, 0.0, 0.0]),
|
||||
("D".to_string(), vec![0.0, 0.0, 0.0]),
|
||||
];
|
||||
let b = detect_boundary(&deltas).unwrap();
|
||||
assert_eq!(b.zone, "B");
|
||||
assert_eq!(b.side, vec!["B".to_string()]);
|
||||
assert!(b.coherence > 0.8, "coherence {}", b.coherence);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_change_means_low_coherence_boundary() {
|
||||
let deltas = vec![
|
||||
("A".to_string(), vec![0.0, 0.0]),
|
||||
("B".to_string(), vec![0.0, 0.0]),
|
||||
("C".to_string(), vec![0.0, 0.0]),
|
||||
];
|
||||
let b = detect_boundary(&deltas).unwrap();
|
||||
// Everything agrees -> no clean boundary.
|
||||
assert!(b.coherence < 0.2, "coherence {}", b.coherence);
|
||||
}
|
||||
}
|
||||
289
crates/ruvector-perception/src/custody.rs
Normal file
289
crates/ruvector-perception/src/custody.rs
Normal file
|
|
@ -0,0 +1,289 @@
|
|||
//! Sensor chain of custody — a tamper-evident, replayable ledger of perception
|
||||
//! events.
|
||||
//!
|
||||
//! Every [`DeltaWitness`] already carries a SHA-256 `evidence_hash` and the
|
||||
//! `prev_hash` of the witness before it, so a sequence of witnesses forms a hash
|
||||
//! chain. This module wraps that chain in an append-only ledger so that every
|
||||
//! action the engine takes can be *replayed* and its provenance *audited* —
|
||||
//! which is what elder-care, medical, industrial, and civic-governance
|
||||
//! deployments require before they trust an automated decision.
|
||||
//!
|
||||
//! ## Honest scope of verification
|
||||
//!
|
||||
//! [`CustodyLedger::verify`] checks **chain linkage**: each record's `prev_hash`
|
||||
//! must equal the prior record's `evidence_hash` (and the first record's
|
||||
//! `prev_hash` must be `None`). If anyone mutates a stored `evidence_hash`, the
|
||||
//! link to the next record breaks and `verify` reports it. This is
|
||||
//! **link-integrity**, *not* a full content re-hash: the raw signal and feature
|
||||
//! bytes that produced each `evidence_hash` are not stored in the witness, so
|
||||
//! this layer cannot recompute the SHA-256 from first principles. Detecting a
|
||||
//! forged-but-internally-consistent hash would require those raw bytes; here we
|
||||
//! detect tampering that breaks the chain.
|
||||
|
||||
use crate::witness::DeltaWitness;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Errors raised while maintaining or auditing the chain of custody.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum CustodyError {
|
||||
/// A record's `prev_hash` did not match the expected prior `evidence_hash`.
|
||||
#[error("broken chain at index {index}: prev_hash {found:?} != expected {expected:?}")]
|
||||
BrokenChain {
|
||||
/// Position in the ledger where the link broke.
|
||||
index: usize,
|
||||
/// The `prev_hash` actually found on the record.
|
||||
found: Option<String>,
|
||||
/// The `evidence_hash` of the prior record (or `None` for the first).
|
||||
expected: Option<String>,
|
||||
},
|
||||
/// No record carries the requested evidence hash.
|
||||
#[error("no record with evidence hash {0}")]
|
||||
NotFound(String),
|
||||
}
|
||||
|
||||
/// One entry in the ledger: the witnessed delta plus an optional outcome.
|
||||
///
|
||||
/// The `outcome` is later feedback attached after the fact (e.g. "confirmed
|
||||
/// fall", "false alarm", "operator acknowledged") so the audit trail records not
|
||||
/// just what was perceived but what actually happened.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct CustodyRecord {
|
||||
/// The perception event, including its evidence/prev hash linkage.
|
||||
pub witness: DeltaWitness,
|
||||
/// Outcome/feedback attached to this event, if any.
|
||||
pub outcome: Option<String>,
|
||||
}
|
||||
|
||||
/// An append-only, tamper-evident ledger of perception events.
|
||||
///
|
||||
/// Linkage is enforced at insert time by [`CustodyLedger::append`] and can be
|
||||
/// re-audited at any time by [`CustodyLedger::verify`].
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct CustodyLedger {
|
||||
records: Vec<CustodyRecord>,
|
||||
}
|
||||
|
||||
impl CustodyLedger {
|
||||
/// Create an empty ledger.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Append a witness to the ledger, enforcing chain linkage.
|
||||
///
|
||||
/// The witness's `prev_hash` MUST equal the last record's `evidence_hash`
|
||||
/// (or be `None` for the very first record); otherwise this returns
|
||||
/// [`CustodyError::BrokenChain`] and the ledger is left unchanged.
|
||||
pub fn append(&mut self, witness: DeltaWitness) -> Result<(), CustodyError> {
|
||||
let expected = self.records.last().map(|r| r.witness.evidence_hash.clone());
|
||||
if witness.prev_hash != expected {
|
||||
return Err(CustodyError::BrokenChain {
|
||||
index: self.records.len(),
|
||||
found: witness.prev_hash,
|
||||
expected,
|
||||
});
|
||||
}
|
||||
self.records.push(CustodyRecord {
|
||||
witness,
|
||||
outcome: None,
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Attach an outcome/feedback to the record with the given evidence hash.
|
||||
///
|
||||
/// Returns [`CustodyError::NotFound`] if no record carries that hash.
|
||||
pub fn record_outcome(
|
||||
&mut self,
|
||||
evidence_hash: &str,
|
||||
outcome: impl Into<String>,
|
||||
) -> Result<(), CustodyError> {
|
||||
let record = self
|
||||
.records
|
||||
.iter_mut()
|
||||
.find(|r| r.witness.evidence_hash == evidence_hash)
|
||||
.ok_or_else(|| CustodyError::NotFound(evidence_hash.to_string()))?;
|
||||
record.outcome = Some(outcome.into());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Re-audit the whole chain: every `prev_hash` must equal the prior record's
|
||||
/// `evidence_hash` (and the first must be `None`).
|
||||
///
|
||||
/// This verifies **chain linkage** — tampering with a stored `evidence_hash`
|
||||
/// breaks the link to the next record and is reported here. It does **not**
|
||||
/// recompute each SHA-256 from raw signal bytes (those are not stored in the
|
||||
/// witness), so it is link-integrity, not full content re-hash. Returns the
|
||||
/// first [`CustodyError::BrokenChain`] encountered.
|
||||
pub fn verify(&self) -> Result<(), CustodyError> {
|
||||
let mut expected: Option<String> = None;
|
||||
for (index, record) in self.records.iter().enumerate() {
|
||||
if record.witness.prev_hash != expected {
|
||||
return Err(CustodyError::BrokenChain {
|
||||
index,
|
||||
found: record.witness.prev_hash.clone(),
|
||||
expected,
|
||||
});
|
||||
}
|
||||
expected = Some(record.witness.evidence_hash.clone());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Number of records in the ledger.
|
||||
pub fn len(&self) -> usize {
|
||||
self.records.len()
|
||||
}
|
||||
|
||||
/// Whether the ledger holds no records.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.records.is_empty()
|
||||
}
|
||||
|
||||
/// Borrow the full record slice (read-only; the ledger stays append-only).
|
||||
pub fn records(&self) -> &[CustodyRecord] {
|
||||
&self.records
|
||||
}
|
||||
|
||||
/// Return the chain of records from the start up to and including the record
|
||||
/// with `evidence_hash` — the replayable provenance of that event.
|
||||
///
|
||||
/// Returns [`CustodyError::NotFound`] if no record carries that hash.
|
||||
pub fn replay_until(&self, evidence_hash: &str) -> Result<Vec<&CustodyRecord>, CustodyError> {
|
||||
let end = self
|
||||
.records
|
||||
.iter()
|
||||
.position(|r| r.witness.evidence_hash == evidence_hash)
|
||||
.ok_or_else(|| CustodyError::NotFound(evidence_hash.to_string()))?;
|
||||
Ok(self.records[..=end].iter().collect())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::modality::Modality;
|
||||
use crate::witness::Action;
|
||||
|
||||
/// Build a witness with explicit hash linkage; raw scores are arbitrary but
|
||||
/// deterministic so tests focus on custody, not perception.
|
||||
fn witness(t: u64, evidence_hash: &str, prev_hash: Option<&str>) -> DeltaWitness {
|
||||
DeltaWitness {
|
||||
t,
|
||||
changed_boundary: format!("zone-{t}"),
|
||||
supporting_modalities: vec![Modality::Rf, Modality::Vibration],
|
||||
contradicting_modalities: vec![Modality::Thermal],
|
||||
novelty: 0.8,
|
||||
coherence: 0.7,
|
||||
contradiction: 0.1,
|
||||
action: Action::Alert,
|
||||
evidence_hash: evidence_hash.to_string(),
|
||||
prev_hash: prev_hash.map(str::to_string),
|
||||
}
|
||||
}
|
||||
|
||||
fn three_link_ledger() -> CustodyLedger {
|
||||
let mut ledger = CustodyLedger::new();
|
||||
ledger.append(witness(0, "h0", None)).unwrap();
|
||||
ledger.append(witness(1, "h1", Some("h0"))).unwrap();
|
||||
ledger.append(witness(2, "h2", Some("h1"))).unwrap();
|
||||
ledger
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn three_link_chain_verifies() {
|
||||
let ledger = three_link_ledger();
|
||||
assert_eq!(ledger.len(), 3);
|
||||
assert!(!ledger.is_empty());
|
||||
assert!(ledger.verify().is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn append_rejects_mismatched_prev_hash() {
|
||||
let mut ledger = CustodyLedger::new();
|
||||
ledger.append(witness(0, "h0", None)).unwrap();
|
||||
// prev_hash should be Some("h0"), but we link it to the wrong place.
|
||||
let err = ledger.append(witness(1, "h1", Some("WRONG"))).unwrap_err();
|
||||
match err {
|
||||
CustodyError::BrokenChain {
|
||||
index,
|
||||
found,
|
||||
expected,
|
||||
} => {
|
||||
assert_eq!(index, 1);
|
||||
assert_eq!(found, Some("WRONG".to_string()));
|
||||
assert_eq!(expected, Some("h0".to_string()));
|
||||
}
|
||||
other => panic!("expected BrokenChain, got {other:?}"),
|
||||
}
|
||||
// The rejected record must not have been stored.
|
||||
assert_eq!(ledger.len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn first_record_must_have_no_prev_hash() {
|
||||
let mut ledger = CustodyLedger::new();
|
||||
let err = ledger.append(witness(0, "h0", Some("h-1"))).unwrap_err();
|
||||
assert!(matches!(err, CustodyError::BrokenChain { index: 0, .. }));
|
||||
assert!(ledger.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn record_outcome_then_find_it() {
|
||||
let mut ledger = three_link_ledger();
|
||||
ledger.record_outcome("h1", "confirmed fall").unwrap();
|
||||
let record = ledger
|
||||
.records()
|
||||
.iter()
|
||||
.find(|r| r.witness.evidence_hash == "h1")
|
||||
.unwrap();
|
||||
assert_eq!(record.outcome.as_deref(), Some("confirmed fall"));
|
||||
// Other records are untouched.
|
||||
assert!(ledger.records()[0].outcome.is_none());
|
||||
|
||||
// An unknown hash is reported as NotFound.
|
||||
let err = ledger.record_outcome("nope", "x").unwrap_err();
|
||||
assert!(matches!(err, CustodyError::NotFound(h) if h == "nope"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_until_returns_prefix_chain() {
|
||||
let ledger = three_link_ledger();
|
||||
let chain = ledger.replay_until("h1").unwrap();
|
||||
assert_eq!(chain.len(), 2);
|
||||
assert_eq!(chain[0].witness.evidence_hash, "h0");
|
||||
assert_eq!(chain[1].witness.evidence_hash, "h1");
|
||||
|
||||
// The full chain is reachable from the last hash.
|
||||
assert_eq!(ledger.replay_until("h2").unwrap().len(), 3);
|
||||
|
||||
// Unknown hashes are NotFound.
|
||||
let err = ledger.replay_until("ghost").unwrap_err();
|
||||
assert!(matches!(err, CustodyError::NotFound(h) if h == "ghost"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_detects_a_corrupted_link() {
|
||||
// Hand-build a ledger whose middle record's evidence_hash has been
|
||||
// mutated *after* insertion, so its link to the next record is broken.
|
||||
// We bypass `append` (which would reject this) to simulate tampering of
|
||||
// already-stored data, exactly what `verify` must catch.
|
||||
let mut ledger = three_link_ledger();
|
||||
// Corrupt h1 -> the third record still points at "h1", so the link
|
||||
// expected at index 2 ("h1_tampered") will mismatch its found prev_hash.
|
||||
ledger.records[1].witness.evidence_hash = "h1_tampered".to_string();
|
||||
|
||||
match ledger.verify() {
|
||||
Err(CustodyError::BrokenChain {
|
||||
index,
|
||||
found,
|
||||
expected,
|
||||
}) => {
|
||||
assert_eq!(index, 2);
|
||||
assert_eq!(found, Some("h1".to_string()));
|
||||
assert_eq!(expected, Some("h1_tampered".to_string()));
|
||||
}
|
||||
other => panic!("expected BrokenChain at index 2, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
259
crates/ruvector-perception/src/engine.rs
Normal file
259
crates/ruvector-perception/src/engine.rs
Normal file
|
|
@ -0,0 +1,259 @@
|
|||
//! The delta engine: turns a window of multi-modal readings into a proof-gated
|
||||
//! [`DeltaWitness`]. The pipeline is `delta → boundary → coherence → proof →
|
||||
//! action` — it models *state transition*, not a fixed task label.
|
||||
|
||||
use crate::coherence::detect_boundary;
|
||||
use crate::modality::Modality;
|
||||
use crate::state::{Reading, WorldState};
|
||||
use crate::witness::{evidence_hash, Action, DeltaWitness, ProofGate};
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Configuration for [`DeltaEngine`].
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct EngineConfig {
|
||||
/// EWMA smoothing for baselines.
|
||||
pub alpha: f32,
|
||||
/// |delta| above which a modality is considered to have responded.
|
||||
pub active_threshold: f32,
|
||||
/// Minimum historical responsiveness for a silent modality to count as a
|
||||
/// contradiction ("it usually reacts here, but didn't").
|
||||
pub responsive_min: f32,
|
||||
/// How many prior changed-zone delta vectors to remember for novelty.
|
||||
pub history_cap: usize,
|
||||
/// Proof-gate thresholds.
|
||||
pub gate: ProofGate,
|
||||
}
|
||||
|
||||
impl Default for EngineConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
alpha: 0.4,
|
||||
active_threshold: 0.4,
|
||||
responsive_min: 0.3,
|
||||
history_cap: 256,
|
||||
gate: ProofGate::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Stateful physical-perception engine.
|
||||
pub struct DeltaEngine {
|
||||
cfg: EngineConfig,
|
||||
state: WorldState,
|
||||
history: Vec<Vec<f64>>, // prior changed-zone delta vectors
|
||||
prev_hash: Option<String>,
|
||||
}
|
||||
|
||||
impl DeltaEngine {
|
||||
/// Create an engine.
|
||||
pub fn new(cfg: EngineConfig) -> Self {
|
||||
let state = WorldState::new(cfg.alpha, cfg.active_threshold);
|
||||
Self {
|
||||
cfg,
|
||||
state,
|
||||
history: Vec::new(),
|
||||
prev_hash: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Borrow the rolling world state (baselines, responsiveness).
|
||||
pub fn state(&self) -> &WorldState {
|
||||
&self.state
|
||||
}
|
||||
|
||||
/// Observe one time window of readings and emit a proof-gated witness.
|
||||
pub fn observe(&mut self, readings: &[Reading], t: u64) -> DeltaWitness {
|
||||
// 1. Per-zone delta vectors over a fixed modality order.
|
||||
let mut by_zone: HashMap<String, HashMap<Modality, f32>> = HashMap::new();
|
||||
for r in readings {
|
||||
by_zone
|
||||
.entry(r.zone.clone())
|
||||
.or_default()
|
||||
.insert(r.modality, r.value);
|
||||
}
|
||||
let mut zones: Vec<String> = by_zone.keys().cloned().collect();
|
||||
zones.sort();
|
||||
|
||||
let delta_vec = |zone: &str| -> Vec<f64> {
|
||||
Modality::ALL
|
||||
.iter()
|
||||
.map(|&m| match by_zone[zone].get(&m) {
|
||||
Some(&v) => (v - self.state.baseline(zone, m)).abs() as f64,
|
||||
None => 0.0,
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
let deltas: Vec<(String, Vec<f64>)> =
|
||||
zones.iter().map(|z| (z.clone(), delta_vec(z))).collect();
|
||||
|
||||
// 2. Boundary via coherence min-cut.
|
||||
let boundary = detect_boundary(&deltas);
|
||||
let (changed, coherence, changed_vec) = match boundary {
|
||||
Some(b) => {
|
||||
let v = deltas
|
||||
.iter()
|
||||
.find(|(z, _)| z == &b.zone)
|
||||
.map(|(_, v)| v.clone());
|
||||
(b.zone, b.coherence, v.unwrap_or_default())
|
||||
}
|
||||
None => {
|
||||
let w = self.finish(readings, &deltas, t, NullWitness::empty());
|
||||
return w;
|
||||
}
|
||||
};
|
||||
|
||||
// 3. Supporting / contradicting modalities in the changed zone.
|
||||
let thr = self.cfg.active_threshold;
|
||||
let mut supporting = Vec::new();
|
||||
let mut contradicting = Vec::new();
|
||||
let mut contradiction = 0.0f32;
|
||||
for &m in &Modality::ALL {
|
||||
let mag = match by_zone[&changed].get(&m) {
|
||||
Some(&v) => (v - self.state.baseline(&changed, m)).abs(),
|
||||
None => 0.0,
|
||||
};
|
||||
if mag >= thr {
|
||||
supporting.push(m);
|
||||
} else if self.state.seen(&changed, m)
|
||||
&& self.state.responsiveness(&changed, m) >= self.cfg.responsive_min
|
||||
{
|
||||
// Usually reacts here, but stayed silent — first-class disagreement.
|
||||
contradicting.push(m);
|
||||
contradiction = contradiction.max(m.physics().spoof_resistance);
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Novelty vs prior changed-zone states.
|
||||
let novelty = self.novelty(&changed_vec);
|
||||
|
||||
// 5. Proof gate -> bounded authority.
|
||||
let action = self.cfg.gate.decide(novelty, coherence, contradiction);
|
||||
|
||||
let w = NullWitness {
|
||||
changed_boundary: changed,
|
||||
supporting,
|
||||
contradicting,
|
||||
novelty,
|
||||
coherence,
|
||||
contradiction,
|
||||
action,
|
||||
changed_vec: Some(changed_vec),
|
||||
};
|
||||
self.finish(readings, &deltas, t, w)
|
||||
}
|
||||
|
||||
fn novelty(&self, vec: &[f64]) -> f32 {
|
||||
if self.history.is_empty() {
|
||||
return 1.0;
|
||||
}
|
||||
let norm = |v: &[f64]| v.iter().map(|x| x * x).sum::<f64>().sqrt();
|
||||
let dist = |a: &[f64], b: &[f64]| -> f64 {
|
||||
a.iter()
|
||||
.zip(b)
|
||||
.map(|(x, y)| (x - y) * (x - y))
|
||||
.sum::<f64>()
|
||||
.sqrt()
|
||||
};
|
||||
let min_d = self
|
||||
.history
|
||||
.iter()
|
||||
.map(|h| dist(h, vec))
|
||||
.fold(f64::INFINITY, f64::min);
|
||||
(min_d / (norm(vec) + 1e-9)).clamp(0.0, 1.0) as f32
|
||||
}
|
||||
|
||||
/// Hash evidence, build the witness, then fold the readings into state and
|
||||
/// remember the changed vector for future novelty.
|
||||
fn finish(
|
||||
&mut self,
|
||||
readings: &[Reading],
|
||||
deltas: &[(String, Vec<f64>)],
|
||||
t: u64,
|
||||
w: NullWitness,
|
||||
) -> DeltaWitness {
|
||||
// Canonical raw + feature bytes for the evidence chain.
|
||||
let mut raw = String::new();
|
||||
let mut sorted: Vec<&Reading> = readings.iter().collect();
|
||||
sorted.sort_by(|a, b| {
|
||||
(a.zone.as_str(), a.modality.name()).cmp(&(b.zone.as_str(), b.modality.name()))
|
||||
});
|
||||
for r in sorted {
|
||||
raw.push_str(&format!("{}:{}:{:.6};", r.zone, r.modality.name(), r.value));
|
||||
}
|
||||
let mut feat = String::new();
|
||||
for (z, v) in deltas {
|
||||
feat.push_str(z);
|
||||
for x in v {
|
||||
feat.push_str(&format!(":{x:.6}"));
|
||||
}
|
||||
feat.push(';');
|
||||
}
|
||||
let hash = evidence_hash(
|
||||
raw.as_bytes(),
|
||||
feat.as_bytes(),
|
||||
&w.changed_boundary,
|
||||
w.novelty,
|
||||
w.coherence,
|
||||
w.contradiction,
|
||||
w.action,
|
||||
self.prev_hash.as_deref(),
|
||||
);
|
||||
|
||||
let witness = DeltaWitness {
|
||||
t,
|
||||
changed_boundary: w.changed_boundary,
|
||||
supporting_modalities: w.supporting,
|
||||
contradicting_modalities: w.contradicting,
|
||||
novelty: w.novelty,
|
||||
coherence: w.coherence,
|
||||
contradiction: w.contradiction,
|
||||
action: w.action,
|
||||
evidence_hash: hash.clone(),
|
||||
prev_hash: self.prev_hash.take(),
|
||||
};
|
||||
self.prev_hash = Some(hash);
|
||||
|
||||
// Remember the changed vector (compress: only store meaningful events).
|
||||
if let Some(v) = w.changed_vec {
|
||||
if v.iter().any(|&x| x as f32 >= self.cfg.active_threshold) {
|
||||
self.history.push(v);
|
||||
if self.history.len() > self.cfg.history_cap {
|
||||
self.history.remove(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fold readings into the rolling baselines.
|
||||
for r in readings {
|
||||
self.state.update(r);
|
||||
}
|
||||
witness
|
||||
}
|
||||
}
|
||||
|
||||
/// Internal scratch for an in-progress witness.
|
||||
struct NullWitness {
|
||||
changed_boundary: String,
|
||||
supporting: Vec<Modality>,
|
||||
contradicting: Vec<Modality>,
|
||||
novelty: f32,
|
||||
coherence: f32,
|
||||
contradiction: f32,
|
||||
action: Action,
|
||||
changed_vec: Option<Vec<f64>>,
|
||||
}
|
||||
|
||||
impl NullWitness {
|
||||
fn empty() -> Self {
|
||||
Self {
|
||||
changed_boundary: String::new(),
|
||||
supporting: Vec::new(),
|
||||
contradicting: Vec::new(),
|
||||
novelty: 0.0,
|
||||
coherence: 0.0,
|
||||
contradiction: 0.0,
|
||||
action: Action::Ignore,
|
||||
changed_vec: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
358
crates/ruvector-perception/src/hypothesis.rs
Normal file
358
crates/ruvector-perception/src/hypothesis.rs
Normal file
|
|
@ -0,0 +1,358 @@
|
|||
//! Multi-modal disagreement engine — *disagreement is information, not noise*.
|
||||
//!
|
||||
//! When modalities contradict each other, the classical reflex is to fuse them
|
||||
//! into a single agreed-upon answer (and throw away the conflict). This module
|
||||
//! does the opposite: it treats a contradiction as a *question* — "why do these
|
||||
//! sensors disagree?" — and answers with **ranked hypotheses** instead of a
|
||||
//! forced consensus.
|
||||
//!
|
||||
//! The same raw disagreement can mean very different things:
|
||||
//!
|
||||
//! - a **real event** the slow channels haven't caught up to yet,
|
||||
//! - a single channel **drifting** out of calibration,
|
||||
//! - a sensor that was physically **relocated** so its readings no longer fit
|
||||
//! the spatial field,
|
||||
//! - an **adversarial replay** where the easy-to-spoof channels were faked while
|
||||
//! the hard-to-spoof physical channels stayed silent,
|
||||
//! - or a transient **environmental artifact** (an echo / reflection).
|
||||
//!
|
||||
//! Each candidate gets an `evidence` score in `[0, 1]` derived from *typed*
|
||||
//! physics ([`Modality::physics`]) plus the qualitative shape of the
|
||||
//! disagreement. We always return all five, sorted by evidence descending, so a
|
||||
//! caller can inspect the full ranked field rather than a single label.
|
||||
|
||||
use crate::modality::Modality;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A candidate explanation for *why* the modalities disagree.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum Hypothesis {
|
||||
/// A genuine physical change; supporting (hard-to-spoof) channels fired
|
||||
/// coherently and the few contradictions are explainable (e.g. latency).
|
||||
RealEvent,
|
||||
/// A single low-spoof-resistance channel slowly wandering out of calibration
|
||||
/// — persistent, lone, and spatially incoherent.
|
||||
SensorDrift,
|
||||
/// A sensor was physically moved: it responds strongly but its readings no
|
||||
/// longer fit the neighbours' spatial pattern (sudden + novel + incoherent).
|
||||
SensorRelocation,
|
||||
/// Easy-to-spoof channels (RF/Optical) report a strong event while the
|
||||
/// hard-to-spoof physical channels (Vibration/Thermal) stayed silent.
|
||||
AdversarialReplay,
|
||||
/// A transient reflection/echo: familiar (low novelty), short-lived, mixed
|
||||
/// support — present but not a durable, coherent event.
|
||||
EnvironmentalArtifact,
|
||||
}
|
||||
|
||||
/// One scored explanation. `evidence` is normalised to `[0, 1]`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct RankedHypothesis {
|
||||
/// The candidate explanation.
|
||||
pub hypothesis: Hypothesis,
|
||||
/// Strength of support for this explanation, in `[0, 1]`.
|
||||
pub evidence: f32,
|
||||
}
|
||||
|
||||
/// Inputs describing a single witnessed disagreement.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DisagreementInput {
|
||||
/// Modalities that *did* respond to the change.
|
||||
pub supporting: Vec<Modality>,
|
||||
/// Modalities that *should* have responded in this zone but stayed silent.
|
||||
pub contradicting: Vec<Modality>,
|
||||
/// How surprising the signal is, `[0, 1]`.
|
||||
pub novelty: f32,
|
||||
/// Cleanliness of the spatial boundary, `[0, 1]` (1 = crisp, 0 = smeared).
|
||||
pub coherence: f32,
|
||||
/// How long the signal has persisted across windows, `[0, 1]`.
|
||||
pub persistence: f32,
|
||||
}
|
||||
|
||||
impl DisagreementInput {
|
||||
/// Mean spoof-resistance of the supporting set (0 if empty).
|
||||
fn supporting_spoof_resistance(&self) -> f32 {
|
||||
mean_spoof_resistance(&self.supporting)
|
||||
}
|
||||
|
||||
/// Mean spoof-resistance of the contradicting set (0 if empty).
|
||||
fn contradicting_spoof_resistance(&self) -> f32 {
|
||||
mean_spoof_resistance(&self.contradicting)
|
||||
}
|
||||
|
||||
/// Fraction of involved modalities that are contradicting, `[0, 1]`.
|
||||
/// 0 when nothing is involved at all.
|
||||
fn contradiction_fraction(&self) -> f32 {
|
||||
let total = self.supporting.len() + self.contradicting.len();
|
||||
if total == 0 {
|
||||
return 0.0;
|
||||
}
|
||||
self.contradicting.len() as f32 / total as f32
|
||||
}
|
||||
}
|
||||
|
||||
/// Mean spoof-resistance of a modality set; 0.0 for an empty set.
|
||||
fn mean_spoof_resistance(set: &[Modality]) -> f32 {
|
||||
if set.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
let sum: f32 = set.iter().map(|m| m.physics().spoof_resistance).sum();
|
||||
sum / set.len() as f32
|
||||
}
|
||||
|
||||
/// Clamp a raw score into `[0, 1]`.
|
||||
fn clamp01(x: f32) -> f32 {
|
||||
x.clamp(0.0, 1.0)
|
||||
}
|
||||
|
||||
/// Rank all candidate explanations for a disagreement by evidence (descending).
|
||||
///
|
||||
/// Always returns exactly five [`RankedHypothesis`] entries. Ties keep the
|
||||
/// canonical declaration order ([`Hypothesis::RealEvent`] first) because the
|
||||
/// sort is stable and the candidates are pushed in that order.
|
||||
pub fn rank_hypotheses(input: &DisagreementInput) -> Vec<RankedHypothesis> {
|
||||
let mut ranked = vec![
|
||||
RankedHypothesis {
|
||||
hypothesis: Hypothesis::RealEvent,
|
||||
evidence: score_real_event(input),
|
||||
},
|
||||
RankedHypothesis {
|
||||
hypothesis: Hypothesis::SensorDrift,
|
||||
evidence: score_sensor_drift(input),
|
||||
},
|
||||
RankedHypothesis {
|
||||
hypothesis: Hypothesis::SensorRelocation,
|
||||
evidence: score_sensor_relocation(input),
|
||||
},
|
||||
RankedHypothesis {
|
||||
hypothesis: Hypothesis::AdversarialReplay,
|
||||
evidence: score_adversarial_replay(input),
|
||||
},
|
||||
RankedHypothesis {
|
||||
hypothesis: Hypothesis::EnvironmentalArtifact,
|
||||
evidence: score_environmental_artifact(input),
|
||||
},
|
||||
];
|
||||
|
||||
// Stable, descending by evidence. `total_cmp` keeps this deterministic even
|
||||
// for NaN-free f32s and never panics. Stability preserves declaration order
|
||||
// on ties.
|
||||
ranked.sort_by(|a, b| b.evidence.total_cmp(&a.evidence));
|
||||
ranked
|
||||
}
|
||||
|
||||
/// **RealEvent**: many trustworthy supporting channels, a crisp boundary, some
|
||||
/// novelty and decent persistence, with few contradictions. We weight by the
|
||||
/// supporting set's mean spoof-resistance (hard-to-spoof agreement is the
|
||||
/// strongest signal a thing actually happened) and penalise by the contradiction
|
||||
/// fraction.
|
||||
fn score_real_event(input: &DisagreementInput) -> f32 {
|
||||
if input.supporting.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
let trust = input.supporting_spoof_resistance();
|
||||
// Reward breadth of support: two trustworthy channels beat one.
|
||||
let breadth = (input.supporting.len() as f32 / 3.0).min(1.0);
|
||||
let persistence_term = 0.5 + 0.5 * clamp01(input.persistence);
|
||||
let novelty_term = 0.5 + 0.5 * clamp01(input.novelty);
|
||||
let base = trust * clamp01(input.coherence) * persistence_term * novelty_term;
|
||||
let breadth_boosted = base * (0.6 + 0.4 * breadth);
|
||||
// Contradictions erode a "real event" reading.
|
||||
clamp01(breadth_boosted * (1.0 - input.contradiction_fraction()))
|
||||
}
|
||||
|
||||
/// **SensorDrift**: a lone low-spoof-resistance channel slowly wandering. High
|
||||
/// when exactly one modality supports, that modality is easy to spoof / noisy
|
||||
/// (low spoof-resistance), the boundary is *incoherent* (drift is not a clean
|
||||
/// spatial edge), and persistence is high (drift is slow and sustained). Novelty
|
||||
/// should be modest — drift creeps, it does not jump.
|
||||
fn score_sensor_drift(input: &DisagreementInput) -> f32 {
|
||||
if input.supporting.len() != 1 {
|
||||
return 0.0;
|
||||
}
|
||||
let weak_channel = 1.0 - input.supporting_spoof_resistance();
|
||||
let incoherence = 1.0 - clamp01(input.coherence);
|
||||
let persistent = clamp01(input.persistence);
|
||||
// Gradual: penalise high novelty (that points at relocation instead).
|
||||
let gradual = 1.0 - clamp01(input.novelty);
|
||||
clamp01(weak_channel * incoherence * persistent * (0.5 + 0.5 * gradual))
|
||||
}
|
||||
|
||||
/// **SensorRelocation**: a sensor moved, so it still responds strongly but its
|
||||
/// readings no longer fit the spatial field. Distinguished from drift by being
|
||||
/// *sudden and novel* rather than gradual: support present, coherence LOW
|
||||
/// (doesn't fit neighbours), novelty HIGH, and at least one contradiction
|
||||
/// (neighbours that should agree don't). Persistence is not required — a
|
||||
/// relocation is a step change.
|
||||
fn score_sensor_relocation(input: &DisagreementInput) -> f32 {
|
||||
if input.supporting.is_empty() || input.contradicting.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
let responding = clamp01(input.supporting_spoof_resistance().max(0.2));
|
||||
let incoherence = 1.0 - clamp01(input.coherence);
|
||||
let sudden = clamp01(input.novelty);
|
||||
let mismatch = input.contradiction_fraction();
|
||||
clamp01(responding * incoherence * sudden * (0.5 + 0.5 * mismatch))
|
||||
}
|
||||
|
||||
/// **AdversarialReplay**: the supporting set is dominated by easy-to-spoof
|
||||
/// channels (RF/Optical) while the *hard-to-spoof* physical channels
|
||||
/// (Vibration/Thermal) are in the contradicting set (silent). A fake can drive
|
||||
/// radios and light but cannot reproduce structural vibration or thermal mass.
|
||||
/// Persistence is usually low/static for a replayed snippet, so low persistence
|
||||
/// adds a little weight.
|
||||
fn score_adversarial_replay(input: &DisagreementInput) -> f32 {
|
||||
if input.supporting.is_empty() || input.contradicting.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
// Supporting must be *easy* to spoof; contradicting must be *hard* to spoof.
|
||||
let support_spoofability = 1.0 - input.supporting_spoof_resistance();
|
||||
let silent_trust = input.contradicting_spoof_resistance();
|
||||
// Only meaningful when the hard channels are the silent ones.
|
||||
if silent_trust <= input.supporting_spoof_resistance() {
|
||||
return 0.0;
|
||||
}
|
||||
let static_signal = 1.0 - clamp01(input.persistence);
|
||||
let core = support_spoofability * silent_trust;
|
||||
clamp01(core * (0.7 + 0.3 * static_signal))
|
||||
}
|
||||
|
||||
/// **EnvironmentalArtifact**: a transient reflection/echo. Familiar rather than
|
||||
/// novel (low novelty), short-lived (low persistence), with moderate coherence
|
||||
/// and mixed support — it shows up but never settles into a durable, trustworthy
|
||||
/// event.
|
||||
fn score_environmental_artifact(input: &DisagreementInput) -> f32 {
|
||||
if input.supporting.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
let familiar = 1.0 - clamp01(input.novelty);
|
||||
let transient = 1.0 - clamp01(input.persistence);
|
||||
// Moderate coherence peaks at ~0.5 (an echo is neither crisp nor formless).
|
||||
let moderate_coherence = 1.0 - (clamp01(input.coherence) - 0.5).abs() * 2.0;
|
||||
// Low-trust support is more echo-like than a hard physical channel.
|
||||
let soft_support = 1.0 - input.supporting_spoof_resistance();
|
||||
clamp01(familiar * transient * (0.4 + 0.6 * moderate_coherence) * (0.5 + 0.5 * soft_support))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn first(input: &DisagreementInput) -> Hypothesis {
|
||||
rank_hypotheses(input)[0].hypothesis
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn returns_all_five_sorted_descending() {
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![Modality::Vibration],
|
||||
contradicting: vec![],
|
||||
novelty: 0.5,
|
||||
coherence: 0.5,
|
||||
persistence: 0.5,
|
||||
};
|
||||
let ranked = rank_hypotheses(&input);
|
||||
assert_eq!(ranked.len(), 5);
|
||||
for w in ranked.windows(2) {
|
||||
assert!(w[0].evidence >= w[1].evidence);
|
||||
assert!((0.0..=1.0).contains(&w[0].evidence));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn many_trustworthy_supporters_imply_real_event() {
|
||||
// Hard-to-spoof channels agree, boundary is crisp, contradictions are
|
||||
// absent — this is what a genuine physical event looks like.
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![Modality::Vibration, Modality::Thermal, Modality::Acoustic],
|
||||
contradicting: vec![],
|
||||
novelty: 0.7,
|
||||
coherence: 0.9,
|
||||
persistence: 0.7,
|
||||
};
|
||||
assert_eq!(first(&input), Hypothesis::RealEvent);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lone_weak_persistent_channel_implies_drift() {
|
||||
// A single easy-to-spoof channel, no clean boundary, sustained over time,
|
||||
// creeping (low novelty): the signature of calibration drift.
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![Modality::Rf],
|
||||
contradicting: vec![],
|
||||
novelty: 0.2,
|
||||
coherence: 0.1,
|
||||
persistence: 0.9,
|
||||
};
|
||||
assert_eq!(first(&input), Hypothesis::SensorDrift);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn easy_channels_loud_hard_channels_silent_imply_replay() {
|
||||
// RF + Optical (easy to spoof) report a strong event, while Vibration +
|
||||
// Thermal (hard to spoof) are silent — a classic replayed/faked signal.
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![Modality::Rf, Modality::Optical],
|
||||
contradicting: vec![Modality::Vibration, Modality::Thermal],
|
||||
novelty: 0.6,
|
||||
coherence: 0.6,
|
||||
persistence: 0.1,
|
||||
};
|
||||
assert_eq!(first(&input), Hypothesis::AdversarialReplay);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sudden_novel_incoherent_with_contradiction_implies_relocation() {
|
||||
// A trustworthy sensor still responds strongly, but suddenly (high
|
||||
// novelty), incoherently, and its neighbours contradict it: it moved.
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![Modality::Vibration],
|
||||
contradicting: vec![Modality::Acoustic],
|
||||
novelty: 0.95,
|
||||
coherence: 0.1,
|
||||
persistence: 0.2,
|
||||
};
|
||||
let ranked = rank_hypotheses(&input);
|
||||
// Relocation should out-rank drift here because the change is sudden.
|
||||
let reloc = ranked
|
||||
.iter()
|
||||
.find(|r| r.hypothesis == Hypothesis::SensorRelocation)
|
||||
.unwrap()
|
||||
.evidence;
|
||||
let drift = ranked
|
||||
.iter()
|
||||
.find(|r| r.hypothesis == Hypothesis::SensorDrift)
|
||||
.unwrap()
|
||||
.evidence;
|
||||
assert!(reloc > drift);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn familiar_transient_implies_environmental_artifact() {
|
||||
// Low novelty, short-lived, moderate coherence, soft support: an echo.
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![Modality::Optical],
|
||||
contradicting: vec![],
|
||||
novelty: 0.05,
|
||||
coherence: 0.5,
|
||||
persistence: 0.05,
|
||||
};
|
||||
assert_eq!(first(&input), Hypothesis::EnvironmentalArtifact);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_supporting_is_deterministic_and_bounded() {
|
||||
let input = DisagreementInput {
|
||||
supporting: vec![],
|
||||
contradicting: vec![Modality::Thermal],
|
||||
novelty: 0.5,
|
||||
coherence: 0.5,
|
||||
persistence: 0.5,
|
||||
};
|
||||
let ranked = rank_hypotheses(&input);
|
||||
assert_eq!(ranked.len(), 5);
|
||||
for r in &ranked {
|
||||
assert!((0.0..=1.0).contains(&r.evidence));
|
||||
}
|
||||
}
|
||||
}
|
||||
241
crates/ruvector-perception/src/identity.rs
Normal file
241
crates/ruvector-perception/src/identity.rs
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
//! Resonant identity layer: continuity recognition for physical objects.
|
||||
//!
|
||||
//! Every physical object emits a *resonant response signature* — a vibration,
|
||||
//! acoustic, or RF-reflection embedding that depends on its mass, geometry,
|
||||
//! material, fastening, and contents. This layer does not ask *"what is this?"*;
|
||||
//! it asks *"is this STILL the same physical thing?"*
|
||||
//!
|
||||
//! By enrolling a known signature and comparing fresh observations against it,
|
||||
//! we detect **identity drift**: a panel has loosened, a pipe has filled with
|
||||
//! water, a bearing has worn, a casing has been tampered with. Small, gradual
|
||||
//! changes (aging, ambient noise) are absorbed by an exponentially-weighted
|
||||
//! moving average (EWMA) so the stored signature tracks slow drift, while a
|
||||
//! sudden large change trips the `changed` flag.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// The result of comparing a fresh signature against an enrolled one.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct IdentityDrift {
|
||||
/// The object identifier this observation pertains to.
|
||||
pub id: String,
|
||||
/// Cosine distance (1 - cosine similarity) clamped to `[0, 1]`.
|
||||
pub drift: f32,
|
||||
/// Whether the drift exceeded the configured threshold (identity changed).
|
||||
pub changed: bool,
|
||||
}
|
||||
|
||||
/// A trusted memory of resonant signatures keyed by object identity.
|
||||
///
|
||||
/// Stores one EWMA-smoothed signature per enrolled object. Observations that
|
||||
/// stay within the drift threshold slowly update the stored signature; large
|
||||
/// jumps are flagged and left to update the memory (the stored signature is
|
||||
/// preserved so a transient tamper does not poison the baseline).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct IdentityMemory {
|
||||
signatures: HashMap<String, Vec<f32>>,
|
||||
drift_threshold: f32,
|
||||
alpha: f32,
|
||||
}
|
||||
|
||||
impl IdentityMemory {
|
||||
/// Create an empty identity memory.
|
||||
///
|
||||
/// `drift_threshold` in `[0, 1]`: cosine-distance above which identity is
|
||||
/// considered changed. `alpha` in `[0, 1]`: EWMA update rate for the stored
|
||||
/// signature when identity is unchanged (higher = faster adaptation). Both
|
||||
/// are clamped to `[0, 1]` defensively.
|
||||
pub fn new(drift_threshold: f32, alpha: f32) -> Self {
|
||||
Self {
|
||||
signatures: HashMap::new(),
|
||||
drift_threshold: drift_threshold.clamp(0.0, 1.0),
|
||||
alpha: alpha.clamp(0.0, 1.0),
|
||||
}
|
||||
}
|
||||
|
||||
/// Enroll (or overwrite) a known object's resonant signature embedding.
|
||||
pub fn enroll(&mut self, id: impl Into<String>, signature: Vec<f32>) {
|
||||
self.signatures.insert(id.into(), signature);
|
||||
}
|
||||
|
||||
/// Whether an id is enrolled.
|
||||
pub fn contains(&self, id: &str) -> bool {
|
||||
self.signatures.contains_key(id)
|
||||
}
|
||||
|
||||
/// Compare a fresh signature to the stored one.
|
||||
///
|
||||
/// Returns `drift` = cosine distance (1 - cosine similarity) clamped to
|
||||
/// `[0, 1]`, and `changed` = `drift > threshold`. If unchanged, the stored
|
||||
/// signature is EWMA-updated (slow adaptation to aging/noise). If the id is
|
||||
/// unknown, the signature is auto-enrolled and `drift = 0.0`,
|
||||
/// `changed = false` is returned. A length mismatch against the stored
|
||||
/// signature is treated as a change (`drift = 1.0`) without updating.
|
||||
pub fn observe(&mut self, id: &str, signature: &[f32]) -> IdentityDrift {
|
||||
let Some(stored) = self.signatures.get(id) else {
|
||||
self.signatures.insert(id.to_string(), signature.to_vec());
|
||||
return IdentityDrift {
|
||||
id: id.to_string(),
|
||||
drift: 0.0,
|
||||
changed: false,
|
||||
};
|
||||
};
|
||||
|
||||
if stored.len() != signature.len() {
|
||||
return IdentityDrift {
|
||||
id: id.to_string(),
|
||||
drift: 1.0,
|
||||
changed: true,
|
||||
};
|
||||
}
|
||||
|
||||
let drift = cosine_distance(stored, signature);
|
||||
let changed = drift > self.drift_threshold;
|
||||
|
||||
if !changed {
|
||||
let alpha = self.alpha;
|
||||
// Update in place: stored = (1 - alpha) * stored + alpha * signature.
|
||||
if let Some(stored_mut) = self.signatures.get_mut(id) {
|
||||
for (s, &fresh) in stored_mut.iter_mut().zip(signature.iter()) {
|
||||
*s = (1.0 - alpha) * *s + alpha * fresh;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
IdentityDrift {
|
||||
id: id.to_string(),
|
||||
drift,
|
||||
changed,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Cosine distance `1 - cos_sim`, clamped to `[0, 1]`.
|
||||
///
|
||||
/// Guards zero norms: if either vector has (near-)zero norm, distance is `1.0`
|
||||
/// when the other vector is non-zero, else `0.0` (both effectively silent =
|
||||
/// indistinguishable).
|
||||
fn cosine_distance(a: &[f32], b: &[f32]) -> f32 {
|
||||
let norm_a = dot(a, a).sqrt();
|
||||
let norm_b = dot(b, b).sqrt();
|
||||
const EPS: f32 = 1e-12;
|
||||
|
||||
let a_zero = norm_a <= EPS;
|
||||
let b_zero = norm_b <= EPS;
|
||||
if a_zero || b_zero {
|
||||
return if a_zero && b_zero { 0.0 } else { 1.0 };
|
||||
}
|
||||
|
||||
let cos_sim = dot(a, b) / (norm_a * norm_b);
|
||||
(1.0 - cos_sim).clamp(0.0, 1.0)
|
||||
}
|
||||
|
||||
/// Dot product of two equal-length slices.
|
||||
fn dot(a: &[f32], b: &[f32]) -> f32 {
|
||||
a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn near_identical_signature_is_unchanged() {
|
||||
let mut mem = IdentityMemory::new(0.1, 0.2);
|
||||
mem.enroll("pump-7", vec![1.0, 2.0, 3.0, 4.0]);
|
||||
|
||||
// Tiny perturbation (sensor noise): same physical object.
|
||||
let result = mem.observe("pump-7", &[1.01, 1.99, 3.02, 3.98]);
|
||||
|
||||
assert!(
|
||||
!result.changed,
|
||||
"near-identical signature should be unchanged"
|
||||
);
|
||||
assert!(
|
||||
result.drift < 0.1,
|
||||
"drift should be low, got {}",
|
||||
result.drift
|
||||
);
|
||||
assert_eq!(result.id, "pump-7");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn large_change_trips_changed() {
|
||||
let mut mem = IdentityMemory::new(0.2, 0.2);
|
||||
mem.enroll("panel-3", vec![1.0, 0.0, 0.0, 0.0]);
|
||||
|
||||
// Orthogonal signature — panel loosened, resonance shifted entirely.
|
||||
let result = mem.observe("panel-3", &[0.0, 1.0, 0.0, 0.0]);
|
||||
|
||||
assert!(result.changed, "orthogonal signature should be a change");
|
||||
assert!(
|
||||
result.drift > 0.2,
|
||||
"drift should be high, got {}",
|
||||
result.drift
|
||||
);
|
||||
// Cosine distance of orthogonal vectors is exactly 1.0.
|
||||
assert!((result.drift - 1.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_id_auto_enrolls() {
|
||||
let mut mem = IdentityMemory::new(0.1, 0.2);
|
||||
assert!(!mem.contains("valve-1"));
|
||||
|
||||
let result = mem.observe("valve-1", &[0.5, 0.5, 0.5]);
|
||||
|
||||
assert!(!result.changed);
|
||||
assert_eq!(result.drift, 0.0);
|
||||
assert!(
|
||||
mem.contains("valve-1"),
|
||||
"observing unknown id should enroll it"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn length_mismatch_is_a_change() {
|
||||
let mut mem = IdentityMemory::new(0.1, 0.2);
|
||||
mem.enroll("bearing-2", vec![1.0, 2.0, 3.0]);
|
||||
|
||||
let result = mem.observe("bearing-2", &[1.0, 2.0]);
|
||||
|
||||
assert!(result.changed);
|
||||
assert_eq!(result.drift, 1.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gradual_drift_absorbed_then_sudden_change_trips() {
|
||||
let mut mem = IdentityMemory::new(0.15, 0.3);
|
||||
mem.enroll("casing-9", vec![1.0, 1.0, 1.0, 1.0]);
|
||||
|
||||
// A slow walk of small perturbations: each step is tiny relative to the
|
||||
// current baseline, so EWMA absorbs it and identity stays the same.
|
||||
let mut current = vec![1.0_f32, 1.0, 1.0, 1.0];
|
||||
for step in 0..20 {
|
||||
let nudge = (step as f32) * 0.01;
|
||||
current = vec![
|
||||
1.0 + nudge,
|
||||
1.0 - nudge * 0.5,
|
||||
1.0 + nudge * 0.3,
|
||||
1.0 - nudge * 0.2,
|
||||
];
|
||||
let r = mem.observe("casing-9", ¤t);
|
||||
assert!(
|
||||
!r.changed,
|
||||
"gradual step {step} should stay unchanged (drift {})",
|
||||
r.drift
|
||||
);
|
||||
}
|
||||
|
||||
// Sudden large change — casing tampered: resonance inverts.
|
||||
let tampered = vec![-1.0, -1.0, -1.0, -1.0];
|
||||
let r = mem.observe("casing-9", &tampered);
|
||||
assert!(
|
||||
r.changed,
|
||||
"sudden inversion should trip changed (drift {})",
|
||||
r.drift
|
||||
);
|
||||
assert!(r.drift > 0.15);
|
||||
}
|
||||
}
|
||||
93
crates/ruvector-perception/src/lib.rs
Normal file
93
crates/ruvector-perception/src/lib.rs
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
//! # ruvector-perception — the layer *under* classification
|
||||
//!
|
||||
//! Current WiFi/edge SOTA is racing toward better *classifiers* (CSI foundation
|
||||
//! models, self-supervised CSI representations, adaptive fusion). This crate
|
||||
//! deliberately does **not** build a better classifier. It builds the substrate
|
||||
//! underneath one:
|
||||
//!
|
||||
//! ```text
|
||||
//! classification → confidence → alert (today)
|
||||
//! delta → boundary → coherence → proof → action (here)
|
||||
//! ```
|
||||
//!
|
||||
//! Instead of asking *"what is this?"* it asks *"what changed, where did the
|
||||
//! boundary move, and is the change coherent enough to act on?"* — and it
|
||||
//! requires **evidence**, not confidence, before it grants any authority.
|
||||
//!
|
||||
//! ## Pipeline
|
||||
//!
|
||||
//! 1. **Delta** ([`state`], [`engine`]) — every reading becomes a delta against a
|
||||
//! rolling multi-modal baseline. No fixed task label (fall/gesture/leak).
|
||||
//! 2. **Boundary** ([`coherence`]) — zones form a coherence graph; dynamic
|
||||
//! min-cut isolates the side that broke away (the moved boundary).
|
||||
//! 3. **Coherence + contradiction** — a modality that *usually* reacts in a zone
|
||||
//! but stayed silent is a first-class contradiction (disagreement is
|
||||
//! information), weighted by the modality's physical spoof-resistance.
|
||||
//! 4. **Proof** ([`witness`]) — a proof gate turns novelty/coherence/
|
||||
//! contradiction into *bounded authority* (Ignore → Observe → Alert →
|
||||
//! Mutate) and emits an auditable SHA-256 evidence chain.
|
||||
//! 5. **Action** — only evidence that is novel, coherent, and uncontradicted may
|
||||
//! escalate; contradicted evidence is capped at *Observe*.
|
||||
//!
|
||||
//! Plus [`absence`]: a *missing* expected continuation (e.g. a bedtime routine
|
||||
//! that never returns) is detected as structural incompleteness, not a threshold.
|
||||
//!
|
||||
//! ## Honest scope
|
||||
//!
|
||||
//! This is the **mechanism** (a trusted-physical-memory engine), demonstrated on
|
||||
//! synthetic multi-modal deltas and reusing [`ruvector_mincut`] for boundary
|
||||
//! detection. It is not validated on real CSI hardware, and it is not a
|
||||
//! classifier — it is the auditable perception layer a classifier (or an agent)
|
||||
//! would sit on top of.
|
||||
//!
|
||||
//! ## Example
|
||||
//!
|
||||
//! ```
|
||||
//! use ruvector_perception::{DeltaEngine, EngineConfig, Reading, Modality, Action};
|
||||
//!
|
||||
//! let mut eng = DeltaEngine::new(EngineConfig::default());
|
||||
//! // (warm up baselines first in real use)
|
||||
//! let w = eng.observe(&[
|
||||
//! Reading::new("table_left_zone", Modality::Rf, 3.0),
|
||||
//! Reading::new("table_left_zone", Modality::Vibration, 3.0),
|
||||
//! ], 0);
|
||||
//! assert_eq!(w.changed_boundary, "table_left_zone");
|
||||
//! let _ = Action::Observe;
|
||||
//! ```
|
||||
|
||||
#![forbid(unsafe_code)]
|
||||
|
||||
pub mod absence;
|
||||
pub mod captcha;
|
||||
pub mod coherence;
|
||||
pub mod custody;
|
||||
pub mod engine;
|
||||
pub mod hypothesis;
|
||||
pub mod identity;
|
||||
pub mod modality;
|
||||
pub mod node;
|
||||
pub mod predict;
|
||||
pub mod reality;
|
||||
pub mod state;
|
||||
pub mod swarm;
|
||||
pub mod topology;
|
||||
pub mod witness;
|
||||
|
||||
pub use absence::{Absence, SequenceMonitor};
|
||||
pub use captcha::{CaptchaVerifier, ChallengeResponse, ObservedResponse, RealityProof, Stimulus};
|
||||
pub use coherence::{detect_boundary, Boundary};
|
||||
pub use custody::{CustodyError, CustodyLedger, CustodyRecord};
|
||||
pub use engine::{DeltaEngine, EngineConfig};
|
||||
pub use hypothesis::{rank_hypotheses, DisagreementInput, Hypothesis, RankedHypothesis};
|
||||
pub use identity::{IdentityDrift, IdentityMemory};
|
||||
pub use modality::{Modality, Physics};
|
||||
pub use node::{NervousSystemNode, NodeEvent};
|
||||
pub use predict::{BoundaryForecast, BoundaryObservation, BoundaryPredictor};
|
||||
pub use reality::{GroundedAnswer, Query, RealityGraph};
|
||||
pub use state::{Reading, WorldState};
|
||||
pub use swarm::{FacilityGraph, FragilityReport};
|
||||
pub use topology::{NodeAssessment, NodeRole, TopologyManager};
|
||||
pub use witness::{evidence_hash, novelty_level, Action, DeltaWitness, ProofGate};
|
||||
|
||||
/// Crate version.
|
||||
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
114
crates/ruvector-perception/src/modality.rs
Normal file
114
crates/ruvector-perception/src/modality.rs
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
//! Physically-typed sensing modalities (substrate-aware: each modality has its
|
||||
//! own latency, decay, and spoof-resistance — edges in the coherence graph are
|
||||
//! not generic, they carry physics).
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A physical sensing modality. The graph is *typed*: an RF edge does not behave
|
||||
/// like a thermal edge.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
pub enum Modality {
|
||||
/// Radio (WiFi CSI, BLE RSSI) — fast, multipath-sensitive, easy to spoof statically.
|
||||
Rf,
|
||||
/// Structural vibration (piezo / accelerometer) — propagation delay, damping.
|
||||
Vibration,
|
||||
/// Acoustic (mic) — echo paths, directionality.
|
||||
Acoustic,
|
||||
/// Thermal — slow diffusion, hysteresis; responds to animate heat sources.
|
||||
Thermal,
|
||||
/// Chemical (gas / QCM / SAW) — very slow, leak/identity cues.
|
||||
Chemical,
|
||||
/// Optical / light modulation.
|
||||
Optical,
|
||||
}
|
||||
|
||||
impl Modality {
|
||||
/// All modalities, for iteration.
|
||||
pub const ALL: [Modality; 6] = [
|
||||
Modality::Rf,
|
||||
Modality::Vibration,
|
||||
Modality::Acoustic,
|
||||
Modality::Thermal,
|
||||
Modality::Chemical,
|
||||
Modality::Optical,
|
||||
];
|
||||
|
||||
/// Short stable name (used in witnesses and hashing).
|
||||
pub fn name(self) -> &'static str {
|
||||
match self {
|
||||
Modality::Rf => "rf",
|
||||
Modality::Vibration => "vibration",
|
||||
Modality::Acoustic => "acoustic",
|
||||
Modality::Thermal => "thermal",
|
||||
Modality::Chemical => "chemical",
|
||||
Modality::Optical => "optical",
|
||||
}
|
||||
}
|
||||
|
||||
/// Typed physics metadata used to weight evidence.
|
||||
pub fn physics(self) -> Physics {
|
||||
match self {
|
||||
Modality::Rf => Physics {
|
||||
latency: 0.01,
|
||||
decay: 0.2,
|
||||
spoof_resistance: 0.3,
|
||||
},
|
||||
Modality::Vibration => Physics {
|
||||
latency: 0.05,
|
||||
decay: 0.5,
|
||||
spoof_resistance: 0.7,
|
||||
},
|
||||
Modality::Acoustic => Physics {
|
||||
latency: 0.03,
|
||||
decay: 0.4,
|
||||
spoof_resistance: 0.6,
|
||||
},
|
||||
Modality::Thermal => Physics {
|
||||
latency: 2.0,
|
||||
decay: 0.95,
|
||||
spoof_resistance: 0.8,
|
||||
},
|
||||
Modality::Chemical => Physics {
|
||||
latency: 5.0,
|
||||
decay: 0.98,
|
||||
spoof_resistance: 0.9,
|
||||
},
|
||||
Modality::Optical => Physics {
|
||||
latency: 0.005,
|
||||
decay: 0.1,
|
||||
spoof_resistance: 0.2,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Physical constants attached to a modality edge.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Physics {
|
||||
/// Characteristic response latency (seconds).
|
||||
pub latency: f32,
|
||||
/// Temporal persistence in `[0, 1]` (how slowly a change fades).
|
||||
pub decay: f32,
|
||||
/// Resistance to static spoofing / replay in `[0, 1]` (higher = harder to fake).
|
||||
pub spoof_resistance: f32,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn names_unique_and_physics_present() {
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
for m in Modality::ALL {
|
||||
assert!(seen.insert(m.name()));
|
||||
let p = m.physics();
|
||||
assert!(p.spoof_resistance >= 0.0 && p.spoof_resistance <= 1.0);
|
||||
}
|
||||
// Thermal is slower and harder to spoof than RF — a real physical prior.
|
||||
assert!(Modality::Thermal.physics().latency > Modality::Rf.physics().latency);
|
||||
assert!(
|
||||
Modality::Thermal.physics().spoof_resistance > Modality::Rf.physics().spoof_resistance
|
||||
);
|
||||
}
|
||||
}
|
||||
157
crates/ruvector-perception/src/node.rs
Normal file
157
crates/ruvector-perception/src/node.rs
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
//! Ambient nervous-system node — the appliance surface.
|
||||
//!
|
||||
//! Wires the perception substrate into one local "coherence node" for a room /
|
||||
//! machine / building. It does **not** stream raw sensor data; it ingests
|
||||
//! readings and emits **deltas, boundaries, coherence, proof-gated witnesses,
|
||||
//! forecasts, and an auditable custody chain** — and answers grounded agent
|
||||
//! queries. Not a camera, not an IoT hub, not a dashboard.
|
||||
|
||||
use crate::custody::{CustodyError, CustodyLedger};
|
||||
use crate::engine::{DeltaEngine, EngineConfig};
|
||||
use crate::predict::{BoundaryForecast, BoundaryObservation, BoundaryPredictor};
|
||||
use crate::reality::{GroundedAnswer, Query, RealityGraph};
|
||||
use crate::state::Reading;
|
||||
use crate::witness::DeltaWitness;
|
||||
|
||||
/// What the node emits per observed window — structure, never raw signal.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct NodeEvent {
|
||||
/// The proof-gated delta witness for this window.
|
||||
pub witness: DeltaWitness,
|
||||
/// Where coherence is forecast to break next (if anywhere).
|
||||
pub forecast: Option<BoundaryForecast>,
|
||||
}
|
||||
|
||||
/// A self-contained ambient perception node.
|
||||
pub struct NervousSystemNode {
|
||||
engine: DeltaEngine,
|
||||
reality: RealityGraph,
|
||||
ledger: CustodyLedger,
|
||||
predictor: BoundaryPredictor,
|
||||
}
|
||||
|
||||
impl NervousSystemNode {
|
||||
/// Build a node. `predict_window` is the per-zone history length used by the
|
||||
/// boundary-break forecaster.
|
||||
pub fn new(config: EngineConfig, predict_window: usize) -> Self {
|
||||
Self {
|
||||
engine: DeltaEngine::new(config),
|
||||
reality: RealityGraph::new(),
|
||||
ledger: CustodyLedger::new(),
|
||||
predictor: BoundaryPredictor::new(predict_window),
|
||||
}
|
||||
}
|
||||
|
||||
/// Observe one window of multi-modal readings. Runs the full pipeline
|
||||
/// (delta → boundary → coherence → proof → action), appends the witness to
|
||||
/// the custody chain, grounds it into the reality graph, updates the
|
||||
/// forecaster, and returns the emitted [`NodeEvent`].
|
||||
pub fn observe(&mut self, readings: &[Reading], t: u64) -> NodeEvent {
|
||||
let witness = self.engine.observe(readings, t);
|
||||
// Maintain the auditable chain (the engine produces a linked witness
|
||||
// chain, so append links cleanly).
|
||||
let _ = self.ledger.append(witness.clone());
|
||||
self.reality.ingest(&witness);
|
||||
if !witness.changed_boundary.is_empty() {
|
||||
self.predictor.observe(&BoundaryObservation::new(
|
||||
witness.changed_boundary.clone(),
|
||||
witness.coherence,
|
||||
witness.contradiction,
|
||||
t,
|
||||
));
|
||||
}
|
||||
let forecast = self.predictor.next_break();
|
||||
NodeEvent { witness, forecast }
|
||||
}
|
||||
|
||||
/// Answer a grounded agent query from physical memory.
|
||||
pub fn query(&self, q: &Query) -> GroundedAnswer {
|
||||
self.reality.query(q)
|
||||
}
|
||||
|
||||
/// The auditable custody ledger (chain of every emitted witness).
|
||||
pub fn ledger(&self) -> &CustodyLedger {
|
||||
&self.ledger
|
||||
}
|
||||
|
||||
/// Verify the integrity of the custody chain.
|
||||
pub fn verify_custody(&self) -> Result<(), CustodyError> {
|
||||
self.ledger.verify()
|
||||
}
|
||||
|
||||
/// The grounding reality graph.
|
||||
pub fn reality(&self) -> &RealityGraph {
|
||||
&self.reality
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::modality::Modality;
|
||||
|
||||
fn warm(node: &mut NervousSystemNode) {
|
||||
// Three zones so the changed-boundary is unambiguous (2-zone min-cut
|
||||
// splits are symmetric and the minority side is arbitrary).
|
||||
for i in 0..8u64 {
|
||||
let hi = (i % 2) as f32;
|
||||
node.observe(
|
||||
&[
|
||||
Reading::new("zone_a", Modality::Rf, hi),
|
||||
Reading::new("zone_a", Modality::Vibration, hi),
|
||||
Reading::new("zone_a", Modality::Thermal, 20.0 + hi),
|
||||
Reading::new("zone_b", Modality::Rf, 0.0),
|
||||
Reading::new("zone_c", Modality::Rf, 0.0),
|
||||
],
|
||||
i,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn node_emits_witness_chain_and_grounds_queries() {
|
||||
let mut node = NervousSystemNode::new(EngineConfig::default(), 16);
|
||||
warm(&mut node);
|
||||
let n_before = node.ledger().len();
|
||||
|
||||
// An RF/vibration event in zone_a (thermal silent).
|
||||
let ev = node.observe(
|
||||
&[
|
||||
Reading::new("zone_a", Modality::Rf, 5.0),
|
||||
Reading::new("zone_a", Modality::Vibration, 5.0),
|
||||
Reading::new("zone_a", Modality::Thermal, 20.5),
|
||||
Reading::new("zone_b", Modality::Rf, 0.0),
|
||||
Reading::new("zone_c", Modality::Rf, 0.0),
|
||||
],
|
||||
100,
|
||||
);
|
||||
assert_eq!(ev.witness.changed_boundary, "zone_a");
|
||||
|
||||
// Custody chain grew and verifies.
|
||||
assert_eq!(node.ledger().len(), n_before + 1);
|
||||
assert!(node.verify_custody().is_ok());
|
||||
|
||||
// The agent can query reality, grounded in a witness evidence hash.
|
||||
let presence = node.query(&Query::Presence {
|
||||
zone: "zone_a".into(),
|
||||
});
|
||||
assert!(presence.yes);
|
||||
assert!(!presence.evidence.is_empty());
|
||||
// A zone with no memory is honestly unknown.
|
||||
assert!(
|
||||
!node
|
||||
.query(&Query::Presence {
|
||||
zone: "unknown".into()
|
||||
})
|
||||
.yes
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_node_is_safe() {
|
||||
let node = NervousSystemNode::new(EngineConfig::default(), 8);
|
||||
assert!(node.ledger().is_empty());
|
||||
assert!(node.verify_custody().is_ok());
|
||||
assert!(!node.query(&Query::Presence { zone: "x".into() }).yes);
|
||||
}
|
||||
}
|
||||
318
crates/ruvector-perception/src/predict.rs
Normal file
318
crates/ruvector-perception/src/predict.rs
Normal file
|
|
@ -0,0 +1,318 @@
|
|||
//! Boundary-first world model.
|
||||
//!
|
||||
//! Conventional world models predict the *full* next state and measure error
|
||||
//! against it. That is expensive and, for a perception substrate, beside the
|
||||
//! point: we do not care what every zone will read next, we care **where
|
||||
//! coherence will break next**. So instead of forecasting state, this module
|
||||
//! forecasts the *boundary*:
|
||||
//!
|
||||
//! ```text
|
||||
//! boundary_{t+1} = f(boundary_t, delta_history, modality_conflict)
|
||||
//! ```
|
||||
//!
|
||||
//! Each zone keeps a short rolling history of an *instability* sample. The
|
||||
//! per-observation sample combines how cleanly a boundary recurs (its
|
||||
//! `coherence`) with how much the modalities disagree about it (its
|
||||
//! `contradiction`):
|
||||
//!
|
||||
//! ```text
|
||||
//! instability = coherence * (1 + contradiction)
|
||||
//! ```
|
||||
//!
|
||||
//! A clean boundary that keeps recurring *with* contradictions is the most
|
||||
//! destabilising: it is consistent enough to be real and conflicted enough to be
|
||||
//! unresolved. From the window we read a *level* (mean) and a *trend* (slope),
|
||||
//! and forecast `level + trend` for the next step. The zone with the highest
|
||||
//! forecast is the one most likely to break next.
|
||||
//!
|
||||
//! The model is deterministic, allocation-light, and uses only `std` + `serde`.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A single observed boundary event for one zone at one time step.
|
||||
///
|
||||
/// `coherence` is how cleanly the boundary separated (see
|
||||
/// [`crate::coherence::Boundary::coherence`]); `contradiction` is how strongly
|
||||
/// the modalities disagreed about it. Both are expected in `[0, 1]` but are
|
||||
/// clamped defensively.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BoundaryObservation {
|
||||
/// The zone this event concerns.
|
||||
pub zone: String,
|
||||
/// Cleanliness of the boundary in `[0, 1]` (high = sharp separation).
|
||||
pub coherence: f32,
|
||||
/// Modality disagreement in `[0, 1]` (high = unresolved conflict).
|
||||
pub contradiction: f32,
|
||||
/// Logical time of the observation.
|
||||
pub t: u64,
|
||||
}
|
||||
|
||||
impl BoundaryObservation {
|
||||
/// Convenience constructor.
|
||||
pub fn new(zone: impl Into<String>, coherence: f32, contradiction: f32, t: u64) -> Self {
|
||||
Self {
|
||||
zone: zone.into(),
|
||||
coherence,
|
||||
contradiction,
|
||||
t,
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-observation instability sample: `coherence * (1 + contradiction)`.
|
||||
///
|
||||
/// Inputs are clamped to `[0, 1]`, so the result lies in `[0, 2]`.
|
||||
fn instability_sample(&self) -> f32 {
|
||||
let coh = self.coherence.clamp(0.0, 1.0);
|
||||
let con = self.contradiction.clamp(0.0, 1.0);
|
||||
coh * (1.0 + con)
|
||||
}
|
||||
}
|
||||
|
||||
/// A forecast of where coherence will break next, for a single zone.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct BoundaryForecast {
|
||||
/// The zone this forecast concerns.
|
||||
pub zone: String,
|
||||
/// Forecast instability for the next step, `(level + trend).max(0.0)`.
|
||||
pub instability: f32,
|
||||
/// Slope of the recent window. Positive = the boundary is worsening.
|
||||
pub trend: f32,
|
||||
}
|
||||
|
||||
/// Predicts which boundary breaks next from a rolling per-zone history.
|
||||
///
|
||||
/// Construct with [`BoundaryPredictor::new`], feed events with
|
||||
/// [`BoundaryPredictor::observe`], then read [`BoundaryPredictor::forecast`] or
|
||||
/// [`BoundaryPredictor::next_break`].
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BoundaryPredictor {
|
||||
/// Rolling window length kept per zone (at least 1).
|
||||
window: usize,
|
||||
/// Per-zone rolling instability samples, oldest first.
|
||||
///
|
||||
/// `BTreeMap` keeps iteration deterministic so equal forecasts keep a stable
|
||||
/// (alphabetical) order after the instability sort.
|
||||
history: BTreeMap<String, Vec<f32>>,
|
||||
}
|
||||
|
||||
impl BoundaryPredictor {
|
||||
/// Create a predictor keeping a rolling window of `window` samples per zone.
|
||||
///
|
||||
/// A `window` of `0` is treated as `1` (a forecast needs at least one
|
||||
/// sample), keeping the type total and panic-free.
|
||||
pub fn new(window: usize) -> Self {
|
||||
Self {
|
||||
window: window.max(1),
|
||||
history: BTreeMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Record an observed boundary event for a zone at time `t`.
|
||||
///
|
||||
/// The derived instability sample is appended to that zone's window; the
|
||||
/// oldest sample is evicted once the window is full. The `t` field is part
|
||||
/// of the public record but does not affect the rolling order, which is the
|
||||
/// order of `observe` calls (callers are expected to feed events in time
|
||||
/// order, as the rest of the pipeline does).
|
||||
pub fn observe(&mut self, obs: &BoundaryObservation) {
|
||||
let sample = obs.instability_sample();
|
||||
let win = self.window;
|
||||
let series = self.history.entry(obs.zone.clone()).or_default();
|
||||
series.push(sample);
|
||||
if series.len() > win {
|
||||
// Drop the oldest sample to keep the rolling window bounded.
|
||||
let overflow = series.len() - win;
|
||||
series.drain(0..overflow);
|
||||
}
|
||||
}
|
||||
|
||||
/// Forecast per-zone instability for the next step, sorted by instability
|
||||
/// descending (ties broken by zone name for determinism).
|
||||
///
|
||||
/// Returns an empty vector if nothing has been observed.
|
||||
pub fn forecast(&self) -> Vec<BoundaryForecast> {
|
||||
let mut out: Vec<BoundaryForecast> = self
|
||||
.history
|
||||
.iter()
|
||||
.filter(|(_, series)| !series.is_empty())
|
||||
.map(|(zone, series)| {
|
||||
let level = mean(series);
|
||||
let trend = slope(series);
|
||||
let instability = (level + trend).max(0.0);
|
||||
BoundaryForecast {
|
||||
zone: zone.clone(),
|
||||
instability,
|
||||
trend,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
out.sort_by(|a, b| {
|
||||
b.instability
|
||||
.partial_cmp(&a.instability)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
.then_with(|| a.zone.cmp(&b.zone))
|
||||
});
|
||||
out
|
||||
}
|
||||
|
||||
/// The single most-likely zone to break next (highest forecast
|
||||
/// instability), or `None` if nothing has been observed.
|
||||
pub fn next_break(&self) -> Option<BoundaryForecast> {
|
||||
self.forecast().into_iter().next()
|
||||
}
|
||||
}
|
||||
|
||||
/// Arithmetic mean of a non-empty slice. Returns `0.0` for an empty slice.
|
||||
fn mean(xs: &[f32]) -> f32 {
|
||||
if xs.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
xs.iter().sum::<f32>() / xs.len() as f32
|
||||
}
|
||||
|
||||
/// Least-squares slope of `xs` over indices `0..len`.
|
||||
///
|
||||
/// Equivalent to the trend of the rolling window: positive means the boundary
|
||||
/// is worsening. Returns `0.0` for fewer than two samples (a single point has
|
||||
/// no trend).
|
||||
fn slope(xs: &[f32]) -> f32 {
|
||||
let n = xs.len();
|
||||
if n < 2 {
|
||||
return 0.0;
|
||||
}
|
||||
let n_f = n as f32;
|
||||
// x is the integer index 0..n; mean_x = (n-1)/2.
|
||||
let mean_x = (n_f - 1.0) / 2.0;
|
||||
let mean_y = mean(xs);
|
||||
let mut num = 0.0f32;
|
||||
let mut den = 0.0f32;
|
||||
for (i, &y) in xs.iter().enumerate() {
|
||||
let dx = i as f32 - mean_x;
|
||||
num += dx * (y - mean_y);
|
||||
den += dx * dx;
|
||||
}
|
||||
if den == 0.0 {
|
||||
0.0
|
||||
} else {
|
||||
num / den
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn rising_zone_has_positive_trend_and_higher_forecast() {
|
||||
let mut p = BoundaryPredictor::new(5);
|
||||
|
||||
// "kitchen" worsens: coherence and contradiction both climb.
|
||||
for (i, (coh, con)) in [(0.1, 0.0), (0.3, 0.2), (0.5, 0.4), (0.7, 0.6), (0.9, 0.8)]
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
{
|
||||
p.observe(&BoundaryObservation::new("kitchen", coh, con, i as u64));
|
||||
}
|
||||
|
||||
// "hallway" stays calm: low coherence, no contradiction.
|
||||
for i in 0..5 {
|
||||
p.observe(&BoundaryObservation::new("hallway", 0.1, 0.0, i as u64));
|
||||
}
|
||||
|
||||
let forecast = p.forecast();
|
||||
assert_eq!(forecast.len(), 2);
|
||||
|
||||
let kitchen = forecast.iter().find(|f| f.zone == "kitchen").unwrap();
|
||||
let hallway = forecast.iter().find(|f| f.zone == "hallway").unwrap();
|
||||
|
||||
// The worsening zone has a clearly positive trend...
|
||||
assert!(
|
||||
kitchen.trend > 0.0,
|
||||
"rising zone should have positive trend, got {}",
|
||||
kitchen.trend
|
||||
);
|
||||
// ...the calm zone is flat...
|
||||
assert!(
|
||||
hallway.trend.abs() < 1e-6,
|
||||
"stable zone should be flat, got {}",
|
||||
hallway.trend
|
||||
);
|
||||
// ...and forecast instability is higher for the worsening zone.
|
||||
assert!(
|
||||
kitchen.instability > hallway.instability,
|
||||
"rising {} should exceed stable {}",
|
||||
kitchen.instability,
|
||||
hallway.instability
|
||||
);
|
||||
|
||||
// Sorted descending: the worsening zone leads.
|
||||
assert_eq!(forecast[0].zone, "kitchen");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_break_returns_the_rising_zone() {
|
||||
let mut p = BoundaryPredictor::new(4);
|
||||
for i in 0..4 {
|
||||
let coh = 0.2 + 0.2 * i as f32;
|
||||
let con = 0.1 * i as f32;
|
||||
p.observe(&BoundaryObservation::new("garage", coh, con, i as u64));
|
||||
p.observe(&BoundaryObservation::new("porch", 0.05, 0.0, i as u64));
|
||||
}
|
||||
|
||||
let next = p.next_break().expect("a break should be predicted");
|
||||
assert_eq!(next.zone, "garage");
|
||||
assert!(next.instability > 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_predictor_yields_nothing() {
|
||||
let p = BoundaryPredictor::new(8);
|
||||
assert!(p.forecast().is_empty());
|
||||
assert!(p.next_break().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn window_evicts_oldest_samples() {
|
||||
let mut p = BoundaryPredictor::new(2);
|
||||
// Early calm sample then two strong ones; with window 2 the calm sample
|
||||
// is evicted, so the level reflects only the recent strong activity.
|
||||
p.observe(&BoundaryObservation::new("z", 0.0, 0.0, 0));
|
||||
p.observe(&BoundaryObservation::new("z", 0.9, 0.9, 1));
|
||||
p.observe(&BoundaryObservation::new("z", 0.9, 0.9, 2));
|
||||
|
||||
let f = p.next_break().unwrap();
|
||||
// Both retained samples are identical => flat trend, high level.
|
||||
assert!(f.trend.abs() < 1e-6);
|
||||
assert!(f.instability > 1.0, "got {}", f.instability);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn instability_sample_is_clamped() {
|
||||
// Out-of-range inputs are clamped to [0, 1] before combining.
|
||||
let obs = BoundaryObservation::new("z", 2.0, 5.0, 0);
|
||||
// coherence -> 1.0, contradiction -> 1.0 => 1.0 * (1 + 1) = 2.0
|
||||
assert!((obs.instability_sample() - 2.0).abs() < 1e-6);
|
||||
|
||||
let neg = BoundaryObservation::new("z", -1.0, -1.0, 0);
|
||||
assert!(neg.instability_sample().abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn forecast_is_deterministic_and_serializable() {
|
||||
let mut p = BoundaryPredictor::new(3);
|
||||
p.observe(&BoundaryObservation::new("a", 0.5, 0.5, 0));
|
||||
p.observe(&BoundaryObservation::new("b", 0.5, 0.5, 0));
|
||||
|
||||
let f = p.forecast();
|
||||
// Equal instability => alphabetical tie-break is stable.
|
||||
assert_eq!(f[0].zone, "a");
|
||||
assert_eq!(f[1].zone, "b");
|
||||
|
||||
let json = serde_json::to_string(&f[0]).unwrap();
|
||||
let back: BoundaryForecast = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(f[0], back);
|
||||
}
|
||||
}
|
||||
250
crates/ruvector-perception/src/reality.rs
Normal file
250
crates/ruvector-perception/src/reality.rs
Normal file
|
|
@ -0,0 +1,250 @@
|
|||
//! Reality graph — grounding layer for agents.
|
||||
//!
|
||||
//! Agents hallucinate because they reason from prompts, not physical state. This
|
||||
//! module lets an agent **query reality**: every answer is backed by witnessed
|
||||
//! evidence (the [`DeltaWitness`] evidence hashes that justify it), not by text
|
||||
//! inference. The agent asks "is anyone in the room? what changed since last
|
||||
//! hour? which sensor is lying? is this action allowed?" and the reality graph
|
||||
//! answers from physical memory.
|
||||
|
||||
use crate::witness::{Action, DeltaWitness};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
/// A grounding question an agent can ask the physical world.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum Query {
|
||||
/// Is something currently happening / present in a zone?
|
||||
Presence { zone: String },
|
||||
/// Which zones changed (acted-upon) at or after time `t`?
|
||||
ChangedSince { t: u64 },
|
||||
/// Which zones carry contradicted / untrusted evidence right now?
|
||||
WhichUntrusted,
|
||||
/// Is escalation (Alert/Mutate) currently permitted in a zone?
|
||||
ActionAllowed { zone: String },
|
||||
/// The most recent witness for a zone.
|
||||
LastWitness { zone: String },
|
||||
}
|
||||
|
||||
/// A witness-grounded answer. `evidence` lists the SHA-256 evidence hashes that
|
||||
/// justify the answer — provenance, not prose.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct GroundedAnswer {
|
||||
/// Boolean verdict (for yes/no queries; `false` when not applicable).
|
||||
pub yes: bool,
|
||||
/// Human-readable, fully grounded explanation.
|
||||
pub detail: String,
|
||||
/// Zones relevant to the answer (sorted).
|
||||
pub zones: Vec<String>,
|
||||
/// Supporting evidence hashes (the witnesses backing this answer).
|
||||
pub evidence: Vec<String>,
|
||||
/// Aggregate coherence of the supporting evidence, `[0, 1]`.
|
||||
pub coherence: f32,
|
||||
}
|
||||
|
||||
impl GroundedAnswer {
|
||||
fn none(detail: impl Into<String>) -> Self {
|
||||
Self {
|
||||
yes: false,
|
||||
detail: detail.into(),
|
||||
zones: Vec::new(),
|
||||
evidence: Vec::new(),
|
||||
coherence: 0.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Physical-memory graph queried by agents. Holds the latest witness per zone.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct RealityGraph {
|
||||
latest: BTreeMap<String, DeltaWitness>,
|
||||
}
|
||||
|
||||
impl RealityGraph {
|
||||
/// Create an empty reality graph.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Fold a witness into physical memory (keyed by its changed boundary zone).
|
||||
pub fn ingest(&mut self, w: &DeltaWitness) {
|
||||
if w.changed_boundary.is_empty() {
|
||||
return;
|
||||
}
|
||||
self.latest.insert(w.changed_boundary.clone(), w.clone());
|
||||
}
|
||||
|
||||
/// Zones known to the reality graph (sorted).
|
||||
pub fn zones(&self) -> Vec<String> {
|
||||
self.latest.keys().cloned().collect()
|
||||
}
|
||||
|
||||
/// Answer a grounding query from physical memory.
|
||||
pub fn query(&self, q: &Query) -> GroundedAnswer {
|
||||
match q {
|
||||
Query::Presence { zone } => match self.latest.get(zone) {
|
||||
Some(w) if w.action != Action::Ignore => GroundedAnswer {
|
||||
yes: true,
|
||||
detail: format!(
|
||||
"activity in {zone}: {} supporting modality(ies), novelty {:.2}, action {:?}",
|
||||
w.supporting_modalities.len(),
|
||||
w.novelty,
|
||||
w.action
|
||||
),
|
||||
zones: vec![zone.clone()],
|
||||
evidence: vec![w.evidence_hash.clone()],
|
||||
coherence: w.coherence,
|
||||
},
|
||||
Some(w) => GroundedAnswer {
|
||||
yes: false,
|
||||
detail: format!("{zone} quiet (last action Ignore)"),
|
||||
zones: vec![zone.clone()],
|
||||
evidence: vec![w.evidence_hash.clone()],
|
||||
coherence: w.coherence,
|
||||
},
|
||||
None => GroundedAnswer::none(format!("no physical memory for {zone}")),
|
||||
},
|
||||
Query::ChangedSince { t } => {
|
||||
let mut zones = Vec::new();
|
||||
let mut evidence = Vec::new();
|
||||
let mut coh = 0.0f32;
|
||||
for (z, w) in &self.latest {
|
||||
if w.t >= *t && w.action != Action::Ignore {
|
||||
zones.push(z.clone());
|
||||
evidence.push(w.evidence_hash.clone());
|
||||
coh = coh.max(w.coherence);
|
||||
}
|
||||
}
|
||||
GroundedAnswer {
|
||||
yes: !zones.is_empty(),
|
||||
detail: format!("{} zone(s) changed since t={t}", zones.len()),
|
||||
zones,
|
||||
evidence,
|
||||
coherence: coh,
|
||||
}
|
||||
}
|
||||
Query::WhichUntrusted => {
|
||||
let mut zones = Vec::new();
|
||||
let mut evidence = Vec::new();
|
||||
let mut coh = 0.0f32;
|
||||
for (z, w) in &self.latest {
|
||||
if w.contradiction > 0.0 {
|
||||
zones.push(z.clone());
|
||||
evidence.push(w.evidence_hash.clone());
|
||||
coh = coh.max(w.contradiction);
|
||||
}
|
||||
}
|
||||
GroundedAnswer {
|
||||
yes: !zones.is_empty(),
|
||||
detail: format!(
|
||||
"{} zone(s) carry contradicted evidence (a modality that usually reacts stayed silent)",
|
||||
zones.len()
|
||||
),
|
||||
zones,
|
||||
evidence,
|
||||
coherence: coh,
|
||||
}
|
||||
}
|
||||
Query::ActionAllowed { zone } => match self.latest.get(zone) {
|
||||
Some(w) => {
|
||||
let allowed = matches!(w.action, Action::Alert | Action::Mutate);
|
||||
GroundedAnswer {
|
||||
yes: allowed,
|
||||
detail: if allowed {
|
||||
format!("escalation permitted in {zone}: evidence is novel, coherent, uncontradicted ({:?})", w.action)
|
||||
} else {
|
||||
format!("escalation NOT permitted in {zone}: action capped at {:?} (contradiction {:.2})", w.action, w.contradiction)
|
||||
},
|
||||
zones: vec![zone.clone()],
|
||||
evidence: vec![w.evidence_hash.clone()],
|
||||
coherence: w.coherence,
|
||||
}
|
||||
}
|
||||
None => GroundedAnswer::none(format!("no physical memory for {zone}; action denied by default")),
|
||||
},
|
||||
Query::LastWitness { zone } => match self.latest.get(zone) {
|
||||
Some(w) => GroundedAnswer {
|
||||
yes: true,
|
||||
detail: format!("last witness for {zone} at t={}, action {:?}", w.t, w.action),
|
||||
zones: vec![zone.clone()],
|
||||
evidence: vec![w.evidence_hash.clone()],
|
||||
coherence: w.coherence,
|
||||
},
|
||||
None => GroundedAnswer::none(format!("no physical memory for {zone}")),
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::modality::Modality;
|
||||
|
||||
fn witness(zone: &str, t: u64, action: Action, contradiction: f32, hash: &str) -> DeltaWitness {
|
||||
DeltaWitness {
|
||||
t,
|
||||
changed_boundary: zone.to_string(),
|
||||
supporting_modalities: vec![Modality::Rf, Modality::Vibration],
|
||||
contradicting_modalities: if contradiction > 0.0 {
|
||||
vec![Modality::Thermal]
|
||||
} else {
|
||||
vec![]
|
||||
},
|
||||
novelty: 0.8,
|
||||
coherence: 0.9,
|
||||
contradiction,
|
||||
action,
|
||||
evidence_hash: hash.to_string(),
|
||||
prev_hash: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn presence_is_grounded_in_a_witness() {
|
||||
let mut rg = RealityGraph::new();
|
||||
rg.ingest(&witness("kitchen", 5, Action::Alert, 0.0, "h1"));
|
||||
let a = rg.query(&Query::Presence {
|
||||
zone: "kitchen".into(),
|
||||
});
|
||||
assert!(a.yes);
|
||||
assert_eq!(a.evidence, vec!["h1".to_string()]);
|
||||
// A zone with no memory is honestly unknown, not hallucinated.
|
||||
let b = rg.query(&Query::Presence {
|
||||
zone: "garage".into(),
|
||||
});
|
||||
assert!(!b.yes);
|
||||
assert!(b.evidence.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn untrusted_and_action_gate() {
|
||||
let mut rg = RealityGraph::new();
|
||||
rg.ingest(&witness("door", 1, Action::Observe, 0.8, "hc")); // contradicted -> Observe
|
||||
rg.ingest(&witness("hall", 2, Action::Mutate, 0.0, "hm")); // clean -> Mutate
|
||||
let untrusted = rg.query(&Query::WhichUntrusted);
|
||||
assert_eq!(untrusted.zones, vec!["door".to_string()]);
|
||||
// Contradicted zone: escalation denied. Clean zone: allowed.
|
||||
assert!(
|
||||
!rg.query(&Query::ActionAllowed {
|
||||
zone: "door".into()
|
||||
})
|
||||
.yes
|
||||
);
|
||||
assert!(
|
||||
rg.query(&Query::ActionAllowed {
|
||||
zone: "hall".into()
|
||||
})
|
||||
.yes
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn changed_since_filters_by_time() {
|
||||
let mut rg = RealityGraph::new();
|
||||
rg.ingest(&witness("a", 1, Action::Alert, 0.0, "ha"));
|
||||
rg.ingest(&witness("b", 9, Action::Alert, 0.0, "hb"));
|
||||
let a = rg.query(&Query::ChangedSince { t: 5 });
|
||||
assert_eq!(a.zones, vec!["b".to_string()]);
|
||||
}
|
||||
}
|
||||
138
crates/ruvector-perception/src/state.rs
Normal file
138
crates/ruvector-perception/src/state.rs
Normal file
|
|
@ -0,0 +1,138 @@
|
|||
//! Physical state history: per-(zone, modality) rolling baselines and how
|
||||
//! *responsive* each sensor usually is in each zone (used to detect a sensor
|
||||
//! that "should have reacted but didn't" — the contradiction signal).
|
||||
|
||||
use crate::modality::Modality;
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// A single sensor sample in one zone at one time window.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Reading {
|
||||
/// Human-readable zone name (e.g. "table_left_zone").
|
||||
pub zone: String,
|
||||
/// Which modality produced the sample.
|
||||
pub modality: Modality,
|
||||
/// Scalar value (already feature-extracted, e.g. band energy).
|
||||
pub value: f32,
|
||||
}
|
||||
|
||||
impl Reading {
|
||||
/// Convenience constructor.
|
||||
pub fn new(zone: impl Into<String>, modality: Modality, value: f32) -> Self {
|
||||
Self {
|
||||
zone: zone.into(),
|
||||
modality,
|
||||
value,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-(zone, modality) running statistics.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
struct Channel {
|
||||
/// EWMA baseline of the value.
|
||||
baseline: f32,
|
||||
/// EWMA of |delta| magnitude — the channel's typical activity.
|
||||
activity: f32,
|
||||
/// Fraction of updates with a significant delta (responsiveness in [0,1]).
|
||||
responsiveness: f32,
|
||||
/// Whether the channel has been initialised.
|
||||
seen: bool,
|
||||
}
|
||||
|
||||
impl Default for Channel {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
baseline: 0.0,
|
||||
activity: 0.0,
|
||||
responsiveness: 0.0,
|
||||
seen: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Rolling multi-modal world state.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct WorldState {
|
||||
channels: HashMap<(String, Modality), Channel>,
|
||||
alpha: f32, // EWMA smoothing
|
||||
active_threshold: f32, // |delta| above this counts as "responded"
|
||||
}
|
||||
|
||||
impl WorldState {
|
||||
/// New state. `alpha` is the EWMA factor (e.g. 0.3); `active_threshold` is
|
||||
/// the |delta| above which a channel is considered to have responded.
|
||||
pub fn new(alpha: f32, active_threshold: f32) -> Self {
|
||||
Self {
|
||||
channels: HashMap::new(),
|
||||
alpha,
|
||||
active_threshold,
|
||||
}
|
||||
}
|
||||
|
||||
/// Current baseline for a channel (0 if unseen).
|
||||
pub fn baseline(&self, zone: &str, m: Modality) -> f32 {
|
||||
self.channels
|
||||
.get(&(zone.to_string(), m))
|
||||
.map(|c| c.baseline)
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
|
||||
/// How responsive a channel historically is, in `[0, 1]`.
|
||||
pub fn responsiveness(&self, zone: &str, m: Modality) -> f32 {
|
||||
self.channels
|
||||
.get(&(zone.to_string(), m))
|
||||
.map(|c| c.responsiveness)
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
|
||||
/// Whether a channel has any history.
|
||||
pub fn seen(&self, zone: &str, m: Modality) -> bool {
|
||||
self.channels
|
||||
.get(&(zone.to_string(), m))
|
||||
.map(|c| c.seen)
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Threshold above which a |delta| counts as a response.
|
||||
pub fn active_threshold(&self) -> f32 {
|
||||
self.active_threshold
|
||||
}
|
||||
|
||||
/// Fold a reading into the rolling state (after its delta has been read).
|
||||
pub fn update(&mut self, r: &Reading) {
|
||||
let key = (r.zone.clone(), r.modality);
|
||||
let a = self.alpha;
|
||||
let thr = self.active_threshold;
|
||||
let ch = self.channels.entry(key).or_default();
|
||||
if !ch.seen {
|
||||
ch.baseline = r.value;
|
||||
ch.seen = true;
|
||||
return;
|
||||
}
|
||||
let delta = (r.value - ch.baseline).abs();
|
||||
let responded = if delta >= thr { 1.0 } else { 0.0 };
|
||||
ch.activity = (1.0 - a) * ch.activity + a * delta;
|
||||
ch.responsiveness = (1.0 - a) * ch.responsiveness + a * responded;
|
||||
ch.baseline = (1.0 - a) * ch.baseline + a * r.value;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn tracks_baseline_and_responsiveness() {
|
||||
let mut s = WorldState::new(0.5, 0.5);
|
||||
// Thermal in zone A reacts repeatedly -> high responsiveness.
|
||||
for v in [0.0, 1.0, 0.0, 1.0, 0.0, 1.0] {
|
||||
s.update(&Reading::new("A", Modality::Thermal, v));
|
||||
}
|
||||
assert!(s.seen("A", Modality::Thermal));
|
||||
assert!(s.responsiveness("A", Modality::Thermal) > 0.4);
|
||||
// An unseen channel is quiet.
|
||||
assert!(!s.seen("A", Modality::Rf));
|
||||
assert_eq!(s.responsiveness("A", Modality::Rf), 0.0);
|
||||
}
|
||||
}
|
||||
284
crates/ruvector-perception/src/swarm.rs
Normal file
284
crates/ruvector-perception/src/swarm.rs
Normal file
|
|
@ -0,0 +1,284 @@
|
|||
//! Swarm-scale min-cut sensing — *where* a coupled system is closest to breaking.
|
||||
//!
|
||||
//! At facility or city scale every room, machine, or router is a node in a
|
||||
//! **coupling graph**: an edge weight is how strongly two nodes hold each other
|
||||
//! in a coherent operating state (shared load, redundant links, correlated
|
||||
//! environment). The operational question is not *"which sensor crossed a
|
||||
//! threshold?"* but *"WHERE is the whole structure closest to fragmenting?"* —
|
||||
//! and that is answered, globally, by the minimum cut.
|
||||
//!
|
||||
//! The global **min-cut value** is the total coupling that would have to fail for
|
||||
//! the facility to split into two pieces: a low value means the system is one
|
||||
//! weak link away from breaking apart (fragile); a high value means it is
|
||||
//! robustly interconnected. The **bottleneck** nodes are those touching a
|
||||
//! crossing edge — the load-bearing joints where the break would happen.
|
||||
//!
|
||||
//! This reuses [`ruvector_mincut`] for the cut. The cut *value* is authoritative;
|
||||
//! the returned partition is best-effort (the engine may peel a single weakly
|
||||
//! connected node rather than return a balanced split), so all decision-relevant
|
||||
//! output keys on the **value** and on the **bottleneck set**, never on an exact
|
||||
//! balanced partition.
|
||||
//!
|
||||
//! ## Example
|
||||
//!
|
||||
//! ```
|
||||
//! use ruvector_perception::FacilityGraph;
|
||||
//!
|
||||
//! let mut g = FacilityGraph::new();
|
||||
//! // Two tight clusters joined by one thin link.
|
||||
//! g.couple("r1", "r2", 10.0);
|
||||
//! g.couple("r2", "r3", 10.0);
|
||||
//! g.couple("r3", "r4", 0.5); // the fragile joint
|
||||
//! g.couple("r4", "r5", 10.0);
|
||||
//! g.couple("r5", "r6", 10.0);
|
||||
//!
|
||||
//! let report = g.fragility().unwrap();
|
||||
//! assert!((report.min_cut - 0.5).abs() < 1e-6);
|
||||
//! assert!(report.bottlenecks.contains(&"r3".to_string())
|
||||
//! || report.bottlenecks.contains(&"r4".to_string()));
|
||||
//! ```
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use ruvector_mincut::MinCutBuilder;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Where a coupled facility is structurally closest to fragmenting.
|
||||
///
|
||||
/// The headline number is [`min_cut`](FragilityReport::min_cut): the total
|
||||
/// coupling weight that would have to fail for the system to split. Lower means
|
||||
/// more fragile. [`bottlenecks`](FragilityReport::bottlenecks) lists the
|
||||
/// load-bearing joints — nodes touching a crossing edge.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct FragilityReport {
|
||||
/// Global min-cut weight = how close the facility is to breaking apart.
|
||||
/// Lower = more fragile.
|
||||
pub min_cut: f64,
|
||||
/// One side of the fragile partition (best-effort; do not rely on balance).
|
||||
pub side_a: Vec<String>,
|
||||
/// The other side of the fragile partition (best-effort; may be empty).
|
||||
pub side_b: Vec<String>,
|
||||
/// Nodes incident to a crossing (cut) edge — the structural bottlenecks.
|
||||
/// Sorted and deduped.
|
||||
pub bottlenecks: Vec<String>,
|
||||
}
|
||||
|
||||
/// A facility-scale coupling graph: nodes are rooms/machines/routers, edges are
|
||||
/// undirected coupling strengths that accumulate across calls.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct FacilityGraph {
|
||||
/// Distinct node names.
|
||||
nodes: BTreeSet<String>,
|
||||
/// Summed undirected coupling, keyed by the ordered `(min, max)` name pair.
|
||||
edges: BTreeMap<(String, String), f64>,
|
||||
}
|
||||
|
||||
impl FacilityGraph {
|
||||
/// Create an empty facility graph.
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Add (or accumulate) an undirected coupling strength between two facility
|
||||
/// nodes. Repeated calls on the same unordered pair **sum** into one total
|
||||
/// coupling weight.
|
||||
///
|
||||
/// Non-positive weights and self-loops (`a == b`) are ignored, matching the
|
||||
/// min-cut engine's requirement of positive weights on distinct endpoints.
|
||||
pub fn couple(&mut self, a: impl Into<String>, b: impl Into<String>, weight: f64) {
|
||||
let a = a.into();
|
||||
let b = b.into();
|
||||
if a == b || !weight.is_finite() || weight <= 0.0 {
|
||||
return;
|
||||
}
|
||||
self.nodes.insert(a.clone());
|
||||
self.nodes.insert(b.clone());
|
||||
let key = if a <= b { (a, b) } else { (b, a) };
|
||||
*self.edges.entry(key).or_insert(0.0) += weight;
|
||||
}
|
||||
|
||||
/// Number of distinct nodes in the graph.
|
||||
pub fn len(&self) -> usize {
|
||||
self.nodes.len()
|
||||
}
|
||||
|
||||
/// Whether the graph has no nodes.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.nodes.is_empty()
|
||||
}
|
||||
|
||||
/// Compute the global minimum cut: the place where the facility is
|
||||
/// structurally closest to fragmenting.
|
||||
///
|
||||
/// Returns `None` if there are fewer than two nodes or no edges. Otherwise
|
||||
/// the [`FragilityReport`] always carries a trustworthy
|
||||
/// [`min_cut`](FragilityReport::min_cut) value; the sides are best-effort and
|
||||
/// `bottlenecks` lists the nodes touching a crossing edge (sorted, deduped).
|
||||
/// Never panics.
|
||||
pub fn fragility(&self) -> Option<FragilityReport> {
|
||||
if self.nodes.len() < 2 || self.edges.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
// Stable name <-> id mapping (BTreeSet iterates in sorted order).
|
||||
let names: Vec<String> = self.nodes.iter().cloned().collect();
|
||||
let id_of: BTreeMap<&str, u64> = names
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, n)| (n.as_str(), i as u64))
|
||||
.collect();
|
||||
|
||||
let edges: Vec<(u64, u64, f64)> = self
|
||||
.edges
|
||||
.iter()
|
||||
.map(|((u, v), w)| (id_of[u.as_str()], id_of[v.as_str()], *w))
|
||||
.collect();
|
||||
|
||||
let mincut = MinCutBuilder::new()
|
||||
.exact()
|
||||
.with_edges(edges)
|
||||
.build()
|
||||
.ok()?;
|
||||
let result = mincut.min_cut();
|
||||
let min_cut = result.value;
|
||||
|
||||
// Best-effort sides from the engine partition. Fall back to "all on one
|
||||
// side" when the engine returns no usable split.
|
||||
let (side_a_ids, side_b_ids): (Vec<u64>, Vec<u64>) = match result.partition {
|
||||
Some((a, b)) if !a.is_empty() && !b.is_empty() => (a, b),
|
||||
_ => ((0..names.len() as u64).collect(), Vec::new()),
|
||||
};
|
||||
|
||||
let mut side_a: Vec<String> = side_a_ids
|
||||
.iter()
|
||||
.map(|&i| names[i as usize].clone())
|
||||
.collect();
|
||||
let mut side_b: Vec<String> = side_b_ids
|
||||
.iter()
|
||||
.map(|&i| names[i as usize].clone())
|
||||
.collect();
|
||||
side_a.sort();
|
||||
side_b.sort();
|
||||
|
||||
// Bottlenecks = endpoints of the WEAKEST link(s) — the fragile joints
|
||||
// where a break would occur. Derived from edge weights, NOT the engine
|
||||
// partition: the engine's `min_cut` value is reliable but the partition
|
||||
// it materialises can be inconsistent with that value (it sometimes
|
||||
// peels a single node), so partition-crossing edges are not trustworthy
|
||||
// bottleneck markers. The weakest edge is the true structural weak point.
|
||||
let min_w = self.edges.values().copied().fold(f64::INFINITY, f64::min);
|
||||
let mut bottleneck_set: BTreeSet<String> = BTreeSet::new();
|
||||
for ((u, v), &w) in &self.edges {
|
||||
if (w - min_w).abs() <= 1e-9 {
|
||||
bottleneck_set.insert(u.clone());
|
||||
bottleneck_set.insert(v.clone());
|
||||
}
|
||||
}
|
||||
Some(FragilityReport {
|
||||
min_cut,
|
||||
side_a,
|
||||
side_b,
|
||||
bottlenecks: bottleneck_set.into_iter().collect(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn thin_link_between_two_clusters_is_the_fragility() {
|
||||
// Cluster {r1,r2,r3} tightly coupled, cluster {r4,r5,r6} tightly coupled,
|
||||
// joined only by the thin link r3<->r4 (weight 0.5).
|
||||
let mut g = FacilityGraph::new();
|
||||
for &(a, b) in &[("r1", "r2"), ("r2", "r3"), ("r1", "r3")] {
|
||||
g.couple(a, b, 10.0);
|
||||
}
|
||||
for &(a, b) in &[("r4", "r5"), ("r5", "r6"), ("r4", "r6")] {
|
||||
g.couple(a, b, 10.0);
|
||||
}
|
||||
g.couple("r3", "r4", 0.5);
|
||||
|
||||
assert_eq!(g.len(), 6);
|
||||
let report = g.fragility().expect("two clusters -> a report");
|
||||
|
||||
// The weakest crossing is exactly the thin link.
|
||||
assert!(
|
||||
(report.min_cut - 0.5).abs() < 1e-6,
|
||||
"min_cut = {}",
|
||||
report.min_cut
|
||||
);
|
||||
// ...and it is far below the intra-cluster coupling.
|
||||
assert!(report.min_cut < 10.0);
|
||||
// The fragile joint is r3 or r4.
|
||||
assert!(
|
||||
report.bottlenecks.contains(&"r3".to_string())
|
||||
|| report.bottlenecks.contains(&"r4".to_string()),
|
||||
"bottlenecks = {:?}",
|
||||
report.bottlenecks
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fewer_than_two_nodes_is_none() {
|
||||
let empty = FacilityGraph::new();
|
||||
assert!(empty.is_empty());
|
||||
assert_eq!(empty.fragility(), None);
|
||||
|
||||
// A single self-loop is ignored, so still no graph.
|
||||
let mut single = FacilityGraph::new();
|
||||
single.couple("only", "only", 5.0);
|
||||
assert!(single.is_empty());
|
||||
assert_eq!(single.fragility(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn uniform_clique_isolates_a_single_node() {
|
||||
// Strongly, uniformly coupled clique over 5 nodes. The cheapest cut is
|
||||
// to isolate one node: (k-1) * weight.
|
||||
let mut g = FacilityGraph::new();
|
||||
let nodes = ["a", "b", "c", "d", "e"];
|
||||
let weight = 2.0;
|
||||
for i in 0..nodes.len() {
|
||||
for j in (i + 1)..nodes.len() {
|
||||
g.couple(nodes[i], nodes[j], weight);
|
||||
}
|
||||
}
|
||||
let report = g.fragility().expect("clique -> a report");
|
||||
|
||||
let isolation_cost = (nodes.len() as f64 - 1.0) * weight; // 4 * 2 = 8
|
||||
assert!(
|
||||
(report.min_cut - isolation_cost).abs() < 1e-6,
|
||||
"min_cut = {}",
|
||||
report.min_cut
|
||||
);
|
||||
assert!(report.min_cut > 0.0);
|
||||
assert!(!report.bottlenecks.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repeated_couplings_sum() {
|
||||
let mut g = FacilityGraph::new();
|
||||
g.couple("x", "y", 1.5);
|
||||
g.couple("y", "x", 2.5); // same unordered pair, reversed
|
||||
// Only one edge, so the only cut separates the two nodes: total = 4.0.
|
||||
let report = g.fragility().expect("two coupled nodes -> a report");
|
||||
assert!(
|
||||
(report.min_cut - 4.0).abs() < 1e-6,
|
||||
"min_cut = {}",
|
||||
report.min_cut
|
||||
);
|
||||
assert_eq!(g.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_positive_and_self_weights_ignored() {
|
||||
let mut g = FacilityGraph::new();
|
||||
g.couple("a", "b", 0.0);
|
||||
g.couple("a", "b", -3.0);
|
||||
g.couple("a", "a", 5.0);
|
||||
assert!(g.is_empty());
|
||||
assert_eq!(g.fragility(), None);
|
||||
}
|
||||
}
|
||||
401
crates/ruvector-perception/src/topology.rs
Normal file
401
crates/ruvector-perception/src/topology.rs
Normal file
|
|
@ -0,0 +1,401 @@
|
|||
//! # Self-healing sensor topology
|
||||
//!
|
||||
//! Sensors are not equal, and which ones *matter* changes over time. This module
|
||||
//! keeps a running **agreement graph** between sensors (how often each pair
|
||||
//! corroborates the other) and lets that graph reorganise itself so the system
|
||||
//! can answer one operational question: *what is each sensor's structural role
|
||||
//! right now?*
|
||||
//!
|
||||
//! Every node is classified into one of four [`NodeRole`]s:
|
||||
//!
|
||||
//! - **Critical** — removing it would fragment the topology. It is the sole (or
|
||||
//! near-sole) strong link bridging two otherwise-disconnected clusters.
|
||||
//! Detected with a dynamic global **min-cut**: a node on the min-cut boundary
|
||||
//! that carries a crossing edge and has few strong alternatives is a bridge.
|
||||
//! - **Redundant** — it has a near-duplicate peer (very high agreement with at
|
||||
//! least one other sensor), so it could be put to sleep without losing
|
||||
//! coverage.
|
||||
//! - **Noisy** — it disagrees with essentially everyone (low mean agreement);
|
||||
//! its readings are not corroborated and should be discounted.
|
||||
//! - **Normal** — none of the above.
|
||||
//!
|
||||
//! The agreement between two sensors is accumulated as an **EWMA** (exponential
|
||||
//! weighted moving average, `alpha = 0.3`) over repeated [`record_agreement`]
|
||||
//! calls, so the topology drifts toward recent behaviour while staying stable.
|
||||
//!
|
||||
//! ```
|
||||
//! use ruvector_perception::topology::{TopologyManager, NodeRole};
|
||||
//!
|
||||
//! let mut topo = TopologyManager::new();
|
||||
//! topo.record_agreement("cam_a", "cam_b", 0.95); // near-duplicates
|
||||
//! topo.record_agreement("cam_a", "mic_x", 0.6);
|
||||
//! topo.record_agreement("cam_b", "mic_x", 0.6);
|
||||
//! let report = topo.assess();
|
||||
//! assert!(report.iter().any(|a| a.role == NodeRole::Redundant));
|
||||
//! ```
|
||||
//!
|
||||
//! [`record_agreement`]: TopologyManager::record_agreement
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// EWMA smoothing factor for accumulated pairwise agreement.
|
||||
const ALPHA: f32 = 0.3;
|
||||
/// Minimum agreement weight for an edge to count as a topology link at all.
|
||||
/// Edges below this floor are treated as "no meaningful link".
|
||||
const EDGE_FLOOR: f32 = 0.05;
|
||||
/// Below this *mean* incident agreement a node is considered [`NodeRole::Noisy`].
|
||||
const NOISY_MEAN: f32 = 0.3;
|
||||
/// At or above this *max* incident agreement a node has a near-duplicate peer
|
||||
/// and is considered [`NodeRole::Redundant`].
|
||||
const REDUNDANT_MAX: f32 = 0.85;
|
||||
/// Minimum number of sensors for articulation (bridge) detection to be meaningful.
|
||||
const MIN_NODES_FOR_BRIDGE: usize = 3;
|
||||
|
||||
/// Structural role of a sensor within the agreement topology.
|
||||
///
|
||||
/// Ordering of precedence when more than one rule fires is documented on
|
||||
/// [`TopologyManager::assess`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
pub enum NodeRole {
|
||||
/// Bridges two clusters; its loss would fragment the topology.
|
||||
Critical,
|
||||
/// Has a near-duplicate peer and could be put to sleep.
|
||||
Redundant,
|
||||
/// Disagrees with (almost) everyone; readings are uncorroborated.
|
||||
Noisy,
|
||||
/// No special structural role.
|
||||
Normal,
|
||||
}
|
||||
|
||||
/// Per-node assessment produced by [`TopologyManager::assess`].
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct NodeAssessment {
|
||||
/// Sensor name.
|
||||
pub node: String,
|
||||
/// Classified structural role.
|
||||
pub role: NodeRole,
|
||||
/// Maximum incident agreement (how close its nearest peer is, in `[0, 1]`).
|
||||
pub redundancy: f32,
|
||||
/// Mean incident agreement across all of its links, in `[0, 1]`.
|
||||
pub agreement: f32,
|
||||
}
|
||||
|
||||
/// Maintains a self-healing sensor agreement graph and classifies node roles.
|
||||
///
|
||||
/// Pairwise agreement is stored once per unordered pair, keyed by the
|
||||
/// lexicographically ordered `(min, max)` name tuple, and accumulated as an
|
||||
/// EWMA. The set of known sensor names is tracked separately so isolated
|
||||
/// sensors (no edges yet) are still assessed.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct TopologyManager {
|
||||
/// EWMA agreement per unordered pair, key = `(min_name, max_name)`.
|
||||
edges: BTreeMap<(String, String), f32>,
|
||||
/// All sensor names ever observed.
|
||||
nodes: BTreeSet<String>,
|
||||
}
|
||||
|
||||
impl TopologyManager {
|
||||
/// Create an empty topology manager.
|
||||
#[must_use]
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Record (and accumulate) a pairwise agreement score in `[0, 1]` between
|
||||
/// two sensors.
|
||||
///
|
||||
/// Repeated calls update the stored value as an EWMA
|
||||
/// (`new = alpha * score + (1 - alpha) * old`), so the topology adapts to
|
||||
/// recent behaviour. The score is clamped to `[0, 1]`. Self-pairs
|
||||
/// (`a == b`) are ignored. Both names are registered as known sensors even
|
||||
/// if the pair is a self-pair.
|
||||
pub fn record_agreement(&mut self, a: impl Into<String>, b: impl Into<String>, score: f32) {
|
||||
let a = a.into();
|
||||
let b = b.into();
|
||||
self.nodes.insert(a.clone());
|
||||
self.nodes.insert(b.clone());
|
||||
if a == b {
|
||||
return; // no self-loops
|
||||
}
|
||||
let score = score.clamp(0.0, 1.0);
|
||||
let key = if a <= b { (a, b) } else { (b, a) };
|
||||
self.edges
|
||||
.entry(key)
|
||||
.and_modify(|w| *w = ALPHA * score + (1.0 - ALPHA) * *w)
|
||||
.or_insert(score);
|
||||
}
|
||||
|
||||
/// Number of known sensors.
|
||||
#[must_use]
|
||||
pub fn len(&self) -> usize {
|
||||
self.nodes.len()
|
||||
}
|
||||
|
||||
/// Whether no sensors are known yet.
|
||||
#[must_use]
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.nodes.is_empty()
|
||||
}
|
||||
|
||||
/// Assess every node's role from the accumulated agreement graph.
|
||||
///
|
||||
/// Output is sorted by node name for determinism. Role precedence when
|
||||
/// multiple rules could apply: **Critical > Redundant > Noisy > Normal**.
|
||||
/// (A bridge that also happens to be noisy is reported Critical, because its
|
||||
/// structural fragility dominates the operational decision.)
|
||||
///
|
||||
/// Graceful degenerate handling: with fewer than two sensors, or with no
|
||||
/// edges above [`EDGE_FLOOR`], every node is [`NodeRole::Normal`] — except a
|
||||
/// truly isolated node (no incident links at all) which is reported
|
||||
/// [`NodeRole::Noisy`], since nothing corroborates it. Never panics.
|
||||
#[must_use]
|
||||
pub fn assess(&self) -> Vec<NodeAssessment> {
|
||||
if self.nodes.is_empty() {
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
// Stable index for each sensor (BTreeSet iterates in sorted order).
|
||||
let names: Vec<String> = self.nodes.iter().cloned().collect();
|
||||
let index: BTreeMap<&str, usize> = names
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, n)| (n.as_str(), i))
|
||||
.collect();
|
||||
let n = names.len();
|
||||
|
||||
// Incident weights per node (only edges above the floor count).
|
||||
let mut incident: Vec<Vec<f32>> = vec![Vec::new(); n];
|
||||
// Adjacency restricted to floor-passing edges, for the bridge rule.
|
||||
let mut adj: Vec<Vec<(usize, f32)>> = vec![Vec::new(); n];
|
||||
for ((a, b), &w) in &self.edges {
|
||||
if w < EDGE_FLOOR {
|
||||
continue;
|
||||
}
|
||||
let (ia, ib) = (index[a.as_str()], index[b.as_str()]);
|
||||
incident[ia].push(w);
|
||||
incident[ib].push(w);
|
||||
adj[ia].push((ib, w));
|
||||
adj[ib].push((ia, w));
|
||||
}
|
||||
|
||||
// Identify the min-cut bridge boundary once for the whole graph.
|
||||
let critical = self.critical_nodes(n, &adj);
|
||||
|
||||
names
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, name)| {
|
||||
let inc = &incident[i];
|
||||
let (agreement, redundancy) = if inc.is_empty() {
|
||||
(0.0_f32, 0.0_f32)
|
||||
} else {
|
||||
let sum: f32 = inc.iter().sum();
|
||||
let mean = sum / inc.len() as f32;
|
||||
let max = inc.iter().copied().fold(0.0_f32, f32::max);
|
||||
(mean, max)
|
||||
};
|
||||
|
||||
let role = if critical.contains(&i) {
|
||||
NodeRole::Critical
|
||||
} else if redundancy >= REDUNDANT_MAX {
|
||||
NodeRole::Redundant
|
||||
} else if inc.is_empty() || agreement < NOISY_MEAN {
|
||||
NodeRole::Noisy
|
||||
} else {
|
||||
NodeRole::Normal
|
||||
};
|
||||
|
||||
NodeAssessment {
|
||||
node: name.clone(),
|
||||
role,
|
||||
redundancy,
|
||||
agreement,
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Determine which node indices are **structural bridges** (articulation
|
||||
/// points): a node whose removal fragments the strong-edge agreement graph
|
||||
/// into more connected components than before. A bridge is the extreme,
|
||||
/// most fragile cut — a single-edge min cut — so losing such a node splits
|
||||
/// the topology.
|
||||
///
|
||||
/// This is robust where a global-min-cut partition is not: it directly tests
|
||||
/// "does removing this node disconnect the graph?", which cleanly separates a
|
||||
/// true inter-cluster bridge (Critical) from a lone outlier that merely
|
||||
/// peels off (Noisy/Redundant). Isolated nodes (no strong edges) are never
|
||||
/// Critical. Needs at least [`MIN_NODES_FOR_BRIDGE`] sensors to be meaningful.
|
||||
fn critical_nodes(&self, n: usize, adj: &[Vec<(usize, f32)>]) -> BTreeSet<usize> {
|
||||
let mut critical = BTreeSet::new();
|
||||
if n < MIN_NODES_FOR_BRIDGE {
|
||||
return critical;
|
||||
}
|
||||
let base = components(n, adj, None);
|
||||
for v in 0..n {
|
||||
if adj[v].is_empty() {
|
||||
continue; // isolated node can't be a bridge
|
||||
}
|
||||
if components(n, adj, Some(v)) > base {
|
||||
critical.insert(v);
|
||||
}
|
||||
}
|
||||
critical
|
||||
}
|
||||
}
|
||||
|
||||
/// Count connected components among non-isolated nodes, optionally excluding one
|
||||
/// `removed` node (and its incident edges). Used for articulation detection.
|
||||
fn components(n: usize, adj: &[Vec<(usize, f32)>], removed: Option<usize>) -> usize {
|
||||
let mut visited = vec![false; n];
|
||||
if let Some(r) = removed {
|
||||
visited[r] = true;
|
||||
}
|
||||
let mut comps = 0;
|
||||
for start in 0..n {
|
||||
if visited[start] || adj[start].is_empty() {
|
||||
continue; // skip visited and truly isolated nodes
|
||||
}
|
||||
comps += 1;
|
||||
let mut stack = vec![start];
|
||||
visited[start] = true;
|
||||
while let Some(u) = stack.pop() {
|
||||
for &(w, _) in &adj[u] {
|
||||
if Some(w) == removed || visited[w] {
|
||||
continue;
|
||||
}
|
||||
visited[w] = true;
|
||||
stack.push(w);
|
||||
}
|
||||
}
|
||||
}
|
||||
comps
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn role_of<'a>(report: &'a [NodeAssessment], node: &str) -> &'a NodeRole {
|
||||
&report
|
||||
.iter()
|
||||
.find(|a| a.node == node)
|
||||
.unwrap_or_else(|| panic!("node {node} missing from report"))
|
||||
.role
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_manager_is_empty_and_safe() {
|
||||
let topo = TopologyManager::new();
|
||||
assert!(topo.is_empty());
|
||||
assert_eq!(topo.len(), 0);
|
||||
assert!(topo.assess().is_empty()); // no panic, empty result
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn near_duplicate_peer_is_redundant() {
|
||||
let mut topo = TopologyManager::new();
|
||||
// a and b are near-duplicates; both also moderately agree with c.
|
||||
topo.record_agreement("a", "b", 0.95);
|
||||
topo.record_agreement("a", "c", 0.6);
|
||||
topo.record_agreement("b", "c", 0.6);
|
||||
let report = topo.assess();
|
||||
|
||||
// At least one of the duplicate pair is flagged Redundant.
|
||||
let redundant = report
|
||||
.iter()
|
||||
.filter(|x| x.role == NodeRole::Redundant)
|
||||
.count();
|
||||
assert!(redundant >= 1, "expected a redundant node, got {report:?}");
|
||||
// The redundant node should be a or b (high mutual agreement).
|
||||
for a in &report {
|
||||
if a.role == NodeRole::Redundant {
|
||||
assert!(a.node == "a" || a.node == "b", "unexpected redundant {a:?}");
|
||||
assert!(a.redundancy >= REDUNDANT_MAX);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lone_disagreeing_node_is_noisy() {
|
||||
let mut topo = TopologyManager::new();
|
||||
// x corroborates y and z strongly; n disagrees with all (~0.1).
|
||||
topo.record_agreement("x", "y", 0.8);
|
||||
topo.record_agreement("x", "z", 0.8);
|
||||
topo.record_agreement("y", "z", 0.8);
|
||||
topo.record_agreement("n", "x", 0.1);
|
||||
topo.record_agreement("n", "y", 0.1);
|
||||
topo.record_agreement("n", "z", 0.1);
|
||||
let report = topo.assess();
|
||||
|
||||
assert_eq!(
|
||||
*role_of(&report, "n"),
|
||||
NodeRole::Noisy,
|
||||
"report: {report:?}"
|
||||
);
|
||||
// The well-corroborated nodes are not Noisy.
|
||||
assert_ne!(*role_of(&report, "x"), NodeRole::Noisy);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bridge_node_between_two_clusters_is_critical() {
|
||||
let mut topo = TopologyManager::new();
|
||||
// Cluster 1: {a, b, c} tightly agree.
|
||||
topo.record_agreement("a", "b", 0.95);
|
||||
topo.record_agreement("a", "c", 0.95);
|
||||
topo.record_agreement("b", "c", 0.95);
|
||||
// Cluster 2: {d, e, f} tightly agree.
|
||||
topo.record_agreement("d", "e", 0.95);
|
||||
topo.record_agreement("d", "f", 0.95);
|
||||
topo.record_agreement("e", "f", 0.95);
|
||||
// Single fragile link joining the clusters: c <-> d.
|
||||
topo.record_agreement("c", "d", 0.6);
|
||||
|
||||
let report = topo.assess();
|
||||
let critical: Vec<&str> = report
|
||||
.iter()
|
||||
.filter(|x| x.role == NodeRole::Critical)
|
||||
.map(|x| x.node.as_str())
|
||||
.collect();
|
||||
|
||||
// The bridge endpoints (c and d) carry the sole crossing link and have
|
||||
// strongly-connected same-side peers; the min-cut should isolate them
|
||||
// as the boundary, marking at least one bridge endpoint Critical.
|
||||
assert!(
|
||||
critical.contains(&"c") || critical.contains(&"d"),
|
||||
"expected a bridge node (c or d) to be Critical, got critical={critical:?} report={report:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ewma_accumulates_repeated_scores() {
|
||||
let mut topo = TopologyManager::new();
|
||||
topo.record_agreement("p", "q", 1.0); // first observation -> stored as-is
|
||||
topo.record_agreement("p", "q", 0.0); // EWMA pulls it down
|
||||
let report = topo.assess();
|
||||
let p = report.iter().find(|x| x.node == "p").unwrap();
|
||||
// After 1.0 then 0.0: 0.3*0.0 + 0.7*1.0 = 0.7.
|
||||
assert!((p.agreement - 0.7).abs() < 1e-4, "got {}", p.agreement);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn output_is_sorted_by_name() {
|
||||
let mut topo = TopologyManager::new();
|
||||
topo.record_agreement("zebra", "alpha", 0.5);
|
||||
topo.record_agreement("mid", "alpha", 0.5);
|
||||
let report = topo.assess();
|
||||
let names: Vec<&str> = report.iter().map(|a| a.node.as_str()).collect();
|
||||
assert_eq!(names, vec!["alpha", "mid", "zebra"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn single_isolated_sensor_is_noisy_not_panic() {
|
||||
let mut topo = TopologyManager::new();
|
||||
topo.record_agreement("solo", "solo", 0.9); // self-pair ignored as edge
|
||||
let report = topo.assess();
|
||||
assert_eq!(report.len(), 1);
|
||||
assert_eq!(report[0].role, NodeRole::Noisy); // nothing corroborates it
|
||||
}
|
||||
}
|
||||
180
crates/ruvector-perception/src/witness.rs
Normal file
180
crates/ruvector-perception/src/witness.rs
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
//! Proof-gated perception. A physical change may only drive an action if it
|
||||
//! passes a proof gate — an auditable evidence chain (raw hash, feature hash,
|
||||
//! novelty, coherence, contradiction, boundary, policy), not a confidence score.
|
||||
|
||||
use crate::modality::Modality;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
/// Bounded authority the engine may exercise on a witnessed change.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum Action {
|
||||
/// Nothing changed worth noting.
|
||||
Ignore,
|
||||
/// Real but ambiguous/contradicted — keep watching, do not escalate.
|
||||
Observe,
|
||||
/// Coherent, novel, uncontradicted — raise an alert.
|
||||
Alert,
|
||||
/// Strong, clean, uncontradicted — allowed to mutate persistent memory.
|
||||
Mutate,
|
||||
}
|
||||
|
||||
/// Thresholds that turn scores into bounded authority.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct ProofGate {
|
||||
/// Below this novelty, ignore (it's business as usual).
|
||||
pub novelty_min: f32,
|
||||
/// Novelty at/above this is "high".
|
||||
pub novelty_high: f32,
|
||||
/// Minimum boundary coherence to trust the localisation.
|
||||
pub coherence_min: f32,
|
||||
/// At/above this contradiction, never escalate beyond Observe.
|
||||
pub contradiction_max: f32,
|
||||
}
|
||||
|
||||
impl Default for ProofGate {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
novelty_min: 0.25,
|
||||
novelty_high: 0.6,
|
||||
coherence_min: 0.5,
|
||||
contradiction_max: 0.34,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ProofGate {
|
||||
/// Decide bounded authority from the three scores. Contradiction caps
|
||||
/// authority at Observe; only clean, novel, uncontradicted evidence escalates.
|
||||
pub fn decide(&self, novelty: f32, coherence: f32, contradiction: f32) -> Action {
|
||||
if novelty < self.novelty_min {
|
||||
return Action::Ignore;
|
||||
}
|
||||
if contradiction >= self.contradiction_max {
|
||||
return Action::Observe; // evidence is internally inconsistent
|
||||
}
|
||||
if coherence < self.coherence_min {
|
||||
return Action::Observe; // can't trust the localisation
|
||||
}
|
||||
if novelty >= self.novelty_high {
|
||||
// Strong, clean, uncontradicted: highest authority only when
|
||||
// contradiction is essentially absent.
|
||||
if contradiction <= self.contradiction_max * 0.25 {
|
||||
Action::Mutate
|
||||
} else {
|
||||
Action::Alert
|
||||
}
|
||||
} else {
|
||||
Action::Observe
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Qualitative novelty bucket for human-readable witnesses.
|
||||
pub fn novelty_level(n: f32, gate: &ProofGate) -> &'static str {
|
||||
if n >= gate.novelty_high {
|
||||
"high"
|
||||
} else if n >= gate.novelty_min {
|
||||
"medium"
|
||||
} else {
|
||||
"low"
|
||||
}
|
||||
}
|
||||
|
||||
/// The structured output of perception — a delta, not a label.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct DeltaWitness {
|
||||
/// Time window index.
|
||||
pub t: u64,
|
||||
/// The zone whose physical state moved.
|
||||
pub changed_boundary: String,
|
||||
/// Modalities that responded coherently.
|
||||
pub supporting_modalities: Vec<Modality>,
|
||||
/// Modalities that should have responded (historically responsive) but
|
||||
/// stayed silent — first-class disagreement.
|
||||
pub contradicting_modalities: Vec<Modality>,
|
||||
/// Novelty vs prior physical states, `[0, 1]`.
|
||||
pub novelty: f32,
|
||||
/// Boundary coherence (localisation cleanliness), `[0, 1]`.
|
||||
pub coherence: f32,
|
||||
/// Contradiction strength, `[0, 1]`.
|
||||
pub contradiction: f32,
|
||||
/// Bounded authority granted by the proof gate.
|
||||
pub action: Action,
|
||||
/// SHA-256 evidence hash for this witness (hex).
|
||||
pub evidence_hash: String,
|
||||
/// Previous witness hash — forms an auditable chain of custody.
|
||||
pub prev_hash: Option<String>,
|
||||
}
|
||||
|
||||
/// Compute the evidence hash binding raw signal, features, scores, boundary,
|
||||
/// policy, and the prior witness into one auditable digest.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn evidence_hash(
|
||||
raw: &[u8],
|
||||
features: &[u8],
|
||||
boundary: &str,
|
||||
novelty: f32,
|
||||
coherence: f32,
|
||||
contradiction: f32,
|
||||
action: Action,
|
||||
prev: Option<&str>,
|
||||
) -> String {
|
||||
let mut h = Sha256::new();
|
||||
h.update(b"rvperception-v1");
|
||||
h.update((raw.len() as u64).to_le_bytes());
|
||||
h.update(raw);
|
||||
h.update((features.len() as u64).to_le_bytes());
|
||||
h.update(features);
|
||||
h.update(boundary.as_bytes());
|
||||
h.update(novelty.to_le_bytes());
|
||||
h.update(coherence.to_le_bytes());
|
||||
h.update(contradiction.to_le_bytes());
|
||||
h.update([action as u8]);
|
||||
if let Some(p) = prev {
|
||||
h.update(p.as_bytes());
|
||||
}
|
||||
let digest = h.finalize();
|
||||
let mut s = String::with_capacity(64);
|
||||
for b in digest {
|
||||
s.push_str(&format!("{b:02x}"));
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn contradiction_caps_authority_at_observe() {
|
||||
let g = ProofGate::default();
|
||||
// High novelty, clean boundary, but contradicted -> Observe, never Alert.
|
||||
assert_eq!(g.decide(0.9, 0.9, 0.5), Action::Observe);
|
||||
// Clean, novel, uncontradicted -> escalates.
|
||||
assert_eq!(g.decide(0.9, 0.9, 0.0), Action::Mutate);
|
||||
// Below novelty floor -> Ignore.
|
||||
assert_eq!(g.decide(0.1, 0.9, 0.0), Action::Ignore);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn evidence_hash_is_deterministic_and_chains() {
|
||||
let a = evidence_hash(b"raw", b"feat", "zoneA", 0.9, 0.8, 0.1, Action::Alert, None);
|
||||
let b = evidence_hash(b"raw", b"feat", "zoneA", 0.9, 0.8, 0.1, Action::Alert, None);
|
||||
assert_eq!(a, b);
|
||||
assert_eq!(a.len(), 64);
|
||||
// Chaining changes the hash.
|
||||
let c = evidence_hash(
|
||||
b"raw",
|
||||
b"feat",
|
||||
"zoneA",
|
||||
0.9,
|
||||
0.8,
|
||||
0.1,
|
||||
Action::Alert,
|
||||
Some(&a),
|
||||
);
|
||||
assert_ne!(a, c);
|
||||
}
|
||||
}
|
||||
102
crates/ruvector-perception/tests/scenarios.rs
Normal file
102
crates/ruvector-perception/tests/scenarios.rs
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
//! The brief's flagship scenarios:
|
||||
//! 1. Move an inert object: RF/vibration/acoustic support, thermal contradicts,
|
||||
//! novelty high, action = observe (a structured delta witness, not a label).
|
||||
//! 2. A bedtime routine whose return never happens -> absence safety signal.
|
||||
|
||||
use ruvector_perception::{
|
||||
novelty_level, Action, DeltaEngine, EngineConfig, Modality, ProofGate, Reading, SequenceMonitor,
|
||||
};
|
||||
|
||||
fn warmup(eng: &mut DeltaEngine) {
|
||||
// Build responsiveness in table_left_zone across RF/vibration/acoustic/thermal
|
||||
// (all historically react here); other zones stay quiet.
|
||||
for i in 0..8u64 {
|
||||
let hi = (i % 2) as f32;
|
||||
let rs = vec![
|
||||
Reading::new("table_left_zone", Modality::Rf, hi),
|
||||
Reading::new("table_left_zone", Modality::Vibration, hi),
|
||||
Reading::new("table_left_zone", Modality::Acoustic, hi),
|
||||
Reading::new("table_left_zone", Modality::Thermal, 20.0 + hi),
|
||||
Reading::new("table_right_zone", Modality::Rf, 0.0),
|
||||
Reading::new("window_zone", Modality::Rf, 0.0),
|
||||
];
|
||||
eng.observe(&rs, i);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inert_object_move_produces_structured_delta_witness() {
|
||||
let mut eng = DeltaEngine::new(EngineConfig::default());
|
||||
warmup(&mut eng);
|
||||
|
||||
// Construct the event relative to learned baselines: RF/vibration/acoustic
|
||||
// jump (object moved), thermal exactly at baseline (no heat -> silent).
|
||||
let bl = |m| eng.state().baseline("table_left_zone", m);
|
||||
let (bl_rf, bl_vib, bl_ac, bl_th) = (
|
||||
bl(Modality::Rf),
|
||||
bl(Modality::Vibration),
|
||||
bl(Modality::Acoustic),
|
||||
bl(Modality::Thermal),
|
||||
);
|
||||
let event = vec![
|
||||
Reading::new("table_left_zone", Modality::Rf, bl_rf + 3.0),
|
||||
Reading::new("table_left_zone", Modality::Vibration, bl_vib + 3.0),
|
||||
Reading::new("table_left_zone", Modality::Acoustic, bl_ac + 3.0),
|
||||
Reading::new("table_left_zone", Modality::Thermal, bl_th), // silent
|
||||
Reading::new(
|
||||
"table_right_zone",
|
||||
Modality::Rf,
|
||||
eng.state().baseline("table_right_zone", Modality::Rf),
|
||||
),
|
||||
Reading::new(
|
||||
"window_zone",
|
||||
Modality::Rf,
|
||||
eng.state().baseline("window_zone", Modality::Rf),
|
||||
),
|
||||
];
|
||||
|
||||
let prev = eng.state().baseline("table_left_zone", Modality::Rf); // touch state
|
||||
let _ = prev;
|
||||
let w = eng.observe(&event, 100);
|
||||
|
||||
// The exact witness shape from the brief.
|
||||
assert_eq!(w.changed_boundary, "table_left_zone");
|
||||
assert!(w.supporting_modalities.contains(&Modality::Rf));
|
||||
assert!(w.supporting_modalities.contains(&Modality::Vibration));
|
||||
assert!(w.supporting_modalities.contains(&Modality::Acoustic));
|
||||
assert!(!w.supporting_modalities.contains(&Modality::Thermal));
|
||||
assert!(
|
||||
w.contradicting_modalities.contains(&Modality::Thermal),
|
||||
"thermal should contradict (usually reacts here, stayed silent): {:?}",
|
||||
w.contradicting_modalities
|
||||
);
|
||||
assert_eq!(novelty_level(w.novelty, &ProofGate::default()), "high");
|
||||
assert!(w.coherence > 0.5, "boundary not clean: {}", w.coherence);
|
||||
// Contradicted evidence is capped at Observe — it does not escalate.
|
||||
assert_eq!(w.action, Action::Observe);
|
||||
// Auditable evidence chain.
|
||||
assert_eq!(w.evidence_hash.len(), 64);
|
||||
assert!(
|
||||
w.prev_hash.is_some(),
|
||||
"witness should chain to the warmup history"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_routine_return_is_a_safety_signal() {
|
||||
let mut routine = SequenceMonitor::new(
|
||||
vec![
|
||||
"bed_exit".into(),
|
||||
"bathroom_path".into(),
|
||||
"return_path".into(),
|
||||
],
|
||||
100,
|
||||
);
|
||||
routine.observe_zone("bed_exit", 0);
|
||||
routine.observe_zone("bathroom_path", 10);
|
||||
// The return edge never appears: the sequence graph stays incomplete.
|
||||
assert!(routine.check(50).is_none());
|
||||
let absence = routine.check(300).expect("overdue return is a signal");
|
||||
assert_eq!(absence.missing_step, "return_path");
|
||||
assert_eq!(absence.after, "bathroom_path");
|
||||
}
|
||||
|
|
@ -80,6 +80,12 @@ ignore = [
|
|||
# incompatible. Dependabot tracking. Re-review on 2026-07-01.
|
||||
"RUSTSEC-2024-0370",
|
||||
|
||||
# proc-macro-error2 — the (also now unmaintained) fork of the above,
|
||||
# pulled transitively via validator_derive -> validator (ruvector-scipix
|
||||
# example). Same crate family as RUSTSEC-2024-0370, no maintained
|
||||
# successor yet. Informational. Re-review on 2026-07-01.
|
||||
"RUSTSEC-2026-0173",
|
||||
|
||||
# number_prefix — unmaintained, used transitively by indicatif. No
|
||||
# known successor; indicatif itself is still maintained. Informational.
|
||||
"RUSTSEC-2025-0119",
|
||||
|
|
|
|||
168
docs/adr/ADR-196-structure-preserving-graph-condensation.md
Normal file
168
docs/adr/ADR-196-structure-preserving-graph-condensation.md
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
---
|
||||
adr: 196
|
||||
title: "Structure-Preserving Graph Condensation (ruvector-graph-condense)"
|
||||
status: accepted
|
||||
date: 2026-06-07
|
||||
authors: [ruvnet, claude]
|
||||
related: [ADR-197]
|
||||
tags: [graph, condensation, coarsening, min-cut, gnn, ruview, worldgraph, hnsw]
|
||||
---
|
||||
|
||||
# ADR-196 — Structure-Preserving Graph Condensation
|
||||
|
||||
## Status
|
||||
|
||||
**Accepted (implemented).** Crate `crates/ruvector-graph-condense` landed on
|
||||
branch `claude/graph-condensation-ruvector-lVAKm`. ADR-197 covers the
|
||||
differentiable min-cut loss added on top.
|
||||
|
||||
## Context
|
||||
|
||||
We want to shrink large feature graphs (a graph plus a per-node embedding and an
|
||||
optional class label) into a much smaller graph that a downstream consumer —
|
||||
GNN training, edge inference, or RuView's `WorldGraph → OccWorld` retraining
|
||||
pipeline — can use in place of the original. Two bodies of evidence shaped the
|
||||
decision:
|
||||
|
||||
### 1. The SOTA literature (graph condensation, 2022–2026)
|
||||
|
||||
The published field — GCond (gradient matching), DosCond (one-step), GCDM
|
||||
(distribution matching), SFGC (structure-free trajectory matching), SGDD
|
||||
(graphon/Laplacian-Energy-Distribution), GEOM (curriculum trajectories), GC-SNTK
|
||||
(kernel ridge regression), GDEM (eigenbasis), DisCo (disentangled, scales to
|
||||
111M nodes) — defines **condensation** as *synthesising a small fake graph* by
|
||||
optimising a bi-level learning objective so a GNN trained on the synthetic graph
|
||||
matches one trained on the original. That paradigm is:
|
||||
|
||||
- **Expensive** — bi-level optimisation, often second-order, hard to scale past
|
||||
~1M nodes; "lossless" results (GEOM) need 1–5% ratios and banks of expert
|
||||
trajectories.
|
||||
- **Supervised** — requires labels `Y'`.
|
||||
- **Provenance-destroying** — a condensed node is synthetic; the mapping back to
|
||||
real nodes is intentionally discarded. This breaks audit, explainability, and
|
||||
link-back.
|
||||
|
||||
The surveys (arXiv:2401.11720, IJCAI'24 arXiv:2402.03358) and benchmarks (GC4NC,
|
||||
GC-Bench) explicitly flag as **under-explored or unpublished**: community
|
||||
detection (not k-means) as a structural prior, min-cut/modularity objectives in
|
||||
the condensation loss, condensation of temporal/streaming graphs, and
|
||||
condensation co-designed for edge deployment. The closest training-free analogs
|
||||
are CGC (clustering, 2025) and GCTD (tensor decomposition, 2025).
|
||||
|
||||
### 2. The RuVector / RuView substrate
|
||||
|
||||
`ruvector-mincut` already ships the relevant primitives with default features:
|
||||
`DynamicGraph` (streaming insert/delete/update), `CommunityDetector` and
|
||||
`GraphPartitioner` (recursive global min cut), `ClusterHierarchy`, and an exact
|
||||
`MinCutBuilder`. RuView (ruvnet/RuView) consumes RuVector's mincut/HNSW/GNN/RVF
|
||||
primitives and records `WorldGraph` JSON snapshots that feed an OccWorld
|
||||
world-model retrainer — but has **no graph condensation anywhere**, giving this
|
||||
work a concrete downstream consumer.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a new crate, `ruvector-graph-condense`, implementing **training-free,
|
||||
structure-preserving, provenance-retaining** graph condensation built on
|
||||
`ruvector-mincut`. Concretely this is closer to **coarsening with synthetic
|
||||
representatives** than to GCond-style condensation, and we say so plainly:
|
||||
|
||||
- Partition the graph into structural **regions**.
|
||||
- Collapse each region to a `CondensedNode { centroid, weight,
|
||||
class_distribution, coherence, representative (medoid), members }`. `members`
|
||||
is retained — the original↔condensed mapping survives.
|
||||
- Rebuild **super-edges** from the *original* graph's boundary edges, so the
|
||||
condensed topology reproduces the source cut structure by construction rather
|
||||
than by training.
|
||||
|
||||
### Region-detection methods (`CondenseMethod`)
|
||||
|
||||
| Method | Mechanism | Reduction | Cost | When to use |
|
||||
|---|---|---|---|---|
|
||||
| **WeakBoundary** (default) | remove edges `< rel·mean_weight`, then union-find connected components | reliable when weights have contrast | linear (single pass) | general default; weighted graphs |
|
||||
| MinCutCommunity | `ruvector_mincut::CommunityDetector` (recursive global min cut) | graph-dependent | **super-linear** | dense clusters + sharp bottlenecks only |
|
||||
| Partition | `ruvector_mincut::GraphPartitioner` bisection | best-effort | super-linear | fixed region budget on clustered graphs |
|
||||
| ConnectedComponents | components only | structural | linear | baseline / pre-separated graphs |
|
||||
| DiffMinCut | trained soft assignment (see ADR-197) | `K`-bounded | iterative GD | learned cut-preserving regions |
|
||||
|
||||
The **default is `WeakBoundary`** because of an empirical finding during
|
||||
implementation: recursive *global* min cut (`CommunityDetector`/`GraphPartitioner`)
|
||||
**degenerates to singleton-peeling** — it shaves off the single lowest-degree
|
||||
boundary vertex each step — on graphs without sharp bottlenecks, giving ~N
|
||||
regions and zero reduction. This is the classic reason the community-detection
|
||||
literature uses modularity/conductance, not raw min cut. We keep the engine
|
||||
methods available (they *are* the literal min-cut-engine integration and work on
|
||||
clearly-bottlenecked graphs) but document the degeneracy and do not default to
|
||||
them.
|
||||
|
||||
### Quality metrics (retrain-free)
|
||||
|
||||
`metrics::evaluate` returns node/edge reduction ratios, `intra_weight_ratio`
|
||||
(fraction of edge weight kept inside regions), mean `coherence`, and weighted
|
||||
`label_purity`. `metrics::cut_inflation` (opt-in, solves an exact min cut on both
|
||||
graphs) reports `mincut(condensed)/mincut(source)`: `1.0` means the source's
|
||||
global min cut survives coarsening exactly.
|
||||
|
||||
### Streaming
|
||||
|
||||
`StreamingCondenser` buffers edges/features into a `DynamicGraph` and
|
||||
re-condenses lazily (on dirty read or every `rebuild_interval` mutations). This
|
||||
is **lazy re-condensation, not sublinear incremental region surgery** — an
|
||||
honest amortisation for growing graphs (e.g. a WorldGraph as it accumulates),
|
||||
with true incremental updates left as future work.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
- Fast: `WeakBoundary` condenses ~2048 nodes in ~4 ms (benchmarked); linear scaling.
|
||||
- Deterministic, label-optional, dependency-light (only `ruvector-mincut` + serde/rand/thiserror).
|
||||
- Interpretable: every super-node carries its `members` and a `coherence` score.
|
||||
- Cuts preserved by construction; `cut_inflation` quantifies fidelity.
|
||||
- Reuses the existing min-cut engine rather than reimplementing graph algorithms.
|
||||
|
||||
**Negative / limitations**
|
||||
- This is *not* accuracy-matched GCond-style condensation; it trades peak
|
||||
downstream GNN accuracy for speed, determinism, and provenance. We do not
|
||||
claim accuracy retention numbers — no GNN-retrain evaluation is in scope.
|
||||
- Engine methods (MinCutCommunity/Partition) are super-linear (~24 s at 96 nodes,
|
||||
measured) and prone to peeling; usable only on small/well-structured graphs.
|
||||
- `WeakBoundary` needs weight contrast; on near-uniform weights it degrades to
|
||||
ConnectedComponents (documented).
|
||||
- Every graph vertex must have a feature vector, or condensation errors
|
||||
(`MissingFeature`).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
1. **Implement GCond/SFGC-style learned condensation.** Rejected for v1:
|
||||
requires an autodiff stack and GNN training loop, is expensive, supervised,
|
||||
and destroys provenance. (ADR-197 adds the differentiable *min-cut* angle,
|
||||
which is the novel, lighter-weight slice of this.)
|
||||
2. **Put condensation inside `ruvector-mincut` or `ruvector-graph`.** Rejected:
|
||||
condensation is a distinct bounded context with its own data model; the
|
||||
workspace convention is one crate per capability.
|
||||
3. **Default to an engine method (MinCutCommunity/Partition).** Rejected after
|
||||
benchmarks showed singleton-peeling and super-linear cost.
|
||||
|
||||
## References
|
||||
|
||||
- Surveys: arXiv:2401.11720 (Graph Condensation: A Survey), arXiv:2402.03358
|
||||
(Graph Reduction, IJCAI'24); benchmarks GC4NC (arXiv:2406.16715), GC-Bench
|
||||
(arXiv:2407.00615).
|
||||
- Methods: GCond (ICLR'22), SFGC (NeurIPS'23), SGDD (NeurIPS'23), GEOM (ICML'24),
|
||||
GDEM (ICML'24), DisCo (2024), CGC (2025), GCTD (WSDM'26).
|
||||
- Substrate: `ruvector-mincut` (`DynamicGraph`, `CommunityDetector`,
|
||||
`GraphPartitioner`, `MinCutBuilder`); RuView (github.com/ruvnet/RuView).
|
||||
- Example: `crates/ruvector-graph-condense/examples/worldgraph.rs` — a RuView
|
||||
`WorldGraph → condense → OccWorld` demo (600 observations → 12 event
|
||||
summaries at 50× reduction, 100% activity purity, cut preserved).
|
||||
- **Accuracy validation** (`gnn_eval` module + `examples/accuracy_eval.rs` +
|
||||
`tests/accuracy.rs`): a gradient-checked 2-layer GCN runs the field's standard
|
||||
protocol (train on condensed, test on original held-out nodes). On a controlled
|
||||
unweighted node-classification task, `DiffMinCut` condensing 360 → 18 nodes
|
||||
(20×) reaches **100% accuracy retention**. Honest scope: controlled synthetic
|
||||
data, not Cora/Citeseer; `WeakBoundary` needs weight contrast (it collapses on
|
||||
uniform-weight graphs, which is why the accuracy path uses `DiffMinCut`).
|
||||
- **WASM deployment**: `crates/ruvector-graph-condense-wasm` exposes the
|
||||
condenser to JS/browser/edge (`wasm32-unknown-unknown`, 667 KB release before
|
||||
wasm-opt). The `parallel` (Rayon) feature is default-on for native and off for
|
||||
wasm (no threads).
|
||||
- ADR-197 (differentiable min-cut loss).
|
||||
180
docs/adr/ADR-197-differentiable-min-cut-condensation-loss.md
Normal file
180
docs/adr/ADR-197-differentiable-min-cut-condensation-loss.md
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
---
|
||||
adr: 197
|
||||
title: "Differentiable Min-Cut Condensation Loss (diffcut)"
|
||||
status: accepted
|
||||
date: 2026-06-07
|
||||
authors: [ruvnet, claude]
|
||||
related: [ADR-196]
|
||||
tags: [graph, condensation, min-cut, normalized-cut, mincutpool, differentiable, gnn]
|
||||
---
|
||||
|
||||
# ADR-197 — Differentiable Min-Cut Condensation Loss
|
||||
|
||||
## Status
|
||||
|
||||
**Accepted (implemented).** Module `crates/ruvector-graph-condense/src/diffcut.rs`
|
||||
plus `CondenseMethod::DiffMinCut`. Builds on ADR-196.
|
||||
|
||||
## Context
|
||||
|
||||
ADR-196 condenses graphs by *detecting* regions (weak-boundary components,
|
||||
recursive min cut, etc.) and collapsing them. The graph-condensation surveys
|
||||
(arXiv:2401.11720, arXiv:2402.03358) and our own SOTA review identified a
|
||||
specific, **genuinely unpublished gap**: while spectral structural terms appear
|
||||
in condensation losses — SGDD's Laplacian Energy Distribution (optimal transport
|
||||
on the spectrum), GDEM's eigenbasis/eigenvalue matching — there is **no
|
||||
published graph-condensation method whose loss is an explicit, differentiable
|
||||
min-cut / normalized-cut / modularity term**. Min-cut objectives are mature in
|
||||
GNN *pooling* (MinCutPool, Bianchi et al. 2020) and in *coarsening*, but using a
|
||||
relaxed-min-cut objective as the condensation mechanism itself is open.
|
||||
|
||||
We want region structure that is **trained to preserve the cut**, not just
|
||||
heuristically detected — without taking on the cost/complexity of a full
|
||||
GCond-style bi-level GNN-gradient-matching pipeline, and without adding a heavy
|
||||
autodiff dependency to a Rust crate that currently depends only on
|
||||
`ruvector-mincut` + serde/rand/thiserror.
|
||||
|
||||
## Decision
|
||||
|
||||
Implement a self-contained **differentiable relaxed-min-cut condenser** with
|
||||
**analytic gradients** (no autodiff framework), after MinCutPool.
|
||||
|
||||
### Objective
|
||||
|
||||
For a soft cluster assignment `S ∈ R^{N×K}` (row-softmax of learned logits `Θ`),
|
||||
weighted adjacency `A`, and degree matrix `D = diag(A·1)`:
|
||||
|
||||
```
|
||||
L_cut = - Tr(Sᵀ A S) / Tr(Sᵀ D S) ∈ [-1, 0] (relaxed normalized cut)
|
||||
L_ortho = ‖ SᵀS / ‖SᵀS‖_F − I_K / √K ‖_F ∈ [0, 2] (anti-collapse / balance)
|
||||
L = L_cut + λ · L_ortho
|
||||
```
|
||||
|
||||
`L_cut` rewards heavy edges inside clusters; `L_ortho` prevents the degenerate
|
||||
"all nodes in one cluster" solution (which by itself drives `L_cut → -1`).
|
||||
|
||||
### Gradients (analytic, all maths in `f64`)
|
||||
|
||||
- `∂L_cut/∂S = -(2/Tr(SᵀDS)) · (A S + L_cut · D S)`
|
||||
- `∂L_ortho/∂S = 2 · S · G_P`, where with `P = SᵀS`, `N_P = ‖P‖_F`,
|
||||
`Q = P/N_P − I/√K`, `Gf = Q/L_ortho`:
|
||||
`G_P = Gf/N_P − (⟨Gf, P⟩_F / N_P³) · P`
|
||||
- Backprop through row-softmax: `∂L/∂Θ_il = S_il · (gS_il − Σ_k gS_ik S_ik)`
|
||||
|
||||
`A S` is computed sparsely from the edge list (`O(nnz · K)` per step); the rest
|
||||
is `O(N·K + K²)`. The loss + analytic gradients live in `cutloss.rs`; the
|
||||
optimiser and orchestration in `diffcut.rs`.
|
||||
|
||||
### Optimisation (the part that makes large K work)
|
||||
|
||||
Plain gradient descent stalls at large `K` (a known property of MinCutPool-style
|
||||
objectives). Three standard ingredients fix it, all defaults:
|
||||
|
||||
1. **Adam** (`Optimizer::Adam`, default) — adaptive per-parameter moments; far
|
||||
more robust than SGD on the ill-conditioned, non-convex cut objective.
|
||||
`Optimizer::Sgd { momentum }` remains available.
|
||||
2. **Warm-start init** (`InitStrategy::WarmStart`, default) — seed the logits
|
||||
from the cheap `WeakBoundary` structural prior (largest regions → own
|
||||
clusters, overflow round-robin, +bias into the logits) and *refine* with the
|
||||
differentiable objective, instead of descending from random noise. This is
|
||||
the coreset/K-Center idea GCond/SFGC use, and it is what makes `K ≫ 2`
|
||||
converge. `InitStrategy::Random` remains available.
|
||||
3. **Restarts** (`restarts`) — keep the lowest-loss run.
|
||||
|
||||
Result: on a 12-event WorldGraph (`examples/worldgraph.rs`) DiffMinCut reaches
|
||||
**100% activity purity, cut preserved (inflation 1.000)** — matching
|
||||
`WeakBoundary` — where plain-GD/random scored ~30%. Training cost fell from
|
||||
~24 s (plain GD, 96 nodes) to milliseconds (Adam, `condense_diffcut` bench:
|
||||
~0.96 ms @ 64, ~6.4 ms @ 192 nodes). Tests `warm_start_recovers_many_clusters`
|
||||
(K=8, purity > 0.85) and `warm_start_beats_random_at_large_k` lock this in.
|
||||
|
||||
### Scale levers (for large / million-node graphs)
|
||||
|
||||
Three further levers, off by default, target very large graphs:
|
||||
|
||||
4. **Early-stopping** (`tolerance`, default `1e-6`) — warm-start lands near the
|
||||
optimum, so most iterations are wasted; stop when the loss plateaus. Test
|
||||
`early_stopping_cuts_iterations`.
|
||||
5. **Parallelism** (`parallel`, Rayon) — the per-iteration `A·S` (CSR,
|
||||
row-parallel) and the `O(N·K²)` `SᵀS` + ortho-gradient loops run in parallel.
|
||||
**Deterministic / bit-identical to sequential** (both use the same chunked
|
||||
partial-sum ordering), proven by `parallel_matches_sequential_exactly`.
|
||||
6. **Edge-minibatching** (`minibatch_edges`) — estimate the gradient from a
|
||||
sampled subset of edges per step (`O(batch·K)` instead of `O(|E|·K)`); the
|
||||
final reported loss is still computed full-batch (exact). Test
|
||||
`minibatch_recovers_structure`.
|
||||
|
||||
Bench (`condense_diffcut_levers`, 1024 nodes, 4 cores, 100 iters): sequential
|
||||
~95 ms, parallel ~83 ms (~1.15×), minibatch(2048) ~77 ms (~1.2×). Gains are
|
||||
modest at this size (Rayon dispatch overhead vs. small per-iteration work) and
|
||||
grow with `N`; the value is enabling graphs that do not fit a single-threaded
|
||||
full-batch budget, not speeding up small ones.
|
||||
|
||||
### Correctness
|
||||
|
||||
The analytic `∂L/∂Θ` is verified against **central finite differences** in
|
||||
`gradient_matches_finite_differences` across **K = 2, 3, 4** (max abs error
|
||||
`< 1e-5`) — the decisive test, proving the K-general formulas, not just K=2.
|
||||
|
||||
### API and integration
|
||||
|
||||
- `DiffCutConfig { num_clusters K, ortho_weight λ, learning_rate, iterations,
|
||||
optimizer, init, restarts, tolerance, parallel, minibatch_edges, seed }`;
|
||||
`DiffCutCondenser::train(&DynamicGraph) -> DiffCutResult`. Default = Adam +
|
||||
warm-start + early-stop, large-K-ready. `DiffCutResult::iterations_run()`
|
||||
reports how many iterations actually ran.
|
||||
- `DiffCutResult::soft_assignment()` (the `N×K` matrix) and `hard_regions()`
|
||||
(argmax grouping → `Vec<Vec<VertexId>>`).
|
||||
- `min_cut_loss(graph, soft, k, λ)` — public, evaluates the loss for any
|
||||
assignment (a quality metric for learned or hand-built assignments).
|
||||
- Wired in as `CondenseMethod::DiffMinCut(DiffCutConfig)`: train the soft
|
||||
assignment, harden to regions via argmax, then flow through ADR-196's existing
|
||||
provenance-preserving super-node/super-edge construction. It is the only region
|
||||
method whose structure is *trained* to preserve the cut.
|
||||
|
||||
Vertices are sorted ascending for a deterministic row order; logit init is
|
||||
seeded — same seed ⇒ identical result (tested).
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
- Fills the specific open gap: a differentiable min-cut term as the condensation
|
||||
mechanism, integrated end-to-end and provenance-preserving.
|
||||
- No new heavy dependency (no candle/burn/tch); pure Rust `f64` maths.
|
||||
- Gradient-checked, deterministic, label-free (uses topology only; features are
|
||||
applied later for centroids).
|
||||
- Recovers planted structure at small *and* large K (barbell exactly; K=8/K=12
|
||||
recovered via Adam + warm-start), and drives the cut term toward −1.
|
||||
- Fast: milliseconds per train (was tens of seconds under plain GD).
|
||||
|
||||
**Negative / limitations**
|
||||
- `K` (cluster count) is a fixed hyperparameter; empty clusters are dropped but
|
||||
`K` must be chosen.
|
||||
- Still slower than `WeakBoundary` (`O(restarts · iterations · nnz · K)`) and
|
||||
non-convex with no formal convergence guarantee, so it is opt-in, not the
|
||||
default. Large-K reliability leans on the warm-start prior; `InitStrategy::
|
||||
Random` at large K remains hard (documented, and what `warm_start_beats_random`
|
||||
measures). `WeakBoundary` stays the default (ADR-196) for speed/simplicity.
|
||||
- Topology-only objective: it optimises the structural cut, not feature/label
|
||||
matching, so it is not a substitute for supervised GCond-style accuracy
|
||||
matching.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
1. **Add an autodiff backend (candle/tch/burn) and a learned GNN condenser.**
|
||||
Rejected: heavy dependency and build cost for a structural objective whose
|
||||
gradients are short closed forms.
|
||||
2. **Spectral objective (SGDD LED / GDEM eigenbasis) instead of min cut.**
|
||||
Rejected for this ADR: those are already published; the min-cut term is the
|
||||
unaddressed gap. (A spectral term remains possible future work.)
|
||||
3. **Only expose the loss as a metric (no training).** Rejected: the request and
|
||||
the novelty are the *trainable* loss; we expose both the metric
|
||||
(`min_cut_loss`) and the optimiser (`DiffCutCondenser`).
|
||||
|
||||
## References
|
||||
|
||||
- Bianchi, Grattarola, Alippi — "Spectral Clustering with GNNs for Graph
|
||||
Pooling" (MinCutPool), ICML 2020.
|
||||
- SGDD (arXiv:2310.09192), GDEM (arXiv:2310.09202) — spectral condensation terms.
|
||||
- Surveys: arXiv:2401.11720, arXiv:2402.03358 (open-problem framing).
|
||||
- ADR-196 (structure-preserving graph condensation; method taxonomy & substrate).
|
||||
151
docs/adr/ADR-198-physical-perception-substrate.md
Normal file
151
docs/adr/ADR-198-physical-perception-substrate.md
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
---
|
||||
adr: 198
|
||||
title: "Physical Perception Substrate — delta → boundary → coherence → proof → action"
|
||||
status: accepted
|
||||
date: 2026-06-08
|
||||
authors: [ruvnet, claude]
|
||||
related: [ADR-196, ADR-197]
|
||||
tags: [perception, sensing, coherence, min-cut, proof-gate, edge-ai, csi, ruview]
|
||||
---
|
||||
|
||||
# ADR-198 — Physical Perception Substrate
|
||||
|
||||
## Status
|
||||
|
||||
**Accepted (initial vertical slice implemented).** Crate
|
||||
`crates/ruvector-perception`.
|
||||
|
||||
## Context
|
||||
|
||||
WiFi/edge sensing SOTA is converging on better **classifiers**: CSI foundation
|
||||
models, self-supervised CSI representations (CSI-JEPA-style), adaptive near-sensor
|
||||
fusion (FusionSense-style), and dynamic-graph anomaly detection (which still
|
||||
flags interpretability + scalability as open). All answer *"what is this?"* and
|
||||
emit *confidence → alert*.
|
||||
|
||||
The wedge is not a better classifier. It is the **layer underneath** one: a
|
||||
trusted-physical-memory engine that answers *"what changed, where did the
|
||||
boundary move, and is the change coherent enough to act on?"* and requires
|
||||
**evidence, not confidence**, before exercising any authority. This reframes the
|
||||
pipeline:
|
||||
|
||||
```
|
||||
classification → confidence → alert (today)
|
||||
delta → boundary → coherence → proof → action (this ADR)
|
||||
```
|
||||
|
||||
It also removes the dependence on a fixed task label (fall / gesture / occupancy
|
||||
/ leak / bearing-failure): it models **state transition itself**.
|
||||
|
||||
## Decision
|
||||
|
||||
Implement the pipeline as a standalone crate built on the dynamic min-cut engine.
|
||||
|
||||
1. **Delta** (`state`, `engine`) — every reading becomes a delta against a
|
||||
rolling per-(zone, modality) baseline (EWMA), plus a learned *responsiveness*
|
||||
(how often that channel reacts in that zone).
|
||||
2. **Boundary** (`coherence`) — zones are nodes in a coherence graph (edge weight
|
||||
= delta-pattern agreement). Dynamic min-cut (`ruvector-mincut`) isolates the
|
||||
side that broke away — the moved boundary, not a class.
|
||||
3. **Contradiction as information** — a modality that *usually* reacts in a zone
|
||||
but stayed silent is a first-class contradiction, weighted by the modality's
|
||||
physical **spoof-resistance** (modalities are physically typed: RF ≠ thermal).
|
||||
This is what flags an inert object-move (RF/vibration/acoustic respond,
|
||||
thermal — which would respond to an animate source — does not).
|
||||
4. **Proof** (`witness`) — a proof gate maps (novelty, coherence, contradiction)
|
||||
to **bounded authority** `Ignore → Observe → Alert → Mutate`, and emits an
|
||||
auditable SHA-256 evidence chain (raw hash, feature hash, scores, boundary,
|
||||
policy, prior-witness hash). Contradicted evidence is **capped at Observe** —
|
||||
it never escalates on confidence alone.
|
||||
5. **Absence** (`absence`) — a *missing* expected continuation (e.g.
|
||||
`bed_exit → bathroom_path → return_path` where the return never arrives) is
|
||||
detected as structural incompleteness, a safety signal, not a threshold.
|
||||
|
||||
The headline output is a `DeltaWitness` (changed_boundary, supporting /
|
||||
contradicting modalities, novelty, coherence, contradiction, action,
|
||||
evidence_hash, prev_hash) — a structured delta, not a label.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
- Task-label-free: detects unknown physical changes without retraining.
|
||||
- Auditable: every action is backed by a replayable evidence chain (matters for
|
||||
elder care / industrial / civic / medical governance).
|
||||
- Interpretable localisation: min-cut says *where* coherence broke and *why*
|
||||
(which modalities support vs contradict) — addressing the open
|
||||
interpretability gap in dynamic-graph anomaly work.
|
||||
- Reuses existing min-cut machinery; small, dependency-light, `#![forbid(unsafe_code)]`.
|
||||
|
||||
**Negative / honest scope**
|
||||
- This is the **mechanism**, demonstrated on **synthetic** multi-modal deltas —
|
||||
not validated on real CSI/hardware, and not benchmarked against CSI-JEPA /
|
||||
FusionSense (different layer). No accuracy claims.
|
||||
- Novelty (nearest-prior distance), contradiction (responsive-but-silent), and
|
||||
coherence (cut cleanliness) are principled **heuristics**, not learned.
|
||||
- Single-window; no temporal model of the delta beyond EWMA baselines and the
|
||||
absence-sequence monitor. Boundary detection is O(zones²) edges + exact min cut
|
||||
(fine for rooms/facilities, not yet city-scale).
|
||||
|
||||
## Capability modules (built on the substrate)
|
||||
|
||||
Five further beyond-classification capabilities from the brief are implemented as
|
||||
self-contained modules (each emits structure, not a label):
|
||||
|
||||
- **`captcha`** — Physical CAPTCHA: a learned per-stimulus multi-modal
|
||||
challenge-response profile; a fresh response is verified within delay/magnitude
|
||||
tolerance, weighted by spoof-resistance, yielding a `RealityProof`. Detects
|
||||
replay/spoof (proof-of-real-physical-field).
|
||||
- **`predict`** — Boundary-first world model: forecasts *where coherence breaks
|
||||
next* (`instability = coherence·(1+contradiction)`, level + least-squares
|
||||
trend) rather than full future states.
|
||||
- **`identity`** — Resonant identity / continuity: per-object EWMA signature;
|
||||
cosine-distance drift detection answers "is this still the same physical
|
||||
thing?" (panel loosened, bearing worn, casing tampered).
|
||||
- **`hypothesis`** — Multi-modal disagreement engine: contradictions produce
|
||||
*ranked hypotheses* (RealEvent / SensorDrift / SensorRelocation /
|
||||
AdversarialReplay / EnvironmentalArtifact), not forced agreement.
|
||||
- **`topology`** — Self-healing sensor topology: an EWMA agreement graph
|
||||
classifies each sensor Critical / Redundant / Noisy / Normal; Critical =
|
||||
articulation point (removal fragments the graph — the extreme single-edge cut).
|
||||
- **`swarm`** — Facility/swarm-scale fragility: rooms/machines/routers as a
|
||||
coupling graph; global min-cut answers "where is the system structurally
|
||||
closest to breaking?" Bottlenecks are derived from the weakest link (edge
|
||||
weights), because the engine's min-cut *value* is reliable but its *partition*
|
||||
is not.
|
||||
- **`custody`** — Sensor chain of custody: a tamper-evident, replayable ledger
|
||||
of witnesses (chain-linkage verification over the SHA-256 evidence hashes;
|
||||
honest scope — link integrity, not raw-signal re-hash).
|
||||
- **`reality`** — Reality-graph agent grounding: an agent *queries reality*
|
||||
(presence / changed-since / which-untrusted / action-allowed) and gets answers
|
||||
**backed by witness evidence hashes**, not prompt inference.
|
||||
- **`node`** — `NervousSystemNode`: the appliance facade wiring engine + reality
|
||||
graph + custody ledger + boundary forecaster. Ingests readings, emits
|
||||
deltas/boundaries/coherence/witnesses/forecasts (never raw signal), and answers
|
||||
grounded queries.
|
||||
|
||||
## Future work (from the brief, not yet built)
|
||||
|
||||
The remaining items are out of pure-software scope: the physical "ambient
|
||||
nervous system" **hardware** node, and replacing the heuristic scorers
|
||||
(novelty / contradiction / coherence) with **learned** models validated on real
|
||||
CSI. Everything above is a mechanism demonstration on synthetic signals.
|
||||
|
||||
Known limitation surfaced during testing: coherence boundary detection is
|
||||
ambiguous with exactly **two** zones (a single-edge min cut splits symmetrically;
|
||||
the minority side is arbitrary). Use ≥3 zones for a well-defined changed
|
||||
boundary — documented and reflected in the tests.
|
||||
|
||||
## Validation
|
||||
|
||||
59 tests (54 unit + 2 integration + 3 doctest), deterministic across repeated
|
||||
runs. Highlights: the brief's exact flagship scenario (inert object move →
|
||||
RF/vibration/acoustic support, thermal contradicts, novelty high, action =
|
||||
observe); the missing-routine-return absence signal; physical-CAPTCHA replay
|
||||
rejection; boundary forecast of a destabilising zone; identity drift on a
|
||||
tampered signature; ranked hypotheses (RealEvent / SensorDrift / AdversarialReplay
|
||||
first under the right evidence); topology roles (bridge → Critical, near-duplicate
|
||||
→ Redundant, lone-disagreer → Noisy); facility fragility (weakest link found);
|
||||
custody chain verify + tamper detection; reality-graph grounded queries; and the
|
||||
end-to-end `NervousSystemNode` (witness chain + grounded query). Built across two
|
||||
parallel agent swarms, then integrated and validated. clippy clean; all source
|
||||
files < 500 lines.
|
||||
Loading…
Add table
Add a link
Reference in a new issue