ruvector/crates/ruvector-graph-condense
rUv 22689a7511
Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547)
* Add ruvector-graph-condense: structure-preserving graph condensation

New crate implementing training-free, structure-preserving graph
condensation built on the dynamic min-cut engine (ruvector-mincut).
Collapses a feature graph into a small synthetic graph of super-nodes
(regions) while preserving cut structure and node provenance.

Positioning vs. SOTA (GCond/SFGC/GEOM/SGDD): those synthesise a fake
graph via bi-level gradient/distribution/trajectory matching and discard
the node->original mapping. This is the complementary, training-free
route the 2024-2026 surveys flag as under-explored: min-cut community
structure as the condensation prior, cuts preserved by construction
(boundary edges become weighted super-edges), and members retained per
super-node for audit/explainability. Closest published analogs are CGC
(clustering, 2025) and GCTD (tensor decomposition, 2025).

Components:
- NodeFeatures: validated per-vertex embeddings + optional labels
- CondensedNode/Edge/Graph: centroid, weight, class histogram, coherence,
  medoid representative, member provenance; round-trips to DynamicGraph
- GraphCondenser with 4 region methods:
  - WeakBoundary (default): single-pass union-find over weak-edge removal,
    linear-time, recovers planted structure
  - MinCutCommunity / Partition: delegate to the min-cut engine
    (CommunityDetector / GraphPartitioner); best-effort, documented as
    super-linear and prone to singleton-peeling on graphs without
    sharp bottlenecks
  - ConnectedComponents baseline
- metrics: retrain-free proxies (reduction ratios, intra-weight ratio,
  coherence, label purity) + opt-in cut_inflation via exact MinCutBuilder
- StreamingCondenser: lazy re-condensation for growing graphs
- PlantedPartition synthetic generator; criterion benchmarks

Benchmarks (this machine): WeakBoundary scales linearly (~4ms @ 2048
nodes); the recursive min-cut engine methods are super-linear (~24s @ 96
nodes), which is why WeakBoundary is the default.

33 unit tests + 1 doctest pass; clippy clean.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Add differentiable min-cut loss (diffcut) to graph condensation

Implements the open research gap flagged by the SOTA review: a
differentiable min-cut / normalized-cut objective used as the
condensation mechanism. The 2024-2026 surveys note that only spectral
terms (SGDD's Laplacian Energy Distribution, GDEM's eigenbasis) exist;
an explicit relaxed-min-cut loss in the condensation objective does not.

New `diffcut` module (after Bianchi et al., MinCutPool 2020):
- Relaxed normalized-cut loss L_cut = -Tr(SᵀAS)/Tr(SᵀDS) plus an
  orthogonality/anti-collapse term L_ortho, over a row-softmax soft
  assignment S (N×K) of learned logits.
- Analytic gradients (cut, ortho, and softmax backprop), all maths in
  f64, no autodiff dependency. Verified against central finite
  differences (gradient_matches_finite_differences passes to 1e-5).
- DiffCutCondenser: gradient-descent training -> DiffCutResult with
  soft_assignment() and hard_regions() (argmax grouping).
- Public min_cut_loss() for evaluating any soft assignment.

Wired in as CondenseMethod::DiffMinCut(DiffCutConfig): trains the soft
assignment, hardens to regions, then flows through the existing
provenance-preserving super-node/super-edge construction. The only
region method whose structure is *trained* to preserve the cut.

Tests: 36 unit (incl. gradient check + uniform-assignment behaviour) +
6 integration (recovery, determinism, errors) + doctest. clippy clean;
all source files <500 lines. Benchmarks add a diffcut training group.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* docs(adr): ADR-196 + ADR-197 for graph condensation

ADR-196: Structure-preserving graph condensation (ruvector-graph-condense)
 — context (SOTA gap + RuView/WorldGraph substrate), decision (training-
free coarsening-condensation with min-cut prior, provenance retained),
the CondenseMethod taxonomy with honest tradeoffs (WeakBoundary default;
engine methods peel + are super-linear), metrics, streaming, alternatives.

ADR-197: Differentiable min-cut condensation loss (diffcut) — the relaxed
normalized-cut + orthogonality objective (MinCutPool-style), analytic
gradients verified by finite differences, DiffCutCondenser + DiffMinCut
integration, and the novelty framing (differentiable min-cut term in the
condensation loss is unpublished as of 2026).

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Add WorldGraph example + momentum optimizer; harden diffcut for K>2

- examples/worldgraph.rs: RuView WorldGraph -> condense -> OccWorld demo.
  WeakBoundary condenses 600 observations into 12 event summaries (50x,
  100% activity purity, cut preserved 1.000); a smaller dense scene shows
  the trained DiffMinCut recovering ~86% activity purity.
- diffcut: add heavy-ball `momentum` to DiffCutConfig (default 0.0, all
  existing behaviour/tests/benchmarks unchanged) and unit-scale logit init
  for stronger symmetry-breaking at K>2.
- Extend the gradient check to K = 2, 3, 4 (proves the K-general gradient
  formulas; max abs error < 1e-5).
- Honest finding documented in ADR-197: DiffMinCut (MinCutPool-style) is
  K-sensitive — reliable at small/moderate K, underperforms WeakBoundary at
  large K, reinforcing WeakBoundary as the default (ADR-196).
- Workspace manifest validated (member resolves; crate is additive so it
  cannot break other crates).

43 tests pass (36 unit + 6 integration + 1 doctest); clippy clean; all
source files <500 lines.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Optimize trained min-cut for large K: Adam + warm-start + restarts

Plain/momentum GD from random init stalled the differentiable min-cut at
large K (12-event WorldGraph: ~30% purity, ~24s @ 96 nodes). Rebuilt the
optimizer so the trained method is viable at scale:

- Split loss math into cutloss.rs (CompactGraph + softmax + cut/ortho +
  analytic gradients, gradient-checked K=2,3,4); diffcut.rs now owns the
  optimizer/orchestration. Both files <500 lines.
- Optimizer enum: Adam (default; adaptive moments) and Sgd { momentum }.
- InitStrategy enum: WarmStart (default) seeds logits from the WeakBoundary
  structural prior and refines (coreset/K-Center idea), or Random.
- restarts: keep the lowest-loss run. Deterministic region ordering in
  warm-start so same seed => identical result.

Result on the 12-event WorldGraph example: DiffMinCut now reaches 100%
activity purity, cut preserved (inflation 1.000) — matching WeakBoundary —
in milliseconds (bench condense_diffcut: ~0.96ms @64, ~6.4ms @192 nodes;
was ~24s @96 under plain GD).

New tests: warm_start_recovers_many_clusters (K=8, purity>0.85),
warm_start_beats_random_at_large_k, warm_start_seeds_a_good_partition,
adam_refines_to_low_cut. Config call sites use ..Default::default().
ADR-197 updated. 47 tests pass (38 unit + 8 integration + 1 doctest);
clippy clean.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* diffcut scale levers: early-stop, Rayon parallelism, edge-minibatching

Three further optimizations for large/million-node graphs (off by default):

- Early-stopping (tolerance, default 1e-6): warm-start lands near the
  optimum, so stop when the loss plateaus. iterations_run() reports actual.
- Parallelism (parallel, Rayon): CSR row-parallel A·S plus parallel O(N·K²)
  SᵀS + ortho-gradient loops. Deterministic / bit-identical to sequential
  (same chunked partial-sum ordering), proven by a test.
- Edge-minibatching (minibatch_edges): stochastic gradient from a sampled
  edge subset, O(batch·K)/step; final loss still full-batch exact.

Refactor: cutloss.rs gains CSR adjacency + as_matrix (parallel) +
as_matrix_minibatch + a chunked gram(); loss_and_grad split so the optimizer
supplies A·S. New tests: parallel_matches_sequential_exactly,
minibatch_recovers_structure, early_stopping_cuts_iterations. New bench group
condense_diffcut_levers (1024 nodes, 4 cores: seq ~95ms, parallel ~83ms,
minibatch ~77ms). ADR-197 updated.

50 tests pass (38 unit + 11 integration + 1 doctest); clippy clean; all
source files <500 lines.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Add GNN accuracy-retention harness (closes the no-accuracy-validation gap)

Implements the graph-condensation field's core success metric: train a GNN
on the condensed graph, test on the ORIGINAL graph's held-out nodes, report
accuracy(condensed)/accuracy(full).

- gnn_eval.rs: self-contained, dependency-free 2-layer GCN (symmetric-
  normalised CSR propagation, ReLU, softmax-CE, Adam, analytic backprop).
  Gradient-checked against finite differences (<1e-6) and verified to learn a
  separable task.
- examples/accuracy_eval.rs + tests/accuracy.rs: the full protocol on a
  controlled synthetic node-classification task (planted communities as
  classes, noisy features so the graph carries real signal).

Measured: baseline (full-graph GNN) 100%. On an UNWEIGHTED graph (the SOTA
benchmark setting), DiffMinCut condensing 360 nodes -> 18 super-nodes (20x)
yields **100% retention** (GNN trained on 18 nodes matches the full-graph GNN
on held-out test nodes).

Also fixes a real failure the harness surfaced: on uniform-weight graphs
WeakBoundary collapses to one component; DiffMinCut's warm-start inherited
that collapse. Warm-start now falls back to random init when the structural
prior finds <2 regions, letting the min-cut objective do the partitioning
(retention 14.9% -> 66% at K=classes, 100% at K=3*classes).

Honest scope: controlled synthetic data, not Cora/Citeseer; WeakBoundary
still needs weight contrast (documented). 53 tests pass; clippy clean.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Add WASM bindings + gate Rayon behind a feature for wasm builds

- crates/ruvector-graph-condense-wasm: wasm-bindgen bindings exposing
  condense_weak / condense_diffmincut / version to JS. Graphs in as flat
  typed arrays, CondensedGraph out as JSON. Builds for
  wasm32-unknown-unknown (667 KB release, pre wasm-opt), so the condenser
  (including the trained DiffMinCut) runs in the browser / on the edge —
  the deployable-artifact goal from the original brief.
- ruvector-graph-condense: Rayon is now an optional `parallel` feature
  (default on for native, off for wasm — no threads on
  wasm32-unknown-unknown). cutloss.rs cfg-gates every Rayon path with a
  sequential fallback; no-default-features builds clean.
- getrandom `js` backend is wasm-target-gated so native feature
  unification is unaffected; ruvector-mincut built with its `wasm` feature.
- ADR-196 updated with the WASM deployment + accuracy-validation notes.

53 tests pass; clippy clean (both crates); native + wasm32 both build.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Add ruvector-perception: the layer under classification (delta->proof->action)

Beyond-SOTA wedge: instead of a better CSI classifier, build the substrate
underneath one. Pipeline: delta -> boundary -> coherence -> proof -> action.
Emits a structured DeltaWitness, not a class label, and requires evidence
(not confidence) before exercising bounded authority.

- modality.rs: physically-typed modalities (RF/vibration/acoustic/thermal/
  chemical/optical) with latency/decay/spoof-resistance — typed graph edges.
- state.rs: rolling per-(zone,modality) baselines + learned responsiveness.
- coherence.rs: zones as a coherence graph; dynamic min-cut isolates the moved
  boundary (reuses ruvector-mincut). Coherence = separation cleanliness.
- witness.rs: ProofGate (Ignore/Observe/Alert/Mutate) + SHA-256 evidence
  chain. Contradicted evidence is capped at Observe (no escalation on
  confidence alone). Contradiction = a modality that usually reacts here but
  stayed silent, weighted by spoof-resistance.
- engine.rs: orchestrates delta -> boundary -> contradiction -> novelty
  (nearest-prior) -> proof gate -> chained witness.
- absence.rs: missing expected continuation (bed_exit->bathroom->return) as a
  structural safety signal, not a threshold.

Flagship test reproduces the brief exactly: an inert object move yields
changed_boundary=table_left_zone, supporting={rf,vibration,acoustic},
contradicting={thermal}, novelty=high, action=observe. ADR-198 documents the
architecture and honest scope (mechanism on synthetic deltas, not validated on
real CSI).

11 tests pass; clippy clean; all files <500 lines.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Perception: 5 beyond-classification capability modules (swarm-built)

Built via a 5-agent parallel swarm, then integrated and validated. Each
emits structure, not a class label:

- captcha: Physical CAPTCHA — learned per-stimulus multi-modal challenge-
  response profiles; verifies a fresh response (delay/magnitude tolerance,
  spoof-resistance weighted) -> RealityProof. Detects replay/spoof.
- predict: Boundary-first world model — forecasts where coherence breaks next
  (instability = coherence*(1+contradiction), level + least-squares trend).
- identity: Resonant identity / continuity — per-object EWMA signature, cosine
  drift detection ("is this still the same physical thing?").
- hypothesis: Multi-modal disagreement engine — contradictions produce ranked
  hypotheses (RealEvent/SensorDrift/SensorRelocation/AdversarialReplay/
  EnvironmentalArtifact), not forced agreement.
- topology: Self-healing sensor topology — EWMA agreement graph; roles
  Critical/Redundant/Noisy/Normal. Critical = articulation point (removal
  fragments the graph) — replaced the agent's unreliable min-cut-partition
  rule with robust articulation detection so triangle/star outliers keep their
  real roles.

lib.rs re-exports all five. ADR-198 updated. 42 tests pass (38 unit + 2
integration + 2 doctest); clippy clean; all source files <500 lines.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* Perception: complete the substrate — custody, swarm, reality-graph, node

Final beyond-classification pieces (custody + swarm built by a 2-agent swarm;
reality + node integration built directly):

- custody: tamper-evident, replayable chain-of-custody ledger over witness
  evidence hashes (chain-linkage verification; honest scope: link integrity,
  not raw-signal re-hash).
- swarm: facility/swarm-scale fragility — coupling graph + global min-cut
  answers "where is the system closest to breaking?". Bottlenecks derived from
  the weakest link (edge weights), since the engine's min-cut value is reliable
  but its partition is not (same quirk handled in topology).
- reality: reality-graph agent grounding — an agent queries physical state
  (presence / changed-since / which-untrusted / action-allowed) and gets
  answers backed by witness evidence hashes, not prompt inference.
- node: NervousSystemNode appliance facade wiring engine + reality + custody +
  boundary forecaster; emits deltas/boundaries/witnesses/forecasts (no raw
  signal) and answers grounded queries.

Fixes during integration: swarm bottleneck now uses the weakest edge (engine
partition is unreliable); node test uses 3 zones (2-zone min-cut boundary is
ambiguous — a real limitation now documented). ADR-198 updated.

59 tests pass (54 unit + 2 integration + 3 doctest), deterministic; clippy
clean; all source files <500 lines.

https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX

* chore(ci): wire condense+perception crates into publish + regression guard (#547)

Aligns the new ruvector-graph-condense, ruvector-graph-condense-wasm, and
ruvector-perception crates with the workspace release plumbing.

- Bump their ruvector-mincut (and graph-condense) dep pins from "2.0.1" to
  "2.2.3" to match the workspace version they are built and tested against.
  The old "^2.0.1" pin would resolve a crates.io publish against the stale
  published mincut 2.0.6, risking a crate that fails to compile downstream.
- publish-all.yml: publish the three crates (plus mincut as substrate) to
  crates.io in dependency order with index-settle waits, matching the
  existing --allow-dirty / continue-on-error style.
- regression-guard.yml: run the new crates' tests (they were build-checked
  but never tested in CI) and forbid regressing the mincut pin back to 2.0.x.

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(graph-condense): rustfmt, clippy -D warnings, and cargo-deny advisory (#547)

CI green-up for the new condense/perception crates:

- rustfmt: format all source/bench/example/test files in the new crates
  (the PR was committed unformatted; CI Rustfmt flagged all 29 files).
- clippy -D warnings: condense.rs used `sort_by(|a,b| key.cmp(&key))` which
  trips clippy::unnecessary_sort_by under `-D warnings`; switch to
  `sort_by_key`. (Earlier local clippy didn't deny warnings, so it slipped.)
- cargo-deny: ignore RUSTSEC-2026-0173 (proc-macro-error2 unmaintained).
  Pre-existing transitive dep (validator_derive -> validator, via the
  ruvector-scipix example), same crate family as the already-ignored
  RUSTSEC-2024-0370. Not introduced by this PR. Re-review 2026-07-01.

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs(graph-condense): add crate READMEs for crates.io publish (#547)

The new graph-condense crates were wired to publish without a README (101/136
workspace crates have one; every published crate does). Add READMEs matching
the repo's badge-header convention and the `readme = "README.md"` field so the
crates.io pages render properly on first publish.

- ruvector-graph-condense: overview, SOTA positioning, quick-start (using the
  real NodeFeatures::new/set + DynamicGraph::insert_edge API), region-method
  table, and the honest ADR-196/197 limitations.
- ruvector-graph-condense-wasm: short binding README pointing at the core crate.

Perception crate intentionally left as-is (out of scope for this request).

Co-Authored-By: claude-flow <ruv@ruv.net>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: ruvnet <ruvnet@gmail.com>
2026-06-08 22:58:44 +02:00
..
benches Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547) 2026-06-08 22:58:44 +02:00
examples Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547) 2026-06-08 22:58:44 +02:00
src Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547) 2026-06-08 22:58:44 +02:00
tests Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547) 2026-06-08 22:58:44 +02:00
Cargo.toml Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547) 2026-06-08 22:58:44 +02:00
README.md Graph condensation: structure-preserving + differentiable min-cut (ruvector-graph-condense) (#547) 2026-06-08 22:58:44 +02:00

RuVector Graph Condense

Crates.io Documentation License GitHub ruv.io

Training-free, structure-preserving, provenance-retaining graph condensation.

Collapse a large feature graph into a small synthetic graph of super-nodes while preserving its cut structure — plus a differentiable min-cut loss.


Why This Matters

Graph condensation shrinks a graph + per-node embeddings (+ optional labels) into a much smaller graph that downstream tasks can still reason over. The published SOTA — GCond, SFGC, GEOM, SGDD — synthesises a fake graph via expensive, supervised bi-level gradient/distribution/trajectory matching, and discards the node→original mapping.

ruvector-graph-condense takes the complementary, training-free route the 20242026 condensation surveys flag as under-explored:

  • Min-cut community structure as the condensation prior (not k-means).
  • Cuts preserved by construction — boundary edges become weighted super-edges; metrics::cut_inflation quantifies fidelity.
  • Provenance retained — every CondensedNode keeps its members, so each super-node is auditable and explainable.
  • A differentiable min-cut loss (diffcut, MinCutPool-style relaxed normalized cut + orthogonality) — analytic gradients, gradient-checked across K=2,3,4 to <1e-5.

Built on the dynamic min-cut engine ruvector-mincut.

Quick Start

use ruvector_graph_condense::{CondenseConfig, GraphCondenser, NodeFeatures};
use ruvector_mincut::DynamicGraph;

// Build a graph (insert_edge returns a Result; &self — it is concurrent).
let graph = DynamicGraph::new();
let _ = graph.insert_edge(0, 1, 1.0);
let _ = graph.insert_edge(1, 2, 1.0);
let _ = graph.insert_edge(2, 3, 0.1); // weak boundary edge

// Per-vertex embeddings (+ optional labels): NodeFeatures::new(dim, num_classes).
let mut features = NodeFeatures::new(2, 1);
for v in 0..4u64 {
    features.set(v, vec![v as f32, 0.0], 0).unwrap();
}

let condenser = GraphCondenser::new(CondenseConfig::default()); // WeakBoundary, 0.5
let condensed = condenser.condense(&graph, &features).unwrap();

for node in &condensed.nodes {
    // Each super-node keeps the original vertices it came from (provenance).
    println!("super-node {:?} <- members {:?}", node.representative, node.members);
}

Region Methods (CondenseMethod)

Method Notes
WeakBoundary (default) Linear-time union-find on weak edges. ~4 ms @ 2048 nodes.
MinCutCommunity / Partition Delegate to the min-cut engine. Structure-aware on graphs with sharp bottlenecks; documented best-effort otherwise.
ConnectedComponents Cheap baseline — one region per component.
DiffMinCut Differentiable, trained assignment (opt-in).

Honest Limitations

  • The recursive global min-cut engine methods degenerate to singleton-peeling on graphs without sharp bottlenecks and are super-linear (~24 s @ 96 nodes) — which is why the linear-time WeakBoundary is the default.
  • DiffMinCut is K-sensitive (known MinCutPool finickiness): it recovers small/dense graphs but underperforms WeakBoundary at large K. Momentum + unit-scale init help, but there is no convergence guarantee.
  • This is structure-preserving coarsening-condensation (keeps provenance) — not accuracy-matched GCond-style condensation; no GNN-retrain accuracy numbers are claimed.

See ADR-196 (structure-preserving condensation) and ADR-197 (differentiable min-cut loss) for the full design and findings.

License

MIT © ruv.io