mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-07-10 01:38:44 +00:00
22689a7511
* Add ruvector-graph-condense: structure-preserving graph condensation
New crate implementing training-free, structure-preserving graph
condensation built on the dynamic min-cut engine (ruvector-mincut).
Collapses a feature graph into a small synthetic graph of super-nodes
(regions) while preserving cut structure and node provenance.
Positioning vs. SOTA (GCond/SFGC/GEOM/SGDD): those synthesise a fake
graph via bi-level gradient/distribution/trajectory matching and discard
the node->original mapping. This is the complementary, training-free
route the 2024-2026 surveys flag as under-explored: min-cut community
structure as the condensation prior, cuts preserved by construction
(boundary edges become weighted super-edges), and members retained per
super-node for audit/explainability. Closest published analogs are CGC
(clustering, 2025) and GCTD (tensor decomposition, 2025).
Components:
- NodeFeatures: validated per-vertex embeddings + optional labels
- CondensedNode/Edge/Graph: centroid, weight, class histogram, coherence,
medoid representative, member provenance; round-trips to DynamicGraph
- GraphCondenser with 4 region methods:
- WeakBoundary (default): single-pass union-find over weak-edge removal,
linear-time, recovers planted structure
- MinCutCommunity / Partition: delegate to the min-cut engine
(CommunityDetector / GraphPartitioner); best-effort, documented as
super-linear and prone to singleton-peeling on graphs without
sharp bottlenecks
- ConnectedComponents baseline
- metrics: retrain-free proxies (reduction ratios, intra-weight ratio,
coherence, label purity) + opt-in cut_inflation via exact MinCutBuilder
- StreamingCondenser: lazy re-condensation for growing graphs
- PlantedPartition synthetic generator; criterion benchmarks
Benchmarks (this machine): WeakBoundary scales linearly (~4ms @ 2048
nodes); the recursive min-cut engine methods are super-linear (~24s @ 96
nodes), which is why WeakBoundary is the default.
33 unit tests + 1 doctest pass; clippy clean.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add differentiable min-cut loss (diffcut) to graph condensation
Implements the open research gap flagged by the SOTA review: a
differentiable min-cut / normalized-cut objective used as the
condensation mechanism. The 2024-2026 surveys note that only spectral
terms (SGDD's Laplacian Energy Distribution, GDEM's eigenbasis) exist;
an explicit relaxed-min-cut loss in the condensation objective does not.
New `diffcut` module (after Bianchi et al., MinCutPool 2020):
- Relaxed normalized-cut loss L_cut = -Tr(SᵀAS)/Tr(SᵀDS) plus an
orthogonality/anti-collapse term L_ortho, over a row-softmax soft
assignment S (N×K) of learned logits.
- Analytic gradients (cut, ortho, and softmax backprop), all maths in
f64, no autodiff dependency. Verified against central finite
differences (gradient_matches_finite_differences passes to 1e-5).
- DiffCutCondenser: gradient-descent training -> DiffCutResult with
soft_assignment() and hard_regions() (argmax grouping).
- Public min_cut_loss() for evaluating any soft assignment.
Wired in as CondenseMethod::DiffMinCut(DiffCutConfig): trains the soft
assignment, hardens to regions, then flows through the existing
provenance-preserving super-node/super-edge construction. The only
region method whose structure is *trained* to preserve the cut.
Tests: 36 unit (incl. gradient check + uniform-assignment behaviour) +
6 integration (recovery, determinism, errors) + doctest. clippy clean;
all source files <500 lines. Benchmarks add a diffcut training group.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* docs(adr): ADR-196 + ADR-197 for graph condensation
ADR-196: Structure-preserving graph condensation (ruvector-graph-condense)
— context (SOTA gap + RuView/WorldGraph substrate), decision (training-
free coarsening-condensation with min-cut prior, provenance retained),
the CondenseMethod taxonomy with honest tradeoffs (WeakBoundary default;
engine methods peel + are super-linear), metrics, streaming, alternatives.
ADR-197: Differentiable min-cut condensation loss (diffcut) — the relaxed
normalized-cut + orthogonality objective (MinCutPool-style), analytic
gradients verified by finite differences, DiffCutCondenser + DiffMinCut
integration, and the novelty framing (differentiable min-cut term in the
condensation loss is unpublished as of 2026).
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add WorldGraph example + momentum optimizer; harden diffcut for K>2
- examples/worldgraph.rs: RuView WorldGraph -> condense -> OccWorld demo.
WeakBoundary condenses 600 observations into 12 event summaries (50x,
100% activity purity, cut preserved 1.000); a smaller dense scene shows
the trained DiffMinCut recovering ~86% activity purity.
- diffcut: add heavy-ball `momentum` to DiffCutConfig (default 0.0, all
existing behaviour/tests/benchmarks unchanged) and unit-scale logit init
for stronger symmetry-breaking at K>2.
- Extend the gradient check to K = 2, 3, 4 (proves the K-general gradient
formulas; max abs error < 1e-5).
- Honest finding documented in ADR-197: DiffMinCut (MinCutPool-style) is
K-sensitive — reliable at small/moderate K, underperforms WeakBoundary at
large K, reinforcing WeakBoundary as the default (ADR-196).
- Workspace manifest validated (member resolves; crate is additive so it
cannot break other crates).
43 tests pass (36 unit + 6 integration + 1 doctest); clippy clean; all
source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Optimize trained min-cut for large K: Adam + warm-start + restarts
Plain/momentum GD from random init stalled the differentiable min-cut at
large K (12-event WorldGraph: ~30% purity, ~24s @ 96 nodes). Rebuilt the
optimizer so the trained method is viable at scale:
- Split loss math into cutloss.rs (CompactGraph + softmax + cut/ortho +
analytic gradients, gradient-checked K=2,3,4); diffcut.rs now owns the
optimizer/orchestration. Both files <500 lines.
- Optimizer enum: Adam (default; adaptive moments) and Sgd { momentum }.
- InitStrategy enum: WarmStart (default) seeds logits from the WeakBoundary
structural prior and refines (coreset/K-Center idea), or Random.
- restarts: keep the lowest-loss run. Deterministic region ordering in
warm-start so same seed => identical result.
Result on the 12-event WorldGraph example: DiffMinCut now reaches 100%
activity purity, cut preserved (inflation 1.000) — matching WeakBoundary —
in milliseconds (bench condense_diffcut: ~0.96ms @64, ~6.4ms @192 nodes;
was ~24s @96 under plain GD).
New tests: warm_start_recovers_many_clusters (K=8, purity>0.85),
warm_start_beats_random_at_large_k, warm_start_seeds_a_good_partition,
adam_refines_to_low_cut. Config call sites use ..Default::default().
ADR-197 updated. 47 tests pass (38 unit + 8 integration + 1 doctest);
clippy clean.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* diffcut scale levers: early-stop, Rayon parallelism, edge-minibatching
Three further optimizations for large/million-node graphs (off by default):
- Early-stopping (tolerance, default 1e-6): warm-start lands near the
optimum, so stop when the loss plateaus. iterations_run() reports actual.
- Parallelism (parallel, Rayon): CSR row-parallel A·S plus parallel O(N·K²)
SᵀS + ortho-gradient loops. Deterministic / bit-identical to sequential
(same chunked partial-sum ordering), proven by a test.
- Edge-minibatching (minibatch_edges): stochastic gradient from a sampled
edge subset, O(batch·K)/step; final loss still full-batch exact.
Refactor: cutloss.rs gains CSR adjacency + as_matrix (parallel) +
as_matrix_minibatch + a chunked gram(); loss_and_grad split so the optimizer
supplies A·S. New tests: parallel_matches_sequential_exactly,
minibatch_recovers_structure, early_stopping_cuts_iterations. New bench group
condense_diffcut_levers (1024 nodes, 4 cores: seq ~95ms, parallel ~83ms,
minibatch ~77ms). ADR-197 updated.
50 tests pass (38 unit + 11 integration + 1 doctest); clippy clean; all
source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add GNN accuracy-retention harness (closes the no-accuracy-validation gap)
Implements the graph-condensation field's core success metric: train a GNN
on the condensed graph, test on the ORIGINAL graph's held-out nodes, report
accuracy(condensed)/accuracy(full).
- gnn_eval.rs: self-contained, dependency-free 2-layer GCN (symmetric-
normalised CSR propagation, ReLU, softmax-CE, Adam, analytic backprop).
Gradient-checked against finite differences (<1e-6) and verified to learn a
separable task.
- examples/accuracy_eval.rs + tests/accuracy.rs: the full protocol on a
controlled synthetic node-classification task (planted communities as
classes, noisy features so the graph carries real signal).
Measured: baseline (full-graph GNN) 100%. On an UNWEIGHTED graph (the SOTA
benchmark setting), DiffMinCut condensing 360 nodes -> 18 super-nodes (20x)
yields **100% retention** (GNN trained on 18 nodes matches the full-graph GNN
on held-out test nodes).
Also fixes a real failure the harness surfaced: on uniform-weight graphs
WeakBoundary collapses to one component; DiffMinCut's warm-start inherited
that collapse. Warm-start now falls back to random init when the structural
prior finds <2 regions, letting the min-cut objective do the partitioning
(retention 14.9% -> 66% at K=classes, 100% at K=3*classes).
Honest scope: controlled synthetic data, not Cora/Citeseer; WeakBoundary
still needs weight contrast (documented). 53 tests pass; clippy clean.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add WASM bindings + gate Rayon behind a feature for wasm builds
- crates/ruvector-graph-condense-wasm: wasm-bindgen bindings exposing
condense_weak / condense_diffmincut / version to JS. Graphs in as flat
typed arrays, CondensedGraph out as JSON. Builds for
wasm32-unknown-unknown (667 KB release, pre wasm-opt), so the condenser
(including the trained DiffMinCut) runs in the browser / on the edge —
the deployable-artifact goal from the original brief.
- ruvector-graph-condense: Rayon is now an optional `parallel` feature
(default on for native, off for wasm — no threads on
wasm32-unknown-unknown). cutloss.rs cfg-gates every Rayon path with a
sequential fallback; no-default-features builds clean.
- getrandom `js` backend is wasm-target-gated so native feature
unification is unaffected; ruvector-mincut built with its `wasm` feature.
- ADR-196 updated with the WASM deployment + accuracy-validation notes.
53 tests pass; clippy clean (both crates); native + wasm32 both build.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Add ruvector-perception: the layer under classification (delta->proof->action)
Beyond-SOTA wedge: instead of a better CSI classifier, build the substrate
underneath one. Pipeline: delta -> boundary -> coherence -> proof -> action.
Emits a structured DeltaWitness, not a class label, and requires evidence
(not confidence) before exercising bounded authority.
- modality.rs: physically-typed modalities (RF/vibration/acoustic/thermal/
chemical/optical) with latency/decay/spoof-resistance — typed graph edges.
- state.rs: rolling per-(zone,modality) baselines + learned responsiveness.
- coherence.rs: zones as a coherence graph; dynamic min-cut isolates the moved
boundary (reuses ruvector-mincut). Coherence = separation cleanliness.
- witness.rs: ProofGate (Ignore/Observe/Alert/Mutate) + SHA-256 evidence
chain. Contradicted evidence is capped at Observe (no escalation on
confidence alone). Contradiction = a modality that usually reacts here but
stayed silent, weighted by spoof-resistance.
- engine.rs: orchestrates delta -> boundary -> contradiction -> novelty
(nearest-prior) -> proof gate -> chained witness.
- absence.rs: missing expected continuation (bed_exit->bathroom->return) as a
structural safety signal, not a threshold.
Flagship test reproduces the brief exactly: an inert object move yields
changed_boundary=table_left_zone, supporting={rf,vibration,acoustic},
contradicting={thermal}, novelty=high, action=observe. ADR-198 documents the
architecture and honest scope (mechanism on synthetic deltas, not validated on
real CSI).
11 tests pass; clippy clean; all files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Perception: 5 beyond-classification capability modules (swarm-built)
Built via a 5-agent parallel swarm, then integrated and validated. Each
emits structure, not a class label:
- captcha: Physical CAPTCHA — learned per-stimulus multi-modal challenge-
response profiles; verifies a fresh response (delay/magnitude tolerance,
spoof-resistance weighted) -> RealityProof. Detects replay/spoof.
- predict: Boundary-first world model — forecasts where coherence breaks next
(instability = coherence*(1+contradiction), level + least-squares trend).
- identity: Resonant identity / continuity — per-object EWMA signature, cosine
drift detection ("is this still the same physical thing?").
- hypothesis: Multi-modal disagreement engine — contradictions produce ranked
hypotheses (RealEvent/SensorDrift/SensorRelocation/AdversarialReplay/
EnvironmentalArtifact), not forced agreement.
- topology: Self-healing sensor topology — EWMA agreement graph; roles
Critical/Redundant/Noisy/Normal. Critical = articulation point (removal
fragments the graph) — replaced the agent's unreliable min-cut-partition
rule with robust articulation detection so triangle/star outliers keep their
real roles.
lib.rs re-exports all five. ADR-198 updated. 42 tests pass (38 unit + 2
integration + 2 doctest); clippy clean; all source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* Perception: complete the substrate — custody, swarm, reality-graph, node
Final beyond-classification pieces (custody + swarm built by a 2-agent swarm;
reality + node integration built directly):
- custody: tamper-evident, replayable chain-of-custody ledger over witness
evidence hashes (chain-linkage verification; honest scope: link integrity,
not raw-signal re-hash).
- swarm: facility/swarm-scale fragility — coupling graph + global min-cut
answers "where is the system closest to breaking?". Bottlenecks derived from
the weakest link (edge weights), since the engine's min-cut value is reliable
but its partition is not (same quirk handled in topology).
- reality: reality-graph agent grounding — an agent queries physical state
(presence / changed-since / which-untrusted / action-allowed) and gets
answers backed by witness evidence hashes, not prompt inference.
- node: NervousSystemNode appliance facade wiring engine + reality + custody +
boundary forecaster; emits deltas/boundaries/witnesses/forecasts (no raw
signal) and answers grounded queries.
Fixes during integration: swarm bottleneck now uses the weakest edge (engine
partition is unreliable); node test uses 3 zones (2-zone min-cut boundary is
ambiguous — a real limitation now documented). ADR-198 updated.
59 tests pass (54 unit + 2 integration + 3 doctest), deterministic; clippy
clean; all source files <500 lines.
https://claude.ai/code/session_01RehxmT96dnBFxStu9LJyKX
* chore(ci): wire condense+perception crates into publish + regression guard (#547)
Aligns the new ruvector-graph-condense, ruvector-graph-condense-wasm, and
ruvector-perception crates with the workspace release plumbing.
- Bump their ruvector-mincut (and graph-condense) dep pins from "2.0.1" to
"2.2.3" to match the workspace version they are built and tested against.
The old "^2.0.1" pin would resolve a crates.io publish against the stale
published mincut 2.0.6, risking a crate that fails to compile downstream.
- publish-all.yml: publish the three crates (plus mincut as substrate) to
crates.io in dependency order with index-settle waits, matching the
existing --allow-dirty / continue-on-error style.
- regression-guard.yml: run the new crates' tests (they were build-checked
but never tested in CI) and forbid regressing the mincut pin back to 2.0.x.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(graph-condense): rustfmt, clippy -D warnings, and cargo-deny advisory (#547)
CI green-up for the new condense/perception crates:
- rustfmt: format all source/bench/example/test files in the new crates
(the PR was committed unformatted; CI Rustfmt flagged all 29 files).
- clippy -D warnings: condense.rs used `sort_by(|a,b| key.cmp(&key))` which
trips clippy::unnecessary_sort_by under `-D warnings`; switch to
`sort_by_key`. (Earlier local clippy didn't deny warnings, so it slipped.)
- cargo-deny: ignore RUSTSEC-2026-0173 (proc-macro-error2 unmaintained).
Pre-existing transitive dep (validator_derive -> validator, via the
ruvector-scipix example), same crate family as the already-ignored
RUSTSEC-2024-0370. Not introduced by this PR. Re-review 2026-07-01.
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs(graph-condense): add crate READMEs for crates.io publish (#547)
The new graph-condense crates were wired to publish without a README (101/136
workspace crates have one; every published crate does). Add READMEs matching
the repo's badge-header convention and the `readme = "README.md"` field so the
crates.io pages render properly on first publish.
- ruvector-graph-condense: overview, SOTA positioning, quick-start (using the
real NodeFeatures::new/set + DynamicGraph::insert_edge API), region-method
table, and the honest ADR-196/197 limitations.
- ruvector-graph-condense-wasm: short binding README pointing at the core crate.
Perception crate intentionally left as-is (out of scope for this request).
Co-Authored-By: claude-flow <ruv@ruv.net>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: ruvnet <ruvnet@gmail.com>
|
||
|---|---|---|
| .. | ||
| adr | ||
| analysis | ||
| api | ||
| architecture | ||
| benchmarks | ||
| cloud-architecture | ||
| cnn | ||
| code-reviews | ||
| dag | ||
| development | ||
| examples | ||
| gnn | ||
| guides | ||
| hailo | ||
| hnsw | ||
| hooks | ||
| implementation | ||
| integration | ||
| nervous-system | ||
| optimization | ||
| plans/subpolynomial-time-mincut | ||
| postgres | ||
| project-phases | ||
| publishing | ||
| research | ||
| reviews | ||
| ruvllm | ||
| rvagent | ||
| sdk | ||
| security | ||
| sparse-inference | ||
| sql | ||
| testing | ||
| training | ||
| .gitkeep | ||
| .nojekyll | ||
| agi-container.md | ||
| C2-shell-execution-hardening.md | ||
| C8_RESULT_VALIDATION_IMPLEMENTATION.md | ||
| consciousness-api.md | ||
| IMPLEMENTATION-C5.md | ||
| index.html | ||
| INDEX.md | ||
| moe-routing-optimization-analysis.md | ||
| README.md | ||
| REPO_STRUCTURE.md | ||
| research-openfang.md | ||
RuVector Documentation
Complete documentation for RuVector, the high-performance Rust vector database with global scale capabilities.
📚 Documentation Structure
docs/
├── adr/ # Architecture Decision Records
├── analysis/ # Research & analysis docs
├── api/ # API references (Rust, Node.js, Cypher)
├── architecture/ # System design docs
├── benchmarks/ # Performance benchmarks & results
├── cloud-architecture/ # Cloud deployment guides
├── code-reviews/ # Code review documentation
├── dag/ # DAG implementation
├── development/ # Developer guides
├── examples/ # SQL examples
├── gnn/ # GNN/Graph implementation
├── guides/ # User guides & tutorials
├── hnsw/ # HNSW index documentation
├── hooks/ # Hooks system documentation
├── implementation/ # Implementation details & summaries
├── integration/ # Integration guides
├── nervous-system/ # Nervous system architecture
├── optimization/ # Performance optimization guides
├── plans/ # Implementation plans
├── postgres/ # PostgreSQL extension docs
├── project-phases/ # Development phases
├── publishing/ # NPM publishing guides
├── research/ # Research documentation
├── ruvllm/ # RuVLLM documentation
├── security/ # Security audits & reports
├── sparse-inference/ # Sparse inference docs
├── sql/ # SQL examples
├── testing/ # Testing documentation
└── training/ # Training & LoRA docs
Getting Started
- guides/GETTING_STARTED.md - Getting started guide
- guides/BASIC_TUTORIAL.md - Basic tutorial
- guides/INSTALLATION.md - Installation instructions
- guides/AGENTICDB_QUICKSTART.md - AgenticDB quick start
- guides/wasm-api.md - WebAssembly API documentation
Architecture & Design
- architecture/ - System architecture details
- cloud-architecture/ - Global cloud deployment
- adr/ - Architecture Decision Records
- nervous-system/ - Nervous system architecture
API Reference
- api/RUST_API.md - Rust API reference
- api/NODEJS_API.md - Node.js API reference
- api/CYPHER_REFERENCE.md - Cypher query reference
Performance & Benchmarks
- benchmarks/ - Performance benchmarks & results
- optimization/ - Performance optimization guides
- analysis/ - Research & analysis docs
Security
- security/ - Security audits & reports
Implementation
- implementation/ - Implementation details & summaries
- integration/ - Integration guides
- code-reviews/ - Code review documentation
Specialized Topics
- gnn/ - GNN/Graph implementation
- hnsw/ - HNSW index documentation
- postgres/ - PostgreSQL extension docs
- ruvllm/ - RuVLLM documentation
- training/ - Training & LoRA docs
Development
- development/CONTRIBUTING.md - Contribution guidelines
- development/MIGRATION.md - Migration guide
- testing/ - Testing documentation
- publishing/ - NPM publishing guides
Research
- research/ - Research documentation
- cognitive-frontier/ - Cognitive frontier research
- gnn-v2/ - GNN v2 research
- latent-space/ - HNSW & attention research
- mincut/ - MinCut algorithm research
🚀 Quick Links
For New Users
- Start with Getting Started Guide
- Try the Basic Tutorial
- Review API Documentation
For Cloud Deployment
- Read Architecture Overview
- Follow Deployment Guide
- Apply Performance Optimizations
For Contributors
- Read Contributing Guidelines
- Review Architecture Decisions
- Check Migration Guide
For Performance Tuning
- Review Optimization Guide
- Run Benchmarks
- Check Analysis
📊 Documentation Status
| Category | Directory | Status |
|---|---|---|
| Getting Started | guides/ | ✅ Complete |
| Architecture | architecture/, adr/ | ✅ Complete |
| API Reference | api/ | ✅ Complete |
| Performance | benchmarks/, optimization/, analysis/ | ✅ Complete |
| Security | security/ | ✅ Complete |
| Implementation | implementation/, integration/ | ✅ Complete |
| Development | development/, testing/ | ✅ Complete |
| Research | research/ | 📚 Ongoing |
Total Documentation: 460+ documents across 60+ directories
🔗 External Resources
- GitHub Repository: https://github.com/ruvnet/ruvector
- Main README: ../README.md
- Changelog: ../CHANGELOG.md
- License: ../LICENSE
Last Updated: 2026-02-26 | Version: 2.0.4 (core) / 0.1.100 (npm) | Status: Production Ready