mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-26 07:44:05 +00:00
ce1afecb22
* feat(ruvector-rabitq-wasm): WASM bindings for RaBitQ via wasm-bindgen
Closes the WASM gap from `docs/research/rabitq-integration/` Tier 2
("WASM / edge: 32× compression makes on-device RAG feasible") and
ADR-157 ("VectorKernel WASM kernel as a Phase 2 goal"). Adds a
`ruvector-rabitq-wasm` sibling crate that exposes `RabitqIndex` to
JavaScript/TypeScript callers (browsers, Cloudflare Workers, Deno,
Bun) via wasm-bindgen.
```js
import init, { RabitqIndex } from "ruvector-rabitq";
await init();
const dim = 768;
const n = 10_000;
const vectors = new Float32Array(n * dim); // populate
const idx = RabitqIndex.build(vectors, dim, 42, 20);
const query = new Float32Array(dim);
const results = idx.search(query, 10); // [{id, distance}, ...]
```
## Surface
- `RabitqIndex.build(vectors: Float32Array, dim, seed, rerank_factor)`
- `idx.search(query: Float32Array, k) → SearchResult[]`
- `idx.len`, `idx.isEmpty`
- `version()` — crate version baked at build time
- `SearchResult { id: u32, distance: f32 }` — mirrors the Python SDK
(PR #381) shape so callers porting code between languages get
identical structures.
## Native compatibility tweak
`ruvector-rabitq` had one rayon call site in
`from_vectors_parallel_with_rotation`. WASM is single-threaded — gated
that path on `cfg(not(target_arch = "wasm32"))` with a sequential
`.into_iter()` fallback for wasm. Output is bit-identical because the
rotation matrix is deterministic (ADR-154); parallel ordering doesn't
affect bytes.
`rayon` is now `[target.'cfg(not(target_arch = "wasm32"))'.dependencies]`
so the wasm build doesn't pull it in. Native build behavior unchanged
(39 / 39 lib tests still pass).
## Crate layout
crates/ruvector-rabitq-wasm/
Cargo.toml cdylib + rlib, wasm-bindgen 0.2, abi-3-friendly
src/lib.rs ~150 LoC of bindings; tests gated to wasm32 via
wasm_bindgen_test (native test would panic in
wasm-bindgen 0.2.117's runtime stub).
## Testing strategy
Native tests of WASM bindings panic by design — `JsValue::from_str`
calls into a wasm-bindgen runtime stub that's `unimplemented!()` on
non-wasm32 targets (since 0.2.117). The right path is
`wasm-pack test --node` or `wasm-pack test --headless --chrome`,
which we'll wire into CI as a follow-up.
The numerical correctness is already covered by `ruvector-rabitq`'s
own test suite. This crate only adds the JS-facing surface.
## Verification (native)
cargo build --workspace → 0 errors
cargo build -p ruvector-rabitq-wasm → clean
cargo clippy -p ruvector-rabitq-wasm --all-targets --no-deps -- -D warnings → exit 0
cargo test -p ruvector-rabitq → 39 / 39 (unchanged)
cargo fmt --all --check → clean
WASM target build (`wasm32-unknown-unknown`) requires `rustup target
add wasm32-unknown-unknown` — not exercised in this PR; will be
covered by a follow-up CI job.
Refs: docs/research/rabitq-integration/ Tier 2, ADR-157
("Optional Accelerator Plane"), PR #381 (Python SDK shape mirror).
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(acorn): add ruvector-acorn crate — ACORN predicate-agnostic filtered HNSW
Implements the ACORN algorithm (Patel et al., SIGMOD 2024, arXiv:2403.04871)
as a standalone Rust crate. ACORN solves filtered vector search recall collapse
at low predicate selectivity by expanding ALL graph neighbors regardless of
predicate outcome, combined with a γ-augmented graph (γ·M neighbors/node).
Three index variants:
- FlatFilteredIndex: post-filter brute-force baseline
- AcornIndex1: ACORN with M=16 standard edges
- AcornIndexGamma: ACORN with 2M=32 edges (γ=2)
Measured (n=5K, D=128, release): ACORN-γ achieves 98.9% recall@10 at 1%
selectivity. cargo build --release and cargo test (12/12) both pass.
https://claude.ai/code/session_0173QrGBttNDWcVXXh4P17if
* perf(acorn): bounded beam, parallel build, flat data, unrolled L2²
Five linked optimizations to ruvector-acorn (≈50% smaller search
working set, ≈6× faster build on 8 cores, comparable or better
recall at every selectivity):
1. **Fix broken bounded-beam eviction in `acorn_search`.**
The previous implementation admitted that its `else` branch was
"wrong" (the comment literally said "this is wrong") and pushed
every neighbor into `candidates` unconditionally, growing the
frontier to O(n). Replace with a correct max-heap eviction:
when `|candidates| >= ef`, only admit a neighbor if it improves
on the farthest pending candidate, evicting that one. This gives
the documented O(ef) memory bound and stops wasted neighbor
expansions at the prune cutoff.
2. **Parallelize the O(n²·D) graph build with rayon.**
The forward pass (each node finds its M nearest predecessors) is
embarrassingly parallel — `into_par_iter` over rows. Back-edge
merge stays serial behind a `Mutex<Vec<u32>>` per node so the
merge is deterministic. ~6× faster on an 8-core box for 5K×128.
3. **Flat row-major vector storage.**
`data: Vec<Vec<f32>>` → `data: Vec<f32>` (length n·dim) with a
`row(i)` accessor. Eliminates the per-vector heap indirection,
keeps the L2² inner loop on contiguous memory the compiler can
vectorize, and trims index size by ~one allocation per row.
4. **`Vec<bool>` for `visited` instead of `HashSet<u32>`.**
O(1) lookup with no hashing or allocator pressure on the hot path.
5. **Hand-unroll L2² by 4.**
Four independent accumulators give LLVM enough room to issue
AVX2/SSE/NEON FMA chains on contemporary x86_64 / aarch64.
3-5× faster for D ≥ 64 in microbenchmarks.
Other:
- `exact_filtered_knn` parallelizes across data via rayon (recall
measurement only — needs `+ Sync` on the predicate).
- `benches/acorn_bench.rs` switches `SmallRng` → `StdRng` (the
workspace doesn't enable rand's `small_rng` feature so the bench
failed to compile).
- `cargo fmt` applied across the crate; CI's Rustfmt check was the
blocking failure on the original PR.
Demo run on x86_64, n=5000, D=128, k=10:
Build: ACORN-γ ≈ 23 ms (was 1.8 s)
Recall: 96.0% @ 1% selectivity (paper: ~98%)
92.0% @ 5% selectivity
79.7% @ 10% selectivity
34.5% @ 50% selectivity (predicate dilutes top-k truth)
QPS: 18 K @ 1% sel, 65 K @ 50% sel
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(acorn): clippy clean-up — sort_by_key, is_empty, redundant closures
CI's `Clippy (deny warnings)` flagged three lints introduced by the
previous optimization commit:
- `unnecessary_sort_by` (graph.rs:158, 176) → use `sort_by_key`
- `len_without_is_empty` (graph.rs) → add `AcornGraph::is_empty`
and `if graph.is_empty()` in search.rs
- `redundant_closure` (main.rs:65, 159, 160) → pass the predicate
directly to `recall_at_k` instead of `|id| pred(id)`
No semantic change.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(wasm): publish @ruvector/rabitq-wasm and @ruvector/acorn-wasm to npm
Two new WASM packages (both v0.1.0, MIT OR Apache-2.0, scoped under
@ruvector). Mirrors the existing @ruvector/graph-wasm packaging
pattern so release tooling treats all three uniformly.
- ADR-161: @ruvector/rabitq-wasm — RaBitQ 1-bit quantized vector
index. 32× embedding compression with deterministic rotation.
Wraps the existing crates/ruvector-rabitq-wasm crate.
- ADR-162: @ruvector/acorn-wasm — ACORN predicate-agnostic filtered
HNSW. 96% recall@10 at 1% selectivity with arbitrary JS predicates.
Adds crates/ruvector-acorn-wasm (new), wrapping the ruvector-acorn
crate from PR #391.
Each crate ships with:
- `build.sh` that runs `wasm-pack build` for web / nodejs / bundler
targets, emitting into npm/packages/{rabitq,acorn}-wasm/{,node/,bundler/}.
- A canonical scoped package.json (kept under git as
package.scoped.json because wasm-pack regenerates package.json from
Cargo metadata on every build).
- A README.md with install + usage for browser, Node.js, and bundler
contexts.
- A `.gitignore` that excludes the wasm-pack-generated artifacts
(.wasm + .js + .d.ts) so only canonical source lives in the repo.
Build sanity:
- `cargo check -p ruvector-acorn-wasm -p ruvector-rabitq-wasm` clean
- `cargo clippy -- -D warnings` clean for both
- `wasm-pack build` succeeds for all three targets on both crates
Published:
- @ruvector/rabitq-wasm@0.1.0 — 40 KB tarball, 71 KB wasm
- @ruvector/acorn-wasm@0.1.0 — 49 KB tarball, ~85 KB wasm
Root README updated with both packages in the npm packages table.
Note: this branch also carries cherry-picks of PR #391's `ruvector-acorn`
crate (commits
|
||
|---|---|---|
| .. | ||
| coherence-engine | ||
| delta-behavior | ||
| quantum-engine | ||
| temporal-tensor-store | ||
| ADR-001-ruvector-core-architecture.md | ||
| ADR-002-ruvllm-integration.md | ||
| ADR-003-simd-optimization-strategy.md | ||
| ADR-004-kv-cache-management.md | ||
| ADR-005-wasm-runtime-integration.md | ||
| ADR-006-memory-management.md | ||
| ADR-007-security-review-technical-debt.md | ||
| ADR-008-mistral-rs-integration.md | ||
| ADR-009-structured-output.md | ||
| ADR-010-function-calling.md | ||
| ADR-011-prefix-caching.md | ||
| ADR-012-security-remediation.md | ||
| ADR-013-huggingface-publishing.md | ||
| ADR-014-coherence-engine.md | ||
| ADR-015-coherence-gated-transformer.md | ||
| ADR-016-delta-behavior-ddd-architecture.md | ||
| ADR-017-temporal-tensor-compression.md | ||
| ADR-024-craftsman-ultra-30b-1bit-bitnet-integration.md | ||
| ADR-025-exo-ai-multiparadigm-integration.md | ||
| ADR-026-rvcow-branching-and-real-cognitive-containers.md | ||
| ADR-027-hnsw-parameterized-query-fix.md | ||
| ADR-028-ehealth-platform-architecture.md | ||
| ADR-029-rvf-canonical-format.md | ||
| ADR-030-rvf-cognitive-container.md | ||
| ADR-031-rvf-example-repository.md | ||
| ADR-032-rvf-wasm-integration.md | ||
| ADR-033-progressive-indexing-hardening.md | ||
| ADR-034-qr-cognitive-seed.md | ||
| ADR-035-capability-report.md | ||
| ADR-036-agi-cognitive-container.md | ||
| ADR-037-publishable-rvf-acceptance-test.md | ||
| ADR-038-npx-ruvector-rvlite-witness-integration.md | ||
| ADR-039-rvf-solver-wasm-agi-integration.md | ||
| ADR-040-causal-atlas-rvf-runtime-planet-detection.md | ||
| ADR-040a-planet-detection-dashboard.md | ||
| ADR-040b-microlensing-graphcut-extensions.md | ||
| ADR-042-Security-RVF-AIDefence-TEE.md | ||
| ADR-043-external-intelligence-providers.md | ||
| ADR-044-ruvector-postgres-v03-extension-upgrade.md | ||
| ADR-045-lean-agentic-integration.md | ||
| ADR-046-graph-transformer-architecture.md | ||
| ADR-047-proof-gated-mutation-protocol.md | ||
| ADR-048-sublinear-graph-attention.md | ||
| ADR-049-verified-training-pipeline.md | ||
| ADR-050-graph-transformer-bindings.md | ||
| ADR-051-physics-informed-graph-layers.md | ||
| ADR-052-biological-graph-layers.md | ||
| ADR-053-temporal-causal-graph-layers.md | ||
| ADR-054-economic-graph-layers.md | ||
| ADR-055-manifold-graph-layers.md | ||
| ADR-056-rvf-knowledge-export.md | ||
| ADR-057-federated-rvf-transfer-learning.md | ||
| ADR-058-hash-security-optimization.md | ||
| ADR-059-shared-brain-google-cloud.md | ||
| ADR-060-shared-brain-capabilities.md | ||
| ADR-061-reasoning-kernel-architecture.md | ||
| ADR-062-brainpedia-architecture.md | ||
| ADR-063-wasm-executable-nodes.md | ||
| ADR-064-pi-brain-infrastructure.md | ||
| ADR-065-npm-publishing-strategy.md | ||
| ADR-066-sse-mcp-transport.md | ||
| ADR-067-mcp-gate-permit-system.md | ||
| ADR-068-domain-expansion-transfer-learning.md | ||
| ADR-069-google-edge-network-deployment.md | ||
| ADR-070-npx-ruvector-unified-integration.md | ||
| ADR-071-npx-ruvector-ecosystem-gap-analysis.md | ||
| ADR-072-rvf-example-management-downloads.md | ||
| ADR-073-pi-platform-security-optimization.md | ||
| ADR-074-ruvllm-neural-embeddings.md | ||
| ADR-075-rvf-agi-stack-brain-integration.md | ||
| ADR-076-agi-capability-wiring-architecture.md | ||
| ADR-077-midstream-brain-integration.md | ||
| ADR-078-npx-ruvector-midstream-integration.md | ||
| ADR-079-sql-audit-script-hardening.md | ||
| ADR-080-npx-ruvector-deep-capability-audit.md | ||
| ADR-081-brain-server-v028-deploy-cli-fixes.md | ||
| ADR-082-brain-security-hardening.md | ||
| ADR-083-brain-training-loops.md | ||
| ADR-084-ruvllm-wasm-publish.md | ||
| ADR-085-neural-trader-ruvector.md | ||
| ADR-086-neural-trader-wasm.md | ||
| ADR-087-ruvix-cognition-kernel.md | ||
| ADR-088-cnn-contrastive-integration.md | ||
| ADR-089-cnn-browser-demo.md | ||
| ADR-090-implementation-checklist.md | ||
| ADR-090-ultra-low-bit-qat-pi-quantization-ddd.md | ||
| ADR-091-implementation-checklist.md | ||
| ADR-091-int8-cnn-quantization-ddd.md | ||
| ADR-092-moe-memory-aware-routing-ddd.md | ||
| ADR-093-daily-discovery-brain-training.md | ||
| ADR-093-deepagents-rust-conversion-overview.md | ||
| ADR-094-deepagents-backend-protocol-traits.md | ||
| ADR-094-pi-shared-web-memory.md | ||
| ADR-095-deepagents-middleware-pipeline.md | ||
| ADR-095-pi-api-v2-capabilities.md | ||
| ADR-096-cloud-pipeline-realtime-optimization.md | ||
| ADR-096-deepagents-tool-system.md | ||
| ADR-097-deepagents-subagent-orchestration.md | ||
| ADR-098-deepagents-memory-skills-summarization.md | ||
| ADR-099-deepagents-cli-acp-server.md | ||
| ADR-100-deepagents-rvf-integration-crate-structure.md | ||
| ADR-101-deepagents-testing-strategy.md | ||
| ADR-102-deepagents-implementation-roadmap.md | ||
| ADR-103-deepagents-review-amendments.md | ||
| ADR-104-rvagent-mcp-skills-topology.md | ||
| ADR-105-rvagent-mcp-implementation-details.md | ||
| ADR-106-ruvix-kernel-rvf-integration.md | ||
| ADR-107-rvagent-native-swarm-wasm.md | ||
| ADR-108-rvagent-ruvbot-integration.md | ||
| ADR-109-backup-disaster-recovery.md | ||
| ADR-110-neural-symbolic-internal-voice.md | ||
| ADR-111-ruvocal-ui-rvagent-integration.md | ||
| ADR-112-rvagent-mcp-server.md | ||
| ADR-113-rvf-app-gallery-ruvix-applications.md | ||
| ADR-114-ruvector-core-hash-placeholders.md | ||
| ADR-115-common-crawl-temporal-compression.md | ||
| ADR-116-spectral-sparsifier-brain-integration.md | ||
| ADR-117-canonical-mincut-pseudo-deterministic.md | ||
| ADR-117-dragnes-dermatology-intelligence-platform.md | ||
| ADR-118-cost-effective-crawl-strategy.md | ||
| ADR-119-historical-crawl-evolutionary-comparison.md | ||
| ADR-120-wet-processing-pipeline.md | ||
| ADR-121-gemini-grounding-integration.md | ||
| ADR-122-rvagent-gemini-grounding-agents.md | ||
| ADR-123-brain-cognitive-enrichment.md | ||
| ADR-124-dynamic-partition-cache.md | ||
| ADR-125-resend-email-brain-integration.md | ||
| ADR-126-google-chat-brain-integration.md | ||
| ADR-127-gist-deep-research-loop.md | ||
| ADR-128-sota-gap-implementations.md | ||
| ADR-129-ruvltra-gcloud-training-turboquant.md | ||
| ADR-130-mcp-sse-decoupling-midstream-queue.md | ||
| ADR-131-consciousness-metrics-crate.md | ||
| ADR-132-e2e-browser-testing-claude-flow.md | ||
| ADR-132-ruvix-hypervisor-core.md | ||
| ADR-133-claude-code-source-analysis.md | ||
| ADR-133-partition-object-model.md | ||
| ADR-134-ruvector-claude-code-deep-integration.md | ||
| ADR-134-witness-schema-log-format.md | ||
| ADR-135-mincut-decompiler-with-witness-chains.md | ||
| ADR-135-proof-verifier-design.md | ||
| ADR-136-gpu-trained-deobfuscation-model.md | ||
| ADR-136-memory-hierarchy-reconstruction.md | ||
| ADR-137-bare-metal-boot-sequence.md | ||
| ADR-137-npm-decompiler-cli-and-mcp.md | ||
| ADR-138-llm-weight-decompiler.md | ||
| ADR-138-seed-hardware-bring-up.md | ||
| ADR-139-appliance-deployment-model.md | ||
| ADR-139-rvagent-claude-code-optimization.md | ||
| ADR-140-agent-runtime-adapter.md | ||
| ADR-141-coherence-engine-kernel-integration.md | ||
| ADR-142-tee-backed-cryptographic-verification.md | ||
| ADR-143-hearmusica-tympan-rust-port.md | ||
| ADR-143-implement-missing-capabilities.md | ||
| ADR-144-candle-whisper-musica-transcription.md | ||
| ADR-144-diskann-vamana-implementation.md | ||
| ADR-144-monorepo-quality-analysis-strategy.md | ||
| ADR-145-wasm-training-pipeline-fixes.md | ||
| ADR-146-diskann-vamana-implementation.md | ||
| ADR-147-stacked-kv-cache-triattention-turboquant.md | ||
| ADR-148-brain-hypothesis-engine.md | ||
| ADR-149-brain-performance-optimizations.md | ||
| ADR-150-pi-brain-ruvltra-tailscale.md | ||
| ADR-151-miller-rabin-prime-optimizations.md | ||
| ADR-153-kalshi-neural-trader-integration.md | ||
| ADR-154-rabitq-rotation-binary-quantization.md | ||
| ADR-155-rulake-datalake-layer.md | ||
| ADR-156-rulake-as-memory-substrate.md | ||
| ADR-157-optional-accelerator-plane.md | ||
| ADR-158-optional-rotation-and-qvcache-positioning.md | ||
| ADR-159-rvagent-a2a-protocol.md | ||
| ADR-161-rabitq-wasm-npm-package.md | ||
| ADR-162-acorn-wasm-npm-package.md | ||