0247c1fc2b
* feat(ruvector): implement missing capabilities (ADR-143) - speculativeEmbed: real FNV-1a hash embedding (128-dim) from file content - ragRetrieve: cosine similarity on embeddings + TF-IDF keyword fallback - contextRank: TF-IDF weighted scoring instead of raw keyword matching - Remove false DiskANN claim (will implement as Rust crate next) Co-Authored-By: claude-flow <ruv@ruv.net> * feat(diskann): Vamana graph + PQ — SSD-friendly billion-scale ANN (ADR-143) New Rust crate: ruvector-diskann Core algorithm (NeurIPS 2019 DiskANN paper): - Vamana graph with α-robust pruning (bounded out-degree R) - k-means++ seeded Product Quantization (M subspaces, 256 centroids) - Asymmetric PQ distance tables for fast candidate filtering - Two-phase search: PQ-filtered beam search → exact re-ranking - Memory-mapped persistence (mmap vectors + binary graph) Performance characteristics: - L2-squared distance with 8-wide loop unrolling (auto-vectorized) - Greedy beam search with bounded visited set - Save/load with flat binary format (mmap-friendly) 9 tests passing: distance, PQ train/encode, Vamana build/search, bounded degree, full index CRUD, PQ-accelerated search, save/load. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(diskann): NAPI-RS bindings + npm package + 14 tests passing Rust core (ruvector-diskann): - 4-accumulator L2 distance for ILP optimization - Recall@10 = 1.000 on 2K vectors - Search latency: 90µs (5K vectors, 128d, k=10) - 14 tests: distance, PQ, Vamana, recall, scale, edge cases NAPI-RS bindings (ruvector-diskann-node): - Sync + async build/search - Batch insert (flat Float32Array) - Save/load, delete, count - Thread-safe via parking_lot::RwLock npm package (@ruvector/diskann): - Platform-specific loader (linux/darwin/win) - TypeScript declarations - Node.js test passing Co-Authored-By: claude-flow <ruv@ruv.net> * ci(diskann): add cross-platform build + publish workflow 5 targets: linux-x64, linux-arm64, darwin-x64, darwin-arm64, win32-x64 Co-Authored-By: claude-flow <ruv@ruv.net> * perf(diskann): FlatVectors + VisitedSet + ILP + optional SIMD/GPU Optimizations applied: - FlatVectors: contiguous f32 slab (eliminates Vec<Vec> indirection) - VisitedSet: O(1) clear via generation counter (replaces HashSet) - 4-accumulator ILP for L2 distance (auto-vectorized) - Flat PQ distance table (cache-line friendly) - Parallel medoid finding via rayon - Zero-copy save (write flat slab directly) - Optional simsimd feature for hardware NEON/AVX2/AVX-512 - Optional gpu feature with Metal/CUDA/Vulkan dispatch stubs Results (5K vectors, 128d): - Search: 90µs → 55µs (1.6x faster) - Build: 6.9s → 6.2s (10% faster) - Recall@10: 0.998 (maintained) - 17 tests passing Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Reuven <cohen@ruv-mac-mini.local>
2.6 KiB
ADR-143: Implement Missing Capabilities in ruvector
Status
Accepted
Date
2026-04-06
Context
A comprehensive audit of the ruvector npm package (v0.2.22) identified 3 gaps where claimed capabilities were either stubs or trivially implemented:
-
Speculative Embedding (parallel-workers.ts) - The
speculativeEmbedworker returned{ embedding: [], confidence: 0.5 }for all files. No actual embedding computation occurred. -
RAG Retrieval (parallel-workers.ts) - The
ragRetrieveandcontextRankworkers used keyword-matching (string.includes()) instead of semantic similarity on embeddings, despite the module claiming "Parallel RAG chunking and retrieval" and "Semantic deduplication." -
DiskANN / Vamana (README, package.json) - Claimed in README ("billion-scale SSD-backed ANN with <10ms latency") and package.json description/keywords, but no implementation exists anywhere in the codebase.
All other 14 modules were verified as real implementations (see release v2.1.1 audit).
Decision
1. Speculative Embedding - Implement real hash-based embedding
Replace the stub with the same multi-hash embedding approach used in intelligence-engine.ts (FNV-1a + positional encoding). This produces deterministic, consistent embeddings from file content without requiring ONNX or native modules. The worker already has access to fs for reading file content.
Embedding dimension: 128 (sufficient for co-edit prediction, avoids overhead of 384-dim).
2. RAG Retrieval - Implement cosine similarity on embeddings
When chunks include embeddings, use cosine similarity for ranking. Fall back to keyword matching only when embeddings are absent. This makes the existing embedding? field on ContextChunk actually functional.
Also upgrade contextRank to use TF-IDF weighting instead of raw keyword matching.
3. DiskANN - Remove false claims, add roadmap note
DiskANN/Vamana requires SSD-backed graph storage with PQ compression — a significant implementation effort that should be a dedicated Rust crate. Rather than ship a stub, remove the claim from README/package.json and add it to a roadmap section. The existing HNSW index (backed by hnsw_rs) already provides fast ANN search for in-memory datasets.
Consequences
- Speculative embedding becomes functional for co-edit prediction use cases
- RAG retrieval produces semantically meaningful results when embeddings are available
- README accurately reflects capabilities (no DiskANN claim without implementation)
- No new dependencies required (all implementations use existing math primitives)