* chore(rvf): sync Cargo.lock with rvf-wire deps (sha3, subtle) https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * fix(rvf-runtime): deterministic tie-breaking in query result ordering Equal-distance vectors were selected and ordered by HashMap iteration order, which changes across process restarts and made query results non-reproducible (flaky smoke_rvlite_adapter_persistence). Break ties by vector id in both the top-k heap eviction and the final sort, in query() and query_with_envelope(). https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * perf(rvf): optimize index/runtime hot paths, fix quant codec and manifest discovery rvf-index: - Cache SIMD distance-kernel dispatch in a OnceLock function-pointer table instead of re-running is_x86_feature_detected! on every call - Rewrite HNSW search_layer with BinaryHeap min/max-heaps (was sorted Vec + O(n) mid-inserts) and a dense Vec<bool> visited bitmap (was a per-call SipHash HashSet); deterministic (distance, id) tie-breaking rvf-runtime: - Replace per-bit CRC32 loops with crc32fast (same IEEE polynomial, byte-identical hashes, ~100x faster) on segment write and verify - Hoist cosine query-norm computation out of the per-vector scan loop - Safety-net scan: single pass with HashSet membership (was O(k*N*neighbors) with Vec::contains) - Bulk little-endian f32 serialization in write_vec_seg (one memcpy per vector instead of per-element appends) - Progressively widen the manifest tail scan (64KB -> 1MB -> 16MB -> whole file): stores with large segment directories were becoming unreadable once the latest manifest fell outside the fixed 64KB window; with regression test rvf-quant: - encode_quant_seg now emits fully decodable payloads (delegates to the real scalar/product encoders; placeholders removed) - decode_quant_seg returns Result instead of panicking on malformed or unknown-type payloads; round-trip and malformed-input tests https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * fix(rvm): bind witness chain to record content; optimize coherence, cap, sched hot paths rvm-witness (security-critical): - The chain hash covered only (prev_hash, sequence) — record content (action, actor, target, payload, timestamp) was never hashed, so verify_chain accepted arbitrarily rewritten history. record_hash is now computed over the 44 content bytes (as its doc always claimed) and the chain binds it: H(prev || seq || record_hash). verify_chain recomputes content hashes; tamper-regression tests added. - HMAC signer keys the Mac template once at construction instead of re-running the key schedule per record (fixed-vector test pins signature bytes) - Witness ring overflow is now observable: total_overwritten counter and needs_drain() accessor rvm-cap: - Nonce replay window: colliding nonces (A + k*4096) could evict and re-admit nonce A. Replaced the two 32KB arrays with one 32KB open-addressed table (8-probe bounded); eviction raises the watermark so it fails closed. Regression test included. rvm-coherence: - internal_weight: O(MAX_EDGES) self-loop scan replaced with O(1) adj_matrix[i][i] read (invariant verified across all mutation paths) - Skip ticks return a cached CoherenceDecision instead of re-running the O(n^2) merge-pair pass over stale data; zero-weight pairs skipped - Mincut: scratch buffers moved into the long-lived bridge (~17KB less stack per call), in-place Stoer-Wagner (no working copy), bitmask membership, column-scan in-neighbors - Compile-time guard: CoherenceGraph MAX_NODES > ADJ_DIM now fails to compile instead of panicking at the 33rd node; u64 weight deltas clamped at the engine boundary rvm-coherence/rvm-partition: - Single-slot hash indexes (id_to_node, edge_index) degraded to permanent O(N) scans after any collision; both now use bounded linear probing with tombstones and probe-proven absence rvm-sched: - enqueue() rejects the HYPERVISOR sentinel id, which previously wedged a run-queue slot permanently; defensive cleanup in switch_next Tests: 733 workspace + 67 rvm-kernel lib pass (baseline 712); 23 new tests including tamper-evidence and collision regressions. https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * feat(rvf): wire HNSW index into the runtime query path (~14x speedup) RvfStore::query was a brute-force O(N*dim) scan; the rvf-index crate was unused by production queries and QualityEnvelope.evidence fabricated layer_a=true. The index is now built lazily on first eligible query, maintained incrementally on ingest, persisted on close() via the existing INDEX_SEG codec (with a versioned, backward-readable trailer for the sparse-id mapping), and validated-or-rebuilt on open. Exact scan remains for small stores (<1024), filtered/COW/membership queries, >25% deleted, and force_exact; deterministic (distance, id) tie-breaking preserved on both paths. evidence.layer_a is now set only when the index served the query. Measured: 21.7ms -> 1.51ms per query at 100k x 64-dim (criterion, release), recall@10 = 0.968 at the ef_search=256 floor (>=0.95 gated by test). +15 tests (recall, index persistence round-trip, evidence honesty, fallback routing, compaction/overwrite invalidation). Co-Authored-By: claude-flow <ruv@ruv.net> * feat(rvm): witness v2 — keyed-BLAKE3 chain with 128-bit links + Merkle sealing v1 records folded chain links to 32 bits and left the head unanchorable. The 96-byte v2 record embeds the predecessor MAC full-width and chains via one keyed-BLAKE3 compression per append (~112ns measured, 9x under the 1us target); keyed MACs detect last-record tampering and unkeyed forgery, which v1 could not. Segment sealing accumulates record MACs into a domain-separated Merkle tree (256/segment) sealed with one signature via the existing signer infra (HMAC/dual-HMAC/Ed25519/TEE), with inclusion proofs — expensive crypto moves off the per-record path and roots are externally anchorable. v1 logs still verify (version-byte dispatch; v1 only as prefix, head anchored into the first v2 record); v1 writing is frozen. blake3 added as pure-Rust no_std. +46 tests covering content/reorder/truncation/wrong-key /forgery tamper modes, v1 compat, proofs, seals, and mixed logs. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(rvm): wire mincut into split decisions; honest partition-switch claim execute_split previously created an empty child and ignored the computed cut. It now resolves the boundary from a cached epoch SplitPlan (or computes on demand) and re-homes move-side neighbors to the child with their edge weights. Two-tier decisions: exact Stoer-Wagner mincut runs as a pressure-triggered epoch task; a new Fennel placer (O(degree), fixed-point gamma=1.5, no_std) handles hot-path placement. Split policy combines pressure and cut quality: mid-band (8000-9500bp) splits only on a cut with conductance <= 5000bp; critical pressure stays an unconditional safety valve. The sub-10us partition-switch claim was a stub certified by a no-op bench (~6ns) reported as 1600x faster than target. The real path needs EL2 assembly the crate forbids; instead the measurable register save/restore lower bound is implemented and benchmarked, the bench is renamed partition_switch_validation_stub with an honesty gate, a canary test fails if HARDWARE_SWITCH_IMPLEMENTED flips without revisiting the claim, and the README row now reads: not validated. +30 tests. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(rvf): RaBitQ binary quantization + Vamana alpha-pruning (opt-in) rvf-quant gains a RaBitQ-style codec: global-centroid centering, 3-round seeded randomized-Hadamard rotation (orthonormal, reproducible from a stored u64 seed), 1-bit sign codes with per-vector norm/dot-correction scalars, and an asymmetric full-precision-query estimator. QUANT_SEG adds versioned type tag 4 (legacy payloads byte-frozen and still decode; unknown versions rejected; decode stays panic-free on untrusted bytes). Query path: opt-in two-stage search (QueryOptions::rabitq, default off) — estimator scan with oversampling (640-candidate floor) then exact f32 rescore; deterministic (distance, id) tie-breaking; falls back to default routing for filtered/COW/IP/cosine queries. Measured recall@10 = 0.972 vs exact on 10k x 128 (gate >= 0.95, test-enforced); code-only compression exactly 32x. rvf-index: Vamana-style robust prune (alpha = 1.2, occluded backfill) at insert and prune time; recall@10 at ef=30 improved 0.986 -> 0.996; construction determinism preserved. +42 tests (1254 passing, no new failures). Co-Authored-By: claude-flow <ruv@ruv.net> * fix(rvf): harden untrusted decode paths against crafted-file DoS An adversarial audit confirmed a crafted .rvf could panic or OOM the process on RvfStore::open(): unvalidated length fields drove Vec::with_capacity before any byte-availability check. decode_payload now bounds id_count by available delta bytes (u64 compare before the usize cast, so 32-bit truncation cannot bypass it); decode_index_seg bounds restart_count/layer_count/neighbor_count by remaining bytes and rejects truncated restart padding (was a reachable slice panic); decode_sketch_seg converts from assert-and-panic to Result with width/depth validated via checked_mul (closes the width=0 + depth=u32::MAX bypass); decode_product size products use checked u64 arithmetic so 32-bit (wasm32) targets cannot wrap usize and read out of bounds. +8 adversarial regression tests. Co-Authored-By: claude-flow <ruv@ruv.net> * perf(rvf): contiguous vector slab, non-blocking index rebuild, unified hashing Vector storage moves from HashMap<u64, Vec<f32>> to a contiguous row-major slab (id->ordinal map, tombstoned deletes, slot reuse only via compaction); HNSW/RaBitQ paths read rows as zero-copy slices and iteration is ordinal- ordered (deterministic across restarts). Brute-force query at 100k x 64: 24.5ms -> 3.8ms (~6.4x). boot() pre-sizes the slab and bulk-copies VEC_SEG payloads (no per-vector allocs): cold open 257ms -> 202ms (-21.5%). mmap deferred (CRC verify touches all bytes anyway; memmap2 not in this workspace) and documented as follow-up. Audit finding 5: index/RaBitQ lazy builds now run with no lock held behind an AtomicBool gate (panic-safe clear-on-drop); concurrent queries fall back to exact scan and keep serving through the entire O(N log N) build. Overwrite still invalidates and unlinks the stale INDEX_SEG. Hashing: the two identical bespoke CRC32-rotation implementations in write_path/read_path now delegate to one source of truth (hashing::legacy_content_hash); on-disk bytes unchanged. Full rvf-wire checksum-registry conformance (XXH3-128 + format-version bump + dual-accept reader) documented as the remaining delta. read_path.rs also carries the audit''s checked vec-seg size arithmetic. +11 tests; suite 1271 passing, no new failures (one pre-existing wall-clock bench assertion flakes under load, passes in isolation). Co-Authored-By: claude-flow <ruv@ruv.net> * chore(release): prepare rvf 0.2.1/0.2.0/0.3.0 crate bumps, npm 0.2.2/0.1.7, measured-benchmark READMEs - rvf-types 0.2.0 -> 0.2.1 (QuantType::RaBitQ format extension) - rvf-index 0.1.0 -> 0.2.0 (Vamana alpha-pruning, hardened INDEX_SEG codec) - rvf-quant 0.1.0 -> 0.2.0 (RaBitQ codec; decode_sketch_seg now returns Result) - rvf-runtime 0.2.0 -> 0.3.0 (HNSW query path, INDEX_SEG trailer, QueryOptions::rabitq, vector slab) - dependent path-dep version reqs updated (cli, import, launch, node, server) - @ruvector/rvf 0.2.0 -> 0.2.2, @ruvector/rvf-wasm 0.1.6 -> 0.1.7 (rebuilt wasm artifact, 1.89 toolchain + wasm-opt -Oz) - READMEs: HNSW/RaBitQ/slab docs with measured numbers (Windows x64, criterion release, 100k x 64-dim); rvm witness v2 bench rows Co-Authored-By: claude-flow <ruv@ruv.net> * fix(robotics): bump rvf-runtime requirement to 0.3 after release bump The rvf-runtime 0.2 -> 0.3.0 version bump updated dependents inside the rvf workspace but missed the root-workspace consumer: ruvector-robotics pins version 0.2 alongside its path dep, which fails cargo resolution against the bumped crate (PR #555 CI: failed to select a version for the requirement rvf-runtime ^0.2). Root Cargo.lock refreshed. Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: ruv <ruvnet@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
rvf — RuVector Format CLI
Standalone command-line tool for creating, inspecting, querying, and managing RVF vector stores. Runs on Windows, macOS, and Linux with zero runtime dependencies.
Install
Pre-built binaries (recommended)
Download from GitHub Releases:
# macOS (Apple Silicon)
curl -L -o rvf https://github.com/ruvnet/ruvector/releases/latest/download/rvf-darwin-arm64
chmod +x rvf && sudo mv rvf /usr/local/bin/
# macOS (Intel)
curl -L -o rvf https://github.com/ruvnet/ruvector/releases/latest/download/rvf-darwin-x64
chmod +x rvf && sudo mv rvf /usr/local/bin/
# Linux x64
curl -L -o rvf https://github.com/ruvnet/ruvector/releases/latest/download/rvf-linux-x64
chmod +x rvf && sudo mv rvf /usr/local/bin/
# Linux ARM64
curl -L -o rvf https://github.com/ruvnet/ruvector/releases/latest/download/rvf-linux-arm64
chmod +x rvf && sudo mv rvf /usr/local/bin/
Windows (PowerShell):
Invoke-WebRequest -Uri https://github.com/ruvnet/ruvector/releases/latest/download/rvf-windows-x64.exe -OutFile rvf.exe
Build from source
Requires Rust:
cargo install --git https://github.com/ruvnet/ruvector.git rvf-cli
Or clone and build:
git clone https://github.com/ruvnet/ruvector.git
cd ruvector
cargo build -p rvf-cli --release
# Binary: target/release/rvf (or rvf.exe on Windows)
Quick start
# Create a 128-dimensional vector store with cosine distance
rvf create mydb.rvf --dimension 128 --metric cosine
# Ingest vectors from JSON
rvf ingest mydb.rvf --input vectors.json
# Search for nearest neighbors
rvf query mydb.rvf --vector "0.1,0.2,0.3,..." --k 10
# Check store status
rvf status mydb.rvf
Running the examples
The repo includes 48 pre-built .rvf example stores in examples/rvf/output/. Use the CLI to inspect, query, and manipulate them:
# List all example stores
ls examples/rvf/output/*.rvf
# Inspect a store
rvf status examples/rvf/output/basic_store.rvf
rvf inspect examples/rvf/output/basic_store.rvf
# Query the semantic search example (500 vectors, 384 dimensions)
rvf status examples/rvf/output/semantic_search.rvf
rvf inspect examples/rvf/output/semantic_search.rvf --json
# Inspect the RAG pipeline store
rvf status examples/rvf/output/rag_pipeline.rvf
# Look at COW lineage (parent -> child)
rvf inspect examples/rvf/output/lineage_parent.rvf
rvf inspect examples/rvf/output/lineage_child.rvf
# Check financial signals store
rvf status examples/rvf/output/financial_signals.rvf
# View the compacted store
rvf status examples/rvf/output/compacted.rvf
# Inspect agent memory store
rvf inspect examples/rvf/output/agent_memory.rvf
# View all stores at once (JSON)
for f in examples/rvf/output/*.rvf; do
echo "--- $(basename $f) ---"
rvf status "$f" 2>/dev/null
done
Available example stores
| Store | Vectors | Dim | Description |
|---|---|---|---|
basic_store.rvf |
100 | 384 | Basic vector store |
semantic_search.rvf |
500 | 384 | Semantic search embeddings |
rag_pipeline.rvf |
300 | 256 | RAG pipeline embeddings |
embedding_cache.rvf |
500 | 384 | Embedding cache |
filtered_search.rvf |
200 | 256 | Filtered search with metadata |
financial_signals.rvf |
100 | 512 | Financial signal vectors |
recommendation.rvf |
100 | 256 | Recommendation engine |
medical_imaging.rvf |
100 | 768 | Medical imaging features |
multimodal_fusion.rvf |
100 | 2048 | Multimodal fusion vectors |
legal_discovery.rvf |
100 | 768 | Legal document embeddings |
progressive_index.rvf |
1000 | 384 | Progressive HNSW index |
quantization.rvf |
1000 | 384 | Quantized vectors |
swarm_knowledge.rvf |
100 | 128 | Swarm intelligence KB |
agent_memory.rvf |
50 | 128 | Agent conversation memory |
experience_replay.rvf |
50 | 64 | RL experience replay buffer |
lineage_parent.rvf |
— | — | COW parent (lineage demo) |
lineage_child.rvf |
— | — | COW child (lineage demo) |
compacted.rvf |
— | — | Post-compaction store |
Commands
create
Create a new empty RVF store.
rvf create store.rvf --dimension 128 --metric cosine
rvf create store.rvf -d 384 -m l2 --profile 1 --json
Options:
-d, --dimension— Vector dimensionality (required)-m, --metric— Distance metric:l2,ip(inner product),cosine(default:l2)-p, --profile— Hardware profile 0-3 (default:0)--json— Output as JSON
ingest
Import vectors from a JSON file.
rvf ingest store.rvf --input data.json
rvf ingest store.rvf -i data.json --batch-size 500 --json
Input JSON format:
[
{"id": 1, "vector": [0.1, 0.2, 0.3, ...]},
{"id": 2, "vector": [0.4, 0.5, 0.6, ...]}
]
query
Search for k nearest neighbors.
rvf query store.rvf --vector "1.0,0.0,0.5,0.3" --k 10
rvf query store.rvf -v "0.5,0.5,0.0,0.0" -k 5 --json
With filters:
rvf query store.rvf -v "1.0,0.0" -k 10 \
--filter '{"eq":{"field":0,"value":{"string":"category_a"}}}'
delete
Delete vectors by ID or filter.
rvf delete store.rvf --ids 1,2,3
rvf delete store.rvf --filter '{"gt":{"field":0,"value":{"u64":100}}}'
status
Show store status.
rvf status store.rvf
rvf status store.rvf --json
inspect
Inspect store segments and lineage.
rvf inspect store.rvf
rvf inspect store.rvf --json
compact
Reclaim dead space from deleted vectors.
rvf compact store.rvf
rvf compact store.rvf --strip-unknown --json
derive
Create a derived child store (COW branching).
rvf derive parent.rvf child.rvf --derivation-type clone
rvf derive parent.rvf child.rvf -t snapshot --json
Derivation types: clone, filter, merge, quantize, reindex, transform, snapshot
freeze
Snapshot-freeze the current state.
rvf freeze store.rvf
verify-witness
Verify the tamper-evident witness chain.
rvf verify-witness store.rvf
verify-attestation
Verify kernel binding and attestation.
rvf verify-attestation store.rvf
serve
Start an HTTP server (requires serve feature).
cargo build -p rvf-cli --features serve
rvf serve store.rvf --port 8080
launch
Boot an RVF file in a QEMU microVM (requires launch feature).
cargo build -p rvf-cli --features launch
rvf launch store.rvf --port 8080 --memory-mb 256
JSON output
All commands support --json for machine-readable output:
rvf status store.rvf --json | jq '.total_vectors'
rvf query store.rvf -v "1,0,0,0" -k 5 --json | jq '.results[].id'
Platform scripts
Platform-specific quickstart scripts are in examples/rvf/scripts/:
# Linux / macOS
bash examples/rvf/scripts/rvf-quickstart.sh
# Windows PowerShell
.\examples\rvf\scripts\rvf-quickstart.ps1
License
MIT OR Apache-2.0