Persist and restore COW map/membership state with strict parent, geometry, hash, and ancestry validation. Expose durable branch/freeze APIs across Node, TypeScript, and MCP; publish architecture-specific native packages through a corrected architecture-neutral wrapper; synchronize release lockfiles.
Refresh all committed Rust locks, eliminate actionable RustSec findings, make the npm graph reproducible and audit-clean, retire vulnerable optional backends, harden RuVocal production dependencies, and repair the affected publishable packages.
Closes#736.
Fix graph replication serialization, PostgreSQL HNSW concurrent-build safety and ef_search propagation, and SONA dimension/statistics correctness. Harden benchmark and CI execution, including cancellation of superseded runs.
Closes#727.
Closes#728.
Closes#729.
Closes#732.
Eliminate shell interpolation from all claude-flow hooks, remove unused vulnerable production dependencies, repair package exports/artifact generation, and prepare @ruvector/agentic-integration 1.0.2.
ADR-194 is already taken on main (ruvector ONNX embedder API & throughput).
Renumber this PR's turbovec ADR to the next free number (254), matching the
canonical record on main. Keeps the fuller PR version (D1–D5 divergences table,
D3/D4 measured-milestone markers) and adds a numbering note. Updates the 13
in-crate ADR-194 references and two stray ADR-193 'future work' pointers so they
no longer resolve to the unrelated ONNX ADR.
Refs #520, #521
Add quantizer_mse_within_paper_bound: draw 400k N(0,1) samples (Box–Muller,
no new deps), quantize via the real quantize_coord path, and assert the
per-coordinate MSE for every width stays under TurboQuant's distortion bound
D_mse ≤ (√3·π/2)·4^(−b) (arXiv:2504.19874) AND within 5% of the Max-1960
Lloyd–Max optimum. A corrupted centroid level trips this far more precisely
than the existing recall>0.5 threshold.
Marks D4 done in ADR-194; updates test count to 17. The full-pipeline
inner-product bound D_prod remains future work (tracked with D5).
Pure rustfmt normalization (rustfmt 1.8.0-stable) of lines that predated
this branch; no semantic change. Isolated from the feature commits so the
crate passes 'cargo fmt --check' cleanly for upstream CI.
Adds BitWidth::Three (8-level Max-1960 optimal N(0,1) reconstruction
levels). pack/unpack, calibration, scoring, and IdMap are width-generic,
so only the centroid table + the enum arms change.
Measured (cargo run --release -p ruvector-turbovec, n=5000 uniform-random,
dim=256, k=10, no rerank, vs exact L2):
3-bit: recall@10 0.767, 112 B/vec, 9.8x compression, bias -0.0000
landing squarely between 2-bit (0.561) and 4-bit (0.879) — a useful
memory/recall midpoint (~22% smaller than 4-bit for ~0.11 recall).
Also refresh ADR-194: add the 3-bit Validation row, mark D3 done, widen
T2 to {2,3,4}, correct the test count to 16, and scope the provenance
note so the measured recall/compression/bias figures are called measured
while the FAISS-competitive claims stay attributed targets.
16 unit + 1 doc-test pass; clippy clean; new code is rustfmt-clean.
Add an explicit 'Divergences from the TurboQuant paper (arXiv:2504.19874)'
section mapping where M1 departs from the paper, so the gaps are reviewable
and the follow-ups are paper-grounded:
- D1: M1 uses a heuristic per-vector c_x scale, not the paper's provably-
unbiased two-stage MSE + 1-bit-QJL-residual estimator. Soften the T4 and
Validation wording accordingly (empirically near-unbiased, not proven).
- D2: M1 quantizes against the N(0,1) limit + empirical TQ+ calibration, not
the paper's exact d-aware Beta-optimal codebooks.
- D3: M1 ships 1/2/4-bit; paper highlights ~2.5/3.5 bpc sweet spots — add 3-bit.
- D4: assert measured distortion under the paper's closed-form bounds as a
stronger test oracle than recall > 0.5.
- D5: estimator variance deferred.
Add milestones M5 (paper-grade QJL-residual estimator) and M6 (Beta-optimal
codebooks); note what M1 already matches (norm-based L2, online ingest,
full-precision query). No code change.
- Cargo.toml: remove unused rand_distr dependency and the redundant
rand dev-dependency (rand is a normal dep for the demo bin + tests).
- Cargo.lock: drop rand_distr from ruvector-turbovec.
- ADR-194: attribute the FAISS-competitive figures to the upstream
RyanCodrai/turbovec project rather than presenting them as this
crate's measured results; point readers to the reproducible
uniform-random Validation table instead.
No code changes; 16 unit + 1 doc-test still pass, clippy clean.
- index: TurboVecIndex::add/search now return RabitqError::DimensionMismatch
in release builds instead of silently accepting/masking wrong-length
vectors (was debug_assert + unwrap_or_default).
- index: finalize() excludes zero vectors from calibration fit so they
don't bias shift/scale toward zero.
- idmap: add_with_id validates dim up front and reports the real length
(was hardcoded got: 0); add_with_ids rejects vectors/ids length
mismatch with new TurboVecError::BatchLenMismatch instead of zip-truncating.
- quantize: pack/unpack document preconditions and debug_assert code-range
and slice-length (proportionate to internal helpers; no Result churn).
- calibrate: fit debug_asserts every row has length dim.
- ADR-194 frontmatter status proposed -> accepted to match body.
Adds 4 tests (wrong-dim reject on add/search, zero-vector calibration
exclusion via self-retrieval, batch-len mismatch, idmap wrong-dim).
16 unit tests + 1 doc-test pass; clippy clean; demo unchanged.
https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5
Research and scope a new crate adapting TurboQuant techniques from
RyanCodrai/turbovec: 2/4-bit Lloyd-Max scalar quantization, TQ+
per-coordinate calibration, length-renormalized unbiased scoring, and a
nibble-LUT FastScan SIMD kernel (AVX-512BW/AVX2/NEON). Reuses
ruvector-rabitq's Hadamard rotation + AnnIndex/VectorKernel traits and
borrows ruvllm's MSE quantizer math, closing the missing 2-4-bit FastScan
ANN regime.
https://claude.ai/code/session_012AzArCzBwxrJp8mUngUcH5
## update_node
Adds to GraphDB — the counterpart to .
Enables in-place property updates on existing nodes without delete+recreate.
This unblocks SUPERSEDES-style versioning where a prior node is marked
without deleting it:
Returns if the node was not found (no error), if
updated. Persists to storage if the feature is enabled.
Closes#666
## keyword_search
Adds to GraphDB — wires the
existing (bm25.rs) into the graph API. Returns top-k node IDs by
BM25 score over a text property for nodes with the given label.
This is the keyword arm of hybrid search — pair with vector ANN for
reciprocal rank fusion. Builds a transient index on each call (suitable
for small-to-medium graphs or one-shot queries).
Closes#667
## Tests
4 new tests, all passing:
- test_update_node — marks a node deprecated, verifies properties changed
- test_update_node_not_found — returns Ok(false) for missing node
- test_keyword_search — ranks relevant docs first, pasta doc doesn't lead
- test_keyword_search_empty_label — empty label returns empty results
Full suite: 21/21 pass (0.11s)
## Context
Filed by Allura (github.com/Allura-Ecosystem/Allura_Memory) during the
RuVector graph cutover (AD-49). Allura uses the graph adapter for governed
memory with SUPERSEDES versioning and hybrid search.
#657: RvfDatabase.query() accepted only a positional numeric count; passing an
options object ({k}/{topK}/{limit}) put the object in the count slot and leaked a
low-level "Failed to convert napi value Object into rust type u32" error. query()
now accepts both forms and validates the count, throwing a clear InvalidArgument
(new SDK code) for a missing/non-positive-integer count before crossing N-API.
#658: NodeBackend.create() on a path that already held a file surfaced the native
FsyncFailed durability error, sending debugging the wrong way. It now checks for an
existing file first and throws a clear FileExists (new SDK code) pointing at open()
or { overwrite: true }; overwrite removes the old file and its .idmap.json sidecar
before creating.
dist/ regenerated via tsc. scripts/smoke-query-create.mjs verifies both against the
built output (11 checks, no native addon required for #657).
Co-authored-by: Leo <noreply@khive.ai>
Co-authored-by: ruvnet <ruv@ruv.net>
The .idmap.json sidecar is load-bearing: delete() resolves string ids through it
and silently drops unresolvable ones, so a lost/torn/corrupt sidecar causes silent
data loss or corruption with no signal. Two swallows made that easy to hit:
saveMappings() wrote the sidecar non-atomically and swallowed all errors. It now
writes to a temp file and renames over the sidecar (so a crash/ENOSPC mid-write
cannot leave partial JSON), and surfaces a failure as SidecarWriteFailed instead of
discarding it — delete()/ingest correctness depends on the map, so it is not
best-effort.
loadMappings() degraded a corrupt sidecar to empty maps, which reset nextLabel to 1
and made subsequent ingests assign labels colliding with existing vectors (silent
corruption), then overwrote the recoverable file on the next save. It now quarantines
the corrupt sidecar (renames it aside) and throws SidecarCorrupt so the caller learns
string-id operations are unsafe, rather than silently corrupting data.
Two new SDK error codes (0xff06/0xff07); 0xff04/0xff05 left reserved. dist/ regenerated
via tsc. scripts/smoke-idmap-durability.mjs verifies atomic write, state restoration on
reopen, and fail-loud+quarantine on corrupt load (3 checks) against the built output.
Co-authored-by: Leo <noreply@khive.ai>
Co-authored-by: ruvnet <ruv@ruv.net>
