The phase-G ablation showed escalation firing on 100% of queries on
concentrated workloads while moving recall@10 only +0.7pp at 2.5x
latency — tight boundary margins there are precision-bound, which wider
traversal cannot fix and the VectorDB f32 verification tier now resolves
(-7pp -> +1.4pp vs the f32 index). Balanced therefore matches
MaxCompression's single-pass traversal (keeping its larger rescore
pool); Quality retains escalation as the safety net for deployments
with no higher-precision tier above the index.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
With Turbo4 quantization active, VectorDB::search now over-fetches 2k
candidates from the quantized index and re-ranks them against the stored
f32 vectors before truncating to k — nearly free, since result
enrichment fetches those vectors anyway.
Ablation justification (20k x 768-D clustered corpus, 200 queries):
4-bit rescoring pins recall@10 at ~0.88 in concentrated neighborhoods
regardless of oversampling (candidates all present; tail ranks are
precision-bound, so wider traversal cannot help). With verification:
config recall@10 P50 us payload
f32 HNSW 0.947 1708 3072 B/vec
t4 maxcomp+verify 0.961 1664 392 B/vec (7.8x)
cascade mc+verify 0.962 1202 496 B/vec (6.2x, 30% faster)
The cascade+verify configuration beats the f32 baseline on recall,
latency, and memory simultaneously.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
First ablation run (20k x 768-D clustered, 200 queries) showed the
precision-bound regime: Turbo4 recall@1 BEATS f32 (0.995 vs 0.945) and
MaxCompression is 1.6x faster than f32 at 7.8x compression, but
recall@10 pins at ~0.88 regardless of oversampling — with mult=8 the
true neighbors are already in the candidate pool; 4-bit rescoring cannot
order ranks 5-10 inside tightly concentrated clusters, so widening ef
(escalation) cannot help. That is exactly the case the ADR-297 §2
FP16/FP32 verification tier exists for: the new configs fetch 2k from
the quantized index and re-rank those ids against original f32 vectors.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
One deterministic clustered corpus, identical HNSW parameters, five
configurations: f32 baseline, Turbo4 direct under MaxCompression/
Balanced/Quality policies, and the RaBitQ1 cascade. Reports recall@1,
recall@10 vs brute-force ground truth, build time, P50/P95 latency,
payload bytes/vector, and adaptive escalation rate — the measurement
plane for the ADR-297 ablation gate.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
- VectorDB now persists a collection-level VectorProvenance record
(codec, codec_version, rotation_seed, dim, metric) in the vector store
itself when Turbo4 is active, and validates it on every reopen: a
mismatched rotation seed, dimension, metric, or codec version refuses
to open instead of silently serving wrong results. Tested: record
written and readable; tampered seed rejects reopen.
- ann_benchmark and memory_benchmark accept "turbo4" so the existing
harness measures the applied 4-bit path alongside none/scalar/binary.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
The ~5 bits/dim active search plane from ADR-297 §2, wired end to end:
- SearchQuantization { Turbo4Direct, RaBitQ1 } on QuantizationConfig::Turbo4.
In cascade mode node data is [bits1 || turbo4] with the 1-bit plane FIRST
(traversal touches only the short cache-friendly prefix); the HNSW walk
scores pure AND+POPCNT bit-plane kernels against the query, graph
construction still scores Turbo4 sections (build quality paid once,
traversal bandwidth every query), and candidates rescore on the shared
Turbo4 plane. One rotation per query serves both representations.
- bits1: flat query-blob form + slice-based scorer callable inside
Distance<u8>::eval (no allocation per candidate).
- New SIMD f32xnibble rescore kernel (pshufb levels -> cvtepi8/cvtdq2ps ->
fmadd, sequential byte order so the f32 query needs no scrambling), with
FMA runtime detection and a tolerance-gated scalar oracle test; rescore()
now dispatches through it. Level grid rounding is ~1% of code error, so
the rescore tier remains the highest-fidelity scorer.
- Cascade tests: recall within 5pp of direct mode on clustered data,
serialization roundtrip in cascade mode (search_quant tag persisted).
- Recall-vs-f32 gate widened 2pp -> 3pp: ADR target 0.5pp + measured
hnsw_rs graph-construction nondeterminism across independent builds.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
- bits1 module: RaBitQ-style sign codes over the SAME Turbo4 rotation
(one rotation pass, two codes; deterministic). Blob = word-padded sign
bits + alpha + c (per-vector MSE-optimal 1-bit scale, c = mean|z|).
Query int8 is decomposed into 8 bit-planes so candidate scoring is pure
AND+POPCNT (9 passes over D/64 words, ~4x less memory traffic than the
nibble kernel — the win when traversal is bandwidth-bound at scale).
Oracle test proves the popcount path equals the naive sign-sum; cascade
test (1-bit top-40 -> Turbo4 rescore -> top-10) gates recall on the
Gaussian worst case.
- Turbo4Codec::encode_dual: both planes from one rotation, byte-identical
to the single encoders (tested).
- AVX2 helpers now carry #[target_feature(enable = "avx2")] explicitly.
Measured kernel throughput (criterion, this hardware): asymmetric
int8xnibble 24.5 Gelem/s at 1536-D (62.6 ns/vector) vs the previous
widen-sequence kernel at 1.19 Gelem/s (1.29 us/vector) — the old inline
helpers without the target-feature attribute compiled to catastrophic
codegen; the bench keeps the old shape as a permanent baseline.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
- AVX2 kernels now use the abs/sign + maddubs idiom (the pshufb-LUT shape
from ADR-296 refinements §3), replacing four cvtepi8_epi16 + two madd
with abs/sign/maddubs/madd per 32 lanes. Exactness proven, not assumed:
the oracle test caught the sign(-128) wrap, fixed by putting the query
on the unsigned-abs side (0x80 reads as +128 there) and the level table
(±127 by construction) on the sign-negated side; bit-exact for the full
i8 input range, saturation-free (pair sums <= 32512).
- criterion bench (benches/kernels.rs) covering scalar dispatch and both
AVX2 variants — the old widen sequence stays in-bench as the baseline so
kernel changes remain measured.
- Turbo4HnswIndex::serialize/deserialize (bincode): codes + mappings +
(dim, metric, rotation_seed, rescore, policy); graph rebuilt from blobs
on load, vectors never re-encoded. Roundtrip test checks identical
rescored self-distances.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
Deep-research pass over primary sources (TurboQuant arXiv:2504.19874,
Qdrant 1.18 quantization docs/blog, RaBitQ SIGMOD 2024 + extended RaBitQ
arXiv:2409.09913, RaBitQ rebuttal arXiv:2604.19528) — findings recorded in
ADR-296 "Refinements from verified research":
- Lloyd-Max reconstructions are systematically short (||r|| = a*sqrt(S) <
a*sqrt(D)), biasing inner-product estimates — the bias TurboQuant fixes
with QJL and Qdrant fixes with RaBitQ renormalization. All three scoring
tiers now scale the level dot by sqrt(D/S) per encoded side and use exact
norms a^2*D — zero storage cost since a, S, D are already in the blob.
- New estimator-bias gate: mean signed relative L2 error across pairs must
stay under 1%.
- Kernel roadmap (maddubs/VPDPBUSD with u8-biased level table) and the
RaBitQ-cascade reinforcement documented for ADR-297 phases C/G.
- Stabilize adaptive-policy test: allow the hnsw_rs graph-construction
nondeterminism noise band when comparing separately built indexes.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
ADR-297 lays out the plane: one EncodedVector interface for every
representation, storage/search precision separation, per-query automatic
precision selection, memory tiers, topology-aware bit allocation, drift
detection, provenance, honest benchmarking, and a three-policy product
surface — gated by an ablation acceptance test (adaptive must beat uniform
Turbo4 by >= 30% memory at <= 0.5pp recall loss and <= 10% P95).
Phase B lands here:
- ruvector-core::encoding — CodecKind + VectorCodec/EncodedQuery traits;
Fp32, Fp16 (in-crate RNE binary16, no half dep), Int8, and Turbo4 plane
codecs; codec_for() registry; VectorProvenance schema (model id, codec
version, rotation seed, source hash, migration lineage).
ruvector-turboquant is now an unconditional core dependency (dep-free,
WASM-safe) so the codec plane exists on every build.
- SearchPolicy { Quality, Balanced, MaxCompression } on
QuantizationConfig::Turbo4 — users pick an outcome, not an algorithm.
- Turbo4HnswIndex adaptive escalation: relative kept/dropped score margin
triggers widened re-search (2-3x ef, 2x rescore pool), stopping when
top-k membership stabilizes; MaxCompression never escalates; telemetry
via adaptive_stats() targets a 5-15% escalation budget.
- clippy --all-targets clean for both crates (fixes CI identity_op).
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
Adds crates/ruvector-turboquant — a dependency-free, WASM-safe Turbo4 codec:
- deterministic randomized rotation (sign/permute/block-FWHT rounds over an
in-crate SplitMix64; bit-stable across platforms, no zero-padding, so codes
stay exactly ceil(D/2) bytes for any even D)
- precomputed 16-level Lloyd-Max tables (N(0,1), Max 1960) with per-vector
standardization alpha = ||v||/sqrt(D)
- packed nibble codes (D/2 + 8 bytes; ~7.9x vs f32 at 1536-D) — the original
float vector is never stored
- three scoring tiers, no reconstruction: symmetric code x code (graph
construction), asymmetric int8-query x code (traversal), exact f32 rescore
(final ranking); AVX2 kernels runtime-dispatched and tested bit-exact
against the scalar oracle
Wires it into ruvector-core (closes the Turbo4 slice of issue #563 —
quantization that is actually applied):
- QuantizationConfig::Turbo4 { rotation_seed, rescore_multiplier }
- Turbo4HnswIndex: hnsw_rs instantiated over u8 packed code blobs; query and
code blobs are structurally disjoint by length, so one Distance functor
gives symmetric construction + asymmetric traversal, then exact rescoring
of k * rescore_multiplier candidates
- VectorDB::new builds the quantized index when Turbo4 + HNSW are configured;
legacy variants keep the not-applied warning
- recall gate: <= 2pp loss vs the f32 HNSW baseline on clustered data, floor
0.75 on the iid-Gaussian concentration worst case
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01XFWB9PKwsZYk5FbjBRY6mk
Records the published @ruvector/rvforge@0.2.0 on npm.
Minor rather than patch: create is a new command that changes what the
package can do. 0.1.0 could verify and package an .rvf but could not
produce one, so the documented flow was unreachable from a clean
install.
Verified against the published tarball rather than the source tree —
installed 0.2.0 from the registry into an empty directory and ran
init --keygen, create, validate --deep. All exit 0.
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
* feat(forge-core): author module for writing signed RVF containers
rvf-forge-core could verify containers but not produce them, so every
test and fixture had to hand-assemble bytes through testkit. The
author module makes writing a first-class operation: ContainerBuilder
assembles segments, computes per-segment digests, and emits a signed
root manifest that this crate's own verifier accepts.
Segment kind decides signing policy rather than the caller: a .wasm
payload becomes an executable WASM segment and is signed individually,
anything else becomes an opaque VEC segment. That keeps rule 3 of the
loading contract — unsigned executable segments are rejected by
default — a property of the writer, not something each caller has to
remember to ask for.
The parity fixture generator now builds its input through this module
instead of a bespoke byte layout, so the TypeScript and Rust sides are
compared against a shared definition of what a valid container is.
138 tests, clippy clean.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
* feat(rvforge): add the create command that writes a signed agent.rvf
Closes the gap that made the published 0.1.0 unusable end to end:
init printed "Next: rvforge pack <agent.rvf>" while creating no such
file, so a first-time user's next command failed with FORGE_E_IO and
there was no supported way to produce the input every other command
needs. The only valid .rvf in the repo lived in tests/fixtures, which
is not in the published tarball.
create reads project metadata and declared capabilities from
rvforge.json and signs with the key init --keygen recorded, so the
common case takes no arguments. With no --from it writes a minimal
but complete skeleton — a META segment declaring the requested
capability classes and a signed root MANIFEST — which is enough for
validate, test, pack, publish and build to run. Walking the whole
pipeline before you have a model to put in it is the point.
--from <dir> adds files as segments in sorted order, so the same
input directory produces the same bytes.
init's next-step line now points at create rather than at a file it
does not write.
Verified from an empty directory against the built CLI: init, create,
validate --deep and test all exit 0 on a self-authored artifact.
253 tests.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
* feat(reader): install, library and update flows over verified artifacts
Takes rvforge-reader from a verification surface to one that manages
installed agents: install, a library of what is installed, and update
with rollback. Each flow re-verifies rather than trusting the step
before it — an artifact that verified at download is verified again
at install and again at load, because the file on disk between those
points is not the same object the check covered.
The dock bridge keeps the trust boundary the Dock exists to enforce.
Chrome the system owns — trust badge, network indicator, pause — is
populated from SystemOwnedStatus only, and agent-supplied text stays
in AgentProvidedStatus and is sanitized before display. A hostile
agent cannot forge an approved badge or claim it has stopped while
running, because the types do not give it a channel to those fields.
Update binds to lineage: an update whose base identity does not match
the installed artifact is refused rather than applied, and rollback
restores the previous version with its state capsule intact.
189 tests, clippy clean.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
* fix(deps): bump rkyv 0.8.16 to 0.8.18 for RUSTSEC-2026-0233/0234/0235
Three advisories published against rkyv 0.8.16: a use-after-free
during deserialization of crafted archives (RUSTSEC-2026-0233), and
out-of-bounds reads from insufficient archive validation for Rc/Arc
(0235) and hash tables (0234).
rkyv is a workspace-wide dependency of ruvector-core, ruvector-graph,
ruvector-router-core and ruvector-sparse-inference. All three
advisories are deserialization-side, which is where untrusted bytes
arrive, so an ignore entry would be the wrong call even though the
existing audit.toml has that mechanism — .cargo/audit.toml states the
policy directly: anything fixable is fixed via a dependency bump
rather than ignored.
Lockfile only, no manifest change. cargo audit exits 0 and the four
dependent crates check clean.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
* docs(rvforge): host the walkthrough on GitHub Pages
Adds docs/rvforge/index.html, served at
https://ruvnet.github.io/RuVector/rvforge/ by the existing Pages
config (main branch, /docs path).
A plain-language explanation of why process isolation does not bound
an agent, then an eight-command walkthrough from authoring a signed
artifact through running it under the capability gate, with seven
inline SVG diagrams that draw themselves on scroll.
Self-contained: no external fonts, scripts, or images, so it renders
under a strict CSP and works offline. Light and dark themes both
honour prefers-color-scheme and an explicit data-theme override.
Content is fully visible without JavaScript — the script only adds
the scroll animation and the artifact-state panel, so a script error
degrades the page rather than blanking it.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
* docs(rvforge): link the walkthrough from the README with a preview
Adds the preview image and points the README at the hosted
walkthrough, as both a clickable image and a plain text link so it
survives renderers that drop images.
The image is referenced by its absolute Pages URL rather than a
relative path, because this README is also published to npm, where a
repo-relative image resolves to nothing.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01ParP55bZs2iTGEGvpnUecx
Three ADRs implemented and hardened across five rounds of adversarial review, plus the fixes that review surfaced.
**ADR-280 — durable RVF metadata.** Delta-encoded generations with a snapshot every 32. The first implementation wrote a full snapshot per commit and replayed every one at open: 600 commits produced a 725 MiB file that could no longer be opened, with no repair path. Now 241 KB of META payload for the same workload, opening in ~4 ms. Review also closed: derive-children that could not be reopened, an 80-byte file driving a 512 MiB allocation, delete() rollback leaving in-memory tombstones that bricked the artifact, ten BufWriter sites discarding flush errors before sync_all, corrupt mid-chain deltas made unopenable (now recovers the longest valid prefix), and an ordering bug where recovery pruning committed without its re-anchoring snapshot so `rvf ingest` printed a repair warning and then destroyed the file.
**ADR-281 — role-aware embeddings.** Query/passage routing with an attested embedding-space identity. Review found the space id hashed CARGO_PKG_VERSION, so a routine version bump would have rejected every persisted corpus and invalidated every cache key — with the test suite structurally blind to it. Now keyed on a dedicated format revision with a golden-id test. Also: three constructors that failed unconditionally with ten unmigrated callers, prompt templates applied from the attested identity rather than hardcoded strings, and ApiEmbedding no longer bypassing templating.
**ADR-282 — nightly research quality gate.** Review found the gate had never completed a single run: the candidate checkout was shallow so its git diff always failed, and a jq quoting bug made the override path dead code. Check-run queries were unpaginated — on a real main commit 8 of 22 failures were invisible, so a red base could be certified green. Schemas are now load-bearing with a hashed dependency closure.
**CI note.** The two red checks are both pre-existing on main, not regressions from this branch: `Tests (core-and-rest)` routinely exceeds its 4-hour window, and `Hooks CI` has failed on main since 2026-08-02 (and in May) on `cp -r node_modules $GITHUB_WORKSPACE/npm/packages/cli/` in hooks-ci.yml — this branch's one-line version sync merely re-triggered its path filter. 72 checks pass.
Follow-ups filed and not blocking: #770, #771, #772.
🤖 Generated with [claude-flow](https://github.com/ruvnet/claude-flow)
Rebuilt from post-ADR-009 main (wire-contract codification). Export
surface identical to 0.1.8; wasm rebuilt and wasm-opt -Oz optimized.
Co-Authored-By: claude-flow <ruv@ruv.net>
Codifies the shipped RVF v1 wire format as the single normative contract: tail-discovered 4096-byte root manifest (no offset-zero header), exact little-endian magic wire bytes (segment 53 46 56 52, root 30 4D 56 52) exported as SEGMENT_MAGIC_BYTES/ROOT_MANIFEST_MAGIC_BYTES, golden byte-vector tests derived from shipped writer output (SHAKE-256 empty-input field matches the NIST vector; root CRC32C FF DD 18 14 verified), supersedes ADR-004/005 wire sections, fixes a tail_scan comment documenting the wrong anchor byte and doc pseudocode that compared wire bytes to literal ASCII, adds a pinned-action CI gate over rvf-types/rvf-wire. No wire bytes changed — existing artifacts, hashes, signatures remain valid.
🤖 Generated with [claude-flow](https://github.com/ruvnet/claude-flow)
Brings the ruvector npm package on main up to the 0.2.40 release content
(published from an unmerged branch: metaharness SDK/CLI/MCP surface, ONNX
embedder improvements, embedding provenance) and bumps to 0.2.41,
published post-PR-#752 with rebuilt NAPI binaries on main.
Co-Authored-By: claude-flow <ruv@ruv.net>
Research docs + target architecture for rvagent as a Hermes-class harness (metaharness + ruflo integration), ADRs 273-279, rvAgent harness repair (tool schemas wired, middleware pipeline, subagents, bootstrap, policy genome), PDX vertical-layout benchmark (not adopted), plus full adversarial code-review fix round: symlink/hard-link write-escape confinement in local tools, real HITL gating in both pipeline construction paths, Gemini parallel-tool-call and schema-compatibility fixes, panic/deadlock hardening.
CI note: Tests (vector-index) failure is the pre-existing flaky ruvector-diskann recall_trigger_holds_under_no_drift probabilistic test (untouched crate; passes 3/3 locally on this head, passed on prior run). Tests (core-and-rest) historically exceeds its window and was not required.
🤖 Generated with [claude-flow](https://github.com/ruvnet/claude-flow)
Persist and restore COW map/membership state with strict parent, geometry, hash, and ancestry validation. Expose durable branch/freeze APIs across Node, TypeScript, and MCP; publish architecture-specific native packages through a corrected architecture-neutral wrapper; synchronize release lockfiles.
Refresh all committed Rust locks, eliminate actionable RustSec findings, make the npm graph reproducible and audit-clean, retire vulnerable optional backends, harden RuVocal production dependencies, and repair the affected publishable packages.
Closes#736.
