* perf(hnsw): 4-accumulator AVX-512 kernels + SIMD wiring into search hot path
- Replace single-accumulator AVX-512 distance kernels with 4-accumulator
versions in simd_intrinsics.rs (euclidean, cosine, dot, manhattan).
On Zen 5 with 4-cycle FMA latency, single-accumulator was latency-bound
(96 cycles for 384-dim); 4-accumulator hides this to ~24 cycles.
- Wire HNSW search hot path in DistanceFn::eval to call simd_intrinsics
directly (inline, no Result wrapping, no simsimd FFI overhead).
- Enable parallel batch insert via hnsw_rs::parallel_insert_slice (rayon).
Measured: 6-10% QPS improvement on 128-dim/1K-vector bench; larger gains
expected on 1M-vector workloads where distance computation dominates.
228 unit tests pass.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* perf(hnsw): gate parallel_insert_slice behind 10K-vector threshold
Rayon-based parallel insert (hnsw_rs::parallel_insert_slice) degrades
graph connectivity for small batches (<10K vectors) because worker
threads can't see each other's in-flight insertions, reducing optimal
neighbor links. Add PARALLEL_THRESHOLD=10_000: use parallel insert only
when the batch is large enough that the graph quality converges.
Below threshold: sequential insert_data (same as before this PR).
Above threshold: parallel_insert_slice for build-time speedup.
228 unit tests pass.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* bench(sift1m): add SIFT-1M fvecs benchmark + hnswlib comparison tooling
Adds two benchmark binaries driven by the real TEXMEX SIFT-1M dataset:
* crates/ruvector-sota-bench/src/bin/sift1m_bench.rs
Reads sift_base.fvecs / sift_query.fvecs / sift_groundtruth.ivecs
directly (no HDF5 required). Sweeps ef_search to produce a
recall@10 vs QPS table used for before/after PR #619 comparison.
* scripts/sift1m_hnswlib_bench.mjs
Same sweep via hnswlib-node (C++ HNSW) to measure the competitive gap.
Cargo.toml: add simd-avx512 feature to sota-bench dependency so the
full optimised kernel path is exercised.
Measured on AMD Ryzen 9 9950X (Zen 5, AVX-512), M=16, efC=200, 1M vecs:
Source Build ef=100 recall ef=100 QPS ef=200 recall ef=200 QPS
before PR 849 s 0.9585 1,849 0.9713 1,058
after PR (#619) 774 s 0.9592 1,768 0.9722 1,024
hnswlib-node 322 s 0.9828 5,339 0.9957 2,897
Build speedup: +9.7 %. Query QPS at 1M-scale: within noise (memory-
bandwidth bound, not compute-bound).
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* style: cargo fmt for sift1m benchmark binary
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* fix(rvf): native COW dual-graph query correct for cosine metric (recall 0.10→~1.0)
Root cause
----------
The manifest format stored `profile_id` at byte [18] of the header but left
byte [19] as a reserved zero — it did NOT persist the `DistanceMetric`. When
`boot()` deserialized a manifest it only restored `epoch`, `dimension`, and
`profile_id`; the metric always stayed at `DistanceMetric::L2` (the
`RvfOptions::default()`).
In the COW dual-graph ANN path (`query_via_index_cow`) the parent store is
lazily opened via `open_readonly()` → `boot()`. Because `boot()` never
restored the metric, every COW child opened its parent with `metric = L2`,
even when the store family was Cosine. The parent HNSW was then built with
the L2 distance function, and parent query results were L2-ordered distances.
Merging those with the child's cosine distances broke the result ordering:
cosine recall@10 measured at ≈ 0.10 for 32-dim random vectors.
Fix
---
* `DistanceMetric` gets two new `pub(crate)` helpers:
- `to_id() -> u8`: L2=0, InnerProduct=1, Cosine=2
- `from_id(u8) -> Self`: reverse mapping (unknown → L2, backward-compatible)
* The manifest write path (`write_manifest_seg_with_identity`) now encodes
the metric into byte [19] of the header (previously a reserved zero).
Old stores have 0x00 there → `from_id(0)` == L2 — correct default.
* `ParsedManifest` gains a `metric: DistanceMetric` field parsed from byte [19].
* `boot()` restores `self.options.metric = manifest.metric` so every
`open()` / `open_readonly()` correctly reflects the stored metric.
Before/after recall
-------------------
| Path | Before fix | After fix |
|---------------------|-----------|-----------|
| COW cosine recall@10 | ≈ 0.10 | 1.0000 |
| COW L2 recall@10 | 1.0000 | 1.0000 |
Regression test
---------------
New test `cow_ann_recall_vs_exact_cosine` in `cow_ann_recall.rs` mirrors the
existing L2 test with `metric = DistanceMetric::Cosine` and cosine ground
truth; asserts recall@10 ≥ 0.95. The L2 test (`cow_ann_recall_vs_exact`)
is unchanged and still passes.
Follow-on
---------
A new `@ruvector/rvf-node` native binding build is needed to ship this fix
to the Node.js surface used by agenticow. Until then, agenticow's existing
L2-normalize workaround (driving the engine with pre-normalized vectors so
L2 and cosine rankings agree) remains correct and safe to keep.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* style: cargo fmt for cosine-metric persistence fix
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* fix(rvf, ruvector): accept dimension/dimensions alias, fix MCP rvf_create, add embed-text stdin (#641)
Issue 1 — store-creation options contract:
- @ruvector/rvf: RvfDatabase.create now accepts `dimension` (singular) as
an alias for the documented `dimensions`; a missing/invalid value fails
fast with an error naming the public `dimensions` option instead of the
native `dimension` field that callers must not use.
- ruvector MCP `rvf_create`: pass `dimensions` (plural) to the SDK so the
tool works at all; accept both spellings in the input schema; only emit
the "Install @ruvector/rvf" hint when the package is actually missing.
Issue 2 — raw text on argv:
- `ruvector embed text` now reads from stdin ("-" sentinel or --stdin)
or from a file (--input-file <path>), keeping sensitive text out of
the process table and audit logs.
Adds tests/test-create-options.js covering the alias, precedence, and
error paths (mock native handle, no N-API addon needed).
Fixes#641
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_017sXWL4ox5bhC86FYwJpmyK
---------
Co-authored-by: ruvnet <ruvnet@gmail.com>
Three backward-compatible TrainingPipeline improvements (minor bump 2.5.7 → 2.6.0):
1. Checkpoint metadata + load validation. saveCheckpoint(path) now writes a
v2 envelope carrying adapter geometry {config:{inputDim,outputDim,rank}} and
{pipelineConfig:{learningRate,batchSize}}. loadCheckpoint() rejects a v2 file
whose geometry does not match the current adapter (returns false, adapter
untouched) instead of silently restoring mis-shaped weights. v1 files carry
no geometry and still load unchanged (back-compat). Adds LoraAdapter
getInputDim()/getOutputDim() to expose geometry that is not part of LoRAConfig.
2. True resume via explicit resumeFrom(path): boolean. It loads the checkpoint
(same v2 shape validation) AND primes the pipeline so the next train()
continues from the restored epoch/step — running the remaining epochs of
config.epochs and fast-forwarding the LR scheduler to the restored step, with
metrics history preserved. Chosen over mutating train() implicitly so that a
plain loadCheckpoint()+train() stays "from scratch" and a no-resume train()
is byte-for-byte the same run as 2.5.7 (same reset, scheduler, result shape).
3. Best-checkpoint retention via config keepBestCheckpoint?: string. When set,
the pipeline writes the current state (same v2 envelope) each time validation
loss improves, so the best-val model survives later degradation. No-op when
validation never runs (validationSplit 0 or no val batches).
Tests: extend test/checkpoint.test.js (v2 round-trip, dim + rank mismatch
rejection, v1 back-compat) and add test/resume.test.js (resume continues to
config total epochs, weights restored not re-initialized, mismatch refuses to
arm resume, keepBestCheckpoint writes on improvement and is a no-op without
validation, plain train() result shape unchanged). Full suite: 107 pass / 3
fail; the 3 failures are the pre-existing native-binding tests in basic.test.js
(query/route/memory), unchanged by this work.
saveCheckpoint() was private, ignored any argument, returned undefined,
and never wrote a file — checkpoints were in-memory only and died with
the process, while callers passing a path got 0 bytes silently
(reported downstream: ruvnet/ruflo#2549).
- saveCheckpoint(path?) is now public: always records the in-memory
checkpoint (training-loop behavior unchanged); with a path it writes
a versioned JSON envelope (parent dirs created) and returns
{index, epoch, step, loss, path, bytes} instead of undefined.
- loadCheckpoint(indexOrPath) loads by in-memory index (back-compat)
or from a checkpoint file, rejecting foreign/malformed JSON.
- 5 tests: metadata shape, non-empty file + byte accounting, disk
round-trip of weights, index back-compat, missing/foreign rejection.
Test suite: 97/101 pass — the 4 failures (query/route/memory native-
binding tests) are pre-existing in a tree without build:native and fail
identically at baseline.
The publish workflow's 'node --test tests/*.test.mjs' contract test failed
(blocking automated publish of 0.2.32 and 0.2.33): the skip guard only wrapped
the BASE initOnnxEmbedder(), but the optimized embedder lazily loads its own
.wasm on first embed() — which throws 'Unknown file extension .wasm' under
Node ESM in restricted CI. That throw escaped the guard and failed the test
instead of skipping, despite the subtest being named '... skipped if model
unavailable'.
Wrap the optimized embed in try/catch: skip on model-load failure, but
re-throw assert.AssertionError so genuine contract regressions (#523 FP16-log,
readiness gates) still fail. Locally (model present) all assertions still run.
* fix(ruvector-npm): accurate MCP tool count + reconcile tool lists (0.2.33)
The package advertised 103 MCP tools but the server registers 97, and the
README listed 12 tools (brain_agi_*, midstream_*) that don't exist. Three
independent sources had also drifted: server TOOLS array (97), the cli
'mcp tools' display list (91, missing the 6 decompile_* tools), and the
README count (103).
- README: 103 → 97; drop the 12 phantom tool bullets; point to
'npx ruvector mcp tools' as the authoritative list
- cli.js 'mcp tools': add the missing 'decompile' group (6 tools) →
now lists 97, matching the server
- mcp-server.js: report version from package.json instead of the
hardcoded (and stale) '0.2.30'
- bump 0.2.32 → 0.2.33
All three sources now reconcile to 97. Full test suite passes
(73 + 7 + 2 + 8 + 2 across the publish-gate files).
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(ci): commit optimizer module, ignore RUSTSEC-2026-0190, ship src/optimizer
Root-causes three pre-existing CI failures on the ruvector npm package:
1. Optimizer test (Cannot find module src/optimizer/index.js): the module
was never committed because .gitignore's bare 'index.js' rule (for napi
artifacts) silently ignored this hand-written module. Add a negation
exception (as already done for @ruvector/sona) and commit index.js —
8 profiles, detectTaskType, applyProfile. Test: 135/135 pass.
2. cargo deny (advisories FAILED): RUSTSEC-2026-0190 (unsoundness in
anyhow's Error::downcast_mut, anyhow 1.0.102, single version in tree).
Added to deny.toml ignore list with justification + re-review date,
consistent with the existing unsoundness exceptions — we never downcast
anyhow::Error to a mismatched mutable type.
3. src/optimizer/ now ships (added to package.json files), so the published
package's 'optimize' command works instead of graceful-degrading.
Pre-existing failures NOT addressed here (separate infra work; main is also
red): Functional smoke / Recall quality / Learning check fail because the
npm CI installs with --no-optional, so @ruvector/rvf + the native HNSW addon
are absent and the engine runs degraded. dependency-review is already
continue-on-error (awaiting a repo Dependency-Graph settings flip).
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(rvf): ANN search across COW branches (dual-graph merge)
Stack on feat/queryable-cow-branches (PR #617). That PR added branch(),
CowEngine, MembershipFilter, and parent_path — but the HNSW/ANN paths were
disabled for COW children (fell back to O(N) exact scan of child's own slab
only, missing parent vectors entirely).
This commit adds sub-linear ANN across the full parent ∪ child-edits view:
Design — dual-graph query + merge (LSM-ANN pattern):
1. Child arm : query child's own HNSW (exact scan when child < 1 024 vectors)
2. Parent arm : lazily open parent store read-only, cache in parent_store
Mutex<Option<Box<RvfStore>>>; query parent's HNSW (built once, no rebuild
per branch)
3. Over-fetch : k' = k × 4 from each arm to absorb tombstones / overrides
4. Merge : child distances override parent for same ID; IDs removed from
membership_filter (tombstoned via child delete) are excluded; re-rank by
distance; return top-k
5. Chained COW : parent.query() walks parent's own HNSW; lineage works
transitively
Key changes to rvf-runtime/src/store.rs:
- Add parent_store: Mutex<Option<Box<RvfStore>>> field (all constructors)
- Fix query_routed early-return: COW children with 0 child-side vectors
must not bail before parent read-through
- New cow_ann_eligible() guard
- New query_via_index_cow() — the dual-graph merge (replaces O(N) fallback)
- New cow_exact_parent_scan() — exact parent read-through for the exact path;
makes query_exact the correct ground-truth for recall comparison
- Update query_exact to call cow_exact_parent_scan for COW children
- Update delete() to tombstone parent IDs from membership_filter so
child-side deletion of inherited parent vectors is correctly reflected
New integration tests (cow_ann_recall.rs, 4 tests):
- cow_ann_recall_vs_exact : 1 200-vector base, branch, add/override/delete;
ANN recall@10 vs exact ground truth — measured 1.0000 (>= 0.95 contract)
- cow_ann_override_correctness: child override returns child distance, not
parent's stale entry
- cow_ann_tombstone_absent : tombstoned ID absent from ANN and exact results
- cow_branch_size_independence: child file (162 bytes) stays << parent
(163 803 bytes) after queries — no HNSW rebuild in child file
Approximation: dual-graph merge is slightly approximate (sub-linear in parent
size, not exact). Measured recall@10 = 1.00 at ef_search=300 on 1 200-vector
L2/32-dim dataset with C=4 over-fetch. force_exact=true always provides
ground truth via cow_exact_parent_scan.
Cost: 2 HNSW queries (child + parent), flat in parent size. Parent HNSW built
once on first COW query then cached. Child HNSW only when child has >= 1 024
vectors. RaBitQ-across-COW deferred (exact fallback used until then).
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
* fix(diskann): make search_returns_self_as_nearest non-flaky
The test used max_degree=16 / beam=16 on a 128-node graph whose initial
topology comes from thread_rng() (VamanaGraph::init_random_graph). With
small M and a random graph, point 5 can end up outside the 16-candidate
window reachable from the medoid in some seedings — causing an intermittent
CI failure unrelated to the caller's changes.
Fix: bump max_degree to 32 and build_beam to 64 (matching production
defaults) so the graph is dense enough to guarantee connectivity on 128
nodes; use n = v.len() as the search beam so the test validates the
"self is retrievable" property exhaustively rather than testing ANN
efficiency (which is covered by other tests).
Fixes pre-existing flaky failure observed in Tests (vector-index) CI job.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_019rVRYrRDKyxYK18kuVrDSf
---------
Co-authored-by: ruvnet <ruvnet@gmail.com>
Wrap the agentic-time layer of the dependency-free `emergent-time` crate in a
tiny wasm-bindgen surface for the browser, edge, and Node.
- crates/emergent-time-wasm: standalone cdylib (workspace-excluded so it carries
its own opt-level="z" / lto / strip / panic=abort release profile and dlmalloc
global allocator, mirroring crates/rvf/rvf-wasm). Hand-rolled getters, no serde,
to keep the wasm tiny.
- SDK surface: AgenticClock (tick → explainable Tick{class,reason,deltaTime,
per-channel}; cumulativeTime, ATI, 7-state health), StateDelta, Tick,
TickClassJs, AgentHealthJs, WindowedDeltaClock + PageHinkleyDetector
change-point detectors, LearnedWeights inference, version().
- Physics core (Wheeler-DeWitt / Page-Wootters / entropic / thermal / Structural
Proper Time) deliberately not wrapped: dense matrices don't serialize cheaply
over the JS boundary and would bloat the wasm. Documented in the README.
- npm/packages/emergent-time: package.json (@ruvector/emergent-time@0.1.0, ESM,
main/module/types → pkg, files include pkg + README, publishConfig public),
detailed README, build.sh pipeline (cargo @1.89 → wasm-bindgen --target web →
wasm-opt -Oz with bulk-memory/nontrapping-float-to-int enabled), and the built
pkg/ (wasm + JS glue + .d.ts).
Validation: wasm raw 62475B / opt 55009B (wasm-tools VALID); Node ESM smoke test
passes end-to-end (AgenticClock Healthy→Drifting→NeedsReplan→Collapsing→
NeedsHumanReview, cumulativeTime 19.36, both detectors fire at the planted jump);
tsc --noEmit --strict on a usage example against the shipped .d.ts exits 0;
npm pack --dry-run lists README.md + .wasm + .js + .d.ts.
Honest scope (mirrors ADR-251): the agentic clock is a diagnostic signal; it does
not establish an early-warning lead over a fair baseline on real traces. Both
fair baselines (windowed z-score, Page-Hinkley) are exported.
Co-authored-by: ruv <ruvnet@users.noreply.github.com>
fix(ruvector): make createEmbedder() init WASM correctly under Node (closes#523)
Root cause: bundler-target wasm-pack wrapper starts with a bare
import that Node ESM cannot resolve, and exports no
init function — both caused by the bundler target generating a wrapper
for bundlers, not for plain Node.
Fix: mirror the proven embed-worker.mjs init path (pathToFileURL +
WebAssembly.instantiate + __wbg_set_wasm + __wbindgen_start). Vectors
are identical to the worker path by construction. Browser/bundler
default() path is preserved unchanged in the else branch.
