ruvector/docs/research/nightly
rUv a4f9991d9d
feat: add k-scoped adaptive ANN calibration (#718)
* research: add nightly survey for adaptive-recall-ann

Identifies adaptive recall-targeted ANN as the 2026-07-23 nightly topic.
Connects vector search, agent memory, edge AI, MCP tool latency SLAs,
and ruFlo workflow recall budgets. No prior nightly covered this angle.

* feat: add ruvector-adaptive-ann Rust proof of concept

Implements RecallTargetedSearch trait with three variants:
- FixedEfSearch (baseline): constant ef=64, ignore recall target
- BinarySearchCalibrated: binary-search ef per query with ground truth
- TableCalibratedSearch: O(1) ef lookup from offline calibration table

Core insight: calibration queries must match production query distribution.
CalibrationTable is a monotone ef→recall mapping from 50-100 held-out queries.

Benchmark: N=3000×D=64, recall_target=0.90
- FixedEf(64): 0.778 recall, 9,497 QPS (misses target)
- BinarySearch: 0.902 recall, 738 QPS (oracle, 13x slower)
- TableCalibrated: 0.940 recall, 4,390 QPS (exceeds target, O(1) ef)

* test: add 7 integration tests for ruvector-adaptive-ann

- beam search at ef=N achieves near-perfect recall
- recall is monotone in ef
- FixedEf(128) achieves minimum recall threshold
- CalibrationTable returns valid ef
- TableCalibratedSearch achieves recall within distribution-mismatch tolerance
- BinarySearchCalibrated achieves per-query target on 12/15 queries
- effective_ef_for_target returns Some for Table, None for Fixed

All 7 tests pass.

* docs: add ADR-272 for adaptive-recall-ann

Documents the calibration table approach, distribution matching constraint,
three implementation variants, benchmark evidence, failure modes, security
considerations, and migration path for adopting recall-targeted search.

ADR-272 status: Proposed.

* bench: capture adaptive-recall-ann benchmark results

cargo run --release -p ruvector-adaptive-ann --bin benchmark
x86_64 Linux, release build, N=3000 D=64 300 queries

FixedEf(64): recall=0.778, mean=105.3µs, QPS=9497
BinarySearch: recall=0.902, mean=1355µs, QPS=738
TableCalibrated: recall=0.940, mean=227.8µs, QPS=4390
All acceptance tests PASSED.

* fix adaptive ANN calibration scope

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-27 09:57:49 -07:00
..
2026-04-23-rabitq feat(rabitq): add RaBitQ rotation-based 1-bit quantization crate (ADR-154) 2026-04-23 07:56:23 +00:00
2026-04-26-acorn-filtered-hnsw research(nightly): ACORN — predicate-agnostic filtered HNSW (#391) 2026-04-27 00:29:37 -04:00
2026-05-12-rairs-ivf research(nightly): rairs-ivf — RAIRS IVF, ruvector's first Inverted File Index (ADR-193) (#459) 2026-05-12 09:47:19 -04:00
2026-05-21-gnn-rerank feat(gnn-rerank): productionize #479 — +10.4pp recall, CI-guarded, hardened, optimized (#582) 2026-06-17 20:18:45 -04:00
2026-05-24-proof-gated-writes feat(proof-gate): productionize #506 — tamper-evident vector writes (Merkle/hash-chain WAL) (#584) 2026-06-17 20:19:47 -04:00
2026-06-13-temporal-coherence-agent-memory research(nightly): temporal-coherence-agent-memory (#564) 2026-06-18 23:28:38 -04:00
2026-06-14-agent-memory-compaction docs: add nightly research README and SEO gist for agent-memory-compaction 2026-06-14 07:22:20 +00:00
2026-06-15-multi-vector-maxsim research: nightly 2026-06-15 — multi-vector MaxSim late interaction (#569) 2026-06-18 23:31:14 -04:00
2026-06-16-coherence-hnsw-search research: add nightly coherence-gated HNSW search PoC (#571) 2026-06-18 23:29:07 -04:00
2026-06-17-hybrid-sparse-dense research(nightly): hybrid sparse-dense search — BM25 + ANN with RRF and RSF (ADR-256) (#576) 2026-06-18 23:28:08 -04:00
2026-06-18-hnsw-delete-repair research: add nightly survey for hnsw-delete-repair 2026-06-18 07:21:39 +00:00
2026-06-19-lsm-ann feat: LSM-ANN write-optimised streaming vector index (ADR-264) (#591) 2026-06-21 18:55:51 -04:00
2026-06-20-pq-adc-search research: add nightly survey for pq-adc-search (#593) 2026-06-21 18:56:06 -04:00
2026-06-21-matryoshka-coarse-fine research(nightly): matryoshka coarse-to-fine ANN search (ADR-264) (#594) 2026-06-21 18:55:59 -04:00
2026-06-24-spann-partition-spill research(nightly): spann-partition-spill — boundary-safe ANN in Rust (#602) 2026-06-25 14:03:59 -04:00
2026-06-25-capability-gated-ann research(nightly): capability-gated-ann — per-vector read access control in ANN search (#604) 2026-06-25 14:05:34 -04:00
2026-07-23-adaptive-recall-ann feat: add k-scoped adaptive ANN calibration (#718) 2026-07-27 09:57:49 -07:00
2026-07-24-recall-bounded-ann feat: add threshold-driven ANN with empirical recall (#719) 2026-07-27 09:57:00 -07:00
2026-07-25-bounded-rag-mincut feat: add bounded RAG graph retrieval research (ADR-272) (#720) 2026-07-27 09:55:59 -07:00
2026-07-26-diverse-beam-ann feat: add measured diverse beam ANN research (ADR-272) (#723) 2026-07-27 09:55:22 -07:00
2026-07-27-speculative-ann-search feat: add audited speculative ANN search (ADR-272) (#725) 2026-07-27 09:54:31 -07:00