fix recall-bounded ANN ids and search budgets

This commit is contained in:
ruvnet 2026-07-27 12:47:03 -04:00
parent ef121c8f4d
commit 6b0df5e1a6
5 changed files with 177 additions and 64 deletions

View file

@ -0,0 +1,16 @@
# ruvector-recall-bounded
Research implementations of threshold-driven similarity search for RuVector.
The crate compares an exact linear scan with two bounded graph-search
heuristics and reports empirical recall against the exact baseline.
The approximate variants do not provide a formal recall guarantee. Their
expansion parameters must be calibrated and audited on representative data.
```sh
cargo test -p ruvector-recall-bounded
cargo run --release -p ruvector-recall-bounded --bin benchmark
```
See `docs/adr/ADR-272-recall-bounded-ann.md` and the associated nightly report
for methodology and limitations.

View file

@ -1,13 +1,13 @@
//! Recall-bounded approximate nearest-neighbour search.
//!
//! Standard ANN search returns a fixed k results regardless of quality.
//! Recall-bounded search returns every vector whose similarity to the query
//! exceeds a caller-supplied threshold θ ∈ (0, 1] with a measurable recall
//! guarantee. Three variants are provided and benchmarked:
//! Recall-bounded search returns vectors whose similarity to the query exceeds
//! a caller-supplied threshold θ with measured empirical recall. Three
//! variants are provided and benchmarked:
//!
//! 1. `LinearScan` — exact brute-force baseline (O(n·d))
//! 2. `HnswBeamSearch` — HNSW-style greedy graph walk with adaptive ef
//! 3. `ThresholdBeam` — beam search with early-stop when beam minimum ≥ θ
//! 3. `ThresholdBeam` — graph search with a fixed expansion budget
use std::collections::{BinaryHeap, HashSet};
@ -43,7 +43,7 @@ pub trait RecallBoundedIndex {
#[inline]
pub fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
debug_assert_eq!(a.len(), b.len());
assert_eq!(a.len(), b.len(), "vector dimension mismatch");
let mut dot = 0.0f32;
let mut na = 0.0f32;
let mut nb = 0.0f32;
@ -120,10 +120,24 @@ impl Default for LinearScan {
impl RecallBoundedIndex for LinearScan {
fn insert(&mut self, entry: Entry) {
if let Some(first) = self.entries.first() {
assert_eq!(
entry.vec.len(),
first.vec.len(),
"vector dimension mismatch"
);
}
self.entries.push(entry);
}
fn search(&self, query: &[f32], threshold: f32) -> Vec<Hit> {
assert!(
(-1.0..=1.0).contains(&threshold),
"threshold must be in [-1, 1]"
);
if let Some(first) = self.entries.first() {
assert_eq!(query.len(), first.vec.len(), "query dimension mismatch");
}
self.entries
.iter()
.filter_map(|e| {
@ -164,6 +178,11 @@ pub struct HnswBeamSearch {
impl HnswBeamSearch {
pub fn new(m: usize, ef_search_base: usize) -> Self {
assert!(m > 0, "m must be greater than zero");
assert!(
ef_search_base > 0,
"ef_search_base must be greater than zero"
);
Self {
entries: Vec::new(),
graph: Vec::new(),
@ -175,7 +194,7 @@ impl HnswBeamSearch {
fn build_neighbours(&self, id: usize) -> Vec<u32> {
let q = &self.entries[id].vec;
let mut pairs: Vec<(u32, u32)> = self
let mut pairs: Vec<(i32, u32)> = self
.entries
.iter()
.enumerate()
@ -183,17 +202,24 @@ impl HnswBeamSearch {
.map(|(j, e)| {
let s = cosine_similarity(q, &e.vec);
// store as fixed-point to use BinaryHeap without Ord on f32
let fp = (s * 1_000_000.0) as u32;
let fp = (s * 1_000_000.0) as i32;
(fp, j as u32)
})
.collect();
pairs.sort_unstable_by(|a, b| b.0.cmp(&a.0));
pairs.sort_unstable_by_key(|pair| std::cmp::Reverse(pair.0));
pairs.iter().take(self.m).map(|(_, id)| *id).collect()
}
}
impl RecallBoundedIndex for HnswBeamSearch {
fn insert(&mut self, entry: Entry) {
if let Some(first) = self.entries.first() {
assert_eq!(
entry.vec.len(),
first.vec.len(),
"vector dimension mismatch"
);
}
let id = self.entries.len();
self.entries.push(entry);
let neighbours = self.build_neighbours(id);
@ -210,7 +236,7 @@ impl RecallBoundedIndex for HnswBeamSearch {
nbr_graph.sort_unstable_by(|&a, &b| {
let sa = cosine_similarity(q, &entries[a as usize].vec);
let sb = cosine_similarity(q, &entries[b as usize].vec);
sb.partial_cmp(&sa).unwrap()
sb.total_cmp(&sa)
});
nbr_graph.truncate(self.m);
}
@ -219,16 +245,31 @@ impl RecallBoundedIndex for HnswBeamSearch {
}
fn search(&self, query: &[f32], threshold: f32) -> Vec<Hit> {
assert!(
(-1.0..=1.0).contains(&threshold),
"threshold must be in [-1, 1]"
);
if self.entries.is_empty() {
return Vec::new();
}
assert_eq!(
query.len(),
self.entries[0].vec.len(),
"query dimension mismatch"
);
let mut ef = self.ef_search_base;
let mut previous_ids: Option<HashSet<u32>> = None;
loop {
let hits = self.greedy_search(query, threshold, ef);
// If we found results or reached ef ceiling, return what we have.
if !hits.is_empty() || ef >= self.ef_search_max {
let ids: HashSet<u32> = hits.iter().map(|hit| hit.id).collect();
if ef >= self.ef_search_max
|| previous_ids
.as_ref()
.is_some_and(|previous| *previous == ids)
{
return hits;
}
previous_ids = Some(ids);
ef = (ef * 2).min(self.ef_search_max);
}
}
@ -257,41 +298,43 @@ impl HnswBeamSearch {
let mut candidates: BinaryHeap<(i32, u32)> = BinaryHeap::new();
let mut visited: HashSet<u32> = HashSet::new();
let mut results: Vec<Hit> = Vec::new();
let mut expanded = 0usize;
candidates.push((ep_fp, 0));
visited.insert(0);
while let Some((sim_fp, cur_id)) = candidates.pop() {
if expanded >= ef {
break;
}
expanded += 1;
let sim = sim_fp as f32 / 1_000_000.0;
if sim >= threshold {
results.push(Hit {
id: cur_id,
id: self.entries[cur_id as usize].id,
similarity: sim,
});
}
// Expand neighbours if within ef budget
if candidates.len() + results.len() < ef {
for &nb in &self.graph[cur_id as usize] {
if visited.insert(nb) {
let nb_sim = cosine_similarity(query, &self.entries[nb as usize].vec);
let nb_fp = (nb_sim * 1_000_000.0) as i32;
candidates.push((nb_fp, nb));
}
for &nb in &self.graph[cur_id as usize] {
if visited.insert(nb) {
let nb_sim = cosine_similarity(query, &self.entries[nb as usize].vec);
let nb_fp = (nb_sim * 1_000_000.0) as i32;
candidates.push((nb_fp, nb));
}
}
}
results.sort_by(|a, b| b.similarity.total_cmp(&a.similarity));
results
}
}
// ─── Variant 3: ThresholdBeam ─────────────────────────────────────────────────
/// Beam search with early stopping.
/// Graph search with a fixed expansion budget.
///
/// Maintains a beam of the `beam_width` most promising unexplored nodes.
/// At each step, expands all nodes in the beam and stops when the minimum
/// similarity in the beam already falls below the threshold (nothing better
/// can be found by following lower-similarity edges).
/// Expands the most promising unexplored nodes and stops after at most
/// `beam_width * 4` expansions. This is an empirical budget, not a recall
/// guarantee or a valid similarity lower bound on unseen graph nodes.
pub struct ThresholdBeam {
entries: Vec<Entry>,
graph: Vec<Vec<u32>>,
@ -301,6 +344,8 @@ pub struct ThresholdBeam {
impl ThresholdBeam {
pub fn new(m: usize, beam_width: usize) -> Self {
assert!(m > 0, "m must be greater than zero");
assert!(beam_width > 0, "beam_width must be greater than zero");
Self {
entries: Vec::new(),
graph: Vec::new(),
@ -311,23 +356,30 @@ impl ThresholdBeam {
fn build_neighbours(&self, id: usize) -> Vec<u32> {
let q = &self.entries[id].vec;
let mut pairs: Vec<(u32, u32)> = self
let mut pairs: Vec<(i32, u32)> = self
.entries
.iter()
.enumerate()
.filter(|(j, _)| *j != id)
.map(|(j, e)| {
let s = cosine_similarity(q, &e.vec);
((s * 1_000_000.0) as u32, j as u32)
((s * 1_000_000.0) as i32, j as u32)
})
.collect();
pairs.sort_unstable_by(|a, b| b.0.cmp(&a.0));
pairs.sort_unstable_by_key(|pair| std::cmp::Reverse(pair.0));
pairs.iter().take(self.m).map(|(_, id)| *id).collect()
}
}
impl RecallBoundedIndex for ThresholdBeam {
fn insert(&mut self, entry: Entry) {
if let Some(first) = self.entries.first() {
assert_eq!(
entry.vec.len(),
first.vec.len(),
"vector dimension mismatch"
);
}
let id = self.entries.len();
self.entries.push(entry);
let nb = self.build_neighbours(id);
@ -346,7 +398,7 @@ impl RecallBoundedIndex for ThresholdBeam {
sorted.sort_unstable_by(|&a, &b| {
let sa = cosine_similarity(&q_vec, &entries[a as usize].vec);
let sb = cosine_similarity(&q_vec, &entries[b as usize].vec);
sb.partial_cmp(&sa).unwrap()
sb.total_cmp(&sa)
});
sorted.truncate(m);
*nbr_graph = sorted;
@ -356,9 +408,18 @@ impl RecallBoundedIndex for ThresholdBeam {
}
fn search(&self, query: &[f32], threshold: f32) -> Vec<Hit> {
assert!(
(-1.0..=1.0).contains(&threshold),
"threshold must be in [-1, 1]"
);
if self.entries.is_empty() {
return Vec::new();
}
assert_eq!(
query.len(),
self.entries[0].vec.len(),
"query dimension mismatch"
);
// Greedy descent: always follow the most similar unvisited neighbour.
// Early stop when the best unvisited candidate is below (threshold * 0.5)
@ -367,17 +428,23 @@ impl RecallBoundedIndex for ThresholdBeam {
let mut candidates: BinaryHeap<(i32, u32)> = BinaryHeap::new();
let mut visited: HashSet<u32> = HashSet::new();
let mut results: Vec<Hit> = Vec::new();
let mut expanded = 0usize;
let expansion_budget = self.beam_width.saturating_mul(4);
let ep_sim = cosine_similarity(query, &self.entries[0].vec);
candidates.push(((ep_sim * 1_000_000.0) as i32, 0));
visited.insert(0);
while let Some((sim_fp, cur_id)) = candidates.pop() {
if expanded >= expansion_budget {
break;
}
expanded += 1;
let sim = sim_fp as f32 / 1_000_000.0;
if sim >= threshold {
results.push(Hit {
id: cur_id,
id: self.entries[cur_id as usize].id,
similarity: sim,
});
}
@ -390,16 +457,8 @@ impl RecallBoundedIndex for ThresholdBeam {
candidates.push(((nb_sim * 1_000_000.0) as i32, nb));
}
}
// Peek at the best remaining candidate; if it's very low and we've
// visited a generous budget, stop early.
if let Some(&(best_fp, _)) = candidates.peek() {
let best = best_fp as f32 / 1_000_000.0;
if best < threshold * 0.5 && visited.len() > self.beam_width * 4 {
break;
}
}
}
results.sort_by(|a, b| b.similarity.total_cmp(&a.similarity));
results
}
@ -541,4 +600,35 @@ mod tests {
assert_eq!(hits[0].id, 0);
let _ = dim;
}
#[test]
fn approximate_indexes_return_opaque_ids() {
let entries: Vec<Entry> = gen_dataset(80, 16, 44)
.into_iter()
.enumerate()
.map(|(position, mut entry)| {
entry.id = 10_000 + position as u32 * 7;
entry
})
.collect();
let query = entries[12].vec.clone();
let expected = entries[12].id;
let hnsw = build_index(HnswBeamSearch::new(12, 32), &entries);
assert!(
hnsw.search(&query, 0.99)
.iter()
.any(|hit| hit.id == expected),
"HNSW search leaked graph positions instead of opaque ids"
);
let threshold = build_index(ThresholdBeam::new(12, 40), &entries);
assert!(
threshold
.search(&query, 0.99)
.iter()
.any(|hit| hit.id == expected),
"threshold search leaked graph positions instead of opaque ids"
);
}
}

View file

@ -21,7 +21,8 @@ The difference is significant:
- **Top-k search** is bounded in output size but unbounded in quality. You always
get exactly k results, some of which may be irrelevant.
- **Recall-bounded search** is bounded in quality but variable in output size. You
get every relevant vector (within a statistical guarantee) and nothing more.
request vectors above a similarity threshold and audit empirical recall
against an exact baseline.
This matters for:
1. **Agent memory** — "fetch every memory about OAuth" must not omit relevant traces.
@ -46,7 +47,7 @@ Add `crates/ruvector-recall-bounded` to the workspace implementing:
2. Three concrete variants benchmarked under identical conditions:
- `LinearScan` — exact O(n·d) brute-force baseline.
- `HnswBeamSearch` — single-layer proximity graph with adaptive ef expansion.
- `ThresholdBeam`beam search that early-stops when the beam minimum falls below θ.
- `ThresholdBeam`graph search with a fixed node-expansion budget.
3. A deterministic `Lcg`-seeded dataset generator (no external deps).
4. A `benchmark` binary that reports mean/p50/p95 latency, throughput, memory, hit count,
and measured recall against the linear-scan ground truth.

View file

@ -1,6 +1,6 @@
# Recall-Bounded Approximate Nearest-Neighbour Search in Rust
**150-char summary:** Quality-first ANN that returns all vectors above a cosine threshold, not a fixed k — measured recall vs. exact baseline, zero external deps, edge-deployable.
**Summary:** Threshold-driven ANN research with empirical recall measured against an exact baseline; approximate variants can miss qualifying vectors.
---
@ -18,7 +18,7 @@ inside `crates/ruvector-recall-bounded`:
|---------|----------|-------------------|----------------|
| `LinearScan` | exact brute force | 1.00 | O(1) per insert |
| `HnswBeamSearch` | graph walk + adaptive ef | ≥ 0.80 | O(n) per insert (PoC) |
| `ThresholdBeam` | beam search + early stop | ≥ 0.65 | O(n) per insert (PoC) |
| `ThresholdBeam` | graph search + fixed expansion budget | ≥ 0.65 in acceptance data | O(n) per insert (PoC) |
All numbers come from `cargo run --release -p ruvector-recall-bounded --bin benchmark`
on an x86_64 Linux host.
@ -148,14 +148,13 @@ n < 10 000 or as a ground-truth reference.
**HnswBeamSearch** — builds a single-layer proximity graph (M neighbours per node).
At query time, performs greedy descent starting from a fixed entry point, expanding
the candidate set. When too few results exceed θ, ef_search doubles up to a ceiling.
This adaptive expansion is the key mechanism: rather than choosing ef_search
statically, it grows until the recall plateau or the ef ceiling is reached.
the candidate set. The search doubles ef until the returned opaque-ID set
stabilises or reaches a ceiling. This remains a heuristic: rather than choosing ef_search
statically, it grows until the result-set plateau or the ef ceiling is reached.
**ThresholdBeam** — maintains a beam of `beam_width` candidates. Expands the beam
at each step by following neighbour edges. Early-stops when the highest-similarity
candidate in the beam falls below θ — because greedy descent will only find
lower-similarity nodes from that point.
at each step by following neighbour edges. Stops after a configured expansion
budget; graph similarity does not provide a safe lower bound on unseen nodes.
---
@ -167,7 +166,7 @@ graph TD
Dispatcher --> |variant 1| LS[LinearScan<br/>O(n·d) exact scan]
Dispatcher --> |variant 2| HB[HnswBeamSearch<br/>graph walk + adaptive ef]
Dispatcher --> |variant 3| TB[ThresholdBeam<br/>beam walk + early stop]
Dispatcher --> |variant 3| TB[ThresholdBeam<br/>fixed expansion budget]
LS --> GT[Ground truth: all hits above θ]
HB --> AH1[Approximate hits]
@ -198,13 +197,12 @@ Fixed ef_search requires the caller to know the result cardinality in advance
the information recall-bounded search avoids. Adaptive ef starts conservative and
expands only when the current ef produces too few qualifying results.
### Why beam early-stop?
### Why a fixed expansion budget?
In a proximity graph, greedy descent reaches successively lower-similarity nodes.
Once the best candidate in the beam is below θ, all future candidates will also be
below θ (by the greedy construction). Early-stopping here avoids wasted expansion.
This is not exact — the graph may have "bridges" to high-similarity clusters that
require traversing a low-similarity node — but it is a useful heuristic in practice.
The budget makes the cost/recall trade-off explicit. A low-scoring frontier
does not prove that unseen graph nodes are also below θ.
The graph may have bridges to high-similarity clusters that require traversing
a low-similarity node, so workload-specific recall audits remain necessary.
### LCG dataset generator

View file

@ -1,8 +1,8 @@
# ruvector 2026: Recall-Bounded ANN Search for High-Performance Rust Agent Memory Retrieval
**150-char SEO summary:** Quality-first ANN that returns every vector above a cosine threshold with measured recall — Rust, zero deps, edge-deployable, agent memory ready.
**Summary:** Threshold-driven ANN research with empirical recall measured against an exact baseline; approximate graph variants can miss qualifying vectors.
**One sentence:** RuVector now exposes a `search_above_threshold(query, θ)` API that retrieves every qualifying vector with measurable recall guarantees — not an arbitrary top-k.
**One sentence:** This PoC exposes a threshold-search API and measures approximate results against an exact scan instead of assuming completeness.
- GitHub: https://github.com/ruvnet/ruvector
- Research branch: `research/nightly/2026-07-24-recall-bounded-ann`
@ -17,7 +17,10 @@ This API is a legacy of image retrieval and recommendation systems, where a rank
An agent fetching its memory to answer *"what do I know about OAuth token refresh?"* does not know k. It knows its confidence floor: retrieve everything with cosine similarity ≥ 0.75. Missing a relevant memory causes a factual error in the agent's reasoning. Returning 10 arbitrary results — some below the confidence floor, some above — forces the agent to do post-filtering that the vector index should be doing.
**Recall-bounded search** is the primitive that fixes this. Instead of `search(query, k) → Vec<Hit>`, the API is `search(query, threshold) → Vec<Hit>` — return every vector whose similarity to the query exceeds θ. The cardinality is determined by the data, not the caller's guess.
**Threshold search** changes the request from `search(query, k)` to
`search(query, threshold)`. Exact scan returns every qualifying vector; the
graph variants return a data-dependent approximate subset whose recall must be
measured.
Current vector databases handle this only as post-filtering: run top-k, then discard results below θ. If k is too small, you miss qualifying vectors. If k is too large, you waste compute. There is no principled way to set k without knowing the answer in advance.
@ -36,7 +39,7 @@ This nightly research implements three Rust variants of recall-bounded search in
| `RecallBoundedIndex` trait | `search(query, threshold) → Vec<Hit>` | Quality-first API contract | Implemented in PoC |
| `LinearScan` variant | Exact O(n·d) brute force | Ground truth oracle | Implemented, measured |
| `HnswBeamSearch` variant | Graph walk + adaptive ef expansion | ~87% recall, faster than exact at scale | Implemented, measured |
| `ThresholdBeam` variant | Greedy descent + early stop | Perfect recall, faster than HnswBeam | Implemented, measured |
| `ThresholdBeam` variant | Greedy descent + fixed expansion budget | Empirical recall/cost trade-off | Implemented, measured |
| `recall(found, gt)` utility | Measures |found ∩ ground_truth| / |ground_truth| | Honest quality accounting | Implemented |
| `Lcg` dataset generator | Deterministic seeded unit vectors | Reproducible benchmarks, zero deps | Implemented |
| Acceptance gate | All variants must hit recall ≥ 0.80 | Prevents shipping low-quality indexes | Implemented, measured |
@ -70,7 +73,10 @@ pub struct LinearScan { entries: Vec<Entry> }
### Variant A: `HnswBeamSearch`
Builds a single-layer proximity graph (M neighbours per node, bidirectional). At query time, performs greedy descent from the entry point, maintaining a max-heap of ef candidates. When too few results exceed θ, ef_search doubles (up to ef_max). The adaptive doubling is the key mechanism — it avoids the user having to pre-tune ef.
Builds a single-layer proximity graph (M neighbours per node, bidirectional).
At query time, it performs greedy descent from the entry point and doubles
`ef_search` until the returned opaque-ID set stabilises or reaches `ef_max`.
This reduces—but does not remove—the need for workload calibration.
```rust
pub struct HnswBeamSearch { entries, graph, m, ef_search_base, ef_search_max }
@ -78,7 +84,9 @@ pub struct HnswBeamSearch { entries, graph, m, ef_search_base, ef_search_max }
### Variant B: `ThresholdBeam`
Greedy descent with early stopping. Maintains a max-heap of candidates by similarity. Expands neighbours freely. Stops when the best remaining candidate falls below × 0.5` and a minimum node budget has been explored. Achieves perfect recall by exploring more of the graph than HnswBeamSearch, trading some speed.
Greedy descent with a fixed expansion budget. It maintains a max-heap of
candidates by similarity and expands at most `beam_width × 4` nodes. No
frontier score is treated as a proof about unseen nodes.
```rust
pub struct ThresholdBeam { entries, graph, m, beam_width }
@ -103,7 +111,7 @@ graph TD
API --> LS[LinearScan<br/>O(n·d) exact]
API --> HB[HnswBeamSearch<br/>adaptive ef, graph walk]
API --> TB[ThresholdBeam<br/>greedy descent + early stop]
API --> TB[ThresholdBeam<br/>fixed expansion budget]
LS --> GT[Ground truth]
HB --> AH1[Approximate hits]
@ -167,7 +175,7 @@ Recall ≥ : 0.80 (acceptance)
1. **ThresholdBeam beats HnswBeam on both speed and recall.** At n=5000: 655μs vs 876μs, recall 1.000 vs 0.873.
2. **Neither graph variant beats LinearScan in this PoC.** Root cause: O(n²) graph construction produces a connectivity structure not better than the linear layout for n < 10k. Production layered HNSW with ef_construction would reverse this.
3. **Recall scales well.** ThresholdBeam maintains perfect recall as n doubles from 2000 to 5000.
3. **Recall was stable in this synthetic run.** That observation is not a guarantee under distribution or graph changes.
4. **Memory overhead is modest.** Graph index costs 1.7× raw vectors.
### Benchmark caveats