ruvector/docs/adr/ADR-149-brain-performance-optimizations.md

ADR-149: Brain Performance Optimizations — SIMD Search, Batch Graph, Incremental LoRA, Quality Gating

Status

Accepted

Date

2026-04-13

Context

The pi.ruv.io brain (10,290 memories, 38M graph edges) performs well at current scale but has four optimization opportunities that compound:

Current Bottleneck	Observed	Root Cause
Search	~0.5ms per query	Scalar cosine similarity over all memories
Graph rebuild	~5-10 min on cold start	Rebuilds all 38M edges from scratch
LoRA training	Full corpus every cycle	Retrains on all 10K memories even when only 5 are new
Search noise	Returns debug/noise entries	No quality filtering in the search path

DiskANN was evaluated (ADR-148 P4) but is not cost-effective at 10K memories — brute-force is 10x faster. The crossover is ~100K memories (~6 months at accelerated ingestion). These four optimizations deliver immediate value at current scale.

Decision

Implement four optimizations in priority order. Each is independent and can ship separately.

P1: SIMD Cosine in Search (2-3x speedup, 1 hour)

Problem: crates/mcp-brain-server/src/graph.rs uses a scalar cosine similarity:

pub fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
    let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
    let norm_a: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
    let norm_b: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
    if norm_a < 1e-10 || norm_b < 1e-10 { return 0.0; }
    dot / (norm_a * norm_b)
}

Fix: Replace with ruvector-core's SIMD-accelerated version which already has NEON (Apple Silicon), AVX2, and AVX-512 implementations:

// In graph.rs or store.rs:
use ruvector_core::simd_intrinsics::cosine_similarity_simd;

// In search_memories():
let sim = cosine_similarity_simd(&query_embedding, &m.embedding);

The ruvector-core SIMD cosine at 128 dimensions processes 4 floats/cycle (NEON) or 8 floats/cycle (AVX2), with 4x loop unrolling. Expected speedup: 2-3x on x86 (Cloud Run), 2x on ARM.

Dependency: Add ruvector-core to mcp-brain-server/Cargo.toml (it's already in the workspace but not a direct dependency of the brain server).

Risk: None. Same math, faster execution. The SIMD functions have 1,486 lines of tests in simd_intrinsics.rs.

P2: Quality-Gated Search (immediate relevance improvement, 30 minutes)

Problem: Search returns all memories regardless of quality. With ~40% of memories being noise (solution category scraped from random websites), the top-k results are polluted.

Fix: Add quality floor to the search path:

// In search_memories():
let quality_floor = 0.05; // Skip bottom-tier noise
for m in memories {
    if m.quality_score.mean() < quality_floor {
        continue; // Skip known noise
    }
    let sim = cosine_similarity_simd(&query_embedding, &m.embedding);
    // ...
}

Also add optional min_quality parameter to the search API:

GET /v1/memories/search?q=seizure+detection&limit=10&min_quality=0.1

Side effect: Reduces the number of memories to scan, giving an additional 30-40% search speedup (skip ~4K noise memories).

Risk: Very low. Quality floor defaults to 0.05 (only skip the absolute bottom). The API parameter is optional — existing calls are unaffected.

P3: Batch Graph Operations (10x faster rebuild, 1 day)

Problem: The 38M-edge graph rebuilds by inserting edges one at a time after Firestore hydration. Each insertion triggers adjacency list updates.

Fix: Batch the graph construction:

// Current (slow): 
for memory in memories {
    for neighbor in find_neighbors(memory) {
        graph.add_edge(memory.id, neighbor.id, similarity);
    }
}

// Optimized (fast):
// 1. Compute all similarities in a single pass (SIMD-accelerated)
// 2. Sort edges by source node
// 3. Build adjacency lists in one allocation
let mut edges: Vec<(NodeId, NodeId, f32)> = Vec::with_capacity(estimated_edges);
for (i, m1) in memories.iter().enumerate() {
    for m2 in &memories[i+1..] {
        let sim = cosine_similarity_simd(&m1.embedding, &m2.embedding);
        if sim > threshold {
            edges.push((m1.id, m2.id, sim));
        }
    }
}
edges.sort_by_key(|e| e.0);
graph.build_from_sorted_edges(&edges);

Additional optimization: Use rayon parallel iterator for the similarity computation:

let edges: Vec<_> = (0..n).into_par_iter()
    .flat_map(|i| {
        (i+1..n).filter_map(move |j| {
            let sim = cosine_similarity_simd(&memories[i].embedding, &memories[j].embedding);
            if sim > threshold { Some((i, j, sim)) } else { None }
        })
    }).collect();

At 10K memories × 128 dims with SIMD, this processes ~50M similarity computations in ~5 seconds on 4 cores (vs minutes for sequential single-edge insertion).

Risk: Low. Graph is rebuilt from scratch — no incremental state to corrupt. If the batch builder fails, fall back to the existing sequential builder.

P4: Incremental LoRA (5x faster training, 1 week)

Problem: Every 5-minute training cycle processes ALL 10K+ memories to extract propositions and compute LoRA updates. Most memories haven't changed — only the last ~5 are new.

Fix: Track a last_trained_at watermark and only process new memories:

struct IncrementalTrainer {
    last_trained_at: chrono::DateTime<chrono::Utc>,
    cumulative_lora: LoraWeights,
}

impl IncrementalTrainer {
    fn train_cycle(&mut self, store: &MemoryStore) {
        // Only process memories created since last training
        let new_memories: Vec<_> = store.all_memories()
            .into_iter()
            .filter(|m| m.created_at > self.last_trained_at)
            .collect();
        
        if new_memories.is_empty() {
            return; // Nothing new — skip entirely
        }
        
        // Extract propositions from new memories only
        let new_propositions = extract_propositions(&new_memories);
        
        // Compute incremental LoRA update
        let delta_lora = compute_lora_delta(&new_propositions, &self.cumulative_lora);
        
        // Apply with EWC to prevent catastrophic forgetting
        self.cumulative_lora = ewc_merge(&self.cumulative_lora, &delta_lora);
        
        self.last_trained_at = chrono::Utc::now();
    }
}

Savings:

• Current: process 10K memories every 5 min = 2M memories/day
• Incremental: process ~50 new memories per cycle = 14K memories/day
• 143x reduction in computation

Risk: Medium. Incremental training may drift from what a full retrain would produce. Mitigate by running a full retrain every 24 hours (the existing nightly job) and using the incremental updates only between full retrains. EWC (already implemented) prevents catastrophic forgetting.

Implementation Order

Week 1:
  Day 1: P1 (SIMD cosine) — wire ruvector-core into brain server
  Day 1: P2 (quality gate) — add quality floor to search path
  Day 2-3: P3 (batch graph) — parallel batch graph builder

Week 2-3:
  P4 (incremental LoRA) — watermark tracking + delta training

P1 and P2 can ship in the same deploy. P3 ships independently. P4 is a separate PR.

Expected Impact

Optimization	Before	After	Speedup
P1: SIMD search	0.5ms/query	0.2ms/query	2.5x
P2: Quality gate	Scan 10.3K memories	Scan ~6K memories	1.7x
P1+P2 combined	0.5ms/query	~0.1ms/query	5x
P3: Graph rebuild	5-10 min cold start	~30 seconds	10-20x
P4: LoRA training	10K memories/cycle	~50 memories/cycle	143x

Combined search improvement: 5x faster with cleaner results. Combined startup improvement: 10-20x faster cold boot. Combined training improvement: 143x less computation per cycle.

Consequences

Positive

• Search drops from 0.5ms to ~0.1ms (5x) — enables real-time use cases
• Cold start drops from 5-10 min to ~30 seconds — faster deploys, less downtime
• Training cycles go from processing 10K memories to ~50 — 143x less compute, lower Cloud Run costs
• Search quality improves immediately from quality gating (skip noise)
• All optimizations are backward-compatible — no API changes, no data migration

Negative

• ruvector-core added as brain server dependency (increases binary size ~2MB)
• Quality gate may hide memories that become relevant later (mitigated by low 0.05 threshold)
• Incremental LoRA may diverge from full retrain (mitigated by nightly full retrain + EWC)

Not Addressed

• DiskANN integration deferred to 100K+ memories (ADR-148 P4)
• Embedding dimension upgrade (128 → 384) deferred to next major version
• Multi-region replication not in scope

References

• ADR-148: Brain Hypothesis Engine (DiskANN at 100K+)
• ADR-146: DiskANN Vamana Implementation
• crates/ruvector-core/src/simd_intrinsics.rs — NEON/AVX2/AVX-512 cosine at lines 42-904
• crates/mcp-brain-server/src/graph.rs — current scalar cosine
• crates/mcp-brain-server/src/store.rs:580-615 — current search implementation