mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-30 12:13:34 +00:00
* feat(ospipe): implement OSpipe screenpipe integration with WASM + TypeScript SDK Adds the OSpipe crate providing a quantum-enhanced screenpipe integration layer: - Rust core library (7 modules): capture, storage, search, pipeline, safety, config, wasm - WASM bindings via wasm-bindgen for browser deployment - TypeScript SDK (@ruvector/ospipe) with SSE streaming and hybrid search - Frame deduplication, PII safety gate, query routing, cosine similarity search - 56 tests passing (24 unit + 32 integration), builds for native + wasm32 - Comprehensive ADR with Windows/macOS/Linux/WASM integration plans - CI stub for cross-platform matrix builds (Linux, Windows, macOS, WASM) Co-Authored-By: claude-flow <ruv@ruv.net> * chore(ospipe): add README, fix clippy warnings, optimize dedup and pipeline - Add comprehensive README.md with features, comparison tables, quick start guides, collapsed configuration reference, and API docs - Fix all default clippy warnings (auto-fix + manual) - Replace Vec with VecDeque in FrameDeduplicator for O(1) eviction - Remove redundant frame.clone() in ingestion pipeline (move instead) - Add is_empty() to WASM OsPipeWasm type - Fix broken intra-doc link for cfg-gated bindings module - Remove unused imports in integration tests (FrameContent, SearchConfig) Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ospipe): integrate graph, attention, GNN, and quantum crates (Phase 2-4) Add four new OSpipe modules integrating RuVector crates: - graph: KnowledgeGraph wrapping ruvector-graph with heuristic entity extraction (URLs, emails, @mentions, capitalized phrases), entity/ relationship CRUD, and frame entity ingestion - search/reranker: AttentionReranker using ruvector-attention scaled dot-product attention for result re-ranking (0.6*attention + 0.4*cosine) - learning: SearchLearner with EWC (ruvector-gnn) for continual learning without catastrophic forgetting, ReplayBuffer for feedback, and EmbeddingQuantizer for age-based vector compression - quantum: QuantumSearch using ruqu-algorithms QAOA for diversity selection, Grover-inspired amplitude boosting, and optimal iteration estimation All modules use cfg-gated dual implementations (native + WASM stub). 60 tests passing (59 integration + 1 doc-test), native + WASM builds clean. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ospipe): complete all 15 gap items — HNSW, persistence, REST API, MMR, safety fixes Implements all remaining OSpipe features from the gap analysis: High — Core functionality: - HNSW indexing via ruvector-core with O(log n) ANN search (HnswVectorStore) - EmbeddingModel trait + RuvectorEmbeddingModel for pluggable embedding backends - JSON-file persistence layer (PersistenceLayer) for frames and config - Axum REST API server matching TypeScript SDK endpoints (/search, /graph, /health, /stats, /route) - Enhanced search pipeline wired into ingestion (router -> rerank -> quantum diversity) Medium — Correctness: - WASM/native routing consistency (aligned keyword sets and priority order) - WASM/native safety consistency (email detection, deny keywords, CC/SSN patterns) - MMR (Maximal Marginal Relevance) reranker for diversity vs relevance tradeoff - Delete and update_metadata APIs on VectorStore and HnswVectorStore - Email redaction preserves surrounding whitespace (tabs, newlines, multi-space) Lower — Polish: - TypeScript SDK: fetchWithRetry with exponential backoff, timeout, AbortSignal - console_error_panic_hook init in WASM module - WASM test scaffold (tests/wasm.rs) - Quantization tiers in config (None -> Scalar -> Product -> Binary by age) - All clippy warnings resolved (0 warnings) 82 tests passing, 1 doc-test passing, 0 clippy warnings. Co-Authored-By: claude-flow <ruv@ruv.net> * chore: update Cargo.lock after OSpipe dependency changes Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ospipe): add server binary, WASM build, version-pin deps for publishing - Add ospipe-server binary with CLI args (--port, --data-dir, --help, --version) - Add tracing-subscriber for structured logging - Version-pin all 9 path dependencies for crates.io readiness - Fix ref -> ref mut for KnowledgeGraph mutable borrow in pipeline - Fix redundant rustdoc link in embedding.rs - Update ospipe-wasm package.json to match wasm-pack output filenames - WASM build produces 145KB binary with full browser API Build artifacts (not committed, in dist/): - ospipe-server-linux-x86_64 (1.8MB) - ospipe-server-linux-arm64 (1.6MB) - ospipe-server-windows-x86_64.exe (3.9MB) - ospipe_bg.wasm (145KB) - @ruvector/ospipe npm tarball (13.9KB) Co-Authored-By: claude-flow <ruv@ruv.net> * docs: add OSpipe to root README, publish ospipe + deps to crates.io Add OSpipe personal AI memory section to root README with features, comparison table, install commands, and Rust quickstart. Published to registries: - ospipe v0.1.0 (crates.io) - ruvector-delta-core v0.1.0 (crates.io) - ruvector-cluster v2.0.2 (crates.io) - ruvector-router-core v2.0.2 (crates.io) - @ruvector/ospipe v0.1.0 (npm) - @ruvector/ospipe-wasm v0.1.0 (npm) Co-Authored-By: claude-flow <ruv@ruv.net> * fix: add uuid dev-dep for tests, bump rvlite to 0.2.1 - Add uuid to OSpipe dev-dependencies to fix version mismatch in integration tests - Bump rvlite npm package to 0.2.1 (0.2.0 blocked by npm) Co-Authored-By: claude-flow <ruv@ruv.net>
283 lines
8.2 KiB
Rust
283 lines
8.2 KiB
Rust
//! WASM integration tests for OSpipe.
|
|
//!
|
|
//! These tests run in a browser-like environment using `wasm-bindgen-test`.
|
|
//! Execute with:
|
|
//!
|
|
//! ```bash
|
|
//! wasm-pack test --headless --chrome -- --test wasm
|
|
//! ```
|
|
|
|
#![cfg(target_arch = "wasm32")]
|
|
|
|
use wasm_bindgen::JsValue;
|
|
use wasm_bindgen_test::*;
|
|
|
|
wasm_bindgen_test_configure!(run_in_browser);
|
|
|
|
use ospipe::wasm::bindings::OsPipeWasm;
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Construction
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_create_instance() {
|
|
let instance = OsPipeWasm::new(384);
|
|
assert_eq!(instance.len(), 0);
|
|
assert!(instance.is_empty());
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_create_with_custom_dimension() {
|
|
let instance = OsPipeWasm::new(128);
|
|
assert_eq!(instance.len(), 0);
|
|
|
|
let stats_json = instance.stats();
|
|
assert!(
|
|
stats_json.contains("\"dimension\":128"),
|
|
"Stats should report dimension 128, got: {}",
|
|
stats_json
|
|
);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Insert + Search roundtrip
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_insert_and_search_roundtrip() {
|
|
let mut instance = OsPipeWasm::new(4);
|
|
|
|
// Insert two vectors.
|
|
let emb_a: Vec<f32> = vec![1.0, 0.0, 0.0, 0.0];
|
|
let emb_b: Vec<f32> = vec![0.0, 1.0, 0.0, 0.0];
|
|
|
|
instance
|
|
.insert("a", &emb_a, r#"{"label":"a"}"#, 1000.0)
|
|
.expect("insert a");
|
|
instance
|
|
.insert("b", &emb_b, r#"{"label":"b"}"#, 2000.0)
|
|
.expect("insert b");
|
|
|
|
assert_eq!(instance.len(), 2);
|
|
assert!(!instance.is_empty());
|
|
|
|
// Searching with emb_a should return "a" as the top hit.
|
|
let results: JsValue = instance.search(&emb_a, 2).expect("search");
|
|
let results_str = js_sys::JSON::stringify(&results)
|
|
.expect("stringify")
|
|
.as_string()
|
|
.expect("as_string");
|
|
|
|
assert!(
|
|
results_str.contains("\"id\":\"a\""),
|
|
"Top result should be 'a', got: {}",
|
|
results_str
|
|
);
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_insert_dimension_mismatch() {
|
|
let mut instance = OsPipeWasm::new(4);
|
|
let wrong_dim: Vec<f32> = vec![1.0, 2.0]; // dimension 2, expects 4
|
|
|
|
let result = instance.insert("bad", &wrong_dim, "{}", 0.0);
|
|
assert!(result.is_err(), "Should reject mismatched dimension");
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Filtered search
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_search_filtered_by_time() {
|
|
let mut instance = OsPipeWasm::new(4);
|
|
|
|
let emb: Vec<f32> = vec![1.0, 0.0, 0.0, 0.0];
|
|
instance
|
|
.insert("early", &emb, "{}", 1000.0)
|
|
.expect("insert early");
|
|
instance
|
|
.insert("late", &emb, "{}", 5000.0)
|
|
.expect("insert late");
|
|
|
|
// Filter to only the early entry (timestamp range [0, 2000]).
|
|
let results: JsValue = instance
|
|
.search_filtered(&emb, 10, 0.0, 2000.0)
|
|
.expect("search_filtered");
|
|
let results_str = js_sys::JSON::stringify(&results)
|
|
.expect("stringify")
|
|
.as_string()
|
|
.expect("as_string");
|
|
|
|
assert!(
|
|
results_str.contains("\"id\":\"early\""),
|
|
"Filtered results should include 'early', got: {}",
|
|
results_str
|
|
);
|
|
assert!(
|
|
!results_str.contains("\"id\":\"late\""),
|
|
"Filtered results should exclude 'late', got: {}",
|
|
results_str
|
|
);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// embed_text
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_embed_text_returns_correct_dimension() {
|
|
let instance = OsPipeWasm::new(384);
|
|
let embedding = instance.embed_text("hello world");
|
|
assert_eq!(
|
|
embedding.len(),
|
|
384,
|
|
"embed_text should return a vector of the configured dimension"
|
|
);
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_embed_text_is_deterministic() {
|
|
let instance = OsPipeWasm::new(64);
|
|
let a = instance.embed_text("test input");
|
|
let b = instance.embed_text("test input");
|
|
assert_eq!(a, b, "Same input text should produce identical embeddings");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_embed_text_different_inputs_differ() {
|
|
let instance = OsPipeWasm::new(64);
|
|
let a = instance.embed_text("alpha");
|
|
let b = instance.embed_text("beta");
|
|
assert_ne!(a, b, "Different inputs should produce different embeddings");
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// safety_check
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_safety_check_allow() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let decision = instance.safety_check("the weather is nice today");
|
|
assert_eq!(decision, "allow");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_safety_check_deny_credit_card() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let decision = instance.safety_check("card number 4111-1111-1111-1111");
|
|
assert_eq!(decision, "deny");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_safety_check_deny_ssn() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let decision = instance.safety_check("my ssn is 123-45-6789");
|
|
assert_eq!(decision, "deny");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_safety_check_redact_password() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let decision = instance.safety_check("my password is hunter2");
|
|
assert_eq!(decision, "redact");
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// route_query
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_route_query_temporal() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let route = instance.route_query("what happened yesterday");
|
|
assert_eq!(route, "Temporal");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_route_query_keyword_short() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let route = instance.route_query("rust");
|
|
assert_eq!(route, "Keyword");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_route_query_keyword_quoted() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let route = instance.route_query("\"exact phrase\"");
|
|
assert_eq!(route, "Keyword");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_route_query_graph() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let route = instance.route_query("things related to authentication module");
|
|
assert_eq!(route, "Graph");
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_route_query_hybrid_default() {
|
|
let instance = OsPipeWasm::new(4);
|
|
let route = instance.route_query("explain how neural networks learn patterns");
|
|
assert_eq!(route, "Hybrid");
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Deduplication
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_is_duplicate_identical() {
|
|
let mut instance = OsPipeWasm::new(4);
|
|
let emb: Vec<f32> = vec![1.0, 0.0, 0.0, 0.0];
|
|
|
|
instance
|
|
.insert("original", &emb, "{}", 0.0)
|
|
.expect("insert");
|
|
|
|
assert!(
|
|
instance.is_duplicate(&emb, 0.99),
|
|
"Identical embedding should be detected as duplicate"
|
|
);
|
|
}
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_is_not_duplicate_orthogonal() {
|
|
let mut instance = OsPipeWasm::new(4);
|
|
let emb_a: Vec<f32> = vec![1.0, 0.0, 0.0, 0.0];
|
|
let emb_b: Vec<f32> = vec![0.0, 1.0, 0.0, 0.0];
|
|
|
|
instance.insert("a", &emb_a, "{}", 0.0).expect("insert");
|
|
|
|
assert!(
|
|
!instance.is_duplicate(&emb_b, 0.5),
|
|
"Orthogonal embedding should not be a duplicate at threshold 0.5"
|
|
);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Stats
|
|
// ---------------------------------------------------------------------------
|
|
|
|
#[wasm_bindgen_test]
|
|
fn test_stats_json() {
|
|
let mut instance = OsPipeWasm::new(16);
|
|
let emb: Vec<f32> = vec![0.0; 16];
|
|
|
|
instance.insert("x", &emb, "{}", 0.0).expect("insert");
|
|
|
|
let stats = instance.stats();
|
|
assert!(stats.contains("\"dimension\":16"), "Stats: {}", stats);
|
|
assert!(
|
|
stats.contains("\"total_embeddings\":1"),
|
|
"Stats: {}",
|
|
stats
|
|
);
|
|
assert!(
|
|
stats.contains("\"memory_estimate_bytes\""),
|
|
"Stats: {}",
|
|
stats
|
|
);
|
|
}
|