mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-30 20:43:38 +00:00
* feat: Add comprehensive dataset discovery framework for RuVector
This commit introduces a powerful dataset discovery framework with
integrations for three high-impact public data sources:
## Core Framework (examples/data/framework/)
- DataIngester: Streaming ingestion with batching and deduplication
- CoherenceEngine: Min-cut based coherence signal computation
- DiscoveryEngine: Pattern detection for emerging structures
## OpenAlex Integration (examples/data/openalex/)
- Research frontier radar: Detect emerging fields via boundary motion
- Cross-domain bridge detection: Find connector subgraphs
- Topic graph construction from citation networks
- Full API client with cursor-based pagination
## Climate Integration (examples/data/climate/)
- NOAA GHCN and NASA Earthdata clients
- Sensor network graph construction
- Regime shift detection using min-cut coherence breaks
- Time series vectorization for similarity search
- Seasonal decomposition analysis
## SEC EDGAR Integration (examples/data/edgar/)
- XBRL financial statement parsing
- Peer network construction
- Coherence watch: Detect fundamental vs narrative divergence
- Filing analysis with sentiment and risk extraction
- Cross-company contagion detection
Each integration leverages RuVector's unique capabilities:
- Vector memory for semantic similarity
- Graph structures for relationship modeling
- Dynamic min-cut for coherence signal computation
- Time series embeddings for pattern matching
Discovery thesis: Detect emerging patterns before they have names,
find non-obvious cross-domain bridges, and map causality chains.
* feat: Add working discovery examples for climate and financial data
- Fix borrow checker issues in coherence analysis modules
- Create standalone workspace for data examples
- Add regime_detector.rs for climate network coherence analysis
- Add coherence_watch.rs for SEC EDGAR narrative-fundamental divergence
- Add frontier_radar.rs template for OpenAlex research discovery
- Update Cargo.toml dependencies for example executability
- Add rand dev-dependency for demo data generation
Examples successfully detect:
- Climate regime shifts via min-cut coherence analysis
- Cross-regional teleconnection patterns
- Fundamental vs narrative divergence in SEC filings
- Sector fragmentation signals in financial data
* feat: Add working discovery examples for climate and financial data
- Add RuVector-native discovery engine with Stoer-Wagner min-cut
- Implement cross-domain pattern detection (climate ↔ finance)
- Add cosine similarity for vector-based semantic matching
- Create cross_domain_discovery example demonstrating:
- 42% cross-domain edge connectivity
- Bridge formation detection with 0.73-0.76 confidence
- Climate and finance correlation hypothesis generation
* perf: Add optimized discovery engine with SIMD and parallel processing
Performance improvements:
- 8.84x speedup for vector insertion via parallel batching
- 2.91x SIMD speedup for cosine similarity (chunked + AVX2)
- Incremental graph updates with adjacency caching
- Early termination in Stoer-Wagner min-cut
Statistical analysis features:
- P-value computation for pattern significance
- Effect size (Cohen's d) calculation
- 95% confidence intervals
- Granger-style temporal causality detection
Benchmark results (248 vectors, 3 domains):
- Cross-domain edges: 34.9% of total graph
- Domain coherence: Climate 0.74, Finance 0.94, Research 0.97
- Detected climate-finance temporal correlations
* feat: Add discovery hunter and comprehensive README tutorial
New features:
- Discovery hunter example with multi-phase pattern detection
- Climate extremes, financial stress, and research data generation
- Cross-domain hypothesis generation
- Anomaly injection testing
Documentation:
- Detailed README with step-by-step tutorial
- API reference for OptimizedConfig and patterns
- Performance benchmarks and best practices
- Troubleshooting guide
* feat: Complete discovery framework with all features
HNSW Indexing (754 lines):
- O(log n) approximate nearest neighbor search
- Configurable M, ef_construction parameters
- Cosine, Euclidean, Manhattan distance metrics
- Batch insertion support
API Clients (888 lines):
- OpenAlex: academic works, authors, topics
- NOAA: climate observations
- SEC EDGAR: company filings
- Rate limiting and retry logic
Persistence (638 lines):
- Save/load engine state and patterns
- Gzip compression (3-10x size reduction)
- Incremental pattern appending
CLI Tool (1,109 lines):
- discover, benchmark, analyze, export commands
- Colored terminal output
- JSON and human-readable formats
Streaming (570 lines):
- Async stream processing
- Sliding and tumbling windows
- Real-time pattern detection
- Backpressure handling
Tests (30 unit tests):
- Stoer-Wagner min-cut verification
- SIMD cosine similarity accuracy
- Statistical significance
- Granger causality
- Cross-domain patterns
Benchmarks:
- CLI: 176 vectors/sec @ 2000 vectors
- SIMD: 6.82M ops/sec (2.06x speedup)
- Vector insertion: 1.61x speedup
- Total: 44.74ms for 248 vectors
* feat: Add visualization, export, forecasting, and real data discovery
Visualization (555 lines):
- ASCII graph rendering with box-drawing characters
- Domain-based ANSI coloring (Climate=blue, Finance=green, Research=yellow)
- Coherence timeline sparklines
- Pattern summary dashboard
- Domain connectivity matrix
Export (650 lines):
- GraphML export for Gephi/Cytoscape
- DOT export for Graphviz
- CSV export for patterns and coherence history
- Filtered export by domain, weight, time range
- Batch export with README generation
Forecasting (525 lines):
- Holt's double exponential smoothing for trend
- CUSUM-based regime change detection (70.67% accuracy)
- Cross-domain correlation forecasting (r=1.000)
- Prediction intervals (95% CI)
- Anomaly probability scoring
Real Data Discovery:
- Fetched 80 actual papers from OpenAlex API
- Topics: climate risk, stranded assets, carbon pricing, physical risk, transition risk
- Built coherence graph: 592 nodes, 1049 edges
- Average min-cut: 185.76 (well-connected research cluster)
* feat: Add medical, real-time, and knowledge graph data sources
New API Clients:
- PubMed E-utilities for medical literature search (NCBI)
- ClinicalTrials.gov v2 API for clinical study data
- FDA OpenFDA for drug adverse events and recalls
- Wikipedia article search and extraction
- Wikidata SPARQL queries for structured knowledge
Real-time Features:
- RSS/Atom feed parsing with deduplication
- News aggregator with multiple source support
- WebSocket and REST polling infrastructure
- Event streaming with configurable windows
Examples:
- medical_discovery: PubMed + ClinicalTrials + FDA integration
- multi_domain_discovery: Climate-health-finance triangulation
- wiki_discovery: Wikipedia/Wikidata knowledge graph
- realtime_feeds: News feed aggregation demo
Tested across 70+ unit tests with all domains integrated.
* feat: Add economic, patent, and ArXiv data source clients
New API Clients:
- FredClient: Federal Reserve economic indicators (GDP, CPI, unemployment)
- WorldBankClient: Global development indicators and climate data
- AlphaVantageClient: Stock market daily prices
- ArxivClient: Scientific preprint search with category and date filters
- UsptoPatentClient: USPTO patent search by keyword, assignee, CPC class
- EpoClient: Placeholder for European patent search
New Domain:
- Domain::Economic for economic/financial indicator data
Updated Exports:
- Domain colors and shapes for Economic in visualization and export
Examples:
- economic_discovery: FRED + World Bank integration demo
- arxiv_discovery: AI/ML/Climate paper search demo
- patent_discovery: Climate tech and AI patent search demo
All 85 tests passing. APIs tested with live endpoints.
* feat: Add Semantic Scholar, bioRxiv/medRxiv, and CrossRef research clients
New Research API Clients:
- SemanticScholarClient: Citation graph analysis, paper search, author lookup
- Methods: search_papers, get_citations, get_references, search_by_field
- Builds citation networks for graph analysis
- BiorxivClient: Life sciences preprints
- Methods: search_recent, search_by_category (neuroscience, genomics, etc.)
- Automatic conversion to Domain::Research
- MedrxivClient: Medical preprints
- Methods: search_covid, search_clinical, search_by_date_range
- Automatic conversion to Domain::Medical
- CrossRefClient: DOI metadata and scholarly communication
- Methods: search_works, get_work, search_by_funder, get_citations
- Polite pool support for better rate limits
All clients include:
- Rate limiting respecting API guidelines
- Retry logic with exponential backoff
- SemanticVector conversion with rich metadata
- Comprehensive unit tests
Examples:
- biorxiv_discovery: Fetch neuroscience and clinical research
- crossref_demo: Search publications, funders, datasets
Total: 104 tests passing, ~2,500 new lines of code
* feat: Add MCP server with STDIO/SSE transport and optimized discovery
MCP Server Implementation (mcp_server.rs):
- JSON-RPC 2.0 protocol with MCP 2024-11-05 compliance
- Dual transport: STDIO for CLI, SSE for HTTP streaming
- 22 discovery tools exposing all data sources:
- Research: OpenAlex, ArXiv, Semantic Scholar, CrossRef, bioRxiv, medRxiv
- Medical: PubMed, ClinicalTrials.gov, FDA
- Economic: FRED, World Bank
- Climate: NOAA
- Knowledge: Wikipedia, Wikidata SPARQL
- Discovery: Multi-source, coherence analysis, pattern detection
- Resources: discovery://patterns, discovery://graph, discovery://history
- Pre-built prompts: cross_domain_discovery, citation_analysis, trend_detection
Binary Entry Point (bin/mcp_discovery.rs):
- CLI arguments with clap
- Configurable discovery parameters
- STDIO/SSE mode selection
Optimized Discovery Runner:
- Parallel data fetching with tokio::join!
- SIMD-accelerated vector operations (1.1M comparisons/sec)
- 6-phase discovery pipeline with benchmarking
- Statistical significance testing (p-values)
- Cross-domain correlation analysis
- CSV export and hypothesis report generation
Performance Results:
- 180 vectors from 3 sources in 7.5s
- 686 edges computed in 8ms
- SIMD throughput: 1,122,216 comparisons/sec
All 106 tests passing.
* feat: Add space, genomics, and physics data source clients
Add exotic data source integrations:
- Space clients: NASA (APOD, NEO, Mars, DONKI), Exoplanet Archive, SpaceX API, TNS Astronomy
- Genomics clients: NCBI (genes, proteins, SNPs), UniProt, Ensembl, GWAS Catalog
- Physics clients: USGS Earthquakes, CERN Open Data, Argo Ocean, Materials Project
New domains: Space, Genomics, Physics, Seismic, Ocean
All 106 tests passing, SIMD benchmark: 208k comparisons/sec
* chore: Update export/visualization and output files
* docs: Add API client inventory and reference documentation
* fix: Update API clients for 2025 endpoint changes
- ArXiv: Switch from HTTP to HTTPS (export.arxiv.org)
- USPTO: Migrate to PatentSearch API v2 (search.patentsview.org)
- Legacy API (api.patentsview.org) discontinued May 2025
- Updated query format from POST to GET
- Note: May require API authentication
- FRED: Require API key (mandatory as of 2025)
- Added error handling for missing API key
- Added response error field parsing
All tests passing, ArXiv discovery confirmed working
* feat: Implement comprehensive 2025 API client library (11,810 lines)
Add 7 new API client modules implementing 35+ data sources:
Academic APIs (1,328 lines):
- OpenAlexClient, CoreClient, EricClient, UnpaywallClient
Finance APIs (1,517 lines):
- FinnhubClient, TwelveDataClient, CoinGeckoClient, EcbClient, BlsClient
Geospatial APIs (1,250 lines):
- NominatimClient, OverpassClient, GeonamesClient, OpenElevationClient
News & Social APIs (1,606 lines):
- HackerNewsClient, GuardianClient, NewsDataClient, RedditClient
Government APIs (2,354 lines):
- CensusClient, DataGovClient, EuOpenDataClient, UkGovClient
- WorldBankGovClient, UNDataClient
AI/ML APIs (2,035 lines):
- HuggingFaceClient, OllamaClient, ReplicateClient
- TogetherAiClient, PapersWithCodeClient
Transportation APIs (1,720 lines):
- GtfsClient, MobilityDatabaseClient
- OpenRouteServiceClient, OpenChargeMapClient
All clients include:
- Async/await with tokio and reqwest
- Mock data fallback for testing without API keys
- Rate limiting with configurable delays
- SemanticVector conversion for RuVector integration
- Comprehensive unit tests (252 total tests passing)
- Full error handling with FrameworkError
* docs: Add API client documentation for new implementations
Add documentation for:
- Geospatial clients (Nominatim, Overpass, Geonames, OpenElevation)
- ML clients (HuggingFace, Ollama, Replicate, Together, PapersWithCode)
- News clients (HackerNews, Guardian, NewsData, Reddit)
- Finance clients implementation notes
* feat: Implement dynamic min-cut tracking system (SODA 2026)
Based on El-Hayek, Henzinger, Li (SODA 2026) subpolynomial dynamic min-cut algorithm.
Core Components (2,626 lines):
- dynamic_mincut.rs (1,579 lines): EulerTourTree, DynamicCutWatcher, LocalMinCutProcedure
- cut_aware_hnsw.rs (1,047 lines): CutAwareHNSW, CoherenceZones, CutGatedSearch
Key Features:
- O(log n) connectivity queries via Euler-tour trees
- n^{o(1)} update time when λ ≤ 2^{(log n)^{3/4}} (vs O(n³) Stoer-Wagner)
- Cut-gated HNSW search that respects coherence boundaries
- Real-time cut monitoring with threshold-based deep evaluation
- Thread-safe structures with Arc<RwLock>
Performance (benchmarked):
- 75x speedup over periodic recomputation
- O(1) min-cut queries vs O(n³) recompute
- ~25µs per edge update
Tests & Benchmarks:
- 36+ unit tests across both modules
- 5 benchmark suites comparing periodic vs dynamic
- Integration with existing OptimizedDiscoveryEngine
This enables real-time coherence tracking in RuVector, transforming
min-cut from an expensive periodic computation to a maintained invariant.
---------
Co-authored-by: Claude <noreply@anthropic.com>
469 lines
12 KiB
Rust
469 lines
12 KiB
Rust
//! Peer network construction for financial coherence analysis
|
|
|
|
use std::collections::HashMap;
|
|
|
|
use chrono::{DateTime, Utc};
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
use crate::{Company, Sector};
|
|
|
|
/// A company node in the peer network
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct CompanyNode {
|
|
/// Company CIK
|
|
pub cik: String,
|
|
|
|
/// Company name
|
|
pub name: String,
|
|
|
|
/// Ticker symbol
|
|
pub ticker: Option<String>,
|
|
|
|
/// Sector
|
|
pub sector: Sector,
|
|
|
|
/// Market cap (if known)
|
|
pub market_cap: Option<f64>,
|
|
|
|
/// Number of peer connections
|
|
pub peer_count: usize,
|
|
|
|
/// Average peer similarity
|
|
pub avg_peer_similarity: f64,
|
|
}
|
|
|
|
/// An edge between peer companies
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct PeerEdge {
|
|
/// Source company CIK
|
|
pub source: String,
|
|
|
|
/// Target company CIK
|
|
pub target: String,
|
|
|
|
/// Similarity score (0-1)
|
|
pub similarity: f64,
|
|
|
|
/// Relationship type
|
|
pub relationship_type: PeerRelationType,
|
|
|
|
/// Edge weight for min-cut
|
|
pub weight: f64,
|
|
|
|
/// Evidence for relationship
|
|
pub evidence: Vec<String>,
|
|
}
|
|
|
|
/// Type of peer relationship
|
|
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
|
|
pub enum PeerRelationType {
|
|
/// Same sector/industry
|
|
SameSector,
|
|
/// Shared institutional investors
|
|
SharedInvestors,
|
|
/// Similar size (market cap)
|
|
SimilarSize,
|
|
/// Supply chain relationship
|
|
SupplyChain,
|
|
/// Competitor
|
|
Competitor,
|
|
/// Multiple relationship types
|
|
Multiple,
|
|
}
|
|
|
|
/// Peer network graph
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct PeerNetwork {
|
|
/// Network identifier
|
|
pub id: String,
|
|
|
|
/// Nodes (companies)
|
|
pub nodes: HashMap<String, CompanyNode>,
|
|
|
|
/// Edges (peer relationships)
|
|
pub edges: Vec<PeerEdge>,
|
|
|
|
/// Creation time
|
|
pub created_at: DateTime<Utc>,
|
|
|
|
/// Network statistics
|
|
pub stats: NetworkStats,
|
|
}
|
|
|
|
/// Network statistics
|
|
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
|
pub struct NetworkStats {
|
|
/// Number of nodes
|
|
pub node_count: usize,
|
|
|
|
/// Number of edges
|
|
pub edge_count: usize,
|
|
|
|
/// Average similarity
|
|
pub avg_similarity: f64,
|
|
|
|
/// Network density
|
|
pub density: f64,
|
|
|
|
/// Average degree
|
|
pub avg_degree: f64,
|
|
|
|
/// Number of connected components
|
|
pub num_components: usize,
|
|
|
|
/// Computed min-cut value
|
|
pub min_cut_value: Option<f64>,
|
|
}
|
|
|
|
impl PeerNetwork {
|
|
/// Create an empty network
|
|
pub fn new(id: &str) -> Self {
|
|
Self {
|
|
id: id.to_string(),
|
|
nodes: HashMap::new(),
|
|
edges: Vec::new(),
|
|
created_at: Utc::now(),
|
|
stats: NetworkStats::default(),
|
|
}
|
|
}
|
|
|
|
/// Add a company node
|
|
pub fn add_node(&mut self, node: CompanyNode) {
|
|
self.nodes.insert(node.cik.clone(), node);
|
|
self.update_stats();
|
|
}
|
|
|
|
/// Add a peer edge
|
|
pub fn add_edge(&mut self, edge: PeerEdge) {
|
|
self.edges.push(edge);
|
|
self.update_stats();
|
|
}
|
|
|
|
/// Get a node by CIK
|
|
pub fn get_node(&self, cik: &str) -> Option<&CompanyNode> {
|
|
self.nodes.get(cik)
|
|
}
|
|
|
|
/// Get peer CIKs for a company
|
|
pub fn get_peers(&self, cik: &str) -> Vec<&str> {
|
|
self.edges
|
|
.iter()
|
|
.filter_map(|e| {
|
|
if e.source == cik {
|
|
Some(e.target.as_str())
|
|
} else if e.target == cik {
|
|
Some(e.source.as_str())
|
|
} else {
|
|
None
|
|
}
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
/// Get edges for a company
|
|
pub fn get_edges_for_company(&self, cik: &str) -> Vec<&PeerEdge> {
|
|
self.edges
|
|
.iter()
|
|
.filter(|e| e.source == cik || e.target == cik)
|
|
.collect()
|
|
}
|
|
|
|
/// Update statistics
|
|
fn update_stats(&mut self) {
|
|
self.stats.node_count = self.nodes.len();
|
|
self.stats.edge_count = self.edges.len();
|
|
|
|
if !self.edges.is_empty() {
|
|
self.stats.avg_similarity = self.edges.iter().map(|e| e.similarity).sum::<f64>()
|
|
/ self.edges.len() as f64;
|
|
}
|
|
|
|
let max_edges = if self.nodes.len() > 1 {
|
|
self.nodes.len() * (self.nodes.len() - 1) / 2
|
|
} else {
|
|
1
|
|
};
|
|
self.stats.density = self.edges.len() as f64 / max_edges as f64;
|
|
|
|
if !self.nodes.is_empty() {
|
|
self.stats.avg_degree = (2 * self.edges.len()) as f64 / self.nodes.len() as f64;
|
|
}
|
|
}
|
|
|
|
/// Convert to format for RuVector min-cut
|
|
pub fn to_mincut_edges(&self) -> Vec<(u64, u64, f64)> {
|
|
let mut node_ids: HashMap<&str, u64> = HashMap::new();
|
|
let mut next_id = 0u64;
|
|
|
|
for cik in self.nodes.keys() {
|
|
node_ids.insert(cik.as_str(), next_id);
|
|
next_id += 1;
|
|
}
|
|
|
|
self.edges
|
|
.iter()
|
|
.filter_map(|e| {
|
|
let src_id = node_ids.get(e.source.as_str())?;
|
|
let tgt_id = node_ids.get(e.target.as_str())?;
|
|
Some((*src_id, *tgt_id, e.weight))
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
/// Get node ID mapping
|
|
pub fn node_id_mapping(&self) -> HashMap<u64, String> {
|
|
let mut mapping = HashMap::new();
|
|
for (i, cik) in self.nodes.keys().enumerate() {
|
|
mapping.insert(i as u64, cik.clone());
|
|
}
|
|
mapping
|
|
}
|
|
}
|
|
|
|
/// Builder for peer networks
|
|
pub struct PeerNetworkBuilder {
|
|
id: String,
|
|
companies: Vec<Company>,
|
|
min_similarity: f64,
|
|
max_peers: usize,
|
|
relationship_types: Vec<PeerRelationType>,
|
|
}
|
|
|
|
impl PeerNetworkBuilder {
|
|
/// Create a new builder
|
|
pub fn new() -> Self {
|
|
Self {
|
|
id: format!("network_{}", Utc::now().timestamp()),
|
|
companies: Vec::new(),
|
|
min_similarity: 0.3,
|
|
max_peers: 20,
|
|
relationship_types: vec![
|
|
PeerRelationType::SameSector,
|
|
PeerRelationType::SimilarSize,
|
|
],
|
|
}
|
|
}
|
|
|
|
/// Set network ID
|
|
pub fn with_id(mut self, id: &str) -> Self {
|
|
self.id = id.to_string();
|
|
self
|
|
}
|
|
|
|
/// Add companies
|
|
pub fn add_companies(mut self, companies: Vec<Company>) -> Self {
|
|
self.companies.extend(companies);
|
|
self
|
|
}
|
|
|
|
/// Set minimum similarity threshold
|
|
pub fn min_similarity(mut self, min: f64) -> Self {
|
|
self.min_similarity = min;
|
|
self
|
|
}
|
|
|
|
/// Set maximum peers per company
|
|
pub fn max_peers(mut self, max: usize) -> Self {
|
|
self.max_peers = max;
|
|
self
|
|
}
|
|
|
|
/// Set relationship types to consider
|
|
pub fn relationship_types(mut self, types: Vec<PeerRelationType>) -> Self {
|
|
self.relationship_types = types;
|
|
self
|
|
}
|
|
|
|
/// Build the network
|
|
pub fn build(self) -> PeerNetwork {
|
|
let mut network = PeerNetwork::new(&self.id);
|
|
|
|
// Add nodes
|
|
for company in &self.companies {
|
|
let sector = company.sic_code
|
|
.as_ref()
|
|
.map(|s| Sector::from_sic(s))
|
|
.unwrap_or(Sector::Other);
|
|
|
|
let node = CompanyNode {
|
|
cik: company.cik.clone(),
|
|
name: company.name.clone(),
|
|
ticker: company.ticker.clone(),
|
|
sector,
|
|
market_cap: None,
|
|
peer_count: 0,
|
|
avg_peer_similarity: 0.0,
|
|
};
|
|
|
|
network.add_node(node);
|
|
}
|
|
|
|
// Add edges based on relationships
|
|
for i in 0..self.companies.len() {
|
|
for j in (i + 1)..self.companies.len() {
|
|
let company_i = &self.companies[i];
|
|
let company_j = &self.companies[j];
|
|
|
|
let (similarity, rel_type) = self.compute_similarity(company_i, company_j);
|
|
|
|
if similarity >= self.min_similarity {
|
|
let edge = PeerEdge {
|
|
source: company_i.cik.clone(),
|
|
target: company_j.cik.clone(),
|
|
similarity,
|
|
relationship_type: rel_type,
|
|
weight: similarity,
|
|
evidence: self.collect_evidence(company_i, company_j),
|
|
};
|
|
|
|
network.add_edge(edge);
|
|
}
|
|
}
|
|
}
|
|
|
|
// Update node statistics
|
|
for (cik, node) in network.nodes.iter_mut() {
|
|
let edges = network.edges
|
|
.iter()
|
|
.filter(|e| e.source == *cik || e.target == *cik)
|
|
.collect::<Vec<_>>();
|
|
|
|
node.peer_count = edges.len();
|
|
if !edges.is_empty() {
|
|
node.avg_peer_similarity = edges.iter().map(|e| e.similarity).sum::<f64>()
|
|
/ edges.len() as f64;
|
|
}
|
|
}
|
|
|
|
network
|
|
}
|
|
|
|
/// Compute similarity between two companies
|
|
fn compute_similarity(&self, a: &Company, b: &Company) -> (f64, PeerRelationType) {
|
|
let mut total_similarity = 0.0;
|
|
let mut relationship_count = 0;
|
|
let mut rel_type = PeerRelationType::SameSector;
|
|
|
|
// Sector similarity
|
|
if self.relationship_types.contains(&PeerRelationType::SameSector) {
|
|
let sector_a = a.sic_code.as_ref().map(|s| Sector::from_sic(s));
|
|
let sector_b = b.sic_code.as_ref().map(|s| Sector::from_sic(s));
|
|
|
|
if sector_a.is_some() && sector_a == sector_b {
|
|
total_similarity += 0.5;
|
|
relationship_count += 1;
|
|
} else if a.sic_code.is_some() && b.sic_code.is_some() {
|
|
// Same SIC division (first digit)
|
|
let sic_a = a.sic_code.as_ref().unwrap();
|
|
let sic_b = b.sic_code.as_ref().unwrap();
|
|
if !sic_a.is_empty() && !sic_b.is_empty() &&
|
|
sic_a.chars().next() == sic_b.chars().next() {
|
|
total_similarity += 0.3;
|
|
relationship_count += 1;
|
|
}
|
|
}
|
|
}
|
|
|
|
// Same state
|
|
if a.state.is_some() && a.state == b.state {
|
|
total_similarity += 0.2;
|
|
relationship_count += 1;
|
|
}
|
|
|
|
let similarity = if relationship_count > 0 {
|
|
total_similarity / relationship_count as f64
|
|
} else {
|
|
0.0
|
|
};
|
|
|
|
if relationship_count > 1 {
|
|
rel_type = PeerRelationType::Multiple;
|
|
}
|
|
|
|
(similarity, rel_type)
|
|
}
|
|
|
|
/// Collect evidence for relationship
|
|
fn collect_evidence(&self, a: &Company, b: &Company) -> Vec<String> {
|
|
let mut evidence = Vec::new();
|
|
|
|
let sector_a = a.sic_code.as_ref().map(|s| Sector::from_sic(s));
|
|
let sector_b = b.sic_code.as_ref().map(|s| Sector::from_sic(s));
|
|
|
|
if sector_a.is_some() && sector_a == sector_b {
|
|
evidence.push(format!("Same sector: {:?}", sector_a.unwrap()));
|
|
}
|
|
|
|
if a.state.is_some() && a.state == b.state {
|
|
evidence.push(format!("Same state: {}", a.state.as_ref().unwrap()));
|
|
}
|
|
|
|
evidence
|
|
}
|
|
}
|
|
|
|
impl Default for PeerNetworkBuilder {
|
|
fn default() -> Self {
|
|
Self::new()
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_empty_network() {
|
|
let network = PeerNetwork::new("test");
|
|
assert_eq!(network.stats.node_count, 0);
|
|
assert_eq!(network.stats.edge_count, 0);
|
|
}
|
|
|
|
#[test]
|
|
fn test_builder() {
|
|
let builder = PeerNetworkBuilder::new()
|
|
.min_similarity(0.5)
|
|
.max_peers(10);
|
|
|
|
let network = builder.build();
|
|
assert!(network.nodes.is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn test_get_peers() {
|
|
let mut network = PeerNetwork::new("test");
|
|
|
|
network.add_node(CompanyNode {
|
|
cik: "A".to_string(),
|
|
name: "Company A".to_string(),
|
|
ticker: None,
|
|
sector: Sector::Technology,
|
|
market_cap: None,
|
|
peer_count: 0,
|
|
avg_peer_similarity: 0.0,
|
|
});
|
|
|
|
network.add_node(CompanyNode {
|
|
cik: "B".to_string(),
|
|
name: "Company B".to_string(),
|
|
ticker: None,
|
|
sector: Sector::Technology,
|
|
market_cap: None,
|
|
peer_count: 0,
|
|
avg_peer_similarity: 0.0,
|
|
});
|
|
|
|
network.add_edge(PeerEdge {
|
|
source: "A".to_string(),
|
|
target: "B".to_string(),
|
|
similarity: 0.8,
|
|
relationship_type: PeerRelationType::SameSector,
|
|
weight: 0.8,
|
|
evidence: vec![],
|
|
});
|
|
|
|
let peers = network.get_peers("A");
|
|
assert_eq!(peers.len(), 1);
|
|
assert_eq!(peers[0], "B");
|
|
}
|
|
}
|