mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-25 15:03:46 +00:00
* feat(postgres): Add 7 advanced AI modules to ruvector-postgres Comprehensive implementation of advanced AI capabilities: ## New Modules (23,541 lines of code) ### 1. Self-Learning / ReasoningBank (`src/learning/`) - Trajectory tracking for query optimization - Pattern extraction using K-means clustering - ReasoningBank for pattern storage and matching - Adaptive search parameter optimization ### 2. Attention Mechanisms (`src/attention/`) - Scaled dot-product attention (core) - Multi-head attention with parallel heads - Flash Attention v2 (memory-efficient) - 10 attention types with PostgresEnum support ### 3. GNN Layers (`src/gnn/`) - Message passing framework - GCN (Graph Convolutional Network) - GraphSAGE with mean/max aggregation - Configurable aggregation methods ### 4. Hyperbolic Embeddings (`src/hyperbolic/`) - Poincaré ball model - Lorentz hyperboloid model - Hyperbolic distance metrics - Möbius operations ### 5. Sparse Vectors (`src/sparse/`) - COO format sparse vector type - Efficient sparse-sparse distance functions - BM25/SPLADE compatible - Top-k pruning operations ### 6. Graph Operations & Cypher (`src/graph/`) - Property graph storage (nodes/edges) - BFS, DFS, Dijkstra traversal - Cypher query parser (AST-based) - Query executor with pattern matching ### 7. Tiny Dancer Routing (`src/routing/`) - FastGRNN neural network - Agent registry with capabilities - Multi-objective routing optimization - Cost/latency/quality balancing ## Docker Infrastructure - Dockerfile with pgrx 0.12.6 and PostgreSQL 16 - docker-compose.yml with test runner - Initialization SQL with test tables - Shell scripts for dev/test/benchmark ## Feature Flags - `learning`, `attention`, `gnn`, `hyperbolic` - `sparse`, `graph`, `routing` - `ai-complete` and `graph-complete` bundles 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(docker): Copy entire workspace for pgrx build 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(docker): Build standalone crate without workspace 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: Update README to enhance clarity and structure * fix(postgres): Resolve compilation errors and Docker build issues - Fix simsimd Option/Result type mismatch in scaled_dot.rs - Fix f32/f64 type conversions in poincare.rs and lorentz.rs - Fix AVX512 missing wrapper functions by using AVX2 fallback - Fix Vec<Vec<f32>> to JsonB for pgrx pg_extern compatibility - Fix DashMap get() to get_mut() for mutable access - Fix router.rs dereference for best_score comparison - Update Dockerfile to copy pre-written SQL file for pgrx - Simplify init.sql to use correct function names - Add postgres-cli npm package for CLI tooling All changes tested successfully in Docker with: - Extension loads with AVX2 SIMD support (8 floats/op) - Distance functions verified working - PostgreSQL 16 container runs successfully 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: Add ruvLLM examples and enhanced postgres-cli Added from claude/ruvector-lfm2-llm-01YS5Tc7i64PyYCLecT9L1dN branch: - examples/ruvLLM: Complete LLM inference system with SIMD optimization - Pretraining, benchmarking, and optimization system - Real SIMD-optimized CPU inference engine - Comprehensive SOTA benchmark suite - Attention mechanisms, memory management, router Enhanced postgres-cli with full ruvector-postgres integration: - Sparse vector operations (BM25, top-k, prune, conversions) - Hyperbolic geometry (Poincare, Lorentz, Mobius operations) - Agent routing (Tiny Dancer system) - Vector quantization (binary, scalar, product) - Enhanced graph and learning commands 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(postgres-cli): Use native ruvector type instead of pgvector - Change createVectorTable to use ruvector type (native RuVector extension) - Add dimensions column for metadata since ruvector is variable-length - Update index creation to use simple btree (HNSW/IVFFlat TBD) - Tested against Docker container with ruvector extension 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat(postgres): Add 53 SQL function definitions for all advanced modules Enable all advanced PostgreSQL extension functions by adding their SQL definitions to the extension file. This exposes all Rust #[pg_extern] functions to PostgreSQL. ## New SQL Functions (53 total) ### Hyperbolic Geometry (8 functions) - ruvector_poincare_distance, ruvector_lorentz_distance - ruvector_mobius_add, ruvector_exp_map, ruvector_log_map - ruvector_poincare_to_lorentz, ruvector_lorentz_to_poincare - ruvector_minkowski_dot ### Sparse Vectors (14 functions) - ruvector_sparse_create, ruvector_sparse_from_dense - ruvector_sparse_dot, ruvector_sparse_cosine, ruvector_sparse_l2_distance - ruvector_sparse_add, ruvector_sparse_scale, ruvector_sparse_to_dense - ruvector_sparse_nnz, ruvector_sparse_dim - ruvector_bm25_score, ruvector_tf_idf, ruvector_sparse_normalize - ruvector_sparse_topk ### GNN - Graph Neural Networks (5 functions) - ruvector_gnn_gcn_layer, ruvector_gnn_graphsage_layer - ruvector_gnn_gat_layer, ruvector_gnn_message_pass - ruvector_gnn_aggregate ### Routing/Agents - "Tiny Dancer" (11 functions) - ruvector_route_query, ruvector_route_with_context - ruvector_calculate_agent_affinity, ruvector_select_best_agent - ruvector_multi_agent_route, ruvector_create_agent_embedding - ruvector_get_routing_stats, ruvector_register_agent - ruvector_update_agent_performance, ruvector_adaptive_route - ruvector_fastgrnn_forward ### Learning/ReasoningBank (7 functions) - ruvector_record_trajectory, ruvector_get_verdict - ruvector_distill_memory, ruvector_adaptive_search - ruvector_learning_feedback, ruvector_get_learning_patterns - ruvector_optimize_search_params ### Graph/Cypher (8 functions) - ruvector_graph_create_node, ruvector_graph_create_edge - ruvector_graph_get_neighbors, ruvector_graph_shortest_path - ruvector_graph_pagerank, ruvector_cypher_query - ruvector_graph_traverse, ruvector_graph_similarity_search ## CLI Updates - Enabled hyperbolic geometry commands in postgres-cli - Added vector distance and normalize commands - Enhanced client with connection pooling and retry logic 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
6.7 KiB
6.7 KiB
RuVector-Postgres Docker Infrastructure
Docker-based development and testing environment for the ruvector-postgres PostgreSQL extension.
Quick Start
Development Environment
# Start development environment
./dev.sh start
# Open psql shell
./dev.sh psql
# Watch for changes and auto-reload
./dev.sh watch
# Stop environment
./dev.sh stop
Running Tests
# Run full test suite
./run-tests.sh
# Run integration tests only
./run-tests.sh --integration
# Keep container running for debugging
./run-tests.sh --keep-running
# Clean rebuild
./run-tests.sh --clean
Scripts Overview
dev.sh - Development Environment
Manages a PostgreSQL development environment with hot-reload support.
Commands:
start- Start development environment (default)stop- Stop development environmentrestart- Restart development environmentlogs- Show PostgreSQL logspsql- Open psql shellwatch- Start file watcher for hot-reload (requires cargo-watch)rebuild- Rebuild and reload extensionstatus- Show container status
Options:
-p, --port PORT- PostgreSQL port (default: 5432)-u, --user USER- PostgreSQL user (default: postgres)-d, --database DB- PostgreSQL database (default: ruvector_dev)-f, --foreground- Start in foreground with logs-h, --help- Show help message
Examples:
# Start on custom port
./dev.sh --port 5433 start
# View logs
./dev.sh logs
# Rebuild extension
./dev.sh rebuild
run-tests.sh - Test Runner
Builds Docker image, runs tests, and manages test infrastructure.
Options:
-b, --build-only- Build Docker image only, don't run tests-t, --test-only- Run tests only (skip build)-i, --integration- Run integration tests only-k, --keep-running- Keep container running after tests-c, --clean- Clean up before starting-v, --keep-volumes- Keep volumes after cleanup-p, --port PORT- PostgreSQL port (default: 5433)-h, --help- Show help message
Examples:
# Build and test
./run-tests.sh
# Integration tests with container kept running
./run-tests.sh --integration --keep-running
# Clean rebuild
./run-tests.sh --clean --build-only
Docker Files
Dockerfile - Main Build File
Multi-stage Docker build for PostgreSQL 16 with pgrx 0.12.6 support.
Features:
- Rust 1.75 with Bookworm base
- PostgreSQL 16 with development headers
- cargo-pgrx 0.12.6 pre-installed
- Optimized layer caching for dependencies
- Health checks built-in
docker-compose.yml - Orchestration
Complete development stack with PostgreSQL and pgAdmin.
Services:
postgres- PostgreSQL 16 with ruvector extensionpgadmin- Web-based database management (port 5050)
Usage:
# Start all services
docker-compose up -d
# View logs
docker-compose logs -f
# Stop services
docker-compose down
# Access pgAdmin
# URL: http://localhost:5050
# Email: admin@ruvector.dev
# Password: admin
init.sql - Database Initialization
SQL script for automatic database setup with:
- Extension creation
- Sample tables and indexes
- Test data
- Performance monitoring views
Development Workflow
1. Initial Setup
# Start development environment
./dev.sh start
# This will:
# - Pull PostgreSQL 16 image
# - Create development database
# - Expose on localhost:5432
# - Show connection string
2. Build Extension
cd /workspaces/ruvector/crates/ruvector-postgres
# Build and install extension
cargo pgrx install --release
3. Test Changes
# Quick test in psql
./dev.sh psql
# In psql:
# CREATE EXTENSION ruvector_postgres;
# SELECT '[1,2,3]'::vector;
4. Hot-Reload Development
# Install cargo-watch (one time)
cargo install cargo-watch
# Start watching for changes
./dev.sh watch
# Now edit code - extension auto-reloads on save!
5. Run Full Test Suite
# Run all tests
./run-tests.sh
# Or run just integration tests
./run-tests.sh --integration
Environment Variables
Development (dev.sh)
POSTGRES_PORT=5432 # PostgreSQL port
POSTGRES_USER=postgres # PostgreSQL user
POSTGRES_PASSWORD=postgres # PostgreSQL password
POSTGRES_DB=ruvector_dev # Database name
Testing (run-tests.sh)
POSTGRES_PORT=5433 # PostgreSQL port (different from dev)
POSTGRES_USER=ruvector # PostgreSQL user
POSTGRES_PASSWORD=ruvector # PostgreSQL password
POSTGRES_DB=ruvector_test # Test database name
KEEP_VOLUMES=false # Keep volumes after cleanup
EXPORT_DB=false # Export database dump
Platform Support
Both scripts support:
- ✅ Linux (Ubuntu, Debian, RHEL, etc.)
- ✅ macOS (Intel and Apple Silicon)
- ✅ Windows (via WSL2)
The scripts automatically detect the platform and adjust behavior accordingly.
Troubleshooting
Port Already in Use
# Check what's using the port
lsof -i :5432
# Use a different port
./dev.sh --port 5433 start
Extension Not Loading
# Rebuild extension
./dev.sh rebuild
# Or manually:
cd /workspaces/ruvector/crates/ruvector-postgres
cargo pgrx install --release
# Then reload in database
./dev.sh psql
# DROP EXTENSION ruvector_postgres CASCADE;
# CREATE EXTENSION ruvector_postgres;
Docker Build Fails
# Clean build
docker system prune -a
./run-tests.sh --clean --build-only
# Check Docker resources
docker info
Tests Fail
# Keep container running to debug
./run-tests.sh --keep-running
# Connect to inspect
./dev.sh psql
# View logs
docker logs ruvector-postgres-test
Performance Tips
Build Optimization
# Use BuildKit for faster builds
export DOCKER_BUILDKIT=1
./run-tests.sh
# Parallel builds
docker build --build-arg MAKEFLAGS="-j$(nproc)" ...
Development Speed
# Use cargo-watch for instant feedback
./dev.sh watch
# Or use cargo-pgrx run for interactive development
cd /workspaces/ruvector/crates/ruvector-postgres
cargo pgrx run pg16
CI/CD Integration
GitHub Actions Example
name: Test RuVector-Postgres
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run tests
run: |
cd crates/ruvector-postgres/docker
./run-tests.sh
GitLab CI Example
test:
image: docker:latest
services:
- docker:dind
script:
- cd crates/ruvector-postgres/docker
- ./run-tests.sh
Resources
License
MIT License - See project root for details