mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-24 05:43:58 +00:00
eb1227047d
Comprehensive implementation of advanced AI capabilities: ## New Modules (23,541 lines of code) ### 1. Self-Learning / ReasoningBank (`src/learning/`) - Trajectory tracking for query optimization - Pattern extraction using K-means clustering - ReasoningBank for pattern storage and matching - Adaptive search parameter optimization ### 2. Attention Mechanisms (`src/attention/`) - Scaled dot-product attention (core) - Multi-head attention with parallel heads - Flash Attention v2 (memory-efficient) - 10 attention types with PostgresEnum support ### 3. GNN Layers (`src/gnn/`) - Message passing framework - GCN (Graph Convolutional Network) - GraphSAGE with mean/max aggregation - Configurable aggregation methods ### 4. Hyperbolic Embeddings (`src/hyperbolic/`) - Poincaré ball model - Lorentz hyperboloid model - Hyperbolic distance metrics - Möbius operations ### 5. Sparse Vectors (`src/sparse/`) - COO format sparse vector type - Efficient sparse-sparse distance functions - BM25/SPLADE compatible - Top-k pruning operations ### 6. Graph Operations & Cypher (`src/graph/`) - Property graph storage (nodes/edges) - BFS, DFS, Dijkstra traversal - Cypher query parser (AST-based) - Query executor with pattern matching ### 7. Tiny Dancer Routing (`src/routing/`) - FastGRNN neural network - Agent registry with capabilities - Multi-objective routing optimization - Cost/latency/quality balancing ## Docker Infrastructure - Dockerfile with pgrx 0.12.6 and PostgreSQL 16 - docker-compose.yml with test runner - Initialization SQL with test tables - Shell scripts for dev/test/benchmark ## Feature Flags - `learning`, `attention`, `gnn`, `hyperbolic` - `sparse`, `graph`, `routing` - `ai-complete` and `graph-complete` bundles 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
350 lines
6.7 KiB
Markdown
350 lines
6.7 KiB
Markdown
# RuVector-Postgres Docker Infrastructure
|
|
|
|
Docker-based development and testing environment for the ruvector-postgres PostgreSQL extension.
|
|
|
|
## Quick Start
|
|
|
|
### Development Environment
|
|
|
|
```bash
|
|
# Start development environment
|
|
./dev.sh start
|
|
|
|
# Open psql shell
|
|
./dev.sh psql
|
|
|
|
# Watch for changes and auto-reload
|
|
./dev.sh watch
|
|
|
|
# Stop environment
|
|
./dev.sh stop
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Run full test suite
|
|
./run-tests.sh
|
|
|
|
# Run integration tests only
|
|
./run-tests.sh --integration
|
|
|
|
# Keep container running for debugging
|
|
./run-tests.sh --keep-running
|
|
|
|
# Clean rebuild
|
|
./run-tests.sh --clean
|
|
```
|
|
|
|
## Scripts Overview
|
|
|
|
### `dev.sh` - Development Environment
|
|
|
|
Manages a PostgreSQL development environment with hot-reload support.
|
|
|
|
**Commands:**
|
|
- `start` - Start development environment (default)
|
|
- `stop` - Stop development environment
|
|
- `restart` - Restart development environment
|
|
- `logs` - Show PostgreSQL logs
|
|
- `psql` - Open psql shell
|
|
- `watch` - Start file watcher for hot-reload (requires cargo-watch)
|
|
- `rebuild` - Rebuild and reload extension
|
|
- `status` - Show container status
|
|
|
|
**Options:**
|
|
- `-p, --port PORT` - PostgreSQL port (default: 5432)
|
|
- `-u, --user USER` - PostgreSQL user (default: postgres)
|
|
- `-d, --database DB` - PostgreSQL database (default: ruvector_dev)
|
|
- `-f, --foreground` - Start in foreground with logs
|
|
- `-h, --help` - Show help message
|
|
|
|
**Examples:**
|
|
```bash
|
|
# Start on custom port
|
|
./dev.sh --port 5433 start
|
|
|
|
# View logs
|
|
./dev.sh logs
|
|
|
|
# Rebuild extension
|
|
./dev.sh rebuild
|
|
```
|
|
|
|
### `run-tests.sh` - Test Runner
|
|
|
|
Builds Docker image, runs tests, and manages test infrastructure.
|
|
|
|
**Options:**
|
|
- `-b, --build-only` - Build Docker image only, don't run tests
|
|
- `-t, --test-only` - Run tests only (skip build)
|
|
- `-i, --integration` - Run integration tests only
|
|
- `-k, --keep-running` - Keep container running after tests
|
|
- `-c, --clean` - Clean up before starting
|
|
- `-v, --keep-volumes` - Keep volumes after cleanup
|
|
- `-p, --port PORT` - PostgreSQL port (default: 5433)
|
|
- `-h, --help` - Show help message
|
|
|
|
**Examples:**
|
|
```bash
|
|
# Build and test
|
|
./run-tests.sh
|
|
|
|
# Integration tests with container kept running
|
|
./run-tests.sh --integration --keep-running
|
|
|
|
# Clean rebuild
|
|
./run-tests.sh --clean --build-only
|
|
```
|
|
|
|
## Docker Files
|
|
|
|
### `Dockerfile` - Main Build File
|
|
|
|
Multi-stage Docker build for PostgreSQL 16 with pgrx 0.12.6 support.
|
|
|
|
**Features:**
|
|
- Rust 1.75 with Bookworm base
|
|
- PostgreSQL 16 with development headers
|
|
- cargo-pgrx 0.12.6 pre-installed
|
|
- Optimized layer caching for dependencies
|
|
- Health checks built-in
|
|
|
|
### `docker-compose.yml` - Orchestration
|
|
|
|
Complete development stack with PostgreSQL and pgAdmin.
|
|
|
|
**Services:**
|
|
- `postgres` - PostgreSQL 16 with ruvector extension
|
|
- `pgadmin` - Web-based database management (port 5050)
|
|
|
|
**Usage:**
|
|
```bash
|
|
# Start all services
|
|
docker-compose up -d
|
|
|
|
# View logs
|
|
docker-compose logs -f
|
|
|
|
# Stop services
|
|
docker-compose down
|
|
|
|
# Access pgAdmin
|
|
# URL: http://localhost:5050
|
|
# Email: admin@ruvector.dev
|
|
# Password: admin
|
|
```
|
|
|
|
### `init.sql` - Database Initialization
|
|
|
|
SQL script for automatic database setup with:
|
|
- Extension creation
|
|
- Sample tables and indexes
|
|
- Test data
|
|
- Performance monitoring views
|
|
|
|
## Development Workflow
|
|
|
|
### 1. Initial Setup
|
|
|
|
```bash
|
|
# Start development environment
|
|
./dev.sh start
|
|
|
|
# This will:
|
|
# - Pull PostgreSQL 16 image
|
|
# - Create development database
|
|
# - Expose on localhost:5432
|
|
# - Show connection string
|
|
```
|
|
|
|
### 2. Build Extension
|
|
|
|
```bash
|
|
cd /workspaces/ruvector/crates/ruvector-postgres
|
|
|
|
# Build and install extension
|
|
cargo pgrx install --release
|
|
```
|
|
|
|
### 3. Test Changes
|
|
|
|
```bash
|
|
# Quick test in psql
|
|
./dev.sh psql
|
|
|
|
# In psql:
|
|
# CREATE EXTENSION ruvector_postgres;
|
|
# SELECT '[1,2,3]'::vector;
|
|
```
|
|
|
|
### 4. Hot-Reload Development
|
|
|
|
```bash
|
|
# Install cargo-watch (one time)
|
|
cargo install cargo-watch
|
|
|
|
# Start watching for changes
|
|
./dev.sh watch
|
|
|
|
# Now edit code - extension auto-reloads on save!
|
|
```
|
|
|
|
### 5. Run Full Test Suite
|
|
|
|
```bash
|
|
# Run all tests
|
|
./run-tests.sh
|
|
|
|
# Or run just integration tests
|
|
./run-tests.sh --integration
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
### Development (`dev.sh`)
|
|
|
|
```bash
|
|
POSTGRES_PORT=5432 # PostgreSQL port
|
|
POSTGRES_USER=postgres # PostgreSQL user
|
|
POSTGRES_PASSWORD=postgres # PostgreSQL password
|
|
POSTGRES_DB=ruvector_dev # Database name
|
|
```
|
|
|
|
### Testing (`run-tests.sh`)
|
|
|
|
```bash
|
|
POSTGRES_PORT=5433 # PostgreSQL port (different from dev)
|
|
POSTGRES_USER=ruvector # PostgreSQL user
|
|
POSTGRES_PASSWORD=ruvector # PostgreSQL password
|
|
POSTGRES_DB=ruvector_test # Test database name
|
|
KEEP_VOLUMES=false # Keep volumes after cleanup
|
|
EXPORT_DB=false # Export database dump
|
|
```
|
|
|
|
## Platform Support
|
|
|
|
Both scripts support:
|
|
- ✅ Linux (Ubuntu, Debian, RHEL, etc.)
|
|
- ✅ macOS (Intel and Apple Silicon)
|
|
- ✅ Windows (via WSL2)
|
|
|
|
The scripts automatically detect the platform and adjust behavior accordingly.
|
|
|
|
## Troubleshooting
|
|
|
|
### Port Already in Use
|
|
|
|
```bash
|
|
# Check what's using the port
|
|
lsof -i :5432
|
|
|
|
# Use a different port
|
|
./dev.sh --port 5433 start
|
|
```
|
|
|
|
### Extension Not Loading
|
|
|
|
```bash
|
|
# Rebuild extension
|
|
./dev.sh rebuild
|
|
|
|
# Or manually:
|
|
cd /workspaces/ruvector/crates/ruvector-postgres
|
|
cargo pgrx install --release
|
|
|
|
# Then reload in database
|
|
./dev.sh psql
|
|
# DROP EXTENSION ruvector_postgres CASCADE;
|
|
# CREATE EXTENSION ruvector_postgres;
|
|
```
|
|
|
|
### Docker Build Fails
|
|
|
|
```bash
|
|
# Clean build
|
|
docker system prune -a
|
|
./run-tests.sh --clean --build-only
|
|
|
|
# Check Docker resources
|
|
docker info
|
|
```
|
|
|
|
### Tests Fail
|
|
|
|
```bash
|
|
# Keep container running to debug
|
|
./run-tests.sh --keep-running
|
|
|
|
# Connect to inspect
|
|
./dev.sh psql
|
|
|
|
# View logs
|
|
docker logs ruvector-postgres-test
|
|
```
|
|
|
|
## Performance Tips
|
|
|
|
### Build Optimization
|
|
|
|
```bash
|
|
# Use BuildKit for faster builds
|
|
export DOCKER_BUILDKIT=1
|
|
./run-tests.sh
|
|
|
|
# Parallel builds
|
|
docker build --build-arg MAKEFLAGS="-j$(nproc)" ...
|
|
```
|
|
|
|
### Development Speed
|
|
|
|
```bash
|
|
# Use cargo-watch for instant feedback
|
|
./dev.sh watch
|
|
|
|
# Or use cargo-pgrx run for interactive development
|
|
cd /workspaces/ruvector/crates/ruvector-postgres
|
|
cargo pgrx run pg16
|
|
```
|
|
|
|
## CI/CD Integration
|
|
|
|
### GitHub Actions Example
|
|
|
|
```yaml
|
|
name: Test RuVector-Postgres
|
|
|
|
on: [push, pull_request]
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v3
|
|
- name: Run tests
|
|
run: |
|
|
cd crates/ruvector-postgres/docker
|
|
./run-tests.sh
|
|
```
|
|
|
|
### GitLab CI Example
|
|
|
|
```yaml
|
|
test:
|
|
image: docker:latest
|
|
services:
|
|
- docker:dind
|
|
script:
|
|
- cd crates/ruvector-postgres/docker
|
|
- ./run-tests.sh
|
|
```
|
|
|
|
## Resources
|
|
|
|
- [pgrx Documentation](https://github.com/pgcentralfoundation/pgrx)
|
|
- [PostgreSQL Docker Hub](https://hub.docker.com/_/postgres)
|
|
- [RuVector Repository](https://github.com/ruvnet/ruvector)
|
|
|
|
## License
|
|
|
|
MIT License - See project root for details
|