mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-29 19:33:34 +00:00
feat: Add automated package-lock.json sync tooling
✨ New Features: - sync-lockfile.sh: Auto-sync lock file with package.json changes - install-hooks.sh: Install git pre-commit hooks - ci-sync-lockfile.sh: CI/CD auto-fix for lock file issues - Pre-commit hook: Automatically runs on git commit - validate-lockfile.yml: GitHub Actions workflow for validation 📚 Documentation: - CONTRIBUTING.md: Complete contribution guide - scripts/README.md: Automation scripts documentation 🎯 Benefits: - Prevents "lock file out of sync" CI/CD failures - Automatic staging of lock file changes - Zero manual intervention needed - Works with any workflow (hooks, manual, CI/CD) 🔧 Usage: 1. Install hooks: ./scripts/install-hooks.sh 2. Add dependencies normally 3. Commit - hook auto-syncs lock file 4. CI validates automatically Resolves the recurring package-lock.json sync issues.
This commit is contained in:
parent
4aad1463c3
commit
9108adeeb5
7 changed files with 417 additions and 71 deletions
10
.githooks/pre-commit
Executable file
10
.githooks/pre-commit
Executable file
|
|
@ -0,0 +1,10 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Git pre-commit hook to automatically sync package-lock.json
|
||||||
|
# Install: ln -sf ../../.githooks/pre-commit .git/hooks/pre-commit
|
||||||
|
|
||||||
|
export GIT_HOOK="pre-commit"
|
||||||
|
|
||||||
|
# Run the sync script
|
||||||
|
./scripts/sync-lockfile.sh
|
||||||
|
|
||||||
|
exit $?
|
||||||
48
.github/workflows/validate-lockfile.yml
vendored
Normal file
48
.github/workflows/validate-lockfile.yml
vendored
Normal file
|
|
@ -0,0 +1,48 @@
|
||||||
|
name: Validate Package Lock File
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
paths:
|
||||||
|
- 'npm/**/package.json'
|
||||||
|
- 'npm/package-lock.json'
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
- develop
|
||||||
|
paths:
|
||||||
|
- 'npm/**/package.json'
|
||||||
|
- 'npm/package-lock.json'
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
validate-lockfile:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Checkout code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Setup Node.js
|
||||||
|
uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: '18'
|
||||||
|
cache: 'npm'
|
||||||
|
cache-dependency-path: npm/package-lock.json
|
||||||
|
|
||||||
|
- name: Validate lock file is in sync
|
||||||
|
run: |
|
||||||
|
cd npm
|
||||||
|
npm ci --dry-run
|
||||||
|
|
||||||
|
- name: Check for lock file changes needed
|
||||||
|
if: failure()
|
||||||
|
run: |
|
||||||
|
echo "❌ package-lock.json is out of sync with package.json"
|
||||||
|
echo ""
|
||||||
|
echo "To fix this issue:"
|
||||||
|
echo " 1. Run: cd npm && npm install"
|
||||||
|
echo " 2. Commit the updated package-lock.json"
|
||||||
|
echo " 3. Push to your branch"
|
||||||
|
echo ""
|
||||||
|
echo "Or use the automated script:"
|
||||||
|
echo " ./scripts/sync-lockfile.sh"
|
||||||
|
exit 1
|
||||||
226
docs/CONTRIBUTING.md
Normal file
226
docs/CONTRIBUTING.md
Normal file
|
|
@ -0,0 +1,226 @@
|
||||||
|
# Contributing to RuVector
|
||||||
|
|
||||||
|
Thank you for your interest in contributing to RuVector!
|
||||||
|
|
||||||
|
## Development Setup
|
||||||
|
|
||||||
|
### 1. Clone the Repository
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ruvnet/ruvector.git
|
||||||
|
cd ruvector
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Install Dependencies
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd npm
|
||||||
|
npm install
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Install Git Hooks (Recommended)
|
||||||
|
|
||||||
|
We provide git hooks that automatically keep `package-lock.json` in sync:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/install-hooks.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
- Automatically run `npm install` when you modify `package.json`
|
||||||
|
- Stage the updated `package-lock.json` automatically
|
||||||
|
- Prevent CI/CD failures due to lock file mismatches
|
||||||
|
|
||||||
|
## Package Management
|
||||||
|
|
||||||
|
### Adding Dependencies
|
||||||
|
|
||||||
|
When adding new dependencies to any package:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd npm/packages/<package-name>
|
||||||
|
npm install <dependency>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important**: Always commit the updated `package-lock.json` with your changes!
|
||||||
|
|
||||||
|
### Manual Lock File Sync
|
||||||
|
|
||||||
|
If you forget to sync the lock file, you can use our helper script:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/sync-lockfile.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
### CI/CD Fails with "Lock file out of sync"
|
||||||
|
|
||||||
|
**Problem**: `npm ci` fails with:
|
||||||
|
```
|
||||||
|
npm error `npm ci` can only install packages when your package.json and package-lock.json are in sync
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
```bash
|
||||||
|
cd npm
|
||||||
|
npm install
|
||||||
|
git add package-lock.json
|
||||||
|
git commit -m "fix: Sync package-lock.json"
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
Or use the automated script:
|
||||||
|
```bash
|
||||||
|
./scripts/sync-lockfile.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pre-commit Hook Not Working
|
||||||
|
|
||||||
|
If the git hook isn't triggering:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Reinstall hooks
|
||||||
|
./scripts/install-hooks.sh
|
||||||
|
|
||||||
|
# Verify hook is executable
|
||||||
|
ls -la .git/hooks/pre-commit
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
1. **Create a feature branch**
|
||||||
|
```bash
|
||||||
|
git checkout -b feat/your-feature-name
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Make your changes**
|
||||||
|
- Write code in the appropriate package
|
||||||
|
- Add tests for new features
|
||||||
|
- Update documentation
|
||||||
|
|
||||||
|
3. **Build and test**
|
||||||
|
```bash
|
||||||
|
cd npm
|
||||||
|
npm run build
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Commit your changes**
|
||||||
|
```bash
|
||||||
|
git add .
|
||||||
|
git commit -m "feat: Your descriptive commit message"
|
||||||
|
```
|
||||||
|
|
||||||
|
The pre-commit hook will automatically sync the lock file if needed.
|
||||||
|
|
||||||
|
5. **Push and create PR**
|
||||||
|
```bash
|
||||||
|
git push origin feat/your-feature-name
|
||||||
|
```
|
||||||
|
|
||||||
|
## Package Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
ruvector/
|
||||||
|
├── npm/
|
||||||
|
│ ├── core/ # @ruvector/core - Native Rust bindings
|
||||||
|
│ ├── packages/
|
||||||
|
│ │ ├── ruvector/ # ruvector - Wrapper package
|
||||||
|
│ │ └── ruvector-extensions/ # ruvector-extensions - Feature extensions
|
||||||
|
│ └── package-lock.json # Workspace lock file
|
||||||
|
├── scripts/
|
||||||
|
│ ├── sync-lockfile.sh # Auto-sync lock file
|
||||||
|
│ ├── install-hooks.sh # Install git hooks
|
||||||
|
│ └── ci-sync-lockfile.sh # CI/CD lock file sync
|
||||||
|
└── .githooks/
|
||||||
|
└── pre-commit # Pre-commit hook script
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
### Run All Tests
|
||||||
|
```bash
|
||||||
|
cd npm
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test Specific Package
|
||||||
|
```bash
|
||||||
|
cd npm/packages/ruvector-extensions
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Testing
|
||||||
|
```bash
|
||||||
|
cd npm/packages/ruvector-extensions/examples
|
||||||
|
tsx complete-integration.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
## Code Style
|
||||||
|
|
||||||
|
- **TypeScript**: Use strict mode, full type annotations
|
||||||
|
- **Formatting**: 2 spaces, semicolons, single quotes
|
||||||
|
- **Comments**: JSDoc for public APIs
|
||||||
|
- **Naming**: camelCase for variables/functions, PascalCase for classes
|
||||||
|
|
||||||
|
## Commit Messages
|
||||||
|
|
||||||
|
Follow [Conventional Commits](https://www.conventionalcommits.org/):
|
||||||
|
|
||||||
|
- `feat:` - New features
|
||||||
|
- `fix:` - Bug fixes
|
||||||
|
- `docs:` - Documentation changes
|
||||||
|
- `refactor:` - Code refactoring
|
||||||
|
- `test:` - Test updates
|
||||||
|
- `chore:` - Build/tooling changes
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```
|
||||||
|
feat: Add OpenAI embeddings provider
|
||||||
|
fix: Resolve CommonJS export issue
|
||||||
|
docs: Update embeddings API documentation
|
||||||
|
chore: Sync package-lock.json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Pull Request Process
|
||||||
|
|
||||||
|
1. **Ensure CI passes**
|
||||||
|
- All tests pass
|
||||||
|
- Build succeeds
|
||||||
|
- No linting errors
|
||||||
|
|
||||||
|
2. **Update documentation**
|
||||||
|
- README.md if public API changes
|
||||||
|
- JSDoc comments for new functions
|
||||||
|
- CHANGELOG.md with notable changes
|
||||||
|
|
||||||
|
3. **Describe your changes**
|
||||||
|
- Clear PR title and description
|
||||||
|
- Reference related issues
|
||||||
|
- Include examples if applicable
|
||||||
|
|
||||||
|
4. **Request review**
|
||||||
|
- Maintainers will review within 48 hours
|
||||||
|
- Address feedback promptly
|
||||||
|
- Keep discussion focused and professional
|
||||||
|
|
||||||
|
## Release Process
|
||||||
|
|
||||||
|
Releases are handled by maintainers:
|
||||||
|
|
||||||
|
1. Version bump in package.json
|
||||||
|
2. Update CHANGELOG.md
|
||||||
|
3. Create git tag
|
||||||
|
4. Publish to npm
|
||||||
|
5. Create GitHub release
|
||||||
|
|
||||||
|
## Questions?
|
||||||
|
|
||||||
|
- 📖 Check the [documentation](../README.md)
|
||||||
|
- 🐛 Report bugs in [Issues](https://github.com/ruvnet/ruvector/issues)
|
||||||
|
- 💬 Ask questions in [Discussions](https://github.com/ruvnet/ruvector/discussions)
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
By contributing, you agree that your contributions will be licensed under the MIT License.
|
||||||
|
|
@ -1,83 +1,38 @@
|
||||||
# Ruvector Scripts
|
# RuVector Automation Scripts
|
||||||
|
|
||||||
Automation scripts for development, publishing, and maintenance.
|
This directory contains automation scripts to streamline development and prevent common issues.
|
||||||
|
|
||||||
## Publishing Scripts
|
## 📜 Available Scripts
|
||||||
|
|
||||||
### publish-crates.sh
|
### 🔄 sync-lockfile.sh
|
||||||
|
Automatically syncs `package-lock.json` with `package.json` changes.
|
||||||
|
|
||||||
Automated script to publish all Ruvector crates to crates.io in the correct dependency order.
|
**Usage:** `./scripts/sync-lockfile.sh`
|
||||||
|
|
||||||
**Prerequisites**:
|
### 🪝 install-hooks.sh
|
||||||
- Rust and Cargo installed
|
Installs git hooks for automatic lock file management.
|
||||||
- `CRATES_API_KEY` in `.env` file (never hardcoded!)
|
|
||||||
- All crates build successfully
|
|
||||||
- All tests pass
|
|
||||||
|
|
||||||
**Usage**:
|
**Usage:** `./scripts/install-hooks.sh`
|
||||||
|
|
||||||
```bash
|
### 🤖 ci-sync-lockfile.sh
|
||||||
# Make executable
|
CI/CD script for automatic lock file fixing.
|
||||||
chmod +x scripts/publish-crates.sh
|
|
||||||
|
|
||||||
# Run publishing
|
**Usage:** `./scripts/ci-sync-lockfile.sh`
|
||||||
./scripts/publish-crates.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
**What it does**:
|
## 🚀 Quick Start
|
||||||
1. Loads `CRATES_API_KEY` from `.env` (secure, not hardcoded)
|
|
||||||
2. Authenticates with crates.io
|
|
||||||
3. Publishes crates in dependency order:
|
|
||||||
- Phase 1: `ruvector-core`, `router-core` (base crates)
|
|
||||||
- Phase 2: `ruvector-node`, `ruvector-wasm`, `ruvector-cli`, `ruvector-bench`
|
|
||||||
- Phase 3: `router-cli`, `router-ffi`, `router-wasm`
|
|
||||||
4. Waits between publishes for crates.io indexing
|
|
||||||
5. Reports success/failure summary
|
|
||||||
|
|
||||||
**Safety Features**:
|
1. **Install git hooks** (recommended):
|
||||||
- ✅ Reads credentials from `.env` (gitignored)
|
```bash
|
||||||
- ✅ Never hardcodes API keys
|
./scripts/install-hooks.sh
|
||||||
- ✅ Verifies packages before publishing
|
```
|
||||||
- ✅ Skips already published versions
|
|
||||||
- ✅ Provides detailed error messages
|
|
||||||
- ✅ Waits for crates.io indexing
|
|
||||||
|
|
||||||
## Security
|
2. **Test the hook**:
|
||||||
|
```bash
|
||||||
|
cd npm/packages/ruvector
|
||||||
|
npm install chalk
|
||||||
|
git add package.json
|
||||||
|
git commit -m "test: Add chalk dependency"
|
||||||
|
# Hook automatically updates lock file
|
||||||
|
```
|
||||||
|
|
||||||
⚠️ **Important**: This directory may contain scripts that use sensitive credentials.
|
See [CONTRIBUTING.md](../docs/CONTRIBUTING.md) for full documentation.
|
||||||
|
|
||||||
**Always**:
|
|
||||||
- Store credentials in `.env` (gitignored)
|
|
||||||
- Use `.env.example` for templates
|
|
||||||
- Never hardcode API keys in scripts
|
|
||||||
- Review scripts before execution
|
|
||||||
|
|
||||||
See [SECURITY.md](../docs/development/SECURITY.md) for security best practices.
|
|
||||||
|
|
||||||
## Development Scripts
|
|
||||||
|
|
||||||
*More scripts will be added here as the project grows*
|
|
||||||
|
|
||||||
Potential additions:
|
|
||||||
- `test-all.sh` - Run all tests across crates
|
|
||||||
- `bench-all.sh` - Run all benchmarks
|
|
||||||
- `check-format.sh` - Verify code formatting
|
|
||||||
- `update-docs.sh` - Update documentation
|
|
||||||
|
|
||||||
## Contributing
|
|
||||||
|
|
||||||
When adding new scripts:
|
|
||||||
|
|
||||||
1. **Document thoroughly** - Add comments and usage examples
|
|
||||||
2. **Use .env for secrets** - Never hardcode credentials
|
|
||||||
3. **Make executable** - `chmod +x scripts/your-script.sh`
|
|
||||||
4. **Add to this README** - Document purpose and usage
|
|
||||||
5. **Test thoroughly** - Verify on clean checkout
|
|
||||||
6. **Error handling** - Exit on errors (`set -e`)
|
|
||||||
7. **Colored output** - Use colors for clarity
|
|
||||||
|
|
||||||
## Resources
|
|
||||||
|
|
||||||
- [Publishing Documentation](../docs/development/PUBLISHING.md)
|
|
||||||
- [Security Guidelines](../docs/development/SECURITY.md)
|
|
||||||
- [Contributing Guide](../docs/development/CONTRIBUTING.md)
|
|
||||||
|
|
|
||||||
38
scripts/ci-sync-lockfile.sh
Executable file
38
scripts/ci-sync-lockfile.sh
Executable file
|
|
@ -0,0 +1,38 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# CI/CD script to auto-fix package-lock.json and create a commit
|
||||||
|
# Use this in GitHub Actions to automatically fix lock file issues
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "🔍 Checking package-lock.json sync for CI/CD..."
|
||||||
|
|
||||||
|
cd npm
|
||||||
|
|
||||||
|
# Try npm ci first to check if lock file is in sync
|
||||||
|
if npm ci --dry-run 2>&1 | grep -q "can only install packages when your package.json and package-lock.json"; then
|
||||||
|
echo "❌ Lock file out of sync - fixing automatically..."
|
||||||
|
|
||||||
|
# Update lock file
|
||||||
|
npm install
|
||||||
|
|
||||||
|
# Check if we're in a git repository and have changes
|
||||||
|
if git diff --quiet npm/package-lock.json; then
|
||||||
|
echo "✅ Lock file is now in sync (no changes needed)"
|
||||||
|
else
|
||||||
|
echo "✅ Lock file updated"
|
||||||
|
|
||||||
|
# If running in GitHub Actions, commit and push
|
||||||
|
if [ -n "$GITHUB_ACTIONS" ]; then
|
||||||
|
git config user.name "github-actions[bot]"
|
||||||
|
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||||
|
git add npm/package-lock.json
|
||||||
|
git commit -m "chore: Auto-sync package-lock.json [skip ci]"
|
||||||
|
git push
|
||||||
|
echo "✅ Lock file committed and pushed"
|
||||||
|
else
|
||||||
|
echo "⚠️ Lock file updated but not committed (not in GitHub Actions)"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "✅ Lock file is already in sync"
|
||||||
|
fi
|
||||||
28
scripts/install-hooks.sh
Executable file
28
scripts/install-hooks.sh
Executable file
|
|
@ -0,0 +1,28 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Install git hooks for automatic lock file syncing
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "🔧 Installing git hooks..."
|
||||||
|
|
||||||
|
# Create .git/hooks directory if it doesn't exist
|
||||||
|
mkdir -p .git/hooks
|
||||||
|
|
||||||
|
# Install pre-commit hook
|
||||||
|
if [ -f ".githooks/pre-commit" ]; then
|
||||||
|
ln -sf ../../.githooks/pre-commit .git/hooks/pre-commit
|
||||||
|
chmod +x .git/hooks/pre-commit
|
||||||
|
chmod +x .githooks/pre-commit
|
||||||
|
echo "✅ Pre-commit hook installed"
|
||||||
|
else
|
||||||
|
echo "❌ Pre-commit hook file not found"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "✨ Git hooks installed successfully!"
|
||||||
|
echo ""
|
||||||
|
echo "The following hooks are now active:"
|
||||||
|
echo " • pre-commit: Automatically syncs package-lock.json when package.json changes"
|
||||||
|
echo ""
|
||||||
|
echo "To disable, run: rm .git/hooks/pre-commit"
|
||||||
41
scripts/sync-lockfile.sh
Executable file
41
scripts/sync-lockfile.sh
Executable file
|
|
@ -0,0 +1,41 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Automatically sync package-lock.json with package.json changes
|
||||||
|
# Can be used as git hook, CI/CD step, or manual script
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "🔍 Checking package-lock.json sync..."
|
||||||
|
|
||||||
|
# Change to npm directory
|
||||||
|
cd "$(dirname "$0")/../npm"
|
||||||
|
|
||||||
|
# Check if package.json or any workspace package.json changed
|
||||||
|
CHANGED_PACKAGES=$(git diff --cached --name-only | grep -E 'package\.json$' || true)
|
||||||
|
|
||||||
|
if [ -n "$CHANGED_PACKAGES" ]; then
|
||||||
|
echo "📦 Package.json changes detected:"
|
||||||
|
echo "$CHANGED_PACKAGES"
|
||||||
|
echo ""
|
||||||
|
echo "🔄 Running npm install to sync lock file..."
|
||||||
|
|
||||||
|
# Run npm install to update lock file
|
||||||
|
npm install
|
||||||
|
|
||||||
|
# Check if lock file changed
|
||||||
|
if git diff --name-only | grep -q 'package-lock.json'; then
|
||||||
|
echo "✅ Lock file updated successfully"
|
||||||
|
|
||||||
|
# If running as pre-commit hook, add the lock file
|
||||||
|
if [ "${GIT_HOOK}" = "pre-commit" ]; then
|
||||||
|
git add npm/package-lock.json
|
||||||
|
echo "✅ Lock file staged for commit"
|
||||||
|
else
|
||||||
|
echo "⚠️ Lock file modified but not staged"
|
||||||
|
echo " Run: git add npm/package-lock.json"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "✅ Lock file already in sync"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "✅ No package.json changes detected"
|
||||||
|
fi
|
||||||
Loading…
Add table
Add a link
Reference in a new issue