mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-25 23:24:03 +00:00
e7f5a391f8
First milestone of the ruvector Python SDK per
docs/sdk/04-milestones.md § "M1 — RaBitQ-only Python wheel". A new
workspace crate `crates/ruvector-py/` exposes ruvector-rabitq as a
Python extension module via PyO3 + maturin with an abi3-py39 wheel
target.
## Surface
```python
import numpy as np
import ruvector
vectors = np.random.randn(10_000, 768).astype(np.float32)
idx = ruvector.RabitqIndex.build(vectors, rerank_factor=20)
results = idx.search(vectors[0], k=10) # → list[(id, distance)]
idx.save("vectors.rbpx")
idx2 = ruvector.RabitqIndex.load("vectors.rbpx")
```
## What ships
- `Cargo.toml`: cdylib crate, pyo3 0.22 with `extension-module` +
`abi3-py39`, numpy 0.22, path dep on `ruvector-rabitq`.
- `pyproject.toml`: maturin build backend, `python-source = "python"`,
`module-name = "ruvector._native"`. PyPI name: `ruvector`.
- `src/lib.rs`: defines the `_native` Python module, registers the
`RabitqIndex` class and `RuVectorError` exception.
- `src/rabitq.rs`: `RabitqIndex` wrapping `RabitqPlusIndex` with
`build` / `search` / `save` / `load` / `__len__` / `__repr__`.
All hot paths release the GIL via `py.allow_threads`.
- `src/error.rs`: maps `RabitqError` → `RuVectorError(PyException)`.
- `python/ruvector/__init__.py`: thin re-export shim from `_native`.
- `python/ruvector/py.typed`: PEP 561 marker.
- Type stubs: `python/ruvector/__init__.pyi` + `stubs/ruvector/__init__.pyi`.
- `tests/test_smoke.py`: pytest coverage of build/search/save/load,
dimension-mismatch error, len/repr, abi3 marker.
- `README.md`: install instructions + 30-second example.
## Real ruvector-rabitq API used
The plan's M1 sketch matched closely. Concrete surface:
- `RabitqPlusIndex::from_vectors_parallel(dim, seed, rerank_factor, items)`
— used in `build()`. Added `seed` kwarg (default 42) since the ctor
requires it.
- `idx.search_with_rerank(query, k, rerank_factor) -> Vec<SearchResult>`
— used in `search()`.
- `persist::save_index` / `persist::load_index` / `persist::MAGIC`
— `.rbpx` v1 wire format. `load()` peeks the 24-byte header to
recover the seed before calling `load_index`.
- `idx.export_items()` — used in `save()` because the seed-based
format needs the items handed back; `RabitqPlusIndex` doesn't
expose `originals_flat` directly.
## Verification
cargo build -p ruvector-py → clean
cargo clippy -p ruvector-py --all-targets --no-deps -- -D warnings → exit 0
cargo test -p ruvector-py → 0 tests, 0 failed (no Rust unit
tests yet; logic is in PyO3
methods that need the Python
interpreter)
`maturin develop` + `pytest` + `mypy --strict` not run — the
sandbox doesn't have those binaries. The Python tests are written
to the M1 acceptance shape and will run as soon as maturin is
present in the dev env.
## Deviations from the M1 plan (docs/sdk/04-milestones.md)
1. One `RabitqIndex` class instead of the plan's four
(`FlatF32Index`, `RabitqIndex`, `RabitqPlusIndex`, `RabitqAsymIndex`).
Adding the others is mechanical follow-up — same register pattern.
2. Single `RuVectorError` exception instead of the subclass tree
(`DimensionMismatch`, `EmptyIndex`, `PersistError`). Subclasses
are M2+ scope per the plan.
3. No `_typing.py`, no `_version.py`. `__version__` sourced from
`env!("CARGO_PKG_VERSION")` via the compiled module.
4. No CI workflow, no Sphinx, no notebook — deferred. Scoped to
"everything needed for pip install to work".
5. `build()` takes a `seed` kwarg (default 42) — not in the M1
sketch but required by the underlying ctor.
## Two pyo3 0.22 quirks worth flagging
- `pyo3::create_exception!` macro emits `cfg(feature = "gil-refs")`
unexpected_cfg warnings. Worked around with `#![allow(unexpected_cfgs)]`
at crate root, comment explains the upstream issue.
- `#[pymethods]` macro expansion triggers
`clippy::useless_conversion` false-positives on `?`-on-PyResult.
Suppressed at crate root with comment.
LoC total: 881 (Cargo.lock excluded; 768 source + 113 lockfile drift).
M1 plan budgeted ~1300 — under because we shipped the user-requested
single-class scope, not the plan's full surface.
Refs: docs/sdk/04-milestones.md M1, docs/sdk/02-strategy.md
Co-Authored-By: claude-flow <ruv@ruv.net>
|
||
|---|---|---|
| .. | ||
| python/ruvector | ||
| src | ||
| stubs/ruvector | ||
| tests | ||
| Cargo.toml | ||
| pyproject.toml | ||
| README.md | ||
ruvector — Python SDK (M1)
Vector similarity search via RaBitQ 1-bit quantization, implemented in Rust
with native NumPy interop. M1 ships exactly one index class —
RabitqIndex — backed by ruvector_rabitq::RabitqPlusIndex (symmetric
1-bit scan + exact f32 rerank).
This crate is the Python wheel half of the ruvector workspace; the
underlying algorithms live in crates/ruvector-rabitq/ and are unchanged
by this binding. The full SDK plan (M1 → M4) is in
docs/sdk/.
Install
Once published to PyPI:
pip install ruvector
For local development from a checkout:
cd crates/ruvector-py
maturin develop --release
pytest tests/
maturin develop builds the Rust cdylib in-place and links it as
ruvector._native so import ruvector works from any Python interpreter
in the active virtualenv. The --release flag matters: a debug build is
~30× slower on the search loop and will fail the latency acceptance test.
30-second example
import numpy as np
import ruvector
# Build an index over 100k random D=128 vectors.
rng = np.random.default_rng(42)
vectors = rng.standard_normal((100_000, 128), dtype=np.float32)
idx = ruvector.RabitqIndex.build(vectors, rerank_factor=20)
# Search the 10 nearest neighbours of a query.
query = vectors[0]
hits = idx.search(query, k=10)
for vid, score in hits:
print(vid, score)
# 0 0.0
# 12345 0.0023
# ...
# Persist and reload.
idx.save("idx.rbpx")
idx2 = ruvector.RabitqIndex.load("idx.rbpx")
assert idx2.search(query, k=10) == hits
API summary
| Call | Returns | Notes |
|---|---|---|
RabitqIndex.build(vectors, *, rerank_factor=20, seed=42) |
RabitqIndex |
vectors: (n, dim) C-contig float32 |
idx.search(query, k, *, rerank_factor=None) |
list[(int, float)] |
(id, score²) ascending; rerank_factor=None uses the build value |
idx.save(path) / RabitqIndex.load(path) |
None / RabitqIndex |
.rbpx v1 format |
len(idx) / idx.dim / idx.memory_bytes / idx.rerank_factor |
int |
diagnostics |
ruvector.RuVectorError |
exception | base of the (future) error tree |
ruvector.__version__ |
str |
mirrors Cargo.toml |
Non-contiguous or wrong-dtype inputs raise TypeError at the boundary
rather than silently copying — predictable beats fast.
Acceptance gates (M1)
Per docs/sdk/04-milestones.md:
pip install ruvector(ormaturin develop) succeeds in <10 s- 100k-vector D=128 search returns in <10 ms (p99 over 100 queries)
- Type stubs validate with
mypy --strict
Links
- SDK plan and milestones — M1 through M4 roadmap
- Binding strategy — why PyO3 + maturin
- API surface sketch — full Python surface
ruvector-rabitq— the Rust crate this wraps
License
Dual MIT / Apache-2.0, matching the rest of the ruvector workspace.