mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-26 16:04:02 +00:00
e7f5a391f8
First milestone of the ruvector Python SDK per
docs/sdk/04-milestones.md § "M1 — RaBitQ-only Python wheel". A new
workspace crate `crates/ruvector-py/` exposes ruvector-rabitq as a
Python extension module via PyO3 + maturin with an abi3-py39 wheel
target.
## Surface
```python
import numpy as np
import ruvector
vectors = np.random.randn(10_000, 768).astype(np.float32)
idx = ruvector.RabitqIndex.build(vectors, rerank_factor=20)
results = idx.search(vectors[0], k=10) # → list[(id, distance)]
idx.save("vectors.rbpx")
idx2 = ruvector.RabitqIndex.load("vectors.rbpx")
```
## What ships
- `Cargo.toml`: cdylib crate, pyo3 0.22 with `extension-module` +
`abi3-py39`, numpy 0.22, path dep on `ruvector-rabitq`.
- `pyproject.toml`: maturin build backend, `python-source = "python"`,
`module-name = "ruvector._native"`. PyPI name: `ruvector`.
- `src/lib.rs`: defines the `_native` Python module, registers the
`RabitqIndex` class and `RuVectorError` exception.
- `src/rabitq.rs`: `RabitqIndex` wrapping `RabitqPlusIndex` with
`build` / `search` / `save` / `load` / `__len__` / `__repr__`.
All hot paths release the GIL via `py.allow_threads`.
- `src/error.rs`: maps `RabitqError` → `RuVectorError(PyException)`.
- `python/ruvector/__init__.py`: thin re-export shim from `_native`.
- `python/ruvector/py.typed`: PEP 561 marker.
- Type stubs: `python/ruvector/__init__.pyi` + `stubs/ruvector/__init__.pyi`.
- `tests/test_smoke.py`: pytest coverage of build/search/save/load,
dimension-mismatch error, len/repr, abi3 marker.
- `README.md`: install instructions + 30-second example.
## Real ruvector-rabitq API used
The plan's M1 sketch matched closely. Concrete surface:
- `RabitqPlusIndex::from_vectors_parallel(dim, seed, rerank_factor, items)`
— used in `build()`. Added `seed` kwarg (default 42) since the ctor
requires it.
- `idx.search_with_rerank(query, k, rerank_factor) -> Vec<SearchResult>`
— used in `search()`.
- `persist::save_index` / `persist::load_index` / `persist::MAGIC`
— `.rbpx` v1 wire format. `load()` peeks the 24-byte header to
recover the seed before calling `load_index`.
- `idx.export_items()` — used in `save()` because the seed-based
format needs the items handed back; `RabitqPlusIndex` doesn't
expose `originals_flat` directly.
## Verification
cargo build -p ruvector-py → clean
cargo clippy -p ruvector-py --all-targets --no-deps -- -D warnings → exit 0
cargo test -p ruvector-py → 0 tests, 0 failed (no Rust unit
tests yet; logic is in PyO3
methods that need the Python
interpreter)
`maturin develop` + `pytest` + `mypy --strict` not run — the
sandbox doesn't have those binaries. The Python tests are written
to the M1 acceptance shape and will run as soon as maturin is
present in the dev env.
## Deviations from the M1 plan (docs/sdk/04-milestones.md)
1. One `RabitqIndex` class instead of the plan's four
(`FlatF32Index`, `RabitqIndex`, `RabitqPlusIndex`, `RabitqAsymIndex`).
Adding the others is mechanical follow-up — same register pattern.
2. Single `RuVectorError` exception instead of the subclass tree
(`DimensionMismatch`, `EmptyIndex`, `PersistError`). Subclasses
are M2+ scope per the plan.
3. No `_typing.py`, no `_version.py`. `__version__` sourced from
`env!("CARGO_PKG_VERSION")` via the compiled module.
4. No CI workflow, no Sphinx, no notebook — deferred. Scoped to
"everything needed for pip install to work".
5. `build()` takes a `seed` kwarg (default 42) — not in the M1
sketch but required by the underlying ctor.
## Two pyo3 0.22 quirks worth flagging
- `pyo3::create_exception!` macro emits `cfg(feature = "gil-refs")`
unexpected_cfg warnings. Worked around with `#![allow(unexpected_cfgs)]`
at crate root, comment explains the upstream issue.
- `#[pymethods]` macro expansion triggers
`clippy::useless_conversion` false-positives on `?`-on-PyResult.
Suppressed at crate root with comment.
LoC total: 881 (Cargo.lock excluded; 768 source + 113 lockfile drift).
M1 plan budgeted ~1300 — under because we shipped the user-requested
single-class scope, not the plan's full surface.
Refs: docs/sdk/04-milestones.md M1, docs/sdk/02-strategy.md
Co-Authored-By: claude-flow <ruv@ruv.net>
91 lines
2.9 KiB
Markdown
91 lines
2.9 KiB
Markdown
# ruvector — Python SDK (M1)
|
||
|
||
Vector similarity search via RaBitQ 1-bit quantization, implemented in Rust
|
||
with native NumPy interop. M1 ships exactly one index class —
|
||
`RabitqIndex` — backed by `ruvector_rabitq::RabitqPlusIndex` (symmetric
|
||
1-bit scan + exact f32 rerank).
|
||
|
||
This crate is the Python wheel half of the ruvector workspace; the
|
||
underlying algorithms live in `crates/ruvector-rabitq/` and are unchanged
|
||
by this binding. The full SDK plan (M1 → M4) is in
|
||
[`docs/sdk/`](../../docs/sdk/).
|
||
|
||
## Install
|
||
|
||
Once published to PyPI:
|
||
|
||
```sh
|
||
pip install ruvector
|
||
```
|
||
|
||
For local development from a checkout:
|
||
|
||
```sh
|
||
cd crates/ruvector-py
|
||
maturin develop --release
|
||
pytest tests/
|
||
```
|
||
|
||
`maturin develop` builds the Rust cdylib in-place and links it as
|
||
`ruvector._native` so `import ruvector` works from any Python interpreter
|
||
in the active virtualenv. The `--release` flag matters: a debug build is
|
||
~30× slower on the search loop and will fail the latency acceptance test.
|
||
|
||
## 30-second example
|
||
|
||
```python
|
||
import numpy as np
|
||
import ruvector
|
||
|
||
# Build an index over 100k random D=128 vectors.
|
||
rng = np.random.default_rng(42)
|
||
vectors = rng.standard_normal((100_000, 128), dtype=np.float32)
|
||
idx = ruvector.RabitqIndex.build(vectors, rerank_factor=20)
|
||
|
||
# Search the 10 nearest neighbours of a query.
|
||
query = vectors[0]
|
||
hits = idx.search(query, k=10)
|
||
for vid, score in hits:
|
||
print(vid, score)
|
||
# 0 0.0
|
||
# 12345 0.0023
|
||
# ...
|
||
|
||
# Persist and reload.
|
||
idx.save("idx.rbpx")
|
||
idx2 = ruvector.RabitqIndex.load("idx.rbpx")
|
||
assert idx2.search(query, k=10) == hits
|
||
```
|
||
|
||
## API summary
|
||
|
||
| Call | Returns | Notes |
|
||
|---|---|---|
|
||
| `RabitqIndex.build(vectors, *, rerank_factor=20, seed=42)` | `RabitqIndex` | `vectors`: `(n, dim)` C-contig `float32` |
|
||
| `idx.search(query, k, *, rerank_factor=None)` | `list[(int, float)]` | `(id, score²)` ascending; `rerank_factor=None` uses the build value |
|
||
| `idx.save(path)` / `RabitqIndex.load(path)` | `None` / `RabitqIndex` | `.rbpx` v1 format |
|
||
| `len(idx)` / `idx.dim` / `idx.memory_bytes` / `idx.rerank_factor` | `int` | diagnostics |
|
||
| `ruvector.RuVectorError` | exception | base of the (future) error tree |
|
||
| `ruvector.__version__` | `str` | mirrors `Cargo.toml` |
|
||
|
||
Non-contiguous or wrong-dtype inputs raise `TypeError` at the boundary
|
||
rather than silently copying — predictable beats fast.
|
||
|
||
## Acceptance gates (M1)
|
||
|
||
Per `docs/sdk/04-milestones.md`:
|
||
|
||
1. `pip install ruvector` (or `maturin develop`) succeeds in <10 s
|
||
2. 100k-vector D=128 search returns in <10 ms (p99 over 100 queries)
|
||
3. Type stubs validate with `mypy --strict`
|
||
|
||
## Links
|
||
|
||
- [SDK plan and milestones](../../docs/sdk/) — M1 through M4 roadmap
|
||
- [Binding strategy](../../docs/sdk/02-strategy.md) — why PyO3 + maturin
|
||
- [API surface sketch](../../docs/sdk/03-api-surface.md) — full Python surface
|
||
- [`ruvector-rabitq`](../ruvector-rabitq/) — the Rust crate this wraps
|
||
|
||
## License
|
||
|
||
Dual MIT / Apache-2.0, matching the rest of the ruvector workspace.
|