mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-08-31 10:15:39 +00:00
3e8429739f
* chore(rvf): sync Cargo.lock with rvf-wire deps (sha3, subtle) https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * fix(rvf-runtime): deterministic tie-breaking in query result ordering Equal-distance vectors were selected and ordered by HashMap iteration order, which changes across process restarts and made query results non-reproducible (flaky smoke_rvlite_adapter_persistence). Break ties by vector id in both the top-k heap eviction and the final sort, in query() and query_with_envelope(). https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * perf(rvf): optimize index/runtime hot paths, fix quant codec and manifest discovery rvf-index: - Cache SIMD distance-kernel dispatch in a OnceLock function-pointer table instead of re-running is_x86_feature_detected! on every call - Rewrite HNSW search_layer with BinaryHeap min/max-heaps (was sorted Vec + O(n) mid-inserts) and a dense Vec<bool> visited bitmap (was a per-call SipHash HashSet); deterministic (distance, id) tie-breaking rvf-runtime: - Replace per-bit CRC32 loops with crc32fast (same IEEE polynomial, byte-identical hashes, ~100x faster) on segment write and verify - Hoist cosine query-norm computation out of the per-vector scan loop - Safety-net scan: single pass with HashSet membership (was O(k*N*neighbors) with Vec::contains) - Bulk little-endian f32 serialization in write_vec_seg (one memcpy per vector instead of per-element appends) - Progressively widen the manifest tail scan (64KB -> 1MB -> 16MB -> whole file): stores with large segment directories were becoming unreadable once the latest manifest fell outside the fixed 64KB window; with regression test rvf-quant: - encode_quant_seg now emits fully decodable payloads (delegates to the real scalar/product encoders; placeholders removed) - decode_quant_seg returns Result instead of panicking on malformed or unknown-type payloads; round-trip and malformed-input tests https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * fix(rvm): bind witness chain to record content; optimize coherence, cap, sched hot paths rvm-witness (security-critical): - The chain hash covered only (prev_hash, sequence) — record content (action, actor, target, payload, timestamp) was never hashed, so verify_chain accepted arbitrarily rewritten history. record_hash is now computed over the 44 content bytes (as its doc always claimed) and the chain binds it: H(prev || seq || record_hash). verify_chain recomputes content hashes; tamper-regression tests added. - HMAC signer keys the Mac template once at construction instead of re-running the key schedule per record (fixed-vector test pins signature bytes) - Witness ring overflow is now observable: total_overwritten counter and needs_drain() accessor rvm-cap: - Nonce replay window: colliding nonces (A + k*4096) could evict and re-admit nonce A. Replaced the two 32KB arrays with one 32KB open-addressed table (8-probe bounded); eviction raises the watermark so it fails closed. Regression test included. rvm-coherence: - internal_weight: O(MAX_EDGES) self-loop scan replaced with O(1) adj_matrix[i][i] read (invariant verified across all mutation paths) - Skip ticks return a cached CoherenceDecision instead of re-running the O(n^2) merge-pair pass over stale data; zero-weight pairs skipped - Mincut: scratch buffers moved into the long-lived bridge (~17KB less stack per call), in-place Stoer-Wagner (no working copy), bitmask membership, column-scan in-neighbors - Compile-time guard: CoherenceGraph MAX_NODES > ADJ_DIM now fails to compile instead of panicking at the 33rd node; u64 weight deltas clamped at the engine boundary rvm-coherence/rvm-partition: - Single-slot hash indexes (id_to_node, edge_index) degraded to permanent O(N) scans after any collision; both now use bounded linear probing with tombstones and probe-proven absence rvm-sched: - enqueue() rejects the HYPERVISOR sentinel id, which previously wedged a run-queue slot permanently; defensive cleanup in switch_next Tests: 733 workspace + 67 rvm-kernel lib pass (baseline 712); 23 new tests including tamper-evidence and collision regressions. https://claude.ai/code/session_01C83hbozEXPgoz9iJN5Smhp * feat(rvf): wire HNSW index into the runtime query path (~14x speedup) RvfStore::query was a brute-force O(N*dim) scan; the rvf-index crate was unused by production queries and QualityEnvelope.evidence fabricated layer_a=true. The index is now built lazily on first eligible query, maintained incrementally on ingest, persisted on close() via the existing INDEX_SEG codec (with a versioned, backward-readable trailer for the sparse-id mapping), and validated-or-rebuilt on open. Exact scan remains for small stores (<1024), filtered/COW/membership queries, >25% deleted, and force_exact; deterministic (distance, id) tie-breaking preserved on both paths. evidence.layer_a is now set only when the index served the query. Measured: 21.7ms -> 1.51ms per query at 100k x 64-dim (criterion, release), recall@10 = 0.968 at the ef_search=256 floor (>=0.95 gated by test). +15 tests (recall, index persistence round-trip, evidence honesty, fallback routing, compaction/overwrite invalidation). Co-Authored-By: claude-flow <ruv@ruv.net> * feat(rvm): witness v2 — keyed-BLAKE3 chain with 128-bit links + Merkle sealing v1 records folded chain links to 32 bits and left the head unanchorable. The 96-byte v2 record embeds the predecessor MAC full-width and chains via one keyed-BLAKE3 compression per append (~112ns measured, 9x under the 1us target); keyed MACs detect last-record tampering and unkeyed forgery, which v1 could not. Segment sealing accumulates record MACs into a domain-separated Merkle tree (256/segment) sealed with one signature via the existing signer infra (HMAC/dual-HMAC/Ed25519/TEE), with inclusion proofs — expensive crypto moves off the per-record path and roots are externally anchorable. v1 logs still verify (version-byte dispatch; v1 only as prefix, head anchored into the first v2 record); v1 writing is frozen. blake3 added as pure-Rust no_std. +46 tests covering content/reorder/truncation/wrong-key /forgery tamper modes, v1 compat, proofs, seals, and mixed logs. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(rvm): wire mincut into split decisions; honest partition-switch claim execute_split previously created an empty child and ignored the computed cut. It now resolves the boundary from a cached epoch SplitPlan (or computes on demand) and re-homes move-side neighbors to the child with their edge weights. Two-tier decisions: exact Stoer-Wagner mincut runs as a pressure-triggered epoch task; a new Fennel placer (O(degree), fixed-point gamma=1.5, no_std) handles hot-path placement. Split policy combines pressure and cut quality: mid-band (8000-9500bp) splits only on a cut with conductance <= 5000bp; critical pressure stays an unconditional safety valve. The sub-10us partition-switch claim was a stub certified by a no-op bench (~6ns) reported as 1600x faster than target. The real path needs EL2 assembly the crate forbids; instead the measurable register save/restore lower bound is implemented and benchmarked, the bench is renamed partition_switch_validation_stub with an honesty gate, a canary test fails if HARDWARE_SWITCH_IMPLEMENTED flips without revisiting the claim, and the README row now reads: not validated. +30 tests. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(rvf): RaBitQ binary quantization + Vamana alpha-pruning (opt-in) rvf-quant gains a RaBitQ-style codec: global-centroid centering, 3-round seeded randomized-Hadamard rotation (orthonormal, reproducible from a stored u64 seed), 1-bit sign codes with per-vector norm/dot-correction scalars, and an asymmetric full-precision-query estimator. QUANT_SEG adds versioned type tag 4 (legacy payloads byte-frozen and still decode; unknown versions rejected; decode stays panic-free on untrusted bytes). Query path: opt-in two-stage search (QueryOptions::rabitq, default off) — estimator scan with oversampling (640-candidate floor) then exact f32 rescore; deterministic (distance, id) tie-breaking; falls back to default routing for filtered/COW/IP/cosine queries. Measured recall@10 = 0.972 vs exact on 10k x 128 (gate >= 0.95, test-enforced); code-only compression exactly 32x. rvf-index: Vamana-style robust prune (alpha = 1.2, occluded backfill) at insert and prune time; recall@10 at ef=30 improved 0.986 -> 0.996; construction determinism preserved. +42 tests (1254 passing, no new failures). Co-Authored-By: claude-flow <ruv@ruv.net> * fix(rvf): harden untrusted decode paths against crafted-file DoS An adversarial audit confirmed a crafted .rvf could panic or OOM the process on RvfStore::open(): unvalidated length fields drove Vec::with_capacity before any byte-availability check. decode_payload now bounds id_count by available delta bytes (u64 compare before the usize cast, so 32-bit truncation cannot bypass it); decode_index_seg bounds restart_count/layer_count/neighbor_count by remaining bytes and rejects truncated restart padding (was a reachable slice panic); decode_sketch_seg converts from assert-and-panic to Result with width/depth validated via checked_mul (closes the width=0 + depth=u32::MAX bypass); decode_product size products use checked u64 arithmetic so 32-bit (wasm32) targets cannot wrap usize and read out of bounds. +8 adversarial regression tests. Co-Authored-By: claude-flow <ruv@ruv.net> * perf(rvf): contiguous vector slab, non-blocking index rebuild, unified hashing Vector storage moves from HashMap<u64, Vec<f32>> to a contiguous row-major slab (id->ordinal map, tombstoned deletes, slot reuse only via compaction); HNSW/RaBitQ paths read rows as zero-copy slices and iteration is ordinal- ordered (deterministic across restarts). Brute-force query at 100k x 64: 24.5ms -> 3.8ms (~6.4x). boot() pre-sizes the slab and bulk-copies VEC_SEG payloads (no per-vector allocs): cold open 257ms -> 202ms (-21.5%). mmap deferred (CRC verify touches all bytes anyway; memmap2 not in this workspace) and documented as follow-up. Audit finding 5: index/RaBitQ lazy builds now run with no lock held behind an AtomicBool gate (panic-safe clear-on-drop); concurrent queries fall back to exact scan and keep serving through the entire O(N log N) build. Overwrite still invalidates and unlinks the stale INDEX_SEG. Hashing: the two identical bespoke CRC32-rotation implementations in write_path/read_path now delegate to one source of truth (hashing::legacy_content_hash); on-disk bytes unchanged. Full rvf-wire checksum-registry conformance (XXH3-128 + format-version bump + dual-accept reader) documented as the remaining delta. read_path.rs also carries the audit''s checked vec-seg size arithmetic. +11 tests; suite 1271 passing, no new failures (one pre-existing wall-clock bench assertion flakes under load, passes in isolation). Co-Authored-By: claude-flow <ruv@ruv.net> * chore(release): prepare rvf 0.2.1/0.2.0/0.3.0 crate bumps, npm 0.2.2/0.1.7, measured-benchmark READMEs - rvf-types 0.2.0 -> 0.2.1 (QuantType::RaBitQ format extension) - rvf-index 0.1.0 -> 0.2.0 (Vamana alpha-pruning, hardened INDEX_SEG codec) - rvf-quant 0.1.0 -> 0.2.0 (RaBitQ codec; decode_sketch_seg now returns Result) - rvf-runtime 0.2.0 -> 0.3.0 (HNSW query path, INDEX_SEG trailer, QueryOptions::rabitq, vector slab) - dependent path-dep version reqs updated (cli, import, launch, node, server) - @ruvector/rvf 0.2.0 -> 0.2.2, @ruvector/rvf-wasm 0.1.6 -> 0.1.7 (rebuilt wasm artifact, 1.89 toolchain + wasm-opt -Oz) - READMEs: HNSW/RaBitQ/slab docs with measured numbers (Windows x64, criterion release, 100k x 64-dim); rvm witness v2 bench rows Co-Authored-By: claude-flow <ruv@ruv.net> * fix(robotics): bump rvf-runtime requirement to 0.3 after release bump The rvf-runtime 0.2 -> 0.3.0 version bump updated dependents inside the rvf workspace but missed the root-workspace consumer: ruvector-robotics pins version 0.2 alongside its path dep, which fails cargo resolution against the bumped crate (PR #555 CI: failed to select a version for the requirement rvf-runtime ^0.2). Root Cargo.lock refreshed. Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: ruv <ruvnet@users.noreply.github.com>
479 lines
25 KiB
Markdown
479 lines
25 KiB
Markdown
# RVM — The Virtual Machine Built for the Agentic Age
|
||
|
||
[](https://www.rust-lang.org)
|
||
[](https://doc.rust-lang.org/reference/names/preludes.html)
|
||
[](LICENSE)
|
||
[](../../docs/adr/)
|
||
[](https://github.com/ruvnet/RuVector/issues/328)
|
||
|
||
### **Agents don't fit in VMs. They need something that understands how they think.**
|
||
|
||
> Part of the [RuVector](https://github.com/ruvnet/RuVector) ecosystem. Uses [RuVix](../../crates/ruvix/) kernel primitives and [RVF](../../crates/rvf/) package format. Designed for [Cognitum](https://cognitum.one) Seed, Appliance, and future chip targets.
|
||
|
||
Traditional hypervisors were built for an era of static server workloads —
|
||
long-running VMs with predictable resource needs. AI agents are different.
|
||
They spawn in milliseconds, communicate in dense, shifting graphs, share
|
||
context across trust boundaries, and die without warning. VMs are the wrong
|
||
abstraction.
|
||
|
||
RVM replaces VMs with **coherence domains** — lightweight, graph-structured
|
||
partitions whose isolation, scheduling, and memory placement are driven by how
|
||
agents actually communicate. When two agents start talking more, RVM moves
|
||
them closer. When trust drops, RVM splits them apart. Every mutation is
|
||
proof-gated. Every action is witnessed. The system *understands* its own
|
||
structure.
|
||
|
||
```
|
||
Agent swarm → [RVM Coherence Engine] → Optimal Placement → Witness Proof
|
||
↑ │
|
||
└──── Agent Communication Graph ─────────────┘
|
||
(< 50µs adaptive re-partitioning)
|
||
```
|
||
|
||
**No KVM. No Linux. No VMs. Bare-metal Rust. Built for agents.**
|
||
|
||
```
|
||
Traditional VM: VM₁ VM₂ VM₃ VM₄ (static, opaque boxes — agents don't fit)
|
||
─────────────────────
|
||
RVM: ┌─A──B─┐ ┌─C─┐ D (dynamic, agent-driven domains)
|
||
│ ↔ │──│ ↔ │──↔ (edges = agent communication weight)
|
||
└──────┘ └───┘ (auto-split when trust or coupling changes)
|
||
```
|
||
|
||
### What Agents Need vs What They Get
|
||
|
||
| What Agents Need | VMs / Containers | RVM |
|
||
|-----------------|-----------------|-----|
|
||
| Sub-millisecond spawn | Seconds to boot | < 10µs partition switch |
|
||
| Dense, shifting comms graph | Static NIC-to-NIC | Graph-weighted CommEdges, auto-rebalanced |
|
||
| Shared context with isolation | All or nothing | Capability-gated shared memory, proof-checked |
|
||
| Per-agent fault containment | Whole-VM crash | F1–F4 graduated rollback, no reboot needed |
|
||
| Auditable every action | External log bolted on | 64-byte witness on every syscall, hash-chained |
|
||
| Hibernate and reconstruct | Kill and restart | Dormant tier → rebuilt from witness log |
|
||
| Run on 64KB MCUs | Needs gigabytes | Seed profile: 64KB–1MB, capability-enforced |
|
||
|
||
---
|
||
|
||
## Why RVM?
|
||
|
||
**Dynamic Re-isolation and Self-Healing Boundaries.** Because RVM uses
|
||
graph-theoretic mincut algorithms, it can dynamically restructure its isolation
|
||
boundaries to match how workloads actually communicate. If an agent in one
|
||
partition begins communicating heavily with an agent in another, RVM
|
||
automatically triggers a partition split and migrates the agent to optimise
|
||
placement — no manual configuration. No existing hypervisor can split or merge
|
||
live partitions along a graph-theoretic cut boundary.
|
||
|
||
**Memory Time Travel and Deep Forensics.** Traditional virtual memory
|
||
permanently overwrites state or blindly swaps it to disk. RVM stores dormant
|
||
memory as a checkpoint combined with a delta-compressed witness trail. Any
|
||
historical state can be perfectly rebuilt on demand — days or weeks later —
|
||
because every privileged action is recorded in a tamper-evident, hash-chained
|
||
witness log. External forensic tools can reconstruct past states to answer
|
||
precise questions such as "which task mutated this vector store between 14:00
|
||
and 14:05 on Tuesday?"
|
||
|
||
**Targeted Fault Rollback Without Global Reboots.** When the kernel detects a
|
||
coherence violation or memory corruption it does not crash. Instead it finds
|
||
the last known-good checkpoint, replays the witness log, explicitly skips the
|
||
mutation that caused the failure, and resumes from a corrected state (DC-14,
|
||
failure classes F1–F3).
|
||
|
||
**Deterministic Multi-Tenant Edge Orchestration.** Existing edge orchestrators
|
||
rely on Linux-based VMs or containers, inheriting scheduling unpredictability
|
||
and no guarantee of bounded latency with provable isolation. RVM enables
|
||
scenarios such as an autonomous vehicle where safety-critical sensor-fusion
|
||
agents (Reflex mode, < 10 µs switch) are strictly isolated from low-priority
|
||
infotainment agents, or a smart factory floor running hard real-time PLC
|
||
control loops safely alongside ML inference agents.
|
||
|
||
**High-Assurance Security on Extreme Microcontrollers.** Through its Seed
|
||
hardware profile (ADR-138), RVM brings capability-enforced isolation,
|
||
proof-gated execution, and witness attestation to deeply constrained IoT
|
||
devices with as little as 64 KB of RAM. Delivering this level of zero-trust,
|
||
auditable security on microcontroller-class hardware is a novel capability not
|
||
provided by any existing embedded operating system.
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
+----------------------------------------------------------+
|
||
| rvm-kernel |
|
||
| |
|
||
| +-----------+ +-----------+ +------------+ |
|
||
| | rvm-boot | | rvm-sched | | rvm-memory | |
|
||
| +-----+-----+ +-----+-----+ +------+-----+ |
|
||
| | | | |
|
||
| +-----+--------------+---------------+------+ |
|
||
| | rvm-partition | |
|
||
| +-----+---------+-----------+----------+----+ |
|
||
| | | | | |
|
||
| +-----+--+ +---+------+ +--+-----+ +--+--------+ |
|
||
| | rvm-cap| |rvm-witness| |rvm-proof| |rvm-security| |
|
||
| +-----+--+ +---+------+ +--+-----+ +--+--------+ |
|
||
| | | | | |
|
||
| +-----+---------+-----------+----------+----+ |
|
||
| | rvm-types | |
|
||
| +-----+-------------------------------------+ |
|
||
| | |
|
||
| +-----+--+ +----------+ +-------------+ |
|
||
| | rvm-hal| | rvm-wasm | |rvm-coherence| |
|
||
| +--------+ +----------+ +-------------+ |
|
||
+----------------------------------------------------------+
|
||
```
|
||
|
||
```
|
||
Layer 4: Persistent State
|
||
witness log │ compressed dormant memory │ RVF checkpoints
|
||
─────────────────────────────────────────────────────────
|
||
Layer 3: Execution Adapters
|
||
bare partition │ WASM partition │ service adapter
|
||
─────────────────────────────────────────────────────────
|
||
Layer 2: Coherence Engine (OPTIONAL — DC-1)
|
||
graph state │ mincut │ pressure scoring │ migration
|
||
─────────────────────────────────────────────────────────
|
||
Layer 1: RVM Core (Rust, no_std)
|
||
partitions │ capabilities │ scheduler │ witnesses
|
||
─────────────────────────────────────────────────────────
|
||
Layer 0: Machine Entry (assembly, <500 LoC)
|
||
reset vector │ trap handlers │ context switch
|
||
```
|
||
|
||
### First-Class Kernel Objects
|
||
|
||
| Object | Purpose |
|
||
|--------|---------|
|
||
| **Partition** | Coherence domain container — unit of scheduling, isolation, and migration |
|
||
| **Capability** | Unforgeable authority token with 7 rights (READ, WRITE, GRANT, REVOKE, EXECUTE, PROVE, GRANT_ONCE) |
|
||
| **Witness** | 64-byte hash-chained audit record emitted by every privileged action |
|
||
| **MemoryRegion** | Typed, tiered, owned memory (Hot/Warm/Dormant/Cold) with move semantics |
|
||
| **CommEdge** | Inter-partition communication channel — weighted edge in the coherence graph |
|
||
| **DeviceLease** | Time-bounded, revocable hardware device access |
|
||
| **CoherenceScore** | Graph-derived locality and coupling metric |
|
||
| **CutPressure** | Isolation signal — high pressure triggers migration or split |
|
||
| **RecoveryCheckpoint** | State snapshot for rollback and reconstruction |
|
||
|
||
---
|
||
|
||
## Crate Structure
|
||
|
||
| Crate | Purpose |
|
||
|-------|---------|
|
||
| `rvm-types` | Foundation types: addresses, IDs, capabilities, witness records, coherence scores |
|
||
| `rvm-hal` | Platform-agnostic hardware abstraction traits (MMU, timer, interrupts) |
|
||
| `rvm-cap` | Capability-based access control with derivation trees and three-tier proof |
|
||
| `rvm-witness` | Append-only witness trail with hash-chain integrity |
|
||
| `rvm-proof` | Proof-gated state transitions (P1/P2/P3 tiers), TEE pipeline, cryptographic signers (Ed25519, HMAC-SHA256) |
|
||
| `rvm-partition` | Partition lifecycle, split/merge, capability tables, communication edges |
|
||
| `rvm-sched` | Coherence-weighted 2-signal scheduler (deadline urgency + cut pressure) |
|
||
| `rvm-memory` | Guest physical address space management with tiered placement |
|
||
| `rvm-coherence` | Unified coherence engine: graph, mincut, scoring, pressure, adaptive, pluggable backends, edge decay |
|
||
| `rvm-boot` | Deterministic 7-phase boot sequence with witness gating |
|
||
| `rvm-wasm` | Optional WebAssembly guest runtime |
|
||
| `rvm-security` | Unified security gate: capability check + proof verification + witness log |
|
||
| `rvm-kernel` | Full integration: coherence engine, IPC→graph feeding, scheduler, split/merge, security gates, tier management |
|
||
|
||
### Dependency Graph
|
||
|
||
```
|
||
rvm-types (foundation, no deps)
|
||
├── rvm-hal
|
||
├── rvm-cap
|
||
├── rvm-witness
|
||
├── rvm-proof ← rvm-cap + rvm-witness
|
||
├── rvm-partition ← rvm-hal + rvm-cap + rvm-witness
|
||
├── rvm-sched ← rvm-partition + rvm-witness
|
||
├── rvm-memory ← rvm-hal + rvm-partition + rvm-witness
|
||
├── rvm-coherence ← rvm-partition + rvm-sched [OPTIONAL]
|
||
├── rvm-boot ← rvm-hal + rvm-partition + rvm-witness + rvm-sched + rvm-memory
|
||
├── rvm-wasm ← rvm-partition + rvm-cap + rvm-witness [OPTIONAL]
|
||
├── rvm-security ← rvm-cap + rvm-proof + rvm-witness
|
||
└── rvm-kernel ← ALL
|
||
```
|
||
|
||
---
|
||
|
||
## Build
|
||
|
||
```bash
|
||
# Check (no_std by default)
|
||
cargo check
|
||
|
||
# Run all 648 tests
|
||
cargo test --workspace --lib
|
||
|
||
# Run 21 criterion benchmarks
|
||
cargo bench
|
||
|
||
# Build with std support
|
||
cargo check --features std
|
||
|
||
# Cross-compile for AArch64 bare-metal
|
||
rustup target add aarch64-unknown-none
|
||
make build # or: cargo build --target aarch64-unknown-none -p rvm-kernel --release
|
||
|
||
# Boot on QEMU (requires qemu-system-aarch64)
|
||
make run # boots at 0x4000_0000, PL011 UART output
|
||
```
|
||
|
||
---
|
||
|
||
## Design Constraints (ADR-132 through ADR-140)
|
||
|
||
| ID | Constraint | Status |
|
||
|----|-----------|--------|
|
||
| DC-1 | Coherence engine is optional; system degrades gracefully | **Implemented** — adaptive engine, static fallback |
|
||
| DC-2 | MinCut budget: 50 µs per epoch | **Implemented** — Stoer-Wagner with iteration budget, ~331ns measured |
|
||
| DC-3 | Capabilities are unforgeable, monotonically attenuated | **Implemented** — constant-time P1, 4096-nonce ring |
|
||
| DC-4 | 2-signal priority: `deadline_urgency + cut_pressure_boost` | **Implemented** |
|
||
| DC-5 | Three systems cleanly separated (kernel + coherence + agents) | **Enforced** — feature-gated |
|
||
| DC-6 | Degraded mode when coherence unavailable | **Implemented** — enter/exit with witnesses, scheduler zeroes CutPressure |
|
||
| DC-7 | Migration timeout enforcement (100 ms) | **Implemented** — MigrationTracker with auto-abort |
|
||
| DC-8 | Capabilities follow objects during partition split | **Implemented** — scored region assignment |
|
||
| DC-9 | Coherence score range [0.0, 1.0] as fixed-point | **Implemented** — u16 basis points |
|
||
| DC-10 | Epoch-based witness batching (no per-switch records) | **Implemented** |
|
||
| DC-11 | Merge requires coherence above threshold + adjacency + resources | **Implemented** — 3-check validation |
|
||
| DC-12 | Max 256 physical VMIDs, multiplexed for >256 partitions | **Implemented** |
|
||
| DC-13 | WASM is optional; native bare partitions are first class | **Enforced** |
|
||
| DC-14 | Failure classes: transient, recoverable, permanent, catastrophic | **Implemented** — F1-F4 with escalation |
|
||
| DC-15 | All types are `no_std`, `forbid(unsafe_code)`, `deny(missing_docs)` | **Enforced** |
|
||
|
||
---
|
||
|
||
## Benchmarks
|
||
|
||
| Operation | ADR Target | Measured | Ratio |
|
||
|-----------|-----------|---------|-------|
|
||
| Witness emit (v1, legacy verify-only format) | < 500 ns | **~17 ns** | 29x faster |
|
||
| Witness v2 append (96 B record, keyed BLAKE3 chain MAC) | < 1 µs | **~112 ns** | 8x faster |
|
||
| Witness v2 segment seal (256 records, Merkle root + sign) | — | **~61 µs** | one signature per segment, off the per-record path |
|
||
| P1 capability verify | < 1 µs | **< 1 ns** | >1000x faster |
|
||
| P2 proof pipeline | < 100 µs | **~996 ns** | 100x faster |
|
||
| Partition switch | < 10 µs | **not validated** | hardware hot path unimplemented (`HARDWARE_SWITCH_IMPLEMENTED == false`); host benches measure only run-queue selection and register-file copy, which are lower bounds |
|
||
| MinCut 16-node | < 50 µs | **~331 ns** | 150x faster |
|
||
| Coherence score (16-node) | budgeted | **~84 ns** | — |
|
||
| Buddy alloc/free cycle | fast | **~184 ns** | — |
|
||
| FNV-1a hash (64 bytes) | fast | **~28 ns** | — |
|
||
| Security gate P1 | fast | **~17 ns** | — |
|
||
| Witness chain verify (64 records) | fast | **~892 ns** | — |
|
||
|
||
Run `cargo bench` for full criterion results with HTML reports.
|
||
|
||
## Implementation Status
|
||
|
||
| Crate | Tests | Key Features |
|
||
|-------|-------|-------------|
|
||
| `rvm-types` | ~40 types | 64-byte `WitnessRecord` (compile-time asserted), ~40 `ActionKind` variants, 34 error variants |
|
||
| `rvm-hal` | 16 | AArch64 EL2: stage-2 page tables, PL011 UART, GICv2, ARM generic timer |
|
||
| `rvm-cap` | 40 | Constant-time P1, nonce ring (4096 + watermark), P3 derivation chain verification, epoch revocation |
|
||
| `rvm-witness` | 29 | SHA-256 hash chain (FNV-1a fallback), HMAC-SHA256 signing, 16MB ring buffer, `StrictSigner`, RLE-compressed replay |
|
||
| `rvm-proof` | 45 | Proof engine, context builder, constant-time P2 (all 6 rules), P3 deep verification (SHA-256 + Merkle + WitnessSigner), TEE pipeline, Ed25519/HMAC-SHA256/DualHmac signers |
|
||
| `rvm-partition` | 86 | Lifecycle state machine, IPC message queues, device leases, scored split/merge, `remove()` |
|
||
| `rvm-sched` | 49 | 2-signal priority, SMP coordinator, VMID-aware switch, `SwitchContext::init()`, degraded fallback |
|
||
| `rvm-memory` | 110 | Buddy allocator with coalescing, 4-tier management, LZ4-style RLE compression, reconstruction |
|
||
| `rvm-coherence` | 59 | Unified coherence engine, pluggable MinCut/Coherence backends, edge decay, bridge to ruvector |
|
||
| `rvm-boot` | 26 | 7-phase measured boot, attestation digest, HAL init, entry point |
|
||
| `rvm-wasm` | 33 | 7-state agent lifecycle, `HostContext` trait, section parser (13 section types), migration |
|
||
| `rvm-security` | 45 | Unified security gate (P1/P2/P3), `SignedSecurityGate` with per-link signature verification, input validation, attestation chain, DMA budget |
|
||
| `rvm-kernel` | 62 | Full integration: IPC→coherence, scheduler, split/merge, security gates, degraded mode, device leases, tier mgmt |
|
||
| **Integration** | 48 | 17 e2e scenarios: agent lifecycle, split pressure, memory tiers, cap chain, boot timing |
|
||
| **Benchmarks** | 21 | Criterion benchmarks for all performance-critical paths |
|
||
| **Total** | **648** | **0 failures, 0 clippy warnings** |
|
||
|
||
### Security Audit Results
|
||
|
||
11 findings from formal security review, 8 fixed in code:
|
||
|
||
| Severity | Finding | Status |
|
||
|----------|---------|--------|
|
||
| Critical | P1 timing side channel | **Fixed** — constant-time bitmask |
|
||
| High | Revocation didn't invalidate descendants | **Fixed** — iterative subtree sync |
|
||
| High | Cross-partition host memory overlap | **Fixed** — global overlap check |
|
||
| Medium | Generation counter wrap aliasing | **Fixed** — skip gen 0 |
|
||
| Medium | next_id overflow | **Fixed** — checked_add |
|
||
| Medium | Recursive revoke stack overflow | **Fixed** — iterative stack |
|
||
| Medium | Incomplete merge preconditions | **Fixed** — full validation |
|
||
| Low | Terminated agent slots never freed | **Fixed** — set None |
|
||
| Medium | Nonce ring too small (64) | **Fixed** — upgraded to 4096 + watermark |
|
||
| Medium | TOCTOU in quota check | **Fixed** — atomic check_and_record |
|
||
| Low | NullSigner always-true | **Fixed** — StrictSigner + deprecation |
|
||
|
||
---
|
||
|
||
<details>
|
||
<summary><b>🔍 RVM vs State of the Art (12 differences)</b></summary>
|
||
|
||
| | RVM | KVM/Firecracker | seL4 | Theseus OS |
|
||
|---|---|---|---|---|
|
||
| **Primary abstraction** | Coherence domains (graph-partitioned) | Virtual machines | Processes + capabilities | Cells (intralingual) |
|
||
| **Isolation driver** | Dynamic mincut + cut pressure | Hardware EPT/NPT | Formal verification + caps | Rust type system |
|
||
| **Scheduling signal** | Structural coherence (graph metrics) | CPU time / fairness | Priority / round-robin | Cooperative |
|
||
| **Memory model** | 4-tier reconstructable (Hot/Warm/Dormant/Cold) | Demand paging | Untyped memory + retype | Single address space |
|
||
| **Audit trail** | Witness-native (64B hash-chained records) | External logging | Not built-in | Not built-in |
|
||
| **Mutation control** | Proof-gated (3-layer: P1/P2/P3) | Unix permissions | Capability tokens | Rust ownership |
|
||
| **Partition operations** | Live split/merge along graph cuts | Not supported | Not supported | Not supported |
|
||
| **Linux dependency** | None — bare-metal | Yes (KVM is a kernel module) | None | None |
|
||
| **Language** | 95-99% Rust, <500 LoC assembly | C | C + Isabelle/HOL proofs | Rust |
|
||
| **Target** | Edge, IoT, agents | Cloud servers | Safety-critical | Research |
|
||
| **Boot time** | < 250ms to first witness | ~125ms (Firecracker) | Varies | N/A |
|
||
| **Partition switch** | < 10µs | ~2-5µs (VM exit) | ~0.5-1µs (IPC) | N/A (no isolation) |
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>✨ 6 Novel Capabilities (No Prior Art)</b></summary>
|
||
|
||
### 1. Kernel-Level Graph Control Loop
|
||
No existing OS uses spectral graph coherence metrics as a scheduling signal. RVM's coherence engine runs mincut algorithms in the kernel's scheduling loop — graph structure directly drives where computation runs, when partitions split, and which memory stays resident.
|
||
|
||
### 2. Reconstructable Memory ("Memory Time Travel")
|
||
RVM explicitly rejects demand paging. Dormant memory is stored as `witness checkpoint + delta compression`, not raw bytes. The system can deterministically reconstruct any historical state from the witness log.
|
||
|
||
### 3. Proof-Gated Infrastructure
|
||
Every state mutation requires a valid proof token verified through a three-tier system: P1 capability (<1µs), P2 policy (<100µs), P3 deep derivation chain verification (walks tree to root, validates ancestor integrity + epoch monotonicity).
|
||
|
||
### 4. Witness-Native OS
|
||
Every privileged action emits a fixed 64-byte, SHA-256 hash-chained record with HMAC-SHA256 signatures. Tamper-evident by construction. Full deterministic replay from any checkpoint.
|
||
|
||
### 5. Live Partition Split/Merge
|
||
Partitions split along graph-theoretic cut boundaries and merge when coherence rises. Capabilities follow ownership (DC-8), regions use weighted scoring (DC-9), merges require 7 preconditions (DC-11).
|
||
|
||
### 6. Edge Security on 64KB RAM
|
||
Capability-based isolation, proof-gated execution, and witness attestation on microcontroller-class hardware (Cortex-M/R, 64KB RAM).
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>🎯 Success Criteria (v1)</b></summary>
|
||
|
||
| # | Criterion | Target |
|
||
|---|-----------|--------|
|
||
| 1 | All 13 crates compile with `#![no_std]` and `#![forbid(unsafe_code)]` | Enforced |
|
||
| 2 | Cold boot to first witness | < 250ms on Appliance hardware |
|
||
| 3 | Hot partition switch | < 10 microseconds |
|
||
| 4 | Witness record is exactly 64 bytes, cache-line aligned | Compile-time asserted |
|
||
| 5 | Capability derivation depth bounded at 8 levels | Enforced |
|
||
| 6 | EMA coherence filter operates without floating-point | Implemented |
|
||
| 7 | Boot sequence is deterministic and witness-gated | Implemented |
|
||
| 8 | Remote memory traffic reduction ≥ 20% vs naive placement | Target |
|
||
| 9 | Fault recovery without global reboot (F1–F3) | Target |
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>🏗️ Implementation Phases</b></summary>
|
||
|
||
### Phase 1: Foundation (M0-M1) — "Can it boot and isolate?"
|
||
- **M0**: Bare-metal Rust boot on QEMU AArch64 virt. Reset → EL2 → serial → MMU → first witness.
|
||
- **M1**: Partition + capability model. Create, destroy, switch. Simple deadline scheduler.
|
||
|
||
### Phase 2: Differentiation (M2-M3) — "Can it prove and witness?"
|
||
- **M2**: Witness logging (64-byte chained records) + P1/P2 proof verifier.
|
||
- **M3**: 2-signal scheduler (deadline + cut_pressure). Flow + Reflex modes. Zero-copy IPC.
|
||
|
||
### Phase 3: Innovation (M4-M5) — "Can it think about coherence?"
|
||
- **M4**: Dynamic mincut integration (DC-2 budget). Live coherence graph. Migration triggers.
|
||
- **M5**: Memory tier management. Reconstruction from dormant state.
|
||
|
||
### Phase 4: Expansion (M6-M7) — "Can agents run on it?"
|
||
- **M6**: WASM agent runtime adapter. Agent lifecycle.
|
||
- **M7**: Seed/Appliance hardware bring-up. All success criteria.
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>🔐 Security Model</b></summary>
|
||
|
||
**Capability-Based Authority.** All access controlled through unforgeable kernel-resident tokens. No ambient authority. Seven rights with monotonic attenuation.
|
||
|
||
**Proof-Gated Mutation.** No memory remap, device mapping, migration, or partition merge without a valid proof token. Three tiers with strict latency budgets.
|
||
|
||
**Witness-Native Audit.** 64-byte records for every mutating operation. Hash-chained for tamper evidence. Deterministic replay from checkpoint + witness log.
|
||
|
||
**Failure Classification.** F1 (agent restart) → F2 (partition reconstruct) → F3 (memory rollback) → F4 (kernel reboot). Each escalation witnessed.
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>🖥️ Target Platforms</b></summary>
|
||
|
||
| Platform | Profile | RAM | Coherence Engine | WASM |
|
||
|----------|---------|-----|-----------------|------|
|
||
| **Seed** | Tiny, persistent, event-driven | 64KB–1MB | No (DC-1) | Optional |
|
||
| **Appliance** | Edge hub, deterministic orchestration | 1–32GB | Yes (full) | Yes |
|
||
| **Chip** | Future Cognitum silicon | Tile-local | Hardware-assisted | Yes |
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>📚 ADR References</b></summary>
|
||
|
||
| ADR | Topic |
|
||
|-----|-------|
|
||
| ADR-132 | RVM top-level architecture and 15 design constraints |
|
||
| ADR-133 | Partition object model and split/merge semantics |
|
||
| ADR-134 | Witness schema and log format (64-byte records) |
|
||
| ADR-135 | Three-tier proof system (P1/P2/P3) |
|
||
| ADR-136 | Memory hierarchy and reconstruction |
|
||
| ADR-137 | Bare-metal boot sequence |
|
||
| ADR-138 | Seed hardware bring-up |
|
||
| ADR-139 | Appliance deployment model |
|
||
| ADR-140 | Agent runtime adapter |
|
||
| ADR-141 | Coherence engine kernel integration and runtime pipeline |
|
||
| ADR-142 | TEE-backed cryptographic verification (SHA-256, Ed25519, HMAC-SHA256, TEE pipeline) |
|
||
|
||
</details>
|
||
|
||
<details>
|
||
<summary><b>🔧 Development</b></summary>
|
||
|
||
### Prerequisites
|
||
|
||
- Rust 1.77+ with `aarch64-unknown-none` target
|
||
- QEMU 8.0+ (for AArch64 virt machine emulation)
|
||
|
||
```bash
|
||
rustup target add aarch64-unknown-none
|
||
brew install qemu # macOS
|
||
```
|
||
|
||
### Project Conventions
|
||
|
||
- `#![no_std]` everywhere — the kernel runs on bare metal
|
||
- `#![forbid(unsafe_code)]` where possible; `unsafe` blocks audited and commented
|
||
- `#![deny(missing_docs)]` — every public API documented
|
||
- Move semantics for memory ownership (`OwnedRegion<P>` is non-copyable)
|
||
- Const generics for fixed-size structures (no heap allocation in kernel paths)
|
||
- Every state mutation emits a witness record
|
||
|
||
</details>
|
||
|
||
---
|
||
|
||
## RuVector Integration
|
||
|
||
| Crate | Role in RVM |
|
||
|-------|-------------|
|
||
| [`ruvector-mincut`](../../crates/ruvector-mincut/) | Partition placement and isolation decisions |
|
||
| [`ruvector-sparsifier`](../../crates/ruvector-sparsifier/) | Compressed shadow graph for Laplacian operations |
|
||
| [`ruvector-solver`](../../crates/ruvector-solver/) | Effective resistance → coherence scores |
|
||
| [`ruvector-coherence`](../../crates/ruvector-coherence/) | Spectral coherence tracking |
|
||
| [`ruvix-*`](../../crates/ruvix/) | Kernel primitives (Task, Capability, Region, Queue, Timer, Proof) |
|
||
| [`rvf`](../../crates/rvf/) | Package format for boot images, checkpoints, and cold storage |
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
Licensed under either of:
|
||
|
||
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or <http://www.apache.org/licenses/LICENSE-2.0>)
|
||
- MIT License ([LICENSE-MIT](LICENSE-MIT) or <http://opensource.org/licenses/MIT>)
|
||
|
||
at your option.
|
||
|
||
---
|
||
|
||
<sub>[EPIC](https://github.com/ruvnet/RuVector/issues/328) · [Research Gist](https://gist.github.com/ruvnet/8082d0b339f05e73cf48b491de5b8ee6) · [pi.ruv.io Brain](https://pi.ruv.io)</sub>
|