Commit graph

112 commits

Author SHA1 Message Date
Claude
ac10792ad0 docs: Deep enhance SOTA research analysis and optimization guide
SOTA Research Analysis (265→568 lines):
- Added 5 new breakthroughs: spectral density estimation, faster effective
  resistance, neural acceleration via sublinear layers, distributed Laplacian
  solvers, sketching-based matrix approximation
- Added randomized numerical linear algebra foundations (Martinsson-Tropp)
- New section: implementation complexity analysis with LOC, difficulty ratings
- New section: error analysis with information-theoretic lower bounds
- New section: hardware evolution impact (M4 Pro, Zen 5, SVE2, RISC-V, CXL)
- Expanded competitive landscape with 8+ vector DBs and academic systems
- New section: research integration roadmap (short/medium/long-term)
- Expanded bibliography to 30 references

Optimization Guide (378→502 lines):
- Added AVX-512 masked SpMV kernel and WASM SIMD128 kernel
- New section: numerical optimization (Kahan summation, mixed precision)
- New section: WASM-specific optimization (memory growth, wasm-opt, workers)
- New section: profiling methodology (perf, flamegraph, roofline model)
- Enhanced optimization checklist with impact/effort/validation columns

https://claude.ai/code/session_01TiqLbr2DaNAntQHaVeLfiR
2026-02-20 04:35:48 +00:00
Claude
7cb1090ccd docs: Add comprehensive ADR and DDD documentation for sublinear-time solver
Add 15 architecture and design documents covering the sublinear-time solver
integration into RuVector's 79-crate ecosystem:

ADR Documents (12):
- ADR-STS-001: Core integration architecture with trait hierarchy and event sourcing
- ADR-STS-002: Algorithm selection and sublinear routing with SONA adaptive learning
- ADR-STS-003: Memory management strategy with arena allocator and HNSW integration
- ADR-STS-004: WASM and cross-platform compilation with SIMD per architecture
- ADR-STS-005: Security model with STRIDE/DREAD analysis and witness chain audit
- ADR-STS-006: Benchmark framework with 6 Criterion.rs suites and CI regression
- ADR-STS-007: Feature flag and progressive rollout strategy
- ADR-STS-008: Error handling and fault tolerance with fallback chains
- ADR-STS-009: Concurrency model with Rayon+SIMD two-level parallelism
- ADR-STS-010: API surface design for Rust/WASM/NAPI/REST/MCP
- SOTA research analysis surveying 20+ papers and competitive landscape
- Optimization guide with SIMD/memory/algorithm/platform strategies

DDD Documents (3):
- Strategic design: 6 bounded contexts, context map, ubiquitous language
- Tactical design: aggregates, entities, value objects, domain services
- Integration patterns: ACLs, shared kernel, published language, event-driven

https://claude.ai/code/session_01TiqLbr2DaNAntQHaVeLfiR
2026-02-20 04:15:46 +00:00
Claude
e18e914810 docs: Add quantum + sublinear solver convergence analysis
Maps 8 convergence points across 5 quantum crates (ruqu-core,
ruqu-algorithms, ruQu, ruqu-exotic, ruqu-wasm) and the sublinear solver:

1. VQE warm-starting from sublinear eigenvector estimates (10x fewer iterations)
2. QAOA spectral parameter initialization via Laplacian eigenvalues
3. Sparse tensor network contraction (100x faster MPS simulation)
4. QEC syndrome decoding via sublinear graph matching (<1us target)
5. Coherence gate enhancement with predictive spectral analysis
6. Interference search with O(log n) amplitude propagation
7. Quantum-classical boundary optimization (automatic resource allocation)
8. DNA→protein→Hamiltonian→VQE triple convergence for drug discovery

Includes quantum advantage map showing where quantum vs sublinear wins.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 03:07:02 +00:00
Claude
0467e5e715 docs: Add DNA + sublinear solver convergence analysis
Maps 7 concrete integration points between rvDNA genomics suite and
sublinear-time-solver: protein contact graph PageRank (500x speedup),
sparse attention solve in RVDNA format, joint variant calling with LD
(+15-30% sensitivity), sublinear Horvath clock regression, HNSW graph
optimization for pangenome k-mer search, network-based cancer detection
(3-5x sensitivity), and DNA storage/computation convergence.

Includes phased integration roadmap and scale impact analysis.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 03:03:08 +00:00
Claude
ee692af790 docs: Add 50-year SOTA vision for ruvector + sublinear-time-solver convergence
10 breakthrough vectors mapping concrete code paths to 50-year-ahead SOTA:
sub-constant time via predictive precomputation, self-discovering algorithms,
photonic-native vector ops, self-booting mathematical universes, neuromorphic
sublinear computing, hyperbolic sublinear geometry, cryptographic proof of
computation, temporal-causal vector spaces, infinite-scale sublinear consensus,
and the convergence of database + intelligence into a single substrate.

5-horizon roadmap from integration (2026) through convergence (2076).

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:49:09 +00:00
Claude
3558ab7910 docs: Add algorithm deep-dive analysis - final document (Agent 10)
Complete mathematical analysis of all 7 sublinear algorithms mapped to
ruvector's 9 subsystems. Top findings: Forward Push for hybrid graph
search (O(1/eps) vs O(k*d^L)), Conjugate Gradient for PDE attention
(quadratic to near-linear), Neumann Series for spectral filtering.

This completes the 15-agent analysis swarm - all documents present:
00-executive-summary, 01-14 covering crates, npm, rvf, examples,
architecture, wasm, mcp, performance, security, algorithms, typescript,
testing, dependencies, and roadmap.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:40:31 +00:00
Claude
279c4315c4 docs: Add testing strategy analysis (Agent 12)
Integration test design, property-based testing for solver correctness,
WASM test strategies, performance regression testing, and CI/CD pipeline
integration recommendations.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:37:48 +00:00
Claude
f52fad813a docs: Add examples integration analysis (Agent 4)
Analysis of 38+ ruvector examples and 46 RVF examples with proposed new
examples for sublinear solver integration, benchmark comparisons, and
tutorial progression.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:36:55 +00:00
Claude
e688ce623b docs: Add WASM integration and use cases roadmap (Agents 6, 14)
Agent 6: WASM build pipeline, SIMD acceleration, memory management,
browser vs Node.js deployment strategies.
Agent 14: Phased integration roadmap, use case mapping, migration
strategy, and success metrics.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:36:11 +00:00
Claude
309a0c6843 docs: Add dependency graph analysis (Agent 13)
Full dependency tree comparison between ruvector (79 workspace members)
and sublinear-time-solver (9 crates), version conflicts, feature flag
compatibility, and bundle size impact.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:35:35 +00:00
Claude
9673b98d25 docs: Add security analysis for sublinear-time-solver integration (Agent 9)
Security posture assessment covering WASM sandbox, serialization safety,
MCP access control, dependency supply chain, and input validation for
solver APIs.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:35:06 +00:00
Claude
1e4b73d864 docs: Add MCP integration and performance analysis (Agents 7, 8)
Agent 7: MCP tool federation, 40+ tool composition, transport layer analysis.
Agent 8: Performance benchmarks, SIMD acceleration, O(log n) complexity gains.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:34:29 +00:00
Claude
27afbd11be docs: Add Rust crates integration analysis (Agent 1)
Detailed analysis of 82 ruvector Rust crates vs 9 sublinear-time-solver
crates, covering dependency overlap (nalgebra, serde, rayon), type
compatibility, and crate-level integration patterns.

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:33:53 +00:00
Claude
fb464e0bfd docs: Add sublinear-time-solver integration analysis (15-agent swarm, partial)
Initial batch of research documents from 15-agent analysis swarm analyzing
integration between ruvector and sublinear-time-solver. Covers NPM packages,
RVF format, architecture, and TypeScript type compatibility.

More documents pending from running agents (crates, WASM, MCP, performance,
security, algorithms, testing, dependencies, roadmap, executive summary).

https://claude.ai/code/session_01WY4MpWoe2LMzkYUHLxhPHX
2026-02-20 02:33:03 +00:00
rUv
125a1003ad feat(rvf): expose AGI components via npm packages
- Create @ruvector/rvf-solver npm package (TypeScript SDK wrapping
  rvf-solver-wasm WASM module with RvfSolver class, Thompson Sampling,
  ReasoningBank, witness chains)
- Add AGI NAPI methods to rvf-node: indexStats, verifyWitness, freeze, metric
- Add store accessors to rvf-runtime: options(), metric(), epoch()
- Update @ruvector/rvf unified SDK to v0.1.8 with solver re-exports
- Update ADRs 032, 036, 037, 039 with AGI npm package details

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-02-17 01:41:13 +00:00
Claude
9cff4e46f2 docs(adr): update ADRs with implementation details from rvf-types
- ADR-029: Add complete segment type registry (23 variants) with source references
- ADR-030: Phase 1 complete — KernelHeader (128B), EbpfHeader (64B), WasmHeader (64B),
  all enums and flag constants implemented with 32+ tests. Updated GOAP world state.
- ADR-032: Add WASM bootstrap types implementation section (WasmHeader, WasmRole,
  WasmTarget, 8 feature flags, 10 tests)
- ADR-036: Status updated to Partially Implemented. Documented AGI container
  implementation (972 lines, 24 tests) including AgiContainerHeader, ExecutionMode,
  AuthorityLevel, ResourceBudget, CoherenceThresholds, ContainerSegments, and
  22 TLV tags with domain expansion integration (0x0112-0x0115)

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-16 02:58:43 +00:00
Claude
0bd75e31b8 feat(rvf): rvf-solver-wasm — self-learning AGI engine compiled to WASM
Compiles the complete three-loop adaptive solver to wasm32-unknown-unknown
(160 KB, no_std + alloc). Preserves all AGI capabilities:

- Thompson Sampling two-signal model (safety Beta + cost EMA)
- 18 context buckets with per-arm bandit stats
- Speculative dual-path execution
- KnowledgeCompiler with signature-based pattern cache
- Three-loop architecture (fast/medium/slow)
- SHAKE-256 witness chain via rvf-crypto

12 WASM exports: create/destroy/train/acceptance/result/policy/witness.
Handle-based API supports 8 concurrent solver instances.

ADR-039 documents the integration architecture.
Benchmark binary validates WASM against native solver.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-16 00:43:12 +00:00
Claude
4bffe12e2b docs(adr): ADR-038 npx ruvector & rvlite witness verification integration
Plans the integration path for .rvf acceptance test verification into
the npm ecosystem:

- npx ruvector rvf verify-witness <file.rvf> (N-API + WASM fallback)
- npx rvlite verify-witness <file.rvf> (WASM via cli-rvf.ts)
- rvlite SDK verifyWitnessChain() for browser-side verification
- MCP tool rvf_verify_witness for Claude Code agents
- 5-phase implementation plan, each independently shippable

Bridges the rvf_witness_verify WASM export (ADR-037) to end users
without requiring the Rust toolchain.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-16 00:17:00 +00:00
Claude
5a9c899f29 feat(rvf): integrate publishable acceptance test with native SHAKE-256 witness chain
Replace standalone SHA-256 chain with rvf-crypto SHAKE-256, add native .rvf
binary output (WITNESS_SEG + META_SEG), and wire witness verification into
rvf-wasm microkernel.

Key changes:
- Feature-gate ed25519 in rvf-crypto for WASM compatibility (sha3 no_std)
- Rewrite WitnessChainBuilder to use shake256_256 + parallel rvf_crypto::WitnessEntry
- Add export_rvf_binary() with WITNESS_SEG (0x0A) + META_SEG (0x07) segments
- Add rvf_witness_verify/rvf_witness_count exports to rvf-wasm
- Add verify-rvf subcommand to acceptance-rvf CLI
- Write ADR-037 documenting architecture and AGI benchmark integration
- Update rvf-crypto, rvf-wasm, and rvf READMEs

86 tests pass (66 lib + 20 integration). rvf-crypto 49 tests pass.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-16 00:13:44 +00:00
Claude
e5458a9c07 refactor(adr-036): optimize AGI container architecture
- Resolve open questions: repo automation as first domain, four-level
  AuthorityLevel enum, per-task ResourceBudget with hard caps,
  CoherenceThresholds with validation
- Add AGI_MAX_CONTAINER_SIZE (16 GiB) with enforcement in validation
- Tighten ContainerSegments::validate: Verify/Live modes now require
  world model data (VEC or INDEX segments), not just kernel/WASM
- Add ContainerError variants: InsufficientAuthority, BudgetExhausted
- Add to_flags support for orchestrator_present and world_model_present
- Add wire format section and cross-references to ADRs 029-033 in doc
- Add 2 new TLV tags: AUTHORITY_CONFIG (0x0110), DOMAIN_PROFILE (0x0111)
- Re-export new types from lib.rs
- Update rvf-runtime tests for tightened validation
- All 222 rvf-types + all rvf-runtime tests pass

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 19:10:00 +00:00
Claude
4282461017 docs(adr-036): AGI cognitive container with Claude Code orchestration
Defines the full system boundary for portable intelligence:
- RuVector as existential substrate (world model, coherence signals)
- RVF as cognitive container format (packaging, witness chains, replay)
- Claude Code as control plane orchestrator (planning, tool use)
- Claude Flow as swarm coordinator (routing, shared memory, learning)

Key mechanisms:
- Structural health gates (min-cut coherence, contradiction pressure)
- Skill promotion with counterexample requirements
- Two execution modes: Replay (bit-identical) and Verify (same grades)
- 10 node types, 9 edge types, 4 invariants for the world model schema
- MCP tools: ruvector_query, ruvector_cypher, rvf_snapshot, eval_run

Acceptance test: same RVF artifact, two machines, 100 tasks,
95+ passing in verify mode, zero policy violations.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 18:41:08 +00:00
Claude
90286c33e5 feat(adr-035): capability report — witness bundles, scorecards, governance
Proof infrastructure for repeatable capability evidence:

- WitnessHeader: 64-byte repr(C) header with task ID, policy hash,
  outcome, governance mode, cost/latency/tokens, HMAC-SHA256 signature
- WitnessBuilder: fluent API to record tool calls, enforce governance
  policy (restricted/approved/autonomous), and build signed bundles
- ParsedWitness: zero-copy parser with verify_all(), parse_trace(),
  evidence_complete() checks
- GovernancePolicy: three enforcement modes with deny/allow lists,
  cost caps, tool call budgets, and deterministic policy hashing
- ScorecardBuilder: aggregate bundles into solve rate, cost/solve,
  median/p95 latency, evidence coverage, policy violations
- ToolCallEntry: per-call trace with hashed args/results, latency,
  cost, tokens, and policy check result

Acceptance criteria from ADR-035:
- solve_rate >= 0.60, policy_violations == 0, evidence_coverage == 1.0

Test counts:
- rvf-types witness: 10 unit tests
- rvf-runtime witness: 14 unit tests
- witness_e2e: 10 integration tests
- Total across all RVF crates: 451 tests passing

Zero external dependencies. Real HMAC-SHA256 signatures.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 18:22:15 +00:00
Claude
afe79e0450 feat(adr-034): zero-dep QR cognitive seed with real crypto and mobile FFI
Complete the QR Cognitive Seed pipeline with zero external dependencies:

- Pure SHA-256 (FIPS 180-4) verified against NIST test vectors
- HMAC-SHA256 (RFC 2104) verified against RFC 4231 test cases
- LZ77 compression (SCF-1 format) with 4KB sliding window
- Seed crypto: content hashing, signing, layer verification
- C FFI (5 extern "C" functions) for App Clip / mobile integration
- SeedBuilder.build_and_sign() with automatic hashing and signing
- ParsedSeed.verify_all() with full integrity and signature checks
- ParsedSeed.decompress_microkernel() using built-in LZ
- 11 end-to-end integration tests with real cryptography
- Updated ADR-034 with App Clip, PWA, Android delivery paths
- Example updated with full real-crypto round-trip demo

Total: 381 tests passing (183 types + 154 runtime + 11 e2e + 33 manifest)

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 17:54:26 +00:00
Claude
8d71a7e7ad feat(adr-034): QR Cognitive Seed — a world inside a world
Implement ADR-034: RVQS binary format for embedding intelligence
in a single QR code (≤2,953 bytes). Scan printed ink to mount a
portable brain with progressive download to full intelligence.

New types (rvf-types/qr_seed.rs):
- SeedHeader (64 bytes, compile-time assertion)
- HostEntry, LayerEntry (28 bytes), 8 seed flag constants
- 8 TLV tag constants, well-known layer identifiers
- Round-trip serialization, 9 unit tests

New runtime (rvf-runtime/qr_seed.rs):
- SeedBuilder: fluent API for constructing RVQS payloads
- ParsedSeed: zero-copy parser with manifest TLV decoding
- DownloadManifest: structured host/layer/token parsing
- BootstrapProgress: phase tracking with recall estimation
- QR capacity enforcement, 12 unit tests

Example (qr_seed_bootstrap.rs):
- Full demo: build → parse → manifest → progressive bootstrap
- Shows 2,724-byte seed with 229 bytes headroom

All 399 tests pass (172 types + 160 runtime + 33 manifest + 34 integration).

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 17:13:01 +00:00
Claude
10c49d5d43 harden(adr-033): QualityEnvelope, triple budget caps, selective scan, fuzz benchmark
- QualityEnvelope as mandatory outer return type (not nestable, not droppable)
- SearchEvidenceSummary, BudgetReport, DegradationReport structs
- QualityPreference enum (Auto/PreferQuality/PreferLatency/AcceptDegraded)
- Triple budget caps: max_scan_time_us, max_scan_candidates, max_distance_ops
- Selective safety net: multi-centroid union + HNSW neighbor expansion + recency
- DoS hardening: budget tokens, negative caching, proof-of-work option
- Three mandatory acceptance tests: schema enforcement, budget cap enforcement,
  graceful degradation under degenerate conditions
- Fuzz benchmark: 4000 queries across 4 classes must respect p95 ceiling and
  preserve monotonic recall improvement across progressive load stages

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 16:00:16 +00:00
Claude
b7b6db2426 fix(adr-033): extend ResultQuality to API boundary, cap brute-force, add malicious manifest test
Three fixes to ADR-033:

1. ResultQuality split into RetrievalQuality (per-candidate) and
   ResponseQuality (per-response at API boundary). ResponseQuality
   survives serialization across JSON/gRPC/MCP. DegradationReason
   provides structured, inspectable evidence for why quality dropped.

2. Brute-force safety net dual-budgeted: max 5ms wall-clock AND max
   50K candidates, whichever hits first. Both configurable via
   QueryOptions. Budget=0 disables fallback entirely. Prevents O(N)
   DoS from adversarial queries on large hot caches.

3. Mandatory acceptance test: malicious tail manifest with valid CRC
   but redirected hotset pointers must fail deterministically under
   Strict policy with a logged, stable error code. Separate test for
   re-signed forgery (wrong signer vs no signature distinction).

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 15:54:39 +00:00
Claude
ffbdd3c482 docs(adr): ADR-033 progressive indexing hardening
Addresses four structural weaknesses in the progressive indexing system:

1. Content-addressed centroid stability — hotset pointers verified by
   SHAKE-256 content hashes, not just byte offsets. Compaction becomes
   physically destructive but logically stable.

2. Adversarial distribution resilience — distance entropy detection
   with adaptive n_probe widening. Silent recall collapse replaced by
   detected degradation with ResultQuality signaling.

3. Honest recall framing — empirical targets scoped to distribution
   classes (natural/synthetic/adversarial). Monotonic recall improvement
   property proven from append-only invariant. Brute-force safety net
   when candidate count is insufficient.

4. Mandatory manifest signatures — SecurityPolicy defaults to Strict.
   No signature = no mount in production. Prevents segment-swap attacks
   on hotset pointers. CRC32C catches corruption; ML-DSA-65 catches
   adversaries.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 15:50:17 +00:00
Claude
4bec65135f feat(rvf): add WASM_SEG (0x10) for self-bootstrapping RVF files
Add the WASM_SEG segment type and complete self-bootstrapping
architecture that allows RVF files to carry their own execution
runtime. When an RVF file embeds a WASM interpreter alongside the
microkernel, the host only needs raw execution capability — making
RVF "run anywhere compute exists."

Changes:
- rvf-types: Add SegmentType::Wasm (0x10), WasmHeader (64-byte),
  WasmRole, WasmTarget enums, and feature flag constants
- rvf-runtime: Add embed_wasm(), extract_wasm(), extract_wasm_all(),
  is_self_bootstrapping() methods on RvfStore, plus write_wasm_seg()
  in the write path
- rvf-wasm: Add bootstrap module with resolve_bootstrap_chain() that
  discovers WASM_SEGs, parses headers, and resolves the optimal
  bootstrap strategy (None/HostRequired/SelfContained/TwoStage/Full)
- docs: Add spec/11-wasm-bootstrap.md with complete wire format,
  bootstrap protocol, size budget analysis, and security model

The three-layer bootstrap stack:
  Layer 0: Raw bytes (.rvf file)
  Layer 1: Embedded WASM interpreter (~50 KB)
  Layer 2: WASM microkernel (~5.5 KB)
  Layer 3: RVF data segments

All 131 rvf-types tests and 72 rvf-runtime tests pass.

https://claude.ai/code/session_01RnwD4x5cbpB7FPvoyYQz8G
2026-02-15 15:36:34 +00:00
rUv
7b8035eb54 feat(rvf): RVF WASM integration, witness auto-append, real verification, prebuilt fallbacks, README examples
* feat(adr): add ADR-032 for RVF WASM integration into npx ruvector and rvlite

Documents phased integration plan: Phase 1 adds RVF as optional dep + CLI
command group to npx ruvector, Phase 2 adds RVF as storage backend for rvlite,
Phase 3 unifies shared WASM backend and MCP bridge.

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(adr): update ADR-032 with invariants, contracts, failure modes, and decision matrix

Adds: single writer rule, crash ordering with epoch reconciliation,
explicit backend selection (no silent fallback), cross-platform compat
rule, phase contracts with success metrics, failure mode test matrix,
hybrid persistence decision matrix, implementation checklist.

Closes #169

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): integrate RVF WASM into npx ruvector and rvlite (ADR-032)

Phase 1 implementation:
- Add @ruvector/rvf as optional dependency to ruvector package
- Create rvf-wrapper.ts with 10 exported functions matching core pattern
- Add 3-tier platform detection (core -> rvf -> stub) with explicit
  --backend rvf override that fails loud if package is missing
- Add 8 rvf CLI subcommands (create, ingest, query, status, segments,
  derive, compact, export) routed through the wrapper
- 5 Rust smoke tests validating persistence across restart, deletion
  persistence, compaction stability, and adapter compatibility

Phase 2 foundations:
- Add rvf-backend feature flag to rvlite Cargo.toml (default off)
- Create epoch reconciliation module for hybrid RVF + IndexedDB sync
- Add @ruvector/rvf-wasm as optional dep to rvlite npm package
- Add rvf-adapter-rvlite to workspace members

All tests green: 237 RVF core, 23 adapter, 4 epoch, 5 smoke.

Refs: #169

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): complete ADR-032 phases 1-3 — epoch, lease, ID map, MCP tools, compat tests

Phase 2 Rust: full epoch reconciliation (EpochTracker with AtomicU64, 23 tests),
writer lease with file lock and PID-based stale detection (12 tests),
direct ID mapping trait with DirectIdMap and OffsetIdMap (20 tests).

Phase 2 JS: createWithRvf/saveToRvf/loadFromRvf factories, BrowserWriterLease
with IndexedDB heartbeat, rvf-migrate and rvf-rebuild CLI commands, epoch sync
helpers. +541 lines to index.ts, new cli-rvf.ts (363 lines).

Phase 3: 3 MCP rvlite tools (rvlite_sql, rvlite_cypher, rvlite_sparql),
CI wasm-dedup-check workflow, 6 cross-platform compat tests, shared peer dep.

Phase 1: 4 RVF smoke integration tests (full lifecycle, cosine, multi-restart,
metadata). Node.js CLI smoke test script.

81 new Rust tests passing. ADR-032 checklist fully complete.

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore: bump versions and fix TS/README for npm publish

- ruvector 0.1.88 → 0.1.97 (match npm registry)
- rvlite 0.2.1 → 0.2.2
- @ruvector/rvf 0.1.0 → 0.1.1
- Fix MCP command in ruvector README (mcp-server → mcp start)
- Fix WASM type conflicts in rvlite index.ts (cast dynamic imports to any)

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): add witness auto-append, real CLI verification, prebuilt fallbacks, and README examples

Five "What's NOT Automatic" gaps fixed:
1. Witness auto-append: WitnessConfig in RvfOptions auto-records ingest/delete/compact
   operations as WITNESS_SEG entries with SHAKE-256 hash chains
2. verify-witness CLI: Real hash chain verification — extracts WITNESS_SEG payloads,
   runs verify_witness_chain() with full SHAKE-256 validation
3. verify-attestation CLI: Real kernel image hash verification and attestation
   witness chain validation
4. Prebuilt kernel fallback: KernelBuilder::from_builtin_minimal() produces valid
   bzImage without Docker
5. Prebuilt eBPF fallback: EbpfCompiler::from_precompiled() produces valid BPF ELF
   without clang; Launcher::check_requirements()/dry_run() for QEMU detection

README examples added to all 3 packages:
- crates/rvf/README.md: Proof of Operations section
- npm/packages/rvf/README.md: 7 real-world examples
- npm/packages/ruvector/README.md: Working cognitive container examples

830 tests passing, workspace compiles cleanly.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-02-14 18:03:26 -05:00
rUv
f8870b3c71 feat(rvf): RuVector Format — Universal Cognitive Container SDK (#166)
* feat(rvf): add RuVector Format universal substrate specification

Research and design for RVF — a streaming, progressive, adaptive, quantum-secure
binary format for vector intelligence. Covers append-only segment model, two-level
tail manifests, temperature tiering, progressive HNSW indexing, epoch-based overlay
system, SIMD-optimized query paths, WASM microkernel for Cognitum tiles, domain
profiles (RVDNA, RVText, RVGraph, RVVision), and post-quantum cryptography.

https://claude.ai/code/session_01DDqjGE51JpsRE3DgUjFyjW

* feat(rvf): add deletion, filtered search, concurrency, and operations specs

Fill four specification gaps in the RVF format design:
- spec/07: Vector deletion lifecycle, JOURNAL_SEG wire format, deletion bitmaps
- spec/08: Filtered search with META_SEG, METAIDX_SEG, filter expression language
- spec/09: Writer locking, reader-writer coordination, versioning, space reclamation
- spec/10: Batch operations API, error codes, network streaming protocol

Also fixes the segment header field conflict between spec/01 and wire/binary-layout.md
(checksum_algo/compression now u8, adds uncompressed_len at 0x38).

https://claude.ai/code/session_01DDqjGE51JpsRE3DgUjFyjW

* feat(rvf): add RuVector Format SDK, 40 examples, MCP server, and documentation

Complete RVF implementation including:
- 12 Rust crates (rvf-types, rvf-wire, rvf-manifest, rvf-index, rvf-quant,
  rvf-crypto, rvf-runtime, rvf-import, rvf-wasm, rvf-node, rvf-server,
  plus integration tests)
- 40 runnable examples covering core storage, agentic AI, production
  patterns, vertical domains, exotic capabilities, runtime targets,
  network/security, POSIX/systems, and network operations
- TypeScript SDK (npm/packages/rvf) with RvfDatabase class
- MCP server (npm/packages/rvf-mcp-server) with stdio and SSE transports
- Node.js N-API bindings (npm/packages/rvf-node)
- WASM package (npm/packages/rvf-wasm)
- ADR-029 (canonical format), ADR-030 (computational container),
  ADR-031 (example repository)
- DNA-style lineage provenance, computational containers (KERNEL_SEG,
  EBPF_SEG), witness chains, TEE attestation, domain profiles
- Superseded ADR annotations for ADR-001, ADR-005, ADR-006, ADR-018-021

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): add CLI, WASM store, generate_all, and 46 output .rvf files

- Add rvf-cli crate (665 lines, 9 subcommands: create/ingest/query/delete/status/inspect/compact/derive/serve)
- Add WASM control plane store (alloc_setup, segment, store modules) for ~46 KB binary
- Add generate_all.rs example producing 46 persistent .rvf files in output/
- Add Node.js N-API bindings for lineage, kernel/eBPF, and inspection
- Add npm TypeScript backend/database/types for RVF integration
- Update READMEs with CLI sections, MCP server docs, and crate map (13 crates)
- All 40 examples verified passing

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): add Claude Code appliance, improve Quick Start, fix API docs

- Add claude_code_appliance.rs: self-booting RVF with SSH + Claude Code
  install (curl -fsSL https://claude.ai/install.sh | bash), 3 SSH users,
  eBPF filter, 20-package manifest, witness chain, lineage snapshot
- Improve Quick Start: Install section (crate/CLI/npm/WASM/MCP), WASM
  browser example, generate_all reference, expanded Rust crate deps
- Fix embed_kernel/embed_ebpf API docs to match actual signatures
  (u8 params with `as u8` cast, 6-param kernel, Option<&[u8]> btf)
- Update generate_all.rs: add claude_code_appliance generator (47 files)
- Regenerate all 47 output .rvf files

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): add RVCOW branching, real kernel/eBPF/launcher, 795 tests

Vector-native copy-on-write branching (ADR-031) with four new segment
types (COW_MAP 0x20, REFCOUNT 0x21, MEMBERSHIP 0x22, DELTA 0x23),
real Linux microkernel builder, QEMU microVM launcher, real eBPF
programs, and 128-byte KernelBinding for tamper-evident kernel-manifest
linkage.

New crates:
- rvf-kernel: Docker-based kernel build, real cpio/newc initramfs builder,
  SHA3-256 verification, prebuilt kernel support (37 tests)
- rvf-launch: QEMU microVM launcher with QMP shutdown, KVM/TCG detection,
  virtio-blk/net port forwarding, kernel extraction (8 tests)
- rvf-ebpf: 3 real BPF C programs (xdp_distance, socket_filter,
  tc_query_route) with clang compilation support (17 tests)

RVCOW runtime:
- CowEngine with read/write paths, write coalescing, snapshot-freeze
- CowMap (flat-array), MembershipFilter (bitmap), CowCompactor
- 3x read performance via pread optimization (1.3us/vector)
- Branch creation: 2.6ms for 10K vectors, child = 162 bytes

Security: 20-finding audit, 7 fixes applied including division-by-zero
guards, integer overflow checks, and KernelBinding::from_bytes_validated().

CLI: 8 new commands (launch, embed-kernel, embed-ebpf, filter, freeze,
verify-witness, verify-attestation, rebuild-refcounts), serve wired to
real rvf-server.

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): update README, add crate/npm READMEs, publish to crates.io and npm

- Rewrite README with cognitive container terminology, grouped features,
  4 comparison tables (vs Docker, Vector DBs, Git LFS, SQLite), updated
  benchmarks, architecture diagram, and 45 examples
- Add READMEs for rvf-kernel, rvf-launch, rvf-ebpf, rvf-import crates
- Add READMEs for @ruvector/rvf, rvf-node, rvf-wasm, rvf-mcp-server npm packages
- Fix Cargo.toml metadata (homepage, readme, categories, keywords) and
  add version specs to all path dependencies for crates.io publishing
- Fix clippy warnings in rvf-kernel/initramfs.rs and rvf-launch/lib.rs
- Published to crates.io: rvf-types, rvf-wire, rvf-manifest, rvf-quant,
  rvf-index, rvf-crypto (remaining crates pending rate limit)
- Published to npm: @ruvector/rvf, @ruvector/rvf-node, @ruvector/rvf-wasm,
  @ruvector/rvf-mcp-server

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore: add rvf-kernel, rvf-ebpf, rvf-launch, rvf-server, rvf-import, rvf-cli to workspace

Include all 15 RVF crates plus integration tests and benchmarks in the
root workspace members list so cargo publish can resolve them by name.

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(rvf): add published packages, cognitive container branding, grouped capabilities

- Add Published Packages section with 13 crates.io + 4 npm tables
- Add Platform Support table (Linux, macOS, Windows, WASM, no_std)
- Expand capability table from 9 to 15 rows in 4 groups
- Rewrite all "How" descriptions in plain language
- Update .rvf diagram to show all 20 segment types
- Rename ADRs: computational container -> cognitive container
- Add emojis to all section headers

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: update root README with RVF cognitive containers, expanded capabilities

- Update intro: "gets smarter + ships as cognitive container"
- Add self-booting microservice row to Pinecone comparison table
- Expand capabilities from 34 to 42 features with dedicated RVF section
- Update "Think of it as" to include Docker comparison and RVF explanation
- Add RVF collapsed group to Ecosystem (13 crates, 4 npm, install commands)
- Add RVF to Platform & Edge section with install commands
- Add RVF npm packages (4) and Rust crates (13) to package reference
- Add RVF rows to feature comparison table (6 new rows)
- Add ADR-030/031 to ADR list
- Add RVF to Installation table, Project Structure
- Update attention mechanisms count from 39 to 40+
- Update npm count to 49+, Rust crates to 83
- Update footer with crates.io and RVF links

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: expand comparison table with emojis, cost, audit, branching, single-file

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs: rewrite comparison table in plain language

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore: clean up empty code change sections in the changes log

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-14 13:14:49 -05:00
rUv
e6d4330008 feat(ruqu): add quantum execution intelligence engine with 5 backends
Transforms ruqu from classical coherence monitor into full-stack quantum execution intelligence engine (~2K to ~24K lines).

New: StateVector, Stabilizer, TensorNetwork, Clifford+T, and Hardware simulation backends. Cost-model planner, surface code decoder (union-find O(n*alpha(n))), QEC scheduler, noise models, OpenQASM 3.0 export, deterministic replay, and cross-backend verification.

PR #161
2026-02-12 12:55:21 -05:00
rUv
244bbffe54 feat: add package.json for rvdna example with WASM bindings and build scripts 2026-02-12 15:32:55 +00:00
rUv
576daba1df Merge origin/main into claude/quantum-engine-adrs-6OsEO - resolve Cargo.toml conflict 2026-02-08 17:04:57 +00:00
rUv
593b44a543 Merge pull request #151 from ruvnet/claude/bitnet-ruvllm-research-Cz4Ot
docs: Add ADR-017 and DDD for Craftsman Ultra 30b 1bit BitNet integration
2026-02-08 11:55:15 -05:00
Claude
eee542e103 docs: Add ADR-018 through ADR-023 and DDD for temporal tensor store
Extends ADR-017 with detailed architecture for block-based temporal
tensor compression. Introduces 6 new ADRs covering storage engine,
tiered quantization formats, temporal scoring algorithm, delta
compression/reconstruction, WASM API, and benchmarking criteria.
Adds comprehensive DDD with 5 bounded contexts mapping to the
proposed 6-crate workspace layout.

ADR-018: Block-based storage engine with BlockKey/BlockMeta model,
         append-only MetaLog, tiered data files, CRC32 checksums
ADR-019: 8/7/5/3-bit quantization formats with two-level scale,
         Tier0 compression-to-zero, codec_bits bit packing
ADR-020: EMA + popcount + recency scoring, hysteresis, budgeted
         maintenance ticks, fast exp approximation
ADR-021: Read/write paths, sparse delta format, delta chain
         compaction, factor-based reconstruction policies
ADR-022: WASM exports (tts_init/put/get/tick/stats), host-imported
         BlockIO, cross-platform strategy (native/Node/browser/edge)
ADR-023: Zipf simulation acceptance test (1M blocks, 10M accesses),
         microbench targets, failure mode catalog

DDD: 5 bounded contexts (Block Management, Quantization, Temporal
     Scoring, Storage Engine, Delta & Reconstruction) with aggregate
     roots, domain events, repository interfaces, and context map.

Total: 8,084 lines across 7 documents.

https://claude.ai/code/session_01Ksy165BL5nGpVoWaAfTE7t
2026-02-08 02:12:38 +00:00
Claude
21dc9866cc docs: Add ADR-QE-014 documenting exotic quantum-classical discoveries
Documents 4 validated Phase 1 discoveries:
- Decoherence trajectory fingerprinting (clustering without similarity)
- Interference-based polysemy resolution (microsecond disambiguation)
- Counterfactual dependency mapping (pipeline importance scoring)
- Phase-coherent swarm coordination (quality > headcount)

Outlines 8 Phase 2 hypotheses for cross-module experiments including
time-dependent disambiguation, QEC on swarm reasoning, counterfactual
search explanation, and the full decohere-interfere-collapse-verify pipeline.

https://claude.ai/code/session_01B1NkbLDWYPaacS9miKsnvW
2026-02-06 15:51:08 +00:00
Claude
5e18e8b8fa docs: Add ADR-QE-013 Deutsch's theorem proof with historical comparison
Complete proof of Deutsch's theorem with phase kickback lemma and
step-by-step derivation. Compares five major formulations:

- Deutsch (1985): original probabilistic version (p=1/2)
- Deutsch-Jozsa (1992): deterministic n-bit, 2 queries
- Cleve-Ekert-Macchiavello-Mosca (1998): deterministic, single query
- Nielsen-Chuang (2000): canonical textbook presentation
- Calude (2006): de-quantization using higher-dimensional classical bits

Includes de-quantization critique (Abbott et al.), classical wave
analogies, and analysis of when quantum advantage is genuine vs
artifactual.

Adds 6 verification tests to ruqu-algorithms confirming all four
oracles produce deterministic correct results via the ruqu-core
simulator, including a phase-kickback amplitude-level check.

https://claude.ai/code/session_01B1NkbLDWYPaacS9miKsnvW
2026-02-06 02:00:35 +00:00
Claude
ce063b3d3f feat: Add quantum simulation engine ADR series (QE-001 to QE-012) and DDD design documents
Comprehensive architecture decision records and domain-driven design documentation
for integrating a Rust-based quantum simulation engine (ruQu) into the ruVector stack.

ADR Series (12 documents):
- QE-001: Core Architecture - pure Rust state-vector simulator decision
- QE-002: Crate Structure - three-crate architecture (ruqu-core, ruqu-wasm, ruqu-algorithms)
- QE-003: WASM Compilation - WebAssembly strategy with 25-qubit limit enforcement
- QE-004: Performance Optimization - SIMD, multithreading, gate fusion, benchmarks
- QE-005: VQE Algorithm - variational eigensolver with exact expectation values
- QE-006: Grover Search - O(1) oracle optimization via direct state vector access
- QE-007: QAOA MaxCut - graph-based optimization with Rzz native gates
- QE-008: Surface Code Error Correction - mid-circuit measurement, syndrome extraction
- QE-009: Tensor Network Evaluation - MPS/contraction for shallow circuits
- QE-010: Observability & Monitoring - metrics, tracing, health checks integration
- QE-011: Memory Gating & Power Management - zero-idle, on-demand allocation
- QE-012: Min-Cut Coherence Integration - syndrome-to-decoder bridge with ruQu

DDD Design (3 documents):
- Strategic Design: 6 bounded contexts, context map, ubiquitous language
- Tactical Design: 6 aggregates, 20+ value objects, 15+ domain events, services
- Integration Patterns: anti-corruption layers, shared kernel, event flows

https://claude.ai/code/session_01B1NkbLDWYPaacS9miKsnvW
2026-02-06 00:39:39 +00:00
Claude
2a21efbb5c feat: Add ADR-017 temporal tensor compression with tiered quantization
Introduces a complete temporal tensor compression system with:

- ADR-017: SOTA research-backed architecture decision record covering
  groupwise symmetric quantization, temporal segment reuse, access-pattern
  driven tier selection (8/7/5/3 bit), and WASM-compatible design

- ruvector-temporal-tensor crate (zero external dependencies):
  - tier_policy: Score-based hot/warm/cold bit-width selection
  - f16: Software IEEE 754 half-precision conversion
  - bitpack: Arbitrary bit-width stream packing (no alignment waste)
  - quantizer: Groupwise symmetric quantization with f16 scales
  - segment: Binary segment format (TQTC) encode/decode
  - compressor: Temporal segment manager with drift detection
  - ffi: WASM/C FFI with handle-based resource management

- ruvector-temporal-tensor-wasm crate for wasm32 targets

- 33 passing unit tests covering all modules

Compression targets: 4x (hot/8-bit), 4.57x (warm/7-bit),
6.4x (warm/5-bit), 10.67x (cold/3-bit) vs f32 baseline.

https://claude.ai/code/session_01U63xtGd5Q8mUevyY7nUSfJ
2026-02-06 00:28:21 +00:00
Claude
a3c7fb54a8 feat: Add RLM embedder, tokenizer, eval gates, trace writer, and security hardening
New modules (4 files, 2,359 lines):
- rlm_embedder.rs (743L): RLM-style recursive sentence transformer with
  3 variants (query-conditioned, corpus-conditioned, contradiction-aware
  twin), merge rule, BaseEmbedder/NeighborRetriever traits, 14 tests
- tokenizer.rs (418L): BPE tokenizer with GGUF vocab loading, encode/decode,
  special token handling, 10 tests
- trace.rs (554L): JSONL trace writer for routing, citation, refusal
  decisions, jaccard similarity, manual JSON serialization, 10 tests
- eval.rs (644L): Three behavioral gates (routing correctness >= 0.85,
  citation precision >= 0.90, refusal F1 >= 0.85), EvalSuite, 12 tests

Documentation:
- AD-24: RLM-Style Recursive Sentence Transformer Embedder — 3 variants,
  merge rule, training strategy, evaluation criteria, appliance fit
- DDD v2.6: 8 new ubiquitous language terms, 4 new open questions (#31-34)
- 3 new positive consequences (#31-33) for RLM embeddings

Security hardening (across 6 existing files):
- Path traversal validation in GGUF export
- Division-by-zero epsilon guards in quantizer
- Bounds validation on public function inputs
- NaN-safe softmax with -inf handling

138 tests pass, 0 compilation errors.
Total bitnet module: 9,632 lines across 16 files.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 15:40:59 +00:00
Claude
ab78e18a87 feat: Add AD-23 Phase-1 distillation, expert cache, and DDD updates
AD-23: Phase-1 Distillation via External GPU Teacher Artifacts
- One-time GPU job produces behavioral artifacts (routing traces,
  sparse logits, preference labels) — not trained weights
- CPU-only refinement: router repair, LoRA correction, EWC++, policy
  optimization using teacher artifacts
- Acceptance criteria: 200-prompt suite, all 3 behavioral gates,
  stability under 10% corpus perturbation

expert_cache.rs: MoE expert hot-set caching (new file)
- ExpertCache with LRU/LFU/Adaptive eviction policies
- MoeBatchScheduler: reorder token execution by expert for cache reuse
- Prefetcher trait for future platform-specific prefetch intrinsics
- 12 tests (92/92 bitnet tests pass)

DDD v2.5: 6 new ubiquitous language terms (Teacher Artifact, Behavioral
Distillation, Router Repair, Sparse Logits, Corpus Perturbation) and
4 new open questions (#27-30) for Phase-1 operability.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 15:12:33 +00:00
Claude
d5219db15c docs: Add AD-22 evaluation infrastructure and behavioral gates
Defines three ship/no-ship gates:
- Gate 1: Routing correctness (>= 85% teacher agreement)
- Gate 2: Citation correctness (precision >= 90%, recall >= 70%)
- Gate 3: Refusal calibration (F1 >= 0.85)

Includes JSONL trace schema, auto-labeling strategy using RuVector
signals (redundancy, cluster disagreement, mincut fragility), and
go/no-go rule requiring all gates to pass on same prompt suite run.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 14:57:32 +00:00
Claude
4685854fb8 docs: Add AD-21 native Rust ternary kernels with WASM SIMD128 target
Research bitnet.cpp Rust port strategy: R3-Engine proves 100% Safe Rust
with dual-target (native AVX-512 + WASM SIMD128) achieving 80-117 tok/s.
Recommend Approach C (reference R3-Engine patterns) over Python codegen.
WASM SIMD128 maps TL1 LUT to v128.swizzle for ~20-40 tok/s in browser.

Resolves open question #5 (WASM viability). Adds 6 new references,
5 new DDD terms, 3 new open questions. DDD updated to v2.4.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 14:07:52 +00:00
Claude
120a94e0a5 feat: Implement Phase 0 PT-BitNet quantizer module
Add bitnet/ module with absmean ternary quantizer, TernaryTensor type,
BITNET_T158 dequantization, and comprehensive test suite (~1600 lines).

Components:
- quantizer.rs: PtBitnetConfig, absmean_ternary(), quantize_tensor()
- ternary_tensor.rs: TernaryTensor, pack/unpack 2-bit ternary encoding
- dequantize.rs: dequantize_bitnet_t158(), block dequant, error metrics
- tests.rs: Packing roundtrips, quantization correctness, edge cases
- gguf/quantization.rs: BitnetT158 = 30 enum variant, block_size, bytes

Implements AD-1 (weight representation), AD-5 (GGUF extension), AD-18
(PT-BitNet quantization) from ADR-017.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 12:40:18 +00:00
Claude
94c760d9cf docs: Add AD-20 SIMD-only training mode for Phase 0.5 in ADR/DDD
Analyze RLM training stack GPU dependencies and document that Phase 0.5
runs entirely on pure CPU SIMD (NEON on aarch64) without Metal GPU.
MicroLoRA, TrainingPipeline, EwcRegularizer, GrpoOptimizer are all pure
ndarray; ContrastiveTrainer has explicit CPU fallback. Only ~2-3x slower
than Metal. Extends platform support to Linux ARM64 and x86 (scalar).

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 07:46:59 +00:00
Claude
d612fa61ff docs: Integrate RLM training stack into Craftsman Ultra ADR/DDD
Add Phase 0.5: RLM Post-Quantization Refinement — a $0 Mac Studio
approach that uses the existing RLM stack (MicroLoRA, GRPO, EWC++,
ContrastiveTrainer, MemoryDistiller, PolicyStore) to refine the
Phase 0 PTQ model by training only FP16 components (~1-2% of params).

ADR-017 changes:
- Added Phase 0.5 to phased decision: A(0C) → RLM Refinement → D → C → B
- Added AD-19: RLM Post-Quantization Refinement architecture
  - Frozen ternary weights + trainable FP16 (LoRA, router, scales)
  - ~200-400M trainable params (1-2% of 30B), 100-500M training tokens
  - 100% RLM code reuse, 0% new training code
  - 2-12 days on Mac Studio Metal, $0 cost
  - Expected quality: ~70-80% of FP16 (up from 55-65% Phase 0 PTQ)
- Full pipeline diagram: Router repair → MicroLoRA injection → Scale opt
- Memory budget analysis: ~12-20 GB active RAM (fits any Mac Studio)
- Training schedule: 3-14 days total wall time
- Added Phase 0.5 exit criteria (11 items)
- Updated infrastructure table with Phase 0.5 row
- Updated consequences with RLM refinement benefits

DDD v2.2 changes:
- Added Section 3.8.1: Phase 0.5 RLM Refinement Mode
- Added 5 ubiquitous language terms (RLM Refinement, Frozen Ternary,
  LoRA Correction, Router Repair)
- Added 3 open questions (LoRA rank, GGUF persistence, Phase continuity)

Key insight: RLM trains ~1% of parameters → needs ~0.25% of the data
(100-500M vs 200B tokens) → Mac Studio Metal is sufficient → $0 cost.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 06:43:52 +00:00
Claude
45d30c7c6e docs: Add Mac Studio as $0 Phase 0 PTQ platform in ADR-017
Update AD-17 and AD-18 to reflect that Phase 0 post-training quantization
runs entirely on Mac Studio (Apple Silicon) at zero cost, eliminating the
need for cloud GPU for the prototype phase.

Key changes:
- Phase 0 cost updated from ~$100 (cloud) to $0 (local Mac Studio)
- AD-18 now includes Mac Studio config compatibility matrix (M4 Max 36-128GB,
  M3 Ultra 96-512GB) with wall time estimates per config
- Added mmap strategy: FP16 weights demand-paged from disk, per-tensor
  quantization uses ~2-4MB working memory regardless of model size
- Metal GPU calibration via existing Candle integration (use_metal: true)
- ARM NEON for TL1 kernel validation (same ISA as production target)
- Updated throughput table with Mac Studio entries and Phase 0 column
- PtBitnetConfig gains use_mmap, use_metal_calibration, max_memory_gb fields
- Phase 0 exit criteria updated for Mac Studio local execution
- Updated infrastructure table: Phase 0 + router validation both $0 local

Mac Studio is ideal for Phase 0 (PTQ in hours, $0) but still infeasible
for Phase 1+ training (200B tokens at 500-1000 tok/s = 6.5 years).
This separation validates the phased cloud-for-training approach.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 06:30:05 +00:00
Claude
5896d6dc39 docs: Add Phase 0 PTQ rapid prototype to Craftsman Ultra ADR/DDD
Research post-training quantization feasibility for GLM-4.7-Flash as a
low-cost ($100, 2-4 hrs) validation step before full distillation ($1,300+).

ADR-017 changes:
- Restructured Option A from "Rejected" to tiered PTQ analysis (0A-0D)
- Added AD-18: PT-BitNet post-training quantization strategy
- Updated phased decision to A(0C) → D → C → B
- Added Phase 0 exit criteria and validation benchmarks
- Documented existing community GGUFs (bartowski, unsloth, ngxson)
- Identified RuvLLM IQ1_S dequant gap (type 19 parsed, not implemented)
- Added PT-BitNet, BitDistill, and STBLLM references

DDD v2.1 changes:
- Added 6 Phase 0 ubiquitous language terms (PT-BitNet, BITNET_T158, etc.)
- Updated Section 3.4 with dual-mode quantization pipeline (PTQ + distillation)
- Updated compatibility matrix with Phase 0 vs Phase 1+ columns
- Added 3 new open questions (calibration corpus, GGUF type, weight migration)

Key finding: IQ1_S ≠ BitNet b1.58. Generic codebook PTQ produces garbled
output; PT-BitNet absmean ternary quantization is viable for kernel validation.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 04:56:28 +00:00
Claude
d5caa4dacf docs: Add AD-17 training infrastructure analysis (cloud GPU vs local SIMD)
ADR-017: Add AD-17 with detailed memory budget analysis showing per-expert
distillation fits in A100 40GB (~15.5GB), full model requires 4×A100 80GB
(~430GB). CPU SIMD training infeasible at 200B+ tokens (~65 years on AVX2).
Recommend GCP 4×A100 spot instances (~$1,300 for Phase 1) or DataCrunch
H100 ($1.99/hr). Includes cost comparison across 6 platforms, per-phase
infrastructure mapping, and required CUDA device dispatch code change for
RealContrastiveTrainer.

DDD: Add section 8.5 Training Infrastructure Model with expert-parallel
GPU topology diagram, what-runs-where matrix, and required code change
summary.

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 04:44:29 +00:00
Claude
261b510d8e docs: Integrate RLM training stack into Craftsman Ultra ADR/DDD
ADR-017 updates:
- Add RLM Training Stack reuse section (GRPO, EWC++, ContrastiveTrainer,
  MemoryDistiller, PolicyStore — ~70% code reuse ratio)
- Add AD-11: GRPO-guided distillation with per-expert reward scaling
- Add AD-12: Contrastive pre-training for expert routing validation
- Add AD-13: EWC++ cross-expert stability during sequential distillation
- Add AD-14: PolicyStore TernaryScale per-layer policy persistence
- Add AD-15: MemoryDistiller trajectory tracking for distillation quality
- Add AD-16: Full pipeline composition with expert-parallel distillation
- Update Options C/D with RLM component mapping tables
- Update consequences, risks, and validation criteria

DDD v2.0 updates:
- Add bounded context 3.8: RLM Training Orchestration (70% reused)
- Add 13 ubiquitous language terms for RLM concepts
- Update context map with RLM relationships
- Update Quantization Pipeline to delegate to RLM Training
- Add 7 new domain events for GRPO, EWC, and distillation lifecycle
- Update module structure with reused vs new file annotations
- Add 6 RLM-specific integration tests
- Add 3 new open questions for RLM scaling

https://claude.ai/code/session_011nTcGcn49b8YKJRVoh4TaK
2026-02-03 04:34:47 +00:00