diff --git a/docs/adr/ADR-256-metaharness-sdk-evaluation.md b/docs/adr/ADR-256-metaharness-sdk-evaluation.md new file mode 100644 index 000000000..e0c46abeb --- /dev/null +++ b/docs/adr/ADR-256-metaharness-sdk-evaluation.md @@ -0,0 +1,245 @@ +--- +adr: 256 +title: "Borrowing `metaharness` concepts into `npx ruvector` using primitives we already ship (router, witness, MCP, SONA, memory)" +status: proposed +date: 2026-06-16 +authors: [ruvnet, claude-flow] +related: [ADR-026, ADR-103, ADR-122, ADR-134, ADR-252, ADR-255] +tags: [cli, npx, metaharness, harness, mcp, agent-sdk, router, witness-chain, startup-latency, dependency-policy, tooling] +--- + +# ADR-256 — Borrowing `metaharness` concepts into `npx ruvector` + +> **Decision in one line.** `metaharness` is a real, same-ecosystem (rUv) npm +> package — but a **v0.1.x harness *generator*, not a runtime SDK** — and the +> capabilities it advertises (cost-optimal model routing, agentic tool surface, +> signed releases, memory + learning loops) are concepts **ruvector already +> ships natively** (Tiny Dancer cost-router, semantic router, MCP server, +> witness chain, SONA, hooks routing). Notably the dependency runs the *other* +> way: `metaharness` optionally depends on `@ruvector/ruvllm` and links +> `@ruvector/emergent-time`. So we **borrow the good concepts and implement them +> with the primitives we already use — we do NOT take the package as a runtime +> dependency.** + +## Context + +`npx ruvector` is the CLI published as the `ruvector` npm package (currently +**v0.2.31**, source at `npm/packages/ruvector/`). Its entry point +`bin/cli.js` is a `commander@11` CLI that **lazy-loads** every heavy dependency +specifically to protect cold-start latency: the Rust/WASM core (`../dist/index.js`), +`@ruvector/gnn`, `@ruvector/attention`, `@ruvector/sona`, `@ruvector/rvf`, +`@ruvector/tiny-dancer`, and `@modelcontextprotocol/sdk`. It already ships an +**MCP server** (`bin/mcp-server.js`) that exposes its commands as an agentic +tool surface to any MCP host (Claude Code, etc.). + +The question raised: *could we improve `npx ruvector` by adopting the "npm +metaharness SDK"?* The term `metaharness` does **not** appear in this monorepo +(only an unrelated `meta-harness` string in tiny-dancer training), so the first +job was to establish what is actually being referred to before deciding. + +A multi-source, evidence-graded research pass (npm registry + GitHub APIs + web) +established the following. + +### Identity finding (Confidence: High, Evidence: H) + +There is an npm package literally named **`metaharness`** (v0.1.14, published +**2026-06-16**), authored by **rUv (ruv@ruv.net)** — the same author/ecosystem +as `ruvector`. Repo: [`ruvnet/agent-harness-generator`](https://github.com/ruvnet/agent-harness-generator). + +It is a **harness generator / factory**, not an embeddable runtime SDK. Tagline: +*"mint a custom AI agent harness from any repo… The model is replaceable. The +harness is the product."* You point it at a repo and it scaffolds an +npm-publishable, branded agent CLI (custom `npx `, multi-agent pods, an MCP +server with default-deny security, Ed25519 witness-signed releases, a memory +namespace + learning loops), targeting six interactive hosts (Claude Code, +Codex, pi.dev, Hermes, OpenClaw, RVM). + +- Bins: `metaharness` → `dist/bin.js`, `harness` → `dist/harness-bin.js` +- Direct deps are tiny (`kolorist`, `prompts`) — no Anthropic SDK, no MCP SDK, + no `commander` bundled. +- Peer: `@metaharness/kernel@0.1.0` (Rust + WASM + NAPI, optional). Optional: + `@ruvector/ruvllm`. Its kernel links `@ruvector/emergent-time` — corroborated + locally by the untracked `docs/research/emergent-time/` dir in git status. +- Maturity: pre-1.0 beta (GitHub tag v0.1.3 vs npm 0.1.14 diverge), 117 stars, + ~3,305 downloads/last month, CI green on Node 20/22 (download count/package.json + Evidence H; feature/maturity claims single-source Evidence M). + +**Name collisions ruled out** (all exist, none are the npm package): Superagentic +`metaharness` (Python/PyPI harness *optimizer*), Databricks **Omnigent** +(meta-harness for *composing/governing* agents), Strands **`harness-sdk`** (a +genuine TS/Python agent-harness SDK — the closest match had the user meant "a +harness SDK" generically), and the **Claude Agent SDK** (Anthropic's agent-loop +SDK). Given the author identity, the `@ruvector/*` linkage, the `npx` framing, +and the local `emergent-time` work, the intended referent is rUv's own npm +`metaharness` (Confidence: High). The "SDK" label is imprecise — the package is a +CLI/generator; the runtime pieces are separate `@metaharness/kernel` / +`@metaharness/router` packages. + +### What `npx ruvector` already ships (the "concepts we use") + +The CLI is not a thin wrapper — it already exposes most of metaharness's harness +concepts through primitives this repo built and maintains: + +- **Cost-optimal model router (Tiny Dancer).** `ruvector tiny-dancer train` + trains a FastGRNN router from a DRACO dataset (rows of `{embedding, scores}`) + into a `.safetensors` model; `tiny-dancer route` scores a query embedding — + *high = the cheap model is good enough → route cheap, else route stronger* + (`bin/cli.js:1879–1950`, ADR-252). This **is** metaharness's "learned + cost-optimal routing," already native and shipping for linux/macos/windows. +- **Semantic router.** `ruvector router --route` over `@ruvector/router` / + `ruvector-router-core` ("Core vector database and neural routing inference + engine") — text → best-matching intent (`bin/cli.js:1832–1879`). +- **Hooks / intelligence routing.** `npx ruvector hooks route ""` routes a + task to an agent/model via the intelligence engine's `route()` (the 3-tier + model routing of ADR-026: Agent-Booster → Haiku → Sonnet/Opus). +- **Agentic tool surface.** The bundled **MCP server** (`bin/mcp-server.js`, + `@modelcontextprotocol/sdk`) already exposes commands as LLM-drivable tools. +- **Witness-signed provenance.** Existing witness chain / manifest machinery + (ADR-103, ADR-134) — the basis for signed releases. +- **Memory + learning loops.** SONA (LoRA/EWC++), ReasoningBank, pi-brain shared + intelligence — persistent, self-learning state. + +### Concept → ruvector primitive (what we borrow vs. what we already have) + +| metaharness concept | ruvector primitive we already use | Action | +|---|---|---| +| Learned cost-optimal model routing | **Tiny Dancer** FastGRNN cost-router (ADR-252) | Reuse; surface as the harness router | +| Semantic intent routing | `@ruvector/router` / `ruvector-router-core` | Reuse | +| Multi-tier model routing | `hooks route` + 3-tier routing (ADR-026) | Reuse | +| Agentic tool exposure (MCP) | Bundled MCP server (`bin/mcp-server.js`) | Reuse; harden to default-deny | +| Ed25519 witness-signed releases | Witness chain / manifest (ADR-103, ADR-134) | Extend to release signing | +| Memory namespace + learning loops | SONA + ReasoningBank + pi-brain | Reuse; document a stable namespace | +| Default-deny security posture | (gap) | **Borrow** as a convention | +| "Harness is the product" framing | (gap) | **Borrow** as documentation framing | + +## Decision + +**Borrow the metaharness *concepts* and deliver them by composing ruvector +primitives we already ship — do NOT take `metaharness`/`@metaharness/*` as a +runtime dependency.** + +Concretely: + +1. **Do NOT add `metaharness`, `@metaharness/kernel`, or `@metaharness/router` + as a runtime dependency of `ruvector`.** It is a build-time generator that + re-scaffolds a *new* CLI, not a retrofit; its kernel is a Rust+WASM+NAPI beta + whose init cost conflicts with the CLI's ms-startup invariant (Evidence H — + grounded in the lazy-load architecture in `cli.js`). The dependency direction + also runs *toward* ruvector (`metaharness` → `@ruvector/ruvllm` / + `@ruvector/emergent-time`), so taking it as a dep would be circular. +2. **Promote the router we already have as the harness's cost-optimal router.** + Use **Tiny Dancer** (ADR-252) as the canonical model-routing primitive and + the **semantic router** for intent dispatch — this is the headline + metaharness feature, already native. No `@metaharness/router` needed. +3. **Treat the shipped MCP server as the canonical agentic surface** and harden + it toward the **default-deny allowlist** posture metaharness advocates. +4. **Borrow three metaharness conventions, building on our own machinery:** + (a) default-deny MCP tool allowlist; (b) Ed25519 witness-signed releases by + *extending* the existing witness chain (ADR-103/ADR-134), not replacing it; + (c) a stable, documented `ruvector` memory namespace over SONA/ReasoningBank. +5. **If an embedded agent loop inside `npx ruvector` is ever wanted** (the CLI + *drives* an LLM — a capability MCP does not provide), prefer the **mature + Claude Agent SDK directly**, behind a lazy-loaded optional subcommand (e.g. + `ruvector agent`), wired to the Tiny Dancer router for model selection. +6. **Re-evaluate a `@metaharness/*` dependency only** if it reaches ≥1.0 with a + stable embeddable contract *and* a benchmark shows it beating Tiny Dancer. + +This is a **borrow-concepts + dependency-policy** decision, not a refactor. +Items 2–4 reuse shipping primitives and are near-zero blast radius. + +## Consequences + +### Positive +- **We get the metaharness feature set by reusing what we own.** The headline + capability — learned cost-optimal routing — is already shipping as Tiny Dancer + (ADR-252); the agentic surface is the bundled MCP server; provenance is the + witness chain. Borrowing concepts costs assembly, not a new engine. +- **Zero startup regression and zero new audit burden** — the ms-startup + invariant verified in `cli.js` is preserved; no Rust+WASM+NAPI beta kernel on + the load path. +- **No circular dependency.** Since `metaharness` depends on `@ruvector/ruvllm` + and `@ruvector/emergent-time`, keeping ruvector dependency-free of + `@metaharness/*` avoids an inverted/circular coupling. +- Reuses prior decisions: ADR-026 (3-tier routing), ADR-103/ADR-134 (witness), + ADR-122 (RVF container), ADR-252 (Tiny Dancer cost-router). + +### Negative +- **Borrowing is integration work, not free.** Surfacing Tiny Dancer + semantic + router as a coherent "harness router," hardening MCP to default-deny, and + extending witness signing to releases are real tasks that must be tracked, not + left implicit. +- We forgo whatever a future shared `@metaharness/router` *might* offer; gated by + decision item 6 (must beat Tiny Dancer on benchmark). +- Two routers (Tiny Dancer cost-router + semantic router) under one "harness + router" framing risks confusion; the docs must state which routes what + (cost/tier vs. intent). + +### Neutral +- No new runtime dependency; `metaharness` stays an external, same-ecosystem + reference whose good ideas we adopt as conventions. Revisit cheaply at ≥1.0. + +## Alternatives considered + +| Option | Verdict | Why | +|---|---|---| +| **Borrow concepts, reuse our primitives** | **Adopt** | Tiny Dancer router + MCP + witness + SONA already deliver the feature set; zero dep, zero startup cost, no circular coupling. | +| Adopt npm `metaharness` generator | Reject | Re-scaffolds a *new* CLI; does not improve a mature one. Headline benefit already native. | +| Depend on `@metaharness/kernel`/`router` | Reject (now) | Rust+WASM+NAPI 0.1.0 beta; overlaps Tiny Dancer + `@ruvector/router` + witness; threatens ms-startup; circular dep. Re-evaluate ≥1.0. | +| Generic harness SDK (Strands / Claude Agent SDK) | Defer | Embedded agent loop is a *new capability*. If wanted, Claude Agent SDK direct, lazy + optional, fed by Tiny Dancer routing. | +| Stricter commander / oclif | Reject | Off-target — CLI ergonomics, not the agentic-harness question. | + +## Rollout sketch (borrow-concepts adoption) + +0. **Now:** document the "harness router" framing — Tiny Dancer (cost/tier) + + semantic router (intent) + `hooks route` as one coherent surface; harden the + MCP server to a default-deny allowlist; document the `ruvector` memory + namespace over SONA/ReasoningBank. +1. **Next:** extend the witness chain (ADR-103/ADR-134) to cover release + signing (Ed25519), matching metaharness's signed-release pattern with our own + machinery. +2. **Guardrail:** add a CI assertion that cold `npx ruvector --help` startup + stays within its current ms budget, so no borrowed feature regresses startup. +3. **Optional future capability:** an embedded agent loop (`ruvector agent`) via + the Claude Agent SDK, lazy + optional, model selection driven by Tiny Dancer. +4. **Gate for any `@metaharness/*` dep:** ≥1.0 + stable embeddable contract + + benchmark beating Tiny Dancer; never bundled into the default load path. + +## Implementation status (as shipped) + +Tracked in issue #574 / PR #575 (branch `adr-256-harness-router`). + +| Rollout item | Status | Where | +|---|---|---| +| 0. Harness router surface | **Done** | `ruvector harness status [--json]` (`bin/cli.js`) — unifies cost router, semantic router, hooks routing, MCP, witness, memory; degrades gracefully | +| 1. Default-deny MCP allowlist | **Done** | `bin/mcp-policy.js` + `mcp-server.js` gate; `RUVECTOR_MCP_ALLOW/DENY/PROFILE`; verified end-to-end over MCP stdio | +| 3. Memory namespace | **Done** | `RUVECTOR_MEMORY_NAMESPACE` (default `ruvector`), surfaced as `memory.namespace` | +| 4. Startup-budget guard | **Done** | `test/startup-budget.js` — abs ceiling + relative delta (harness adds ≈ +0–3ms) | +| Benchmarked / verified | **Done** | full `npm test` green (cli 73/0, mcp-policy 8/0, startup 2/0, db-workflow, integration, sigterm) | +| 2. Witness-signed releases | **Convention** | documented extension of the existing witness chain (ADR-103/ADR-134); no new signing engine built | +| 5. Embedded `ruvector agent` loop | **Deferred** | optional future capability via Claude Agent SDK, per the decision | + +No `@metaharness/*` runtime dependency was added; the borrowed capabilities are +delivered entirely by primitives ruvector already ships. + +## Links +- npm `metaharness` (v0.1.14, 2026-06-16, author rUv): + https://registry.npmjs.org/metaharness/latest · + downloads: https://api.npmjs.org/downloads/point/last-month/metaharness +- Generator repo: https://github.com/ruvnet/agent-harness-generator +- Name collisions (NOT this package): Superagentic metaharness + (https://superagenticai.github.io/metaharness/) · Databricks Omnigent + (https://www.databricks.com/blog/introducing-omnigent-meta-harness-combine-control-and-share-your-agents) · + Strands harness-sdk (https://github.com/strands-agents/harness-sdk) · + awesome-harness-engineering (https://github.com/ai-boost/awesome-harness-engineering) +- Local grounding: `npm/packages/ruvector/bin/cli.js` — lazy-load startup + architecture, Tiny Dancer cost-router (`:1879–1950`), semantic router + (`:1832–1879`), `hooks route` (`:4391`); `ruvector-router-core` ("Core vector + database and neural routing inference engine"); `package.json` (v0.2.31, + `@modelcontextprotocol/sdk` + `@ruvector/router` peer dep); `bin/mcp-server.js` + (shipped MCP surface); git status (`docs/research/emergent-time/` untracked, + corroborating the `@ruvector/emergent-time` linkage). +- Prior decisions: ADR-026 (3-tier model routing), ADR-103 (witness manifest), + ADR-122 (RVF container), ADR-134 (witness chain), ADR-252 (Tiny Dancer + FastGRNN cost-router), ADR-255 (OIA alignment profile). +- Source: deep-research brief (npm/GitHub registry APIs + web) cross-referenced + with the ruvector CLI source. diff --git a/npm/packages/ruvector/README.md b/npm/packages/ruvector/README.md index 80d401241..d0d963f71 100644 --- a/npm/packages/ruvector/README.md +++ b/npm/packages/ruvector/README.md @@ -91,6 +91,38 @@ claude mcp add ruvector -- npx ruvector mcp start - `midstream_status`, `midstream_attractor`, `midstream_scheduler` — Streaming analysis - `midstream_benchmark`, `midstream_search`, `midstream_health` — Latency benchmarks + health +**MCP tool-access policy (default-deny, ADR-256):** restrict the exposed/callable +tool surface with environment variables — useful for least-privilege deployments. + +```bash +# Only expose specific tools (everything else is denied) +RUVECTOR_MCP_ALLOW="hooks_route,hooks_recall" npx ruvector mcp start + +# Block specific tools (deny wins over allow) +RUVECTOR_MCP_DENY="hooks_force_learn" npx ruvector mcp start + +# Apply a curated read-only profile (safe, non-mutating subset) +RUVECTOR_MCP_PROFILE=readonly npx ruvector mcp start +``` + +Precedence is **DENY > ALLOW/PROFILE > allow-all**. With no policy set, all tools +are available (backward compatible). Inspect the active posture with +`npx ruvector harness status --json` (see `mcp.accessControl`). + +### Harness Router (ADR-256) + +`ruvector harness` surfaces the unified routing/agentic primitives ruvector ships — +cost-optimal model routing (Tiny Dancer), semantic routing, hooks routing, the MCP +server, witness-signed provenance, and SONA memory — in one place: + +```bash +npx ruvector harness status # human-readable surface + availability +npx ruvector harness status --json # structured, for tooling/CI +``` + +Memory + learning loops use a stable namespace (default `ruvector`), overridable per +deployment with `RUVECTOR_MEMORY_NAMESPACE` and reported under `memory.namespace`. + ### Brain AGI Commands Access all 8 AGI subsystems deployed at π.ruv.io: diff --git a/npm/packages/ruvector/bin/cli.js b/npm/packages/ruvector/bin/cli.js index ec6472287..2bb2a42f2 100755 --- a/npm/packages/ruvector/bin/cli.js +++ b/npm/packages/ruvector/bin/cli.js @@ -10101,6 +10101,137 @@ const optimizeCmd = program.command('optimize') console.log(''); }); +// ============================================================================= +// Harness Commands - unified "harness router" surface (ADR-256) +// Borrows metaharness concepts using primitives ruvector already ships: +// cost router (tiny-dancer) + semantic router + hooks routing + MCP + witness +// Read-only status surface; degrades gracefully when optional deps are absent. +// ============================================================================= + +function buildHarnessSurface() { + const primitives = {}; + + // Cost-optimal model router — Tiny Dancer FastGRNN (ADR-252) + try { + const td = require('@ruvector/tiny-dancer'); + primitives.costRouter = { + name: '@ruvector/tiny-dancer', + role: 'cost-optimal model routing (cheap vs strong)', + available: true, + version: typeof td.version === 'function' ? td.version() : null, + usage: 'npx ruvector tiny-dancer score --query ', + }; + } catch { + primitives.costRouter = { + name: '@ruvector/tiny-dancer', + role: 'cost-optimal model routing (cheap vs strong)', + available: false, + install: 'npm install @ruvector/tiny-dancer', + }; + } + + // Semantic intent router — @ruvector/router / ruvector-router-core + let semanticAvailable = false; + try { require.resolve('@ruvector/router'); semanticAvailable = true; } catch { semanticAvailable = false; } + primitives.semanticRouter = { + name: '@ruvector/router', + role: 'semantic intent routing', + available: semanticAvailable, + ...(semanticAvailable ? { usage: 'npx ruvector router --route ""' } : { install: 'npm install @ruvector/router' }), + }; + + // Multi-tier intelligence routing — bundled (ADR-026) + primitives.hooksRouting = { + name: 'hooks route', + role: '3-tier task→agent/model routing (ADR-026)', + available: true, + usage: 'npx ruvector hooks route ""', + }; + + // Agentic tool surface — bundled MCP server (with ADR-256 default-deny policy) + const mcpPath = path.join(__dirname, 'mcp-server.js'); + let mcpPolicy = { configured: false }; + try { + const { buildToolPolicy } = require('./mcp-policy.js'); + const p = buildToolPolicy(process.env); + mcpPolicy = { + configured: p.configured, + profile: p.profile || null, + allow: p.allowSet ? p.allowSet.size : 0, + deny: p.deny.size, + }; + } catch { /* policy module optional */ } + primitives.mcp = { + name: 'mcp-server', + role: 'agentic tool surface (Model Context Protocol)', + available: fs.existsSync(mcpPath), + usage: 'npx ruvector mcp start', + policy: mcpPolicy, + accessControl: mcpPolicy.configured ? 'default-deny (configured)' : 'allow-all (set RUVECTOR_MCP_ALLOW/PROFILE)', + }; + + // Signed provenance — witness chain (ADR-103 / ADR-134) + primitives.witness = { + name: 'witness-chain', + role: 'signed provenance / release signing (ADR-103, ADR-134)', + available: true, + }; + + // Memory + learning loops — SONA / ReasoningBank (stable namespace, ADR-256 step 3) + primitives.memory = { + name: 'sona+reasoningbank', + role: 'persistent memory + self-learning loops', + available: true, + namespace: (process.env.RUVECTOR_MEMORY_NAMESPACE || 'ruvector').trim() || 'ruvector', + }; + + const values = Object.values(primitives); + return { + adr: 'ADR-256', + decision: 'borrow metaharness concepts using primitives ruvector already ships', + primitives, + summary: { + available: values.filter((p) => p.available).length, + total: values.length, + }, + }; +} + +const harnessCmd = program + .command('harness') + .description('Unified "harness router" surface — cost router + semantic router + hooks routing + MCP + witness (ADR-256)'); + +function printHarnessStatus(opts) { + const surface = buildHarnessSurface(); + if (opts && opts.json) { + console.log(JSON.stringify(surface, null, 2)); + return; + } + console.log(chalk.cyan('\n═══════════════════════════════════════════════════════════════')); + console.log(chalk.cyan(' RuVector Harness Router (ADR-256)')); + console.log(chalk.cyan('═══════════════════════════════════════════════════════════════\n')); + console.log(chalk.gray(' ' + surface.decision + '\n')); + for (const p of Object.values(surface.primitives)) { + const badge = p.available ? chalk.green('● available') : chalk.yellow('○ optional '); + console.log(` ${badge} ${chalk.white(p.name)}${p.version ? chalk.dim(' v' + p.version) : ''}`); + console.log(` ${chalk.dim(p.role)}`); + if (p.available && p.usage) console.log(` ${chalk.dim(p.usage)}`); + if (!p.available && p.install) console.log(` ${chalk.dim('install: ' + p.install)}`); + } + console.log(''); + console.log(chalk.cyan(` ${surface.summary.available}/${surface.summary.total} primitives available\n`)); +} + +harnessCmd + .command('status') + .alias('info') + .description('Show the unified harness routing surface and primitive availability') + .option('--json', 'Output as JSON') + .action((opts) => printHarnessStatus(opts)); + +// Bare `ruvector harness` defaults to status +harnessCmd.action(() => printHarnessStatus({})); + program.parse(); diff --git a/npm/packages/ruvector/bin/mcp-policy.js b/npm/packages/ruvector/bin/mcp-policy.js new file mode 100644 index 000000000..a1d282aef --- /dev/null +++ b/npm/packages/ruvector/bin/mcp-policy.js @@ -0,0 +1,95 @@ +'use strict'; + +/** + * MCP tool-access policy (ADR-256 — default-deny posture). + * + * Borrows metaharness's "default-deny allowlist" security concept using a small, + * pure, testable module. Operators restrict the exposed/callable MCP tool surface + * via environment variables: + * + * RUVECTOR_MCP_ALLOW="hooks_route,hooks_recall" → only these are exposed/callable + * RUVECTOR_MCP_DENY="hooks_security_scan" → these are blocked (wins over allow) + * RUVECTOR_MCP_PROFILE=readonly → a curated safe subset + * + * Precedence: DENY > ALLOW/PROFILE > (default allow-all). + * With no policy configured, all registered tools are allowed (backward compatible); + * `policy.configured` is false so the server can nudge toward an allowlist. + * + * This module is dependency-free and side-effect-free so it can be unit-tested + * without spawning an MCP stdio server. + */ + +/** Curated read-only / low-risk tool subsets. */ +const MCP_PROFILES = { + readonly: [ + 'ruvector', + 'hooks_stats', + 'hooks_recall', + 'hooks_route', + 'hooks_route_enhanced', + 'hooks_suggest_context', + 'hooks_capabilities', + 'hooks_export', + 'hooks_doctor', + 'hooks_attention_info', + 'hooks_gnn_info', + ], +}; + +/** Split a comma/space separated env list into a clean array. */ +function parseList(value) { + if (!value || typeof value !== 'string') return []; + return value + .split(/[,\s]+/) + .map((s) => s.trim()) + .filter(Boolean); +} + +/** + * Build a tool-access policy from an env-like object (defaults to process.env). + * Returns { allowSet: Set|null, deny: Set, profile: string|null, configured: bool }. + */ +function buildToolPolicy(env) { + const e = env || process.env; + const allow = parseList(e.RUVECTOR_MCP_ALLOW); + const deny = new Set(parseList(e.RUVECTOR_MCP_DENY)); + const profileName = e.RUVECTOR_MCP_PROFILE && e.RUVECTOR_MCP_PROFILE.trim(); + + let allowSet = null; + if (profileName && MCP_PROFILES[profileName]) { + allowSet = new Set(MCP_PROFILES[profileName]); + } + if (allow.length) { + allowSet = new Set([...(allowSet || []), ...allow]); + } + + return { + allowSet, + deny, + profile: profileName && MCP_PROFILES[profileName] ? profileName : null, + // An unknown profile name with no allow/deny is treated as "not configured". + configured: Boolean(allowSet || deny.size), + }; +} + +/** Is a tool name permitted under the given policy? DENY always wins. */ +function isToolAllowed(name, policy) { + if (!policy) return true; + if (policy.deny.has(name)) return false; + if (policy.allowSet) return policy.allowSet.has(name); + return true; +} + +/** Filter a TOOLS array (objects with a `.name`) down to the allowed subset. */ +function filterAllowedTools(tools, policy) { + if (!Array.isArray(tools)) return []; + return tools.filter((t) => t && isToolAllowed(t.name, policy)); +} + +module.exports = { + MCP_PROFILES, + parseList, + buildToolPolicy, + isToolAllowed, + filterAllowedTools, +}; diff --git a/npm/packages/ruvector/bin/mcp-server.js b/npm/packages/ruvector/bin/mcp-server.js index 7c4185444..bf568f3d2 100644 --- a/npm/packages/ruvector/bin/mcp-server.js +++ b/npm/packages/ruvector/bin/mcp-server.js @@ -26,6 +26,10 @@ const path = require('path'); const fs = require('fs'); const { execSync, execFileSync } = require('child_process'); +// ADR-256: default-deny MCP tool-access policy (RUVECTOR_MCP_ALLOW/DENY/PROFILE) +const { buildToolPolicy, isToolAllowed, filterAllowedTools } = require('./mcp-policy.js'); +const MCP_TOOL_POLICY = buildToolPolicy(process.env); + // ── Security Helpers ──────────────────────────────────────────────────────── /** @@ -1692,15 +1696,30 @@ const TOOLS = [ } ]; -// List tools handler +// List tools handler — only expose tools permitted by the access policy (ADR-256) server.setRequestHandler(ListToolsRequestSchema, async () => { - return { tools: TOOLS }; + return { tools: filterAllowedTools(TOOLS, MCP_TOOL_POLICY) }; }); // Call tool handler server.setRequestHandler(CallToolRequestSchema, async (request) => { const { name, arguments: args } = request.params; + // ADR-256 default-deny gate: refuse tools excluded by RUVECTOR_MCP_ALLOW/DENY/PROFILE + if (!isToolAllowed(name, MCP_TOOL_POLICY)) { + return { + content: [{ + type: 'text', + text: JSON.stringify({ + success: false, + error: `Tool '${name}' is denied by the MCP access policy (ADR-256). ` + + `Adjust RUVECTOR_MCP_ALLOW / RUVECTOR_MCP_DENY / RUVECTOR_MCP_PROFILE to permit it.`, + }, null, 2), + }], + isError: true, + }; + } + try { switch (name) { case 'hooks_stats': { diff --git a/npm/packages/ruvector/package.json b/npm/packages/ruvector/package.json index 5b4a5d4c0..9197e8004 100644 --- a/npm/packages/ruvector/package.json +++ b/npm/packages/ruvector/package.json @@ -12,7 +12,7 @@ "verify-dist": "node scripts/verify-dist.js", "prepack": "npm run build && npm run verify-dist", "prepublishOnly": "npm run build && npm run verify-dist", - "test": "node test/integration.js && node test/cli-commands.js && node test/db-workflow.js && node test/sigterm-cleanup.js" + "test": "node test/integration.js && node test/cli-commands.js && node test/db-workflow.js && node test/sigterm-cleanup.js && node test/mcp-policy.js && node test/startup-budget.js" }, "keywords": [ "vector", diff --git a/npm/packages/ruvector/test/cli-commands.js b/npm/packages/ruvector/test/cli-commands.js index b4b998fcb..3dca28213 100644 --- a/npm/packages/ruvector/test/cli-commands.js +++ b/npm/packages/ruvector/test/cli-commands.js @@ -552,6 +552,51 @@ test('demo --help shows options', () => { assert(out.includes('demo'), 'Should show demo info'); }); +// ---- Section 24: Harness router surface (ADR-256) ------------------------ +console.log('\n--- 24. Harness router surface (ADR-256) ---\n'); + +test('harness --help describes the unified surface', () => { + const out = run('harness --help'); + assert(out.includes('harness'), 'Should show harness command'); + assert(/status/.test(out), 'Should list the status subcommand'); +}); + +test('harness status --json returns a valid structured surface', () => { + const out = run('harness status --json'); + const surface = JSON.parse(out); + assert(surface.adr === 'ADR-256', 'adr field should be ADR-256'); + assert(surface.primitives && typeof surface.primitives === 'object', 'should have primitives'); + // The router concepts ADR-256 borrows must be present + for (const key of ['costRouter', 'semanticRouter', 'hooksRouting', 'mcp', 'witness', 'memory']) { + assert(surface.primitives[key], `primitives.${key} should exist`); + assert(typeof surface.primitives[key].available === 'boolean', `${key}.available should be boolean`); + assert(typeof surface.primitives[key].role === 'string', `${key}.role should be a string`); + } + // Bundled primitives are always available regardless of optional deps + assert(surface.primitives.hooksRouting.available === true, 'hooks routing is bundled'); + assert(surface.primitives.mcp.available === true, 'mcp server is bundled'); + assert(surface.summary && surface.summary.total === Object.keys(surface.primitives).length, + 'summary.total should match primitive count'); + // ADR-256 step 1 + 3: MCP access-control posture and a stable memory namespace + assert(typeof surface.primitives.mcp.accessControl === 'string', 'mcp.accessControl should be reported'); + assert(surface.primitives.mcp.policy && typeof surface.primitives.mcp.policy.configured === 'boolean', + 'mcp.policy.configured should be a boolean'); + assert(typeof surface.primitives.memory.namespace === 'string' && surface.primitives.memory.namespace.length > 0, + 'memory.namespace should default to a non-empty value'); +}); + +test('harness honors RUVECTOR_MEMORY_NAMESPACE override', () => { + const out = run('harness status --json', { env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1', RUVECTOR_MEMORY_NAMESPACE: 'team-x' } }); + const surface = JSON.parse(out); + assert(surface.primitives.memory.namespace === 'team-x', 'should reflect the namespace override'); +}); + +test('bare harness command prints status without crashing', () => { + const { code, stdout } = runSafe('harness'); + assert(code === 0, 'bare harness should exit 0'); + assert(/Harness Router/.test(stdout), 'should print the harness router header'); +}); + // --------------------------------------------------------------------------- // Summary // --------------------------------------------------------------------------- diff --git a/npm/packages/ruvector/test/mcp-policy.js b/npm/packages/ruvector/test/mcp-policy.js new file mode 100644 index 000000000..8f495a73c --- /dev/null +++ b/npm/packages/ruvector/test/mcp-policy.js @@ -0,0 +1,127 @@ +#!/usr/bin/env node + +/** + * Unit tests for the MCP tool-access policy (ADR-256 default-deny posture). + * Pure module — no MCP stdio server required. + */ + +const assert = require('assert'); +const { + MCP_PROFILES, + parseList, + buildToolPolicy, + isToolAllowed, + filterAllowedTools, +} = require('../bin/mcp-policy.js'); + +let passed = 0; +let failed = 0; +const failures = []; + +function test(name, fn) { + try { + fn(); + passed++; + console.log(` PASS ${name}`); + } catch (err) { + failed++; + failures.push({ name, error: err.message || String(err) }); + console.log(` FAIL ${name}`); + console.log(` ${err.message || err}`); + } +} + +const TOOLS = [ + { name: 'ruvector' }, + { name: 'hooks_route' }, + { name: 'hooks_recall' }, + { name: 'hooks_security_scan' }, + { name: 'hooks_force_learn' }, +]; + +console.log('\n--- MCP tool-access policy (ADR-256) ---\n'); + +test('parseList handles comma and whitespace separators', () => { + assert.deepStrictEqual(parseList('a, b ,c'), ['a', 'b', 'c']); + assert.deepStrictEqual(parseList('a b\tc'), ['a', 'b', 'c']); + assert.deepStrictEqual(parseList(''), []); + assert.deepStrictEqual(parseList(undefined), []); +}); + +test('no env configured → allow-all, not configured', () => { + const p = buildToolPolicy({}); + assert.strictEqual(p.configured, false); + assert.strictEqual(p.allowSet, null); + for (const t of TOOLS) assert.strictEqual(isToolAllowed(t.name, p), true); + assert.strictEqual(filterAllowedTools(TOOLS, p).length, TOOLS.length); +}); + +test('RUVECTOR_MCP_ALLOW restricts to the listed tools (deny rest)', () => { + const p = buildToolPolicy({ RUVECTOR_MCP_ALLOW: 'hooks_route, hooks_recall' }); + assert.strictEqual(p.configured, true); + assert.strictEqual(isToolAllowed('hooks_route', p), true); + assert.strictEqual(isToolAllowed('hooks_recall', p), true); + assert.strictEqual(isToolAllowed('hooks_security_scan', p), false); + assert.strictEqual(isToolAllowed('ruvector', p), false); + assert.deepStrictEqual( + filterAllowedTools(TOOLS, p).map((t) => t.name).sort(), + ['hooks_recall', 'hooks_route'], + ); +}); + +test('RUVECTOR_MCP_DENY blocks specific tools, allows the rest', () => { + const p = buildToolPolicy({ RUVECTOR_MCP_DENY: 'hooks_force_learn' }); + assert.strictEqual(p.configured, true); + assert.strictEqual(isToolAllowed('hooks_force_learn', p), false); + assert.strictEqual(isToolAllowed('hooks_route', p), true); +}); + +test('DENY wins over ALLOW (precedence)', () => { + const p = buildToolPolicy({ + RUVECTOR_MCP_ALLOW: 'hooks_route,hooks_force_learn', + RUVECTOR_MCP_DENY: 'hooks_force_learn', + }); + assert.strictEqual(isToolAllowed('hooks_route', p), true); + assert.strictEqual(isToolAllowed('hooks_force_learn', p), false); +}); + +test('RUVECTOR_MCP_PROFILE=readonly applies the curated subset', () => { + const p = buildToolPolicy({ RUVECTOR_MCP_PROFILE: 'readonly' }); + assert.strictEqual(p.profile, 'readonly'); + assert.strictEqual(p.configured, true); + for (const name of MCP_PROFILES.readonly) { + assert.strictEqual(isToolAllowed(name, p), true, `${name} should be allowed in readonly`); + } + // A mutating tool not in the profile is denied + assert.strictEqual(isToolAllowed('hooks_force_learn', p), false); +}); + +test('profile + extra ALLOW are unioned', () => { + const p = buildToolPolicy({ + RUVECTOR_MCP_PROFILE: 'readonly', + RUVECTOR_MCP_ALLOW: 'hooks_force_learn', + }); + assert.strictEqual(isToolAllowed('hooks_force_learn', p), true); + assert.strictEqual(isToolAllowed('hooks_stats', p), true); +}); + +test('unknown profile with no allow/deny → not configured (allow-all)', () => { + const p = buildToolPolicy({ RUVECTOR_MCP_PROFILE: 'does-not-exist' }); + assert.strictEqual(p.profile, null); + assert.strictEqual(p.configured, false); + assert.strictEqual(isToolAllowed('hooks_force_learn', p), true); +}); + +console.log('\n' + '='.repeat(60)); +console.log(`\nResults: ${passed} passed, ${failed} failed\n`); +if (failures.length > 0) { + console.log('Failures:'); + for (const f of failures) console.log(` - ${f.name}: ${f.error}`); + console.log(''); +} +if (failed > 0) { + console.log('SOME TESTS FAILED\n'); + process.exit(1); +} else { + console.log('ALL TESTS PASSED\n'); +} diff --git a/npm/packages/ruvector/test/startup-budget.js b/npm/packages/ruvector/test/startup-budget.js new file mode 100644 index 000000000..9e41f6659 --- /dev/null +++ b/npm/packages/ruvector/test/startup-budget.js @@ -0,0 +1,88 @@ +#!/usr/bin/env node + +/** + * Startup-budget CI guard (ADR-256 step 4). + * + * The ruvector CLI lazy-loads every heavy dependency to keep cold start fast. + * ADR-256 borrows harness features (the `harness` router surface, MCP policy) + * into the CLI — this guard fails CI if any of them regress startup. + * + * Strategy (robust across machines / loaded CI boxes): + * 1. ABSOLUTE ceiling — `--help` cold start must stay under a generous cap. + * 2. RELATIVE delta — `harness status --json` must not add more than a small + * delta over the `--help` baseline (the real regression signal: eager-loading + * a heavy module into the harness path would blow this out). + * + * Most of the floor (~100ms) is the Node process spawn itself, which is why the + * relative delta is the meaningful check. Budgets are overridable via env. + */ +'use strict'; + +const { execSync } = require('child_process'); +const path = require('path'); +const assert = require('assert'); + +const CLI = path.join(__dirname, '..', 'bin', 'cli.js'); +const CWD = path.join(__dirname, '..'); + +const ABS_BUDGET_MS = Number(process.env.RUVECTOR_STARTUP_BUDGET_MS || 2000); +const DELTA_BUDGET_MS = Number(process.env.RUVECTOR_STARTUP_DELTA_MS || 120); +const SAMPLES = Number(process.env.RUVECTOR_STARTUP_SAMPLES || 5); + +function timeMs(args) { + const start = process.hrtime.bigint(); + try { + execSync(`node ${CLI} ${args}`, { stdio: ['pipe', 'pipe', 'pipe'], timeout: 20000, cwd: CWD }); + } catch { + // non-zero exit still yields a valid timing + } + return Number(process.hrtime.bigint() - start) / 1e6; +} + +function median(args) { + const t = []; + for (let i = 0; i < SAMPLES; i++) t.push(timeMs(args)); + t.sort((a, b) => a - b); + return t[Math.floor(t.length / 2)]; +} + +let passed = 0; +let failed = 0; +const failures = []; +function test(name, fn) { + try { fn(); passed++; console.log(` PASS ${name}`); } + catch (err) { failed++; failures.push({ name, error: err.message || String(err) }); console.log(` FAIL ${name}`); console.log(` ${err.message || err}`); } +} + +console.log('\n--- Startup-budget guard (ADR-256) ---\n'); +console.log(` samples=${SAMPLES} abs_budget=${ABS_BUDGET_MS}ms delta_budget=${DELTA_BUDGET_MS}ms\n`); + +// Warm up (filesystem cache, AV scan of the file, etc.) +timeMs('--version'); + +const helpMs = median('--help'); +const harnessMs = median('harness status --json'); +const delta = harnessMs - helpMs; + +console.log(` --help (cold): ${helpMs.toFixed(0)}ms`); +console.log(` harness status --json: ${harnessMs.toFixed(0)}ms (Δ ${delta >= 0 ? '+' : ''}${delta.toFixed(0)}ms vs --help)\n`); + +test(`--help cold start under ${ABS_BUDGET_MS}ms absolute budget`, () => { + assert(helpMs < ABS_BUDGET_MS, + `--help startup ${helpMs.toFixed(0)}ms exceeds ${ABS_BUDGET_MS}ms (set RUVECTOR_STARTUP_BUDGET_MS to override)`); +}); + +test(`harness status adds < ${DELTA_BUDGET_MS}ms over --help baseline (no lazy-load regression)`, () => { + assert(delta < DELTA_BUDGET_MS, + `harness status added ${delta.toFixed(0)}ms over --help — a heavy module may have leaked into the startup path`); +}); + +console.log('\n' + '='.repeat(60)); +console.log(`\nResults: ${passed} passed, ${failed} failed\n`); +if (failures.length > 0) { + console.log('Failures:'); + for (const f of failures) console.log(` - ${f.name}: ${f.error}`); + console.log(''); +} +if (failed > 0) { console.log('STARTUP BUDGET EXCEEDED\n'); process.exit(1); } +else { console.log('STARTUP WITHIN BUDGET\n'); }