mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-07-09 17:28:42 +00:00
ADR-256: harness router surface (borrow metaharness concepts) (#575)
* feat(ruvector): ADR-256 harness router surface + tracking (#574) Borrow metaharness concepts using primitives ruvector already ships. - Add `ruvector harness status [--json]` — unified read-only view of the routing surface (Tiny Dancer cost router + semantic router + hooks routing + MCP + witness + memory), degrading gracefully when optional deps are absent. Implements ADR-256 rollout step 0. - Add ADR-256 (borrow-concepts decision, concept→primitive mapping). - Add CLI tests (Section 24): harness --help, status --json structure, bare-command behavior. Full suite: 72 passed, 0 failed. Refs #574 Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ruvector): ADR-256 default-deny MCP tool-access policy (#574) Borrow metaharness's default-deny allowlist concept with our own machinery. - New pure, testable bin/mcp-policy.js: RUVECTOR_MCP_ALLOW / RUVECTOR_MCP_DENY / RUVECTOR_MCP_PROFILE=readonly. Precedence DENY > ALLOW/PROFILE > allow-all. No policy set = backward-compatible allow-all (policy.configured=false). - Wire into mcp-server.js: ListTools now returns only permitted tools; CallTool gates denied tools with an isError response before dispatch. - harness status --json now reports mcp.policy + accessControl posture. - Tests: test/mcp-policy.js (8 unit tests) wired into npm test; verified end-to-end over MCP stdio (readonly profile exposes 10 safe tools, filters hooks_force_learn). CLI suite still 72/0. Refs #574 Co-Authored-By: claude-flow <ruv@ruv.net> * test(ruvector): ADR-256 startup-budget guard + harness/MCP-policy docs (#574) - New test/startup-budget.js wired into npm test: absolute ceiling on `--help` cold start + relative delta guard ensuring `harness status` adds < 120ms over baseline (catches a heavy module leaking into the startup path). Measured here: --help 127ms, harness +3ms. Env-overridable. - README: document the default-deny MCP policy env vars (RUVECTOR_MCP_ALLOW/DENY/PROFILE) and the `harness` router command. Refs #574 Co-Authored-By: claude-flow <ruv@ruv.net> * feat(ruvector): ADR-256 memory namespace + full verification, ADR status (#574) - harness surface reports a stable memory namespace (RUVECTOR_MEMORY_NAMESPACE, default `ruvector`); CLI tests assert the default + override and the MCP accessControl/policy fields. - README documents the memory namespace. - ADR-256: add "Implementation status (as shipped)" — items 0/1/3/4 done, benchmarked + full npm test green; item 2 as a documented convention; item 5 deferred. No @metaharness/* runtime dep. Full suite: cli 73/0, mcp-policy 8/0, startup-budget 2/0, db-workflow/integration/sigterm green. Refs #574 Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: ruv <ruvnet@users.noreply.github.com>
This commit is contained in:
parent
716dbede1d
commit
d5347d514b
9 changed files with 785 additions and 3 deletions
245
docs/adr/ADR-256-metaharness-sdk-evaluation.md
Normal file
245
docs/adr/ADR-256-metaharness-sdk-evaluation.md
Normal file
|
|
@ -0,0 +1,245 @@
|
|||
---
|
||||
adr: 256
|
||||
title: "Borrowing `metaharness` concepts into `npx ruvector` using primitives we already ship (router, witness, MCP, SONA, memory)"
|
||||
status: proposed
|
||||
date: 2026-06-16
|
||||
authors: [ruvnet, claude-flow]
|
||||
related: [ADR-026, ADR-103, ADR-122, ADR-134, ADR-252, ADR-255]
|
||||
tags: [cli, npx, metaharness, harness, mcp, agent-sdk, router, witness-chain, startup-latency, dependency-policy, tooling]
|
||||
---
|
||||
|
||||
# ADR-256 — Borrowing `metaharness` concepts into `npx ruvector`
|
||||
|
||||
> **Decision in one line.** `metaharness` is a real, same-ecosystem (rUv) npm
|
||||
> package — but a **v0.1.x harness *generator*, not a runtime SDK** — and the
|
||||
> capabilities it advertises (cost-optimal model routing, agentic tool surface,
|
||||
> signed releases, memory + learning loops) are concepts **ruvector already
|
||||
> ships natively** (Tiny Dancer cost-router, semantic router, MCP server,
|
||||
> witness chain, SONA, hooks routing). Notably the dependency runs the *other*
|
||||
> way: `metaharness` optionally depends on `@ruvector/ruvllm` and links
|
||||
> `@ruvector/emergent-time`. So we **borrow the good concepts and implement them
|
||||
> with the primitives we already use — we do NOT take the package as a runtime
|
||||
> dependency.**
|
||||
|
||||
## Context
|
||||
|
||||
`npx ruvector` is the CLI published as the `ruvector` npm package (currently
|
||||
**v0.2.31**, source at `npm/packages/ruvector/`). Its entry point
|
||||
`bin/cli.js` is a `commander@11` CLI that **lazy-loads** every heavy dependency
|
||||
specifically to protect cold-start latency: the Rust/WASM core (`../dist/index.js`),
|
||||
`@ruvector/gnn`, `@ruvector/attention`, `@ruvector/sona`, `@ruvector/rvf`,
|
||||
`@ruvector/tiny-dancer`, and `@modelcontextprotocol/sdk`. It already ships an
|
||||
**MCP server** (`bin/mcp-server.js`) that exposes its commands as an agentic
|
||||
tool surface to any MCP host (Claude Code, etc.).
|
||||
|
||||
The question raised: *could we improve `npx ruvector` by adopting the "npm
|
||||
metaharness SDK"?* The term `metaharness` does **not** appear in this monorepo
|
||||
(only an unrelated `meta-harness` string in tiny-dancer training), so the first
|
||||
job was to establish what is actually being referred to before deciding.
|
||||
|
||||
A multi-source, evidence-graded research pass (npm registry + GitHub APIs + web)
|
||||
established the following.
|
||||
|
||||
### Identity finding (Confidence: High, Evidence: H)
|
||||
|
||||
There is an npm package literally named **`metaharness`** (v0.1.14, published
|
||||
**2026-06-16**), authored by **rUv (ruv@ruv.net)** — the same author/ecosystem
|
||||
as `ruvector`. Repo: [`ruvnet/agent-harness-generator`](https://github.com/ruvnet/agent-harness-generator).
|
||||
|
||||
It is a **harness generator / factory**, not an embeddable runtime SDK. Tagline:
|
||||
*"mint a custom AI agent harness from any repo… The model is replaceable. The
|
||||
harness is the product."* You point it at a repo and it scaffolds an
|
||||
npm-publishable, branded agent CLI (custom `npx <name>`, multi-agent pods, an MCP
|
||||
server with default-deny security, Ed25519 witness-signed releases, a memory
|
||||
namespace + learning loops), targeting six interactive hosts (Claude Code,
|
||||
Codex, pi.dev, Hermes, OpenClaw, RVM).
|
||||
|
||||
- Bins: `metaharness` → `dist/bin.js`, `harness` → `dist/harness-bin.js`
|
||||
- Direct deps are tiny (`kolorist`, `prompts`) — no Anthropic SDK, no MCP SDK,
|
||||
no `commander` bundled.
|
||||
- Peer: `@metaharness/kernel@0.1.0` (Rust + WASM + NAPI, optional). Optional:
|
||||
`@ruvector/ruvllm`. Its kernel links `@ruvector/emergent-time` — corroborated
|
||||
locally by the untracked `docs/research/emergent-time/` dir in git status.
|
||||
- Maturity: pre-1.0 beta (GitHub tag v0.1.3 vs npm 0.1.14 diverge), 117 stars,
|
||||
~3,305 downloads/last month, CI green on Node 20/22 (download count/package.json
|
||||
Evidence H; feature/maturity claims single-source Evidence M).
|
||||
|
||||
**Name collisions ruled out** (all exist, none are the npm package): Superagentic
|
||||
`metaharness` (Python/PyPI harness *optimizer*), Databricks **Omnigent**
|
||||
(meta-harness for *composing/governing* agents), Strands **`harness-sdk`** (a
|
||||
genuine TS/Python agent-harness SDK — the closest match had the user meant "a
|
||||
harness SDK" generically), and the **Claude Agent SDK** (Anthropic's agent-loop
|
||||
SDK). Given the author identity, the `@ruvector/*` linkage, the `npx` framing,
|
||||
and the local `emergent-time` work, the intended referent is rUv's own npm
|
||||
`metaharness` (Confidence: High). The "SDK" label is imprecise — the package is a
|
||||
CLI/generator; the runtime pieces are separate `@metaharness/kernel` /
|
||||
`@metaharness/router` packages.
|
||||
|
||||
### What `npx ruvector` already ships (the "concepts we use")
|
||||
|
||||
The CLI is not a thin wrapper — it already exposes most of metaharness's harness
|
||||
concepts through primitives this repo built and maintains:
|
||||
|
||||
- **Cost-optimal model router (Tiny Dancer).** `ruvector tiny-dancer train`
|
||||
trains a FastGRNN router from a DRACO dataset (rows of `{embedding, scores}`)
|
||||
into a `.safetensors` model; `tiny-dancer route` scores a query embedding —
|
||||
*high = the cheap model is good enough → route cheap, else route stronger*
|
||||
(`bin/cli.js:1879–1950`, ADR-252). This **is** metaharness's "learned
|
||||
cost-optimal routing," already native and shipping for linux/macos/windows.
|
||||
- **Semantic router.** `ruvector router --route` over `@ruvector/router` /
|
||||
`ruvector-router-core` ("Core vector database and neural routing inference
|
||||
engine") — text → best-matching intent (`bin/cli.js:1832–1879`).
|
||||
- **Hooks / intelligence routing.** `npx ruvector hooks route "<task>"` routes a
|
||||
task to an agent/model via the intelligence engine's `route()` (the 3-tier
|
||||
model routing of ADR-026: Agent-Booster → Haiku → Sonnet/Opus).
|
||||
- **Agentic tool surface.** The bundled **MCP server** (`bin/mcp-server.js`,
|
||||
`@modelcontextprotocol/sdk`) already exposes commands as LLM-drivable tools.
|
||||
- **Witness-signed provenance.** Existing witness chain / manifest machinery
|
||||
(ADR-103, ADR-134) — the basis for signed releases.
|
||||
- **Memory + learning loops.** SONA (LoRA/EWC++), ReasoningBank, pi-brain shared
|
||||
intelligence — persistent, self-learning state.
|
||||
|
||||
### Concept → ruvector primitive (what we borrow vs. what we already have)
|
||||
|
||||
| metaharness concept | ruvector primitive we already use | Action |
|
||||
|---|---|---|
|
||||
| Learned cost-optimal model routing | **Tiny Dancer** FastGRNN cost-router (ADR-252) | Reuse; surface as the harness router |
|
||||
| Semantic intent routing | `@ruvector/router` / `ruvector-router-core` | Reuse |
|
||||
| Multi-tier model routing | `hooks route` + 3-tier routing (ADR-026) | Reuse |
|
||||
| Agentic tool exposure (MCP) | Bundled MCP server (`bin/mcp-server.js`) | Reuse; harden to default-deny |
|
||||
| Ed25519 witness-signed releases | Witness chain / manifest (ADR-103, ADR-134) | Extend to release signing |
|
||||
| Memory namespace + learning loops | SONA + ReasoningBank + pi-brain | Reuse; document a stable namespace |
|
||||
| Default-deny security posture | (gap) | **Borrow** as a convention |
|
||||
| "Harness is the product" framing | (gap) | **Borrow** as documentation framing |
|
||||
|
||||
## Decision
|
||||
|
||||
**Borrow the metaharness *concepts* and deliver them by composing ruvector
|
||||
primitives we already ship — do NOT take `metaharness`/`@metaharness/*` as a
|
||||
runtime dependency.**
|
||||
|
||||
Concretely:
|
||||
|
||||
1. **Do NOT add `metaharness`, `@metaharness/kernel`, or `@metaharness/router`
|
||||
as a runtime dependency of `ruvector`.** It is a build-time generator that
|
||||
re-scaffolds a *new* CLI, not a retrofit; its kernel is a Rust+WASM+NAPI beta
|
||||
whose init cost conflicts with the CLI's ms-startup invariant (Evidence H —
|
||||
grounded in the lazy-load architecture in `cli.js`). The dependency direction
|
||||
also runs *toward* ruvector (`metaharness` → `@ruvector/ruvllm` /
|
||||
`@ruvector/emergent-time`), so taking it as a dep would be circular.
|
||||
2. **Promote the router we already have as the harness's cost-optimal router.**
|
||||
Use **Tiny Dancer** (ADR-252) as the canonical model-routing primitive and
|
||||
the **semantic router** for intent dispatch — this is the headline
|
||||
metaharness feature, already native. No `@metaharness/router` needed.
|
||||
3. **Treat the shipped MCP server as the canonical agentic surface** and harden
|
||||
it toward the **default-deny allowlist** posture metaharness advocates.
|
||||
4. **Borrow three metaharness conventions, building on our own machinery:**
|
||||
(a) default-deny MCP tool allowlist; (b) Ed25519 witness-signed releases by
|
||||
*extending* the existing witness chain (ADR-103/ADR-134), not replacing it;
|
||||
(c) a stable, documented `ruvector` memory namespace over SONA/ReasoningBank.
|
||||
5. **If an embedded agent loop inside `npx ruvector` is ever wanted** (the CLI
|
||||
*drives* an LLM — a capability MCP does not provide), prefer the **mature
|
||||
Claude Agent SDK directly**, behind a lazy-loaded optional subcommand (e.g.
|
||||
`ruvector agent`), wired to the Tiny Dancer router for model selection.
|
||||
6. **Re-evaluate a `@metaharness/*` dependency only** if it reaches ≥1.0 with a
|
||||
stable embeddable contract *and* a benchmark shows it beating Tiny Dancer.
|
||||
|
||||
This is a **borrow-concepts + dependency-policy** decision, not a refactor.
|
||||
Items 2–4 reuse shipping primitives and are near-zero blast radius.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- **We get the metaharness feature set by reusing what we own.** The headline
|
||||
capability — learned cost-optimal routing — is already shipping as Tiny Dancer
|
||||
(ADR-252); the agentic surface is the bundled MCP server; provenance is the
|
||||
witness chain. Borrowing concepts costs assembly, not a new engine.
|
||||
- **Zero startup regression and zero new audit burden** — the ms-startup
|
||||
invariant verified in `cli.js` is preserved; no Rust+WASM+NAPI beta kernel on
|
||||
the load path.
|
||||
- **No circular dependency.** Since `metaharness` depends on `@ruvector/ruvllm`
|
||||
and `@ruvector/emergent-time`, keeping ruvector dependency-free of
|
||||
`@metaharness/*` avoids an inverted/circular coupling.
|
||||
- Reuses prior decisions: ADR-026 (3-tier routing), ADR-103/ADR-134 (witness),
|
||||
ADR-122 (RVF container), ADR-252 (Tiny Dancer cost-router).
|
||||
|
||||
### Negative
|
||||
- **Borrowing is integration work, not free.** Surfacing Tiny Dancer + semantic
|
||||
router as a coherent "harness router," hardening MCP to default-deny, and
|
||||
extending witness signing to releases are real tasks that must be tracked, not
|
||||
left implicit.
|
||||
- We forgo whatever a future shared `@metaharness/router` *might* offer; gated by
|
||||
decision item 6 (must beat Tiny Dancer on benchmark).
|
||||
- Two routers (Tiny Dancer cost-router + semantic router) under one "harness
|
||||
router" framing risks confusion; the docs must state which routes what
|
||||
(cost/tier vs. intent).
|
||||
|
||||
### Neutral
|
||||
- No new runtime dependency; `metaharness` stays an external, same-ecosystem
|
||||
reference whose good ideas we adopt as conventions. Revisit cheaply at ≥1.0.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
| Option | Verdict | Why |
|
||||
|---|---|---|
|
||||
| **Borrow concepts, reuse our primitives** | **Adopt** | Tiny Dancer router + MCP + witness + SONA already deliver the feature set; zero dep, zero startup cost, no circular coupling. |
|
||||
| Adopt npm `metaharness` generator | Reject | Re-scaffolds a *new* CLI; does not improve a mature one. Headline benefit already native. |
|
||||
| Depend on `@metaharness/kernel`/`router` | Reject (now) | Rust+WASM+NAPI 0.1.0 beta; overlaps Tiny Dancer + `@ruvector/router` + witness; threatens ms-startup; circular dep. Re-evaluate ≥1.0. |
|
||||
| Generic harness SDK (Strands / Claude Agent SDK) | Defer | Embedded agent loop is a *new capability*. If wanted, Claude Agent SDK direct, lazy + optional, fed by Tiny Dancer routing. |
|
||||
| Stricter commander / oclif | Reject | Off-target — CLI ergonomics, not the agentic-harness question. |
|
||||
|
||||
## Rollout sketch (borrow-concepts adoption)
|
||||
|
||||
0. **Now:** document the "harness router" framing — Tiny Dancer (cost/tier) +
|
||||
semantic router (intent) + `hooks route` as one coherent surface; harden the
|
||||
MCP server to a default-deny allowlist; document the `ruvector` memory
|
||||
namespace over SONA/ReasoningBank.
|
||||
1. **Next:** extend the witness chain (ADR-103/ADR-134) to cover release
|
||||
signing (Ed25519), matching metaharness's signed-release pattern with our own
|
||||
machinery.
|
||||
2. **Guardrail:** add a CI assertion that cold `npx ruvector --help` startup
|
||||
stays within its current ms budget, so no borrowed feature regresses startup.
|
||||
3. **Optional future capability:** an embedded agent loop (`ruvector agent`) via
|
||||
the Claude Agent SDK, lazy + optional, model selection driven by Tiny Dancer.
|
||||
4. **Gate for any `@metaharness/*` dep:** ≥1.0 + stable embeddable contract +
|
||||
benchmark beating Tiny Dancer; never bundled into the default load path.
|
||||
|
||||
## Implementation status (as shipped)
|
||||
|
||||
Tracked in issue #574 / PR #575 (branch `adr-256-harness-router`).
|
||||
|
||||
| Rollout item | Status | Where |
|
||||
|---|---|---|
|
||||
| 0. Harness router surface | **Done** | `ruvector harness status [--json]` (`bin/cli.js`) — unifies cost router, semantic router, hooks routing, MCP, witness, memory; degrades gracefully |
|
||||
| 1. Default-deny MCP allowlist | **Done** | `bin/mcp-policy.js` + `mcp-server.js` gate; `RUVECTOR_MCP_ALLOW/DENY/PROFILE`; verified end-to-end over MCP stdio |
|
||||
| 3. Memory namespace | **Done** | `RUVECTOR_MEMORY_NAMESPACE` (default `ruvector`), surfaced as `memory.namespace` |
|
||||
| 4. Startup-budget guard | **Done** | `test/startup-budget.js` — abs ceiling + relative delta (harness adds ≈ +0–3ms) |
|
||||
| Benchmarked / verified | **Done** | full `npm test` green (cli 73/0, mcp-policy 8/0, startup 2/0, db-workflow, integration, sigterm) |
|
||||
| 2. Witness-signed releases | **Convention** | documented extension of the existing witness chain (ADR-103/ADR-134); no new signing engine built |
|
||||
| 5. Embedded `ruvector agent` loop | **Deferred** | optional future capability via Claude Agent SDK, per the decision |
|
||||
|
||||
No `@metaharness/*` runtime dependency was added; the borrowed capabilities are
|
||||
delivered entirely by primitives ruvector already ships.
|
||||
|
||||
## Links
|
||||
- npm `metaharness` (v0.1.14, 2026-06-16, author rUv):
|
||||
https://registry.npmjs.org/metaharness/latest ·
|
||||
downloads: https://api.npmjs.org/downloads/point/last-month/metaharness
|
||||
- Generator repo: https://github.com/ruvnet/agent-harness-generator
|
||||
- Name collisions (NOT this package): Superagentic metaharness
|
||||
(https://superagenticai.github.io/metaharness/) · Databricks Omnigent
|
||||
(https://www.databricks.com/blog/introducing-omnigent-meta-harness-combine-control-and-share-your-agents) ·
|
||||
Strands harness-sdk (https://github.com/strands-agents/harness-sdk) ·
|
||||
awesome-harness-engineering (https://github.com/ai-boost/awesome-harness-engineering)
|
||||
- Local grounding: `npm/packages/ruvector/bin/cli.js` — lazy-load startup
|
||||
architecture, Tiny Dancer cost-router (`:1879–1950`), semantic router
|
||||
(`:1832–1879`), `hooks route` (`:4391`); `ruvector-router-core` ("Core vector
|
||||
database and neural routing inference engine"); `package.json` (v0.2.31,
|
||||
`@modelcontextprotocol/sdk` + `@ruvector/router` peer dep); `bin/mcp-server.js`
|
||||
(shipped MCP surface); git status (`docs/research/emergent-time/` untracked,
|
||||
corroborating the `@ruvector/emergent-time` linkage).
|
||||
- Prior decisions: ADR-026 (3-tier model routing), ADR-103 (witness manifest),
|
||||
ADR-122 (RVF container), ADR-134 (witness chain), ADR-252 (Tiny Dancer
|
||||
FastGRNN cost-router), ADR-255 (OIA alignment profile).
|
||||
- Source: deep-research brief (npm/GitHub registry APIs + web) cross-referenced
|
||||
with the ruvector CLI source.
|
||||
|
|
@ -91,6 +91,38 @@ claude mcp add ruvector -- npx ruvector mcp start
|
|||
- `midstream_status`, `midstream_attractor`, `midstream_scheduler` — Streaming analysis
|
||||
- `midstream_benchmark`, `midstream_search`, `midstream_health` — Latency benchmarks + health
|
||||
|
||||
**MCP tool-access policy (default-deny, ADR-256):** restrict the exposed/callable
|
||||
tool surface with environment variables — useful for least-privilege deployments.
|
||||
|
||||
```bash
|
||||
# Only expose specific tools (everything else is denied)
|
||||
RUVECTOR_MCP_ALLOW="hooks_route,hooks_recall" npx ruvector mcp start
|
||||
|
||||
# Block specific tools (deny wins over allow)
|
||||
RUVECTOR_MCP_DENY="hooks_force_learn" npx ruvector mcp start
|
||||
|
||||
# Apply a curated read-only profile (safe, non-mutating subset)
|
||||
RUVECTOR_MCP_PROFILE=readonly npx ruvector mcp start
|
||||
```
|
||||
|
||||
Precedence is **DENY > ALLOW/PROFILE > allow-all**. With no policy set, all tools
|
||||
are available (backward compatible). Inspect the active posture with
|
||||
`npx ruvector harness status --json` (see `mcp.accessControl`).
|
||||
|
||||
### Harness Router (ADR-256)
|
||||
|
||||
`ruvector harness` surfaces the unified routing/agentic primitives ruvector ships —
|
||||
cost-optimal model routing (Tiny Dancer), semantic routing, hooks routing, the MCP
|
||||
server, witness-signed provenance, and SONA memory — in one place:
|
||||
|
||||
```bash
|
||||
npx ruvector harness status # human-readable surface + availability
|
||||
npx ruvector harness status --json # structured, for tooling/CI
|
||||
```
|
||||
|
||||
Memory + learning loops use a stable namespace (default `ruvector`), overridable per
|
||||
deployment with `RUVECTOR_MEMORY_NAMESPACE` and reported under `memory.namespace`.
|
||||
|
||||
### Brain AGI Commands
|
||||
|
||||
Access all 8 AGI subsystems deployed at π.ruv.io:
|
||||
|
|
|
|||
|
|
@ -10101,6 +10101,137 @@ const optimizeCmd = program.command('optimize')
|
|||
console.log('');
|
||||
});
|
||||
|
||||
// =============================================================================
|
||||
// Harness Commands - unified "harness router" surface (ADR-256)
|
||||
// Borrows metaharness concepts using primitives ruvector already ships:
|
||||
// cost router (tiny-dancer) + semantic router + hooks routing + MCP + witness
|
||||
// Read-only status surface; degrades gracefully when optional deps are absent.
|
||||
// =============================================================================
|
||||
|
||||
function buildHarnessSurface() {
|
||||
const primitives = {};
|
||||
|
||||
// Cost-optimal model router — Tiny Dancer FastGRNN (ADR-252)
|
||||
try {
|
||||
const td = require('@ruvector/tiny-dancer');
|
||||
primitives.costRouter = {
|
||||
name: '@ruvector/tiny-dancer',
|
||||
role: 'cost-optimal model routing (cheap vs strong)',
|
||||
available: true,
|
||||
version: typeof td.version === 'function' ? td.version() : null,
|
||||
usage: 'npx ruvector tiny-dancer score <model> --query <embedding>',
|
||||
};
|
||||
} catch {
|
||||
primitives.costRouter = {
|
||||
name: '@ruvector/tiny-dancer',
|
||||
role: 'cost-optimal model routing (cheap vs strong)',
|
||||
available: false,
|
||||
install: 'npm install @ruvector/tiny-dancer',
|
||||
};
|
||||
}
|
||||
|
||||
// Semantic intent router — @ruvector/router / ruvector-router-core
|
||||
let semanticAvailable = false;
|
||||
try { require.resolve('@ruvector/router'); semanticAvailable = true; } catch { semanticAvailable = false; }
|
||||
primitives.semanticRouter = {
|
||||
name: '@ruvector/router',
|
||||
role: 'semantic intent routing',
|
||||
available: semanticAvailable,
|
||||
...(semanticAvailable ? { usage: 'npx ruvector router --route "<text>"' } : { install: 'npm install @ruvector/router' }),
|
||||
};
|
||||
|
||||
// Multi-tier intelligence routing — bundled (ADR-026)
|
||||
primitives.hooksRouting = {
|
||||
name: 'hooks route',
|
||||
role: '3-tier task→agent/model routing (ADR-026)',
|
||||
available: true,
|
||||
usage: 'npx ruvector hooks route "<task>"',
|
||||
};
|
||||
|
||||
// Agentic tool surface — bundled MCP server (with ADR-256 default-deny policy)
|
||||
const mcpPath = path.join(__dirname, 'mcp-server.js');
|
||||
let mcpPolicy = { configured: false };
|
||||
try {
|
||||
const { buildToolPolicy } = require('./mcp-policy.js');
|
||||
const p = buildToolPolicy(process.env);
|
||||
mcpPolicy = {
|
||||
configured: p.configured,
|
||||
profile: p.profile || null,
|
||||
allow: p.allowSet ? p.allowSet.size : 0,
|
||||
deny: p.deny.size,
|
||||
};
|
||||
} catch { /* policy module optional */ }
|
||||
primitives.mcp = {
|
||||
name: 'mcp-server',
|
||||
role: 'agentic tool surface (Model Context Protocol)',
|
||||
available: fs.existsSync(mcpPath),
|
||||
usage: 'npx ruvector mcp start',
|
||||
policy: mcpPolicy,
|
||||
accessControl: mcpPolicy.configured ? 'default-deny (configured)' : 'allow-all (set RUVECTOR_MCP_ALLOW/PROFILE)',
|
||||
};
|
||||
|
||||
// Signed provenance — witness chain (ADR-103 / ADR-134)
|
||||
primitives.witness = {
|
||||
name: 'witness-chain',
|
||||
role: 'signed provenance / release signing (ADR-103, ADR-134)',
|
||||
available: true,
|
||||
};
|
||||
|
||||
// Memory + learning loops — SONA / ReasoningBank (stable namespace, ADR-256 step 3)
|
||||
primitives.memory = {
|
||||
name: 'sona+reasoningbank',
|
||||
role: 'persistent memory + self-learning loops',
|
||||
available: true,
|
||||
namespace: (process.env.RUVECTOR_MEMORY_NAMESPACE || 'ruvector').trim() || 'ruvector',
|
||||
};
|
||||
|
||||
const values = Object.values(primitives);
|
||||
return {
|
||||
adr: 'ADR-256',
|
||||
decision: 'borrow metaharness concepts using primitives ruvector already ships',
|
||||
primitives,
|
||||
summary: {
|
||||
available: values.filter((p) => p.available).length,
|
||||
total: values.length,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const harnessCmd = program
|
||||
.command('harness')
|
||||
.description('Unified "harness router" surface — cost router + semantic router + hooks routing + MCP + witness (ADR-256)');
|
||||
|
||||
function printHarnessStatus(opts) {
|
||||
const surface = buildHarnessSurface();
|
||||
if (opts && opts.json) {
|
||||
console.log(JSON.stringify(surface, null, 2));
|
||||
return;
|
||||
}
|
||||
console.log(chalk.cyan('\n═══════════════════════════════════════════════════════════════'));
|
||||
console.log(chalk.cyan(' RuVector Harness Router (ADR-256)'));
|
||||
console.log(chalk.cyan('═══════════════════════════════════════════════════════════════\n'));
|
||||
console.log(chalk.gray(' ' + surface.decision + '\n'));
|
||||
for (const p of Object.values(surface.primitives)) {
|
||||
const badge = p.available ? chalk.green('● available') : chalk.yellow('○ optional ');
|
||||
console.log(` ${badge} ${chalk.white(p.name)}${p.version ? chalk.dim(' v' + p.version) : ''}`);
|
||||
console.log(` ${chalk.dim(p.role)}`);
|
||||
if (p.available && p.usage) console.log(` ${chalk.dim(p.usage)}`);
|
||||
if (!p.available && p.install) console.log(` ${chalk.dim('install: ' + p.install)}`);
|
||||
}
|
||||
console.log('');
|
||||
console.log(chalk.cyan(` ${surface.summary.available}/${surface.summary.total} primitives available\n`));
|
||||
}
|
||||
|
||||
harnessCmd
|
||||
.command('status')
|
||||
.alias('info')
|
||||
.description('Show the unified harness routing surface and primitive availability')
|
||||
.option('--json', 'Output as JSON')
|
||||
.action((opts) => printHarnessStatus(opts));
|
||||
|
||||
// Bare `ruvector harness` defaults to status
|
||||
harnessCmd.action(() => printHarnessStatus({}));
|
||||
|
||||
program.parse();
|
||||
|
||||
|
||||
|
|
|
|||
95
npm/packages/ruvector/bin/mcp-policy.js
Normal file
95
npm/packages/ruvector/bin/mcp-policy.js
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
'use strict';
|
||||
|
||||
/**
|
||||
* MCP tool-access policy (ADR-256 — default-deny posture).
|
||||
*
|
||||
* Borrows metaharness's "default-deny allowlist" security concept using a small,
|
||||
* pure, testable module. Operators restrict the exposed/callable MCP tool surface
|
||||
* via environment variables:
|
||||
*
|
||||
* RUVECTOR_MCP_ALLOW="hooks_route,hooks_recall" → only these are exposed/callable
|
||||
* RUVECTOR_MCP_DENY="hooks_security_scan" → these are blocked (wins over allow)
|
||||
* RUVECTOR_MCP_PROFILE=readonly → a curated safe subset
|
||||
*
|
||||
* Precedence: DENY > ALLOW/PROFILE > (default allow-all).
|
||||
* With no policy configured, all registered tools are allowed (backward compatible);
|
||||
* `policy.configured` is false so the server can nudge toward an allowlist.
|
||||
*
|
||||
* This module is dependency-free and side-effect-free so it can be unit-tested
|
||||
* without spawning an MCP stdio server.
|
||||
*/
|
||||
|
||||
/** Curated read-only / low-risk tool subsets. */
|
||||
const MCP_PROFILES = {
|
||||
readonly: [
|
||||
'ruvector',
|
||||
'hooks_stats',
|
||||
'hooks_recall',
|
||||
'hooks_route',
|
||||
'hooks_route_enhanced',
|
||||
'hooks_suggest_context',
|
||||
'hooks_capabilities',
|
||||
'hooks_export',
|
||||
'hooks_doctor',
|
||||
'hooks_attention_info',
|
||||
'hooks_gnn_info',
|
||||
],
|
||||
};
|
||||
|
||||
/** Split a comma/space separated env list into a clean array. */
|
||||
function parseList(value) {
|
||||
if (!value || typeof value !== 'string') return [];
|
||||
return value
|
||||
.split(/[,\s]+/)
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a tool-access policy from an env-like object (defaults to process.env).
|
||||
* Returns { allowSet: Set|null, deny: Set, profile: string|null, configured: bool }.
|
||||
*/
|
||||
function buildToolPolicy(env) {
|
||||
const e = env || process.env;
|
||||
const allow = parseList(e.RUVECTOR_MCP_ALLOW);
|
||||
const deny = new Set(parseList(e.RUVECTOR_MCP_DENY));
|
||||
const profileName = e.RUVECTOR_MCP_PROFILE && e.RUVECTOR_MCP_PROFILE.trim();
|
||||
|
||||
let allowSet = null;
|
||||
if (profileName && MCP_PROFILES[profileName]) {
|
||||
allowSet = new Set(MCP_PROFILES[profileName]);
|
||||
}
|
||||
if (allow.length) {
|
||||
allowSet = new Set([...(allowSet || []), ...allow]);
|
||||
}
|
||||
|
||||
return {
|
||||
allowSet,
|
||||
deny,
|
||||
profile: profileName && MCP_PROFILES[profileName] ? profileName : null,
|
||||
// An unknown profile name with no allow/deny is treated as "not configured".
|
||||
configured: Boolean(allowSet || deny.size),
|
||||
};
|
||||
}
|
||||
|
||||
/** Is a tool name permitted under the given policy? DENY always wins. */
|
||||
function isToolAllowed(name, policy) {
|
||||
if (!policy) return true;
|
||||
if (policy.deny.has(name)) return false;
|
||||
if (policy.allowSet) return policy.allowSet.has(name);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Filter a TOOLS array (objects with a `.name`) down to the allowed subset. */
|
||||
function filterAllowedTools(tools, policy) {
|
||||
if (!Array.isArray(tools)) return [];
|
||||
return tools.filter((t) => t && isToolAllowed(t.name, policy));
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
MCP_PROFILES,
|
||||
parseList,
|
||||
buildToolPolicy,
|
||||
isToolAllowed,
|
||||
filterAllowedTools,
|
||||
};
|
||||
|
|
@ -26,6 +26,10 @@ const path = require('path');
|
|||
const fs = require('fs');
|
||||
const { execSync, execFileSync } = require('child_process');
|
||||
|
||||
// ADR-256: default-deny MCP tool-access policy (RUVECTOR_MCP_ALLOW/DENY/PROFILE)
|
||||
const { buildToolPolicy, isToolAllowed, filterAllowedTools } = require('./mcp-policy.js');
|
||||
const MCP_TOOL_POLICY = buildToolPolicy(process.env);
|
||||
|
||||
// ── Security Helpers ────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
|
|
@ -1692,15 +1696,30 @@ const TOOLS = [
|
|||
}
|
||||
];
|
||||
|
||||
// List tools handler
|
||||
// List tools handler — only expose tools permitted by the access policy (ADR-256)
|
||||
server.setRequestHandler(ListToolsRequestSchema, async () => {
|
||||
return { tools: TOOLS };
|
||||
return { tools: filterAllowedTools(TOOLS, MCP_TOOL_POLICY) };
|
||||
});
|
||||
|
||||
// Call tool handler
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
const { name, arguments: args } = request.params;
|
||||
|
||||
// ADR-256 default-deny gate: refuse tools excluded by RUVECTOR_MCP_ALLOW/DENY/PROFILE
|
||||
if (!isToolAllowed(name, MCP_TOOL_POLICY)) {
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
success: false,
|
||||
error: `Tool '${name}' is denied by the MCP access policy (ADR-256). ` +
|
||||
`Adjust RUVECTOR_MCP_ALLOW / RUVECTOR_MCP_DENY / RUVECTOR_MCP_PROFILE to permit it.`,
|
||||
}, null, 2),
|
||||
}],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
switch (name) {
|
||||
case 'hooks_stats': {
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
"verify-dist": "node scripts/verify-dist.js",
|
||||
"prepack": "npm run build && npm run verify-dist",
|
||||
"prepublishOnly": "npm run build && npm run verify-dist",
|
||||
"test": "node test/integration.js && node test/cli-commands.js && node test/db-workflow.js && node test/sigterm-cleanup.js"
|
||||
"test": "node test/integration.js && node test/cli-commands.js && node test/db-workflow.js && node test/sigterm-cleanup.js && node test/mcp-policy.js && node test/startup-budget.js"
|
||||
},
|
||||
"keywords": [
|
||||
"vector",
|
||||
|
|
|
|||
|
|
@ -552,6 +552,51 @@ test('demo --help shows options', () => {
|
|||
assert(out.includes('demo'), 'Should show demo info');
|
||||
});
|
||||
|
||||
// ---- Section 24: Harness router surface (ADR-256) ------------------------
|
||||
console.log('\n--- 24. Harness router surface (ADR-256) ---\n');
|
||||
|
||||
test('harness --help describes the unified surface', () => {
|
||||
const out = run('harness --help');
|
||||
assert(out.includes('harness'), 'Should show harness command');
|
||||
assert(/status/.test(out), 'Should list the status subcommand');
|
||||
});
|
||||
|
||||
test('harness status --json returns a valid structured surface', () => {
|
||||
const out = run('harness status --json');
|
||||
const surface = JSON.parse(out);
|
||||
assert(surface.adr === 'ADR-256', 'adr field should be ADR-256');
|
||||
assert(surface.primitives && typeof surface.primitives === 'object', 'should have primitives');
|
||||
// The router concepts ADR-256 borrows must be present
|
||||
for (const key of ['costRouter', 'semanticRouter', 'hooksRouting', 'mcp', 'witness', 'memory']) {
|
||||
assert(surface.primitives[key], `primitives.${key} should exist`);
|
||||
assert(typeof surface.primitives[key].available === 'boolean', `${key}.available should be boolean`);
|
||||
assert(typeof surface.primitives[key].role === 'string', `${key}.role should be a string`);
|
||||
}
|
||||
// Bundled primitives are always available regardless of optional deps
|
||||
assert(surface.primitives.hooksRouting.available === true, 'hooks routing is bundled');
|
||||
assert(surface.primitives.mcp.available === true, 'mcp server is bundled');
|
||||
assert(surface.summary && surface.summary.total === Object.keys(surface.primitives).length,
|
||||
'summary.total should match primitive count');
|
||||
// ADR-256 step 1 + 3: MCP access-control posture and a stable memory namespace
|
||||
assert(typeof surface.primitives.mcp.accessControl === 'string', 'mcp.accessControl should be reported');
|
||||
assert(surface.primitives.mcp.policy && typeof surface.primitives.mcp.policy.configured === 'boolean',
|
||||
'mcp.policy.configured should be a boolean');
|
||||
assert(typeof surface.primitives.memory.namespace === 'string' && surface.primitives.memory.namespace.length > 0,
|
||||
'memory.namespace should default to a non-empty value');
|
||||
});
|
||||
|
||||
test('harness honors RUVECTOR_MEMORY_NAMESPACE override', () => {
|
||||
const out = run('harness status --json', { env: { ...process.env, FORCE_COLOR: '0', NO_COLOR: '1', RUVECTOR_MEMORY_NAMESPACE: 'team-x' } });
|
||||
const surface = JSON.parse(out);
|
||||
assert(surface.primitives.memory.namespace === 'team-x', 'should reflect the namespace override');
|
||||
});
|
||||
|
||||
test('bare harness command prints status without crashing', () => {
|
||||
const { code, stdout } = runSafe('harness');
|
||||
assert(code === 0, 'bare harness should exit 0');
|
||||
assert(/Harness Router/.test(stdout), 'should print the harness router header');
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Summary
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
|
|||
127
npm/packages/ruvector/test/mcp-policy.js
Normal file
127
npm/packages/ruvector/test/mcp-policy.js
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Unit tests for the MCP tool-access policy (ADR-256 default-deny posture).
|
||||
* Pure module — no MCP stdio server required.
|
||||
*/
|
||||
|
||||
const assert = require('assert');
|
||||
const {
|
||||
MCP_PROFILES,
|
||||
parseList,
|
||||
buildToolPolicy,
|
||||
isToolAllowed,
|
||||
filterAllowedTools,
|
||||
} = require('../bin/mcp-policy.js');
|
||||
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
const failures = [];
|
||||
|
||||
function test(name, fn) {
|
||||
try {
|
||||
fn();
|
||||
passed++;
|
||||
console.log(` PASS ${name}`);
|
||||
} catch (err) {
|
||||
failed++;
|
||||
failures.push({ name, error: err.message || String(err) });
|
||||
console.log(` FAIL ${name}`);
|
||||
console.log(` ${err.message || err}`);
|
||||
}
|
||||
}
|
||||
|
||||
const TOOLS = [
|
||||
{ name: 'ruvector' },
|
||||
{ name: 'hooks_route' },
|
||||
{ name: 'hooks_recall' },
|
||||
{ name: 'hooks_security_scan' },
|
||||
{ name: 'hooks_force_learn' },
|
||||
];
|
||||
|
||||
console.log('\n--- MCP tool-access policy (ADR-256) ---\n');
|
||||
|
||||
test('parseList handles comma and whitespace separators', () => {
|
||||
assert.deepStrictEqual(parseList('a, b ,c'), ['a', 'b', 'c']);
|
||||
assert.deepStrictEqual(parseList('a b\tc'), ['a', 'b', 'c']);
|
||||
assert.deepStrictEqual(parseList(''), []);
|
||||
assert.deepStrictEqual(parseList(undefined), []);
|
||||
});
|
||||
|
||||
test('no env configured → allow-all, not configured', () => {
|
||||
const p = buildToolPolicy({});
|
||||
assert.strictEqual(p.configured, false);
|
||||
assert.strictEqual(p.allowSet, null);
|
||||
for (const t of TOOLS) assert.strictEqual(isToolAllowed(t.name, p), true);
|
||||
assert.strictEqual(filterAllowedTools(TOOLS, p).length, TOOLS.length);
|
||||
});
|
||||
|
||||
test('RUVECTOR_MCP_ALLOW restricts to the listed tools (deny rest)', () => {
|
||||
const p = buildToolPolicy({ RUVECTOR_MCP_ALLOW: 'hooks_route, hooks_recall' });
|
||||
assert.strictEqual(p.configured, true);
|
||||
assert.strictEqual(isToolAllowed('hooks_route', p), true);
|
||||
assert.strictEqual(isToolAllowed('hooks_recall', p), true);
|
||||
assert.strictEqual(isToolAllowed('hooks_security_scan', p), false);
|
||||
assert.strictEqual(isToolAllowed('ruvector', p), false);
|
||||
assert.deepStrictEqual(
|
||||
filterAllowedTools(TOOLS, p).map((t) => t.name).sort(),
|
||||
['hooks_recall', 'hooks_route'],
|
||||
);
|
||||
});
|
||||
|
||||
test('RUVECTOR_MCP_DENY blocks specific tools, allows the rest', () => {
|
||||
const p = buildToolPolicy({ RUVECTOR_MCP_DENY: 'hooks_force_learn' });
|
||||
assert.strictEqual(p.configured, true);
|
||||
assert.strictEqual(isToolAllowed('hooks_force_learn', p), false);
|
||||
assert.strictEqual(isToolAllowed('hooks_route', p), true);
|
||||
});
|
||||
|
||||
test('DENY wins over ALLOW (precedence)', () => {
|
||||
const p = buildToolPolicy({
|
||||
RUVECTOR_MCP_ALLOW: 'hooks_route,hooks_force_learn',
|
||||
RUVECTOR_MCP_DENY: 'hooks_force_learn',
|
||||
});
|
||||
assert.strictEqual(isToolAllowed('hooks_route', p), true);
|
||||
assert.strictEqual(isToolAllowed('hooks_force_learn', p), false);
|
||||
});
|
||||
|
||||
test('RUVECTOR_MCP_PROFILE=readonly applies the curated subset', () => {
|
||||
const p = buildToolPolicy({ RUVECTOR_MCP_PROFILE: 'readonly' });
|
||||
assert.strictEqual(p.profile, 'readonly');
|
||||
assert.strictEqual(p.configured, true);
|
||||
for (const name of MCP_PROFILES.readonly) {
|
||||
assert.strictEqual(isToolAllowed(name, p), true, `${name} should be allowed in readonly`);
|
||||
}
|
||||
// A mutating tool not in the profile is denied
|
||||
assert.strictEqual(isToolAllowed('hooks_force_learn', p), false);
|
||||
});
|
||||
|
||||
test('profile + extra ALLOW are unioned', () => {
|
||||
const p = buildToolPolicy({
|
||||
RUVECTOR_MCP_PROFILE: 'readonly',
|
||||
RUVECTOR_MCP_ALLOW: 'hooks_force_learn',
|
||||
});
|
||||
assert.strictEqual(isToolAllowed('hooks_force_learn', p), true);
|
||||
assert.strictEqual(isToolAllowed('hooks_stats', p), true);
|
||||
});
|
||||
|
||||
test('unknown profile with no allow/deny → not configured (allow-all)', () => {
|
||||
const p = buildToolPolicy({ RUVECTOR_MCP_PROFILE: 'does-not-exist' });
|
||||
assert.strictEqual(p.profile, null);
|
||||
assert.strictEqual(p.configured, false);
|
||||
assert.strictEqual(isToolAllowed('hooks_force_learn', p), true);
|
||||
});
|
||||
|
||||
console.log('\n' + '='.repeat(60));
|
||||
console.log(`\nResults: ${passed} passed, ${failed} failed\n`);
|
||||
if (failures.length > 0) {
|
||||
console.log('Failures:');
|
||||
for (const f of failures) console.log(` - ${f.name}: ${f.error}`);
|
||||
console.log('');
|
||||
}
|
||||
if (failed > 0) {
|
||||
console.log('SOME TESTS FAILED\n');
|
||||
process.exit(1);
|
||||
} else {
|
||||
console.log('ALL TESTS PASSED\n');
|
||||
}
|
||||
88
npm/packages/ruvector/test/startup-budget.js
Normal file
88
npm/packages/ruvector/test/startup-budget.js
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Startup-budget CI guard (ADR-256 step 4).
|
||||
*
|
||||
* The ruvector CLI lazy-loads every heavy dependency to keep cold start fast.
|
||||
* ADR-256 borrows harness features (the `harness` router surface, MCP policy)
|
||||
* into the CLI — this guard fails CI if any of them regress startup.
|
||||
*
|
||||
* Strategy (robust across machines / loaded CI boxes):
|
||||
* 1. ABSOLUTE ceiling — `--help` cold start must stay under a generous cap.
|
||||
* 2. RELATIVE delta — `harness status --json` must not add more than a small
|
||||
* delta over the `--help` baseline (the real regression signal: eager-loading
|
||||
* a heavy module into the harness path would blow this out).
|
||||
*
|
||||
* Most of the floor (~100ms) is the Node process spawn itself, which is why the
|
||||
* relative delta is the meaningful check. Budgets are overridable via env.
|
||||
*/
|
||||
'use strict';
|
||||
|
||||
const { execSync } = require('child_process');
|
||||
const path = require('path');
|
||||
const assert = require('assert');
|
||||
|
||||
const CLI = path.join(__dirname, '..', 'bin', 'cli.js');
|
||||
const CWD = path.join(__dirname, '..');
|
||||
|
||||
const ABS_BUDGET_MS = Number(process.env.RUVECTOR_STARTUP_BUDGET_MS || 2000);
|
||||
const DELTA_BUDGET_MS = Number(process.env.RUVECTOR_STARTUP_DELTA_MS || 120);
|
||||
const SAMPLES = Number(process.env.RUVECTOR_STARTUP_SAMPLES || 5);
|
||||
|
||||
function timeMs(args) {
|
||||
const start = process.hrtime.bigint();
|
||||
try {
|
||||
execSync(`node ${CLI} ${args}`, { stdio: ['pipe', 'pipe', 'pipe'], timeout: 20000, cwd: CWD });
|
||||
} catch {
|
||||
// non-zero exit still yields a valid timing
|
||||
}
|
||||
return Number(process.hrtime.bigint() - start) / 1e6;
|
||||
}
|
||||
|
||||
function median(args) {
|
||||
const t = [];
|
||||
for (let i = 0; i < SAMPLES; i++) t.push(timeMs(args));
|
||||
t.sort((a, b) => a - b);
|
||||
return t[Math.floor(t.length / 2)];
|
||||
}
|
||||
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
const failures = [];
|
||||
function test(name, fn) {
|
||||
try { fn(); passed++; console.log(` PASS ${name}`); }
|
||||
catch (err) { failed++; failures.push({ name, error: err.message || String(err) }); console.log(` FAIL ${name}`); console.log(` ${err.message || err}`); }
|
||||
}
|
||||
|
||||
console.log('\n--- Startup-budget guard (ADR-256) ---\n');
|
||||
console.log(` samples=${SAMPLES} abs_budget=${ABS_BUDGET_MS}ms delta_budget=${DELTA_BUDGET_MS}ms\n`);
|
||||
|
||||
// Warm up (filesystem cache, AV scan of the file, etc.)
|
||||
timeMs('--version');
|
||||
|
||||
const helpMs = median('--help');
|
||||
const harnessMs = median('harness status --json');
|
||||
const delta = harnessMs - helpMs;
|
||||
|
||||
console.log(` --help (cold): ${helpMs.toFixed(0)}ms`);
|
||||
console.log(` harness status --json: ${harnessMs.toFixed(0)}ms (Δ ${delta >= 0 ? '+' : ''}${delta.toFixed(0)}ms vs --help)\n`);
|
||||
|
||||
test(`--help cold start under ${ABS_BUDGET_MS}ms absolute budget`, () => {
|
||||
assert(helpMs < ABS_BUDGET_MS,
|
||||
`--help startup ${helpMs.toFixed(0)}ms exceeds ${ABS_BUDGET_MS}ms (set RUVECTOR_STARTUP_BUDGET_MS to override)`);
|
||||
});
|
||||
|
||||
test(`harness status adds < ${DELTA_BUDGET_MS}ms over --help baseline (no lazy-load regression)`, () => {
|
||||
assert(delta < DELTA_BUDGET_MS,
|
||||
`harness status added ${delta.toFixed(0)}ms over --help — a heavy module may have leaked into the startup path`);
|
||||
});
|
||||
|
||||
console.log('\n' + '='.repeat(60));
|
||||
console.log(`\nResults: ${passed} passed, ${failed} failed\n`);
|
||||
if (failures.length > 0) {
|
||||
console.log('Failures:');
|
||||
for (const f of failures) console.log(` - ${f.name}: ${f.error}`);
|
||||
console.log('');
|
||||
}
|
||||
if (failed > 0) { console.log('STARTUP BUDGET EXCEEDED\n'); process.exit(1); }
|
||||
else { console.log('STARTUP WITHIN BUDGET\n'); }
|
||||
Loading…
Add table
Add a link
Reference in a new issue