mirror of
https://github.com/NeuralNomadsAI/CodeNomad.git
synced 2026-10-21 05:06:37 +00:00
feat(missions): add a "Compare two models" Mission template (#1026)
## Summary Adds a fifth Mission template, **Compare two models** (`compare`). It runs the same bounded task on two different models and ends with a blind verdict. - **Creation:** the user picks **model A** and **model B** in the main creation form. Both are required and must name different models (provider + model id; a reasoning variant alone is not a different model). This is enforced on every creation path: HTTP, prepare, recurrence create and agent-started missions. - **Roles:** `model-a`, `model-b` and `comparator`. - A and B share one instruction text and report contract. - The coordinator writes the task, inputs and criteria once, and gives both byte-identical briefs. - The server refuses a run task whose brief differs from its sibling's, or whose per-task execution would change its role's model. - **Blind judge, always on:** the comparator works on neutral X/Y labels. - It reports one versioned `comparison` artifact: the A/B task references, per-criterion scores for both runs, sourced observations, and a verdict `a | b | equivalent | inconclusive`. - Completion requires both runs and that comparison. - **Judge model:** frozen once at creation, like every other role. - An explicit judge is kept; otherwise the judge copies the coordinator's effective model. When that model is unknown, both stay on the native default. - Replays and retries therefore reuse the stored profile. - The editor warns when the judge's model equals A or B. - **Readers:** the verdict, both model names, a score table and evidence by path. Hashes stay under Technical details, for current and archived recurring passages. The template came out of evaluating Harness2's comparison loop. It deliberately has **no** lesson, human approval, budget or duration handling, and leaves the Wayfinder human gate unchanged. Up to five models, per-run worktrees and launching from the composer's model picker (OpenChamber-style multi-run) are left for later. `AGENTS.md` records the single editor exception: the A/B selectors sit outside the collapsed Options disclosure. All ten locales are updated. ## Size and upkeep 62 files, +837/-96: | Part | Lines | |---|---| | Product code (about 16 files are one-line template-list hooks) | +280/-57 | | Locales (13 keys × 10) | +131 | | Tests | +409 | | Docs | +17 | `missions/control.ts` stays oversized (~1,500 lines). ## Validation - Full server suite: 2,426 tests pass, 7 existing skips. Root and server typechecks, the server build and the plugin build pass. - UI: 45 unit tests and 36 browser tests pass. - The native trajectory fixture, extended with a compare case, passes on OpenCode 2.0.31. It checks two distinct models, a one-newline brief difference refused with the journal unchanged, a real blind judgment and a completed mission. The native contract and recurring Q fixtures also pass. - Regressions cover: - distinct-model refusals on every creation path; - brief and override refusals; - artifact validation and the completion policy; - judge freezing at creation, revision replay, and retry after a coordinator model change; - editor inheritance and readable evidence. - Independent review rounds ended with no P0-P3 findings. - The editor and reader were checked from screenshots of the real Solid components in the browser harness, not in a packaged app.
This commit is contained in:
parent
b6221e9232
commit
debbdce33f
62 changed files with 837 additions and 96 deletions
|
|
@ -7,7 +7,7 @@ description: Coordinate bounded native tasks through the CodeNomad mission map,
|
|||
|
||||
Prefer ordinary native delegation for bounded work. Use a mission only when the user needs a durable shared task map, evidence gates, and explicit coordinator decisions; a mission is not required for an ordinary helper.
|
||||
|
||||
1. Call `mission.inspect` with `start.objective` and one template: `custom`, `debug`, `wayfinder`, or `delivery`.
|
||||
1. Call `mission.inspect` with `start.objective` and one template: `custom`, `debug`, `wayfinder`, `delivery`, or `compare` (two required different model profiles: `model-a` and `model-b`; coordinator-written protocol, identical briefs, always a blind `comparator`, verdict only).
|
||||
2. Read the returned playbook. The starting session is the coordinator and is the only session allowed to call `mission.delegate`.
|
||||
3. Give every task a stable lowercase `taskKey`. Declare its role, real `blockedBy` dependencies, and execution mode separately from execution. Native is the default for new declarations; blocked declarations do not dispatch or require completed prerequisite reports yet.
|
||||
4. Prefer a fresh native child for independent work; reuse an exact specialist only when its context helps and continuation is eligible. Use an independent root only for a concrete `location`, `lifetime`, `existing-root`, or `playbook` reason and explanation. Native continuation names the exact live native `reuseFromTaskKey` with the same parent, never an arbitrary root or a guessed session. This selects context, not a `blockedBy` dependency; do not add a dependency just to reuse a session. Debugging resolvers still require dependency-connected implementer and review evidence. Admission must prove the exact current native execution ended/returned and the actual bound session is idle; a report alone proves neither. Historical tasks without an execution mode retain their root-only behavior: omit `targetSessionID` to create a root, or supply a same-project root to reuse it. Do not automatically convert them.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue