mirror of https://github.com/MoonshotAI/kimi-code.git synced 2026-08-22 23:26:12 +00:00

feat: isolate the full-text search index from the session index and the main thread (#2701 )

* feat(minidb): instrument open lifecycle with phase timings and status

Add MiniDb.lifecycleStatus() exposing the no-generation/generation-load/
wal-catch-up/full-rebuild/ready/degraded state machine plus per-phase
timings (generation candidate load, store/non-text/text image load,
postings integrity check, WAL scan/apply, full recovery, text rebuild
hosting), so snapshot load, WAL catch-up and full rebuild can be told
apart in diagnostics.

Also add a repeatable open-lifecycle bench (small data, large WAL delta,
large full-text generation, corrupt generation) and fixtures proving a
healthy generation open performs no full-corpus tokenization while a
corrupt or missing generation falls back. Log search-index and
query-store open diagnostics in kap-server and agent-core-v2 so a
listSessions call can be attributed to the database it touches.

No persistence format or product behavior change.

* feat(agent-core-v2): isolate the session index from the global search index

Harden the separation between the session read model and the full-text
search index so session operations never depend on search availability:

- Reject text index definitions in MiniDbQueryStore at definition level,
  keeping the session query-store a structural-only read model with no
  postings/tokenizer artifacts, and assert its generation carries no
  full-text files.
- Share one authoritative scan between the first list and the initial
  projection (single-flight) instead of scanning twice; reads may only
  join an in-flight scan, and every fallback read folds the mirror's
  pending queue so read-your-writes holds while preparing.
- Keep withReadModel() fallback semantics pinned by tests:
  uninitialized/preparing reads hit authoritative metadata immediately,
  ready reads use the read model, degraded keeps falling back with a
  diagnosable status reason.
- Guard session metadata writes so a mirror failure degrades only the
  read model and never fails the session lifecycle.
- Prove via tests that listSessions/--resume/--continue never open the
  global search DB (including when search-index is unopenable), and that
  only real full-text search requests report building/stale/degraded.

* perf(minidb): slice open-time work so it never blocks the main thread

Make the whole generation-open path cooperative:

- Replace the synchronous postings/store CRC verification with chunked
  async variants (readGenerationFileCheckedAsync, verifyFileIntegrityAsync)
  that keep the exact bytes/crc-mismatch error semantics.
- Give the WAL-delta apply a primitive-op + wall-clock budget
  (walApplySlicer), so a batch frame unrolling into thousands of ops can
  no longer run as one uninterruptible slice; torn-tail, corrupt-batch
  and read-only behaviors are unchanged.
- Slice the big attach loops: Store.bulkLoadRefsAsync +
  SkipList.bulkLoadAsync for the store image, async parsers and
  loadImageAsync for secondary/compound images, and
  TextIndex.attachImageAsync for the docs/dictionary map construction.
- Queue text builds on worker-slot pressure (WorkerSlots.acquireBounded,
  bounded by MiniDb.textBuildSlotWaitMs, abort-aware) instead of falling
  back to an unbounded inline build; a persisted drought hosts the
  bounded inline core as the explicit last resort with stats accounting.

Bench (bench/open-lifecycle, seed 42): event-loop delay max across the
four open scenarios drops from 45/734/331/492 ms to ~12-28 ms with wall
time flat or better.

* feat(kap-server): run the global search index in a dedicated worker

Move the whole search-index MiniDb lifecycle (open, generation load,
WAL replay, sync, rebuild, compaction) off the main thread into a
long-lived worker_threads host, so it never shares the event loop with
TUI input:

- Add a versioned request/response protocol and worker entry hosting a
  host-agnostic SearchIndexCore; the same core also backs an inline
  backend kept as the explicit rollback
  (KIMI_CODE_EXPERIMENTAL_SEARCH_WORKER=false, flag default ON).
- The worker exclusively owns the search-index handle. The lock token
  is reported at acquire time (new MiniDb OpenOptions.onLockAcquired
  hook) and reaped on dirty exit; an orphan-lock detector (same-pid
  lock row whose token no live holder owns) recovers the window where
  the token report is lost, so a mid-open crash can never freeze the
  index into a silent permanent read-only.
- Crash handling: in-flight requests are rejected with typed errors,
  respawn uses capped exponential backoff, per-request watchdogs
  terminate wedged workers, and beginClose propagates into the worker
  so dispose stays bounded during a long sync. Page tokens pin a
  boot-salted generation, so tokens issued before a transparent worker
  restart fail closed with invalid_page_token.
- The main process keeps the sync coordinator (debounce/coalescing/
  single-flight), live transcript routing, query normalization and
  page-token codec; searches keep reading the published generation and
  report building/stale/degraded instead of waiting for sync/rebuild.
- Wire the worker into the CLI packaging: self-contained worker bundles
  for npm dist and the SEA asset manifest/installer/smoke check, plus a
  dev runtime (type-stripping + .ts resolve hook) scoped to worker
  execArgv.

* feat(kap-server): model search and session-index lifecycles explicitly

Consolidate the two-index separation into explicit, diagnosable
lifecycles:

- Surface the global search state machine (stopped / opening / building
  / ready / degraded / closing) end to end: SearchIndexCore.lifecycleState,
  SearchWorkerHost lifecycle snapshots cached from RPC responses (and
  invalidated across worker generations), a never-throwing status()
  carrying the lifecycle, and a synchronous lifecycleReport() that
  neither kicks the open nor spawns the worker. Corrupt search-index
  rebuilds are announced with a dedicated warn log so building, stale,
  degraded, corrupt and worker-unavailable stay distinguishable.
- Turn MiniDb read-only replica catch-up fully cooperative:
  catchUpWalAsync scans frames with the windowed async scanner and
  yields per primitive op on the shared walApplySlicer budget, while a
  per-instance catchUpChain serializes concurrent catch-ups so each
  caller keeps its atomic watermark advance. The stale synchronous
  implementations are removed.
- Pin the dependency direction and availability timing with tests:
  session list/create/resume survive a corrupt or unopenable search
  index (also end-to-end with a dead query-store), search generation
  reuse and stale-serving keep working across restarts, concurrent cold
  callers open the index / spawn the worker exactly once, resume-then-
  fetchSessions performs no duplicate authoritative scan, and a clean
  dispose releases the lock and settles at stopped.
- Document the experimental flag surface (persistence_minidb_readmodel,
  search_worker) in the root guide.

* feat(agent-core-v2): default the session read model on and roll out the separation

Rollout and validation for the index separation plan:

- Flip persistence_minidb_readmodel to default ON (rollback via
  KIMI_CODE_EXPERIMENTAL_PERSISTENCE_MINIDB_READMODEL=false or the
  experimental config section); session list/--resume/--continue now
  always go through the isolated session read model with the
  authoritative fallback. Test harnesses pin the flag off where shared
  fixtures require hermetic homes, while the dedicated suites keep
  explicit on/off coverage.
- Add a probe proving the main thread stays responsive while the
  search worker rebuilds and swaps a generation (reindex), completing
  the TUI responsiveness matrix.
- Record the rollout state in the agent-core-v2 guide (session index
  section) and the root flag line.
- Add changesets for the CLI (worker isolation, session index
  independence) and minidb (cooperative open lifecycle).

Validation: full suites green across minidb (551), agent-core-v2
(4760), kap-server (1005), node-sdk (343), klient (91) and the CLI app
(2567); open-lifecycle bench event-loop delay max is down from
45/734/331/492 ms to ~16-22 ms across the four scenarios with wall
time flat or better.

* fix(agent-core-v2): evict deleted sessions from the mirror queue and drain the index on close

Two issues surfaced by the read-model default in the acp-server suite:

- ISessionIndex.remove only deleted from the query store, but a summary
  still queued in the mirror was folded back into reads (and re-written
  by the next flush), resurrecting a deleted session in listings. The
  mirror now exposes evict(id): drop the queued summary and wait out an
  in-flight flush before the store delete.
- RunningAcpServer.close and SDKRpcClientV2.close disposed the engine
  without awaiting the asynchronous mirror flush / query-store close,
  so a host removing homeDir right after close() raced in-flight shard
  closes (ENOTEMPTY). Both now follow the kap-server shutdown order:
  drain the mirror while the store is open, dispose, then await the
  drains.

* fix(minidb): pause active expiry during the sliced bulk load

The store's active-expire timer is armed at construction, so during a
sliced bulkLoadRefsAsync a tick can fire mid-load: it reaps a TTL key
from the map while the order skiplist is still the old empty one, and
the final bulkLoadAsync then rebuilds order from the stale orderEntries
snapshot — resurrecting the expired key in the ordered index (and
duplicating it if the key is later set again). The sync bulkLoadRefs had
no yield windows, so guard the async path with a bulkLoading flag that
defers expiry ticks until the load settles (finally-safe).

* chore: consolidate changesets into the TUI startup freeze fix

2026-08-07 07:38:16 +08:00

11 KiB

Raw Permalink Blame History

Repository-level Agent Guide

Reply in the same language as the user.

This is a TypeScript monorepo built for agent-assisted development. Keep the root AGENTS.md limited to hot-path rules: the project map, hard constraints, and workflow requirements — things every task needs to know.

Working Principles

Think from first principles. Start from real requirements, code facts, and verification results; if the goal is unclear, discuss it with the user first.
Treat code, not documentation, as the source of truth. Unless the user explicitly says otherwise, do not read ordinary Markdown just to understand the implementation.
Before making code changes, read the relevant code and the most recent constraints, and follow the nearest AGENTS.md in the directory tree.
Keep changes focused. Do not slip in unrelated refactors along the way.
When committing, do not add any co-author attribution, and do not reveal the identity of the agent in commit messages, PR descriptions, or any explanatory text.

Project Map

apps/kimi-code: the CLI / TUI application. It consumes core capabilities through @moonshot-ai/kimi-code-sdk and must not depend directly on @moonshot-ai/agent-core. When writing or modifying its terminal UI, use the write-tui skill (.agents/skills/write-tui/SKILL.md).
the browser web UI: its source no longer lives in this repo. It is developed in the code-app repo (apps/web) and shipped as the committed, prebuilt bundle apps/kimi-code/dist-web (gitignored, force-added), synced from code-app with KIMI_CODE_REPO=<this checkout> pnpm run sync:web — sync and commit the bundle in the same change whenever the web UI should ship differently. apps/kimi-code/scripts/check-web-assets.mjs guards packaging against a missing bundle. To hack on the web UI against this repo's server, run pnpm dev:server here and point code-app's pnpm dev:web at it via KIMI_SERVER_URL.
apps/vis, apps/vis/server, apps/vis/web: visual debugging tools for sessions and replays.
apps/kimi-inspect: web inspector for the kap-server /api/v1/debug RPC surface — workspace/session browser, per-session transcript chat, per-scope Service panels, and the DI unit inspection view. See apps/kimi-inspect/AGENTS.md.
packages/agent-core: the unified agent engine, including Agent, Session, profile, skills, tools, plan, permission, background, records, the in-process DI service layer (src/services/), and other core capabilities. See packages/agent-core/AGENTS.md.
packages/agent-core-v2: the DI × Scope agent engine (the v2 port behind kap-server). Four LifecycleScope tiers — App / Workspace / Session / Agent (app/scopes.ts) — plus the L3 unit layer (Service/Fiber units, collection contribution points, the Feature seam in src/features/); there is no App-level session lifecycle facade — callers compose ISessionIndex → IWorkspaceLifecycleService.handlerFor → the handler. See packages/agent-core-v2/AGENTS.md and use the agent-core-dev skill (.agents/skills/agent-core-dev/SKILL.md) when developing here.
packages/node-sdk: the public TypeScript SDK and harness.
packages/kosong: the LLM / provider abstraction layer.
packages/kaos: the execution environment and file/process abstractions.
packages/oauth: Kimi OAuth and managed auth utilities.
packages/telemetry: shared client-side telemetry infrastructure.
packages/transcript: the isomorphic transcript rendering data layer — L1 agent-granular store, L2 idempotent operations, L3 off/turn/block/delta subscription granularity, L4 framework-free view registry, plus turn-cursor pagination. Pure TypeScript (browser-safe, no engine imports); the sole owner of the transcript contract types (src/contract/) and the op-batch sequencing contract. See packages/transcript/AGENTS.md.
packages/kap-server: the Kimi Code server, backed by @moonshot-ai/agent-core-v2; exposes sessions over REST + WebSocket (/api/v1 + /api/v1/ws), plus the /api/v1/debug/* reflection RPC surface (--debug-endpoints, loopback bind + bearer auth). See packages/kap-server/AGENTS.md.
packages/klient: the client SDK — a contract-driven facade over agent-core-v2 (global.* / session(id).* / agent(id).*, zod-validated); transport via subpath entry (@moonshot-ai/klient/ipc|memory, both return the same Klient); also hosts the e2e suites. See packages/klient/AGENTS.md.
packages/tree-sitter-bash: a pure-TypeScript bash parser (no runtime deps, no wasm); parse(source, { timeoutMs, maxNodes }) runs under a deterministic budget and returns a discriminated ParseResult — callers must treat aborted/hasError trees as "cannot analyze" and degrade. Parser only, no safety judgments; see the package README's "Known differences" section.
packages/minidb: the embedded JSON document store (MiniDb) behind kap-server's search index — snapshot + WAL persistence with an exclusive write lock, a larger-than-RAM full-text layer, and persistent index generations. See packages/minidb/AGENTS.md.

Environment Requirements

Node.js: >=24.15.0 (from the root package.json engines; .nvmrc is 24.15.0, used by nvm / fnm / mise to pick the minimum recommended version).
pnpm: 10.33.0 (from the root package.json packageManager).
pnpm install will fail when the Node version is not satisfied, because .npmrc sets engine-strict=true.

Monorepo Workspace Maintenance

pnpm-workspace.yaml is the source of truth for workspace membership, but flake.nix also contains hardcoded workspacePaths and workspaceNames lists.
Whenever you add or remove a workspace package, you MUST update both pnpm-workspace.yaml and flake.nix — for every package, including leaf / test / e2e packages that nothing depends on.
- pnpm-workspace.yaml uses globs (packages/*, apps/*), so most packages land there automatically; flake.nix is fully manual and is where omissions happen.
- Missing a path in flake.nix's workspacePaths will silently drop files from the Nix build's src fileset.
- Missing a name in flake.nix's workspaceNames will break pnpmConfigHook because dependencies for that workspace will not be fetched.
The automated "Check flake.nix workspace sync" (scripts/check-nix-workspace.mjs) only validates the transitive dependency closure of @moonshot-ai/kimi-code. A leaf package outside that closure (e.g. an e2e package nobody imports) slips through even when it is missing from flake.nix. A green check is therefore NOT proof that flake.nix is fully in sync — keep it updated by hand on every add/remove, do not rely on the check to catch omissions.

General Coding Rules

For optional object properties, pass undefined directly instead of using conditional spread.
- YES: { user }
- NO: { ...(user ? { user } : undefined) }
Optional object properties do not need to additionally allow undefined in the type.
- YES: interface Options { user?: User }
- NO: interface Options { user?: User | undefined }
Internal methods with only a single parameter should not be turned into options objects just for stylistic uniformity.
Except for a package's index.ts, other index.ts files should prefer export * from './module';.
Do not add too many new test files. Prefer adding tests to the existing test file of the corresponding component or module.
When a test fails because of a user modification, default to fixing the test first; do not change the implementation to satisfy an old test unless the implementation truly has a bug.
Do not sacrifice code quality for external compatibility unless the user explicitly asks for it. Breaking changes go through changesets and a major bump, gated by the rule below.

Experimental Features

Gate a not-yet-public feature behind an experimental flag. Flags are env-driven and default off: KIMI_CODE_EXPERIMENTAL_<NAME> toggles one, KIMI_CODE_EXPERIMENTAL_FLAG enables all. Release by flipping the entry's default to true.
- packages/agent-core (v1): add the flag to the central registry at packages/agent-core/src/flags/registry.ts, then check it with flags.enabled('my-feature').
- packages/agent-core-v2 and kap-server modules: there is no central catalog — declare the flag in the owning domain via registerFlagDefinition at import time (see packages/agent-core-v2/docs/flag.md), then check it with IFlagService.enabled(id). Current search-index-separation flags: persistence_minidb_readmodel (session read model, default on) and search_worker (global search worker host, default on).

Where to Update Instructions

Hard rules that affect almost every task: update the root AGENTS.md.
Rules that only affect a specific directory: update the nearest sub-directory AGENTS.md.
Project-map entries stay at 1–2 sentences; deep package docs live in the package's own AGENTS.md.
Keep instruction updates focused and supported by code facts.

Workflow Requirements

Prefer rg / rg --files when reading code.
When designing changes, follow existing boundaries and local patterns first.
In public text and test data, replace real internal identifiers with neutral placeholders such as example.com, example.test, and YOUR_API_KEY. Before opening a PR, ask a read-only agent to audit the diff for context-specific internal identifiers.
When creating a PR, the PR title must follow Conventional Commit style, e.g. chore: remove legacy format commands.
When an AI agent opens or updates a PR, fill in .github/pull_request_template.md — link the related issue or explain the problem, then describe what changed. Do not leave placeholder text or submit a generic summary of the diff.
Do not submit vague AI-generated PR text. The human author must understand the change well enough to explain the code, edge cases, and why the approach fits this repository.
After finishing a task and before submitting a PR, you must run the gen-changesets skill (see .agents/skills/gen-changesets/SKILL.md) and generate a changeset under .changeset/ according to its rules.
When generating a changeset, never decide on a major bump on your own — stop, explain, and get explicit user confirmation first; default to minor, fall back to patch. See .agents/skills/gen-changesets/SKILL.md.
Prefer importing via import ... from '#/...', which serves the same purpose as import ... from '@/...'.
Do not commit throwaway scratch or exploratory files. Never stage:
- Agent working notes or handoff/summary documents (e.g. HANDOVER-*.md, HANDOFF-*.md, handoff.md).
- Throwaway UI/UX prototypes or design mockups (e.g. *-designs.html, *-mockup.html, *-demo(s).html) at the repo root or under a design/ folder. The only tracked .html files should be Vite index.html entrypoints. Before committing or opening a PR, run git status and git diff --staged --stat and remove anything matching these patterns. Put scratch work under .tmp/ (gitignored) instead of the repo root or the source tree.

11 KiB Raw Permalink Blame History Unescape Escape