Commit graph

7 commits

Author SHA1 Message Date
Haozhe
7cd64766c8
feat: isolate the full-text search index from the session index and the main thread (#2701)
* feat(minidb): instrument open lifecycle with phase timings and status

Add MiniDb.lifecycleStatus() exposing the no-generation/generation-load/
wal-catch-up/full-rebuild/ready/degraded state machine plus per-phase
timings (generation candidate load, store/non-text/text image load,
postings integrity check, WAL scan/apply, full recovery, text rebuild
hosting), so snapshot load, WAL catch-up and full rebuild can be told
apart in diagnostics.

Also add a repeatable open-lifecycle bench (small data, large WAL delta,
large full-text generation, corrupt generation) and fixtures proving a
healthy generation open performs no full-corpus tokenization while a
corrupt or missing generation falls back. Log search-index and
query-store open diagnostics in kap-server and agent-core-v2 so a
listSessions call can be attributed to the database it touches.

No persistence format or product behavior change.

* feat(agent-core-v2): isolate the session index from the global search index

Harden the separation between the session read model and the full-text
search index so session operations never depend on search availability:

- Reject text index definitions in MiniDbQueryStore at definition level,
  keeping the session query-store a structural-only read model with no
  postings/tokenizer artifacts, and assert its generation carries no
  full-text files.
- Share one authoritative scan between the first list and the initial
  projection (single-flight) instead of scanning twice; reads may only
  join an in-flight scan, and every fallback read folds the mirror's
  pending queue so read-your-writes holds while preparing.
- Keep withReadModel() fallback semantics pinned by tests:
  uninitialized/preparing reads hit authoritative metadata immediately,
  ready reads use the read model, degraded keeps falling back with a
  diagnosable status reason.
- Guard session metadata writes so a mirror failure degrades only the
  read model and never fails the session lifecycle.
- Prove via tests that listSessions/--resume/--continue never open the
  global search DB (including when search-index is unopenable), and that
  only real full-text search requests report building/stale/degraded.

* perf(minidb): slice open-time work so it never blocks the main thread

Make the whole generation-open path cooperative:

- Replace the synchronous postings/store CRC verification with chunked
  async variants (readGenerationFileCheckedAsync, verifyFileIntegrityAsync)
  that keep the exact bytes/crc-mismatch error semantics.
- Give the WAL-delta apply a primitive-op + wall-clock budget
  (walApplySlicer), so a batch frame unrolling into thousands of ops can
  no longer run as one uninterruptible slice; torn-tail, corrupt-batch
  and read-only behaviors are unchanged.
- Slice the big attach loops: Store.bulkLoadRefsAsync +
  SkipList.bulkLoadAsync for the store image, async parsers and
  loadImageAsync for secondary/compound images, and
  TextIndex.attachImageAsync for the docs/dictionary map construction.
- Queue text builds on worker-slot pressure (WorkerSlots.acquireBounded,
  bounded by MiniDb.textBuildSlotWaitMs, abort-aware) instead of falling
  back to an unbounded inline build; a persisted drought hosts the
  bounded inline core as the explicit last resort with stats accounting.

Bench (bench/open-lifecycle, seed 42): event-loop delay max across the
four open scenarios drops from 45/734/331/492 ms to ~12-28 ms with wall
time flat or better.

* feat(kap-server): run the global search index in a dedicated worker

Move the whole search-index MiniDb lifecycle (open, generation load,
WAL replay, sync, rebuild, compaction) off the main thread into a
long-lived worker_threads host, so it never shares the event loop with
TUI input:

- Add a versioned request/response protocol and worker entry hosting a
  host-agnostic SearchIndexCore; the same core also backs an inline
  backend kept as the explicit rollback
  (KIMI_CODE_EXPERIMENTAL_SEARCH_WORKER=false, flag default ON).
- The worker exclusively owns the search-index handle. The lock token
  is reported at acquire time (new MiniDb OpenOptions.onLockAcquired
  hook) and reaped on dirty exit; an orphan-lock detector (same-pid
  lock row whose token no live holder owns) recovers the window where
  the token report is lost, so a mid-open crash can never freeze the
  index into a silent permanent read-only.
- Crash handling: in-flight requests are rejected with typed errors,
  respawn uses capped exponential backoff, per-request watchdogs
  terminate wedged workers, and beginClose propagates into the worker
  so dispose stays bounded during a long sync. Page tokens pin a
  boot-salted generation, so tokens issued before a transparent worker
  restart fail closed with invalid_page_token.
- The main process keeps the sync coordinator (debounce/coalescing/
  single-flight), live transcript routing, query normalization and
  page-token codec; searches keep reading the published generation and
  report building/stale/degraded instead of waiting for sync/rebuild.
- Wire the worker into the CLI packaging: self-contained worker bundles
  for npm dist and the SEA asset manifest/installer/smoke check, plus a
  dev runtime (type-stripping + .ts resolve hook) scoped to worker
  execArgv.

* feat(kap-server): model search and session-index lifecycles explicitly

Consolidate the two-index separation into explicit, diagnosable
lifecycles:

- Surface the global search state machine (stopped / opening / building
  / ready / degraded / closing) end to end: SearchIndexCore.lifecycleState,
  SearchWorkerHost lifecycle snapshots cached from RPC responses (and
  invalidated across worker generations), a never-throwing status()
  carrying the lifecycle, and a synchronous lifecycleReport() that
  neither kicks the open nor spawns the worker. Corrupt search-index
  rebuilds are announced with a dedicated warn log so building, stale,
  degraded, corrupt and worker-unavailable stay distinguishable.
- Turn MiniDb read-only replica catch-up fully cooperative:
  catchUpWalAsync scans frames with the windowed async scanner and
  yields per primitive op on the shared walApplySlicer budget, while a
  per-instance catchUpChain serializes concurrent catch-ups so each
  caller keeps its atomic watermark advance. The stale synchronous
  implementations are removed.
- Pin the dependency direction and availability timing with tests:
  session list/create/resume survive a corrupt or unopenable search
  index (also end-to-end with a dead query-store), search generation
  reuse and stale-serving keep working across restarts, concurrent cold
  callers open the index / spawn the worker exactly once, resume-then-
  fetchSessions performs no duplicate authoritative scan, and a clean
  dispose releases the lock and settles at stopped.
- Document the experimental flag surface (persistence_minidb_readmodel,
  search_worker) in the root guide.

* feat(agent-core-v2): default the session read model on and roll out the separation

Rollout and validation for the index separation plan:

- Flip persistence_minidb_readmodel to default ON (rollback via
  KIMI_CODE_EXPERIMENTAL_PERSISTENCE_MINIDB_READMODEL=false or the
  experimental config section); session list/--resume/--continue now
  always go through the isolated session read model with the
  authoritative fallback. Test harnesses pin the flag off where shared
  fixtures require hermetic homes, while the dedicated suites keep
  explicit on/off coverage.
- Add a probe proving the main thread stays responsive while the
  search worker rebuilds and swaps a generation (reindex), completing
  the TUI responsiveness matrix.
- Record the rollout state in the agent-core-v2 guide (session index
  section) and the root flag line.
- Add changesets for the CLI (worker isolation, session index
  independence) and minidb (cooperative open lifecycle).

Validation: full suites green across minidb (551), agent-core-v2
(4760), kap-server (1005), node-sdk (343), klient (91) and the CLI app
(2567); open-lifecycle bench event-loop delay max is down from
45/734/331/492 ms to ~16-22 ms across the four scenarios with wall
time flat or better.

* fix(agent-core-v2): evict deleted sessions from the mirror queue and drain the index on close

Two issues surfaced by the read-model default in the acp-server suite:

- ISessionIndex.remove only deleted from the query store, but a summary
  still queued in the mirror was folded back into reads (and re-written
  by the next flush), resurrecting a deleted session in listings. The
  mirror now exposes evict(id): drop the queued summary and wait out an
  in-flight flush before the store delete.
- RunningAcpServer.close and SDKRpcClientV2.close disposed the engine
  without awaiting the asynchronous mirror flush / query-store close,
  so a host removing homeDir right after close() raced in-flight shard
  closes (ENOTEMPTY). Both now follow the kap-server shutdown order:
  drain the mirror while the store is open, dispose, then await the
  drains.

* fix(minidb): pause active expiry during the sliced bulk load

The store's active-expire timer is armed at construction, so during a
sliced bulkLoadRefsAsync a tick can fire mid-load: it reaps a TTL key
from the map while the order skiplist is still the old empty one, and
the final bulkLoadAsync then rebuilds order from the stale orderEntries
snapshot — resurrecting the expired key in the ordered index (and
duplicating it if the key is later set again). The sync bulkLoadRefs had
no yield windows, so guard the async path with a bulkLoading flag that
defers expiry ticks until the load settles (finally-safe).

* chore: consolidate changesets into the TUI startup freeze fix
2026-08-07 07:38:16 +08:00
liruifengv
541ddd2d89
chore(web): replace apps/kimi-web with the code-app web bundle (#2599)
* chore(web): remove apps/kimi-web in favor of the code-app web bundle

The web UI source now lives in the code-app repo (apps/web); this repo
only ships the prebuilt bundle at apps/kimi-code/dist-web, synced from
code-app via \.

- delete apps/kimi-web (source, tests, docs)
- root package.json: drop dev:web and the kimi-web typecheck leg
- apps/kimi-code: drop the workspace dep and the build-from-source step;
  replace copy-web-assets.mjs with check-web-assets.mjs so packaging
  fails fast when the committed bundle is missing
- CI: _native-build verifies the committed bundle instead of building
  from source; ci.yml and pkg-pr-new.yml drop the kimi-web legs
- docs: AGENTS.md project map, changeset README, gen-changesets and
  sync-changelog skills now key web UI entries on dist-web

* chore(web): stop ignoring dist-web now that the bundle is committed

The ignore entry predates the code-app sync flow, when dist-web was a
local build artifact. The bundle is now the canonical, committed form of
the web UI, so ignoring it only forces every sync to git add -f.

* docs(skills): drop the web-specific changeset and changelog rules

The web: prefix convention and the web-specific dedup guidance belonged
to the in-repo web app. With the source moved to code-app, web UI
changes follow the same generic rules as any other CLI-bundle change.

* chore(web): add changesets for the web UI changes in the bundle

* chore(web): collapse the bundle changesets into one umbrella entry

* ci: unbreak nix and lint after the kimi-web removal

- flake.nix: drop apps/kimi-web from the source fileset and package
  lists, and verify the committed dist-web in the native build phase
  instead of building the web app from source
- .oxlintrc.json: exclude dist-web (a committed build artifact) now
  that .gitignore no longer hides it from the linter

* ci(nix): update pnpmDeps hash for the post-kimi-web lockfile

* chore(web): resync dist-web from code-app main

Bundle rebuilt from code-app main (upstream parity ports #175, pinned
sidebar #176) with the CLI version 0.32.0 embedded.
2026-08-05 13:38:30 +08:00
Haozhe
119a33f7f1
feat(minidb): persistent index generations and lifecycle hardening (#2604)
Some checks are pending
CI / test (2) (push) Waiting to run
CI / test (3) (push) Waiting to run
CI / test (4) (push) Waiting to run
CI / test (5) (push) Waiting to run
CI / test-pi-tui (push) Waiting to run
CI / test-windows (push) Waiting to run
CI / lint (push) Waiting to run
CI / build (push) Waiting to run
CI / test (1) (push) Waiting to run
CI / typecheck (push) Waiting to run
Nix Build / Check flake.nix workspace sync (push) Waiting to run
Nix Build / nix build .#kimi-code (push) Blocked by required conditions
Release / Release (push) Waiting to run
Release / Deploy docs (push) Blocked by required conditions
Release / Native release artifact (push) Blocked by required conditions
Release / Publish native release assets (push) Blocked by required conditions
* perf(minidb): skip idle everysec fsyncs and add lifecycle stats

- everysec WAL now fsyncs on the timer only while dirty (tracked by a
  write/sync generation watermark); close() keeps its unconditional
  final sync, and background sync failures surface via walFsyncErrors
  plus a sticky lastWalFsyncError instead of being silently swallowed
- add WAL queue/group-commit counters (walQueuedBytes,
  walMaxQueuedBytes, walGroupCommits, walGroupCommitFrames) and
  lifecycle phase stats: recovery bytes/frames/duration, index/text
  rebuild durations, compaction total/snapshot/rotation/postings
  durations, rotation pause, and query candidates/decoded/sorted
  rows; add a syncIntervalMs open option threaded through compaction
  WAL rotation
- rewrite the bench on fixed-seed synthetic data with a stable
  machine-readable JSON report (cold open 10k/50k/100k, word/ngram
  search, idle-fsync acceptance, 100k compaction, event-loop delay,
  peak heap/RSS per scenario) and pin the schema in
  test/bench-json.test.ts; add app-side baselines with loose
  complexity budgets in sessionIndex and searchService tests
- fix ClusterDb lock-pool closeAll() leaking in-flight shard opens
  and drain the query store's async close on server shutdown,
  eliminating the ENOTEMPTY directory-teardown race

* perf(minidb): bound startup rebuild and steady-state hot paths

- rebuild all derived indexes in one shared store walk: a single decode
  per record fans out to staged builders, dt rebuild reads record
  metadata only, and index-less opens no longer decode at all
- rank full-text results with a bounded min-heap plus a stable key
  tie-break instead of sorting every candidate
- remove/overwrite text docs via a docID -> delta-terms reverse map
  instead of scanning the whole delta vocabulary
- validate unique batches incrementally against touched postings
  instead of copying the full per-index owner map
- reap due TTL entries from the expiry heap on the write path instead
  of a full-store sweep per write

* feat(session-index): add minidb read model with keyset pagination

- add ISessionIndex read-model lifecycle (prepare/status, ready/degraded
  states) behind the persistence_minidb_readmodel experimental flag
- add ISessionIndexMirror write side recording fresh summaries into a
  bounded, coalescing queue after the authoritative document is durable
- replace the offset cursor with before/after keyset pagination; rename
  list/countActive to listRecent/count
- extend IQueryStore with ordered columns and pageByColumn, plus
  getMany/listKeys/dropCollection
- wire the read model through kap-server routes and start, and update the
  klient sessions contract
- index every session for global search instead of the 500 most recent

* feat(kap-server): bound search sync lifecycle, pagination, and query budgets

- split search requests from sync work: searchIndex() no longer awaits
  runSync/reopen/reindex; a single-flight sync coordinator with debounce
  and backpressure runs in the background, stale generations keep
  serving with explicit stale/degraded state, and refresh/sync/reindex
  failures surface via lastRefreshError instead of being swallowed
- scope file-meta keys by session id (\0meta\file\<sessionId>\<hash>)
  with lazy + one-shot background migration from the legacy hash-only
  keys, so one session sync only touches its own meta rows
- make authoritative scans incremental (mtime/ino/size rescan
  conditions, unchanged files no longer rewrite meta) and read wire
  deltas in 1 MiB chunks instead of whole-file buffer + split
- replace offset pagination with versioned v2 keyset page tokens
  (fingerprint + index generation + sort boundary); generation changes
  fail old tokens with invalid_page_token, legacy v1 offset tokens are
  served once and upgraded, and pages collect via bounded top-K instead
  of full sort + offset skip
- add query budgets enforced at the postings/score stage: max query
  terms, literal length cap, postings visit budget (minidb
  searchBounded/maxVisits with prefix decoding that never fabricates
  hits and skips the postings LRU), candidate caps, deadline and text
  budget; truncation is reported via incomplete reasons
  candidate_cap/postings_budget/deadline
- reopen read-only dbs by opening the next handle before closing the
  previous one so a failed refresh keeps the old generation serving;
  failed opens now self-heal through search traffic

100k-message bench: first page p95 < 300ms and page-100 cost on par
with page 1; event-loop delay during queries stays sub-millisecond.

* fix(minidb): poison, roll back, and recover the WAL on write failures

- give WAL writes a commit point: a failed flushBatch poisons the WAL
  (WAL_POISONED, tracked separately as walWriteErrors vs walFsyncErrors),
  rejects queued frames in reverse enqueue order, and stops scheduling
  further batches; everysec background sync failures stay non-rejecting
  per stage-1 semantics
- recover in place to a known-safe point: a serialized recovery chain
  truncates the WAL back to the first un-acked frame, rebuilds
  size/nextOffset, and clears the poison; writes queue behind the
  recovery gate (zero-cost when idle), a failed truncate flips the
  instance into an explicit writeDisabled state, and a stale truncate
  offset (WAL file replaced by a rotation) skips the truncate
- roll failed flush groups back as a unit: frames are stamped with
  their batchId, MiniDb keeps per-group earliest pre-state, and the
  first rejection restores every key of the group (rejected writes no
  longer reappear after reopen, and in-memory state matches reopen for
  any failure interleaving); the per-op seq guard remains for
  cross-group and rotation-retry races
- wrap applyOp and the following in-memory mutations so a contract
  violation poisons the WAL and rolls the group back instead of
  escaping as a half-commit; frames never enqueued (seal race) roll
  back per-op without poisoning
- tag errors past the commit point with ambiguous: true so callers can
  distinguish "definitely not applied" from "maybe applied but revoked"
- close() waits for the recovery chain to go idle and backup() fences
  behind in-flight recovery before copying files

Controlled A/B bench (22 alternating iterations, 100k concurrent sets):
write-path throughput regression is within the 2% budget.

* fix(minidb): turn the file lock into an instance-owned serialized lease

- distinguish lock ownership by instance instead of pid: every acquire
  mints a pid:uuid token carried by lock/bid/watch files, inspect().mine
  compares tokens, liveness still follows pid, tokenless legacy files
  keep the old stale-takeover path, and hasLiveForeignWatch excludes
  self by token so same-process contenders see each other (closing the
  double-win takeover and the cross-instance release); a live same-pid
  lock is still respected, and re-acquiring a held lock is idempotent
- serialize acquire/renew/release through a per-instance promise-chain
  mutex: renew re-checks held inside the chain and release waits for an
  in-flight renew, eliminating the renew/rename-after-unlink ghost lock
- make MiniDb.close() a state machine (open/closing/closed) with a
  shared closePromise: cleanup runs per-resource try/catch in
  dependency order (text indexes, store, valueReader, WAL, lock),
  aggregates every cleanup error into an AggregateError, stays in
  'closing' on failure so a retry finishes the cleanup, and no longer
  leaks the lock when the WAL close fails; a rejected in-flight
  compaction no longer escapes the cleanup pass

* fix(minidb): keep readers on one consistent file generation

- add an internal persistent-files module as the single source of truth
  for the persisted file set (snapshot, WAL, sidecars, postings
  pattern, fingerprint subset); lock-pool fingerprints, persistentFiles,
  open stale-tmp cleanup, and backup/restore filtering all derive from
  it, and fingerprints upgrade to dev:ino:size:mtimeMs so compound
  sidecar changes can no longer hide from cluster readers
- pair snapshot and WAL generations during recovery (transitional
  stat-pairing until stage-5 manifests): each pass anchors the fds it
  scans, re-stats afterwards, tolerates append-only WAL growth, retries
  bounded times on generation churn with a clean store reset, and
  throws RECOVERY_GENERATION_CHURN when churn exceeds the budget; the
  disk-mode ValueReader attach re-validates inodes so stale offsets
  never read a replaced file
- make the rotation directory fsyncs strict: failures abort the
  rotation through the existing rollback path instead of being
  swallowed, while platforms without directory fsync degrade once with
  a warn and stats.dirFsyncUnsupported

* fix(minidb): serialize index-definition sidecar mutations, persist before publish

- extract the promise-chain mutex into a shared createSerializer() and
  give each sidecar family (secondary/compound/text) its own chain:
  create/drop run uninterruptibly (memory change + rebuild + persist),
  different families stay independent, and the data write path never
  shares these chains
- reverse the publication order to staged -> persist -> publish: a
  create stages the definition, rebuilds via the staged builder,
  persists the sidecar including the new definition, then publishes
  atomically; any failure discards the staged state leaving live and
  sidecar untouched (no phantom indexes, retry-safe); a drop persists
  the sidecar without the definition before removing it live; text
  index create/drop adopt the same pattern, replacing the hand-rolled
  unwind, and a dropping marker keeps compaction postings rebuilds out
  of the persist window
- feed staged indexes from the incremental write path (add/remove/
  checkUnique/checkUniqueBatch visit live+staged) so writes landing in
  the persist window are not lost at publish; queries still see live
  only
- harden writeFileAtomic: instance-unique tmp names (.tmp-pid-seq),
  a strict fsyncDir after rename so a successful persist is crash
  durable, and whitelist-based stale-tmp cleanup that never touches
  lock tmp files

* fix(minidb): validate writes before any side effect, canonicalize values once

- canonical value at the write boundary: the json codec re-parses the
  encoded bytes once and every downstream consumer (unique checks,
  secondary/compound/text indexes, dt extraction) sees exactly the
  persisted representation, so getter/toJSON/Proxy documents can no
  longer diverge between the index view and the storage view
- reorder the set/batch pipeline so every fallible check happens before
  any visible side effect: prepare (key/ttl checks, encoding, canonical
  decode, index field extraction, tokenization) -> unique checks ->
  ensureMemoryFor eviction -> commit; a constraint failure now leaves
  the database untouched (no more evicted victims on rejected inserts),
  and applyOp is structurally pure against pre-validated data
- tokenize at the prepare boundary: TextIndex gains prepareAdd/
  addPrepared and the buildQueue carries validated key+tokens mutations
  instead of raw docs, so a throwing custom tokenizer can no longer
  poison the live view or the queue, and custom-tokenizer output is
  rejected per token over 0xffff bytes before it can permanently break
  postings rebuilds; prepared tokens are keyed by index instance so a
  same-name drop+create mid-write re-tokenizes instead of crossing
  tokenizers
- strict batch structure validation: scanBatchOpRefs/decodeBatchOps
  reject unknown op types, out-of-bounds lengths, and trailing bytes
  (offset must equal body length), so a valid-CRC but malformed batch is
  skipped as a unit and counted via RecoveryInfo.corruptBatches instead
  of being partially applied

Bench vs the stage-1 baseline: json write throughput regression is
within the 5% budget (median ~2-4% depending on the measurement).

* feat(minidb): add OpTracker drain primitive and atomic backup, harden tests

- introduce the internal OpTracker (close gate + in-flight counter with
  enter/leave/close/whenIdle and reference-counted pause/resume) and
  drive every shutdown/drain path from it: WAL background syncs are
  tracked so close() waits out an in-flight sync before closing the fd,
  cluster lock-pool closeAll() closes the gates and drains busy
  callbacks before closing handles, and MiniDb writes pass a write gate
- make backup() atomic with a defined linearization point: pause the
  write gate, drain in-flight writes (every acknowledged write is now
  included), copy to a sibling temp dir with per-file fsyncs, write the
  manifest last as the commit marker, and rename into place; failures
  clean up and leave no partial backup, and concurrent writes are
  rejected with BACKUP_IN_PROGRESS
- reap emptied compound-index groups on remove (the groups map no
  longer grows monotonically), move the open-time mkdir behind the
  readOnly check so a read-only open of a missing directory fails with
  ENOENT instead of creating it, and never run a destructive rebuild
  for a read-only open failure (explicit or onLockFail fallback)
- consolidate every review fault-injection repro into the formal suite
  behind deterministic barrier helpers (programmable writev/sync/
  rename/tokenize hooks) and convert the six timing-based tests to
  barrier/tick-driven assertions; the .tmp repro scripts are removed

The converted timing tests and the full suite pass 50 repeat runs
(including under CPU load injection) with zero flakes.

* feat(minidb): persist derived indexes as atomic generations, open from WAL delta

- checkpoint the store, dt/secondary/compound indexes, and text
  dictionary/postings/docs into immutable generations under
  generations/g-NNNNNN published atomically (tmp build, per-file
  checksums and fsyncs, dir rename, CURRENT swap, strict dir fsyncs);
  the manifest records the format version, WAL/snapshot checkpoint
  anchors, per-index definition hashes, and codec/value-mode
  compatibility
- open now loads the published generation and replays only the WAL
  delta after its checkpoint: no full value decode, corpus
  tokenization, or postings rewrite on a normal reopen (warm opens are
  3.5-13.8x faster at 100k/1M records); a definition change rebuilds
  only the affected index, and corrupt generation files fall back to
  the previous generation or the legacy full recovery without ever
  touching the authoritative snapshot/WAL
- build generations transactionally with compaction (rotation plus
  derived state publish as one unit, replacing the synchronous
  rebuildTextPostings tail), capture concurrent writes through a sealed
  op queue with byte/op caps, hard-link clean postings and the snapshot
  into the new generation, and repoint every live text base into the
  CURRENT generation after publish
- cluster/read-only refresh watches CURRENT and the WAL watermark:
  pure generation publishes keep readers on incremental catch-up while
  rotations reopen onto the new generation; writers building the next
  generation never disturb readers of the current one
- legacy databases open through the old path unchanged and gain their
  first generation in the background; OpenOptions.indexGenerations:
  false fully restores the pre-generation behavior

* feat(minidb): workerize text-index builds and split MiniDb into facets

- split the monolithic src/index.ts into facet modules (mini-db, types,
  value-codec, memory-guard, backup, query-engine, text-registry,
  wal-group, generation-builder/loader, write-path, read-path,
  index-admin, lifecycle, stats) and move text-index.ts to text-index/
- run corpus-scale text-index builds off the main thread via the bounded
  worker engine (src/worker/), exported through the new worker-runtime
  subpath, with inline fallback for small corpora and rollback switches
- defer the open-time fallback text rebuild into a maintenance task;
  searches on a not-yet-committed base raise TextIndexBuildingError
- add the unified maintenance scheduler, bounded async read surface,
  and a maintenance bench
- kap-server search: switch to searchBoundedAsync and serve the
  building page while the index base rebuilds after fallback recovery
- kimi-code: install the SEA-bundled minidb text-build worker at
  startup, bundle it via the native asset scripts, and add the
  startup-trace util plus the KIMI_TUI_INPUT_LATENCY debug probe

* fix(minidb): treat win32 EPERM as unsupported directory fsync

- extract isUnsupportedDirectoryFsyncError and cover win32 EPERM
- drop the one-shot console.warn; stats.dirFsyncUnsupported carries the degraded state

* fix(kap-server): harden search-index dispose and drain lifecycle

- dispose() now closes an OpTracker gate and drains in-flight sync/refresh
  passes before closing the db, so no background write can hit a closed
  handle; the deleteSessionDocs loop and trailing stats write skip once
  the gate closes (review #20)
- drainGlobalSearchDisposals loops to a fixpoint so disposals registered
  while a drain is in flight are also awaited (review #21)
- pin the post-open failure semantics with a regression test: a failed
  text-index setup closes the handle and the next open reacquires the
  writer lock instead of self-locking read-only (review #19)
- export OpTracker from the minidb root for the search service's drain

* chore: fix oxlint type-aware lint errors
2026-08-04 20:38:42 +08:00
liruifengv
7859b0afe8
feat(kimi-code): vendor @moonshot-ai/pi-tui (#1254)
* chore(kimi-code): upgrade pi-tui to 0.78.1 and adapt native helpers

Bump @earendil-works/pi-tui from ^0.74.0 to ^0.78.1. pi-tui 0.75.5 replaced its koffi-based Windows VT input with a bundled native helper, and 0.76.0 added a darwin native helper for Terminal.app Shift+Enter.

- SEA build: teach native-deps to collect pi-tui's per-target .node files, drop the koffi registry, and add a native-file-only collect mode so only package.json + the target .node ship (28 -> 2 files).

- Redirect pi-tui's absolute-path native require() into the native-asset cache through the Module._load hook, and extend the native smoke test to actually load the helper.

- npm package: ship pi-tui's native/ directory so macOS Terminal.app Shift+Enter and Windows Shift+Tab keep working for npm installs.

* chore: add changeset for pi-tui upgrade

* chore: vendor @earendil-works/pi-tui 0.80.2

Fork the upstream pi-tui source into packages/pi-tui for local modification. Pristine snapshot of @earendil-works/pi-tui@0.80.2; the apps/kimi-code dependency on ^0.78.1 from npm is left unchanged.

* feat(kimi-code): integrate vendored @moonshot-ai/pi-tui

Replace the npm @earendil-works/pi-tui dependency with the vendored @moonshot-ai/pi-tui workspace package so the fork can be modified locally.

- Point apps/kimi-code imports and native-deps at @moonshot-ai/pi-tui.

- Make pi-tui source-first (exports -> src, publishConfig.exports -> dist, mirroring node-sdk) and strict-clean: bracket access for process.env / named capture groups, an override modifier, and non-null assertions for noUncheckedIndexedAccess.

- Bump the root tsconfig target to ES2024 and enable allowImportingTsExtensions (needed for pi-tui's /v regex and .ts imports, which node --test requires).

- Add packages/pi-tui to flake.nix workspaces and exclude the vendored source from oxlint.

* fix(pi-tui): export package.json for native asset resolution

The SEA native-asset collector resolves the package root via
require.resolve('@moonshot-ai/pi-tui/package.json'). The vendored
package's exports field only exposed ".", which blocked the
"./package.json" subpath and broke build:native:sea with
ERR_PACKAGE_PATH_NOT_EXPORTED.

* chore(kimi-code): sync pi-tui native prebuilds at build time

Copy packages/pi-tui/native prebuilds into apps/kimi-code/native
during build instead of tracking a manual copy in git. Only the
.node prebuilds are copied (not the C sources); the directory is
now a build artifact covered by .gitignore.

* fix(pi-tui): avoid destructive full redraw during streaming

When the first changed line is above the viewport, the differential
renderer fell back to fullRender(true), which clears scrollback and
yanks the user's viewport. On Windows Terminal this jumps to the
absolute top (microsoft/Terminal#20370).

Clamp the diff to the visible viewport when content length is
unchanged (spinner tick / markdown reflow above the viewport), so
streaming no longer triggers a full redraw in those cases. Length
changes still fall back to fullRender to reset the viewport.

* fix(kimi-code): update pi-tui imports in files merged from main

Two files added on main (effort-selector, plugin-command) still
imported the old @earendil-works/pi-tui package name; point them at
the vendored @moonshot-ai/pi-tui.

* chore(nix): update pnpmDeps hash after lockfile regen

* fix(pi-tui): clamp above-viewport diff even when content shrinks

Previously, when the first changed line was above the viewport and
content length changed (e.g. spinner removed at end of streaming), the
renderer fell back to fullRender(true), which clears scrollback and
yanks the viewport to the absolute top on Windows Terminal.

Always clamp the diff to the visible viewport instead, preserving the
user's scroll position. Stale bytes remain in scrollback but are not
visible.

* fix(kimi-code): keep activity placeholder to avoid streaming shrink

When streaming ends, removing the activity spinner shrank the content
by two rows, which (combined with transient→final code highlighting
above the viewport) triggered a destructive full redraw. Keep a one-row
placeholder in the activity pane when idle so the content does not
fully shrink.

* chore: refine streaming scroll changeset wording

* chore: remove obsolete pi-tui native helpers changeset

* chore: add pi-tui changesets and document the pi-tui changelog rule

Add changesets for the fork integration, package manifest export, and
viewport clamp fix so the vendored pi-tui keeps its own changelog.

Update the gen-changesets skill to treat @moonshot-ai/pi-tui as a
special internal package that lists itself instead of the CLI, with a
separate CLI changeset only when the change is user-visible there.

* chore(nix): update pnpmDeps hash after merging main

* fix(changeset): drop private kimi-code-docs from google-genai changeset
2026-07-01 20:23:35 +08:00
qer
3554f7e7d6
feat(plugins): source Superpowers from GitHub and show update badges (#1066)
Some checks are pending
CI / test (push) Waiting to run
CI / lint (push) Waiting to run
CI / build (push) Waiting to run
CI / typecheck (push) Waiting to run
Nix Build / Check flake.nix workspace sync (push) Waiting to run
Nix Build / nix build .#kimi-code (push) Blocked by required conditions
Release / Native release artifact (push) Blocked by required conditions
Release / Release (push) Waiting to run
Release / Deploy docs (push) Blocked by required conditions
Release / Publish native release assets (push) Blocked by required conditions
* feat(plugins): source Superpowers from GitHub and show update badges

Source the Superpowers plugin from its GitHub release (v6.0.3) instead of a vendored copy, and drop the explicit version field.

Derive marketplace entry versions from GitHub source URLs when the version field is omitted, keeping the source URL the single source of truth.

Show update badges for installed plugins on the /plugins Installed tab.

* docs(plugins): document Installed tab update badges

* fix(plugins): stamp GitHub source version in CDN catalog

Older CLIs only read the explicit marketplace version and cannot derive it from a GitHub source URL. When publishing the CDN catalog, stamp the version derived from a pinned GitHub source so those clients still surface update badges.

The source plugins/marketplace.json keeps no explicit version; the version is derived at build time instead.

* feat(plugins): resolve latest version for bare GitHub sources at runtime

Point the Superpowers marketplace entry at the bare GitHub repo URL so it tracks the latest release instead of a pinned tag.

When a marketplace entry omits version and its source is a bare GitHub repo URL, resolve the latest release tag at load time (via the /releases/latest redirect) to fill the version for update detection.

Revert the build-time version stamping; it is no longer needed. Older CLIs that only read the explicit catalog version will no longer see update badges for Superpowers, since the catalog no longer carries one.

* feat(plugins): make Enter update and add I for details on Installed tab

On the Installed tab, Enter now installs the available update when one is present, and falls back to opening plugin details otherwise.

Add the I key to always open plugin details, so details remain reachable when Enter is occupied by an update. Update the installed hint, docs and changeset accordingly.

* feat(plugins): show installing state inside the plugins panel

Move the "Installing … from marketplace" notice from a transient status message into the plugins panel itself, so the user sees progress in the interactive card while an install or update is in flight.

* feat(plugins): highlight reload hint and add dev:cli:marketplace

Highlight "Run /new or /reload to apply plugin changes." in warning color after plugin install and remove, and make the two notices symmetric.

Add a root dev:cli:marketplace script that points the dev CLI at the production marketplace instead of the local dev server.

* fix(plugins): dedupe install success notice

Drop the redundant showNotice on marketplace installs so the success message is shown only once, symmetric with remove.

* fix(plugins): reset installing state on install failure

When a marketplace or Custom-tab install rejects, clear the installing state and return to the list so the user can retry, instead of leaving the panel stuck on the one-way "Installing…" view.
2026-06-24 21:58:13 +08:00
qer
ebf6e8181e
feat: add plugin manager and official plugins (#119)
* feat: add plugin manager and official plugins

* fix(agent-core): honor plugin capability overrides

* fix: restrict plugin zip root detection

* Update apps/kimi-code/src/constant/app.ts

Co-authored-by: liruifengv <liruifeng1024@gmail.com>
Signed-off-by: qer <wbxl2000@outlook.com>

---------

Signed-off-by: qer <wbxl2000@outlook.com>
Co-authored-by: liruifengv <liruifeng1024@gmail.com>
2026-05-27 22:47:33 +08:00
Kaiyi
842e699a64 Kimi For Coding 2026-05-22 15:54:50 +08:00