mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-08-14 19:24:54 +00:00
feat(serve): observe daemon and child memory against real denominators (#8423)
* feat(serve): observe daemon memory pressure against a real denominator The daemon samples its own RSS and heap but has nothing to divide them by, so nothing in `/daemon/status` says whether a figure is fine or nearly fatal. #8245 landed the denominator (`limits.memory`); this turns it into a reading. `runtime.memory.pressure` reports `level`, `ratio`, `source`, and the six raw figures behind them. The level is the worse of two independent ratios, because the two failure modes are independent: a container dies by RSS against its cgroup limit, while a process on a large host can exhaust V8's heap long before RSS is a meaningful fraction of the machine. Reporting only one hides whichever failure the deployment is actually heading for. `source` names which ratio produced the level, and `unknown` says the daemon could not measure itself — which a consumer must not read as healthy. The denominator is `availableMemoryMb`, not `effectiveBudgetMb`: pressure asks how close this process is to being killed, and what kills it is the cgroup limit or host memory. An operator's budget is a policy number, so classifying against it would report `critical` for a daemon in no danger. `--memory-pressure-mode` is `off | observe`, default `observe`. Both modes report every figure; only `observe` also raises the `daemon_memory_pressure` warning, so `off` leaves the top-level `status` rollup untouched — the thresholds are inherited from an interactive-CLI monitor and are not yet calibrated for a long-running daemon, and a deployment that alerts on `status` needs the reading without the verdict. There is deliberately no `enforce`: nothing here remediates, and a value a caller can pass but never use is a dead switch. It arrives with the enforcement. Scope is the daemon root process only. `childRssCoverage` still reads `primary_only` and says so on the wire; aggregate child RSS and channel workers are separate measurements and land separately. Severity is `warning` at every level including `critical`, because `error` would make `rollupStatus` return `error` for the whole daemon — too strong a claim to stake on uncalibrated thresholds. Refs #8051. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(serve): report aggregate ACP child RSS, not just the primary's (#8462) * test(serve): close the under-determined assertions review probed The automated review mutation-probed this diff and found several assertions that were live but under-determined — each mutant it names kept the whole suite green. All confirmed locally, and all now fail: - Deleting `level !== 'normal'` from the issue gate raised daemon_memory_pressure on a healthy daemon and flipped top-level status to warning on every response — the exact false positive `--memory-pressure-mode off` exists to opt out of. Now covered on both sides: nothing raised at a realistic denominator, exactly one warning at a denominator sized to land this process in `soft`. - Summing children over `list()` instead of `listManaged()` dropped a draining-but-process-holding workspace while `activeAcpChildren` still counted it. The draining bridge now reports RSS, so the byte count can only come from that child. - The message's denominator ternary had no coverage; inverting it sent an operator hunting RSS growth during a heap-driven incident. - A truthiness guard on `ageMs` turned a measured-fresh reading (age exactly 0, when a status read lands in the sampler's millisecond) into `null`, which the field's own docs say never means fresh. - The multi-contributor age test listed ages ascending, so a plain-overwrite accumulator produced the same answer as Math.max. Reordered descending, which kills last-wins and first-wins both. Two declaration-only hunks — the issue-code union member and the `pressure` field — were guarded by tsc alone, which vitest does not run. Both are now pinned at runtime by asserting the code string and the full key set. Also fixes a real display defect: `toFixed(0)` renders a ratio of 0.795 as "hard at 80%", and 80% is critical's documented threshold. One decimal, so the number and the level cannot contradict each other. And corrects a JSDoc claim of mine that was simply wrong: `pressure` is absent not only for direct-embed but on the bootstrap /daemon/status route, which omits runtime.memory wholesale even though the budget is resolved — and that window is not just startup, since a daemon whose runtime fails to start serves the bootstrap app for its lifetime. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(serve): model a per-child heap partition of the daemon budget (#8508) * feat(serve): add the child-heap admission primitives, unwired Groundwork for #8182 step 2. Nothing calls any of this yet, so no child is sized differently and no spawn is refused. `ProcessRegistry.committedProcessCount` counts attached children plus reservations that have not attached. That is the figure admission has to key on: `reserve()` inserts its token synchronously before `spawn()`, so two racing spawns each see the other, while neither appears in `activeProcessCount` until its child attaches. A child leaves the count on exit rather than when `terminate()` starts, so a channel swap counts twice while the old process winds down — deliberate, since its memory is still resident. `getAcpMemoryArgs(explicitMb?)` takes an optional share that bypasses both the module cache and the raise-only guard. Both bypasses are load-bearing. The cache, because the share depends on how many children are live now rather than on the host. The guard, because a budget-derived share is normally *below* the daemon's own heap limit, so routing it through `targetMB > currentLimitMB` would drop the flag, silently restore the overcommit, and leave every test green — the trap against a multi-GB runner, and mutation-checking it by reinstating the guard fails two tests. `createChildHeapPolicy` holds the mode, the budget, and the would-be refusal counter, and answers `decide(concurrentChildren)`. The refusal is derived from the unclamped quotient, not from `recommendedChildShareMb`, because that function clamps *up* to the 512 MB floor: past the point where the pool stops covering the count its answer saturates and can no longer distinguish "barely does not fit" from "wildly does not fit". `ChildHeapPoolExhaustedError` with both transport mappings — REST 503 with Retry-After, ACP `child_heap_pool_exhausted` — added together, since the two mappings are hand-written and drift silently otherwise. Refusing at spawn rather than at registration is the correction #8182 demands: registration allocates nothing, so this surfaces as "no new session in this workspace right now", which is true and retryable. Refs #8182. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(serve): size each ACP child by concurrently live children Wires the primitives from the previous commit into the spawn path, behind `--child-heap-mode off | observe | enforce`, default `observe`. Under `enforce` a child's `--max-old-space-size` is a share of the child pool divided by the children concurrently committed at the moment it spawns — read from the shared ProcessRegistry after `reserve()`, so two racing spawns each see the other. When the pool cannot cover another child at the 512 MB floor the spawn is refused with ChildHeapPoolExhaustedError, which is what turns a per-child ceiling into an aggregate bound: concurrent children can never exceed pool/512. Keyed on concurrency, never on registrations. A dormant workspace has no child, so it costs nothing — the specific correction #8182 records against the withdrawn proposal, which would have shrunk a lone live child to 614 MB because of 24 idle registrations. Default `observe` computes the share and the admission decision and applies neither, counting the refusals that would have happened. The divisor has never been checked against a real multi-workspace deployment, and a non-zero count is how an operator learns enforcement would have broken them without being broken. It also catches the case worth worrying about: a channel swap counts the dying child alongside its replacement, so on a saturated pool enforcement could refuse a restart and leave that workspace with no child at all. Excluding terminating children would authorise real overcommit to dodge a hypothetical refusal, so the count reports it instead. Ceilings already granted are not revisited — V8 cannot lower them — so granted ceilings transiently exceed the pool. Acceptable: the flag is a ceiling, not a reservation, and a workspace with no live sessions has no child and picks up the current share on its next spawn. `limits.memory.enforced` stops being a required literal `false`. #8245 made it one so a client could never mistake that namespace for enforcement that had not shipped; it has now, so the field is a boolean derived from the mode — and stays `false` under `observe`, which applies nothing. Refs #8182. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(serve): correct the claims child-heap enforcement makes false Two sentences in the protocol doc described the memory section as unconditionally observational: "a required `enforced: false`", and "no child spawn argument derives from these values, and no request is refused on their basis". Both are false under `--child-heap-mode enforce`, so both are rewritten rather than left to rot — `enforced` is now documented as the boolean that answers exactly this, and the refusal is documented with its wire shape on both transports. Also documents `childHeap.refusals` as the calibration signal, since a would-be-refusal count is useless if operators do not know to read it before switching to `enforce`; the flag row in the three operator docs; and the design doc's Part 1, which listed applying a share as a compatibility risk without recording how that was resolved. The end-to-end test asserts the policy reaches a real booted daemon's status with `enforced: false` under the default mode — the wire type in that test is a hand-written mirror, so its `enforced: false` literal had to widen too, which is the check that caught the type not being widened everywhere. Refs #8182. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(serve): cover both branches of the enforced tripwire `enforced` was only ever asserted false — the unit tests build no policy and the end-to-end daemon runs the default `observe` mode, so the branch that makes the field worth having was untested. Hardcoding it back to `false` passed everything. Also pins `childHeap: null` as distinct from a policy in `off` mode: the first says no policy exists (direct-embed, or the bootstrap window before the runtime is built), the second says one exists and computes nothing. Refs #8182. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(serve): partition the child pool so granted ceilings stay inside it Review was right that the previous design did not deliver the aggregate bound it claimed. Sizing each child by the count live at *its* spawn bounds the child count but not the memory: V8 cannot lower a running child's ceiling, so grants accumulate as P + P/2 + P/3 + ... = P x H(n). Reproduced exactly — 9557 MB authorised against a 3687 MB pool at seven children on an 8 GB host, and 61355 MB against 15360 MB at the limit on 32 GB. That is 2.6x and 4x the pool, which is what the policy exists to prevent. Grant accounting alone does not fix it: the first child would take the whole pool and the second would be refused immediately. Keeping the invariant requires early children not to receive the whole pool, so the ceiling is now a fixed partition — childPoolMb / maxConcurrentChildren, constant for every child, with maxConcurrentChildren itself derived from the pool and capped at MAX_DAEMON_WORKSPACES. The sum is then n x ceiling <= pool by construction, with no ledger of outstanding grants and no dependence on arrival order. Tested as an invariant across four host sizes: fill the daemon to its admission limit and the authorised total still fits. The cost is deliberate and now documented rather than hidden: a lone workspace on a 32 GB host gets 614 MB rather than the pool, because any child may still be running when the house fills. An 8 GB host admits seven concurrent children at 526 MB each. Also from review: - The policy is no longer built for an injected `deps.bridge`. That bridge carries its own channel and never reaches the factory the policy rides on, so status could report `enforced: true` while nothing was being sized. - Both transport mappings now have direct tests. They are hand-written beside each other and drift silently; the spawn-policy tests cannot catch a wire regression. - Swept the "does not size any child" claim, which enforce makes false, out of the CLI help text, ServeOptions docs, the two operator tables, and the e2e header comment. The 17-configuration table realigns wholesale because that cell was its widest — whitespace only. Refs #8182. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(serve): model the child heap partition, defer applying it Review established that the refusal counter cannot tell an operator whether enforcement is safe, and that is the ground the enforcing mode stood on. While observing, children run on the host-derived ceiling (16384 MB on a 32 GB host), so a workload needing 2 GB of old space is healthy with zero refusals and OOMs the moment a 614 MB partition is applied. The counter measures admission pressure, not ceiling adequacy. Rather than ship a switch with no safe way to decide when to turn it on, `enforce` is removed. `--child-heap-mode` is `off | observe`, and the mode that would apply the partition arrives with the measurement that justifies it: peak old-space per child, compared against the modeled ceiling. That is a real measurement chain — the child reports rss and cpu today, and `--max-old-space-size` bounds old space specifically, so neither rss nor heapUsed answers the question. With nothing applying the partition, the machinery that existed only to apply it goes too rather than shipping unreachable: `getAcpMemoryArgs(explicitMb?)`, `ChildHeapPoolExhaustedError` and both transport mappings, and `limits.memory.enforced` reverts to the required literal `false` it was before. The spawn path is untouched again; the factory asks the policy what it would decide purely so the count is real. Also fixes the zero-pool defect review found, which the removed clamp caused: forcing at least one admissible child on a 512 MB host — where the root reserve consumes the whole 256 MB budget — produced a ceiling of 0, and `--max-old-space-size=0` is V8's *default* heap, not a zero ceiling. A pool that cannot cover one child at the floor now reports `maxConcurrentChildren: 0` and `perChildCeilingMb: null`, and the test that enshrined the old behaviour is inverted. Status now publishes `maxConcurrentChildren` and `perChildCeilingMb`, so an operator can judge the partition against their own workload — the substitute for a counter that cannot judge it for them. Every claim that a zero refusal count means the partition is safe to apply is removed from the flag help, the operator docs, the protocol doc, and the design doc. Refs #8182. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * fix(serve): repair the child-heap assertion and the reservation leak Three findings review raised against #8508 after the partition became observation-only, all still live on this branch now that it has merged. The status assertion in `run-qwen-serve.test.ts` failed on head: it used `toEqual` against `{ mode, refusals }` while the wire also carries `maxConcurrentChildren` and `perChildCeilingMb`, so the suite was red at 217 passed / 1 failed. The local type restating the wire shape was short the same two fields. Both are filled in, and the assertion stays `toEqual` so an unannounced field still fails it — the two derived figures get matchers because this suite boots a real daemon and the pool follows the machine. What they have to satisfy is now pinned separately: a fixed ceiling times the number admitted must fit inside the pool it partitions, which is the whole reason the partition bounds anything. `decide()` and `getAcpMemoryArgs()` ran between `reserve()` and the `try` that cancels the reservation. `childHeapPolicy` is a public `createSpawnChannelFactory` option, so `decide()` is caller code and may throw; the spawn then rejected with the token held for the process lifetime, inflating `committedProcessCount` for every later spawn. Both calls move inside the `try`. The regression test is mutation-verified — reverting the move gives `expected 1 to be +0`. `ServeOptions.memoryBudgetMb` still promised a `childHeapMode: 'enforce'` that sizes children and refuses spawns. No such mode exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(serve): report no child-heap partition under `off` `snapshot()` returned `maxConcurrentChildren` and `perChildCeilingMb` unconditionally, so a daemon run with `--child-heap-mode off` still published a partition — 7 children at 526 MB on an 8 GB host — under a mode whose documentation says "do not model it". Review raised it, and it mattered more than it looked: with `enforce` gone, `off` and `observe` differed only in whether `refusals` incremented, so nothing on the wire distinguished a model that was switched off from one in force. Both figures are now `null` under `off`, which required widening `maxConcurrentChildren` to `number | null` in the daemon type and the SDK mirror. `null` rather than `0`: zero is already the computed answer for a pool too small to host one child at the 512 MB floor, and collapsing the two would tell an operator who disabled the model that their host cannot run anything. That leaves three distinguishable states — no policy at all (`childHeap: null`), a policy modeling nothing (`mode: 'off'` with null figures), and a live model — and each now has a test. The `off` unit test previously asserted only `refusals`, so its name ("models nothing at all when off") promised more than it checked. It now covers the figures, with a sibling test pinning 7 / 526 under `observe` on the same budget so nulling them unconditionally cannot satisfy both. Mutation-verified in both directions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(serve): never model a child heap ceiling below the documented minimum `perChildCeilingMb` is `min(floor(pool / maxConcurrentChildren), legacyChildCeilingMb)`. The first term is at least `MIN_CHILD_HEAP_MB` by construction; the second is `floor(available / 2)` and is not, so the `Math.min` could publish a ceiling *below* the `minChildHeapMb` sitting beside it in the same snapshot: avail=768 --memory-budget-mb 1024 pool=512 legacyCeil=384 perChild=384 avail=1023 --memory-budget-mb 1024 pool=767 legacyCeil=511 perChild=511 Unreachable from a derived budget — the pool reaches 0 first — but an explicit budget has a floor of 1024 while available memory does not, and `docs/users/qwen-serve.md` tells operators on exactly these hosts to pass that flag. The documented remedy is what reaches the band. Refuse the model rather than shrink under the floor, with `maxConcurrentChildren` zeroed in lockstep: a ceiling no child may run at is not a partition, and "one child fits" beside a null ceiling is the same contradiction from the other side. Nothing is applied today so the impact was a wrong published figure, but this is the number the partition asks to be judged by and the one an `enforce` mode would hand to `--max-old-space-size`. The existing matrix resolves derived budgets only, which is why the mutation sweep came back clean; add the `budgetMb` axis, asserting in each case the shape that makes it reachable, and pin the inclusive boundary (1024/1024 -> one child at 512) so nulling unconditionally cannot pass instead. Also, in the same review pass: - Split usable-gauge handling into numerator and denominator. Coercing an unusable numerator to 0 published `rssBytes: 0, rssRatio: 0, level: 'normal', source: 'rss'` — a daemon that measured nothing, indistinguishable from an idle one, which is the confusion `source: 'unknown'` and `sampled: 0` exist to prevent everywhere else here. An unusable numerator now retires its own side. Zero stays a reading for a numerator and not for a denominator. - Document that `rssRatio` divides by host total under `availableMemorySource: 'host'`, so it is a lower bound on real pressure there — a denominator problem no threshold calibration addresses. - Document that `refusals` counts channel swaps at full occupancy (the terminating child is counted until it exits) and equals the total spawn count on a host too small to model a partition. Deliberately not fixed by giving the comparison swap headroom, which would admit a 26th ceiling against a 25-child pool. - Keep the sampler's rejection handler as a documented backstop — the shipped `refreshChildResource` never rejects, but it is an optional `async` interface member, so a foreign implementation throwing early would otherwise surface as an unhandled rejection — and give it the workspace so it is attributable across the fan-out. - Test hygiene: drop a duplicated `enforced` assertion; replace a host- dependent `expect.any(Number)` with a key-set pin plus a branch, since a small host now legitimately reports no partition; use `vi.spyOn(Date, 'now')` over direct assignment; reuse the exported `ChildHeapMode` on the child-heap side, leaving the independent `memoryPressureMode` switch alone. Reported by @wenshao. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>
This commit is contained in:
jinye
GitHub
parent
650e085fec
commit
2eb5cd6df5
30 changed files with 2121 additions and 78 deletions
|
|
@ -6,42 +6,44 @@ This page collects every setting that affects the `qwen serve` daemon and its ad
|
|||
|
||||
## CLI flags (`qwen serve`)
|
||||
|
||||
| Flag | Type | Default | Effect |
|
||||
| --------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `--hostname <host>` | string | `127.0.0.1` | Bind address. Loopback values: `127.0.0.1`, `localhost`, `::1`, `[::1]`. Non-loopback requires a bearer token at boot. `host:port` input is rejected with guidance to use `--port`. |
|
||||
| `--port <n>` | number | `4170` | Listen port; `0` means ephemeral. |
|
||||
| `--token <s>` | string | env | Bearer token. Overrides `QWEN_SERVER_TOKEN` and is trimmed at boot. It appears in the process command line, so prefer env in deployments. |
|
||||
| `--require-auth` | boolean | `false` | Extends bearer auth to loopback and `/health`; boot refuses to start without a token. |
|
||||
| `--workspace <dir>` | absolute path / repeatable | `process.cwd()` | Startup workspace runtime; repeat to register additional isolated runtimes. The first is primary. Every value must be absolute and a directory; canonicalized at boot. |
|
||||
| `--memory-project-scope <mode>` | `git-root` / `workspace` | `git-root` | Project-memory partitioning. `git-root` shares memory among workspaces at the same Git root; `workspace` isolates by exact workspace directory. Overrides `QWEN_CODE_MEMORY_PROJECT_SCOPE`. |
|
||||
| `--max-sessions <n>` | number | `32` | Per-workspace active session cap. `0` / `Infinity` means unlimited; `NaN` / negative values throw. |
|
||||
| `--max-total-sessions <n>` | number | derived for multiple startup/restored workspaces | Daemon-wide active session cap. When omitted, a finite default is derived once from the per-workspace cap and startup/restored workspace count. `0` / `Infinity` means unlimited. |
|
||||
| `--max-pending-prompts-per-session <n>` | number | `5` | Accepted but pending/running prompt cap per session. Excess prompt returns 503. `0` / `Infinity` means unlimited; negative or non-integer values throw. |
|
||||
| `--max-connections <n>` | number | `256` | HTTP listener `server.maxConnections`; `0` / `Infinity` means unlimited. |
|
||||
| `--enable-session-shell` | boolean | `false` | Enables direct `POST /session/:id/shell` execution. Requires bearer token, and every call must carry a session-bound `X-Qwen-Client-Id`. |
|
||||
| `--event-ring-size <n>` | number | `8000` | Per-session SSE replay ring; soft cap is `1_000_000`. |
|
||||
| `--compacted-replay-max-bytes <n>` | positive integer | `4194304` | Byte cap for the bounded in-memory replay snapshot returned by `POST /session/:id/load`; hard cap is `268435456`. |
|
||||
| `--memory-budget-mb <n>` | integer in `[1024, 1048576]` | 50% of cgroup-constrained or host memory, capped at the flag maximum (1048576 MB) | Total memory budget for the daemon process tree, capped at resolved available memory. Observed and reported under `limits.memory` in daemon status; it does not size any child process. Boot rejects out-of-range values. |
|
||||
| `--http-bridge` | boolean | `true` | Stage 1 bridge mode. `--no-http-bridge` still falls back to http-bridge and prints to stderr. |
|
||||
| `--mcp-client-budget <n>` | positive integer | unset | Sets `WorkspaceMcpBudget.clientBudget` and forwards it to the ACP child through `childEnvOverrides`. |
|
||||
| `--mcp-budget-mode <m>` | `off` / `warn` / `enforce` | `warn` when budget is set, otherwise `off` | Sets `WorkspaceMcpBudget.mode`; `enforce` requires `--mcp-client-budget`. |
|
||||
| `--external-tool-guard-mode <m>` | `off` / `required` | `off` | Enables the managed ACP external pre-execution Guard. `required` fails startup unless its loopback provider completes the v1 handshake. |
|
||||
| `--external-tool-guard-endpoint <url>` | loopback HTTP(S) origin | unset | Provider origin used only in `required` mode. It must be origin-only and use `127.0.0.1`, `localhost`, or `::1`; paths, credentials, redirects, and proxy routing are rejected. |
|
||||
| `--external-tool-guard-timeout-ms <n>` | integer `100..30000` | `3000` | Per-handshake and per-prepare deadline. A timeout fails startup during the handshake or fails the invocation closed during a turn. |
|
||||
| `--allow-origin <pattern>` | repeatable string | unset | Cross-origin allowlist that replaces the default CORS denial. `*` allows any origin but requires a token. |
|
||||
| `--allow-private-auth-base-url` | boolean | `false` | Allows `/workspace/auth/provider` to install localhost / private-network auth provider `baseUrl`; use only in trusted local development. |
|
||||
| `--prompt-deadline-ms <n>` | positive integer | unset | Server-side prompt wallclock limit in ms. Timeout aborts and returns an error. |
|
||||
| `--writer-idle-timeout-ms <n>` | positive integer | unset | Per-SSE-connection idle timeout in ms. The daemon closes the SSE connection when no event is sent for this duration. |
|
||||
| `--channel-idle-timeout-ms <n>` | non-negative integer | `0` | How long to keep the ACP child alive after the last session closes. `0` means reclaim immediately. |
|
||||
| `--initialize-timeout-ms <n>` | positive integer | `10000` | ACP child request timeout, including the initialize handshake (ms). |
|
||||
| `--session-reap-interval-ms <n>` | non-negative integer | `60000` | Session reaper scan interval; `0` disables it. |
|
||||
| `--session-idle-timeout-ms <n>` | non-negative integer | `1800000` | Disconnected-session idle reaping time; `0` disables it. |
|
||||
| `--rate-limit` / `--no-rate-limit` | boolean | env / off | Enables per-tier HTTP rate limiting for prompt, mutation, and read routes. |
|
||||
| `--rate-limit-prompt <n>` | positive integer | `10` | Prompt request limit per window; requires rate limiting to be enabled. |
|
||||
| `--rate-limit-mutation <n>` | positive integer | `30` | Mutation request limit per window; requires rate limiting to be enabled. |
|
||||
| `--rate-limit-read <n>` | positive integer | `120` | Read request limit per window; requires rate limiting to be enabled. |
|
||||
| `--rate-limit-window-ms <n>` | integer `>= 1000` | `60000` | Rate limit window length; requires rate limiting to be enabled. |
|
||||
| no flag | - | - | `QWEN_SERVE_NO_MCP_POOL=1` fully disables the pool. |
|
||||
| Flag | Type | Default | Effect |
|
||||
| --------------------------------------- | ---------------------------- | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `--hostname <host>` | string | `127.0.0.1` | Bind address. Loopback values: `127.0.0.1`, `localhost`, `::1`, `[::1]`. Non-loopback requires a bearer token at boot. `host:port` input is rejected with guidance to use `--port`. |
|
||||
| `--port <n>` | number | `4170` | Listen port; `0` means ephemeral. |
|
||||
| `--token <s>` | string | env | Bearer token. Overrides `QWEN_SERVER_TOKEN` and is trimmed at boot. It appears in the process command line, so prefer env in deployments. |
|
||||
| `--require-auth` | boolean | `false` | Extends bearer auth to loopback and `/health`; boot refuses to start without a token. |
|
||||
| `--workspace <dir>` | absolute path / repeatable | `process.cwd()` | Startup workspace runtime; repeat to register additional isolated runtimes. The first is primary. Every value must be absolute and a directory; canonicalized at boot. |
|
||||
| `--memory-project-scope <mode>` | `git-root` / `workspace` | `git-root` | Project-memory partitioning. `git-root` shares memory among workspaces at the same Git root; `workspace` isolates by exact workspace directory. Overrides `QWEN_CODE_MEMORY_PROJECT_SCOPE`. |
|
||||
| `--max-sessions <n>` | number | `32` | Per-workspace active session cap. `0` / `Infinity` means unlimited; `NaN` / negative values throw. |
|
||||
| `--max-total-sessions <n>` | number | derived for multiple startup/restored workspaces | Daemon-wide active session cap. When omitted, a finite default is derived once from the per-workspace cap and startup/restored workspace count. `0` / `Infinity` means unlimited. |
|
||||
| `--max-pending-prompts-per-session <n>` | number | `5` | Accepted but pending/running prompt cap per session. Excess prompt returns 503. `0` / `Infinity` means unlimited; negative or non-integer values throw. |
|
||||
| `--max-connections <n>` | number | `256` | HTTP listener `server.maxConnections`; `0` / `Infinity` means unlimited. |
|
||||
| `--enable-session-shell` | boolean | `false` | Enables direct `POST /session/:id/shell` execution. Requires bearer token, and every call must carry a session-bound `X-Qwen-Client-Id`. |
|
||||
| `--event-ring-size <n>` | number | `8000` | Per-session SSE replay ring; soft cap is `1_000_000`. |
|
||||
| `--compacted-replay-max-bytes <n>` | positive integer | `4194304` | Byte cap for the bounded in-memory replay snapshot returned by `POST /session/:id/load`; hard cap is `268435456`. |
|
||||
| `--memory-budget-mb <n>` | integer in `[1024, 1048576]` | 50% of cgroup-constrained or host memory, capped at the flag maximum (1048576 MB) | Total memory budget for the daemon process tree, capped at resolved available memory. Observed and reported under `limits.memory` in daemon status; it does not size any child process. Boot rejects out-of-range values. |
|
||||
| `--memory-pressure-mode <mode>` | `off` \| `observe` | `observe` | Whether the daemon derives a memory-pressure level from its own RSS and V8 heap. Both modes report `runtime.memory.pressure`; only `observe` raises `daemon_memory_pressure`. Root process only; no remediation. |
|
||||
| `--child-heap-mode <mode>` | `off` \| `observe` | `observe` | Whether the daemon models a per-child heap partition of the budget. `observe` reports it and counts spawns past it; nothing is applied. `off` publishes no partition at all — `maxConcurrentChildren` and `perChildCeilingMb` are both `null`. |
|
||||
| `--http-bridge` | boolean | `true` | Stage 1 bridge mode. `--no-http-bridge` still falls back to http-bridge and prints to stderr. |
|
||||
| `--mcp-client-budget <n>` | positive integer | unset | Sets `WorkspaceMcpBudget.clientBudget` and forwards it to the ACP child through `childEnvOverrides`. |
|
||||
| `--mcp-budget-mode <m>` | `off` / `warn` / `enforce` | `warn` when budget is set, otherwise `off` | Sets `WorkspaceMcpBudget.mode`; `enforce` requires `--mcp-client-budget`. |
|
||||
| `--external-tool-guard-mode <m>` | `off` / `required` | `off` | Enables the managed ACP external pre-execution Guard. `required` fails startup unless its loopback provider completes the v1 handshake. |
|
||||
| `--external-tool-guard-endpoint <url>` | loopback HTTP(S) origin | unset | Provider origin used only in `required` mode. It must be origin-only and use `127.0.0.1`, `localhost`, or `::1`; paths, credentials, redirects, and proxy routing are rejected. |
|
||||
| `--external-tool-guard-timeout-ms <n>` | integer `100..30000` | `3000` | Per-handshake and per-prepare deadline. A timeout fails startup during the handshake or fails the invocation closed during a turn. |
|
||||
| `--allow-origin <pattern>` | repeatable string | unset | Cross-origin allowlist that replaces the default CORS denial. `*` allows any origin but requires a token. |
|
||||
| `--allow-private-auth-base-url` | boolean | `false` | Allows `/workspace/auth/provider` to install localhost / private-network auth provider `baseUrl`; use only in trusted local development. |
|
||||
| `--prompt-deadline-ms <n>` | positive integer | unset | Server-side prompt wallclock limit in ms. Timeout aborts and returns an error. |
|
||||
| `--writer-idle-timeout-ms <n>` | positive integer | unset | Per-SSE-connection idle timeout in ms. The daemon closes the SSE connection when no event is sent for this duration. |
|
||||
| `--channel-idle-timeout-ms <n>` | non-negative integer | `0` | How long to keep the ACP child alive after the last session closes. `0` means reclaim immediately. |
|
||||
| `--initialize-timeout-ms <n>` | positive integer | `10000` | ACP child request timeout, including the initialize handshake (ms). |
|
||||
| `--session-reap-interval-ms <n>` | non-negative integer | `60000` | Session reaper scan interval; `0` disables it. |
|
||||
| `--session-idle-timeout-ms <n>` | non-negative integer | `1800000` | Disconnected-session idle reaping time; `0` disables it. |
|
||||
| `--rate-limit` / `--no-rate-limit` | boolean | env / off | Enables per-tier HTTP rate limiting for prompt, mutation, and read routes. |
|
||||
| `--rate-limit-prompt <n>` | positive integer | `10` | Prompt request limit per window; requires rate limiting to be enabled. |
|
||||
| `--rate-limit-mutation <n>` | positive integer | `30` | Mutation request limit per window; requires rate limiting to be enabled. |
|
||||
| `--rate-limit-read <n>` | positive integer | `120` | Read request limit per window; requires rate limiting to be enabled. |
|
||||
| `--rate-limit-window-ms <n>` | integer `>= 1000` | `60000` | Rate limit window length; requires rate limiting to be enabled. |
|
||||
| no flag | - | - | `QWEN_SERVE_NO_MCP_POOL=1` fully disables the pool. |
|
||||
|
||||
## Environment variables
|
||||
|
||||
|
|
|
|||
|
|
@ -145,6 +145,46 @@ New OTel metric names:
|
|||
- `qwen-code.daemon.prompt.queue_wait`, histogram in milliseconds.
|
||||
- `qwen-code.daemon.pipe.message_bytes`, histogram in bytes with `direction=inbound|outbound`.
|
||||
|
||||
### 11. Is the daemon under memory pressure?
|
||||
|
||||
```bash
|
||||
curl -s 'http://127.0.0.1:4170/daemon/status' | \
|
||||
jq '.runtime.memory.pressure'
|
||||
```
|
||||
|
||||
`level` is `normal` / `soft` / `hard` / `critical`, classified from `ratio` —
|
||||
the worse of `rssRatio` (RSS against detected cgroup/host memory, which is what
|
||||
the OOM killer watches) and `heapRatio` (V8 heap used against this process's
|
||||
`heap_size_limit` — the whole heap, not only the old space that
|
||||
`--max-old-space-size` names). `source` says which one produced it. Check `source` before acting:
|
||||
`unknown` means the daemon could measure neither side, so `normal` there is the
|
||||
absence of a reading, not evidence of health. A side is only reported when both
|
||||
its numerator and its denominator were usable, so `source` is also what tells a
|
||||
zero `rssBytes` / `heapUsedBytes` apart from a real one.
|
||||
|
||||
**`rssRatio` is only as good as its denominator, and
|
||||
`limits.memory.availableMemorySource` is what grades it.** Under a cgroup
|
||||
(`constrained`) it is exactly the limit the OOM killer enforces, so the ratio
|
||||
means what it says. On bare metal (`host`) it is the size of the whole machine,
|
||||
while the daemon actually dies when the _machine_ runs out — which depends on
|
||||
every other process on the box. A daemon holding 20% of a 64 GB host beside a
|
||||
55 GB neighbour reports `level: normal, source: rss` right up until it is
|
||||
killed. Under `source: 'host'`, read `rssRatio` as a **lower bound** on real
|
||||
pressure. This is separate from the thresholds being uncalibrated: no threshold
|
||||
choice fixes a denominator that is measuring the wrong thing.
|
||||
|
||||
Two further things this does **not** cover. It is the daemon **root** process only, so
|
||||
a daemon whose `qwen --acp` children are the ones growing can report `normal`
|
||||
throughout — read `runtime.memory.children` beside it, which sums the live
|
||||
children's own RSS (and says via `sampled` how many actually reported).
|
||||
And nothing remediates: leaving `normal` raises a `daemon_memory_pressure`
|
||||
warning and changes no behaviour.
|
||||
|
||||
Under `--memory-pressure-mode off` every figure above is still reported and the
|
||||
issue is not raised, so the top-level `status` stays whatever it would have
|
||||
been. Use `off` while calibrating thresholds against a real workload, or if you
|
||||
alert on `status` and do not want an uncalibrated signal moving it.
|
||||
|
||||
## Flow
|
||||
|
||||
### Typical triage flow
|
||||
|
|
|
|||
|
|
@ -80,7 +80,9 @@ The CLI is defined in **`packages/cli/src/commands/serve.ts`**:
|
|||
| `--token <s>` | string | env / none | Non-loopback and `--require-auth` | Bearer token; trimmed once. **It appears in `/proc/<pid>/cmdline`, so prefer `QWEN_SERVER_TOKEN`**. Boot stderr also warns about this. |
|
||||
| `--max-sessions <n>` | number | `32` | - | Per-workspace active session cap. Excess spawn returns 503. `0` means unlimited. `NaN` / negative values throw. |
|
||||
| `--max-total-sessions <n>` | number | derived for multiple startup/restored workspaces | - | Daemon-wide active session cap. When omitted, a finite default is derived once from the per-workspace cap and startup/restored workspace count; dynamic registration does not recompute it. `0` means unlimited. |
|
||||
| `--memory-budget-mb <n>` | integer in `[1024, 1048576]` | 50% of cgroup/host memory | Observation only | Total memory budget for the daemon process tree, capped at resolved available memory. Reported under `limits.memory`; does not size any child. |
|
||||
| `--memory-budget-mb <n>` | integer in `[1024, 1048576]` | 50% of cgroup/host memory | Observation only | Total memory budget for the daemon process tree, capped at resolved available memory. Reported under `limits.memory`; modeled into a partition that nothing applies. |
|
||||
| `--memory-pressure-mode <mode>` | `off` \| `observe` | `observe` | Observation only | Reports `runtime.memory.pressure` in both modes; only `observe` raises the `daemon_memory_pressure` issue. Root process only. |
|
||||
| `--child-heap-mode <mode>` | `off` \| `observe` | `observe` | Observation only | Under `observe`, reports the modeled partition under `limits.memory.childHeap`; applies nothing and refuses nothing. Under `off`, that block's two figures are `null`. |
|
||||
| `--max-pending-prompts-per-session <n>` | number | `5` | - | Accepted but pending/running prompt cap per session. Excess prompt returns 503. `0` / `Infinity` means unlimited. Negative or non-integer values throw. |
|
||||
| `--workspace <dir>` | string / repeatable | `process.cwd()` | - | Startup workspace runtime; repeat to register additional isolated runtimes. The first is primary. Each value **must be an absolute path, must exist, and must be a directory**. Boot canonicalizes every value via `canonicalizeWorkspace`. `POST /session` with a mismatched `cwd` returns `400 workspace_mismatch`. |
|
||||
| `--max-connections <n>` | number | `256` | - | Listener-level `server.maxConnections`. `0` / `Infinity` means unlimited. `NaN` / negative values fail boot to avoid fail-open behavior. |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue