4 commits

Author	SHA1	Message	Date
tanzhenxin	2786afb2b5	fix(core): compact on the window ceiling, not the max of the threshold ladder (#6583) ... * fix(core): compact on the window ceiling (min), not the max of the ladder computeThresholds combined the proportional term (pct*window) and the absolute term (effectiveWindow - AUTOCOMPACT_BUFFER) with Math.max, which pushed the auto-compaction trigger toward the top of the window on large windows — a 1M-token window compacted at ~97%, leaving ~33K headroom. The absolute term is structurally a ceiling ("compact before the prompt leaves too little room for the summarization side-query, which needs up to SUMMARY_RESERVE of output"), so it composes with Math.min, matching the claude-code reference (services/compact/autoCompact.ts, which uses Math.min and whose default trigger is the absolute term alone). auto = absoluteCeiling > 0 ? min(pct*window, absoluteCeiling) : pct*window warn = max(0, auto - WARN_BUFFER) // WARN_PCT_OFFSET retired hard = unchanged Effect: large windows compact at ~85% (the DEFAULT_PCT ceiling) instead of ~97%; small/mid windows keep room to run compaction (a 128K window's summary now provably fits); sub-33K windows are unchanged. A lower context.autoCompactThreshold now pulls compaction earlier on large windows, matching the reference's Math.min override semantics. Updates the threshold unit tests, the settings schema description, and the user docs to describe the setting as a ceiling on the trigger. * refactor(core): trim threshold doc comments; name the hard-edge term Post-review cleanup (no behavior change): - Collapse the duplicated regime explanation shared between the DEFAULT_PCT and computeThresholds doc comments into one canonical block; point the constant's doc at computeThresholds. - Rename rawHard -> hardEdge and note it is the window-edge ceiling, so the two roles of the hard tier (window edge vs. auto + HARD_BUFFER) are legible. - Shorten the context.autoCompactThreshold description in settings.md to the concise schema wording (also un-widens the docs table).	2026-07-09 19:25:21 +08:00
顾盼	a8a6ad2d06	feat(core)!: redesign auto-compaction thresholds with three-tier ladder (#4345) ... * feat(core)!: redesign auto-compaction thresholds with three-tier ladder Replaces the single 70% proportional threshold with a three-tier ladder (warn/auto/hard) that combines proportional fallback with absolute reservation. Large-window models (>=128K) now reserve ~33K instead of 30% of the window, freeing tens of thousands of context tokens that the old formula wasted. Other improvements bundled in the same redesign: - Compression sideQuery now disables thinking and caps maxOutputTokens at 20K, matching claude-code so the buffer math is predictable across providers (Anthropic/OpenAI/Gemini handle thinking budgets inconsistently) - Failure handling upgraded from one-shot permanent lock to a 3-strike circuit breaker; reactive overflow still latches immediately - New estimatePromptTokens helper closes the lag-by-one-turn and first-send-is-0 gaps in lastPromptTokenCount - Hard-tier rescue pulls reactive overflow recovery forward to before the API call, saving an oversized round-trip - /context command displays the three-tier ladder + current tier - tipRegistry's context-* tips track the new thresholds instead of fixed 50/80/95 percentages BREAKING CHANGE: chatCompression.contextPercentageThreshold setting is removed. Settings files containing the field log a one-line deprecation warning at startup and the value is ignored; behaviour is now controlled by built-in thresholds via the new computeThresholds() function. Design: docs/design/auto-compaction-threshold-redesign.md Plan: docs/plans/2026-05-14-auto-compaction-threshold-redesign.md * test(core): fix leftover hasFailedCompressionAttempt option in compress test A pre-existing test case at chatCompressionService.test.ts:678 still passed `hasFailedCompressionAttempt: false` in the CompressOptions shape; rebasing onto current main surfaced this as a typecheck error because the field was renamed to `consecutiveFailures` (Task 7 of the three-tier ladder migration). Update to `consecutiveFailures: 0` — semantically equivalent, the test asserts the side-query is called when `force: true`, no other behaviour change. * fix(core): drop compaction summary when output hits maxOutputTokens cap Adds a defensive guard in ChatCompressionService.compress() that detects when the side-query summary hit COMPACT_MAX_OUTPUT_TOKENS (20K). In that case the summary is likely truncated mid-content, so we drop it and return NOOP rather than persist a half-summary. The next send re-tries; reactive overflow still catches the catastrophic case where the API rejects the next request as too large. Documented in the design doc as risk #2; the bot reviewer on PR #4168 correctly pushed for it to land alongside the threshold redesign rather than as a follow-up since the new 20K cap is what makes truncation likely in the first place. * fix(cli): render three-tier thresholds in /context TUI view The Task 11 redesign updated the non-interactive text formatter (formatContextUsageText) but left ContextUsage.tsx — the interactive React component that real /context users see — unchanged. As a result the TUI still showed the old single "Autocompact buffer" line and none of the new warn/auto/hard ladder. Adds a "Compaction thresholds" section after the per-category breakdown: - Effective window - Warn / Auto / Hard threshold rows with a ▶ marker on the row the current usage has crossed - Current tier label coloured by severity (safe→green, warn/auto→ yellow, hard→red) The existing progress bar legend (Used / Free / Autocompact buffer) is preserved because it's tied to the three-segment progress bar visualisation; the new section adds the absolute numbers + tier badge on top of that. Caught by the tmux e2e test (PR #4168 ci-monitor follow-up). Pre-fix the assertion 'Compaction thresholds' missed completely from the TUI; post-fix the new section renders correctly for fresh and live sessions on 1M / 200K / 128K windows. * fix(core,cli): address PR #4168 review batch 4 Behavior fixes: - MAX_TOKENS truncation guard now returns COMPRESSION_FAILED_EMPTY_SUMMARY instead of NOOP so the consecutive-failure breaker actually trips after repeated max-length summaries (R1.1). - Reactive overflow failure increments consecutiveFailures by 1 instead of latching to MAX in one shot, so a transient network blip doesn't permanently disable auto-compaction. The hard-tier rescue resets the counter, which remains the designated recovery path (R1.2). - /context current-tier classification uses rawOverhead (system + tools + memory + skills) as the tier input when API data is not yet available, rather than 0 — large inherited contexts no longer silently show 'safe' (R2.2). Performance: - sendMessageStream computes effectiveTokens ONCE and passes it through TryCompressOptions.precomputedEffectiveTokens, so the cheap-gate inside service.compress doesn't redo the estimation. Also fixes the imageTokenEstimate inconsistency between the rescue and cheap-gate paths (R1.3 + R1.4). - Steady-state path (lastPromptTokenCount > 0) skips the costly getHistory(true) clone — estimatePromptTokens only needs the user message in that branch. Code hygiene: - BYTES_PER_TOKEN → CHARS_PER_TOKEN (inputs are char counts, not byte counts; CJK text would mislead under the old name) (R3.1). - Drop dead getContextUsagePercent helper + index re-export — no callers in source after the threshold rewire (R1.5). - Add a comment on estimatePromptTokens' first-send fallback documenting the ~15-20K under-estimate (system prompt + tools + skills) and that reactive overflow is the safety net (R3.3). Tests: - New CLI ContextUsage.test.tsx exercises the React renderer for the three-tier section: section presence, ▶ marker placement per tier, current-tier label coloring (R1.6). - New chatCompressionService.test.ts case pins that a stale contextPercentageThreshold: 0 value in user settings no longer short-circuits compaction (R2.1). - New tokenEstimation.test.ts case covers functionResponse (distinct nested-parts branch from functionCall) (R3.5). - New geminiChat.test.ts integration test exercises the real ChatCompressionService — not a mock — for the first-send-after- inherited-history scenario where lastPromptTokenCount=0 and only the full-history estimate can cross the auto threshold (R3.4). Declined: R3.2 (change `>=` to `>` on the MAX_TOKENS guard). The current operator catches the at-cap case as suspicious, which is intentional — landing exactly at the output cap is far more likely truncation than clean stop given p99.99 ≈ 17K. With R1.1 in place, persistent truncations trip the breaker after MAX_CONSECUTIVE_FAILURES so the worst case is bounded. * fix(core,cli): address PR #4168 review batch 5 - R5.1: tighten /context tier comment + TODO. The rawOverhead-based fix doesn't cover `--continue` restores with many history messages (since rawOverhead excludes messagesTokens). UI may still show 'safe' for one render until the first send. Documented inline and added a TODO to plumb chat history into collectContextData for same-source-of-truth as the cheap-gate. - R5.2a: add TODO(finish_reason) at the truncation guard. The `>= cap` heuristic false-positives on legitimate at-cap summaries; the proper signal is finish_reason which runSideQuery doesn't surface today. - R5.2b: split telemetry — new CompressionStatus.COMPRESSION_FAILED_OUTPUT_TRUNCATED enum value. Distinct from EMPTY_SUMMARY so logs/telemetry can tell prompt-quality failures (tune prompt / splitter) from capacity failures (raise cap / shrink splitter input). isCompressionFailureStatus() treats both as failures so the breaker behavior is unchanged. - R5.3: expand consecutiveFailures JSDoc to clarify it tracks "non-force, non-hard-rescue consecutive failures" — hard-rescue resets the counter and force=true skips increments, so the counter is the "regular path" health signal only; reactive overflow is the real safety net for the force-only paths. - R5.4: document the CompressOptions field rename (hasFailedCompressionAttempt: boolean → consecutiveFailures: number) as an SDK breaking change in the design doc with migration guide. * fix(core): disambiguate hard-rescue from manual /compress orphan-strip Self-review (dual reviewer / pr-triage round 1) caught a correctness regression in the hard-rescue path: `sendMessageStream` calls `tryCompress(force=true)` from inside the pre-push window when `effectiveTokens >= hard`. The service's orphan-strip predicate at `chatCompressionService.ts:426-429` gated on `force` alone, which conflated two distinct call shapes: - manual `/compress` (force=true, trigger='manual'): user-initiated between turns; trailing model funcCall IS orphaned because no funcResponse is coming - hard-rescue (force=true, trigger='auto'): automatic mid-turn; trailing model funcCall is ACTIVE because its matching funcResponse is sitting in the pending `userContent` waiting to be pushed The strip fired for both, so a hard-rescue triggered mid tool-use loop would drop the active funcCall. After compression returned and `userContent` (the funcResponse) was pushed, the next API request carried tool_result with no matching tool_use → provider validation error. The in-code comment at L422-424 already documented this exact constraint for the auto-compress case (`force=false`), but reusing `force=true` for hard-rescue silently violated the same constraint. Fix: - Gate `hasOrphanedFuncCall` on `compactTrigger === 'manual'` instead of `force`. The trigger field already disambiguates intent. - `sendMessageStream` hard-rescue now passes `trigger: 'auto'` explicitly (without it, `force=true` defaults to `trigger='manual'` via the `?? (force ? 'manual' : 'auto')` resolver). Sibling audit for "force=true non-manual callsites": - `GeminiClient.tryCompressChat` (manual /compress): correct — manual - `sendMessageStream` hard-rescue: fixed in this commit - `sendMessageStream` reactive overflow catch: already passes trigger='auto'; runs AFTER API call (userContent in history), so if it observes a trailing funcCall it IS orphaned but findCompressSplitPoint handles the case without needing the strip RED-first regression test added: `preserves trailing model+funcCall under hard-rescue (force=true + trigger=auto)` in `chatCompressionService.test.ts`. Failed against pre-fix code (the strip dropped the funcCall); passes against the fix. Adjacent fixes from the same triage round: - `docs/users/configuration/settings.md`: the `chatCompression.contextPercentageThreshold` row still said "use 0 to disable compression entirely" — code has ignored the value since the removal commit. Marked the row REMOVED with migration guidance pointing at the design doc. - `packages/core/src/config/config.ts`: the deprecation warning now tells users how to silence it (remove the key) and where to read current behavior, instead of just announcing the removal. - `docs/design/auto-compaction-threshold-redesign.md`: closed Open Question 2 (small-window hard/auto collapse) — decision is to NOT annotate `/context`, with rationale on file. Tests: 2395 core tests passing, typecheck clean. * docs(core): fix tier-collapse direction in auto-compaction design doc Self-review on the 50bac974b commit caught a direction error in the M2a Open Question 2 closure note: said `currentTier` skips `'hard'` and goes to `'auto'` on collapsed windows, which is backwards. `contextCommand.ts:43-44` checks `tokens >= thresholds.hard` first (no `hard > auto` guard — that fix lives in a separate follow-up), so when `hard === auto` the `'hard'` branch matches first and the `'auto'` band is the empty one. Updated the rationale to describe the actual collapse direction and cite the source-of-truth file:line. Conclusion of the open question (don't annotate `/context`) is unchanged — only the explanation is corrected. * refactor(core): extract shared in-flight funcCall fixture in compression tests The auto-compress and hard-rescue tests for "trailing funcCall is active, not orphaned" shared a byte-identical 4-message history and mock setup. Pull both into setupInFlightFuncCallFixture() inside the describe block so each test only contains the scenario name, the compress() call shape, and its own assertions. Net -29 LOC, no behavior change. * fix(core,cli): address PR #4345 round-2 review feedback - geminiChat: remove pre-call consecutiveFailures reset in hard-rescue. force=true already bypasses the breaker check in chatCompressionService; the pre-reset was redundant on success (post-call L614 already handles it) and *broke* the breaker on failure paths — hard-rescue failures don't increment via tryCompress (force=true skips that branch), only the reactive overflow path at L992 explicitly increments. With the pre-reset the counter oscillated 0↔1 every send and MAX_CONSECUTIVE_FAILURES=3 was unreachable. Wrote a RED test asserting the forwarded counter is the latched value, not zero; the test failed against the old code and passes with the reset removed. - geminiChat: log hard-tier-rescue triggers via debugLogger.warn including effectiveTokens, hard, and the current consecutiveFailures so operators debugging "compaction stopped working" have a breadcrumb. - chatCompressionService: clamp effectiveWindow to >= 0 in computeThresholds so the value surfaced in /context stays meaningful for tiny windows (window < SUMMARY_RESERVE). auto/warn/hard outputs are unaffected because each is Math.max(proportional, absolute) and the proportional branch dominates whenever the absolute branch goes negative. - turn.ts: rewrite COMPRESSION_FAILED_OUTPUT_TRUNCATED docstring. Drop the misleading "compression succeeded" framing (the summary is dropped and isCompressionFailureStatus returns true) and reference the full enum name COMPRESSION_FAILED_EMPTY_SUMMARY instead of the abbreviation. - contextCommand.test.ts: reword the no-API-data-session test comment. collectContextData classifies estimated sessions against rawOverhead; with default fixtures rawOverhead lands in `safe`, but heavy system-prompt / skill / MCP loads can push it into warn/auto/hard. - design doc Background: prepend a blockquote clarifying the section describes pre-redesign behavior and that the inline file:line references point at code before PR #4345 (which removes them). - ui/types: replace the duplicated ContextThresholds interface with a type alias to the core's CompactionThresholds. Field-by-field copy in contextCommand.ts becomes a direct spread. ContextUsage.tsx keeps its CompactionThresholds React component name — the alias avoids the collision a direct import would have caused. - contextCommand: interpolate the actual reserve value into the "(window − 20K reserve)" annotation so SUMMARY_RESERVE retuning doesn't leave the text stale. * fix(core): address PR #4345 round-3 + round-4 review feedback R3-1: rewrite the stale "Hard-tier rescue resets the counter" comment in the reactive-overflow path. The R2 commit removed the pre-call reset from hard-rescue; the only counter-reset path is now the post-call COMPRESSED branch in tryCompress. Two contradicting comments in the same file would mislead a future maintainer tracing the lifecycle. R3-2: rewrite the JSDoc on CompactionThresholds.hard. The "(resets failure counter)" phrasing was true under the pre-R2 design; after R2 the hard threshold force-triggers compaction and bypasses the breaker, but does not reset the counter (which only happens on COMPRESSED success via the post-call branch). The type is consumed by both geminiChat and the CLI UI (via ContextThresholds alias), so the authoritative description had to match the actual contract. R3-3: add a Step 3 to the hard-rescue regression test. The test title claims "success recovers via the post-call branch" but the original Steps 1-2 only verified the latched counter was forwarded INTO the call. Step 3 follows up with a below-hard send and asserts the forwarded counter is 0 — proving geminiChat.ts:614 ran on the COMPRESSED result. R3-4: assert effectiveWindow === 0 on the existing extreme-small-window test and add a separate zero-window edge case. The Math.max(0, ...) clamp from R2 was previously unasserted; a regression that removed the clamp would go undetected. R4-1: forward originalTokenCount on the breaker-NOOP path in chatCompressionService.compress() to match the adjacent threshold-NOOP path (L368-369). Returning {originalTokenCount: 0, newTokenCount: 0} masked "breaker tripped at N tokens" as "empty session" in telemetry dashboards. R4-2a: add debugLogger.warn at the two consecutiveFailures increment sites (cheap-gate path L586 and reactive-overflow path L955) when the counter reaches MAX_CONSECUTIVE_FAILURES. The breaker is one of the PR's headline safety features but, prior to this round, had zero observability when it tripped. Required importing MAX_CONSECUTIVE_FAILURES into geminiChat.ts. R4-3: programmatically link tokenEstimation.ts's CHARS_PER_TOKEN to compactionInputSlimming.ts's TOKEN_TO_CHAR_RATIO. Both are 4 today and represent the same generic char/token conversion. Exporting from compactionInputSlimming and aliasing in tokenEstimation eliminates the silent-drift hazard the JSDoc already warned about. Declined (round-weighted bar at round 4): - R3-5: debugLogger test for hard-rescue trigger — observability test coverage is overthinking at round 3+; the log is informational. - R4-2b: expose breaker state in /context — new feature; out of scope. - R4-4: render test for auto-tier marker — test coverage gap on working code, defer to follow-up PR per round-weighted bar. - R4-5a: extract makeFakeChat/makeFakeConfig shared factory — pure test refactor at round 4, not a fix. - R4-5b: direct unit test for precomputedEffectiveTokens — exercised indirectly via hard-rescue path tests in geminiChat.test.ts. - R4-6: truncation-guard fallback test for missing candidatesTokenCount — code already has a TODO acknowledging the heuristic is imperfect (chatCompressionService.ts:549-553); defer. * fix(core): address PR #4345 round-5 review feedback R5-1: assert breaker-NOOP forwards originalTokenCount. R4-1 changed the breaker-NOOP return from `{0, 0}` to `{originalTokenCount, originalTokenCount}` so telemetry can distinguish "breaker tripped at N tokens" from "empty session", but the existing test only checked compressionStatus and newHistory. Now seeds a non-zero originalTokenCount (120K) and asserts both fields forward it. R5-2: forward originalTokenCount on the empty-history NOOP. This was sibling drift on R4-1 — I fixed the cited breaker-NOOP site but missed the empty-history NOOP. Of 5 NOOP return sites in chatCompressionService, 4 now forward originalTokenCount (breaker, threshold-gate, post-split, min-compression-fraction) and 1 (this one) was still returning `{0, 0}`, breaking the project-wide invariant. Now consistent. R5-3: replace 10 stale line-number references with semantic anchors. After the R3+R4 push, the line refs in my R2/R3 comments (`geminiChat.ts:614`, `chatCompressionService.ts:339`, `line 992`, `L627`, `line 944`) no longer pointed at their original targets — `geminiChat.ts:614` now points at `setSystemInstruction`'s body, completely unrelated to compaction. The pattern itself is fragile; semantic phrasing ("the post-call reset in tryCompress's COMPRESSED handler") doesn't drift when lines shift. 347/347 affected core tests passing locally; typecheck clean. * fix(core): address PR #4345 round-6 review feedback (R6 sweep) R6-1: rewrite the stale JSDoc bullet on `consecutiveFailures` (the "Hard-tier rescue failures" bullet). The old wording said "the counter is reset to 0 BEFORE the rescue call" — that contradicted R5 which explicitly removed the pre-call reset. Now the bullet matches the actual behavior: counter is NOT pre-reset, force=true bypasses the breaker, post-call COMPRESSED handler resets on success, reactive overflow is the explicit-increment safety net. My R5 stale-comment sweep only grep'd inline `//` comments; this JSDoc on the field declaration slipped through. Re-audited "reset to 0 BEFORE" / "pre-reset" across both packages — single site remaining. R6-7: assert `passedOpts.trigger === 'auto'` in the hard-rescue test. This field is the orphan-strip safety wire added by the C1 fix (the service's `compactTrigger === 'manual'` check would otherwise strip the trailing active funcCall mid tool-loop). The test asserted force and pendingUserMessage but not the trigger; a refactor dropping the 'auto' from `trigger: shouldForceFromHard ? 'auto' : undefined` would silently break orphan-strip safety. Now regression-guarded with a single-line expect. 164/164 affected core tests passing locally. Declined per round-weighted bar (round 6 defaults Suggestion / Test coverage / Style to overthinking): - R6-2/3/6: test-coverage gaps on working code — defer to follow-up - R6-4: redundant truthy guard on always-set fields — style nit - R6-5: text-vs-UI inconsistency on /context — existing test enforces current behavior; treat as design decision (offer follow-up if reviewer escalates) - R6-8 (tipRegistry small-window context-high): explicitly closed in design doc's Open Question 2 — small windows have empty context-high band by design; UI work is out-of-scope for this PR - R6-9: wasted clone on rare fallback path — Suggestion-level perf - R6-10 (CompressionMessage missing case): file not in this PR's diff; reviewer themselves proposed it as follow-up	2026-05-25 21:11:08 +08:00
Shaojin Wen	b3bc42931e	feat: add contextual tips system with post-response context awareness (#2904) ... * feat: add contextual tips system with post-response context awareness Add a context-aware tips system that proactively shows helpful tips based on session state. Post-response tips warn when context usage exceeds 80% or 95%, suggesting /compress. Startup tips rotate across sessions via LRU scheduling with cross-session persistence (~/.qwen/tip_history.json). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use value import for runtime values in useContextualTips Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback - Use lastSessionTimestamp instead of totalShown for cross-session LRU - Move getTipHistory singleton from Tips.tsx to services/tips/index.ts - Defer TipHistory.load() when hideTips is true (no side effects) - Use os.tmpdir() in tests for cross-platform portability - Add proper translations for de/ja/pt/ru locale files - Accept TipHistory | null in useContextualTips Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address Copilot review feedback - Validate tips field type in TipHistory.load() to handle corrupted JSON - Split approval-mode tip into platform-specific variants using ctx.platform - Add afterEach cleanup for temp files in all test suites - Guard useContextualTips against null tipHistory Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: import shared DEFAULT_TOKEN_LIMIT, harden tipHistory, set file permissions - Import DEFAULT_TOKEN_LIMIT from @qwen-code/qwen-code-core instead of hardcoding 1_048_576 in tipRegistry.ts and useContextualTips.ts - Add normalizeEntry() to defensively handle corrupted tip history entries - Write tip_history.json with mode 0o600 for privacy on multi-user systems Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused compressionThreshold from TipContext compressionThreshold was defined in TipContext but never used by any tip's isRelevant check. Remove it to avoid misleading consumers into thinking tips respect the user's compression settings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: sanitize sessionCount and getLastShown against corrupted tip history - Validate sessionCount is finite and non-negative in TipHistory.load() - Use normalizeEntry() in getLastShown() for corrupted lastSessionTimestamp Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add contextual tips user documentation Add docs/users/features/tips.md covering startup tips, post-response context warnings, tip history persistence, and the hideTips setting. Update settings.md description and register the new page in _meta.ts. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-13 17:40:27 +08:00
tanzhenxin	1b7d153a4c	feat(cli): Increase /insight feature exposure via weighted tips ... - Add weighted tip system to make certain tips appear more frequently - Set /insight tip to weight 3 (3x more likely than regular tips) - Add i18n translations for the new tip across all supported languages - Add comprehensive unit tests for weighted tip selection Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>	2026-02-28 18:08:37 +08:00