2505 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Ryker_Feng	5ba25b06ec	feat(mcp): add MCP routing hints (#4004) ... * feat: add MCP routing hints * test: isolate mcp routing prompt config * fix: address mcp routing review feedback	2026-07-09 16:26:31 +08:00
Zheng Feng	bc9ee9645c	feat(deploy): first-class Helm chart for Kubernetes deployment (#3987) ... * feat(helm): add production-ready Helm chart for Kubernetes deployment Adds deploy/helm/deer-flow, a native-Kubernetes translation of the production docker-compose stack, plus CI to publish its images and chart. * ci(release): gate releases on version-source consistency Add a reusable verify-versions workflow invoked by both chart.yaml and container.yaml on v* tags. It runs scripts/verify_versions.sh against the tag and fails the release — skipping all image and chart publishing — when Chart.yaml (version + appVersion), backend/pyproject.toml, or frontend/package.json don't all match the tag. Add scripts/verify_versions.sh (the check, also runnable locally) and scripts/bump_version.sh (bumps all four sources in lockstep, then self-verifies). Document the release flow in RELEASING.md and link it from AGENTS.md. * fix(deploy): address Helm chart review feedback (#3987) Three review items from willem-bd: 1. nginx IPv6 listen strip never matched. The sed pattern required a `;` immediately after `2026`, but the rendered config emits `listen [::]:2026 default_server;` (space + `default_server` before the `;`), so the line was never deleted and nginx crash-looped on pods without IPv6 (`socket() :::2026 failed (97: Address family not supported)`). Drop the trailing `;` from the pattern so it matches. Same latent bug fixed in docker-compose-dev.yaml. 2. Passwords were spliced into DSNs verbatim, so a password containing URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and a confusing parse error. Add a `deer-flow.urlEscape` helper (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral treats the replacement as a regex template so `[`/`]`/`?` break it) and apply it to the password in the postgres and redis DSNs. The raw `postgres-password` / `redis-password` keys stay unencoded - they back POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment. 3. NODE_HOST defaulted to "gateway", which can never route: the gateway Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a user who skips the caveat gets unreachable sandboxes with no error at install time. Default NODE_HOST to the provisioner pod's node IP via the downward API (status.hostIP) - a NodePort is exposed on every node, so <node-IP>:<NodePort> routes from the gateway on most clusters. `provisioner.nodeHost` remains an override for CNIs/policies that block pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart README. (#3929 remains the long-term fix - ClusterIP + cluster-DNS URL removes NODE_HOST and the NodePort exposure entirely.) Validated with helm lint, helm template (incl. a special-char password rendering the encoded DSNs), and a sed pattern-match check. * fix(deploy): address round-2 Helm chart review feedback (#3987) Three "Medium" items from willem-bd: 1. No helm lint / helm template gate before publish. A template regression ships as an immutable OCI artifact (GHCR won't overwrite --version), so gate packaging on `helm lint` + `helm template --include-crds` in chart.yaml before `helm package`. (ct lint / helm-unittest deferred.) 2. Action pinning inconsistent + PR body overstates it. SHA-pin actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance (v2.4.0, e8998f94) across the publishing workflows (chart.yaml, container.yaml, verify-versions.yml), matching the existing docker/* SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the "SHA-pinned actions" claim accurate. Other pre-existing workflows left untouched (out of scope for this PR). 3. Provisioner RBAC broader than needed. Dropped the unused update/patch verbs and the pods/exec + events rules from the provisioner Role - audited against docker/provisioner/app.py, which only calls get/create/delete on pods and get/list/create/delete on services. Fixed NOTES.txt to accurately describe the grant instead of understating it as "create Pods and Services". The remaining scope concern - verbs apply to all Pods in the namespace, not just sandbox Pods - is still deferred (RBAC can't scope by label; needs a dedicated namespace or admission control), now noted in NOTES.txt and README. Validated with helm lint + helm template (narrowed Role renders with exactly get/list/watch/create/delete). * feat(helm): enable sandbox+web tools out of the box The chart's default config loaded zero agent tools (config.tools empty -> "Total tools loaded: 0"), so a fresh install gave an agent that could do nothing useful. Add tool_groups + tools to the default config block: - web: web_search (ddg), web_fetch (jina), image_search - no API key - file:read: ls, read_file, glob, grep - file:write: write_file, str_replace - bash The file/bash tools run inside the AIO sandbox the chart already configures; the web tools need outbound internet from the gateway pod (swap backends or drop entries for air-gapped clusters - see config.example.yaml). Also bump config_version 15 -> 19 to match config.example.yaml (the chart had drifted behind). NOTES.txt and the README example updated to match. * ci(helm): add chart validation + config_version drift check on PR Extend the chart workflow with a PR-triggered validate-chart job that runs helm lint, helm template --include-crds, and a config_version drift check: it parses config_version from both config.example.yaml and the chart's values.yaml and fails the build (with a ::error:: naming the files to bump) if the chart is behind the example. This catches the kind of drift this PR is fixing - the chart sat at v15 while the example moved to v19 - before it can merge again. verify-versions and publish-chart stay tag-only; publish-chart now needs: [verify-versions, validate-chart]. validate-chart runs on both PRs and tag pushes: the tag arm is required because a job that `needs` a skipped job is itself skipped under the default success() check, so validate-chart must actually run on tag pushes or publish-chart would never fire. * Bump config version to 20	2026-07-09 15:40:53 +08:00
Yi Deng	b36d7194d9	fix(security): neutralize prompt-injection tags in remote tool results (#4002) ... * fix(security): neutralize prompt-injection tags in remote tool results User input is already neutralized for framework/injection tags, but tool results are not. Remote content fetched by web_fetch/web_search is equally untrusted and can carry a forged <system-reminder> block that reaches the model verbatim as authoritative context. Extract a shared neutralize_untrusted_tags() primitive from InputSanitizationMiddleware and apply it to remote-content tool results (web_fetch/web_search/image_search) via a new ToolResultSanitizationMiddleware. Local tool output (bash/read_file) is left untouched so legitimate code/file content is never mangled. * test: update subagent middleware count for tool-result sanitizer The new ToolResultSanitizationMiddleware adds one entry to the shared runtime chain (11 -> 12). Update the subagent count assertion, use a lazy import for neutralize_untrusted_tags so the module loads even when tests stub the input-sanitization module, and document the new middleware in AGENTS.md. * fix(security): address review — sanitize bare str list items; document MCP scope - Neutralize bare str elements inside a ToolMessage content list (previously only {type:text} dict blocks were rewritten), matching the str-in-list shape ToolOutputBudgetMiddleware._message_text already anticipates. - Document the name-based allowlist limitation: MCP remote-content tools registered under arbitrary names (e.g. fetch_url) are not covered; a name heuristic is avoided to prevent mangling local tool output, with metadata tagging tracked as a follow-up. Add a regression test pinning this boundary.	2026-07-09 14:45:25 +08:00
hataa	c9fb9768d4	fix(subagents): unify guardrail caps on additive stop_reason + add token_budget (#3875 Phase 2) (#3980) ... Phase 2 of #3875. Two guardrail axes can end a subagent run early — the turn budget (GraphRecursionError) and the token budget (TokenBudgetMiddleware) — and both now surface *why* through one additive `subagent_stop_reason` field instead of a status enum. This completes and course-corrects Phase 1 (#3949), which shipped the turn-budget cap as a `max_turns_reached` status enum. The agreed Phase 2 design replaces that enum with an optional `stop_reason` field (token_capped | turn_capped | loop_capped): a new enum value would break v1 consumers, while an additive field is ignored by older frontends and ledger readers. `max_turns_reached` and SubagentStatus.MAX_TURNS_REACHED are removed. - subagents.token_budget config (default enabled, 2,000,000 tokens, warn 0.7) with per-agent override; TokenBudgetMiddleware is now attached in build_subagent_runtime_middlewares so the cost-ceiling backstop engages for every subagent. The hard-stop does not raise — it strips tool_calls and lets the run finish with a final answer, recording the cap on a per-run consume_stop_reason() accessor. - executor.py: on normal completion it reads consume_stop_reason() and stamps completed + token_capped when the budget fired; on GraphRecursionError it recovers the last AIMessage partial (completed + turn_capped) or, if nothing usable survived, failed + turn_capped. SubagentResult gains stop_reason. - status_contract.py / contracts/subagent_status_contract.json (v2) / frontend subtask-result.ts: additive subagent_stop_reason field, pinned by test_status_values_match_contract / test_stop_reason_values_match_contract. - task_tool.py + delegation_ledger.py: drop the max_turns_reached paths; the ledger captures stop_reason and renders model-facing "capped" guidance so the lead reuses a capped completion knowingly. The 2,000,000-token default is deliberately loose (tighten to taste) — it would have roughly halved the reported 4.4M burn while leaving legitimate deep-research runs (max_turns=150) room. Subagent summarization is a follow-up.	2026-07-08 22:26:06 +08:00
Ryker_Feng	c640b52a7d	feat(frontend): render slash-skill activations as inline chips (#3981) ... * feat(frontend): render slash-skill activations as inline chips Show an explicit `/skill` activation as a compact inline chip in both the composer and the chat transcript instead of raw slash text. - Composer: selecting a skill suggestion stores it as a removable chip aligned inline with the textarea; the leading `/skill ` prefix is reattached only at submit time, so the backend activation protocol is unchanged. Backspace on an empty input or the chip's close button clears it; history navigation is disabled while a chip is active. - Transcript: human messages that begin with `/skill` render the skill as a read-only chip followed by the task text. - Add a shared `core/skills/slash.ts` (`parseSlashSkillReference` + `resolveSlashSkillDisplay`) mirroring the backend `slash.py` gate, so the transcript only shows a chip when the skill actually exists and is enabled. This removes a duplicated regex/reserved-name list and keeps display semantics consistent with backend activation. Add unit tests for the shared slash parser and extend the chat e2e to assert the composer still submits `/skill <task>` after showing a chip. * chore(frontend): format chat e2e test * refactor(skills): address slash-skill chip review feedback Follow-up to the inline slash-skill chip PR, resolving three second-order review findings: - Drive the reserved-command set and skill-name grammar from a shared contracts/slash_skill_contract.json instead of a hand-copied "keep in sync" pair. slash.ts and slash.py now reference the fixture, and contract tests on both sides fail CI if either drifts. - Extract a shared SlashSkillChip so the composer and transcript chips stay in lockstep, and normalize the off-scale /8 and /12 opacity steps to the standard /10 and /20 tokens. - Split HumanMessageText into a pure parse gate plus a slash-only subtree that owns the useSkills() lookup, so a skill-enabled toggle no longer re-renders every plain-text human turn. Verified: frontend eslint + tsc clean, pnpm test 572 pass (incl. new slash-contract test); backend slash contract + slash-skills tests 31 pass. * style(tests): sort slash skill contract imports * fix(composer): inline the slash-skill text so the chip aligns with input Address the "composer body layout change" review on #3981 by rendering the active skill as an inline chip in the same text flow as the prompt, rather than a separate flex row that drifted the box model across states. - Render the chip + prompt inside one leading-6 wrapper and edit the prompt through a `contentEditable` span, so the chip sits inline with the first line and long/multi-line input wraps naturally back to the container edge. - Align the chip with `align-top`: its h-6 (24px) height matches the text line height, so chip and first-line centers coincide exactly (measured delta 0), fixing the chip being raised above the baseline. - Restore the placeholder in chip mode via a `data-empty` CSS `::before`, which also gives the empty editable span width so it is no longer treated as hidden. - Widen the IME helper to `HTMLElement` and route the span's keydown/paste through the shared skill-suggestion, prompt-history, backspace-to-clear, and IME-composition handlers so contentEditable behaves like the textarea. - Extend chat.spec.ts to drive the inline skill editor instead of the textarea after a chip is shown. * style(frontend): fix composer class order formatting * fix(composer): break long unbroken input inside the slash-skill row The inline slash-skill editor wrapped with `break-words` (overflow-wrap: break-word), which only moves an over-long token to the next line before breaking it. A long unbroken string therefore started on the line below the chip, and when the string contained a break opportunity such as a hyphen the browser wrapped there and pushed the remaining run to the next line, leaving a wide gap on the right. Switch to `break-all` (word-break: break-all) so the text fills each line from the chip and packs tightly regardless of hyphens or CJK.	2026-07-08 21:58:33 +08:00
Ryker_Feng	01dc067997	feat: add composer input polishing (#3986) ... * feat: add composer input polishing * Revert "Merge branch 'main' into feat/input-polish" This reverts commit 5b6ceccf0db3092bc62fde3b05e7816829601756, reversing changes made to 45fbc57fef5fa5fd878cf0176c37f3e3bc7ebef6. * Merge main into feat/input-polish * style(frontend): format input helper polish guard * fix(input-polish): address composer polish review findings Frontend - Add a cancel affordance to the in-flight polish status pill that calls abortInputPolishRequest(), so a slow/hung provider no longer hard-locks the composer for up to stream_chunk_timeout with a page reload (and draft loss) as the only escape. - Reset promptHistoryIndexRef/promptHistoryDraftRef when a rewrite is applied (and on undo), so a stale history-browse index can no longer let the next ArrowDown silently overwrite the polished draft. - Disable polishing while an open human-input card is present, matching the frontend/AGENTS.md rule that composer entry points defer to the card so card-reply metadata is preserved. - canPolishInput now reuses parseGoalCommand/parseCompactCommand instead of a third hardcoded reserved-command regex, and drops the phantom /help entry (no /help parser exists in the composer), so future builtins only need to be taught to the existing parsers. Backend - Extract the non-graph one-shot LLM path (build model + inject Langfuse metadata + system/user invoke + text extract) into deerflow.utils.oneshot_llm.run_oneshot_llm, shared by the input-polish and suggestions routers so tracing-metadata and invocation shape cannot drift between the two copies. - strip_think_blocks gains truncate_unclosed (default True, preserving the suggestions/goal JSON-prep behavior); input polish passes False so a draft that legitimately contains a literal <think> substring is no longer truncated into a partial rewrite or a spurious 503. - Validate the empty-check and max_chars boundary against the same stripped view of the draft that is sent to the model, so the user-facing length boundary and the model input can no longer disagree. Tests / docs - Backend: literal-<think> preservation, whitespace-only rejection, and normalized-length/model-input agreement cases; suggestions tests repoint the create_chat_model patch to the shared helper module. - Frontend: helper unit tests updated for the /help/reserved-command change; a new Playwright case covers cancelling an in-flight polish request. - backend/AGENTS.md documents the shared one-shot helper and the polish normalization/think-tag behavior. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-08 17:10:27 +08:00
Vanzeren	e3137b13cf	fix(provisioner): gate legacy skills mount by user visibility (#3985) ... * fix(provisioner): gate legacy skills mount by user visibility * fix(test): aio sandbox provider * fix: use shared legacy skill visibility helper for sandbox mounts	2026-07-08 14:41:16 +08:00
Yufeng He	a4a88fda19	fix(mcp): synchronize session pool singleton lifecycle (#3797) ... * fix(mcp): synchronize session pool singleton lifecycle Parity follow-up to #3778 (skill storage) and #3730 (sandbox provider). get_session_pool() already serialised creation with _pool_lock, but its fast-path check and final return read the global separately, and reset_session_pool() cleared it with no lock. A reset_mcp_tools_cache() (reachable via the /api/mcp/cache/reset admin endpoint) racing a concurrent get could null the singleton between the None-check and the return, handing the caller None despite the -> MCPSessionPool annotation. Build and return the pool inside _pool_lock with a double check, and clear it under the same lock in reset_session_pool(). The critical section is tiny and never awaits, so holding the threading.Lock is safe from both the async and sync/worker-thread paths. No behavior change for single-threaded callers. Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-07 23:09:06 +08:00
AochenShen99	658c39ccf7	feat(skills): Add native SkillScan phase 1 for skills (#3033) ... * Add phase 1 skill static scanning * Rework SkillScan phase 1 as native scanner * refactor(skillscan): align phase 1 with trimmed RFC contract - SecurityFinding: 7 fields (rule_id, severity, file, line, message, remediation, evidence); category/analyzer derive from the rule_id prefix, confidence/column/fingerprint/metadata removed - scan_archive_preflight()/scan_skill_dir() are pure functions: no ScanContext, no policy schema; CRITICAL-blocks is a code constant and skill_scan.enabled is applied by enforce_static_scan()/callers - secret-* evidence is redacted before findings leave the scanner - de-dup keys on (rule_id, file, line) so repeated occurrences keep distinct locations for agent self-correction - cloud-metadata detection consolidated into network-cloud-metadata - nested zip members get a one-level stdlib magic-byte peek; an executable member escalates package-nested-archive to CRITICAL - install metadata sidecar removed (Phase 7 decides if it is needed) - rule specs moved next to their analyzers; skillscan/rules/ removed - tests updated + new anchors: redaction, dedup lines, nested-zip escalation, single cloud-metadata rule, bundled-skill zero-CRITICAL * fix(skillscan): tighten reverse-shell/secret/archive scan rules from review Address PR #3033 review feedback on the native SkillScan analyzers: - Reverse-shell false positives: split shell detection by signal strength (/dev/tcp/, nc -e stay CRITICAL; bash -i, mkfifo -> new HIGH shell-reverse-shell-heuristic, warn->LLM). The Python check is now AST-anchored on real socket.socket/os.dup2/subprocess call sites instead of raw-text substring matching, so prose/docstrings no longer hard-block. - Secret evidence: _redact_secret_evidence returns [redacted] with no secret bytes (was value[:6], which leaked 2 real token bytes past the prefix). - Archive DoS: cap outer archive member count (_MAX_ARCHIVE_MEMBERS=4096); scan_archive_preflight early-aborts with a package-too-many-members CRITICAL finding (routes through the existing blocked->400 fail-closed path). - shell-destructive-command: broaden the rm -rf matcher to sensitive system roots (/home, /usr, /*, --no-preserve-root /) while leaving safe subpaths unflagged. - Dead code: collapse _decode_text_for_analysis to a single decode path and drop the unused _TEXT_SUFFIXES set and _has_text_shebang helper. - local_skill_storage: document why the host_path branch keeps app_config possibly-None (lazy kill-switch resolution; avoids eager get_app_config in config-free environments such as CI). Tests: new negative/positive coverage in test_skillscan_native.py. Full backend suite 6616 passed, 26 skipped.	2026-07-07 21:44:28 +08:00
Xinmin Zeng	857fb96269	fix(sandbox): stop setup-sandbox from pre-pulling the stale :latest sandbox image (#3983) ... * fix(sandbox): stop setup-sandbox from pre-pulling the stale :latest image scripts/setup-sandbox.sh's fallback (used whenever config.yaml has no uncommented sandbox.image) pulled the volces mirror's :latest tag. We confirmed in #3921/#3922 that this tag is frozen on the pre-1.9.3 all-in-one-sandbox digest (1.0.0.156), which lacks the /v1/bash/* routes required-secrets skills need — so the one script whose entire job is 'pre-pull a working sandbox image' was pre-pulling a known-broken one. Pin the fallback to :1.11.0 instead. Also update config.example.yaml's commented image: example and 'Recommended' line to the same version, so uncommenting the example doesn't reproduce the same trap. Out of scope on purpose: aio_sandbox_provider.py's DEFAULT_IMAGE constant (the harness-level default for AioSandboxProvider itself) is a separate, broader default-image decision already flagged to maintainers in #3921 — this PR only fixes the pre-pull helper script. Reported in #3914 (a real user deleted their stale local image, reran make setup-sandbox, and got the same broken :latest image back). * fix(sandbox): make setup-sandbox warn when the pull won't affect the runtime image Self-review caught a real gap in the previous commit: AioSandboxProvider resolves its image as `sandbox_config.image or DEFAULT_IMAGE` (aio_sandbox_provider.py:214), and DEFAULT_IMAGE is deliberately left untouched (still the frozen :latest, per #3921 — that's a maintainer decision, not this PR's scope). So when config.yaml has no uncommented sandbox.image, pre-pulling :1.11.0 alone creates a NEW inconsistency: the script reports success pulling a modern image, but the sandbox that actually starts still falls back to the broken :latest — silently leaving the user's underlying required-secrets/bash.exec problem unfixed, which is worse than the previous consistent-but-broken behavior (pre-pull :latest, run :latest). Make the unconfigured path loud about this instead of silent: print the exact config.yaml snippet needed to make the pulled image actually take effect.	2026-07-07 21:05:28 +08:00
Ryker_Feng	26d7a5970d	feat: add manual context compaction (#3969) ... * feat: add manual context compaction * fix: harden manual context compaction	2026-07-07 19:55:33 +08:00
AnoobFeng	8fbcdf821b	Fix/card tool message bug (#3976) ... * fix(runtime): add final reconciliation for missed tool messages * fix(gateway): persist hidden human input card responses Persist allowlisted hidden human_input_response messages in RunJournal so Human Input Cards can recover answered state from run_events after checkpoint compaction. Keep generic internal hidden messages filtered and add regression coverage for ask_clarification responses.	2026-07-07 14:39:04 +08:00
dependabot[bot]	d2d91c1c78	build(deps): bump starlette from 1.0.1 to 1.3.1 in /backend (#3974) ... Bumps [starlette](https://github.com/Kludex/starlette) from 1.0.1 to 1.3.1. - [Release notes](https://github.com/Kludex/starlette/releases) - [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md) - [Commits](https://github.com/Kludex/starlette/compare/1.0.1...1.3.1) --- updated-dependencies: - dependency-name: starlette dependency-version: 1.3.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-07 13:49:02 +08:00
dependabot[bot]	4b25fe6d95	build(deps): bump aiohttp from 3.14.0 to 3.14.1 in /backend (#3973) ... --- updated-dependencies: - dependency-name: aiohttp dependency-version: 3.14.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-07 13:40:12 +08:00
dependabot[bot]	0808738c58	build(deps): bump langgraph-checkpoint from 4.0.2 to 4.1.1 in /backend (#3972) ... Bumps [langgraph-checkpoint](https://github.com/langchain-ai/langgraph) from 4.0.2 to 4.1.1. - [Release notes](https://github.com/langchain-ai/langgraph/releases) - [Commits](https://github.com/langchain-ai/langgraph/compare/checkpoint==4.0.2...checkpoint==4.1.1) --- updated-dependencies: - dependency-name: langgraph-checkpoint dependency-version: 4.1.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-07 09:25:21 +08:00
sqsge	927b833ef4	fix(guardrails): propagate internal owner attribution to guardrail context (#3839) ... * fix guardrail attribution for internal owner runs * fix guardrail owner attribution mismatch	2026-07-07 08:05:26 +08:00
Huixin615	4915b5e4cf	fix(frontend): enable regenerate in custom agent chats (#3967)	2026-07-07 07:10:48 +08:00
AnoobFeng	47b0f604f4	feat(frontend):enhance the ask_clarification interaction with visualized card (#3956) ... * feat(frontend): add structured human input cards for ask_clarification Implement a reusable Human Input Card flow for ask_clarification while keeping the existing text fallback for older clients and IM channels. Backend: - Add structured ToolMessage.artifact.human_input payloads for clarification requests. - Preserve ToolMessage.content as the readable Markdown/text fallback. - Normalize clarification options from native lists, JSON strings, plain strings, mixed scalar values, None, and missing options. - Derive input_mode as choice_with_other when options exist, otherwise free_text. - Keep disable_clarification non-interactive behavior as a plain ToolMessage with no human_input artifact. - Cover artifact persistence and Gateway message metadata preservation in tests. Frontend: - Add human input protocol types, runtime guards, extractors, response builders, and thread-state helpers. - Add reusable HumanInputCard with option buttons, free-text input, pending, read-only, disabled, and answered states. - Render structured clarification cards from artifact.human_input, with Markdown fallback for malformed or legacy tool messages. - Preserve line breaks in structured question/context/option text. - Hide submitted clarification bridge messages from the chat UI via additional_kwargs.hide_from_ui. - Send structured human_input_response metadata through the fourth sendMessage options argument, preserving run context in the third argument. - Wire submissions for normal chats, custom agent chats, agent bootstrap chats, and sidecar chats. - Derive answered state from raw thread.messages so hidden replies still update the original card. - Clear pending state when the hidden reply arrives, dispatch is dropped, or a later async stream failure appears on thread.error. * perf(frontend): optimize HumanInputCard UI interactions - Support Enter key to submit text input (Shift+Enter for newline) - Render question and context fields as Markdown instead of plain text - Replace deprecated FormEventHandler type with structural typing * test(frontend): add unit test cover optimize HumanInputCard UI interactions * feat(frontend): disabled chatbox when has new human-input-card * fix(style): lint error fix * fix: sanitize hidden human input replies - Preserve IME composition safety for human input card Enter submits - Treat hidden human input responses as genuine user messages for sanitization - Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages - Add regression coverage for card IME handling and hidden reply sanitization * fix: tighten human input response validation - Reject empty hidden human input response values - Remove invalid list ARIA role from human input card options - Add backend coverage for empty response payloads --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 22:34:41 +08:00
Vanzeren	823c47bcc2	fix(backend): stop classifying UTF-16 markdown files as binary (#3966) ... * fix: detect utf16 markdown workspace diffs * fix: tighten binary detection in workspace diff scanner * fix: decode utf-8-sig before utf-8 to strip bom from diff content	2026-07-06 22:24:14 +08:00
Ryker_Feng	34f87f6c92	fix(sidecar): panel button deletes the side chat instead of hiding it (#3961) ... * fix(sidecar): make panel button delete the side chat instead of hiding it The side chat panel's top-right button called `sidecar.close()`, which only hid the panel — duplicating the header `SidecarTrigger` toggle. Repurpose the button so the panel owns deletion and the header toggle owns hide/show. - When a conversation exists, the button shows a trash icon and opens a confirmation dialog before deleting via `useDeleteThread` (backend cascade). - In the draft state (references only, no thread yet) the header trigger is not rendered, so the button falls back to a plain close (X) that discards the draft without a confirm — there is nothing persisted to delete. Also fix a latent double-delete bug surfaced by the new dialog: `useDeleteThread` deletes through the LangGraph route (which drops the thread_meta row) and then calls `deleteLocalThreadData`, which hits the same gateway handler whose `require_existing=True` guard now 404s. Treat that 404 as idempotent success. The recent-chat delete used fire-and-forget `mutate`, so it swallowed this error; the sidecar's `mutateAsync` surfaced it as a toast. The e2e mock now mirrors the gateway ownership guard (second DELETE 404s once the thread is gone) so the regression stays covered. Adds tests for delete, draft-state close, and header-owned hide/show. * fix(sidecar): lock delete dialog while deletion is in flight The delete confirmation dialog disabled Cancel during an in-flight delete but still allowed dismissal via the overlay, Esc, and the built-in close (X). Those paths could imply the delete was cancelled when it was actually still running. Ignore dismissals and drop the close button while `isDeleting` so only the (disabled) Cancel path controls the dialog.	2026-07-06 22:14:44 +08:00
Tu Naichao	48e5856c24	Docs/i18n backfill missing sections (#3965) ... * docs: backfill missing i18n sections in README_zh.md - Add Scheduled Tasks and Session Goals sections plus TOC entries - Backfill WeChat iLink channel (intro/table/config/.env/setup) and embedded client goal API - Fix URL adjacency to full-width parens * docs: backfill missing sections in README_ja.md - Add Session Goals, Scheduled Tasks, Terminal Workbench (TUI) sections - Add Embedded client goal API (set_goal/get_goal/clear_goal) and DELETE-endpoint note - Add WeChat and WeCom IM channels (intro/table/config/.env/setup) - Add TOC entries for the three new sections * docs: backfill missing sections in README_fr.md - Add Session Goals, Scheduled Tasks, Atelier terminal (TUI) sections - Add Embedded client goal API (set_goal/get_goal/clear_goal) and DELETE-endpoint note - Add WeChat and WeCom IM channels (intro/table/config/.env/setup) - Add TOC entries for the three new sections * docs: backfill missing sections in README_ru.md - Add Session Goals, Scheduled Tasks, Terminal Workbench (TUI) sections - Add Embedded client goal API (set_goal/get_goal/clear_goal) and DELETE-endpoint note - Add WeChat and WeCom IM channels (intro/table/config/.env/setup) - Add TOC entries for the three new sections	2026-07-06 21:21:11 +08:00
Ryker_Feng	2ebe5cf048	fix(composer): stop bottom mask strip from clipping the focus ring (#3962) ... On conversation pages the composer renders an opaque `bg-background` strip just below itself to mask scrolled content peeking past the rounded corners. The strip was a child of `PromptInput`, the element that draws the focus ring. A parent's box-shadow always paints beneath its own descendants, so the strip covered the bottom ~3px of the ring — the blue focus outline looked cut off along the bottom edge whenever the composer sat flush at the viewport bottom. Move the strip out of `PromptInput` to be a sibling with a lower stacking order and give the composer `relative z-10`, so the ring composites above the strip. The strip still masks the same region; only the paint order changes. Welcome mode is unaffected (it never renders the strip).	2026-07-06 21:11:52 +08:00
He Wang	fd41fdb065	feat(middleware): add structured tool result meta and tool-progress state machine (#3601) ... * feat(middleware): add structured tool result meta and tool-progress state machine feat: - Add tool_result_meta.py: ToolResultMeta dataclass (status/error_type/retryable/ recoverable_by_model/recommended_next_action/source) + normalize_tool_result and stamp_exception_meta utilities; classifies every ToolMessage regardless of path - Add ToolProgressMiddleware: per-(thread_id, tool_name) state machine ACTIVE → WARNED (hint injected as HumanMessage) → BLOCKED (call short-circuited); Jaccard near-duplicate detection for repeated successful results; auth/config/internal errors bypass WARNED and go directly to BLOCKED; LRU-bounded thread state store - Add ToolProgressConfig: all thresholds configurable (stagnation_threshold, warn_escalation_count, jaccard_similarity_threshold, exempt_tools, etc.); disabled by default (enabled: false) - Wire ToolProgressMiddleware as outer wrapper around ToolErrorHandlingMiddleware in _build_runtime_middlewares so it receives results already carrying deerflow_tool_meta fix: - ToolErrorHandlingMiddleware now calls stamp_exception_meta on exception path and normalize_tool_result on success path so every ToolMessage carries deerflow_tool_meta test: - Add test_tool_result_meta.py: 26 cases covering all classification paths, stamp_exception_meta, and normalize_tool_result Command passthrough - Add test_tool_progress_middleware.py: 27 cases including full async paths, Jaccard duplicate detection, LRU eviction, hint injection, and malformed meta passthrough - Extend test_tool_error_handling_middleware.py: middleware ordering invariant and meta stamping on exception docs: - Add tool_progress section to config.example.yaml with all fields and descriptions - Update CLAUDE.md middleware chain documentation (entries 8-9) Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(middleware): recoverable errors stay WARNED; fix auth keyword shadowing fix: - WARNED is terminal for recoverable_by_model=True errors (no_results, not_found, permission); hint re-injected on each problem call instead of escalating to BLOCKED, so the model can retry with different parameters (e.g. fresh query, new URL) without being hard-blocked by a prior stagnation count. Non-recoverable (rate_limited, transient) still escalate WARNED → BLOCKED after warn_escalation_count more problems; auth/config/internal remain immediately BLOCKED. - Remove bare "api key" keyword from auth classification rule so "no api key configured" correctly classifies as config (not auth), producing the accurate block-reason text for the model. docs: - CLAUDE.md: document all three ToolProgressMiddleware transition paths - config.example.yaml: update inline state-machine comment to match new paths test: - test_recoverable_errors_stay_warned_indefinitely: WARNED never escalates for recoverable errors regardless of how many problem calls accumulate - test_recoverable_error_re_injects_hint_past_escalation: hints continue past the escalation zone for recoverable errors - test_no_api_key_is_config_not_auth: regression guard for keyword shadowing fix Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(tool_result_meta): add JSON error extraction and fix source classification fix: - Fix non-standard error path: source was "exception" but should be "tool_return" - Add _extract_json_error_text to isolate JSON error fields from noisy JSON bodies (e.g. Brave Search {"error": "...", "query": "..."} — query keywords no longer pollute error classification) - Add success-path JSON extraction to catch tools that return HTTP 200 with a JSON error body (status="success" but {"error": "API key not configured"}) - Add _SEMANTIC_ZERO_ERROR_STRINGS frozenset to suppress false positives from tools that use {"error": "none"} / {"error": "null"} / {"error": "ok"} as success signals - Document that stamp_exception_meta always overwrites existing TOOL_META_KEY (exception-derived classification is authoritative over tool return-time stamps) test: - Add parametrized regression tests for all semantic-zero error strings - Add tests for non-standard error path source field - Add tests for JSON error extraction (nonstd, success-path, numeric, falsy values) - Correct test comment for test_no_api_key_is_config_not_auth Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(tool_progress_middleware): fix 6 bugs, add terminal guard and structured logging fix: - H1: fix exempt_tools empty-set silently ignored — use `is not None` instead of truthiness check so ToolProgressConfig(exempt_tools=set()) correctly disables all exemptions - Fix _get_block_reason creating phantom LRU entries via _get_state (write path); now uses dict.get + explicit move_to_end on read path only - Fix _pending memory leak: LRU eviction of _phase_states now synchronously removes all (evicted_thread, *) keys from _pending - Fix _assess_and_transition missing terminal guard for blocked state — a recoverable error result could silently demote blocked → warned in concurrent-race scenarios; early return preserves terminal semantics - Fix recent_word_sets window: stored [-5:] but is_near_duplicate only compared [-3:]; align to [-3:] and change type list→tuple (prevents accidental in-place mutation across dataclasses.replace shallow copies) - Fix _format_hint missing "success" key and "continue" action: Jaccard near-duplicate results produced the generic fallback instead of a specific actionable message feat: - Add structured state-transition logging (ACTIVE/WARNED/BLOCKED transitions, blocked intercepts, hint injection debug log) test: - Add regression tests for all 6 bug fixes (H1, phantom LRU, pending leak, terminal guard, window alignment, format_hint near-dup) - Add Jaccard near-threshold boundary test (7/9 vs 8/9 Jaccard) - Add production min_words=10 skip test for short content - Add exempt_tools empty-set and None round-trip tests - Add _augment_request deduplication test - Add before_agent current-run preservation test - Add structured logging tests (WARNED/BLOCKED/ACTIVE/intercepted/debug) Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * chore(config): remove unused backward-compat fields from ToolProgressConfig Remove max_calls_per_intent and window_size fields that were marked "Retained for backward compatibility; not used by the current state machine" when the state machine was introduced. Pydantic v2 ignores unknown fields by default, so existing config.yaml files with these keys remain valid. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com> * fix(tool_progress): address PR review and multi-agent review findings fix: - Remove <80-char length gate for partial_success; only _PARTIAL_MARKERS now - Add word-boundary regex for numeric HTTP codes (401/403/404/500) to avoid false positives like "500ms" or "4010 rows" triggering hard-block - Add "task" to default exempt_tools (delegation primitive, not a search tool) - Remove move_to_end() from _get_block_reason read path; blocked threads were permanently warm in LRU, starving active threads of eviction slots - Add _reset_blocked_states in before_agent: scope BLOCKED and WARNED states to a single run; clear recent_word_sets so stale Jaccard windows don't cause false near-duplicate detections in the next run - Compute word_set() lazily (only for success results); cap content at 8192 chars to bound memory and CPU cost on large tool results - Remove unused retryable field from ToolResultMeta (no consumer existed) - Add isinstance-based ordering guard and warning log for missing meta - Fix JSON-without-error-key fallback: use _UNKNOWN_ERROR instead of classifying incidental field values (e.g. {"user_id": 401} → auth → stop) - Fix _extract_json_error_text: use json.dumps for dict/list error fields instead of str() which produced Python repr matching config rules spuriously - Add "no results found"/"no content found"/"no images found" to _PARTIAL_MARKERS so success responses with empty results trigger stagnation detection - Fix immediate-block path to increment consecutive_problems (was left at 0) - Fix _queue_assessment: skip phantom _pending entries for evicted threads - Bump config_version 13→16 (upstream added 14/15; tool_progress is additive) test: - Update test_short_content_is_partial → test_short_terse_success_is_not_partial - Add parametrized test_numeric_keyword_word_boundary (8 positive + negative cases) - Add test_before_agent_resets_blocked_states_for_new_run (strengthened assertions) - Add test_before_agent_resets_warned_states_for_new_run - Add test_missing_meta_on_non_exempt_tool_emits_warning - Add test_middleware_ordering_guard_raises_when_progress_is_inner - Add test_auth_error_immediately_blocked asserts consecutive_problems == 1 - Add tests for JSON-without-error-key, dict error field, no-results partial_success Co-Authored-By: Claude <noreply@anthropic.com> * fix(tool_progress): address second PR review — perf, architecture doc, concurrency note fix: - Extract content.lower() once before _PARTIAL_MARKERS check in normalize_tool_message; previously computed up to 7× per call inside the generator (once per marker) docs: - Add division-of-labor paragraph to ToolProgressMiddleware module docstring explaining coexistence with LoopDetectionMiddleware: result-quality guard (per-tool BLOCK) vs call-pattern guard (whole-turn hard-stop); no shared state, no double-stop risk - Add threading.Lock comment explaining why asyncio.Lock is not used (short critical sections, must also protect sync wrap_tool_call path from subagent executor threads) - Update backend/CLAUDE.md entry 8 with division-of-labor summary; fix entry 9 (remove stale retryable field reference, add missing recoverable_by_model/source) test: - Add test_tool_progress_and_loop_detection_coexist_without_interfering: drives both middlewares to WARNED state simultaneously, verifies independent state, independent hint queues, and no cross-contamination; uses snapshot copy for final assertion Co-Authored-By: Claude <noreply@anthropic.com> * fix(tool_progress): reset all tool states at run boundary; fix semantic-zero test validity fix: - _reset_run_states (formerly _reset_blocked_states) drops the phase filter and resets all tracked (thread, tool) pairs unconditionally at before_agent; ACTIVE tools with sub-threshold consecutive_problems or cached recent_word_sets no longer bleed into the next run, preventing spurious WARNED transitions on clean R2 calls - test_normalize_json_semantic_zero_error_string_not_treated_as_error: replace {error_value!r} f-string (produces invalid JSON with single quotes) with json.dumps so _extract_json_error_text actually parses the payload and the _SEMANTIC_ZERO_ERROR_STRINGS guard is exercised, not bypassed at json.loads test: - add test_before_agent_resets_active_state_consecutive_problems_and_word_sets to lock the ACTIVE-phase run-boundary reset: drives tool to active/cp=1/ws≠() in R1, asserts both fields are zero/empty after before_agent fires for R2 * docs(tool_progress): document intentional per-run reset vs LoopDetection thread-scoped retention Addresses reviewer observation in PR #3601 that _reset_run_states diverges from LoopDetectionMiddleware's cross-run scoping policy without explanation. Expands the _reset_run_states docstring to record the intentional design choice: ToolProgressMiddleware resets per-run because result-quality errors (rate_limited, transient) are time-bound and may resolve between turns — retaining stale counters would risk false-positive BLOCKED calls. LoopDetectionMiddleware retains history across runs because call-pattern loops are time-invariant. The divergence is by design, not oversight. Co-Authored-By: Claude <noreply@anthropic.com> * fix(middleware): restore ReadBeforeWriteMiddleware as outermost write gate A merge conflict resolution had accidentally placed ReadBeforeWriteMiddleware after ToolErrorHandlingMiddleware (inner), reversing the original intent from b81334cc where it was the outermost write gate before ToolErrorHandling. fix: - Restore ReadBeforeWriteMiddleware to outer position: ReadBeforeWrite → ToolProgress → ToolErrorHandling. Blocked writes now return immediately without consuming a ToolProgress slot. - Add normalize_tool_result call on blocked ToolMessages so they carry deerflow_tool_meta (recoverable_by_model=True) even though they bypass ToolErrorHandlingMiddleware. test: - Add test_blocked_write_has_deerflow_tool_meta (sync + async) to lock the normalize_tool_result behavior on blocked writes. - Fix chain order assertions in TestChainWiring and test_build_lead_runtime_middlewares_chain_order_matches_agents_md. docs: - Renumber AGENTS.md items: 10→ReadBeforeWrite, 11→ToolProgress, 12→ToolErrorHandling; update descriptions to reflect outermost-gate design. - Fix stale cross-reference: LoopDetectionMiddleware (item 23) → (item 25). Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 20:57:41 +08:00
MeloMei	f122594419	refactor(frontend): extract placeholder detection utility with unit tests (follow-up to #3764) (#3783) ... * refactor(frontend): extract placeholder detection utility with unit tests * fix(frontend): pass prompt text directly to onSelectPlaceholder to avoid stale DOM read The onSelectPlaceholder callback was reading textarea.value immediately after textInput.setInput(prompt), but React state updates are async so the DOM had not yet reflected the new value. This caused the placeholder auto-selection to silently fail when the textarea was previously empty. Fix: accept the new text as a parameter instead of reading from the DOM. * fix(frontend): resolve duplicate findSuggestionTemplatePlaceholder identifier After rebasing onto main, the function existed both inline in input-box-helpers.ts (from #3764) and as an import from our new placeholders module, causing TS2300 duplicate identifier errors. - Remove duplicate import in input-box.tsx - Replace inline function in input-box-helpers with re-export from @/core/suggestions/placeholders - Export SUGGESTION_TEMPLATE_PLACEHOLDER_PATTERN from placeholders module * refactor(frontend): remove dead hasUnreplacedPlaceholder export No production call site uses this boolean wrapper — both existing checks need the {start,end} range from findSuggestionTemplatePlaceholder. Drop the function and its two unit tests.	2026-07-06 15:44:24 +08:00
heart-scalpel	8cde7f258e	feat(gateway): refuse multi-worker startup on non-Postgres backends (#3960) ... Why: issue #3948 identifies four correctness breakers when GATEWAY_WORKERS > 1. Work item 1 adds a startup gate that refuses to boot when database.backend is not postgres, giving operators a clear error instead of silent SQLite write-lock corruption. The gate runs inside langgraph_runtime() before init_engine_from_config, so a misconfigured deploy never opens a listener or writes to disk. Non-integer env values fall back to 1 so uvicorn's own validation is unaffected.	2026-07-06 15:20:27 +08:00
Tianye Song	28f2b07b79	feat(memory): add staleness review to prune silently-outdated facts (#3860) ... * feat(memory): add staleness review to prune silently-outdated facts Facts created long ago may become outdated without any future conversation explicitly contradicting them ("Silent Staleness"). This adds a staleness review mechanism that surfaces aged facts to the LLM during the normal memory-update call so it can semantically judge whether each is still valid. - New MemoryConfig fields: staleness_review_enabled, staleness_age_days, staleness_min_candidates, staleness_max_removals_per_cycle, staleness_protected_categories - New STALENESS_REVIEW_PROMPT section injected into MEMORY_UPDATE_PROMPT when enough stale candidates exist - New staleFactsToRemove output field in the LLM response schema - Safety cap limits max removals per cycle, keeping lowest-confidence entries when the LLM returns more than the cap - Correction facts (category=correction) are protected by default - Observability via structured logging of each removal with reason - 32 unit tests covering parsing, selection, triggers, formatting, normalization, safety cap, and integration * fix(memory): add deterministic guardrail for staleness removals _apply_updates previously removed any fact id the LLM returned in staleFactsToRemove without verifying it was in the actual staleness candidate set. An LLM slip could silently delete protected-category facts (e.g. correction) or fresh facts, defeating the stated guarantee. Now intersect stale_ids_to_remove with _select_stale_candidates before the safety cap, making the protection independent of both model behavior and the staleness_review_enabled flag. Add three regression tests: - test_protected_category_fact_refused_at_apply - test_non_aged_fact_refused_at_apply - test_guardrail_runs_when_staleness_review_disabled * docs(memory): sync AGENTS.md staleness config + simplify datetime parsing Address reviewer feedback from PR #3860: - Add staleness workflow step and 5 new config fields to backend/AGENTS.md - Simplify _parse_fact_datetime: drop manual Z→+00:00 replace, Python 3.12+ fromisoformat handles Z natively --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 15:09:56 +08:00
Ryker_Feng	cb83edb0da	fix(sidecar): correct text-selection toolbar actions inside the side chat (#3959) ... * fix(sidecar): correct text-selection toolbar actions inside the side chat The sidecar panel reuses MessageList, but it did not distinguish the side chat surface from the main conversation, so selecting text inside the side chat behaved as if it were the main list: - The "Ask in side chat" action was shown even though the user is already in the side chat, which is a no-op interaction. - "Add to conversation" routed the snippet to the main composer's quotes (conversationQuotes) instead of the side chat's own composer, so the reference landed in the wrong input box. Add a `sidecarSurface` prop to MessageList. On the sidecar surface, hide "Ask in side chat" and route "Add to conversation" to `sidecar.openContext` so the snippet attaches to the side chat's own composer (activeReferences). Main-list behavior is unchanged. * docs(sidecar): drop AGENTS.md note for the toolbar surface change The sidecarSurface behavior is self-evident from the code; no dedicated Interaction Ownership entry is needed.	2026-07-06 14:37:31 +08:00
Huixin615	eb5eb9c574	fix(docker): keep config mounts stable across host saves (#3954) ... * fix(docker): keep mutable config mounts stable * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-06 08:00:49 +08:00
hataa	0664ea2243	fix(subagents): surface turn-budget cap as MAX_TURNS_REACHED with partial result (#3875 Phase 2) (#3949) ... * fix(subagents): surface turn-budget cap as MAX_TURNS_REACHED with partial result (#3875) Phase 2 of #3875. When a subagent exhausts its turn budget (recursion_limit == max_turns), LangGraph raises GraphRecursionError from agent.astream. The generic except Exception in _aexecute misclassified it as FAILED and discarded the partial work already streamed into final_state, so the lead could not tell 'broken subagent' from 'out of budget' and got an empty failure. Catch GraphRecursionError specifically (before the generic handler) and set a distinct SubagentStatus.MAX_TURNS_REACHED terminal status, recovering the partial result from the last streamed chunk via a shared _extract_final_result helper (refactored out of the normal-completion path so both paths render content identically). Extend the cross-language status contract so the new value travels on additional_kwargs.subagent_status: a capped run is result-bearing, so make_subagent_additional_kwargs / read_subagent_result_metadata carry subagent_result_brief + subagent_result_sha256 (the recovered work, like completed) AND the cap notice on subagent_error -- the one status that carries both. task_tool.py returns it via the shared _task_result_command; the delegation ledger prefers the partial result_brief and renders model-facing guidance (reuse / retry tighter / raise max_turns). Frontend collapses max_turns_reached to the failed pill with the cap notice on error. No agent-loop, runner, or persistence behavior touched; default max_turns is unchanged. * refactor(subagents): consolidate content-stringify onto shared helper Address review feedback on #3949 (willem-bd, copilot-pull-request-reviewer): - executor.py: drop the private `_stringify_message_content` — a third near-duplicate of `utils/messages.py::message_content_to_text`. `_extract_final_result` now delegates to that canonical helper; the "No response generated" sentinel is pushed down to the consumer (the shared helper returns "" for no-text, matching every other call site). - task_tool.py: align the live `task_failed` event's error string with the canonical "Reached max_turns=N" used by the logger, the structured `error=`, and the executor (was "Reached max turns (N)"). Behavior for real AIMessage content is unchanged; only atypical edge inputs (consecutive bare-string list items; empty content) now match the canonical helper that every other call site already uses. `extract_response_text` is intentionally left as-is: it filters by OpenAI content-block `type`, a different shape with many callers and its own tests. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-07-06 07:55:32 +08:00
Ryker_Feng	186c6ea463	feat: add branching support for assistant turns (#3950) ... * feat: add assistant turn branching * fix(threads): skip workspace clone when branching from historical turn Workspace files are not checkpointed, so cloning them onto a branch rooted at an older assistant turn leaked files created in a later timeline. Restrict the best-effort workspace copy to branches taken from the latest turn; historical-turn branches now report workspace_clone_mode="skipped_historical_turn" and keep only the restored message history. * style(frontend): fix prettier formatting in e2e mock-api Collapse the branch-title normalization chain onto a single line to satisfy the frontend lint (prettier --check) CI gate. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 07:40:41 +08:00
Vanzeren	67c8ade375	feat(sandbox): warm pool for boxlite (#3951) ... * feat: add boxlite SDK as harness dependency * test: add fake SimpleBox fixtures for BoxLite warm pool tests * feat: add deterministic sandbox_id and warm pool fields to BoxliteProvider * feat: pass deterministic sandbox_id to SimpleBox name * feat: warm pool lifecycle — park on release, reclaim on acquire - release(): parks VMs in _warm_pool with timestamp instead of closing - _reclaim_warm_pool(): health checks warm boxes via echo ok - acquire(): tries warm pool reclaim before creating new boxes - Deterministic sandbox_id ensures thread isolation * feat(boxlite): idle reaper, replica enforcement, warm-pool shutdown/reset - Task 6: idle reaper daemon thread destroys expired warm-pool boxes - Task 7: replica enforcement evicts oldest warm-pool box when at capacity - Task 8: shutdown() stops idle checker first, destroys all boxes (active+warm); reset() clears warm pool Tests: 17 passed (4 new: idle reaper, replica enforcement, shutdown, reset) * fix(boxlite): harden warm pool lifecycle races * docs(boxlite): document warm pool configuration * fix(sandbox): log warning when evicting oldest warm box is failed Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(boxlite): stop provider lifecycle on reset * fix(boxlite): make runtime an optional dependency * test(boxlite): remove unused warm pool lookup * docs(boxlite): clarify optional runtime support * refactor: extract shared WarmPoolLifecycleMixin for sandbox warm-pool lifecycle Introduce deerflow.community.warm_pool_lifecycle.WarmPoolLifecycleMixin owning idle-checker loop, warm-pool expiry, oldest-warm eviction, replica counting, and soft-cap logging. Move AioSandboxProvider and BoxliteProvider onto the mixin; keep AIO active-idle cleanup local and delegate only warm-pool expiry to the shared helper. BoxliteProvider also gains: - Prefixed box names (deer-flow-boxlite-*) for startup orphan reconciliation - _reconcile_orphans() adopting surviving boxes from a prior process - Pinned timeout forwarding: command timeout now bounds both BoxLite SDK exec(timeout=...) and the loop bridge .result(timeout) - reset() reworked as lightweight registry clear (boxes -> warm pool, no close, no idle-reaper stop, no loop close) so reset_sandbox_provider() config switches are safe; shutdown() remains the teardown path Backward-compatible: AIO DEFAULT_IDLE_TIMEOUT/DEFAULT_REPLICAS/ IDLE_CHECK_INTERVAL stay importable; Boxlite IDLE_CHECK_INTERVAL stays monkeypatchable. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-06 07:21:12 +08:00
Ryker_Feng	7a985f441c	Fix(frontend): Fix mobile workspace and accessibility blockers (#3740) ... * fix(frontend): address mobile workspace polish blockers * fix(frontend): prevent mobile landing overflow * fix(frontend): keep landing sections within mobile viewport Section titles used a fixed text-5xl with no word breaking, and the <section> flex items had default min-width:auto, so wider Linux font metrics in CI pushed content past the viewport (scrollWidth 345 > 320). Make titles/subtitles responsive with break-words, constrain section width with min-w-0, and add an overflow-x-clip guard on the page root. * fix(frontend): address review feedback on mobile landing PR - add hamburger Sheet nav below sm: so mobile users keep docs/blog access - switch useIsMobile to useSyncExternalStore to avoid hydration swap flash - memoize artifactContent so it isn't rebuilt on every streamed token - use slot-based key in HeroWordRotate; drop flex-wrap so SuperAgent never orphans at 320px - delete unused word-rotate.tsx dead code - move section gutter to <main> for a uniform mobile padding contract - harden e2e: per-viewport overflow tests, locale-stable artifact selector, focus-ring asserts light vs dark differ --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-06 07:15:35 +08:00
ly-wang19	f0f9dd6656	feat(setup): ask whether OpenAI-compatible gateway models support thinking (#3428) ... The "Other OpenAI-compatible" wizard provider lets users supply a custom base_url and model name but never asked whether that model supports thinking/reasoning, so the generated config.yaml always left supports_thinking at its default of false — even for reasoning models behind the gateway. Add an explicit ask_thinking_support flag on LLMProvider (enabled for the "other" provider) plus a pure with_thinking_support() helper. When the flag is set, the LLM step prompts via ask_yes_no; confirming wires the standard OpenAI-compatible enable/disable thinking toggles, declining records supports_thinking=false. Provider definitions are copied with dataclasses.replace, never mutated. Adds unit tests for the helper and the interactive step. Closes #3162 Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-05 19:55:39 +08:00
Ryker_Feng	00161811bd	feat: add workspace change review for agent runs (#3945) ... * feat: add workspace change review * chore: format workspace change files * fix: optimize workspace change summaries * style: refine workspace change badge scale * fix: restore workspace change user context import * fix(frontend): gate workspace change badge to assistant messages Only pass run_id to assistant MessageListItems so the workspace-change badge can never render under a user's prompt, which carries the same run_id. Assert single badge render in the E2E flow. * fix(workspace-changes): address review feedback on diff parsing and badge - Restrict unified-diff header detection to "+++ "/"--- " (trailing space) in both backend _count_diff_lines and frontend getWorkspaceChangeLineClass so content lines beginning with +++/--- are counted/styled correctly - Gate WorkspaceChangeBadge to ai messages so tool messages folded into an assistant group don't render a duplicate badge - Add regression tests for both diff-classification fixes - Apply prettier formatting to workspace-changes files flagged by CI --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-05 15:38:01 +08:00
Prax Lannister	f6a910dec9	fix(models): normalize api_base->base_url for ChatOpenAI + warn on unknown config keys (#3790) ... ModelConfig is `extra="allow"`, so a config key like `api_base` (which config.example.yaml uses for other model classes, e.g. PatchedChatDeepSeek and Moonshot) gets copied onto a `langchain_openai:ChatOpenAI` model by users. LangChain's OpenAI client does not reject the unknown kwarg — it transfers it into `model_kwargs` (with a UserWarning), which is then spread into every `Completions.create()` call and rejected by the OpenAI SDK at REQUEST time with an opaque `unexpected keyword argument 'api_base'` error. The endpoint override is also silently dropped, so the model targets the wrong base URL. Changes in factory.py, mirroring the existing OpenAI-compatible helpers: - `_normalize_openai_base_url`: renames `api_base` -> `base_url` for the OpenAI-compatible family (ChatOpenAI + PatchedChatOpenAI); when an endpoint key is already present, drops the alias with a warning. Runs before the stream_usage/stream_chunk_timeout heuristics so they see the canonical key. - `_warn_unknown_model_settings`: scoped to the same OpenAI-compatible family (where the model_kwargs divert-and-crash actually happens and the field/alias set is accurate), logs an actionable warning for unrecognized config keys. - The three OpenAI-compatible helpers now share `_OPENAI_COMPAT_USE_PATHS` instead of disagreeing on the literal class string. Adds a note to docs/CONFIGURATION.md and 9 tests covering normalization (ChatOpenAI + PatchedChatOpenAI, both-set precedence for base_url and openai_api_base, non-OpenAI class untouched, no-op when unset) and the unknown-key warning (fires on a typo, silent on a clean config and on non-OpenAI providers like ChatAnthropic).	2026-07-05 10:17:25 +08:00
Nan Gao	c05c1899b5	fix(frontend): Fix uploaded file metadata in message copy (#3944) ... * fix uploaded file metadata copy * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-05 10:04:14 +08:00
Yi Deng	12eda6c701	fix(web_fetch): add SSRF guard for self-hosted providers (#3942) ... Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-05 09:54:55 +08:00
Willem Jiang	972096db88	fix(unittest): fix the unit test error TestConsoleUsage with date time setting (#3946)	2026-07-05 09:40:16 +08:00
AnoobFeng	e9de187408	fix(provisioner): keep k8s calls off event loop (#3941) ... * fix(provisioner): keep k8s calls off event loop * test(provisioner): scan blocking IO by module name * test(provisioner): exercise create path so BlockBuster has real work	2026-07-05 08:45:27 +08:00
dorianzheng	358bacad89	feat(sandbox): add BoxLite micro-VM sandbox provider (scaffold) (#3940) ... * feat(sandbox): add BoxLite micro-VM sandbox provider (scaffold) Community SandboxProvider backed by BoxLite, a daemonless OCI-native micro-VM runtime. execute_command is wired end to end through a private asyncio loop that bridges BoxLite's async SDK to DeerFlow's sync Sandbox contract; the remaining Sandbox methods are stubbed pending approach review. Opt-in via sandbox.use (pip install boxlite); a packaged [boxlite] extra will follow. Existing providers are unchanged. Refs #3936, #3439, #3213 * feat(sandbox): implement full Sandbox contract for BoxLite backend Rename the community BoxLite integration to read as a compute backend rather than "a sandbox": module boxlite_sandbox -> boxlite, BoxliteSandboxProvider -> BoxliteProvider, BoxliteSandbox -> BoxliteBox. Implement the file surface DeerFlow's default tool path assumes -- read_file, write_file, update_file, download_file, list_dir, glob, grep -- as shell commands inside the box (cat/find/grep/chunked base64), reusing deerflow.sandbox.search and busybox-portable flags. Removes the reachable NotImplementedError regression once the provider is selected. download_file keeps the /mnt/user-data prefix + traversal guards; the provider materialises those virtual dirs on box start. Refs #3936, #3940 * fix(sandbox): resolve CI + review findings on BoxLite backend - provider: import DEFAULT_SKILLS_CONTAINER_PATH instead of the "/mnt/skills" literal (backend-unit-tests guard), and drop the redundant env-var re-resolution -- AppConfig.resolve_env_variables already resolves $VARS and raises on missing. - box: grep now passes the raw pattern to grep (-F/-E); it previously handed the re.escape'd pattern to grep -F, so a literal search of e.g. foo.bar looked for foo\.bar and never matched. - box: execute_command now calls _validate_extra_env(env), matching the Sandbox POSIX env-key contract that the local/e2b/aio sandboxes enforce. - add tests/test_boxlite_provider.py (CI-safe, no BoxLite): actionable ImportError on the lazy import, clean acquire-failure + shutdown, the traversal / download-prefix guards, and env-key rejection. Refs #3936, #3940	2026-07-05 00:17:30 +08:00
Ryker_Feng	6a4e5a3bb2	feat(frontend): add side conversations for quoted follow-ups (#3934) ... * feat(frontend): add side conversations for quoted follow-ups * style(frontend): apply prettier formatting to sidecar-chat files * fix(frontend): surface sidecar cascade cleanup failures via console.warn Previously deleteSidecarThreadsForParent silently swallowed both lookup errors and per-thread deletion failures, so parent thread deletions could succeed while orphaning sidecar threads with no signal to the caller. Log a warning that includes the parent id and the failed thread ids/reasons so the leak is discoverable in telemetry, matching the existing console.warn/error pattern in this file. * fix(frontend): address all sidecar review feedback Resolve every reviewer comment on PR #3934: - input-box/hooks/sidecar-panel: clear quoted references only via an `onSent` callback that fires after the in-flight guard, so a dropped send no longer silently discards quotes (willem-bd #3550). - message-list: flip the selection toolbar below the selection when it would clip above the viewport (willem-bd #3551). - reference-metadata/thread/input-box: keep referenced ids, roles, and count arrays 1:1 parallel instead of deduping ids (willem-bd #3552). - message-list: widen selection containment to the shared assistant-turn container and hint when a selection crosses messages (willem-bd #3553). - sidecar/api: coalesce concurrent sidecar creates for one parent behind a single in-flight promise to prevent duplicates (willem-bd #3554). - sidecar-trigger/context: force-restore on trigger click so a sidecar deleted elsewhere self-heals instead of opening a dead thread (willem-bd #3555). - threads/hooks: surface sidecar cascade cleanup failures via console.warn for both lookup and per-thread deletes (Copilot). Add unit + e2e coverage for parallel metadata, atomic create, and trigger self-healing.	2026-07-05 00:12:16 +08:00
d33kayyy	84bbdf6e00	fix(langfuse): resolve trace user from runtime context (#3794) ... * fix(langfuse): resolve trace user from runtime context The worker built langfuse_user_id from get_effective_user_id(), which reads the request-scoped _current_user ContextVar. For runs invoked over an internal token on behalf of an end user, that ContextVar is never the end user, so traces recorded langfuse_user_id="default". Switch to resolve_runtime_user_id(runtime), matching the sandbox middleware/tools sites: it reads runtime.context["user_id"] (the owner carried in the run request's context, which survives background-task boundaries) and falls back to get_effective_user_id() for no-auth / browser paths. Caller-supplied metadata still wins via inject_langfuse_metadata's setdefault. * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-07-05 00:07:27 +08:00
Xinmin Zeng	4d660b202a	feat(skills): bind request-scoped secrets for autonomously-invoked skills (A+) (#3938) ... * feat(skills): bind request-scoped secrets for in-context (autonomously invoked) skills Extends the #3861 binding point A (slash-activation only) to A+: the injection set is recomputed on every model call from two unioned sources — the run's most recent slash activation (persisted on the run context so the tool loop keeps the binding) and skills the model actually loaded in this thread (ThreadState.skill_context), re-validated against the live registry each call. Authorization stays three-gated regardless of activation style: skill enabled by the operator, values supplied per-request by the caller in context.secrets (never persisted server-side, never from the host env), names declared in the skill's required-secrets frontmatter. Because the set is replaced per call, eviction from skill_context or a caller that stops supplying a value revokes injection on the next call. New frontmatter field secrets-autonomous (default true) lets a skill restrict binding to explicit slash activation; malformed values fail closed to false. Binding changes are recorded as a middleware:skill_secrets journal event carrying names only. Design informed by a survey of peer systems (Claude Code, Codex CLI, opencode, pi, deepagents, hermes-agent, QwenPaw) and specs (agentskills.io, MCP 2025-11-25): the industry trust boundary is enable-time consent plus caller-scoped credentials, not per-invocation ceremony; no surveyed system scopes secrets to an activation turn. Part of #3914 * refactor(skills): centralize secret context keys, document intentional per-call reload Review follow-ups (no behavior change): move the two private binding keys (__slash_skill_secret_source, __skill_secrets_binding_audit) into secret_context.py and add them to REDACTED_CONTEXT_KEYS so the redaction allowlist stays a complete guard even though both keys hold names only. Document why _in_context_secret_sources reloads skills every call rather than caching: load_skills re-reads enabled state so an operator disabling a skill revokes its binding on the next model call — an mtime cache would miss enable/disable toggles and keep injecting after a disable. * fix(skills): match in-context secret bindings by path only, never by name Review finding (confused deputy): _in_context_secret_sources fell back to name matching when a skill_context path did not resolve. DeerFlow lets a custom skill shadow a same-named public/legacy one (load_skills de-dupes by name, custom wins), so a thread that read public/foo could bind the custom foo's declared secrets although the custom skill was never loaded in the thread. The recent user-isolation path changes make by-path misses (and thus the dangerous fallback) more likely. Drop the by-name fallback: match strictly by the exact container file path the model read; an unresolved path simply does not bind (the safe direction). Regression tests cover the shadowing case and a stale path. Part of #3914 * fix(skills): resolve secret-binding sources via registry; strip caller __-keys Security review (willem-bd, #3938): 1. Forged `__slash_skill_secret_source` bypassed the enabled/allowlist/ secrets-autonomous gates. runtime.context is caller-mergeable, and the slash source was trusted as authoritative (its stored requirements were injected directly). Now the slash source records only the activated skill's canonical container path, and BOTH the slash and in-context sources resolve the live registry skill by normalized path each call (_resolve_registry_skill) — binding only that real, enabled, allowlisted skill's own declared secrets. A forged path resolves to nothing. As defense in depth, build_run_config strips caller-supplied __-prefixed context keys at the gateway boundary. 2. Malformed caller requirements crashed the run (unguarded tuple unpack / DoS). The middleware no longer unpacks caller-provided requirement data at all — declarations come from the registry — so a malformed source fails closed instead of raising. 3. Path-normalization asymmetry silently disabled in-context binding on a trailing-slash container_path config. Both the registry keys and the lookup path are now posixpath.normpath'd. Regression tests: forged source rejected, forged-but-real path ignores caller requirements + allowlist, malformed source fails closed, trailing- slash config binds, gateway strips __-keys. Part of #3914 * docs(skills): correct _SLASH_SECRET_SOURCE_KEY comment and note fail-closed trade-off Post-review cleanup: the key now stores only the canonical container path (the comment still described the pre-fix skill-name+requirements shape), and document that a transient registry-load failure fails closed (drops the binding for that call) rather than trusting stale data. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 23:34:32 +08:00
codeingforcoffee	4669d3c089	feat(gateway): cache-aware cost accounting (#3920) ... * feat(gateway): cache-aware cost accounting + /api/console observability endpoints - Capture prompt-cache hits (usage_metadata.input_token_details.cache_read) in RunJournal and SubagentTokenCollector as a sparse cache_read_tokens key in token_usage_by_model (JSON field — no schema migration; legacy bucket shapes unchanged) - New read-only /api/console router: GET /stats (headline counters), GET /runs (cross-thread paginated history joined with thread titles), GET /usage (zero-filled daily token series + per-model breakdown); user-scoped, 503 on the memory database backend - Optional models[*].pricing (currency, input_per_million, output_per_million, input_cache_hit_per_million) powers real spend estimation; cache-hit input tokens are billed at the hit price (omitted hit price falls back to the miss price as a conservative upper bound); unpriced models yield cost: null - create_chat_model strips the presentation-only pricing block so it never reaches the provider client (unknown kwargs are forwarded into the completion payload and break live calls) - Tests: console router SQLite round-trips, journal/collector cache capture incl. a DeepSeek raw-usage pin test, factory strip regression Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: address review feedback on cost sum and sparse cache_read_tokens - console.py: replace the walrus-in-generator total-cost sum with an explicit loop (review noted the multi-line form reads ambiguously) - token_collector.py: omit cache_read_tokens from usage records when the provider reported no cache hits, matching the journal's sparse per-model bucket shape; absent is treated as 0 downstream - add a regression test pinning the sparse record shape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: coffeeFish <codeingforcoffee@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-04 23:14:46 +08:00
Tianye Song	15454b6fec	feat(skills): deferred skill discovery via describe_skill tool (#3775) ... Replace the full-metadata <available_skills> system-prompt block with a compact <skill_index> (names only) and an on-demand describe_skill tool when skills.deferred_discovery: true (default: false / backward compat). New modules: - skills/catalog.py — SkillCatalog (immutable, searchable; select: has no cap, keyword/prefix search caps at MAX_RESULTS=5) - skills/describe.py — build_describe_skill_tool(catalog) closure; build_skill_search_setup() wires SkillSearchSetup into both the LangGraph agent factory (agent.py) and DeerFlowClient (client.py) Changes: - Skill @dataclass(frozen=True); allowed_tools/required_secrets list→tuple - Skill First prompt line gated on skill_names (deferred vs legacy wording) - get_skills_prompt_section: short-circuit storage on deferred path; merge user_id (upstream) + skill_names (this PR) params - describe_skill tool parameter named "name" (matches prompt wording) - select: branch removes [:MAX_RESULTS] cap (exact request, not ranking) - AGENTS.md: document deferred_discovery config field + new modules Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 23:09:29 +08:00
Huixin615	c22c955c2d	fix: batch Feishu file messages into one thread (#3753) ... * fix: batch feishu file messages * fix: narrow Feishu file batching	2026-07-04 23:06:28 +08:00
Zheng Feng	dcb2e687d5	feat(channels): add GitHub as a webhook-driven channel (#3754) ... * feat(channels): add GitHub event-driven agents (#3754) Add a webhook-driven GitHub channel with fail-closed webhook routing, deterministic per-agent PR/issue threads, mention-gated trigger fan-out, GitHub App token injection for sandboxed gh/git commands, and backend/AGENTS.md documentation. * fix(llm-middleware): classify bare IndexError as transient Upstream chat providers occasionally return 200 OK with an empty generations list (observed against Volces "coding" on ark.cn-beijing.volces.com). When that happens, langchain_core.language_models.chat_models.ainvoke raises ``IndexError: list index out of range`` at ``llm_result.generations[0][0].message`` and kills the run. Treat a bare IndexError reaching the middleware as a transient upstream-payload glitch and route it through the existing retry/backoff path instead of failing the whole agent run. The retry budget and backoff schedule are unchanged. Adds three regression tests covering the classifier and both the recover-on-retry and exhausted-retries paths. * fix(runtime): ignore stale LLM fallback markers from prior runs When a run on a thread ends with the LLM-error-handling middleware emitting a `deerflow_error_fallback`-marked AIMessage (e.g. after the IndexError empty-generations classification fix lands), that message is persisted to the thread's checkpoint as part of the messages channel. LangGraph replays the full message history in `stream_mode="values"` chunks, so every subsequent run on the same thread re-streams the stale fallback marker — and the worker's chunk scanner faithfully picks it up, flipping `RunStatus.success` to `RunStatus.error` for runs that themselves had no LLM failure at all. Snapshot the set of pre-existing message ids from the pre-run checkpoint and thread it through `_extract_llm_error_fallback_message` / `_try_extract_from_message` as a filter. Markers on history messages are ignored; markers on fresh messages produced during this run still trip the error path. Falls back to an empty set when the checkpointer is absent or the snapshot can't be captured, preserving the prior behavior on first-run / no-state paths. Adds unit tests for the new filter (helper-level and `_collect_pre_existing_message_ids`) plus an integration test exercising the full `run_agent` path with a stale history checkpointer. * fix(channels): make github channel fire-and-forget to avoid httpx.ReadTimeout on long runs GitHub agent runs (clone -> edit -> test -> push -> PR) routinely exceed the langgraph_sdk default 300s read deadline. The manager's runs.wait call kept an HTTP stream open for the entire run lifetime, so the long run blew up with httpx.ReadTimeout and the outer except branch then released the dedupe key and emitted a false 'internal error' outbound. The GitHub channel's outbound send is log-only by design: agents post to the issue/PR via the gh CLI in the sandbox when they choose to comment or create a PR. There is nothing for the manager to ferry back, so the long-poll was pure overhead. This change adds ChannelRunPolicy.fire_and_forget (default False) and sets it True for the github channel. When fire_and_forget is True, _handle_chat dispatches via client.runs.create (short POST, returns once the run is pending) instead of client.runs.wait, and skips the response-extraction + outbound-publish block. ConflictError on a busy thread still trips the standard THREAD_BUSY_MESSAGE path so behavior on the busy case is preserved for any future non-github fire-and-forget channel. Other (non-github) channels are unchanged: their policy defaults fire_and_forget=False and they continue to dispatch via runs.wait. Adds 6 regression tests in tests/test_channels.py::TestGithubFireAndForget: - Default ChannelRunPolicy.fire_and_forget is False. - The github policy registers fire_and_forget=True. - github inbound calls runs.create, not runs.wait, with the right kwargs. - github inbound publishes no outbound on success. - ConflictError from runs.create still emits THREAD_BUSY_MESSAGE. - Non-github channels (slack) still dispatch via runs.wait. * test(lead-agent): accept user_id kwarg in skill-policy test stubs The two GitHub-channel tests added in #3754 stubbed _load_enabled_skills_for_tool_policy with a lambda that only accepted `available_skills` and `app_config`, but the real function (and its call site in agent.py) also passes `user_id`. This raised TypeError on every run, failing backend-unit-tests. Add `user_id=None` to match the three sibling stubs in the same file. * refactor(gateway): disambiguate context-key set names The two frozensets _INTERNAL_ONLY_CONTEXT_KEYS and _CONTEXT_ONLY_KEYS shared a confusable "CONTEXT_ONLY" token in different orders, and the first broke the _CONTEXT_<X>_KEYS pattern of its sibling _CONTEXT_CONFIGURABLE_KEYS. Rename to make the distinct axes explicit: _CONTEXT_INTERNAL_CALLER_KEYS - WHO: internal callers (scheduler) only _CONTEXT_RUNTIME_ONLY_KEYS - WHERE: runtime context only, never configurable Pure rename, no behavior change.	2026-07-04 22:56:24 +08:00
Xinmin Zeng	4fc08b4f15	feat: add scheduled tasks MVP (#3898) ... * feat: add scheduled tasks MVP * fix: harden scheduled task execution semantics * feat(scheduled-tasks): preset-driven schedule form with timezone and live preview Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom) plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local for one-time tasks, a timezone selector defaulting to the browser timezone, and a live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers). * feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin Localize the rest of the scheduled-tasks page (filters, detail pane, actions, edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four built-in recipe templates (GitHub Trending, news digest, issue triage, weekly report) exposed as a chip row that pre-fills title + prompt + schedule. Pin Playwright locale to en-US so E2E selectors stay stable against i18n. No backend change, no new deps. * fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test Merge with main surfaced three CI failures: - 0003_scheduled_tasks create_table collided with legacy test seeds that build from full metadata; guard with inspector.has_table so the revision no-ops when the table already exists (0004/0005 are already idempotent via _helpers.py). - persistence bootstrap concurrency/regression tests pinned HEAD to main's 0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable. - once-task router test used a fixed past run_at and tripped the must-be-in-the-future validation; use a future date. * address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences - fetchThreadScheduledTasks now checks response.ok like the other fetchers. - trigger endpoint returns 502 (not 409) when dispatch fails outright, so clients can distinguish a real conflict from a server-side failure. - E2E mock normalizes scheduled-task objects with context_mode/last_thread_id and nullable thread_id, matching the backend contract the UI renders against. - Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its revision id (file was renamed in spirit already; filename now follows). - CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence after the Scheduler notes so the sections render correctly. * fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review * fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review - defer a once task's terminal status to the run-completion hook; the task stays running until the real outcome, and a startup sweep cancels once tasks orphaned by a crash (launch-time 'completed' could stick forever) - record interrupted runs as a distinct 'interrupted' run status with a readable message; an interrupted once task ends 'cancelled', not 'failed' - enforce overlap_policy=skip for fresh_thread_per_run via an active-run pre-check (same-thread ConflictError can never fire across fresh threads) - protect terminal run statuses from the late launch-path 'running' write - honor context.non_interactive only for internally-authenticated callers; arbitrary clients can no longer strip ask_clarification - fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the offset at the resolved instant (once tasks fired an hour late around spring-forward and the create->edit round-trip diverged) - drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway API error helper between channels and scheduled-tasks frontends * fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics - scrub internal-only context keys (non_interactive) from the assembled run config for non-internal callers: gating body.context alone left the same key smuggle-able through the free-form body.config copied verbatim by build_run_config - guard update_after_launch with protect_terminal so the launch bookkeeping write cannot clobber a once task already finalized by a fast-failing run's completion hook (parent-row sibling of the run-row guard) - reject a manual trigger while the task has an active run (409) instead of launching a duplicate concurrent run on fresh_thread_per_run - re-arm a terminal once task to enabled when PATCH pushes run_at into the future; previously the endpoint returned 200 with a next_run_at that could never be claimed - make max_concurrent_runs a real global cap: each poll claims only into the remaining budget of active (queued/running) scheduled runs - paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the thread filter of /threads/{id}/scheduled-tasks into SQL - stamp context.user_id on scheduler-launched runs, matching IM channels, so user-scoped guardrail providers see the owning user --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>	2026-07-04 21:51:57 +08:00
Vanzeren	6060d95ee0	fix(wizard): update DeepSeek provider models to v4 (#3939) ... Update DeepSeek references from deprecated model names to the V4 lineup: - deepseek-reasoner → deepseek-v4-pro - deepseek-chat → deepseek-v4-flash Keep docs and frontend mocks aligned with the wizard provider list.	2026-07-04 21:44:22 +08:00
hataa	b85c672cc1	fix(channels): offload blocking filesystem IO in Wechat channel (#3925) ... WechatChannel made synchronous filesystem calls (mkdir, write_text, read_bytes, Path.replace, unlink) directly inside async entry points: _poll_loop, _bind_via_qrcode, _ensure_authenticated, _extract_image_file, _extract_file_item, start, _send_image_attachment, _send_file_attachment. Under slow disks, large files, or concurrent load these blocked the asyncio event loop and stalled the channel worker. Construction was also blocking: __init__ called _load_state() (os.stat + read_text) synchronously, and ChannelService._start_channel() instantiates the channel directly on the async path, so constructing WechatChannel in an async context raised BlockingError. Persisted state (auth token + cursor) is now loaded in start() via asyncio.to_thread, leaving __init__ IO-free. Offload each call to a thread via asyncio.to_thread, matching the existing pattern in channels/manager.py and dingtalk.py. The sync helpers (_save_state, _save_auth_state, _load_auth_state, _stage_downloaded_file) keep their signatures; only the async call sites wrap them. Adds tests/blocking_io/test_wechat_channel_state.py as a regression anchor covering the IO-free constructor (the production _start_channel path), the staging write path, and the auth-state read path. Detected by `make detect-blocking-io`.	2026-07-04 21:35:05 +08:00