deer-flow/backend/docs
Ryker_Feng ebc09ce130
feat(mcp): auto-promote deferred MCP tools from routing hints (#4019)
* feat(mcp): auto-promote deferred MCP tools from routing hints

When tool_search.enabled=true defers MCP tool schemas, PR1 routing hints
still require the model to spend a tool_search discovery round trip before
it can call the tool the routing metadata already points at. This adds a
McpRoutingMiddleware that matches the latest user message against PR1
routing keywords and promotes the matching deferred schemas before the
model call, removing that round trip.

Design (soft routing, opt-in, additive):
- Matches only the latest real HumanMessage (shared is_real_user_message
  helper, reused by SkillActivationMiddleware so the two cannot drift);
  case-insensitive substring match, no tokenizer dependency.
- Ordering: priority desc, then tool name asc; capped by the new global
  tool_search.auto_promote_top_k (default 3, clamped 1..5). Does not add or
  consume a per-tool auto_promote_top_k (PR1 schema unchanged); a per-tool
  value is ignored with a DEBUG note.
- Returns a plain {"promoted": ...} state update (not a Command) and relies
  on ThreadState.merge_promoted for union/dedupe, so auto-promote and a
  model-triggered tool_search converge on the same catalog hash.
- Installed before DeferredToolFilterMiddleware on every deferred-tool path
  (lead agent, subagent, embedded client, webhook via shared builders);
  a construction-time assert rejects the reversed order. catalog_hash is
  None / no routing index is a complete no-op, so bootstrap and ACP skip it.
- Privacy: never executes tools, never promotes policy-filtered tools, adds
  no routing keywords or matched tool names to trace metadata or INFO/WARN
  logs.

No behavior change when tool_search.enabled=false.

Tests: index construction, matching semantics, middleware state updates,
same-cycle deferred-filter interaction, lead/subagent/embedded-client
builder wiring + order invariant, config clamping, config.example.yaml
parseability, and privacy assertions.

* refactor(mcp): address auto-promote review nits

- executor: access app_config.tool_search.auto_promote_top_k directly to match
  the lead-agent and embedded-client paths (drop the over-defensive getattr that
  masked missing config); update the subagent test mock to carry tool_search.
- tool_search / mcp_routing_middleware: cross-reference the duplicated routing
  priority/keyword normalization between the builder and the middleware's
  defensive _normalize_index so they cannot silently drift.
- MCP_SERVER.md: document that auto-promote keyword matching is a case-insensitive
  substring test (not word-boundary), advising distinctive keywords.
2026-07-10 07:54:36 +08:00
..
API.md fix(gateway): clamp client-supplied recursion_limit to prevent runaway runs (#3903) 2026-07-02 11:38:21 +08:00
APPLE_CONTAINER.md Fix command syntax for container image pull (#1349) 2026-03-26 00:14:08 +08:00
ARCHITECTURE.md fix: add MCP tools cache reset endpoint (#3602) 2026-06-16 23:20:20 +08:00
AUTH_DESIGN.md feat(auth): add OIDC SSO support (#3506) 2026-06-21 15:47:53 +08:00
AUTH_TEST_DOCKER_GAP.md docs: clean gateway runtime transition remnants (#3334) 2026-06-02 10:03:28 +08:00
AUTH_TEST_PLAN.md docs: clean standalone LangGraph server remnants (#3301) 2026-05-29 11:36:45 +08:00
AUTH_UPGRADE.md docs: clean gateway runtime transition remnants (#3334) 2026-06-02 10:03:28 +08:00
AUTO_TITLE_GENERATION.md fix(title): avoid default LLM call before stream end (#3885) 2026-06-30 18:58:02 +08:00
BLOCKING_IO_DETECTION.md feat(skill): add blocking-io-guard — SOP skill for blocking-IO triage and runtime anchors (#3503) 2026-06-12 10:20:38 +08:00
CONFIGURATION.md feat(mcp): add MCP routing hints (#4004) 2026-07-09 16:26:31 +08:00
FILE_UPLOAD.md fix(uploads): enforce streaming upload limits in gateway (#2589) 2026-05-01 20:19:30 +08:00
GUARDRAILS.md fix(guardrails): propagate internal owner attribution to guardrail context (#3839) 2026-07-07 08:05:26 +08:00
IM_CHANNEL_CONNECTIONS.md fix(channels): harden runtime credential management APIs (#3581) 2026-06-18 10:45:33 +08:00
MCP_SERVER.md feat(mcp): auto-promote deferred MCP tools from routing hints (#4019) 2026-07-10 07:54:36 +08:00
memory-settings-sample.json feat: support manual add and edit for memory facts (#1538) 2026-03-29 23:53:23 +08:00
MEMORY_IMPROVEMENTS.md feat(memory): add memory.token_counting config to avoid tiktoken network dependency (#3429) (#3465) 2026-06-10 23:26:15 +08:00
MEMORY_IMPROVEMENTS_SUMMARY.md refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
MEMORY_SETTINGS_REVIEW.md feat: support manual add and edit for memory facts (#1538) 2026-03-29 23:53:23 +08:00
middleware-execution-flow.md feat(loop-detection): defer warning injection (#2752) 2026-05-21 14:36:07 +08:00
PATH_EXAMPLES.md refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
plan_mode_usage.md refactor(lead-agent): make build_middlewares public to drop the last cross-module private import (#3458) 2026-06-09 11:56:28 +08:00
README.md chore: add sandbox memory profiling tools (#3249) 2026-06-03 22:02:27 +08:00
REPLAY_E2E.md fix(replay-e2e): key fixtures by caller and conversation (#3453) 2026-06-09 21:58:31 +08:00
rfc-create-deerflow-agent.md feat: add create_deerflow_agent SDK entry point (Phase 1) (#1203) 2026-03-29 15:31:18 +08:00
rfc-extract-shared-modules.md refactor: extract shared skill installer and upload manager to harness (#1202) 2026-03-25 16:28:33 +08:00
rfc-grep-glob-tools.md feat(sandbox): add built-in grep and glob tools (#1784) 2026-04-03 16:03:06 +08:00
SANDBOX_MEMORY_PROFILING.md chore: add sandbox memory profiling tools (#3249) 2026-06-03 22:02:27 +08:00
SETUP.md fix(harness): resolve runtime paths from project root (#2642) 2026-05-01 22:19:50 +08:00
SSO.md feat(auth): add OIDC SSO support (#3506) 2026-06-21 15:47:53 +08:00
STREAMING.md fix(redis): stream retention recovery (#3933) 2026-07-04 16:07:42 +08:00
summarization.md feat: emit structured runtime metadata (follow-up#3887) (#3906) 2026-07-04 11:27:19 +08:00
task_tool_improvements.md refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
TITLE_GENERATION_IMPLEMENTATION.md fix: generate fallback title for interrupted first-turn runs (#3874) 2026-07-02 00:13:01 +08:00
TODO.md docs: clean standalone LangGraph server remnants (#3301) 2026-05-29 11:36:45 +08:00
TUI.md feat(runtime): implement goal continuations (#3858) 2026-07-03 08:49:33 +08:00

Documentation

This directory contains detailed documentation for the DeerFlow backend.

Document Description
ARCHITECTURE.md System architecture overview
API.md Complete API reference
AUTH_DESIGN.md User authentication, CSRF, and per-user isolation design
CONFIGURATION.md Configuration options
SETUP.md Quick setup guide

Feature Documentation

Document Description
STREAMING.md Token-level streaming design: Gateway vs DeerFlowClient paths, stream_mode semantics, per-id dedup
FILE_UPLOAD.md File upload functionality
PATH_EXAMPLES.md Path types and usage examples
SANDBOX_MEMORY_PROFILING.md Sandbox memory baseline and runtime comparison guide
summarization.md Context summarization feature
plan_mode_usage.md Plan mode with TodoList
AUTO_TITLE_GENERATION.md Automatic title generation

Development

Document Description
TODO.md Planned features and known issues

Getting Started

  1. New to DeerFlow? Start with SETUP.md for quick installation
  2. Configuring the system? See CONFIGURATION.md
  3. Understanding the architecture? Read ARCHITECTURE.md
  4. Building integrations? Check API.md for API reference

Document Organization

docs/
├── README.md                  # This file
├── ARCHITECTURE.md            # System architecture
├── API.md                     # API reference
├── AUTH_DESIGN.md             # User authentication and isolation design
├── CONFIGURATION.md           # Configuration guide
├── SETUP.md                   # Setup instructions
├── FILE_UPLOAD.md             # File upload feature
├── PATH_EXAMPLES.md           # Path usage examples
├── summarization.md           # Summarization feature
├── plan_mode_usage.md         # Plan mode feature
├── STREAMING.md               # Token-level streaming design
├── AUTO_TITLE_GENERATION.md   # Title generation
├── TITLE_GENERATION_IMPLEMENTATION.md  # Title implementation details
└── TODO.md                    # Roadmap and issues