vrr/deer-flow
2
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-07-09 16:08:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

feat(community): add GroundRoute web search + fetch engine (#3675)

Browse source 
* feat(groundroute): add GroundRoute community web_search + web_fetch tools

GroundRoute is a meta search layer over six engines (Serper, Brave, Exa,
Tavily, Firecrawl, Perplexity) with price-based routing and failover. This
adds a self-contained community engine module (httpx only, no new required
deps) mirroring community/brave + community/tavily:
- web_search: POST /v1/search, normalize to {title,url,snippet,source_engine}.
- web_fetch: fetch a URL via mode=page.
- unit tests covering normalization, auth, clamping, and graceful errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(groundroute): register GroundRoute search + fetch in wizard and config

Add GroundRoute to the setup wizard provider lists (SEARCH_PROVIDERS +
WEB_FETCH_PROVIDERS) and as commented web_search + web_fetch examples in
config.example.yaml, mirroring tavily/serper/brave so SEARCH_API can select it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(groundroute): apply repo ruff format (line-length 240)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(groundroute): add GroundRoute to tools docs and config reference

Adds GroundRoute as a web_search and web_fetch option in the en + zh
tools.mdx pages (new tab alongside Tavily/Brave/Exa/etc.) and documents
GROUNDROUTE_API_KEY in CONFIGURATION.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(groundroute): define empty groundroute extra for clean install

The docs install line 'uv add deerflow-harness[groundroute]' (mirroring the
tavily/exa/firecrawl pattern) referenced an undefined extra, which uv accepts
but warns about. GroundRoute needs no extra packages (httpx is a core dep), so
declare an empty 'groundroute' extra in deerflow-harness optional-dependencies
so the documented command resolves without a warning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(groundroute): per-tool api key + honor caller max_results (review)

Address maintainer review on PR #3675:
- _get_api_key(tool_name): web_fetch now reads the web_fetch config block's key
  instead of always web_search, so a flow that pairs GroundRoute fetch with a
  different search engine authenticates correctly. Mirrors serper/exa/firecrawl.
- web_search honors a caller-supplied max_results (sentinel default None),
  falling back to the configured value only when omitted, so the documented
  parameter is no longer silently discarded.
- warn-once is now keyed per tool. Tests cover both fixes (web_fetch key,
  agent max_results honored).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
jp0xz 2026-06-21 09:55:10 +02:00 • committed by GitHub
parent ee8ad1bc67
commit a6dd2876f0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 587 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

5
backend/docs/CONFIGURATION.md
View file

@ -234,8 +234,8 @@ tools:
```
**Built-in Tools**:
- `web_search` - Search the web (DuckDuckGo, Tavily, Brave, Exa, InfoQuest, Firecrawl, fastCRW)
- `web_fetch` - Fetch web pages (Jina AI, Exa, InfoQuest, Firecrawl, fastCRW)
- `web_search` - Search the web (DuckDuckGo, Tavily, Brave, Exa, InfoQuest, Firecrawl, fastCRW, GroundRoute)
- `web_fetch` - Fetch web pages (Jina AI, Exa, InfoQuest, Firecrawl, fastCRW, GroundRoute)
- `image_search` - Search for reference images (DuckDuckGo, InfoQuest, Serper)
- `ls` - List directory contents
- `read_file` - Read file contents
@ -416,6 +416,7 @@ models:
- `TAVILY_API_KEY` - Tavily search API key
- `BRAVE_SEARCH_API_KEY` - Brave Search API key
- `SERPER_API_KEY` - Serper (Google Search/Images API) key for `web_search` and `image_search`
- `GROUNDROUTE_API_KEY` - GroundRoute meta-search API key for `web_search` and `web_fetch` (routes across Serper, Brave, Exa, Tavily, Firecrawl, Perplexity with gain-share pricing)
- `DEER_FLOW_PROJECT_ROOT` - Project root for relative runtime paths
- `DEER_FLOW_CONFIG_PATH` - Custom config file path
- `DEER_FLOW_EXTENSIONS_CONFIG_PATH` - Custom extensions config file path

3
backend/packages/harness/deerflow/community/groundroute/__init__.py Normal file
View file

@ -0,0 +1,3 @@
from .tools import web_fetch_tool, web_search_tool
__all__ = ["web_fetch_tool", "web_search_tool"]

166
backend/packages/harness/deerflow/community/groundroute/tools.py Normal file
View file

@ -0,0 +1,166 @@
"""GroundRoute community web search + fetch tools.
GroundRoute (https://groundroute.ai) is a meta search layer: one API in front of
six search engines (Serper, Brave, Exa, Tavily, Firecrawl, Perplexity). It routes
each query to the cheapest engine that clears a quality bar and caches repeats, so
high-volume research runs keep working when one engine is down and pay no more than
going to a single engine direct. Pricing is gain-share: the caller keeps about half
of any cache savings.
This module is self-contained (httpx only, no GroundRoute SDK). The /v1/search
request and response mapping mirrors the GroundRoute MCP server and the verified
Langflow component:
results[] = {url, title, snippet, content, source_engine, published_at}
`web_search` returns a normalized JSON list of {title, url, snippet, source_engine}.
`web_fetch` reads one URL via GroundRoute mode=page and returns its extracted text.
"""
import json
import logging
import os
import httpx
from langchain.tools import tool
from deerflow.config import get_app_config
logger = logging.getLogger(__name__)
_GROUNDROUTE_ENDPOINT = "https://api.groundroute.ai/v1/search"
_DEFAULT_MAX_RESULTS = 5
# GroundRoute clamps max_results to 1-50 server-side; clamp here too to mirror it.
_MAX_RESULTS_CAP = 50
_TIMEOUT_S = 30.0
_FETCH_SNIPPET_LIMIT = 4096
# Warn at most once per tool ("web_search" / "web_fetch") about a missing key.
_api_key_warned: set[str] = set()
def _get_api_key(tool_name: str) -> str | None:
"""Resolve the GroundRoute key from a given tool's config block, then the env var.
`tool_name` is the config section to read (web_search vs web_fetch) so a flow that
runs GroundRoute for fetch but a different engine for search still reads the right
key. Mirrors serper/exa/firecrawl, which all take the tool name.
"""
config = get_app_config().get_tool_config(tool_name)
if config is not None:
api_key = (config.model_extra or {}).get("api_key")
if isinstance(api_key, str) and api_key.strip():
return api_key.strip()
return os.getenv("GROUNDROUTE_API_KEY")
def _coerce_max_results(value: object, *, default: int = _DEFAULT_MAX_RESULTS) -> int:
try:
coerced = int(value)
except (TypeError, ValueError):
logger.warning("Invalid GroundRoute max_results=%r; using default %s", value, default)
coerced = default
return max(1, min(coerced, _MAX_RESULTS_CAP))
def _missing_key_error(tool_name: str, **context: str) -> str:
if tool_name not in _api_key_warned:
_api_key_warned.add(tool_name)
logger.warning(
"GroundRoute API key is not set for '%s'. Set GROUNDROUTE_API_KEY in your environment or provide api_key in config.yaml. Get a free key at https://groundroute.ai/keys",
tool_name,
)
return json.dumps({"error": "GROUNDROUTE_API_KEY is not configured", **context}, ensure_ascii=False)
def _post_search(api_key: str, body: dict) -> dict:
with httpx.Client(timeout=_TIMEOUT_S) as client:
response = client.post(
_GROUNDROUTE_ENDPOINT,
json=body,
headers={"Authorization": f"Bearer {api_key}"},
)
response.raise_for_status()
return response.json()
@tool("web_search", parse_docstring=True)
def web_search_tool(query: str, max_results: int | None = None) -> str:
"""Search the web for information using GroundRoute.
GroundRoute routes the query across six search engines and returns the result
set from the engine it selected, with failover if one engine is unavailable.
Args:
query: Search keywords describing what you want to find. Be specific for better results.
max_results: Maximum number of search results to return. If omitted, uses the configured value (default 5). Clamped to 1-50.
"""
# Honor the caller-supplied max_results; fall back to config only when omitted.
if max_results is None:
config = get_app_config().get_tool_config("web_search")
if config is not None:
max_results = (config.model_extra or {}).get("max_results")
count = _DEFAULT_MAX_RESULTS if max_results is None else _coerce_max_results(max_results)
api_key = _get_api_key("web_search")
if not api_key:
return _missing_key_error("web_search", query=query)
try:
data = _post_search(api_key, {"query": query, "max_results": count})
except httpx.HTTPStatusError as e:
logger.error("GroundRoute API returned HTTP %s: %s", e.response.status_code, e.response.text)
return json.dumps(
{"error": f"GroundRoute API error: HTTP {e.response.status_code}", "query": query},
ensure_ascii=False,
)
except Exception as e:
logger.error("GroundRoute search failed: %s: %s", type(e).__name__, e)
return json.dumps({"error": str(e), "query": query}, ensure_ascii=False)
results = data.get("results") or []
if not results:
return json.dumps({"error": "No results found", "query": query}, ensure_ascii=False)
normalized_results = [
{
"title": r.get("title", ""),
"url": r.get("url", ""),
"snippet": r.get("snippet", ""),
"source_engine": r.get("source_engine", ""),
}
for r in results
]
return json.dumps(normalized_results, indent=2, ensure_ascii=False)
@tool("web_fetch", parse_docstring=True)
def web_fetch_tool(url: str) -> str:
"""Fetch the contents of a web page at a given URL via GroundRoute.
Only fetch EXACT URLs that have been provided directly by the user or have been returned in results from the web_search and web_fetch tools.
This tool can NOT access content that requires authentication, such as private Google Docs or pages behind login walls.
Do NOT add www. to URLs that do NOT have them.
URLs must include the schema: https://example.com is a valid URL while example.com is an invalid URL.
Args:
url: The URL to fetch the contents of.
"""
api_key = _get_api_key("web_fetch")
if not api_key:
return _missing_key_error("web_fetch", url=url)
try:
data = _post_search(api_key, {"query": url, "mode": "page", "max_results": 1})
except httpx.HTTPStatusError as e:
logger.error("GroundRoute fetch returned HTTP %s: %s", e.response.status_code, e.response.text)
return f"Error: GroundRoute API error: HTTP {e.response.status_code}"
except Exception as e:
logger.error("GroundRoute fetch failed: %s: %s", type(e).__name__, e)
return f"Error: {e}"
results = data.get("results") or []
if not results:
return "Error: No results found"
result = results[0]
content = result.get("content") or result.get("snippet") or ""
title = result.get("title", "")
return f"# {title}\n\n{content[:_FETCH_SNIPPET_LIMIT]}"

4
backend/packages/harness/pyproject.toml
View file

@ -40,6 +40,10 @@ dependencies = [
]
[project.optional-dependencies]
# GroundRoute needs no extra packages (httpx is already a core dependency). This
# empty extra exists so the documented `uv add 'deerflow-harness[groundroute]'`
# install command resolves cleanly without an "unknown extra" warning.
groundroute = []
ollama = ["langchain-ollama>=0.3.0"]
postgres = [
"asyncpg>=0.29",

330
backend/tests/test_groundroute_tools.py Normal file
View file

@ -0,0 +1,330 @@
"""Unit tests for the GroundRoute community web search + fetch tools."""
import json
from unittest.mock import MagicMock, patch
import httpx
import pytest
@pytest.fixture(autouse=True)
def reset_api_key_warned():
"""Reset the per-tool warning set before and after each test."""
import deerflow.community.groundroute.tools as gr_mod
gr_mod._api_key_warned = set()
yield
gr_mod._api_key_warned = set()
@pytest.fixture
def mock_config_with_key():
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
tool_config = MagicMock()
tool_config.model_extra = {"api_key": "test-gr-key", "max_results": 5}
mock.return_value.get_tool_config.return_value = tool_config
yield mock
@pytest.fixture
def mock_config_no_key():
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
tool_config = MagicMock()
tool_config.model_extra = {}
mock.return_value.get_tool_config.return_value = tool_config
yield mock
def _make_search_response(payload: dict) -> MagicMock:
mock_resp = MagicMock()
mock_resp.json.return_value = payload
mock_resp.raise_for_status = MagicMock()
return mock_resp
def _patch_post(mock_resp: MagicMock):
"""Patch httpx.Client so the context-managed .post returns mock_resp."""
patcher = patch("deerflow.community.groundroute.tools.httpx.Client")
mock_client_cls = patcher.start()
mock_client_cls.return_value.__enter__.return_value.post.return_value = mock_resp
return patcher, mock_client_cls
def _per_tool_config(**by_tool):
"""Build a get_app_config mock whose get_tool_config returns a distinct config per tool name."""
configs = {}
for tool_name, extra in by_tool.items():
cfg = MagicMock()
cfg.model_extra = extra
configs[tool_name] = cfg
app = MagicMock()
app.get_tool_config.side_effect = lambda name: configs.get(name)
return app
class TestGetApiKey:
def test_returns_config_key_when_present(self):
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
tool_config = MagicMock()
tool_config.model_extra = {"api_key": "from-config"}
mock.return_value.get_tool_config.return_value = tool_config
from deerflow.community.groundroute.tools import _get_api_key
assert _get_api_key("web_search") == "from-config"
def test_falls_back_to_env_when_config_key_empty(self):
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
tool_config = MagicMock()
tool_config.model_extra = {"api_key": " "}
mock.return_value.get_tool_config.return_value = tool_config
with patch.dict("os.environ", {"GROUNDROUTE_API_KEY": "env-key"}, clear=True):
from deerflow.community.groundroute.tools import _get_api_key
assert _get_api_key("web_search") == "env-key"
def test_returns_none_when_no_key_anywhere(self):
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
mock.return_value.get_tool_config.return_value = None
with patch.dict("os.environ", {}, clear=True):
from deerflow.community.groundroute.tools import _get_api_key
assert _get_api_key("web_search") is None
def test_reads_the_named_tools_config_block(self):
"""web_fetch must read the web_fetch block, not web_search (multi-engine flows)."""
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
mock.return_value = _per_tool_config(
web_search={"api_key": "search-key"},
web_fetch={"api_key": "fetch-key"},
)
from deerflow.community.groundroute.tools import _get_api_key
assert _get_api_key("web_search") == "search-key"
assert _get_api_key("web_fetch") == "fetch-key"
class TestWebSearchTool:
def test_basic_search_returns_normalized_list_with_source_engine(self, mock_config_with_key):
payload = {
"request_id": "r1",
"results": [
{"url": "https://ex.com/a", "title": "A", "snippet": "s1", "source_engine": "serper"},
{"url": "https://ex.com/b", "title": "B", "snippet": "s2", "source_engine": "exa"},
],
}
patcher, _ = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_search_tool
result = web_search_tool.invoke({"query": "vector databases"})
parsed = json.loads(result)
finally:
patcher.stop()
assert isinstance(parsed, list)
assert len(parsed) == 2
assert parsed[0] == {
"title": "A",
"url": "https://ex.com/a",
"snippet": "s1",
"source_engine": "serper",
}
assert {r["source_engine"] for r in parsed} == {"serper", "exa"}
def test_uses_web_search_config_key(self):
"""web_search authenticates with the web_search config block's key."""
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
mock.return_value = _per_tool_config(
web_search={"api_key": "search-key"},
web_fetch={"api_key": "fetch-key"},
)
payload = {"results": [{"url": "u", "title": "t", "snippet": "s", "source_engine": "exa"}]}
patcher, mock_client_cls = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_search_tool
web_search_tool.invoke({"query": "hello world"})
call = mock_client_cls.return_value.__enter__.return_value.post.call_args
finally:
patcher.stop()
assert call.kwargs["headers"]["Authorization"] == "Bearer search-key"
assert call.kwargs["json"]["query"] == "hello world"
def test_agent_max_results_is_honored_over_config(self):
"""A caller-supplied max_results wins over the config value (not silently discarded)."""
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
tool_config = MagicMock()
tool_config.model_extra = {"api_key": "k", "max_results": 5}
mock.return_value.get_tool_config.return_value = tool_config
payload = {"results": [{"url": "u", "title": "t", "snippet": "s", "source_engine": "exa"}]}
patcher, mock_client_cls = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_search_tool
web_search_tool.invoke({"query": "test", "max_results": 20})
body = mock_client_cls.return_value.__enter__.return_value.post.call_args.kwargs["json"]
finally:
patcher.stop()
assert body["max_results"] == 20
def test_config_max_results_used_when_caller_omits(self, mock_config_with_key):
"""When the caller omits max_results, the configured value is used."""
payload = {"results": [{"url": "u", "title": "t", "snippet": "s", "source_engine": "exa"}]}
patcher, mock_client_cls = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_search_tool
web_search_tool.invoke({"query": "test"})
body = mock_client_cls.return_value.__enter__.return_value.post.call_args.kwargs["json"]
finally:
patcher.stop()
assert body["max_results"] == 5
def test_max_results_clamped_to_cap(self):
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
tool_config = MagicMock()
tool_config.model_extra = {"api_key": "k", "max_results": "500"}
mock.return_value.get_tool_config.return_value = tool_config
payload = {"results": [{"url": "u", "title": "t", "snippet": "s", "source_engine": "exa"}]}
patcher, mock_client_cls = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_search_tool
web_search_tool.invoke({"query": "test"})
body = mock_client_cls.return_value.__enter__.return_value.post.call_args.kwargs["json"]
finally:
patcher.stop()
assert body["max_results"] == 50
def test_empty_results_returns_error_json(self, mock_config_with_key):
patcher, _ = _patch_post(_make_search_response({"results": []}))
try:
from deerflow.community.groundroute.tools import web_search_tool
parsed = json.loads(web_search_tool.invoke({"query": "no results"}))
finally:
patcher.stop()
assert parsed["error"] == "No results found"
assert parsed["query"] == "no results"
def test_missing_api_key_returns_error_json(self, mock_config_no_key):
with patch.dict("os.environ", {}, clear=True):
from deerflow.community.groundroute.tools import web_search_tool
parsed = json.loads(web_search_tool.invoke({"query": "test"}))
assert "error" in parsed
assert "GROUNDROUTE_API_KEY" in parsed["error"]
def test_missing_api_key_logs_warning_once(self, mock_config_no_key, caplog):
import logging
with patch.dict("os.environ", {}, clear=True):
from deerflow.community.groundroute.tools import web_search_tool
with caplog.at_level(logging.WARNING, logger="deerflow.community.groundroute.tools"):
web_search_tool.invoke({"query": "q1"})
web_search_tool.invoke({"query": "q2"})
warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
assert len(warnings) == 1
def test_http_error_returns_structured_error(self, mock_config_with_key):
mock_resp = MagicMock()
mock_resp.raise_for_status.side_effect = httpx.HTTPStatusError("402", request=MagicMock(), response=MagicMock(status_code=402, text="Payment Required"))
patcher, _ = _patch_post(mock_resp)
try:
from deerflow.community.groundroute.tools import web_search_tool
parsed = json.loads(web_search_tool.invoke({"query": "test"}))
finally:
patcher.stop()
assert "error" in parsed
assert "402" in parsed["error"]
def test_network_exception_returns_error_json(self, mock_config_with_key):
patcher, mock_client_cls = _patch_post(MagicMock())
mock_client_cls.return_value.__enter__.return_value.post.side_effect = Exception("timeout")
try:
from deerflow.community.groundroute.tools import web_search_tool
parsed = json.loads(web_search_tool.invoke({"query": "test"}))
finally:
patcher.stop()
assert "error" in parsed
def test_partial_fields_default_to_empty_string(self, mock_config_with_key):
patcher, _ = _patch_post(_make_search_response({"results": [{}]}))
try:
from deerflow.community.groundroute.tools import web_search_tool
parsed = json.loads(web_search_tool.invoke({"query": "test"}))
finally:
patcher.stop()
assert parsed[0] == {"title": "", "url": "", "snippet": "", "source_engine": ""}
class TestWebFetchTool:
def test_fetch_returns_titled_content(self, mock_config_with_key):
payload = {"results": [{"title": "Page", "content": "Body text", "url": "https://ex.com"}]}
patcher, mock_client_cls = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_fetch_tool
result = web_fetch_tool.invoke({"url": "https://ex.com"})
body = mock_client_cls.return_value.__enter__.return_value.post.call_args.kwargs["json"]
finally:
patcher.stop()
assert result == "# Page\n\nBody text"
# web_fetch uses mode=page with the URL as the query.
assert body["mode"] == "page"
assert body["query"] == "https://ex.com"
def test_fetch_uses_web_fetch_config_key(self):
"""web_fetch must authenticate with the web_fetch config block's key, not web_search's."""
with patch("deerflow.community.groundroute.tools.get_app_config") as mock:
mock.return_value = _per_tool_config(
web_search={"api_key": "search-key"},
web_fetch={"api_key": "fetch-key"},
)
payload = {"results": [{"title": "P", "content": "b", "url": "https://ex.com"}]}
patcher, mock_client_cls = _patch_post(_make_search_response(payload))
try:
from deerflow.community.groundroute.tools import web_fetch_tool
web_fetch_tool.invoke({"url": "https://ex.com"})
call = mock_client_cls.return_value.__enter__.return_value.post.call_args
finally:
patcher.stop()
assert call.kwargs["headers"]["Authorization"] == "Bearer fetch-key"
def test_fetch_missing_key_returns_error(self, mock_config_no_key):
with patch.dict("os.environ", {}, clear=True):
from deerflow.community.groundroute.tools import web_fetch_tool
parsed = json.loads(web_fetch_tool.invoke({"url": "https://ex.com"}))
assert "error" in parsed
assert "GROUNDROUTE_API_KEY" in parsed["error"]
def test_fetch_no_results_returns_error_string(self, mock_config_with_key):
patcher, _ = _patch_post(_make_search_response({"results": []}))
try:
from deerflow.community.groundroute.tools import web_fetch_tool
result = web_fetch_tool.invoke({"url": "https://ex.com"})
finally:
patcher.stop()
assert result == "Error: No results found"

21
config.example.yaml
View file

@ -570,6 +570,18 @@ tools:
# max_results: 5
# # api_key: $FIRECRAWL_API_KEY
# Web search tool (uses GroundRoute, requires GROUNDROUTE_API_KEY)
# GroundRoute is a meta search layer: one API in front of six engines (Serper,
# Brave, Exa, Tavily, Firecrawl, Perplexity). It routes each query to the cheapest
# engine that clears a quality bar and fails over if one is down. Pricing is
# gain-share (you keep about half of any cache savings). Get a key at
# https://groundroute.ai/keys
# - name: web_search
# group: web
# use: deerflow.community.groundroute.tools:web_search_tool
# max_results: 5 # Clamped to 1-50 by GroundRoute
# # api_key: $GROUNDROUTE_API_KEY # Optional if the env var is set
# Web search tool (uses fastCRW - Firecrawl-compatible web scraper, single binary,
# self-host or cloud. Cloud requires CRW_API_KEY; self-host may need no key.)
# - name: web_search
@ -630,6 +642,15 @@ tools:
# use: deerflow.community.firecrawl.tools:web_fetch_tool
# # api_key: $FIRECRAWL_API_KEY
# Web fetch tool (uses GroundRoute, requires GROUNDROUTE_API_KEY)
# Fetches a page's extracted text via GroundRoute mode=page.
# NOTE: Only one web_fetch provider can be active at a time.
# Comment out the Jina AI web_fetch entry above before enabling this one.
# - name: web_fetch
# group: web
# use: deerflow.community.groundroute.tools:web_fetch_tool
# # api_key: $GROUNDROUTE_API_KEY
# Web fetch tool (uses fastCRW - Firecrawl-compatible web scraper, single binary,
# self-host or cloud. Cloud requires CRW_API_KEY; self-host may need no key.)
# NOTE: Only one web_fetch provider can be active at a time.

24
frontend/src/content/en/harness/tools.mdx
View file

@ -113,7 +113,7 @@ Community tools connect the agent to external services. They are configured in `
### Web search
<Tabs items={["DuckDuckGo (default)", "Tavily", "Brave", "Exa", "InfoQuest", "Firecrawl"]}>
<Tabs items={["DuckDuckGo (default)", "Tavily", "Brave", "Exa", "InfoQuest", "Firecrawl", "GroundRoute"]}>
<Tabs.Tab>
```yaml
tools:
@ -172,12 +172,25 @@ Firecrawl-powered search and crawl. Requires a [Firecrawl](https://firecrawl.dev
Install: `cd backend && uv add 'deerflow-harness[firecrawl]'`
</Tabs.Tab>
<Tabs.Tab>
```yaml
tools:
- use: deerflow.community.groundroute.tools:web_search_tool
api_key: $GROUNDROUTE_API_KEY
```
GroundRoute is a meta search layer: one API in front of six engines (Serper, Brave, Exa, Tavily, Firecrawl, Perplexity). Routes each query to the cheapest engine that clears a quality bar and fails over if one is down. Pricing is gain-share, you keep about half of any cache savings.
Requires a [GroundRoute](https://groundroute.ai) API key. Also supports `web_fetch`.
Install: `cd backend && uv add 'deerflow-harness[groundroute]'`
</Tabs.Tab>
</Tabs>
### Web fetch (page content extraction)
<Tabs items={["Jina AI (default)", "Exa", "InfoQuest", "Firecrawl"]}>
<Tabs items={["Jina AI (default)", "Exa", "InfoQuest", "Firecrawl", "GroundRoute"]}>
<Tabs.Tab>
```yaml
tools:
@ -206,6 +219,13 @@ tools:
tools:
- use: deerflow.community.firecrawl.tools:web_fetch_tool
api_key: $FIRECRAWL_API_KEY
```
</Tabs.Tab>
<Tabs.Tab>
```yaml
tools:
- use: deerflow.community.groundroute.tools:web_fetch_tool
api_key: $GROUNDROUTE_API_KEY
```
</Tabs.Tab>
</Tabs>

24
frontend/src/content/zh/harness/tools.mdx
View file

@ -110,7 +110,7 @@ tools:
### 网络搜索
<Tabs items={["DuckDuckGo（默认）", "Tavily", "Brave", "Exa", "InfoQuest", "Firecrawl"]}>
<Tabs items={["DuckDuckGo（默认）", "Tavily", "Brave", "Exa", "InfoQuest", "Firecrawl", "GroundRoute"]}>
<Tabs.Tab>
```yaml
tools:
@ -167,11 +167,24 @@ tools:
```
Firecrawl 驱动的搜索和爬取。需要 [Firecrawl](https://firecrawl.dev) API Key。
</Tabs.Tab>
<Tabs.Tab>
```yaml
tools:
- use: deerflow.community.groundroute.tools:web_search_tool
api_key: $GROUNDROUTE_API_KEY
```
GroundRoute 是一个元搜索层：一个 API 接入六个搜索引擎（Serper、Brave、Exa、Tavily、Firecrawl、Perplexity）。为每个查询路由到满足质量门槛的最低成本引擎，并在引擎故障时自动切换。定价采用收益分成模式，您可保留约一半的缓存节省。
需要 [GroundRoute](https://groundroute.ai) API Key。同时支持 `web_fetch`。
安装：`cd backend && uv add 'deerflow-harness[groundroute]'`
</Tabs.Tab>
</Tabs>
### 网页内容抓取
<Tabs items={["Jina AI（默认）", "Exa"]}>
<Tabs items={["Jina AI（默认）", "Exa", "GroundRoute"]}>
<Tabs.Tab>
```yaml
tools:
@ -185,6 +198,13 @@ tools:
tools:
- use: deerflow.community.exa.tools:web_fetch_tool
api_key: $EXA_API_KEY
```
</Tabs.Tab>
<Tabs.Tab>
```yaml
tools:
- use: deerflow.community.groundroute.tools:web_fetch_tool
api_key: $GROUNDROUTE_API_KEY
```
</Tabs.Tab>
</Tabs>

16
scripts/wizard/providers.py
View file

@ -526,6 +526,14 @@ SEARCH_PROVIDERS: list[SearchProvider] = [
env_var="BRAVE_SEARCH_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="groundroute",
display_name="GroundRoute",
description="One key across six engines, price-routed with failover, API key required",
use="deerflow.community.groundroute.tools:web_search_tool",
env_var="GROUNDROUTE_API_KEY",
extra_config={"max_results": 5},
),
]
WEB_FETCH_PROVIDERS: list[WebProvider] = [
@ -563,6 +571,14 @@ WEB_FETCH_PROVIDERS: list[WebProvider] = [
env_var="FIRECRAWL_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="groundroute",
display_name="GroundRoute",
description="Page fetch via routed engines, API key required",
use="deerflow.community.groundroute.tools:web_fetch_tool",
env_var="GROUNDROUTE_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="fastcrw",
display_name="fastCRW",