mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-09 16:08:31 +00:00
* feat(channels): add GitHub event-driven agents (#3754) Add a webhook-driven GitHub channel with fail-closed webhook routing, deterministic per-agent PR/issue threads, mention-gated trigger fan-out, GitHub App token injection for sandboxed gh/git commands, and backend/AGENTS.md documentation. * fix(llm-middleware): classify bare IndexError as transient Upstream chat providers occasionally return 200 OK with an empty generations list (observed against Volces "coding" on ark.cn-beijing.volces.com). When that happens, langchain_core.language_models.chat_models.ainvoke raises ``IndexError: list index out of range`` at ``llm_result.generations[0][0].message`` and kills the run. Treat a bare IndexError reaching the middleware as a transient upstream-payload glitch and route it through the existing retry/backoff path instead of failing the whole agent run. The retry budget and backoff schedule are unchanged. Adds three regression tests covering the classifier and both the recover-on-retry and exhausted-retries paths. * fix(runtime): ignore stale LLM fallback markers from prior runs When a run on a thread ends with the LLM-error-handling middleware emitting a `deerflow_error_fallback`-marked AIMessage (e.g. after the IndexError empty-generations classification fix lands), that message is persisted to the thread's checkpoint as part of the messages channel. LangGraph replays the full message history in `stream_mode="values"` chunks, so every subsequent run on the same thread re-streams the stale fallback marker — and the worker's chunk scanner faithfully picks it up, flipping `RunStatus.success` to `RunStatus.error` for runs that themselves had no LLM failure at all. Snapshot the set of pre-existing message ids from the pre-run checkpoint and thread it through `_extract_llm_error_fallback_message` / `_try_extract_from_message` as a filter. Markers on history messages are ignored; markers on fresh messages produced during this run still trip the error path. Falls back to an empty set when the checkpointer is absent or the snapshot can't be captured, preserving the prior behavior on first-run / no-state paths. Adds unit tests for the new filter (helper-level and `_collect_pre_existing_message_ids`) plus an integration test exercising the full `run_agent` path with a stale history checkpointer. * fix(channels): make github channel fire-and-forget to avoid httpx.ReadTimeout on long runs GitHub agent runs (clone -> edit -> test -> push -> PR) routinely exceed the langgraph_sdk default 300s read deadline. The manager's runs.wait call kept an HTTP stream open for the entire run lifetime, so the long run blew up with httpx.ReadTimeout and the outer except branch then released the dedupe key and emitted a false 'internal error' outbound. The GitHub channel's outbound send is log-only by design: agents post to the issue/PR via the gh CLI in the sandbox when they choose to comment or create a PR. There is nothing for the manager to ferry back, so the long-poll was pure overhead. This change adds ChannelRunPolicy.fire_and_forget (default False) and sets it True for the github channel. When fire_and_forget is True, _handle_chat dispatches via client.runs.create (short POST, returns once the run is pending) instead of client.runs.wait, and skips the response-extraction + outbound-publish block. ConflictError on a busy thread still trips the standard THREAD_BUSY_MESSAGE path so behavior on the busy case is preserved for any future non-github fire-and-forget channel. Other (non-github) channels are unchanged: their policy defaults fire_and_forget=False and they continue to dispatch via runs.wait. Adds 6 regression tests in tests/test_channels.py::TestGithubFireAndForget: - Default ChannelRunPolicy.fire_and_forget is False. - The github policy registers fire_and_forget=True. - github inbound calls runs.create, not runs.wait, with the right kwargs. - github inbound publishes no outbound on success. - ConflictError from runs.create still emits THREAD_BUSY_MESSAGE. - Non-github channels (slack) still dispatch via runs.wait. * test(lead-agent): accept user_id kwarg in skill-policy test stubs The two GitHub-channel tests added in #3754 stubbed _load_enabled_skills_for_tool_policy with a lambda that only accepted `available_skills` and `app_config`, but the real function (and its call site in agent.py) also passes `user_id`. This raised TypeError on every run, failing backend-unit-tests. Add `user_id=None` to match the three sibling stubs in the same file. * refactor(gateway): disambiguate context-key set names The two frozensets _INTERNAL_ONLY_CONTEXT_KEYS and _CONTEXT_ONLY_KEYS shared a confusable "CONTEXT_ONLY" token in different orders, and the first broke the _CONTEXT_<X>_KEYS pattern of its sibling _CONTEXT_CONFIGURABLE_KEYS. Rename to make the distinct axes explicit: _CONTEXT_INTERNAL_CALLER_KEYS - WHO: internal callers (scheduler) only _CONTEXT_RUNTIME_ONLY_KEYS - WHERE: runtime context only, never configurable Pure rename, no behavior change.
221 lines
7.8 KiB
Python
221 lines
7.8 KiB
Python
"""Tests for the GitHub binding block on :class:`AgentConfig`.
|
|
|
|
Verifies the new ``github:`` block parses correctly when present, is ``None``
|
|
when absent (so every existing agent continues to load unchanged), and that
|
|
``load_agent_config`` round-trips through YAML correctly.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
import yaml
|
|
|
|
from deerflow.config.agents_config import (
|
|
AgentConfig,
|
|
GitHubAgentConfig,
|
|
GitHubBinding,
|
|
GitHubTriggerConfig,
|
|
load_agent_config,
|
|
)
|
|
|
|
|
|
def test_github_field_defaults_to_none() -> None:
|
|
cfg = AgentConfig(name="solo")
|
|
assert cfg.github is None
|
|
|
|
|
|
def test_github_block_parses_full_shape() -> None:
|
|
cfg = AgentConfig(
|
|
name="coding-llm-gateway",
|
|
github={
|
|
"installation_id": 123456,
|
|
"bindings": [
|
|
{
|
|
"repo": "zhfeng/llm-gateway",
|
|
"triggers": {
|
|
"pull_request": {"actions": ["opened", "reopened"]},
|
|
"issue_comment": {
|
|
"require_mention": True,
|
|
"allow_authors": ["zhfeng"],
|
|
"mention_login": "coding-llm-gateway-bot",
|
|
},
|
|
},
|
|
}
|
|
],
|
|
},
|
|
)
|
|
assert isinstance(cfg.github, GitHubAgentConfig)
|
|
assert cfg.github.installation_id == 123456
|
|
assert len(cfg.github.bindings) == 1
|
|
binding = cfg.github.bindings[0]
|
|
assert isinstance(binding, GitHubBinding)
|
|
assert binding.repo == "zhfeng/llm-gateway"
|
|
assert set(binding.triggers.keys()) == {"pull_request", "issue_comment"}
|
|
pr = binding.triggers["pull_request"]
|
|
assert isinstance(pr, GitHubTriggerConfig)
|
|
assert pr.actions == ["opened", "reopened"]
|
|
assert pr.require_mention is False
|
|
assert pr.allow_authors == []
|
|
comment = binding.triggers["issue_comment"]
|
|
assert comment.require_mention is True
|
|
assert comment.allow_authors == ["zhfeng"]
|
|
assert comment.mention_login == "coding-llm-gateway-bot"
|
|
|
|
|
|
def test_github_block_minimal_uses_defaults() -> None:
|
|
cfg = AgentConfig(name="x", github={})
|
|
assert cfg.github is not None
|
|
assert cfg.github.installation_id is None
|
|
assert cfg.github.bindings == []
|
|
# recursion_limit defaults to None — the channel default (250) is
|
|
# applied later by ChannelManager._resolve_run_params, not here.
|
|
assert cfg.github.recursion_limit is None
|
|
|
|
|
|
def test_github_recursion_limit_parses() -> None:
|
|
cfg = AgentConfig(name="refactorer", github={"recursion_limit": 500})
|
|
assert cfg.github is not None
|
|
assert cfg.github.recursion_limit == 500
|
|
|
|
|
|
def test_github_trigger_invalid_type_raises() -> None:
|
|
with pytest.raises(Exception): # noqa: PT011 — pydantic ValidationError subclasses Exception
|
|
AgentConfig(
|
|
name="x",
|
|
github={"bindings": [{"repo": "a/b", "triggers": {"pull_request": "not-an-object"}}]},
|
|
)
|
|
|
|
|
|
def test_github_bindings_must_have_repo() -> None:
|
|
with pytest.raises(Exception): # noqa: PT011
|
|
AgentConfig(name="x", github={"bindings": [{"triggers": {}}]})
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# YAML round-trip via load_agent_config
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _write_agent(base: Path, user_id: str, name: str, body: dict) -> None:
|
|
agent_dir = base / "users" / user_id / "agents" / name
|
|
agent_dir.mkdir(parents=True, exist_ok=True)
|
|
(agent_dir / "config.yaml").write_text(yaml.safe_dump(body), encoding="utf-8")
|
|
|
|
|
|
def test_load_agent_config_reads_github_block(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
|
|
# Reset the singleton so the new HOME is picked up.
|
|
from deerflow.config import paths as paths_module
|
|
|
|
monkeypatch.setattr(paths_module, "_paths", None)
|
|
|
|
body = {
|
|
"name": "coding-llm-gateway",
|
|
"model": "claude-sonnet-4-6",
|
|
"github": {
|
|
"installation_id": 999,
|
|
"bindings": [
|
|
{
|
|
"repo": "zhfeng/llm-gateway",
|
|
"triggers": {
|
|
"pull_request": {"actions": ["opened"]},
|
|
},
|
|
}
|
|
],
|
|
},
|
|
}
|
|
_write_agent(tmp_path, "default", "coding-llm-gateway", body)
|
|
|
|
cfg = load_agent_config("coding-llm-gateway", user_id="default")
|
|
assert cfg is not None
|
|
assert cfg.github is not None
|
|
assert cfg.github.installation_id == 999
|
|
assert cfg.github.bindings[0].repo == "zhfeng/llm-gateway"
|
|
assert cfg.github.bindings[0].triggers["pull_request"].actions == ["opened"]
|
|
|
|
|
|
def test_load_agent_config_without_github_block_is_none(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
|
|
from deerflow.config import paths as paths_module
|
|
|
|
monkeypatch.setattr(paths_module, "_paths", None)
|
|
|
|
_write_agent(tmp_path, "default", "plain-agent", {"name": "plain-agent"})
|
|
cfg = load_agent_config("plain-agent", user_id="default")
|
|
assert cfg is not None
|
|
assert cfg.github is None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Single-binding-per-repo validator (PR feedback R3)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_duplicate_bindings_same_repo_rejected() -> None:
|
|
"""Two bindings on the same repo must fail validation.
|
|
|
|
Pre-R3 the dispatcher silently picked the FIRST binding for the repo,
|
|
so a second binding with a different ``triggers:`` map would never fire
|
|
its events. Fail loudly at config load instead.
|
|
"""
|
|
with pytest.raises(ValueError, match="duplicate repos"):
|
|
GitHubAgentConfig(
|
|
bindings=[
|
|
GitHubBinding(repo="a/b", triggers={"pull_request": GitHubTriggerConfig()}),
|
|
GitHubBinding(repo="a/b", triggers={"issue_comment": GitHubTriggerConfig()}),
|
|
],
|
|
)
|
|
|
|
|
|
def test_duplicate_bindings_error_lists_offending_repo() -> None:
|
|
"""The error mentions every duplicate repo so operators can locate them."""
|
|
with pytest.raises(ValueError) as excinfo:
|
|
GitHubAgentConfig(
|
|
bindings=[
|
|
GitHubBinding(repo="x/one"),
|
|
GitHubBinding(repo="x/two"),
|
|
GitHubBinding(repo="x/one"),
|
|
GitHubBinding(repo="x/two"),
|
|
],
|
|
)
|
|
msg = str(excinfo.value)
|
|
assert "x/one" in msg
|
|
assert "x/two" in msg
|
|
|
|
|
|
def test_distinct_repo_bindings_allowed() -> None:
|
|
"""One agent with bindings on different repos is fine (the common case)."""
|
|
cfg = GitHubAgentConfig(
|
|
bindings=[
|
|
GitHubBinding(repo="a/one"),
|
|
GitHubBinding(repo="a/two"),
|
|
GitHubBinding(repo="b/three"),
|
|
],
|
|
)
|
|
assert [b.repo for b in cfg.bindings] == ["a/one", "a/two", "b/three"]
|
|
|
|
|
|
def test_load_agent_config_rejects_duplicate_repo_bindings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""End-to-end: load_agent_config surfaces the validator error from YAML."""
|
|
monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path))
|
|
from deerflow.config import paths as paths_module
|
|
|
|
monkeypatch.setattr(paths_module, "_paths", None)
|
|
|
|
agent_dir = tmp_path / "users" / "default" / "agents" / "broken"
|
|
agent_dir.mkdir(parents=True)
|
|
body = {
|
|
"name": "broken",
|
|
"github": {
|
|
"bindings": [
|
|
{"repo": "a/b", "triggers": {"pull_request": {}}},
|
|
{"repo": "a/b", "triggers": {"issue_comment": {}}},
|
|
],
|
|
},
|
|
}
|
|
(agent_dir / "config.yaml").write_text(yaml.safe_dump(body), encoding="utf-8")
|
|
|
|
with pytest.raises(ValueError, match="duplicate repos"):
|
|
load_agent_config("broken", user_id="default")
|