vrr/deer-flow
2
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-07-09 16:08:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
deer-flow/AGENTS.md
Zheng Feng bc9ee9645c
Some checks are pending
Backend Blocking IO / backend-blocking-io (push) Waiting to run
Details
Unit Tests / backend-unit-tests (push) Waiting to run
Details
Frontend Unit Tests / frontend-unit-tests (push) Waiting to run
Details
Lint Check / lint-backend (push) Waiting to run
Details
Lint Check / lint-frontend (push) Waiting to run
Details
Replay E2E (front-back contract) / Layer 1 — backend golden (no API key) (push) Waiting to run
Details
Replay E2E (front-back contract) / Layer 2 — full-stack render (no API key) (push) Waiting to run
Details
feat(deploy): first-class Helm chart for Kubernetes deployment (#3987) 
* feat(helm): add production-ready Helm chart for Kubernetes deployment

Adds deploy/helm/deer-flow, a native-Kubernetes translation of the
production docker-compose stack, plus CI to publish its images and chart.

* ci(release): gate releases on version-source consistency

Add a reusable verify-versions workflow invoked by both chart.yaml and
container.yaml on v* tags. It runs scripts/verify_versions.sh against the
tag and fails the release — skipping all image and chart publishing — when
Chart.yaml (version + appVersion), backend/pyproject.toml, or
frontend/package.json don't all match the tag.

Add scripts/verify_versions.sh (the check, also runnable locally) and
scripts/bump_version.sh (bumps all four sources in lockstep, then
self-verifies). Document the release flow in RELEASING.md and link it from
AGENTS.md.

* fix(deploy): address Helm chart review feedback (#3987)

Three review items from willem-bd:

1. nginx IPv6 listen strip never matched. The sed pattern required a `;`
   immediately after `2026`, but the rendered config emits
   `listen [::]:2026 default_server;` (space + `default_server` before the
   `;`), so the line was never deleted and nginx crash-looped on pods
   without IPv6 (`socket() :::2026 failed (97: Address family not
   supported)`). Drop the trailing `;` from the pattern so it matches.
   Same latent bug fixed in docker-compose-dev.yaml.

2. Passwords were spliced into DSNs verbatim, so a password containing
   URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and
   a confusing parse error. Add a `deer-flow.urlEscape` helper
   (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral
   treats the replacement as a regex template so `[`/`]`/`?` break it) and
   apply it to the password in the postgres and redis DSNs. The raw
   `postgres-password` / `redis-password` keys stay unencoded - they back
   POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment.

3. NODE_HOST defaulted to "gateway", which can never route: the gateway
   Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a
   user who skips the caveat gets unreachable sandboxes with no error at
   install time. Default NODE_HOST to the provisioner pod's node IP via
   the downward API (status.hostIP) - a NodePort is exposed on every node,
   so <node-IP>:<NodePort> routes from the gateway on most clusters.
   `provisioner.nodeHost` remains an override for CNIs/policies that block
   pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart
   README. (#3929 remains the long-term fix - ClusterIP + cluster-DNS URL
   removes NODE_HOST and the NodePort exposure entirely.)

Validated with helm lint, helm template (incl. a special-char password
rendering the encoded DSNs), and a sed pattern-match check.

* fix(deploy): address round-2 Helm chart review feedback (#3987)

Three "Medium" items from willem-bd:

1. No helm lint / helm template gate before publish. A template regression
   ships as an immutable OCI artifact (GHCR won't overwrite --version), so
   gate packaging on `helm lint` + `helm template --include-crds` in
   chart.yaml before `helm package`. (ct lint / helm-unittest deferred.)

2. Action pinning inconsistent + PR body overstates it. SHA-pin
   actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance
   (v2.4.0, e8998f94) across the publishing workflows (chart.yaml,
   container.yaml, verify-versions.yml), matching the existing docker/*
   SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the
   "SHA-pinned actions" claim accurate. Other pre-existing workflows left
   untouched (out of scope for this PR).

3. Provisioner RBAC broader than needed. Dropped the unused update/patch
   verbs and the pods/exec + events rules from the provisioner Role -
   audited against docker/provisioner/app.py, which only calls
   get/create/delete on pods and get/list/create/delete on services. Fixed
   NOTES.txt to accurately describe the grant instead of understating it as
   "create Pods and Services". The remaining scope concern - verbs apply to
   all Pods in the namespace, not just sandbox Pods - is still deferred
   (RBAC can't scope by label; needs a dedicated namespace or admission
   control), now noted in NOTES.txt and README.

Validated with helm lint + helm template (narrowed Role renders with
exactly get/list/watch/create/delete).

* feat(helm): enable sandbox+web tools out of the box

The chart's default config loaded zero agent tools (config.tools empty ->
"Total tools loaded: 0"), so a fresh install gave an agent that could do
nothing useful. Add tool_groups + tools to the default config block:

- web: web_search (ddg), web_fetch (jina), image_search - no API key
- file:read: ls, read_file, glob, grep
- file:write: write_file, str_replace
- bash

The file/bash tools run inside the AIO sandbox the chart already
configures; the web tools need outbound internet from the gateway pod
(swap backends or drop entries for air-gapped clusters - see
config.example.yaml).

Also bump config_version 15 -> 19 to match config.example.yaml (the chart
had drifted behind). NOTES.txt and the README example updated to match.

* ci(helm): add chart validation + config_version drift check on PR

Extend the chart workflow with a PR-triggered validate-chart job that runs
helm lint, helm template --include-crds, and a config_version drift check:
it parses config_version from both config.example.yaml and the chart's
values.yaml and fails the build (with a ::error:: naming the files to bump)
if the chart is behind the example. This catches the kind of drift this
PR is fixing - the chart sat at v15 while the example moved to v19 - before
it can merge again.

verify-versions and publish-chart stay tag-only; publish-chart now
needs: [verify-versions, validate-chart]. validate-chart runs on both
PRs and tag pushes: the tag arm is required because a job that `needs`
a skipped job is itself skipped under the default success() check, so
validate-chart must actually run on tag pushes or publish-chart would
never fire.

* Bump config version to 20
2026-07-09 15:40:53 +08:00

129 lines
7.4 KiB
Markdown
Raw Permalink Blame History

# AGENTS.md
This file provides guidance to AI coding agents (Claude Code, Codex, and others) when working with code in this repository. It is the source of truth; the sibling `CLAUDE.md` imports it via `@AGENTS.md`.
It is the **monorepo orientation layer**: it maps the whole repo and points to the
module guides that own the depth. For anything inside a module, read that module's
guide rather than expecting full detail here:
- **[backend/AGENTS.md](backend/AGENTS.md)** — backend depth: harness/app split, agent &
middleware chain, sandbox, MCP, skills, memory, IM channels, persistence/migrations,
config system, test layout.
- **[frontend/AGENTS.md](frontend/AGENTS.md)** — frontend depth: Next.js App Router layout,
thread/streaming data flow, code style, commands.
## What is DeerFlow
DeerFlow is a LangGraph-based AI super-agent system with a full-stack architecture. The
backend runs a "super agent" with sandboxed execution, persistent memory, subagent
delegation, and extensible tools (built-in, MCP, community), all per-thread isolated. The
frontend is a Next.js chat UI. External IM platforms (Feishu, Slack, Telegram, Discord,
DingTalk) bridge into the same agent through the Gateway.
## Service Topology
A single `make dev` / Docker stack runs four cooperating services:
| Service | Port | Role |
| --------------- | ------ | ------------------------------------------------------------------- |
| **Nginx** | `2026` | Unified reverse-proxy entry point — open this in the browser |
| **Gateway API** | `8001` | FastAPI REST API + embedded LangGraph-compatible agent runtime |
| **Frontend** | `3000` | Next.js web interface |
| **Provisioner** | `8002` | Optional — only when sandbox is configured for provisioner/K8s mode |
Nginx is the single public entry: it serves the frontend and proxies `/api/langgraph/*`
to the Gateway's LangGraph runtime, rewriting it to Gateway's native `/api/*` routes; all
other `/api/*` go straight to the Gateway REST routers. See
[backend/AGENTS.md](backend/AGENTS.md) for the runtime and router detail.
## Repository Map
```
deer-flow/
├── Makefile # Root orchestration: drives the full stack (dev/start/stop, docker, setup)
├── config.example.yaml # Template → copy to config.yaml (gitignored) at repo root
├── extensions_config.example.json # Template → copy to extensions_config.json (gitignored): MCP servers + skills
├── backend/ # Python backend — see backend/AGENTS.md
│ ├── Makefile # Per-module backend commands (dev, gateway, test, lint, migrate-rev)
│ ├── packages/harness/ # deerflow-harness package (import: deerflow.*) — agent framework
│ └── app/ # FastAPI Gateway + IM channels (import: app.*)
├── frontend/ # Next.js frontend (pnpm) — see frontend/AGENTS.md
├── docker/ # docker-compose files, nginx config, provisioner
├── skills/ # Agent skills: public/ (committed), custom/ (gitignored)
├── contracts/ # Cross-component JSON contracts (e.g. subagent status)
├── scripts/ # Root orchestration scripts invoked by the Makefile (check, configure, doctor, support_bundle, serve, nginx, docker, deploy, setup_wizard)
├── tests/ # Root-level tests (currently tests/skills/ — public skill tests)
└── docs/ # Cross-cutting docs, plans, and design notes
```
Runtime config lives at the **repo root**: copy `config.example.yaml` → `config.yaml`
(main app config) and `extensions_config.example.json` → `extensions_config.json` (MCP
servers + skills). Both real files are gitignored and may be edited at runtime via the
Gateway API. Config schema and resolution order are documented in
[backend/AGENTS.md](backend/AGENTS.md).
Scheduled-task note:
- The scheduled-task MVP adds a workspace page at `/workspace/scheduled-tasks` plus a background scheduler service gated by `config.yaml -> scheduler.enabled`.
- Scheduled background runs are intentionally non-interactive: they execute through the normal run lifecycle, but the lead-agent toolset excludes `ask_clarification` when `context.non_interactive=true`. The key is honored only for internally-authenticated callers (the scheduler launch path); client-supplied `context.non_interactive` is dropped.
## Commands: Root vs. Module
**Root `make` targets drive the whole stack** (run from the repo root):
```bash
make setup # Interactive setup wizard (recommended for new users)
make doctor # Check configuration and system requirements
make support-bundle # Generate redacted troubleshooting summary, AI issue draft, and optional zip
make config # Generate local config files from the examples
make check # Check that required tools are installed
make install # Install all dependencies (frontend + backend + pre-commit hooks)
make dev # Start all services with hot-reload (Gateway + Frontend + Nginx)
make start # Start all services in production mode (local, optimized)
make stop # Stop all running services
make up / down # Build/stop the production Docker stack (browser at localhost:2026)
make docker-start / docker-stop / docker-logs # Docker development environment
```
Run `make help` for the full list.
**Per-module commands drive a single module** (run inside that module):
```bash
# Backend (see backend/AGENTS.md for the full set)
cd backend && make dev # Gateway API with reload (port 8001)
cd backend && make test # Backend test suite
cd backend && make lint # ruff check
cd backend && make format # ruff format
# Frontend (see frontend/AGENTS.md for the full set)
cd frontend && pnpm dev # Dev server with Turbopack (port 3000)
cd frontend && pnpm check # Lint + type check (run before committing)
cd frontend && pnpm test # Unit tests
```
Rule of thumb: **root `make` = the full application**; **`backend/Makefile` and `frontend/`
(`pnpm`) = per-module work.**
## Where to Go Next
- Backend work → **[backend/AGENTS.md](backend/AGENTS.md)**
- Frontend work → **[frontend/AGENTS.md](frontend/AGENTS.md)**
- Setup & install → **[Install.md](Install.md)**, **[CONTRIBUTING.md](CONTRIBUTING.md)**
- Project overview & usage → **[README.md](README.md)** (translations: `README_zh.md`,
`README_ja.md`, `README_fr.md`, `README_ru.md`)
- Security policy → **[SECURITY.md](SECURITY.md)**
- Changes → **[CHANGELOG.md](CHANGELOG.md)**
- Cutting a release → **[RELEASING.md](RELEASING.md)**
## Cross-Cutting Conventions
These apply repo-wide; module guides own the module-specific detail.
- **Documentation update policy** — keep docs in sync with code: update `README.md` for
user-facing changes and the relevant `AGENTS.md` for development/architecture changes in
the same change set.
- **Test-driven development** — features and bug fixes ship with tests. Backend tests live
in `backend/tests/` (TDD is mandatory there; see [backend/AGENTS.md](backend/AGENTS.md));
frontend tests live in `frontend/tests/`.
- **Format before pushing** — run `make format` (backend) / `pnpm check` (frontend). Backend
CI enforces `ruff format --check`, so formatting must be clean before a push.
Reference in a new issue View git blame Copy permalink