mirror of https://github.com/bytedance/deer-flow.git synced 2026-07-09 16:08:31 +00:00

* feat: add scheduled tasks MVP

* fix: harden scheduled task execution semantics

* feat(scheduled-tasks): preset-driven schedule form with timezone and live preview

Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom)
plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local
for one-time tasks, a timezone selector defaulting to the browser timezone, and a
live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and
edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers).

* feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin

Localize the rest of the scheduled-tasks page (filters, detail pane, actions,
edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four
built-in recipe templates (GitHub Trending, news digest, issue triage, weekly
report) exposed as a chip row that pre-fills title + prompt + schedule. Pin
Playwright locale to en-US so E2E selectors stay stable against i18n. No backend
change, no new deps.

* fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test

Merge with main surfaced three CI failures:
- 0003_scheduled_tasks create_table collided with legacy test seeds that
  build from full metadata; guard with inspector.has_table so the revision
  no-ops when the table already exists (0004/0005 are already idempotent via
  _helpers.py).
- persistence bootstrap concurrency/regression tests pinned HEAD to main's
  0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable.
- once-task router test used a fixed past run_at and tripped the
  must-be-in-the-future validation; use a future date.

* address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences

- fetchThreadScheduledTasks now checks response.ok like the other fetchers.
- trigger endpoint returns 502 (not 409) when dispatch fails outright, so
  clients can distinguish a real conflict from a server-side failure.
- E2E mock normalizes scheduled-task objects with context_mode/last_thread_id
  and nullable thread_id, matching the backend contract the UI renders against.
- Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its
  revision id (file was renamed in spirit already; filename now follows).
- CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence
  after the Scheduler notes so the sections render correctly.

* fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review

* fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review

- defer a once task's terminal status to the run-completion hook; the task
  stays running until the real outcome, and a startup sweep cancels once
  tasks orphaned by a crash (launch-time 'completed' could stick forever)
- record interrupted runs as a distinct 'interrupted' run status with a
  readable message; an interrupted once task ends 'cancelled', not 'failed'
- enforce overlap_policy=skip for fresh_thread_per_run via an active-run
  pre-check (same-thread ConflictError can never fire across fresh threads)
- protect terminal run statuses from the late launch-path 'running' write
- honor context.non_interactive only for internally-authenticated callers;
  arbitrary clients can no longer strip ask_clarification
- fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the
  offset at the resolved instant (once tasks fired an hour late around
  spring-forward and the create->edit round-trip diverged)
- drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway
  API error helper between channels and scheduled-tasks frontends

* fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics

- scrub internal-only context keys (non_interactive) from the assembled run
  config for non-internal callers: gating body.context alone left the same
  key smuggle-able through the free-form body.config copied verbatim by
  build_run_config
- guard update_after_launch with protect_terminal so the launch bookkeeping
  write cannot clobber a once task already finalized by a fast-failing run's
  completion hook (parent-row sibling of the run-row guard)
- reject a manual trigger while the task has an active run (409) instead of
  launching a duplicate concurrent run on fresh_thread_per_run
- re-arm a terminal once task to enabled when PATCH pushes run_at into the
  future; previously the endpoint returned 200 with a next_run_at that could
  never be claimed
- make max_concurrent_runs a real global cap: each poll claims only into the
  remaining budget of active (queued/running) scheduled runs
- paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the
  thread filter of /threads/{id}/scheduled-tasks into SQL
- stamp context.user_id on scheduler-launched runs, matching IM channels, so
  user-scoped guardrail providers see the owning user

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

2026-07-04 21:51:57 +08:00

7.4 KiB

Raw Blame History

AGENTS.md

This file provides guidance to AI coding agents (Claude Code, Codex, and others) when working with code in this repository. It is the source of truth; the sibling CLAUDE.md imports it via @AGENTS.md.

It is the monorepo orientation layer: it maps the whole repo and points to the module guides that own the depth. For anything inside a module, read that module's guide rather than expecting full detail here:

backend/AGENTS.md — backend depth: harness/app split, agent & middleware chain, sandbox, MCP, skills, memory, IM channels, persistence/migrations, config system, test layout.
frontend/AGENTS.md — frontend depth: Next.js App Router layout, thread/streaming data flow, code style, commands.

What is DeerFlow

DeerFlow is a LangGraph-based AI super-agent system with a full-stack architecture. The backend runs a "super agent" with sandboxed execution, persistent memory, subagent delegation, and extensible tools (built-in, MCP, community), all per-thread isolated. The frontend is a Next.js chat UI. External IM platforms (Feishu, Slack, Telegram, Discord, DingTalk) bridge into the same agent through the Gateway.

Service Topology

A single make dev / Docker stack runs four cooperating services:

Service	Port	Role
Nginx	`2026`	Unified reverse-proxy entry point — open this in the browser
Gateway API	`8001`	FastAPI REST API + embedded LangGraph-compatible agent runtime
Frontend	`3000`	Next.js web interface
Provisioner	`8002`	Optional — only when sandbox is configured for provisioner/K8s mode

Nginx is the single public entry: it serves the frontend and proxies /api/langgraph/* to the Gateway's LangGraph runtime, rewriting it to Gateway's native /api/* routes; all other /api/* go straight to the Gateway REST routers. See backend/AGENTS.md for the runtime and router detail.

Repository Map

deer-flow/
├── Makefile                        # Root orchestration: drives the full stack (dev/start/stop, docker, setup)
├── config.example.yaml             # Template → copy to config.yaml (gitignored) at repo root
├── extensions_config.example.json  # Template → copy to extensions_config.json (gitignored): MCP servers + skills
├── backend/                        # Python backend — see backend/AGENTS.md
│   ├── Makefile                    # Per-module backend commands (dev, gateway, test, lint, migrate-rev)
│   ├── packages/harness/           # deerflow-harness package (import: deerflow.*) — agent framework
│   └── app/                        # FastAPI Gateway + IM channels (import: app.*)
├── frontend/                       # Next.js frontend (pnpm) — see frontend/AGENTS.md
├── docker/                         # docker-compose files, nginx config, provisioner
├── skills/                         # Agent skills: public/ (committed), custom/ (gitignored)
├── contracts/                      # Cross-component JSON contracts (e.g. subagent status)
├── scripts/                        # Root orchestration scripts invoked by the Makefile (check, configure, doctor, support_bundle, serve, nginx, docker, deploy, setup_wizard)
├── tests/                          # Root-level tests (currently tests/skills/ — public skill tests)
└── docs/                           # Cross-cutting docs, plans, and design notes

Runtime config lives at the repo root: copy config.example.yaml → config.yaml (main app config) and extensions_config.example.json → extensions_config.json (MCP servers + skills). Both real files are gitignored and may be edited at runtime via the Gateway API. Config schema and resolution order are documented in backend/AGENTS.md.

Scheduled-task note:

The scheduled-task MVP adds a workspace page at /workspace/scheduled-tasks plus a background scheduler service gated by config.yaml -> scheduler.enabled.
Scheduled background runs are intentionally non-interactive: they execute through the normal run lifecycle, but the lead-agent toolset excludes ask_clarification when context.non_interactive=true. The key is honored only for internally-authenticated callers (the scheduler launch path); client-supplied context.non_interactive is dropped.

Commands: Root vs. Module

Root make targets drive the whole stack (run from the repo root):

make setup       # Interactive setup wizard (recommended for new users)
make doctor      # Check configuration and system requirements
make support-bundle  # Generate redacted troubleshooting summary, AI issue draft, and optional zip
make config      # Generate local config files from the examples
make check       # Check that required tools are installed
make install     # Install all dependencies (frontend + backend + pre-commit hooks)
make dev         # Start all services with hot-reload (Gateway + Frontend + Nginx)
make start       # Start all services in production mode (local, optimized)
make stop        # Stop all running services
make up / down   # Build/stop the production Docker stack (browser at localhost:2026)
make docker-start / docker-stop / docker-logs   # Docker development environment

Run make help for the full list.

Per-module commands drive a single module (run inside that module):

# Backend (see backend/AGENTS.md for the full set)
cd backend && make dev        # Gateway API with reload (port 8001)
cd backend && make test       # Backend test suite
cd backend && make lint       # ruff check
cd backend && make format     # ruff format

# Frontend (see frontend/AGENTS.md for the full set)
cd frontend && pnpm dev       # Dev server with Turbopack (port 3000)
cd frontend && pnpm check     # Lint + type check (run before committing)
cd frontend && pnpm test      # Unit tests

Rule of thumb: root make = the full application; backend/Makefile and frontend/ (pnpm) = per-module work.

Where to Go Next

Backend work → backend/AGENTS.md
Frontend work → frontend/AGENTS.md
Setup & install → Install.md, CONTRIBUTING.md
Project overview & usage → README.md (translations: README_zh.md, README_ja.md, README_fr.md, README_ru.md)
Security policy → SECURITY.md
Changes → CHANGELOG.md

Cross-Cutting Conventions

These apply repo-wide; module guides own the module-specific detail.

Documentation update policy — keep docs in sync with code: update README.md for user-facing changes and the relevant AGENTS.md for development/architecture changes in the same change set.
Test-driven development — features and bug fixes ship with tests. Backend tests live in backend/tests/ (TDD is mandatory there; see backend/AGENTS.md); frontend tests live in frontend/tests/.
Format before pushing — run make format (backend) / pnpm check (frontend). Backend CI enforces ruff format --check, so formatting must be clean before a push.

7.4 KiB Raw Blame History