deer-flow/backend/docs
Xinmin Zeng 4fc08b4f15
feat: add scheduled tasks MVP (#3898)
* feat: add scheduled tasks MVP

* fix: harden scheduled task execution semantics

* feat(scheduled-tasks): preset-driven schedule form with timezone and live preview

Replace the raw cron input with a preset Select (hourly/daily/weekly/monthly/custom)
plus structured inputs (time picker, weekday toggles, day-of-month), datetime-local
for one-time tasks, a timezone selector defaulting to the browser timezone, and a
live human-readable preview. Reuses one ScheduledTaskScheduleInput for create and
edit; backend contract unchanged; zero new deps (pure Intl + DST-safe offset helpers).

* feat(scheduled-tasks): full-page i18n + recipe templates + E2E locale pin

Localize the rest of the scheduled-tasks page (filters, detail pane, actions,
edit form, run list, enum values) via t.scheduledTasks.* in en/zh. Add four
built-in recipe templates (GitHub Trending, news digest, issue triage, weekly
report) exposed as a chip row that pre-fills title + prompt + schedule. Pin
Playwright locale to en-US so E2E selectors stay stable against i18n. No backend
change, no new deps.

* fix(scheduled-tasks): idempotent 0003 migration, update head constants, future-date once test

Merge with main surfaced three CI failures:
- 0003_scheduled_tasks create_table collided with legacy test seeds that
  build from full metadata; guard with inspector.has_table so the revision
  no-ops when the table already exists (0004/0005 are already idempotent via
  _helpers.py).
- persistence bootstrap concurrency/regression tests pinned HEAD to main's
  0002_runs_token_usage; bump to the new head 0005_scheduled_task_thread_nullable.
- once-task router test used a fixed past run_at and tripped the
  must-be-in-the-future validation; use a future date.

* address review: ok-check, 502 for trigger failure, mock fields, migration filename, doc fences

- fetchThreadScheduledTasks now checks response.ok like the other fetchers.
- trigger endpoint returns 502 (not 409) when dispatch fails outright, so
  clients can distinguish a real conflict from a server-side failure.
- E2E mock normalizes scheduled-task objects with context_mode/last_thread_id
  and nullable thread_id, matching the backend contract the UI renders against.
- Rename 0002_scheduled_tasks.py -> 0003_scheduled_tasks.py to match its
  revision id (file was renamed in spirit already; filename now follows).
- CONFIGURATION.md: close the Tool Groups yaml fence and drop the stray fence
  after the Scheduler notes so the sections render correctly.

* fix(scheduled-tasks): harden lease, poller, config, and frontend UX after review

* fix(scheduled-tasks): harden run lifecycle, overlap skip, non_interactive gating, and DST conversion after review

- defer a once task's terminal status to the run-completion hook; the task
  stays running until the real outcome, and a startup sweep cancels once
  tasks orphaned by a crash (launch-time 'completed' could stick forever)
- record interrupted runs as a distinct 'interrupted' run status with a
  readable message; an interrupted once task ends 'cancelled', not 'failed'
- enforce overlap_policy=skip for fresh_thread_per_run via an active-run
  pre-check (same-thread ConflictError can never fire across fresh threads)
- protect terminal run statuses from the late launch-path 'running' write
- honor context.non_interactive only for internally-authenticated callers;
  arbitrary clients can no longer strip ask_clarification
- fix DST-stale timezone offset in zonedLocalToUtcIso by re-deriving the
  offset at the resolved instant (once tasks fired an hour late around
  spring-forward and the create->edit round-trip diverged)
- drop dead ScheduledTaskRunRepository.update_by_run_id; share one Gateway
  API error helper between channels and scheduled-tasks frontends

* fix(scheduled-tasks): close review round-3 gaps in guards, concurrency, and API ergonomics

- scrub internal-only context keys (non_interactive) from the assembled run
  config for non-internal callers: gating body.context alone left the same
  key smuggle-able through the free-form body.config copied verbatim by
  build_run_config
- guard update_after_launch with protect_terminal so the launch bookkeeping
  write cannot clobber a once task already finalized by a fast-failing run's
  completion hook (parent-row sibling of the run-row guard)
- reject a manual trigger while the task has an active run (409) instead of
  launching a duplicate concurrent run on fresh_thread_per_run
- re-arm a terminal once task to enabled when PATCH pushes run_at into the
  future; previously the endpoint returned 200 with a next_run_at that could
  never be claimed
- make max_concurrent_runs a real global cap: each poll claims only into the
  remaining budget of active (queued/running) scheduled runs
- paginate GET /scheduled-tasks/{id}/runs (limit<=200, offset) and push the
  thread filter of /threads/{id}/scheduled-tasks into SQL
- stamp context.user_id on scheduler-launched runs, matching IM channels, so
  user-scoped guardrail providers see the owning user

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-04 21:51:57 +08:00
..
API.md fix(gateway): clamp client-supplied recursion_limit to prevent runaway runs (#3903) 2026-07-02 11:38:21 +08:00
APPLE_CONTAINER.md Fix command syntax for container image pull (#1349) 2026-03-26 00:14:08 +08:00
ARCHITECTURE.md fix: add MCP tools cache reset endpoint (#3602) 2026-06-16 23:20:20 +08:00
AUTH_DESIGN.md feat(auth): add OIDC SSO support (#3506) 2026-06-21 15:47:53 +08:00
AUTH_TEST_DOCKER_GAP.md docs: clean gateway runtime transition remnants (#3334) 2026-06-02 10:03:28 +08:00
AUTH_TEST_PLAN.md docs: clean standalone LangGraph server remnants (#3301) 2026-05-29 11:36:45 +08:00
AUTH_UPGRADE.md docs: clean gateway runtime transition remnants (#3334) 2026-06-02 10:03:28 +08:00
AUTO_TITLE_GENERATION.md fix(title): avoid default LLM call before stream end (#3885) 2026-06-30 18:58:02 +08:00
BLOCKING_IO_DETECTION.md feat(skill): add blocking-io-guard — SOP skill for blocking-IO triage and runtime anchors (#3503) 2026-06-12 10:20:38 +08:00
CONFIGURATION.md feat: add scheduled tasks MVP (#3898) 2026-07-04 21:51:57 +08:00
FILE_UPLOAD.md fix(uploads): enforce streaming upload limits in gateway (#2589) 2026-05-01 20:19:30 +08:00
GUARDRAILS.md feat(guardrails): expose authenticated runtime context in GuardrailRequest (#3665) 2026-06-21 16:08:25 +08:00
IM_CHANNEL_CONNECTIONS.md fix(channels): harden runtime credential management APIs (#3581) 2026-06-18 10:45:33 +08:00
MCP_SERVER.md feat(mcp): add per-server tool_call_timeout for MCP tool calls (#3843) 2026-07-03 15:39:12 +08:00
memory-settings-sample.json feat: support manual add and edit for memory facts (#1538) 2026-03-29 23:53:23 +08:00
MEMORY_IMPROVEMENTS.md feat(memory): add memory.token_counting config to avoid tiktoken network dependency (#3429) (#3465) 2026-06-10 23:26:15 +08:00
MEMORY_IMPROVEMENTS_SUMMARY.md refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
MEMORY_SETTINGS_REVIEW.md feat: support manual add and edit for memory facts (#1538) 2026-03-29 23:53:23 +08:00
middleware-execution-flow.md feat(loop-detection): defer warning injection (#2752) 2026-05-21 14:36:07 +08:00
PATH_EXAMPLES.md refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
plan_mode_usage.md refactor(lead-agent): make build_middlewares public to drop the last cross-module private import (#3458) 2026-06-09 11:56:28 +08:00
README.md chore: add sandbox memory profiling tools (#3249) 2026-06-03 22:02:27 +08:00
REPLAY_E2E.md fix(replay-e2e): key fixtures by caller and conversation (#3453) 2026-06-09 21:58:31 +08:00
rfc-create-deerflow-agent.md feat: add create_deerflow_agent SDK entry point (Phase 1) (#1203) 2026-03-29 15:31:18 +08:00
rfc-extract-shared-modules.md refactor: extract shared skill installer and upload manager to harness (#1202) 2026-03-25 16:28:33 +08:00
rfc-grep-glob-tools.md feat(sandbox): add built-in grep and glob tools (#1784) 2026-04-03 16:03:06 +08:00
SANDBOX_MEMORY_PROFILING.md chore: add sandbox memory profiling tools (#3249) 2026-06-03 22:02:27 +08:00
SETUP.md fix(harness): resolve runtime paths from project root (#2642) 2026-05-01 22:19:50 +08:00
SSO.md feat(auth): add OIDC SSO support (#3506) 2026-06-21 15:47:53 +08:00
STREAMING.md fix(redis): stream retention recovery (#3933) 2026-07-04 16:07:42 +08:00
summarization.md feat: emit structured runtime metadata (follow-up#3887) (#3906) 2026-07-04 11:27:19 +08:00
task_tool_improvements.md refactor: split backend into harness (deerflow.*) and app (app.*) (#1131) 2026-03-14 22:55:52 +08:00
TITLE_GENERATION_IMPLEMENTATION.md fix: generate fallback title for interrupted first-turn runs (#3874) 2026-07-02 00:13:01 +08:00
TODO.md docs: clean standalone LangGraph server remnants (#3301) 2026-05-29 11:36:45 +08:00
TUI.md feat(runtime): implement goal continuations (#3858) 2026-07-03 08:49:33 +08:00

Documentation

This directory contains detailed documentation for the DeerFlow backend.

Document Description
ARCHITECTURE.md System architecture overview
API.md Complete API reference
AUTH_DESIGN.md User authentication, CSRF, and per-user isolation design
CONFIGURATION.md Configuration options
SETUP.md Quick setup guide

Feature Documentation

Document Description
STREAMING.md Token-level streaming design: Gateway vs DeerFlowClient paths, stream_mode semantics, per-id dedup
FILE_UPLOAD.md File upload functionality
PATH_EXAMPLES.md Path types and usage examples
SANDBOX_MEMORY_PROFILING.md Sandbox memory baseline and runtime comparison guide
summarization.md Context summarization feature
plan_mode_usage.md Plan mode with TodoList
AUTO_TITLE_GENERATION.md Automatic title generation

Development

Document Description
TODO.md Planned features and known issues

Getting Started

  1. New to DeerFlow? Start with SETUP.md for quick installation
  2. Configuring the system? See CONFIGURATION.md
  3. Understanding the architecture? Read ARCHITECTURE.md
  4. Building integrations? Check API.md for API reference

Document Organization

docs/
├── README.md                  # This file
├── ARCHITECTURE.md            # System architecture
├── API.md                     # API reference
├── AUTH_DESIGN.md             # User authentication and isolation design
├── CONFIGURATION.md           # Configuration guide
├── SETUP.md                   # Setup instructions
├── FILE_UPLOAD.md             # File upload feature
├── PATH_EXAMPLES.md           # Path usage examples
├── summarization.md           # Summarization feature
├── plan_mode_usage.md         # Plan mode feature
├── STREAMING.md               # Token-level streaming design
├── AUTO_TITLE_GENERATION.md   # Title generation
├── TITLE_GENERATION_IMPLEMENTATION.md  # Title implementation details
└── TODO.md                    # Roadmap and issues