deer-flow/backend/docs/TUI.md
DanielWalnut 25ea6970a6
feat(runtime): implement goal continuations (#3858)
* implement goal continuations

* fix(goal): address review findings for goal continuations

- goal: key the no-progress breaker on a signature of the latest visible
  assistant evidence instead of the evaluator's volatile free-text, so it
  actually fires on stalled turns; thread the signature through every
  worker persist / no-progress call site
- goal: align _stand_down_reason default caps with should_continue_goal
  (8 / 2) so the two gate functions agree on goals missing the fields
- runtime: offload the synchronous checkpointer fallback via
  asyncio.to_thread (goal.py + worker.py) to keep blocking IO off the loop
- frontend: i18n the GoalStatus "Goal" label (goalLabel in en/zh/types)
- frontend: extract pure composer helpers into input-box-helpers.ts with
  unit tests (parseGoalCommand, readGoalResponseError, skill suggestions)
- tests: cover the evidence-based no-progress and default-cap behavior
- docs: align backend/AGENTS.md goal paragraph with actual behavior
- e2e: prettier-format chat.spec.ts (fixes the lint-frontend CI failure)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(frontend): hide goal continuation counter until the agent continues

The goal status bar rendered a raw "0/8" before any auto-continuation, which
read as a mysterious score. Now the counter is hidden until
continuation_count > 0, then shows "Continuing N/M" with a tooltip explaining
the auto-continuation cap.

- Extract getGoalContinuationDisplay into a pure helper (hides at 0) + unit tests
- Add goalContinuing / goalContinuationTooltip i18n keys (en/zh/types)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(goal): address review findings for goal continuations

Frontend correctness
- Fix the optimistic /goal result permanently shadowing server goal state:
  the streamed continuation counter never surfaced for a goal set in-session.
  Extract a shared useActiveGoal hook (used by both chat pages) that reconciles
  the optimistic copy with server state via a goalReconciliationKey, de-duping
  the copy-pasted goal block across the two pages.
- Stop /goal status|clear failures from escaping handleSubmit as unhandled
  rejections (handleGoalCommand now returns success; the run only starts when a
  goal was actually saved).
- Use a function replacer for the goal-status toast so an objective containing
  $&/$1 isn't treated as a replacement pattern.

Backend cleanliness / correctness
- De-duplicate four byte-identical helpers (_call_checkpointer_method,
  _message_type, _additional_kwargs, _is_visible_message) by importing them
  from runtime.goal instead of re-defining them in the run worker.
- Remove the dead `checkpoint_tuple.tasks` durability guard (CheckpointTuple has
  no tasks field) and document that pending_writes is the durability signal.
- Decompose the 176-line _prepare_goal_continuation_input: extract
  _reread_goal_and_checkpoint and a _persist closure so the thread-unchanged
  guard and stand-down persistence aren't open-coded three times. Document the
  last-writer-wins write-window limitation as a follow-up.
- Add a shared parse_goal_command helper and use it from the TUI and IM-channel
  /goal handlers (one place for the status/clear/set semantics).

Tests
- Restore the 11 command-registry tests dropped by the previous goal change
  (filter_commands ranking/description, build_registry builtins/skills, resolve
  cases) alongside the new goal tests.
- Add coverage for the IM-channel _handle_goal_command, the TUI _handle_goal
  handler, parse_goal_command, and goalReconciliationKey.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix goal review feedback

* fix goal continuation checkpoint races

* prioritize goal commands while streaming

Route composer submits through a shared helper so /goal commands can be handled before the streaming stop shortcut, while ordinary streaming submits still stop the active run.

Testing: cd frontend && pnpm exec rstest run tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; cd frontend && pnpm check

* preserve goal status during clarification

Keep omitted stream goal fields distinct from explicit null clears so clarification interrupts do not hide an active thread goal that is still present in the checkpoint.

Testing: pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check

* style: format active goal hook

Run Prettier on use-active-goal.ts to satisfy the frontend lint workflow formatting gate.

Testing: pnpm format; pnpm exec rstest run tests/unit/components/workspace/use-active-goal.test.ts tests/unit/components/workspace/input-box-helpers.test.ts tests/unit/components/workspace/goal-status-helpers.test.ts; pnpm check; git diff --check

* fix goal review followups

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 08:49:33 +08:00

5.3 KiB

DeerFlow Terminal Workbench (TUI)

deerflow is a terminal-native workbench for the DeerFlow harness. It runs embedded over DeerFlowClient — no Gateway, frontend, nginx, or Docker services required — while honoring the same config.yaml, checkpointer, skills, memory, MCP, and sandbox settings as the rest of DeerFlow.

DeerFlow TUI

Install & run

The TUI ships as an optional extra so the core harness install stays lean:

uv pip install 'deerflow-harness[tui]'    # or: pip install textual

Launch modes:

Command Behavior
deerflow Launch the TUI when stdin/stdout are TTYs
deerflow --tui Force the TUI (clear diagnostic if textual is missing)
deerflow --cli Force headless/classic mode for one invocation
deerflow chat Same TUI conversation surface
deerflow --continue Resume the most recent thread
deerflow --resume THREAD Resume a thread by id
deerflow --print "question" Headless one-shot answer to stdout
deerflow --json "question" Headless newline-delimited StreamEvents
echo "q" | deerflow --print Read the message from stdin
DEER_FLOW_TUI=1 deerflow Force the TUI via environment

If no TTY is available and no headless flag is given, deerflow prints guidance instead of hanging.

Surface

  • Header — model, thread, project root, skill/tool counts.
  • Transcript — user prompts, assistant answers, and compact tool cards (⚙ Read path ✓) with dimmed result previews. Finalized assistant messages render as Markdown (headings, bold, lists, code, links); the actively-streaming message stays plain text to avoid reflow jumpiness and snaps to Markdown when it completes. Transcript re-renders are coalesced (~16 fps) so streaming stays smooth on long threads.
  • Status line — run state + animated spinner, model, thread title, token usage, and an esc interrupt hint while a run is active.
  • Composer — rounded input box. / opens the command palette.

Keys

Key Action
Enter Send message / accept palette selection
/ Open the slash-command palette
/ Palette navigation, or input history when the palette is closed
Tab Complete the highlighted command (adds a trailing space)
Esc Close the palette / overlay
Ctrl+C Interrupt the active run, or quit when idle
Ctrl+L Redraw · Ctrl+U clear composer

Slash commands

/help /new /goal /threads (/switch) /model /skills /tools /mcp /memory /uploads /usage /config /quit, plus /<skill-name> task to activate any enabled skill for the current turn (same semantics as elsewhere in DeerFlow). /model and /threads open modal pickers. Use /goal <condition> to set the active thread goal, /goal to show it, and /goal clear to clear it.

Architecture

The TUI is a UI shell over the existing embedded harness — it does not fork agent behavior.

cli.py          launch-mode planning (pure) + headless print/json + entry point
session.py      builds DeerFlowClient (+ checkpointer) and the persistence writer
runtime.py      StreamEvent  ->  reducer actions  (pure translate + threaded driver)
view_state.py   ViewState + reduce(state, action)  (pure, the testable heart)
message_format  compact tool summaries / truncation (pure)
command_registry slash-command registry + resolve (pure)
input_history   bounded ↑/↓ history (pure)
render.py       Rich renderers for header / transcript / status / palette (pure)
theme.py        palette + symbols
app.py          Textual App: composes widgets, drives runs on a worker thread,
                marshals actions back to the UI thread, renders ViewState
persistence.py  writes threads_meta so sessions appear in the Web UI (below)

DeerFlowClient.stream() is a synchronous generator, so the app runs it on a Textual worker thread and marshals each yielded action back to the UI thread via call_from_thread. The pure layers (everything except app.py) have no Textual dependency and are unit-tested directly with synthetic StreamEvents.

Web UI visibility (shared persistence)

The Web UI lists conversations from the threads_meta SQL table (filtered by user_id), not from the checkpointer. An embedded run only writes the checkpointer, so a TUI thread would otherwise be invisible in the sidebar.

persistence.py closes that gap: on the first turn of a thread it writes a threads_meta row — owned by the local default user ("default") — into the same database the Gateway reads, and syncs the generated title afterward. This requires only the shared threads_meta store (built via deerflow.persistence.engine.init_engine_from_config), not the Gateway process. When the database backend is memory (no SQL store) the writer degrades to a silent no-op and the TUI still works.

All DB work runs on one long-lived background event loop, because a SQLAlchemy async engine is bound to the loop that created it.

Tests

Pure layers are TDD'd in backend/tests/test_tui_*.py; the Textual app, slash palette, and modal overlays are exercised through Textual's pilot harness with a fake in-process session (no live model). test_tui_persistence.py proves the threads_meta write/read round-trip.

cd backend && PYTHONPATH=. uv run pytest tests/ -k tui -q