diff --git a/README.md b/README.md index 692777768..5e81ec6b4 100644 --- a/README.md +++ b/README.md @@ -69,6 +69,7 @@ DeerFlow has newly integrated the intelligent search and crawling toolset indepe - [Long-Term Memory](#long-term-memory) - [Recommended Models](#recommended-models) - [Embedded Python Client](#embedded-python-client) + - [Terminal Workbench (TUI)](#terminal-workbench-tui) - [Documentation](#documentation) - [⚠️ Security Notice](#️-security-notice) - [Improper Deployment May Introduce Security Risks](#improper-deployment-may-introduce-security-risks) @@ -713,6 +714,26 @@ client.upload_files("thread-1", ["./report.pdf"]) # {"success": True, "files": All dict-returning methods are validated against Gateway Pydantic response models in CI (`TestGatewayConformance`), ensuring the embedded client stays in sync with the HTTP API schemas. See `backend/packages/harness/deerflow/client.py` for full API documentation. +## Terminal Workbench (TUI) + +`deerflow` is a terminal-native workbench for people who live in the shell. It runs **embedded** over `DeerFlowClient` — no Gateway, frontend, nginx, or Docker required — while honoring the same `config.yaml`, checkpointer, skills, memory, MCP, and sandbox settings as the rest of DeerFlow. + +![DeerFlow TUI](docs/tui/tui-preview.svg) + +```bash +uv pip install 'deerflow-harness[tui]' # optional 'textual' dependency + +deerflow # launch the terminal UI (TTY required) +deerflow --continue # resume the most recent thread +deerflow --resume THREAD # resume a thread by id +deerflow --print "summarize this repo" # headless one-shot answer to stdout +deerflow --json "hello" # headless newline-delimited StreamEvents +``` + +A keyboard-driven chat surface with a streaming transcript (Markdown-rendered answers), compact tool-activity cards, a `/` slash-command palette, `/model` and `/threads` pickers, input history, and `Esc` / `Ctrl+C` interrupt. Sessions opened in the TUI also appear in the Web UI sidebar — it writes the shared thread store under the local default user, so terminal and web stay in sync **without running the Gateway**. + +See [backend/docs/TUI.md](backend/docs/TUI.md) for the full guide. + ## Documentation - [Contributing Guide](CONTRIBUTING.md) - Development environment setup and workflow diff --git a/README_zh.md b/README_zh.md index eb5795bc0..e2af683bb 100644 --- a/README_zh.md +++ b/README_zh.md @@ -54,6 +54,7 @@ https://github.com/user-attachments/assets/a8bcadc4-e040-4cf2-8fda-dd768b999c18 - [长期记忆](#长期记忆) - [推荐模型](#推荐模型) - [内嵌 Python Client](#内嵌-python-client) + - [终端工作台 (TUI)](#终端工作台-tui) - [文档](#文档) - [⚠️ 安全使用](#️-安全使用) - [参与贡献](#参与贡献) @@ -542,6 +543,26 @@ client.upload_files("thread-1", ["./report.pdf"]) # {"success": True, "files": 所有返回 dict 的方法都会在 CI 中通过 Gateway 的 Pydantic 响应模型校验（`TestGatewayConformance`），以确保内嵌 client 始终和 HTTP API schema 保持同步。完整 API 说明见 `backend/packages/harness/deerflow/client.py`。 +## 终端工作台 (TUI) + +`deerflow` 是一个面向终端用户的工作台，**内嵌**运行在 `DeerFlowClient` 之上——无需启动 Gateway、前端、nginx 或 Docker，同时沿用与 DeerFlow 其它部分相同的 `config.yaml`、checkpointer、技能、记忆、MCP 和沙箱配置。 + +![DeerFlow TUI](docs/tui/tui-preview.svg) + +```bash +uv pip install 'deerflow-harness[tui]' # 可选的 'textual' 依赖 + +deerflow # 启动终端 UI（需要 TTY） +deerflow --continue # 恢复最近一次会话 +deerflow --resume THREAD # 按 id 恢复指定会话 +deerflow --print "总结一下这个仓库" # 无头模式，结果打印到 stdout +deerflow --json "hello" # 无头模式，输出按行分隔的 StreamEvent +``` + +键盘驱动的对话界面：流式渲染的对话区（回答按 Markdown 渲染）、紧凑的工具活动卡片、`/` 斜杠命令面板、`/model` 与 `/threads` 选择器、输入历史，以及 `Esc` / `Ctrl+C` 打断。在 TUI 里开启的会话也会出现在 Web UI 侧边栏——它会以本地默认用户身份写入共享的会话存储，因此终端与网页保持同步，**无需运行 Gateway**。 + +完整说明见 [backend/docs/TUI.md](backend/docs/TUI.md)。 + ## 文档 - [贡献指南](CONTRIBUTING.md) - 开发环境搭建与协作流程 diff --git a/backend/README.md b/backend/README.md index cb48b93a0..51a79a37a 100644 --- a/backend/README.md +++ b/backend/README.md @@ -203,6 +203,18 @@ make dev Direct access: Gateway at http://localhost:8001 +**Terminal Workbench (TUI)** — a terminal-native UI over the embedded harness, +no services required: + +```bash +uv pip install 'deerflow-harness[tui]' # optional 'textual' dependency +deerflow # launch the TUI +deerflow --print "summarize this repo" # headless one-shot +``` + +Sessions opened in the TUI appear in the Web UI sidebar (it writes the shared +`threads_meta` store under the local default user). See [docs/TUI.md](docs/TUI.md). + --- ## Project Structure diff --git a/backend/docs/TUI.md b/backend/docs/TUI.md new file mode 100644 index 000000000..a65d722bf --- /dev/null +++ b/backend/docs/TUI.md @@ -0,0 +1,119 @@ +# DeerFlow Terminal Workbench (TUI) + +`deerflow` is a terminal-native workbench for the DeerFlow harness. It runs +**embedded** over `DeerFlowClient` — no Gateway, frontend, nginx, or Docker +services required — while honoring the same `config.yaml`, checkpointer, skills, +memory, MCP, and sandbox settings as the rest of DeerFlow. + +![DeerFlow TUI](../../docs/tui/tui-preview.svg) + +## Install & run + +The TUI ships as an optional extra so the core harness install stays lean: + +```bash +uv pip install 'deerflow-harness[tui]' # or: pip install textual +``` + +Launch modes: + +| Command | Behavior | +|---|---| +| `deerflow` | Launch the TUI when stdin/stdout are TTYs | +| `deerflow --tui` | Force the TUI (clear diagnostic if `textual` is missing) | +| `deerflow --cli` | Force headless/classic mode for one invocation | +| `deerflow chat` | Same TUI conversation surface | +| `deerflow --continue` | Resume the most recent thread | +| `deerflow --resume THREAD` | Resume a thread by id | +| `deerflow --print "question"` | Headless one-shot answer to stdout | +| `deerflow --json "question"` | Headless newline-delimited `StreamEvent`s | +| `echo "q" \| deerflow --print` | Read the message from stdin | +| `DEER_FLOW_TUI=1 deerflow` | Force the TUI via environment | + +If no TTY is available and no headless flag is given, `deerflow` prints guidance +instead of hanging. + +## Surface + +- **Header** — model, thread, project root, skill/tool counts. +- **Transcript** — user prompts, assistant answers, and compact tool cards + (`⚙ Read path ✓`) with dimmed result previews. Finalized assistant messages + render as Markdown (headings, bold, lists, code, links); the actively-streaming + message stays plain text to avoid reflow jumpiness and snaps to Markdown when + it completes. Transcript re-renders are coalesced (~16 fps) so streaming stays + smooth on long threads. +- **Status line** — run state + animated spinner, model, thread title, token + usage, and an `esc interrupt` hint while a run is active. +- **Composer** — rounded input box. `/` opens the command palette. + +### Keys + +| Key | Action | +|---|---| +| `Enter` | Send message / accept palette selection | +| `/` | Open the slash-command palette | +| `↑` / `↓` | Palette navigation, or input history when the palette is closed | +| `Tab` | Complete the highlighted command (adds a trailing space) | +| `Esc` | Close the palette / overlay | +| `Ctrl+C` | Interrupt the active run, or quit when idle | +| `Ctrl+L` | Redraw · `Ctrl+U` clear composer | + +### Slash commands + +`/help` `/new` `/threads` (`/switch`) `/model` `/skills` `/tools` `/mcp` +`/memory` `/uploads` `/usage` `/config` `/quit`, plus `/ task` to +activate any enabled skill for the current turn (same semantics as elsewhere in +DeerFlow). `/model` and `/threads` open modal pickers. + +## Architecture + +The TUI is a UI shell over the existing embedded harness — it does **not** fork +agent behavior. + +``` +cli.py launch-mode planning (pure) + headless print/json + entry point +session.py builds DeerFlowClient (+ checkpointer) and the persistence writer +runtime.py StreamEvent -> reducer actions (pure translate + threaded driver) +view_state.py ViewState + reduce(state, action) (pure, the testable heart) +message_format compact tool summaries / truncation (pure) +command_registry slash-command registry + resolve (pure) +input_history bounded ↑/↓ history (pure) +render.py Rich renderers for header / transcript / status / palette (pure) +theme.py palette + symbols +app.py Textual App: composes widgets, drives runs on a worker thread, + marshals actions back to the UI thread, renders ViewState +persistence.py writes threads_meta so sessions appear in the Web UI (below) +``` + +`DeerFlowClient.stream()` is a **synchronous** generator, so the app runs it on a +Textual worker *thread* and marshals each yielded action back to the UI thread +via `call_from_thread`. The pure layers (everything except `app.py`) have no +Textual dependency and are unit-tested directly with synthetic `StreamEvent`s. + +## Web UI visibility (shared persistence) + +The Web UI lists conversations from the `threads_meta` SQL table (filtered by +`user_id`), **not** from the checkpointer. An embedded run only writes the +checkpointer, so a TUI thread would otherwise be invisible in the sidebar. + +`persistence.py` closes that gap: on the first turn of a thread it writes a +`threads_meta` row — owned by the local default user (`"default"`) — into the +**same** database the Gateway reads, and syncs the generated title afterward. +This requires only the shared `threads_meta` store (built via +`deerflow.persistence.engine.init_engine_from_config`), **not** the Gateway +process. When the database backend is `memory` (no SQL store) the writer +degrades to a silent no-op and the TUI still works. + +All DB work runs on one long-lived background event loop, because a SQLAlchemy +async engine is bound to the loop that created it. + +## Tests + +Pure layers are TDD'd in `backend/tests/test_tui_*.py`; the Textual app, slash +palette, and modal overlays are exercised through Textual's pilot harness with a +fake in-process session (no live model). `test_tui_persistence.py` proves the +`threads_meta` write/read round-trip. + +```bash +cd backend && PYTHONPATH=. uv run pytest tests/ -k tui -q +``` diff --git a/backend/packages/harness/deerflow/tui/__init__.py b/backend/packages/harness/deerflow/tui/__init__.py new file mode 100644 index 000000000..6fa6eceaa --- /dev/null +++ b/backend/packages/harness/deerflow/tui/__init__.py @@ -0,0 +1 @@ +"""DeerFlow terminal workbench (TUI), embedded over DeerFlowClient.""" diff --git a/backend/packages/harness/deerflow/tui/__main__.py b/backend/packages/harness/deerflow/tui/__main__.py new file mode 100644 index 000000000..40897eb14 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/__main__.py @@ -0,0 +1,6 @@ +"""Allow ``python -m deerflow.tui`` to launch the workbench.""" + +from .cli import main + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/backend/packages/harness/deerflow/tui/app.py b/backend/packages/harness/deerflow/tui/app.py new file mode 100644 index 000000000..75cb720d2 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/app.py @@ -0,0 +1,648 @@ +"""The Textual application — a terminal workbench over the embedded harness. + +The app keeps a single immutable :class:`ViewState` and re-renders it through the +pure renderers. Agent runs execute on a worker *thread* (``DeerFlowClient.stream`` +is a synchronous generator); each yielded action is marshalled back onto the UI +thread via ``call_from_thread`` and folded into the reducer. +""" + +from __future__ import annotations + +import uuid +from functools import partial + +from textual.app import App, ComposeResult +from textual.binding import Binding +from textual.containers import Vertical, VerticalScroll +from textual.screen import ModalScreen +from textual.widgets import Input, Label, OptionList, Static +from textual.widgets.option_list import Option + +from .input_history import InputHistory +from .render import render_header, render_status, render_transcript +from .runtime import stream_actions +from .theme import SYMBOLS, THEME +from .view_state import ( + ClearRows, + RunEnded, + RunStarted, + SystemMessage, + ThreadTitle, + UserSubmitted, + initial_state, + reduce, +) +from .widgets.composer import ComposerInput + +_HELP_TEXT = "Commands: /new /threads /model /skills /tools /mcp /memory /usage /config /quit\nKeys: Enter send · Ctrl+C interrupt or quit · Ctrl+L redraw · / commands · Esc close overlay" + + +class SelectScreen(ModalScreen): + """A centered modal that returns the id of the chosen option (or None).""" + + BINDINGS = [Binding("escape", "cancel", "Close")] + + def __init__(self, title: str, options: list[tuple[str, str]]) -> None: + super().__init__() + self._title = title + self._options = options + + def compose(self) -> ComposeResult: + with Vertical(id="dialog"): + yield Label(self._title, id="dialog-title") + yield OptionList(*[Option(label, id=oid) for oid, label in self._options], id="dialog-list") + + def on_mount(self) -> None: + self.query_one(OptionList).focus() + + def on_option_list_option_selected(self, event: OptionList.OptionSelected) -> None: + self.dismiss(event.option_id) + + def action_cancel(self) -> None: + self.dismiss(None) + + +class DeerFlowTUI(App): + CSS = f""" + Screen {{ + background: {THEME.bg}; + color: {THEME.text}; + }} + #header {{ + height: 1; + padding: 0 1; + background: {THEME.panel}; + }} + #scroll {{ + height: 1fr; + padding: 1 2; + background: {THEME.bg}; + scrollbar-size-vertical: 1; + }} + #transcript {{ + width: 100%; + height: auto; + }} + #status {{ + height: 1; + padding: 0 1; + background: {THEME.panel}; + color: {THEME.muted}; + }} + #palette {{ + height: auto; + max-height: 10; + margin: 0 1; + padding: 0 1; + background: {THEME.panel}; + border: round {THEME.border}; + display: none; + }} + #palette.open {{ + display: block; + }} + #composer {{ + height: 3; + margin: 0 1 1 1; + border: round {THEME.border}; + background: {THEME.panel}; + }} + #composer:focus {{ + border: round {THEME.primary}; + }} + SelectScreen {{ + align: center middle; + }} + SelectScreen #dialog {{ + width: 72; + max-width: 90%; + height: auto; + max-height: 80%; + padding: 1 2; + background: {THEME.panel}; + border: round {THEME.primary}; + }} + SelectScreen #dialog-title {{ + color: {THEME.primary}; + text-style: bold; + padding: 0 0 1 0; + }} + SelectScreen OptionList {{ + background: {THEME.panel}; + border: none; + height: auto; + max-height: 20; + }} + SelectScreen OptionList > .option-list--option-highlighted {{ + background: {THEME.primary}; + color: {THEME.bg}; + }} + """ + + BINDINGS = [ + Binding("ctrl+c", "interrupt", "Interrupt / Quit", priority=True, show=True), + Binding("ctrl+l", "redraw", "Redraw", show=False), + Binding("ctrl+u", "clear_composer", "Clear input", show=False), + # Up/Down drive the palette when it's open, otherwise input history. + # Tab/Enter/Esc only act when the palette is open. check_action gates all + # of these so they never steal keys from a modal overlay or the composer. + Binding("down", "nav_down", show=False, priority=True), + Binding("up", "nav_up", show=False, priority=True), + Binding("tab", "palette_complete", show=False, priority=True), + Binding("escape", "escape", show=False, priority=True), + Binding("enter", "palette_accept", show=False, priority=True), + ] + + def __init__(self, session, plan) -> None: + super().__init__() + self.session = session + self.plan = plan + self.state = initial_state() + self._conv_thread_id: str | None = None + self._model = "" + self._skill_names: list[str] = [] + self._skills = 0 + self._spinner_idx = 0 + self._streaming = False + self._cancelled = False + self._skills_meta: list[dict] = [] + self._model_override: str | None = None + self._palette_open = False + self._palette_items: list = [] + self._palette_index = 0 + self._history = InputHistory() + self._transcript_dirty = False + + # ----- composition --------------------------------------------------- # + + def compose(self) -> ComposeResult: + yield Static(id="header") + with VerticalScroll(id="scroll"): + yield Static(id="transcript") + yield Static(id="status") + yield Static(id="palette") + yield ComposerInput(placeholder="Message DeerFlow… ( / for commands )", id="composer") + + def on_mount(self) -> None: + self._load_session_info() + self._refresh_all() + self.set_interval(0.1, self._tick_spinner) + self.set_interval(0.06, self._flush_transcript) # coalesce streaming re-renders + self.query_one("#composer", Input).focus() + if self.plan and getattr(self.plan, "message", None): + self._send_to_agent(self.plan.message) + + # ----- session info -------------------------------------------------- # + + def _load_session_info(self) -> None: + self._conv_thread_id = self.session.resolve_thread(self.plan) if self.plan else None + client = self.session.client + try: + models = client.list_models().get("models", []) + self._model = next((m.get("display_name") or m.get("name") for m in models if m.get("name")), "") + except Exception: # noqa: BLE001 - header is best-effort + self._model = "" + try: + skills = client.list_skills(enabled_only=True).get("skills", []) + self._skills_meta = [s for s in skills if s.get("name")] + self._skill_names = [s["name"] for s in self._skills_meta] + self._skills = len(self._skill_names) + except Exception: # noqa: BLE001 + self._skills_meta = [] + self._skill_names = [] + self._skills = 0 + + # ----- input --------------------------------------------------------- # + + def on_input_submitted(self, event: Input.Submitted) -> None: + text = event.value.strip() + event.input.value = "" + self._close_palette() + if not text: + return + self._history.add(text) + self._handle_submit(text) + + def on_input_changed(self, event: Input.Changed) -> None: + value = event.value + if value.startswith("/") and " " not in value: + from .command_registry import build_registry, filter_commands + + items = filter_commands(build_registry(self._skills_meta), value[1:]) + # The candidate set changed, so the previous highlight index is stale — + # reset to the top rather than clamping to a now-different command. + self._palette_index = 0 + self._open_palette(items) + else: + self._close_palette() + + # ----- slash command palette ----------------------------------------- # + + def check_action(self, action: str, parameters): # noqa: D401 - Textual hook + custom = {"nav_up", "nav_down", "palette_complete", "palette_accept", "escape"} + if action in custom: + # A modal overlay (e.g. the model/thread picker) is on top — never + # intercept its keys; let the overlay handle them natively. + if len(self.screen_stack) > 1: + return None + # nav (history), Tab and Esc are always consumed (Tab can't move focus + # off the composer; Esc closes the palette or interrupts a run). Enter + # falls through to the Input when the palette is closed so it submits. + if action in {"nav_up", "nav_down", "palette_complete", "escape"}: + return True + return True if self._palette_open else None + return True + + def action_nav_up(self) -> None: + if self._palette_open: + self.action_palette_up() + else: + self._history_move(self._history.up(self.query_one("#composer", Input).value)) + + def action_nav_down(self) -> None: + if self._palette_open: + self.action_palette_down() + else: + self._history_move(self._history.down()) + + def _history_move(self, value: str) -> None: + composer = self.query_one("#composer", Input) + composer.value = value + composer.cursor_position = len(value) + + def _open_palette(self, items: list) -> None: + if not items: + self._close_palette() + return + self._palette_items = items + self._palette_index = min(self._palette_index, len(items) - 1) + self._palette_open = True + palette = self.query_one("#palette", Static) + palette.add_class("open") + self._render_palette() + + def _close_palette(self) -> None: + if not self._palette_open and not self._palette_items: + return + self._palette_open = False + self._palette_items = [] + self._palette_index = 0 + self.query_one("#palette", Static).remove_class("open") + + def _render_palette(self) -> None: + from .render import render_palette + + self.query_one("#palette", Static).update(render_palette(self._palette_items, self._palette_index)) + + def _current_palette_item(self): + if 0 <= self._palette_index < len(self._palette_items): + return self._palette_items[self._palette_index] + return None + + def action_palette_down(self) -> None: + if self._palette_items: + self._palette_index = min(self._palette_index + 1, len(self._palette_items) - 1) + self._render_palette() + + def action_palette_up(self) -> None: + if self._palette_items: + self._palette_index = max(self._palette_index - 1, 0) + self._render_palette() + + def action_palette_complete(self) -> None: + # When the palette is open, Tab completes the highlighted command. + # When it's closed, Tab is a no-op here (consumed) so focus stays in the + # composer instead of moving to the scroll region. + if self._palette_open: + self._fill_from_palette() + + def action_palette_accept(self) -> None: + item = self._current_palette_item() + if item is None: + return + if getattr(item, "category", "") == "skill": + # Skills need a task argument; fill and let the user keep typing. + self._fill_from_palette() + return + self._close_palette() + self.query_one("#composer", Input).value = "" + self._handle_submit(f"/{item.name}") + + def _fill_from_palette(self) -> None: + item = self._current_palette_item() + if item is None: + return + composer = self.query_one("#composer", Input) + composer.value = f"/{item.name} " + composer.cursor_position = len(composer.value) + self._close_palette() + + def _handle_submit(self, text: str) -> None: + from .command_registry import resolve + + res = resolve(text, skills=self._skill_names) + if res.kind == "builtin": + self._handle_builtin(res.name, res.args) + return + if res.kind == "unknown": + self._dispatch(SystemMessage(f"Unknown command /{res.name}. Try /help.", tone="error")) + return + # plain message or skill activation (/skill task) both go to the agent, + # which applies skill-activation semantics on the raw text. + self._send_to_agent(text) + + def _handle_builtin(self, name: str, args: str) -> None: + if name == "quit": + self.exit() + elif name == "help": + self._dispatch(SystemMessage(_HELP_TEXT)) + elif name == "new": + self._conv_thread_id = None + self.state = initial_state() + self._dispatch(SystemMessage("Started a new thread.")) + elif name == "clear": + self._dispatch(ClearRows()) + elif name == "model": + self._open_model_picker() + elif name in {"threads", "switch"}: + self._open_thread_switcher() + elif name == "resume": + self._resume_thread(args) + elif name == "skills": + self._show_skills() + elif name == "mcp": + self._show_mcp() + elif name == "memory": + self._show_memory() + elif name == "usage": + self._show_usage() + elif name == "config": + self._show_config() + elif name == "tools": + self._dispatch(SystemMessage("Tools are listed in the agent's runtime; use /mcp for MCP servers.")) + elif name == "uploads": + self._show_uploads() + elif name == "artifacts": + self._dispatch(SystemMessage("Artifacts appear inline as the agent writes them.", tone="info")) + elif name == "details": + self._dispatch(SystemMessage("Verbose activity is always shown in this build.", tone="info")) + else: + self._dispatch(SystemMessage(f"/{name} is not available yet.", tone="info")) + + # ----- overlays + info commands -------------------------------------- # + + def _open_model_picker(self) -> None: + try: + models = self.session.client.list_models().get("models", []) + except Exception: # noqa: BLE001 + models = [] + options = [(m["name"], (m.get("display_name") or m["name"])) for m in models if m.get("name")] + if not options: + self._dispatch(SystemMessage("No models configured.", tone="error")) + return + + def on_choice(choice: str | None) -> None: + if choice: + self._model_override = choice + self._model = choice + self._dispatch(SystemMessage(f"Model set to {choice}.")) + self._refresh_header() + + self.push_screen(SelectScreen("Select model", options), on_choice) + + def _open_thread_switcher(self) -> None: + try: + threads = self.session.recent_threads(limit=20) + except Exception: # noqa: BLE001 + threads = [] + options: list[tuple[str, str]] = [] + for thread in threads: + tid = thread.get("thread_id") + if not tid: + continue + title = thread.get("title") or "untitled" + options.append((tid, f"{title} · {tid[:8]}")) + if not options: + self._dispatch(SystemMessage("No saved threads yet.")) + return + + def on_choice(choice: str | None) -> None: + if choice: + self._switch_to_thread(choice) + + self.push_screen(SelectScreen("Resume thread", options), on_choice) + + def _resume_thread(self, ref: str) -> None: + """/resume [id-or-title]: switch to a thread, or open the picker if blank.""" + ref = ref.strip() + if not ref: + self._open_thread_switcher() + return + self._switch_to_thread(self.session.resolve_ref(ref)) + + def _switch_to_thread(self, thread_id: str) -> None: + self._conv_thread_id = thread_id + self.state = initial_state() + self._dispatch(SystemMessage(f"Resumed thread {thread_id[:8]}.")) + self._refresh_header() + + def _show_skills(self) -> None: + names = ", ".join(self._skill_names) or "none" + self._dispatch(SystemMessage(f"Enabled skills ({self._skills}): {names}")) + + def _show_mcp(self) -> None: + try: + servers = self.session.client.get_mcp_config().get("mcp_servers", {}) + except Exception: # noqa: BLE001 + self._dispatch(SystemMessage("Could not read MCP config.", tone="error")) + return + if not servers: + self._dispatch(SystemMessage("No MCP servers configured.")) + return + lines = [f"{name}: {'on' if cfg.get('enabled') else 'off'}" for name, cfg in servers.items()] + self._dispatch(SystemMessage("MCP servers — " + " · ".join(lines))) + + def _show_memory(self) -> None: + try: + data = self.session.client.get_memory() + except Exception: # noqa: BLE001 + self._dispatch(SystemMessage("Could not read memory.", tone="error")) + return + facts = data.get("facts", []) if isinstance(data, dict) else [] + top = (data.get("topOfMind") if isinstance(data, dict) else "") or "—" + self._dispatch(SystemMessage(f"Memory: {len(facts)} facts · top of mind: {top}")) + + def _show_usage(self) -> None: + usage = self.state.usage or {} + if not usage: + self._dispatch(SystemMessage("No token usage recorded yet.")) + return + parts = ", ".join(f"{k}={v}" for k, v in usage.items()) + self._dispatch(SystemMessage(f"Token usage — {parts}")) + + def _show_config(self) -> None: + import os + + self._dispatch(SystemMessage(f"cwd: {os.getcwd()} model: {self._model or 'default'}")) + + def _show_uploads(self) -> None: + if not self._conv_thread_id: + self._dispatch(SystemMessage("Start a thread before listing uploads.")) + return + try: + uploads = self.session.client.list_uploads(self._conv_thread_id).get("files", []) + except Exception: # noqa: BLE001 + self._dispatch(SystemMessage("Could not list uploads.", tone="error")) + return + if not uploads: + self._dispatch(SystemMessage("No uploads in this thread.")) + return + names = ", ".join(f.get("filename", "?") for f in uploads) + self._dispatch(SystemMessage(f"Uploads ({len(uploads)}): {names}")) + + # ----- agent run ----------------------------------------------------- # + + def _send_to_agent(self, text: str) -> None: + if self._streaming: + self._dispatch(SystemMessage("Still working — wait for the current run to finish.", tone="info")) + return + if self._conv_thread_id is None: + self._conv_thread_id = str(uuid.uuid4()) + self._cancelled = False + self._dispatch(UserSubmitted(text)) + self.run_worker( + partial(self._stream_worker, text, self._conv_thread_id), + thread=True, + exclusive=True, + group="agent", + ) + + def _stream_worker(self, text: str, thread_id: str) -> None: + kwargs: dict = {} + if self._model_override: + kwargs["model_name"] = self._model_override + + writer = getattr(self.session, "writer", None) + if writer is not None: + # Make this terminal session visible in the Web UI sidebar by writing a + # threads_meta row under the local default user (best-effort, no-op on + # memory backends). Done on this worker thread to keep the UI responsive. + writer.ensure_created(thread_id, assistant_id="lead-agent", metadata={"source": "tui"}) + + latest_title: str | None = None + for action in stream_actions(self.session.client, text, thread_id=thread_id, **kwargs): + if self._cancelled: + break + if isinstance(action, ThreadTitle): + latest_title = action.title + self.call_from_thread(self._on_action, action) + + # Only persist a title for a run that completed normally — an interrupted + # run may only have emitted the title middleware's first, truncated guess. + if writer is not None and latest_title and not self._cancelled: + writer.set_title(thread_id, latest_title) + + def _on_action(self, action) -> None: + self.state = reduce(self.state, action) + if isinstance(action, RunStarted): + self._streaming = True + self._transcript_dirty = True + elif isinstance(action, RunEnded): + self._streaming = False + # Flush now so the finished message snaps to its Markdown rendering. + self._transcript_dirty = False + self._refresh_transcript() + else: + # Coalesce rapid streaming deltas; _flush_transcript renders them. + self._transcript_dirty = True + self._refresh_status() + + def _flush_transcript(self) -> None: + if self._transcript_dirty: + self._transcript_dirty = False + self._refresh_transcript() + + # ----- key actions --------------------------------------------------- # + + def action_interrupt(self) -> None: + if self._streaming: + self._interrupt_run() + else: + self.exit() + + def action_escape(self) -> None: + if self._palette_open: + self._close_palette() + elif self._streaming: + self._interrupt_run() + + def _interrupt_run(self) -> None: + self._cancelled = True + self.workers.cancel_group(self, "agent") + self._streaming = False + self.state = reduce(self.state, RunEnded()) + self._dispatch(SystemMessage("Interrupted.", tone="info")) + + def action_redraw(self) -> None: + self.refresh(layout=True) + self._refresh_all() + + def action_clear_composer(self) -> None: + self.query_one("#composer", Input).value = "" + + # ----- rendering ----------------------------------------------------- # + + def _dispatch(self, action) -> None: + self.state = reduce(self.state, action) + self._refresh_transcript() + self._refresh_status() + + def _tick_spinner(self) -> None: + if self._streaming: + self._spinner_idx = (self._spinner_idx + 1) % len(SYMBOLS["spinner"]) + self._refresh_status() + + def _thread_label(self) -> str: + if not self._conv_thread_id: + return "new thread" + return f"thread {self._conv_thread_id[:8]}" + + def _refresh_all(self) -> None: + self._refresh_header() + self._refresh_transcript() + self._refresh_status() + + def _refresh_header(self) -> None: + import os + + self.query_one("#header", Static).update( + render_header( + model=self._model, + thread_label=self._thread_label(), + cwd=os.getcwd(), + skills=self._skills, + ) + ) + + def _refresh_transcript(self) -> None: + self.query_one("#transcript", Static).update(render_transcript(self.state)) + self.query_one("#scroll", VerticalScroll).scroll_end(animate=False) + + def _refresh_status(self) -> None: + spinner = SYMBOLS["spinner"][self._spinner_idx] if self._streaming else "" + self.query_one("#status", Static).update(render_status(self.state, model=self._model, thread_label=self._thread_label(), spinner=spinner)) + + +def run_tui(plan) -> int: + """Construct the embedded session and run the app. Returns a process exit code.""" + from .session import open_session + + session = open_session() + app = DeerFlowTUI(session, plan) + try: + app.run() + finally: + # Stop the background DB loop + dispose the engine so repeated run_tui + # calls in one process don't leak loops / connection pools. + session.close() + return 0 diff --git a/backend/packages/harness/deerflow/tui/cli.py b/backend/packages/harness/deerflow/tui/cli.py new file mode 100644 index 000000000..b7c57ac69 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/cli.py @@ -0,0 +1,237 @@ +"""Command-line entry point and launch-mode planning for the DeerFlow TUI. + +``plan_launch`` is a pure decision function (fully unit-tested): given argv, TTY +state and the environment, it decides whether to open the terminal UI or run a +headless one-shot. ``main`` wires that decision to the embedded ``DeerFlowClient`` +and lazily imports the Textual app only when actually launching the UI, so the +``deerflow`` console script still runs headless commands without Textual present. +""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +from collections.abc import Sequence +from dataclasses import dataclass +from typing import Literal + +_UNSET = object() + +Mode = Literal["tui", "print", "json", "headless-help"] + + +@dataclass +class LaunchPlan: + mode: Mode + message: str | None = None + read_stdin: bool = False + thread_id: str | None = None + continue_recent: bool = False + forced_tui: bool = False + reason: str = "" + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="deerflow", + description="DeerFlow terminal workbench — a TUI over the embedded DeerFlow harness.", + add_help=True, + ) + parser.add_argument("message", nargs="*", help="initial prompt for the TUI, or message in --cli mode") + parser.add_argument( + "--print", + dest="print", + nargs="?", + const=None, + default=_UNSET, + metavar="MESSAGE", + help="headless one-shot: print the final answer and exit (reads stdin if no MESSAGE)", + ) + parser.add_argument( + "--json", + dest="json", + nargs="?", + const=None, + default=_UNSET, + metavar="MESSAGE", + help="headless streaming: emit newline-delimited JSON StreamEvents and exit", + ) + parser.add_argument("--tui", action="store_true", help="force the terminal UI (error if unavailable)") + parser.add_argument("--cli", action="store_true", help="force headless/classic mode for one invocation") + parser.add_argument("--continue", dest="continue_recent", action="store_true", help="resume the most recent thread") + parser.add_argument("--resume", dest="resume", metavar="THREAD", default=None, help="resume a thread by id or title") + return parser + + +def _strip_chat(argv: Sequence[str]) -> list[str]: + """Accept an optional leading ``chat`` subcommand as an alias for the default surface.""" + argv = list(argv) + if argv and argv[0] == "chat": + return argv[1:] + return argv + + +def _truthy(value: object) -> bool: + return isinstance(value, str) and value.strip().lower() in {"1", "true", "yes", "on"} + + +def plan_launch( + argv: Sequence[str], + *, + stdin_isatty: bool, + stdout_isatty: bool, + env: dict[str, str], +) -> LaunchPlan: + """Decide what surface to launch. Pure: no I/O, no client construction.""" + args = build_parser().parse_args(_strip_chat(argv)) + positional = " ".join(args.message).strip() or None + resume = args.resume + continue_recent = bool(args.continue_recent) + + if args.print is not _UNSET: + message = args.print if isinstance(args.print, str) else None + if message is None and stdin_isatty: + return LaunchPlan(mode="headless-help", reason="--print needs a MESSAGE argument or piped stdin.") + return LaunchPlan(mode="print", message=message, read_stdin=message is None, thread_id=resume, continue_recent=continue_recent) + + if args.json is not _UNSET: + message = args.json if isinstance(args.json, str) else None + if message is None and stdin_isatty: + return LaunchPlan(mode="headless-help", reason="--json needs a MESSAGE argument or piped stdin.") + return LaunchPlan(mode="json", message=message, read_stdin=message is None, thread_id=resume, continue_recent=continue_recent) + + if args.cli: + if positional: + return LaunchPlan(mode="print", message=positional, thread_id=resume, continue_recent=continue_recent) + # Mirror --print: a piped message or --continue is enough to run headless. + if continue_recent or not stdin_isatty: + return LaunchPlan(mode="print", message=None, read_stdin=True, thread_id=resume, continue_recent=continue_recent) + return LaunchPlan( + mode="headless-help", + reason='--cli needs a message. Try: deerflow --print "your question".', + ) + + forced_tui = bool(args.tui) + if forced_tui or _truthy(env.get("DEER_FLOW_TUI")) or (stdin_isatty and stdout_isatty): + return LaunchPlan( + mode="tui", + message=positional, + thread_id=resume, + continue_recent=continue_recent, + forced_tui=forced_tui, + ) + + return LaunchPlan( + mode="headless-help", + message=positional, + thread_id=resume, + continue_recent=continue_recent, + reason="No interactive terminal detected. Use --print MESSAGE for one-shot output, or --tui to force the UI.", + ) + + +# --------------------------------------------------------------------------- # +# Runtime dispatch (not unit-tested here; covered by smoke + integration). +# --------------------------------------------------------------------------- # + +_HEADLESS_HELP = """\ +deerflow — DeerFlow terminal workbench + + deerflow launch the terminal UI (TTY required) + deerflow --tui force the terminal UI + deerflow --continue resume the most recent thread in the UI + deerflow --resume THREAD resume a thread by id or title + deerflow --print "question" one-shot answer to stdout + deerflow --json "question" stream newline-delimited JSON events + echo "question" | deerflow --print +""" + + +def _resolve_message(plan: LaunchPlan) -> str: + if plan.read_stdin: + return sys.stdin.read().strip() + return plan.message or "" + + +def main(argv: Sequence[str] | None = None) -> int: + argv = list(sys.argv[1:] if argv is None else argv) + plan = plan_launch( + argv, + stdin_isatty=sys.stdin.isatty(), + stdout_isatty=sys.stdout.isatty(), + env=dict(os.environ), + ) + + if plan.mode == "headless-help": + if plan.reason: + print(plan.reason, file=sys.stderr) + print(_HEADLESS_HELP, file=sys.stderr) + return 0 if not plan.reason else 2 + + if plan.mode == "print": + return _run_print(plan) + + if plan.mode == "json": + return _run_json(plan) + + return _run_tui(plan) + + +def _make_session(): + # Imported lazily so the pure planning path never imports the heavy harness. + # Headless one-shots never use the threads_meta writer, so skip persistence + # (no background loop / engine / connection pool just to discard it). + from .session import open_session + + return open_session(persistence=False) + + +def _run_print(plan: LaunchPlan) -> int: + message = _resolve_message(plan) + if not message: + print("No message provided.", file=sys.stderr) + return 2 + session = _make_session() + thread_id = session.resolve_thread(plan) + answer = session.client.chat(message, thread_id=thread_id) + print(answer) + return 0 + + +def _run_json(plan: LaunchPlan) -> int: + message = _resolve_message(plan) + if not message: + print("No message provided.", file=sys.stderr) + return 2 + session = _make_session() + thread_id = session.resolve_thread(plan) + for event in session.client.stream(message, thread_id=thread_id): + payload = {"type": event.type, "data": event.data} + sys.stdout.write(json.dumps(payload, ensure_ascii=False, default=str) + "\n") + sys.stdout.flush() + return 0 + + +def _run_tui(plan: LaunchPlan) -> int: + try: + # Absolute import (not `from .app`) so the harness import-boundary check, + # which records relative module names verbatim, doesn't mistake the sibling + # `deerflow.tui.app` module for the forbidden top-level `app` package. + from deerflow.tui.app import run_tui + except ModuleNotFoundError as exc: # textual missing + if getattr(exc, "name", "") == "textual" or "textual" in str(exc): + msg = "The terminal UI needs the optional 'textual' dependency.\nInstall it with: uv pip install 'deerflow-harness[tui]' (or: pip install textual)\n" + if plan.forced_tui: + print(msg, file=sys.stderr) + return 1 + print(msg + "\nFalling back to headless help:\n", file=sys.stderr) + print(_HEADLESS_HELP, file=sys.stderr) + return 0 + raise + return run_tui(plan) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/backend/packages/harness/deerflow/tui/command_registry.py b/backend/packages/harness/deerflow/tui/command_registry.py new file mode 100644 index 000000000..016f4e77f --- /dev/null +++ b/backend/packages/harness/deerflow/tui/command_registry.py @@ -0,0 +1,115 @@ +"""Slash-command registry for the DeerFlow TUI (pure). + +Normalizes two command sources into one searchable list: + +* **Built-ins** — TUI-owned affordances (``/help``, ``/model``, ``/threads`` …). +* **Skills** — one ``/`` per enabled skill, preserving DeerFlow's + existing slash-skill activation semantics. + +The picker filters this list; :func:`resolve` classifies a submitted line as a +built-in command, a skill activation, an unknown command, or a plain message. +No Textual dependency. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Literal + + +@dataclass(frozen=True) +class Command: + name: str # without leading slash + description: str + category: Literal["builtin", "skill"] = "builtin" + + +@dataclass(frozen=True) +class Resolution: + kind: Literal["builtin", "skill", "unknown", "message"] + name: str = "" + args: str = "" + text: str = "" + + +# Built-in commands, ordered for display in /help and the picker. +BUILTIN_COMMANDS: tuple[Command, ...] = ( + Command("help", "Show commands and keybindings"), + Command("new", "Start a fresh thread"), + Command("threads", "Open the thread switcher"), + Command("switch", "Open the thread switcher"), + Command("resume", "Resume a thread by id or title"), + Command("model", "Open the model picker"), + Command("skills", "Browse enabled and available skills"), + Command("tools", "Show built-in, MCP and sandbox tools"), + Command("mcp", "Show MCP server status"), + Command("memory", "Show memory status and injected facts"), + Command("uploads", "Show uploaded files for this thread"), + Command("artifacts", "Show generated artifacts"), + Command("details", "Toggle verbose activity rendering"), + Command("usage", "Show token usage and context"), + Command("config", "Show resolved config paths and overrides"), + Command("quit", "Exit the TUI"), +) + +_BUILTIN_NAMES = frozenset(c.name for c in BUILTIN_COMMANDS) + + +def build_registry(skills: list[dict]) -> list[Command]: + """Merge built-ins with one command per enabled skill.""" + commands = list(BUILTIN_COMMANDS) + for skill in skills: + if not skill.get("enabled", False): + continue + name = skill.get("name") + if not name or name in _BUILTIN_NAMES: + continue + commands.append(Command(name=name, description=skill.get("description", "") or "", category="skill")) + return commands + + +def filter_commands(commands: list[Command], query: str) -> list[Command]: + """Filter + rank commands for the picker. + + Ranking: name-prefix matches first, then name-substring, then + description-substring. Original order is preserved within a rank tier. + """ + q = query.strip().lower() + if not q: + return commands + + prefix: list[Command] = [] + substring: list[Command] = [] + description: list[Command] = [] + for command in commands: + name = command.name.lower() + if name.startswith(q): + prefix.append(command) + elif q in name: + substring.append(command) + elif q in command.description.lower(): + description.append(command) + return prefix + substring + description + + +def resolve(text: str, skills: list[str] | None = None) -> Resolution: + """Classify a submitted input line.""" + stripped = text.strip() + if not stripped.startswith("/"): + return Resolution(kind="message", text=text) + + body = stripped[1:] + name, _, args = body.partition(" ") + name = name.strip() + args = args.strip() + + if not name: + return Resolution(kind="unknown", name="") + + if name in _BUILTIN_NAMES: + return Resolution(kind="builtin", name=name, args=args) + + if skills and name in skills: + return Resolution(kind="skill", name=name, args=args) + + return Resolution(kind="unknown", name=name, args=args) diff --git a/backend/packages/harness/deerflow/tui/input_history.py b/backend/packages/harness/deerflow/tui/input_history.py new file mode 100644 index 000000000..ed88f01ae --- /dev/null +++ b/backend/packages/harness/deerflow/tui/input_history.py @@ -0,0 +1,58 @@ +"""Bounded composer input history with up/down navigation (pure). + +No persistence and no Textual dependency here; the app may seed/save entries +elsewhere. Navigation stashes the in-progress draft so walking back through +history and forward again restores what the user was typing. +""" + +from __future__ import annotations + +DEFAULT_LIMIT = 200 + + +class InputHistory: + def __init__(self, entries: list[str] | None = None, limit: int = DEFAULT_LIMIT) -> None: + self._limit = max(1, limit) + self._entries: list[str] = list(entries or [])[-self._limit :] + self._cursor: int | None = None # None => not navigating + self._draft: str = "" + + def entries(self) -> list[str]: + return list(self._entries) + + def add(self, text: str) -> None: + """Record a submitted entry. Ignores blank and consecutive-duplicate lines.""" + self._cursor = None + self._draft = "" + if not text.strip(): + return + if self._entries and self._entries[-1] == text: + return + self._entries.append(text) + if len(self._entries) > self._limit: + self._entries = self._entries[-self._limit :] + + def up(self, draft: str = "") -> str: + """Move one entry older. Returns the entry (or ``draft`` if empty).""" + if not self._entries: + return draft + if self._cursor is None: + self._draft = draft + self._cursor = len(self._entries) - 1 + elif self._cursor > 0: + self._cursor -= 1 + return self._entries[self._cursor] + + def down(self) -> str: + """Move one entry newer. Past the newest entry, restores the draft.""" + if self._cursor is None: + return self._draft + if self._cursor < len(self._entries) - 1: + self._cursor += 1 + return self._entries[self._cursor] + self._cursor = None + return self._draft + + def reset(self) -> None: + self._cursor = None + self._draft = "" diff --git a/backend/packages/harness/deerflow/tui/message_format.py b/backend/packages/harness/deerflow/tui/message_format.py new file mode 100644 index 000000000..8dcfe348f --- /dev/null +++ b/backend/packages/harness/deerflow/tui/message_format.py @@ -0,0 +1,108 @@ +"""Compact, human-friendly formatting for tool activity in the TUI. + +Pure helpers: given a tool name + args (or a tool result), produce short, +readable strings for the transcript instead of dumping raw JSON. No Textual +dependency. +""" + +from __future__ import annotations + +import json +from typing import Any + +# Friendly titles for built-in tools. Anything not listed falls back to a +# humanized version of the raw name. +_TOOL_TITLES: dict[str, str] = { + "read_file": "Read", + "write_file": "Write", + "edit_file": "Edit", + "str_replace": "Edit", + "bash": "Bash", + "shell": "Shell", + "command": "Run", + "web_search": "Search", + "web_fetch": "Fetch", + "todo_write": "Todo", + "task": "Subagent", + "ls": "List", + "glob": "Find", + "grep": "Search", +} + +# Per-tool: which arg holds the single most salient value to show inline. +_DETAIL_KEYS: dict[str, tuple[str, ...]] = { + "read_file": ("path", "file_path", "filename"), + "write_file": ("path", "file_path", "filename"), + "edit_file": ("path", "file_path", "filename"), + "bash": ("command", "cmd"), + "shell": ("command", "cmd"), + "command": ("command", "cmd"), + "web_search": ("query", "q"), + "grep": ("pattern", "query"), + "glob": ("pattern",), + "web_fetch": ("url",), +} + +# Generic arg keys to try when a tool isn't in _DETAIL_KEYS. +_GENERIC_DETAIL_KEYS = ("path", "file_path", "command", "query", "url", "pattern", "name") + +DEFAULT_DETAIL_LIMIT = 80 +DEFAULT_RESULT_LIMIT = 160 + + +def truncate(text: str, limit: int) -> str: + """Truncate ``text`` to ``limit`` chars, appending an ellipsis marker.""" + if len(text) <= limit: + return text + return text[:limit].rstrip() + "…" + + +def summarize_tool_title(tool_name: str) -> str: + if not tool_name or not tool_name.strip(): + return "Tool" + if tool_name in _TOOL_TITLES: + return _TOOL_TITLES[tool_name] + return _humanize(tool_name) + + +def format_tool_detail(tool_name: str, args: Any, limit: int = DEFAULT_DETAIL_LIMIT) -> str: + """Return a short inline detail for a tool call (e.g. the path or command).""" + if not isinstance(args, dict) or not args: + return "" + + keys = _DETAIL_KEYS.get(tool_name, ()) + _GENERIC_DETAIL_KEYS + for key in keys: + value = args.get(key) + if isinstance(value, str) and value.strip(): + return truncate(_one_line(value), limit) + + # Fallback: compact JSON of the args. + try: + compact = json.dumps(args, ensure_ascii=False, separators=(",", ":")) + except (TypeError, ValueError): + compact = str(args) + return truncate(compact, limit) + + +def format_tool_result(result: Any, limit: int = DEFAULT_RESULT_LIMIT) -> str: + """Return a one-line, truncated preview of a tool result.""" + if result is None: + return "" + if not isinstance(result, str): + try: + result = json.dumps(result, ensure_ascii=False, separators=(",", ":")) + except (TypeError, ValueError): + result = str(result) + return truncate(_one_line(result), limit) + + +def _one_line(text: str) -> str: + """Collapse all runs of whitespace (incl. newlines) into single spaces.""" + return " ".join(text.split()) + + +def _humanize(name: str) -> str: + cleaned = name.replace("_", " ").replace("-", " ").strip() + if not cleaned: + return name + return " ".join(word[:1].upper() + word[1:] for word in cleaned.split()) diff --git a/backend/packages/harness/deerflow/tui/persistence.py b/backend/packages/harness/deerflow/tui/persistence.py new file mode 100644 index 000000000..122d3eac1 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/persistence.py @@ -0,0 +1,111 @@ +"""Shared-persistence wiring so terminal sessions show up in the Web UI. + +The Web UI lists conversations from the ``threads_meta`` SQL table (filtered by +``user_id``), not from the checkpointer. An embedded run only writes the +checkpointer, so a TUI thread would be invisible in the sidebar. This module +closes that gap: it writes a ``threads_meta`` row (owned by the local default +user) into the **same** database the Gateway reads — without requiring the +Gateway process to be running. + +Everything here is best-effort: when the database is memory-backed or +unavailable, the writer degrades to a no-op and the TUI keeps working. + +The SQLAlchemy async engine is bound to the event loop that created it, so all +DB work runs on one long-lived background loop (``_LoopThread``) rather than a +fresh ``asyncio.run`` per call (which would bind connections to throwaway loops). +""" + +from __future__ import annotations + +import asyncio +import threading +from collections.abc import Awaitable +from typing import Any + +from deerflow.runtime.user_context import DEFAULT_USER_ID + + +class _LoopThread: + """A daemon thread running a single asyncio event loop for DB work.""" + + def __init__(self) -> None: + self._loop = asyncio.new_event_loop() + self._thread = threading.Thread(target=self._run, name="deerflow-tui-db", daemon=True) + self._thread.start() + + def _run(self) -> None: + asyncio.set_event_loop(self._loop) + self._loop.run_forever() + + def run(self, coro: Awaitable[Any], *, timeout: float = 15.0) -> Any: + future = asyncio.run_coroutine_threadsafe(coro, self._loop) + return future.result(timeout) + + def close(self) -> None: + self._loop.call_soon_threadsafe(self._loop.stop) + + +class ThreadMetaWriter: + """Writes/updates ``threads_meta`` rows for the local default user. + + All methods swallow errors: persistence visibility is a convenience, never a + reason to break the conversation. + """ + + def __init__(self, loop: _LoopThread, store: Any) -> None: + self._loop = loop + self._store = store + self.user_id = DEFAULT_USER_ID + + @property + def enabled(self) -> bool: + return self._store is not None + + def ensure_created(self, thread_id: str, *, assistant_id: str | None = None, metadata: dict | None = None) -> None: + if not self._store or not thread_id: + return + try: + self._loop.run(self._ensure_created(thread_id, assistant_id, metadata)) + except Exception: # noqa: BLE001 - best-effort + pass + + async def _ensure_created(self, thread_id: str, assistant_id: str | None, metadata: dict | None) -> None: + existing = await self._store.get(thread_id, user_id=self.user_id) + if existing is None: + await self._store.create( + thread_id, + assistant_id=assistant_id, + user_id=self.user_id, + metadata=metadata or {"source": "tui"}, + ) + + def set_title(self, thread_id: str, title: str) -> None: + if not self._store or not thread_id or not title: + return + try: + self._loop.run(self._store.update_display_name(thread_id, title, user_id=self.user_id)) + except Exception: # noqa: BLE001 - best-effort + pass + + +def build_persistence() -> tuple[_LoopThread, ThreadMetaWriter]: + """Initialise the shared engine on a background loop and return a writer. + + Returns a ``ThreadMetaWriter`` that is a no-op when the configured database + backend is ``memory`` (no SQL session factory) or initialisation fails. + """ + loop = _LoopThread() + store = None + try: + from deerflow.config.app_config import get_app_config + from deerflow.persistence.engine import get_session_factory, init_engine_from_config + from deerflow.persistence.thread_meta import make_thread_store + + config = get_app_config() + loop.run(init_engine_from_config(config.database)) + session_factory = get_session_factory() + if session_factory is not None: + store = make_thread_store(session_factory) + except Exception: # noqa: BLE001 - degrade to no-op writer + store = None + return loop, ThreadMetaWriter(loop, store) diff --git a/backend/packages/harness/deerflow/tui/render.py b/backend/packages/harness/deerflow/tui/render.py new file mode 100644 index 000000000..195d9ced9 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/render.py @@ -0,0 +1,152 @@ +"""Pure Rich renderers for the transcript, status line and header. + +These take a :class:`ViewState` (plus light session info) and return Rich +renderables. No Textual import, so they can be unit-tested by rendering to a +Rich ``Console`` and inspecting the text. +""" + +from __future__ import annotations + +from rich.console import Group, RenderableType +from rich.markdown import Markdown +from rich.table import Table +from rich.text import Text + +from .theme import SYMBOLS, THEME +from .view_state import AssistantRow, Row, SystemRow, ToolRow, UserRow, ViewState + +_EMPTY_HINT = "Type a message to begin. Press / for commands, ? for help." + +_TOOL_STATUS_SYMBOL = {"running": SYMBOLS["running"], "ok": SYMBOLS["ok"], "error": SYMBOLS["error"]} +_TOOL_STATUS_STYLE = {"running": THEME.warning, "ok": THEME.accent, "error": THEME.error} + + +def render_transcript(state: ViewState) -> RenderableType: + if not state.rows: + return Text(_EMPTY_HINT, style=f"italic {THEME.dim}") + + # Only the message being generated right now renders as plain text (to avoid + # Markdown reflow jumpiness). Every other message — all history — renders as + # Markdown, so a follow-up turn never reverts prior answers to raw text. + blocks: list[RenderableType] = [] + for row in state.rows: + streaming_now = state.streaming and isinstance(row, AssistantRow) and row.id is not None and row.id == state.streaming_id + blocks.append(render_row(row, as_markdown=not streaming_now)) + blocks.append(Text("")) # one blank line between blocks for breathing room + return Group(*blocks[:-1]) + + +def render_row(row: Row, *, as_markdown: bool = True) -> RenderableType: + if isinstance(row, UserRow): + text = Text() + text.append(f"{SYMBOLS['user']} ", style=f"bold {THEME.user}") + text.append(row.text, style=f"bold {THEME.user}") + return text + + if isinstance(row, AssistantRow): + if not row.error and as_markdown and row.text.strip(): + return _assistant_markdown(row.text) + style = THEME.error if row.error else THEME.assistant + text = Text() + text.append(f"{SYMBOLS['assistant']} ", style=f"bold {style}") + text.append(row.text or "…", style=style) + return text + + if isinstance(row, ToolRow): + return _render_tool(row) + + if isinstance(row, SystemRow): + style = THEME.error if row.tone == "error" else THEME.dim + return Text(f"{SYMBOLS['system']} {row.text}", style=f"italic {style}") + + return Text(str(row)) + + +def _assistant_markdown(text: str) -> RenderableType: + """A ``●`` speaker marker aligned to the top of the Markdown-rendered body.""" + grid = Table.grid(padding=(0, 1, 0, 0)) + grid.add_column(width=1, vertical="top") # marker + grid.add_column(ratio=1) # markdown body + grid.add_row( + Text(SYMBOLS["assistant"], style=f"bold {THEME.assistant}"), + Markdown(text), + ) + return grid + + +def _render_tool(row: ToolRow) -> RenderableType: + head = Text() + head.append(f" {SYMBOLS['tool']} ", style=THEME.tool) + head.append(row.title, style=f"bold {THEME.tool}") + if row.detail: + head.append(f" {row.detail}", style=THEME.dim) + head.append(f" {_TOOL_STATUS_SYMBOL[row.status]}", style=_TOOL_STATUS_STYLE[row.status]) + + if row.result and row.status != "running": + return Group(head, Text(f" {row.result}", style=THEME.dim)) + return head + + +def render_status(state: ViewState, *, model: str, thread_label: str, spinner: str = "", elapsed: str = "") -> Text: + text = Text(no_wrap=True, overflow="ellipsis") + if state.streaming: + text.append(f"{spinner} working", style=f"bold {THEME.warning}") + else: + text.append("● ready", style=f"bold {THEME.accent}") + if state.title: + text.append(" ") + text.append(state.title, style=f"italic {THEME.muted}") + text.append(" ") + text.append(model or "default", style=THEME.primary) + text.append(" ") + text.append(thread_label, style=THEME.muted) + if elapsed: + text.append(" ") + text.append(elapsed, style=THEME.dim) + usage = state.usage or {} + total = usage.get("total_tokens") + if total: + text.append(" ") + text.append(f"{total} tok", style=THEME.dim) + if state.streaming: + text.append(" esc interrupt", style=THEME.dim) + return text + + +def render_palette(items, index: int, limit: int = 8) -> RenderableType: + """Render the slash-command picker: a windowed list with one highlighted row.""" + if not items: + return Text("") + index = max(0, min(index, len(items) - 1)) + start = index - limit + 1 if index >= limit else 0 + window = items[start : start + limit] + selected_in_window = index - start + + lines: list[RenderableType] = [] + for i, command in enumerate(window): + selected = i == selected_in_window + line = Text(no_wrap=True, overflow="ellipsis") + line.append("▌ " if selected else " ", style=THEME.primary) + line.append(f"/{command.name}", style=(f"bold {THEME.primary}" if selected else THEME.text)) + if command.description: + line.append(" ") + line.append(command.description, style=THEME.dim) + lines.append(line) + if len(items) > limit: + lines.append(Text(f" … {len(items) - limit} more", style=f"italic {THEME.dim}")) + return Group(*lines) + + +def render_header(*, model: str, thread_label: str, cwd: str, skills: int = 0) -> Text: + text = Text(no_wrap=True, overflow="ellipsis") + text.append(" DeerFlow ", style=f"bold {THEME.bg} on {THEME.primary}") + text.append(" ") + text.append(model or "default", style=f"bold {THEME.primary}") + text.append(" · ", style=THEME.dim) + text.append(thread_label, style=THEME.muted) + text.append(" · ", style=THEME.dim) + text.append(cwd, style=THEME.dim) + if skills: + text.append(" · ", style=THEME.dim) + text.append(f"{skills} skills", style=THEME.dim) + return text diff --git a/backend/packages/harness/deerflow/tui/runtime.py b/backend/packages/harness/deerflow/tui/runtime.py new file mode 100644 index 000000000..6b5568461 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/runtime.py @@ -0,0 +1,130 @@ +"""Runtime bridge between ``DeerFlowClient`` streaming and the view-state reducer. + +Two layers, both kept free of Textual: + +* :func:`translate` — pure: one ``StreamEvent`` -> zero or more reducer actions. +* :func:`stream_actions` — drives ``client.stream()`` and yields a bracketed + action sequence (``RunStarted`` … translated actions … ``RunEnded``), turning + model errors into an ``AssistantError`` row instead of crashing. + +The Textual app runs :func:`stream_actions` in a worker thread and applies each +yielded action to the reducer on the UI thread. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from typing import Any, Protocol + +from .view_state import ( + Action, + AssistantDelta, + AssistantError, + RunEnded, + RunStarted, + ThreadTitle, + ToolResult, + ToolStarted, +) + + +class _StreamEventLike(Protocol): + type: str + data: dict + + +class _ClientLike(Protocol): + def stream(self, message: str, *, thread_id: str | None = None, **kwargs: Any) -> Iterator[Any]: + """Yield streaming events for *message* (see ``DeerFlowClient.stream``).""" + + +def translate(event: _StreamEventLike) -> list[Action]: + """Map a single ``StreamEvent`` to reducer actions. Pure.""" + if event.type == "messages-tuple": + return _translate_message(event.data) + if event.type == "end": + usage = event.data.get("usage") if isinstance(event.data, dict) else None + return [RunEnded(usage=usage)] + if event.type == "values" and isinstance(event.data, dict): + title = event.data.get("title") + if isinstance(title, str) and title.strip(): + return [ThreadTitle(title=title.strip())] + return [] + # "custom" events are not rendered incrementally. + return [] + + +def _translate_message(data: Any) -> list[Action]: + if not isinstance(data, dict): + return [] + + message_type = data.get("type") + actions: list[Action] = [] + + if message_type == "ai": + text = _extract_text(data.get("content")) + if text: + actions.append(AssistantDelta(id=_as_str(data.get("id")), text=text)) + for tool_call in data.get("tool_calls") or []: + if not isinstance(tool_call, dict): + continue + actions.append( + ToolStarted( + tool_call_id=_as_str(tool_call.get("id")), + tool_name=_as_str(tool_call.get("name")), + args=tool_call.get("args") or {}, + ) + ) + elif message_type == "tool": + is_error = bool(data.get("is_error")) or data.get("status") == "error" + actions.append( + ToolResult( + tool_call_id=_as_str(data.get("tool_call_id")), + content=_extract_text(data.get("content")), + is_error=is_error, + tool_name=_as_str(data.get("name")), + ) + ) + + return actions + + +def _as_str(value: Any) -> str: + # Provider stream chunks can carry an explicit ``None`` id/name (the key is + # present, so ``.get(k, "")`` would return None, and ``str(None) == "None"`` + # — a truthy value that would defeat the empty-id guard downstream). + return "" if value is None else str(value) + + +def stream_actions(client: _ClientLike, message: str, *, thread_id: str | None = None, **kwargs: Any) -> Iterator[Action]: + """Yield a bracketed action stream for one agent run. + + Always begins with ``RunStarted`` and ends with ``RunEnded`` (even on error, + where an ``AssistantError`` row is emitted first). + """ + yield RunStarted() + try: + for event in client.stream(message, thread_id=thread_id, **kwargs): + yield from translate(event) + if event.type == "end": + return # RunEnded already emitted by translate() + yield RunEnded() + except Exception as exc: # noqa: BLE001 - surface any model/runtime error in-UI + yield AssistantError(str(exc) or exc.__class__.__name__) + yield RunEnded() + + +def _extract_text(content: Any) -> str: + if content is None: + return "" + if isinstance(content, str): + return content + if isinstance(content, list): + parts: list[str] = [] + for block in content: + if isinstance(block, str): + parts.append(block) + elif isinstance(block, dict) and block.get("type") == "text" and isinstance(block.get("text"), str): + parts.append(block["text"]) + return "".join(parts) + return str(content) diff --git a/backend/packages/harness/deerflow/tui/session.py b/backend/packages/harness/deerflow/tui/session.py new file mode 100644 index 000000000..e1effe32a --- /dev/null +++ b/backend/packages/harness/deerflow/tui/session.py @@ -0,0 +1,92 @@ +"""Embedded session wiring for the TUI. + +Owns construction of the ``DeerFlowClient`` (with a persistent checkpointer), +thread resolution for ``--continue`` / ``--resume`` (by id **or** title), and the +shared-persistence writer that makes terminal sessions visible in the Web UI (see +``deerflow.tui.persistence``). +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import TYPE_CHECKING + +if TYPE_CHECKING: # avoid importing the heavy client during pure planning + from deerflow.client import DeerFlowClient + + from .cli import LaunchPlan + from .persistence import ThreadMetaWriter, _LoopThread + + +@dataclass +class Session: + client: DeerFlowClient + writer: ThreadMetaWriter | None = None + _loop: _LoopThread | None = None + + def resolve_thread(self, plan: LaunchPlan) -> str | None: + """Resolve the thread id to run against, honoring --resume / --continue.""" + if plan.thread_id: + return self.resolve_ref(plan.thread_id) + if plan.continue_recent: + threads = self.client.list_threads(limit=1).get("thread_list", []) + if threads: + return threads[0].get("thread_id") + return None + + def resolve_ref(self, ref: str) -> str: + """Resolve a thread reference (id or title) to a thread id. + + Matches an existing thread by id first, then by exact title. Falls back to + the literal ref (treated as an id) when nothing matches, so an unknown id + still continues/creates that namespace. + """ + try: + threads = self.client.list_threads(limit=100).get("thread_list", []) + except Exception: # noqa: BLE001 - resolution is best-effort + return ref + if any(t.get("thread_id") == ref for t in threads): + return ref + for thread in threads: + if (thread.get("title") or "") == ref: + return thread.get("thread_id") or ref + return ref + + def recent_threads(self, limit: int = 20) -> list[dict]: + return self.client.list_threads(limit=limit).get("thread_list", []) + + def close(self) -> None: + """Stop the background DB loop and dispose the engine (best-effort).""" + loop = self._loop + if loop is None: + return + self._loop = None + try: + from deerflow.persistence.engine import close_engine + + loop.run(close_engine()) + except Exception: # noqa: BLE001 - teardown is best-effort + pass + loop.close() + + +def open_session(persistence: bool = True) -> Session: + """Build an embedded session backed by the configured checkpointer. + + ``persistence`` controls the shared ``threads_meta`` writer (and its background + DB loop/engine). Headless one-shots never use the writer, so they pass + ``persistence=False`` to avoid standing up an event loop + connection pool only + to discard it. + """ + from deerflow.client import DeerFlowClient + from deerflow.runtime.checkpointer.provider import get_checkpointer + + checkpointer = get_checkpointer() + client = DeerFlowClient(checkpointer=checkpointer) + if not persistence: + return Session(client=client) + + from .persistence import build_persistence + + loop, writer = build_persistence() + return Session(client=client, writer=writer, _loop=loop) diff --git a/backend/packages/harness/deerflow/tui/theme.py b/backend/packages/harness/deerflow/tui/theme.py new file mode 100644 index 000000000..4084ed324 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/theme.py @@ -0,0 +1,42 @@ +"""Restrained colour + symbol palette for the TUI. + +A Tokyo-Night-ish palette: calm, readable on dark terminals, with a few accent +hues to distinguish speakers and tool state. Rich-compatible hex colours so the +same constants drive both Rich renderables and Textual CSS variables. +""" + +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class Theme: + bg: str = "#1a1b26" + panel: str = "#1f2335" + border: str = "#2f334d" + text: str = "#c0caf5" + dim: str = "#565f89" + muted: str = "#737aa2" + + primary: str = "#7dcfff" # headings / app accent + user: str = "#7aa2f7" # user speaker + assistant: str = "#c0caf5" # assistant speaker + tool: str = "#bb9af7" # tool activity + accent: str = "#9ece6a" # success / ok + warning: str = "#e0af68" # running / caution + error: str = "#f7768e" # errors + + +THEME = Theme() + +SYMBOLS = { + "user": "›", + "assistant": "●", + "tool": "⚙", + "running": "◐", + "ok": "✓", + "error": "✗", + "system": "·", + "spinner": ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"], +} diff --git a/backend/packages/harness/deerflow/tui/view_state.py b/backend/packages/harness/deerflow/tui/view_state.py new file mode 100644 index 000000000..178e55b99 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/view_state.py @@ -0,0 +1,306 @@ +"""Pure view-state reducer for the DeerFlow TUI. + +This module has **no** Textual / rendering dependency. It models the visible +conversation as an immutable list of typed rows and a small set of actions, +and exposes a single pure ``reduce(state, action) -> state`` function. + +Keeping this layer pure makes the interesting behaviour (streaming deltas, +tool cards, error rows) testable with plain ``pytest`` and a handful of +synthetic actions, independent of any terminal. + +The runtime bridge (``deerflow.tui.runtime``) is responsible for translating +``DeerFlowClient`` ``StreamEvent`` objects into these actions; the Textual app +renders ``ViewState`` into widgets. Both sides depend on this module, not on +each other. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field, replace +from typing import Literal + +from .message_format import format_tool_detail, format_tool_result, summarize_tool_title + +# --------------------------------------------------------------------------- # +# Rows — the immutable units the transcript is built from. +# --------------------------------------------------------------------------- # + + +@dataclass(frozen=True) +class UserRow: + text: str + kind: Literal["user"] = "user" + + +@dataclass(frozen=True) +class AssistantRow: + text: str + id: str | None = None + error: bool = False + kind: Literal["assistant"] = "assistant" + + +@dataclass(frozen=True) +class ToolRow: + tool_call_id: str + tool_name: str + title: str + detail: str = "" + result: str = "" + status: Literal["running", "ok", "error"] = "running" + kind: Literal["tool"] = "tool" + + +@dataclass(frozen=True) +class SystemRow: + text: str + tone: Literal["info", "error"] = "info" + kind: Literal["system"] = "system" + + +Row = UserRow | AssistantRow | ToolRow | SystemRow + + +# --------------------------------------------------------------------------- # +# Actions — the only ways the state can change. +# --------------------------------------------------------------------------- # + + +@dataclass(frozen=True) +class UserSubmitted: + text: str + + +@dataclass(frozen=True) +class RunStarted: + pass + + +@dataclass(frozen=True) +class RunEnded: + usage: dict | None = None + + +@dataclass(frozen=True) +class AssistantDelta: + id: str + text: str + + +@dataclass(frozen=True) +class AssistantError: + text: str + + +@dataclass(frozen=True) +class ToolStarted: + tool_call_id: str + tool_name: str + args: dict = field(default_factory=dict) + + +@dataclass(frozen=True) +class ToolResult: + tool_call_id: str + content: str + is_error: bool = False + tool_name: str = "" + + +@dataclass(frozen=True) +class SystemMessage: + text: str + tone: Literal["info", "error"] = "info" + + +@dataclass(frozen=True) +class ThreadTitle: + title: str + + +@dataclass(frozen=True) +class ClearRows: + pass + + +Action = UserSubmitted | RunStarted | RunEnded | AssistantDelta | AssistantError | ToolStarted | ToolResult | SystemMessage | ThreadTitle | ClearRows + + +# --------------------------------------------------------------------------- # +# State. +# --------------------------------------------------------------------------- # + + +@dataclass(frozen=True) +class ViewState: + rows: tuple[Row, ...] = () + streaming: bool = False + usage: dict | None = None + title: str | None = None + # Id of the message currently being generated this turn. Only this row renders + # as plain text while streaming; everything else (history) stays Markdown. + streaming_id: str | None = None + + +def initial_state(rows: tuple[Row, ...] = ()) -> ViewState: + return ViewState(rows=tuple(rows)) + + +# --------------------------------------------------------------------------- # +# Reducer. +# --------------------------------------------------------------------------- # + + +def _append(state: ViewState, row: Row) -> ViewState: + return replace(state, rows=state.rows + (row,)) + + +def reduce(state: ViewState, action: Action) -> ViewState: + """Return a new ``ViewState`` after applying ``action``. Pure.""" + + if isinstance(action, UserSubmitted): + return _append(state, UserRow(text=action.text)) + + if isinstance(action, RunStarted): + # New turn: no message is actively streaming yet (the client re-emits + # prior messages first; those must not be treated as the active one). + return replace(state, streaming=True, streaming_id=None) + + if isinstance(action, RunEnded): + return replace( + state, + streaming=False, + streaming_id=None, + usage=action.usage if action.usage is not None else state.usage, + ) + + if isinstance(action, AssistantDelta): + return _apply_assistant_delta(state, action) + + if isinstance(action, AssistantError): + return _append(state, AssistantRow(text=action.text, error=True)) + + if isinstance(action, ToolStarted): + return _apply_tool_started(state, action) + + if isinstance(action, ToolResult): + return _apply_tool_result(state, action) + + if isinstance(action, SystemMessage): + return _append(state, SystemRow(text=action.text, tone=action.tone)) + + if isinstance(action, ThreadTitle): + return replace(state, title=action.title) + + if isinstance(action, ClearRows): + return replace(state, rows=(), title=None, streaming_id=None) + + return state + + +def _apply_assistant_delta(state: ViewState, action: AssistantDelta) -> ViewState: + """Update the assistant row with this id (anywhere in the transcript), or + start a new one. + + On a thread with history, the client re-emits every prior message on each + new turn (its dedup is per-turn), and a re-emitted *older* message can arrive + after a newer one has started — so we must match by id across the whole + transcript, not just the most recent assistant row, or prior answers get + duplicated. + + Updates also merge by content rather than blindly concatenating, to absorb + full re-sends / cumulative snapshots vs. genuine incremental deltas: + + * new text == accumulated, or starts with it -> cumulative/re-send: replace + * accumulated starts with new text -> stale/shorter re-send: keep + * otherwise -> a real delta: append + """ + + rows = list(state.rows) + for i, row in enumerate(rows): + # ``not row.error``: error rows are appended without an id, so they never + # match here anyway — the guard is belt-and-suspenders to keep an error + # row from being merged into if a future change ever gives it an id. + if isinstance(row, AssistantRow) and row.id == action.id and not row.error: + merged = _merge_stream_text(row.text, action.text) + if merged == row.text: + # No-op re-send (e.g. a values snapshot re-emitting history) — + # don't mark this as the actively-streaming message. + return state + rows[i] = replace(row, text=merged) + return _mark_streaming(replace(state, rows=tuple(rows)), action.id) + return _mark_streaming(_append(state, AssistantRow(text=action.text, id=action.id)), action.id) + + +def _mark_streaming(state: ViewState, message_id: str) -> ViewState: + """Record the actively-streaming message id (only while a run is active).""" + if state.streaming: + return replace(state, streaming_id=message_id) + return state + + +def _merge_stream_text(existing: str, incoming: str) -> str: + if not existing: + return incoming + if incoming.startswith(existing): + return incoming # cumulative snapshot or exact full re-send + if existing.startswith(incoming): + return existing # shorter/stale re-send + return existing + incoming # genuine incremental delta + + +def _apply_tool_started(state: ViewState, action: ToolStarted) -> ViewState: + """Create or update a tool card, de-duplicated by ``tool_call_id``. + + Streaming tool calls arrive as multiple chunks for one call id (name first, + then growing arguments), and the client may re-emit the call via a values + snapshot. Chunks with no id are argument-fragment noise and are dropped. + """ + if not action.tool_call_id: + return state + + rows = list(state.rows) + for i, row in enumerate(rows): + if isinstance(row, ToolRow) and row.tool_call_id == action.tool_call_id: + name = action.tool_name or row.tool_name + detail = format_tool_detail(name, action.args) or row.detail + rows[i] = replace(row, tool_name=name, title=summarize_tool_title(name), detail=detail) + return replace(state, rows=tuple(rows)) + + return _append( + state, + ToolRow( + tool_call_id=action.tool_call_id, + tool_name=action.tool_name, + title=summarize_tool_title(action.tool_name), + detail=format_tool_detail(action.tool_name, action.args), + status="running", + ), + ) + + +def _apply_tool_result(state: ViewState, action: ToolResult) -> ViewState: + if not action.tool_call_id: + return state + + rows = list(state.rows) + for i, row in enumerate(rows): + if isinstance(row, ToolRow) and row.tool_call_id == action.tool_call_id: + rows[i] = replace( + row, + status="error" if action.is_error else "ok", + result=format_tool_result(action.content), + ) + return replace(state, rows=tuple(rows)) + + # No matching tool card (started chunks missed) -> surface the result anyway. + return _append( + state, + ToolRow( + tool_call_id=action.tool_call_id, + tool_name=action.tool_name, + title=summarize_tool_title(action.tool_name), + status="error" if action.is_error else "ok", + result=format_tool_result(action.content), + ), + ) diff --git a/backend/packages/harness/deerflow/tui/widgets/__init__.py b/backend/packages/harness/deerflow/tui/widgets/__init__.py new file mode 100644 index 000000000..20536067c --- /dev/null +++ b/backend/packages/harness/deerflow/tui/widgets/__init__.py @@ -0,0 +1 @@ +"""Textual widgets for the DeerFlow TUI.""" diff --git a/backend/packages/harness/deerflow/tui/widgets/composer.py b/backend/packages/harness/deerflow/tui/widgets/composer.py new file mode 100644 index 000000000..48049ae51 --- /dev/null +++ b/backend/packages/harness/deerflow/tui/widgets/composer.py @@ -0,0 +1,23 @@ +"""Composer input with a wide-character (CJK) cursor fix. + +Textual's ``Input._cursor_offset`` adds an unconditional ``+1`` when the cursor +is at the end of the value. After double-width (CJK) characters that overshoots +by one cell, which misplaces the *hardware / IME* cursor — the visible drift when +typing Chinese in terminals such as iTerm2. (The on-screen block cursor is drawn +separately in ``render_line`` via character-index styling and is unaffected; this +only corrects the terminal cursor anchor that the IME candidate window follows.) + +English input never exercises this because it doesn't go through an IME, which is +why the drift only shows up for CJK. +""" + +from __future__ import annotations + +from textual.widgets import Input + + +class ComposerInput(Input): + @property + def _cursor_offset(self) -> int: + # True cell offset of the cursor, without Textual's end-of-value +1. + return self._position_to_cell(self.cursor_position) diff --git a/backend/packages/harness/pyproject.toml b/backend/packages/harness/pyproject.toml index a4bc6b28f..20e3a9f42 100644 --- a/backend/packages/harness/pyproject.toml +++ b/backend/packages/harness/pyproject.toml @@ -39,7 +39,13 @@ dependencies = [ "cryptography>=48.0.1", ] +[project.scripts] +deerflow = "deerflow.tui.cli:main" + [project.optional-dependencies] +# Terminal workbench (TUI). Kept optional so the core harness install stays lean; +# the `deerflow` console script degrades to headless help when textual is absent. +tui = ["textual>=0.80"] # GroundRoute needs no extra packages (httpx is already a core dependency). This # empty extra exists so the documented `uv add 'deerflow-harness[groundroute]'` # install command resolves cleanly without an "unknown extra" warning. diff --git a/backend/pyproject.toml b/backend/pyproject.toml index 8560563a1..0b4ca357a 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -34,6 +34,9 @@ dev = [ "pytest>=9.0.3", "pytest-asyncio>=1.3.0", "ruff>=0.14.11", + # TUI runtime dep (also declared as the deerflow-harness[tui] extra); kept in + # the dev group so the terminal workbench can be run and tested locally / in CI. + "textual>=0.80", ] [tool.pytest.ini_options] diff --git a/backend/tests/test_tui_app.py b/backend/tests/test_tui_app.py new file mode 100644 index 000000000..d9848540e --- /dev/null +++ b/backend/tests/test_tui_app.py @@ -0,0 +1,148 @@ +"""Integration tests for the Textual app via the pilot harness. + +Uses a fake in-process session so no real model is invoked. Exercises the full +loop: keypress -> submit -> worker thread -> stream_actions -> reducer -> state. +""" + +import asyncio + +import pytest + +from deerflow.client import StreamEvent +from deerflow.tui.app import DeerFlowTUI +from deerflow.tui.cli import LaunchPlan + + +class _FakeClient: + def list_models(self): + return {"models": [{"name": "fake-model", "display_name": "Fake Model"}]} + + def list_skills(self, enabled_only=False): + return {"skills": [{"name": "tdd", "enabled": True}]} + + def stream(self, message, *, thread_id=None, **kwargs): + yield StreamEvent(type="messages-tuple", data={"type": "ai", "content": "Hello ", "id": "m1"}) + yield StreamEvent(type="messages-tuple", data={"type": "ai", "content": "world", "id": "m1"}) + yield StreamEvent(type="end", data={"usage": {"total_tokens": 3}}) + + +class _FakeSession: + def __init__(self): + self.client = _FakeClient() + + def resolve_thread(self, plan): + return None + + +async def _wait_until(predicate, pilot, *, timeout=3.0): + deadline = 0.0 + while deadline < timeout: + await pilot.pause() + if predicate(): + return True + await asyncio.sleep(0.02) + deadline += 0.02 + return predicate() + + +@pytest.mark.asyncio +async def test_app_runs_a_turn_and_renders_streamed_assistant(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + await pilot.press("h", "i") + await pilot.press("enter") + await _wait_until( + lambda: not app._streaming and any(r.kind == "assistant" for r in app.state.rows), + pilot, + ) + + kinds = [r.kind for r in app.state.rows] + assert "user" in kinds + assert "assistant" in kinds + assistant = [r for r in app.state.rows if r.kind == "assistant"][-1] + assert assistant.text == "Hello world" + assert app.state.usage == {"total_tokens": 3} + + +@pytest.mark.asyncio +async def test_app_assigns_thread_id_on_first_send(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + assert app._conv_thread_id is None + await pilot.press("y", "o") + await pilot.press("enter") + await _wait_until(lambda: app._conv_thread_id is not None, pilot) + assert app._conv_thread_id is not None + + +@pytest.mark.asyncio +async def test_help_command_renders_system_row_without_calling_agent(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in "/help": + await pilot.press(ch) + await pilot.press("enter") + await _wait_until(lambda: any(r.kind == "system" for r in app.state.rows), pilot) + + assert any(r.kind == "system" for r in app.state.rows) + # /help must not produce a user turn or start streaming. + assert not any(r.kind == "user" for r in app.state.rows) + + +@pytest.mark.asyncio +async def test_up_arrow_recalls_previous_input_from_history(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in "remember me": + await pilot.press("space" if ch == " " else ch) + await pilot.press("enter") + await _wait_until(lambda: any(r.kind == "user" for r in app.state.rows), pilot) + # Composer is empty after submit; Up should recall the last entry. + await pilot.press("up") + await pilot.pause() + assert app.query_one("#composer").value == "remember me" + + +@pytest.mark.asyncio +async def test_escape_interrupts_an_active_run(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + app._streaming = True + app._refresh_status() + await pilot.press("escape") + await pilot.pause() + assert app._streaming is False + assert any(r.kind == "system" and "Interrupt" in r.text for r in app.state.rows) + + +@pytest.mark.asyncio +async def test_tab_keeps_focus_on_composer_when_palette_closed(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + composer = app.query_one("#composer") + assert app.focused is composer + await pilot.press("tab") + await pilot.pause() + # Tab must not move focus off the composer to the scroll region. + assert app.focused is composer + + +@pytest.mark.asyncio +async def test_unknown_command_shows_error_system_row(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in "/nope": + await pilot.press(ch) + await pilot.press("enter") + await _wait_until( + lambda: any(r.kind == "system" and getattr(r, "tone", "") == "error" for r in app.state.rows), + pilot, + ) + assert any(r.kind == "system" and r.tone == "error" for r in app.state.rows) diff --git a/backend/tests/test_tui_cli.py b/backend/tests/test_tui_cli.py new file mode 100644 index 000000000..47740c9b8 --- /dev/null +++ b/backend/tests/test_tui_cli.py @@ -0,0 +1,106 @@ +"""Tests for TUI launch-mode planning + arg parsing (pure, no Textual).""" + +from deerflow.tui.cli import LaunchPlan, plan_launch + + +def plan(argv, *, stdin_tty=True, stdout_tty=True, env=None): + return plan_launch(argv, stdin_isatty=stdin_tty, stdout_isatty=stdout_tty, env=env or {}) + + +def test_bare_command_on_tty_launches_tui(): + p = plan([]) + assert p.mode == "tui" + assert p.forced_tui is False + + +def test_non_tty_with_no_message_falls_back_to_headless_help(): + p = plan([], stdin_tty=False, stdout_tty=False) + assert p.mode == "headless-help" + assert p.reason + + +def test_print_with_message(): + p = plan(["--print", "summarize this repo"]) + assert p.mode == "print" + assert p.message == "summarize this repo" + assert p.read_stdin is False + + +def test_print_without_value_reads_stdin_when_piped(): + p = plan(["--print"], stdin_tty=False) + assert p.mode == "print" + assert p.read_stdin is True + + +def test_json_mode(): + p = plan(["--json", "hello"]) + assert p.mode == "json" + assert p.message == "hello" + + +def test_force_tui_even_without_tty(): + p = plan(["--tui"], stdin_tty=False, stdout_tty=False) + assert p.mode == "tui" + assert p.forced_tui is True + + +def test_env_var_forces_tui(): + p = plan([], stdin_tty=False, stdout_tty=False, env={"DEER_FLOW_TUI": "1"}) + assert p.mode == "tui" + + +def test_cli_flag_with_message_runs_print(): + p = plan(["--cli", "do", "this", "thing"]) + assert p.mode == "print" + assert p.message == "do this thing" + + +def test_cli_flag_without_message_is_headless_help(): + p = plan(["--cli"]) + assert p.mode == "headless-help" + + +def test_cli_continue_runs_headless_reading_stdin(): + p = plan(["--cli", "--continue"], stdin_tty=False) + assert p.mode == "print" + assert p.read_stdin is True + assert p.continue_recent is True + + +def test_cli_with_piped_stdin_runs_headless(): + p = plan(["--cli"], stdin_tty=False) + assert p.mode == "print" + assert p.read_stdin is True + + +def test_continue_recent_flag(): + p = plan(["--continue"]) + assert p.mode == "tui" + assert p.continue_recent is True + + +def test_resume_specific_thread(): + p = plan(["--resume", "thread-abc"]) + assert p.mode == "tui" + assert p.thread_id == "thread-abc" + + +def test_chat_subcommand_is_accepted(): + p = plan(["chat"]) + assert p.mode == "tui" + + +def test_chat_subcommand_with_print(): + p = plan(["chat", "--print", "hi"]) + assert p.mode == "print" + assert p.message == "hi" + + +def test_positional_message_becomes_initial_tui_prompt(): + p = plan(["explain", "the", "codebase"]) + assert p.mode == "tui" + assert p.message == "explain the codebase" + + +def test_plan_is_a_launch_plan_instance(): + assert isinstance(plan([]), LaunchPlan) diff --git a/backend/tests/test_tui_cli_main.py b/backend/tests/test_tui_cli_main.py new file mode 100644 index 000000000..efb5a6826 --- /dev/null +++ b/backend/tests/test_tui_cli_main.py @@ -0,0 +1,48 @@ +"""Integration tests for ``main`` dispatch (headless paths), with a fake session.""" + +import json + +from deerflow.client import StreamEvent +from deerflow.tui import cli + + +class _FakeClient: + def chat(self, message, *, thread_id=None, **kwargs): + return f"answer:{message}" + + def stream(self, message, *, thread_id=None, **kwargs): + yield StreamEvent(type="messages-tuple", data={"type": "ai", "content": "hi", "id": "m1"}) + yield StreamEvent(type="end", data={"usage": {"total_tokens": 1}}) + + +class _FakeSession: + def __init__(self): + self.client = _FakeClient() + + def resolve_thread(self, plan): + return None + + +def test_main_print_outputs_chat_answer(monkeypatch, capsys): + monkeypatch.setattr(cli, "_make_session", _FakeSession) + rc = cli.main(["--print", "hello"]) + assert rc == 0 + assert "answer:hello" in capsys.readouterr().out + + +def test_main_json_emits_ndjson_stream_events(monkeypatch, capsys): + monkeypatch.setattr(cli, "_make_session", _FakeSession) + rc = cli.main(["--json", "hello"]) + assert rc == 0 + lines = [ln for ln in capsys.readouterr().out.splitlines() if ln.strip()] + payloads = [json.loads(ln) for ln in lines] + assert payloads[0]["type"] == "messages-tuple" + assert payloads[-1]["type"] == "end" + + +def test_main_headless_help_returns_2_and_prints_usage(monkeypatch, capsys): + # On a TTY with no message and no piped stdin, --cli has nothing to run. + monkeypatch.setattr(cli.sys.stdin, "isatty", lambda: True) + rc = cli.main(["--cli"]) + assert rc == 2 + assert "deerflow" in capsys.readouterr().err diff --git a/backend/tests/test_tui_command_registry.py b/backend/tests/test_tui_command_registry.py new file mode 100644 index 000000000..151fd9176 --- /dev/null +++ b/backend/tests/test_tui_command_registry.py @@ -0,0 +1,94 @@ +"""Tests for the slash-command registry (pure).""" + +from deerflow.tui.command_registry import ( + BUILTIN_COMMANDS, + build_registry, + filter_commands, + resolve, +) + + +def _skills(): + return [ + {"name": "brainstorming", "description": "Explore ideas", "enabled": True}, + {"name": "tdd", "description": "Test driven dev", "enabled": True}, + {"name": "disabled-one", "description": "off", "enabled": False}, + ] + + +def test_build_registry_includes_all_builtins(): + registry = build_registry([]) + names = {c.name for c in registry} + for builtin in BUILTIN_COMMANDS: + assert builtin.name in names + + +def test_build_registry_adds_enabled_skill_commands_only(): + registry = build_registry(_skills()) + skill_names = {c.name for c in registry if c.category == "skill"} + assert "brainstorming" in skill_names + assert "tdd" in skill_names + assert "disabled-one" not in skill_names + + +def test_filter_empty_query_returns_all(): + registry = build_registry([]) + assert filter_commands(registry, "") == registry + + +def test_filter_matches_name_substring_case_insensitive(): + registry = build_registry([]) + results = filter_commands(registry, "MOD") + assert any(c.name == "model" for c in results) + + +def test_filter_matches_description(): + registry = build_registry(_skills()) + results = filter_commands(registry, "explore") + assert any(c.name == "brainstorming" for c in results) + + +def test_filter_ranks_prefix_matches_before_substring(): + registry = build_registry([]) + results = filter_commands(registry, "me") + # "memory" (prefix) should rank above a command that only contains "me" + names = [c.name for c in results] + assert "memory" in names + assert names.index("memory") == 0 + + +def test_resolve_plain_text_is_message(): + res = resolve("hello there") + assert res.kind == "message" + assert res.text == "hello there" + + +def test_resolve_builtin_command(): + res = resolve("/model") + assert res.kind == "builtin" + assert res.name == "model" + + +def test_resolve_builtin_with_args(): + res = resolve("/resume thread-123") + assert res.kind == "builtin" + assert res.name == "resume" + assert res.args == "thread-123" + + +def test_resolve_skill_activation(): + res = resolve("/tdd write the test first", skills=["tdd"]) + assert res.kind == "skill" + assert res.name == "tdd" + assert res.args == "write the test first" + + +def test_resolve_unknown_command(): + res = resolve("/definitely-not-a-command", skills=["tdd"]) + assert res.kind == "unknown" + assert res.name == "definitely-not-a-command" + + +def test_resolve_bare_slash_is_unknown_empty(): + res = resolve("/") + assert res.kind == "unknown" diff --git a/backend/tests/test_tui_composer.py b/backend/tests/test_tui_composer.py new file mode 100644 index 000000000..84cca12e0 --- /dev/null +++ b/backend/tests/test_tui_composer.py @@ -0,0 +1,46 @@ +"""Tests for the CJK-aware composer cursor offset.""" + +import pytest +from textual.app import App, ComposeResult + +from deerflow.tui.widgets.composer import ComposerInput + + +class _Harness(App): + def compose(self) -> ComposeResult: + yield ComposerInput(id="c") + + +@pytest.mark.asyncio +async def test_cursor_offset_after_cjk_has_no_off_by_one(): + app = _Harness() + async with app.run_test() as pilot: + await pilot.pause() + comp = app.query_one("#c", ComposerInput) + comp.value = "总结一下" # 4 wide chars = 8 cells + comp.cursor_position = len(comp.value) + # Stock Input would report 9 here (unconditional +1); the fix reports 8, + # so the hardware/IME cursor sits exactly after the last character. + assert comp._cursor_offset == 8 + + +@pytest.mark.asyncio +async def test_cursor_offset_mid_text_is_unchanged(): + app = _Harness() + async with app.run_test() as pilot: + await pilot.pause() + comp = app.query_one("#c", ComposerInput) + comp.value = "总结一下" + comp.cursor_position = 3 # not at end -> 3 wide chars = 6 cells + assert comp._cursor_offset == 6 + + +@pytest.mark.asyncio +async def test_cursor_offset_ascii_end_is_exact(): + app = _Harness() + async with app.run_test() as pilot: + await pilot.pause() + comp = app.query_one("#c", ComposerInput) + comp.value = "abcd" + comp.cursor_position = 4 + assert comp._cursor_offset == 4 diff --git a/backend/tests/test_tui_input_history.py b/backend/tests/test_tui_input_history.py new file mode 100644 index 000000000..34062d7a6 --- /dev/null +++ b/backend/tests/test_tui_input_history.py @@ -0,0 +1,61 @@ +"""Tests for bounded composer input history (pure).""" + +from deerflow.tui.input_history import InputHistory + + +def test_add_ignores_empty_and_whitespace(): + h = InputHistory() + h.add("") + h.add(" \n") + assert h.entries() == [] + + +def test_add_ignores_consecutive_duplicate(): + h = InputHistory() + h.add("same") + h.add("same") + assert h.entries() == ["same"] + + +def test_add_keeps_non_consecutive_duplicates(): + h = InputHistory() + h.add("a") + h.add("b") + h.add("a") + assert h.entries() == ["a", "b", "a"] + + +def test_add_bounds_to_limit_dropping_oldest(): + h = InputHistory(limit=3) + for text in ["1", "2", "3", "4"]: + h.add(text) + assert h.entries() == ["2", "3", "4"] + + +def test_up_walks_back_and_stops_at_oldest(): + h = InputHistory(["first", "second", "third"]) + assert h.up() == "third" + assert h.up() == "second" + assert h.up() == "first" + assert h.up() == "first" # clamped at oldest + + +def test_down_walks_forward_then_restores_draft(): + h = InputHistory(["first", "second"]) + assert h.up(draft="my draft") == "second" + assert h.up() == "first" + assert h.down() == "second" + assert h.down() == "my draft" # past newest -> stashed draft + + +def test_up_with_empty_history_returns_draft(): + h = InputHistory() + assert h.up(draft="keep") == "keep" + + +def test_add_resets_navigation_cursor(): + h = InputHistory(["old"]) + h.up() + h.add("new") + # After adding, up() starts again from the newest entry. + assert h.up() == "new" diff --git a/backend/tests/test_tui_message_format.py b/backend/tests/test_tui_message_format.py new file mode 100644 index 000000000..6575276f3 --- /dev/null +++ b/backend/tests/test_tui_message_format.py @@ -0,0 +1,53 @@ +"""Tests for compact tool-activity formatting helpers (pure).""" + +from deerflow.tui.message_format import ( + format_tool_detail, + format_tool_result, + summarize_tool_title, + truncate, +) + + +def test_summarize_known_tool_titles(): + assert summarize_tool_title("read_file") == "Read" + assert summarize_tool_title("write_file") == "Write" + assert summarize_tool_title("bash") == "Bash" + + +def test_summarize_unknown_tool_falls_back_to_humanized_name(): + assert summarize_tool_title("my_custom_tool") == "My Custom Tool" + + +def test_format_tool_detail_extracts_salient_arg(): + assert format_tool_detail("read_file", {"path": "src/app.py"}) == "src/app.py" + assert format_tool_detail("bash", {"command": "ls -la"}) == "ls -la" + assert format_tool_detail("web_search", {"query": "deerflow tui"}) == "deerflow tui" + + +def test_format_tool_detail_unknown_args_compact_json(): + detail = format_tool_detail("mystery", {"a": 1, "b": 2}) + assert "a" in detail and "1" in detail + + +def test_format_tool_detail_empty_args_is_empty_string(): + assert format_tool_detail("bash", {}) == "" + + +def test_truncate_short_text_unchanged(): + assert truncate("hello", 80) == "hello" + + +def test_truncate_long_text_adds_marker(): + out = truncate("x" * 200, 50) + assert len(out) <= 50 + 1 # marker char + assert out.endswith("…") + + +def test_format_tool_result_collapses_whitespace_and_truncates(): + result = format_tool_result("line1\n\n line2 \n", limit=80) + assert "line1" in result and "line2" in result + assert "\n" not in result + + +def test_format_tool_result_handles_non_string(): + assert format_tool_result({"ok": True}) != "" diff --git a/backend/tests/test_tui_overlays.py b/backend/tests/test_tui_overlays.py new file mode 100644 index 000000000..1c2682c0d --- /dev/null +++ b/backend/tests/test_tui_overlays.py @@ -0,0 +1,127 @@ +"""Integration tests for modal overlays: /model picker and /threads switcher.""" + +import asyncio + +import pytest + +from deerflow.client import StreamEvent +from deerflow.tui.app import DeerFlowTUI, SelectScreen +from deerflow.tui.cli import LaunchPlan + + +class _FakeClient: + def list_models(self): + return {"models": [{"name": "fast", "display_name": "Fast"}, {"name": "smart", "display_name": "Smart"}]} + + def list_skills(self, enabled_only=False): + return {"skills": []} + + def list_threads(self, limit=10): + return { + "thread_list": [ + {"thread_id": "thread-aaaaaaaa", "title": "Refactor bridge"}, + {"thread_id": "thread-bbbbbbbb", "title": "Write docs"}, + ] + } + + def stream(self, *args, **kwargs): + yield StreamEvent(type="end", data={}) + + +class _FakeSession: + def __init__(self): + self.client = _FakeClient() + + def resolve_thread(self, plan): + return None + + def recent_threads(self, limit=20): + return self.client.list_threads(limit=limit)["thread_list"] + + def resolve_ref(self, ref): + threads = self.client.list_threads(limit=100)["thread_list"] + if any(t["thread_id"] == ref for t in threads): + return ref + for t in threads: + if (t.get("title") or "") == ref: + return t["thread_id"] + return ref + + +async def _settle(pilot, predicate, timeout=2.0): + elapsed = 0.0 + while elapsed < timeout: + await pilot.pause() + if predicate(): + return True + await asyncio.sleep(0.02) + elapsed += 0.02 + return predicate() + + +@pytest.mark.asyncio +async def test_resume_command_with_title_switches_thread(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + # Resolve a thread by its title (not id) — the by-title resume path. + app.query_one("#composer").value = "/resume Refactor bridge" + await pilot.press("enter") + await _settle(pilot, lambda: app._conv_thread_id == "thread-aaaaaaaa") + assert app._conv_thread_id == "thread-aaaaaaaa" + assert any(r.kind == "system" and "Resumed" in r.text for r in app.state.rows) + + +def test_resume_without_arg_routes_to_thread_switcher(): + # /resume with no id/title falls back to the thread switcher. + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + calls = [] + app._open_thread_switcher = lambda: calls.append("switcher") + app._handle_builtin("resume", "") + assert calls == ["switcher"] + + +@pytest.mark.asyncio +async def test_model_command_opens_picker_and_sets_override(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in ("slash", "m", "o", "d", "e", "l"): + await pilot.press(ch) + await _settle(pilot, lambda: app._palette_open and any(c.name == "model" for c in app._palette_items)) + await pilot.press("enter") # run /model -> opens picker + await _settle(pilot, lambda: isinstance(app.screen, SelectScreen)) + await pilot.press("enter") # choose first model (Fast) + await _settle(pilot, lambda: app._model_override is not None) + assert app._model_override == "fast" + assert any(r.kind == "system" and "Fast" in r.text or "fast" in r.text for r in app.state.rows) + + +@pytest.mark.asyncio +async def test_threads_command_opens_switcher_and_resumes(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in ("slash", "t", "h", "r", "e", "a", "d", "s"): + await pilot.press(ch) + await _settle(pilot, lambda: app._palette_open and any(c.name == "threads" for c in app._palette_items)) + await pilot.press("enter") # run /threads -> opens switcher + await _settle(pilot, lambda: isinstance(app.screen, SelectScreen)) + await pilot.press("enter") # choose first thread + await _settle(pilot, lambda: app._conv_thread_id == "thread-aaaaaaaa") + assert app._conv_thread_id == "thread-aaaaaaaa" + + +@pytest.mark.asyncio +async def test_picker_escape_cancels_without_change(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in ("slash", "m", "o", "d", "e", "l"): + await pilot.press(ch) + await _settle(pilot, lambda: app._palette_open) + await pilot.press("enter") + await _settle(pilot, lambda: isinstance(app.screen, SelectScreen)) + await pilot.press("escape") + await _settle(pilot, lambda: not isinstance(app.screen, SelectScreen)) + assert app._model_override is None diff --git a/backend/tests/test_tui_palette.py b/backend/tests/test_tui_palette.py new file mode 100644 index 000000000..7c99d30c2 --- /dev/null +++ b/backend/tests/test_tui_palette.py @@ -0,0 +1,116 @@ +"""Integration tests for the slash-command palette via the pilot harness.""" + +import asyncio + +import pytest + +from deerflow.client import StreamEvent +from deerflow.tui.app import DeerFlowTUI +from deerflow.tui.cli import LaunchPlan + + +class _FakeClient: + def list_models(self): + return {"models": [{"name": "m"}]} + + def list_skills(self, enabled_only=False): + return {"skills": [{"name": "tdd", "description": "Test first", "enabled": True}]} + + def stream(self, *args, **kwargs): + yield StreamEvent(type="end", data={}) + + +class _FakeSession: + def __init__(self): + self.client = _FakeClient() + + def resolve_thread(self, plan): + return None + + +async def _settle(pilot, predicate, timeout=2.0): + elapsed = 0.0 + while elapsed < timeout: + await pilot.pause() + if predicate(): + return True + await asyncio.sleep(0.02) + elapsed += 0.02 + return predicate() + + +@pytest.mark.asyncio +async def test_typing_slash_opens_palette_with_matches(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + await pilot.press("slash", "h", "e") + await _settle(pilot, lambda: app._palette_open) + assert app._palette_open + assert "help" in [c.name for c in app._palette_items] + + +@pytest.mark.asyncio +async def test_palette_index_resets_when_filter_changes(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + await pilot.press("slash", "m") # memory / mcp / model … + await _settle(pilot, lambda: app._palette_open and len(app._palette_items) > 1) + await pilot.press("down", "down") + await pilot.pause() + assert app._palette_index > 0 + await pilot.press("e") # filter narrows ("/me") -> highlight must reset + await pilot.pause() + assert app._palette_index == 0 + + +@pytest.mark.asyncio +async def test_palette_enter_runs_builtin_and_closes(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + for ch in ("slash", "h", "e", "l", "p"): + await pilot.press(ch) + await _settle(pilot, lambda: app._palette_open and bool(app._palette_items)) + await pilot.press("enter") + await _settle(pilot, lambda: any(r.kind == "system" for r in app.state.rows)) + assert any(r.kind == "system" for r in app.state.rows) + assert not app._palette_open + # /help is a builtin and must not have triggered an agent user turn. + assert not any(r.kind == "user" for r in app.state.rows) + + +@pytest.mark.asyncio +async def test_escape_closes_palette(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + await pilot.press("slash", "m") + await _settle(pilot, lambda: app._palette_open) + await pilot.press("escape") + await _settle(pilot, lambda: not app._palette_open) + assert not app._palette_open + + +@pytest.mark.asyncio +async def test_skill_command_tab_completes_with_trailing_space(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + await pilot.press("slash", "t", "d", "d") + await _settle(pilot, lambda: app._palette_open and any(c.name == "tdd" for c in app._palette_items)) + await pilot.press("tab") + await _settle(pilot, lambda: not app._palette_open) + value = app.query_one("#composer").value + assert value == "/tdd " + + +@pytest.mark.asyncio +async def test_normal_text_does_not_open_palette(): + app = DeerFlowTUI(_FakeSession(), LaunchPlan(mode="tui")) + async with app.run_test() as pilot: + await pilot.pause() + await pilot.press("h", "i") + await pilot.pause() + assert not app._palette_open diff --git a/backend/tests/test_tui_palette_render.py b/backend/tests/test_tui_palette_render.py new file mode 100644 index 000000000..e6f9bc184 --- /dev/null +++ b/backend/tests/test_tui_palette_render.py @@ -0,0 +1,44 @@ +"""Tests for the slash-command palette renderer (pure).""" + +from rich.console import Console + +from deerflow.tui.command_registry import build_registry +from deerflow.tui.render import render_palette + + +def _text(renderable) -> str: + console = Console(width=80, no_color=True) + with console.capture() as cap: + console.print(renderable) + return cap.get() + + +def test_empty_items_render_nothing(): + assert _text(render_palette([], 0)).strip() == "" + + +def test_lists_commands_with_descriptions(): + registry = build_registry([]) + out = _text(render_palette(registry, 0, limit=5)) + assert "/help" in out + assert "Show commands" in out + + +def test_highlight_marker_present_on_selected_row(): + registry = build_registry([]) + out = _text(render_palette(registry, 0, limit=5)) + assert "▌" in out + + +def test_windowing_shows_more_indicator_when_truncated(): + registry = build_registry([]) + out = _text(render_palette(registry, 0, limit=3)) + assert "more" in out + + +def test_window_follows_selection_index(): + registry = build_registry([]) + # Selecting an index beyond the first window must keep that command visible. + target = registry[6] + out = _text(render_palette(registry, 6, limit=4)) + assert f"/{target.name}" in out diff --git a/backend/tests/test_tui_persistence.py b/backend/tests/test_tui_persistence.py new file mode 100644 index 000000000..0d4d1fd4a --- /dev/null +++ b/backend/tests/test_tui_persistence.py @@ -0,0 +1,63 @@ +"""Tests for the shared-persistence writer (thread_meta visibility). + +Uses the in-memory ThreadMetaStore so no SQL engine is required, but exercises +the real async store + background-loop wiring used by the TUI. +""" + +import pytest +from langgraph.store.memory import InMemoryStore + +from deerflow.persistence.thread_meta import make_thread_store +from deerflow.tui.persistence import ThreadMetaWriter, _LoopThread + + +@pytest.fixture +def writer_store_loop(): + loop = _LoopThread() + store = make_thread_store(None, store=InMemoryStore()) + writer = ThreadMetaWriter(loop, store) + try: + yield writer, store, loop + finally: + loop.close() + + +def test_writer_is_enabled_with_a_store(writer_store_loop): + writer, _store, _loop = writer_store_loop + assert writer.enabled is True + assert writer.user_id == "default" + + +def test_ensure_created_writes_row_owned_by_default_user(writer_store_loop): + writer, store, loop = writer_store_loop + writer.ensure_created("th-1", assistant_id="lead-agent", metadata={"source": "tui"}) + rows = loop.run(store.search(user_id="default")) + assert "th-1" in [r["thread_id"] for r in rows] + + +def test_ensure_created_is_idempotent(writer_store_loop): + writer, store, loop = writer_store_loop + writer.ensure_created("th-1") + writer.ensure_created("th-1") + rows = loop.run(store.search(user_id="default")) + assert sum(1 for r in rows if r["thread_id"] == "th-1") == 1 + + +def test_set_title_updates_display_name(writer_store_loop): + writer, store, loop = writer_store_loop + writer.ensure_created("th-1") + writer.set_title("th-1", "Refactor the bridge") + row = loop.run(store.get("th-1", user_id="default")) + assert row["display_name"] == "Refactor the bridge" + + +def test_disabled_writer_is_a_silent_noop(): + loop = _LoopThread() + try: + writer = ThreadMetaWriter(loop, None) + assert writer.enabled is False + # Must not raise even though there is no store. + writer.ensure_created("x", assistant_id="lead-agent") + writer.set_title("x", "title") + finally: + loop.close() diff --git a/backend/tests/test_tui_render.py b/backend/tests/test_tui_render.py new file mode 100644 index 000000000..f17b8be90 --- /dev/null +++ b/backend/tests/test_tui_render.py @@ -0,0 +1,118 @@ +"""Smoke tests for the pure Rich renderers — they must render without error +and include the expected text.""" + +from rich.console import Console + +from deerflow.tui.render import render_header, render_status, render_transcript +from deerflow.tui.view_state import ( + AssistantDelta, + RunEnded, + RunStarted, + SystemMessage, + ToolResult, + ToolStarted, + UserSubmitted, + initial_state, + reduce, +) + + +def _render_to_text(renderable) -> str: + console = Console(width=100, no_color=True) + with console.capture() as capture: + console.print(renderable) + return capture.get() + + +def test_render_empty_transcript_shows_hint(): + out = _render_to_text(render_transcript(initial_state())) + assert "Type a message" in out + + +def test_render_transcript_includes_all_row_kinds(): + state = initial_state() + state = reduce(state, UserSubmitted("hello there")) + state = reduce(state, AssistantDelta(id="m1", text="hi back")) + state = reduce(state, ToolStarted(tool_call_id="t1", tool_name="read_file", args={"path": "a.py"})) + state = reduce(state, ToolResult(tool_call_id="t1", content="file body", is_error=False)) + state = reduce(state, SystemMessage("a note")) + + out = _render_to_text(render_transcript(state)) + assert "hello there" in out + assert "hi back" in out + assert "Read" in out and "a.py" in out + assert "file body" in out + assert "a note" in out + + +def test_finalized_assistant_renders_markdown(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="**bold** text\n\n## A Heading\n\n- item one")) + out = _render_to_text(render_transcript(state)) + # Markdown is rendered: the syntax markers are consumed, the content remains. + assert "bold" in out + assert "**bold**" not in out + assert "A Heading" in out + assert "## A Heading" not in out + assert "item one" in out + + +def test_actively_streaming_assistant_stays_plain(): + state = initial_state() + state = reduce(state, RunStarted()) + state = reduce(state, AssistantDelta(id="m1", text="**partial heading ##")) + out = _render_to_text(render_transcript(state)) + # The streaming row must NOT be markdown-rendered (avoids reflow jumpiness). + assert "**partial heading ##" in out + + +def test_prior_message_stays_markdown_when_a_followup_run_starts(): + # Regression: sending a follow-up must NOT revert a finalized markdown answer + # to raw text. Between RunStarted and the new answer's first delta (and during + # the client's re-emit of prior messages), the previous answer is the last + # assistant row — it must still render as Markdown. + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="**bold answer**")) + state = reduce(state, RunEnded()) + state = reduce(state, UserSubmitted("follow up")) + state = reduce(state, RunStarted()) + state = reduce(state, AssistantDelta(id="m1", text="**bold answer**")) # re-emit, no new answer yet + + out = _render_to_text(render_transcript(state)) + assert "**bold answer**" not in out # still Markdown-rendered + assert "bold answer" in out + + +def test_only_the_actively_streaming_message_is_plain(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="**done**")) + state = reduce(state, RunEnded()) + state = reduce(state, RunStarted()) + state = reduce(state, AssistantDelta(id="m2", text="**streaming now")) + + out = _render_to_text(render_transcript(state)) + assert "**streaming now" in out # active m2 stays plain + assert "**done**" not in out # finalized m1 stays Markdown + assert "done" in out + + +def test_render_status_ready_and_working(): + ready = _render_to_text(render_status(initial_state(), model="gpt", thread_label="new")) + assert "ready" in ready + + state = reduce(initial_state(), RunStarted()) + working = _render_to_text(render_status(state, model="gpt", thread_label="new", spinner="*")) + assert "working" in working + + +def test_render_status_shows_token_usage(): + state = reduce(initial_state(), RunEnded(usage={"total_tokens": 42})) + out = _render_to_text(render_status(state, model="gpt", thread_label="t1")) + assert "42 tok" in out + + +def test_render_header_includes_model_and_cwd(): + out = _render_to_text(render_header(model="claude", thread_label="new", cwd="/tmp/proj", skills=3)) + assert "DeerFlow" in out + assert "claude" in out + assert "/tmp/proj" in out diff --git a/backend/tests/test_tui_runtime.py b/backend/tests/test_tui_runtime.py new file mode 100644 index 000000000..9b69fcf75 --- /dev/null +++ b/backend/tests/test_tui_runtime.py @@ -0,0 +1,145 @@ +"""Tests for the runtime bridge: StreamEvent -> reducer actions. + +The translation layer is pure and is exercised here against real +``StreamEvent`` objects plus a fake client, with no Textual involved. +""" + +from deerflow.client import StreamEvent +from deerflow.tui.runtime import stream_actions, translate +from deerflow.tui.view_state import ( + AssistantDelta, + AssistantError, + RunEnded, + RunStarted, + ThreadTitle, + ToolResult, + ToolStarted, + initial_state, + reduce, +) + + +def test_translate_ai_text_delta(): + event = StreamEvent(type="messages-tuple", data={"type": "ai", "content": "Hello", "id": "m1"}) + actions = translate(event) + assert actions == [AssistantDelta(id="m1", text="Hello")] + + +def test_translate_ai_tool_call_emits_tool_started_not_empty_delta(): + event = StreamEvent( + type="messages-tuple", + data={ + "type": "ai", + "content": "", + "id": "m1", + "tool_calls": [{"name": "read_file", "args": {"path": "a.py"}, "id": "t1"}], + }, + ) + actions = translate(event) + assert actions == [ToolStarted(tool_call_id="t1", tool_name="read_file", args={"path": "a.py"})] + + +def test_translate_ai_content_blocks_list_extracts_text(): + event = StreamEvent( + type="messages-tuple", + data={"type": "ai", "content": [{"type": "text", "text": "abc"}, {"type": "text", "text": "def"}], "id": "m9"}, + ) + actions = translate(event) + assert actions == [AssistantDelta(id="m9", text="abcdef")] + + +def test_translate_tool_call_with_none_id_yields_empty_id(): + # Some providers' first tool-call chunk has id=None; it must coerce to "" (not + # "None"), so the empty-id guard in the reducer drops the noise chunk. + event = StreamEvent( + type="messages-tuple", + data={"type": "ai", "content": "", "id": "m1", "tool_calls": [{"id": None, "name": None, "args": {}}]}, + ) + assert translate(event) == [ToolStarted(tool_call_id="", tool_name="", args={})] + + +def test_translate_tool_result_with_none_id_yields_empty_id(): + event = StreamEvent(type="messages-tuple", data={"type": "tool", "content": "x", "name": None, "tool_call_id": None}) + assert translate(event) == [ToolResult(tool_call_id="", content="x", is_error=False, tool_name="")] + + +def test_translate_tool_result_with_error_status(): + event = StreamEvent( + type="messages-tuple", + data={"type": "tool", "content": "boom", "name": "bash", "tool_call_id": "t1", "status": "error"}, + ) + actions = translate(event) + assert actions == [ToolResult(tool_call_id="t1", content="boom", is_error=True, tool_name="bash")] + + +def test_translate_end_event_carries_usage(): + usage = {"input_tokens": 3, "output_tokens": 7, "total_tokens": 10} + actions = translate(StreamEvent(type="end", data={"usage": usage})) + assert actions == [RunEnded(usage=usage)] + + +def test_translate_values_surfaces_title_only(): + assert translate(StreamEvent(type="values", data={"title": "My Thread", "messages": []})) == [ThreadTitle(title="My Thread")] + assert translate(StreamEvent(type="values", data={"title": None, "messages": []})) == [] + assert translate(StreamEvent(type="custom", data={"anything": 1})) == [] + + +class _FakeClient: + def __init__(self, events): + self._events = events + self.calls = [] + + def stream(self, message, *, thread_id=None, **kwargs): + self.calls.append((message, thread_id)) + yield from self._events + + +def test_stream_actions_brackets_with_run_started_and_ended(): + client = _FakeClient( + [ + StreamEvent(type="messages-tuple", data={"type": "ai", "content": "Hi", "id": "m1"}), + StreamEvent(type="end", data={"usage": {"total_tokens": 5}}), + ] + ) + actions = list(stream_actions(client, "hello", thread_id="th-1")) + assert isinstance(actions[0], RunStarted) + assert isinstance(actions[-1], RunEnded) + assert client.calls == [("hello", "th-1")] + + +def test_stream_actions_reduces_to_expected_transcript(): + client = _FakeClient( + [ + StreamEvent(type="messages-tuple", data={"type": "ai", "content": "Let me look. ", "id": "m1"}), + StreamEvent( + type="messages-tuple", + data={"type": "ai", "content": "", "id": "m1", "tool_calls": [{"name": "read_file", "args": {"path": "a.py"}, "id": "t1"}]}, + ), + StreamEvent(type="messages-tuple", data={"type": "tool", "content": "file body", "name": "read_file", "tool_call_id": "t1"}), + StreamEvent(type="messages-tuple", data={"type": "ai", "content": "Done.", "id": "m2"}), + StreamEvent(type="end", data={"usage": {"total_tokens": 9}}), + ] + ) + state = initial_state() + for action in stream_actions(client, "go"): + state = reduce(state, action) + + kinds = [r.kind for r in state.rows] + assert kinds == ["assistant", "tool", "assistant"] + assert state.rows[0].text == "Let me look. " + assert state.rows[1].status == "ok" + assert state.rows[2].text == "Done." + assert state.streaming is False + assert state.usage == {"total_tokens": 9} + + +class _BoomClient: + def stream(self, message, *, thread_id=None, **kwargs): + yield StreamEvent(type="messages-tuple", data={"type": "ai", "content": "partial", "id": "m1"}) + raise RuntimeError("model down") + + +def test_stream_actions_surfaces_exception_as_error_then_ends(): + actions = list(stream_actions(_BoomClient(), "go")) + assert any(isinstance(a, AssistantError) and "model down" in a.text for a in actions) + assert isinstance(actions[-1], RunEnded) diff --git a/backend/tests/test_tui_session.py b/backend/tests/test_tui_session.py new file mode 100644 index 000000000..0811d0a2c --- /dev/null +++ b/backend/tests/test_tui_session.py @@ -0,0 +1,59 @@ +"""Tests for Session thread resolution (id-or-title) and lifecycle.""" + +from deerflow.tui.cli import LaunchPlan +from deerflow.tui.session import Session + + +class _Client: + def __init__(self, threads): + self._threads = threads + + def list_threads(self, limit=10): + return {"thread_list": self._threads} + + +def _session(threads): + return Session(client=_Client(threads)) + + +THREADS = [ + {"thread_id": "thread-aaaa", "title": "Bug triage"}, + {"thread_id": "thread-bbbb", "title": "Write docs"}, +] + + +def test_resolve_ref_matches_id(): + assert _session(THREADS).resolve_ref("thread-bbbb") == "thread-bbbb" + + +def test_resolve_ref_matches_title(): + assert _session(THREADS).resolve_ref("Bug triage") == "thread-aaaa" + + +def test_resolve_ref_unknown_falls_back_to_literal_id(): + assert _session(THREADS).resolve_ref("brand-new-id") == "brand-new-id" + + +def test_resolve_thread_resolves_title_from_plan(): + plan = LaunchPlan(mode="tui", thread_id="Write docs") + assert _session(THREADS).resolve_thread(plan) == "thread-bbbb" + + +def test_resolve_thread_continue_returns_most_recent(): + plan = LaunchPlan(mode="tui", continue_recent=True) + assert _session(THREADS).resolve_thread(plan) == "thread-aaaa" + + +def test_close_is_a_noop_without_a_loop(): + # Headless sessions have no persistence loop; close() must be safe. + _session(THREADS).close() + + +def test_close_stops_the_background_loop(): + from deerflow.tui.persistence import ThreadMetaWriter, _LoopThread + + loop = _LoopThread() + session = Session(client=_Client(THREADS), writer=ThreadMetaWriter(loop, None), _loop=loop) + session.close() + assert session._loop is None + session.close() # idempotent diff --git a/backend/tests/test_tui_view_state.py b/backend/tests/test_tui_view_state.py new file mode 100644 index 000000000..e06250f78 --- /dev/null +++ b/backend/tests/test_tui_view_state.py @@ -0,0 +1,199 @@ +"""Tests for the pure TUI view-state reducer. + +The reducer is the testable heart of the TUI: a pure function mapping +(state, action) -> state, with no Textual / rendering dependency. +""" + +from deerflow.tui.view_state import ( + AssistantDelta, + AssistantError, + ClearRows, + RunEnded, + RunStarted, + SystemMessage, + ToolResult, + ToolStarted, + UserSubmitted, + initial_state, + reduce, +) + + +def test_user_submitted_appends_user_row(): + state = reduce(initial_state(), UserSubmitted("hello world")) + assert len(state.rows) == 1 + row = state.rows[0] + assert row.kind == "user" + assert row.text == "hello world" + + +def test_run_started_and_ended_toggle_streaming_and_store_usage(): + state = reduce(initial_state(), RunStarted()) + assert state.streaming is True + + usage = {"input_tokens": 10, "output_tokens": 5, "total_tokens": 15} + state = reduce(state, RunEnded(usage=usage)) + assert state.streaming is False + assert state.usage == usage + + +def test_assistant_delta_creates_then_extends_same_id_row(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="Hel")) + state = reduce(state, AssistantDelta(id="m1", text="lo")) + assert len(state.rows) == 1 + assert state.rows[0].kind == "assistant" + assert state.rows[0].text == "Hello" + assert state.rows[0].id == "m1" + + +def test_assistant_delta_with_new_id_after_tool_creates_separate_row(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="thinking")) + state = reduce(state, ToolStarted(tool_call_id="t1", tool_name="read_file", args={"path": "a.py"})) + state = reduce(state, ToolResult(tool_call_id="t1", content="ok", is_error=False)) + state = reduce(state, AssistantDelta(id="m2", text="done")) + + kinds = [r.kind for r in state.rows] + assert kinds == ["assistant", "tool", "assistant"] + assert state.rows[0].text == "thinking" + assert state.rows[2].text == "done" + + +def test_tool_started_appends_running_row_and_result_marks_ok(): + state = initial_state() + state = reduce(state, ToolStarted(tool_call_id="t1", tool_name="read_file", args={"path": "x.py"})) + assert state.rows[0].kind == "tool" + assert state.rows[0].status == "running" + assert state.rows[0].tool_name == "read_file" + + state = reduce(state, ToolResult(tool_call_id="t1", content="file body", is_error=False)) + assert state.rows[0].status == "ok" + assert "file body" in state.rows[0].result + + +def test_tool_result_with_error_marks_error_status(): + state = initial_state() + state = reduce(state, ToolStarted(tool_call_id="t1", tool_name="bash", args={})) + state = reduce(state, ToolResult(tool_call_id="t1", content="boom", is_error=True)) + assert state.rows[0].status == "error" + + +def test_tool_result_without_prior_started_creates_a_completed_row(): + # Defensive: if the tool_started chunks were skipped/missed, a tool result + # should still surface as a completed card rather than vanish. + state = initial_state() + state = reduce(state, ToolResult(tool_call_id="ghost", content="x", is_error=False, tool_name="bash")) + tools = [r for r in state.rows if r.kind == "tool"] + assert len(tools) == 1 + assert tools[0].status == "ok" + + +def test_tool_result_without_call_id_is_ignored(): + state = reduce(initial_state(), ToolResult(tool_call_id="", content="x", is_error=False)) + assert state.rows == () + + +# --- streaming robustness: the client can re-emit the same id (values +# re-synthesis) and stream partial tool-call chunks. The reducer must not +# duplicate text or tool cards. --- + + +def test_assistant_delta_skips_full_resend_of_same_id(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="Hey there!\nWhat's up?")) + state = reduce(state, AssistantDelta(id="m1", text="Hey there!\nWhat's up?")) + assert [r.kind for r in state.rows] == ["assistant"] + assert state.rows[0].text == "Hey there!\nWhat's up?" + + +def test_assistant_delta_treats_cumulative_snapshot_as_replace(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="Hel")) + state = reduce(state, AssistantDelta(id="m1", text="Hel lo world")) + assert state.rows[0].text == "Hel lo world" + + +def test_streaming_id_tracks_active_message_not_reemitted_history(): + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="answer one")) + state = reduce(state, RunEnded()) + assert state.streaming_id is None + + state = reduce(state, RunStarted()) + assert state.streaming_id is None # new turn: nothing actively streaming yet + state = reduce(state, AssistantDelta(id="m1", text="answer one")) # re-emit (no-op) + assert state.streaming_id is None # re-emit of history must not mark active + state = reduce(state, AssistantDelta(id="m2", text="new content")) # real new content + assert state.streaming_id == "m2" + + state = reduce(state, RunEnded()) + assert state.streaming_id is None + + +def test_assistant_resend_of_older_message_updates_in_place_not_duplicated(): + # On a thread with history, the client re-emits every prior message on each + # new turn. A values snapshot can re-emit an OLDER message's full text AFTER + # a newer message has already started — the reducer must update the old row + # by id, not append a verbatim duplicate at the end. + state = initial_state() + state = reduce(state, AssistantDelta(id="m1", text="First answer.")) + state = reduce(state, UserSubmitted("second question")) + state = reduce(state, AssistantDelta(id="m2", text="Second answer.")) + state = reduce(state, AssistantDelta(id="m1", text="First answer.")) # re-emit of old m1 + + assistants = [r for r in state.rows if r.kind == "assistant"] + assert [a.text for a in assistants] == ["First answer.", "Second answer."] + + +def test_tool_started_dedupes_by_call_id(): + state = initial_state() + state = reduce(state, ToolStarted(tool_call_id="tc1", tool_name="bash", args={"cmd": "l"})) + state = reduce(state, ToolStarted(tool_call_id="tc1", tool_name="bash", args={"cmd": "ls -la"})) + tools = [r for r in state.rows if r.kind == "tool"] + assert len(tools) == 1 + assert "ls -la" in tools[0].detail + + +def test_tool_started_with_empty_call_id_is_ignored(): + state = initial_state() + state = reduce(state, ToolStarted(tool_call_id="", tool_name="", args={})) + assert state.rows == () + + +def test_tool_started_fills_name_on_a_later_chunk(): + state = initial_state() + state = reduce(state, ToolStarted(tool_call_id="tc1", tool_name="", args={})) + state = reduce(state, ToolStarted(tool_call_id="tc1", tool_name="web_search", args={"query": "x"})) + tools = [r for r in state.rows if r.kind == "tool"] + assert len(tools) == 1 + assert tools[0].tool_name == "web_search" + assert tools[0].title == "Search" + + +def test_assistant_error_appends_error_row(): + state = reduce(initial_state(), AssistantError("model exploded")) + assert state.rows[0].kind == "assistant" + assert state.rows[0].error is True + assert state.rows[0].text == "model exploded" + + +def test_system_message_appends_with_tone(): + state = reduce(initial_state(), SystemMessage("heads up", tone="error")) + assert state.rows[0].kind == "system" + assert state.rows[0].tone == "error" + + +def test_clear_rows_empties_transcript(): + state = initial_state() + state = reduce(state, UserSubmitted("hi")) + state = reduce(state, ClearRows()) + assert state.rows == () + + +def test_reduce_is_pure_does_not_mutate_input_state(): + state = reduce(initial_state(), UserSubmitted("first")) + before_len = len(state.rows) + # Reducing again must not mutate the previous state object. + _ = reduce(state, UserSubmitted("second")) + assert len(state.rows) == before_len diff --git a/backend/uv.lock b/backend/uv.lock index fd71a5fe5..8c2ecc08b 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -792,6 +792,7 @@ dev = [ { name = "pytest" }, { name = "pytest-asyncio" }, { name = "ruff" }, + { name = "textual" }, ] [package.metadata] @@ -824,6 +825,7 @@ dev = [ { name = "pytest", specifier = ">=9.0.3" }, { name = "pytest-asyncio", specifier = ">=1.3.0" }, { name = "ruff", specifier = ">=0.14.11" }, + { name = "textual", specifier = ">=0.80" }, ] [[package]] @@ -879,6 +881,9 @@ postgres = [ pymupdf = [ { name = "pymupdf4llm" }, ] +tui = [ + { name = "textual" }, +] [package.metadata] requires-dist = [ @@ -920,9 +925,10 @@ requires-dist = [ { name = "readabilipy", specifier = ">=0.3.0" }, { name = "sqlalchemy", extras = ["asyncio"], specifier = ">=2.0,<3.0" }, { name = "tavily-python", specifier = ">=0.7.17" }, + { name = "textual", marker = "extra == 'tui'", specifier = ">=0.80" }, { name = "tiktoken", specifier = ">=0.8.0" }, ] -provides-extras = ["groundroute", "ollama", "postgres", "pymupdf"] +provides-extras = ["tui", "groundroute", "ollama", "postgres", "pymupdf"] [[package]] name = "defusedxml" @@ -2084,6 +2090,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b4/72/c2e973066da57e9f6720c229364e673d89c884fac65c265a08e2c32eed3c/lark_oapi-1.5.5-py3-none-any.whl", hash = "sha256:c953d3f87e5b43d9e99cdee7c2d962568ac05d5c01ef57ad662fbb5d4ec0e69f", size = 6995394, upload-time = "2026-04-21T04:00:42.216Z" }, ] +[[package]] +name = "linkify-it-py" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "uc-micro-py" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/2e/c9/06ea13676ef354f0af6169587ae292d3e2406e212876a413bf9eece4eb23/linkify_it_py-2.1.0.tar.gz", hash = "sha256:43360231720999c10e9328dc3691160e27a718e280673d444c38d7d3aaa3b98b", size = 29158, upload-time = "2026-03-01T07:48:47.683Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b4/de/88b3be5c31b22333b3ca2f6ff1de4e863d8fe45aaea7485f591970ec1d3e/linkify_it_py-2.1.0-py3-none-any.whl", hash = "sha256:0d252c1594ecba2ecedc444053db5d3a9b7ec1b0dd929c8f1d74dce89f86c05e", size = 19878, upload-time = "2026-03-01T07:48:46.098Z" }, +] + [[package]] name = "lxml" version = "6.1.0" @@ -2206,6 +2224,23 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ca/54/2e39566a131b13f6d8d193f974cb6a34e81bb7cc2fa6f7e03de067b36588/mammoth-1.11.0-py2.py3-none-any.whl", hash = "sha256:c077ab0d450bd7c0c6ecd529a23bf7e0fa8190c929e28998308ff4eada3f063b", size = 54752, upload-time = "2025-09-19T10:35:18.699Z" }, ] +[[package]] +name = "markdown-it-py" +version = "4.2.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mdurl" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" }, +] + +[package.optional-dependencies] +linkify = [ + { name = "linkify-it-py" }, +] + [[package]] name = "markdown-to-mrkdwn" version = "0.3.2" @@ -2355,6 +2390,27 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/9c/46/f6b4ad632c67ef35209a66127e4bddc95759649dd595f71f13fba11bdf9a/mcp-1.27.0-py3-none-any.whl", hash = "sha256:5ce1fa81614958e267b21fb2aa34e0aea8e2c6ede60d52aba45fd47246b4d741", size = 215967, upload-time = "2026-04-02T14:48:07.24Z" }, ] +[[package]] +name = "mdit-py-plugins" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + [[package]] name = "mpmath" version = "1.3.0" @@ -2996,6 +3052,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ff/6e/cf826fae916b8658848d7b9f38d88da6396895c676e8086fc0988073aaf8/pillow-12.2.0-cp314-cp314t-win_arm64.whl", hash = "sha256:aa88ccfe4e32d362816319ed727a004423aab09c5cea43c01a4b435643fa34eb", size = 2556579, upload-time = "2026-04-01T14:45:52.529Z" }, ] +[[package]] +name = "platformdirs" +version = "4.10.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/47/e4501f49c178ae1d9f4a75073fda4204f52647993f075a9db4d14930e0c5/platformdirs-4.10.0.tar.gz", hash = "sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7", size = 31224, upload-time = "2026-05-28T03:32:53.587Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/81/e6/cd9575ac904136b3cbf7aa7ee819ef86eedb7274e46f230e94ea4342e729/platformdirs-4.10.0-py3-none-any.whl", hash = "sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a", size = 22743, upload-time = "2026-05-28T03:32:52.175Z" }, +] + [[package]] name = "pluggy" version = "1.6.0" @@ -3825,6 +3890,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl", hash = "sha256:cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", size = 54481, upload-time = "2023-05-01T04:11:28.427Z" }, ] +[[package]] +name = "rich" +version = "15.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" }, +] + [[package]] name = "rpds-py" version = "0.30.0" @@ -4163,6 +4241,23 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d7/c1/eb8f9debc45d3b7918a32ab756658a0904732f75e555402972246b0b8e71/tenacity-9.1.4-py3-none-any.whl", hash = "sha256:6095a360c919085f28c6527de529e76a06ad89b23659fa881ae0649b867a9d55", size = 28926, upload-time = "2026-02-07T10:45:32.24Z" }, ] +[[package]] +name = "textual" +version = "8.2.7" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py", extra = ["linkify"] }, + { name = "mdit-py-plugins" }, + { name = "platformdirs" }, + { name = "pygments" }, + { name = "rich" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9b/7a/c519db0aba5024f86e71e9631810bfdd6866ed2c8695bd7fa34b90e7ef59/textual-8.2.7.tar.gz", hash = "sha256:658f568ff81e30ed43890c3e07520390e5cf1b4763822006e060656b0a88f105", size = 1859249, upload-time = "2026-05-19T10:52:49.531Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a8/f5/c1e18bc0707300a0e90204343abbf7d7acd6fb7ebe03a6d4893b99a234b8/textual-8.2.7-py3-none-any.whl", hash = "sha256:4caaa13a90bc4cf9c6c862c067ccd34fe84e9c161710a2a907a8026313b6bd73", size = 731129, upload-time = "2026-05-19T10:52:51.773Z" }, +] + [[package]] name = "tiktoken" version = "0.12.0" @@ -4261,6 +4356,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ce/e4/dccd7f47c4b64213ac01ef921a1337ee6e30e8c6466046018326977efd95/tzdata-2026.2-py2.py3-none-any.whl", hash = "sha256:bbe9af844f658da81a5f95019480da3a89415801f6cc966806612cc7169bffe7", size = 349321, upload-time = "2026-04-24T15:22:05.876Z" }, ] +[[package]] +name = "uc-micro-py" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/78/67/9a363818028526e2d4579334460df777115bdec1bb77c08f9db88f6389f2/uc_micro_py-2.0.0.tar.gz", hash = "sha256:c53691e495c8db60e16ffc4861a35469b0ba0821fe409a8a7a0a71864d33a811", size = 6611, upload-time = "2026-03-01T06:31:27.526Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/61/73/d21edf5b204d1467e06500080a50f79d49ef2b997c79123a536d4a17d97c/uc_micro_py-2.0.0-py3-none-any.whl", hash = "sha256:3603a3859af53e5a39bc7677713c78ea6589ff188d70f4fee165db88e22b242c", size = 6383, upload-time = "2026-03-01T06:31:26.257Z" }, +] + [[package]] name = "urllib3" version = "2.7.0" diff --git a/docs/superpowers/specs/2026-06-13-deerflow-tui.md b/docs/superpowers/specs/2026-06-13-deerflow-tui.md new file mode 100644 index 000000000..bc32a6644 --- /dev/null +++ b/docs/superpowers/specs/2026-06-13-deerflow-tui.md @@ -0,0 +1,257 @@ +# DeerFlow TUI - Product and Engineering Spec + +**Date**: 2026-06-13 +**Status**: Draft, ready for RFC discussion +**Scope**: Hermes-like terminal UI for the DeerFlow 2.0 harness + +**Revision 2026-06-24**: Reworked runtime ownership and session persistence after RFC #3540 feedback. The TUI runs embedded but reuses the same session/persistence layer (`ThreadMetaStore`) so terminal sessions are visible in the Web UI without running the Gateway. See [Runtime and Session Persistence](#runtime-and-session-persistence). + +--- + +## Problem Statement + +DeerFlow has a capable embedded Python client and a full Gateway/Web UI, but no first-class terminal workbench. A plain command set would help scripts, but it would not cover the interactive developer experience we actually want: a persistent, keyboard-driven TUI where users can chat, switch threads, inspect tools, manage files, view live execution, trigger slash commands, and keep context without opening the browser. + +Hermes Agent's TUI is a useful reference point: the terminal UI is a modern interactive surface backed by the same runtime as the classic CLI, shares sessions and slash commands, supports modal overlays, non-blocking input, live session switching, status indicators, and a TTY-aware fallback path. + +DeerFlow needs the same product shape: + +1. A terminal-native primary interface for users who live in the shell. +2. The same agent behavior, memory, skills, MCP tools, sandbox behavior, and thread persistence as the web app and embedded client, converging on one shared session/persistence layer rather than a per-surface copy. +3. A richer interactive surface than one-shot commands can provide. +4. Scriptable headless commands for automation, but not as the center of the product. + +## Solution + +Build a first-party `deerflow` TUI backed by `DeerFlowClient`. + +The default interactive path should be: + +```bash +deerflow +deerflow --tui +deerflow chat +deerflow --continue +deerflow --resume THREAD_ID +``` + +The headless path should remain available for scripts: + +```bash +deerflow chat --print "summarize this repo" +deerflow models list --json +deerflow threads list --json +``` + +The v1 TUI should run in embedded mode only. It should not require Gateway, frontend, nginx, or Docker services to be running, though it must honor the same `config.yaml`, `extensions_config.json`, runtime paths, sandbox configuration, tracing configuration, skills, memory, and checkpointer settings used by the rest of DeerFlow. + +Remote HTTP/Gateway mode can be added later as a transport option. The first implementation should ship a dependable local terminal workbench. + +## Runtime and Session Persistence + +This is the most load-bearing decision in the spec, and the RFC discussion (#3540) surfaced two facts about the current codebase that the original draft glossed over: + +1. **There is no single runtime today — there are two run paths that share only the agent factory.** The Web/Gateway surface executes runs through `run_agent()` (async `astream` + `StreamBridge` for SSE). `DeerFlowClient.stream()` executes runs synchronously and in-process. Both build the agent through the same `make_lead_agent` / `create_agent()` factory, but the run, streaming, and persistence orchestration around that factory is implemented twice and is not shared. +2. **Web UI session visibility is driven by `ThreadMetaStore`, not the checkpointer.** The Web UI lists conversations via `GET /api/threads/search`, which reads the `thread_meta` SQL table filtered by `user_id`. The Gateway creates a `thread_meta` row (with the authenticated `user_id`) on the first run of a thread. `DeerFlowClient` writes only to the checkpointer and never creates a `thread_meta` row, so a thread created from the TUI is invisible in the Web UI sidebar by default — even though both surfaces can load the same checkpoint if they already know the `thread_id`. + +### v1 decision + +The TUI stays **embedded** (no Gateway, frontend, nginx, or Docker dependency) but **reuses the same persistence layer** instead of a private copy: + +1. The embedded run path writes `thread_meta` to the same store the Web UI reads, so terminal sessions appear in the Web UI sidebar **without the Gateway process running**. Web UI visibility requires the shared `thread_meta` store and a `user_id` — it does **not** require the Gateway HTTP API. +2. Because the embedded process has no authenticated user, v1 attributes embedded `thread_meta` rows to a single **local default user identity** resolved from config. Full multi-user auth/session switching stays out of scope. +3. `thread_meta` creation, token-usage tracking, and thread-title sync are factored into one shared module used by **both** `run_agent()` and `DeerFlowClient`, so the two run paths cannot drift on session bookkeeping. This converges "single runtime" from the agent-factory layer down to the session/persistence layer, which is where multi-surface consistency actually lives. +4. Reusing the Gateway HTTP API as the TUI transport was considered and deferred: it would give Web UI visibility and user scoping "for free" but requires a running service stack and an auth token, which defeats the standalone local-workbench goal. It remains a later transport option, not the v1 path. + +## Reference Behavior From Hermes + +The DeerFlow TUI should borrow product ideas, not implementation details, from Hermes: + +1. Bare command launches an interactive terminal interface. +2. TUI is backed by the same runtime as non-interactive commands. +3. Sessions/threads are shared between TUI and headless invocations. +4. Slash commands work the same way across surfaces. +5. Input remains usable while the runtime initializes or while a run is active. +6. Model picker, thread picker, clarification prompts, and approvals render as overlays. +7. Tool and skill initialization appears progressively instead of blocking startup. +8. Alternate-screen rendering avoids scrollback clutter and flicker. +9. TTY detection falls back to single-query/headless behavior when interactive rendering is unavailable. +10. Interrupts can stop or redirect the active run without corrupting persisted thread state. + +## User Stories + +1. As a DeerFlow user, I want `deerflow` to open an interactive terminal UI, so that I can work without a browser. +2. As a DeerFlow user, I want to continue the most recent thread, so that I can resume work quickly. +3. As a DeerFlow user, I want to resume a specific thread by id or title, so that I can return to older work. +4. As a DeerFlow user, I want a fixed composer with multiline editing, so that long prompts are comfortable to write. +5. As a DeerFlow user, I want slash-command autocomplete, so that skills and commands are discoverable. +6. As a DeerFlow user, I want `/model` to open a model picker, so that model switching is not a fragile text-only workflow. +7. As a DeerFlow user, I want `/threads` or `/switch` to open a live thread switcher, so that I can move between active conversations. +8. As a DeerFlow user, I want `/skills` to browse enabled and available skills, so that I can understand what the agent can do. +9. As a DeerFlow user, I want tool calls and results to stream into a collapsible activity panel, so that I can inspect execution without losing the main answer. +10. As a DeerFlow user, I want uploads and generated artifacts visible in a side panel, so that file-heavy workflows stay manageable. +11. As a DeerFlow user, I want memory status and relevant injected facts visible on demand, so that persistent context is not mysterious. +12. As a DeerFlow user, I want MCP servers and built-in tools visible on demand, so that tool availability is clear. +13. As a DeerFlow user, I want status-line indicators for model, thread, token usage, run state, and sandbox mode, so that I know what environment I am driving. +14. As a DeerFlow user, I want to interrupt a running agent with `Ctrl+C` or a new message, so that I can redirect work without restarting the TUI. +15. As a DeerFlow user, I want clarification prompts to appear as focused overlays, so that I can answer them quickly. +16. As a DeerFlow user, I want file path paste handling, so that pasted paths can become uploads or prompt text intentionally. +17. As a DeerFlow user, I want command history and prompt drafts to survive transient UI redraws, so that I do not lose work. +18. As an automation author, I want headless `--print` and `--json` modes, so that TUI work does not remove scriptability. +19. As a maintainer, I want TUI tests at the UI-driver boundary, so that layout refactors do not silently break command behavior. +20. As a package consumer, I want an installed `deerflow` binary to launch the TUI, so that the harness feels like a product, not only a library. + +## TUI Surface + +### Launch Modes + +| Command | Behavior | +|---|---| +| `deerflow` | Launch TUI when stdin/stdout are TTYs. | +| `deerflow --tui` | Force TUI and fail with a clear diagnostic if unavailable. | +| `deerflow --cli` | Force headless/classic command mode for one invocation. | +| `deerflow chat` | Launch the same TUI conversation surface. | +| `deerflow chat --print MESSAGE` | Single-query headless mode. | +| `deerflow --continue` | Resume most recent thread. | +| `deerflow --resume THREAD_ID_OR_TITLE` | Resume a specific thread. | +| `DEER_FLOW_TUI=1 deerflow` | Force TUI through environment. | + +If TUI dependencies or a TTY are missing, bare `deerflow` should print a diagnostic and fall back to headless guidance rather than hanging or crashing. + +### Layout + +The TUI should use a predictable terminal workbench layout: + +1. Header/banner: project root, active model, sandbox mode, memory state, enabled tool groups, enabled skills, MCP server summary. +2. Main transcript: user messages, streamed assistant deltas, final answers, and compact summaries of tool activity. +3. Activity panel: collapsible tool calls, tool results, subagent status, uploads, artifact writes, and custom stream events. +4. Side/session panel: current thread title/id, recent threads, live threads, uploaded files, generated artifacts. +5. Composer: multiline input, slash-command autocomplete, file attach affordances, paste handling. +6. Status line: run state, model, token usage, elapsed time, thread id/title, pending uploads, active tool/subagent count. + +### Slash Commands + +Slash commands should be TUI-owned affordances over existing DeerFlow capabilities: + +| Command | TUI behavior | +|---|---| +| `/help` | Overlay with categorized commands and keybindings. | +| `/new` | Start a fresh thread. | +| `/resume THREAD` | Resume a thread. | +| `/threads` or `/switch` | Open thread switcher. | +| `/model` | Open model picker. | +| `/skills` | Open skill browser. | +| `/ ...` | Activate a skill for the current turn, preserving existing slash-skill semantics. | +| `/tools` | Show built-in, MCP, sandbox, and community tool availability. | +| `/mcp` | Show MCP server status. | +| `/memory` | Show memory status and injected facts. | +| `/uploads` | Show uploaded files for the current thread. | +| `/artifacts` | Show generated artifacts and save/open actions. | +| `/details` | Toggle verbose activity rendering. | +| `/usage` | Open token usage/context panel. | +| `/config` | Show resolved config paths and active overrides. | +| `/quit` | Exit the TUI. | + +### Keybindings + +Initial keybindings: + +| Key | Behavior | +|---|---| +| `Enter` | Send current prompt when composer is single-line or submit is explicit. | +| `Shift+Enter` / terminal-supported equivalent | Insert newline. | +| `Ctrl+C` | Interrupt active run; when idle, ask to quit or clear composer. | +| `Esc` | Close overlay or cancel current selection. | +| `Ctrl+L` | Redraw UI. | +| `Ctrl+R` | Open thread/session switcher. | +| `Ctrl+O` | Open file attach picker. | +| `Ctrl+D` | Toggle details/activity panel. | +| `Ctrl+U` | Clear composer. | +| `Ctrl+P` / `Ctrl+N` | Navigate history/autocomplete. | + +Keybindings should be documented in `/help` and must degrade gracefully across terminals. + +## Headless Command Surface + +The TUI is the primary product, but scriptable commands should remain: + +1. `deerflow chat --print MESSAGE` +2. `deerflow chat --stdin --print` +3. `deerflow chat --json MESSAGE` +4. `deerflow models list|get` +5. `deerflow skills list|get|enable|disable|install` +6. `deerflow mcp list|export|apply` +7. `deerflow memory status|show|export|import|clear|fact add|fact update|fact delete` +8. `deerflow uploads add|list|delete` +9. `deerflow artifacts get` +10. `deerflow threads list|get` + +Headless JSON streaming should preserve `StreamEvent` semantics as newline-delimited JSON. + +## Implementation Decisions + +1. The console script name is `deerflow`. +2. The TUI is the default interactive surface for TTY sessions. +3. The TUI is backed by the embedded `DeerFlowClient` run path, not a new third runtime. Today `DeerFlowClient` and the Gateway's `run_agent()` share only the agent factory; v1 converges them on a shared session/persistence layer (see [Runtime and Session Persistence](#runtime-and-session-persistence)) instead of adding another divergent copy. +4. Thread persistence, streaming semantics, uploads, artifacts, skills, MCP, memory, and cache invalidation remain owned by `DeerFlowClient` and existing harness modules. Session indexing (`thread_meta`) plus token-usage and title bookkeeping move into one shared module so the embedded TUI writes the same session records the Web UI reads, attributed to a single local default `user_id`. Web UI visibility therefore needs the shared store, not the Gateway process. +5. The first TUI should prefer a Python-native TUI stack unless a prototype proves a Node/React terminal stack is materially better for DeerFlow. Python-native keeps packaging and runtime closer to the harness. +6. If a Node TUI is chosen, it must be launched by the Python `deerflow` command and communicate through a narrow local protocol so `DeerFlowClient` remains the runtime owner. +7. The TUI should support alternate-screen rendering, but headless mode and non-TTY fallback must remain available. +8. The TUI should render tool activity as first-class UI state rather than mixing every event into the transcript. +9. The implementation should avoid adding a Gateway dependency to local TUI mode. +10. The implementation must update user-facing README usage and backend development guidance when code lands. + +## Testing Decisions + +Good TUI tests should validate behavior at the highest practical seam: + +1. Headless parser/dispatcher tests through `main(argv)`. +2. TUI app state tests using a mocked `DeerFlowClient` and synthetic `StreamEvent` objects. +3. Snapshot-like tests for layout state, not brittle terminal screenshots, for core panels and overlays. +4. Keybinding tests for send, interrupt, overlay close, thread switch, details toggle, and file attach. +5. Slash-command tests for command routing, autocomplete, and skill activation handoff. +6. JSON contract tests for headless mode. +7. Error tests for missing TTY, missing optional TUI dependency, invalid config, missing model, upload failures, and artifact path errors. +8. Packaging smoke test proving the console entry point launches the correct surface. + +Optional live tests: + +1. A skipped-by-default live TUI smoke test can run only when a valid local config and credentials are present. +2. Live tests should not be required in CI. + +## Risks + +1. Terminal compatibility: keybindings, paste handling, mouse support, and alternate screen differ across terminal emulators. +2. Streaming complexity: a pleasant UI needs careful state management for transcript, tool events, uploads, artifacts, and run lifecycle. +3. Config timing: global overrides must be applied before modules cache configuration. +4. Dependency footprint: a full TUI stack may add runtime dependencies that need packaging scrutiny. +5. Scope creep: a TUI can easily become a second web UI. V1 should focus on chat, threads, model/skill/tool visibility, uploads, artifacts, and interruptibility. +6. Run-path divergence: `run_agent()` and `DeerFlowClient` orchestrate runs separately. Factoring session indexing and bookkeeping into a shared module is required to keep terminal and web sessions consistent; skipping it reintroduces the visibility gap and silent drift. + +## Out of Scope + +1. Replacing the web UI. +2. Running or supervising Gateway/frontend/nginx services. +3. Remote HTTP transport to an already-running DeerFlow server. +4. Full desktop app. +5. Voice mode. +6. Shell completion generation. +7. Native Windows `cmd.exe` support beyond what the chosen TUI stack can reasonably support. +8. Multi-user auth/session switching. +9. Direct raw sandbox shell commands outside normal agent tool execution. +10. Reimplementing setup wizard or doctor checks before their logic is made reusable. + +## Documentation Requirements + +When implementation lands, update: + +1. Quick Start docs with `deerflow`, `deerflow --tui`, `deerflow --continue`, and `deerflow chat --print`. +2. Backend development docs with the TUI architecture, module ownership, and test command. +3. Embedded client docs to explain that the TUI is a front-end over `DeerFlowClient`. +4. Any package/install docs that mention available console scripts. + +## Further Notes + +The core product decision is TUI-first, CLI-second. `deerflow` should feel like a terminal workbench for the DeerFlow harness. Headless commands still matter, but they support automation rather than define the interactive product. + +The most important technical decision is runtime ownership. The TUI should be a UI shell over the existing embedded harness. It should not fork agent behavior away from `DeerFlowClient`, because that would split tests, persistence, streaming semantics, and tool behavior across surfaces. Concretely, v1 closes the existing split by moving session indexing and run bookkeeping into a layer shared by both `run_agent()` and `DeerFlowClient`, so terminal and web sessions stay consistent (see [Runtime and Session Persistence](#runtime-and-session-persistence)). diff --git a/docs/tui/tui-markdown.svg b/docs/tui/tui-markdown.svg new file mode 100644 index 000000000..ea64beef8 --- /dev/null +++ b/docs/tui/tui-markdown.svg @@ -0,0 +1,232 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + DeerFlow TUI — markdown + + + + + + + + + +  DeerFlow kimi-k2.6  ·  thread b0bc1580  ·  /Users/taohe/workspace/deer-flow/backend  ·  12 skills + +› 世界杯小组赛第二轮总结一下，C罗表现怎么样 + +●葡萄牙 5-0 横扫乌兹别克斯坦，强势反弹！C罗梅开二度。                                             + +🌍 整体形势 + +第二轮结束后，已有 6 支球队提前锁定 32 强 来源:                                                  + + • 欧洲：德国（E组，两战 9 进球）、法国、挪威                                                    + • 南美：阿根廷                                                                                  + • 北美：美国、墨西哥 — 东道主强势                                                               + +📅 接下来看点 + +小组赛第三轮（6月25–27日）是生死战。葡萄牙 vs 哥伦比亚 打平即可出线。                            + +▌ C罗 41 岁用一场大胜宣告"王者归来"。 + +------------------------------------------------------------------------------------------------ + +Sources + + 1 联合早报 — C罗双响 + 2 FIFA 官网 — 积分榜 + + + + + + + + + +● ready世界杯次轮总结kimi-k2.6thread b0bc15804096 tok +╭────────────────────────────────────────────────────────────────────────────────────────────────────╮ +│Message DeerFlow…   ( / for commands )│ +╰────────────────────────────────────────────────────────────────────────────────────────────────────╯ + + + + diff --git a/docs/tui/tui-palette.svg b/docs/tui/tui-palette.svg new file mode 100644 index 000000000..49a17cb95 --- /dev/null +++ b/docs/tui/tui-palette.svg @@ -0,0 +1,205 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + DeerFlow TUI — slash palette + + + + + + + + + +  DeerFlow claude-opus-4.8  ·  thread 4f2a8c1d  ·  /Users/taohe/workspace/deer-flow/backend  ·  12 + +› Add a /usage panel to the TUI status line + +● I'll wire token usage into the status renderer and add a test. + +  ⚙ Read  deerflow/tui/render.py   ✓ +    def render_status(state, *, model, thread_label, spinner): ... + +  ⚙ Bash  pytest tests/test_tui_render.py -q   ✓ +    5 passed in 0.10s + +● Done. The status line now shows live token usage and an interrupt hint. + + + + + + + +● readyAdd /usage panelclaude-opus-4.8thread 4f2a8c1d2048 tok +╭────────────────────────────────────────────────────────────────────────────────────────────────────╮ +│▌ /modelOpen the model picker│ +│/mcpShow MCP server status│ +│/memoryShow memory status and injected facts│ +│/resumeResume a thread by id or title│ +│/helpShow commands and keybindings│ +│/toolsShow built-in, MCP and sandbox tools│ +│/tddTest-driven development│ +╰────────────────────────────────────────────────────────────────────────────────────────────────────╯ +╭────────────────────────────────────────────────────────────────────────────────────────────────────╮ +│/m│ +╰────────────────────────────────────────────────────────────────────────────────────────────────────╯ + + + + diff --git a/docs/tui/tui-preview.svg b/docs/tui/tui-preview.svg new file mode 100644 index 000000000..7c139efcd --- /dev/null +++ b/docs/tui/tui-preview.svg @@ -0,0 +1,195 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + DeerFlow TUI + + + + + + + + + +  DeerFlow claude-opus-4.8  ·  thread 4f2a8c1d  ·  /Users/taohe/workspace/deer-flow/backend  ·  1 + +› Refactor the streaming bridge and add a regression test + +● I'll read the bridge, then add a focused test. + +  ⚙ Read  deerflow/tui/runtime.py   ✓ +    def stream_actions(client, message): yield RunStarted() + +  ⚙ Bash  pytest tests/test_tui_runtime.py -q   ✓ +    9 passed in 0.20s + +● Done — 9 tests pass. Every run is now bracketed with RunStarted/RunEnded, and errors surface as  +an assistant row instead of crashing. + + + + + + + + + + + + + +● readyclaude-opus-4.8thread 4f2a8c1d1234 tok +╭────────────────────────────────────────────────────────────────────────────────────────────────────╮ +│explain the title middleware│ +╰────────────────────────────────────────────────────────────────────────────────────────────────────╯ + + + +