mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-09 16:08:31 +00:00
Some checks are pending
Backend Blocking IO / backend-blocking-io (push) Waiting to run
Unit Tests / backend-unit-tests (push) Waiting to run
E2E Tests / e2e-tests (push) Waiting to run
Frontend Unit Tests / frontend-unit-tests (push) Waiting to run
Lint Check / lint-backend (push) Waiting to run
Lint Check / lint-frontend (push) Waiting to run
Replay E2E (front-back contract) / Layer 1 — backend golden (no API key) (push) Waiting to run
Replay E2E (front-back contract) / Layer 2 — full-stack render (no API key) (push) Waiting to run
* fix: generate title for interrupted first turn * test(title): cover partial-exchange + dict-form messages Harden the interrupted-run fallback path added in 19fc34fd: - TitleMiddleware._should_generate_title now accepts a lone first-turn user message when allow_partial_exchange=True, so the worker can still derive a title if cancellation lands before any AI chunk is checkpointed. - runtime/runs/worker._ensure_interrupted_title computes the next checkpoint step defensively (treat missing/non-int step as 0) and renames a shadowed ckpt_config local for readability. - Add four unit tests in tests/test_title_middleware_core_logic.py: partial-exchange allows user-only, partial-exchange still respects an existing title, dict-form messages are recognized, and the sync fallback path derives a title from dict-form messages — matching what channel_values stores in the checkpoint. Refs #3859. * fix: persist interrupted-title via channel_versions bump Address PR #3874 review feedback: ``_ensure_interrupted_title`` previously called ``aput(..., new_versions={})``. LangGraph's DB-backed savers (``PostgresSaver`` and the v4 ``SqliteSaver`` blob layout) strip inline ``channel_values`` from ``put`` and only persist blobs for channels named in ``new_versions`` — so the fallback ``title`` channel was dropped on read-back and ``threads_meta.display_name`` stayed ``"Untitled"`` after refresh on those backends. The original in-memory e2e passed because ``InMemorySaver`` keeps the inline snapshot verbatim. Fix mirrors ``_rollback_to_pre_run_checkpoint`` in the same file: bump ``channel_versions["title"]`` (via ``checkpointer.get_next_version`` when available, else int/string fallbacks), persist the new version on the checkpoint, and declare it in ``new_versions`` so the DB savers actually write the blob. Regression coverage in ``tests/test_run_worker_rollback.py``: - ``test_ensure_interrupted_title_bumps_channel_version_and_declares_it_in_new_versions`` — exact ``aput`` invariants: ``new_versions == {"title": 1}``, the written checkpoint's ``channel_versions["title"]`` is bumped, and the pre-existing ``messages`` version is preserved. - ``test_ensure_interrupted_title_bumps_existing_string_version`` — string-shaped prior version (some savers use UUID-style versions); bumped value must differ from the prior, no overwrite-in-place. - ``test_ensure_interrupted_title_skips_when_title_already_set`` — title short-circuit; no extra ``aput``. - ``test_ensure_interrupted_title_returns_none_when_no_checkpoint`` — no checkpoint yet; returns ``None`` without writing. - ``test_ensure_interrupted_title_round_trip_with_real_sqlite_checkpointer`` — full round-trip against a real ``AsyncSqliteSaver`` on a disk-backed DB, then closes and re-opens the saver to simulate a fresh connection. The fallback title must still be present on the second ``aget_tuple``. This is the exact scenario the review flagged. Validated locally with the full backend suite: 5195 passed, 18 skipped. Refs #3859. Addresses review on #3874. * test(worker, title): harden interrupted-title fallback for every saver Defensive coverage on top of the channel_versions fix (commit 05253957), addressing edge cases surfaced during a second-pass review of #3874. Worker: - Extract version bump into ``_bump_channel_version(checkpointer, current)`` with explicit fallbacks for int / float / numeric-string / UUID-shaped string / None / bool, AND a wrap-around defense when the saver's ``get_next_version`` raises or returns an unchanged value. The invariant is: returned version MUST differ from the prior. Without this, a saver bug (or a custom backend) could leave ``new_versions={"title": v}`` no-op on DB savers — the very class of bug the original review pointed out. Title middleware: - Coerce ``state.get("messages")`` from ``None`` to ``[]`` on both ``_should_generate_title`` and ``_build_title_prompt``. A partially-initialized checkpoint can carry ``messages=None`` on the channel_values channel (the worker reads raw channel_values, not BaseMessages), and the default kwarg only protects against a missing key. Repro: ``TypeError: 'NoneType' object is not iterable`` from the next() generator — confirmed by reverting the fix and watching ``test_*_handles_none_messages_channel`` go red. Tests (TDD-verified red→green for the new asserts): - ``test_run_worker_rollback.py``: * ``_bump_channel_version`` — 8 tests covering every version type (int, float, numeric string, UUID-style string, None, bool) and every saver-side fault mode (no ``get_next_version`` / raising / stuck on identity). * ``test_ensure_interrupted_title_*`` — 5 additional helper boundary tests: title.enabled=false short-circuit; empty messages list; messages=None; aput-error propagation (helper contract: caller swallows, not the helper); idempotency on a real InMemorySaver across two invocations. * ``test_ensure_interrupted_title_preserves_non_title_channel_versions`` — pins that ``new_versions`` only contains ``"title"`` and that other channels' versions are untouched (regression anchor for a sloppier draft that bumped every channel). * ``test_worker_finally_block_swallows_helper_exceptions`` — pins the integration contract: even if the helper raises, the worker's threads_meta status sync still runs and ``publish_end`` is still awaited so the SSE stream closes cleanly. - ``test_title_middleware_core_logic.py``: * 4 additional tests: ``messages=None`` on both ``_should_generate_title`` and ``_build_title_prompt``; the ``role: user`` / ``role: assistant`` (OpenAI-style) dict normalization; partial-exchange path with a dict-form message. Verification: - ``PYTHONPATH=. uv run pytest tests/ -x --ignore=tests/blocking_io -q`` → 5215 passed, 18 skipped. - ``ruff check`` + ``ruff format --check`` clean on every touched file. - Red/green TDD verification: temporarily reverted the ``new_versions={}`` fix → 4 new tests went red as expected; restored and the suite is green again. Same red/green dance for the ``messages=None`` coercion. Refs #3859. Addresses second-pass review on #3874. * fix(title): ignore dict context reminders in fallback * fix(worker): link interrupted-title checkpoint to its parent The title-bump checkpoint written by ``_ensure_interrupted_title`` was landing without a ``parent_checkpoint_id`` — a real orphan in the LangGraph history graph. Reproduction (disk-backed AsyncSqliteSaver): [seed] checkpoint_id = 1f173dbc... [helper] wrote title = "Why is the sky blue?" [issue 1] new checkpoint = 1f173dbc..., parent = None [issue 1] is new checkpoint orphaned? True Root cause: ``_ensure_interrupted_title`` built ``write_config`` as ``{"thread_id": ..., "checkpoint_ns": ...}`` only. ``BaseCheckpointSaver`` implementations read ``configurable.checkpoint_id`` from that config as the *parent* id when inserting (see ``langgraph/checkpoint/sqlite/aio.py`` ``aput``: ``config["configurable"].get("checkpoint_id")`` becomes the ``parent_checkpoint_id`` column). With no value, the saver writes NULL — the new checkpoint is a tree root. Consequences: - Any future LangGraph ``runs.resume_from`` / time-travel feature has no backward edge to walk past the title-bump. - History-visualization UIs built on ``alist()`` render the title-bump as a sibling of the prior checkpoint, not its descendant. Fix: read ``checkpoint_id`` off the tuple's own config and thread it into ``write_config["configurable"]["checkpoint_id"]`` before calling ``aput``, the same pattern every middleware-driven write uses. Three new regression tests against real ``AsyncSqliteSaver`` (disk-backed, fresh connections so we exercise the on-disk read path): - ``test_ensure_interrupted_title_links_new_checkpoint_to_its_parent`` — asserts ``latest.parent_config["configurable"]["checkpoint_id"]`` equals the seeded checkpoint id. TDD red-green verified: reverting the fix flips this test red with ``AssertionError: title-bump checkpoint must have a parent_config``. - ``test_ensure_interrupted_title_appears_in_history_with_audit_marker`` — pins the audit contract: the title-bump entry in ``alist()`` carries ``metadata.source == "update"`` and ``metadata.writes`` contains ``runtime_interrupt_title``. This is a deliberate design choice — we do NOT hide the entry from history (audit trail belongs in the saver), but its source and writes marker MUST be unambiguous so UIs/tools can identify it. - ``test_ensure_interrupted_title_survives_immediate_next_turn`` — cancel → immediate user follow-up scenario. Simulates the agent's next turn appending a (user, ai) pair without touching the title channel, then opens a fresh saver and verifies the title is still present after the next-turn checkpoint write. Pins the channel-version-blob invariant established by commit 05253957 — without the ``new_versions={"title": v}`` declaration there, the title blob would vanish from the DB and this test would read back ``None``. Verification: - ``PYTHONPATH=. uv run pytest tests/ -x --ignore=tests/blocking_io -q`` → 5222 passed, 15 skipped. - ``ruff check`` + ``ruff format --check`` clean on every touched file. - Reproduction script confirms ``parent_checkpoint_id`` is now non-null and the next-turn read-back preserves the fallback title. Refs #3859. * Revert "fix(worker): link interrupted-title checkpoint to its parent" This reverts commit c763ed9334781db1acdce0f5f33d663d8d5f80ff. * test: trim over-engineered test coverage Reduce review surface area on PR #3874 by dropping defensive tests that don't pin a real invariant. After self-review: - ``_bump_channel_version``: 8 tests → 2 (happy path + saver-error fallback). Dropped float / bool / numeric-string / UUID-string / missing-get-next-version / stuck-get-next-version branches — those are speculative scaffolding for savers we don't ship. - ``_ensure_interrupted_title``: dropped ``returns_none_when_title_disabled``, ``returns_none_with_no_user_message``, ``returns_none_when_no_checkpoint`` — boundary guards already exercised by the e2e test and the ``handles_none_messages_channel`` regression anchor. Net: -107 lines of test code. Remaining coverage still pins every red-green-verified invariant (channel_versions bump, string-version bump, idempotency, sqlite round-trip, non-title channel preservation, aput-error contract, worker finally swallowing, partial-exchange). Verification: 5209 passed, 15 skipped. * fix: harden interrupted title finalization * fix: serialize interrupted title finalization * fix: preserve interrupt semantics during title finalization * fix: preserve delayed interrupted title recovery --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
6.9 KiB
6.9 KiB
自动 Title 生成功能实现总结
✅ 已完成的工作
1. 核心实现文件
packages/harness/deerflow/agents/thread_state.py
- ✅ 添加
title: str | None = None字段到ThreadState
packages/harness/deerflow/config/title_config.py (新建)
- ✅ 创建
TitleConfig配置类 - ✅ 支持配置:enabled, max_words, max_chars, model_name, prompt_template
- ✅ 提供
get_title_config()和set_title_config()函数 - ✅ 提供
load_title_config_from_dict()从配置文件加载
packages/harness/deerflow/agents/middlewares/title_middleware.py (新建)
- ✅ 创建
TitleMiddleware类 - ✅ 实现
_should_generate_title()检查是否需要生成 - ✅ 默认使用本地 fallback 生成标题,避免流式回复结束前等待额外 LLM 调用;显式配置
model_name时可使用 LLM 标题 - ✅ 实现
after_model()/aafter_model()钩子,在首次对话后自动触发 - ✅ 包含 fallback 策略(LLM 未配置或失败时使用用户消息前几个字符)
packages/harness/deerflow/config/app_config.py
- ✅ 导入
load_title_config_from_dict - ✅ 在
from_file()中加载 title 配置
packages/harness/deerflow/agents/lead_agent/agent.py
- ✅ 导入
TitleMiddleware - ✅ 注册到
middleware列表:[SandboxMiddleware(), TitleMiddleware()]
2. 配置文件
config.yaml
- ✅ 添加 title 配置段:
title:
enabled: true
max_words: 6
max_chars: 60
model_name: null # null = 快速本地 fallback;填模型名才启用 LLM 标题
3. 文档
docs/AUTO_TITLE_GENERATION.md (新建)
- ✅ 完整的功能说明文档
- ✅ 实现方式和架构设计
- ✅ 配置说明
- ✅ 客户端使用示例(TypeScript)
- ✅ 工作流程图(Mermaid)
- ✅ 故障排查指南
- ✅ State vs Metadata 对比
TODO.md
- ✅ 添加功能完成记录
4. 测试
tests/test_title_generation.py (新建)
- ✅ 配置类测试
- ✅ Middleware 初始化测试
- ✅ TODO: 集成测试(需要 mock Runtime)
🎯 核心设计决策
为什么使用 State 而非 Metadata?
| 方面 | State (✅ 采用) | Metadata (❌ 未采用) |
|---|---|---|
| 持久化 | 自动(通过 checkpointer) | 取决于实现,不可靠 |
| 版本控制 | 支持时间旅行 | 不支持 |
| 类型安全 | TypedDict 定义 | 任意字典 |
| 标准化 | LangGraph 核心机制 | 扩展功能 |
工作流程
用户发送首条消息
↓
Agent 处理并返回回复
↓
TitleMiddleware.after_model()/aafter_model() 触发
↓
检查:是否首次对话?是否已有 title?
↓
默认从首条用户消息生成本地 fallback title
↓
如果显式配置 title.model_name,才调用 LLM 生成更精炼的 title
↓
返回 {"title": "..."} 更新 state
↓
Checkpointer 自动持久化(如果配置了)
↓
客户端从 state.values.title 读取
📋 使用指南
后端配置
- 启用/禁用功能
# config.yaml
title:
enabled: true # 设为 false 禁用
- 自定义配置
title:
enabled: true
max_words: 8 # 标题最多 8 个词
max_chars: 80 # 标题最多 80 个字符
model_name: null # null = 快速本地 fallback;填模型名才启用 LLM 标题
- 配置持久化(可选)
如果需要在本地开发时持久化 title:
# checkpointer.py
from langgraph.checkpoint.sqlite import SqliteSaver
checkpointer = SqliteSaver.from_conn_string("deerflow.db")
// langgraph.json
{
"graphs": {
"lead_agent": "deerflow.agents:lead_agent"
},
"checkpointer": "checkpointer:checkpointer"
}
客户端使用
// 获取 thread title
const state = await client.threads.getState(threadId);
const title = state.values.title || "New Conversation";
// 显示在对话列表
<li>{title}</li>
⚠️ 注意:Title 在 state.values.title,而非 thread.metadata.title
🧪 测试
# 运行测试
pytest tests/test_title_generation.py -v
# 运行所有测试
pytest
🔍 故障排查
Title 没有生成?
- 检查配置:
title.enabled = true - 确认是首次对话(1 个用户消息 + 1 个助手回复)
- 如果显式配置了
title.model_name,检查标题模型是否可用;未配置时会走本地 fallback
Title 生成但看不到?
- 确认读取位置:
state.values.title(不是thread.metadata.title) - 检查 API 响应是否包含 title
- 重新获取 state
Title 重启后丢失?
- 本地开发需要配置 checkpointer
- LangGraph Platform 会自动持久化
- 检查数据库确认 checkpointer 工作正常
中断首轮后仍显示默认标题?
runtime/runs/worker.py会在 interrupted-run cleanup 中保持 run 处于 finalizing 状态,避免同线程新 run 在 fallback title 写入期间覆盖 checkpoint- 如果取消发生在可用 checkpoint 写入前,worker 会使用本次
graph_input中的首条用户消息生成本地 fallback title - fallback title 写入前会重新读取 latest checkpoint;如果同线程状态已经前进,只对最新 snapshot 做 title-only 更新,避免旧消息重新成为 latest
📊 性能影响
- 默认延迟:默认
title.model_name: null不会发起额外 LLM 调用,仅从首条用户消息生成本地 fallback title - 显式 LLM 标题延迟:只有配置
title.model_name时,首轮回复后才会等待一次标题模型调用 - 并发安全:在
after_model()/aafter_model()中更新 state,不需要客户端额外请求 - 资源消耗:每个 thread 只生成一次
优化建议
- 默认保持
model_name: null,避免流式回复结束前的额外 LLM 等待 - 如需更精炼标题,再显式配置较快的标题模型
- 减少
max_words和max_chars,并让 prompt 保持简洁
🚀 下一步
- 补充 prompt template 的集成测试
- 支持多语言 title 生成
- 添加 title 重新生成功能
- 监控 title 生成成功率和延迟
📚 相关资源
实现完成时间: 2026-01-14