mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-09 16:08:31 +00:00
feat(frontend):enhance the ask_clarification interaction with visualized card (#3956)
Some checks are pending
Backend Blocking IO / backend-blocking-io (push) Waiting to run
Unit Tests / backend-unit-tests (push) Waiting to run
E2E Tests / e2e-tests (push) Waiting to run
Frontend Unit Tests / frontend-unit-tests (push) Waiting to run
Lint Check / lint-backend (push) Waiting to run
Lint Check / lint-frontend (push) Waiting to run
Replay E2E (front-back contract) / Layer 1 — backend golden (no API key) (push) Waiting to run
Replay E2E (front-back contract) / Layer 2 — full-stack render (no API key) (push) Waiting to run
Some checks are pending
Backend Blocking IO / backend-blocking-io (push) Waiting to run
Unit Tests / backend-unit-tests (push) Waiting to run
E2E Tests / e2e-tests (push) Waiting to run
Frontend Unit Tests / frontend-unit-tests (push) Waiting to run
Lint Check / lint-backend (push) Waiting to run
Lint Check / lint-frontend (push) Waiting to run
Replay E2E (front-back contract) / Layer 1 — backend golden (no API key) (push) Waiting to run
Replay E2E (front-back contract) / Layer 2 — full-stack render (no API key) (push) Waiting to run
* feat(frontend): add structured human input cards for ask_clarification Implement a reusable Human Input Card flow for ask_clarification while keeping the existing text fallback for older clients and IM channels. Backend: - Add structured ToolMessage.artifact.human_input payloads for clarification requests. - Preserve ToolMessage.content as the readable Markdown/text fallback. - Normalize clarification options from native lists, JSON strings, plain strings, mixed scalar values, None, and missing options. - Derive input_mode as choice_with_other when options exist, otherwise free_text. - Keep disable_clarification non-interactive behavior as a plain ToolMessage with no human_input artifact. - Cover artifact persistence and Gateway message metadata preservation in tests. Frontend: - Add human input protocol types, runtime guards, extractors, response builders, and thread-state helpers. - Add reusable HumanInputCard with option buttons, free-text input, pending, read-only, disabled, and answered states. - Render structured clarification cards from artifact.human_input, with Markdown fallback for malformed or legacy tool messages. - Preserve line breaks in structured question/context/option text. - Hide submitted clarification bridge messages from the chat UI via additional_kwargs.hide_from_ui. - Send structured human_input_response metadata through the fourth sendMessage options argument, preserving run context in the third argument. - Wire submissions for normal chats, custom agent chats, agent bootstrap chats, and sidecar chats. - Derive answered state from raw thread.messages so hidden replies still update the original card. - Clear pending state when the hidden reply arrives, dispatch is dropped, or a later async stream failure appears on thread.error. * perf(frontend): optimize HumanInputCard UI interactions - Support Enter key to submit text input (Shift+Enter for newline) - Render question and context fields as Markdown instead of plain text - Replace deprecated FormEventHandler type with structural typing * test(frontend): add unit test cover optimize HumanInputCard UI interactions * feat(frontend): disabled chatbox when has new human-input-card * fix(style): lint error fix * fix: sanitize hidden human input replies - Preserve IME composition safety for human input card Enter submits - Treat hidden human input responses as genuine user messages for sanitization - Keep hidden card replies in memory filtering while excluding malformed/internal hidden messages - Add regression coverage for card IME handling and hidden reply sanitization * fix: tighten human input response validation - Reject empty hidden human input response values - Remove invalid list ARIA role from human input card options - Add backend coverage for empty response payloads --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
823c47bcc2
commit
47b0f604f4
24 changed files with 1775 additions and 33 deletions
|
|
@ -249,7 +249,7 @@ Lead-agent middlewares are assembled in strict order across three functions: the
|
|||
26. **TokenBudgetMiddleware** - *(optional, if `token_budget.enabled`)* Enforces per-run token limits
|
||||
27. **Custom middlewares** - *(optional)* Any `custom_middlewares` passed to `build_middlewares` are injected here, before the safety/clarification tail
|
||||
28. **SafetyFinishReasonMiddleware** - *(optional, if `safety_finish_reason.enabled`)* Suppresses tool execution when the provider safety-terminated the response (e.g. `finish_reason=content_filter`); registered after custom middlewares so LangChain's reverse-order `after_model` dispatch runs it first
|
||||
29. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
|
||||
29. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, writes a readable `ToolMessage.content` fallback plus structured `ToolMessage.artifact.human_input` request payload, and interrupts via `Command(goto=END)` (must be last)
|
||||
|
||||
### Configuration System
|
||||
|
||||
|
|
@ -388,7 +388,7 @@ Proxied through nginx: `/api/langgraph/*` → Gateway LangGraph-compatible runti
|
|||
2. **MCP tools** - From enabled MCP servers (lazy initialized, cached with mtime invalidation)
|
||||
3. **Built-in tools**:
|
||||
- `present_files` - Make output files visible to user (only `/mnt/user-data/outputs`)
|
||||
- `ask_clarification` - Request clarification (intercepted by ClarificationMiddleware → interrupts)
|
||||
- `ask_clarification` - Request clarification (intercepted by ClarificationMiddleware, which preserves text fallback and adds `artifact.human_input` for Web UI Human Input Cards)
|
||||
- `view_image` - Read image as base64 (added only if model supports vision)
|
||||
- `setup_agent` - Bootstrap-only: persist a brand-new custom agent's `SOUL.md` and `config.yaml`. Bound only when `is_bootstrap=True`.
|
||||
- `update_agent` - Custom-agent-only: persist self-updates to the current agent's `SOUL.md` / `config.yaml` from inside a normal chat (partial update + atomic write). Bound when `agent_name` is set and `is_bootstrap=False`.
|
||||
|
|
|
|||
76
backend/packages/harness/deerflow/agents/human_input.py
Normal file
76
backend/packages/harness/deerflow/agents/human_input.py
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
"""Structured human-input message metadata helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import Literal, TypedDict
|
||||
|
||||
HUMAN_INPUT_RESPONSE_KEY = "human_input_response"
|
||||
|
||||
|
||||
class HumanInputTextResponse(TypedDict):
|
||||
version: Literal[1]
|
||||
kind: Literal["human_input_response"]
|
||||
source: str
|
||||
request_id: str
|
||||
response_kind: Literal["text"]
|
||||
value: str
|
||||
|
||||
|
||||
class HumanInputOptionResponse(TypedDict):
|
||||
version: Literal[1]
|
||||
kind: Literal["human_input_response"]
|
||||
source: str
|
||||
request_id: str
|
||||
response_kind: Literal["option"]
|
||||
option_id: str
|
||||
value: str
|
||||
|
||||
|
||||
HumanInputResponse = HumanInputTextResponse | HumanInputOptionResponse
|
||||
|
||||
|
||||
def _non_empty_string(value: object) -> str | None:
|
||||
return value if isinstance(value, str) and value.strip() else None
|
||||
|
||||
|
||||
def read_human_input_response(additional_kwargs: Mapping[str, object] | None) -> HumanInputResponse | None:
|
||||
"""Read a valid human-input response payload from message metadata."""
|
||||
if not additional_kwargs:
|
||||
return None
|
||||
raw = additional_kwargs.get(HUMAN_INPUT_RESPONSE_KEY)
|
||||
if not isinstance(raw, Mapping):
|
||||
return None
|
||||
if raw.get("version") != 1 or raw.get("kind") != "human_input_response":
|
||||
return None
|
||||
|
||||
source = _non_empty_string(raw.get("source"))
|
||||
request_id = _non_empty_string(raw.get("request_id"))
|
||||
value = _non_empty_string(raw.get("value"))
|
||||
if source is None or request_id is None or value is None:
|
||||
return None
|
||||
|
||||
response_kind = raw.get("response_kind")
|
||||
if response_kind == "text":
|
||||
return {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": source,
|
||||
"request_id": request_id,
|
||||
"response_kind": "text",
|
||||
"value": value,
|
||||
}
|
||||
if response_kind == "option":
|
||||
option_id = _non_empty_string(raw.get("option_id"))
|
||||
if option_id is None:
|
||||
return None
|
||||
return {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": source,
|
||||
"request_id": request_id,
|
||||
"response_kind": "option",
|
||||
"option_id": option_id,
|
||||
"value": value,
|
||||
}
|
||||
return None
|
||||
|
|
@ -6,6 +6,8 @@ import re
|
|||
from copy import copy
|
||||
from typing import Any
|
||||
|
||||
from deerflow.agents.human_input import read_human_input_response
|
||||
|
||||
_UPLOAD_BLOCK_RE = re.compile(r"<uploaded_files>[\s\S]*?</uploaded_files>\n*", re.IGNORECASE)
|
||||
_CORRECTION_PATTERNS = (
|
||||
re.compile(r"\bthat(?:'s| is) (?:wrong|incorrect)\b", re.IGNORECASE),
|
||||
|
|
@ -66,7 +68,8 @@ def filter_messages_for_memory(messages: list[Any]) -> list[Any]:
|
|||
# hide_from_ui and must never reach the memory-updating LLM — otherwise
|
||||
# framework-internal text pollutes long-term memory (and the p0 __memory
|
||||
# payload could trigger a self-amplification loop).
|
||||
if getattr(msg, "additional_kwargs", {}).get("hide_from_ui"):
|
||||
additional_kwargs = getattr(msg, "additional_kwargs", {}) or {}
|
||||
if additional_kwargs.get("hide_from_ui") and read_human_input_response(additional_kwargs) is None:
|
||||
continue
|
||||
content_str = extract_message_text(msg)
|
||||
if "<uploaded_files>" in content_str:
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import json
|
|||
import logging
|
||||
from collections.abc import Callable
|
||||
from hashlib import sha256
|
||||
from typing import override
|
||||
from typing import Any, override
|
||||
|
||||
from langchain.agents import AgentState
|
||||
from langchain.agents.middleware import AgentMiddleware
|
||||
|
|
@ -44,6 +44,60 @@ class ClarificationMiddleware(AgentMiddleware[ClarificationMiddlewareState]):
|
|||
digest = sha256(formatted_message.encode("utf-8")).hexdigest()[:16]
|
||||
return f"clarification:{digest}"
|
||||
|
||||
def _normalize_options(self, raw_options: Any) -> list[str]:
|
||||
"""Normalize tool-provided options into displayable string values."""
|
||||
options = raw_options
|
||||
|
||||
# Some models (e.g. Qwen3-Max) serialize array parameters as JSON strings
|
||||
# instead of native arrays. Deserialize and normalize so `options`
|
||||
# is always a list for the rendering logic below.
|
||||
if isinstance(options, str):
|
||||
try:
|
||||
options = json.loads(options)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
options = [options]
|
||||
|
||||
if options is None:
|
||||
return []
|
||||
if not isinstance(options, list):
|
||||
options = [options]
|
||||
|
||||
return [str(option) for option in options]
|
||||
|
||||
def _build_human_input_payload(self, args: dict[str, Any], *, tool_call_id: str, request_id: str) -> dict[str, Any]:
|
||||
"""Build the structured UI payload while keeping ToolMessage.content as fallback."""
|
||||
options = self._normalize_options(args.get("options", []))
|
||||
clarification_type = str(args.get("clarification_type", "missing_info"))
|
||||
|
||||
payload: dict[str, Any] = {
|
||||
"version": 1,
|
||||
"kind": "human_input_request",
|
||||
"source": "ask_clarification",
|
||||
"request_id": request_id,
|
||||
"clarification_type": clarification_type,
|
||||
"question": str(args.get("question") or ""),
|
||||
"input_mode": "choice_with_other" if options else "free_text",
|
||||
}
|
||||
|
||||
if tool_call_id:
|
||||
payload["tool_call_id"] = tool_call_id
|
||||
|
||||
if "context" in args:
|
||||
context = args.get("context")
|
||||
payload["context"] = None if context is None else str(context)
|
||||
|
||||
if options:
|
||||
payload["options"] = [
|
||||
{
|
||||
"id": f"option-{index}",
|
||||
"label": option,
|
||||
"value": option,
|
||||
}
|
||||
for index, option in enumerate(options, 1)
|
||||
]
|
||||
|
||||
return payload
|
||||
|
||||
def _is_chinese(self, text: str) -> bool:
|
||||
"""Check if text contains Chinese characters.
|
||||
|
||||
|
|
@ -67,21 +121,7 @@ class ClarificationMiddleware(AgentMiddleware[ClarificationMiddlewareState]):
|
|||
question = args.get("question", "")
|
||||
clarification_type = args.get("clarification_type", "missing_info")
|
||||
context = args.get("context")
|
||||
options = args.get("options", [])
|
||||
|
||||
# Some models (e.g. Qwen3-Max) serialize array parameters as JSON strings
|
||||
# instead of native arrays. Deserialize and normalize so `options`
|
||||
# is always a list for the rendering logic below.
|
||||
if isinstance(options, str):
|
||||
try:
|
||||
options = json.loads(options)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
options = [options]
|
||||
|
||||
if options is None:
|
||||
options = []
|
||||
elif not isinstance(options, list):
|
||||
options = [options]
|
||||
options = self._normalize_options(args.get("options", []))
|
||||
|
||||
# Type-specific icons
|
||||
type_icons = {
|
||||
|
|
@ -174,13 +214,17 @@ class ClarificationMiddleware(AgentMiddleware[ClarificationMiddlewareState]):
|
|||
# Get the tool call ID
|
||||
tool_call_id = request.tool_call.get("id", "")
|
||||
|
||||
request_id = self._stable_message_id(tool_call_id, formatted_message)
|
||||
human_input_payload = self._build_human_input_payload(args, tool_call_id=tool_call_id, request_id=request_id)
|
||||
|
||||
# Create a ToolMessage with the formatted question
|
||||
# This will be added to the message history
|
||||
tool_message = ToolMessage(
|
||||
id=self._stable_message_id(tool_call_id, formatted_message),
|
||||
id=request_id,
|
||||
content=formatted_message,
|
||||
tool_call_id=tool_call_id,
|
||||
name="ask_clarification",
|
||||
artifact={"human_input": human_input_payload},
|
||||
)
|
||||
|
||||
# Return a Command that:
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ from langchain.agents.middleware.types import (
|
|||
from langchain_core.messages import HumanMessage
|
||||
from langgraph.errors import GraphBubbleUp
|
||||
|
||||
from deerflow.agents.human_input import read_human_input_response
|
||||
from deerflow.utils.messages import ORIGINAL_USER_CONTENT_KEY, message_content_to_text
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
|
@ -90,15 +91,15 @@ def _escape_tag_match(match: re.Match) -> str:
|
|||
def _is_genuine_user_message(message: object) -> bool:
|
||||
"""Return True for real user messages, excluding system-injected HumanMessages.
|
||||
|
||||
System-injected context is marked via ``hide_from_ui`` — the same convention
|
||||
used by DynamicContextMiddleware and TodoMiddleware.
|
||||
``hide_from_ui`` is also used by hidden UI replies from HumanInputCard, so
|
||||
only skip hidden HumanMessages that do not carry a valid user response.
|
||||
"""
|
||||
if not isinstance(message, HumanMessage):
|
||||
return False
|
||||
if message.additional_kwargs.get("hide_from_ui"):
|
||||
return False
|
||||
if message.name == _SUMMARY_MESSAGE_NAME:
|
||||
return False
|
||||
if message.additional_kwargs.get("hide_from_ui") and read_human_input_response(message.additional_kwargs) is None:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -122,6 +122,99 @@ class TestFormatClarificationMessage:
|
|||
assert "4. None" in result
|
||||
|
||||
|
||||
class TestHumanInputPayload:
|
||||
"""Tests for structured human input request payloads."""
|
||||
|
||||
def test_payload_with_native_options(self, middleware):
|
||||
payload = middleware._build_human_input_payload(
|
||||
{
|
||||
"question": "Which environment should I deploy to?",
|
||||
"clarification_type": "approach_choice",
|
||||
"context": "Need the target environment for config.",
|
||||
"options": ["development", "staging", "production"],
|
||||
},
|
||||
tool_call_id="call-abc",
|
||||
request_id="clarification:call-abc",
|
||||
)
|
||||
|
||||
assert payload == {
|
||||
"version": 1,
|
||||
"kind": "human_input_request",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"tool_call_id": "call-abc",
|
||||
"clarification_type": "approach_choice",
|
||||
"question": "Which environment should I deploy to?",
|
||||
"context": "Need the target environment for config.",
|
||||
"input_mode": "choice_with_other",
|
||||
"options": [
|
||||
{"id": "option-1", "label": "development", "value": "development"},
|
||||
{"id": "option-2", "label": "staging", "value": "staging"},
|
||||
{"id": "option-3", "label": "production", "value": "production"},
|
||||
],
|
||||
}
|
||||
|
||||
def test_payload_with_json_string_options(self, middleware):
|
||||
payload = middleware._build_human_input_payload(
|
||||
{
|
||||
"question": "Pick one",
|
||||
"clarification_type": "approach_choice",
|
||||
"options": json.dumps(["Option A", 2, True, None]),
|
||||
},
|
||||
tool_call_id="call-abc",
|
||||
request_id="clarification:call-abc",
|
||||
)
|
||||
|
||||
assert payload["input_mode"] == "choice_with_other"
|
||||
assert payload["options"] == [
|
||||
{"id": "option-1", "label": "Option A", "value": "Option A"},
|
||||
{"id": "option-2", "label": "2", "value": "2"},
|
||||
{"id": "option-3", "label": "True", "value": "True"},
|
||||
{"id": "option-4", "label": "None", "value": "None"},
|
||||
]
|
||||
|
||||
def test_payload_with_plain_string_option(self, middleware):
|
||||
payload = middleware._build_human_input_payload(
|
||||
{
|
||||
"question": "Pick one",
|
||||
"clarification_type": "approach_choice",
|
||||
"options": "just one option",
|
||||
},
|
||||
tool_call_id="call-abc",
|
||||
request_id="clarification:call-abc",
|
||||
)
|
||||
|
||||
assert payload["input_mode"] == "choice_with_other"
|
||||
assert payload["options"] == [{"id": "option-1", "label": "just one option", "value": "just one option"}]
|
||||
|
||||
def test_payload_without_options_is_free_text(self, middleware):
|
||||
payload = middleware._build_human_input_payload(
|
||||
{
|
||||
"question": "Tell me more",
|
||||
"clarification_type": "missing_info",
|
||||
"options": None,
|
||||
},
|
||||
tool_call_id="call-abc",
|
||||
request_id="clarification:call-abc",
|
||||
)
|
||||
|
||||
assert payload["input_mode"] == "free_text"
|
||||
assert "options" not in payload
|
||||
|
||||
def test_payload_missing_options_is_free_text(self, middleware):
|
||||
payload = middleware._build_human_input_payload(
|
||||
{
|
||||
"question": "Tell me more",
|
||||
"clarification_type": "missing_info",
|
||||
},
|
||||
tool_call_id="call-abc",
|
||||
request_id="clarification:call-abc",
|
||||
)
|
||||
|
||||
assert payload["input_mode"] == "free_text"
|
||||
assert "options" not in payload
|
||||
|
||||
|
||||
class TestClarificationCommandIdempotency:
|
||||
"""Clarification tool-call retries should not duplicate messages in state."""
|
||||
|
||||
|
|
@ -147,12 +240,41 @@ class TestClarificationCommandIdempotency:
|
|||
assert first_message.id == "clarification:call-clarify-1"
|
||||
assert second_message.id == first_message.id
|
||||
assert second_message.tool_call_id == first_message.tool_call_id
|
||||
assert first_message.artifact["human_input"]["request_id"] == "clarification:call-clarify-1"
|
||||
assert first_message.artifact["human_input"]["tool_call_id"] == "call-clarify-1"
|
||||
assert first_message.artifact["human_input"]["clarification_type"] == "approach_choice"
|
||||
assert first_message.artifact["human_input"]["input_mode"] == "choice_with_other"
|
||||
|
||||
merged = add_messages(add_messages([], [first_message]), [second_message])
|
||||
|
||||
assert len(merged) == 1
|
||||
assert merged[0].id == "clarification:call-clarify-1"
|
||||
assert merged[0].content == first_message.content
|
||||
assert merged[0].artifact == first_message.artifact
|
||||
|
||||
def test_tool_message_model_dump_preserves_human_input_artifact(self, middleware):
|
||||
request = SimpleNamespace(
|
||||
tool_call={
|
||||
"name": "ask_clarification",
|
||||
"id": "call-clarify-1",
|
||||
"args": {
|
||||
"question": "Which environment should I use?",
|
||||
"clarification_type": "approach_choice",
|
||||
"options": ["dev", "prod"],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
result = middleware.wrap_tool_call(request, lambda _req: pytest.fail("handler should not be called"))
|
||||
message = result.update["messages"][0]
|
||||
dumped = message.model_dump()
|
||||
|
||||
assert dumped["artifact"]["human_input"]["request_id"] == "clarification:call-clarify-1"
|
||||
assert dumped["artifact"]["human_input"]["options"] == [
|
||||
{"id": "option-1", "label": "dev", "value": "dev"},
|
||||
{"id": "option-2", "label": "prod", "value": "prod"},
|
||||
]
|
||||
assert "Which environment should I use?" in dumped["content"]
|
||||
|
||||
|
||||
class TestClarificationDisabled:
|
||||
|
|
@ -178,6 +300,7 @@ class TestClarificationDisabled:
|
|||
|
||||
assert isinstance(result, ToolMessage)
|
||||
assert result.tool_call_id == "call-clarify-1"
|
||||
assert result.artifact is None
|
||||
|
||||
def test_disabled_message_tells_agent_to_proceed(self, middleware):
|
||||
request = self._request(runtime_context={"disable_clarification": True})
|
||||
|
|
|
|||
|
|
@ -132,6 +132,37 @@ def test_normalize_input_preserves_additional_kwargs_and_id():
|
|||
assert msg.additional_kwargs == {"files": files, "custom": "keep-me"}
|
||||
|
||||
|
||||
def test_normalize_input_preserves_human_input_response_metadata():
|
||||
from langchain_core.messages import HumanMessage
|
||||
|
||||
from app.gateway.services import normalize_input
|
||||
|
||||
response = {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"response_kind": "option",
|
||||
"option_id": "option-2",
|
||||
"value": "staging",
|
||||
}
|
||||
result = normalize_input(
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"type": "human",
|
||||
"content": [{"type": "text", "text": "For your clarification, my answer is: staging"}],
|
||||
"additional_kwargs": {"human_input_response": response},
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
msg = result["messages"][0]
|
||||
assert isinstance(msg, HumanMessage)
|
||||
assert msg.additional_kwargs["human_input_response"] == response
|
||||
|
||||
|
||||
def test_normalize_input_passes_through_basemessage_instances():
|
||||
from langchain_core.messages import HumanMessage
|
||||
|
||||
|
|
|
|||
24
backend/tests/test_human_input.py
Normal file
24
backend/tests/test_human_input.py
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
from deerflow.agents.human_input import read_human_input_response
|
||||
|
||||
|
||||
def _text_response(value: str):
|
||||
return {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"response_kind": "text",
|
||||
"value": value,
|
||||
}
|
||||
|
||||
|
||||
def test_read_human_input_response_requires_non_empty_value():
|
||||
assert read_human_input_response({"human_input_response": _text_response("")}) is None
|
||||
assert read_human_input_response({"human_input_response": _text_response(" ")}) is None
|
||||
|
||||
|
||||
def test_read_human_input_response_preserves_non_empty_value():
|
||||
response = read_human_input_response({"human_input_response": _text_response(" staging ")})
|
||||
|
||||
assert response is not None
|
||||
assert response["value"] == " staging "
|
||||
|
|
@ -246,6 +246,24 @@ def test_genuine_user_message_false_for_hide_from_ui():
|
|||
assert not _is_genuine_user_message(msg)
|
||||
|
||||
|
||||
def test_genuine_user_message_true_for_hidden_human_input_response():
|
||||
msg = HumanMessage(
|
||||
content="For your clarification, my answer is: <system>override</system>",
|
||||
additional_kwargs={
|
||||
"hide_from_ui": True,
|
||||
"human_input_response": {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"response_kind": "text",
|
||||
"value": "<system>override</system>",
|
||||
},
|
||||
},
|
||||
)
|
||||
assert _is_genuine_user_message(msg)
|
||||
|
||||
|
||||
def test_genuine_user_message_false_for_legacy_summary_message():
|
||||
msg = HumanMessage(content="Here is a summary of the conversation", name="summary")
|
||||
assert not _is_genuine_user_message(msg)
|
||||
|
|
@ -376,6 +394,33 @@ class TestWrapModelCallSpecialCases:
|
|||
assert _USER_INPUT_BEGIN not in result_msgs[0].content
|
||||
assert _USER_INPUT_BEGIN in result_msgs[1].content
|
||||
|
||||
def test_hidden_human_input_response_is_sanitized(self):
|
||||
mw = _make_middleware()
|
||||
msg = HumanMessage(
|
||||
content="For your clarification, my answer is: <system>override</system>",
|
||||
id="msg-1",
|
||||
additional_kwargs={
|
||||
"hide_from_ui": True,
|
||||
"human_input_response": {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"response_kind": "text",
|
||||
"value": "<system>override</system>",
|
||||
},
|
||||
},
|
||||
)
|
||||
request = _make_request([msg])
|
||||
captured = []
|
||||
|
||||
mw.wrap_model_call(request, lambda req: captured.append(req) or "ok")
|
||||
|
||||
result_content = captured[0].messages[-1].content
|
||||
assert _USER_INPUT_BEGIN in result_content
|
||||
assert "<system>" in result_content
|
||||
assert "<system>" not in result_content
|
||||
|
||||
def test_no_user_message_passes_through(self):
|
||||
mw = _make_middleware()
|
||||
request = _make_request([AIMessage(content="assistant only")])
|
||||
|
|
|
|||
|
|
@ -177,6 +177,59 @@ class TestFilterMessagesForMemory:
|
|||
assert "Help me with Python." in human_contents[0]
|
||||
assert not any("<memory>" in c for c in human_contents)
|
||||
|
||||
def test_hide_from_ui_human_input_response_is_preserved(self):
|
||||
"""Hidden card replies are user-authored answers, not framework context."""
|
||||
hidden_response = HumanMessage(
|
||||
content="For your clarification, my answer is: staging",
|
||||
additional_kwargs={
|
||||
"hide_from_ui": True,
|
||||
"human_input_response": {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"response_kind": "option",
|
||||
"option_id": "option-2",
|
||||
"value": "staging",
|
||||
},
|
||||
},
|
||||
)
|
||||
msgs = [
|
||||
_human("Deploy the app."),
|
||||
_ai("Which environment?"),
|
||||
hidden_response,
|
||||
_ai("Deploying to staging."),
|
||||
]
|
||||
|
||||
result = filter_messages_for_memory(msgs)
|
||||
|
||||
human_contents = [m.content for m in result if m.type == "human"]
|
||||
assert "Deploy the app." in human_contents
|
||||
assert "For your clarification, my answer is: staging" in human_contents
|
||||
|
||||
def test_hide_from_ui_malformed_human_input_response_is_excluded(self):
|
||||
hidden_response = HumanMessage(
|
||||
content="For your clarification, my answer is: staging",
|
||||
additional_kwargs={
|
||||
"hide_from_ui": True,
|
||||
"human_input_response": {
|
||||
"version": 1,
|
||||
"kind": "human_input_response",
|
||||
"source": "ask_clarification",
|
||||
"request_id": "clarification:call-abc",
|
||||
"response_kind": "option",
|
||||
"value": "staging",
|
||||
},
|
||||
},
|
||||
)
|
||||
msgs = [_human("Deploy the app."), _ai("Which environment?"), hidden_response]
|
||||
|
||||
result = filter_messages_for_memory(msgs)
|
||||
|
||||
human_contents = [m.content for m in result if m.type == "human"]
|
||||
assert "Deploy the app." in human_contents
|
||||
assert "For your clarification, my answer is: staging" not in human_contents
|
||||
|
||||
def test_hide_from_ui_false_is_preserved(self):
|
||||
"""Messages without hide_from_ui (or with it set to False) are kept."""
|
||||
visible_msg = HumanMessage(content="Visible message", additional_kwargs={"hide_from_ui": False})
|
||||
|
|
|
|||
|
|
@ -71,6 +71,8 @@ The frontend is a stateful chat application. Users create **threads** (conversat
|
|||
|
||||
`/goal` is a built-in composer command, not a skill activation. `src/components/workspace/input-box.tsx` intercepts `/goal`, `/goal clear`, and `/goal <condition>` before normal chat submission, calling Gateway `GET/PUT/DELETE /api/threads/{thread_id}/goal`. Setting `/goal <condition>` also submits the condition text as the next user task so the agent starts running immediately; status and clear do not start a run. Goal requests are tied to the current `threadId` with an `AbortController`, so switching threads or unmounting the composer aborts in-flight goal requests and stale responses cannot update the new thread's goal state. The chat pages render `GoalStatus` above the composer from `AgentThreadState.goal`, with local optimistic state until the next stream `values` update arrives.
|
||||
|
||||
Human input requests are a structured message protocol layered on normal chat history. The backend writes request payloads to `ToolMessage.artifact.human_input`, `src/core/messages/human-input.ts` owns the runtime validators/types, and `src/components/workspace/messages/human-input-card.tsx` renders the reusable card. `MessageList` owns answered/latest/pending state for visible cards, but derives answered responses from raw `thread.messages` because replies are hidden; pending cards clear when the hidden reply appears, when dispatch is dropped, or when a new `thread.error` reports an async stream failure. Page-level submit callbacks must send a normal human message and put `hide_from_ui: true` plus the response payload in the fourth `sendMessage(..., options)` argument as `options.additionalKwargs`; the third argument remains run context such as `{ agent_name }`. Composer entry points should disable normal bottom input while `hasOpenHumanInputRequest(...)` is true so users answer through the card and preserve response metadata.
|
||||
|
||||
### Key Patterns
|
||||
|
||||
- **Server Components by default**, `"use client"` only for interactive components
|
||||
|
|
@ -84,6 +86,7 @@ The frontend is a stateful chat application. Users create **threads** (conversat
|
|||
- `src/app/workspace/chats/[thread_id]/page.tsx` owns composer busy-state wiring.
|
||||
- `src/app/workspace/chats/[thread_id]/page.tsx` owns branch-from-turn submission and navigation; sidecar `MessageList` instances do not receive the branch action.
|
||||
- `src/app/workspace/chats/[thread_id]/page.tsx` and `src/app/workspace/agents/[agent_name]/chats/[thread_id]/page.tsx` own active-goal display state for their composer overlays.
|
||||
- `src/components/workspace/messages/message-list.tsx` owns human-input card answered/latest/pending gating; entry pages only translate a submitted card response into `sendMessage` calls.
|
||||
- `src/core/threads/hooks.ts` owns pre-submit upload state and thread submission.
|
||||
|
||||
## Code Style
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
import { BotIcon, PlusSquare } from "lucide-react";
|
||||
import { useParams, useRouter } from "next/navigation";
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import { useCallback, useEffect, useMemo, useState } from "react";
|
||||
|
||||
import type { PromptInputMessage } from "@/components/ai-elements/prompt-input";
|
||||
import { Button } from "@/components/ui/button";
|
||||
|
|
@ -32,6 +32,13 @@ import { Tooltip } from "@/components/workspace/tooltip";
|
|||
import { useActiveGoal } from "@/components/workspace/use-active-goal";
|
||||
import { useAgent } from "@/core/agents";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
buildHumanInputResponseText,
|
||||
hasOpenHumanInputRequest,
|
||||
type HumanInputRequest,
|
||||
type HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
import { isHiddenFromUIMessage } from "@/core/messages/utils";
|
||||
import { useModels } from "@/core/models/hooks";
|
||||
import { useNotification } from "@/core/notification/hooks";
|
||||
import { useLocalSettings, useThreadSettings } from "@/core/settings";
|
||||
|
|
@ -168,6 +175,31 @@ export default function AgentChatPage() {
|
|||
[sendMessage, threadId, agent_name],
|
||||
);
|
||||
|
||||
const handleSubmitHumanInput = useCallback(
|
||||
async (request: HumanInputRequest, response: HumanInputResponse) => {
|
||||
let sent = false;
|
||||
await sendMessage(
|
||||
threadId,
|
||||
{
|
||||
text: buildHumanInputResponseText(request, response),
|
||||
files: [],
|
||||
},
|
||||
{ agent_name },
|
||||
{
|
||||
additionalKwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
onSent: () => {
|
||||
sent = true;
|
||||
},
|
||||
},
|
||||
);
|
||||
return sent;
|
||||
},
|
||||
[agent_name, sendMessage, threadId],
|
||||
);
|
||||
|
||||
const handleStop = useCallback(async () => {
|
||||
await thread.stop();
|
||||
}, [thread]);
|
||||
|
|
@ -180,6 +212,14 @@ export default function AgentChatPage() {
|
|||
threadId,
|
||||
thread.values.goal,
|
||||
);
|
||||
const hasOpenHumanInputCard = useMemo(
|
||||
() =>
|
||||
hasOpenHumanInputRequest(
|
||||
thread.messages,
|
||||
(message) => !isHiddenFromUIMessage(message),
|
||||
),
|
||||
[thread.messages],
|
||||
);
|
||||
|
||||
return (
|
||||
<ThreadContext.Provider value={{ thread, isMock }}>
|
||||
|
|
@ -253,6 +293,11 @@ export default function AgentChatPage() {
|
|||
loadMoreHistory={loadMoreHistory}
|
||||
isHistoryLoading={isHistoryLoading}
|
||||
tokenUsageInlineMode={tokenUsageInlineMode}
|
||||
onSubmitHumanInput={
|
||||
isMock || env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true"
|
||||
? undefined
|
||||
: handleSubmitHumanInput
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
|
||||
|
|
@ -322,7 +367,9 @@ export default function AgentChatPage() {
|
|||
}
|
||||
disabled={
|
||||
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" ||
|
||||
isUploading
|
||||
isUploading ||
|
||||
hasOpenHumanInputCard ||
|
||||
(!isNewThread && isHistoryLoading)
|
||||
}
|
||||
onContextChange={(context) =>
|
||||
setSettings("context", context)
|
||||
|
|
|
|||
|
|
@ -38,6 +38,13 @@ import {
|
|||
getAgent,
|
||||
} from "@/core/agents/api";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
buildHumanInputResponseText,
|
||||
hasOpenHumanInputRequest,
|
||||
type HumanInputRequest,
|
||||
type HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
import { isHiddenFromUIMessage } from "@/core/messages/utils";
|
||||
import { useThreadStream } from "@/core/threads/hooks";
|
||||
import { uuid } from "@/core/utils/uuid";
|
||||
import { isIMEComposing } from "@/lib/ime";
|
||||
|
|
@ -110,6 +117,14 @@ export default function NewAgentPage() {
|
|||
});
|
||||
},
|
||||
});
|
||||
const hasOpenHumanInputCard = useMemo(
|
||||
() =>
|
||||
hasOpenHumanInputRequest(
|
||||
thread.messages,
|
||||
(message) => !isHiddenFromUIMessage(message),
|
||||
),
|
||||
[thread.messages],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (typeof window === "undefined" || step !== "chat") {
|
||||
|
|
@ -208,14 +223,43 @@ export default function NewAgentPage() {
|
|||
const handleChatSubmit = useCallback(
|
||||
async (text: string) => {
|
||||
const trimmed = text.trim();
|
||||
if (!trimmed || thread.isLoading) return;
|
||||
if (!trimmed || thread.isLoading || hasOpenHumanInputCard) return;
|
||||
await sendMessage(
|
||||
threadId,
|
||||
{ text: trimmed, files: [] },
|
||||
{ agent_name: agentName },
|
||||
);
|
||||
},
|
||||
[agentName, sendMessage, thread.isLoading, threadId],
|
||||
[agentName, hasOpenHumanInputCard, sendMessage, thread.isLoading, threadId],
|
||||
);
|
||||
|
||||
const handleSubmitHumanInput = useCallback(
|
||||
async (request: HumanInputRequest, response: HumanInputResponse) => {
|
||||
if (!agentName) {
|
||||
return false;
|
||||
}
|
||||
|
||||
let sent = false;
|
||||
await sendMessage(
|
||||
threadId,
|
||||
{
|
||||
text: buildHumanInputResponseText(request, response),
|
||||
files: [],
|
||||
},
|
||||
{ agent_name: agentName },
|
||||
{
|
||||
additionalKwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
onSent: () => {
|
||||
sent = true;
|
||||
},
|
||||
},
|
||||
);
|
||||
return sent;
|
||||
},
|
||||
[agentName, sendMessage, threadId],
|
||||
);
|
||||
|
||||
const handleSaveAgent = useCallback(async () => {
|
||||
|
|
@ -365,6 +409,9 @@ export default function NewAgentPage() {
|
|||
className={cn("size-full", showSaveHint ? "pt-4" : "pt-10")}
|
||||
threadId={threadId}
|
||||
thread={thread}
|
||||
onSubmitHumanInput={
|
||||
agentName ? handleSubmitHumanInput : undefined
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
|
||||
|
|
@ -394,15 +441,18 @@ export default function NewAgentPage() {
|
|||
</div>
|
||||
) : (
|
||||
<PromptInput
|
||||
disabled={thread.isLoading || hasOpenHumanInputCard}
|
||||
onSubmit={({ text }) => void handleChatSubmit(text)}
|
||||
>
|
||||
<PromptInputTextarea
|
||||
autoFocus
|
||||
placeholder={t.agents.createPageSubtitle}
|
||||
disabled={thread.isLoading}
|
||||
disabled={thread.isLoading || hasOpenHumanInputCard}
|
||||
/>
|
||||
<PromptInputFooter className="justify-end">
|
||||
<PromptInputSubmit disabled={thread.isLoading} />
|
||||
<PromptInputSubmit
|
||||
disabled={thread.isLoading || hasOpenHumanInputCard}
|
||||
/>
|
||||
</PromptInputFooter>
|
||||
</PromptInput>
|
||||
)}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
"use client";
|
||||
|
||||
import { useRouter } from "next/navigation";
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { toast } from "sonner";
|
||||
|
||||
import { type PromptInputMessage } from "@/components/ai-elements/prompt-input";
|
||||
|
|
@ -34,6 +34,13 @@ import { TokenUsageIndicator } from "@/components/workspace/token-usage-indicato
|
|||
import { useActiveGoal } from "@/components/workspace/use-active-goal";
|
||||
import { Welcome } from "@/components/workspace/welcome";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
buildHumanInputResponseText,
|
||||
hasOpenHumanInputRequest,
|
||||
type HumanInputRequest,
|
||||
type HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
import { isHiddenFromUIMessage } from "@/core/messages/utils";
|
||||
import { useModels } from "@/core/models/hooks";
|
||||
import { useNotification } from "@/core/notification/hooks";
|
||||
import { useLocalSettings, useThreadSettings } from "@/core/settings";
|
||||
|
|
@ -170,6 +177,30 @@ export default function ChatPage() {
|
|||
},
|
||||
[sendMessage, threadId],
|
||||
);
|
||||
const handleSubmitHumanInput = useCallback(
|
||||
async (request: HumanInputRequest, response: HumanInputResponse) => {
|
||||
let sent = false;
|
||||
await sendMessage(
|
||||
threadId,
|
||||
{
|
||||
text: buildHumanInputResponseText(request, response),
|
||||
files: [],
|
||||
},
|
||||
undefined,
|
||||
{
|
||||
additionalKwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
onSent: () => {
|
||||
sent = true;
|
||||
},
|
||||
},
|
||||
);
|
||||
return sent;
|
||||
},
|
||||
[sendMessage, threadId],
|
||||
);
|
||||
const handleStop = useCallback(async () => {
|
||||
await thread.stop();
|
||||
}, [thread]);
|
||||
|
|
@ -213,6 +244,14 @@ export default function ChatPage() {
|
|||
threadId,
|
||||
thread.values.goal,
|
||||
);
|
||||
const hasOpenHumanInputCard = useMemo(
|
||||
() =>
|
||||
hasOpenHumanInputRequest(
|
||||
thread.messages,
|
||||
(message) => !isHiddenFromUIMessage(message),
|
||||
),
|
||||
[thread.messages],
|
||||
);
|
||||
|
||||
return (
|
||||
<ThreadContext.Provider value={{ thread, isMock }}>
|
||||
|
|
@ -275,6 +314,11 @@ export default function ChatPage() {
|
|||
!thread.isLoading
|
||||
}
|
||||
onRegenerateMessage={handleRegenerate}
|
||||
onSubmitHumanInput={
|
||||
isMock || env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true"
|
||||
? undefined
|
||||
: handleSubmitHumanInput
|
||||
}
|
||||
canBranch={
|
||||
!isNewThread &&
|
||||
!isMock &&
|
||||
|
|
@ -351,7 +395,9 @@ export default function ChatPage() {
|
|||
disabled={
|
||||
isMock ||
|
||||
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true" ||
|
||||
isUploading
|
||||
isUploading ||
|
||||
hasOpenHumanInputCard ||
|
||||
(!isNewThread && isHistoryLoading)
|
||||
}
|
||||
onContextChange={(context) =>
|
||||
setSettings("context", context)
|
||||
|
|
|
|||
244
frontend/src/components/workspace/messages/human-input-card.tsx
Normal file
244
frontend/src/components/workspace/messages/human-input-card.tsx
Normal file
|
|
@ -0,0 +1,244 @@
|
|||
"use client";
|
||||
|
||||
import {
|
||||
CheckCircle2Icon,
|
||||
Loader2Icon,
|
||||
MessageCircleQuestionMarkIcon,
|
||||
} from "lucide-react";
|
||||
import { useId, useState, type KeyboardEvent } from "react";
|
||||
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Textarea } from "@/components/ui/textarea";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
createHumanInputOptionResponse,
|
||||
createHumanInputTextResponse,
|
||||
type HumanInputOption,
|
||||
type HumanInputRequest,
|
||||
type HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
import { isIMEComposing } from "@/lib/ime";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
import { MarkdownContent } from "./markdown-content";
|
||||
|
||||
export type HumanInputSubmitResult = boolean | void;
|
||||
|
||||
export function shouldSubmitHumanInputTextOnKeyDown(
|
||||
event: KeyboardEvent<HTMLTextAreaElement>,
|
||||
isComposing = false,
|
||||
) {
|
||||
return (
|
||||
event.key === "Enter" &&
|
||||
!event.shiftKey &&
|
||||
!isIMEComposing(event, isComposing)
|
||||
);
|
||||
}
|
||||
|
||||
export function HumanInputCard({
|
||||
request,
|
||||
disabled = false,
|
||||
pending = false,
|
||||
answeredResponse = null,
|
||||
onSubmit,
|
||||
}: {
|
||||
request: HumanInputRequest;
|
||||
disabled?: boolean;
|
||||
pending?: boolean;
|
||||
answeredResponse?: HumanInputResponse | null;
|
||||
onSubmit?: (
|
||||
response: HumanInputResponse,
|
||||
) => HumanInputSubmitResult | Promise<HumanInputSubmitResult>;
|
||||
}) {
|
||||
const { t } = useI18n();
|
||||
const [text, setText] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
const [isComposing, setIsComposing] = useState(false);
|
||||
const titleId = useId();
|
||||
const textInputId = useId();
|
||||
const allowText =
|
||||
request.input_mode === "free_text" ||
|
||||
request.input_mode === "choice_with_other";
|
||||
const options = request.options ?? [];
|
||||
const readOnly = !onSubmit;
|
||||
const isDisabled =
|
||||
disabled || pending || Boolean(answeredResponse) || readOnly;
|
||||
const statusLabel = answeredResponse
|
||||
? t.humanInput.answered
|
||||
: pending
|
||||
? t.humanInput.pending
|
||||
: readOnly
|
||||
? t.humanInput.readOnly
|
||||
: null;
|
||||
|
||||
const submitResponse = async (response: HumanInputResponse) => {
|
||||
if (isDisabled || !onSubmit) {
|
||||
return;
|
||||
}
|
||||
setError("");
|
||||
const result = await onSubmit(response);
|
||||
if (result !== false && response.response_kind === "text") {
|
||||
setText("");
|
||||
}
|
||||
};
|
||||
|
||||
const handleOptionClick = (option: HumanInputOption) => {
|
||||
void submitResponse(createHumanInputOptionResponse(request, option));
|
||||
};
|
||||
|
||||
const handleTextSubmit = (event: { preventDefault(): void }) => {
|
||||
event.preventDefault();
|
||||
const value = text.trim();
|
||||
if (!value) {
|
||||
setError(t.humanInput.emptyError);
|
||||
return;
|
||||
}
|
||||
void submitResponse(createHumanInputTextResponse(request, value));
|
||||
};
|
||||
|
||||
const handleTextKeyDown = (event: KeyboardEvent<HTMLTextAreaElement>) => {
|
||||
if (shouldSubmitHumanInputTextOnKeyDown(event, isComposing)) {
|
||||
event.preventDefault();
|
||||
const value = text.trim();
|
||||
if (!value) {
|
||||
setError(t.humanInput.emptyError);
|
||||
return;
|
||||
}
|
||||
void submitResponse(createHumanInputTextResponse(request, value));
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<section
|
||||
aria-labelledby={titleId}
|
||||
className="border-border bg-card/70 text-card-foreground rounded-lg border p-4 shadow-xs"
|
||||
data-testid="human-input-card"
|
||||
>
|
||||
<div className="flex items-start gap-3">
|
||||
<div className="bg-primary/10 text-primary mt-0.5 flex size-8 shrink-0 items-center justify-center rounded-md">
|
||||
<MessageCircleQuestionMarkIcon className="size-4" />
|
||||
</div>
|
||||
<div className="min-w-0 flex-1 space-y-3">
|
||||
<div className="flex flex-wrap items-start justify-between gap-2">
|
||||
<div className="min-w-0 space-y-1">
|
||||
<h2 id={titleId} className="text-sm leading-5 font-medium">
|
||||
{request.title ?? t.toolCalls.needYourHelp}
|
||||
</h2>
|
||||
{request.context ? (
|
||||
<div className="text-muted-foreground text-sm leading-6">
|
||||
<MarkdownContent
|
||||
content={request.context}
|
||||
isLoading={false}
|
||||
/>
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
{statusLabel ? (
|
||||
<Badge
|
||||
className={cn(
|
||||
"h-6 rounded-md px-2",
|
||||
pending && "gap-1.5",
|
||||
answeredResponse &&
|
||||
"border-primary/20 bg-primary/10 text-primary",
|
||||
)}
|
||||
variant={answeredResponse ? "outline" : "secondary"}
|
||||
>
|
||||
{pending ? (
|
||||
<Loader2Icon className="size-3 animate-spin" />
|
||||
) : null}
|
||||
{answeredResponse ? (
|
||||
<CheckCircle2Icon className="size-3" />
|
||||
) : null}
|
||||
{statusLabel}
|
||||
</Badge>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<div className="text-foreground text-sm leading-6">
|
||||
<MarkdownContent content={request.question} isLoading={false} />
|
||||
</div>
|
||||
|
||||
{options.length > 0 ? (
|
||||
<div className="grid gap-2">
|
||||
{options.map((option) => (
|
||||
<Button
|
||||
key={option.id}
|
||||
className="min-h-11 w-full justify-start rounded-md px-3 py-2 text-left leading-5 whitespace-normal"
|
||||
disabled={isDisabled}
|
||||
type="button"
|
||||
variant="outline"
|
||||
onClick={() => handleOptionClick(option)}
|
||||
>
|
||||
<span className="min-w-0 wrap-break-word whitespace-pre-wrap">
|
||||
{option.label}
|
||||
</span>
|
||||
</Button>
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
{allowText ? (
|
||||
<form className="space-y-2" onSubmit={handleTextSubmit}>
|
||||
<label className="sr-only" htmlFor={textInputId}>
|
||||
{t.humanInput.otherLabel}
|
||||
</label>
|
||||
<Textarea
|
||||
id={textInputId}
|
||||
aria-invalid={Boolean(error)}
|
||||
aria-describedby={error ? `${textInputId}-error` : undefined}
|
||||
className="min-h-20 resize-y text-sm"
|
||||
disabled={isDisabled}
|
||||
placeholder={t.humanInput.otherPlaceholder}
|
||||
value={text}
|
||||
onChange={(event) => {
|
||||
setText(event.target.value);
|
||||
if (error) {
|
||||
setError("");
|
||||
}
|
||||
}}
|
||||
onCompositionEnd={() => setIsComposing(false)}
|
||||
onCompositionStart={() => setIsComposing(true)}
|
||||
onKeyDown={handleTextKeyDown}
|
||||
/>
|
||||
<div className="flex min-h-9 flex-wrap items-center justify-between gap-2">
|
||||
{error ? (
|
||||
<p
|
||||
className="text-destructive text-sm"
|
||||
id={`${textInputId}-error`}
|
||||
>
|
||||
{error}
|
||||
</p>
|
||||
) : answeredResponse ? (
|
||||
<p
|
||||
className="text-muted-foreground text-sm"
|
||||
aria-live="polite"
|
||||
>
|
||||
{t.humanInput.answeredValue(answeredResponse.value)}
|
||||
</p>
|
||||
) : (
|
||||
<span />
|
||||
)}
|
||||
<Button
|
||||
className="min-w-24"
|
||||
disabled={isDisabled}
|
||||
type="submit"
|
||||
variant="secondary"
|
||||
>
|
||||
{pending ? (
|
||||
<Loader2Icon className="size-4 animate-spin" />
|
||||
) : null}
|
||||
{t.humanInput.submit}
|
||||
</Button>
|
||||
</div>
|
||||
</form>
|
||||
) : answeredResponse ? (
|
||||
<p className="text-muted-foreground text-sm" aria-live="polite">
|
||||
{t.humanInput.answeredValue(answeredResponse.value)}
|
||||
</p>
|
||||
) : null}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
|
@ -29,6 +29,13 @@ import {
|
|||
} from "@/components/ai-elements/reasoning";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
deriveHumanInputThreadState,
|
||||
extractHumanInputRequest,
|
||||
shouldClearPendingHumanInputOnThreadError,
|
||||
type HumanInputRequest,
|
||||
type HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
import {
|
||||
buildTokenDebugSteps,
|
||||
type TokenUsageInlineMode,
|
||||
|
|
@ -45,6 +52,7 @@ import {
|
|||
hasPresentFiles,
|
||||
hasReasoning,
|
||||
isAssistantMessageGroupStreaming,
|
||||
isHiddenFromUIMessage,
|
||||
} from "@/core/messages/utils";
|
||||
import { useRehypeSplitWordsIntoSpans } from "@/core/rehype";
|
||||
import {
|
||||
|
|
@ -65,6 +73,10 @@ import { CopyButton } from "../copy-button";
|
|||
import { useMaybeSidecar } from "../sidecar/context";
|
||||
import { Tooltip } from "../tooltip";
|
||||
|
||||
import {
|
||||
HumanInputCard,
|
||||
type HumanInputSubmitResult,
|
||||
} from "./human-input-card";
|
||||
import { MarkdownContent } from "./markdown-content";
|
||||
import { MessageGroup } from "./message-group";
|
||||
import { MessageListItem } from "./message-list-item";
|
||||
|
|
@ -208,6 +220,7 @@ export function MessageList({
|
|||
loadMoreHistory,
|
||||
isHistoryLoading,
|
||||
onRegenerateMessage,
|
||||
onSubmitHumanInput,
|
||||
onBranchTurn,
|
||||
canRegenerate = false,
|
||||
canBranch = false,
|
||||
|
|
@ -229,6 +242,10 @@ export function MessageList({
|
|||
messageId: string,
|
||||
supersededMessageIds: string[],
|
||||
) => void | Promise<void>;
|
||||
onSubmitHumanInput?: (
|
||||
request: HumanInputRequest,
|
||||
response: HumanInputResponse,
|
||||
) => HumanInputSubmitResult | Promise<HumanInputSubmitResult>;
|
||||
onBranchTurn?: (
|
||||
messageId: string,
|
||||
messageIds: string[],
|
||||
|
|
@ -258,6 +275,9 @@ export function MessageList({
|
|||
const [regeneratingMessageId, setRegeneratingMessageId] = useState<
|
||||
string | null
|
||||
>(null);
|
||||
const [pendingHumanInputRequestIds, setPendingHumanInputRequestIds] =
|
||||
useState<Set<string>>(() => new Set());
|
||||
const previousHumanInputThreadError = useRef<unknown>(thread.error);
|
||||
const [branchingMessageId, setBranchingMessageId] = useState<string | null>(
|
||||
null,
|
||||
);
|
||||
|
|
@ -293,6 +313,84 @@ export function MessageList({
|
|||
[messages, thread.getMessagesMetadata, thread.isLoading],
|
||||
);
|
||||
|
||||
const humanInputState = useMemo(
|
||||
() =>
|
||||
deriveHumanInputThreadState(
|
||||
messages,
|
||||
(message) => !isHiddenFromUIMessage(message),
|
||||
),
|
||||
[messages],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (pendingHumanInputRequestIds.size === 0) {
|
||||
return;
|
||||
}
|
||||
setPendingHumanInputRequestIds((previous) => {
|
||||
const next = new Set(previous);
|
||||
for (const requestId of previous) {
|
||||
if (humanInputState.answeredResponses.has(requestId)) {
|
||||
next.delete(requestId);
|
||||
}
|
||||
}
|
||||
return next.size === previous.size ? previous : next;
|
||||
});
|
||||
}, [humanInputState.answeredResponses, pendingHumanInputRequestIds.size]);
|
||||
|
||||
useEffect(() => {
|
||||
const previousError = previousHumanInputThreadError.current;
|
||||
previousHumanInputThreadError.current = thread.error;
|
||||
|
||||
if (
|
||||
!shouldClearPendingHumanInputOnThreadError({
|
||||
currentError: thread.error,
|
||||
pendingRequestCount: pendingHumanInputRequestIds.size,
|
||||
previousError,
|
||||
})
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
// `sendMessage` can return after dispatching while the SDK stream later
|
||||
// reports an async error through `thread.error`. In that case the hidden
|
||||
// human reply never reaches history, so unlock the card for retry.
|
||||
setPendingHumanInputRequestIds(new Set());
|
||||
}, [pendingHumanInputRequestIds.size, thread.error]);
|
||||
|
||||
const clearPendingHumanInput = useCallback((requestId: string) => {
|
||||
setPendingHumanInputRequestIds((previous) => {
|
||||
if (!previous.has(requestId)) {
|
||||
return previous;
|
||||
}
|
||||
const next = new Set(previous);
|
||||
next.delete(requestId);
|
||||
return next;
|
||||
});
|
||||
}, []);
|
||||
|
||||
const handleSubmitHumanInput = useCallback(
|
||||
async (request: HumanInputRequest, response: HumanInputResponse) => {
|
||||
setPendingHumanInputRequestIds((previous) => {
|
||||
const next = new Set(previous);
|
||||
next.add(request.request_id);
|
||||
return next;
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await onSubmitHumanInput?.(request, response);
|
||||
if (result === false) {
|
||||
clearPendingHumanInput(request.request_id);
|
||||
}
|
||||
return result;
|
||||
} catch (error) {
|
||||
clearPendingHumanInput(request.request_id);
|
||||
toast.error(error instanceof Error ? error.message : String(error));
|
||||
return false;
|
||||
}
|
||||
},
|
||||
[clearPendingHumanInput, onSubmitHumanInput],
|
||||
);
|
||||
|
||||
const latestAssistantGroupId = useMemo(() => {
|
||||
if (thread.isLoading) {
|
||||
return null;
|
||||
|
|
@ -686,7 +784,52 @@ export function MessageList({
|
|||
);
|
||||
} else if (group.type === "assistant:clarification") {
|
||||
const message = group.messages[0];
|
||||
if (message && hasContent(message)) {
|
||||
if (!message) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const humanInputRequest = extractHumanInputRequest(message);
|
||||
if (humanInputRequest) {
|
||||
const answeredResponse =
|
||||
humanInputState.answeredResponses.get(
|
||||
humanInputRequest.request_id,
|
||||
) ?? null;
|
||||
const pending = pendingHumanInputRequestIds.has(
|
||||
humanInputRequest.request_id,
|
||||
);
|
||||
return (
|
||||
<div key={group.id} className="w-full">
|
||||
<HumanInputCard
|
||||
answeredResponse={answeredResponse}
|
||||
disabled={
|
||||
thread.isLoading ||
|
||||
pending ||
|
||||
Boolean(answeredResponse) ||
|
||||
humanInputState.latestOpenRequestId !==
|
||||
humanInputRequest.request_id ||
|
||||
!onSubmitHumanInput
|
||||
}
|
||||
pending={pending}
|
||||
request={humanInputRequest}
|
||||
onSubmit={
|
||||
onSubmitHumanInput
|
||||
? (response) =>
|
||||
handleSubmitHumanInput(
|
||||
humanInputRequest,
|
||||
response,
|
||||
)
|
||||
: undefined
|
||||
}
|
||||
/>
|
||||
{renderTokenUsage({
|
||||
messages: group.messages,
|
||||
turnUsageMessages,
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
if (hasContent(message)) {
|
||||
return (
|
||||
<div key={group.id} className="w-full">
|
||||
<MarkdownContent
|
||||
|
|
|
|||
|
|
@ -49,6 +49,13 @@ import {
|
|||
DropdownMenuLabel,
|
||||
} from "@/components/ui/dropdown-menu";
|
||||
import { useI18n } from "@/core/i18n/hooks";
|
||||
import {
|
||||
buildHumanInputResponseText,
|
||||
hasOpenHumanInputRequest,
|
||||
type HumanInputRequest,
|
||||
type HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
import { isHiddenFromUIMessage } from "@/core/messages/utils";
|
||||
import { useModels } from "@/core/models/hooks";
|
||||
import type { Model } from "@/core/models/types";
|
||||
import { useLocalSettings } from "@/core/settings";
|
||||
|
|
@ -202,6 +209,14 @@ export function SidecarPanel({ className }: { className?: string }) {
|
|||
|
||||
const hasPendingReferences = sidecar.activeReferences.length > 0;
|
||||
const hasSidecarThread = Boolean(sidecar.sidecarThreadId);
|
||||
const hasOpenHumanInputCard = useMemo(
|
||||
() =>
|
||||
hasOpenHumanInputRequest(
|
||||
thread.messages,
|
||||
(message) => !isHiddenFromUIMessage(message),
|
||||
),
|
||||
[thread.messages],
|
||||
);
|
||||
const tokenUsageInlineMode = tokenUsageEnabled
|
||||
? localSettings.tokenUsage.inlineMode
|
||||
: "off";
|
||||
|
|
@ -211,6 +226,8 @@ export function SidecarPanel({ className }: { className?: string }) {
|
|||
creatingThread ||
|
||||
Boolean(queuedSubmit) ||
|
||||
isUploading ||
|
||||
hasOpenHumanInputCard ||
|
||||
(hasSidecarThread && isHistoryLoading) ||
|
||||
(sidecar.isMock ?? false) ||
|
||||
env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true";
|
||||
|
||||
|
|
@ -340,6 +357,7 @@ export function SidecarPanel({ className }: { className?: string }) {
|
|||
message: PromptInputMessage,
|
||||
references: SidecarReference[],
|
||||
onSent?: () => void,
|
||||
additionalKwargs?: Record<string, unknown>,
|
||||
) => {
|
||||
const contexts = references.map((reference) => reference.context);
|
||||
const parentConversation = buildParentConversationContext(
|
||||
|
|
@ -363,6 +381,7 @@ export function SidecarPanel({ className }: { className?: string }) {
|
|||
...(contexts.length > 0
|
||||
? buildReferenceMessageMetadata(contexts)
|
||||
: {}),
|
||||
...additionalKwargs,
|
||||
},
|
||||
onSent,
|
||||
});
|
||||
|
|
@ -370,6 +389,37 @@ export function SidecarPanel({ className }: { className?: string }) {
|
|||
[parentThread.messages, sendMessage, sidecar.parentThreadId],
|
||||
);
|
||||
|
||||
const handleSubmitHumanInput = useCallback(
|
||||
async (request: HumanInputRequest, response: HumanInputResponse) => {
|
||||
if (!sidecar.sidecarThreadId) {
|
||||
return false;
|
||||
}
|
||||
|
||||
let sent = false;
|
||||
const pendingReferences = [...sidecar.activeReferences];
|
||||
await submitToSidecarThread(
|
||||
sidecar.sidecarThreadId,
|
||||
{
|
||||
text: buildHumanInputResponseText(request, response),
|
||||
files: [],
|
||||
},
|
||||
pendingReferences,
|
||||
() => {
|
||||
sent = true;
|
||||
if (pendingReferences.length > 0) {
|
||||
sidecar.clearActiveReferences();
|
||||
}
|
||||
},
|
||||
{
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
);
|
||||
return sent;
|
||||
},
|
||||
[sidecar, submitToSidecarThread],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!queuedSubmit || !sidecar.sidecarThreadId || thread.isLoading) {
|
||||
return;
|
||||
|
|
@ -546,6 +596,11 @@ export function SidecarPanel({ className }: { className?: string }) {
|
|||
sidecarSurface
|
||||
initialScroll="instant"
|
||||
resizeScroll="instant"
|
||||
onSubmitHumanInput={
|
||||
sidecar.isMock || env.NEXT_PUBLIC_STATIC_WEBSITE_ONLY === "true"
|
||||
? undefined
|
||||
: handleSubmitHumanInput
|
||||
}
|
||||
/>
|
||||
) : (
|
||||
<ConversationEmptyState
|
||||
|
|
|
|||
|
|
@ -538,6 +538,17 @@ export const enUS: Translations = {
|
|||
skillInstallTooltip: "Install skill and make it available to DeerFlow",
|
||||
},
|
||||
|
||||
humanInput: {
|
||||
answered: "Answered",
|
||||
pending: "Sending...",
|
||||
readOnly: "Read only",
|
||||
otherLabel: "Other answer",
|
||||
otherPlaceholder: "Type another answer...",
|
||||
submit: "Submit",
|
||||
emptyError: "Enter an answer before submitting.",
|
||||
answeredValue: (value: string) => `Answered: ${value}`,
|
||||
},
|
||||
|
||||
// Subtasks
|
||||
uploads: {
|
||||
uploading: "Uploading...",
|
||||
|
|
|
|||
|
|
@ -436,6 +436,17 @@ export interface Translations {
|
|||
skillInstallTooltip: string;
|
||||
};
|
||||
|
||||
humanInput: {
|
||||
answered: string;
|
||||
pending: string;
|
||||
readOnly: string;
|
||||
otherLabel: string;
|
||||
otherPlaceholder: string;
|
||||
submit: string;
|
||||
emptyError: string;
|
||||
answeredValue: (value: string) => string;
|
||||
};
|
||||
|
||||
// Uploads
|
||||
uploads: {
|
||||
uploading: string;
|
||||
|
|
|
|||
|
|
@ -521,6 +521,17 @@ export const zhCN: Translations = {
|
|||
skillInstallTooltip: "安装技能并使其可在 DeerFlow 中使用",
|
||||
},
|
||||
|
||||
humanInput: {
|
||||
answered: "已回答",
|
||||
pending: "发送中...",
|
||||
readOnly: "只读",
|
||||
otherLabel: "其他回答",
|
||||
otherPlaceholder: "输入其他回答...",
|
||||
submit: "提交",
|
||||
emptyError: "请输入回答后再提交。",
|
||||
answeredValue: (value: string) => `已回答:${value}`,
|
||||
},
|
||||
|
||||
uploads: {
|
||||
uploading: "上传中...",
|
||||
uploadingFiles: "文件上传中,请稍候...",
|
||||
|
|
|
|||
326
frontend/src/core/messages/human-input.ts
Normal file
326
frontend/src/core/messages/human-input.ts
Normal file
|
|
@ -0,0 +1,326 @@
|
|||
import type { Message } from "@langchain/langgraph-sdk";
|
||||
|
||||
export type HumanInputMode =
|
||||
| "free_text"
|
||||
| "single_choice"
|
||||
| "choice_with_other";
|
||||
|
||||
export type HumanInputOption = {
|
||||
id: string;
|
||||
label: string;
|
||||
value: string;
|
||||
};
|
||||
|
||||
export type HumanInputRequest = {
|
||||
version: 1;
|
||||
kind: "human_input_request";
|
||||
source: "ask_clarification" | string;
|
||||
request_id: string;
|
||||
tool_call_id?: string;
|
||||
clarification_type?: string;
|
||||
title?: string;
|
||||
question: string;
|
||||
context?: string | null;
|
||||
input_mode: HumanInputMode;
|
||||
options?: HumanInputOption[];
|
||||
};
|
||||
|
||||
export type HumanInputResponse =
|
||||
| {
|
||||
version: 1;
|
||||
kind: "human_input_response";
|
||||
source: string;
|
||||
request_id: string;
|
||||
response_kind: "option";
|
||||
option_id: string;
|
||||
value: string;
|
||||
}
|
||||
| {
|
||||
version: 1;
|
||||
kind: "human_input_response";
|
||||
source: string;
|
||||
request_id: string;
|
||||
response_kind: "text";
|
||||
value: string;
|
||||
};
|
||||
|
||||
export type HumanInputThreadState = {
|
||||
answeredResponses: Map<string, HumanInputResponse>;
|
||||
latestOpenRequestId: string | null;
|
||||
};
|
||||
|
||||
export function shouldClearPendingHumanInputOnThreadError({
|
||||
currentError,
|
||||
pendingRequestCount,
|
||||
previousError,
|
||||
}: {
|
||||
currentError: unknown;
|
||||
pendingRequestCount: number;
|
||||
previousError: unknown;
|
||||
}) {
|
||||
return (
|
||||
pendingRequestCount > 0 &&
|
||||
currentError != null &&
|
||||
!Object.is(currentError, previousError)
|
||||
);
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null;
|
||||
}
|
||||
|
||||
function isNonEmptyString(value: unknown): value is string {
|
||||
return typeof value === "string" && value.trim().length > 0;
|
||||
}
|
||||
|
||||
function isHumanInputMode(value: unknown): value is HumanInputMode {
|
||||
return (
|
||||
value === "free_text" ||
|
||||
value === "single_choice" ||
|
||||
value === "choice_with_other"
|
||||
);
|
||||
}
|
||||
|
||||
function readOptionalString(value: unknown) {
|
||||
return typeof value === "string" ? value : undefined;
|
||||
}
|
||||
|
||||
function parseOptions(value: unknown): HumanInputOption[] | undefined {
|
||||
if (value === undefined) {
|
||||
return undefined;
|
||||
}
|
||||
if (!Array.isArray(value)) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const options: HumanInputOption[] = [];
|
||||
for (const option of value) {
|
||||
if (!isRecord(option)) {
|
||||
return undefined;
|
||||
}
|
||||
const id = option.id;
|
||||
const label = option.label;
|
||||
const optionValue = option.value;
|
||||
if (
|
||||
!isNonEmptyString(id) ||
|
||||
!isNonEmptyString(label) ||
|
||||
typeof optionValue !== "string"
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
options.push({ id, label, value: optionValue });
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
export function parseHumanInputRequest(
|
||||
value: unknown,
|
||||
): HumanInputRequest | null {
|
||||
if (!isRecord(value)) {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
value.version !== 1 ||
|
||||
value.kind !== "human_input_request" ||
|
||||
!isNonEmptyString(value.source) ||
|
||||
!isNonEmptyString(value.request_id) ||
|
||||
!isNonEmptyString(value.question) ||
|
||||
!isHumanInputMode(value.input_mode)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const options = parseOptions(value.options);
|
||||
if (value.options !== undefined && options === undefined) {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
(value.input_mode === "single_choice" ||
|
||||
value.input_mode === "choice_with_other") &&
|
||||
(!options || options.length === 0)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const context = value.context;
|
||||
if (
|
||||
context !== undefined &&
|
||||
context !== null &&
|
||||
typeof context !== "string"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
version: 1,
|
||||
kind: "human_input_request",
|
||||
source: value.source,
|
||||
request_id: value.request_id,
|
||||
...(readOptionalString(value.tool_call_id)
|
||||
? { tool_call_id: readOptionalString(value.tool_call_id) }
|
||||
: {}),
|
||||
...(readOptionalString(value.clarification_type)
|
||||
? { clarification_type: readOptionalString(value.clarification_type) }
|
||||
: {}),
|
||||
...(readOptionalString(value.title)
|
||||
? { title: readOptionalString(value.title) }
|
||||
: {}),
|
||||
question: value.question,
|
||||
...(context !== undefined ? { context } : {}),
|
||||
input_mode: value.input_mode,
|
||||
...(options ? { options } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
export function parseHumanInputResponse(
|
||||
value: unknown,
|
||||
): HumanInputResponse | null {
|
||||
if (!isRecord(value)) {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
value.version !== 1 ||
|
||||
value.kind !== "human_input_response" ||
|
||||
!isNonEmptyString(value.source) ||
|
||||
!isNonEmptyString(value.request_id) ||
|
||||
!isNonEmptyString(value.value)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (value.response_kind === "option") {
|
||||
if (!isNonEmptyString(value.option_id)) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: value.source,
|
||||
request_id: value.request_id,
|
||||
response_kind: "option",
|
||||
option_id: value.option_id,
|
||||
value: value.value,
|
||||
};
|
||||
}
|
||||
|
||||
if (value.response_kind === "text") {
|
||||
return {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: value.source,
|
||||
request_id: value.request_id,
|
||||
response_kind: "text",
|
||||
value: value.value,
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
export function extractHumanInputRequest(
|
||||
message: Message,
|
||||
): HumanInputRequest | null {
|
||||
if (message.type !== "tool") {
|
||||
return null;
|
||||
}
|
||||
const artifact = Reflect.get(message, "artifact");
|
||||
if (!isRecord(artifact)) {
|
||||
return null;
|
||||
}
|
||||
return parseHumanInputRequest(artifact.human_input);
|
||||
}
|
||||
|
||||
export function extractHumanInputResponse(
|
||||
message: Message,
|
||||
): HumanInputResponse | null {
|
||||
if (message.type !== "human") {
|
||||
return null;
|
||||
}
|
||||
const additionalKwargs = message.additional_kwargs;
|
||||
if (!isRecord(additionalKwargs)) {
|
||||
return null;
|
||||
}
|
||||
return parseHumanInputResponse(additionalKwargs.human_input_response);
|
||||
}
|
||||
|
||||
export function deriveHumanInputThreadState(
|
||||
messages: Message[],
|
||||
isVisibleMessage: (message: Message) => boolean = (message) =>
|
||||
message.additional_kwargs?.hide_from_ui !== true,
|
||||
): HumanInputThreadState {
|
||||
const answeredResponses = new Map<string, HumanInputResponse>();
|
||||
const seenRequestIds = new Set<string>();
|
||||
const requestOrder: string[] = [];
|
||||
|
||||
for (const message of messages) {
|
||||
if (isVisibleMessage(message)) {
|
||||
const request = extractHumanInputRequest(message);
|
||||
if (request) {
|
||||
seenRequestIds.add(request.request_id);
|
||||
requestOrder.push(request.request_id);
|
||||
}
|
||||
}
|
||||
|
||||
const response = extractHumanInputResponse(message);
|
||||
if (
|
||||
response &&
|
||||
seenRequestIds.has(response.request_id) &&
|
||||
!answeredResponses.has(response.request_id)
|
||||
) {
|
||||
answeredResponses.set(response.request_id, response);
|
||||
}
|
||||
}
|
||||
|
||||
const latestOpenRequestId =
|
||||
[...requestOrder]
|
||||
.reverse()
|
||||
.find((requestId) => !answeredResponses.has(requestId)) ?? null;
|
||||
|
||||
return { answeredResponses, latestOpenRequestId };
|
||||
}
|
||||
|
||||
export function hasOpenHumanInputRequest(
|
||||
messages: Message[],
|
||||
isVisibleMessage?: (message: Message) => boolean,
|
||||
) {
|
||||
return (
|
||||
deriveHumanInputThreadState(messages, isVisibleMessage)
|
||||
.latestOpenRequestId !== null
|
||||
);
|
||||
}
|
||||
|
||||
export function createHumanInputOptionResponse(
|
||||
request: HumanInputRequest,
|
||||
option: HumanInputOption,
|
||||
): HumanInputResponse {
|
||||
return {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: request.source,
|
||||
request_id: request.request_id,
|
||||
response_kind: "option",
|
||||
option_id: option.id,
|
||||
value: option.value,
|
||||
};
|
||||
}
|
||||
|
||||
export function createHumanInputTextResponse(
|
||||
request: HumanInputRequest,
|
||||
value: string,
|
||||
): HumanInputResponse {
|
||||
return {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: request.source,
|
||||
request_id: request.request_id,
|
||||
response_kind: "text",
|
||||
value,
|
||||
};
|
||||
}
|
||||
|
||||
export function buildHumanInputResponseText(
|
||||
request: HumanInputRequest,
|
||||
response: HumanInputResponse,
|
||||
) {
|
||||
return `For your clarification "${request.question}", my answer is: ${response.value}`;
|
||||
}
|
||||
|
|
@ -0,0 +1,136 @@
|
|||
import { describe, expect, it } from "@rstest/core";
|
||||
import { createElement, type KeyboardEvent } from "react";
|
||||
import { renderToStaticMarkup } from "react-dom/server";
|
||||
|
||||
import {
|
||||
HumanInputCard,
|
||||
shouldSubmitHumanInputTextOnKeyDown,
|
||||
} from "@/components/workspace/messages/human-input-card";
|
||||
import { I18nContext } from "@/core/i18n/context";
|
||||
import type {
|
||||
HumanInputRequest,
|
||||
HumanInputResponse,
|
||||
} from "@/core/messages/human-input";
|
||||
|
||||
const request: HumanInputRequest = {
|
||||
version: 1,
|
||||
kind: "human_input_request",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
tool_call_id: "call-abc",
|
||||
clarification_type: "approach_choice",
|
||||
question: "Which environment should I deploy to?",
|
||||
context: "Need the target environment.",
|
||||
input_mode: "choice_with_other",
|
||||
options: [
|
||||
{ id: "option-1", label: "development", value: "development" },
|
||||
{ id: "option-2", label: "staging", value: "staging" },
|
||||
],
|
||||
};
|
||||
|
||||
describe("HumanInputCard", () => {
|
||||
it("renders request text, options, and the other-answer input", () => {
|
||||
const html = renderCard();
|
||||
|
||||
expect(html).toContain("Need your help");
|
||||
expect(html).toContain("Need the target environment.");
|
||||
expect(html).toContain("Which environment should I deploy to?");
|
||||
expect(html).toContain("development");
|
||||
expect(html).toContain("staging");
|
||||
expect(html).toContain("Other answer");
|
||||
expect(html).toContain("Type another answer...");
|
||||
});
|
||||
|
||||
it("renders answered state as disabled with the selected value", () => {
|
||||
const response: HumanInputResponse = {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "option",
|
||||
option_id: "option-2",
|
||||
value: "staging",
|
||||
};
|
||||
const html = renderCard({ answeredResponse: response });
|
||||
|
||||
expect(html).toContain("Answered");
|
||||
expect(html).toContain("Answered: staging");
|
||||
expect(html).toContain("disabled");
|
||||
});
|
||||
|
||||
it("renders read-only state when no submit handler is available", () => {
|
||||
const html = renderCard({ onSubmit: undefined });
|
||||
|
||||
expect(html).toContain("Read only");
|
||||
expect(html).toContain("disabled");
|
||||
});
|
||||
|
||||
it("renders markdown in question field (bold, lists)", () => {
|
||||
const html = renderCard({
|
||||
request: {
|
||||
...request,
|
||||
question:
|
||||
"你想写什么样的小说?\n\n1. **题材/类型**:科幻、奇幻\n2. **篇幅**:短篇、中篇",
|
||||
input_mode: "free_text",
|
||||
options: undefined,
|
||||
},
|
||||
});
|
||||
|
||||
expect(html).toContain("题材/类型");
|
||||
expect(html).toContain("篇幅");
|
||||
expect(html).not.toContain("**题材/类型**");
|
||||
expect(html).not.toContain("**篇幅**");
|
||||
});
|
||||
|
||||
it("does not submit text with Enter while IME composition is active", () => {
|
||||
expect(shouldSubmitHumanInputTextOnKeyDown(keyEvent())).toBe(true);
|
||||
expect(
|
||||
shouldSubmitHumanInputTextOnKeyDown(keyEvent({ shiftKey: true })),
|
||||
).toBe(false);
|
||||
expect(
|
||||
shouldSubmitHumanInputTextOnKeyDown(keyEvent({ isComposing: true })),
|
||||
).toBe(false);
|
||||
expect(
|
||||
shouldSubmitHumanInputTextOnKeyDown(keyEvent({ keyCode: 229 })),
|
||||
).toBe(false);
|
||||
expect(shouldSubmitHumanInputTextOnKeyDown(keyEvent(), true)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
function renderCard(props: Partial<Parameters<typeof HumanInputCard>[0]> = {}) {
|
||||
return renderToStaticMarkup(
|
||||
createElement(
|
||||
I18nContext.Provider,
|
||||
{
|
||||
value: {
|
||||
locale: "en-US",
|
||||
setLocale: () => undefined,
|
||||
},
|
||||
},
|
||||
createElement(HumanInputCard, {
|
||||
request,
|
||||
onSubmit: () => undefined,
|
||||
...props,
|
||||
}),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
function keyEvent({
|
||||
isComposing = false,
|
||||
key = "Enter",
|
||||
keyCode = 13,
|
||||
shiftKey = false,
|
||||
}: {
|
||||
isComposing?: boolean;
|
||||
key?: string;
|
||||
keyCode?: number;
|
||||
shiftKey?: boolean;
|
||||
} = {}) {
|
||||
return {
|
||||
key,
|
||||
keyCode,
|
||||
nativeEvent: { isComposing },
|
||||
shiftKey,
|
||||
} as unknown as KeyboardEvent<HTMLTextAreaElement>;
|
||||
}
|
||||
223
frontend/tests/unit/core/messages/human-input.test.ts
Normal file
223
frontend/tests/unit/core/messages/human-input.test.ts
Normal file
|
|
@ -0,0 +1,223 @@
|
|||
import type { Message } from "@langchain/langgraph-sdk";
|
||||
import { expect, test } from "@rstest/core";
|
||||
|
||||
import {
|
||||
buildHumanInputResponseText,
|
||||
createHumanInputOptionResponse,
|
||||
createHumanInputTextResponse,
|
||||
deriveHumanInputThreadState,
|
||||
extractHumanInputRequest,
|
||||
extractHumanInputResponse,
|
||||
hasOpenHumanInputRequest,
|
||||
shouldClearPendingHumanInputOnThreadError,
|
||||
} from "@/core/messages/human-input";
|
||||
|
||||
const requestPayload = {
|
||||
version: 1,
|
||||
kind: "human_input_request",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
tool_call_id: "call-abc",
|
||||
clarification_type: "approach_choice",
|
||||
question: "Which environment should I deploy to?",
|
||||
context: "Need the target environment.",
|
||||
input_mode: "choice_with_other",
|
||||
options: [
|
||||
{ id: "option-1", label: "development", value: "development" },
|
||||
{ id: "option-2", label: "staging", value: "staging" },
|
||||
],
|
||||
};
|
||||
|
||||
test("extractHumanInputRequest reads a valid tool artifact payload", () => {
|
||||
const message = {
|
||||
type: "tool",
|
||||
name: "ask_clarification",
|
||||
content: "fallback",
|
||||
artifact: {
|
||||
human_input: requestPayload,
|
||||
},
|
||||
} as unknown as Message;
|
||||
|
||||
expect(extractHumanInputRequest(message)).toEqual(requestPayload);
|
||||
});
|
||||
|
||||
test("extractHumanInputRequest rejects malformed artifacts", () => {
|
||||
const message = {
|
||||
type: "tool",
|
||||
name: "ask_clarification",
|
||||
content: "fallback",
|
||||
artifact: {
|
||||
human_input: {
|
||||
...requestPayload,
|
||||
options: [{ id: "option-1", label: "missing value" }],
|
||||
},
|
||||
},
|
||||
} as unknown as Message;
|
||||
|
||||
expect(extractHumanInputRequest(message)).toBeNull();
|
||||
});
|
||||
|
||||
test("extractHumanInputResponse reads valid human message metadata", () => {
|
||||
const response = {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "option",
|
||||
option_id: "option-2",
|
||||
value: "staging",
|
||||
};
|
||||
const message = {
|
||||
type: "human",
|
||||
content: "For your clarification, my answer is: staging",
|
||||
additional_kwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
} as unknown as Message;
|
||||
|
||||
expect(extractHumanInputResponse(message)).toEqual(response);
|
||||
});
|
||||
|
||||
test("derives answered card state from hidden human input responses", () => {
|
||||
const response = {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "option",
|
||||
option_id: "option-2",
|
||||
value: "staging",
|
||||
};
|
||||
const state = deriveHumanInputThreadState([
|
||||
{
|
||||
type: "tool",
|
||||
name: "ask_clarification",
|
||||
content: "fallback",
|
||||
artifact: {
|
||||
human_input: requestPayload,
|
||||
},
|
||||
} as unknown as Message,
|
||||
{
|
||||
type: "human",
|
||||
content: "For your clarification, my answer is: staging",
|
||||
additional_kwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
} as unknown as Message,
|
||||
]);
|
||||
|
||||
expect(state.answeredResponses.get("clarification:call-abc")).toEqual(
|
||||
response,
|
||||
);
|
||||
expect(state.latestOpenRequestId).toBeNull();
|
||||
});
|
||||
|
||||
test("detects whether a thread has an open human input request", () => {
|
||||
const requestMessage = {
|
||||
type: "tool",
|
||||
name: "ask_clarification",
|
||||
content: "fallback",
|
||||
artifact: {
|
||||
human_input: requestPayload,
|
||||
},
|
||||
} as unknown as Message;
|
||||
const responseMessage = {
|
||||
type: "human",
|
||||
content: "For your clarification, my answer is: staging",
|
||||
additional_kwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "option",
|
||||
option_id: "option-2",
|
||||
value: "staging",
|
||||
},
|
||||
},
|
||||
} as unknown as Message;
|
||||
|
||||
expect(hasOpenHumanInputRequest([requestMessage])).toBe(true);
|
||||
expect(hasOpenHumanInputRequest([requestMessage, responseMessage])).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
test("detects new thread errors that should unlock pending human input cards", () => {
|
||||
const previousError = new Error("old failure");
|
||||
const currentError = new Error("stream failed");
|
||||
|
||||
expect(
|
||||
shouldClearPendingHumanInputOnThreadError({
|
||||
currentError,
|
||||
pendingRequestCount: 1,
|
||||
previousError: undefined,
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
shouldClearPendingHumanInputOnThreadError({
|
||||
currentError,
|
||||
pendingRequestCount: 0,
|
||||
previousError: undefined,
|
||||
}),
|
||||
).toBe(false);
|
||||
expect(
|
||||
shouldClearPendingHumanInputOnThreadError({
|
||||
currentError: previousError,
|
||||
pendingRequestCount: 1,
|
||||
previousError,
|
||||
}),
|
||||
).toBe(false);
|
||||
expect(
|
||||
shouldClearPendingHumanInputOnThreadError({
|
||||
currentError: undefined,
|
||||
pendingRequestCount: 1,
|
||||
previousError: currentError,
|
||||
}),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
test("creates option and text responses for a request", () => {
|
||||
const request = extractHumanInputRequest({
|
||||
type: "tool",
|
||||
name: "ask_clarification",
|
||||
content: "fallback",
|
||||
artifact: {
|
||||
human_input: requestPayload,
|
||||
},
|
||||
} as unknown as Message);
|
||||
|
||||
expect(request).not.toBeNull();
|
||||
const optionResponse = createHumanInputOptionResponse(
|
||||
request!,
|
||||
request!.options![1]!,
|
||||
);
|
||||
const textResponse = createHumanInputTextResponse(
|
||||
request!,
|
||||
"Use blue-green deployment",
|
||||
);
|
||||
|
||||
expect(optionResponse).toEqual({
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "option",
|
||||
option_id: "option-2",
|
||||
value: "staging",
|
||||
});
|
||||
expect(textResponse).toEqual({
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "text",
|
||||
value: "Use blue-green deployment",
|
||||
});
|
||||
expect(buildHumanInputResponseText(request!, optionResponse)).toBe(
|
||||
'For your clarification "Which environment should I deploy to?", my answer is: staging',
|
||||
);
|
||||
});
|
||||
|
|
@ -60,3 +60,39 @@ test("keeps uploaded files on the visible user message only", () => {
|
|||
],
|
||||
});
|
||||
});
|
||||
|
||||
test("keeps human input response metadata on the hidden user message", () => {
|
||||
const response = {
|
||||
version: 1,
|
||||
kind: "human_input_response",
|
||||
source: "ask_clarification",
|
||||
request_id: "clarification:call-abc",
|
||||
response_kind: "option",
|
||||
option_id: "option-2",
|
||||
value: "staging",
|
||||
};
|
||||
|
||||
const messages = buildThreadSubmitMessages({
|
||||
text: 'For your clarification "Which environment?", my answer is: staging',
|
||||
additionalKwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
});
|
||||
|
||||
expect(messages).toEqual([
|
||||
{
|
||||
type: "human",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: 'For your clarification "Which environment?", my answer is: staging',
|
||||
},
|
||||
],
|
||||
additional_kwargs: {
|
||||
hide_from_ui: true,
|
||||
human_input_response: response,
|
||||
},
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue