mirror of
https://github.com/shareAI-lab/learn-claude-code.git
synced 2026-08-27 01:02:08 +00:00
Rewrite s16 workflow chapter for clarity and teaching fidelity
Retell the Runtime lesson as a progressive story (two doors, kitchen primitives, null-isolation, longest-prefix resume) in EN/ZH/JA, and align the mini-runtime with Claude Code / Pi semantics so the docs and code agree. Co-authored-by: Xinlu Lai <CrazyBoyM@users.noreply.github.com>
This commit is contained in:
Cursor Agent
parent
eb4307f4e4
commit
e28bec6dd4
5 changed files with 695 additions and 548 deletions
|
|
@ -1,51 +1,57 @@
|
|||
# s16: Workflow Runtime — モデルが単一 step を決め、script が orchestration を決める
|
||||
# s16: Workflow Runtime — レシピをコードに書く
|
||||
|
||||
[English](README.md) · [中文](README.zh.md) · [日本語](README.ja.md)
|
||||
|
||||
s01 → ... → s14 → [s15](../s15_integrated_harness/) → `s16` → [s17](../s17_goal_loop/)
|
||||
|
||||
> *「1 回の tool_use で、一式の orchestration を実行する」* — `Workflow` ツールが復元可能な script runtime を起動し、多数の agent call を協調させます。
|
||||
> *「ターンごとのチャットは、10 秒ごとにシェフへメールするようなものです。Workflow は厨房が従えるレシピです。」*
|
||||
>
|
||||
> **Harness 層**: Orchestration — single-agent loop の上で保存済み multi-agent script を実行します。
|
||||
> **Harness 層**: Orchestration — single-agent loop の上で multi-agent script を実行します。
|
||||
|
||||
---
|
||||
|
||||
s01 から s15 まで、各 round で model が呼び出す tools を決めます。tool results が `messages[]` に入ると、model は更新された context から次の step を決めます。次の経路が前の step の発見に依存する task に向いています。
|
||||
友だちとチャットだけで料理している場面を想像してください。「玉ねぎを切って」と送り、返事を待ち、「できた？」、次は「フライパンを……」。一品ならまだよいです。二十卓の宴会では、チャット自体がボトルネックになります。手順を忘れ、同じ指示を繰り返し、スマホが落ちたら最初からやり直しです。
|
||||
|
||||
一方、固定された流れを繰り返す task もあります。code review なら、複数の観点を同時に調べ、各 finding を検証し、重複をまとめて severity 順に並べます。実行前に step と順序が分かっている場合、host には次の 3 つが必要です。
|
||||
ふつうの「モデルが指揮者」な会話も同じです。**Workflow** は書かれたレシピです。厨房（runtime）がそれに従い、助手（subagent）が判断し、途中の器はカウンターに置かれます —— グループチャットの中ではありません。
|
||||
|
||||
- **並行性**: 1 件ずつ順番に待たないこと。
|
||||
- **安定した結果構造**: 個々の agent answer が変わっても構造を保つこと。
|
||||
- **復元可能性**: 途中で止まっても、完了済みの部分を最初からやり直さないこと。
|
||||
## 問題
|
||||
|
||||
この orchestration が conversation history にしか存在しなければ、順序と checkpoint も history にしか残りません。saved workflow は固定 flow を code に置き、完了した call を journal に記録します。
|
||||
s01 から s15 まで、各ラウンドでモデルが次の tool を選びます。直前の発見で次の道が変わるタスクには向いています。
|
||||
|
||||
## 計画は chat のラウンドを重ねず、コードに書く
|
||||
一方、形が先に分かっている仕事もあります。
|
||||
|
||||
harness の tool pool に `Workflow` ツールを追加します。host は `agent() / parallel() / pipeline() / phase()` で構成した trusted script を登録します。model が渡すのは saved workflow name、argument、任意の resume run ID だけで、実行可能 code や metadata は渡しません。
|
||||
- 複数の観点で多くのファイルを review する
|
||||
- 調査 → 検証 → 統合
|
||||
- N 個のモジュールを同じやり方で移行する
|
||||
|
||||
workflow は 1 回の `tool_use` として main loop に入ります。script の実行中、runtime は lifecycle event と progress event を出し、各 step を disk journal へ記録します。script が終わると、この call は launch 情報、result、task state を返します。script の中間結果は変数に保存され、conversation history を使いません。`resume_from_run_id` で再開すると、変更されていない `agent()` は journal の結果を再利用します。
|
||||
計画を `messages[]` の中だけで「覚えている」と、三つのことが起きます。orchestration の雑音で context が埋まる、途中で計画がずれる、落ちたら完了済みの作業までやり直す。
|
||||
|
||||
必要なのは並行性、安定した結果の形、そして再開です。会話履歴だけにそれを預けるのは弱いです。
|
||||
|
||||
## 一息でいうアイデア
|
||||
|
||||
**計画をコードへ移します。** Subagent は相変わらず判断します。script がループ、扇状の分配、マージを持ちます。中間結果は変数にあり、会話には入りません。
|
||||
|
||||
|
||||
|
||||
```python
|
||||
SAMPLE_META = {"name": "review-changes", "description": "コード変更を review", "phases": ["Review", "Verify"]}
|
||||
1 回の `Workflow` tool call が、その script 実行を始めます。実行中に lifecycle / progress event が出て、最後に launch 情報・result・task state を含む tool result が返ります。
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify) # 各 dimension が独立して audit → verify を通る
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"{len(confirmed)} 件の実在する問題を確認")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
## ふたつの入口
|
||||
|
||||
## Workflow ツール: 1 回の call で run 全体を実行する
|
||||
Claude Code は、workflow の始め方について正直です。
|
||||
|
||||
`Workflow` は s15 host の既存 tool pool に追加されます。ユーザーが保存済み workflow の実行を求めるか、タスクが既知の orchestration に一致したときにモデルがこのツールを選びます。adapter は name を host-owned `WORKFLOWS` registry で解決し、trusted metadata と function を runtime へ渡します。s15 の他の tools も同じ loop で利用できます。
|
||||
| 入口 | 渡すもの | いつ使うか |
|
||||
|------|----------|------------|
|
||||
| **Dynamic** | オーケストレーション用の JavaScript（`script`、あとから `scriptPath`） | モデルが**このタスク用**にレシピを書く |
|
||||
| **Saved** | `name` + `args` | 良いレシピを例えば `.claude/workflows/` に保存し、名前で再実行する |
|
||||
|
||||
model-facing schema が受け取るのは `name`、`args`、`resume_from_run_id` です。unknown name や不正 argument は error tool result として返し、host loop を終了させません。その後 runtime が登録済み metadata を検証し、permission check を通し、local workflow task を登録して、script の実行前に `async_launched` を出します。progress event と最後の `task_notification` が続き、call は JSON-safe な launch 情報、result、task state を返します。
|
||||
厨房は同じです。Dynamic は「今レシピを書く」、Saved は「カード箱から引く」です。
|
||||
|
||||
**このレッスンは Python の teaching runtime です。** 同じアイデアを、1 行ずつ読める形で示します。デモは名前で saved workflow を登録します。概念は Claude Code の script 世界と 1:1 です。「モデルは実行可能コードを渡せない」と Claude Code について主張するのは誤りでした。ここでは単に、完全な JS インタプリタを埋め込まないだけです。
|
||||
|
||||
```python
|
||||
# Teaching adapter: saved の入口（name + args）。
|
||||
# Claude Code は script / scriptPath / resumeFromRunId も受け付ける。
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"input_schema": {
|
||||
|
|
@ -54,192 +60,148 @@ WORKFLOW_TOOL = {
|
|||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
"resumeFromRunId": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta, script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {"launched": out["launched"], "result": out["result"],
|
||||
"task": serialize_task(out["task"])}
|
||||
```
|
||||
|
||||
## Workflow metadata: 起動前に検証する
|
||||
## プリミティブを学校のバザーで
|
||||
|
||||
各 saved workflow は `name`、`description`、任意の `phases` を持つ trusted metadata を登録します。runtime は workflow code を実行する前に検証します。`name` と `description` は task と UI の表示に使い、`phases` は progress 表示の group 名を定義します。これらは model input ではなく host registry に属します。
|
||||
学校のバザーでたくさんのケーキを焼くとします。各テーブルは 混ぜる → 焼く → 箱詰め。助手が味見と判断をし、レシピが順番を決めます。
|
||||
|
||||
不正な登録内容は launch 前に `WorkflowInputError` になります。s12 の cron 式検証と同じ考えです。不正な saved workflow が実行時まで進んでから壊れないようにします。
|
||||
| Primitive | 厨房での意味 |
|
||||
|-----------|--------------|
|
||||
| `agent(prompt, {schema, label, phase})` | 助手ひとりに一つの仕事を頼む |
|
||||
| `pipeline(items, *stages)` | **既定。** 各ケーキが自分で混ぜ→焼き→箱詰めを通る。A が箱詰め中でも、B はまだ混ぜているかもしれない |
|
||||
| `parallel(thunks)` | **すべての**トレイが戻るまで待つ — 次の段が本当に全部の結果を必要とするときだけ |
|
||||
| `phase(title)` | 進捗ボードに「今は焼き工程」と出す |
|
||||
| `log(message)` | 短いステータスを一声 |
|
||||
| `workflow(name, args)` | 小さなレシピを呼ぶ（ネストは 1 段） |
|
||||
| `args` | この run に渡す材料リスト |
|
||||
| `budget` | 使える「オーブン分」（token） |
|
||||
|
||||
runtime は `meta.name` をローカル artifact のファイル名に使うため、英数字で始まり、英数字、`.`、`_`、`-` のみからなる 1-64 文字の安全な slug も要求する。
|
||||
既定は `pipeline` です。次の段が直前の結果をすべてまとめて必要とするときだけ `parallel` を使います —— 全トレイを味見してから採点表を書く、といった場合です。
|
||||
|
||||
```python
|
||||
def validate_meta(meta):
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta は object literal でなければなりません")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta には name と description が必要です")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError("meta.name は安全な 1-64 文字の slug が必要です")
|
||||
if "phases" in meta and (
|
||||
not isinstance(meta["phases"], list)
|
||||
or not all(isinstance(p, str) and p for p in meta["phases"])
|
||||
):
|
||||
raise WorkflowInputError("meta.phases は空でない文字列だけを含む必要があります")
|
||||
return meta
|
||||
# 各 dimension が独立して audit → verify を通る（stage 間に barrier なし）。
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
```
|
||||
|
||||
## Orchestration primitive
|
||||
## 答えを機械が読める形に
|
||||
|
||||
script は少数の orchestration primitive だけを公開する `ExecutionState` を受け取り、ファイルを直接読み書きせず、shell も実行しません。default の interactive mode では `agent()` を host と同じ real API client に接続し、各 workflow agent は arguments で渡された内容だけを読みます。`demo` と unit test は `MockAgentRunner` を使い、event と journal replay を繰り返し確認できるようにします。
|
||||
|
||||
| Primitive | 役割 |
|
||||
|------|------|
|
||||
| `agent(prompt, {schema, label, phase})` | 1 つの subagent を派遣 |
|
||||
| `parallel(thunks)` | **barrier**: すべての task を並行実行し、全結果が戻るまで待つ |
|
||||
| `pipeline(items, *stages)` | 各 item を **barrier なし**で stage ごとに実行し、終わった item から先へ進める |
|
||||
| `phase(title)` | 現在の progress phase を記録し、progress bar を更新 |
|
||||
| `log(message)` | progress log を 1 行出力 |
|
||||
| `workflow(name, args)` | nested sub-workflow（1 階層だけ） |
|
||||
|
||||
各 item が同じ stage を独立して通る場合は `pipeline` を使えます。item A が stage 3 にいる間、item B はまだ stage 1 かもしれません。次の処理が前の group の全結果を必要とする場合は `parallel` を使います。
|
||||
助手が散文で返してくると、次の stage は finding と verdict を reliably に対応づけられません。`schema` を渡します。runtime は JSON を求め、検証し、だめなら**1 回だけ**再試行します。それでもだめならその call はエラーになります（下の null 分離を参照）。
|
||||
|
||||
```python
|
||||
async def pipeline(self, items, *stages):
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages: # 各 item がすべての stage を独立して完走
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
out = await ctx.agent(
|
||||
f"この変更に {dimension} 関連の問題がないか確認してください:\n{changes}",
|
||||
schema=FINDINGS_SCHEMA,
|
||||
label=f"audit:{dimension}",
|
||||
)
|
||||
# out は "findings" を持つ dict であり、段落ではない
|
||||
```
|
||||
|
||||
## 構造化出力: Subagent に散文を返させない
|
||||
あなたとの会話は自然言語でよいです。パイプラインには合うソケットが必要です。
|
||||
|
||||
`agent({schema})` は、schema に一致する JSON object だけを返すよう workflow agent に要求します。runtime は結果を parse、validate し、不一致なら 1 回 retry します。下流コードは prose から field を取り出さず、object を受け取れます。
|
||||
## 助手がひとり失敗したとき
|
||||
|
||||
s05 では tool argument を全面的に信頼できないと説明しました。ここでは同じ教訓を逆向きに使います。subagent の出力も全面的には信頼できません。orchestration boundary で検証し、1 回 retry の機会を与え、不確実性を後続 flow の外へ止めます。
|
||||
トレイがひとつ焦げても、艦隊全体を止めてはいけません。
|
||||
|
||||
- **`parallel`**: 失敗した thunk はそのスロットで `null` / `None` になります。gather 自体は reject しません。
|
||||
- **`pipeline`**: 失敗した stage は**その item** を `null` / `None` にし、残りの stage をスキップします。他の item は進み続けます。
|
||||
|
||||
マージ前に注意して絞り込みます。よくあるのは `if r` / `.filter(Boolean)` です。
|
||||
|
||||
```python
|
||||
run = await asyncio.to_thread(self.runner.run, prompt, schema, label)
|
||||
result = run.value
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok: # 1 回だけ注意して retry、それでも不正なら error
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run, prompt + "\n\n有効な JSON を返してください。", schema, label
|
||||
)
|
||||
result = retry.value
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) の出力が不正です: {err}")
|
||||
verdicts = await ctx.parallel([...]) # いくつかは None かもしれない
|
||||
confirmed = [
|
||||
f for f, v in zip(findings, verdicts)
|
||||
if v and v.get("isReal")
|
||||
]
|
||||
```
|
||||
|
||||
## Task state と progress event
|
||||
## Journal と resume
|
||||
|
||||
`LocalWorkflowTask` は status と token usage を管理し、SDK style の event stream を外へ出します。`task_started` → phase change、subagent start、log を含む一連の `task_progress` → 完了または失敗に加え、output file、agent 数、token 数を含む最後の `task_notification` です。
|
||||
各 run には `runId` があります。`agent()` が終わるたびに、runtime は disk 上の journal へ 1 行追記します。ノートだと思ってください。助手がオーブンから戻った順ではなく、あなたが**呼んだ**順です。
|
||||
|
||||
demo はこれらの event を順番に表示し、最後の notification の後で task state を返します。
|
||||
resume（`resume_from_run_id` / `resumeFromRunId`）では script をまた先頭から走らせますが：
|
||||
|
||||
```python
|
||||
class LocalWorkflowTask:
|
||||
def progress_event(self, ptype, **data): # phase/subagent/log
|
||||
self.progress.append({"type": ptype, **data})
|
||||
print(f" progress {ptype} ...")
|
||||
1. 呼び出し順で、各 `agent()` を次の journal 行と照合します。
|
||||
2. **最長の未変更プレフィックス** → cache hit（即座に再生）。
|
||||
3. **最初の**変更または未完了 call でプレフィックスが切れます。
|
||||
4. **それ以降はすべて live** — journal の後ろに古い key が残っていても、黙って hit しません。
|
||||
|
||||
本物の JS workflow runtime が `Date.now()` / `Math.random()` / 引数なしの `new Date()` を禁じるのはこのためです。非決定的な時計や乱数は prompt や呼び出し順を変え、ノートが合わなくなります。この Python デモは完全なサンドボックスではありません —— それでも script は決定的に書いてください。
|
||||
|
||||
```text
|
||||
journal: [A ✓] [B ✓] [C ✓] [D ✓]
|
||||
resume: A hit → B hit → C 変更 → D は live（古い D への silent hit なし）
|
||||
```
|
||||
|
||||
## 保存: Snapshot + journal で中断から再開する
|
||||
## サンプルを歩く: `review-changes`
|
||||
|
||||
runtime は各 run を `s16_workflow_runtime/.runtime/` に保存します。`<runId>.json` snapshot、`<runId>.output.json` output、`<runId>.journal.jsonl` journal、`<runId>.lock` coordination file です。fresh run は journal を開く前に exclusive file creation で新しい `runId` を予約します。run lock は実行と最終永続化が終わるまで保持するため、別 process は同じ run を同時に resume できません。snapshot に workflow name、arguments、task state を記録し、resume は保存済み snapshot と journal を先に検証してから、成功済み artifact を変更します。
|
||||
4 つの review dimension が同じ 2 段階の道を通ります。
|
||||
|
||||
journal は checkpoint resume の中心で、各 `agent()` の結果を 1 行ずつ記録します。
|
||||
|
||||
```python
|
||||
class WorkflowJournal:
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
```text
|
||||
correctness ── audit ── verify ──┐
|
||||
security ── audit ── verify ──┤── 確認済み finding を統合
|
||||
performance ── audit ── verify ──┤
|
||||
style ── audit ── verify ──┘
|
||||
```
|
||||
|
||||
## Resume: runId から続行し、変更のないものを再利用する
|
||||
|
||||
`resume_from_run_id` を渡して workflow を再度呼ぶと script を再実行しますが、各 `agent()` は決定的な semantic key を計算します。journal に key があれば、再実行せず cached result を返します。変更された call と、それに依存する後続 step だけが本当に動きます。
|
||||
|
||||
key は concurrency の完了順に依存してはいけません。`parallel` と `pipeline` の Agent は不定の順番で完了します。「何番目に完了したか」を key にすると、次回の cache が別の call へ対応してしまいます。そのため key は競合する counter ではなく、call の内容、つまり type、label、prompt、schema の stable hash です。
|
||||
|
||||
```python
|
||||
def key(self, kind, label, prompt, schema):
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
# agent() の内部:
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
self.task.progress_event("workflow_agent", label=label, status="cached")
|
||||
return cached
|
||||
```
|
||||
|
||||
## Stable call key
|
||||
|
||||
resume では、現在の各 `agent()` call を以前の journal record と対応付ける必要があります。stable hash は変更されていない workflow code と arguments に同じ call key を与えます。real model の出力は変化しても、call 内容が同じなら journal に保存済みの result を使います。
|
||||
|
||||
## 実際に動かす
|
||||
|
||||
sample workflow `review-changes` は `pipeline` を使い、各 review dimension を独立して audit → verify へ通します。interactive mode は real API を使い、`args.changes` から review 対象を読みます。`demo` は固定 runner data で pipeline、validation、journal、resume を示します。
|
||||
1. **Review** — 各 dimension の auditor が構造化 findings を返します。
|
||||
2. **Verify** — 各 finding を敵対的チェッカーへ（verify stage 内で `parallel`）。
|
||||
3. 本物とされたものだけ残し、severity で並べます。
|
||||
|
||||
```python
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
|
||||
async def audit(_v, dimension, _i):
|
||||
out = await ctx.agent(f"この変更に {dimension} 関連の問題がないか確認してください:\n{changes}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _i):
|
||||
ctx.phase("Verify")
|
||||
verdicts = await ctx.parallel([ # 各 finding を独立して verify
|
||||
(lambda f=f: ctx.agent(f"変更内容に照らして finding を検証してください:\n{changes}\n\n{f}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}"))
|
||||
for f in audited["findings"]])
|
||||
return {"dimension": dimension,
|
||||
"confirmed": [f for f, v in zip(audited["findings"], verdicts) if v and v["isReal"]]}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
...
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"{len(confirmed)} 件の実在する問題を確認")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
|
||||
## s15 からの変更点
|
||||
## s15 へのつなぎ方
|
||||
|
||||
| | s15 Integrated Harness | s16 Workflow Runtime |
|
||||
|--|-----------|---------------------|
|
||||
| loop | 1 つ、モデル駆動 | main loop は不変。tool の背後で script orchestration を実行 |
|
||||
| 次の step を決めるもの | モデルが毎ラウンド判断 | script が orchestration flow を事前に定義 |
|
||||
| multi-agent | s06 subagent を一度だけ派遣 | agent-runner boundary を通る scripted、resumable call |
|
||||
| 新しい仕組み | — | orchestration primitive、host registry と tool adapter、task lifecycle、progress event、journal/resume、structured output |
|
||||
s15 は依然として host loop です。s16 が足すのは一つの tool、`Workflow` だけです。モデル（またはあなた）が saved name を渡し、adapter が registry を解決して script を走らせます。
|
||||
|
||||
s16 は main loop を置き換えません。tool layer に `Workflow` を公開し、背後で local workflow runtime を起動します。saved script が agent-runner boundary を通じて N 回の call を協調させます。s06 の subagent はモデルがその場で 1 回派遣し、s16 は orchestration を resumable な host code にします。
|
||||
| | Claude Code / Pi（製品） | この teaching CLI |
|
||||
|--|--------------------------|-------------------|
|
||||
| Script 言語 | サンドボックス内の JavaScript | 読める Python 関数 |
|
||||
| Dynamic 入口 | モデルが `script` を書く / `scriptPath` を編集 | 文書で説明。デモは saved の `name` |
|
||||
| 実行中の host | バックグラウンド + 通知でセッションが応答し続ける | 観察しやすいよう `demo` / `resume` は前景 |
|
||||
| アイデア | 同じ primitives、journal、prefix resume | teaching model — 簡略化は明示する |
|
||||
|
||||
main loop が workflow エンジンになるわけではありません。`bash` や `task` を借りるのと同じく、tool をひとつ借ります。
|
||||
|
||||
## 試してみる
|
||||
|
||||
```bash
|
||||
python s16_workflow_runtime/code.py # main model と Workflow agent の両方が real API を使う
|
||||
python s16_workflow_runtime/code.py demo # deterministic fixture と event stream を確認
|
||||
python s16_workflow_runtime/code.py resume # 前回の runId から resume。すべての agent() が journal cache に当たる
|
||||
python s16_workflow_runtime/code.py # s15 host + Workflow tool（real API）
|
||||
python s16_workflow_runtime/code.py demo # 固定 fixture: phase と agent を観察
|
||||
python s16_workflow_runtime/code.py resume # 同じ runId。prefix はすべて cache hit になるはず
|
||||
```
|
||||
|
||||
default command では、model に changes を読ませ、その text を `args.changes` に入れて保存済み `review-changes` workflow を実行させます。main model と workflow agent の両方が real API を使います。`demo` は固定 runner data で lifecycle と resume を繰り返し観察でき、すべて cache hit した resume は `agents=0 tokens=0` と表示されます。
|
||||
見るポイント:
|
||||
|
||||
## 次へ
|
||||
- `workflow_phase` が Review、続いて Verify
|
||||
- 各 `workflow_agent` が初回は `done`、完全 resume では `cached`
|
||||
- 末尾の短い confirmed リスト。全 hit の resume は `agents=0 tokens=0`
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) は、より小さな独立 loop で goal が達成されたかを確認し、次の round が必要かを判断します。
|
||||
## s15 との対比 → 次は s17
|
||||
|
||||
<!-- translation-sync: zh@v10, en@v10, ja@v10 -->
|
||||
| | s15 Integrated Harness | s16 Workflow Runtime |
|
||||
|--|------------------------|----------------------|
|
||||
| loop | 1 つ、モデル駆動 | 同じ loop。1 つの tool が script を実行 |
|
||||
| 次の step を決めるもの | モデルが毎ラウンド | script がバッチの形を持つ |
|
||||
| multi-agent | 一度きりの subagent | script 化・再開可能な `agent()` |
|
||||
| 失敗 / resume | 会話メモリ頼り | null 分離 + journal prefix |
|
||||
|
||||
**s16 = バッチの回し方。s17 = ゴール全体が終わったかどうか。**
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) は独立した評価器に聞きます。止めるべきか、もう一ターンか。
|
||||
|
||||
<!-- translation-sync: zh@v11, en@v11, ja@v11 -->
|
||||
|
|
|
|||
|
|
@ -1,51 +1,57 @@
|
|||
# s16: Workflow Runtime — The Model Decides Each Step; a Script Decides the Orchestration
|
||||
# s16: Workflow Runtime — Put the Recipe in Code
|
||||
|
||||
[English](README.md) · [中文](README.zh.md) · [日本語](README.ja.md)
|
||||
|
||||
s01 → ... → s14 → [s15](../s15_integrated_harness/) → `s16` → [s17](../s17_goal_loop/)
|
||||
|
||||
> *"One tool_use runs an entire orchestration"* — The `Workflow` tool starts a recoverable script runtime that coordinates many agent calls.
|
||||
> *"Chatting turn-by-turn is like texting the chef every ten seconds. A workflow is a recipe the kitchen can follow."*
|
||||
>
|
||||
> **Harness layer**: Orchestration — run saved multi-agent scripts above the single-agent loop.
|
||||
> **Harness layer**: Orchestration — run a multi-agent script above the single-agent loop.
|
||||
|
||||
---
|
||||
|
||||
From s01 through s15, the model decides which tools to call in each round. Their results enter `messages[]`, and the model decides the next step from the updated context. This works well when the path depends on what the previous step discovers.
|
||||
Imagine you are cooking with a friend over text. You send “chop the onions,” wait, ask “are they done?”, then “now the pan…”. It works for one dish. For a feast with twenty dishes, that chat becomes the bottleneck: you forget steps, repeat yourself, and if the phone dies you start over.
|
||||
|
||||
Some tasks repeat a fixed sequence. A code review may inspect several dimensions concurrently, verify each finding, combine duplicates, and sort the result. The sequence and dependencies are known before execution. Here the host needs three things:
|
||||
That is ordinary model-as-orchestrator chatting. A **workflow** is the written recipe: the kitchen (runtime) follows it, helpers (subagents) do judgment, and intermediate bowls sit on the counter — not in the group chat.
|
||||
|
||||
- **Parallelism**, rather than waiting for one item at a time;
|
||||
- **A stable result structure**, even when individual agent answers vary;
|
||||
- **Recoverability**, so an interruption does not rerun work that is already complete.
|
||||
## The problem
|
||||
|
||||
If this orchestration exists only in conversation history, its ordering and checkpoints also exist only in that history. A saved workflow puts the fixed sequence in code and records completed calls in a journal.
|
||||
From s01 through s15, the model picks the next tool each round. That shines when the path depends on what you just discovered.
|
||||
|
||||
## Put the Plan in Code, Not in a Sequence of Chat Turns
|
||||
Some jobs already know their shape:
|
||||
|
||||
Add a `Workflow` tool to the harness tool pool. The host registers trusted scripts built from `agent()`, `parallel()`, `pipeline()`, and `phase()`. The model supplies only a saved workflow name, arguments, and an optional run ID to resume; it does not send executable code or metadata.
|
||||
- review many files on several dimensions
|
||||
- research, then verify, then merge
|
||||
- migrate N modules the same way
|
||||
|
||||
The workflow enters the main loop as one `tool_use`. As the script runs, the runtime emits lifecycle and progress events and records every step in a journal on disk. When the script finishes, the call returns the launch envelope, result, and task state. Intermediate script results live in variables instead of taking space in conversation history. When restarted with `resume_from_run_id`, unchanged `agent()` calls hit the journal cache and reuse previous results.
|
||||
If the model keeps “remembering” the plan inside `messages[]`, three things go wrong: context fills with orchestration noise, the plan drifts mid-run, and a crash means redoing finished work.
|
||||
|
||||
You want parallelism, stable result shapes, and a way to resume. Chat history is a weak place to store all three.
|
||||
|
||||
## The idea in one breath
|
||||
|
||||
**Move the plan into code.** Subagents still think. The script owns loops, fan-out, and merge. Intermediate results live in variables, not in the conversation.
|
||||
|
||||
|
||||
|
||||
```python
|
||||
SAMPLE_META = {"name": "review-changes", "description": "Review code changes", "phases": ["Review", "Verify"]}
|
||||
One `Workflow` tool call starts that scripted run. Lifecycle and progress events fire while it works; one tool result comes back with launch info, the result, and task state.
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify) # Each dimension independently runs audit → verify
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"Confirmed {len(confirmed)} real issues")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
## Two doors into a workflow
|
||||
|
||||
## The Workflow Tool: One Call, One Complete Run
|
||||
Claude Code is honest about how a workflow starts:
|
||||
|
||||
`Workflow` is added to the s15 host's existing tool pool. The user can request a saved workflow, or the model can select it when a task matches a known orchestration. The adapter resolves the name through the host-owned `WORKFLOWS` registry, then passes its trusted metadata and function to the runtime. The other s15 tools remain available in the same loop.
|
||||
| Door | What you pass | When |
|
||||
|------|----------------|------|
|
||||
| **Dynamic** | A JavaScript orchestration script (`script`, or later `scriptPath`) | The model writes a recipe for *this* task |
|
||||
| **Saved** | `name` + `args` | A good recipe lives under e.g. `.claude/workflows/` and you rerun it |
|
||||
|
||||
The model-facing schema accepts `name`, `args`, and `resume_from_run_id`. Unknown names and malformed arguments become an error tool result instead of ending the host loop. The runtime then validates the registered metadata, checks permissions, registers a local workflow task, and emits `async_launched` before running the script. Progress events follow, then the final `task_notification`; the call returns JSON-safe launch information, result, and task state.
|
||||
Same kitchen either way. Dynamic is “write the recipe now.” Saved is “pull the card from the box.”
|
||||
|
||||
**This lesson is a Python teaching runtime.** It shows the same ideas so you can read every line. Our demo registers a saved workflow by name; the concepts map 1:1 to Claude Code’s script world. We do **not** claim “the model cannot submit executable code” — that was wrong for Claude Code. We simply skip embedding a full JS interpreter here.
|
||||
|
||||
```python
|
||||
# Teaching adapter: saved door (name + args).
|
||||
# Claude Code also accepts script / scriptPath / resumeFromRunId.
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"input_schema": {
|
||||
|
|
@ -54,192 +60,148 @@ WORKFLOW_TOOL = {
|
|||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
"resumeFromRunId": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta, script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {"launched": out["launched"], "result": out["result"],
|
||||
"task": serialize_task(out["task"])}
|
||||
```
|
||||
|
||||
## Workflow Metadata: Validate Before Launch
|
||||
## Primitives, taught with a kitchen story
|
||||
|
||||
Each saved workflow registers trusted metadata with `name`, `description`, and optional `phases`. The runtime validates it before executing workflow code. `name` and `description` identify the task in the UI, while `phases` names groups in the progress display. These fields belong to the host registry, not to model input.
|
||||
You are running a school bake sale. Each table needs mix → bake → box. Helpers taste and judge; the recipe decides the order.
|
||||
|
||||
Invalid registration raises `WorkflowInputError` before launch. This is the same idea as validating cron expressions in s12: do not wait until execution to discover a bad saved workflow.
|
||||
| Primitive | Kitchen meaning |
|
||||
|-----------|-----------------|
|
||||
| `agent(prompt, {schema, label, phase})` | Ask one helper to do one job |
|
||||
| `pipeline(items, *stages)` | **Default.** Each cake goes through mix→bake→box on its own. Cake A can be boxing while cake B is still mixing |
|
||||
| `parallel(thunks)` | Wait until **every** tray comes back — only when the next step needs all of them together |
|
||||
| `phase(title)` | Announce “we’re in baking now” on the progress board |
|
||||
| `log(message)` | Shout a short status line |
|
||||
| `workflow(name, args)` | Call a smaller recipe (one level deep) |
|
||||
| `args` | The ingredients list passed into this run |
|
||||
| `budget` | How many “oven minutes” (tokens) you may spend |
|
||||
|
||||
Because the runtime uses `meta.name` in local artifact filenames, it also requires a 1-64 character safe slug containing letters, numbers, `.`, `_`, or `-`.
|
||||
Default to `pipeline`. Reach for `parallel` only when the next step truly needs every prior result at once — like tasting all trays before writing the scorecard.
|
||||
|
||||
```python
|
||||
def validate_meta(meta):
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta must be an object literal")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta requires name and description")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError("meta.name must be a safe 1-64 character slug")
|
||||
if "phases" in meta and (
|
||||
not isinstance(meta["phases"], list)
|
||||
or not all(isinstance(p, str) and p for p in meta["phases"])
|
||||
):
|
||||
raise WorkflowInputError("meta.phases must contain non-empty strings")
|
||||
return meta
|
||||
# Each dimension walks audit → verify on its own (no barrier between stages).
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
```
|
||||
|
||||
## Orchestration Primitives
|
||||
## Make answers machine-readable
|
||||
|
||||
A script receives an `ExecutionState` exposing a small set of orchestration primitives. It does not read files or run shell commands directly. The default interactive mode connects `agent()` to the same real API client as the host, and each workflow agent reads only the content supplied through workflow arguments. `demo` and unit tests use `MockAgentRunner` so events and journal replay are repeatable.
|
||||
|
||||
| Primitive | Purpose |
|
||||
|------|------|
|
||||
| `agent(prompt, {schema, label, phase})` | Dispatch one subagent |
|
||||
| `parallel(thunks)` | **Barrier**: run every task concurrently and wait until all results return |
|
||||
| `pipeline(items, *stages)` | Run each item through stages **without a barrier**; finished items proceed immediately |
|
||||
| `phase(title)` | Mark the current progress phase and update the progress display |
|
||||
| `log(message)` | Emit a progress log line |
|
||||
| `workflow(name, args)` | Run a nested sub-workflow, one level only |
|
||||
|
||||
Use `pipeline` when each item independently crosses the same stages. Item A may reach stage three while item B is still in stage one. Use `parallel` when the next step needs every result from the preceding group.
|
||||
If a helper returns a poem, the next stage cannot reliably zip findings to verdicts. Pass a `schema`: the runtime asks for JSON, validates it, and retries **once**. Fail again and that call errors (see null-isolation below).
|
||||
|
||||
```python
|
||||
async def pipeline(self, items, *stages):
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages: # Each item independently completes every stage
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
out = await ctx.agent(
|
||||
f"Inspect this change for {dimension} issues:\n{changes}",
|
||||
schema=FINDINGS_SCHEMA,
|
||||
label=f"audit:{dimension}",
|
||||
)
|
||||
# out is a dict with "findings", not a paragraph
|
||||
```
|
||||
|
||||
## Structured Output: Do Not Let Subagents Return Essays
|
||||
Free-form prose is fine for chatting with you. Pipelines need sockets that fit.
|
||||
|
||||
`agent({schema})` asks a workflow agent to return only a JSON object matching the schema. The runtime parses and validates the result, then retries once if it does not match. Downstream code receives an object instead of extracting fields from prose.
|
||||
## When one helper fails
|
||||
|
||||
s05 warned that tool arguments cannot be trusted completely. This is the same lesson in reverse: subagent output cannot be trusted completely either. Validate at the orchestration boundary, give one retry, and keep uncertainty out of the rest of the flow.
|
||||
A fleet should not stop because one tray burned.
|
||||
|
||||
- **`parallel`**: a failing thunk becomes `null` / `None` in that slot; the gather itself does not reject.
|
||||
- **`pipeline`**: a failing stage drops **that item** to `null` / `None` and skips its remaining stages; other items keep going.
|
||||
|
||||
Filter with care — usually `if r` / `.filter(Boolean)` — before you merge.
|
||||
|
||||
```python
|
||||
run = await asyncio.to_thread(self.runner.run, prompt, schema, label)
|
||||
result = run.value
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok: # Retry once with a reminder, then fail
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run, prompt + "\n\nReturn valid JSON.", schema, label
|
||||
)
|
||||
result = retry.value
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) returned invalid output: {err}")
|
||||
verdicts = await ctx.parallel([...]) # some entries may be None
|
||||
confirmed = [
|
||||
f for f, v in zip(findings, verdicts)
|
||||
if v and v.get("isReal")
|
||||
]
|
||||
```
|
||||
|
||||
## Task State and Progress Events
|
||||
## Journal + resume
|
||||
|
||||
`LocalWorkflowTask` maintains status and token usage and emits an SDK-style event stream: `task_started` → a sequence of `task_progress` events containing phase changes, subagent starts, and log batches → one final `task_notification` reporting completion or failure, plus the output file and agent and token counts.
|
||||
Every run gets a `runId`. As each `agent()` finishes, the runtime appends a line to a journal on disk. Think of a notebook that lists helpers in the order you *called* them, not the order they wandered back from the oven.
|
||||
|
||||
The demo prints these events in order and returns the task state after the final notification.
|
||||
On resume (`resume_from_run_id` / `resumeFromRunId`), the script runs from the top again, but:
|
||||
|
||||
```python
|
||||
class LocalWorkflowTask:
|
||||
def progress_event(self, ptype, **data): # Phase/subagent/log
|
||||
self.progress.append({"type": ptype, **data})
|
||||
print(f" progress {ptype} ...")
|
||||
1. Compare each `agent()` call, in call order, to the next journal line.
|
||||
2. **Longest unchanged prefix** → cache hits (instant replay).
|
||||
3. At the **first** changed or unfinished call, the prefix breaks.
|
||||
4. **Everything after that runs live** — even if an old key still sits later in the journal.
|
||||
|
||||
That is why real JS workflow runtimes ban `Date.now()`, `Math.random()`, and bare `new Date()`: nondeterministic clocks and dice change prompts or call order, and the notebook no longer matches. This Python demo does not fully sandbox that — still write deterministic scripts.
|
||||
|
||||
```text
|
||||
journal: [A ✓] [B ✓] [C ✓] [D ✓]
|
||||
resume: A hit → B hit → C changed → D runs live (no silent hit on old D)
|
||||
```
|
||||
|
||||
## Storage: Snapshot + Journal for Resuming after Interruptions
|
||||
## Walk the sample: `review-changes`
|
||||
|
||||
The runtime stores each run under `s16_workflow_runtime/.runtime/`: a `<runId>.json` snapshot, `<runId>.output.json` output, `<runId>.journal.jsonl` journal, and `<runId>.lock` coordination file. Every fresh run reserves a new `runId` with exclusive file creation before opening its journal. The run lock stays held through execution and final persistence, so another process cannot resume the same run at the same time. Its snapshot records the workflow name, arguments, and task state; resume validates the saved snapshot and journal before changing either successful artifact.
|
||||
Four review dimensions walk the same two-stage path:
|
||||
|
||||
The journal is the core of checkpointed resume. It records every `agent()` result one line at a time:
|
||||
|
||||
```python
|
||||
class WorkflowJournal:
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
```text
|
||||
correctness ── audit ── verify ──┐
|
||||
security ── audit ── verify ──┤── merge confirmed findings
|
||||
performance ── audit ── verify ──┤
|
||||
style ── audit ── verify ──┘
|
||||
```
|
||||
|
||||
## Resume: Continue by runId and Reuse Everything Unchanged
|
||||
|
||||
Calling the workflow again with `resume_from_run_id` reruns the script, but every `agent()` computes a deterministic semantic key. If that key is present in the journal, it returns the cached result without executing again. Every unchanged call hits the cache; only a changed call and the downstream steps that depend on it actually rerun.
|
||||
|
||||
The key detail is that keys cannot depend on concurrency order. Agents in `parallel` and `pipeline` finish in nondeterministic order. If "the nth completion" became the key, cache entries would map to the wrong calls on the next run. A key therefore uses a stable hash of call content, including type, label, prompt, and schema, rather than a shared counter:
|
||||
|
||||
```python
|
||||
def key(self, kind, label, prompt, schema):
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
# Inside agent():
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
self.task.progress_event("workflow_agent", label=label, status="cached")
|
||||
return cached
|
||||
```
|
||||
|
||||
## Stable Call Keys
|
||||
|
||||
On resume, the runtime must match each current `agent()` call with its earlier journal record. A stable hash gives unchanged workflow code and arguments the same call key. Real model output may vary; when the call content has not changed, resume uses the result already saved in the journal.
|
||||
|
||||
## See It Run
|
||||
|
||||
The sample `review-changes` workflow uses `pipeline` to send each review dimension independently through audit → verify. Interactive mode uses the real API and reads the material to review from `args.changes`. `demo` uses fixed runner data to show pipeline, validation, journal, and resume behavior.
|
||||
1. **Review** — each dimension’s auditor returns structured findings.
|
||||
2. **Verify** — each finding gets an adversarial checker (`parallel` inside the verify stage).
|
||||
3. Keep only findings marked real; sort by severity.
|
||||
|
||||
```python
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
|
||||
async def audit(_v, dimension, _i):
|
||||
out = await ctx.agent(f"Inspect this change for {dimension} issues:\n{changes}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _i):
|
||||
ctx.phase("Verify")
|
||||
verdicts = await ctx.parallel([ # Verify every finding independently
|
||||
(lambda f=f: ctx.agent(f"Verify this finding against the change:\n{changes}\n\n{f}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}"))
|
||||
for f in audited["findings"]])
|
||||
return {"dimension": dimension,
|
||||
"confirmed": [f for f, v in zip(audited["findings"], verdicts) if v and v["isReal"]]}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
...
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"confirmed {len(confirmed)} real finding(s)")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
|
||||
## Changes from s15
|
||||
## How this plugs into s15
|
||||
|
||||
| | s15 Integrated Harness | s16 Workflow Runtime |
|
||||
|--|-----------|---------------------|
|
||||
| Loop | One model-driven loop | Main loop unchanged; a tool runs scripted orchestration |
|
||||
| Who decides the next step | Model decides each round | Script declares the orchestration in advance |
|
||||
| Multiple agents | One-shot s06 subagents | Scripted, resumable calls through an agent-runner boundary |
|
||||
| New mechanisms | — | Script primitives, host registry and tool adapter, task lifecycle, progress events, journal/resume, structured output |
|
||||
s15 is still the host loop. s16 adds one tool: `Workflow`. The model (or you) asks for a saved name; the adapter resolves the registry and runs the script.
|
||||
|
||||
s16 does not replace the main loop. It exposes `Workflow` at the tool layer and starts a local workflow runtime behind it: one saved script coordinates N calls through an agent-runner boundary. An s06 subagent is dispatched once at the model's discretion; s16 turns the orchestration into resumable host code.
|
||||
| | Claude Code / Pi (product) | This teaching CLI |
|
||||
|--|----------------------------|-------------------|
|
||||
| Script language | JavaScript in a sandbox | Python functions you can read |
|
||||
| Dynamic door | Model writes `script` / edits `scriptPath` | Explained in docs; demo uses saved `name` |
|
||||
| Host while running | Background + notification; session stays responsive | `demo` / `resume` run in the foreground for clarity |
|
||||
| Ideas | Same primitives, journal, prefix resume | Teaching model — precise where we simplify |
|
||||
|
||||
## Try It
|
||||
The main loop does not become a workflow engine. It borrows one tool, the way it borrows `bash` or `task`.
|
||||
|
||||
## Try it
|
||||
|
||||
```bash
|
||||
python s16_workflow_runtime/code.py # Both the main model and Workflow agents use the real API
|
||||
python s16_workflow_runtime/code.py demo # Deterministic review-changes fixture and event stream
|
||||
python s16_workflow_runtime/code.py resume # Resume by the last runId; every agent() hits the journal cache
|
||||
python s16_workflow_runtime/code.py # s15 host + Workflow tool (real API)
|
||||
python s16_workflow_runtime/code.py demo # fixed fixture: watch phases + agents
|
||||
python s16_workflow_runtime/code.py resume # same runId; prefix should be all cache hits
|
||||
```
|
||||
|
||||
In the default command, ask the model to read the changes, place that text in `args.changes`, and run the saved `review-changes` workflow. Both the main model and workflow agents use the real API. The `demo` command uses fixed runner data so lifecycle and resume behavior can be observed repeatedly. A resumed demo reports `agents=0 tokens=0` when every call hits the cache.
|
||||
What to watch for:
|
||||
|
||||
## Next
|
||||
- `workflow_phase` lines for Review, then Verify
|
||||
- each `workflow_agent` flip from `done` (first run) to `cached` (full resume)
|
||||
- a short confirmed list at the end; full resume shows `agents=0 tokens=0`
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) uses a smaller, independent loop to check whether a stated goal has been reached and decide whether another turn is needed.
|
||||
## Relative to s15 → next is s17
|
||||
|
||||
<!-- translation-sync: zh@v10, en@v10, ja@v10 -->
|
||||
| | s15 Integrated Harness | s16 Workflow Runtime |
|
||||
|--|------------------------|----------------------|
|
||||
| Loop | One model-driven loop | Same loop; one tool runs a script |
|
||||
| Who decides the next step | Model, each round | Script owns the batch shape |
|
||||
| Multi-agent | One-shot subagents | Scripted, resumable `agent()` calls |
|
||||
| Failure / resume | Conversation memory | Null-isolation + journal prefix |
|
||||
|
||||
**s16 = how a batch runs. s17 = whether the whole goal is done.**
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) asks an independent evaluator: should we stop, or take another turn?
|
||||
|
||||
<!-- translation-sync: zh@v11, en@v11, ja@v11 -->
|
||||
|
|
|
|||
|
|
@ -1,51 +1,57 @@
|
|||
# s16: Workflow Runtime — 模型决定单步，脚本决定编排
|
||||
# s16: Workflow Runtime — 把菜谱写进代码
|
||||
|
||||
[English](README.md) · [中文](README.zh.md) · [日本語](README.ja.md)
|
||||
|
||||
s01 → ... → s14 → [s15](../s15_integrated_harness/) → `s16` → [s17](../s17_goal_loop/)
|
||||
|
||||
> *"一次 tool_use，跑完一整套编排"* — `Workflow` 工具启动一个可恢复的脚本运行时，协调多次 agent 调用。
|
||||
> *“一轮轮聊天，像每隔十秒给厨师发一条短信。Workflow 是厨房能照着做的菜谱。”*
|
||||
>
|
||||
> **Harness 层**: 编排 — 在单 agent 循环之上，执行保存好的多 agent 脚本。
|
||||
> **Harness 层**: 编排 — 在单 agent 循环之上，跑一套多 agent 脚本。
|
||||
|
||||
---
|
||||
|
||||
从 s01 到 s15，每一轮都由模型决定调用哪些工具。工具结果进入 `messages[]` 后，模型再根据更新后的上下文决定下一步。当后续路径取决于上一步发现了什么时，这种方式很合适。
|
||||
想象你和朋友用微信一起做饭。你发“先切洋葱”，等他回，再问“切好了吗？”，然后“热锅……”。一道菜还行；要办二十桌宴席，聊天就成了瓶颈：步骤记丢、反复叮嘱，手机一死还得从头来。
|
||||
|
||||
有些任务会重复一套固定流程。例如代码审查可以同时检查多个维度，再逐条验证发现、合并重复项并按严重程度排序。执行前已经知道步骤及其先后关系，这时宿主需要三样东西：
|
||||
普通“模型当总指挥”的对话就是这样。**Workflow** 是写好的菜谱：厨房（runtime）按谱做，帮手（子 agent）负责判断，中间结果放在台面上的碗里 —— 不塞进群聊记录。
|
||||
|
||||
- **并行**，别一个一个串着等；
|
||||
- **稳定的结果结构**，即使每个 agent 的回答会变化；
|
||||
- **可恢复**，跑到一半断了，已经做完的部分别从头再来。
|
||||
## 问题在哪
|
||||
|
||||
如果这套编排只存在于对话历史里，步骤顺序和检查点也只存在于历史里。保存好的 workflow 把固定流程写进代码，并在 journal 中记录已经完成的调用。
|
||||
从 s01 到 s15，每一轮都由模型决定下一步调用什么工具。当“下一步取决于刚才发现了什么”时，这很合适。
|
||||
|
||||
## 计划写在代码里，不是靠聊天一轮轮凑
|
||||
有些任务的形状事先就知道：
|
||||
|
||||
在 harness 的工具池里加入一个 `Workflow` 工具。宿主注册由 `agent() / parallel() / pipeline() / phase()` 组成的可信脚本。模型只提供保存好的 workflow 名称、参数和可选的续跑 run ID，不会提交可执行代码或元数据。
|
||||
- 按多个维度审查很多文件
|
||||
- 先调研，再验证，再合并
|
||||
- 用同一种方式迁移 N 个模块
|
||||
|
||||
workflow 以一次 `tool_use` 进入主循环。脚本运行时，runtime 会发出生命周期和进度事件，并把每一步写进磁盘上的 journal。脚本结束后，这次调用返回启动信息、结果和任务状态。脚本里的中间结果存在变量里，不会塞进对话历史。下次用 `resume_from_run_id` 重启时，没改过的 `agent()` 会直接使用 journal 中的结果。
|
||||
如果模型只能把计划“记”在 `messages[]` 里，会发生三件事：编排噪音占满上下文、中途计划漂移、崩了就得把做完的活重做一遍。
|
||||
|
||||
你需要并行、稳定的结果形状，以及能续跑。把这三样只寄存在对话历史里，太脆弱。
|
||||
|
||||
## 一句话说清想法
|
||||
|
||||
**把计划写进代码。** 子 agent 仍然负责判断；脚本负责循环、分发和合并。中间结果存在变量里，不进对话。
|
||||
|
||||
|
||||
|
||||
```python
|
||||
SAMPLE_META = {"name": "review-changes", "description": "审查代码改动", "phases": ["Review", "Verify"]}
|
||||
一次 `Workflow` 工具调用启动这次脚本运行。运行中会发出生命周期和进度事件；最后一条工具结果带回启动信息、结果和任务状态。
|
||||
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify) # 每个维度独立走 审计 → 验证
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"确认了 {len(confirmed)} 个真实问题")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
## 两扇门
|
||||
|
||||
## Workflow 工具：一次调用，完成整次运行
|
||||
Claude Code 对“工作流怎么启动”是诚实的：
|
||||
|
||||
`Workflow` 会加入 s15 宿主已有的工具池。用户可以要求运行一个保存好的 workflow，模型也可以在任务匹配已知编排时选择这个工具。适配器会用名称查询宿主管理的 `WORKFLOWS` registry，再把可信的元数据和函数交给运行时；s15 的其他工具仍在同一个循环里可用。
|
||||
| 门 | 你传什么 | 什么时候用 |
|
||||
|----|----------|------------|
|
||||
| **动态（Dynamic）** | 一段编排用的 JavaScript（`script`，或之后的 `scriptPath`） | 模型为**这次任务**现写菜谱 |
|
||||
| **已保存（Saved）** | `name` + `args` | 好用的菜谱放进例如 `.claude/workflows/`，按名字再跑 |
|
||||
|
||||
模型可见的 schema 只接受 `name`、`args` 和 `resume_from_run_id`。名称未知或参数格式错误时，适配器会返回错误工具结果，不会让宿主循环退出。随后运行时校验已经注册的元数据、经过权限检查、注册本地 workflow 任务，并在执行脚本前发出 `async_launched`。进度事件和最终的 `task_notification` 随后到达；调用返回可写入 JSON 的启动信息、结果和任务状态。
|
||||
同一间厨房。动态是“现在写菜谱”，已保存是“从卡片盒里抽一张”。
|
||||
|
||||
**本课是一个 Python 教学运行时。** 用同样的想法，但每行你都能读懂。演示按名字注册一个已保存的 workflow；概念和 Claude Code 的脚本世界一一对应。我们**不会**再说“模型不能提交可执行代码”——那是对 Claude Code 的误述。这里只是不嵌入完整的 JS 解释器。
|
||||
|
||||
```python
|
||||
# 教学适配器：已保存这扇门（name + args）。
|
||||
# Claude Code 还接受 script / scriptPath / resumeFromRunId。
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"input_schema": {
|
||||
|
|
@ -54,192 +60,148 @@ WORKFLOW_TOOL = {
|
|||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
"resumeFromRunId": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
},
|
||||
}
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta, script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
)
|
||||
return {"launched": out["launched"], "result": out["result"],
|
||||
"task": serialize_task(out["task"])}
|
||||
```
|
||||
|
||||
## Workflow 元数据：启动前先校验
|
||||
## 原语：用一次义卖来讲
|
||||
|
||||
每个保存好的 workflow 都会注册一份可信元数据，包含 `name`、`description` 和可选的 `phases`。运行时会在执行 workflow 代码前校验它：`name` 和 `description` 用来标识任务，`phases` 给进度显示分组命名。这些字段属于宿主 registry，不是模型输入。
|
||||
学校义卖要烤很多蛋糕。每张桌子都要：搅拌 → 烘烤 → 装箱。帮手负责尝和判断；菜谱决定顺序。
|
||||
|
||||
注册内容不合法时，运行时会在启动前抛出 `WorkflowInputError`。这和 s12 校验 cron 表达式是一个思路：保存好的 workflow 有问题，就不要等到执行时才发现。
|
||||
| 原语 | 在厨房里的意思 |
|
||||
|------|----------------|
|
||||
| `agent(prompt, {schema, label, phase})` | 请一个帮手做一件事 |
|
||||
| `pipeline(items, *stages)` | **默认。** 每块蛋糕自己走完搅拌→烘烤→装箱。A 在装箱时，B 可能还在搅拌 |
|
||||
| `parallel(thunks)` | 等**所有**托盘都回来 —— 只有下一步真的需要全部结果时才用 |
|
||||
| `phase(title)` | 在进度板上宣布“现在进入烘烤” |
|
||||
| `log(message)` | 喊一句短状态 |
|
||||
| `workflow(name, args)` | 套用一份更小的菜谱（只嵌一层） |
|
||||
| `args` | 这次运行的“食材清单” |
|
||||
| `budget` | 还能烧多少“烤箱分钟”（token） |
|
||||
|
||||
运行时会把 `meta.name` 用在本地产物文件名中，因此还要求它是 1-64 个字符的安全 slug，只能包含字母、数字、`.`、`_`、`-`。
|
||||
默认用 `pipeline`。只有下一步必须凑齐上一阶段全部结果时，才用 `parallel` —— 比如要先尝完所有托盘再写评分表。
|
||||
|
||||
```python
|
||||
def validate_meta(meta):
|
||||
if not isinstance(meta, dict):
|
||||
raise WorkflowInputError("meta 必须是对象字面量")
|
||||
if not meta.get("name") or not meta.get("description"):
|
||||
raise WorkflowInputError("meta 必须包含 name 和 description")
|
||||
if not isinstance(meta["name"], str) or not WORKFLOW_NAME_RE.fullmatch(meta["name"]):
|
||||
raise WorkflowInputError("meta.name 必须是 1-64 字符的安全 slug")
|
||||
if "phases" in meta and (
|
||||
not isinstance(meta["phases"], list)
|
||||
or not all(isinstance(p, str) and p for p in meta["phases"])
|
||||
):
|
||||
raise WorkflowInputError("meta.phases 必须包含非空字符串")
|
||||
return meta
|
||||
# 每个审查维度独立走 审计 → 验证（阶段之间不等齐）。
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
```
|
||||
|
||||
## 编排原语
|
||||
## 让答案机器能读
|
||||
|
||||
脚本收到一个只暴露少量编排原语的 `ExecutionState`，本身不直接读写文件，也不运行 shell。默认交互模式把 `agent()` 接到与宿主相同的真实 API client；每个子 agent 只读取 workflow 参数中提供的内容。`demo` 和单元测试使用 `MockAgentRunner`，便于重复观察事件和 journal。
|
||||
|
||||
| 原语 | 作用 |
|
||||
|------|------|
|
||||
| `agent(prompt, {schema, label, phase})` | 派一个子 agent 干活 |
|
||||
| `parallel(thunks)` | **等齐屏障**：所有任务并行跑完，一起等结果回来 |
|
||||
| `pipeline(items, *stages)` | 每个 item 分阶段跑，**不等齐**，跑完一个往下走一个 |
|
||||
| `phase(title)` | 标记当前进度阶段（更新进度条） |
|
||||
| `log(message)` | 打一行进度日志 |
|
||||
| `workflow(name, args)` | 嵌套子工作流（只支持一层） |
|
||||
|
||||
每个 item 都要独立经过相同步骤时，可以使用 `pipeline`。item A 跑到第 3 阶段时，item B 可能还在第 1 阶段；下一步必须同时使用上一阶段全部结果时，再使用 `parallel` 等待所有调用完成。
|
||||
如果帮手回来写散文，下一阶段就很难把 finding 和 verdict 一一对应。传入 `schema`：运行时要求 JSON、做校验，不对就**重试一次**。再不对，这次调用报错（见下面的空值隔离）。
|
||||
|
||||
```python
|
||||
async def pipeline(self, items, *stages):
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages: # 每个 item 独立跑完所有 stage
|
||||
value = await stage(value, item, idx)
|
||||
return value
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
out = await ctx.agent(
|
||||
f"检查这段变更里有没有{dimension}相关的问题：\n{changes}",
|
||||
schema=FINDINGS_SCHEMA,
|
||||
label=f"audit:{dimension}",
|
||||
)
|
||||
# out 是带 "findings" 的字典，不是一段话
|
||||
```
|
||||
|
||||
## 结构化输出：别让子 agent 回来写散文
|
||||
跟你聊天可以用自然语言；流水线需要接口对得上。
|
||||
|
||||
`agent({schema})` 会要求子 agent 只返回匹配 schema 的 JSON 对象。运行时解析并校验结果，不符合时重试一次。这样下游代码拿到的是对象，不必再从自然语言中提取字段。
|
||||
## 一个帮手失败时
|
||||
|
||||
s05 就说过，工具的参数不能全信；这里是同一个道理反过来：子 agent 的输出也不能全信。加一层校验，不对就给一次机会重试，把不确定性挡在编排层外面。
|
||||
不能因为一个托盘糊了，整支队伍停工。
|
||||
|
||||
- **`parallel`**：失败的 thunk 在该槽位变成 `null` / `None`；整个 gather 不会因此拒绝。
|
||||
- **`pipeline`**：某个 stage 失败时，**该 item** 变成 `null` / `None`，并跳过它后面的 stage；其他 item 继续。
|
||||
|
||||
合并前要小心过滤 —— 常见写法是 `if r` / `.filter(Boolean)`。
|
||||
|
||||
```python
|
||||
run = await asyncio.to_thread(self.runner.run, prompt, schema, label)
|
||||
result = run.value
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok: # 提醒一次重试，再不对就报错
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run, prompt + "\n\n返回合法的 JSON。", schema, label
|
||||
)
|
||||
result = retry.value
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) 输出不合法: {err}")
|
||||
verdicts = await ctx.parallel([...]) # 有些位置可能是 None
|
||||
confirmed = [
|
||||
f for f, v in zip(findings, verdicts)
|
||||
if v and v.get("isReal")
|
||||
]
|
||||
```
|
||||
|
||||
## 任务状态和进度事件
|
||||
## Journal 与续跑
|
||||
|
||||
`LocalWorkflowTask` 维护状态和 token 用量，向外发一条 SDK 风格的事件流：`task_started` → 一串 `task_progress`（包含阶段切换、子 agent 启动和日志输出）→ 最后一个 `task_notification`（完成或失败，带输出文件、agent 数和 token 数）。
|
||||
每次运行都有一个 `runId`。每个 `agent()` 结束后，运行时往磁盘上的 journal 追加一行。把它想成笔记本：按你**召唤**帮手的顺序记，而不是按他们从烤箱回来的先后。
|
||||
|
||||
演示会按顺序打印这些事件，并在最终通知后返回任务状态。
|
||||
续跑时（`resume_from_run_id` / `resumeFromRunId`），脚本仍从开头执行，但是：
|
||||
|
||||
```python
|
||||
class LocalWorkflowTask:
|
||||
def progress_event(self, ptype, **data): # 阶段/子agent/日志
|
||||
self.progress.append({"type": ptype, **data})
|
||||
print(f" 进度 {ptype} ...")
|
||||
1. 按调用顺序，把每次 `agent()` 和下一条 journal 记录比对。
|
||||
2. **最长未改前缀** → 缓存命中（直接回放）。
|
||||
3. 遇到**第一个**改过或未完成的调用，前缀断开。
|
||||
4. **之后全部实跑** —— 即使 journal 更后面还躺着旧 key，也不能偷懒命中。
|
||||
|
||||
所以真正的 JS workflow 运行时会禁止 `Date.now()`、`Math.random()` 和裸的 `new Date()`：不确定的时钟和骰子会改 prompt 或调用顺序，笔记本就对不上了。这个 Python 演示不会完整沙箱这些 —— 但脚本仍应写成确定性的。
|
||||
|
||||
```text
|
||||
journal: [A ✓] [B ✓] [C ✓] [D ✓]
|
||||
续跑: A 命中 → B 命中 → C 改过 → D 实跑（不会悄悄命中旧的 D）
|
||||
```
|
||||
|
||||
## 存储：快照 + journal，断了能续
|
||||
## 跟着示例走：`review-changes`
|
||||
|
||||
运行时把每次运行的数据存在 `s16_workflow_runtime/.runtime/`：快照 `<runId>.json`、输出 `<runId>.output.json`、journal `<runId>.journal.jsonl` 和协调文件 `<runId>.lock`。每次新运行都会在打开 journal 前，用排他式文件创建预留新的 `runId`。整次执行和最终持久化期间都持有 run lock，另一个进程不能同时 resume 同一次运行。快照记录 workflow 名称、参数和任务状态；resume 会先验证已保存的快照和 journal，再改动原有的成功产物。
|
||||
四个审查维度走同一条两阶段路径：
|
||||
|
||||
journal 是断点续跑的核心，它一条一条记下来每个 `agent()` 的结果：
|
||||
|
||||
```python
|
||||
class WorkflowJournal:
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
```text
|
||||
correctness ── 审计 ── 验证 ──┐
|
||||
security ── 审计 ── 验证 ──┤── 合并确认过的问题
|
||||
performance ── 审计 ── 验证 ──┤
|
||||
style ── 审计 ── 验证 ──┘
|
||||
```
|
||||
|
||||
## resume：用 runId 续跑，没改的直接用缓存
|
||||
|
||||
带着 `resume_from_run_id` 再次调用 workflow 时，脚本会重新执行，但每个 `agent()` 都会计算一个确定的语义 key：key 在 journal 里有记录，就直接返回缓存结果；只有改过的调用以及依赖它的后续步骤才会真的运行。
|
||||
|
||||
这里有个关键点：key 不能依赖并发顺序。`parallel` 和 `pipeline` 里 agent 完成的顺序是不确定的，用"第几个完成"当 key，两次跑缓存就对错位了。所以 key 是根据调用内容（类型、标签、prompt、schema）算的稳定哈希，不是一个会竞争的计数器：
|
||||
|
||||
```python
|
||||
def key(self, kind, label, prompt, schema):
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
# agent() 内部：
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
self.task.progress_event("workflow_agent", label=label, status="cached")
|
||||
return cached
|
||||
```
|
||||
|
||||
## 稳定调用键
|
||||
|
||||
续跑时，运行时需要把当前 `agent()` 与 journal 中的旧调用对应起来。稳定哈希让同一份 workflow 和同样的参数产生相同的调用 key。真实模型的回答可以变化；只要调用内容没有变化，resume 就直接使用 journal 中已经保存的结果。
|
||||
|
||||
## 跑起来看看
|
||||
|
||||
示例 workflow `review-changes` 用 `pipeline` 让每个审查维度独立走“审计 → 验证”。默认交互模式使用真实 API，并从 `args.changes` 读取待审查内容；`demo` 使用固定 runner 数据来展示 pipeline、结构校验、journal 和续跑。
|
||||
1. **Review** — 每个维度的审计员返回结构化 findings。
|
||||
2. **Verify** — 每条 finding 交给对抗性检查（在 verify 阶段里用 `parallel`）。
|
||||
3. 只保留被标成真实的问题，再按严重程度排序。
|
||||
|
||||
```python
|
||||
async def sample_workflow(ctx, args):
|
||||
ctx.phase("Review")
|
||||
changes = args.get("changes", "")
|
||||
|
||||
async def audit(_v, dimension, _i):
|
||||
out = await ctx.agent(f"检查这段变更里有没有{dimension}相关的问题：\n{changes}",
|
||||
schema=FINDINGS_SCHEMA, label=f"audit:{dimension}", phase="Review")
|
||||
return {"dimension": dimension, "findings": out["findings"]}
|
||||
|
||||
async def verify(audited, dimension, _i):
|
||||
ctx.phase("Verify")
|
||||
verdicts = await ctx.parallel([ # 每条发现独立做对抗性验证
|
||||
(lambda f=f: ctx.agent(f"根据变更内容验证这条 finding：\n{changes}\n\n{f}",
|
||||
schema=VERDICT_SCHEMA, label=f"verify:{dimension}:{f['title']}"))
|
||||
for f in audited["findings"]])
|
||||
return {"dimension": dimension,
|
||||
"confirmed": [f for f, v in zip(audited["findings"], verdicts) if v and v["isReal"]]}
|
||||
|
||||
results = await ctx.pipeline(DIMENSIONS, audit, verify)
|
||||
...
|
||||
confirmed = [f for r in results if r for f in r["confirmed"]]
|
||||
ctx.log(f"确认了 {len(confirmed)} 个真实问题")
|
||||
return {"confirmed": confirmed}
|
||||
```
|
||||
|
||||
## 相对 s15 的变更
|
||||
## 怎样接到 s15
|
||||
|
||||
| | s15 Agent Harness 集成 | s16 Workflow Runtime |
|
||||
|--|-----------|---------------------|
|
||||
| 循环 | 单个、模型驱动 | 主循环不变；工具背后执行脚本编排 |
|
||||
| 谁决定下一步 | 模型逐轮决定 | 脚本预先写好编排流程 |
|
||||
| 多 agent | s06 子 agent，一次性派出去 | 通过 agent-runner 边界执行脚本化、可续跑的调用 |
|
||||
| 新增机制 | — | 编排原语、宿主 registry 与工具适配器、任务生命周期、进度事件、journal/续跑、结构化输出 |
|
||||
s15 仍是宿主循环。s16 只多一个工具：`Workflow`。模型（或你）给出已保存的名字；适配器查 registry，再跑脚本。
|
||||
|
||||
s16 不替换主循环，它只是在工具层暴露 `Workflow`，背后启动一个本地 workflow 运行时：一份保存好的脚本通过 agent-runner 边界协调 N 次调用。s06 的子 agent 是模型临场派一次；s16 把编排写成可续跑的宿主代码。
|
||||
| | Claude Code / Pi（产品） | 本课教学 CLI |
|
||||
|--|--------------------------|--------------|
|
||||
| 脚本语言 | 沙箱里的 JavaScript | 可读的 Python 函数 |
|
||||
| 动态门 | 模型写 `script` / 改 `scriptPath` | 文档说明；演示走已保存的 `name` |
|
||||
| 运行时宿主 | 后台 + 通知，会话保持可响应 | `demo` / `resume` 前台跑，方便观察 |
|
||||
| 想法 | 同一套原语、journal、前缀续跑 | 教学模型 —— 简化处会说清楚 |
|
||||
|
||||
主循环不会变成 workflow 引擎。它只是多借一把工具，就像借 `bash` 或 `task` 一样。
|
||||
|
||||
## 试一下
|
||||
|
||||
```bash
|
||||
python s16_workflow_runtime/code.py # 主模型和 Workflow 子 agent 都使用真实 API
|
||||
python s16_workflow_runtime/code.py demo # 运行确定性的 review-changes 测试数据并观察事件流
|
||||
python s16_workflow_runtime/code.py resume # 用上次的 runId 续跑，每个 agent() 都命中 journal 缓存
|
||||
python s16_workflow_runtime/code.py # s15 宿主 + Workflow 工具（真实 API）
|
||||
python s16_workflow_runtime/code.py demo # 固定数据：观察阶段和 agent
|
||||
python s16_workflow_runtime/code.py resume # 同一个 runId；前缀应全部缓存命中
|
||||
```
|
||||
|
||||
默认命令里，可以先让模型读取改动，再把内容放进 `args.changes` 并运行保存好的 `review-changes` workflow。主模型和 workflow 子 agent 都使用真实 API。`demo` 命令使用固定 runner 数据，便于重复观察生命周期和续跑；续跑命中全部缓存时显示 `agents=0 tokens=0`。
|
||||
留意这些：
|
||||
|
||||
## 接下来
|
||||
- `workflow_phase`：先 Review，再 Verify
|
||||
- 每个 `workflow_agent`：第一次是 `done`，完整续跑变成 `cached`
|
||||
- 结尾有一份简短的确认列表；全命中续跑显示 `agents=0 tokens=0`
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) 会使用一个更小、独立的循环检查既定目标是否已经达成，并据此决定是否还需要下一轮。
|
||||
## 相对 s15 → 下一站 s17
|
||||
|
||||
<!-- translation-sync: zh@v10, en@v10, ja@v10 -->
|
||||
| | s15 Agent Harness 集成 | s16 Workflow Runtime |
|
||||
|--|------------------------|----------------------|
|
||||
| 循环 | 单个、模型驱动 | 同一循环；一个工具跑脚本 |
|
||||
| 谁决定下一步 | 模型逐轮决定 | 脚本规定整批形状 |
|
||||
| 多 agent | 一次性子 agent | 可脚本化、可续跑的 `agent()` |
|
||||
| 失败 / 续跑 | 靠对话记忆 | 空值隔离 + journal 前缀 |
|
||||
|
||||
**s16 = 一批活怎么跑。s17 = 整个目标算不算做完。**
|
||||
|
||||
[s17 Goal Loop](../s17_goal_loop/) 会问一个独立判断器：该停，还是再来一轮？
|
||||
|
||||
<!-- translation-sync: zh@v11, en@v11, ja@v11 -->
|
||||
|
|
|
|||
|
|
@ -1,6 +1,12 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
s16: Workflow Runtime - run a saved orchestration through one tool call.
|
||||
s16: Workflow Runtime — a teaching model of Claude Code Dynamic Workflows.
|
||||
|
||||
Claude Code's Workflow tool accepts script / scriptPath / name / args /
|
||||
resumeFromRunId. The model can write a JavaScript orchestration script for the
|
||||
task (dynamic), or rerun a saved one by name. This lesson is a small Python
|
||||
runtime that shows the same ideas line-by-line. The demo registers one saved
|
||||
workflow by name; we do not embed a JS interpreter.
|
||||
|
||||
Run:
|
||||
python s16_workflow_runtime/code.py
|
||||
|
|
@ -319,15 +325,32 @@ RUNNER_FACTORY = MockAgentRunner
|
|||
|
||||
# -- Journal --
|
||||
class WorkflowJournal:
|
||||
"""Append-only <runId>.journal.jsonl. On resume, agent() calls whose
|
||||
semantic key is already present are replayed from cache instead of re-run."""
|
||||
"""Append-only <runId>.journal.jsonl with longest-unchanged-prefix resume.
|
||||
|
||||
Replay walks agent() calls in *invocation* order (the contract Claude Code
|
||||
and mature Pi ports use). Cache hits continue only while each call's
|
||||
semantic key matches the next journal entry. The first mismatch or missing
|
||||
entry breaks the prefix: every later call runs live, even if an older key
|
||||
still exists further down the journal.
|
||||
|
||||
Why real JS runtimes ban Date.now() / Math.random() / bare new Date(): those
|
||||
make call order or prompts nondeterministic, so resume cannot match the
|
||||
journal. This Python teaching runtime does not sandbox that — write
|
||||
deterministic scripts anyway.
|
||||
"""
|
||||
|
||||
def __init__(self, run_id, resume, store=None):
|
||||
store = STORE if store is None else store
|
||||
store.mkdir(parents=True, exist_ok=True)
|
||||
self.path = store / f"{run_id}.journal.jsonl"
|
||||
self.resume = resume
|
||||
self.entries = []
|
||||
self.cache = {}
|
||||
self._cursor = 0 # next call-order index to assign
|
||||
self._write_cursor = 0 # next index to flush to disk
|
||||
self._pending = {} # idx -> (key, value) awaiting in-order flush
|
||||
self._prefix_broken = False
|
||||
self._did_truncate = False
|
||||
if resume:
|
||||
if not self.path.exists():
|
||||
raise WorkflowInputError(f"resume journal not found for {run_id}")
|
||||
|
|
@ -344,24 +367,54 @@ class WorkflowJournal:
|
|||
raise WorkflowInputError(
|
||||
f"invalid resume journal record at line {line_number}"
|
||||
) from exc
|
||||
self.entries.append({"key": rec["key"], "value": rec["value"]})
|
||||
self.cache[rec["key"]] = rec["value"]
|
||||
self._f = self.path.open("a")
|
||||
else:
|
||||
self._f = self.path.open("w") # fresh run truncates
|
||||
|
||||
def key(self, kind, label, prompt, schema):
|
||||
# Deterministic semantic key, independent of concurrency order, so a
|
||||
# parallel/pipeline call gets the same key on resume.
|
||||
# Content hash identifies *this* call. Prefix matching uses call order;
|
||||
# the hash must stay stable across resume (not Python's salted hash()).
|
||||
basis = f"{kind}|{label}|{prompt}|{json.dumps(schema, sort_keys=True)}"
|
||||
return f"{kind}-{_stable_hash(basis) % 10**10:010d}"
|
||||
|
||||
def cached(self, key):
|
||||
return self.cache.get(key, MISS)
|
||||
def try_replay(self, key):
|
||||
"""Reserve the next call-order slot. Return (cached_value_or_MISS, idx)."""
|
||||
idx = self._cursor
|
||||
self._cursor += 1
|
||||
if self._prefix_broken or not self.resume:
|
||||
return MISS, idx
|
||||
if idx >= len(self.entries) or self.entries[idx]["key"] != key:
|
||||
self._prefix_broken = True
|
||||
return MISS, idx
|
||||
return self.entries[idx]["value"], idx
|
||||
|
||||
def record(self, key, value):
|
||||
self._f.write(json.dumps({"key": key, "value": value}) + "\n")
|
||||
self._f.flush()
|
||||
self.cache[key] = value
|
||||
def record(self, idx, key, value):
|
||||
"""Record a live result at call-order index idx; flush in order."""
|
||||
if self.resume and self._prefix_broken and not self._did_truncate:
|
||||
# Keep only the unchanged prefix; rewrite the file from there.
|
||||
self.entries = self.entries[:idx]
|
||||
self._f.close()
|
||||
self._f = self.path.open("w")
|
||||
for rec in self.entries:
|
||||
self._f.write(json.dumps({"key": rec["key"], "value": rec["value"]}) + "\n")
|
||||
self._f.flush()
|
||||
self._write_cursor = len(self.entries)
|
||||
self._did_truncate = True
|
||||
|
||||
self._pending[idx] = (key, value)
|
||||
while self._write_cursor in self._pending:
|
||||
k, v = self._pending.pop(self._write_cursor)
|
||||
self._f.write(json.dumps({"key": k, "value": v}) + "\n")
|
||||
self._f.flush()
|
||||
rec = {"key": k, "value": v}
|
||||
if self._write_cursor < len(self.entries):
|
||||
self.entries[self._write_cursor] = rec
|
||||
else:
|
||||
self.entries.append(rec)
|
||||
self.cache[k] = v
|
||||
self._write_cursor += 1
|
||||
|
||||
def close(self):
|
||||
self._f.close()
|
||||
|
|
@ -439,6 +492,9 @@ class ExecutionState:
|
|||
self._phase = None
|
||||
self._phases_seen = set()
|
||||
self._limits = limits or ExecutionLimits()
|
||||
# Serializes call-order tickets so parallel agent() invocations still
|
||||
# get a stable prefix position (creation order, not completion order).
|
||||
self._order_lock = asyncio.Lock()
|
||||
|
||||
def phase(self, title):
|
||||
"""Start a phase; subsequent agent()s group under it. Upsert: emitting the
|
||||
|
|
@ -453,69 +509,125 @@ class ExecutionState:
|
|||
self.task.progress_event("workflow_log", message=message)
|
||||
|
||||
async def agent(self, prompt, schema=None, label=None, phase=None):
|
||||
"""Spawn one subagent. With a schema, force StructuredOutput + validate
|
||||
(retry once). On resume, a cached key short-circuits the run."""
|
||||
"""Spawn one subagent. With a schema, validate (+ one retry).
|
||||
|
||||
Resume uses longest unchanged prefix in agent() call order: hits continue
|
||||
until the first changed/missing call; everything after runs live.
|
||||
"""
|
||||
label = label or (prompt[:24] + "...")
|
||||
self._limits.claim_agent()
|
||||
if self.budget.remaining() <= 0:
|
||||
raise WorkflowInputError("token budget exceeded")
|
||||
|
||||
key = self.journal.key("agent", label, prompt, schema)
|
||||
cached = self.journal.cached(key)
|
||||
if cached is not MISS:
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(cached)
|
||||
if not ok:
|
||||
raise WorkflowInputError(
|
||||
f"cached agent output failed schema validation: {err}"
|
||||
)
|
||||
self.task.progress_event("workflow_agent", label=label,
|
||||
phase=phase or self._phase, status="cached")
|
||||
return cached
|
||||
|
||||
async with self._limits.semaphore:
|
||||
run = await asyncio.to_thread(
|
||||
self.runner.run, prompt, schema, label
|
||||
)
|
||||
result = run.value
|
||||
tokens = run.tokens
|
||||
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run,
|
||||
prompt + "\n\nReturn valid JSON.",
|
||||
schema,
|
||||
label,
|
||||
async with self._order_lock:
|
||||
cached, call_idx = self.journal.try_replay(key)
|
||||
if cached is not MISS:
|
||||
# None is a recorded failure (null-isolation); replay it as-is.
|
||||
if cached is not None and schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(cached)
|
||||
if not ok:
|
||||
raise WorkflowInputError(
|
||||
f"cached agent output failed schema validation: {err}"
|
||||
)
|
||||
self.task.progress_event(
|
||||
"workflow_agent",
|
||||
label=label,
|
||||
phase=phase or self._phase,
|
||||
status="cached",
|
||||
)
|
||||
result = retry.value
|
||||
tokens += retry.tokens
|
||||
return cached
|
||||
|
||||
result = None
|
||||
tokens = 0
|
||||
live_error = None
|
||||
try:
|
||||
async with self._limits.semaphore:
|
||||
run = await asyncio.to_thread(
|
||||
self.runner.run, prompt, schema, label
|
||||
)
|
||||
result = run.value
|
||||
tokens = run.tokens
|
||||
|
||||
if schema is not None:
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(f"agent({{schema}}) invalid output: {err}")
|
||||
retry = await asyncio.to_thread(
|
||||
self.runner.run,
|
||||
prompt + "\n\nReturn valid JSON.",
|
||||
schema,
|
||||
label,
|
||||
)
|
||||
result = retry.value
|
||||
tokens += retry.tokens
|
||||
ok, err = SimpleJsonSchema(schema).validate(result)
|
||||
if not ok:
|
||||
raise WorkflowInputError(
|
||||
f"agent({{schema}}) invalid output: {err}"
|
||||
)
|
||||
|
||||
self.budget.add(tokens)
|
||||
self.task.usage["agents"] += 1
|
||||
self.task.usage["tokens"] += tokens
|
||||
except Exception as exc:
|
||||
# Fill the call-order slot with null so later parallel records can
|
||||
# flush; parallel/pipeline turn the raise into a null slot.
|
||||
result = None
|
||||
live_error = exc
|
||||
|
||||
async with self._order_lock:
|
||||
self.journal.record(call_idx, key, result)
|
||||
|
||||
if live_error is not None:
|
||||
self.task.progress_event(
|
||||
"workflow_agent",
|
||||
label=label,
|
||||
phase=phase or self._phase,
|
||||
status="null",
|
||||
)
|
||||
raise live_error
|
||||
|
||||
self.budget.add(tokens)
|
||||
self.task.usage["agents"] += 1
|
||||
self.task.usage["tokens"] += tokens
|
||||
self.journal.record(key, result)
|
||||
self.task.progress_event("workflow_agent", label=label,
|
||||
phase=phase or self._phase, status="done")
|
||||
return result
|
||||
|
||||
async def parallel(self, thunks):
|
||||
"""BARRIER: run all thunks concurrently and fail if any thunk fails."""
|
||||
return await asyncio.gather(*[thunk() for thunk in thunks])
|
||||
"""BARRIER: run all thunks concurrently; wait for every result.
|
||||
|
||||
A failing thunk becomes None in that slot — the gather itself does not
|
||||
reject. Filter with care (e.g. [x for x in results if x]).
|
||||
"""
|
||||
async def isolate(thunk):
|
||||
try:
|
||||
return await thunk()
|
||||
except Exception as exc:
|
||||
self.log(f"parallel step → null ({type(exc).__name__})")
|
||||
return None
|
||||
|
||||
return await asyncio.gather(*[isolate(thunk) for thunk in thunks])
|
||||
|
||||
async def pipeline(self, items, *stages):
|
||||
"""Per-item staged flow, NO barrier between stages: item A can be in
|
||||
stage 3 while item B is still in stage 1. Each stage gets
|
||||
(prev_result, original_item, index). A throwing stage fails the workflow."""
|
||||
(prev_result, original_item, index).
|
||||
|
||||
A failing stage drops that item to None and skips its remaining stages;
|
||||
other items keep going.
|
||||
"""
|
||||
async def run_item(item, idx):
|
||||
value = item
|
||||
for stage in stages:
|
||||
value = await stage(value, item, idx)
|
||||
try:
|
||||
value = await stage(value, item, idx)
|
||||
except Exception as exc:
|
||||
self.log(
|
||||
f"pipeline item {idx} → null ({type(exc).__name__})"
|
||||
)
|
||||
return None
|
||||
if value is None:
|
||||
return None
|
||||
return value
|
||||
|
||||
return await asyncio.gather(*[run_item(it, i) for i, it in enumerate(items)])
|
||||
|
||||
async def workflow(self, name, args=None):
|
||||
|
|
@ -713,15 +825,22 @@ async def sample_workflow(ctx, args):
|
|||
# Saved workflow registry
|
||||
WORKFLOWS = {SAMPLE_META["name"]: (SAMPLE_META, sample_workflow)}
|
||||
|
||||
# Teaching adapter: saved-workflow path (name + args). Claude Code also accepts
|
||||
# script / scriptPath for dynamic JS the model writes; we keep this surface
|
||||
# small and map the same resume idea via resume_from_run_id / resumeFromRunId.
|
||||
WORKFLOW_TOOL = {
|
||||
"name": "Workflow",
|
||||
"description": "Run a saved workflow by name. Pass input in args.",
|
||||
"description": (
|
||||
"Run a saved workflow by name. Pass input in args. "
|
||||
"Optional resume_from_run_id (alias: resumeFromRunId) continues a prior run."
|
||||
),
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"args": {"type": "object"},
|
||||
"resume_from_run_id": {"type": "string"},
|
||||
"resumeFromRunId": {"type": "string"},
|
||||
},
|
||||
"required": ["name"],
|
||||
"additionalProperties": False,
|
||||
|
|
@ -741,20 +860,34 @@ def serialize_task(task):
|
|||
}
|
||||
|
||||
|
||||
async def run_workflow(name, args=None, resume_from_run_id=None):
|
||||
"""Model-facing adapter: resolve trusted code from the host registry."""
|
||||
async def run_workflow(
|
||||
name,
|
||||
args=None,
|
||||
resume_from_run_id=None,
|
||||
resumeFromRunId=None,
|
||||
):
|
||||
"""Model-facing adapter: resolve a saved workflow from the host registry.
|
||||
|
||||
Claude Code's dynamic door passes script/scriptPath instead of name; this
|
||||
teaching runtime stays on the saved-name path so every line stays readable.
|
||||
"""
|
||||
if not isinstance(name, str):
|
||||
raise WorkflowInputError("workflow name must be a string")
|
||||
if name not in WORKFLOWS:
|
||||
raise WorkflowInputError(f"unknown workflow '{name}'")
|
||||
if args is not None and not isinstance(args, dict):
|
||||
raise WorkflowInputError("workflow args must be an object")
|
||||
if resume_from_run_id and resumeFromRunId and resume_from_run_id != resumeFromRunId:
|
||||
raise WorkflowInputError(
|
||||
"resume_from_run_id and resumeFromRunId disagree"
|
||||
)
|
||||
resume_id = resume_from_run_id or resumeFromRunId
|
||||
meta, script_fn = WORKFLOWS[name]
|
||||
out = await WorkflowTool().call(
|
||||
meta,
|
||||
script_fn,
|
||||
args=args,
|
||||
resume_from_run_id=resume_from_run_id,
|
||||
resume_from_run_id=resume_id,
|
||||
)
|
||||
return {
|
||||
"launched": out["launched"],
|
||||
|
|
@ -814,9 +947,14 @@ async def run_demo(argv):
|
|||
if not resume_id:
|
||||
print("nothing to resume; run `python code.py demo` first.")
|
||||
return
|
||||
print(f"resuming {resume_id}; unchanged agent() calls use the journal cache\n")
|
||||
print(
|
||||
f"resuming {resume_id}\n"
|
||||
"longest unchanged agent() prefix → cache hit; "
|
||||
"first change breaks the prefix\n"
|
||||
)
|
||||
else:
|
||||
print("launching workflow `review-changes`\n")
|
||||
print("launching saved workflow `review-changes`")
|
||||
print("watch phases: Review → Verify, then a confirmed list\n")
|
||||
|
||||
out = await WORKFLOW_HANDLERS["Workflow"](
|
||||
name="review-changes",
|
||||
|
|
|
|||
|
|
@ -117,8 +117,8 @@ def test_workflow_runtime_enforces_budget_and_shared_agent_cap(
|
|||
async def fail_stage(_value, _item, _index):
|
||||
raise RuntimeError("stage failed")
|
||||
|
||||
with pytest.raises(RuntimeError, match="stage failed"):
|
||||
await state.pipeline(["item"], fail_stage)
|
||||
# Null-isolation: a failing stage drops that item, not the whole run.
|
||||
assert await state.pipeline(["item"], fail_stage) == [None]
|
||||
|
||||
try:
|
||||
asyncio.run(run())
|
||||
|
|
@ -482,16 +482,139 @@ def test_workflow_default_entry_extends_the_real_s15_host(
|
|||
)
|
||||
|
||||
|
||||
def test_workflow_tool_adapter_rejects_model_supplied_code() -> None:
|
||||
def test_workflow_tool_adapter_uses_saved_name_surface() -> None:
|
||||
"""Teaching adapter stays on name/args; Claude Code also has script/scriptPath."""
|
||||
workflow = load_lesson(
|
||||
"workflow_schema_test", ROOT / "s16_workflow_runtime" / "code.py"
|
||||
)
|
||||
properties = workflow.WORKFLOW_TOOL["input_schema"]["properties"]
|
||||
|
||||
assert set(properties) == {"name", "args", "resume_from_run_id"}
|
||||
assert set(properties) == {
|
||||
"name",
|
||||
"args",
|
||||
"resume_from_run_id",
|
||||
"resumeFromRunId",
|
||||
}
|
||||
assert "description" not in properties
|
||||
assert "script" not in properties
|
||||
assert "scriptPath" not in properties
|
||||
with pytest.raises(workflow.WorkflowInputError, match="name must be a string"):
|
||||
asyncio.run(workflow.run_workflow({"name": "review-changes"}))
|
||||
with pytest.raises(workflow.WorkflowInputError, match="unknown workflow"):
|
||||
asyncio.run(workflow.run_workflow("missing"))
|
||||
|
||||
|
||||
def test_parallel_and_pipeline_isolate_failures(tmp_path: Path) -> None:
|
||||
workflow = load_lesson(
|
||||
"workflow_null_isolation_test", ROOT / "s16_workflow_runtime" / "code.py"
|
||||
)
|
||||
journal = workflow.WorkflowJournal(
|
||||
"wf_null-iso_0001", resume=False, store=tmp_path
|
||||
)
|
||||
task = workflow.LocalWorkflowTask("task", "wf_null-iso_0001", {})
|
||||
state = workflow.ExecutionState(
|
||||
task, journal, workflow.MockAgentRunner(), workflow.Budget(), {}
|
||||
)
|
||||
|
||||
async def ok():
|
||||
return await state.agent("ok", label="ok")
|
||||
|
||||
async def boom():
|
||||
raise RuntimeError("helper crashed")
|
||||
|
||||
async def stage_ok(value, _item, _index):
|
||||
return f"ok:{value}"
|
||||
|
||||
async def stage_fail(value, _item, _index):
|
||||
if value == "bad":
|
||||
raise RuntimeError("stage crashed")
|
||||
return f"next:{value}"
|
||||
|
||||
async def stage_later(value, _item, _index):
|
||||
return f"final:{value}"
|
||||
|
||||
async def run():
|
||||
parallel_out = await state.parallel([ok, boom, ok])
|
||||
pipeline_out = await state.pipeline(
|
||||
["good", "bad", "also"], stage_fail, stage_later
|
||||
)
|
||||
return parallel_out, pipeline_out
|
||||
|
||||
try:
|
||||
parallel_out, pipeline_out = asyncio.run(run())
|
||||
finally:
|
||||
journal.close()
|
||||
|
||||
assert parallel_out[1] is None
|
||||
assert parallel_out[0] is not None and parallel_out[2] is not None
|
||||
assert pipeline_out == ["final:next:good", None, "final:next:also"]
|
||||
|
||||
|
||||
def test_resume_uses_longest_unchanged_prefix(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
workflow = load_lesson(
|
||||
"workflow_prefix_resume_test", ROOT / "s16_workflow_runtime" / "code.py"
|
||||
)
|
||||
monkeypatch.setattr(workflow, "STORE", tmp_path)
|
||||
run_id = "wf_prefix-test_0000000000001a7b"
|
||||
monkeypatch.setattr(workflow, "create_run_id", lambda _meta: run_id)
|
||||
calls: list[str] = []
|
||||
|
||||
class TrackingRunner:
|
||||
def run(self, prompt, schema=None, label=None):
|
||||
calls.append(label or prompt)
|
||||
return workflow.RunnerOutput({"label": label, "prompt": prompt}, 1)
|
||||
|
||||
monkeypatch.setattr(workflow, "RUNNER_FACTORY", TrackingRunner)
|
||||
meta = {"name": "prefix-test", "description": "prefix resume"}
|
||||
|
||||
async def script_v1(ctx, _args):
|
||||
a = await ctx.agent("one", label="a")
|
||||
b = await ctx.agent("two", label="b")
|
||||
c = await ctx.agent("three", label="c")
|
||||
return [a, b, c]
|
||||
|
||||
async def script_v2(ctx, _args):
|
||||
# a unchanged, b's prompt changes → prefix breaks; c must run live
|
||||
# even though an old "c" key exists further down the journal.
|
||||
a = await ctx.agent("one", label="a")
|
||||
b = await ctx.agent("two-changed", label="b")
|
||||
c = await ctx.agent("three", label="c")
|
||||
return [a, b, c]
|
||||
|
||||
first = asyncio.run(workflow.WorkflowTool().call(meta, script_v1))
|
||||
assert first["task"].status == "completed"
|
||||
assert calls == ["a", "b", "c"]
|
||||
|
||||
calls.clear()
|
||||
resumed = asyncio.run(
|
||||
workflow.WorkflowTool().call(
|
||||
meta, script_v2, resume_from_run_id=run_id
|
||||
)
|
||||
)
|
||||
assert resumed["task"].status == "completed"
|
||||
assert calls == ["b", "c"], "after the first miss, later calls must run live"
|
||||
assert resumed["result"][0] == {"label": "a", "prompt": "one"}
|
||||
assert resumed["result"][1]["prompt"] == "two-changed"
|
||||
assert resumed["task"].usage["agents"] == 2
|
||||
|
||||
|
||||
def test_resume_from_run_id_camel_case_alias(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
workflow = load_lesson(
|
||||
"workflow_resume_alias_test", ROOT / "s16_workflow_runtime" / "code.py"
|
||||
)
|
||||
monkeypatch.setattr(workflow, "STORE", tmp_path)
|
||||
first = asyncio.run(
|
||||
workflow.run_workflow("review-changes", {"budget": None})
|
||||
)
|
||||
resumed = asyncio.run(
|
||||
workflow.run_workflow(
|
||||
"review-changes",
|
||||
resumeFromRunId=first["task"]["runId"],
|
||||
)
|
||||
)
|
||||
assert resumed["task"]["status"] == "completed"
|
||||
assert resumed["task"]["usage"]["agents"] == 0
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue