mirror of
https://github.com/musistudio/claude-code-router.git
synced 2026-10-19 17:03:59 +00:00
Fix October routing and gateway issues and remove dependency patches
Preserve Windows Pi dispatch arguments, serialize raw trace adoption, recover unexpected core exits without overriding manual stop/start, and avoid unused token estimation and repeated route script body copies. Preserve custom token counts and select the configured protocol for bare runtime provider IDs. Remove all gateway dependency patches and build directly from upstream commit 0a2c0abd48d2144ed0ac2ce4d954a661c51ffcb1. Include reproducing regressions and update desktop/Docker release packaging and development records. Validation: CCR targeted regressions 270 passed, 2 Windows skips; upstream full suite 931 passed, 7 skips; bundled HTTP/SSE/SQLite/process tests 4 passed. Type checks and both production builds passed.
This commit is contained in:
parent
499b7f6310
commit
915c5f8e9a
30 changed files with 1029 additions and 654 deletions
|
|
@ -1,13 +1,13 @@
|
|||
name: Prepare release gateway
|
||||
description: Build the pinned gateway with CCR's regression fixes for release artifacts.
|
||||
description: Build and verify upstream gateway source for release artifacts.
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Check out pinned gateway source
|
||||
- name: Check out gateway source
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: The-NeXT-AI/ai-gateway
|
||||
ref: f061ad4371b6cf99d5e248746eac1e4ffdda8980
|
||||
ref: 0a2c0abd48d2144ed0ac2ce4d954a661c51ffcb1
|
||||
path: .tmp/release-gateway
|
||||
|
||||
- name: Set up Node.js
|
||||
|
|
@ -15,20 +15,17 @@ runs:
|
|||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Apply gateway regression fixes
|
||||
shell: bash
|
||||
working-directory: .tmp/release-gateway
|
||||
run: |
|
||||
git apply ../../patches/ai-gateway/0001-fix-tool-ids-and-fusion-traces.patch
|
||||
git apply ../../patches/ai-gateway/0002-complete-native-replay-and-response-status.patch
|
||||
|
||||
- name: Build and verify patched gateway
|
||||
- name: Build and verify gateway
|
||||
shell: bash
|
||||
working-directory: .tmp/release-gateway
|
||||
run: |
|
||||
# Push the upstream fixes before creating a CCR release tag.
|
||||
test -f src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
test -f src/raw-trace.test.ts
|
||||
npm ci
|
||||
npm run typecheck
|
||||
npx vitest run src/gateway/routes.test.ts src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
npm run typecheck:all
|
||||
npx vitest run src/gateway/routes.test.ts src/adapters/builtins/anthropic-tool-id.test.ts src/raw-trace.test.ts
|
||||
npm run build
|
||||
mkdir -p ../../docker/local-ai-gateway
|
||||
npm pack --ignore-scripts --pack-destination ../../docker/local-ai-gateway
|
||||
|
|
|
|||
2
.github/workflows/docker-release.yml
vendored
2
.github/workflows/docker-release.yml
vendored
|
|
@ -54,7 +54,7 @@ jobs:
|
|||
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||
echo "major_minor=${version%.*}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Prepare patched gateway package
|
||||
- name: Prepare gateway package
|
||||
uses: ./.github/actions/prepare-release-gateway
|
||||
|
||||
- name: Set up QEMU
|
||||
|
|
|
|||
26
.github/workflows/release.yml
vendored
26
.github/workflows/release.yml
vendored
|
|
@ -38,19 +38,19 @@ jobs:
|
|||
retention-days: 1
|
||||
|
||||
gateway:
|
||||
name: Build patched gateway
|
||||
name: Build gateway
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check out repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Prepare patched gateway package
|
||||
- name: Prepare gateway package
|
||||
uses: ./.github/actions/prepare-release-gateway
|
||||
|
||||
- name: Upload patched gateway package
|
||||
- name: Upload gateway package
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: release-patched-gateway
|
||||
name: release-gateway
|
||||
path: docker/local-ai-gateway/the-next-ai-ai-gateway-*.tgz
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
|
|
@ -88,13 +88,13 @@ jobs:
|
|||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Download patched gateway package
|
||||
- name: Download gateway package
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: release-patched-gateway
|
||||
name: release-gateway
|
||||
path: .tmp/gateway-package
|
||||
|
||||
- name: Install patched gateway
|
||||
- name: Install gateway
|
||||
shell: bash
|
||||
run: npm install --no-save --package-lock=false .tmp/gateway-package/*.tgz
|
||||
|
||||
|
|
@ -214,13 +214,13 @@ jobs:
|
|||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Download patched gateway package
|
||||
- name: Download gateway package
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: release-patched-gateway
|
||||
name: release-gateway
|
||||
path: .tmp/gateway-package
|
||||
|
||||
- name: Install patched gateway
|
||||
- name: Install gateway
|
||||
shell: bash
|
||||
run: npm install --no-save --package-lock=false .tmp/gateway-package/*.tgz
|
||||
|
||||
|
|
@ -262,13 +262,13 @@ jobs:
|
|||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Download patched gateway package
|
||||
- name: Download gateway package
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: release-patched-gateway
|
||||
name: release-gateway
|
||||
path: .tmp/gateway-package
|
||||
|
||||
- name: Install patched gateway
|
||||
- name: Install gateway
|
||||
shell: bash
|
||||
run: npm install --no-save --package-lock=false .tmp/gateway-package/*.tgz
|
||||
|
||||
|
|
|
|||
41
docs/ai-gateway-development.md
Normal file
41
docs/ai-gateway-development.md
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
# 直接使用上游 ai-gateway
|
||||
|
||||
Gateway 修复和对应测试直接维护在 `/Users/jinhuilee/products/next-ai/gateway`,当前分支为 `feat/v1.0.22`。CCR 直接使用这个上游目录的构建产物。
|
||||
|
||||
上游包含 Anthropic 工具 ID 及历史回放修复、Fusion 流日志与 HTTP 状态修复、raw trace 队列容量限制、原生 Anthropic 虚拟模型流透传,以及对应的三个回归测试文件。
|
||||
|
||||
## 本地构建和验证
|
||||
|
||||
先在上游目录运行:
|
||||
|
||||
```sh
|
||||
cd /Users/jinhuilee/products/next-ai/gateway
|
||||
npm run typecheck
|
||||
npm run typecheck:all
|
||||
npx vitest run src/gateway/routes.test.ts src/adapters/builtins/anthropic-tool-id.test.ts src/raw-trace.test.ts
|
||||
npm run build
|
||||
```
|
||||
|
||||
再在 CCR 目录运行:
|
||||
|
||||
```sh
|
||||
cd /Users/jinhuilee/products/CCR/claude-code-router
|
||||
npm run build:assets
|
||||
node build/test.mjs core
|
||||
CCR_TEST_GATEWAY_ENTRY="$PWD/packages/core/dist/main/next-ai-gateway.js" node build/run-tests.mjs core
|
||||
```
|
||||
|
||||
本机已有构建逻辑默认优先使用 `../../next-ai/gateway`。其他目录布局可通过 `CCR_GATEWAY_SOURCE_DIR` 指定上游源码目录。Docker 的本地打包脚本同样直接构建这个上游目录。
|
||||
|
||||
## 发布顺序
|
||||
|
||||
先推送上游修复,再创建 CCR 发布。`prepare-release-gateway` 直接检出上游修复提交 `0a2c0abd48d2144ed0ac2ce4d954a661c51ffcb1`,检查回归测试文件存在,执行类型检查和三个回归套件,然后打包供桌面及 Docker 发布使用。修复由上游源码提供。
|
||||
|
||||
当前 npm 依赖范围仍为 `^1.0.21`;上游工作区修改不会自动进入注册表中的已发布包。通过 npm 发布 CCR 时,应显式使用修复后的上游源码构建;上游正式发布后再升级注册表依赖。
|
||||
|
||||
## 本次迁移验证
|
||||
|
||||
- 先迁入复现测试,在原上游源码上得到 5 项失败;修改生产源码后三个定向回归套件 163 项全部通过。
|
||||
- 上游全量:931 项通过、7 项跳过、0 失败;生产类型检查、包含测试的类型检查及构建通过。
|
||||
- CCR 无源码路径覆盖的默认构建通过,实际打包产物的 4 项 HTTP / SSE / SQLite / core 恢复回归全部通过。
|
||||
- 发布 YAML、action shell 语法及各平台 artifact 引用检查通过;两个仓库的 diff 检查通过。
|
||||
48
docs/issue-fixes-2026-10.md
Normal file
48
docs/issue-fixes-2026-10.md
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
# 2026 年 10 月未处理 bug 修复记录
|
||||
|
||||
当前分支:`feat/v3.1.3`。范围是 10 月 issue 审查中尚未修复的问题;当前分支已合入的 #1858、#1865、#1866 保持现有实现。
|
||||
|
||||
每项新修复均先增加复现测试,记录失败,然后修改生产代码并验证通过。#1861 按作者要求先验证真实归属,未添加掩盖上游归属的响应 model 改写。
|
||||
|
||||
| 条目 | 修改前的复现 | 修复及最终结果 |
|
||||
| --- | --- | --- |
|
||||
| #1851 | Pi dispatch 参数测试和 Windows wrapper 生成测试均失败;selector 会被传入客户端。 | Windows Pi wrapper 复用现有 CLI 参数处理与 Windows 命令转义;只消费 profile/surface 分隔参数,保留提示词与 CLI 参数。独立 wrapper 中输入 `Pi` 仍是正常提示词。 |
|
||||
| #1852 | 原生 Anthropic 虚拟模型 SSE 只有 message_start、message_delta、message_stop,缺少全部 content_block 事件;跨协议对照场景已通过。 | 同协议原生流使用现有透传路径;跨协议保持转换。源码回归保留思考、签名、文本、工具与用量;实际打包产物通过 JSON/SSE、计费开关和 UTF-8 分片测试。 |
|
||||
| #1867 | 扫描接管与上传并发时上传得到 ENOENT/503;扫描间隔短于静默等待期时损坏 bundle 不过期;阻塞 sink 时 20 个 capture 全部排队;意外退出不重启。合并复核又复现:已开始的恢复任务在手动 stop/start 之后完成,覆盖手动操作。 | 接管与保留操作共享串行队列;不重写相同 manifest,继续 fsync;producer 最多保留 uploaderConcurrency × 3 个 capture,超过时跳过 body capture 并记录原因;重试结束释放容量;core 意外退出以 1、2、4…30 秒退避重启。手动 stop/start 取消待执行恢复,并等待已开始的恢复结束后再清理或启动,两个新增竞态测试先失败后通过。真实进程终止后能恢复健康状态并轮换内部鉴权。 |
|
||||
| #1868 | 构造脚本输入会在主线程 JSON.parse 整个 2 MB body。 | 保留一次 JSON 快照,把字符串发送到 worker,移除主线程整包 parse/深拷贝和 worker 的重复 stringify;仍保持原始快照、冻结、超时及脚本能力隔离。没有增加预过滤配置或新规则类型。 |
|
||||
| #1869 | 没有消费者时仍对完整对话执行两次 token 正则扫描。合并复核又复现:脚本未保留 custom router 设置或缓存的 tokenCount。 | request.tokenCount 惰性计算并缓存;尚未计算的值由脚本在 worker 首次读取时计算,已有值直接传递,保留显式 0 和自定义覆盖。新增兼容性测试先失败后通过。保留条件别名、custom router、已有 provider 路由设置、客户插件的 token 输入及 count_tokens。没有消费者时路由诊断 tokenCount 为 0。 |
|
||||
| #1861 | 在实际 host → core → 两个独立模拟上游路径中检查请求目的地、最终响应和 SQLite。 | Chat Completions / Responses × JSON / SSE 均返回、记录 Backup/backup;未复现独立的响应 model 错误,当前实现无需新响应改写。保留回归测试。 |
|
||||
| #1847(关联 PR) | 核验 #1861 时发现 Responses 的 primary 请求被裸 provider ID shortcut 送到 Chat 路径;既有 PR 的九个协议/credential 场景中七个失败。 | 裸 provider ID 通过协议和 credential 选择;已完整指定的内部 selector 保持有效。更新现有 executor 的公开名称断言为配置的 runtime ID。37 个相关测试通过,真实 fallback 两条腿的协议也正确。 |
|
||||
|
||||
## 最终验证
|
||||
|
||||
- 合并复核后的 CCR 定向回归(增加已运行恢复取消、token 覆盖兼容性,以及全部 router-builtins 场景):272 项,270 通过,2 项 Windows 原生命令执行测试在 macOS 上跳过,0 失败。
|
||||
- Gateway 定向回归:163 项全部通过;直接迁入上游后,全量回归为 938 项,931 通过、7 项既有跳过、0 失败。
|
||||
- CCR 打包产物实际 HTTP / SQLite / managed core 进程回归:4 项全部通过。每项包含多个 JSON/SSE、协议或计费组合断言。
|
||||
- 按 CCR 定向、gateway 全量和打包产物集成回归计(不重复计算 gateway 定向测试):1214 项,1205 通过、9 跳过、0 失败。
|
||||
- CCR 与 gateway 的生产 TypeScript 检查及 gateway 包含测试的完整 TypeScript 检查通过;gateway build 与 CCR `build:assets` 通过。
|
||||
- 所有 gateway 修复及测试已直接迁入 `/Users/jinhuilee/products/next-ai/gateway`;CCR 直接消费上游源码及构建产物。发布 composite action 固定构建上游修复提交 `0a2c0abd48d2144ed0ac2ce4d954a661c51ffcb1` 并验证回归测试。
|
||||
|
||||
## 重现构建
|
||||
|
||||
按 [上游 gateway 开发与发布](./ai-gateway-development.md) 直接构建 gateway,再使用:
|
||||
|
||||
```sh
|
||||
npm run build:assets
|
||||
node build/test.mjs core
|
||||
CCR_TEST_GATEWAY_ENTRY=/absolute/path/to/claude-code-router/packages/core/dist/main/next-ai-gateway.js npm run test:core
|
||||
```
|
||||
|
||||
本次定向运行使用项目安装的 Electron Node runtime,匹配 better-sqlite3 的 ABI;本机系统 Node 的 ABI 与已安装 SQLite 模块不匹配。HTTP 测试仅使用 loopback 模拟上游,没有真实 API 付费调用。Windows wrapper 生成和参数处理已在 macOS 验证;两个原生 cmd.exe 测试需要 Windows 环境。没有进行数小时线上 OOM 压测。
|
||||
|
||||
## 合并建议
|
||||
|
||||
除 #1851 的 Windows 原生命令路径外,当前定向回归及构建未发现剩余阻塞问题。整批合并前应在 Windows 实际运行 Pi dispatch 参数测试与两个 cmd.exe 测试;现有 Windows 发布 job 只构建发布,不会自动运行这些测试。其余修改可以先合并,需要分别提交上游 gateway 修复和 CCR 修改,发布前先推送上游修复,确保构建消费正确版本。
|
||||
|
||||
## 直接迁入上游的验证
|
||||
|
||||
先迁入 routes 和 raw-trace 复现测试,在上游原始生产源码上得到 5 项失败、155 项通过;随后直接修改生产源码并迁入工具 ID 测试,三个回归套件共 163 项全部通过。上游全量测试 931 通过、7 跳过、0 失败;生产与包含测试的完整 TypeScript 检查以及构建通过。
|
||||
|
||||
CCR 默认构建解析到 `/Users/jinhuilee/products/next-ai/gateway`,没有设置源码覆盖路径。使用这个上游构建的最终 CCR 产物,4 项实际 HTTP / SSE / SQLite / core 进程回归全部通过。两库 diff 检查通过,发布 YAML、action shell 语法和各平台 gateway artifact 引用一致。全仓依赖维护配置和文档检查通过,均采用直接构建上游源码的方式。
|
||||
|
||||
上游修复已包含在提交 `0a2c0ab` 中,10 个源码及回归文件与受测版本一致。CCR 修复、复现测试、发布依赖和维护记录随本记录一并提交。本次未执行推送或 GitHub 关闭、合并、评论。
|
||||
|
|
@ -4,6 +4,8 @@
|
|||
|
||||
后续已逐条按原文补测,并发现、修复五处遗漏。最新验收结果、测试数量及未完成事项以 [原 Issue 复现验收](./2026-09-27-verification.md) 为准;下文保留首轮记录。
|
||||
|
||||
2026-10-09 更新:下文所述网关修复和测试已直接迁入上游 gateway,CCR 直接消费上游构建产物。当前构建和发布方式见 [上游 gateway 开发与发布](../ai-gateway-development.md)。
|
||||
|
||||
## 逐项结果
|
||||
|
||||
| Issue | 问题 | 复核时状态与本次处理 |
|
||||
|
|
@ -16,13 +18,13 @@
|
|||
| [#1819](https://github.com/musistudio/claude-code-router/issues/1819) | preset 遮蔽探测成功的协议端点 | 优先级改为手动、detected、preset;探测成功的协议不再生成重复 preset。配置并发保护同时防止旧快照回写丢失能力。 |
|
||||
| [#1810](https://github.com/musistudio/claude-code-router/issues/1810) | OAuth 刷新后仍用旧令牌 | 动态 hook 与静态插件使用不同 key;静态插件不再覆盖认证头;运行请求和 Claude OAuth 探测读取最新凭据。 |
|
||||
| [#1809](https://github.com/musistudio/claude-code-router/issues/1809) | 本地模型供应商和费用归属错误 | 运行时供应商别名映射回配置名称;原始日志路径传递自定义价格。任意本地模型别名没有通用市价,应在模型元数据配置价格。 |
|
||||
| [#1808](https://github.com/musistudio/claude-code-router/issues/1808) | 非法 tool ID 污染 Anthropic 历史 | 底层网关补丁:JSON/SSE 输出及历史回放统一确定性映射,保持调用与结果成对。需采用打过补丁的网关构建。 |
|
||||
| [#1808](https://github.com/musistudio/claude-code-router/issues/1808) | 非法 tool ID 污染 Anthropic 历史 | 底层网关修复:JSON/SSE 输出及历史回放统一确定性映射,保持调用与结果成对。需采用包含修复的上游网关构建。 |
|
||||
| [#1807](https://github.com/musistudio/claude-code-router/issues/1807) | 日志 worker OOM 无限重试 | 写入 worker 限制堆大小;崩溃批次拆分并限制单条重试;修复等待启动后的并发发送;持久化已接受任务的重试预算。 |
|
||||
| [#1806](https://github.com/musistudio/claude-code-router/issues/1806) | Fusion 流式请求没有日志 | 底层网关补齐流式完成后的 raw trace,独立于 billing 开关。需采用打过补丁的网关构建。 |
|
||||
| [#1806](https://github.com/musistudio/claude-code-router/issues/1806) | Fusion 流式请求没有日志 | 底层网关补齐流式完成后的 raw trace,独立于 billing 开关。需采用包含修复的上游网关构建。 |
|
||||
| [#1805](https://github.com/musistudio/claude-code-router/issues/1805) | Fusion 日志重复报告 | 已关闭;由 #1806 覆盖。 |
|
||||
| [#1804](https://github.com/musistudio/claude-code-router/issues/1804) | 回退仍请求失败的主模型 | 当前分支已有 `aad384b3`,复核并运行 upstream-executor 回归。 |
|
||||
| [#1803](https://github.com/musistudio/claude-code-router/issues/1803) | Anthropic 健康检查错误 | 维护者确认 v3.1.1 已修复。 |
|
||||
| [#1801](https://github.com/musistudio/claude-code-router/issues/1801) | 日志状态 0 全部显示失败 | [#1788](https://github.com/musistudio/claude-code-router/pull/1788) 未合并。本次显示未知状态为中性;错误过滤和统计排除未知;保留显式传输错误与 HTTP 错误。底层补丁补全 Fusion HTTP 状态。 |
|
||||
| [#1801](https://github.com/musistudio/claude-code-router/issues/1801) | 日志状态 0 全部显示失败 | [#1788](https://github.com/musistudio/claude-code-router/pull/1788) 未合并。本次显示未知状态为中性;错误过滤和统计排除未知;保留显式传输错误与 HTTP 错误。底层网关修复补全 Fusion HTTP 状态。 |
|
||||
| [#1800](https://github.com/musistudio/claude-code-router/issues/1800) | 热力图底行被裁切 | Issue 已关闭,但 [#1820](https://github.com/musistudio/claude-code-router/pull/1820) 仍开放。本次按实际月份标签高度计算首页与托盘网格尺寸。 |
|
||||
| [#1799](https://github.com/musistudio/claude-code-router/issues/1799) | 上下文超限信息被包装 | 当前分支已有 `503a8c17`,复核并运行 error-detail 回归。 |
|
||||
| [#1798](https://github.com/musistudio/claude-code-router/issues/1798) | autoMode 与 apiKeyHelper 冲突 | 当前分支已有 `503a8c17`,复核并运行 profile-service 回归。 |
|
||||
|
|
@ -31,8 +33,8 @@
|
|||
|
||||
## 交付与限制
|
||||
|
||||
- 网关源代码补丁和构建方式见 [`patches/ai-gateway/README.md`](../../patches/ai-gateway/README.md)。本次已使用补丁构建完整 CCR 生产资源,并用本机模拟上游验证 Fusion JSON/SSE、工具 ID、日志及 HTTP 200 元数据。
|
||||
- 网关补丁尚未进入公开 npm 版本;普通安装依赖并不会自动包含它。发布 CCR 前需要合入并发布网关补丁、升级依赖,或明确使用补丁源码构建。
|
||||
- 网关修复和构建方式见 [上游 gateway 开发与发布](../ai-gateway-development.md)。本次已构建完整 CCR 生产资源,并用本机模拟上游验证 Fusion JSON/SSE、工具 ID、日志及 HTTP 200 元数据。
|
||||
- 网关修复尚未进入公开 npm 版本;普通安装依赖并不会自动包含它。发布 CCR 前需先提交并推送上游修复,使用该源码构建;上游正式发布后再升级依赖。
|
||||
- 插件市场仓库 404 需要维护者变更仓库可见性或提供有效地址,文档无法使远端仓库上线。
|
||||
- 既有历史日志缺失的 HTTP 状态无法可靠还原;显示未知,不伪造成功。供应商与价格修复适用于后续入库记录,没有批量改写用户历史数据库。
|
||||
- Jev 示例在明确启用后才发送最新用户文本摘要到配置的分类服务;测试使用模拟响应,未使用真实 API Key。
|
||||
|
|
@ -45,4 +47,4 @@
|
|||
- 核心最终全量:1069 项,1062 通过、6 项平台/可选环境跳过、1 项已有失败(见下)。日志专项:132 通过、1 项跳过。新增的并发配置、OAuth、端点优先级、重试预算、工具结果配对和 Jev 示例回归均通过。
|
||||
- 全量测试发现一项已有失败:Kimi 配置测试对 `DeepSeek/deepseek-v4-flash` 期望不含 `image_in`,仓库当前模型目录却包含图片输入能力。已在未修改的 `86dfac1c` 快照复现相同失败;未改动产品行为以迎合旧断言。
|
||||
|
||||
最终收尾时,一次重复的生产网关冒烟验证因自动审批服务额度不足而未执行;此前相同网关补丁的生产 JSON/SSE 冒烟已通过,最终生产资源重建也通过。未绕过审批限制。
|
||||
最终收尾时,一次重复的生产网关冒烟验证因自动审批服务额度不足而未执行;此前相同网关修复的生产 JSON/SSE 冒烟已通过,最终生产资源重建也通过。未绕过审批限制。
|
||||
|
|
|
|||
|
|
@ -1,16 +1,18 @@
|
|||
# 原 Issue 复现验收(2026-09-27)
|
||||
|
||||
本次重新读取此前 20 个 Issue 的正文和评论,从报告者的复现条件设计测试。下表的“通过”表示当前源码、指定网关补丁和受控测试环境通过该场景;不代表已发布,也不代表在报告者的真实账号或设备上复测。此前仅凭维护者回复或单元测试得出的结论,以本记录为准。
|
||||
本次重新读取此前 20 个 Issue 的正文和评论,从报告者的复现条件设计测试。下表的“通过”表示当前源码、指定网关修复和受控测试环境通过该场景;不代表已发布,也不代表在报告者的真实账号或设备上复测。此前仅凭维护者回复或单元测试得出的结论,以本记录为准。
|
||||
|
||||
收尾核对日期:2026-09-28。
|
||||
|
||||
2026-10-09 更新:本记录中的网关修复及测试已直接迁入上游 gateway;历史验证结果保留,当前直接消费上游构建产物。构建和发布方式见 [上游 gateway 开发与发布](../ai-gateway-development.md)。
|
||||
|
||||
## 本次测试找到并补齐的遗漏
|
||||
|
||||
| 原 Issue | 补测前失败 | 本次修复与补测后结果 |
|
||||
| --- | --- | --- |
|
||||
| #1807 | 查询 worker 没有堆上限;并发启动可能创建多个查询 worker | 查询 worker 限制堆;等待写入 worker 后重新检查;提前退出拒绝挂起请求。真实 V8 OOM 后查询可恢复。写入 OOM 隔离一条 poison,100 条健康记录继续处理。 |
|
||||
| #1801 | 普通模型 JSON 返回 HTTP 200,最终 SQLite 却为 0 | 第二份网关补丁沿普通转换、工具循环及解析重试传递实际响应状态和头。SQLite 验证为 200。 |
|
||||
| #1808 | 新工具 ID 已净化,但旧的非法历史在原生 Anthropic 直通分支仍返回 400 | 第二份网关补丁在直通请求中成对映射调用和结果,保留 thinking 签名及其他内容。新会话和旧污染历史回放均通过。 |
|
||||
| #1801 | 普通模型 JSON 返回 HTTP 200,最终 SQLite 却为 0 | 网关修复沿普通转换、工具循环及解析重试传递实际响应状态和头。SQLite 验证为 200。 |
|
||||
| #1808 | 新工具 ID 已净化,但旧的非法历史在原生 Anthropic 直通分支仍返回 400 | 网关修复在直通请求中成对映射调用和结果,保留 thinking 签名及其他内容。新会话和旧污染历史回放均通过。 |
|
||||
| #1810 | “获取模型”中的 GET 已更新 token,后续协议探测仍发送占位 token;缓存未感知 token 变化 | 协议探测也经过实时认证;缓存 key 包含 token 哈希。同一网关 PID 内切换凭据,两次 models/protocol 请求都使用新 token,普通 API Key 保持正确。 |
|
||||
| #1819 评论 | 主模型 429、第二次 404、最后模型解析 400,客户端只看到最后一次 | 汇总 CCR 回退历史到最终错误的 attempts、fallbackReason 和 message。新用例在旧实现中只得到 `[400]`,修复后得到 `[429,404,400]`,请求处理流水线验证客户端正文也保留三次失败。 |
|
||||
|
||||
|
|
@ -28,13 +30,13 @@
|
|||
| [#1819](https://github.com/musistudio/claude-code-router/issues/1819) | 同域不同协议路径;preset 不应遮蔽 detected;旧保存不丢能力;链尾错误不掩盖前因 | `unit/providers/provider-preset-utils.test.mjs` 验证排序与手动优先;`recent-issues-wire.test.mjs` 捕获真实 `/backup/api/v3/chat/completions` 与 `/anthropic/v1/messages`;RPC 验证能力保留;error-detail/request-pipeline-query 验证 `[429,404,400]`。通过所列场景。 |
|
||||
| [#1810](https://github.com/musistudio/claude-code-router/issues/1810) | 运行后轮换凭据、不重启、不重新导入;获取模型也跟随 | `integration/gateway/recent-issues-wire.test.mjs` 加载实际静态插件和动态 hook,两次 HTTP 使用不同 token;models 与协议探测更新;普通 Key 对照通过。另有 Keychain 单测。未用真实 OAuth 账号做远端续期。 |
|
||||
| [#1809](https://github.com/musistudio/claude-code-router/issues/1809) | Llama CPP 的 Qwen GGUF 别名能按供应商筛选并计费 | 同一 wire 测试:每次 100 输入/20 输出 token,自定义价格 2/6 美元每百万 token,SQLite 和概览供应商筛选得到每次 `$0.00032`;全供应商数量一致。通过。任意本地别名仍需明确配置价格;未补写历史记录。 |
|
||||
| [#1808](https://github.com/musistudio/claude-code-router/issues/1808) | OpenAI 返回 `functions.Bash:0`,Claude 历史切到原生 Anthropic | JSON/SSE 出站 ID 正则有效;把净化 ID 和旧非法 ID 分别回放到会拒绝非法/不成对 ID 的原生模拟上游。通过,**依赖两份网关补丁,尚未发布依赖版本**。 |
|
||||
| [#1808](https://github.com/musistudio/claude-code-router/issues/1808) | OpenAI 返回 `functions.Bash:0`,Claude 历史切到原生 Anthropic | JSON/SSE 出站 ID 正则有效;把净化 ID 和旧非法 ID 分别回放到会拒绝非法/不成对 ID 的原生模拟上游。通过,**依赖上游网关修复,尚未发布依赖版本**。 |
|
||||
| [#1807](https://github.com/musistudio/claude-code-router/issues/1807) | worker OOM、poison 批次反复重放、重启仍阻塞健康日志 | `integration/observability/request-log-runtime.test.mjs` 真实 OOM、堆上限、100 条健康尾部记录、恢复查询;`unit/observability/raw-trace-sync.test.mjs` 验证重启后预算与 dead letter。通过。未执行数小时生产流量压力测试,不能承诺任意负载无 OOM。 |
|
||||
| [#1806](https://github.com/musistudio/claude-code-router/issues/1806) | 相同入口/上游的 direct/Fusion × JSON/SSE 四组,逐次查库 | wire 测试再交叉 billing true/false,共八组;每次客户端 200、SQLite 新增一条、120 token(usage 放在 finish_reason 后)、成本正确。通过,**需要网关补丁**。 |
|
||||
| [#1806](https://github.com/musistudio/claude-code-router/issues/1806) | 相同入口/上游的 direct/Fusion × JSON/SSE 四组,逐次查库 | wire 测试再交叉 billing true/false,共八组;每次客户端 200、SQLite 新增一条、120 token(usage 放在 finish_reason 后)、成本正确。通过,**需要上游网关修复**。 |
|
||||
| [#1805](https://github.com/musistudio/claude-code-router/issues/1805) | 同上,重复报告 | 同一四组×计费矩阵覆盖。 |
|
||||
| [#1804](https://github.com/musistudio/claude-code-router/issues/1804) | 主模型配额 403 后真实网络是否访问备用上游/模型 | wire 测试加载 CCR 路由插件并运行回退执行器,JSON/SSE 均捕获 `primary/k3` 后访问 `backup/glm-5.3`。通过。 |
|
||||
| [#1803](https://github.com/musistudio/claude-code-router/issues/1803) | 原文 `anthropic/claude-ccr-h467573696f6e2f474c4d20352e3320566973696f6e` 探测 ID | wire 测试保留额度耗尽的 Anthropic 供应商,发送该完整 ID,实际只访问选中的 Backup/glm-5.3 并成功。通过;未启动真实 Claude 桌面端。 |
|
||||
| [#1801](https://github.com/musistudio/claude-code-router/issues/1801) | 成功显示 0/红色,真正传输/HTTP 错误应保留 | 八组 wire 测试确认新增行 200;日志/用量测试区分 unknown、传输错误和 HTTP 错误;UI 中性状态测试通过。历史 0 不伪造为 200。普通路径遗漏已补齐,**需要网关补丁**。 |
|
||||
| [#1801](https://github.com/musistudio/claude-code-router/issues/1801) | 成功显示 0/红色,真正传输/HTTP 错误应保留 | 八组 wire 测试确认新增行 200;日志/用量测试区分 unknown、传输错误和 HTTP 错误;UI 中性状态测试通过。历史 0 不伪造为 200。普通路径遗漏已补齐,**需要上游网关修复**。 |
|
||||
| [#1800](https://github.com/musistudio/claude-code-router/issues/1800) | 宽布局/改变尺寸时七行热力图不得被裁切 | `tests/e2e/recent-issues.spec.ts` 实际 Chrome 测量:首页 1200/1920/900/1440px;托盘构建产物 360/680/420px。七行、非零尺寸与裁切边界检查均通过。 |
|
||||
| [#1799](https://github.com/musistudio/claude-code-router/issues/1799) | 262144 上下文,输入 230462、输出 32000 的原始 vLLM 错误 | `unit/gateway/error-detail.test.mjs` 使用原文错误并断言 Claude 可识别格式,同时覆盖 OpenAI/LiteLLM、最终尝试与无效数值。通过。未让真实 Claude CLI 执行自动 compact。 |
|
||||
| [#1798](https://github.com/musistudio/claude-code-router/issues/1798) | Claude 写回 autoMode:true,重新生成仍导致 apiKeyHelper 失效 | `integration/profiles/profile-service.test.mjs` 生成真实 settings 文件→模拟会话写回→再次 apply,移除布尔 autoMode 并保留 helper;结构化设置保留。通过文件生命周期;未运行 Windows Claude CLI 登录。 |
|
||||
|
|
@ -43,13 +45,13 @@
|
|||
|
||||
## 可复现命令
|
||||
|
||||
先按 [`patches/ai-gateway/README.md`](../../patches/ai-gateway/README.md) 应用两份补丁并构建网关。以下是本机 Electron ABI 对应的运行方式;其他环境可使用与 better-sqlite3 匹配的 Node。
|
||||
先按 [上游 gateway 开发与发布](../ai-gateway-development.md) 直接构建上游网关。以下是本机 Electron ABI 对应的运行方式;其他环境可使用与 better-sqlite3 匹配的 Node。
|
||||
|
||||
```sh
|
||||
node build/test.mjs core ui
|
||||
node build/run-tests.mjs core
|
||||
node build/run-tests.mjs ui
|
||||
CCR_GATEWAY_SOURCE_DIR=/absolute/path/to/patched/gateway npm run build:assets
|
||||
npm run build:assets
|
||||
ELECTRON_RUN_AS_NODE=1 CCR_TEST_GATEWAY_ENTRY="$PWD/packages/core/dist/main/next-ai-gateway.js" \
|
||||
node_modules/electron/dist/Electron.app/Contents/MacOS/Electron --test \
|
||||
.test-dist/core/test/integration/gateway/recent-issues-wire.test.js
|
||||
|
|
@ -64,15 +66,15 @@ wire 用例未设置 `CCR_TEST_GATEWAY_ENTRY` 时会明确跳过,避免普通
|
|||
|
||||
| 检查 | 最终结果 |
|
||||
| --- | --- |
|
||||
| CCR TypeScript 检查、生产资源构建 | 通过;构建显式使用应用两份补丁的网关源码。 |
|
||||
| CCR TypeScript 检查、生产资源构建 | 通过;构建显式使用包含修复的网关源码。 |
|
||||
| 核心全量测试 | 1076 项:1068 通过、7 跳过、1 项既有失败(见下)。 |
|
||||
| UI 单元测试 | 212 项通过。 |
|
||||
| Chrome 浏览器回归 | 3 项通过:真实配置 RPC 并发保存、首页热力图、托盘热力图。 |
|
||||
| 网关 TypeScript 检查、生产构建 | 通过。 |
|
||||
| 网关 routes 与 tool ID 回归 | 158 项通过。 |
|
||||
| HTTP / spool / SQLite 集成回归 | 单独启用后通过;分别验证网关自身构建产物,以及最终 CCR 生产包内的 `next-ai-gateway.js`。覆盖八组用量/成本矩阵及工具历史、回退、编码探测 ID、OAuth 轮换。 |
|
||||
| 补丁可复现性 | 从网关基线 `f061ad4` 的干净快照依次应用两份补丁成功;关键生产文件与受测源码一致。 |
|
||||
| 网关修复验证 | 当时从网关基线 `f061ad4` 的干净快照完成修复验证;修复和受测源码现已直接迁入上游目录。 |
|
||||
|
||||
核心唯一失败为 `profile service writes a multi-model Kimi CLI home that points inference to CCR`:当前模型目录为 `DeepSeek/deepseek-v4-flash` 包含 `image_in`,旧断言要求没有该能力。此前已在未修改的 `86dfac1c` 快照复现相同失败,本轮未修改该行为或断言。7 项跳过包含需要显式指定网关构建产物的 wire 用例,该用例已另行执行通过;Windows 原生用例仍受平台限制。
|
||||
|
||||
结论:所列受控回归场景通过并补齐五处遗漏,不能将全部 20 个 Issue 标记为完全解决。远端 marketplace 仍为 404;Windows 原生执行、Jev 真实效果及表中注明的真实客户端/账号场景尚未验收;网关补丁也尚未发布到依赖版本。
|
||||
结论:所列受控回归场景通过并补齐五处遗漏,不能将全部 20 个 Issue 标记为完全解决。远端 marketplace 仍为 404;Windows 原生执行、Jev 真实效果及表中注明的真实客户端/账号场景尚未验收;上游网关修复也尚未发布到依赖版本。
|
||||
|
|
|
|||
|
|
@ -95,6 +95,10 @@ function botBridge() {
|
|||
|
||||
async function main() {
|
||||
const args = directProfileDispatchArgs(process.argv.slice(2));
|
||||
if (process.env.CCR_PI_WRAPPER === "1") {
|
||||
await runPiCliWrapper(args);
|
||||
return;
|
||||
}
|
||||
if (process.env.CCR_OPENCODE_BOT_WORKER === "1" || args[0] === "opencode-bot-worker") {
|
||||
await runOpenCodeBotWorker(args);
|
||||
return;
|
||||
|
|
@ -132,6 +136,22 @@ function directProfileDispatchArgs(args) {
|
|||
return forwarded;
|
||||
}
|
||||
|
||||
async function runPiCliWrapper(args) {
|
||||
const realCli = expandHome(nonEmptyEnv("CCR_REAL_PI_BIN") || "pi");
|
||||
const child = spawnAgentCli(realCli, [
|
||||
"--provider", nonEmptyEnv("CCR_PI_PROVIDER") || "claude-code-router",
|
||||
"--model", nonEmptyEnv("CCR_PI_MODEL"), ...args
|
||||
], {
|
||||
env: withoutKeys(process.env, ["CCR_PI_WRAPPER", "CCR_REAL_PI_BIN", "CCR_PI_PROVIDER", "CCR_PI_MODEL", "CCR_CLI_DIRECT_PROFILE_DISPATCH"]),
|
||||
stdio: "inherit"
|
||||
});
|
||||
child.on("error", (error) => {
|
||||
process.stderr.write("Failed to start " + realCli + ": " + formatError(error) + "\n");
|
||||
});
|
||||
const exit = await waitForChildResult(child);
|
||||
process.exitCode = exit.exitCode;
|
||||
}
|
||||
|
||||
async function runClaudeCodeCliWrapper(args) {
|
||||
const realCli = expandHome(nonEmptyEnv("CCR_REAL_CLAUDE_CODE_BIN") || nonEmptyEnv("CCR_CLAUDE_CODE_BIN") || nonEmptyEnv("CODEXL_CLAUDE_CODE_BIN") || "claude");
|
||||
const realArgs = claudeCodeCliWrapperArgs(args);
|
||||
|
|
|
|||
|
|
@ -115,6 +115,10 @@ class GatewayService {
|
|||
getGlobalBillingConfig: () => pluginService.getCoreGatewayConfig().billing
|
||||
});
|
||||
private child?: ChildProcess;
|
||||
private coreRestartAttempts = 0;
|
||||
private coreRestartGeneration = 0;
|
||||
private coreRestartTimer?: NodeJS.Timeout;
|
||||
private coreRecovery?: Promise<GatewayStatus>;
|
||||
private config?: AppConfig;
|
||||
private coreAuthToken = "";
|
||||
private externalGatewayApiKey?: string;
|
||||
|
|
@ -146,6 +150,12 @@ class GatewayService {
|
|||
}
|
||||
|
||||
async start(config: AppConfig): Promise<GatewayStatus> {
|
||||
this.cancelCoreRestart();
|
||||
await this.coreRecovery?.catch(() => undefined);
|
||||
return this.startGateway(config);
|
||||
}
|
||||
|
||||
private async startGateway(config: AppConfig, recoveryGeneration?: number): Promise<GatewayStatus> {
|
||||
const coreHostError = loopbackCoreHostError(config.gateway.coreHost);
|
||||
if (coreHostError) {
|
||||
this.status = {
|
||||
|
|
@ -155,7 +165,8 @@ class GatewayService {
|
|||
};
|
||||
return this.status;
|
||||
}
|
||||
await this.stop({ nextConfig: config });
|
||||
await this.stopGateway({ nextConfig: config });
|
||||
if (recoveryGeneration !== undefined && recoveryGeneration !== this.coreRestartGeneration) return this.getStatus();
|
||||
this.config = config;
|
||||
const coreAuthToken = generateCoreGatewayAuthToken();
|
||||
const scriptValidationErrors = await this.routeScriptRuntime.prepare(config.Router.rules);
|
||||
|
|
@ -236,6 +247,7 @@ class GatewayService {
|
|||
if (await isCoreGatewayHealthy(this.status.coreEndpoint)) {
|
||||
throw new Error(`Core gateway endpoint is already in use: ${this.status.coreEndpoint}`);
|
||||
}
|
||||
if (recoveryGeneration !== undefined && recoveryGeneration !== this.coreRestartGeneration) return this.getStatus();
|
||||
const runtimeId = randomUUID();
|
||||
const spawnedGateway = spawnGatewayProcess(
|
||||
withCoreRuntimeEndpoint(config, runtimeHost, runtimePort),
|
||||
|
|
@ -269,6 +281,7 @@ class GatewayService {
|
|||
assertManagedGatewayStartupContinues(managedChild, startupFailure);
|
||||
}
|
||||
|
||||
if (recoveryGeneration !== undefined && recoveryGeneration !== this.coreRestartGeneration) return this.getStatus();
|
||||
this.status = {
|
||||
...this.status,
|
||||
coreManagedExternally: this.status.coreManagedExternally,
|
||||
|
|
@ -280,7 +293,8 @@ class GatewayService {
|
|||
this.runtimeConfigReloadError = undefined;
|
||||
return this.status;
|
||||
} catch (error) {
|
||||
await this.stop();
|
||||
if (recoveryGeneration !== undefined && recoveryGeneration !== this.coreRestartGeneration) return this.getStatus();
|
||||
await this.stopGateway();
|
||||
this.status = {
|
||||
...this.status,
|
||||
lastError: formatError(error),
|
||||
|
|
@ -347,6 +361,14 @@ class GatewayService {
|
|||
}
|
||||
|
||||
async stop(options: GatewayStopOptions = {}): Promise<GatewayStatus> {
|
||||
this.cancelCoreRestart();
|
||||
// Recovery may have started resources before observing cancellation. Finish
|
||||
// that attempt before cleanup so it cannot recreate resources after stop.
|
||||
await this.coreRecovery?.catch(() => undefined);
|
||||
return this.stopGateway(options);
|
||||
}
|
||||
|
||||
private async stopGateway(options: GatewayStopOptions = {}): Promise<GatewayStatus> {
|
||||
const child = this.child;
|
||||
const childCoreEndpoint = child ? this.status.coreEndpoint : "";
|
||||
this.child = undefined;
|
||||
|
|
@ -533,10 +555,16 @@ class GatewayService {
|
|||
if (this.child !== child || this.status.state === "stopped") {
|
||||
return;
|
||||
}
|
||||
const wasRunning = this.status.state === "running";
|
||||
const generation = this.coreRestartGeneration;
|
||||
if (this.status.lastStartedAt && Date.now() - Date.parse(this.status.lastStartedAt) >= 60_000) {
|
||||
this.coreRestartAttempts = 0;
|
||||
}
|
||||
this.child = undefined;
|
||||
setExternalLiveTokenRateSnapshot(coreGatewayLiveTokenRateSourceId);
|
||||
this.coreAuthToken = "";
|
||||
await removeManagedCoreGatewayMarkerBestEffort();
|
||||
if (generation !== this.coreRestartGeneration) return;
|
||||
this.status = {
|
||||
...this.status,
|
||||
coreManagedExternally: undefined,
|
||||
|
|
@ -544,6 +572,35 @@ class GatewayService {
|
|||
pid: undefined,
|
||||
state: "error"
|
||||
};
|
||||
if (wasRunning && this.config) this.scheduleCoreRestart(generation);
|
||||
}
|
||||
|
||||
private cancelCoreRestart(): void {
|
||||
this.coreRestartGeneration += 1;
|
||||
this.coreRestartAttempts = 0;
|
||||
if (this.coreRestartTimer) clearTimeout(this.coreRestartTimer);
|
||||
this.coreRestartTimer = undefined;
|
||||
}
|
||||
|
||||
private scheduleCoreRestart(generation: number): void {
|
||||
if (generation !== this.coreRestartGeneration || this.coreRestartTimer) return;
|
||||
const delayMs = Math.min(30_000, 1_000 * 2 ** Math.min(this.coreRestartAttempts++, 5));
|
||||
console.warn(`[gateway] Core gateway exited unexpectedly; restarting in ${delayMs}ms.`);
|
||||
this.coreRestartTimer = setTimeout(async () => {
|
||||
this.coreRestartTimer = undefined;
|
||||
if (generation !== this.coreRestartGeneration || !this.config) return;
|
||||
const recovery = this.startGateway(this.config, generation);
|
||||
this.coreRecovery = recovery;
|
||||
try {
|
||||
const status = await recovery;
|
||||
if (status.state === "error") this.scheduleCoreRestart(generation);
|
||||
} catch (error) {
|
||||
console.warn(`[gateway] Core gateway restart failed: ${formatError(error)}`);
|
||||
this.scheduleCoreRestart(generation);
|
||||
} finally {
|
||||
if (this.coreRecovery === recovery) this.coreRecovery = undefined;
|
||||
}
|
||||
}, delayMs);
|
||||
}
|
||||
|
||||
private handleCoreGatewayMessage(child: ChildProcess, message: unknown): void {
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ import type { RouteTraceObserver } from "@ccr/core/observability/route-trace";
|
|||
import { applyCompiledRouteRewrite, isBodyModelCompiledRewrite, type CompiledRouteRewrite } from "@ccr/core/routing/rewrite";
|
||||
import { buildRouteScriptInput } from "@ccr/core/routing/route-script-context";
|
||||
import { normalizeRouteScriptResult } from "@ccr/core/routing/route-script-result";
|
||||
import { calculateTokenCount } from "@ccr/core/routing/token-estimator";
|
||||
import type { RouteScriptRuntime } from "@ccr/core/routing/route-script-runtime";
|
||||
import { profileApiKeyId } from "@ccr/core/profiles/api-key";
|
||||
import { isModelAllowedForProfile } from "@ccr/core/profiles/model-allowlist";
|
||||
|
|
@ -52,11 +53,16 @@ const requireFromHere = createRequire(__filename);
|
|||
export class ClaudeCodeRouterPlugin {
|
||||
private readonly compiled: CompiledRouterConfig;
|
||||
private readonly event = new EventEmitter();
|
||||
private readonly requiresProviderTokenCount: boolean;
|
||||
private readonly scriptRuntime?: RouteScriptRuntime;
|
||||
|
||||
constructor(private readonly config: AppConfig, options: ClaudeCodeRouterPluginOptions = {}) {
|
||||
this.compiled = compileRouterConfig(config, { scriptValidationErrors: options.scriptValidationErrors });
|
||||
this.scriptRuntime = options.scriptRuntime;
|
||||
// Provider selection consumes a plain token value after routing, including
|
||||
// Fusion base models. Keep that value when its routing setting is enabled.
|
||||
this.requiresProviderTokenCount = config.Providers.some(provider =>
|
||||
Object.values(provider.modelMetadata ?? {}).some(metadata => metadata.openRouterDiscountRouting?.enabled));
|
||||
}
|
||||
|
||||
async routeRequest(input: {
|
||||
|
|
@ -87,9 +93,13 @@ export class ClaudeCodeRouterPlugin {
|
|||
matches: (candidate) => builtInAgentRouteMatches(candidate, this.config, "claude-code")
|
||||
}]);
|
||||
const sessionId = resolveSessionId(body, input.headers);
|
||||
const tokenCount = calculateTokenCount(body.messages, body.system, body.tools);
|
||||
let tokenCount: number | undefined;
|
||||
request.sessionId = sessionId;
|
||||
request.tokenCount = tokenCount;
|
||||
Object.defineProperty(request, "tokenCount", {
|
||||
enumerable: true,
|
||||
get: () => tokenCount ??= calculateTokenCount(body.messages, body.system, body.tools),
|
||||
set: (value: number) => { tokenCount = value; }
|
||||
});
|
||||
|
||||
const customRouteStartedAt = Date.now();
|
||||
const requestedCustomModel = await this.resolveCustomRoute(request);
|
||||
|
|
@ -115,6 +125,7 @@ export class ClaudeCodeRouterPlugin {
|
|||
const routeDecisionStartedAt = Date.now();
|
||||
const runtimeDiagnostics: RouteDiagnostic[] = [];
|
||||
const configuredDecision = await resolveConfiguredRouteDecision(request, this.config, this.compiled, customModel, {
|
||||
knownTokenCount: () => tokenCount,
|
||||
runtimeDiagnostics,
|
||||
scriptRuntime: this.scriptRuntime,
|
||||
trace: input.trace
|
||||
|
|
@ -183,7 +194,7 @@ export class ClaudeCodeRouterPlugin {
|
|||
reason: configuredDecision.reason,
|
||||
sessionId,
|
||||
source: configuredDecision.source,
|
||||
tokenCount
|
||||
tokenCount: this.requiresProviderTokenCount ? request.tokenCount ?? 0 : tokenCount ?? 0
|
||||
}
|
||||
};
|
||||
}
|
||||
|
|
@ -279,6 +290,7 @@ function isPathInside(file: string, root: string): boolean {
|
|||
}
|
||||
|
||||
type RouteResolutionRuntime = {
|
||||
knownTokenCount?: () => number | undefined;
|
||||
runtimeDiagnostics: RouteDiagnostic[];
|
||||
scriptRuntime?: RouteScriptRuntime;
|
||||
trace?: RouteTraceObserver;
|
||||
|
|
@ -1246,9 +1258,14 @@ async function resolveRouterRule(
|
|||
});
|
||||
return undefined;
|
||||
}
|
||||
const context = buildRouteScriptInput(request, { profileId: options.auth.profileId });
|
||||
const context = buildRouteScriptInput(request, {
|
||||
deferTokenCount: true,
|
||||
profileId: options.auth.profileId,
|
||||
tokenCount: runtime.knownTokenCount?.()
|
||||
});
|
||||
const startedAtMs = Date.now();
|
||||
const execution = await runtime.scriptRuntime.execute(rule.id, rule.script, context);
|
||||
if (execution.tokenCount !== undefined) request.tokenCount = execution.tokenCount;
|
||||
if (execution.status !== "ok") {
|
||||
const timeout = execution.status === "timeout";
|
||||
const diagnostic: RouteDiagnostic = {
|
||||
|
|
@ -1630,59 +1647,6 @@ function isSubagentModelPlaceholder(model: string): boolean {
|
|||
return model.trim().toLowerCase() === ccrSubagentModelPlaceholder;
|
||||
}
|
||||
|
||||
function calculateTokenCount(messages: unknown, system: unknown, tools: unknown): number {
|
||||
return countMessageTokens(messages) + countSystemTokens(system) + countToolTokens(tools);
|
||||
}
|
||||
|
||||
function countMessageTokens(messages: unknown): number {
|
||||
if (!Array.isArray(messages)) {
|
||||
return 0;
|
||||
}
|
||||
return messages.reduce((total, message) => total + countUnknownTokens(message), 0);
|
||||
}
|
||||
|
||||
function countSystemTokens(system: unknown): number {
|
||||
return countUnknownTokens(system);
|
||||
}
|
||||
|
||||
function countToolTokens(tools: unknown): number {
|
||||
if (!Array.isArray(tools)) {
|
||||
return 0;
|
||||
}
|
||||
return tools.reduce((total, tool) => total + countUnknownTokens(tool), 0);
|
||||
}
|
||||
|
||||
function countUnknownTokens(value: unknown): number {
|
||||
if (typeof value === "string") {
|
||||
return estimateTextTokens(value);
|
||||
}
|
||||
|
||||
if (typeof value === "number" || typeof value === "boolean") {
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
return value.reduce((total, item) => total + countUnknownTokens(item), 0);
|
||||
}
|
||||
|
||||
if (!isRecord(value)) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
let total = 0;
|
||||
for (const [key, item] of Object.entries(value)) {
|
||||
total += estimateTextTokens(key);
|
||||
total += countUnknownTokens(item);
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
function estimateTextTokens(text: string): number {
|
||||
const asciiWords = text.match(/[A-Za-z0-9_]+|[^\sA-Za-z0-9_]/g)?.length ?? 0;
|
||||
const cjkChars = text.match(/[\u3400-\u9fff]/g)?.length ?? 0;
|
||||
return Math.max(1, Math.ceil((asciiWords + cjkChars) * 1.15));
|
||||
}
|
||||
|
||||
function resolveSessionId(body: Record<string, unknown>, headers: Record<string, HeaderValue>): string | undefined {
|
||||
const fromHeader = readHeader(headers["x-claude-code-session-id"]) || readHeader(headers["x-claude-session-id"]);
|
||||
if (fromHeader) {
|
||||
|
|
|
|||
|
|
@ -1171,10 +1171,7 @@ function isCoreGatewayRuntimeProviderName(
|
|||
if (!normalized) {
|
||||
return false;
|
||||
}
|
||||
const runtimeId = providerRuntimeId(provider).toLowerCase();
|
||||
if (normalized === runtimeId) {
|
||||
return true;
|
||||
}
|
||||
// Bare provider IDs are shared aliases; resolve their protocol and credential.
|
||||
const capabilityName = providerCapabilityInternalName(provider, protocol).toLowerCase();
|
||||
return normalized === capabilityName || normalized.startsWith(`${capabilityName}::cred:`);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -400,6 +400,9 @@ export class GatewayRequestPipeline {
|
|||
routeFallback = routed.decision.fallback ?? routeFallback;
|
||||
routedSessionId = routed.decision.sessionId;
|
||||
routedTokenCount = routed.decision.tokenCount;
|
||||
if (routedTokenCount === 0 && pluginService.hasGatewayRequestTransforms({ includeBuiltIns: false })) {
|
||||
routedTokenCount = this.plugin.countTokens(routed.body).input_tokens;
|
||||
}
|
||||
if (routed.decision.model) {
|
||||
headers[ccrRoutedModelHeader] = sanitizeHeaderValue(routed.decision.model);
|
||||
routedModel = routed.decision.model;
|
||||
|
|
|
|||
|
|
@ -152,6 +152,8 @@ export class RawTraceSynchronizer {
|
|||
private storageReady = false;
|
||||
private sourceScanGeneration = 0;
|
||||
private readonly sourceObservations = new Map<string, RawTraceSourceObservation>();
|
||||
// Uploads, source/staging recovery, and retention must not move the same
|
||||
// directory while another operation is still making its durable publication.
|
||||
private storageMutation = Promise.resolve();
|
||||
|
||||
constructor(private readonly dependencies: RawTraceSynchronizerDependencies) {}
|
||||
|
|
@ -208,12 +210,12 @@ export class RawTraceSynchronizer {
|
|||
}
|
||||
let stored: StoredRawTraceBundle;
|
||||
try {
|
||||
stored = await persistRawTraceBundle(
|
||||
stored = await this.withStorageMutation(() => persistRawTraceBundle(
|
||||
manifest,
|
||||
spoolDirectory,
|
||||
this.dependencies.syncPartFile,
|
||||
this.dependencies.measureDirectorySize
|
||||
);
|
||||
));
|
||||
} catch (error) {
|
||||
console.warn(`[gateway] Failed to durably accept raw trace bundle: ${formatError(error)}`);
|
||||
sendJson(response, 503, { accepted: false, ok: false, reason: "spool_unavailable" });
|
||||
|
|
@ -571,12 +573,12 @@ export class RawTraceSynchronizer {
|
|||
for await (const directory of iterateRawTraceStagingDirectories(this.spoolDirectory())) {
|
||||
if (!await pathExists(join(directory, rawTraceReadyFileName))) continue;
|
||||
try {
|
||||
await publishStagedRawTraceBundle(
|
||||
await this.withStorageMutation(() => publishStagedRawTraceBundle(
|
||||
directory,
|
||||
this.spoolDirectory(),
|
||||
this.dependencies.syncPartFile,
|
||||
this.dependencies.measureDirectorySize
|
||||
);
|
||||
));
|
||||
} catch (error) {
|
||||
console.warn(`[gateway] Failed to publish recovered raw trace staging ${directory}: ${formatError(error)}`);
|
||||
}
|
||||
|
|
@ -611,12 +613,12 @@ export class RawTraceSynchronizer {
|
|||
}
|
||||
continue;
|
||||
}
|
||||
const stored = await persistRawTraceBundle(
|
||||
const stored = await this.withStorageMutation(() => persistRawTraceBundle(
|
||||
manifest,
|
||||
this.spoolDirectory(),
|
||||
this.dependencies.syncPartFile,
|
||||
this.dependencies.measureDirectorySize
|
||||
);
|
||||
));
|
||||
this.sourceObservations.delete(directory);
|
||||
await this.registerStoredBundle(stored);
|
||||
} catch (error) {
|
||||
|
|
@ -630,12 +632,12 @@ export class RawTraceSynchronizer {
|
|||
for await (const directory of iterateRawTraceStagingDirectories(this.spoolDirectory())) {
|
||||
try {
|
||||
if (await pathExists(join(directory, rawTraceReadyFileName))) {
|
||||
const stored = await publishStagedRawTraceBundle(
|
||||
const stored = await this.withStorageMutation(() => publishStagedRawTraceBundle(
|
||||
directory,
|
||||
this.spoolDirectory(),
|
||||
this.dependencies.syncPartFile,
|
||||
this.dependencies.measureDirectorySize
|
||||
);
|
||||
));
|
||||
this.sourceObservations.delete(directory);
|
||||
await this.registerStoredBundle(stored);
|
||||
continue;
|
||||
|
|
@ -1023,6 +1025,14 @@ async function writeDurableManifest(
|
|||
directory: string,
|
||||
manifest: Record<string, unknown>
|
||||
): Promise<void> {
|
||||
// Retrying an unchanged manifest must not look like fresh producer activity
|
||||
// to the abandoned-source sweep. fsync still precedes any durable ACK.
|
||||
const existing = await readManifestFile(directory);
|
||||
if (existing && JSON.stringify(existing) === JSON.stringify(manifest)) {
|
||||
await syncFile(join(directory, "manifest.json"));
|
||||
await syncDirectory(directory);
|
||||
return;
|
||||
}
|
||||
await writeDurableJsonFile(directory, "manifest.json", manifest);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1901,12 +1901,16 @@ function writePiWrapper(
|
|||
mkdirSync(binDir, { mode: privateDirMode, recursive: true });
|
||||
const configResult = writePiGatewayConfig(CONFIGDIR, config, profile, token, defaultModel);
|
||||
const file = piWrapperPath(profile);
|
||||
const runtimeFile = path.join(binDir, codexMiddlewareRuntimeFilename());
|
||||
const runtimeResult = process.platform === "win32"
|
||||
? writeGeneratedFileIfChanged(runtimeFile, codexCliMiddlewareRuntimeScript(), { mode: publicExecutableMode })
|
||||
: { changed: false };
|
||||
const content = process.platform === "win32"
|
||||
? piWrapperCmdScript(config, profile, configResult)
|
||||
? piWrapperCmdScript(config, profile, configResult, runtimeFile)
|
||||
: piWrapperShellScript(config, profile, configResult);
|
||||
const writeResult = writeGeneratedFileIfChanged(file, content, { mode: privateExecutableMode });
|
||||
return {
|
||||
changed: configResult.changed || writeResult.changed,
|
||||
changed: configResult.changed || writeResult.changed || runtimeResult.changed,
|
||||
configFile: configResult.file,
|
||||
file
|
||||
};
|
||||
|
|
@ -1944,7 +1948,8 @@ function piWrapperShellScript(
|
|||
function piWrapperCmdScript(
|
||||
config: AppConfig,
|
||||
profile: ProfileConfig,
|
||||
piConfig: { model: string; profileHome: string; providerId: string; sessionDir: string }
|
||||
piConfig: { model: string; profileHome: string; providerId: string; sessionDir: string },
|
||||
runtimeFile: string
|
||||
): string {
|
||||
const realPi = profile.env?.CCR_PI_BIN?.trim() || profile.env?.PI_BIN?.trim() || "pi";
|
||||
const envExports = Object.entries(profileEnv(profile))
|
||||
|
|
@ -1953,6 +1958,7 @@ function piWrapperCmdScript(
|
|||
const noProxyHosts = gatewayNoProxyHosts(config);
|
||||
return [
|
||||
"@echo off",
|
||||
"setlocal",
|
||||
...envExports,
|
||||
`set "NO_PROXY=%NO_PROXY%,${cmdValue(noProxyHosts)}"`,
|
||||
`set "no_proxy=%no_proxy%,${cmdValue(noProxyHosts)}"`,
|
||||
|
|
@ -1960,8 +1966,11 @@ function piWrapperCmdScript(
|
|||
cmdSetLine("PI_CODING_AGENT_SESSION_DIR", piConfig.sessionDir),
|
||||
cmdSetLine("PI_SKIP_VERSION_CHECK", profile.env?.PI_SKIP_VERSION_CHECK?.trim() || "1"),
|
||||
cmdSetLine("CCR_PROFILE_SURFACE", "cli"),
|
||||
`${cmdQuote(realPi)} --provider ${cmdQuote(piConfig.providerId)} --model ${cmdQuote(piConfig.model)} %*`,
|
||||
"exit /b %ERRORLEVEL%",
|
||||
cmdSetLine("CCR_PI_WRAPPER", "1"),
|
||||
cmdSetLine("CCR_REAL_PI_BIN", realPi),
|
||||
cmdSetLine("CCR_PI_PROVIDER", piConfig.providerId),
|
||||
cmdSetLine("CCR_PI_MODEL", piConfig.model),
|
||||
...nodeRuntimeCmdExecLines(runtimeFile),
|
||||
""
|
||||
].join("\r\n");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,6 +3,8 @@ import type { RouteRequest } from "@ccr/core/routing/contracts";
|
|||
const maxLastUserTextChars = 16 * 1024;
|
||||
const maxSystemTextChars = 8 * 1024;
|
||||
const maxToolNames = 128;
|
||||
type BodySnapshot = { json: string; body?: Record<string, unknown> };
|
||||
const bodySnapshots = new WeakMap<RouteScriptInput, BodySnapshot>();
|
||||
|
||||
export type RouteScriptInput = {
|
||||
apiKeyId?: string;
|
||||
|
|
@ -20,12 +22,14 @@ export type RouteScriptInput = {
|
|||
systemText: string;
|
||||
toolNames: string[];
|
||||
};
|
||||
tokenCount: number;
|
||||
tokenCount?: number;
|
||||
url: string;
|
||||
};
|
||||
|
||||
export type BuildRouteScriptInputOptions = {
|
||||
deferTokenCount?: boolean;
|
||||
profileId?: string;
|
||||
tokenCount?: number;
|
||||
};
|
||||
|
||||
export function buildRouteScriptInput(
|
||||
|
|
@ -33,10 +37,11 @@ export function buildRouteScriptInput(
|
|||
options: BuildRouteScriptInputOptions = {}
|
||||
): RouteScriptInput {
|
||||
const apiKeyId = readHeader(request.headers, "x-auth-api-key-id");
|
||||
const snapshot: BodySnapshot = { json: JSON.stringify(request.body) };
|
||||
const input: RouteScriptInput = {
|
||||
...(apiKeyId ? { apiKeyId } : {}),
|
||||
...(request.builtInSubagentModel ? { builtInSubagentModel: request.builtInSubagentModel } : {}),
|
||||
body: cloneJson(request.body) as Record<string, unknown>,
|
||||
body: {},
|
||||
headers: requestHeaders(request.headers),
|
||||
method: request.method,
|
||||
...(typeof request.body.model === "string" ? { model: request.body.model } : {}),
|
||||
|
|
@ -49,12 +54,32 @@ export function buildRouteScriptInput(
|
|||
systemText: truncateText(textFromUnknown(request.body.system), maxSystemTextChars),
|
||||
toolNames: toolNames(request.body.tools).slice(0, maxToolNames)
|
||||
},
|
||||
tokenCount: request.tokenCount ?? 0,
|
||||
tokenCount: options.deferTokenCount ? options.tokenCount : request.tokenCount ?? 0,
|
||||
url: request.url
|
||||
};
|
||||
// Retain an immutable JSON snapshot for transport, without parsing and then
|
||||
// cloning the entire conversation again on the host's main thread.
|
||||
bodySnapshots.set(input, snapshot);
|
||||
Object.defineProperty(input, "body", {
|
||||
configurable: true,
|
||||
enumerable: true,
|
||||
get: () => snapshot.body ??= JSON.parse(snapshot.json) as Record<string, unknown>,
|
||||
set: (body: Record<string, unknown>) => { snapshot.body = body; }
|
||||
});
|
||||
return input;
|
||||
}
|
||||
|
||||
export function serializeRouteScriptInput(input: RouteScriptInput): string {
|
||||
const snapshot = bodySnapshots.get(input);
|
||||
if (!snapshot) return JSON.stringify(input);
|
||||
const metadata: Record<string, unknown> = {};
|
||||
for (const key of Object.keys(input)) {
|
||||
if (key !== "body") metadata[key] = input[key as keyof RouteScriptInput];
|
||||
}
|
||||
const bodyJson = snapshot.body ? JSON.stringify(snapshot.body) : snapshot.json;
|
||||
return `${JSON.stringify(metadata).slice(0, -1)},"body":${bodyJson}}`;
|
||||
}
|
||||
|
||||
function requestHeaders(headers: RouteRequest["headers"]): Record<string, string | string[]> {
|
||||
const result: Record<string, string | string[]> = {};
|
||||
for (const [name, rawValue] of Object.entries(headers)) {
|
||||
|
|
@ -109,10 +134,6 @@ function containsImage(value: unknown, depth = 0): boolean {
|
|||
return Object.values(value).some((item) => containsImage(item, depth + 1));
|
||||
}
|
||||
|
||||
function cloneJson(value: unknown): unknown {
|
||||
return JSON.parse(JSON.stringify(value)) as unknown;
|
||||
}
|
||||
|
||||
function truncateText(value: string, maxChars: number): string {
|
||||
return value.length <= maxChars ? value : value.slice(0, maxChars);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ import {
|
|||
type RouterRule,
|
||||
type RouterRuleScript
|
||||
} from "@ccr/core/contracts/app";
|
||||
import type { RouteScriptInput } from "@ccr/core/routing/route-script-context";
|
||||
import { serializeRouteScriptInput, type RouteScriptInput } from "@ccr/core/routing/route-script-context";
|
||||
import type {
|
||||
ResolvedRouteScript,
|
||||
RouteScriptWorkerRequest,
|
||||
|
|
@ -27,6 +27,7 @@ const workerOldGenerationMb = 64;
|
|||
const workerYoungGenerationMb = 16;
|
||||
|
||||
export type RouteScriptExecutionResult = {
|
||||
tokenCount?: number;
|
||||
durationMs: number;
|
||||
error?: string;
|
||||
status: "circuit-open" | "error" | "ok" | "queue-full" | "timeout";
|
||||
|
|
@ -138,14 +139,15 @@ export class RouteScriptRuntime {
|
|||
}
|
||||
try {
|
||||
const response = await this.nextSlot().rpc({
|
||||
input,
|
||||
inputJson: serializeRouteScriptInput(input),
|
||||
requestId: ++this.nextRequestId,
|
||||
script: resolved,
|
||||
type: "execute"
|
||||
}, resolved.timeoutMs + 250);
|
||||
if (response.status === "ok") {
|
||||
if (circuitBreakerEnabled) this.recordSuccess(circuitKey);
|
||||
return { durationMs: response.durationMs, status: "ok", value: response.result };
|
||||
return { durationMs: response.durationMs, status: "ok", value: response.result,
|
||||
...(response.tokenCount !== undefined ? { tokenCount: response.tokenCount } : {}) };
|
||||
}
|
||||
if (circuitBreakerEnabled) this.recordFailure(circuitKey, Date.now());
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -1,12 +1,10 @@
|
|||
import type { RouteScriptInput } from "@ccr/core/routing/route-script-context";
|
||||
|
||||
export type ResolvedRouteScript = {
|
||||
source: string;
|
||||
timeoutMs: number;
|
||||
};
|
||||
|
||||
export type RouteScriptWorkerRequest = {
|
||||
input?: RouteScriptInput;
|
||||
inputJson?: string;
|
||||
requestId: number;
|
||||
script: ResolvedRouteScript;
|
||||
type: "execute" | "validate";
|
||||
|
|
@ -17,6 +15,7 @@ export type RouteScriptWorkerResponse = {
|
|||
error?: string;
|
||||
requestId: number;
|
||||
result?: unknown;
|
||||
tokenCount?: number;
|
||||
status: "error" | "ok" | "timeout";
|
||||
type: "response";
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { calculateTokenCount } from "@ccr/core/routing/token-estimator";
|
||||
import { promises as fs } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
|
@ -37,14 +38,16 @@ async function evaluateRequest(request: RouteScriptWorkerRequest): Promise<Route
|
|||
});
|
||||
const route = compileScript(request.script.source, request.requestId, context);
|
||||
if (request.type === "validate") return successResponse(request.requestId, startedAt);
|
||||
if (!request.input) return failureResponse(request.requestId, startedAt, "Script input is required.");
|
||||
if (!request.inputJson) return failureResponse(request.requestId, startedAt, "Script input is required.");
|
||||
|
||||
const deadline = Date.now() + request.script.timeoutMs;
|
||||
let tokenCount: number | undefined;
|
||||
Object.assign(context, {
|
||||
__routeBridge: createRouteScriptBridge(deadline),
|
||||
__routeBridge: createRouteScriptBridge(deadline, (body: Record<string, unknown>) =>
|
||||
tokenCount ??= calculateTokenCount(body.messages, body.system, body.tools)),
|
||||
__routeEnvironmentJson: JSON.stringify(routeScriptEnvironment()),
|
||||
__routeFunction: route,
|
||||
__routeInputJson: JSON.stringify(request.input)
|
||||
__routeInputJson: request.inputJson
|
||||
});
|
||||
|
||||
const invocation = new vm.Script(routeInvocationSource, {
|
||||
|
|
@ -53,7 +56,8 @@ async function evaluateRequest(request: RouteScriptWorkerRequest): Promise<Route
|
|||
const synchronousBudget = Math.max(1, request.script.timeoutMs);
|
||||
const pendingResult = invocation.runInContext(context, { timeout: synchronousBudget });
|
||||
const result = await withDeadline(Promise.resolve(pendingResult), deadline);
|
||||
return successResponse(request.requestId, startedAt, serializeResult(result));
|
||||
return { ...successResponse(request.requestId, startedAt, serializeResult(result)),
|
||||
...(tokenCount !== undefined ? { tokenCount } : {}) };
|
||||
} catch (error) {
|
||||
return failureResponse(
|
||||
request.requestId,
|
||||
|
|
@ -89,7 +93,9 @@ const routeInvocationSource = `(() => {
|
|||
const deepFreeze = (value) => {
|
||||
if ((typeof value === "object" || typeof value === "function") && value !== null && !Object.isFrozen(value)) {
|
||||
Object.freeze(value);
|
||||
for (const nested of Object.values(value)) deepFreeze(nested);
|
||||
for (const descriptor of Object.values(Object.getOwnPropertyDescriptors(value))) {
|
||||
if ("value" in descriptor) deepFreeze(descriptor.value);
|
||||
}
|
||||
}
|
||||
return value;
|
||||
};
|
||||
|
|
@ -129,11 +135,19 @@ const routeInvocationSource = `(() => {
|
|||
},
|
||||
hash
|
||||
});
|
||||
const input = deepFreeze(JSON.parse(inputJson));
|
||||
const input = JSON.parse(inputJson);
|
||||
if (input.tokenCount === undefined) {
|
||||
let tokenCount;
|
||||
Object.defineProperty(input, "tokenCount", {
|
||||
enumerable: true,
|
||||
get: () => tokenCount ??= bridge.estimateTokens(input.body)
|
||||
});
|
||||
}
|
||||
deepFreeze(input);
|
||||
return route(input, api);
|
||||
})()`;
|
||||
|
||||
function createRouteScriptBridge(deadline: number): Readonly<Record<string, unknown>> {
|
||||
function createRouteScriptBridge(deadline: number, estimateTokens: (body: Record<string, unknown>) => number): Readonly<Record<string, unknown>> {
|
||||
const methods: Record<string, (...args: unknown[]) => unknown> = {
|
||||
"fetch": (url, options) => controlledFetch(String(url), options, deadline),
|
||||
"fs.exists": async (file) => {
|
||||
|
|
@ -174,6 +188,7 @@ function createRouteScriptBridge(deadline: number): Readonly<Record<string, unkn
|
|||
"fs.writeText": async (file, value) => writeTextFile(String(file), String(value))
|
||||
};
|
||||
return Object.freeze({
|
||||
estimateTokens,
|
||||
invoke: async (method: unknown, encodedArguments: unknown) => {
|
||||
if (typeof method !== "string" || typeof encodedArguments !== "string" || !Object.hasOwn(methods, method)) {
|
||||
throw new Error("Unsupported route script capability call.");
|
||||
|
|
|
|||
54
packages/core/src/routing/token-estimator.ts
Normal file
54
packages/core/src/routing/token-estimator.ts
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
import { isRecord } from "@ccr/core/gateway/internal/value";
|
||||
|
||||
export function calculateTokenCount(messages: unknown, system: unknown, tools: unknown): number {
|
||||
return countMessageTokens(messages) + countSystemTokens(system) + countToolTokens(tools);
|
||||
}
|
||||
|
||||
function countMessageTokens(messages: unknown): number {
|
||||
if (!Array.isArray(messages)) {
|
||||
return 0;
|
||||
}
|
||||
return messages.reduce((total, message) => total + countUnknownTokens(message), 0);
|
||||
}
|
||||
|
||||
function countSystemTokens(system: unknown): number {
|
||||
return countUnknownTokens(system);
|
||||
}
|
||||
|
||||
function countToolTokens(tools: unknown): number {
|
||||
if (!Array.isArray(tools)) {
|
||||
return 0;
|
||||
}
|
||||
return tools.reduce((total, tool) => total + countUnknownTokens(tool), 0);
|
||||
}
|
||||
|
||||
function countUnknownTokens(value: unknown): number {
|
||||
if (typeof value === "string") {
|
||||
return estimateTextTokens(value);
|
||||
}
|
||||
|
||||
if (typeof value === "number" || typeof value === "boolean") {
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
return value.reduce((total, item) => total + countUnknownTokens(item), 0);
|
||||
}
|
||||
|
||||
if (!isRecord(value)) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
let total = 0;
|
||||
for (const [key, item] of Object.entries(value)) {
|
||||
total += estimateTextTokens(key);
|
||||
total += countUnknownTokens(item);
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
function estimateTextTokens(text: string): number {
|
||||
const asciiWords = text.match(/[A-Za-z0-9_]+|[^\sA-Za-z0-9_]/g)?.length ?? 0;
|
||||
const cjkChars = text.match(/[\u3400-\u9fff]/g)?.length ?? 0;
|
||||
return Math.max(1, Math.ceil((asciiWords + cjkChars) * 1.15));
|
||||
}
|
||||
|
|
@ -15,6 +15,40 @@ test("generated Codex CLI middleware runtime is valid JavaScript", () => {
|
|||
execFileSync(process.execPath, ["--check", file], { stdio: "pipe" });
|
||||
});
|
||||
|
||||
test("#1851 Pi dispatch consumes only the profile selector and preserves the prompt", () => {
|
||||
const dir = mkdtempSync(path.join(os.tmpdir(), "ccr-pi-dispatch-"));
|
||||
try {
|
||||
const runtime = writeRuntimeScript(dir);
|
||||
const output = path.join(dir, "args.json");
|
||||
const capture = path.join(dir, "capture.js");
|
||||
const fakePi = path.join(dir, process.platform === "win32" ? "pi.cmd" : "pi");
|
||||
writeFileSync(capture, 'require("node:fs").writeFileSync(process.env.CCR_PI_TEST_OUTPUT, JSON.stringify({ args: process.argv.slice(2), marker: process.env.CCR_CLI_DIRECT_PROFILE_DISPATCH }));');
|
||||
if (process.platform === "win32") writeFileSync(fakePi, `@echo off\r\n"${process.execPath}" "${capture}" %*\r\n`);
|
||||
else {
|
||||
writeFileSync(fakePi, `#!/bin/sh\nexec "${process.execPath}" "${capture}" "$@"\n`);
|
||||
chmodSync(fakePi, 0o700);
|
||||
}
|
||||
for (const [args, direct, expected] of [
|
||||
[["Pi"], true, []],
|
||||
[["pi"], true, []],
|
||||
[["Pi Work", "cli", "--", "hello world", "--continue"], true, ["hello world", "--continue"]],
|
||||
[["Pi", "--", 'quote " and & % !', "--model", "custom"], true, ['quote " and & % !', "--model", "custom"]],
|
||||
[["Pi"], false, ["Pi"]],
|
||||
]) {
|
||||
const result = spawnSync(process.execPath, [runtime, ...args], {
|
||||
encoding: "utf8", timeout: 10000,
|
||||
env: { ...process.env, CCR_PI_WRAPPER: "1", CCR_REAL_PI_BIN: fakePi,
|
||||
CCR_REAL_CODEX_CLI_PATH: fakePi, CCR_PI_PROVIDER: "ccr", CCR_PI_MODEL: "Provider/model",
|
||||
CCR_PI_TEST_OUTPUT: output, CCR_CLI_DIRECT_PROFILE_DISPATCH: direct ? "1" : "", CCR_CODEX_DEFAULT_ARGS: "--version" },
|
||||
});
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
const captured = JSON.parse(readFileSync(output, "utf8"));
|
||||
assert.deepEqual(captured.args, ["--provider", "ccr", "--model", "Provider/model", ...expected]);
|
||||
assert.equal(captured.marker, undefined);
|
||||
}
|
||||
} finally { rmSync(dir, { force: true, recursive: true }); }
|
||||
});
|
||||
|
||||
test("generated Codex middleware fallback retains the catalog's tool commentary instructions", () => {
|
||||
const model = "uuroute/gpt-5.5";
|
||||
const fallback = evaluateRuntimeFunction("modelCatalogConfigItem")(model, 0);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,199 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { spawn } from "node:child_process";
|
||||
import { createServer } from "node:http";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import { createDefaultAppConfig } from "@ccr/core/config/default-config.ts";
|
||||
import { compileCoreGatewayConfig } from "@ccr/core/gateway/core-runtime/config-compiler.ts";
|
||||
import { ClaudeCodeRouterPlugin } from "@ccr/core/gateway/claude-code-router-plugin.ts";
|
||||
import { GatewayRequestPipeline } from "@ccr/core/gateway/request/pipeline.ts";
|
||||
import { closeRequestLogRuntime, flushRequestLogRuntime, requestLogRuntime } from "@ccr/core/observability/request-log-store.ts";
|
||||
|
||||
import { gatewayService } from "@ccr/core/gateway/application/gateway-service.ts";
|
||||
import { pluginService } from "@ccr/core/plugins/service.ts";
|
||||
|
||||
const entry = process.env.CCR_TEST_GATEWAY_ENTRY;
|
||||
const skip = !entry && "set CCR_TEST_GATEWAY_ENTRY to the gateway production entry";
|
||||
|
||||
test("#1850/#1861 fallback attribution reaches OpenAI JSON/SSE clients and request logs", { skip, timeout: 30000 }, async () => {
|
||||
const root = mkdtempSync(path.join(tmpdir(), "ccr-october-fallback-"));
|
||||
requestLogRuntime.options.workerFile = path.resolve(__dirname, "../../../runtime/request-log-worker.js");
|
||||
const calls = [];
|
||||
const tokenCounts = [];
|
||||
const originalTransforms = pluginService.gatewayRequestTransforms;
|
||||
pluginService.gatewayRequestTransforms = [{ pluginId: "customer", id: "token-probe",
|
||||
transform: input => { tokenCounts.push(input.tokenCount); return undefined; } }];
|
||||
const upstream = createServer(async (req, res) => {
|
||||
const body = await readJson(req);
|
||||
calls.push({ path: req.url, model: body.model });
|
||||
if (req.url.startsWith("/primary/")) {
|
||||
res.writeHead(429, { "content-type": "application/json" });
|
||||
res.end(JSON.stringify({ error: { message: "primary quota exhausted" } }));
|
||||
return;
|
||||
}
|
||||
const responses = req.url.endsWith("/responses");
|
||||
const payload = responses
|
||||
? { id: "resp_backup", object: "response", model: "backup", status: "completed", output: [{ id: "msg_backup", type: "message", role: "assistant", status: "completed", content: [{ type: "output_text", text: "from backup", annotations: [] }] }], usage: { input_tokens: 2, output_tokens: 1, total_tokens: 3 } }
|
||||
: { id: "chatcmpl_backup", object: "chat.completion", model: "backup", choices: [{ index: 0, message: { role: "assistant", content: "from backup" }, finish_reason: "stop" }], usage: { prompt_tokens: 2, completion_tokens: 1, total_tokens: 3 } };
|
||||
res.writeHead(200, { "content-type": body.stream ? "text/event-stream" : "application/json" });
|
||||
if (!body.stream) res.end(JSON.stringify(payload));
|
||||
else if (responses) res.end(`event: response.completed\ndata: ${JSON.stringify({ type: "response.completed", response: payload })}\n\n`);
|
||||
else res.end(`data: ${JSON.stringify({ ...payload, object: "chat.completion.chunk", choices: [{ index: 0, delta: { role: "assistant", content: "from backup" }, finish_reason: "stop" }] })}\n\ndata: [DONE]\n\n`);
|
||||
});
|
||||
let gateway;
|
||||
let host;
|
||||
try {
|
||||
const url = await listen(upstream);
|
||||
const config = createDefaultAppConfig();
|
||||
config.Providers = ["Primary", "Backup"].map(name => ({
|
||||
id: name.toLowerCase(), name, api_key: "test", models: [name.toLowerCase()], type: "openai_chat_completions",
|
||||
api_base_url: `${url}/${name.toLowerCase()}/v1`,
|
||||
capabilities: ["openai_chat_completions", "openai_responses"].map(type => ({ type, source: "detected", baseUrl: `${url}/${name.toLowerCase()}/v1` })),
|
||||
}));
|
||||
config.Router.builtInRules = { "claude-code": { enabled: false }, codex: { enabled: false } };
|
||||
config.Router.fallback = { mode: "model-chain", models: ["Backup/backup"], retryCount: 0 };
|
||||
config.observability = { ...config.observability, requestLogs: true, requestLogBodyCapture: "all", requestLogSuccessSampleRate: 1 };
|
||||
const compiled = await compileCoreGatewayConfig(config, "trace", "billing", "auth");
|
||||
const router = compiled.plugins.find(plugin => plugin.key === "ccr-router");
|
||||
router.modulePath = path.resolve(__dirname, "../../../runtime/router-plugin.js");
|
||||
gateway = await startGateway(root, { providers: compiled.providers, plugins: [router] });
|
||||
const plugin = new ClaudeCodeRouterPlugin(config);
|
||||
const pipeline = new GatewayRequestPipeline({
|
||||
getConfig: () => config, getPlugin: () => plugin, getCoreAuthToken: () => "auth",
|
||||
getBrowserWebSearchMcpIntegration: () => undefined,
|
||||
getStatus: () => ({ coreEndpoint: gateway.url, endpoint: "http://127.0.0.1" }),
|
||||
});
|
||||
host = createServer((req, res) => void pipeline.proxyRequest(req, res, req.url).catch(error => { res.writeHead(500); res.end(String(error)); }));
|
||||
const hostUrl = await listen(host);
|
||||
for (const route of ["/v1/chat/completions", "/v1/responses"]) for (const stream of [false, true]) {
|
||||
const before = calls.length;
|
||||
const response = await fetch(hostUrl + route, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ model: "Primary/primary", stream, messages: [{ role: "user", content: "hi" }], input: "hi" }) });
|
||||
const text = await response.text();
|
||||
assert.equal(response.status, 200, text);
|
||||
assert.match(text, /"model":"backup"/);
|
||||
assert.match(text, /from backup/);
|
||||
assert.doesNotMatch(text, /"model":"primary"/);
|
||||
assert.deepEqual(calls.slice(before), [{ path: `/primary${route}`, model: "primary" }, { path: `/backup${route}`, model: "backup" }]);
|
||||
}
|
||||
assert.equal(tokenCounts.length, 4);
|
||||
assert.ok(tokenCounts.every(count => count > 0), `#1869 plugin token input: ${JSON.stringify(tokenCounts)}`);
|
||||
await until(async () => { await flushRequestLogRuntime(1000); return (await requestLogRuntime.list({ pageSize: 100 })).items.length === 4; });
|
||||
const rows = (await requestLogRuntime.list({ pageSize: 100 })).items;
|
||||
assert.ok(rows.every(row => row.provider === "Backup" && row.responseModel === "backup" && row.resolvedModel.includes("backup")), JSON.stringify(rows));
|
||||
} finally {
|
||||
pluginService.gatewayRequestTransforms = originalTransforms;
|
||||
await close(host);
|
||||
if (gateway) await stopGateway(gateway);
|
||||
await close(upstream);
|
||||
await closeRequestLogRuntime();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("#1852 built gateway preserves native virtual-model content and usage with billing on/off", { skip, timeout: 30000 }, async () => {
|
||||
const root = mkdtempSync(path.join(tmpdir(), "ccr-october-native-stream-"));
|
||||
const upstream = createServer(async (req, res) => {
|
||||
const body = await readJson(req);
|
||||
assert.equal(body.model, "native");
|
||||
const payload = { id: "msg_native", type: "message", role: "assistant", model: "native", content: [{ type: "text", text: "native answer 你好 🌏" }], stop_reason: "end_turn", usage: { input_tokens: 5, output_tokens: 7 } };
|
||||
res.writeHead(200, { "content-type": body.stream ? "text/event-stream" : "application/json" });
|
||||
if (!body.stream) { res.end(JSON.stringify(payload)); return; }
|
||||
const events = [
|
||||
["message_start", { type: "message_start", message: { ...payload, content: [], stop_reason: null, usage: { input_tokens: 5, output_tokens: 0 } } }],
|
||||
["content_block_start", { type: "content_block_start", index: 0, content_block: { type: "thinking", thinking: "" } }],
|
||||
["content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "thinking_delta", thinking: "native reasoning" } }],
|
||||
["content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "signature_delta", signature: "native signature" } }],
|
||||
["content_block_stop", { type: "content_block_stop", index: 0 }],
|
||||
["content_block_start", { type: "content_block_start", index: 1, content_block: { type: "text", text: "" } }],
|
||||
["content_block_delta", { type: "content_block_delta", index: 1, delta: { type: "text_delta", text: "native answer 你好 🌏" } }],
|
||||
["content_block_stop", { type: "content_block_stop", index: 1 }],
|
||||
["message_delta", { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 7 } }],
|
||||
["message_stop", { type: "message_stop" }],
|
||||
];
|
||||
// Send fragmented UTF-8/SSE chunks through actual HTTP, not Fastify inject.
|
||||
const text = events.map(([event, data]) => `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`).join("");
|
||||
const bytes = Buffer.from(text);
|
||||
for (let offset = 0; offset < bytes.length; offset += 37) res.write(bytes.subarray(offset, offset + 37));
|
||||
res.end();
|
||||
});
|
||||
let gateway;
|
||||
try {
|
||||
const upstreamUrl = await listen(upstream);
|
||||
for (const billing of [false, true]) {
|
||||
gateway = await startGateway(root, { providers: [{ name: "native", type: "anthropic_messages", baseUrl: upstreamUrl, apiKey: "test", models: ["native"] }], billing: { enabled: billing }, virtualModelProfiles: [{ id: "alias", key: "alias", displayName: "Alias", enabled: true, match: { exactAliases: ["alias"], prefixes: [], suffixes: [] }, baseModel: { mode: "fixed", fixedModel: "native/native" }, tools: [], execution: { mode: "tool_loop", maxTurns: 2, maxToolCalls: 2, clientToolsPolicy: "allow", streamMode: "buffered" }, materialization: { enabled: true, includeInGatewayModels: true } }] });
|
||||
for (const stream of [false, true]) {
|
||||
const response = await fetch(gateway.url + "/v1/messages", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ model: "alias", max_tokens: 32, stream, messages: [{ role: "user", content: "hi" }] }) });
|
||||
const text = await response.text();
|
||||
assert.equal(response.status, 200, text);
|
||||
assert.match(text, /native answer 你好 🌏/);
|
||||
assert.match(text, /"output_tokens":7/);
|
||||
if (stream) { assert.match(text, /native reasoning/); assert.match(text, /native signature/); }
|
||||
}
|
||||
await stopGateway(gateway); gateway = undefined;
|
||||
}
|
||||
} finally {
|
||||
if (gateway) await stopGateway(gateway);
|
||||
await close(upstream);
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("#1867 a killed managed core restarts and restores the real service health", { skip, timeout: 30000 }, async () => {
|
||||
const originalEntry = process.env.CCR_GATEWAY_ENTRY;
|
||||
process.env.CCR_GATEWAY_ENTRY = entry;
|
||||
const service = new gatewayService.constructor();
|
||||
const upstream = createServer((req, res) => { req.resume(); res.end("{}"); });
|
||||
try {
|
||||
const upstreamUrl = await listen(upstream);
|
||||
const publicReserve = createServer();
|
||||
await listen(publicReserve);
|
||||
const publicPort = publicReserve.address().port;
|
||||
const coreReserve = createServer();
|
||||
await listen(coreReserve);
|
||||
const corePort = coreReserve.address().port;
|
||||
await close(publicReserve); await close(coreReserve);
|
||||
const config = createDefaultAppConfig();
|
||||
config.Providers = [{ name: "Provider", id: "provider", models: ["native"], type: "anthropic_messages", api_base_url: upstreamUrl, api_key: "test" }];
|
||||
config.gateway = { ...config.gateway, enabled: true, host: "127.0.0.1", port: publicPort, coreHost: "127.0.0.1", corePort };
|
||||
config.Router.fallback = { mode: "model-chain", models: ["Provider/native"], retryCount: 0 };
|
||||
config.mediaTools.enabled = false;
|
||||
config.proxy.enabled = false; config.proxy.captureNetwork = false;
|
||||
config.contextArchive.enabled = false;
|
||||
config.observability.requestLogs = false; config.observability.requestLogBodyCapture = "none";
|
||||
config.profile.enabled = false; config.profile.profiles = [];
|
||||
config.plugins = [];
|
||||
const started = await service.start(config);
|
||||
assert.equal(started.state, "running", started.lastError);
|
||||
const pid = started.pid;
|
||||
const auth = service.coreAuthToken;
|
||||
service.child.kill("SIGKILL");
|
||||
await until(() => service.getStatus().state === "running" && service.getStatus().pid !== pid, 15000);
|
||||
assert.ok(service.getStatus().pid);
|
||||
assert.notEqual(service.coreAuthToken, auth);
|
||||
const health = await fetch(`http://127.0.0.1:${publicPort}/health`);
|
||||
assert.equal(health.status, 200);
|
||||
assert.equal((await health.json()).status, "running");
|
||||
} finally {
|
||||
await service.stop();
|
||||
await close(upstream);
|
||||
if (originalEntry === undefined) delete process.env.CCR_GATEWAY_ENTRY;
|
||||
else process.env.CCR_GATEWAY_ENTRY = originalEntry;
|
||||
}
|
||||
});
|
||||
|
||||
async function readJson(request) { let text = ""; for await (const chunk of request) text += chunk; return JSON.parse(text || "{}"); }
|
||||
async function listen(server) { await new Promise((resolve, reject) => { server.once("error", reject); server.listen(0, "127.0.0.1", resolve); }); return `http://127.0.0.1:${server.address().port}`; }
|
||||
async function close(server) { if (server?.listening) await new Promise(resolve => server.close(resolve)); }
|
||||
async function until(predicate, timeout = 10000) { const deadline = Date.now() + timeout; do { if (await predicate()) return; await new Promise(resolve => setTimeout(resolve, 25)); } while (Date.now() < deadline); assert.fail("Timed out waiting for gateway/logs."); }
|
||||
async function startGateway(root, config) {
|
||||
const reserve = createServer(); const url = await listen(reserve); const port = reserve.address().port; await close(reserve);
|
||||
const file = path.join(root, "gateway.json");
|
||||
writeFileSync(file, JSON.stringify({ host: "127.0.0.1", port, auth: { enabled: false }, rawTrace: { enabled: false }, billing: { enabled: false }, logging: { level: "error", accessLog: false }, ...config }));
|
||||
const child = spawn(process.execPath, [entry], { env: { ...process.env, GATEWAY_CONFIG_PATH: file }, stdio: ["ignore", "pipe", "pipe"] });
|
||||
let output = ""; child.stdout.on("data", chunk => output += chunk); child.stderr.on("data", chunk => output += chunk);
|
||||
try { await until(async () => { if (child.exitCode !== null) throw new Error(output); try { return (await fetch(url + "/health")).ok; } catch { return false; } }); return { child, url }; }
|
||||
catch (error) { child.kill(); throw new Error(`${error}\n${output}`); }
|
||||
}
|
||||
async function stopGateway({ child }) { if (child.exitCode !== null) return; child.kill(); await new Promise(resolve => { const timer = setTimeout(() => { child.kill("SIGKILL"); resolve(); }, 2000); child.once("exit", () => { clearTimeout(timer); resolve(); }); }); }
|
||||
|
|
@ -1627,6 +1627,23 @@ test("profile service writes a Pi config and wrapper that points inference to CC
|
|||
assert.equal(existsSync(path.join(profilePiHome, "sessions")), true);
|
||||
});
|
||||
|
||||
test("#1851 Windows Pi wrapper uses the argument-aware CLI runtime", { skip: !process.env.CCR_INTERNAL_HOME_DIR }, async () => {
|
||||
const originalPlatform = Object.getOwnPropertyDescriptor(process, "platform");
|
||||
const config = createDefaultAppConfig();
|
||||
config.Providers = [{ name: "Provider", models: ["model"], type: "openai_chat_completions", api_base_url: "https://example.test/v1" }];
|
||||
config.APIKEYS = [{ id: "profile:pi-dispatch", key: "test", name: "Pi" }];
|
||||
config.profile.profiles = [{ agent: "pi", enabled: true, env: {}, id: "pi-dispatch", model: "Provider/model", name: "Pi", scope: "ccr", surface: "cli" }];
|
||||
try {
|
||||
Object.defineProperty(process, "platform", { value: "win32", configurable: true });
|
||||
const result = await applyProfileFixture(config);
|
||||
assert.equal(result.clients[0].ok, true);
|
||||
const content = readFileSync(path.join(CONFIGDIR, "bin", "ccr-pi-wrapper-pi-dispatch.cmd"), "utf8");
|
||||
assert.match(content, /CCR_PI_WRAPPER=1/);
|
||||
assert.match(content, /ccr-codex-cli-middleware\.js/);
|
||||
assert.doesNotMatch(content, /"pi" --provider .*%\*/);
|
||||
} finally { Object.defineProperty(process, "platform", originalPlatform); }
|
||||
});
|
||||
|
||||
test("profile service writes an OpenCode CLI wrapper and shared CLI/App config", { skip: !process.env.CCR_INTERNAL_HOME_DIR }, async () => {
|
||||
const profileId = "opencode-gateway-test";
|
||||
const config = createDefaultAppConfig();
|
||||
|
|
|
|||
128
packages/core/test/unit/gateway/core-restart.test.mjs
Normal file
128
packages/core/test/unit/gateway/core-restart.test.mjs
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { gatewayService } from "@ccr/core/gateway/application/gateway-service.ts";
|
||||
|
||||
// A fresh service without starting HTTP, plugins, or child processes.
|
||||
function createService() {
|
||||
return new gatewayService.constructor();
|
||||
}
|
||||
|
||||
function crashedService() {
|
||||
const service = createService();
|
||||
const child = {};
|
||||
service.child = child;
|
||||
service.config = { gateway: { host: "127.0.0.1", port: 0 } };
|
||||
service.status = { state: "running", endpoint: "", coreEndpoint: "", networkEndpoints: [] };
|
||||
return { child, service };
|
||||
}
|
||||
|
||||
test("#1867 unexpected core exit restarts with bounded backoff", async (t) => {
|
||||
t.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const { child, service } = crashedService();
|
||||
let restarts = 0;
|
||||
service.startGateway = async () => {
|
||||
restarts += 1;
|
||||
service.status.state = "error";
|
||||
return service.getStatus();
|
||||
};
|
||||
await service.handleCoreGatewayTermination(child, undefined, "heap OOM");
|
||||
assert.equal(service.getStatus().state, "error");
|
||||
t.mock.timers.tick(999);
|
||||
assert.equal(restarts, 0);
|
||||
t.mock.timers.tick(1);
|
||||
await Promise.resolve();
|
||||
await Promise.resolve();
|
||||
assert.equal(restarts, 1);
|
||||
t.mock.timers.tick(1999);
|
||||
assert.equal(restarts, 1);
|
||||
t.mock.timers.tick(1);
|
||||
await Promise.resolve();
|
||||
await Promise.resolve();
|
||||
assert.equal(restarts, 2);
|
||||
for (let attempt = 0; attempt < 10; attempt += 1) {
|
||||
t.mock.timers.tick(30_000);
|
||||
await Promise.resolve();
|
||||
await Promise.resolve();
|
||||
}
|
||||
assert.equal(restarts, 12);
|
||||
});
|
||||
|
||||
test("#1867 explicit stop cancels a scheduled core restart", async (t) => {
|
||||
t.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const { child, service } = crashedService();
|
||||
let restarts = 0;
|
||||
service.startGateway = async () => { restarts += 1; return service.getStatus(); };
|
||||
service.stopGateway = async () => { service.status.state = "stopped"; return service.getStatus(); };
|
||||
await service.handleCoreGatewayTermination(child, undefined, "heap OOM");
|
||||
await service.stop();
|
||||
t.mock.timers.tick(60_000);
|
||||
await Promise.resolve();
|
||||
assert.equal(restarts, 0);
|
||||
});
|
||||
|
||||
test("#1867 explicit stop cleans up after an already running recovery", async (t) => {
|
||||
t.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const { child, service } = crashedService();
|
||||
const operations = [];
|
||||
let releaseRecovery;
|
||||
const recoveryGate = new Promise((resolve) => { releaseRecovery = resolve; });
|
||||
service.startGateway = async () => {
|
||||
await recoveryGate;
|
||||
operations.push("recovery completed");
|
||||
service.status.state = "running";
|
||||
return service.getStatus();
|
||||
};
|
||||
service.stopGateway = async () => {
|
||||
operations.push("stopped");
|
||||
service.status.state = "stopped";
|
||||
return service.getStatus();
|
||||
};
|
||||
await service.handleCoreGatewayTermination(child, undefined, "heap OOM");
|
||||
t.mock.timers.tick(1_000);
|
||||
const stopping = service.stop();
|
||||
releaseRecovery();
|
||||
await stopping;
|
||||
await Promise.resolve();
|
||||
assert.deepEqual(operations, ["recovery completed", "stopped"]);
|
||||
assert.equal(service.getStatus().state, "stopped");
|
||||
t.mock.timers.tick(60_000);
|
||||
assert.equal(operations.length, 2);
|
||||
});
|
||||
|
||||
test("#1867 explicit start waits for an already running recovery", async (t) => {
|
||||
t.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const { child, service } = crashedService();
|
||||
const operations = [];
|
||||
let releaseRecovery;
|
||||
const recoveryGate = new Promise((resolve) => { releaseRecovery = resolve; });
|
||||
service.startGateway = async (_config, generation) => {
|
||||
if (generation !== undefined) {
|
||||
await recoveryGate;
|
||||
operations.push("recovery completed");
|
||||
} else {
|
||||
operations.push("manual start");
|
||||
}
|
||||
service.status.state = "running";
|
||||
return service.getStatus();
|
||||
};
|
||||
await service.handleCoreGatewayTermination(child, undefined, "heap OOM");
|
||||
t.mock.timers.tick(1_000);
|
||||
const starting = service.start(service.config);
|
||||
releaseRecovery();
|
||||
await starting;
|
||||
await Promise.resolve();
|
||||
assert.deepEqual(operations, ["recovery completed", "manual start"]);
|
||||
t.mock.timers.tick(60_000);
|
||||
assert.equal(operations.length, 2);
|
||||
});
|
||||
|
||||
test("#1867 an obsolete child's exit cannot restart its replacement", async (t) => {
|
||||
t.mock.timers.enable({ apis: ["setTimeout"] });
|
||||
const { service } = crashedService();
|
||||
let restarts = 0;
|
||||
service.startGateway = async () => { restarts += 1; return service.getStatus(); };
|
||||
await service.handleCoreGatewayTermination({}, undefined, "old child");
|
||||
t.mock.timers.tick(60_000);
|
||||
assert.equal(restarts, 0);
|
||||
assert.equal(service.getStatus().state, "running");
|
||||
});
|
||||
|
|
@ -26,6 +26,75 @@ import { ccrRemoteControlPathPrefix } from "@ccr/core/gateway/remote-control-ser
|
|||
import { gatewayRuntimeConfigControlPath, gatewayRuntimeConfigRevision } from "@ccr/core/gateway/runtime-config-control.ts";
|
||||
import { createGatewayPlugin } from "@ccr/core/gateway/core-runtime/router-plugin.ts";
|
||||
import { providerRuntimeId } from "@ccr/core/routing/model-registry.ts";
|
||||
import { sanitizeHeaderValue, toCoreGatewayProviders } from "@ccr/core/providers/runtime-topology.ts";
|
||||
|
||||
test("CCR route resolver preserves the selected protocol for bare provider IDs", async (t) => {
|
||||
const chat = { type: "openai_chat_completions", baseUrl: "https://example.test/v1" };
|
||||
const responses = { type: "openai_responses", baseUrl: "https://example.test/v1" };
|
||||
const anthropic = { type: "anthropic_messages", baseUrl: "https://example.test/anthropic" };
|
||||
const credentials = [
|
||||
{ id: "first", apiKey: "test-first", priority: 1 },
|
||||
{ id: "second", apiKey: "test-second", priority: 2 }
|
||||
];
|
||||
for (const scenario of [
|
||||
{ name: "Kimi prefers Anthropic despite OpenAI appearing first" },
|
||||
{ name: "GLM also prefers Anthropic", model: "glm-5.3" },
|
||||
{
|
||||
name: "OpenAI clients prefer OpenAI despite Anthropic appearing first",
|
||||
capabilities: [anthropic, responses, chat], path: "/v1/chat/completions", target: "multi::openai_chat_completions"
|
||||
},
|
||||
{
|
||||
name: "Responses clients prefer Responses over both other protocols",
|
||||
capabilities: [anthropic, chat, responses], path: "/v1/responses", target: "multi::openai_responses"
|
||||
},
|
||||
{
|
||||
name: "missing Anthropic capability still permits conversion",
|
||||
capabilities: [chat], target: "multi::openai_chat_completions"
|
||||
},
|
||||
{
|
||||
name: "bare IDs select a concrete credential",
|
||||
credentials, target: "multi::anthropic_messages::cred:first"
|
||||
},
|
||||
{
|
||||
name: "explicit credential selection is preserved",
|
||||
credentials, selector: "multi::anthropic_messages::cred:second",
|
||||
target: "multi::anthropic_messages::cred:second"
|
||||
},
|
||||
{ name: "legacy providers keep their bare runtime ID", capabilities: [], target: "multi" },
|
||||
{
|
||||
name: "legacy providers with credentials select a concrete credential",
|
||||
capabilities: [], credentials, target: "multi::anthropic_messages::cred:first"
|
||||
}
|
||||
]) {
|
||||
await t.test(scenario.name, async () => {
|
||||
const config = createDefaultAppConfig();
|
||||
const provider = {
|
||||
id: "multi", name: "供应商 multi", type: "anthropic_messages",
|
||||
api_base_url: anthropic.baseUrl, models: ["kimi-k3", "glm-5.3"],
|
||||
capabilities: scenario.capabilities ?? [chat, responses, anthropic], credentials: scenario.credentials
|
||||
};
|
||||
config.Providers = [provider];
|
||||
const plugin = await createGatewayPlugin({ plugin: { config: { appConfig: config } } });
|
||||
const resolver = plugin.routeResolvers.find((item) => item.key === ccrRouterRouteResolverKey);
|
||||
const model = scenario.model ?? "kimi-k3";
|
||||
const target = scenario.target ?? "multi::anthropic_messages";
|
||||
const requestBody = { model: `${target}/${model}`, messages: [{ role: "user", content: "hello" }] };
|
||||
// The outer pipeline has qualified the body model, but its route header
|
||||
// still contains the original selector after non-ASCII names are sanitized.
|
||||
const resolved = resolver.resolve({
|
||||
request: {
|
||||
headers: { [ccrRoutedModelHeader]: sanitizeHeaderValue(`${scenario.selector ?? provider.name}/${model}`) },
|
||||
method: "POST", url: scenario.path ?? "/v1/messages"
|
||||
},
|
||||
requestBody
|
||||
});
|
||||
assert.equal(resolved.targetProviderName, target);
|
||||
assert.ok(toCoreGatewayProviders(provider).some((item) => item.name === resolved.targetProviderName));
|
||||
assert.equal(resolved.model, model);
|
||||
assert.deepEqual(resolved.requestBody, { ...requestBody, model });
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
test("CCR router core plugin exposes route endpoint and beforeRouting transform", async () => {
|
||||
const config = createDefaultAppConfig();
|
||||
|
|
|
|||
|
|
@ -659,7 +659,7 @@ test("model-chain fallback refreshes the routed model before core gateway resolu
|
|||
{
|
||||
headerModel: "Primary/k3",
|
||||
model: "k3",
|
||||
provider: "Primary"
|
||||
provider: "primary"
|
||||
},
|
||||
{
|
||||
headerModel: "fallback/glm-5.3",
|
||||
|
|
|
|||
|
|
@ -974,6 +974,74 @@ function directorySyncError(code) {
|
|||
return error;
|
||||
}
|
||||
|
||||
test("#1867 source sweep and upload share one durable bundle adoption", async () => {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), "ccr-raw-trace-adoption-race-"));
|
||||
const spoolDirectory = path.join(dir, "spool");
|
||||
const source = path.join(spoolDirectory, "source");
|
||||
mkdirSync(source, { recursive: true });
|
||||
const bodyFile = path.join(source, "upstream_request.json");
|
||||
writeFileSync(bodyFile, '{"model":"same-bundle"}');
|
||||
const manifest = { parts: [{ filePath: bodyFile, partType: "upstream_request" }], requestId: "shared-adoption", turnKey: "shared-request" };
|
||||
writeFileSync(path.join(source, "manifest.json"), JSON.stringify(manifest));
|
||||
let releaseSync;
|
||||
const syncGate = new Promise(resolve => { releaseSync = resolve; });
|
||||
let syncing = false;
|
||||
const synchronizer = new RawTraceSynchronizer({
|
||||
enqueueUpdate: async () => false,
|
||||
getConfig: createConfig,
|
||||
replayIntervalMs: 10000,
|
||||
spoolDirectory,
|
||||
syncPartFile: async () => { syncing = true; await syncGate; }
|
||||
});
|
||||
try {
|
||||
await synchronizer.start();
|
||||
await waitFor(() => syncing);
|
||||
const upload = sendRawTrace(synchronizer, manifest);
|
||||
// Let the upload reach the source already moved into staging by the sweep.
|
||||
await new Promise(resolve => setTimeout(resolve, 20));
|
||||
releaseSync();
|
||||
const result = await upload;
|
||||
assert.equal(result.statusCode, 202, JSON.stringify(result.body));
|
||||
assert.equal(result.body.durable, true);
|
||||
assert.equal(readdirSync(path.join(spoolDirectory, ".ccr-inbox")).length, 1);
|
||||
assert.equal(readdirSync(path.join(spoolDirectory, ".ccr-staging")).length, 0);
|
||||
} finally {
|
||||
releaseSync?.();
|
||||
await synchronizer.stop();
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("#1867 broken source bundles expire even below inbox capacity", async () => {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), "ccr-raw-trace-broken-source-"));
|
||||
const spoolDirectory = path.join(dir, "spool");
|
||||
const source = path.join(spoolDirectory, "broken");
|
||||
mkdirSync(source, { recursive: true });
|
||||
writeFileSync(path.join(source, "manifest.json"), JSON.stringify({
|
||||
parts: [{ filePath: path.join(source, "missing.json"), partType: "upstream_request" }],
|
||||
requestId: "broken-source", turnKey: "broken-request"
|
||||
}));
|
||||
const synchronizer = new RawTraceSynchronizer({
|
||||
bundleMaxAgeMs: 20,
|
||||
enqueueUpdate: async () => false,
|
||||
getConfig: createConfig,
|
||||
inboxMaxBundles: 100,
|
||||
replayIntervalMs: 5,
|
||||
sourceBundleGraceMs: 60,
|
||||
sourceScanIntervalMs: 1,
|
||||
spoolDirectory
|
||||
});
|
||||
try {
|
||||
await synchronizer.start();
|
||||
await waitFor(() => !existsSync(source), 250);
|
||||
await waitFor(() => readdirSync(path.join(spoolDirectory, ".ccr-dead-letter")).length === 1, 250);
|
||||
assert.equal(readdirSync(path.join(spoolDirectory, ".ccr-inbox")).length, 0);
|
||||
} finally {
|
||||
await synchronizer.stop();
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
function createConfig() {
|
||||
const config = createDefaultAppConfig();
|
||||
config.observability.requestLogs = true;
|
||||
|
|
|
|||
|
|
@ -68,6 +68,139 @@ function scriptInput(script, body = {}) {
|
|||
});
|
||||
}
|
||||
|
||||
function tokenProbeBody() {
|
||||
return { model: "Provider/alpha", messages: [{ role: "user", content: "token-estimate-probe " + "x ".repeat(100_000) }] };
|
||||
}
|
||||
function tokenProbe(t) {
|
||||
let scans = 0;
|
||||
const original = String.prototype.match;
|
||||
t.mock.method(String.prototype, "match", function (...args) {
|
||||
if (this.includes("token-estimate-probe")) scans += 1;
|
||||
return original.apply(this, args);
|
||||
});
|
||||
return () => scans;
|
||||
}
|
||||
function routingInput(body = tokenProbeBody()) {
|
||||
return { body, bodyOwnership: "owned", headers: {}, method: "POST", url: "/v1/messages" };
|
||||
}
|
||||
|
||||
test("#1869 routes without token consumers skip conversation estimation; count_tokens remains accurate", async (t) => {
|
||||
const plugin = new ClaudeCodeRouterPlugin(routingConfig());
|
||||
const scans = tokenProbe(t);
|
||||
const result = await plugin.routeRequest(routingInput());
|
||||
assert.equal(scans(), 0);
|
||||
assert.equal(result.decision.tokenCount, 0);
|
||||
assert.ok(plugin.countTokens(tokenProbeBody()).input_tokens > 100_000);
|
||||
assert.equal(scans(), 2);
|
||||
});
|
||||
|
||||
test("#1869 token condition aliases estimate once and keep selecting the same route", async (t) => {
|
||||
for (const key of ["tokenCount", "token_count"]) {
|
||||
const plugin = new ClaudeCodeRouterPlugin(routingConfig([{
|
||||
id: key, name: key, type: "condition", enabled: true,
|
||||
condition: { left: `request.${key}`, operator: ">", right: "100000" },
|
||||
rewrites: [{ key: "request.body.model", operation: "set", value: "Provider/beta" }],
|
||||
}]));
|
||||
const expected = plugin.countTokens(tokenProbeBody()).input_tokens;
|
||||
const scans = tokenProbe(t);
|
||||
const result = await plugin.routeRequest(routingInput());
|
||||
assert.equal(result.decision.model, "Provider/beta");
|
||||
assert.equal(result.decision.tokenCount, expected);
|
||||
assert.equal(scans(), 2);
|
||||
t.mock.restoreAll();
|
||||
}
|
||||
});
|
||||
|
||||
test("#1869 scripts estimate tokens lazily in the worker only when read", async (t) => {
|
||||
const runtime = new RouteScriptRuntime({ workerCount: 1, workerFile });
|
||||
try {
|
||||
for (const readTokens of [false, true]) {
|
||||
const script = routeScript(readTokens ? 'return input.tokenCount > 100000 ? { model: "Provider/beta" } : null;' : 'return null;');
|
||||
const plugin = new ClaudeCodeRouterPlugin(routingConfig([scriptRule("tokens", script)]), { scriptRuntime: runtime });
|
||||
const expected = readTokens ? plugin.countTokens(tokenProbeBody()).input_tokens : 0;
|
||||
const scans = tokenProbe(t);
|
||||
const result = await plugin.routeRequest(routingInput());
|
||||
assert.equal(scans(), 0, "host must not scan a conversation for script-only token consumers");
|
||||
assert.equal(result.decision.tokenCount, expected);
|
||||
assert.equal(result.decision.model, readTokens ? "Provider/beta" : "Provider/alpha");
|
||||
t.mock.restoreAll();
|
||||
}
|
||||
} finally { await runtime.close(); }
|
||||
});
|
||||
|
||||
test("#1869 provider routing settings retain the token estimate for downstream selection", async () => {
|
||||
const config = routingConfig();
|
||||
config.Providers[0].modelMetadata = { alpha: { openRouterDiscountRouting: { enabled: true } } };
|
||||
const plugin = new ClaudeCodeRouterPlugin(config);
|
||||
const expected = plugin.countTokens(tokenProbeBody()).input_tokens;
|
||||
const result = await plugin.routeRequest(routingInput());
|
||||
assert.equal(result.decision.tokenCount, expected);
|
||||
});
|
||||
|
||||
test("#1869 custom routers retain the token getter without forcing unused estimates", async (t) => {
|
||||
const file = path.join(routeScriptDirectory, "token-router.cjs");
|
||||
for (const readTokens of [false, true]) {
|
||||
writeFileSync(file, readTokens ? 'module.exports = request => request.tokenCount > 100000 && request.tokenCount > 100000 ? "Provider/beta" : undefined;' : 'module.exports = () => undefined;');
|
||||
const plugin = new ClaudeCodeRouterPlugin({ ...routingConfig(), CUSTOM_ROUTER_PATH: file });
|
||||
const scans = tokenProbe(t);
|
||||
const result = await plugin.routeRequest(routingInput());
|
||||
assert.equal(scans(), readTokens ? 2 : 0);
|
||||
assert.equal(result.decision.model, readTokens ? "Provider/beta" : "Provider/alpha");
|
||||
t.mock.restoreAll();
|
||||
}
|
||||
});
|
||||
|
||||
test("#1869 scripts preserve custom router token overrides and cached estimates", async (t) => {
|
||||
const runtime = new RouteScriptRuntime({ workerCount: 1, workerFile });
|
||||
const file = path.join(routeScriptDirectory, "token-override-router.cjs");
|
||||
try {
|
||||
for (const scenario of [
|
||||
{ router: 'module.exports = request => { request.tokenCount = 7; };', predicate: "input.tokenCount === 7", count: 7, scans: 0 },
|
||||
{ router: 'module.exports = request => { request.tokenCount = 0; };', predicate: "input.tokenCount === 0", count: 0, scans: 0 },
|
||||
{ router: 'module.exports = request => { void request.tokenCount; request.body.messages = []; };', predicate: "input.tokenCount > 100000", scans: 2 }
|
||||
]) {
|
||||
writeFileSync(file, scenario.router);
|
||||
const script = routeScript(`return ${scenario.predicate} ? { model: "Provider/beta" } : null;`);
|
||||
const config = { ...routingConfig([scriptRule("override", script)]), CUSTOM_ROUTER_PATH: file };
|
||||
const plugin = new ClaudeCodeRouterPlugin(config, { scriptRuntime: runtime });
|
||||
const expected = scenario.count ?? plugin.countTokens(tokenProbeBody()).input_tokens;
|
||||
const scans = tokenProbe(t);
|
||||
const result = await plugin.routeRequest(routingInput());
|
||||
assert.equal(result.decision.model, "Provider/beta");
|
||||
assert.equal(result.decision.tokenCount, expected);
|
||||
assert.equal(scans(), scenario.scans);
|
||||
t.mock.restoreAll();
|
||||
}
|
||||
} finally { await runtime.close(); }
|
||||
});
|
||||
|
||||
test("#1868 preparing a script input does not parse a copy of the full request on the host", (t) => {
|
||||
const originalParse = JSON.parse;
|
||||
let bodyCopies = 0;
|
||||
t.mock.method(JSON, "parse", (text, ...args) => {
|
||||
if (typeof text === "string" && text.includes("large-copy-probe")) bodyCopies += 1;
|
||||
return originalParse(text, ...args);
|
||||
});
|
||||
const body = { model: "Provider/alpha", messages: [{ role: "user", content: "large-copy-probe " + "x".repeat(2_000_000) }] };
|
||||
const input = scriptInput({}, body);
|
||||
assert.equal(input.model, "Provider/alpha");
|
||||
assert.equal(bodyCopies, 0);
|
||||
});
|
||||
|
||||
test("#1868 worker receives an isolated snapshot even if the original body changes", async () => {
|
||||
const runtime = new RouteScriptRuntime({ workerCount: 1, workerFile });
|
||||
const script = routeScript('return { model: input.body.model, text: input.body.messages[0].content, frozen: Object.isFrozen(input.body.messages[0]) };');
|
||||
const body = { model: "Provider/alpha", messages: [{ role: "user", content: "original" }] };
|
||||
const input = scriptInput(script, body);
|
||||
body.model = "Provider/beta";
|
||||
body.messages[0].content = "mutated";
|
||||
try {
|
||||
const execution = await runtime.execute("snapshot", script, input);
|
||||
assert.equal(execution.status, "ok", execution.error);
|
||||
assert.deepEqual(execution.value, { model: "Provider/alpha", text: "original", frozen: true });
|
||||
} finally { await runtime.close(); }
|
||||
});
|
||||
|
||||
test("route script input exposes the complete request body and headers", () => {
|
||||
const script = routeScript("return null;");
|
||||
const input = scriptInput(script, {
|
||||
|
|
|
|||
|
|
@ -1,306 +0,0 @@
|
|||
From c6cba47ffe2d9349112d14675d83fc5cdd937bff Mon Sep 17 00:00:00 2001
|
||||
From: musistudio <m@musiiot.top>
|
||||
Date: Sun, 27 Sep 2026 06:43:11 +0800
|
||||
Subject: [PATCH] Fix Anthropic tool IDs and Fusion stream raw trace capture
|
||||
|
||||
---
|
||||
.../builtins/anthropic-tool-id.test.ts | 39 +++++++++++++++++++
|
||||
src/adapters/builtins/anthropic-tool-id.ts | 8 ++++
|
||||
src/adapters/builtins/source/formatters.ts | 3 +-
|
||||
.../builtins/target/anthropic-messages.ts | 9 +++--
|
||||
src/gateway/handler.ts | 27 +++++++++----
|
||||
src/gateway/routes.test.ts | 17 +++++++-
|
||||
src/gateway/streaming-conversion.ts | 3 +-
|
||||
7 files changed, 92 insertions(+), 14 deletions(-)
|
||||
create mode 100644 src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
create mode 100644 src/adapters/builtins/anthropic-tool-id.ts
|
||||
|
||||
diff --git a/src/adapters/builtins/anthropic-tool-id.test.ts b/src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
new file mode 100644
|
||||
index 0000000..479b045
|
||||
--- /dev/null
|
||||
+++ b/src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
@@ -0,0 +1,39 @@
|
||||
+import { expect, it } from 'vitest';
|
||||
+import { anthropicToolId } from './anthropic-tool-id';
|
||||
+import { formatAnthropicMessagesResponse } from './source/formatters';
|
||||
+import { anthropicMessagesTargetAdapter } from './target/anthropic-messages';
|
||||
+import { createOptimisticOpenAIChatStreamRelay, relayOptimisticOpenAIChatStreamToolCalls } from '../../gateway/streaming-conversion';
|
||||
+
|
||||
+it('preserves valid IDs and maps distinct invalid IDs deterministically', () => {
|
||||
+ expect(anthropicToolId('toolu_valid-1')).toBe('toolu_valid-1');
|
||||
+ expect(anthropicToolId('Bash:0')).toMatch(/^[A-Za-z0-9_-]+$/);
|
||||
+ expect(anthropicToolId('Bash:0')).not.toBe(anthropicToolId('Bash/0'));
|
||||
+ expect(anthropicToolId('Bash:0')).toBe(anthropicToolId('Bash:0'));
|
||||
+});
|
||||
+
|
||||
+it('sanitizes JSON and streamed Anthropic tool calls and pairs replayed results', () => {
|
||||
+ const call = { type: 'function_call', id: 'Bash:0', call_id: 'Bash:0', name: 'Bash', arguments: '{}', status: 'completed' } as const;
|
||||
+ const response = { id: 'message', model: 'model', object: 'response', status: 'completed', output: [call], usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 } } as any;
|
||||
+ const formatted = formatAnthropicMessagesResponse(response) as any;
|
||||
+ const id = formatted.content[0].id;
|
||||
+ expect(id).toBe(anthropicToolId('Bash:0'));
|
||||
+ const relay = createOptimisticOpenAIChatStreamRelay({ adapterKey: 'anthropic_messages' } as any, { model: 'model', input: 'hello' });
|
||||
+ expect(relay).toBeTruthy();
|
||||
+ const frames = relayOptimisticOpenAIChatStreamToolCalls(relay!, [call]).join('');
|
||||
+ expect(frames).toContain(`"id":"${id}"`);
|
||||
+ expect(frames).not.toContain('Bash:0');
|
||||
+ const built = anthropicMessagesTargetAdapter.buildRequestFromStandard({
|
||||
+ request: { headers: {} } as any,
|
||||
+ standardRequest: { model: 'model', input: [
|
||||
+ { type: 'message', role: 'assistant', content: [{ type: 'tool_use', id: 'Bash:0', name: 'Bash', input: {} }] },
|
||||
+ { type: 'message', role: 'user', content: [{ type: 'tool_result', tool_use_id: 'Bash:0', content: 'ok' }] }
|
||||
+ ] },
|
||||
+ config: { anthropicApiKey: 'test', anthropicBaseUrl: 'https://example.test' } as any
|
||||
+ });
|
||||
+ expect(built.ok).toBe(true);
|
||||
+ if (built.ok) {
|
||||
+ const body = built.value.body as any;
|
||||
+ expect(body.messages[0].content[0].id).toBe(id);
|
||||
+ expect(body.messages[1].content[0].tool_use_id).toBe(id);
|
||||
+ }
|
||||
+});
|
||||
diff --git a/src/adapters/builtins/anthropic-tool-id.ts b/src/adapters/builtins/anthropic-tool-id.ts
|
||||
new file mode 100644
|
||||
index 0000000..5c40e80
|
||||
--- /dev/null
|
||||
+++ b/src/adapters/builtins/anthropic-tool-id.ts
|
||||
@@ -0,0 +1,8 @@
|
||||
+import { createHash, randomUUID } from 'node:crypto';
|
||||
+
|
||||
+/** Keep calls and results paired, including when replaying a different protocol's history. */
|
||||
+export function anthropicToolId(value: string | undefined): string {
|
||||
+ if (!value) return `toolu_${randomUUID().replace(/-/g, '')}`;
|
||||
+ if (/^[A-Za-z0-9_-]+$/.test(value)) return value;
|
||||
+ return `toolu_ccr_${createHash('sha256').update(value).digest('hex').slice(0, 48)}`;
|
||||
+}
|
||||
diff --git a/src/adapters/builtins/source/formatters.ts b/src/adapters/builtins/source/formatters.ts
|
||||
index a415dec..63ff75a 100644
|
||||
--- a/src/adapters/builtins/source/formatters.ts
|
||||
+++ b/src/adapters/builtins/source/formatters.ts
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { StandardResponse, StandardResponseReasoning } from '../../../types';
|
||||
+import { anthropicToolId } from '../anthropic-tool-id';
|
||||
import { anthropicInputTokens } from '../../../shared/usage';
|
||||
import {
|
||||
mapFinishReasonToAnthropic,
|
||||
@@ -196,7 +197,7 @@ function collectAnthropicContentBlocks(response: StandardResponse): Array<Record
|
||||
|
||||
const block: Record<string, unknown> = {
|
||||
type: 'tool_use',
|
||||
- id: item.call_id || item.id,
|
||||
+ id: anthropicToolId(item.call_id || item.id),
|
||||
name: item.name,
|
||||
input: parseFunctionArguments(item.arguments)
|
||||
};
|
||||
diff --git a/src/adapters/builtins/target/anthropic-messages.ts b/src/adapters/builtins/target/anthropic-messages.ts
|
||||
index 337f274..8440d86 100644
|
||||
--- a/src/adapters/builtins/target/anthropic-messages.ts
|
||||
+++ b/src/adapters/builtins/target/anthropic-messages.ts
|
||||
@@ -359,7 +359,7 @@ function standardContentToAnthropicBlocks(
|
||||
if (item.type === 'tool_use') {
|
||||
blocks.push({
|
||||
type: 'tool_use',
|
||||
- id: item.id,
|
||||
+ id: anthropicToolId(item.id),
|
||||
name: mapStandardToolNameToTargetName(item.name, toolContext.functionTools),
|
||||
input: isObject(item.input) ? item.input : {}
|
||||
});
|
||||
@@ -369,7 +369,7 @@ function standardContentToAnthropicBlocks(
|
||||
if (item.type === 'tool_search_call') {
|
||||
blocks.push({
|
||||
type: 'tool_use',
|
||||
- id: item.call_id,
|
||||
+ id: anthropicToolId(item.call_id),
|
||||
name: 'ToolSearch',
|
||||
input: normalizeAnthropicToolInput(item.arguments)
|
||||
});
|
||||
@@ -384,7 +384,7 @@ function standardContentToAnthropicBlocks(
|
||||
}));
|
||||
blocks.push({
|
||||
type: 'tool_result',
|
||||
- tool_use_id: item.call_id,
|
||||
+ tool_use_id: anthropicToolId(item.call_id),
|
||||
content: references.length > 0 ? references : serializeStandardToolSearchOutput(item)
|
||||
});
|
||||
continue;
|
||||
@@ -417,7 +417,7 @@ function standardContentToAnthropicBlocks(
|
||||
: [];
|
||||
const toolResultBlock: Record<string, unknown> = {
|
||||
type: 'tool_result',
|
||||
- tool_use_id: item.tool_use_id,
|
||||
+ tool_use_id: anthropicToolId(item.tool_use_id),
|
||||
content:
|
||||
referenceBlocks.length > 0
|
||||
? [
|
||||
@@ -732,3 +732,4 @@ function isAnthropicToolChoiceNone(toolChoice: unknown): boolean {
|
||||
|
||||
return asString(toolChoice.type) === 'none';
|
||||
}
|
||||
+import { anthropicToolId } from '../anthropic-tool-id';
|
||||
diff --git a/src/gateway/handler.ts b/src/gateway/handler.ts
|
||||
index 21e0fe1..b92ba3f 100644
|
||||
--- a/src/gateway/handler.ts
|
||||
+++ b/src/gateway/handler.ts
|
||||
@@ -3218,7 +3218,8 @@ async function handleVirtualModelRequest(
|
||||
isStreaming,
|
||||
model,
|
||||
rewrittenBaseRequest,
|
||||
- lastUpstreamRequest
|
||||
+ lastUpstreamRequest,
|
||||
+ upstreamResponse
|
||||
);
|
||||
}
|
||||
|
||||
@@ -3277,7 +3278,8 @@ async function handleVirtualModelRequest(
|
||||
isStreaming,
|
||||
model,
|
||||
rewrittenBaseRequest,
|
||||
- lastUpstreamRequest
|
||||
+ lastUpstreamRequest,
|
||||
+ upstreamResponse
|
||||
);
|
||||
}
|
||||
|
||||
@@ -4587,7 +4589,8 @@ async function* runOptimisticVirtualModelStream(input: {
|
||||
publishOptimisticVirtualModelBillingEvent(
|
||||
input,
|
||||
finalResponse,
|
||||
- lastAttemptSequence
|
||||
+ lastAttemptSequence,
|
||||
+ buildUpstreamRawTraceCapture(upstreamRequest, upstreamResponse, finalResponse)
|
||||
);
|
||||
const visibleFunctionCalls = extractFunctionCallsFromStandardResponse(finalResponse);
|
||||
if (usesNativeAnthropicTarget) {
|
||||
@@ -4619,7 +4622,8 @@ async function* runOptimisticVirtualModelStream(input: {
|
||||
publishOptimisticVirtualModelBillingEvent(
|
||||
input,
|
||||
finalResponse,
|
||||
- lastAttemptSequence
|
||||
+ lastAttemptSequence,
|
||||
+ buildUpstreamRawTraceCapture(upstreamRequest, upstreamResponse, finalResponse)
|
||||
);
|
||||
if (usesNativeAnthropicTarget) {
|
||||
yield* relayOptimisticAnthropicDeferredContent(relay, turnResult, []);
|
||||
@@ -4764,7 +4768,8 @@ function publishOptimisticVirtualModelBillingEvent(
|
||||
model: string;
|
||||
},
|
||||
response: StandardResponse,
|
||||
- attemptSequence: number
|
||||
+ attemptSequence: number,
|
||||
+ rawTraceCapture: GatewayRawTraceCapture
|
||||
): void {
|
||||
recordGatewaySchedulingUsage({
|
||||
config: input.config,
|
||||
@@ -4775,6 +4780,8 @@ function publishOptimisticVirtualModelBillingEvent(
|
||||
});
|
||||
|
||||
if (!input.config.billing.enabled || !input.reply) {
|
||||
+ publishRawTraceCaptureSafe(input.request, input.config, input.targetProvider,
|
||||
+ response.model || input.model, input.targetProviderConfig, rawTraceCapture);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -4813,7 +4820,8 @@ function publishOptimisticVirtualModelBillingEvent(
|
||||
buildGatewayBillingTraceSnapshot(input.request, input.reply, {
|
||||
billingTrace: input.config.billing.trace,
|
||||
standardResponse: response
|
||||
- })
|
||||
+ }),
|
||||
+ rawTraceCapture
|
||||
).catch(() => undefined);
|
||||
}
|
||||
|
||||
@@ -4831,7 +4839,8 @@ async function sendVirtualModelResponse(
|
||||
streaming: boolean,
|
||||
model: string,
|
||||
standardRequest?: StandardRequest,
|
||||
- upstreamRequest?: UpstreamRequest
|
||||
+ upstreamRequest?: UpstreamRequest,
|
||||
+ upstreamResponse?: Response
|
||||
) {
|
||||
attachTargetRoutingHeaders(reply, targetProvider, targetProviderConfig?.name, fallbackAttempts, targetProviderConfig);
|
||||
|
||||
@@ -4864,6 +4873,8 @@ async function sendVirtualModelResponse(
|
||||
}),
|
||||
{
|
||||
upstreamRequest,
|
||||
+ upstreamResponseStatus: upstreamResponse?.status,
|
||||
+ upstreamResponseHeaders: upstreamResponse ? normalizeHeadersForRawTrace(upstreamResponse.headers) : undefined,
|
||||
upstreamResponseBody: sourcePayload
|
||||
}
|
||||
);
|
||||
@@ -4889,6 +4900,8 @@ async function sendVirtualModelResponse(
|
||||
}),
|
||||
{
|
||||
upstreamRequest,
|
||||
+ upstreamResponseStatus: upstreamResponse?.status,
|
||||
+ upstreamResponseHeaders: upstreamResponse ? normalizeHeadersForRawTrace(upstreamResponse.headers) : undefined,
|
||||
upstreamResponseBody: sourceStandardResponse
|
||||
}
|
||||
);
|
||||
diff --git a/src/gateway/routes.test.ts b/src/gateway/routes.test.ts
|
||||
index 73aec02..692b7fc 100644
|
||||
--- a/src/gateway/routes.test.ts
|
||||
+++ b/src/gateway/routes.test.ts
|
||||
@@ -11351,7 +11351,7 @@ export function createGatewayPlugin() {
|
||||
}
|
||||
});
|
||||
|
||||
- it('optimistically streams virtual model content while intercepting internal tool calls', async () => {
|
||||
+ it.each([true, false])('optimistically streams virtual model content and records raw trace with billing=%s', async (billingEnabled) => {
|
||||
const firstStream = [
|
||||
'data: {"id":"chatcmpl_virtual_websearch_optimistic_1","object":"chat.completion.chunk","model":"glm-5","choices":[{"index":0,"delta":{"role":"assistant"}}]}\n\n',
|
||||
'data: {"id":"chatcmpl_virtual_websearch_optimistic_1","object":"chat.completion.chunk","model":"glm-5","choices":[{"index":0,"delta":{"reasoning_content":"think "}}]}\n\n',
|
||||
@@ -11452,6 +11452,11 @@ export function createGatewayPlugin() {
|
||||
]
|
||||
);
|
||||
|
||||
+ const spoolDir = await mkdtemp(join(tmpdir(), 'gateway-fusion-trace-'));
|
||||
+ config.billing.enabled = billingEnabled;
|
||||
+ config.rawTrace = { ...config.rawTrace, enabled: true, mode: 'wire_raw', spoolDir, sync: { ...config.rawTrace.sync, enabled: false } };
|
||||
+ await initializeRawTraceManager(config.rawTrace);
|
||||
+
|
||||
const app = Fastify({ logger: false });
|
||||
registerGatewayRoutes(app, config, createGatewayRuntime(config, toolProvider as any));
|
||||
await app.ready();
|
||||
@@ -11499,8 +11504,18 @@ export function createGatewayPlugin() {
|
||||
expect(completedLine).toBeDefined();
|
||||
const completedEvent = JSON.parse(String(completedLine).slice('data: '.length));
|
||||
expect(completedEvent.response.id).toBe('chatcmpl_virtual_websearch_optimistic_1');
|
||||
+ const { bundleDir, manifest } = await waitForRawTraceManifest(spoolDir);
|
||||
+ const parts = manifest.parts as Array<{ partType: string; filePath: string }>;
|
||||
+ const metadata = JSON.parse(await readFile(parts.find(p => p.partType === 'upstream_response_metadata')!.filePath, 'utf8'));
|
||||
+ const body = JSON.parse(await readFile(join(bundleDir, 'upstream_response.json'), 'utf8'));
|
||||
+ expect(metadata.statusCode).toBe(200);
|
||||
+ expect(body.usage.input_tokens).toBe(11);
|
||||
+ expect(body.usage.output_tokens).toBe(7);
|
||||
+ expect(body.output_text).toBe('fresh answer');
|
||||
} finally {
|
||||
+ await closeRawTraceManager();
|
||||
await app.close();
|
||||
+ await rm(spoolDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
diff --git a/src/gateway/streaming-conversion.ts b/src/gateway/streaming-conversion.ts
|
||||
index b3c4fa4..85f9757 100644
|
||||
--- a/src/gateway/streaming-conversion.ts
|
||||
+++ b/src/gateway/streaming-conversion.ts
|
||||
@@ -5600,7 +5600,7 @@ function buildAnthropicToolUseStartFrame(toolCall: PendingAnthropicToolCall): st
|
||||
index: toolCall.blockIndex,
|
||||
content_block: {
|
||||
type: 'tool_use',
|
||||
- id: toolCall.id,
|
||||
+ id: anthropicToolId(toolCall.id),
|
||||
name: toolCall.name,
|
||||
input: {}
|
||||
}
|
||||
@@ -6930,3 +6930,4 @@ function encodeSseDataLines(data: unknown): string {
|
||||
const lines = serialized.split('\n');
|
||||
return lines.map((line) => `data: ${line}`).join('\n');
|
||||
}
|
||||
+import { anthropicToolId } from '../adapters/builtins/anthropic-tool-id';
|
||||
--
|
||||
2.50.1 (Apple Git-155)
|
||||
|
||||
|
|
@ -1,171 +0,0 @@
|
|||
diff --git a/src/adapters/builtins/anthropic-tool-id.test.ts b/src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
--- a/src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
+++ b/src/adapters/builtins/anthropic-tool-id.test.ts
|
||||
@@ -2,6 +2,7 @@
|
||||
import { anthropicToolId } from './anthropic-tool-id';
|
||||
import { formatAnthropicMessagesResponse } from './source/formatters';
|
||||
import { anthropicMessagesTargetAdapter } from './target/anthropic-messages';
|
||||
+import { anthropicMessagesSourceAdapter } from './source/anthropic-messages';
|
||||
import { createOptimisticOpenAIChatStreamRelay, relayOptimisticOpenAIChatStreamToolCalls } from '../../gateway/streaming-conversion';
|
||||
|
||||
it('preserves valid IDs and maps distinct invalid IDs deterministically', () => {
|
||||
@@ -9,6 +10,22 @@
|
||||
expect(anthropicToolId('Bash:0')).toMatch(/^[A-Za-z0-9_-]+$/);
|
||||
expect(anthropicToolId('Bash:0')).not.toBe(anthropicToolId('Bash/0'));
|
||||
expect(anthropicToolId('Bash:0')).toBe(anthropicToolId('Bash:0'));
|
||||
+});
|
||||
+
|
||||
+it('repairs old poisoned Anthropic histories on the native passthrough path without mutating signed blocks', () => {
|
||||
+ const thinking = { type: 'thinking', thinking: 'opaque', signature: 'keep-exactly' };
|
||||
+ const body = { model: 'native', messages: [
|
||||
+ { role: 'assistant', content: [thinking, { type: 'tool_use', id: 'functions.Bash:0', name: 'Bash', input: {} }] },
|
||||
+ { role: 'user', content: [{ type: 'tool_result', tool_use_id: 'functions.Bash:0', content: 'ok', is_error: false }] }
|
||||
+ ] };
|
||||
+ const built = anthropicMessagesSourceAdapter.buildPassthroughRequest({ body, request: { headers: {} }, config: { anthropicApiKey: 'test', anthropicBaseUrl: 'https://example.test' } } as any);
|
||||
+ expect(built.ok).toBe(true);
|
||||
+ if (!built.ok) return;
|
||||
+ const messages = (built.value.body as any).messages;
|
||||
+ expect(messages[0].content[0]).toEqual(thinking);
|
||||
+ expect(messages[0].content[1].id).toMatch(/^[A-Za-z0-9_-]+$/);
|
||||
+ expect(messages[1].content[0].tool_use_id).toBe(messages[0].content[1].id);
|
||||
+ expect(body.messages[0].content[1]).toHaveProperty('id', 'functions.Bash:0');
|
||||
});
|
||||
|
||||
it('sanitizes JSON and streamed Anthropic tool calls and pairs replayed results', () => {
|
||||
diff --git a/src/adapters/builtins/source/anthropic-messages.ts b/src/adapters/builtins/source/anthropic-messages.ts
|
||||
--- a/src/adapters/builtins/source/anthropic-messages.ts
|
||||
+++ b/src/adapters/builtins/source/anthropic-messages.ts
|
||||
@@ -1,7 +1,8 @@
|
||||
import type { SourceAdapter } from '../../../types';
|
||||
import { ok } from '../../../types';
|
||||
-import { asBoolean } from '../../../utils';
|
||||
+import { asBoolean, isObject } from '../../../utils';
|
||||
import { buildAnthropicHeaders } from '../common';
|
||||
+import { anthropicToolId } from '../anthropic-tool-id';
|
||||
import { formatAnthropicMessagesResponse } from './formatters';
|
||||
import { parseAnthropicMessagesRequest } from './parsers';
|
||||
|
||||
@@ -26,7 +27,22 @@
|
||||
return ok({
|
||||
url: `${input.config.anthropicBaseUrl}/v1/messages`,
|
||||
headers: headersResult.value,
|
||||
- body: input.body
|
||||
+ body: {
|
||||
+ ...input.body,
|
||||
+ messages: Array.isArray(input.body.messages) ? input.body.messages.map((message: unknown) => !isObject(message) || Array.isArray(message) ? message : ({
|
||||
+ ...message,
|
||||
+ content: Array.isArray(message.content) ? message.content.map((block: unknown) => {
|
||||
+ if (!isObject(block) || Array.isArray(block)) return block;
|
||||
+ if (block.type === 'tool_use' || block.type === 'server_tool_use') {
|
||||
+ return { ...block, id: anthropicToolId(typeof block.id === 'string' ? block.id : undefined) };
|
||||
+ }
|
||||
+ if (typeof block.tool_use_id === 'string') {
|
||||
+ return { ...block, tool_use_id: anthropicToolId(block.tool_use_id) };
|
||||
+ }
|
||||
+ return block;
|
||||
+ }) : message.content
|
||||
+ })) : input.body.messages
|
||||
+ }
|
||||
});
|
||||
}
|
||||
};
|
||||
diff --git a/src/gateway/handler.ts b/src/gateway/handler.ts
|
||||
--- a/src/gateway/handler.ts
|
||||
+++ b/src/gateway/handler.ts
|
||||
@@ -248,6 +248,7 @@
|
||||
standardRequest: StandardRequest;
|
||||
standardResponse: StandardResponse;
|
||||
upstreamRequest: UpstreamRequest;
|
||||
+ upstreamResponse: Response;
|
||||
attemptSequence: number;
|
||||
upstreamAttemptSequence: number;
|
||||
}
|
||||
@@ -1911,6 +1912,7 @@
|
||||
}
|
||||
|
||||
let responseUpstreamRequest = upstreamRequest;
|
||||
+ let responseUpstreamResponse = upstreamResponse;
|
||||
let responseAttemptSequence = currentAttemptSequence;
|
||||
let transformedPayload = responsePluginResult.value;
|
||||
let standardPayload = await normalizeOpenAIPayloadForResponseParseRecovery(
|
||||
@@ -1966,6 +1968,7 @@
|
||||
}
|
||||
if (retryResult.ok) {
|
||||
responseUpstreamRequest = retryResult.upstreamRequest;
|
||||
+ responseUpstreamResponse = retryResult.upstreamResponse;
|
||||
responseAttemptSequence = retryResult.attemptSequence;
|
||||
transformedPayload = retryResult.transformedPayload;
|
||||
standardPayload = retryResult.standardPayload;
|
||||
@@ -2002,6 +2005,7 @@
|
||||
initialStandardRequest: standardRequest,
|
||||
initialStandardResponse: standardResponseResult.value,
|
||||
initialUpstreamRequest: responseUpstreamRequest,
|
||||
+ initialUpstreamResponse: responseUpstreamResponse,
|
||||
initialUpstreamResponseBody: transformedPayload,
|
||||
initialAttemptSequence: responseAttemptSequence,
|
||||
state: {
|
||||
@@ -2050,6 +2054,8 @@
|
||||
}),
|
||||
{
|
||||
upstreamRequest: transparentToolExecutionResult.upstreamRequest,
|
||||
+ upstreamResponseStatus: transparentToolExecutionResult.upstreamResponse.status,
|
||||
+ upstreamResponseHeaders: normalizeHeadersForRawTrace(transparentToolExecutionResult.upstreamResponse.headers),
|
||||
upstreamResponseBody: sourcePayload
|
||||
}
|
||||
);
|
||||
@@ -2159,6 +2165,7 @@
|
||||
initialStandardRequest: StandardRequest;
|
||||
initialStandardResponse: StandardResponse;
|
||||
initialUpstreamRequest: UpstreamRequest;
|
||||
+ initialUpstreamResponse: Response;
|
||||
initialUpstreamResponseBody: unknown;
|
||||
initialAttemptSequence: number;
|
||||
state: {
|
||||
@@ -2171,6 +2178,7 @@
|
||||
let workingRequest = input.initialStandardRequest;
|
||||
let lastResponse = input.initialStandardResponse;
|
||||
let lastUpstreamRequest = input.initialUpstreamRequest;
|
||||
+ let lastUpstreamResponse = input.initialUpstreamResponse;
|
||||
let lastAttemptSequence = input.initialAttemptSequence;
|
||||
let lastUpstreamResponseBody = input.initialUpstreamResponseBody;
|
||||
let aggregatedUsage = lastResponse.usage;
|
||||
@@ -2181,6 +2189,7 @@
|
||||
standardRequest: workingRequest,
|
||||
standardResponse: lastResponse,
|
||||
upstreamRequest: lastUpstreamRequest,
|
||||
+ upstreamResponse: lastUpstreamResponse,
|
||||
attemptSequence: lastAttemptSequence,
|
||||
upstreamAttemptSequence
|
||||
};
|
||||
@@ -2200,6 +2209,7 @@
|
||||
usage: aggregatedUsage
|
||||
},
|
||||
upstreamRequest: lastUpstreamRequest,
|
||||
+ upstreamResponse: lastUpstreamResponse,
|
||||
attemptSequence: lastAttemptSequence,
|
||||
upstreamAttemptSequence
|
||||
};
|
||||
@@ -2281,6 +2291,7 @@
|
||||
usage: aggregatedUsage
|
||||
},
|
||||
upstreamRequest: lastUpstreamRequest,
|
||||
+ upstreamResponse: lastUpstreamResponse,
|
||||
attemptSequence: lastAttemptSequence,
|
||||
upstreamAttemptSequence
|
||||
};
|
||||
@@ -2391,6 +2402,7 @@
|
||||
|
||||
const { upstreamRequest, upstreamResponse } = upstreamDispatchResult;
|
||||
lastUpstreamRequest = upstreamRequest;
|
||||
+ lastUpstreamResponse = upstreamResponse;
|
||||
upstreamAttemptSequence += 1;
|
||||
lastAttemptSequence = upstreamAttemptSequence;
|
||||
completedModelTurns += 1;
|
||||
@@ -2521,6 +2533,7 @@
|
||||
}
|
||||
if (retryResult.ok) {
|
||||
lastUpstreamRequest = retryResult.upstreamRequest;
|
||||
+ lastUpstreamResponse = retryResult.upstreamResponse;
|
||||
lastUpstreamResponseBody = retryResult.transformedPayload;
|
||||
lastAttemptSequence = retryResult.attemptSequence;
|
||||
lastResponse = retryResult.standardResponse;
|
||||
|
|
@ -1,37 +0,0 @@
|
|||
# Gateway fixes for CCR issues #1801, #1806 and #1808
|
||||
|
||||
The runtime dependency needs source changes as well as the CCR changes in this branch.
|
||||
|
||||
- Base: `@the-next-ai/ai-gateway` source commit `f061ad4` (the `feat/v1.0.22` branch, package version still 1.0.21).
|
||||
- Fix: `c6cba47` on local gateway branch `codex/ccr-recent-issues`.
|
||||
- Reviewable patch: `0001-fix-tool-ids-and-fusion-traces.patch`, including regression tests.
|
||||
- Follow-up after reproducing the original issues: `0002-complete-native-replay-and-response-status.patch`. This repairs old invalid tool IDs on the native Anthropic passthrough path and retains the final upstream HTTP status/headers for ordinary converted responses, including transparent tool turns and parse retries.
|
||||
- The patch sanitizes Anthropic tool IDs consistently in JSON, SSE and replayed call/result pairs, captures completed optimistic Fusion streams with billing enabled or disabled, and retains real upstream status/headers for buffered Fusion responses.
|
||||
|
||||
## Build from source
|
||||
|
||||
In a gateway source checkout at the base commit, apply both patches in order, install its dependencies, and build:
|
||||
|
||||
```sh
|
||||
git am /absolute/path/to/0001-fix-tool-ids-and-fusion-traces.patch
|
||||
git apply /absolute/path/to/0002-complete-native-replay-and-response-status.patch
|
||||
npm run typecheck
|
||||
npm run build
|
||||
```
|
||||
|
||||
Then build CCR:
|
||||
|
||||
```sh
|
||||
CCR_GATEWAY_SOURCE_DIR=/absolute/path/to/patched/gateway npm run build:assets
|
||||
```
|
||||
|
||||
The second audit used `/tmp/ccr-audit-gateway` as an isolated source snapshot. Temporary directories are not deliverables; the two patches and the committed regression suites are the reproducible artifacts.
|
||||
|
||||
Do not assume a normal dependency install contains these fixes yet. The main dependency constraint remains `^1.0.21`. The desktop and Docker release workflows build the pinned source with both patches and verify its gateway regression tests. Desktop builds install that package on every platform; Docker builds consume it through `docker/local-ai-gateway`. Local CLI publishing must likewise build with an explicit patched `CCR_GATEWAY_SOURCE_DIR` until the gateway changes are released in the registry dependency.
|
||||
|
||||
## Verification
|
||||
|
||||
- Gateway TypeScript check and build passed.
|
||||
- Gateway routes and tool ID suites: 158 tests passed.
|
||||
- CCR's `packages/core/test/integration/gateway/recent-issues-wire.test.mjs` exercises the built gateway via real loopback HTTP and consumes its spool with `RawTraceSynchronizer` and the real SQLite writer. It covers direct/Fusion × JSON/SSE × billing on/off, usage after `finish_reason`, provider-filtered costs, native tool replay, fallback wire destinations, the original encoded Fusion verification ID, and live OAuth rotation/probes. Set `CCR_TEST_GATEWAY_ENTRY` to the absolute built `dist/index.js` (or CCR's bundled `packages/core/dist/main/next-ai-gateway.js`) when running it.
|
||||
- No model credentials or real upstream calls were needed.
|
||||
Loading…
Add table
Add a link
Reference in a new issue