mirror of
https://github.com/openclaw/openclaw.git
synced 2026-10-03 17:53:39 +00:00
feat(models): backport dormant ordered catalog recommendations
Apply explicitly requested #158863 /09a60518fc, crediting Ayaan Zaidi (obviyus). Reuse Vincent Koc's dormant parser (#156518) and opt-in paired generator (#156530). Retain only the optional wire-schema and negative-test compatibility from #156535, not runtime/client/store activation, StandalonePricing infrastructure or billing policy. Provider recommendations preserve order and trim/validate IDs against the same provider. Invalid optional manifest lists are omitted; strict v2 rejects invalid remote lists. V1 output excludes the field and keeps its released contract. No bundled manifest or runtime/picker activates recommendations. Fix the review-exposed paired-output failure: prepare both candidate/previous sets before replacing final paths and retain cross-referenced recovery after interruption or uncertain publication. Do not automatically roll back over foreign replacements. Cleanup after success warns without claiming failure. This is recoverability, not multi-file atomicity or power-loss durability. Fixed-input actual default/priced v1 output remains byte-identical; v2 output remains identical across the settlement repair, and the parser matches the requested upstream source. 59 focused and165 caller cases pass; settlement adds real boundary/interruption proof,13 paired tests and73 publisher tests. Affected types, lint, format, MDX/docs links and frozeneb377acratchets pass. Final independent P0-P2 review is scoped-clean on exact tree2e67a787ae. No SQLite schema, package version, dependency, runtime activation or public publication change. Version remains2026.9.6 and fs-safe0.20.0. Requested npm preflight/stable FRV still require their separate source/version/admission gates; no CI run or platform clearance is claimed by this commit. ## Work sessions - Original discussion, review and integration: https://team.openclaw.ai/chat/roboclaw/dashboard/db4d78ac-b19b-4f64-adfb-0ca7973a62e4 - Dormant catalog implementation and verification: https://team.openclaw.ai/chat/roboclaw/subagent/5bc0e124-d2e6-4193-9f9b-d2767d326b30 - Paired-output correction and verification: https://team.openclaw.ai/chat/roboclaw/subagent/90593a20-97bf-4134-9221-ff30d55a618a Co-authored-by: Ayaan Zaidi <hi@obviy.us> Co-authored-by: Vincent Koc <vincentkoc@ieee.org> Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com> Co-authored-by: vyctorbrzezowski <51521767+vyctorbrzezowski@users.noreply.github.com> Co-authored-by: steipete <58493+steipete@users.noreply.github.com> Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
This commit is contained in:
roboclaw-bot
parent
f65230d92a
commit
45f97d88dd
9 changed files with 1268 additions and 51 deletions
|
|
@ -124,6 +124,29 @@ Provider fields:
|
|||
| `defaultUtilityModel` | `string` | Optional provider-recommended small model id for short internal utility tasks (titles, progress narration). Used when `agents.defaults.utilityModel` is unset and this provider serves the agent's primary model. |
|
||||
| `models` | `object[]` | Required model rows. Rows without an `id` are ignored. |
|
||||
|
||||
`recommendedModels` is an optional ordered shortlist of distinct model
|
||||
ids from this provider's `models`. Ids are trimmed and must be non-empty. The field
|
||||
is reserved for picker ordering and is not yet used. It is published only in catalog
|
||||
v2, never v1. Invalid manifest lists are omitted; invalid remote v2 lists are rejected.
|
||||
|
||||
The catalog generator opts into local paired output with `--out <v1-file> --out-v2 <v2-file>`.
|
||||
It validates both bundles and prepares candidate bytes and previous-file backups
|
||||
before replacing either output. Paired destinations must be distinct regular files
|
||||
or absent; output-directory aliases are resolved before preparation. The v1-only
|
||||
writer is unchanged.
|
||||
|
||||
Each file is replaced separately: this is **not** a multi-file atomic transaction.
|
||||
Use a single publisher and do not serve or deploy the pair until the command succeeds.
|
||||
If publication fails or the process stops between replacements, inspect the
|
||||
`.catalog-pair-*` directories beside both outputs. Each contains `next.json`,
|
||||
`previous.json` when the output existed, and `RECOVERY.txt` mapping both destinations
|
||||
and recovery directories. Stop competing writers, compare the current outputs with
|
||||
these artifacts, and explicitly restore or finish the pair before retrying. There
|
||||
is no automatic rollback or replay that could overwrite another writer's replacement.
|
||||
Identity checks detect observed changes but are not filesystem compare-and-swap;
|
||||
this protocol does not promise power-loss durability. After successful publication,
|
||||
cleanup failures warn with retained paths without reporting the pair as unpublished.
|
||||
|
||||
Model fields:
|
||||
|
||||
| Field | Type | What it means |
|
||||
|
|
|
|||
|
|
@ -4,6 +4,36 @@ import { normalizeModelCatalog, normalizeModelCatalogProviderRows } from "./inde
|
|||
import { buildModelCatalogMergeKey, buildModelCatalogRef } from "./model-catalog-refs.js";
|
||||
|
||||
describe("model catalog normalization", () => {
|
||||
it.each([
|
||||
{ input: [" model-2 ", "model-0"], expected: ["model-2", "model-0"] },
|
||||
{ input: ["missing"], expected: undefined },
|
||||
{ input: ["model-0", " model-0 "], expected: undefined },
|
||||
{ input: [" "], expected: undefined },
|
||||
{ input: [42], expected: undefined },
|
||||
{ input: [], expected: undefined },
|
||||
{ input: "model-0", expected: undefined },
|
||||
{ input: ["foreign-only"], expected: undefined },
|
||||
{
|
||||
input: Array.from({ length: 12 }, (_, index) => `model-${11 - index}`),
|
||||
expected: Array.from({ length: 12 }, (_, index) => `model-${11 - index}`),
|
||||
},
|
||||
])("normalizes a complete recommendation list or omits it: $input", ({ input, expected }) => {
|
||||
const catalog = normalizeModelCatalog(
|
||||
{
|
||||
providers: {
|
||||
openai: {
|
||||
recommendedModels: input,
|
||||
models: Array.from({ length: 12 }, (_, index) => ({ id: `model-${index}` })),
|
||||
},
|
||||
foreign: { recommendedModels: ["foreign-only"], models: [{ id: "foreign-only" }] },
|
||||
},
|
||||
},
|
||||
{ ownedProviders: new Set(["openai"]) },
|
||||
);
|
||||
expect(catalog?.providers?.openai?.recommendedModels).toEqual(expected);
|
||||
expect(catalog?.providers).not.toHaveProperty("foreign");
|
||||
});
|
||||
|
||||
it("normalizes catalog ownership, aliases, suppressions, and row fields", () => {
|
||||
const catalog = normalizeModelCatalog(
|
||||
{
|
||||
|
|
|
|||
|
|
@ -509,12 +509,22 @@ function normalizeModelCatalogProvider(value: unknown): ModelCatalogProvider | u
|
|||
const headers = normalizeStringMap(value.headers);
|
||||
const defaultModel = normalizeOptionalString(value.defaultModel) ?? "";
|
||||
const defaultUtilityModel = normalizeOptionalString(value.defaultUtilityModel) ?? "";
|
||||
const recommended = Array.isArray(value.recommendedModels)
|
||||
? value.recommendedModels.map(normalizeOptionalString)
|
||||
: [];
|
||||
const recommendedModels =
|
||||
recommended.length > 0 &&
|
||||
new Set(recommended).size === recommended.length &&
|
||||
recommended.every((id): id is string => Boolean(id) && models.some((model) => model.id === id))
|
||||
? recommended
|
||||
: [];
|
||||
return {
|
||||
...(baseUrl ? { baseUrl } : {}),
|
||||
...(api ? { api } : {}),
|
||||
...(headers ? { headers } : {}),
|
||||
...(defaultModel ? { defaultModel } : {}),
|
||||
...(defaultUtilityModel ? { defaultUtilityModel } : {}),
|
||||
...(recommendedModels.length > 0 ? { recommendedModels } : {}),
|
||||
models,
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -259,6 +259,8 @@ export type ModelCatalogProvider = {
|
|||
defaultModel?: string;
|
||||
/** Provider-recommended small model id for short internal utility tasks. */
|
||||
defaultUtilityModel?: string;
|
||||
/** Ordered provider shortlist reserved for picker ordering; not yet used. */
|
||||
recommendedModels?: string[];
|
||||
models: ModelCatalogModel[];
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
parseRemoteModelCatalogBundle,
|
||||
parseRemoteModelCatalogBundleV2,
|
||||
validateAndSanitizeRemoteModelCatalogBundle,
|
||||
validateAndSanitizeRemoteModelCatalogBundleV2,
|
||||
} from "./remote-catalog-bundle.js";
|
||||
|
||||
const validBundle = {
|
||||
|
|
@ -113,3 +115,232 @@ describe("remote model catalog bundle", () => {
|
|||
).toThrow("contextWindowDefault must reference a declared contextWindows option");
|
||||
});
|
||||
});
|
||||
|
||||
const validBundleV2 = {
|
||||
schemaVersion: 2,
|
||||
generatedAt: 1_753_500_000_000,
|
||||
sourceCommit: "abc123",
|
||||
providers: {
|
||||
first: { defaultModel: "vendor/model" },
|
||||
second: { defaultUtilityModel: "vendor/model" },
|
||||
},
|
||||
models: [
|
||||
{
|
||||
id: "vendor/model",
|
||||
provider: "first",
|
||||
input: ["text"],
|
||||
pricing: {
|
||||
status: "known",
|
||||
currency: "USD",
|
||||
unit: "million_tokens",
|
||||
source: "native-feed",
|
||||
input: 0,
|
||||
output: 0,
|
||||
},
|
||||
},
|
||||
{ id: "vendor/model", provider: "second", pricing: { status: "unknown" } },
|
||||
],
|
||||
} as const;
|
||||
|
||||
describe("remote model catalog v2", () => {
|
||||
it("preserves ordered provider recommendations only in v2", () => {
|
||||
const providers = { first: { recommendedModels: [" other ", "vendor/model"] }, second: {} };
|
||||
const bundle = parseRemoteModelCatalogBundleV2({
|
||||
...validBundleV2,
|
||||
providers,
|
||||
models: [
|
||||
...validBundleV2.models,
|
||||
{ id: "other", provider: "first", pricing: { status: "unknown" } },
|
||||
],
|
||||
});
|
||||
expect(bundle.providers.first?.recommendedModels).toEqual(["other", "vendor/model"]);
|
||||
expect(() =>
|
||||
parseRemoteModelCatalogBundle({
|
||||
...validBundle,
|
||||
providers: {
|
||||
anthropic: { ...validBundle.providers.anthropic, recommendedModels: ["claude-test"] },
|
||||
},
|
||||
}),
|
||||
).toThrow("recommendedModels");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["unknown id", ["missing"]],
|
||||
["duplicate id", ["vendor/model", " vendor/model "]],
|
||||
["empty id", [" "]],
|
||||
["non-string id", [42]],
|
||||
["non-list", "vendor/model"],
|
||||
["other provider", ["second-only"]],
|
||||
])("rejects recommended models with %s", (_name, recommendedModels) => {
|
||||
expect(() =>
|
||||
parseRemoteModelCatalogBundleV2({
|
||||
...validBundleV2,
|
||||
providers: { first: { recommendedModels }, second: {} },
|
||||
models: [
|
||||
...validBundleV2.models,
|
||||
{ id: "second-only", provider: "second", pricing: { status: "unknown" } },
|
||||
],
|
||||
}),
|
||||
).toThrow();
|
||||
});
|
||||
|
||||
it("keeps native ids distinct by provider and unknown prices distinct from free", () => {
|
||||
const bundle = parseRemoteModelCatalogBundleV2(validBundleV2);
|
||||
expect(bundle.providers).toEqual(validBundleV2.providers);
|
||||
expect(bundle.models.map(({ id, provider, pricing }) => ({ id, provider, pricing }))).toEqual([
|
||||
{
|
||||
id: "vendor/model",
|
||||
provider: "first",
|
||||
pricing: {
|
||||
status: "known",
|
||||
currency: "USD",
|
||||
unit: "million_tokens",
|
||||
source: "native-feed",
|
||||
input: 0,
|
||||
output: 0,
|
||||
},
|
||||
},
|
||||
{ id: "vendor/model", provider: "second", pricing: { status: "unknown" } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("preserves partial rates, context tiers, and authoritative unavailable prices", () => {
|
||||
const tiers = [
|
||||
{ input: 2, output: 8, cacheRead: 0.5, cacheWrite: 0, range: [0, 200_000] },
|
||||
{ input: 4, output: 12, cacheRead: 1, cacheWrite: 0, range: [200_000] },
|
||||
];
|
||||
const bundle = parseRemoteModelCatalogBundleV2({
|
||||
...validBundleV2,
|
||||
models: [
|
||||
{
|
||||
id: "partial",
|
||||
provider: "first",
|
||||
pricing: { status: "known", currency: "USD", unit: "million_tokens", input: 2 },
|
||||
},
|
||||
{
|
||||
id: "tiered",
|
||||
provider: "first",
|
||||
pricing: {
|
||||
status: "known",
|
||||
currency: "USD",
|
||||
unit: "million_tokens",
|
||||
tieredPricing: tiers,
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "unpriced",
|
||||
provider: "first",
|
||||
pricing: { status: "unavailable", source: "native-feed" },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(bundle.models[0]?.pricing).toEqual({
|
||||
status: "known",
|
||||
currency: "USD",
|
||||
unit: "million_tokens",
|
||||
input: 2,
|
||||
});
|
||||
expect(bundle.models[1]?.pricing).toMatchObject({ tieredPricing: tiers });
|
||||
expect(bundle.models[2]?.pricing).toEqual({ status: "unavailable", source: "native-feed" });
|
||||
});
|
||||
|
||||
it("retains model metadata while stripping nested transport overrides", () => {
|
||||
const bundle = validateAndSanitizeRemoteModelCatalogBundleV2({
|
||||
...validBundleV2,
|
||||
models: [
|
||||
{
|
||||
...validBundleV2.models[0],
|
||||
contextWindows: [{ id: "large", label: "Large", contextWindow: 200_000 }],
|
||||
contextWindowDefault: "large",
|
||||
compat: {
|
||||
supportsTools: true,
|
||||
nested: { baseUrl: "https://bad.test", headers: { X: "bad" } },
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(bundle.models[0]).toMatchObject({
|
||||
contextWindowDefault: "large",
|
||||
contextWindows: [{ id: "large", label: "Large", contextWindow: 200_000 }],
|
||||
compat: { supportsTools: true, nested: {} },
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves provider identities that resemble transport fields", () => {
|
||||
const bundle = validateAndSanitizeRemoteModelCatalogBundleV2({
|
||||
...validBundleV2,
|
||||
providers: { headers: {}, baseUrl: {} },
|
||||
models: [
|
||||
{ ...validBundleV2.models[0], provider: "headers" },
|
||||
{ ...validBundleV2.models[1], provider: "baseUrl" },
|
||||
],
|
||||
});
|
||||
expect(Object.keys(bundle.providers)).toEqual(["headers", "baseUrl"]);
|
||||
expect(bundle.models.map((model) => model.provider)).toEqual(["headers", "baseUrl"]);
|
||||
expect(() => parseRemoteModelCatalogBundleV2(bundle)).not.toThrow();
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: "duplicate tuple",
|
||||
models: [validBundleV2.models[0], { ...validBundleV2.models[0], id: " vendor/model " }],
|
||||
error: "duplicate provider/model",
|
||||
},
|
||||
{
|
||||
name: "undeclared provider",
|
||||
models: [{ ...validBundleV2.models[0], provider: "missing" }],
|
||||
error: "undeclared model provider",
|
||||
},
|
||||
{
|
||||
name: "transport override",
|
||||
models: [{ ...validBundleV2.models[0], baseUrl: "https://bad.test" }],
|
||||
error: "baseUrl",
|
||||
},
|
||||
{
|
||||
name: "invalid context choice",
|
||||
models: [{ ...validBundleV2.models[0], contextWindowDefault: "missing" }],
|
||||
error: "contextWindowDefault",
|
||||
},
|
||||
{
|
||||
name: "unknown price with rates",
|
||||
models: [{ ...validBundleV2.models[0], pricing: { status: "unknown", input: 0 } }],
|
||||
error: "input",
|
||||
},
|
||||
{
|
||||
name: "empty known price",
|
||||
models: [
|
||||
{
|
||||
...validBundleV2.models[0],
|
||||
pricing: { status: "known", currency: "USD", unit: "million_tokens" },
|
||||
},
|
||||
],
|
||||
error: "known pricing must contain rates",
|
||||
},
|
||||
{
|
||||
name: "negative rate",
|
||||
models: [
|
||||
{ ...validBundleV2.models[0], pricing: { ...validBundleV2.models[0].pricing, input: -1 } },
|
||||
],
|
||||
error: "input",
|
||||
},
|
||||
])("rejects $name", ({ models, error }) => {
|
||||
expect(() => parseRemoteModelCatalogBundleV2({ ...validBundleV2, models })).toThrow(error);
|
||||
});
|
||||
|
||||
it("keeps each version strict and rejects a detached pricing map in v2", () => {
|
||||
expect(() => parseRemoteModelCatalogBundle(validBundleV2)).toThrow();
|
||||
expect(() => parseRemoteModelCatalogBundleV2(validBundle)).toThrow();
|
||||
expect(() => parseRemoteModelCatalogBundleV2({ ...validBundleV2, schemaVersion: 3 })).toThrow();
|
||||
expect(() => parseRemoteModelCatalogBundleV2({ ...validBundleV2, pricing: {} })).toThrow();
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ name: "unsourced rate", upstreamPricing: { "vendor/model": { input: 1, output: 2 } } },
|
||||
{ name: "bare model key", upstreamPricing: { model: { input: 1, output: 2, source: "x" } } },
|
||||
])("rejects standalone pricing with $name", ({ upstreamPricing }) => {
|
||||
expect(() => parseRemoteModelCatalogBundleV2({ ...validBundleV2, upstreamPricing })).toThrow();
|
||||
expect(() =>
|
||||
parseRemoteModelCatalogBundleV2({ ...validBundleV2, providerPricing: upstreamPricing }),
|
||||
).toThrow();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import {
|
|||
MODEL_CATALOG_MAX_CONTEXT_WINDOWS,
|
||||
MODEL_CATALOG_THINKING_LEVELS,
|
||||
} from "./model-catalog-types.js";
|
||||
import type { ModelCatalogProvider } from "./model-catalog-types.js";
|
||||
import type { ModelCatalogModel, ModelCatalogProvider } from "./model-catalog-types.js";
|
||||
|
||||
export const REMOTE_CATALOG_MAX_FUTURE_SKEW_MS = 24 * 60 * 60_000;
|
||||
|
||||
|
|
@ -52,47 +52,52 @@ const contextWindowOptionSchema = z
|
|||
})
|
||||
.strict();
|
||||
|
||||
const modelSchema = z
|
||||
.object({
|
||||
id: z.string().trim().min(1),
|
||||
name: z.string().optional(),
|
||||
api: z.enum(MODEL_CATALOG_APIS).optional(),
|
||||
baseUrl: z.string().optional(),
|
||||
headers: stringMapSchema.optional(),
|
||||
input: z.array(z.enum(["text", "image", "document"])).optional(),
|
||||
reasoning: z.boolean().optional(),
|
||||
contextWindow: z.number().finite().positive().optional(),
|
||||
contextWindows: z
|
||||
.array(contextWindowOptionSchema)
|
||||
.max(MODEL_CATALOG_MAX_CONTEXT_WINDOWS)
|
||||
.optional(),
|
||||
contextWindowDefault: z.string().trim().min(1).optional(),
|
||||
contextTokens: z.number().int().positive().optional(),
|
||||
maxTokens: z.number().finite().positive().optional(),
|
||||
thinkingLevelMap: z
|
||||
.partialRecord(z.enum(MODEL_CATALOG_THINKING_LEVELS), z.string().nullable())
|
||||
.optional(),
|
||||
cost: costSchema.optional(),
|
||||
compat: z.record(z.string(), z.unknown()).optional(),
|
||||
mediaInput: z.record(z.string(), z.unknown()).optional(),
|
||||
status: z.enum(["available", "preview", "deprecated", "disabled"]).optional(),
|
||||
statusReason: z.string().optional(),
|
||||
replaces: z.array(z.string()).optional(),
|
||||
replacedBy: z.string().optional(),
|
||||
tags: z.array(z.string()).optional(),
|
||||
})
|
||||
.superRefine((model, context) => {
|
||||
if (
|
||||
model.contextWindowDefault &&
|
||||
!model.contextWindows?.some((option) => option.id === model.contextWindowDefault)
|
||||
) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
message: "contextWindowDefault must reference a declared contextWindows option",
|
||||
path: ["contextWindowDefault"],
|
||||
});
|
||||
}
|
||||
});
|
||||
const modelFieldsSchema = z.object({
|
||||
id: z.string().trim().min(1),
|
||||
name: z.string().optional(),
|
||||
api: z.enum(MODEL_CATALOG_APIS).optional(),
|
||||
baseUrl: z.string().optional(),
|
||||
headers: stringMapSchema.optional(),
|
||||
input: z.array(z.enum(["text", "image", "document"])).optional(),
|
||||
reasoning: z.boolean().optional(),
|
||||
contextWindow: z.number().finite().positive().optional(),
|
||||
contextWindows: z
|
||||
.array(contextWindowOptionSchema)
|
||||
.max(MODEL_CATALOG_MAX_CONTEXT_WINDOWS)
|
||||
.optional(),
|
||||
contextWindowDefault: z.string().trim().min(1).optional(),
|
||||
contextTokens: z.number().int().positive().optional(),
|
||||
maxTokens: z.number().finite().positive().optional(),
|
||||
thinkingLevelMap: z
|
||||
.partialRecord(z.enum(MODEL_CATALOG_THINKING_LEVELS), z.string().nullable())
|
||||
.optional(),
|
||||
cost: costSchema.optional(),
|
||||
compat: z.record(z.string(), z.unknown()).optional(),
|
||||
mediaInput: z.record(z.string(), z.unknown()).optional(),
|
||||
status: z.enum(["available", "preview", "deprecated", "disabled"]).optional(),
|
||||
statusReason: z.string().optional(),
|
||||
replaces: z.array(z.string()).optional(),
|
||||
replacedBy: z.string().optional(),
|
||||
tags: z.array(z.string()).optional(),
|
||||
});
|
||||
|
||||
function validateContextWindowDefault(
|
||||
model: { contextWindowDefault?: string; contextWindows?: Array<{ id: string }> },
|
||||
context: z.RefinementCtx,
|
||||
) {
|
||||
if (
|
||||
model.contextWindowDefault &&
|
||||
!model.contextWindows?.some((option) => option.id === model.contextWindowDefault)
|
||||
) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
message: "contextWindowDefault must reference a declared contextWindows option",
|
||||
path: ["contextWindowDefault"],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const modelSchema = modelFieldsSchema.superRefine(validateContextWindowDefault);
|
||||
|
||||
export const remoteModelCatalogProviderSchema = z
|
||||
.object({
|
||||
|
|
@ -142,6 +147,136 @@ export type RemoteModelCatalogBundle = Omit<
|
|||
providers: Record<string, ModelCatalogProvider>;
|
||||
};
|
||||
|
||||
// "Known" describes the supplied rates, not a complete tariff. Preserve omitted
|
||||
// components so ingestion retains the existing partial-cost contract.
|
||||
const knownPricingV2Schema = costSchema
|
||||
.extend({
|
||||
status: z.literal("known"),
|
||||
currency: z.literal("USD"),
|
||||
unit: z.literal("million_tokens"),
|
||||
source: z.string().trim().min(1).optional(),
|
||||
})
|
||||
.refine(
|
||||
(pricing) =>
|
||||
pricing.input !== undefined ||
|
||||
pricing.output !== undefined ||
|
||||
pricing.cacheRead !== undefined ||
|
||||
pricing.cacheWrite !== undefined ||
|
||||
Boolean(pricing.tieredPricing?.length),
|
||||
{ message: "known pricing must contain rates" },
|
||||
);
|
||||
|
||||
const pricingV2Schema = z.discriminatedUnion("status", [
|
||||
knownPricingV2Schema,
|
||||
z
|
||||
.object({
|
||||
status: z.enum(["unknown", "unavailable"]),
|
||||
source: z.string().trim().min(1).optional(),
|
||||
})
|
||||
.strict(),
|
||||
]);
|
||||
|
||||
export type RemoteModelCatalogPricingV2 = z.infer<typeof pricingV2Schema>;
|
||||
|
||||
// Standalone rates cover models outside `models`, in USD per million tokens.
|
||||
// `upstreamPricing` is keyed by the source's vendor/model ID and serves passthrough
|
||||
// gateways plus same-vendor lookups. The entry's own rate wins; `alternatives` keep
|
||||
// later sources' rates for providers that allow only those sources. `passthroughOnly`
|
||||
// entries are owned by catalog rows and serve gateways alone. `providerPricing` holds
|
||||
// provider-owned rates for models without a catalog row.
|
||||
const sourcedPricingV2Schema = hostedPricingSchema
|
||||
.extend({ source: z.string().trim().min(1) })
|
||||
.strict();
|
||||
const upstreamPricingV2Schema = sourcedPricingV2Schema
|
||||
.extend({
|
||||
passthroughOnly: z.literal(true).optional(),
|
||||
alternatives: z.array(sourcedPricingV2Schema).min(1).optional(),
|
||||
})
|
||||
.strict();
|
||||
const standalonePricingKeySchema = z
|
||||
.string()
|
||||
.regex(/^[^/\s]+\/\S+$/u, "standalone pricing keys must be provider/model");
|
||||
|
||||
const modelV2Schema = modelFieldsSchema
|
||||
.omit({ cost: true, baseUrl: true, headers: true })
|
||||
.extend({ provider: z.string().trim().min(1), pricing: pricingV2Schema })
|
||||
.strict()
|
||||
.superRefine(validateContextWindowDefault);
|
||||
|
||||
export const remoteModelCatalogBundleV2Schema = remoteModelCatalogBundleSchema
|
||||
.omit({ providers: true, pricing: true })
|
||||
.extend({
|
||||
schemaVersion: z.literal(2),
|
||||
providers: z.record(
|
||||
z.string().trim().min(1),
|
||||
z
|
||||
.object({
|
||||
api: z.enum(MODEL_CATALOG_APIS).optional(),
|
||||
defaultModel: z.string().optional(),
|
||||
defaultUtilityModel: z.string().optional(),
|
||||
recommendedModels: z
|
||||
.array(z.string().trim().min(1))
|
||||
.refine((ids) => new Set(ids).size === ids.length, "duplicate recommended model id")
|
||||
.optional(),
|
||||
})
|
||||
.strict(),
|
||||
),
|
||||
models: z.array(modelV2Schema).min(1),
|
||||
upstreamPricing: z.record(standalonePricingKeySchema, upstreamPricingV2Schema).optional(),
|
||||
providerPricing: z.record(standalonePricingKeySchema, sourcedPricingV2Schema).optional(),
|
||||
})
|
||||
.superRefine((bundle, context) => {
|
||||
const providers = new Map<string, Set<string>>();
|
||||
for (const [index, model] of bundle.models.entries()) {
|
||||
if (!Object.hasOwn(bundle.providers, model.provider)) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
message: `undeclared model provider: ${model.provider}`,
|
||||
path: ["models", index, "provider"],
|
||||
});
|
||||
}
|
||||
const ids = providers.get(model.provider) ?? new Set<string>();
|
||||
if (ids.has(model.id)) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
message: `duplicate provider/model: ${model.provider}/${model.id}`,
|
||||
path: ["models", index, "id"],
|
||||
});
|
||||
}
|
||||
ids.add(model.id);
|
||||
providers.set(model.provider, ids);
|
||||
}
|
||||
for (const [provider, entry] of Object.entries(bundle.providers)) {
|
||||
for (const [index, id] of (entry.recommendedModels ?? []).entries()) {
|
||||
if (!providers.get(provider)?.has(id)) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
message: `recommended model must reference a model of provider ${provider}: ${id}`,
|
||||
path: ["providers", provider, "recommendedModels", index],
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
export type RemoteModelCatalogModelV2 = Omit<
|
||||
ModelCatalogModel,
|
||||
"cost" | "baseUrl" | "headers" | "upstreamModel"
|
||||
> & {
|
||||
provider: string;
|
||||
pricing: RemoteModelCatalogPricingV2;
|
||||
};
|
||||
|
||||
export type RemoteModelCatalogBundleV2 = Omit<
|
||||
z.infer<typeof remoteModelCatalogBundleV2Schema>,
|
||||
"models"
|
||||
> & { models: RemoteModelCatalogModelV2[] };
|
||||
|
||||
export function parseRemoteModelCatalogBundleV2(value: unknown): RemoteModelCatalogBundleV2 {
|
||||
// SAFETY: the schema validates every model field; shared catalog types narrow compat metadata.
|
||||
return remoteModelCatalogBundleV2Schema.parse(value) as RemoteModelCatalogBundleV2;
|
||||
}
|
||||
|
||||
export function parseRemoteModelCatalogBundle(value: unknown): RemoteModelCatalogBundle {
|
||||
return remoteModelCatalogBundleSchema.parse(value) as RemoteModelCatalogBundle;
|
||||
}
|
||||
|
|
@ -177,3 +312,18 @@ export function validateAndSanitizeRemoteModelCatalogBundle(
|
|||
): RemoteModelCatalogBundle {
|
||||
return sanitizeRemoteModelCatalogBundle(parseRemoteModelCatalogBundle(value));
|
||||
}
|
||||
|
||||
export function validateAndSanitizeRemoteModelCatalogBundleV2(
|
||||
value: unknown,
|
||||
): RemoteModelCatalogBundleV2 {
|
||||
const bundle = parseRemoteModelCatalogBundleV2(value);
|
||||
// Provider dictionary keys are identities, even when named "headers" or "baseUrl".
|
||||
// Their strict defaults contain no transport overrides; only model metadata needs stripping.
|
||||
return {
|
||||
...bundle,
|
||||
models: bundle.models.map((model) => {
|
||||
// SAFETY: strict v2 fields exclude transport keys; only freeform metadata can contain them.
|
||||
return stripRemoteTransportOverrides(model) as RemoteModelCatalogModelV2;
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
|
|
|||
175
scripts/lib/publish-model-catalog-files.mts
Normal file
175
scripts/lib/publish-model-catalog-files.mts
Normal file
|
|
@ -0,0 +1,175 @@
|
|||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import {
|
||||
readRegularFileSync,
|
||||
sameFileIdentity,
|
||||
writeSiblingTempFile,
|
||||
} from "@openclaw/fs-safe/advanced";
|
||||
|
||||
type Output = { file: string; content: string };
|
||||
type Snapshot = ReturnType<typeof readRegularFileSync> | undefined;
|
||||
type Recovery = {
|
||||
dir: string;
|
||||
identity: fs.Stats;
|
||||
files: Map<string, fs.Stats>;
|
||||
};
|
||||
|
||||
function snapshot(file: string): Snapshot {
|
||||
return fs.lstatSync(file, { throwIfNoEntry: false })
|
||||
? readRegularFileSync({ filePath: file })
|
||||
: undefined;
|
||||
}
|
||||
|
||||
function assertUnchanged(file: string, previous: Snapshot): void {
|
||||
const current = snapshot(file);
|
||||
if (
|
||||
previous
|
||||
? !current ||
|
||||
!sameFileIdentity(previous.stat, current.stat) ||
|
||||
!previous.buffer.equals(current.buffer)
|
||||
: current
|
||||
) {
|
||||
throw new Error(`Catalog output changed during preparation: ${file}`);
|
||||
}
|
||||
}
|
||||
|
||||
function saveRecoveryFile(recovery: Recovery, name: string, content: string | Buffer): void {
|
||||
const file = path.join(recovery.dir, name);
|
||||
// Capture ownership before writing: even a partial write remains cleanable.
|
||||
const fd = fs.openSync(file, "wx", 0o600);
|
||||
try {
|
||||
recovery.files.set(file, fs.fstatSync(fd));
|
||||
fs.writeFileSync(fd, content);
|
||||
fs.fsyncSync(fd);
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
}
|
||||
|
||||
function cleanupRecovery(recovery: Recovery): void {
|
||||
if (!sameFileIdentity(recovery.identity, fs.lstatSync(recovery.dir))) {
|
||||
throw new Error("recovery directory was replaced");
|
||||
}
|
||||
// No recursive removal and no exit hook: interrupted publication must retain
|
||||
// its backups. Preserve observed substitutes and unknown children.
|
||||
for (const [file, identity] of recovery.files) {
|
||||
const current = fs.lstatSync(file, { throwIfNoEntry: false });
|
||||
if (current) {
|
||||
if (!sameFileIdentity(identity, current)) {
|
||||
throw new Error("recovery file was replaced");
|
||||
}
|
||||
fs.unlinkSync(file);
|
||||
}
|
||||
}
|
||||
fs.rmdirSync(recovery.dir);
|
||||
}
|
||||
|
||||
/** Local artifact publication, not a transaction across two filesystem names. */
|
||||
export async function publishModelCatalogPair(
|
||||
outputs: [Output, Output],
|
||||
warn: (message: string) => void,
|
||||
): Promise<void> {
|
||||
// Canonical parents catch aliases without changing the v1-only writer contract.
|
||||
const prepared = outputs.map((output) => {
|
||||
fs.mkdirSync(path.dirname(output.file), { recursive: true });
|
||||
const parent = fs.realpathSync(path.dirname(output.file));
|
||||
return { ...output, file: path.join(parent, path.basename(output.file)), parent };
|
||||
});
|
||||
if (new Set(prepared.map((output) => output.file)).size !== outputs.length) {
|
||||
throw new Error("--out and --out-v2 must name different files");
|
||||
}
|
||||
const plans = prepared.map((output) =>
|
||||
Object.assign(output, { previous: snapshot(output.file) }),
|
||||
);
|
||||
const recoveries: Array<Recovery & (typeof plans)[number]> = [];
|
||||
let publicationStarted = false;
|
||||
try {
|
||||
for (const output of plans) {
|
||||
const dir = fs.mkdtempSync(path.join(output.parent, ".catalog-pair-"));
|
||||
recoveries.push({ ...output, dir, identity: fs.lstatSync(dir), files: new Map() });
|
||||
}
|
||||
for (const [index, recovery] of recoveries.entries()) {
|
||||
saveRecoveryFile(recovery, "next.json", recovery.content);
|
||||
const previous = recovery.previous;
|
||||
if (previous) {
|
||||
saveRecoveryFile(recovery, "previous.json", previous.buffer);
|
||||
}
|
||||
saveRecoveryFile(
|
||||
recovery,
|
||||
"RECOVERY.txt",
|
||||
[
|
||||
"Catalog pair recovery artifacts; not proof of publication.",
|
||||
...recoveries.map((entry, i) => `output ${i + 1}: ${entry.file}; recovery: ${entry.dir}`),
|
||||
`This directory belongs to output ${index + 1}.`,
|
||||
previous
|
||||
? `previous.json holds original bytes; mode ${(previous.stat.mode & 0o777).toString(8)}.`
|
||||
: "The output was absent before this attempt.",
|
||||
"next.json holds the validated candidate bytes.",
|
||||
"Stop writers and inspect BOTH outputs before restoring or completing the pair.",
|
||||
"Do not blindly overwrite a replacement or rerun publication to recover.",
|
||||
"Retained artifacts are never automatically replayed or removed by a later run.",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
if (
|
||||
!fs.readFileSync(path.join(recovery.dir, "next.json")).equals(Buffer.from(recovery.content))
|
||||
) {
|
||||
throw new Error("prepared catalog bytes changed");
|
||||
}
|
||||
}
|
||||
// Finish both preparations before any final-path mutation. Recheck each
|
||||
// destination again at publication; this is cooperative, not rename CAS.
|
||||
recoveries.forEach((output) => assertUnchanged(output.file, output.previous));
|
||||
publicationStarted = true;
|
||||
for (const output of recoveries) {
|
||||
await writeSiblingTempFile({
|
||||
dir: output.parent,
|
||||
chmodDir: false,
|
||||
producerIsolation: "private-directory",
|
||||
mode: output.previous ? output.previous.stat.mode & 0o777 : undefined,
|
||||
syncTempFile: true,
|
||||
writeTemp: async (tempPath) => {
|
||||
fs.writeFileSync(tempPath, output.content, {
|
||||
flag: "wx",
|
||||
mode: output.previous ? output.previous.stat.mode & 0o777 : 0o666,
|
||||
});
|
||||
},
|
||||
resolveFinalPath: () => {
|
||||
assertUnchanged(output.file, output.previous);
|
||||
return output.file;
|
||||
},
|
||||
});
|
||||
}
|
||||
} catch (cause) {
|
||||
if (publicationStarted) {
|
||||
// A failed rename/verification can already have published. Never roll back
|
||||
// over a foreign replacement; retain both old/new sets for reconciliation.
|
||||
throw new Error(
|
||||
`Catalog pair publication incomplete; inspect both outputs. Recovery retained: ${recoveries.map((entry) => entry.dir).join(", ")}. Cause: ${String(cause)}`,
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
throw cause;
|
||||
} finally {
|
||||
if (!publicationStarted) {
|
||||
for (const recovery of recoveries) {
|
||||
try {
|
||||
cleanupRecovery(recovery);
|
||||
} catch (error) {
|
||||
warn(`Catalog recovery cleanup failed; retained ${recovery.dir}: ${String(error)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Publication succeeded. Cleanup failure is a warning, not a false failed
|
||||
// publication or an excuse to roll back an already visible pair.
|
||||
for (const recovery of recoveries) {
|
||||
try {
|
||||
cleanupRecovery(recovery);
|
||||
} catch (error) {
|
||||
warn(
|
||||
`Catalog pair published; recovery cleanup failed; retained ${recovery.dir}: ${String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -18,9 +18,12 @@ import { isRecord } from "@openclaw/normalization-core/record-coerce";
|
|||
import type { ModelCatalogModel } from "../packages/model-catalog-core/src/model-catalog-types.js";
|
||||
import type {
|
||||
RemoteModelCatalogBundle,
|
||||
RemoteModelCatalogBundleV2,
|
||||
RemoteModelCatalogPricing,
|
||||
RemoteModelCatalogPricingV2,
|
||||
} from "../packages/model-catalog-core/src/remote-catalog-bundle.js";
|
||||
import { importToolingTypeScript } from "./lib/import-tooling-typescript.mts";
|
||||
import { publishModelCatalogPair } from "./lib/publish-model-catalog-files.mts";
|
||||
import { resolveRepoRoot } from "./lib/repo-root.mjs";
|
||||
|
||||
type ModelCatalogManifestInput = {
|
||||
|
|
@ -55,6 +58,7 @@ type ModelsDevModel = Record<string, unknown> & {
|
|||
type ModelCatalogHydrationCounts = { added: number; filled: number; skipped: number };
|
||||
type ModelCatalogHydrationResult = Record<string, ModelCatalogHydrationCounts>;
|
||||
type ModelCatalogSourceLoader = (url: string, label: string) => Promise<unknown>;
|
||||
type PricingSelection = Pick<RemoteModelCatalogPricingV2, "status" | "source">;
|
||||
const MODEL_CATALOG_MIN_VERSION = "2026.7.0";
|
||||
export const MODEL_CATALOG_MIN_MODELS = 200;
|
||||
|
||||
|
|
@ -87,6 +91,7 @@ export function parsePublishModelCatalogArgs(args: string[]) {
|
|||
let dryRun = false;
|
||||
let pricing = false;
|
||||
let out: string | undefined;
|
||||
let outV2: string | undefined;
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const arg = args[index];
|
||||
if (arg === "--dry-run") {
|
||||
|
|
@ -102,12 +107,17 @@ export function parsePublishModelCatalogArgs(args: string[]) {
|
|||
index += 1;
|
||||
continue;
|
||||
}
|
||||
if (arg === "--out-v2") {
|
||||
outV2 = requireOptionValue(args, index, arg);
|
||||
index += 1;
|
||||
continue;
|
||||
}
|
||||
throw new Error(`unknown argument: ${arg}`);
|
||||
}
|
||||
if (!dryRun && !out) {
|
||||
throw new Error("provide --out <file> or --dry-run");
|
||||
}
|
||||
return { dryRun, pricing, ...(out ? { out } : {}) };
|
||||
return { dryRun, pricing, ...(out ? { out } : {}), ...(outV2 ? { outV2 } : {}) };
|
||||
}
|
||||
|
||||
export function readModelCatalogManifests(
|
||||
|
|
@ -131,16 +141,20 @@ export function readModelCatalogManifests(
|
|||
.toSorted((left, right) => left.pluginId.localeCompare(right.pluginId));
|
||||
}
|
||||
|
||||
async function loadClientBundleValidator() {
|
||||
async function loadClientBundleValidator(version: 1 | 2 = 1) {
|
||||
const modulePath = path.join(
|
||||
defaultRootDir,
|
||||
"packages/model-catalog-core/src/remote-catalog-bundle.ts",
|
||||
);
|
||||
const module = await importToolingTypeScript(pathToFileURL(modulePath).href, import.meta.url);
|
||||
if (typeof module.validateAndSanitizeRemoteModelCatalogBundle !== "function") {
|
||||
const name =
|
||||
version === 1
|
||||
? "validateAndSanitizeRemoteModelCatalogBundle"
|
||||
: "validateAndSanitizeRemoteModelCatalogBundleV2";
|
||||
if (typeof module[name] !== "function") {
|
||||
throw new Error("remote catalog bundle validator export is unavailable");
|
||||
}
|
||||
return module.validateAndSanitizeRemoteModelCatalogBundle;
|
||||
return module[name];
|
||||
}
|
||||
|
||||
export async function assembleModelCatalogBundle(options: {
|
||||
|
|
@ -160,7 +174,12 @@ export async function assembleModelCatalogBundle(options: {
|
|||
if (Object.hasOwn(providers, providerId)) {
|
||||
throw new Error(`provider ${providerId} is declared by more than one plugin manifest`);
|
||||
}
|
||||
providers[providerId] = provider;
|
||||
if (isRecord(provider)) {
|
||||
const { recommendedModels: _recommendedModels, ...v1Provider } = provider;
|
||||
providers[providerId] = v1Provider;
|
||||
} else {
|
||||
providers[providerId] = provider;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -727,6 +746,7 @@ export async function enrichModelCatalogPricing(options: {
|
|||
manifests: ModelCatalogManifestInput[];
|
||||
fetchImpl?: typeof fetch;
|
||||
loadSource?: ModelCatalogSourceLoader;
|
||||
pricingSelections?: WeakMap<ModelCatalogModel, PricingSelection>;
|
||||
}): Promise<{ modelsEnriched: number; pricingEntries: number }> {
|
||||
const policies = readPricingPolicies(options.manifests);
|
||||
const sources = await fetchPricingSources(
|
||||
|
|
@ -758,10 +778,12 @@ export async function enrichModelCatalogPricing(options: {
|
|||
);
|
||||
if (chosen?.pricing) {
|
||||
model.cost = chosen.pricing;
|
||||
options.pricingSelections?.set(model, { status: "known", source: chosen.source.id });
|
||||
enriched += 1;
|
||||
} else if (chosen) {
|
||||
// Keep the metadata row: removing it would revive the bundled seed's stale price.
|
||||
delete model.cost;
|
||||
options.pricingSelections?.set(model, { status: "unavailable", source: chosen.source.id });
|
||||
process.stderr.write(
|
||||
`[${SCRIPT_LABEL}] warning: ${chosen.source.label} pricing unavailable for ${providerId}/${model.id}; preserving metadata without cost\n`,
|
||||
);
|
||||
|
|
@ -840,6 +862,93 @@ export function serializeModelCatalogBundle(bundle: PublishedModelCatalogBundle)
|
|||
return `${JSON.stringify(sortCatalogValue({ ...bundle, providers }), null, 2)}\n`;
|
||||
}
|
||||
|
||||
export async function assembleModelCatalogBundleV2(
|
||||
bundle: PublishedModelCatalogBundle,
|
||||
pricingSelections: WeakMap<ModelCatalogModel, PricingSelection>,
|
||||
manifests: ModelCatalogManifestInput[] = [],
|
||||
): Promise<RemoteModelCatalogBundleV2> {
|
||||
const recommendations = new Map<string, string[]>();
|
||||
for (const entry of manifests) {
|
||||
const catalog = normalizeModelCatalog(entry.manifest.modelCatalog, {
|
||||
ownedProviders: new Set(entry.manifest.providers ?? []),
|
||||
});
|
||||
for (const [id, provider] of Object.entries(catalog?.providers ?? {})) {
|
||||
if (provider.recommendedModels?.length) {
|
||||
recommendations.set(id, provider.recommendedModels);
|
||||
}
|
||||
}
|
||||
}
|
||||
const providers: RemoteModelCatalogBundleV2["providers"] = {};
|
||||
const models: RemoteModelCatalogBundleV2["models"] = [];
|
||||
for (const [providerId, provider] of Object.entries(bundle.providers)) {
|
||||
const recommendedModels = recommendations.get(providerId);
|
||||
providers[providerId] = {
|
||||
api: provider.api,
|
||||
defaultModel: provider.defaultModel,
|
||||
defaultUtilityModel: provider.defaultUtilityModel,
|
||||
...(recommendedModels?.length ? { recommendedModels } : {}),
|
||||
};
|
||||
for (const model of provider.models) {
|
||||
const { cost, ...metadata } = model;
|
||||
delete metadata.baseUrl;
|
||||
delete metadata.headers;
|
||||
delete metadata.upstreamModel;
|
||||
const selection = pricingSelections.get(model);
|
||||
const pricing: RemoteModelCatalogPricingV2 =
|
||||
cost && (selection?.status === "known" || hasKnownPricing(cost))
|
||||
? {
|
||||
status: "known",
|
||||
currency: "USD",
|
||||
unit: "million_tokens",
|
||||
...cost,
|
||||
...(selection?.source ? { source: selection.source } : {}),
|
||||
}
|
||||
: {
|
||||
status: selection?.status === "unavailable" ? "unavailable" : "unknown",
|
||||
...(selection?.source ? { source: selection.source } : {}),
|
||||
};
|
||||
models.push({ ...metadata, provider: providerId, pricing });
|
||||
}
|
||||
}
|
||||
const validateBundle = await loadClientBundleValidator(2);
|
||||
// The first supporting release is not assigned yet. schemaVersion gates v2;
|
||||
// never copy v1's older client floor onto a new wire contract.
|
||||
return validateBundle({
|
||||
schemaVersion: 2,
|
||||
generatedAt: bundle.generatedAt,
|
||||
sourceCommit: bundle.sourceCommit,
|
||||
providers,
|
||||
models,
|
||||
});
|
||||
}
|
||||
|
||||
export function serializeModelCatalogBundleV2(bundle: RemoteModelCatalogBundleV2): string {
|
||||
const models = bundle.models
|
||||
.toSorted(
|
||||
(left, right) =>
|
||||
left.provider.localeCompare(right.provider) || left.id.localeCompare(right.id),
|
||||
)
|
||||
.map(({ id, provider, ...metadata }) =>
|
||||
Object.assign(
|
||||
{ id, provider },
|
||||
Object.fromEntries(
|
||||
Object.entries(metadata)
|
||||
.toSorted(([left], [right]) => left.localeCompare(right))
|
||||
.map(([key, value]) => [key, sortCatalogValue(value)]),
|
||||
),
|
||||
),
|
||||
);
|
||||
return `${JSON.stringify(
|
||||
Object.fromEntries(
|
||||
Object.entries(bundle)
|
||||
.toSorted(([left], [right]) => left.localeCompare(right))
|
||||
.map(([key, value]) => [key, key === "models" ? models : sortCatalogValue(value)]),
|
||||
),
|
||||
null,
|
||||
2,
|
||||
)}\n`;
|
||||
}
|
||||
|
||||
function resolveSourceCommit(rootDir: string): string {
|
||||
return execFileSync("git", ["rev-parse", "HEAD"], {
|
||||
cwd: rootDir,
|
||||
|
|
@ -848,7 +957,7 @@ function resolveSourceCommit(rootDir: string): string {
|
|||
}).trim();
|
||||
}
|
||||
|
||||
async function runPublishModelCatalog(
|
||||
export async function runPublishModelCatalog(
|
||||
options: {
|
||||
args?: string[];
|
||||
fetchImpl?: typeof fetch;
|
||||
|
|
@ -859,22 +968,44 @@ async function runPublishModelCatalog(
|
|||
) {
|
||||
const rootDir = options.rootDir ?? defaultRootDir;
|
||||
const args = parsePublishModelCatalogArgs(options.args ?? process.argv.slice(2));
|
||||
if (
|
||||
args.out &&
|
||||
args.outV2 &&
|
||||
path.resolve(rootDir, args.out) === path.resolve(rootDir, args.outV2)
|
||||
) {
|
||||
throw new Error("--out and --out-v2 must name different files");
|
||||
}
|
||||
const generatedAt = (options.now ?? Date.now)();
|
||||
const sourceCommit = options.sourceCommit ?? resolveSourceCommit(rootDir);
|
||||
const manifests = readModelCatalogManifests({ rootDir });
|
||||
let bundle = await assembleModelCatalogBundle({ manifests, generatedAt, sourceCommit });
|
||||
const pricingSelections = new WeakMap<ModelCatalogModel, PricingSelection>();
|
||||
// Capture seed ownership before hydration/enrichment can replace its cost.
|
||||
for (const provider of Object.values(bundle.providers)) {
|
||||
for (const model of provider.models) {
|
||||
if (model.cost && hasKnownPricing(model.cost)) {
|
||||
pricingSelections.set(model, { status: "known", source: "manifest" });
|
||||
}
|
||||
}
|
||||
}
|
||||
const loadSource = createModelCatalogSourceLoader(options.fetchImpl);
|
||||
const hydrationResult = await hydrateModelCatalogFromModelsDev({ bundle, manifests, loadSource });
|
||||
const pricingResult = args.pricing
|
||||
? await enrichModelCatalogPricing({ bundle, manifests, loadSource })
|
||||
? await enrichModelCatalogPricing({ bundle, manifests, loadSource, pricingSelections })
|
||||
: { modelsEnriched: 0, pricingEntries: 0 };
|
||||
// Validate after all enrichment so metadata-only and dry-run output obey the
|
||||
// same client contract as priced catalogs.
|
||||
const validateBundle = await loadClientBundleValidator();
|
||||
// Project while selection facts still refer to the assembled model objects.
|
||||
const bundleV2 = args.outV2
|
||||
? await assembleModelCatalogBundleV2(bundle, pricingSelections, manifests)
|
||||
: undefined;
|
||||
bundle = validateBundle(bundle);
|
||||
const summary = summarizeModelCatalogBundle(bundle);
|
||||
const serialized = serializeModelCatalogBundle(bundle);
|
||||
const bundleBytes = Buffer.byteLength(serialized);
|
||||
const serializedV2 = bundleV2 ? serializeModelCatalogBundleV2(bundleV2) : undefined;
|
||||
const bundleV2Bytes = serializedV2 ? Buffer.byteLength(serializedV2) : 0;
|
||||
if (bundleBytes > BUNDLE_SIZE_WARNING_BYTES) {
|
||||
process.stderr.write(
|
||||
`[${SCRIPT_LABEL}] warning: bundle size ${bundleBytes} bytes exceeds ${BUNDLE_SIZE_WARNING_BYTES} bytes\n`,
|
||||
|
|
@ -885,6 +1016,11 @@ async function runPublishModelCatalog(
|
|||
`catalog bundle ${bundleBytes} bytes exceeds client limit ${CLIENT_BUNDLE_LIMIT_BYTES} bytes`,
|
||||
);
|
||||
}
|
||||
if (bundleV2Bytes > CLIENT_BUNDLE_LIMIT_BYTES) {
|
||||
throw new Error(
|
||||
`catalog v2 bundle ${bundleV2Bytes} bytes exceeds client limit ${CLIENT_BUNDLE_LIMIT_BYTES} bytes`,
|
||||
);
|
||||
}
|
||||
const hydrationSummary = Object.entries(hydrationResult)
|
||||
.toSorted(([left], [right]) => left.localeCompare(right))
|
||||
.map(
|
||||
|
|
@ -901,8 +1037,21 @@ async function runPublishModelCatalog(
|
|||
throw new Error("output path is required outside dry-run mode");
|
||||
}
|
||||
const outputFile = path.resolve(rootDir, args.out);
|
||||
fs.mkdirSync(path.dirname(outputFile), { recursive: true });
|
||||
fs.writeFileSync(outputFile, serialized);
|
||||
if (args.outV2 && serializedV2) {
|
||||
await publishModelCatalogPair(
|
||||
[
|
||||
{ file: outputFile, content: serialized },
|
||||
{ file: path.resolve(rootDir, args.outV2), content: serializedV2 },
|
||||
],
|
||||
(message) => process.stderr.write(`[${SCRIPT_LABEL}] warning: ${message}\n`),
|
||||
);
|
||||
process.stdout.write(
|
||||
`[${SCRIPT_LABEL}] published schemaVersion=2 models=${summary.models} bundleBytes=${bundleV2Bytes} out=${args.outV2}\n`,
|
||||
);
|
||||
} else {
|
||||
fs.mkdirSync(path.dirname(outputFile), { recursive: true });
|
||||
fs.writeFileSync(outputFile, serialized);
|
||||
}
|
||||
process.stdout.write(`[${SCRIPT_LABEL}] published ${stats} out=${args.out}\n${hydrationSummary}`);
|
||||
return { bundle, summary, pricingEnriched: pricingResult.modelsEnriched, wrote: true };
|
||||
}
|
||||
|
|
|
|||
447
test/scripts/publish-model-catalog-v2.test.ts
Normal file
447
test/scripts/publish-model-catalog-v2.test.ts
Normal file
|
|
@ -0,0 +1,447 @@
|
|||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
parseRemoteModelCatalogBundle,
|
||||
parseRemoteModelCatalogBundleV2,
|
||||
type RemoteModelCatalogBundle,
|
||||
} from "../../packages/model-catalog-core/src/remote-catalog-bundle.js";
|
||||
import {
|
||||
assembleModelCatalogBundleV2,
|
||||
parsePublishModelCatalogArgs,
|
||||
runPublishModelCatalog,
|
||||
serializeModelCatalogBundle,
|
||||
serializeModelCatalogBundleV2,
|
||||
} from "../../scripts/publish-model-catalog.mts";
|
||||
import { useAutoCleanupTempDirTracker } from "../helpers/temp-dir.js";
|
||||
|
||||
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
function fixtureRoot() {
|
||||
const root = tempDirs.make("openclaw-catalog-v2-");
|
||||
const dir = path.join(root, "extensions", "fixture");
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
const seeds = Array.from({ length: 100 }, (_, index) => ({ id: `seed-${index}` }));
|
||||
fs.writeFileSync(
|
||||
path.join(dir, "openclaw.plugin.json"),
|
||||
JSON.stringify({
|
||||
providers: ["anthropic", "openai", "fixture-native"],
|
||||
modelCatalog: {
|
||||
modelsDev: { "fixture-native": "upstream" },
|
||||
providers: {
|
||||
anthropic: {
|
||||
recommendedModels: ["missing"],
|
||||
models: seeds.map((model, index) =>
|
||||
index === 0 ? { ...model, cost: { input: 0.5 } } : model,
|
||||
),
|
||||
},
|
||||
openai: {
|
||||
defaultModel: "seed-0",
|
||||
recommendedModels: [" seed-2 ", "seed-0"],
|
||||
models: seeds,
|
||||
},
|
||||
"fixture-native": {
|
||||
models: [
|
||||
{ id: "free", cost: { input: 9, output: 9 } },
|
||||
{ id: "paid" },
|
||||
{ id: "withdrawn", cost: { input: 8, output: 8 } },
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
modelPricing: { providers: { "fixture-native": { openCode: { provider: "upstream" } } } },
|
||||
}),
|
||||
);
|
||||
return root;
|
||||
}
|
||||
|
||||
function fixtureFetch() {
|
||||
return vi.fn<typeof fetch>(async (url) => {
|
||||
if (url === "https://models.opencode.ai/api.json") {
|
||||
return Response.json({
|
||||
upstream: {
|
||||
id: "upstream",
|
||||
models: {
|
||||
free: { id: "free", cost: { input: 0, output: 0 } },
|
||||
paid: { id: "paid", cost: { input: 2, output: 3 } },
|
||||
extra: { id: "extra", cost: { input: 5, output: 6 } },
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
return Response.json({ data: [] });
|
||||
});
|
||||
}
|
||||
|
||||
function pairFixture(existing = true) {
|
||||
const rootDir = fixtureRoot();
|
||||
const outputs: [string, string] = [
|
||||
path.join(rootDir, "v1/catalog.json"),
|
||||
path.join(rootDir, "v2/catalog.json"),
|
||||
];
|
||||
outputs.forEach((file, index) => {
|
||||
fs.mkdirSync(path.dirname(file));
|
||||
if (existing) {
|
||||
fs.writeFileSync(file, `previous v${index + 1}`);
|
||||
}
|
||||
});
|
||||
const run = () =>
|
||||
runPublishModelCatalog({
|
||||
rootDir,
|
||||
sourceCommit: "fixture",
|
||||
now: () => 42,
|
||||
fetchImpl: fixtureFetch(),
|
||||
args: ["--out", outputs[0], "--out-v2", outputs[1]],
|
||||
});
|
||||
const recovery = (index: number) => {
|
||||
const output = outputs[index];
|
||||
if (!output) {
|
||||
throw new Error("fixture output index is invalid");
|
||||
}
|
||||
const parent = path.dirname(output);
|
||||
return fs
|
||||
.readdirSync(parent)
|
||||
.filter((name) => name.startsWith(".catalog-pair-"))
|
||||
.map((name) => path.join(parent, name));
|
||||
};
|
||||
vi.spyOn(process.stdout, "write").mockImplementation(() => true);
|
||||
const warnings = vi.spyOn(process.stderr, "write").mockImplementation(() => true);
|
||||
return { outputs, run, recovery, warnings };
|
||||
}
|
||||
|
||||
describe("publish model catalog v2", () => {
|
||||
it("rejects a real second-parent failure before replacing v1", async () => {
|
||||
const fixture = pairFixture();
|
||||
fs.unlinkSync(fixture.outputs[1]);
|
||||
fs.rmdirSync(path.dirname(fixture.outputs[1]));
|
||||
fs.writeFileSync(path.dirname(fixture.outputs[1]), "not a directory");
|
||||
await expect(fixture.run()).rejects.toThrow(/EEXIST|ENOTDIR/u);
|
||||
expect(fs.readFileSync(fixture.outputs[0], "utf8")).toBe("previous v1");
|
||||
expect(fixture.recovery(0)).toEqual([]);
|
||||
});
|
||||
|
||||
it("keeps both existing outputs when the second prepared write fails", async () => {
|
||||
const fixture = pairFixture();
|
||||
const write = fs.writeFileSync;
|
||||
vi.spyOn(fs, "writeFileSync").mockImplementation((file, data, options) => {
|
||||
if (
|
||||
typeof file === "number" &&
|
||||
typeof data === "string" &&
|
||||
data.includes('"schemaVersion": 2')
|
||||
) {
|
||||
write(file, "partial");
|
||||
throw new Error("fixture second prepare ENOSPC");
|
||||
}
|
||||
return write(file, data, options);
|
||||
});
|
||||
await expect(fixture.run()).rejects.toThrow("fixture second prepare ENOSPC");
|
||||
fixture.outputs.forEach((file, index) => {
|
||||
expect(fs.readFileSync(file, "utf8")).toBe(`previous v${index + 1}`);
|
||||
expect(fixture.recovery(index)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
"retains old/new recovery on a second rename failure (existing=%s)",
|
||||
async (existing) => {
|
||||
const fixture = pairFixture(existing);
|
||||
const rename = fs.promises.rename;
|
||||
vi.spyOn(fs.promises, "rename").mockImplementation(async (source, destination) => {
|
||||
if (destination === fixture.outputs[1]) {
|
||||
throw new Error("fixture second rename EIO");
|
||||
}
|
||||
return rename(source, destination);
|
||||
});
|
||||
await expect(fixture.run()).rejects.toThrow("Recovery retained:");
|
||||
expect(JSON.parse(fs.readFileSync(fixture.outputs[0], "utf8")).schemaVersion).toBe(1);
|
||||
if (existing) {
|
||||
expect(fs.readFileSync(fixture.outputs[1], "utf8")).toBe("previous v2");
|
||||
} else {
|
||||
expect(fs.existsSync(fixture.outputs[1])).toBe(false);
|
||||
}
|
||||
fixture.outputs.forEach((file, index) => {
|
||||
const dirs = fixture.recovery(index);
|
||||
expect(dirs).toHaveLength(1);
|
||||
const [dir] = dirs;
|
||||
if (!dir) {
|
||||
throw new Error("fixture recovery is missing");
|
||||
}
|
||||
expect(JSON.parse(fs.readFileSync(path.join(dir, "next.json"), "utf8")).schemaVersion).toBe(
|
||||
index + 1,
|
||||
);
|
||||
const note = fs.readFileSync(path.join(dir, "RECOVERY.txt"), "utf8");
|
||||
expect(note).toContain(file);
|
||||
expect(note).toContain(fixture.outputs[1 - index]);
|
||||
if (existing) {
|
||||
expect(fs.readFileSync(path.join(dir, "previous.json"), "utf8")).toBe(
|
||||
`previous v${index + 1}`,
|
||||
);
|
||||
} else {
|
||||
expect(note).toContain("was absent");
|
||||
expect(fs.existsSync(path.join(dir, "previous.json"))).toBe(false);
|
||||
}
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("retains recovery after a partial second publication write", async () => {
|
||||
const fixture = pairFixture();
|
||||
const write = fs.writeFileSync;
|
||||
vi.spyOn(fs, "writeFileSync").mockImplementation((file, data, options) => {
|
||||
if (
|
||||
typeof file === "string" &&
|
||||
typeof data === "string" &&
|
||||
data.includes('"schemaVersion": 2')
|
||||
) {
|
||||
write(file, "partial", options);
|
||||
throw new Error("fixture publication write ENOSPC");
|
||||
}
|
||||
return write(file, data, options);
|
||||
});
|
||||
await expect(fixture.run()).rejects.toThrow("fixture publication write ENOSPC");
|
||||
expect(JSON.parse(fs.readFileSync(fixture.outputs[0], "utf8")).schemaVersion).toBe(1);
|
||||
expect(fs.readFileSync(fixture.outputs[1], "utf8")).toBe("previous v2");
|
||||
expect(fixture.recovery(0)).toHaveLength(1);
|
||||
expect(fixture.recovery(1)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("preserves an observed destination replacement before the second publication", async () => {
|
||||
const fixture = pairFixture();
|
||||
const rename = fs.promises.rename;
|
||||
vi.spyOn(fs.promises, "rename").mockImplementation(async (source, destination) => {
|
||||
await rename(source, destination);
|
||||
if (destination === fixture.outputs[0]) {
|
||||
fs.unlinkSync(fixture.outputs[1]);
|
||||
fs.writeFileSync(fixture.outputs[1], "foreign replacement");
|
||||
}
|
||||
});
|
||||
await expect(fixture.run()).rejects.toThrow("output changed during preparation");
|
||||
expect(fs.readFileSync(fixture.outputs[1], "utf8")).toBe("foreign replacement");
|
||||
expect(fixture.recovery(0)).toHaveLength(1);
|
||||
expect(fixture.recovery(1)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("does not roll back foreign replacements after an uncertain second rename", async () => {
|
||||
const fixture = pairFixture();
|
||||
const rename = fs.promises.rename;
|
||||
vi.spyOn(fs.promises, "rename").mockImplementation(async (source, destination) => {
|
||||
await rename(source, destination);
|
||||
if (destination === fixture.outputs[1]) {
|
||||
for (const file of fixture.outputs) {
|
||||
fs.unlinkSync(file);
|
||||
fs.writeFileSync(file, "foreign replacement");
|
||||
}
|
||||
throw new Error("fixture post-rename failure");
|
||||
}
|
||||
});
|
||||
await expect(fixture.run()).rejects.toThrow("fixture post-rename failure");
|
||||
fixture.outputs.forEach((file, index) => {
|
||||
expect(fs.readFileSync(file, "utf8")).toBe("foreign replacement");
|
||||
expect(fixture.recovery(index)).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
it("reports cleanup failure without rejecting a successfully published pair", async () => {
|
||||
const fixture = pairFixture();
|
||||
const unlink = fs.unlinkSync;
|
||||
vi.spyOn(fs, "unlinkSync").mockImplementation((file) => {
|
||||
if (String(file).includes(".catalog-pair-")) {
|
||||
throw new Error("fixture cleanup EACCES");
|
||||
}
|
||||
unlink(file);
|
||||
});
|
||||
await expect(fixture.run()).resolves.toMatchObject({ wrote: true });
|
||||
fixture.outputs.forEach((file, index) => {
|
||||
expect(JSON.parse(fs.readFileSync(file, "utf8")).schemaVersion).toBe(index + 1);
|
||||
expect(fixture.recovery(index)).toHaveLength(1);
|
||||
});
|
||||
expect(fixture.warnings.mock.calls.flat().join("")).toContain(
|
||||
"pair published; recovery cleanup failed",
|
||||
);
|
||||
});
|
||||
|
||||
it("replaces an existing pair and removes only its recovery artifacts", async () => {
|
||||
const fixture = pairFixture();
|
||||
const parent = path.dirname(fixture.outputs[0]);
|
||||
const parentMode = fs.statSync(parent).mode;
|
||||
fs.writeFileSync(path.join(parent, "unrelated"), "preserve");
|
||||
await expect(fixture.run()).resolves.toMatchObject({ wrote: true });
|
||||
fixture.outputs.forEach((file, index) => {
|
||||
expect(JSON.parse(fs.readFileSync(file, "utf8")).schemaVersion).toBe(index + 1);
|
||||
expect(fixture.recovery(index)).toEqual([]);
|
||||
});
|
||||
expect(fs.statSync(parent).mode).toBe(parentMode);
|
||||
expect(fs.readFileSync(path.join(parent, "unrelated"), "utf8")).toBe("preserve");
|
||||
});
|
||||
|
||||
it("adds an explicit second output while keeping --out as v1", () => {
|
||||
expect(parsePublishModelCatalogArgs(["--out", "v1.json", "--out-v2", "v2.json"])).toEqual({
|
||||
dryRun: false,
|
||||
pricing: false,
|
||||
out: "v1.json",
|
||||
outV2: "v2.json",
|
||||
});
|
||||
expect(() => parsePublishModelCatalogArgs(["--out-v2", "v2.json"])).toThrow("provide --out");
|
||||
expect(() => parsePublishModelCatalogArgs(["--out", "v1.json", "--out-v2"])).toThrow(
|
||||
"requires a value",
|
||||
);
|
||||
});
|
||||
|
||||
it("projects only metadata rows with partial costs, tiers, and native tuple identity", async () => {
|
||||
const bundle: RemoteModelCatalogBundle = {
|
||||
schemaVersion: 1,
|
||||
generatedAt: 1,
|
||||
sourceCommit: "fixture",
|
||||
minVersion: "2026.7.0",
|
||||
providers: {
|
||||
alpha: {
|
||||
models: [
|
||||
{ id: "vendor/model", cost: { input: 2 } },
|
||||
{ id: "zero", cost: { input: 0, output: 0 } },
|
||||
{
|
||||
id: "tier",
|
||||
cost: {
|
||||
tieredPricing: [{ input: 3, output: 4, cacheRead: 0, cacheWrite: 0, range: [0] }],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
beta: { models: [{ id: "vendor/model" }] },
|
||||
},
|
||||
pricing: { "alpha/extra": { input: 9, output: 9 } },
|
||||
};
|
||||
const before = serializeModelCatalogBundle(bundle);
|
||||
const v2 = await assembleModelCatalogBundleV2(bundle, new WeakMap());
|
||||
expect(v2.models).toHaveLength(4);
|
||||
expect(v2.models[0]).toMatchObject({
|
||||
id: "vendor/model",
|
||||
provider: "alpha",
|
||||
pricing: { status: "known", input: 2 },
|
||||
});
|
||||
expect(v2.models[0]?.pricing).not.toHaveProperty("source");
|
||||
expect(v2.models[0]?.pricing).not.toHaveProperty("output");
|
||||
expect(v2.models[1]?.pricing).toEqual({ status: "unknown" });
|
||||
expect(v2.models[2]?.pricing).toMatchObject({
|
||||
status: "known",
|
||||
tieredPricing: bundle.providers.alpha?.models[2]?.cost?.tieredPricing,
|
||||
});
|
||||
expect(v2.models[3]).toMatchObject({
|
||||
id: "vendor/model",
|
||||
provider: "beta",
|
||||
pricing: { status: "unknown" },
|
||||
});
|
||||
expect(v2).not.toHaveProperty("pricing");
|
||||
expect(v2).not.toHaveProperty("minVersion");
|
||||
expect(serializeModelCatalogBundle(bundle)).toBe(before);
|
||||
const serialized = serializeModelCatalogBundleV2(v2);
|
||||
const parsed = parseRemoteModelCatalogBundleV2(JSON.parse(serialized));
|
||||
expect(parsed.models).toHaveLength(4);
|
||||
expect(Object.keys(JSON.parse(serialized).models[0]).slice(0, 2)).toEqual(["id", "provider"]);
|
||||
expect(serializeModelCatalogBundleV2({ ...v2, models: v2.models.toReversed() })).toBe(
|
||||
serialized,
|
||||
);
|
||||
});
|
||||
|
||||
it("writes paired catalogs from one source snapshot, preserving authoritative zero and withdrawal", async () => {
|
||||
const rootDir = fixtureRoot();
|
||||
const fetchImpl = fixtureFetch();
|
||||
vi.spyOn(process.stdout, "write").mockImplementation(() => true);
|
||||
vi.spyOn(process.stderr, "write").mockImplementation(() => true);
|
||||
await runPublishModelCatalog({
|
||||
rootDir,
|
||||
fetchImpl,
|
||||
now: () => 42,
|
||||
sourceCommit: "fixture",
|
||||
args: ["--pricing", "--out", "v1/catalog.json", "--out-v2", "v2/catalog.json"],
|
||||
});
|
||||
const v1 = parseRemoteModelCatalogBundle(
|
||||
JSON.parse(fs.readFileSync(path.join(rootDir, "v1/catalog.json"), "utf8")),
|
||||
);
|
||||
const v2 = parseRemoteModelCatalogBundleV2(
|
||||
JSON.parse(fs.readFileSync(path.join(rootDir, "v2/catalog.json"), "utf8")),
|
||||
);
|
||||
expect(v1.generatedAt).toBe(v2.generatedAt);
|
||||
expect(v1.sourceCommit).toBe(v2.sourceCommit);
|
||||
expect(v1.minVersion).toBe("2026.7.0");
|
||||
expect(v2.models).toHaveLength(203);
|
||||
expect(v2.providers.openai?.defaultModel).toBe("seed-0");
|
||||
expect(v2.providers.openai?.recommendedModels).toEqual(["seed-2", "seed-0"]);
|
||||
expect(v1.providers.openai).not.toHaveProperty("recommendedModels");
|
||||
expect(v2.providers.anthropic).not.toHaveProperty("recommendedModels");
|
||||
expect(
|
||||
v2.models.find((model) => model.provider === "anthropic" && model.id === "seed-0")?.pricing,
|
||||
).toEqual({
|
||||
status: "known",
|
||||
currency: "USD",
|
||||
unit: "million_tokens",
|
||||
input: 0.5,
|
||||
source: "manifest",
|
||||
});
|
||||
expect(v2.models.find((model) => model.id === "free")?.pricing).toMatchObject({
|
||||
status: "known",
|
||||
input: 0,
|
||||
output: 0,
|
||||
source: "openCode",
|
||||
});
|
||||
expect(v2.models.find((model) => model.id === "paid")?.pricing).toMatchObject({
|
||||
status: "known",
|
||||
input: 2,
|
||||
output: 3,
|
||||
source: "openCode",
|
||||
});
|
||||
expect(v2.models.find((model) => model.id === "withdrawn")?.pricing).toEqual({
|
||||
status: "unavailable",
|
||||
source: "openCode",
|
||||
});
|
||||
expect(v1.pricing?.["fixture-native/extra"]).toBeDefined();
|
||||
expect(v2.models.some((model) => model.id === "extra")).toBe(false);
|
||||
expect(
|
||||
fetchImpl.mock.calls.filter(([url]) => url === "https://models.opencode.ai/api.json"),
|
||||
).toHaveLength(1);
|
||||
await runPublishModelCatalog({
|
||||
rootDir,
|
||||
fetchImpl: fixtureFetch(),
|
||||
now: () => 42,
|
||||
sourceCommit: "fixture",
|
||||
args: ["--pricing", "--out", "v1-only.json"],
|
||||
});
|
||||
expect(fs.readFileSync(path.join(rootDir, "v1-only.json"), "utf8")).toBe(
|
||||
fs.readFileSync(path.join(rootDir, "v1/catalog.json"), "utf8"),
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps both prior files on source failure and never writes in dry-run mode", async () => {
|
||||
const rootDir = fixtureRoot();
|
||||
const out = path.join(rootDir, "v1.json");
|
||||
const outV2 = path.join(rootDir, "v2.json");
|
||||
fs.writeFileSync(out, "previous v1");
|
||||
fs.writeFileSync(outV2, "previous v2");
|
||||
const args = ["--pricing", "--out", out, "--out-v2", outV2];
|
||||
vi.spyOn(process.stdout, "write").mockImplementation(() => true);
|
||||
vi.spyOn(process.stderr, "write").mockImplementation(() => true);
|
||||
await expect(
|
||||
runPublishModelCatalog({
|
||||
rootDir,
|
||||
sourceCommit: "fixture",
|
||||
args,
|
||||
fetchImpl: async () => {
|
||||
throw new Error("fixture outage");
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow("fixture outage");
|
||||
await runPublishModelCatalog({
|
||||
rootDir,
|
||||
sourceCommit: "fixture",
|
||||
args: [...args, "--dry-run"],
|
||||
fetchImpl: fixtureFetch(),
|
||||
});
|
||||
expect(fs.readFileSync(out, "utf8")).toBe("previous v1");
|
||||
expect(fs.readFileSync(outV2, "utf8")).toBe("previous v2");
|
||||
await expect(
|
||||
runPublishModelCatalog({ rootDir, args: ["--out", "same.json", "--out-v2", "./same.json"] }),
|
||||
).rejects.toThrow("different files");
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue