mirror of
https://github.com/openclaw/openclaw.git
synced 2026-10-03 01:29:56 +00:00
feat: use protected model credentials in Crabbox applications (#156207)
* feat: use protected model credentials in Crabbox applications * refactor: use the shared model egress lazy runtime owner
This commit is contained in:
parent
da98b092e4
commit
741a1c84cd
23 changed files with 1038 additions and 97 deletions
|
|
@ -48,6 +48,12 @@ Cloud workers are opt-in. Until you configure a profile, clients hide the Cloud
|
|||
| Transcript and live session state | Gateway, fed by the worker's replayable event stream | Gateway through the normal local harness path |
|
||||
| Workspace file state | Changed on the box; reconciled by the Gateway | Changed remotely; reconciled by the Gateway |
|
||||
|
||||
Applications that make their own model API calls need a separate credential
|
||||
route. For an exclusively owned coordinator-backed Linux lease, use
|
||||
[`openclaw crabbox run`](/gateway/secrets/secret-store-and-egress#model-credentials-for-crabbox-commands)
|
||||
to keep the configured API key on the host while the application uses a protected
|
||||
egress bridge.
|
||||
|
||||
The bundled Crabbox cloud provider advertises both `worker-turn` and `remote-exec` through its enrolled node transport, so the same cloud profile is available to both harnesses. Codex can also use an explicitly authorized paired device or a provider that retains an SSH-backed remote-execution carrier. A profile that advertises only one mode remains unavailable to the other runtime.
|
||||
|
||||
After Crabbox setup, the cloud node dials the Gateway's public TLS endpoint over outbound WebSocket. Worker control, Codex remote execution, and workspace transfer use authenticated node or worker channels, not a Gateway-created reverse tunnel or rsync. Crabbox itself may still require SSH reachability while its CLI runs the provider-owned setup command. Outbound internet access and setup reachability follow the selected backend's network policy; configure them in Crabbox.
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ summary: "The shared secret store, the default-off secret egress proxy, and file
|
|||
read_when:
|
||||
- Storing team-wide secrets and environment values in the shared secret store
|
||||
- Enabling the destination-bound secret egress proxy or its traffic allowlist
|
||||
- Running a Crabbox application with a configured model credential kept on the host
|
||||
title: "Shared secret store and egress proxy"
|
||||
---
|
||||
|
||||
|
|
@ -145,9 +146,110 @@ Current limits:
|
|||
- Identity-scoped secrets are not supported; only the team store participates.
|
||||
- Allowed-host policy is exact-hostname authorization only. It does not validate the resolved IP or prevent an allowed origin from reflecting credentials.
|
||||
- Plain HTTP is refused; it is not upgraded or substituted.
|
||||
- Secret egress applies only to Gateway-hosted exec. Sandbox and remote `node` exec receive neither proxy variables nor sentinels, so shared-store `secret` entries are unavailable there. Provider-native harness subprocesses also do not use this proxy.
|
||||
- Automatic shared-store secret egress applies only to Gateway-hosted exec. Sandbox and remote `node` exec receive neither proxy variables nor sentinels, so shared-store `secret` entries are unavailable there. Provider-native harness subprocesses also do not use this proxy. The explicit Crabbox command below grants a configured model credential separately.
|
||||
- Background subprocesses retain their original secret snapshot until they exit or are stopped. Changes to stored credentials or destination bindings require a new run and a new command; stop existing commands to revoke their older grants immediately.
|
||||
|
||||
## Model credentials for Crabbox commands
|
||||
|
||||
The Crabbox plugin lets a foreground application in a Linux lease call an
|
||||
OpenAI-compatible API using a credential kept on the host. Cloud agents already
|
||||
keep their own inference and provider authentication on the Gateway; this command
|
||||
is for API calls made by the application itself.
|
||||
|
||||
### Requirements
|
||||
|
||||
Run the command on the host that owns the configured credential, from the local
|
||||
project directory that owns the lease. Use an exclusively owned,
|
||||
coordinator-backed Linux lease with a configured Crabbox login and no active
|
||||
egress session. The Crabbox binary must support `egress run` with
|
||||
`--upstream-proxy-env`; the command checks support before reading the credential.
|
||||
Use `--binary <path>` to select another binary.
|
||||
|
||||
The selected provider must use an API-key SecretRef in
|
||||
`models.providers.<provider>.apiKey`. File, environment, exec, and shared-store
|
||||
SecretRefs use the existing resolver without copying the credential into another
|
||||
store. The model must resolve to an `openai-responses` or `openai-completions`
|
||||
route with an HTTPS endpoint on port 443. Endpoint URLs cannot contain credentials,
|
||||
a query, or a fragment. Auth-profile and OAuth credentials, custom headers,
|
||||
request proxy/TLS overrides, disabled auth headers, and local-service
|
||||
configuration are unsupported.
|
||||
|
||||
### Prepare and run
|
||||
|
||||
Sync files, hydrate the workspace, and install dependencies through the normal
|
||||
Crabbox workflow first. The model command passes `--no-sync --no-hydrate`, so it
|
||||
uses the prepared workspace and cannot fetch dependencies through its
|
||||
model-host-only bridge. Keep the same local project directory for preparation
|
||||
and execution: `--id` selects a lease but does not override Crabbox's repository
|
||||
claim or workspace selection.
|
||||
|
||||
For an existing lease and configured OpenAI SecretRef, this example checks that
|
||||
`curl` is available, then makes a Responses API request without reading the key:
|
||||
|
||||
```bash
|
||||
cd ~/path/to/project
|
||||
crabbox run --id <lease-id> -- curl --version
|
||||
openclaw crabbox run --id <lease-id> --model openai/gpt-5.6-sol -- sh -c '
|
||||
curl --fail-with-body --silent --show-error "${OPENAI_BASE_URL%/}/responses" \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data "{\"model\":\"$OPENAI_MODEL\",\"input\":\"Reply with OK.\"}"
|
||||
'
|
||||
```
|
||||
|
||||
Success returns the provider's response JSON. Replace `sh -c ...` with your
|
||||
application command, such as `node test-app.js`. Use the endpoint appropriate to
|
||||
the configured route; a Completions-only provider needs its matching API call.
|
||||
`--provider <backend>` selects a Crabbox backend, while `--model` selects the
|
||||
model provider. `--timeout <seconds>` bounds setup and execution together
|
||||
(1–86400 seconds, default 600). Cancellation and timeout revoke credential access
|
||||
immediately and give Crabbox 75 seconds for graceful cleanup.
|
||||
|
||||
OpenClaw resolves the selected model's configured or provider-owned API endpoint
|
||||
and starts an isolated secret proxy. Crabbox's native `egress run` owns the
|
||||
foreground bridge, remote command, and session cleanup. OpenClaw supplies
|
||||
`OPENAI_API_KEY` as an opaque sentinel,
|
||||
`OPENAI_BASE_URL`, and `OPENAI_MODEL`, plus HTTP proxy settings and a temporary
|
||||
public CA bundle. The API key, upstream proxy authentication, and CA private key
|
||||
stay on the host. The bridge permits only the selected hostname; this does not
|
||||
block a program from opening direct sockets. Model selection sets the app's
|
||||
default environment, not a limit on the provider credential's API operations or
|
||||
models.
|
||||
|
||||
The application's HTTP client must honor both proxy and CA settings. `curl` and
|
||||
Python's default `urllib.request` opener use the injected environment. For Node.js,
|
||||
use a runtime supporting `NODE_USE_ENV_PROXY` (for example Node.js 24+) with
|
||||
`NODE_EXTRA_CA_CERTS`. A custom Node dispatcher, Python opener, or SDK client may
|
||||
override these defaults; configure its proxy and trust explicitly if needed.
|
||||
Disabling certificate verification or ignoring the proxy does not establish
|
||||
protected model access.
|
||||
|
||||
### Lifetime and recovery
|
||||
|
||||
Cancellation and timeout revoke credential use immediately, before command
|
||||
cleanup settles. Completion closes the grant and bridge and stops the matching
|
||||
lease-side egress client. The lease and prepared workspace remain available.
|
||||
Keep the foreground command running for the entire app lifetime; detached apps
|
||||
lose model access when it exits. Ordinary remote `exec`, `background`, and
|
||||
sandbox commands do not acquire this grant.
|
||||
|
||||
An old binary is refused with an update message. An active-egress error requires
|
||||
an idle lease or stopping an existing session you own. A repository-claim error
|
||||
means you must return to the lease's owning local project directory. Do not
|
||||
reclaim another job's lease to bypass either check.
|
||||
|
||||
If cleanup cannot confirm settlement, the command fails. Inspect
|
||||
`crabbox egress status --id <lease-id>` and, when the failed command reported a
|
||||
session ID, retry `crabbox egress stop --id <lease-id> --session <egress-session-id>`
|
||||
for that session. Also confirm the remote workload has stopped before reusing
|
||||
the lease, or release the disposable lease through its normal owner. Revocation
|
||||
is not proof that an unreachable remote process has exited.
|
||||
|
||||
This standalone command does not require `secrets.egressProxy.enabled`, change
|
||||
Gateway configuration, or restart the Gateway. After a completed or canceled
|
||||
command, start a new command to obtain a fresh grant; its old sentinel and CA
|
||||
files are not reusable credentials.
|
||||
|
||||
## File-backed API keys
|
||||
|
||||
Do not put `file:...` strings in the config `env` block. That block is literal and non-overriding, so `file:...` is never resolved there.
|
||||
|
|
|
|||
|
|
@ -10,6 +10,41 @@ sidebarTitle: "Model helpers"
|
|||
|
||||
Call a model, resolve model-selection policy, and resolve provider auth without importing host internals. Part of the [Plugin runtime helpers](/plugins/sdk-runtime) reference.
|
||||
|
||||
## Protected model egress for standalone commands
|
||||
|
||||
`withConfiguredModelEgress` from `openclaw/plugin-sdk/secret-egress-runtime`
|
||||
runs an explicit standalone command for an official plugin with a destination-bound credential
|
||||
sentinel. It resolves the selected provider's configured API-key SecretRef
|
||||
through the normal secret resolver, including file-backed keys, and uses the
|
||||
provider's model route policy for its HTTPS endpoint on port 443. It supports
|
||||
OpenAI-compatible Responses and Completions routes. OAuth, auth-profile
|
||||
references, custom request headers, and custom request transport are unsupported.
|
||||
This is a private JavaScript-only host binding for bundled and separately
|
||||
published official plugins, not a third-party plugin API.
|
||||
|
||||
```typescript
|
||||
const { withConfiguredModelEgress } = await import("openclaw/plugin-sdk/secret-egress-runtime");
|
||||
await withConfiguredModelEgress({ config, provider, model, signal }, async (egress) => {
|
||||
// Keep hostEnv on the credential-owning host for the authenticated bridge.
|
||||
// Send only sentinel, baseUrl, model, and public caBundle to the remote app.
|
||||
await runRemoteAppThroughBridge(egress, signal);
|
||||
});
|
||||
```
|
||||
|
||||
The callback receives `sentinel`, `baseUrl`, `model`, `allowedHosts`, `hostEnv`,
|
||||
and the public `caBundle` contents. Supply `onOutput(text, stream)` to receive
|
||||
live output with resolved credentials redacted, including values split between
|
||||
chunks; pass the callback's `onOutputChunk` to the command runner. The caller owns its bridge and remote
|
||||
process, must honor cancellation, and must join both before its callback
|
||||
settles. Cancellation revokes proxy access immediately; callback completion or
|
||||
failure revokes access, stops the isolated proxy, and removes its private CA
|
||||
directory. Credentials and proxy grants never enter the shared secret store.
|
||||
|
||||
This is an explicit command-scoped proxy using the existing secret egress
|
||||
implementation. It does not require enabling or restarting the Gateway's
|
||||
persistent egress proxy. Import the SDK module only in the command execution
|
||||
path; importing it alone does not load model or secret runtime code.
|
||||
|
||||
## Prepared simple completions
|
||||
|
||||
The `openclaw/plugin-sdk/simple-completion-runtime` helpers support preparing a
|
||||
|
|
|
|||
|
|
@ -268,6 +268,7 @@ usage endpoint failed or returned no usable usage data.
|
|||
| `plugin-sdk/channel-secret-basic-runtime` | Narrow secret-contract exports and target-registry builders for non-TTS channel/plugin secret surfaces |
|
||||
| `plugin-sdk/channel-secret-tts-runtime` | Private-local after July 2026; Narrow nested channel TTS secret assignment helpers |
|
||||
| `plugin-sdk/secret-ref-runtime` | Narrow SecretRef typing, resolution, setup-plan construction, and setup CLI scaffolding for plugin-owned secret providers |
|
||||
| `plugin-sdk/secret-egress-runtime` | Private official-plugin runtime; command-scoped protected model credentials, public CA transfer, and revocable HTTPS egress through `withConfiguredModelEgress`; not a third-party plugin API |
|
||||
| `plugin-sdk/security-runtime` | Deprecated broad barrel for trust, DM gating, root-bounded file/path helpers including create-only writes, sync/async atomic file replacement, sibling temp writes, cross-device move fallback, private file-store helpers, symlink-parent guards, external-content, `redactSensitiveText`, constant-time secret comparison, and secret-collection helpers; prefer focused security/SSRF/secret subpaths |
|
||||
| `plugin-sdk/ssrf-policy` | Host allowlist and private-network SSRF policy helpers |
|
||||
| `plugin-sdk/ssrf-dispatcher` | Private-local after July 2026; Narrow pinned-dispatcher helpers without the broad infra runtime surface |
|
||||
|
|
|
|||
|
|
@ -33,16 +33,18 @@ export default definePluginEntry({
|
|||
tags: ["cloud", "desktop"],
|
||||
});
|
||||
api.registerCli(
|
||||
async ({ program }) => {
|
||||
async ({ program, config }) => {
|
||||
const { registerCrabboxWarmImageCommands } =
|
||||
await import("./src/crabbox-worker-warm-image-cli.js");
|
||||
registerCrabboxWarmImageCommands(program, api.runtime.state);
|
||||
const { registerCrabboxModelRunCommand } = await import("./src/crabbox-model-run-cli.js");
|
||||
registerCrabboxModelRunCommand({ program, config });
|
||||
},
|
||||
{
|
||||
descriptors: [
|
||||
{
|
||||
name: "crabbox",
|
||||
description: "Inspect and recover Crabbox warm images",
|
||||
description: "Run model-backed commands and manage Crabbox warm images",
|
||||
hasSubcommands: true,
|
||||
},
|
||||
],
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
"cliCommands": [
|
||||
{
|
||||
"name": "crabbox",
|
||||
"description": "Inspect and recover Crabbox warm images",
|
||||
"description": "Run model-backed commands and manage Crabbox warm images",
|
||||
"hasSubcommands": true
|
||||
}
|
||||
],
|
||||
|
|
|
|||
|
|
@ -52,6 +52,39 @@ capability and the actual completed state. A portal that the user's browser
|
|||
cannot reach is not a completed preview. Preserve the separate-origin portal
|
||||
transport; never expose arbitrary application scripts on the Gateway origin.
|
||||
|
||||
## Apps that call a model API
|
||||
|
||||
Cloud-agent inference already uses Gateway-held authentication. An application
|
||||
inside the lease making its own API calls needs a separate protected route.
|
||||
For an exclusively owned coordinator-backed Linux lease, use the host CLI:
|
||||
|
||||
```sh
|
||||
openclaw crabbox run --id <lease-id> --model <provider/model> -- <command> <args>
|
||||
```
|
||||
|
||||
Run from the credential-owning host and the local project directory that owns
|
||||
the lease. Prepare source and dependencies first: the command skips sync and
|
||||
hydration, and its bridge permits only the model host. A lease ID does not
|
||||
override Crabbox's repository claim. Require no active egress session and a
|
||||
Crabbox binary supporting `egress run --upstream-proxy-env`. Crabbox owns the
|
||||
foreground bridge, remote command, and session cleanup.
|
||||
|
||||
The provider must have a configured API-key SecretRef and an OpenAI-compatible
|
||||
HTTPS endpoint on port 443. Auth-profile/OAuth credentials, custom headers, and
|
||||
request transport overrides are unsupported. The CLI injects a sentinel as
|
||||
`OPENAI_API_KEY`, plus `OPENAI_BASE_URL`, `OPENAI_MODEL`, proxy settings, and
|
||||
public CA trust. The actual key and upstream proxy credentials stay on the host.
|
||||
Use a client that honors the proxy and CA environment; Node.js needs
|
||||
`NODE_USE_ENV_PROXY` support, while Python's default `urllib.request` opener
|
||||
honors the environment. Custom SDK clients may need explicit proxy/trust setup.
|
||||
|
||||
Keep this command alive for the full app lifetime. Do not detach the app or
|
||||
copy a key into the box. Ordinary tool `exec` and `background` do not acquire
|
||||
this model grant. Cancellation revokes access before cleanup; if settlement is
|
||||
uncertain, inspect the named session and remote process before reusing the lease.
|
||||
This path needs no Gateway restart or persistent egress setting. See the
|
||||
[setup, runnable example, and recovery guide](https://docs.openclaw.ai/gateway/secrets/secret-store-and-egress#model-credentials-for-crabbox-commands).
|
||||
|
||||
## Follow-ups and cleanup
|
||||
|
||||
Reopen the current environment or portal for "show me again". A viewer reconnect
|
||||
|
|
|
|||
65
extensions/crabbox/src/crabbox-model-run-cli.ts
Normal file
65
extensions/crabbox/src/crabbox-model-run-cli.ts
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
||||
import { resolveCrabboxBinary } from "./crabbox-binary.js";
|
||||
import { runCrabboxModelCommand } from "./crabbox-model-run.js";
|
||||
|
||||
type CliContext = Parameters<Parameters<OpenClawPluginApi["registerCli"]>[0]>[0];
|
||||
|
||||
export function registerCrabboxModelRunCommand({
|
||||
program,
|
||||
config,
|
||||
}: Pick<CliContext, "program" | "config">): void {
|
||||
const crabbox =
|
||||
program.commands.find((command) => command.name() === "crabbox") ?? program.command("crabbox");
|
||||
crabbox.description("Run model-backed commands and manage Crabbox warm images");
|
||||
crabbox
|
||||
.command("run")
|
||||
.description(
|
||||
"Run a foreground command on an exclusive Linux lease using a protected model credential",
|
||||
)
|
||||
.requiredOption("--id <lease>", "Existing, exclusively owned Crabbox lease")
|
||||
.requiredOption("--model <provider/model>", "Configured OpenAI-compatible API-key model")
|
||||
.option("--provider <provider>", "Crabbox backend override; otherwise use the lease's provider")
|
||||
.option("--binary <path>", "Crabbox binary with upstream proxy support")
|
||||
.option("--timeout <seconds>", "Total command and setup deadline in seconds", "600")
|
||||
.argument("<command...>", "Remote executable and arguments, following --")
|
||||
.action(
|
||||
async (
|
||||
argv: string[],
|
||||
options: {
|
||||
id: string;
|
||||
model: string;
|
||||
provider?: string;
|
||||
binary?: string;
|
||||
timeout: string;
|
||||
},
|
||||
) => {
|
||||
const seconds = Number(options.timeout);
|
||||
if (!Number.isSafeInteger(seconds) || seconds < 1 || seconds > 86_400) {
|
||||
throw new Error("--timeout must be an integer between 1 and 86400 seconds");
|
||||
}
|
||||
const controller = new AbortController();
|
||||
const cancel = () => controller.abort(new Error("Crabbox model command interrupted"));
|
||||
process.once("SIGINT", cancel);
|
||||
process.once("SIGTERM", cancel);
|
||||
try {
|
||||
const result = await runCrabboxModelCommand({
|
||||
config,
|
||||
binary: resolveCrabboxBinary({ explicit: options.binary, pathEnv: process.env.PATH }),
|
||||
id: options.id,
|
||||
model: options.model,
|
||||
provider: options.provider,
|
||||
argv,
|
||||
timeoutMs: seconds * 1000,
|
||||
signal: controller.signal,
|
||||
onOutput: (text, stream) => {
|
||||
process[stream].write(text);
|
||||
},
|
||||
});
|
||||
process.exitCode = result.termination === "exit" ? (result.code ?? 1) : 1;
|
||||
} finally {
|
||||
process.off("SIGINT", cancel);
|
||||
process.off("SIGTERM", cancel);
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
122
extensions/crabbox/src/crabbox-model-run.test.ts
Normal file
122
extensions/crabbox/src/crabbox-model-run.test.ts
Normal file
|
|
@ -0,0 +1,122 @@
|
|||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import type { CommandOptions, SpawnResult } from "openclaw/plugin-sdk/process-runtime";
|
||||
import type { ConfiguredModelEgress } from "openclaw/plugin-sdk/secret-egress-runtime";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const mocks = vi.hoisted(() => ({ run: vi.fn(), withEgress: vi.fn() }));
|
||||
vi.mock("openclaw/plugin-sdk/process-runtime", () => ({ runCommandWithTimeout: mocks.run }));
|
||||
vi.mock("openclaw/plugin-sdk/secret-egress-runtime", () => ({
|
||||
withConfiguredModelEgress: mocks.withEgress,
|
||||
}));
|
||||
const { runCrabboxModelCommand } = await import("./crabbox-model-run.js");
|
||||
const execFileAsync = promisify(execFile);
|
||||
const capabilityHelp = " -upstream-proxy-env string\n";
|
||||
const egress: ConfiguredModelEgress = {
|
||||
sentinel: "oc-sent-v2.synthetic-placeholder.end",
|
||||
baseUrl: "https://api.example.com/v1/",
|
||||
model: "example-model",
|
||||
allowedHosts: ["api.example.com"],
|
||||
hostEnv: { HTTPS_PROXY: "http://openclaw:proxy-password-fixture@127.0.0.1:43210" },
|
||||
caBundle: "public-ca-fixture\n",
|
||||
};
|
||||
const options = {
|
||||
config: {},
|
||||
binary: "crabbox-fixture",
|
||||
id: "cbx_owned",
|
||||
model: "example/example-model",
|
||||
argv: ["node", "test-app.js"],
|
||||
timeoutMs: 60_000,
|
||||
};
|
||||
function result(stdout = "", code = 0): SpawnResult {
|
||||
return { stdout, stderr: "", code, signal: null, killed: false, termination: "exit" };
|
||||
}
|
||||
beforeEach(() => {
|
||||
mocks.run.mockReset();
|
||||
mocks.withEgress.mockReset();
|
||||
mocks.withEgress.mockImplementation(async (_options, run) => await run(egress));
|
||||
});
|
||||
|
||||
describe("Crabbox protected model command", () => {
|
||||
it("delegates one native job with a host-only proxy grant and sentinel-only app environment", async () => {
|
||||
let remoteInput = "";
|
||||
mocks.run.mockImplementation(async (argv: string[], params: CommandOptions) => {
|
||||
if (argv.includes("--help")) {
|
||||
return result(capabilityHelp);
|
||||
}
|
||||
expect(argv).toEqual([
|
||||
"crabbox-fixture",
|
||||
"egress",
|
||||
"run",
|
||||
"--id",
|
||||
"cbx_owned",
|
||||
"--allow",
|
||||
"api.example.com",
|
||||
"--upstream-proxy-env",
|
||||
"CRABBOX_MODEL_PROXY",
|
||||
"--no-sync",
|
||||
"--no-hydrate",
|
||||
"--script-stdin",
|
||||
"--",
|
||||
"node",
|
||||
"test-app.js",
|
||||
]);
|
||||
expect(params.env?.CRABBOX_MODEL_PROXY).toBe(egress.hostEnv.HTTPS_PROXY);
|
||||
expect(params.env?.CRABBOX_ENV_ALLOW).toBe(",");
|
||||
expect(params.killGraceMs).toBeGreaterThan(60_000);
|
||||
expect(argv.join(" ")).not.toContain("proxy-password-fixture");
|
||||
remoteInput = String(params.input);
|
||||
expect(remoteInput).not.toContain("proxy-password-fixture");
|
||||
return result("app finished\n", 7);
|
||||
});
|
||||
expect(await runCrabboxModelCommand(options)).toMatchObject({
|
||||
code: 7,
|
||||
stdout: "app finished\n",
|
||||
});
|
||||
expect(mocks.run).toHaveBeenCalledTimes(2);
|
||||
|
||||
const fixture = `const fs = require('fs'); console.log(JSON.stringify({
|
||||
key: process.env.OPENAI_API_KEY, baseUrl: process.env.OPENAI_BASE_URL,
|
||||
model: process.env.OPENAI_MODEL, proxy: process.env.HTTPS_PROXY,
|
||||
ca: fs.readFileSync(process.env.SSL_CERT_FILE, 'utf8'), path: process.env.SSL_CERT_FILE
|
||||
}));`;
|
||||
const child = execFile("bash", ["-s", "--", process.execPath, "-e", fixture], {
|
||||
env: { ...process.env, NODE_OPTIONS: undefined },
|
||||
});
|
||||
const completion = new Promise<string>((resolve, reject) => {
|
||||
let output = "";
|
||||
child.stdout?.on("data", (chunk) => {
|
||||
output += String(chunk);
|
||||
});
|
||||
child.once("error", reject);
|
||||
child.once("exit", (code) =>
|
||||
code === 0 ? resolve(output) : reject(new Error(`fixture exited ${code}`)),
|
||||
);
|
||||
});
|
||||
child.stdin!.end(remoteInput);
|
||||
const observed = JSON.parse(await completion);
|
||||
expect(observed).toMatchObject({
|
||||
key: egress.sentinel,
|
||||
baseUrl: egress.baseUrl,
|
||||
model: egress.model,
|
||||
proxy: "http://127.0.0.1:3128",
|
||||
ca: `${egress.caBundle}\n`,
|
||||
});
|
||||
await expect(execFileAsync("test", ["-e", observed.path])).rejects.toThrow();
|
||||
});
|
||||
|
||||
it.each(["", " --upstream-proxy-env-name string\n per-upstream-proxy-env help"])(
|
||||
"refuses incompatible native commands before credential access",
|
||||
async (help) => {
|
||||
mocks.run.mockResolvedValue(result(help));
|
||||
await expect(runCrabboxModelCommand(options)).rejects.toThrow("lacks native egress run");
|
||||
expect(mocks.withEgress).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it("reports unconfirmed native process cleanup as failure", async () => {
|
||||
mocks.run.mockResolvedValueOnce(result(capabilityHelp));
|
||||
mocks.run.mockResolvedValueOnce({ ...result(), cleanup: "uncertain" });
|
||||
await expect(runCrabboxModelCommand(options)).rejects.toThrow("could not confirm");
|
||||
});
|
||||
});
|
||||
128
extensions/crabbox/src/crabbox-model-run.ts
Normal file
128
extensions/crabbox/src/crabbox-model-run.ts
Normal file
|
|
@ -0,0 +1,128 @@
|
|||
import { randomBytes } from "node:crypto";
|
||||
import { redactSensitiveText } from "openclaw/plugin-sdk/logging-core";
|
||||
import { parseModelRef } from "openclaw/plugin-sdk/model-ref-parse";
|
||||
import type { OpenClawConfig } from "openclaw/plugin-sdk/plugin-entry";
|
||||
import {
|
||||
runCommandWithTimeout,
|
||||
type CommandOptions,
|
||||
type SpawnResult,
|
||||
} from "openclaw/plugin-sdk/process-runtime";
|
||||
import {
|
||||
withConfiguredModelEgress,
|
||||
type ConfiguredModelEgress,
|
||||
} from "openclaw/plugin-sdk/secret-egress-runtime";
|
||||
|
||||
const UPSTREAM_PROXY_ENV = "CRABBOX_MODEL_PROXY";
|
||||
const MAX_OUTPUT_BYTES = 1024 * 1024;
|
||||
|
||||
export type CrabboxModelRunOptions = {
|
||||
config: OpenClawConfig;
|
||||
binary: string;
|
||||
id: string;
|
||||
model: string;
|
||||
provider?: string;
|
||||
argv: string[];
|
||||
timeoutMs: number;
|
||||
signal?: AbortSignal;
|
||||
onOutput?: (text: string, stream: "stdout" | "stderr") => void;
|
||||
};
|
||||
|
||||
function shellQuote(value: string): string {
|
||||
if (value.includes("\0")) {
|
||||
throw new Error("Crabbox model command cannot contain NUL bytes");
|
||||
}
|
||||
return `'${value.replaceAll("'", `'"'"'`)}'`;
|
||||
}
|
||||
|
||||
function remoteCommandScript(egress: ConfiguredModelEgress): string {
|
||||
const delimiter = `MODEL_EGRESS_CA_${randomBytes(12).toString("hex")}`;
|
||||
return `#!/bin/bash
|
||||
set -euo pipefail
|
||||
umask 077
|
||||
egress_dir=$(mktemp -d)
|
||||
trap 'rm -rf -- "$egress_dir"' EXIT
|
||||
cat > "$egress_dir/ca.pem" <<'${delimiter}'
|
||||
${egress.caBundle}
|
||||
${delimiter}
|
||||
export OPENAI_API_KEY=${shellQuote(egress.sentinel)}
|
||||
export OPENAI_BASE_URL=${shellQuote(egress.baseUrl)}
|
||||
export OPENAI_MODEL=${shellQuote(egress.model)}
|
||||
export HTTPS_PROXY=http://127.0.0.1:3128 HTTP_PROXY=http://127.0.0.1:3128
|
||||
export https_proxy="$HTTPS_PROXY" http_proxy="$HTTP_PROXY"
|
||||
export NO_PROXY=localhost,127.0.0.1,::1 no_proxy=localhost,127.0.0.1,::1
|
||||
unset ALL_PROXY all_proxy
|
||||
export NODE_USE_ENV_PROXY=1
|
||||
export NODE_EXTRA_CA_CERTS="$egress_dir/ca.pem" SSL_CERT_FILE="$egress_dir/ca.pem"
|
||||
export CURL_CA_BUNDLE="$egress_dir/ca.pem" REQUESTS_CA_BUNDLE="$egress_dir/ca.pem"
|
||||
export GIT_SSL_CAINFO="$egress_dir/ca.pem"
|
||||
"$@"
|
||||
`;
|
||||
}
|
||||
|
||||
export async function runCrabboxModelCommand(params: CrabboxModelRunOptions): Promise<SpawnResult> {
|
||||
const model = parseModelRef(params.model, "");
|
||||
if (!params.model.includes("/") || !model?.provider || !model.model) {
|
||||
throw new Error("--model must name an explicit provider/model");
|
||||
}
|
||||
const signal = AbortSignal.any([
|
||||
AbortSignal.timeout(params.timeoutMs),
|
||||
...(params.signal ? [params.signal] : []),
|
||||
]);
|
||||
const command = (
|
||||
args: string[],
|
||||
options: Pick<CommandOptions, "env" | "input" | "onOutputChunk"> = {},
|
||||
) =>
|
||||
runCommandWithTimeout([params.binary, "egress", "run", ...args], {
|
||||
timeoutMs: params.timeoutMs,
|
||||
maxOutputBytes: MAX_OUTPUT_BYTES,
|
||||
killProcessTree: true,
|
||||
requireProcessTreeExtinction: true,
|
||||
// Native egress owns remote cleanup after SIGTERM; credential use ends immediately.
|
||||
killGraceMs: 75_000,
|
||||
signal,
|
||||
...options,
|
||||
env: { CRABBOX_ENV_ALLOW: ",", [UPSTREAM_PROXY_ENV]: undefined, ...options.env },
|
||||
});
|
||||
const help = await command(["--help"]);
|
||||
if (
|
||||
help.termination !== "exit" ||
|
||||
help.code !== 0 ||
|
||||
!/(?:^|\n)\s+-{1,2}upstream-proxy-env(?:\s|$)/u.test(`${help.stdout}\n${help.stderr}`)
|
||||
) {
|
||||
throw new Error("This Crabbox binary lacks native egress run support; update Crabbox first");
|
||||
}
|
||||
return await withConfiguredModelEgress(
|
||||
{ config: params.config, ...model, signal, onOutput: params.onOutput },
|
||||
async (egress) => {
|
||||
const result = await command(
|
||||
[
|
||||
"--id",
|
||||
params.id,
|
||||
...(params.provider ? ["--provider", params.provider] : []),
|
||||
"--allow",
|
||||
egress.allowedHosts.join(","),
|
||||
"--upstream-proxy-env",
|
||||
UPSTREAM_PROXY_ENV,
|
||||
"--no-sync",
|
||||
"--no-hydrate",
|
||||
"--script-stdin",
|
||||
"--",
|
||||
...params.argv,
|
||||
],
|
||||
{
|
||||
env: { [UPSTREAM_PROXY_ENV]: egress.hostEnv.HTTPS_PROXY },
|
||||
input: remoteCommandScript(egress),
|
||||
onOutputChunk: egress.onOutputChunk,
|
||||
},
|
||||
);
|
||||
if (result.cleanup === "uncertain") {
|
||||
throw new Error("Crabbox could not confirm that the command stopped");
|
||||
}
|
||||
return {
|
||||
...result,
|
||||
stdout: redactSensitiveText(result.stdout),
|
||||
stderr: redactSensitiveText(result.stderr),
|
||||
};
|
||||
},
|
||||
);
|
||||
}
|
||||
|
|
@ -954,6 +954,9 @@
|
|||
],
|
||||
"openclaw/plugin-sdk/agent-harness-attempt-runtime": [
|
||||
"../packages/plugin-sdk/dist/src/plugin-sdk/agent-harness-attempt-runtime.d.ts"
|
||||
],
|
||||
"openclaw/plugin-sdk/secret-egress-runtime": [
|
||||
"../packages/plugin-sdk/dist/src/plugin-sdk/secret-egress-runtime.d.ts"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -938,6 +938,9 @@
|
|||
],
|
||||
"openclaw/plugin-sdk/agent-harness-attempt-runtime": [
|
||||
"../../packages/plugin-sdk/dist/src/plugin-sdk/agent-harness-attempt-runtime.d.ts"
|
||||
],
|
||||
"openclaw/plugin-sdk/secret-egress-runtime": [
|
||||
"../../packages/plugin-sdk/dist/src/plugin-sdk/secret-egress-runtime.d.ts"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -439,7 +439,8 @@
|
|||
"!dist/plugin-sdk/realtime-voice-playback.d.ts",
|
||||
"!dist/plugin-sdk/text-grapheme.d.ts",
|
||||
"!dist/plugin-sdk/agent-harness-session-runtime.d.ts",
|
||||
"!dist/plugin-sdk/agent-harness-attempt-runtime.d.ts"
|
||||
"!dist/plugin-sdk/agent-harness-attempt-runtime.d.ts",
|
||||
"!dist/plugin-sdk/secret-egress-runtime.d.ts"
|
||||
],
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
|
|
@ -847,6 +848,9 @@
|
|||
"types": "./dist/plugin-sdk/secret-ref-runtime.d.ts",
|
||||
"default": "./dist/plugin-sdk/secret-ref-runtime.js"
|
||||
},
|
||||
"./plugin-sdk/secret-egress-runtime": {
|
||||
"default": "./dist/plugin-sdk/secret-egress-runtime.js"
|
||||
},
|
||||
"./plugin-sdk/secret-file-runtime": {
|
||||
"default": "./dist/plugin-sdk/secret-file-runtime.js"
|
||||
},
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@
|
|||
"channel-secret-owner-runtime",
|
||||
"channel-secret-tts-runtime",
|
||||
"secret-ref-runtime",
|
||||
"secret-egress-runtime",
|
||||
"secret-file-runtime",
|
||||
"security-runtime",
|
||||
"gateway-config-runtime",
|
||||
|
|
|
|||
|
|
@ -168,6 +168,7 @@
|
|||
"runtime-doctor-migrations",
|
||||
"runtime-fetch",
|
||||
"sandbox",
|
||||
"secret-egress-runtime",
|
||||
"secret-file-runtime",
|
||||
"secret-provider-alias",
|
||||
"secure-random-runtime",
|
||||
|
|
|
|||
|
|
@ -1,64 +0,0 @@
|
|||
// Fleet container stream redaction preserves chunked output without leaking split secrets.
|
||||
import { StringDecoder } from "node:string_decoder";
|
||||
|
||||
// Longest suffix of `text` that is a proper prefix of any secret. Retaining it
|
||||
// across emissions guarantees no complete secret is ever split between two
|
||||
// emitted chunks, and emitted text can never grow into a match later.
|
||||
function secretPrefixSuffixLength(text: string, redactValues: readonly string[]): number {
|
||||
let longest = 0;
|
||||
for (const value of redactValues) {
|
||||
const max = Math.min(value.length - 1, text.length);
|
||||
for (let length = max; length > longest; length -= 1) {
|
||||
if (value.startsWith(text.slice(text.length - length))) {
|
||||
longest = length;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
return longest;
|
||||
}
|
||||
|
||||
export function createRedactingStreamWriter(
|
||||
target: NodeJS.WriteStream,
|
||||
redactValues: readonly string[],
|
||||
): { write: (chunk: Buffer) => boolean; flush: () => void } {
|
||||
const decoder = new StringDecoder("utf8");
|
||||
let pending = "";
|
||||
const redact = (text: string): string => {
|
||||
let redacted = text;
|
||||
for (const value of redactValues) {
|
||||
if (value) {
|
||||
redacted = redacted.replaceAll(value, "<redacted>");
|
||||
}
|
||||
}
|
||||
return redacted;
|
||||
};
|
||||
// Returns the raw target.write() backpressure signal so callers can pause
|
||||
// the child stream instead of buffering a noisy follow stream without bound.
|
||||
const emit = (text: string): boolean => {
|
||||
if (!text) {
|
||||
return true;
|
||||
}
|
||||
return target.write(redact(text));
|
||||
};
|
||||
return {
|
||||
// Emit everything except a possible secret prefix at the tail on every
|
||||
// chunk, so unterminated output (progress lines, prompts) streams live
|
||||
// instead of stalling until a newline arrives.
|
||||
write: (chunk) => {
|
||||
pending += decoder.write(chunk);
|
||||
const keep = secretPrefixSuffixLength(pending, redactValues);
|
||||
const cut = pending.length - keep;
|
||||
if (cut <= 0) {
|
||||
return true;
|
||||
}
|
||||
const writable = emit(pending.slice(0, cut));
|
||||
pending = pending.slice(cut);
|
||||
return writable;
|
||||
},
|
||||
flush: () => {
|
||||
emit(pending + decoder.end());
|
||||
pending = "";
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -29,7 +29,6 @@ const profileMocks = vi.hoisted(() => ({
|
|||
vi.mock("./cell-profile.js", () => profileMocks);
|
||||
|
||||
import type { CellContainerProfile } from "./cell-profile.js";
|
||||
import { createRedactingStreamWriter } from "./containers.redaction.js";
|
||||
import { createFleetContainerRuntime } from "./containers.runtime.js";
|
||||
|
||||
type FleetContainerCommandExecutor = NonNullable<Parameters<typeof createFleetContainerRuntime>[0]>;
|
||||
|
|
@ -599,32 +598,6 @@ describe("fleet container runtime", () => {
|
|||
expect(executor.mock.calls[0]?.[1].includes("--internal")).toBe(expected);
|
||||
});
|
||||
|
||||
it("redacts secrets from streamed log output, including across chunk boundaries", () => {
|
||||
const written: string[] = [];
|
||||
const target = { write: (text: string) => written.push(text) } as unknown as NodeJS.WriteStream;
|
||||
const writer = createRedactingStreamWriter(target, ["gw-secret-token"]);
|
||||
writer.write(Buffer.from("boot ok\ntoken=gw-sec"));
|
||||
writer.write(Buffer.from("ret-token done\ntail without newline"));
|
||||
writer.flush();
|
||||
const output = written.join("");
|
||||
expect(output).toContain("token=<redacted> done");
|
||||
expect(output).toContain("tail without newline");
|
||||
expect(output).not.toContain("gw-secret-token");
|
||||
});
|
||||
|
||||
it("never splits a secret across a forced long-line flush", () => {
|
||||
const written: string[] = [];
|
||||
const target = { write: (text: string) => written.push(text) } as unknown as NodeJS.WriteStream;
|
||||
const writer = createRedactingStreamWriter(target, ["gw-secret-token"]);
|
||||
// An unterminated line ending exactly in a secret prefix at the flush point.
|
||||
writer.write(Buffer.from(`${"x".repeat(64 * 1024)}gw-sec`));
|
||||
writer.write(Buffer.from("ret-token trailing"));
|
||||
writer.flush();
|
||||
const output = written.join("");
|
||||
expect(output).toContain("<redacted> trailing");
|
||||
expect(output).not.toContain("gw-secret-token");
|
||||
});
|
||||
|
||||
it("parses hardened inspect fields and Docker network internal state", async () => {
|
||||
const executor = vi.fn<FleetContainerCommandExecutor>(async (_runtime, args) =>
|
||||
args[0] === "network"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { spawn } from "node:child_process";
|
||||
import { isRecord, isStringRecord } from "@openclaw/normalization-core/record-coerce";
|
||||
import { withContainerEnvFile } from "../infra/container-env-file.js";
|
||||
import { createRedactingStreamWriter } from "../logging/redacting-stream.js";
|
||||
import { attachChildProcessBridge } from "../process/child-process-bridge.js";
|
||||
import { runCommandWithTimeout } from "../process/exec.js";
|
||||
import {
|
||||
|
|
@ -11,7 +12,6 @@ import {
|
|||
type CellContainerProfile,
|
||||
type FleetContainerRuntimeName,
|
||||
} from "./cell-profile.js";
|
||||
import { createRedactingStreamWriter } from "./containers.redaction.js";
|
||||
|
||||
type FleetContainerCommandOptions = {
|
||||
allowFailure?: boolean;
|
||||
|
|
|
|||
67
src/logging/redacting-stream.test.ts
Normal file
67
src/logging/redacting-stream.test.ts
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
import { describe, expect, it, vi } from "vitest";
|
||||
import { createRedactingStreamWriter } from "./redacting-stream.js";
|
||||
|
||||
describe("createRedactingStreamWriter", () => {
|
||||
it.each([
|
||||
{ values: ["abcabc"], input: "abcabc", expected: "<redacted>" },
|
||||
{ values: ["abcabc"], input: "abcabcabcabc", expected: "<redacted><redacted>" },
|
||||
{ values: ["abc", "abcdef"], input: "abcdef abc!", expected: "<redacted> <redacted>!" },
|
||||
{ values: ["bc", "abcdef"], input: "abcdef abc", expected: "<redacted> a<redacted>" },
|
||||
{ values: ["abcabc", "redact"], input: "abcabc redact", expected: "<redacted> <redacted>" },
|
||||
{ values: ["", "🦞密🦞"], input: "🦞密🦞 🦞!", expected: "<redacted> 🦞!" },
|
||||
{ values: ["gw-secret-token"], input: "partial gw-sec", expected: "partial gw-sec" },
|
||||
{ values: [], input: "progress 🦞", expected: "progress 🦞" },
|
||||
])("redacts $input independently of byte chunk boundaries", ({ values, input, expected }) => {
|
||||
const bytes = Buffer.from(input);
|
||||
const partitions = Array.from({ length: bytes.length + 1 }, (_, index) => [
|
||||
bytes.subarray(0, index),
|
||||
bytes.subarray(index),
|
||||
]);
|
||||
partitions.push(Array.from(bytes, (byte) => Buffer.from([byte])));
|
||||
for (const chunks of partitions) {
|
||||
let output = "";
|
||||
const writer = createRedactingStreamWriter(
|
||||
{
|
||||
write: (text) => {
|
||||
output += text;
|
||||
return true;
|
||||
},
|
||||
},
|
||||
values,
|
||||
);
|
||||
for (const chunk of chunks) {
|
||||
writer.write(chunk);
|
||||
}
|
||||
writer.flush();
|
||||
expect(output).toBe(expected);
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["boot ok\ntoken=", "x".repeat(64 * 1024)])(
|
||||
"streams unterminated progress with bounded carry and forwards backpressure (%#)",
|
||||
(prefix) => {
|
||||
const write = vi
|
||||
.fn<(text: string) => boolean>()
|
||||
.mockReturnValueOnce(false)
|
||||
.mockReturnValue(true);
|
||||
const writer = createRedactingStreamWriter({ write }, ["gw-secret-token"]);
|
||||
expect(writer.write(Buffer.from(`${prefix}gw-sec`))).toBe(false);
|
||||
expect(write.mock.calls).toEqual([[prefix]]);
|
||||
expect(writer.write(Buffer.from("ret-token done\ntail without newline"))).toBe(true);
|
||||
expect(write.mock.calls).toEqual([[prefix], ["<redacted> done\ntail without newline"]]);
|
||||
writer.flush();
|
||||
expect(write).toHaveBeenCalledTimes(2);
|
||||
},
|
||||
);
|
||||
|
||||
it("holds an ambiguous secret prefix until the match is complete", () => {
|
||||
const write = vi.fn<(text: string) => boolean>().mockReturnValue(false);
|
||||
const writer = createRedactingStreamWriter({ write }, ["abc", "abcabc"]);
|
||||
expect(writer.write(Buffer.from("abc"))).toBe(true);
|
||||
expect(write).not.toHaveBeenCalled();
|
||||
expect(writer.write(Buffer.from("abc"))).toBe(false);
|
||||
expect(write.mock.calls).toEqual([["<redacted>"]]);
|
||||
writer.flush();
|
||||
expect(write).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
48
src/logging/redacting-stream.ts
Normal file
48
src/logging/redacting-stream.ts
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
import { StringDecoder } from "node:string_decoder";
|
||||
|
||||
export function createRedactingStreamWriter(
|
||||
target: { write(text: string): boolean },
|
||||
redactValues: readonly string[],
|
||||
): { write: (chunk: Buffer) => boolean; flush: () => void } {
|
||||
const decoder = new StringDecoder("utf8");
|
||||
const values = redactValues.filter(Boolean).toSorted((left, right) => right.length - left.length);
|
||||
const firstCharacters = new Set(values.map((value) => value.charAt(0)));
|
||||
let pending = "";
|
||||
const emit = (ending: boolean): boolean => {
|
||||
let output = "";
|
||||
let cursor = 0;
|
||||
while (cursor < pending.length) {
|
||||
const value = firstCharacters.has(pending.charAt(cursor))
|
||||
? values.find(
|
||||
(candidate) =>
|
||||
pending.startsWith(candidate, cursor) ||
|
||||
(!ending &&
|
||||
pending.length - cursor < candidate.length &&
|
||||
candidate.startsWith(pending.slice(cursor))),
|
||||
)
|
||||
: undefined;
|
||||
if (!value) {
|
||||
output += pending[cursor];
|
||||
cursor += 1;
|
||||
} else if (cursor + value.length > pending.length) {
|
||||
// A longer possible match owns the tail until it completes or the stream ends.
|
||||
break;
|
||||
} else {
|
||||
output += "<redacted>";
|
||||
cursor += value.length;
|
||||
}
|
||||
}
|
||||
pending = pending.slice(cursor);
|
||||
return !output || target.write(output);
|
||||
};
|
||||
return {
|
||||
write: (chunk) => {
|
||||
pending += decoder.write(chunk);
|
||||
return emit(false);
|
||||
},
|
||||
flush: () => {
|
||||
pending += decoder.end();
|
||||
emit(true);
|
||||
},
|
||||
};
|
||||
}
|
||||
17
src/plugin-sdk/secret-egress-runtime.ts
Normal file
17
src/plugin-sdk/secret-egress-runtime.ts
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
import type {
|
||||
ConfiguredModelEgress,
|
||||
ConfiguredModelEgressOptions,
|
||||
} from "../secrets/model-egress.js";
|
||||
import { createLazyRuntimeModule } from "../shared/lazy-runtime.js";
|
||||
|
||||
export type { ConfiguredModelEgress, ConfiguredModelEgressOptions };
|
||||
|
||||
const loadModelEgressRuntime = createLazyRuntimeModule(() => import("../secrets/model-egress.js"));
|
||||
|
||||
/** Private official-plugin runtime for a standalone job's protected model credential. */
|
||||
export async function withConfiguredModelEgress<T>(
|
||||
options: ConfiguredModelEgressOptions,
|
||||
run: (egress: ConfiguredModelEgress) => Promise<T>,
|
||||
): Promise<T> {
|
||||
return (await loadModelEgressRuntime()).withConfiguredModelEgress(options, run);
|
||||
}
|
||||
203
src/secrets/model-egress.test.ts
Normal file
203
src/secrets/model-egress.test.ts
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import { createDeferredCore } from "../shared/deferred.js";
|
||||
import type { SecretEgressProxyHandle } from "./egress-proxy/proxy-server.js";
|
||||
import { withConfiguredModelEgress } from "./model-egress.js";
|
||||
import { looksLikeSecretSentinel, resolveSecretSentinel } from "./sentinel.js";
|
||||
|
||||
const startProxy = vi.hoisted(() =>
|
||||
vi.fn<typeof import("./egress-proxy/proxy-server.js").startSecretEgressProxyServer>(),
|
||||
);
|
||||
vi.mock("./egress-proxy/proxy-server.js", () => ({
|
||||
startSecretEgressProxyServer: startProxy,
|
||||
}));
|
||||
|
||||
const tempDirs = useAutoCleanupTempDirTracker(afterEach);
|
||||
const credential = "synthetic-model-egress-api-key";
|
||||
const publicCa = "synthetic-public-ca-bundle";
|
||||
let config: OpenClawConfig;
|
||||
let proxy: SecretEgressProxyHandle;
|
||||
const revoke = vi.fn();
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.clearAllMocks();
|
||||
vi.stubEnv("OPENAI_BASE_URL", undefined);
|
||||
vi.stubEnv("OPENCLAW_SECRET_SENTINELS", "off");
|
||||
const dir = tempDirs.make("model-egress-test-");
|
||||
const secretPath = path.join(dir, "key.txt");
|
||||
const caPath = path.join(dir, "ca.pem");
|
||||
await fs.writeFile(secretPath, credential, { mode: 0o600 });
|
||||
await fs.writeFile(caPath, publicCa);
|
||||
config = {
|
||||
secrets: {
|
||||
providers: { model_key: { source: "file", path: secretPath, mode: "singleValue" } },
|
||||
},
|
||||
models: {
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: { source: "file", provider: "model_key", id: "value" },
|
||||
baseUrl: "",
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
proxy = {
|
||||
caCertPath: caPath,
|
||||
proxyOrigin: "http://127.0.0.1:12345",
|
||||
getCertificateStatus: () => ({
|
||||
state: "ready",
|
||||
caExpiresAt: "2030-01-01",
|
||||
failedCertificates: 0,
|
||||
}),
|
||||
registerProcess: vi.fn(() => ({
|
||||
env: {
|
||||
HTTPS_PROXY: "http://openclaw:synthetic-proxy-token@127.0.0.1:12345",
|
||||
NODE_EXTRA_CA_CERTS: caPath,
|
||||
},
|
||||
revoke,
|
||||
})),
|
||||
stop: vi.fn(async () => {}),
|
||||
};
|
||||
startProxy.mockResolvedValue(proxy);
|
||||
});
|
||||
|
||||
afterEach(() => vi.unstubAllEnvs());
|
||||
|
||||
function runOptions() {
|
||||
return { config, provider: "openai", model: "gpt-5.5" };
|
||||
}
|
||||
|
||||
describe("configured model egress", () => {
|
||||
it("uses the provider-owned default route and seals a file-backed credential only for its host", async () => {
|
||||
// Authored provider config can omit its default transport, even though materialized config has one.
|
||||
Reflect.deleteProperty(config.models!.providers!.openai!, "baseUrl");
|
||||
const result = await withConfiguredModelEgress(runOptions(), async (egress) => {
|
||||
expect(egress).toMatchObject({
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
model: "gpt-5.5",
|
||||
allowedHosts: ["api.openai.com"],
|
||||
caBundle: publicCa,
|
||||
});
|
||||
expect(looksLikeSecretSentinel(egress.sentinel)).toBe(true);
|
||||
expect(resolveSecretSentinel(egress.sentinel)).toBe(credential);
|
||||
expect(JSON.stringify(egress)).not.toContain(credential);
|
||||
expect(proxy.registerProcess).toHaveBeenCalledWith([
|
||||
{
|
||||
name: "openai model API key",
|
||||
sentinel: egress.sentinel,
|
||||
allowedHosts: ["api.openai.com"],
|
||||
},
|
||||
]);
|
||||
expect(revoke).not.toHaveBeenCalled();
|
||||
return 17;
|
||||
});
|
||||
expect(result).toBe(17);
|
||||
expect(revoke).toHaveBeenCalled();
|
||||
expect(proxy.stop).toHaveBeenCalledOnce();
|
||||
const options = startProxy.mock.calls[0]![0];
|
||||
expect(options.allowedHosts).toEqual(["api.openai.com"]);
|
||||
await expect(fs.stat(options.caDir)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
|
||||
it("binds a configured compatible endpoint instead of the provider default", async () => {
|
||||
config.models!.providers!.openai!.baseUrl = "https://inference.example.test/tenant/v1";
|
||||
await withConfiguredModelEgress(runOptions(), async (egress) => {
|
||||
expect(egress.baseUrl).toBe("https://inference.example.test/tenant/v1");
|
||||
expect(egress.allowedHosts).toEqual(["inference.example.test"]);
|
||||
});
|
||||
});
|
||||
|
||||
it("streams progress before completion and redacts credentials split between chunks", async () => {
|
||||
const stdout: string[] = [];
|
||||
const stderr: string[] = [];
|
||||
await withConfiguredModelEgress(
|
||||
{
|
||||
...runOptions(),
|
||||
onOutput: (text, stream) => (stream === "stdout" ? stdout : stderr).push(text),
|
||||
},
|
||||
async (egress) => {
|
||||
egress.onOutputChunk!(Buffer.from("ready "), "stdout");
|
||||
expect(stdout.join("")).toBe("ready ");
|
||||
egress.onOutputChunk!(Buffer.from(credential.slice(0, 8)), "stdout");
|
||||
egress.onOutputChunk!(Buffer.from(credential.slice(8)), "stdout");
|
||||
egress.onOutputChunk!(Buffer.from("diagnostic"), "stderr");
|
||||
expect(stdout.join("")).toBe("ready <redacted>");
|
||||
expect(stderr.join("")).toBe("diagnostic");
|
||||
},
|
||||
);
|
||||
expect(stdout.join("")).not.toContain(credential);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["literal or profile credential", { apiKey: "profile-id" }],
|
||||
["OAuth authentication", { auth: "oauth" }],
|
||||
["custom headers", { headers: { Authorization: "synthetic-header" } }],
|
||||
["custom request transport", { request: { proxy: { url: "http://proxy.example.test" } } }],
|
||||
["unencrypted endpoint", { baseUrl: "http://api.openai.com/v1" }],
|
||||
["unsupported endpoint port", { baseUrl: "https://inference.example.test:8443/v1" }],
|
||||
["credential-bearing endpoint", { baseUrl: "https://user:password@inference.example.test/v1" }],
|
||||
["subscription route", { api: "openai-chatgpt-responses" }],
|
||||
])("rejects %s before starting a proxy", async (_label, override) => {
|
||||
Object.assign(config.models!.providers!.openai!, override);
|
||||
const run = vi.fn();
|
||||
await expect(withConfiguredModelEgress(runOptions(), run)).rejects.toThrow();
|
||||
expect(startProxy).not.toHaveBeenCalled();
|
||||
expect(run).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("revokes synchronously on cancellation while the remote job is still settling", async () => {
|
||||
const controller = new AbortController();
|
||||
const entered = createDeferredCore();
|
||||
const settle = createDeferredCore();
|
||||
const running = withConfiguredModelEgress(
|
||||
{ ...runOptions(), signal: controller.signal },
|
||||
async () => {
|
||||
entered.resolve();
|
||||
await settle.promise;
|
||||
},
|
||||
);
|
||||
await entered.promise;
|
||||
controller.abort(new Error("job cancelled"));
|
||||
expect(revoke).toHaveBeenCalled();
|
||||
expect(proxy.stop).not.toHaveBeenCalled();
|
||||
settle.resolve();
|
||||
await expect(running).rejects.toThrow("job cancelled");
|
||||
expect(proxy.stop).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("refuses late admission after cancellation during proxy preparation", async () => {
|
||||
const controller = new AbortController();
|
||||
const entered = createDeferredCore();
|
||||
const prepared = createDeferredCore();
|
||||
startProxy.mockImplementationOnce(async () => {
|
||||
entered.resolve();
|
||||
await prepared.promise;
|
||||
return proxy;
|
||||
});
|
||||
const run = vi.fn();
|
||||
const running = withConfiguredModelEgress({ ...runOptions(), signal: controller.signal }, run);
|
||||
await entered.promise;
|
||||
controller.abort(new Error("preparation cancelled"));
|
||||
prepared.resolve();
|
||||
await expect(running).rejects.toThrow("preparation cancelled");
|
||||
expect(proxy.registerProcess).not.toHaveBeenCalled();
|
||||
expect(run).not.toHaveBeenCalled();
|
||||
expect(proxy.stop).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("revokes and removes the private CA directory when the command fails", async () => {
|
||||
await expect(
|
||||
withConfiguredModelEgress(runOptions(), async () => {
|
||||
throw new Error("remote command failed");
|
||||
}),
|
||||
).rejects.toThrow("remote command failed");
|
||||
expect(revoke).toHaveBeenCalled();
|
||||
expect(proxy.stop).toHaveBeenCalledOnce();
|
||||
const options = startProxy.mock.calls[0]![0];
|
||||
await expect(fs.stat(options.caDir)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
});
|
||||
191
src/secrets/model-egress.ts
Normal file
191
src/secrets/model-egress.ts
Normal file
|
|
@ -0,0 +1,191 @@
|
|||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { normalizeProviderId } from "@openclaw/model-catalog-core/provider-id";
|
||||
import { resolveProviderConfigSecretInput } from "../agents/model-auth-provider-config.js";
|
||||
import { findConfiguredProviderModel } from "../config/model-provider-config.js";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import { createRedactingStreamWriter } from "../logging/redacting-stream.js";
|
||||
import { captureSecretRedactionRegistrySnapshot } from "../logging/secret-redaction-registry.js";
|
||||
import {
|
||||
createProviderModelCatalogIdNormalizer,
|
||||
resolveProviderModelRoutes,
|
||||
} from "../plugins/provider-model-routes.js";
|
||||
import { startSecretEgressProxyServer } from "./egress-proxy/proxy-server.js";
|
||||
import { normalizeExactAllowedHost } from "./exact-hostname.js";
|
||||
import { resolveSecretRefString } from "./resolve.js";
|
||||
import { sealSecretSentinel } from "./sentinel.js";
|
||||
|
||||
export type ConfiguredModelEgress = {
|
||||
/** Opaque API-key substitute for the remote application. */
|
||||
sentinel: string;
|
||||
baseUrl: string;
|
||||
model: string;
|
||||
allowedHosts: readonly string[];
|
||||
/** Authenticated loopback proxy environment; keep it on the credential-owning host. */
|
||||
hostEnv: Readonly<Record<string, string>>;
|
||||
/** Public trust bundle contents, suitable for the remote application's CA file. */
|
||||
caBundle: string;
|
||||
onOutputChunk?: (chunk: Buffer, stream: "stdout" | "stderr") => void;
|
||||
};
|
||||
|
||||
export type ConfiguredModelEgressOptions = {
|
||||
config: OpenClawConfig;
|
||||
provider: string;
|
||||
model: string;
|
||||
signal?: AbortSignal;
|
||||
onOutput?: (text: string, stream: "stdout" | "stderr") => void;
|
||||
};
|
||||
|
||||
function hasEntries(value: object | undefined): boolean {
|
||||
return value !== undefined && Object.keys(value).length > 0;
|
||||
}
|
||||
|
||||
function resolveModelEgressSelection(params: ConfiguredModelEgressOptions) {
|
||||
const provider = normalizeProviderId(params.provider);
|
||||
const model = params.model.trim();
|
||||
if (!provider || !model) {
|
||||
throw new Error("Model egress requires an explicit provider and model");
|
||||
}
|
||||
const { providerConfig, ref } = resolveProviderConfigSecretInput(params.config, provider);
|
||||
if (!providerConfig) {
|
||||
throw new Error("Model egress requires a configured provider");
|
||||
}
|
||||
if (!ref || (providerConfig.auth !== undefined && providerConfig.auth !== "api-key")) {
|
||||
throw new Error(
|
||||
"Model egress requires a configured API-key SecretRef; auth profiles and OAuth are unsupported",
|
||||
);
|
||||
}
|
||||
const configuredModel = findConfiguredProviderModel(
|
||||
providerConfig,
|
||||
provider,
|
||||
model,
|
||||
createProviderModelCatalogIdNormalizer(provider),
|
||||
);
|
||||
if (
|
||||
hasEntries(providerConfig.headers) ||
|
||||
hasEntries(configuredModel?.headers) ||
|
||||
hasEntries(providerConfig.request) ||
|
||||
providerConfig.authHeader === false ||
|
||||
providerConfig.localService !== undefined
|
||||
) {
|
||||
throw new Error(
|
||||
"Model egress does not support custom request headers, authentication, proxy, TLS, or local-service configuration",
|
||||
);
|
||||
}
|
||||
const resolution = resolveProviderModelRoutes({
|
||||
provider,
|
||||
modelId: model,
|
||||
config: params.config,
|
||||
});
|
||||
if (resolution?.kind === "incompatible") {
|
||||
throw new Error(resolution.message);
|
||||
}
|
||||
const route = resolution
|
||||
? resolution.kind === "routes"
|
||||
? resolution.routes.find((candidate) => candidate.authRequirement === "api-key")
|
||||
: undefined
|
||||
: {
|
||||
api: configuredModel?.api ?? providerConfig.api,
|
||||
baseUrl: configuredModel?.baseUrl ?? providerConfig.baseUrl,
|
||||
};
|
||||
if (!route?.baseUrl || (route.api !== "openai-responses" && route.api !== "openai-completions")) {
|
||||
throw new Error("Model egress requires an OpenAI-compatible API-key model route");
|
||||
}
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(route.baseUrl);
|
||||
} catch {
|
||||
throw new Error("Model egress requires a valid HTTPS model base URL");
|
||||
}
|
||||
if (
|
||||
url.protocol !== "https:" ||
|
||||
url.port ||
|
||||
url.username ||
|
||||
url.password ||
|
||||
url.search ||
|
||||
url.hash
|
||||
) {
|
||||
throw new Error(
|
||||
"Model egress requires an HTTPS model base URL on port 443 without credentials, query, or fragment",
|
||||
);
|
||||
}
|
||||
return {
|
||||
provider,
|
||||
model,
|
||||
ref,
|
||||
baseUrl: url.href,
|
||||
allowedHosts: [normalizeExactAllowedHost(url.hostname)],
|
||||
};
|
||||
}
|
||||
|
||||
/** Owns one explicit CLI job's protected model credential and its revocable egress path. */
|
||||
export async function withConfiguredModelEgress<T>(
|
||||
params: ConfiguredModelEgressOptions,
|
||||
run: (egress: ConfiguredModelEgress) => Promise<T>,
|
||||
): Promise<T> {
|
||||
params.signal?.throwIfAborted();
|
||||
const selection = resolveModelEgressSelection(params);
|
||||
const apiKey = await resolveSecretRefString(selection.ref, { config: params.config });
|
||||
params.signal?.throwIfAborted();
|
||||
if (Buffer.byteLength(apiKey) > 64 * 1024) {
|
||||
throw new Error("Model egress API key exceeds the protected credential size limit");
|
||||
}
|
||||
const sentinel = sealSecretSentinel(apiKey, { label: `model-egress:${selection.provider}` });
|
||||
await using resources = new AsyncDisposableStack();
|
||||
const caDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-model-egress-"));
|
||||
resources.defer(() => fs.rm(caDir, { recursive: true, force: true }));
|
||||
params.signal?.throwIfAborted();
|
||||
const proxy = await startSecretEgressProxyServer({
|
||||
caDir,
|
||||
allowedHosts: selection.allowedHosts,
|
||||
onAudit: () => {},
|
||||
});
|
||||
resources.defer(() => proxy.stop());
|
||||
params.signal?.throwIfAborted();
|
||||
const grant = proxy.registerProcess([
|
||||
{
|
||||
name: `${selection.provider} model API key`,
|
||||
sentinel,
|
||||
allowedHosts: selection.allowedHosts,
|
||||
},
|
||||
]);
|
||||
resources.defer(() => {
|
||||
grant.revoke();
|
||||
params.signal?.removeEventListener("abort", grant.revoke);
|
||||
});
|
||||
params.signal?.addEventListener("abort", grant.revoke, { once: true });
|
||||
params.signal?.throwIfAborted();
|
||||
const caBundle = await fs.readFile(grant.env.NODE_EXTRA_CA_CERTS!, "utf8");
|
||||
params.signal?.throwIfAborted();
|
||||
const values = [...captureSecretRedactionRegistrySnapshot().values, grant.env.HTTPS_PROXY!];
|
||||
const output = (stream: "stdout" | "stderr") =>
|
||||
createRedactingStreamWriter(
|
||||
{
|
||||
write(text) {
|
||||
params.onOutput?.(text, stream);
|
||||
return true;
|
||||
},
|
||||
},
|
||||
values,
|
||||
);
|
||||
const stdout = output("stdout");
|
||||
const stderr = output("stderr");
|
||||
resources.defer(stdout.flush);
|
||||
resources.defer(stderr.flush);
|
||||
const result = await run({
|
||||
sentinel,
|
||||
baseUrl: selection.baseUrl,
|
||||
model: selection.model,
|
||||
allowedHosts: selection.allowedHosts,
|
||||
hostEnv: grant.env,
|
||||
caBundle,
|
||||
onOutputChunk: params.onOutput
|
||||
? (chunk, stream) => {
|
||||
(stream === "stdout" ? stdout : stderr).write(chunk);
|
||||
}
|
||||
: undefined,
|
||||
});
|
||||
params.signal?.throwIfAborted();
|
||||
return result;
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue