OmniRoute/scripts/dev/sync-env.mjs
Diego Rodrigues de Sa e Souza f5073604b3
Release/v3.8.0 (#2430)
* fix(cli-tools): guard modelId type before calling indexOf

E2E shakedown v3.8.0: cli-tools quebrava com TypeError quando dynamicModels
continha entradas sem .id (objeto retornado diretamente em vez de string).

* fix(offline): avoid SSR/CSR hydration mismatch on navigator.onLine

Replace useState+lazy-initializer with useSyncExternalStore so the server
snapshot (() => false) and client snapshot (() => navigator.onLine) are
declared separately. React hydrates with the server value and switches to
the real online status client-side without a mismatch.

* chore(i18n): add missing en.json keys for translator, cli-tools, memory, onboarding

Adds 58 missing keys identified by the new dashboard audit script:
- cliTools: 18 custom CLI builder keys (CustomCliCard)
- translator: 24 keys covering stream transformer, live monitor, test bench
- memory: 12 health/pagination/dialog keys
- onboarding.tier: 8 keys for the tier tour walkthrough

Also adds scripts/i18n/audit-dashboard-pages.mjs which scans all dashboard
pages, reports t() calls referencing missing en.json keys, and flags
candidate hardcoded JSX/attribute strings.

* chore(i18n): replace hardcoded UI text with t() calls across dashboard (round 1)

Subagents refactored 8 high-impact dashboard pages, replacing 81 of the
407 hardcoded English/PT strings flagged by the audit with proper
useTranslations() lookups. Added 73 corresponding keys to en.json across
the home, apiManager, providers, settings, and usage namespaces.

Pages affected:
- BudgetTab (27 → 0)
- HomePageClient (2 → 0)
- RoutingTab (25 → 7)
- ResilienceTab (38 → 18)
- SystemStorageTab (42 → 21)
- providers/[id] (17 → 15)
- ApiManagerPageClient (14 → 13)
- OneproxyTab (13 → 10)

Also adds two helper scripts:
- scripts/i18n/extract-keys-from-diff.mjs — extracts new keys from git diff
- scripts/i18n/merge-keys.mjs — merges a pending-keys JSON into en.json

Remaining hardcoded strings will be addressed in follow-up rounds.

* chore(i18n): replace hardcoded UI text with t() calls across dashboard (round 2)

Continues round 1 (commit 8d34f4c65). Round-2 subagents refactored
additional dashboard pages, replacing 77 more hardcoded strings with
useTranslations() lookups. Added 79 corresponding keys to en.json
across the a2aDashboard, agents, analytics, apiManager, cliTools,
common, and settings namespaces.

Pages affected:
- a2a/page (new useTranslations + 6 keys)
- agent-skills/page (new useTranslations + 9 keys)
- AutoRoutingAnalyticsTab (new useTranslations + 6 keys)
- AppearanceTab (8 → 6 remaining)
- OneproxyTab (10 → 0)
- ResilienceTab (18 → 0 missing key)
- RoutingTab (7 → 0 missing key)
- VisionBridgeSettingsTab (new useTranslations + 6 keys)
- CopilotToolCard (7 → 0 missing key)
- ApiManagerPageClient (13 → 0 missing key)
- gamification/admin (new useTranslations + 7 keys)

Hardcoded total: 326 → 249. Real missing keys: 0 (the 6 still flagged
are false positives in exampleTemplates.tsx where t is passed as a
parameter — keys exist at translator.templatePayloads.*).

* chore(i18n): replace hardcoded UI text with t() calls across dashboard (round 3)

Round-3 subagents and manual edits refactored 9 more dashboard pages
(plus 2 small extras), replacing ~80 hardcoded strings with
useTranslations() lookups. Added 79 corresponding keys to en.json
across analytics, cloudAgents, combos, common, health, settings, and
usage namespaces.

Pages affected:
- analytics/ComboHealthTab (new useTranslations + 15 keys)
- analytics/CompressionAnalyticsTab (new useTranslations + 11 keys)
- settings/SystemStorageTab (21 → 0 missing key)
- tokens/page (new useTranslations + 13 keys)
- usage/BudgetTab (9 missing fixed)
- health/page (manual: 6 keys)
- cloud-agents/page (manual: 3 keys)
- combos/page (manual: 1 key)

Hardcoded total: 249 → 164. Real missing keys: 0 (6 remaining are
exampleTemplates.tsx false positives).

Also adds scripts/i18n/build-pending-from-missing.mjs which reads
_audit.json and locates English values from HEAD to rebuild
_pending-keys.json after race-condition resets between subagent edits.

* chore(i18n): localize remaining dashboard settings labels

Replace hardcoded labels in compression and resilience settings with
translation lookups to continue the dashboard i18n cleanup.

Add the v3.8.0 dashboard shakedown runbook to document the manual
smoke-test process and known dev environment pitfalls.

* chore(i18n): replace hardcoded UI text with t() calls across dashboard (round 4)

Round-4 subagent + manual key-resolution refactored remaining strings in
3 high-traffic settings/API tabs, plus extracted English values for
keys that were already added as t() calls but lost during the previous
en.json race-condition resets.

Pages affected:
- api-manager/ApiManagerPageClient (7 → 0 missing key)
- settings/CompressionSettingsTab (8 → 0 missing key)
- settings/MemorySkillsTab (8 → 0 missing key)
- settings/ResilienceTab (4 more keys recovered)

Hardcoded total: 164 → 140. Real missing keys: 0 (6 remaining are the
exampleTemplates.tsx false positives — t passed as parameter).

* chore(i18n): replace hardcoded UI text with t() calls across dashboard (round 5)

Round-5 agent began processing the remaining smaller dashboard files.
Added 5 more keys to en.json for providers/[id]/page.tsx OAuth flow
labels and the cross-OS auto-detection hint.

Pages affected:
- providers/[id]/page.tsx (5 keys)

Hardcoded total: 140 → 136. Real missing keys: 0.

* chore(i18n): resolve last 2 missing providers/[id] keys

Adds providerDetailMyClaudeAccountPlaceholder and
providerDetailPathAutoDetected — the final user-visible labels in the
providers/[id] page that the round-5 subagent rewrote to t() calls
without yet adding to en.json.

Real missing keys: 0 (6 remaining are exampleTemplates.tsx false
positives — t is passed as a parameter so the audit cannot resolve the
namespace; keys do exist at translator.templatePayloads.*).

* chore(i18n): replace hardcoded UI text with t() calls across dashboard (round 6 — 10 parallel agents)

Round-6 dispatched 10 parallel subagents covering all 57 remaining
dashboard files. Each agent worked on a disjoint file set to avoid
en.json race conditions. Added ~60 new i18n keys across 9 namespaces
covering small UI labels, table headers, search placeholders, and
empty-state messages.

Major changes:
- analytics: SearchAnalyticsTab, ProviderUtilizationTab, DiversityScoreCard, CompressionAnalyticsTab (new useTranslations + keys)
- batch: BatchDetailModal, BatchListTab, FileDetailModal, FilesListTab (new useTranslations + keys)
- settings: CliproxyapiSettingsTab, PayloadRulesTab, ModelCooldownsCard, AppearanceTab, PricingTab (mostly new useTranslations)
- endpoint: TokenSaverCard, ApiEndpointsTab, EndpointPageClient
- cache: CachePerformance, IdempotencyLayer, ReasoningCacheTab, MediaPageClient, page
- combos: IntelligentComboPanel, page
- playground: ChatPlayground, SearchPlayground
- providers: ProviderCard
- onboarding: TierFlowDiagram
- changelog: ChangelogViewer
- home: ProviderTopology, TierCoverageWidget, BootstrapBanner, BadgeToast
- usage: BudgetTab, BudgetTelemetryCards, QuotaTable
- quotaShare: QuotaSharePageClient
- profile: page
- leaderboard: page
- skills: page

Hardcoded total: 131 → 60. Real missing keys: 0 plus 1 false-positive
for combos.modePack (lookup via prop-passed t).

* chore(i18n): finalize round-6 keys for batch/cache/endpoint/usage

Adds the remaining keys produced by parallel agents A4, A6, A8, A9:
- common: batch-related labels (BatchDetailModal, BatchListTab,
  FileDetailModal, FilesListTab, page) + profile/leaderboard
- cache: hit rate, latency, retry, avg chars
- endpoint: token saver, API endpoints, copy URL, cloud/local labels
- usage: noSpend, activeSessions, quotaAlerts, budget timing
- skills: install/marketplace/filter
- proxyRegistry/quotaShare/mcpDashboard: misc labels

Hardcoded total: 60 → 48. Real missing keys: 0 (modePack remaining is a
false positive — combos.modePack exists but the audit can't resolve it
since IntelligentComboPanel receives t as a prop).

* fix(playground): dedupe filteredModels to avoid duplicate React key warning

The /v1/models endpoint can return the same model id twice (e.g., when a
model is listed by both an alias and its canonical provider), which made
the <Select> emit two <option> elements with the same key — triggering
"Encountered two children with the same key, codex/gpt-5.5".

Replace the chained filter + map with a single pass that skips ids
already added.

* fix(playground): guard against non-string model ids before .split/.startsWith

The /v1/models endpoint can include synthetic entries (combos, locals,
in-progress imports) with a null/undefined id. The playground used to
call m.id.split("/") in the provider-discovery loop, which threw on the
first non-string entry; the surrounding .catch(() => {}) silently
swallowed the error, so the provider/model/account dropdowns ended up
empty even though /v1/models returned thousands of valid entries.

- Skip entries without a string id before split/startsWith.
- Log the rejection in the .catch handler so future regressions are
  visible in DevTools instead of silently emptying the UI.

* fix(playground): guard ChatPlayground filteredModels for non-string ids

Same root cause as commit 49fe356b9: ChatPlayground filtered models
with m.id.startsWith(...) which crashed on null/undefined ids returned
by /v1/models (synthetic combo entries). Apply the same defensive guard
and dedupe used in the parent page.

* fix(claude): drop orphan tool_result after fixToolAdjacency strip (discussion #2410)

Discussion #2410 reports Claude returning 400 for sequences like:
  assistant: tool_use(id=X)
  user: <plain text>           ← breaks adjacency
  user: tool_result(id=X)

The previous round added `fixToolAdjacency` (commit 44d9abac9) which
correctly strips the orphan tool_use from the assistant message. But
that left the now-unmatched tool_result intact, so the upstream
rejected the request with:

  messages.N.content.M: unexpected `tool_use_id` found in `tool_result`
  blocks: X. Each tool_result block must have a corresponding tool_use
  block in the previous message.

Fix: after running `fixToolAdjacency`, re-run `fixToolPairs` to drop
the orphaned tool_result blocks. All three call sites updated:
  - contextManager.purifyHistory (both inside the binary-search loop
    and the final pass)
  - BaseExecutor message-prep (Claude path)
  - claudeCodeCompatible request signer

Also tightens an unrelated dynamic-key access in
readNestedString (claudeCodeCompatible) to satisfy the prototype-
pollution scanner triggered by the post-tool semgrep hook.

* fix(mitm): point runtime manager re-export to js entrypoint

Use the emitted `.js` path for the runtime manager re-export so dynamic
runtime loading resolves correctly outside the Turbopack alias handling.

* docs: add AgentRouter setup guide (#2422)

Integrated into release/v3.8.0 — AgentRouter setup guide docs.

* feat: add new feature on combos - falloverBeforeRetry (#2417)

Integrated into release/v3.8.0 — falloverBeforeRetry for per-model quota skipping in combos.

* feat(batch): implement 10 feature requests harvested  (#2414)

Integrated into release/v3.8.0 — batch of 10 feature requests: llama.cpp local provider, upstream error exposure, Termux detection, providers rotate CLI, t3.chat web skeleton, Zed Docker integration, Kiro multi-account OAuth isolation, auto-combo cost blending, auto-combo context filter, combo provider-level exhaustion tracking (#1731). Conflicts with #2417 (falloverBeforeRetry) resolved.

* fix(gamification): resolve SQL bug, auth gap, pagination, and anomaly scoring (#2421)

Integrated into release/v3.8.0 — 6 critical gamification bug fixes: SQL SELECT in checkActionCountBadges, federation auth enforcement, leaderboard pagination offset, real z-score computation, addXp level calculation, and barrel index.ts

* docs(changelog): add post-release entries for #2414 #2417 #2421 #2422

- feat(batch): T3-Chat-Web executor, exhaustedProviders set (#1731), Zed Docker
- feat(combos): falloverBeforeRetry + setTry loop (#2417 — @hartmark)
- fix(gamification): SQL SELECT bug, federation auth, pagination, z-score (#2421 — @oyi77)
- docs: AgentRouter setup guide (#2422 — @leninejunior)

* fix(security): resolve CodeQL random/password-hash alerts and sync docs & tests

* feat/fix: integrate PRs #2423, #2425, #2427, #2428 with test & security fixes

* docs(changelog): credit contributors for PRs #2423, #2425, #2427, #2428

* fix(mitm): drop .js extension on manager.runtime re-export for webpack build (#2425)

Merged into release/v3.8.0

* fix: persist STORAGE_ENCRYPTION_KEY across upgrades (closes #1622) (#2428)

Merged into release/v3.8.0

* fix: auto-reset apiKeyHealth on successful connection test (#2427)

Merged into release/v3.8.0

* fix: support Antigravity image generation, Add Gemini 3.5 Flash (#2423)

Merged into release/v3.8.0

---------

Co-authored-by: diegosouzapw <diego.souza.pw@gmail.com>
Co-authored-by: Lenine Júnior <lenine@engrene.com.br>
Co-authored-by: Markus Hartung <mail@hartmark.se>
Co-authored-by: Paijo <14921983+oyi77@users.noreply.github.com>
Co-authored-by: Anton <39598727+NomenAK@users.noreply.github.com>
Co-authored-by: Chewji <126886556+Chewji9875@users.noreply.github.com>
Co-authored-by: clousky2020 <33016567+clousky2020@users.noreply.github.com>
Co-authored-by: backryun <bakryun0718@proton.me>
2026-05-20 09:12:49 -03:00

341 lines
11 KiB
JavaScript

#!/usr/bin/env node
/**
* OmniRoute — Environment Sync
*
* Ensures .env exists and contains the selected keys from .env.example.
* Runs on installs and can be executed manually via `npm run env:sync`.
*
* Rules:
* - Never overwrites existing values in .env
* - Auto-generates cryptographic secrets if blank in .env.example
* - Copies default values from .env.example for new keys
* - Skips commented lines from .env.example
*/
import { copyFileSync, existsSync, readFileSync, writeFileSync } from "node:fs";
import { randomBytes } from "node:crypto";
import { createRequire } from "node:module";
import { dirname, join, resolve } from "node:path";
import { homedir } from "node:os";
import { fileURLToPath } from "node:url";
const require = createRequire(import.meta.url);
const CRYPTO_SECRETS = {
JWT_SECRET: () => randomBytes(64).toString("hex"),
API_KEY_SECRET: () => randomBytes(32).toString("hex"),
// STORAGE_ENCRYPTION_KEY: Generated at server startup instead of postinstall.
// Generated in bin/omniroute.mjs:ensureStorageEncryptionKey() and persisted to
// ~/.omniroute/.env to survive across upgrades. This prevents credential loss
// when upgrading OmniRoute (issue #1622).
MACHINE_ID_SALT: () => `omniroute-${randomBytes(8).toString("hex")}`,
};
/**
* Keys that MUST NOT be regenerated when existing encrypted data exists in the DB.
* Generating a new key would make all previously-encrypted credentials unrecoverable.
*
* Note: STORAGE_ENCRYPTION_KEY is no longer auto-generated in postinstall.
* It's generated at server startup in bin/omniroute.mjs and persisted to
* ~/.omniroute/.env to survive across upgrades.
* @see https://github.com/diegosouzapw/OmniRoute/issues/1622
*/
const ENCRYPTION_BOUND_KEYS = new Set([]);
// ── Resolve DATA_DIR (mirrors bootstrap-env.mjs / dataPaths.ts) ─────────────
function resolveDataDir(env = process.env) {
const configured = env.DATA_DIR?.trim();
if (configured) return resolve(configured);
if (process.platform === "win32") {
const appData = env.APPDATA || join(homedir(), "AppData", "Roaming");
return join(appData, "omniroute");
}
const xdg = env.XDG_CONFIG_HOME?.trim();
if (xdg) return join(resolve(xdg), "omniroute");
return join(homedir(), ".omniroute");
}
/**
* Check whether the SQLite database already contains credentials encrypted
* under a previous STORAGE_ENCRYPTION_KEY. If so, generating a new key would
* make them permanently unrecoverable (AES-GCM auth-tag mismatch).
*/
function hasEncryptedCredentials(dataDir) {
const dbPath = join(dataDir, "storage.sqlite");
if (!existsSync(dbPath)) return false;
try {
const Database = require("better-sqlite3");
const db = new Database(dbPath, { readonly: true, fileMustExist: true });
try {
const row = db
.prepare(
`SELECT 1
FROM provider_connections
WHERE access_token LIKE 'enc:v1:%'
OR refresh_token LIKE 'enc:v1:%'
OR api_key LIKE 'enc:v1:%'
OR id_token LIKE 'enc:v1:%'
LIMIT 1`
)
.get();
return !!row;
} finally {
db.close();
}
} catch {
// If we can't open the DB (e.g. missing better-sqlite3 during install),
// err on the side of caution: don't block secret generation.
return false;
}
}
export function parseEnvFile(filePath) {
if (!existsSync(filePath)) return new Map();
const content = readFileSync(filePath, "utf8");
const entries = new Map();
for (const line of content.split(/\r?\n/)) {
const parsed = parseEnvEntry(line);
if (!parsed) continue;
const [key, value] = parsed;
entries.set(key, value);
}
return entries;
}
function parseEnvEntry(line) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) return null;
const eqIndex = trimmed.indexOf("=");
if (eqIndex < 1) return null;
const key = trimmed.slice(0, eqIndex).trim();
const value = unquoteEnvValue(trimmed.slice(eqIndex + 1).trim());
return [key, value];
}
function unquoteEnvValue(value) {
if (value.length < 2) return value;
const quote = value[0];
if ((quote !== '"' && quote !== "'") || value[value.length - 1] !== quote) return value;
return value.slice(1, -1);
}
function parseExampleEntries(content, scope = "full") {
const entries = new Map();
const lines = content.split(/\r?\n/);
if (scope === "oauth") {
let inOauthSection = false;
for (const line of lines) {
const trimmed = line.trim();
if (/OAUTH PROVIDER CREDENTIALS/i.test(trimmed)) {
inOauthSection = true;
continue;
}
if (!inOauthSection) continue;
if (/Provider User-Agent Overrides/i.test(trimmed)) break;
const parsed = parseEnvEntry(line);
if (!parsed) continue;
const [key, value] = parsed;
entries.set(key, value);
}
return entries;
}
for (const line of lines) {
const parsed = parseEnvEntry(line);
if (!parsed) continue;
const [key, value] = parsed;
entries.set(key, value);
}
return entries;
}
export function getEnvSyncPlan({ rootDir, scope = "full" } = {}) {
const root = rootDir || dirname(dirname(fileURLToPath(import.meta.url)));
const envExamplePath = join(root, ".env.example");
const envPath = join(root, ".env");
if (!existsSync(envExamplePath)) {
return {
available: false,
created: false,
added: 0,
missingEntries: [],
};
}
const exampleEntries = parseExampleEntries(readFileSync(envExamplePath, "utf8"), scope);
const currentEntries = parseEnvFile(envPath);
const missingEntries = [];
// Check once whether encrypted data exists — avoids repeated DB opens
let _encryptedDataExists;
function encryptedDataExists() {
if (_encryptedDataExists === undefined) {
try {
_encryptedDataExists = hasEncryptedCredentials(resolveDataDir());
} catch {
_encryptedDataExists = false;
}
}
return _encryptedDataExists;
}
for (const [key, defaultValue] of exampleEntries) {
if (currentEntries.has(key)) continue;
if (CRYPTO_SECRETS[key] && !defaultValue) {
// Guard: never generate a new encryption key if the DB already has
// credentials encrypted under the previous key (#1622)
if (ENCRYPTION_BOUND_KEYS.has(key) && encryptedDataExists()) {
missingEntries.push({
key,
value: "",
generated: false,
blocked: true,
});
continue;
}
missingEntries.push({ key, value: CRYPTO_SECRETS[key](), generated: true });
continue;
}
missingEntries.push({ key, value: defaultValue, generated: false });
}
return {
available: true,
created: !existsSync(envPath),
added: missingEntries.length,
missingEntries,
};
}
function replaceBlankSecret(content, key, value) {
const pattern = new RegExp(`^${key}=\\s*$`, "m");
return pattern.test(content) ? content.replace(pattern, `${key}=${value}`) : content;
}
export function syncEnv({ rootDir, quiet = false, scope = "full" } = {}) {
const log = quiet ? () => {} : (message) => process.stderr.write(`[sync-env] ${message}\n`);
const root = rootDir || dirname(dirname(fileURLToPath(import.meta.url)));
const envExamplePath = join(root, ".env.example");
const envPath = join(root, ".env");
if (!existsSync(envExamplePath)) {
log("⚠️ .env.example not found — skipping sync");
return { created: false, added: 0 };
}
const exampleEntries = parseExampleEntries(readFileSync(envExamplePath, "utf8"), scope);
if (!existsSync(envPath)) {
if (scope === "full") {
copyFileSync(envExamplePath, envPath);
let content = readFileSync(envPath, "utf8");
let generated = 0;
// Check once whether encrypted data exists — avoids repeated DB opens
let dbHasEncrypted;
try {
dbHasEncrypted = hasEncryptedCredentials(resolveDataDir());
} catch {
dbHasEncrypted = false;
}
for (const [key, generator] of Object.entries(CRYPTO_SECRETS)) {
// Guard: never generate a new encryption key if the DB already has
// credentials encrypted under the previous key (#1622)
if (ENCRYPTION_BOUND_KEYS.has(key) && dbHasEncrypted) {
log(
`⚠️ ${key} NOT generated — encrypted credentials exist in DB. ` +
`Restore your previous key via ~/.omniroute/server.env, ~/.omniroute/.env, ` +
`or the STORAGE_ENCRYPTION_KEY environment variable.`
);
continue;
}
const nextContent = replaceBlankSecret(content, key, generator());
if (nextContent !== content) {
content = nextContent;
generated++;
log(`${key} auto-generated`);
}
}
writeFileSync(envPath, content, "utf8");
log(
`✨ Created .env from .env.example (${exampleEntries.size} keys, ${generated} secrets generated)`
);
return { created: true, added: exampleEntries.size };
}
const { missingEntries } = getEnvSyncPlan({ rootDir: root, scope });
const content = [
"# ── Auto-added by sync-env (oauth defaults) ──",
...missingEntries.map((entry) => `${entry.key}=${entry.value}`),
"",
].join("\n");
writeFileSync(envPath, content, "utf8");
log(`✨ Created .env with oauth defaults (${missingEntries.length} keys)`);
return { created: true, added: missingEntries.length };
}
const { missingEntries } = getEnvSyncPlan({ rootDir: root, scope });
if (missingEntries.length === 0) {
log("✅ .env is up to date (0 keys added)");
return { created: false, added: 0 };
}
const appendLines = [
"",
`# ── Auto-added by sync-env (${new Date().toISOString().slice(0, 10)}) ──`,
];
for (const entry of missingEntries) {
if (entry.blocked) {
log(
`⚠️ ${entry.key} NOT generated — encrypted credentials exist in DB. ` +
`Restore your previous key via ~/.omniroute/server.env, ~/.omniroute/.env, ` +
`or the STORAGE_ENCRYPTION_KEY environment variable.`
);
continue;
}
appendLines.push(`${entry.key}=${entry.value}`);
log(
`${entry.generated ? "✨" : "📦"} ${entry.key}${entry.generated ? " (auto-generated)" : ""}`
);
}
appendLines.push("");
const currentContent = readFileSync(envPath, "utf8");
writeFileSync(envPath, `${currentContent.trimEnd()}\n${appendLines.join("\n")}`, "utf8");
log(`📦 Synced .env — added ${missingEntries.length} missing keys`);
return { created: false, added: missingEntries.length };
}
if (process.argv[1]?.endsWith("sync-env.mjs")) {
syncEnv({ scope: process.argv.includes("--oauth-only") ? "oauth" : "full" });
}