Commit graph

1548 commits

Author SHA1 Message Date
iamtoruk
f92949c081 docs(pricing): bound the codex repricing drift in verify:upgrade, fix stale comments
The COST_CHANGED_BY_DESIGN carve-out in compare.mjs left codex cost entirely
unasserted after #1075/#1078. It now requires the upgraded cost to be strictly
lower than baseline and within 25% of it, and the row verdict says "repriced"
instead of the misleading "identical (cost N% drift)".

grok.ts's comment on the reasoning/output split still claimed provider-side
splitting was the repo's only mechanism; it now also names
billableOutputTokens/REASONING_INCLUDED_IN_OUTPUT (models.ts), which is the
other half since #1078. usage-aggregator.ts's "folds reasoning into output"
comment was true pre-#1078 but is backwards for codex now (reasoning is
already inside output, not added to it).

Test exemplars for the "reasoning is additive" case used hermes, whose
upstream is OpenAI-shaped and may not stay a safe example; swapped to gemini,
which documents "thoughts" as genuinely separate output.

CHANGELOG's #1075 entry gets one line noting days whose codex transcripts
have aged out keep their pre-fix totals via the daily-cache never-lose guard,
matching the disclosure already given for #1040.
2026-08-21 12:49:05 -07:00
iamtoruk
7876c8e9d7 fix(pricing): version the pricing cache and drop codex-credits' dead reasoning param
The pricing cache written to disk had no schema version, so a cache written
by a pre-#1078 binary lacked cacheWriteCostIsExplicit on every entry. Reading
it back resolved the missing key to undefined (falsy), silently reintroducing
the surcharge-fabrication bug #1078 killed for up to CACHE_TTL_MS after an
upgrade. loadCachedPricing now rejects any cache whose version doesn't match
the current schema instead of reading it verbatim.

codexCredits() still accepted an optional reasoningTokens param that added it
to output - the exact double-count #1078 removed from every real caller. The
only caller never passed it; deleted it so it can't be reintroduced by
accident.

parser.ts's activeGeneratedTokens fallback went through billableOutputTokens
in #1078, but codex is the only caller of activeDurationMs/activeGeneratedTokens
and always sets both together, so the fallback branch is unreachable for it.
Reverted to reduce diff noise.
2026-08-21 12:48:55 -07:00
Resham Joshi
91ddf58d15
Merge pull request #1078 from getagentseal/fix/codex-pricing-1075
fix(codex): stop double-billing reasoning output, price cache writes at the explicit rate only
2026-08-21 12:37:08 -07:00
iamtoruk
fda7e8024d fix(codex): stop double-billing reasoning output, price cache writes at the explicit rate only
Reasoning tokens are a subset of output_tokens for OpenAI models, not an
extra bucket: on a 1,396-rollout corpus all 134,316 token_count events
carrying a total satisfy input + output == total. CodeBurn added
reasoning_output_tokens on top when pricing a codex call, in the
cache-rehydration re-price, and in the models/audit display sums. That
overstated codex cost by $166.03 (3.5%) and displayed output tokens by
34.6% on that corpus. Both cost sites and the display sums now go through
one shared billableOutputTokens() so a cold parse and a warm read cannot
drift apart.

cache_write_input_tokens (codex PR #33454) was never read and
cacheCreationInputTokens was hardcoded to 0. It is now carved out of the
uncached-input bucket and clamped to it, but routed to the cache-write
bucket ONLY when the pricing source publishes an explicit cache-write rate.
buildCosts fabricates 1.25x input when a source omits one, which is correct
for Anthropic and would have invented a surcharge OpenAI never charged on
gpt-5.5 / 5.4 / 5.3-codex / gpt-5. ModelCosts now carries
cacheWriteCostIsExplicit so that distinction survives getModelCosts.

A cost change invalidates persisted output: codex-results.json v10 -> v11
(stores costUSD verbatim), the codex parse version moves (the token-bucket
change does not self-heal on read), and the daily cache goes 20 -> 23 (21 is
claimed by the #946 landing branch and 22 by PR #1056). The upgrade-path
corpus asserts codex tokens and calls exactly and reports the repricing.

Closes #1075
2026-08-21 11:50:23 -07:00
Resham Joshi
e12fb39e3f
Merge pull request #1064 from avs-io/fix/test-isolate-provider-homes
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (22) (push) Waiting to run
Tests / test (22.13.0) (push) Waiting to run
test: isolate provider-home env vars so developer shells cannot leak sessions
2026-08-21 06:48:47 -07:00
Aditya Vikram Singh
094f9f1d43 test: consume runtime CLEARED/REDIRECTED arrays in the isolation guard
Extra High MERGE AFTER FIX: source-scraping the setup file treated a
comment containing 'HERMES_HOME' as isolation. The lists now live in a
side-effect-free module that applyIsolation and the declaration test
both import. Comment-only sabotage fails with hermes:HERMES_HOME.
2026-08-21 18:58:21 +05:30
Aditya Vikram Singh
f55f98726d test: isolate provider-home env vars so developer shells cannot leak sessions
HERMES_HOME and eight sibling PROVIDER_ENV_VARS data-dir overrides were
fingerprinted for cache invalidation but never CLEARED by the vitest
setup file, so a Hermes-shell laptop parsed real sessions in fixtures.
A static guard fails closed when the map grows another undeclared home.
2026-08-21 18:45:15 +05:30
Resham Joshi
f0c6e58008
Merge pull request #1049 from avs-io/fix/optimize-provider-remediation-copy
fix(optimize): scope remediation copy to --provider
2026-08-21 05:23:03 -07:00
Resham Joshi
52714d6241
Merge pull request #1063 from getagentseal/snap/scope-personal-files-main
snap: scope personal-files to the log subdirectories each provider actually reads
2026-08-21 03:57:36 -07:00
iamtoruk
c73d8ff6c8 snap: scope the entries main gained after the tightening
The ported declaration was written against main as of c9e6e2ec. Main has since
added dsh and carries five entries that commit never saw, all of them still bare
tool roots, so the port dropped them rather than reintroduce what the store
rejected. Each is restored at the path its provider opens:

  .dsh/sessions                          dsh.ts reads <DSH_HOME>/sessions only
  .kiro/sessions                         CLI store at sessions/cli, v2 IDE store
                                         at sessions/<hash> — siblings
  .quickwork/{profiles.json,sessions,metrics}
                                         profiles.json names the profile bases;
                                         the legacy layout is sessions/sessions.db
                                         plus metrics/
  .config/Claude/local-agent-mode-sessions
                                         Claude Desktop's local-agent-mode store
  .config/Open Design/{runs,data/runs,namespaces}
                                         the three discovery roots open-design.ts
                                         probes under its data dir

.lingtai and .lingtai-tui stay dropped. A LingTai ledger lives at
.lingtai/<agent>/logs/token_ledger.jsonl, and personal-files has no wildcard for
the agent segment; .lingtai-tui only exists to enumerate project homes that could
not be read anyway. Goose is narrowed to .local/share/goose/sessions, which holds
the only file it opens.

Four entries stay whole roots because the provider reads a file sitting directly
in the root: .config/github-copilot (JetBrains stores nest under a variable
<ide>/<kind>/<storeId>) and .local/share/{opencode,crush,kilo}. The new
app/scripts/snap-grants.test.ts asserts every other entry is at least one level
below its tool root, so a bare root cannot come back unnoticed.
2026-08-21 03:42:09 -07:00
iamtoruk
3168699927 snap: scope personal-files to log subdirectories
Snap Store review found the declaration requested each tool's entire root
directory. Those roots hold configuration and, in several cases, credentials,
and personal-files read is recursive, so the request granted read of every AI
tool's credential store.

Every path is now the subdirectory the provider actually opens: .claude/projects,
.codex/sessions, .cline/data, .vibe/logs/session, .hermes/profiles, .mux/sessions
and so on; two are single files (.forge/.forge.db, .zcode/cli/db/db.sqlite). The
editor entries name only the extension folders that hold transcripts rather than
the editor's whole configuration. No bare tool root remains.

One credential file is requested openly instead of implicitly: .claude/.credentials.json,
read-only, used to call Anthropic's usage endpoint for the live plan gauge.

Codex's equivalent needs read-write on the Codex CLI's auth.json to rotate the
token, so neither that file nor a Codex root is declared and the Codex live
gauge is disabled under $SNAP. Codex usage and cost analytics are unaffected;
they come from the session rollouts.

Also declares five providers that were missing entirely and would have shown no
data under the snap: opencode, crush, goose, kilo, kimi-code. Drops .lingtai,
whose per-agent directory sits above the log folder and cannot be expressed
without wildcards.

Ports 4fe760aafe0492a674476c1ca64bf0dbdc673dde onto current main; the entries
main gained since that commit are reconciled separately.
2026-08-21 03:38:25 -07:00
Aditya Vikram Singh
c12c1c6267 fix(optimize): do not claim a session scan for non-Claude providers
Maintainer review on #1049: empty-state copy under --provider codex
named detectors that scanSessions never ran. Say the session-scan
detectors do not cover that provider yet. Drop the dead TUI provider
threading; optimize view is still gated to all|claude.
2026-08-21 15:58:05 +05:30
Resham Joshi
ed02748063
Merge pull request #1055 from avs-io/fix/1045-session-meta-depth
fix(codex): read session_meta string fields at payload depth 1
2026-08-21 02:53:14 -07:00
Resham Joshi
be86a4bf95
Merge pull request #1060 from getagentseal/test/hermetic-deepseek-pricing
test: make pricing and FX hermetic so upstream reprices can't turn CI red
2026-08-21 02:32:24 -07:00
iamtoruk
577ff0b9ee test: skip the live Frankfurter fetch so FX moves can't shift assertions
Mirrors the pricing guard: CODEBURN_FX_NO_FETCH short-circuits getExchangeRate
after the cache read and before the fetch, returning the same USD-equivalent
rate an unreachable network already yields. Tests that seed their own
exchange-rate.json (serve-stdio's EUR case) keep working unchanged.
2026-08-21 02:23:41 -07:00
iamtoruk
4866332b91 test: price off the bundled snapshot so upstream reprices can't turn tests red
loadPricing() fetched the live LiteLLM table during tests and live data wins
over the bundled snapshot, so DeepSeek v4 assertions went red when upstream
dropped the off-peak discount. CODEBURN_PRICING_SNAPSHOT_ONLY skips the fetch;
env-isolation sets it for the whole suite.
2026-08-21 02:21:31 -07:00
Aditya Vikram Singh
497f655623 fix(codex): read session_meta string fields at payload depth 1
#1040 fixed nested provenance.model. The compact Buffer path still
took the first cwd/session_id/originator/name anywhere in the payload.
Same depth-1 window for every session_meta string field. Bump parse
fingerprint and CODEX_CACHE_VERSION so present sources re-parse.
2026-08-20 08:52:30 +05:30
Resham Joshi
a48d78f9d2
Merge pull request #1040 from timdp/codex-model-attribution-fix
Some checks failed
CI / semgrep (push) Has been cancelled
Tests / test (22) (push) Has been cancelled
macOS Menubar CI / test (push) Has been cancelled
Tests / test (22.13.0) (push) Has been cancelled
Fix Codex model attribution after session metadata
2026-08-19 13:49:52 -07:00
iamtoruk
134e662013 Merge remote-tracking branch 'origin/main' into pr1040-rebase 2026-08-19 13:46:28 -07:00
Resham Joshi
c42de88f74
Merge pull request #1052 from getagentseal/ci/lock-suite-step-timeout
ci: give the quarantined cache-lock step its own timeout
2026-08-19 13:46:25 -07:00
iamtoruk
5130041839 ci: give the quarantined lock suite its own timeout
The step is continue-on-error, but when it stalls it runs into the job's
15-minute budget and the whole job is reported cancelled, hiding the
parallel suite's result (#1040 three times in a row).
2026-08-19 13:35:35 -07:00
iamtoruk
f9f79b0660 Merge remote-tracking branch 'origin/main' into pr1040-rebase
# Conflicts:
#	CHANGELOG.md
2026-08-19 13:18:09 -07:00
Resham Joshi
7862aabd47
Merge pull request #1039 from avs-io/fix/hermes-glm53-pr-links
fix(hermes): unwrap known routers; fail closed on unknown models
2026-08-19 12:29:21 -07:00
Resham Joshi
496b3f258e
Merge pull request #1041 from avs-io/fix/plan-budget-not-live-quota
fix(plans): call sticker-price bars a budget, not live quota
2026-08-19 12:27:34 -07:00
iamtoruk
fec3519ff1 Merge remote-tracking branch 'origin/main' into pr1041-rebase
# Conflicts:
#	CHANGELOG.md
2026-08-19 12:08:25 -07:00
iamtoruk
9ae4250ef3 changelog: resolve merge markers 2026-08-19 12:08:22 -07:00
iamtoruk
8d93a6b40a Merge remote-tracking branch 'origin/main' into pr1040-rebase
# Conflicts:
#	CHANGELOG.md
2026-08-19 12:07:57 -07:00
Aditya Vikram Singh
8b79927d29 fix(optimize): preserve Claude copy and use provider display names
Extra High on #1049: keep the shipped Claude Code empty-state and
TUI header strings; resolve agent nouns from Provider.displayName;
cover all five cross-provider detector labels.
2026-08-20 00:37:41 +05:30
iamtoruk
02059c961c Merge remote-tracking branch 'origin/main' into pr1039-rebase
# Conflicts:
#	src/models.ts
2026-08-19 12:07:02 -07:00
Resham Joshi
96fc940798
Merge pull request #1037 from avs-io/fix/menubar-keychain-credential-cache
fix(menubar): migrate Claude/Codex caches to namespaced Keychain
2026-08-19 12:05:47 -07:00
Resham Joshi
fe94377a13
Merge pull request #1046 from getagentseal/ci/mac-menubar-swift-tests
ci: run the macOS menubar Swift tests on every mac/ change
2026-08-19 12:05:34 -07:00
Resham Joshi
cab937c2c7
Merge pull request #1038 from avs-io/fix/tui-resize-coalesce
fix(tui): coalesce resize bursts without dropping updates
2026-08-19 12:05:03 -07:00
Resham Joshi
0290c3ee49
Merge pull request #1048 from getagentseal/fix/mimo-alias-landing
fix(models): alias bare MiMo 2.5 ids to the LiteLLM Xiaomi rows (lands #1042 + review fixes)
2026-08-19 12:04:40 -07:00
iamtoruk
dba9a45825 test(mac): stop the concurrent-timeout test starving the pool it waits on
The test blocked on a DispatchSemaphore with a 15s deadline, commented as
running "on a real thread, not the cooperative pool". Swift Testing invokes
synchronous test bodies from a task on the cooperative pool, so the wait
parked one of activeProcessorCount workers on the very task group it was
waiting for. With 16 cores locally there is slack; on the 3-core macos-latest
runner, alongside the rest of the parallel suite, the group made no progress
at all and the wait expired.

Await the group directly instead, which also lets the compiler reject the
blocking wait (unavailable from async contexts), and bound the test with
.timeLimit rather than a hand-rolled wall clock. Assert each child came back
with a signal status, so the test now proves the timeout killed every hung
process instead of only that the group returned.

Reproduced by parking all but 3 cooperative threads for the run: 3/3 failures
at 15.0s before, 3/3 passes after. 10x full suite under CPU load: 160/160.
2026-08-19 11:58:39 -07:00
Aditya Vikram Singh
fb477f9916 fix(optimize): scope remediation copy to --provider
Cross-provider optimize findings and destination headers were
Claude-hardcoded after #1002 already scoped the detectors.
2026-08-20 00:20:27 +05:30
iamtoruk
0dff0b66d5 docs(menubar): state what the Keychain move actually guarantees
The Codex settings copy implied the cached credential was app-private. It is
a normal login-Keychain item: reachable by programs running as you, with no
per-app ACL. The real win is that it is no longer a world-readable 0644 file,
so say that instead.

Also documents why readAfterSecuringPermissions repairs permissions before
validating content (validating first would read the secret while it is still
world-readable, which is the window the function exists to close), and why the
Keychain service names are deliberately not derived from CFBundleIdentifier
(the Electron app hardcodes the same strings).

Adds the #1037 changelog entry.
2026-08-19 11:43:44 -07:00
iamtoruk
0f7bfb3eb2 fix(menubar): make a failed disconnect leave a consistent state
disconnect() cleared the usage block before anyone knew whether the delete
had worked, and AppStore then returned early on failure — so a failed
disconnect cleared some state, left the rest, and still posted
subscriptionDisconnected. It also carried a second !isSuccess branch that the
early return had already made unreachable.

Both services now return the delete result and only clear the usage block on
success, so a failure changes nothing at all: the provider stays connected,
Disconnect stays available, and the banner asks for a retry. That matches the
success path's ordering instead of half-applying it.

Errors reaching the generic catches now render localizedDescription rather
than String(describing:), so a Keychain failure shows its message instead of
an enum dump with the raw item name in it.
2026-08-19 11:43:44 -07:00
iamtoruk
0fd8419bfd fix(menubar): keep the newer credential copy and drop the sticky retry flag
Three fixes in the store read path.

A locked keychain no longer reads as a disconnect. currentRecord() treated
any failure from readOurCache() as fatal, and a nil as "the item vanished",
which cleared isBootstrapCompleted. .unavailable now falls back to the last
known record and leaves the flag set.

Recency. A Keychain hit always won and the legacy file was then unlinked,
even when the file was newer. This service name has been in use since May
2026, so an upgrading install can hold a months-old item beside a file the
pre-migration build wrote today; the older token won and the newer copy was
deleted. Both stores now compare first (expiresAt for Claude, lastRefresh for
Codex) and adopt the later one before anything is removed. Codex matters most
here: serving a spent rotating refresh token ends in a terminal invalid_grant.

lastLegacyCleanupFailed is gone. It was set on every cleanup path and read
only by tests, never surfaced. The retry it was meant to signal already
happens, because the unlink is attempted on every successful read.

Also serializes migrate + unlink under the existing SafeFile.withExclusiveLock
so two menubar instances cannot race on the same legacy file, and drops a
leftover no-op local.
2026-08-19 11:43:32 -07:00
iamtoruk
e213e192b4 fix(menubar): never let a Keychain read raise UI on the refresh timer
Cache reads run on the background quota timer, so they must not be able to
put a panel on screen. Measured on macOS 15 against a throwaway keychain:
with the keychain locked, SecItemCopyMatching blocks on an unlock panel even
when the query carries kSecUseAuthenticationUI: ...Fail or a non-interactive
LAContext. Both of those govern the data-protection keychain; unlocking a
file-based keychain is something securityd drives itself. The only reliable
suppression is not issuing the read, so check lock state first and report
.unavailable instead.

.unavailable is separate from readFailed on purpose: a locked keychain means
"cannot look right now", not "the item is gone", and callers must not turn it
into a disconnect. It also carries a readable errorDescription so a -25308
reaching the UI reads as "Keychain unavailable" rather than a struct dump.

SecKeychainGetStatus is soft-deprecated with no replacement that reports
file-keychain lock state; annotating the warning away only moves it to the
call site, so it is left visible with a comment.

Adds the first test that touches a real Keychain, against a throwaway service
name no build reads, skipped when the host has no usable Keychain.
2026-08-19 11:43:22 -07:00
iamtoruk
bb5b71dff1 feat(models): name the MiMo v2 Flash row
It was the only MiMo row still rendering as its raw slug next to
"MiMo v2.5" and "MiMo v2.5 Pro". SORTED_SHORT_NAMES is longest-first, so
the two v2.5 entries keep their own labels.
2026-08-19 11:33:13 -07:00
iamtoruk
6558a4feee docs(changelog): note the plan budget copy fix
User-visible on two surfaces and the branch had no entry.
2026-08-19 11:31:47 -07:00
iamtoruk
9752f02636 fix(hermes): memoize the origin lookup; cover PR links end to end
githubOwnerRepoFromRoot re-read .git/config once per session, so every
session in the same repo paid for the same two syscalls. Memoize it per repo
root for the life of the process.

The prLinks plumbing from a provider call through the session cache into the
session summary had no test above the provider boundary; add one that runs
the real parseAllSessions pipeline against a temp HERMES_HOME. It fails
against the pre-change parser.
2026-08-19 11:31:40 -07:00
iamtoruk
e6cac54661 fix(models): derive the vendor-namespace allowlist from the pricing catalog
The hand-written KNOWN_NAMESPACES set dropped pricing for vendor prefixes
LiteLLM itself indexes: x-ai/, nousresearch/, zhipu/, litellm_proxy/ and
openai_like/ all priced on main and went unpriced here. Derive the set from
the loaded pricing keys instead, so a vendor the catalog knows is never lost
to a stale list, and keep only the spellings no catalog carries as explicit
extras: the routing wrappers, the client-side kimi/ and mimo/ prefixes, and
the litellm_proxy/ + openai_like/ routes. Local runners are excluded on
purpose, so an unlisted ollama tag cannot strip down to a priced cloud row.

xiaomi/ stops being a routing wrapper: it is the namespace LiteLLM prices
MiMo under, and BUILTIN_ALIASES maps the bare MiMo ids INTO it, so peeling
pulled against the alias. It stays known via the derived set.

Also: a user price override for a bare id now wins over the catalog row a
routed spelling of it would otherwise hit, and a namespaced GLM-5.3 is no
longer LABELLED GLM-5.2 by the sibling alias it prices through.

Tests assert the allowlist through a price override on a synthetic id, so
they cannot rot with the snapshot; the glm-5.4 assertion that pinned on the
snapshot NOT carrying a model is dropped.
2026-08-19 11:31:34 -07:00
iamtoruk
911bd3f486 fix(plans): say monthly budget, not calendar month, and fit 80 columns
The budget window comes from computePeriodFromResetDay, which builds an
anniversary period from plan.resetDay (1-28, settable per plan with
`codeburn plan set --reset-day`). "Calendar-month budget" and "Next
calendar reset" are therefore wrong for anyone who moved the reset day,
which is the same class of inaccuracy this change set exists to remove.
Say "budget" and "Next budget reset" instead, and use one wording across
the TUI and the desktop cards.

Both TUI lines truncate end-first at the terminal width. The headline had
grown past the point where an 80-column terminal still showed the
percentage, so it drops "vs ... /mo" for "/ $300.00 budget", and the
status line drops the clause repeating "budget" from the headline. At 80
columns the longest label (custom plans carry their provider) now fits
the percentage, and the status line still shows the projection.
2026-08-19 11:30:07 -07:00
iamtoruk
6bd86dbc36 ci: build, test and package the macOS menubar on every mac/ change
release-menubar.yml only packages; the 170+ Swift tests gated nothing.
2026-08-19 11:29:14 -07:00
iamtoruk
52ceceb211 cache: re-derive the daily rollup for the codex session_meta model fix
The codex parse version and CODEX_CACHE_VERSION bumps in #1040 make codex
sessions re-parse, but the daily cache has no per-provider invalidation, so
every day already finalized keeps its old per-model rows - and usage-aggregator
serves every day before today from that cache, with ten-year retention. Raise
DAILY_CACHE_VERSION and MIN_SUPPORTED_VERSION to 20 so history re-derives once
off the warm session cache.

The re-derivation test now seeds v19, the last shipped version, so it models the
real 19 -> 20 path, and the upgrade-path check expects daily-cache.v20.json.

Measured on a real 110-day cache: no day lost value, none disappeared, 100 came
back identical, and 9 grok days rose by $19.80 in total from rollups an earlier
parse change had left stale. Every codex model row was unchanged - that corpus
predates the provenance field the fix corrects.
2026-08-19 11:28:40 -07:00
iamtoruk
ef040a4c4d test(models): pin the shipped MiMo v2 Flash crash; name the base 2.5 row
The `mimo-v2-flash -> xiaomi/mimo-v2-flash` alias shipped before this
branch and already cycled through display-name resolution, so
getShortModelName threw RangeError on every real MiMo v2 Flash session.
The new cycle-safe resolver fixes it, but nothing pinned the ids that
actually crashed in production: cover the four spellings found in a real
session cache, including the unnamespaced `mimo/mimo-v2-flash`.

Add the base `mimo-v2.5` display name so the row reads next to
"MiMo v2.5 Pro" instead of showing a raw slug; SORTED_SHORT_NAMES is
longest-first, so the Pro tier still wins its own entry.
2026-08-19 11:26:22 -07:00
iamtoruk
a4384d28f2 Merge remote-tracking branch 'origin/main' into pr1041-rebase 2026-08-19 11:19:11 -07:00
iamtoruk
6f1098d215 Merge remote-tracking branch 'origin/main' into pr1042-rebase 2026-08-19 11:14:22 -07:00
iamtoruk
952a301877 Merge remote-tracking branch 'origin/main' into pr1040-rebase 2026-08-19 11:13:42 -07:00