Commit graph

13 commits

Author SHA1 Message Date
Resham Joshi
354492e500
Copilot: count every agent round and bill exact credits in VS Code chat sessions (#1752)
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (22) (push) Waiting to run
Tests / test (22.13.0) (push) Waiting to run
Tests / test (24) (push) Waiting to run
Tests / test (26) (push) Waiting to run
Tests / test-platforms (macos-latest) (push) Waiting to run
Tests / test-platforms (windows-latest) (push) Waiting to run
Tests / desktop (push) Waiting to run
Tests / vscode (push) Waiting to run
* Copilot: count every agent round and exact credits in VS Code chat sessions

Copilot Chat 0.69 keeps only the last model call of an agent request in result.metadata. Read the request's modelTotals (input, cache reads, output over every call) first, then root completionTokens, then metadata. Carry copilotCredits as nanoAiu so plan credits are exact. Bump the copilot parse tag so cached chat sessions re-parse.

modelKeyMatches no longer lets claude-sonnet-4 capture claude-sonnet-4-6.

* Copilot: bill chat-session credits exactly and skip agent-host twins

A chat-session request that records copilotCredits now costs credits x $0.01 through costFromBilling; requests without it stay priced from tokens. A chat session whose id matches a session-state dir is left to the agent-host path so it is counted once.

* Copilot: drop agent-host twins at serve time, rate chat requests one by one, re-derive past days

A chat-session journal request is left out when the same session has billed agent-host usage (session-state calls with tokens or nanoAiu, or session-store rows). Plan credits rate chat-session requests individually, so older requests in a mixed session keep their estimate. DAILY_CACHE_VERSION 75 with copilot pending re-derive so settled days pick up the new counts.
2026-10-13 01:02:04 +02:00
Resham Joshi
756511e8ea
fix(copilot): count agent-host requests and their output (#1651)
Some checks are pending
CI / semgrep (push) Waiting to run
Tests / test (22) (push) Waiting to run
Tests / test (22.13.0) (push) Waiting to run
Tests / test (24) (push) Waiting to run
Tests / test (26) (push) Waiting to run
Tests / test-platforms (macos-latest) (push) Waiting to run
Tests / test-platforms (windows-latest) (push) Waiting to run
Tests / desktop (push) Waiting to run
* fix(copilot): count VS Code agent-host requests and their output

VS Code now runs Copilot's agent through the Copilot CLI engine. Its
assistant.message events carry no outputTokens (CLI 1.0.8x writes the same
shape), so every request was skipped and the shutdown rollup, which assumed
per-turn events own output, dropped it: 0 calls and 0 output.

Tokenless messages now count as calls with zero tokens, and a leg with no
per-turn output takes its output from the rollup. The rollup also carries
modelMetrics totalNanoAiu, so the session's credits are exact. Tokens and
credits appear when the session ends.

* test(copilot): serve an open agent-host session

* fix(copilot): serve store-session output via the residual, count credit requests

CLI 1.0.8x sessions with session-store rows lost their output: the rows
replace the rollup and carry output only for compaction requests. The
per-leg residual now carries rollup output minus compaction-row output.

The credits note counted store rows, rollups and per-turn twins as separate
requests. It now counts requests by behavioral weight, rated when the
session carries an exact figure.

* fix(copilot): keep paired store-row credits, running nanoAiu delta

A behavioral store row (compaction or crash tail) made the credit sum drop
every paired row. Store rows now always count; a rollup counts only when
nothing behavioral carries nanoAiu.

totalNanoAiu is a running bill that does not reset at compaction, so its
per-leg delta uses its own sentinel instead of the token reset.

copilot's PENDING_REDERIVE contract moves to 56: a store row and its
message straddling midnight can move one call to the next day.
2026-10-06 18:03:51 +02:00
iamtoruk
62cb4f8630 test: isolate the real home on Windows and make the CLI suite pass there
os.homedir() reads USERPROFILE on Windows and never HOME, so the
isolation setup and every test that moved home were still resolving the
developer's real home directory. Fixtures came back empty, real sessions
leaked into counts, and the code under test wrote plans, receipts and a
sync ledger into the real ~/.config/codeburn during a run.

USERPROFILE now joins the redirected variables and a shared setHome()
helper moves HOME and USERPROFILE together; every spawn env carries both.
The remaining Windows-only failures were POSIX-shaped fixtures: chmod
tricks that only toggle a read-only attribute on NTFS, pid 1 as an
always-alive lock holder, splitting paths on '/', bare tar, mode-bit
expectations, and per-file timeouts too short for the cost of a tsx spawn.
Each gets a platform-aware equivalent so nothing is skipped on Windows and
the POSIX branches stay byte-for-byte what they were.

Full run on Windows: 267 files passed, 2 skipped, 3592 tests passed.
2026-09-02 13:43:57 -07:00
iamtoruk
71bd7607f3 docs(plan): calibrate the Copilot credit estimate wording against real session-store rows (#1199) 2026-08-31 03:13:04 -07:00
iamtoruk
616a1fada2 fix(plan): move the Copilot credits bar onto the estimate and print whole credits (#1199) 2026-08-31 03:10:29 -07:00
iamtoruk
d879d6590b fix(plan): estimate Copilot AI credits from tokens and say when the total is incomplete (#1199)
Only Copilot CLI session-store rows carry total_nano_aiu, so a typical user
has a handful of exactly rated requests and hundreds of unrated ones, and
codeburn plan showed the exact-only sum as if it were the whole bill.

Price every unrated request from its tokens at listed API rates (1 credit =
$0.01) and carry that as estimatedCredits alongside creditRatedCalls /
creditUnratedCalls. A session that already carries an exact figure is never
estimated on top of: total_nano_aiu bills that request's whole token set,
JSONL twin included. spentCredits, the bar and percentUsed stay on GitHub's
exact figures. The plan headline, and status JSON via creditsNote, now say
how much of the total is estimated.
2026-08-31 03:07:38 -07:00
Aditya Vikram Singh
57e897c74a fix(plan): close Copilot nanoAiu twins once per session
foldCopilotSupplementaryTurns will not fold a supplementary twin across
a local-day boundary, so the pair can occupy separate turns. Credit
spend now groups at session scope.
2026-08-24 18:35:59 +05:30
Aditya Vikram Singh
55155c10ed fix(plan): do not double Copilot credits when both twins have nanoAiu
Store-row nanoAiu is still the bill when the JSONL twin has none.
USD plans keep the cost-or-calls retain predicate.
2026-08-23 20:54:21 +05:30
Aditya Vikram Singh
0c7d92a312 feat: Copilot AI Credit plan math from total_nano_aiu
Spent credits are finite nanoAiu / 1e9. Token-priced costUSD does not
fill the Copilot bar. Thread nanoAiu through cachedCallToApiCall with
no cache-version bump.
2026-08-23 19:12:04 +05:30
Matthew Kelch
a5a0ddaeac
feat(copilot): read per-request input/cache from session-store.db
The Copilot CLI and the GitHub Copilot desktop app both write
~/.copilot/session-store.db unconditionally; its assistant_usage_events
table holds one row per API request. Until now input/cache tokens for
these surfaces came only from the session.shutdown rollups in
events.jsonl, which are written only on clean shutdown (a crash loses
the whole leg's input/cache accounting) and lump each session leg into
one per-model total. The rollup also RESETS its counters at in-session
compaction (traced on a clean single-process 107-request session whose
sole rollup covered exactly its five post-compaction requests), so even
cleanly-closed long sessions were truncated; on a long-history machine
the store recovered ~35% of real Copilot spend lost to crashes and
compaction resets. The DB rows are per-request, crash-proof, and carry
real timestamps.

The store's input_tokens is cache-INCLUSIVE (input + cache_read +
cache_write), the same convention as the shutdown rollups — verified
against each row's token_details_json and by reconciling per-session
sums against the CLI's own footers and rollups across two machines
(1,380+ rows, 8 models, CLI 1.0.70–1.0.79, schema_version 6): every
divergence was a rollup gap. Emitted calls mirror the shutdown-call
contract: input/cache/reasoning only, output 0 — per-turn output stays
owned by the events.jsonl assistant.message calls.

Rollup-vs-store precedence is RECONCILED at serve time, per
(session, model), and only there. Both representations always parse and
cache; parseProviderSources aggregates the cached calls and, wherever
store rows exist for a (session, model), drops the rollup calls and
serves the rows plus per-leg RESIDUAL calls: each rollup leg subtracts
only the rows in its own interval — rows commit strictly before their
leg's shutdown line, so a leg at time T covers exactly the rows in
(previous leg's T, T] — and any remainder (per token component, floored
at zero) serves once at that leg's own timestamp. A store missing
requests a leg covered — adopted mid-session, rows pruned before ever
being read — therefore still serves that tail exactly once ON THAT
LEG'S DAY, a crash-tail row the rollup never saw can never cancel it,
and a complete store serves pure per-request granularity with every
residual retired to zero. The decision reads only cached contents, never discovery:
deleting or resetting the store changes nothing served, so finalized
daily history can never flip on an absence epoch; cached rows of a
deleted store remain the record until the 90-day orphan age-out (which
exempts still-discovered paths). The serve set is the one coherent
snapshot — nothing a writer does between discovery and a parse can
change what one pass sees — and read-time precedence heals persisted
duplication (stale epochs, runtimes without node:sqlite, restored
files) instead of preserving it, following the buildDurablePeriod
pattern.

Store rows and rollups carry supplementary accounting weight. A rollup
(or its residual) is aggregate accounting, never a request: zero
api-call/model-call/turn weight, tokens and cost fully retained. A
store row is one real request, but when it pairs with a served per-turn
call it is supplementary too; rows pair with same-model per-turn calls
by timestamp adjacency (monotone matching, tight 2-minute window — the
two are written at the same completion moment, and a wide window would
let a crash-only row pair against a neighbor whose own row is missing),
computed once over the FULL serve set so a date-range boundary that
separates a row from its call cannot double the request across adjacent
day queries. Only the unpaired rows — store-only requests, exactly
where crash-lost requests sit — count. Supplementary-only turns fold
into the nearest behavioral turn within 30 minutes; with no behavioral
turn to fold into they stay separate weightless turns, each on its own
day, with apiCalls 0 — and the session emission gate admits
usage-bearing zero-call sessions. The weight
propagates into the daily cache: aggregateProjectsIntoDays applies the
same rule to every calls counter and category-turn count it seals, so
v19 history and live summaries can never disagree about what was a
request.

A changed source whose read defers on the busy shape (locked, EACCES,
corrupt mid-replace — discovery still emits the source; only true
absence or a schema mismatch reads as absent) now marks session
hydration incomplete, so the daily backfill holds its watermark instead
of finalizing a day the deferred rows never reached; an unchanged
unreadable store defers nothing. The verdict travels with its result —
the 180s memo and the serve burst-reuse restore the hydration verdict
their cached data was parsed under, so a memoized partial parse cannot
inherit a later parse's complete — and a discovered source whose
FINGERPRINT cannot be read (EACCES on a present file) defers instead of
silently skipping, while a genuinely deleted file stays a silent skip. Copilot reasoning tokens are no longer
double-billed at the report layer: they are a subset of the output the
per-turn calls already price, and copilot joins claude in the
reasoning-inside-output case of the query-time cost recompute.

Store dedup keys are content-discriminated —
copilot-store:<sid>:<rowId>:<fnv1a64(created_at|tokens|model)> —
because AUTOINCREMENT prevents id reuse only within one database
lifetime: a same-path DB reset reusing row ids now mints new keys
instead of the durable union swallowing the new usage, while a
byte-identical re-insert still collapses (64-bit: 32-bit FNV
collisions between plausible token tuples are constructible). Every
call of a session serves under one project label resolved at serve
time — the session-state-derived label when the serve set knows it,
else the store rows' own — so neither rows cached before events.jsonl
existed nor an events.jsonl orphaned by a session-state prune can
split the session across two grouping keys.

CODEBURN_COPILOT_SESSION_STORE_DB is read but deliberately NOT
fingerprinted, per the #927 ruling (any copilot fingerprint change
drops cached entries whose path still exists, destroying pruned history
only the cache holds); the read is allowlisted in the #927 guard, and
serve-time reconciliation makes repointing safe without a fingerprint —
the new store's rows parse on sight and the old path's entries persist
as durable orphans. The copilot parse version appends session-store-v2
and the daily cache bumps v17 → v19: per-day attribution, call counts
and costs all change against pre-store builds. 19, not 18: an earlier
pushed head of this PR already claimed v18 under different accounting,
and the carry-forward would adopt those days as finalized without
re-deriving them.

Verified by A/B on snapshots of two real stores, a live SIGKILL crash
test (row present, no rollup, tokens recovered exactly), live resumes
whose warm-cache deltas matched new rows to the token, upgrade-healing
at 4,800-session scale, and serve-level regressions pinning every
maintainer finding from six review rounds: the rows-then-shutdown race,
stale-cache healing, age-out exemption, absence-epoch identity,
progressive row landing with residual retirement, behavioral weight
across all four pinned scenarios, the hydration fence, project
unification in both directions, the same-path reset, mixed
coverage (crash tail vs covered-leg gap), multi-leg residual day
attribution, range-invariant pairing, memo-scoped hydration verdicts,
and the fingerprint-failure fence.
2026-08-17 17:52:01 -04:00
Tiago Santos
75c32e6d65
fix: fix and improve test isolation and collision with environment (#530)
* fix: fix and improve test isolation and collision with environment

* docs: remove unnecessary comment

* test(env-isolation): clear CODEBURN_FORCE_MACOS_MAJOR and pin TZ

Two env vars read in src/ were not isolated: CODEBURN_FORCE_MACOS_MAJOR
(now cleared so it cannot leak between tests) and TZ (now pinned to UTC,
since clearing it falls back to the OS zone and would shift date buckets
versus a clean CI runner).

---------

Co-authored-by: AgentSeal <hello@agentseal.org>
2026-06-20 13:42:10 +02:00
ozymandiashh
a04c0cbfaa Track multiple provider plans 2026-05-11 16:33:33 +03:00
Trevin Chow
553cf2d706 feat(plan): subscription plan tracking with usage progress bar
Adds `codeburn plan set <id>` to configure a subscription plan (Claude Pro,
Claude Max, Cursor Pro, or custom). When set, the Overview panel renders
an API-equivalent progress bar against subscription price with a
projected month-end cost.

Closes the loudest demand signal on the repo: issue #11 ("Subscription
vs API Use") from two independent voices, plus the routing-decision use
case raised in #12.

- src/config.ts: extends CodeburnConfig with Plan, adds readPlan/savePlan/clearPlan
- src/plans.ts: presets (claude-pro $20, claude-max $200, cursor-pro $20)
- src/plan-usage.ts: getPlanUsage, resetDay-aware period math (1-28),
  median-of-7-day-trailing projection
- src/cli.ts: `codeburn plan [show|set|reset]` subcommand, plan wired
  into JSON outputs for report/today/month/status (only when active)
- src/dashboard.tsx: Plan row in Overview, color-coded (green under 80%,
  orange near, red over), with days-until-reset
- README.md: Plans section with honest framing (API-equivalent vs
  subscription price, not token allowance)
- tests/plan-usage.test.ts, tests/plans.test.ts, tests/cli-plan.test.ts:
  period math, presets, CLI round-trip

Resets respect resetDay across month boundaries. Uses median daily spend
(not mean) so one huge day doesn't distort the month-end projection.

Fixes #11

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 14:55:07 -07:00