codeburn/docs/sync/DEVELOPER.md

Sync — Developer Documentation

Architecture, protocol, server contract, and testing for codeburn sync.

Architecture

Developer machine                          Remote backend
──────────────────                         ──────────────
~/.config/codeburn/sync.json  (config)
~/.config/codeburn/.sync-token (credential)
~/.cache/codeburn/sync-ledger.json (sent-ledger)

codeburn sync push
  │
  ├─ Read config → baseUrl, clientId, issuer, tracesPath
  ├─ Read refresh token from OS store
  ├─ POST {issuer}/oauth2/token (refresh_token grant) → access_token
  ├─ Collect ParsedProviderCall[] for window
  ├─ Filter against sent-ledger (only unsent calls)
  ├─ Build OTLP/HTTP JSON payload
  ├─ POST {baseUrl}{tracesPath} with Bearer token
  ├─ On success → append deduplicationKeys to ledger
  └─ Update lastSync in config

Discovery Protocol

Server discovery document

GET {baseUrl}/.well-known/codeburn-export.json

{
  "version": 1,
  "issuer": "https://cognito-idp.us-west-2.amazonaws.com/us-west-2_XXXX",
  "client_id": "70e6sgst2ju6ff9dnrmv4l1tcb",
  "scopes": ["openid", "email"],
  "traces_path": "/v1/traces",
  "max_batch_size": 1000
}

Field	Required	Default	Description
version	No	1	Client rejects version > 1
issuer	Yes	—	OIDC issuer URL. Client fetches {issuer}/.well-known/openid-configuration
client_id	Yes	—	OAuth client ID for this deployment
scopes	No	["openid"]	Scopes to request. offline_access added dynamically if IdP supports it
traces_path	No	/v1/traces	Path for OTLP POST
max_batch_size	No	1000	Max spans per HTTP request

Why not proxy .well-known/openid-configuration?

OIDC requires the issuer claim inside the discovery doc to match the URL it was fetched from. Serving Cognito's doc from a different domain violates this constraint. The codeburn-export.json doc decouples the metrics endpoint from the identity provider.

OIDC Authentication

Flow: Authorization Code + PKCE

1. Client generates code_verifier (32 random bytes, base64url) and code_challenge (SHA-256 of verifier, base64url)
2. Client starts callback server on 127.0.0.1:19876 (fallback: 19877, 19878)
3. Browser opens {authorization_endpoint}?response_type=code&client_id=...&redirect_uri=http://127.0.0.1:{port}/callback&code_challenge=...&code_challenge_method=S256&state=...&scope=...
4. User logs in at IdP → IdP redirects to http://127.0.0.1:{port}/callback?code=...&state=...
5. Callback server validates state, extracts code
6. Client POSTs to {token_endpoint} with grant_type=authorization_code, code, code_verifier, redirect_uri, client_id
7. IdP returns access_token + refresh_token

Fixed ports

Cognito (and Okta) do exact string comparison on callback URLs. Ephemeral ports fail. We register three fixed ports: 19876, 19877, 19878. The client tries in order, falling back if a port is in use.

RFC 8252 recommends 127.0.0.1 (IP literal) over localhost to avoid IPv6 ::1 resolution.

Token refresh

On every sync push:

1. Read refresh token from OS store
2. POST {token_endpoint} with grant_type=refresh_token
3. Store whatever refresh token the server returns (handles rotation transparently)
4. On invalid_grant → stop, prompt user to re-run sync setup

Scope resolution

• Request scopes from codeburn-export.json
• Add offline_access only if scopes_supported in OIDC discovery includes it
• Cognito rejects offline_access as invalid_scope — it issues refresh tokens without it

Credential Storage

Platform	Method	Implementation
macOS	Keychain	security add-generic-password / find-generic-password
Linux	libsecret	secret-tool store / secret-tool lookup
Windows	DPAPI	PowerShell ConvertTo-SecureString / ConvertFrom-SecureString
Fallback	File	~/.config/codeburn/.sync-token with 0600 permissions

No native modules (keytar is archived). Shell out to OS CLIs. Fallback reported honestly in sync status.

OTLP Encoding

Strict protobuf-JSON mapping of ExportTraceServiceRequest. lowerCamelCase fields, hex-encoded IDs, integer enums.

Span identity (deterministic)

span_id   = first 8 bytes of SHA-256(deduplicationKey) → hex (16 chars)
trace_id  = first 16 bytes of SHA-256(sessionId)       → hex (32 chars)

Re-sends are byte-identical. Server-side dedup is defense-in-depth.

Resource attributes

{
  "resource": {
    "attributes": [
      { "key": "codeburn.device_id", "value": { "stringValue": "<SHA-256(hostname+username)[:16]>" } },
      { "key": "codeburn.coverage_through", "value": { "stringValue": "2026-08-24" } }
    ]
  }
}

codeburn.coverage_through is optional. It is the ISO date the local corpus is complete through, taken from the daily-cache watermark, and is stamped only when a complete local parse finalized that watermark (complete and watermarkTrusted both set). Receivers should treat its absence as "coverage unknown", never as "no history".

Span attributes

{
  "attributes": [
    { "key": "ai.provider", "value": { "stringValue": "kiro" } },
    { "key": "ai.model", "value": { "stringValue": "claude-sonnet-4-6" } },
    { "key": "ai.input_tokens", "value": { "intValue": "12500" } },
    { "key": "ai.output_tokens", "value": { "intValue": "3200" } },
    { "key": "ai.cost_usd", "value": { "doubleValue": 0.085 } },
    { "key": "ai.project", "value": { "stringValue": "my-app" } },
    { "key": "ai.tools", "value": { "arrayValue": { "values": [{ "stringValue": "Edit" }] } } },
    { "key": "ai.speed", "value": { "stringValue": "standard" } },
    { "key": "ai.cost_estimated", "value": { "boolValue": true } },
    { "key": "ai.work_unit_id", "value": { "stringValue": "ff1b1358ef64c52f80e50e7ae47ca176" } },
    { "key": "ai.session_role", "value": { "stringValue": "child" } },
    { "key": "ai.lineage_evidence", "value": { "stringValue": "provider-recorded" } },
    { "key": "ai.cache_read_tokens", "value": { "intValue": "800" } },
    { "key": "ai.cache_write_tokens", "value": { "intValue": "200" } },
    { "key": "ai.call_count", "value": { "intValue": "3" } },
    { "key": "ai.session_duration_ms", "value": { "intValue": "61000" } },
    { "key": "ai.subscription_covered", "value": { "boolValue": true } }
  ]
}

Every attribute from ai.work_unit_id down is optional and sent only when its value is proven; an old receiver ignoring unknown attributes loses nothing:

• ai.work_unit_id, ai.session_role, ai.lineage_evidence are emitted together or not at all, and only for sessions whose provider durably recorded lineage (#1140). ai.work_unit_id is deriveTraceId of the root session id resolved by the #1145 work-unit resolver: the exact trace-id derivation above, so a unit's identity matches the root trace already on the wire. Lineage is never inferred: a session without recorded lineage carries none of the three, and a child whose parent is out of range (or whose link is ambiguous or cyclic) fails closed and carries none either.
• ai.cache_read_tokens / ai.cache_write_tokens are the provider-recorded cache token counts, billable-consistent with ai.input_tokens (cache read takes the display layer's max(cacheReadInputTokens, cachedInputTokens) convention across the Anthropic and OpenAI vocabularies). Each is sent only when non-zero.
• ai.call_count is the number of usage spans the span's session contributes in the synced window. ai.session_duration_ms is the last-minus-first provider-recorded event time of that session, omitted when either timestamp is missing or out of order.
• ai.subscription_covered is the plan/proxy-path machinery's decision: true when a configured plan covers the call's provider or the session's provider-recorded cwd sits under a configured proxy path, false when both are ruled out, and omitted when the machinery cannot decide (no plan match and no cwd to check).

ai.project is optional. Usage spans include it only when CodeBurn can derive one safe basename from a provider-recorded absolute working directory. Attribution spans derive it only from the normalized git.repo; PR-only evidence omits it. Receivers must group a missing project as unattributed and must not require the field. When usage and attribution spans for one trace carry different safe basenames (for example, a fork checkout whose directory name differs from the upstream repository), the attribution span's normalized git.repo basename is authoritative for project aggregation. The usage cwd basename is provisional; receivers must not count both as separate projects.

ai.output_tokens is the billable output total. For providers that meter reasoning separately from response tokens, CodeBurn includes that reasoning in this field; providers whose response count already includes reasoning are left unchanged.

Sent-Ledger

Client-side deduplication source of truth at ~/.cache/codeburn/sync-ledger.json.

Format: JSON array of { key: string, ts: string } objects.

Push logic: collect all calls in window → subtract ledger entries → send remainder → append to ledger on success.

Pruning: entries older than 6 months removed on every push.

Why not a watermark? Timestamp watermarks silently skip late-arriving calls (long sessions, providers that update rows). The ledger is exact.

Partial success

OTLP returns partial_success.rejected_spans in the response body. Because OTLP does not identify which spans were rejected, the client ledgers nothing for a partially-rejected batch — the entire batch retries on the next push. This is safe: span IDs are deterministic (derived from the deduplication key), so servers that store by span ID treat re-sent spans as idempotent upserts.

Rate limiting (429)

A push runs to completion — there is no routine per-push cap (only a 50,000-call safety valve). Server rate limits are the intended brake:

• On HTTP 429 the client honors Retry-After (delta-seconds or HTTP-date), capped at 120 seconds per wait, defaulting to 5 seconds when the header is absent
• The same batch is retried up to 3 consecutive times; if the server is still rate-limiting after that, the push stops and the remaining (unledgered) calls are sent on the next push
• On 401 or 5xx the push stops immediately with the same resume-on-next-push behavior

Server contract

The backend must implement:

1. GET {baseUrl}/.well-known/codeburn-export.json — returns the discovery doc (public, no auth)
2. POST {baseUrl}{traces_path} — accepts OTLP/HTTP JSON with Bearer token
   • Validate JWT (issued by the configured IdP)
   • Derive developer identity from token's sub claim
   • Accept startTimeUnixNano up to 6 months in the past
   • Return standard OTLP response body

No PII is included in the payload. The server derives identity solely from the authenticated token.

Testing

Unit tests (tests/sync.test.ts)

26 tests covering pure functions: discovery parsing, PKCE generation, auth URL construction, scope resolution, callback server, config read/write. No network, no browser.

Mock IdP e2e (tests/sync-e2e.test.ts)

6 tests with a localhost mock IdP server. Exercises the full auth round-trip, token refresh, rotation, revocation — fully offline, runs in CI.

Headless browser e2e (tests/sync-headless-e2e.test.ts)

1 test with Playwright headless Chromium against real Cognito. Proves the actual browser PKCE flow works including Cognito Hosted UI form submission and localhost redirect.

Developer-only — requires:

• Deployed test backend (CDK stack at ../codeburn-sync-backend/)
• Cognito user with confirmed password
• Environment variables: CODEBURN_SYNC_URL, CODEBURN_SYNC_EMAIL, CODEBURN_SYNC_PASSWORD
• Playwright Chromium installed (PLAYWRIGHT_BROWSERS_PATH)

Skipped by default when env vars are not set. Never runs in CI.

Test CDK stack (codeburn-sync-backend/)

Minimal AWS backend for the headless e2e test:

• Cognito User Pool (PKCE, fixed callback ports)
• HTTP API with JWT authorizer
• Discovery Lambda (serves codeburn-export.json)
• Ingest Lambda (logs OTLP spans to CloudWatch)

Deploy: npx cdk deploy --profile andklee-dev Cost: ~$0/mo idle (pay-per-request)

This is a test fixture, not a production reference. Any OIDC provider + OTLP-accepting endpoint satisfies the server contract.