codeburn/docs/sync/DEVELOPER.md

Sync — Developer Documentation

Architecture, protocol, server contract, and testing for codeburn sync.

Architecture

Developer machine                          Remote backend
──────────────────                         ──────────────
~/.config/codeburn/sync.json  (config)
~/.config/codeburn/.sync-token (credential)
~/.cache/codeburn/sync-ledger.json (sent-ledger)

codeburn sync push
  │
  ├─ Read config → baseUrl, clientId, issuer, tracesPath
  ├─ Read refresh token from OS store
  ├─ POST {issuer}/oauth2/token (refresh_token grant) → access_token
  ├─ Collect ParsedProviderCall[] for window
  ├─ Filter against sent-ledger (only unsent calls)
  ├─ Build OTLP/HTTP JSON payload
  ├─ POST {baseUrl}{tracesPath} with Bearer token
  ├─ On success → append deduplicationKeys to ledger
  └─ Update lastSync in config

Discovery Protocol

Server discovery document

GET {baseUrl}/.well-known/codeburn-export.json

{
  "version": 1,
  "issuer": "https://cognito-idp.us-west-2.amazonaws.com/us-west-2_XXXX",
  "client_id": "70e6sgst2ju6ff9dnrmv4l1tcb",
  "scopes": ["openid", "email"],
  "traces_path": "/v1/traces",
  "max_batch_size": 1000
}

Field	Required	Default	Description
version	No	1	Client rejects version > 1
issuer	Yes	—	OIDC issuer URL. Client fetches {issuer}/.well-known/openid-configuration
client_id	Yes	—	OAuth client ID for this deployment
scopes	No	["openid"]	Scopes to request. offline_access added dynamically if IdP supports it
traces_path	No	/v1/traces	Path for OTLP POST
max_batch_size	No	1000	Max spans per HTTP request

Why not proxy .well-known/openid-configuration?

OIDC requires the issuer claim inside the discovery doc to match the URL it was fetched from. Serving Cognito's doc from a different domain violates this constraint. The codeburn-export.json doc decouples the metrics endpoint from the identity provider.

OIDC Authentication

Flow: Authorization Code + PKCE

1. Client generates code_verifier (32 random bytes, base64url) and code_challenge (SHA-256 of verifier, base64url)
2. Client starts callback server on 127.0.0.1:19876 (fallback: 19877, 19878)
3. Browser opens {authorization_endpoint}?response_type=code&client_id=...&redirect_uri=http://127.0.0.1:{port}/callback&code_challenge=...&code_challenge_method=S256&state=...&scope=...
4. User logs in at IdP → IdP redirects to http://127.0.0.1:{port}/callback?code=...&state=...
5. Callback server validates state, extracts code
6. Client POSTs to {token_endpoint} with grant_type=authorization_code, code, code_verifier, redirect_uri, client_id
7. IdP returns access_token + refresh_token

Fixed ports

Cognito (and Okta) do exact string comparison on callback URLs. Ephemeral ports fail. We register three fixed ports: 19876, 19877, 19878. The client tries in order, falling back if a port is in use.

RFC 8252 recommends 127.0.0.1 (IP literal) over localhost to avoid IPv6 ::1 resolution.

Token refresh

On every sync push:

1. Read refresh token from OS store
2. POST {token_endpoint} with grant_type=refresh_token
3. Store whatever refresh token the server returns (handles rotation transparently)
4. On invalid_grant → stop, prompt user to re-run sync setup

Scope resolution

• Request scopes from codeburn-export.json
• Add offline_access only if scopes_supported in OIDC discovery includes it
• Cognito rejects offline_access as invalid_scope — it issues refresh tokens without it

Credential Storage

Platform	Method	Implementation
macOS	Keychain	security add-generic-password / find-generic-password
Linux	libsecret	secret-tool store / secret-tool lookup
Windows	DPAPI	PowerShell ConvertTo-SecureString / ConvertFrom-SecureString
Fallback	File	~/.config/codeburn/.sync-token with 0600 permissions

No native modules (keytar is archived). Shell out to OS CLIs. Fallback reported honestly in sync status.

OTLP Encoding

Strict protobuf-JSON mapping of ExportTraceServiceRequest. lowerCamelCase fields, hex-encoded IDs, integer enums.

Span identity (deterministic)

All ids are HMAC-SHA256 keyed by the per-install host privacy key (decision D1, see packages/cli/src/privacy-key.ts) with a role prefix — never a bare SHA-256, which would be confirmable by dictionary attack:

span_id   = first 8 bytes of HMAC-SHA256(privacyKey, "sync-span:" + deduplicationKey) → hex (16 chars)
trace_id  = first 16 bytes of HMAC-SHA256(privacyKey, "sync-trace:" + sessionId)      → hex (32 chars)

The key is generated once per install, persisted in the codeburn config dir, and never leaves the host, so re-sends are byte-identical on the same machine. Sync REQUIRES that persisted key: push aborts with an error if the config dir is unwritable (no per-process fallback key) or a key file exists but does not hold a valid key — corrupt content, a zero-byte file (a partial write), or an unreadable file, with no silent regeneration in any of those cases. Only "no file at all" may be created, and that first create is exclusive (O_CREAT|O_EXCL): concurrent first pushes collide, the loser re-reads and adopts the winner's key, so two processes can never mint different keys and mix ids derived under each. Either degradation — a per-process fallback key, or a silent re-key — would re-key every id between processes and break the partial-rejection retry guarantee below. Deliberately deleting the key file (or changing the derivation) re-keys every id: spans already sent under the old construction no longer correlate with new ones. A corrupt key file is the one case that never re-keys silently — the push stops and the operator must fix the disk or delete the file on purpose.

Resource attributes

{
  "resource": {
    "attributes": [
      { "key": "codeburn.device_id", "value": { "stringValue": "<HMAC-SHA256(privacyKey, \"sync-device:\" + hostname + \"\\x1f\" + username)[:16]> (\\x1f = ASCII Unit Separator)" } }
    ]
  }
}

Span attributes

{
  "attributes": [
    { "key": "ai.provider", "value": { "stringValue": "kiro" } },
    { "key": "ai.model", "value": { "stringValue": "claude-sonnet-4-6" } },
    { "key": "ai.input_tokens", "value": { "intValue": "12500" } },
    { "key": "ai.output_tokens", "value": { "intValue": "3200" } },
    { "key": "ai.cost_usd", "value": { "doubleValue": 0.085 } },
    { "key": "ai.project", "value": { "stringValue": "my-app" } },
    { "key": "ai.tools", "value": { "arrayValue": { "values": [{ "stringValue": "Edit" }] } } },
    { "key": "ai.speed", "value": { "stringValue": "standard" } },
    { "key": "ai.cost_estimated", "value": { "boolValue": true } }
  ]
}

Sent-Ledger

Client-side deduplication source of truth at ~/.cache/codeburn/sync-ledger.json.

Format: JSON array of { key: string, ts: string } objects.

Push logic: collect all calls in window → subtract ledger entries → send remainder → append to ledger on success.

Pruning: entries older than 6 months removed on every push.

Why not a watermark? Timestamp watermarks silently skip late-arriving calls (long sessions, providers that update rows). The ledger is exact.

Partial success

OTLP returns partial_success.rejected_spans in the response body. Because OTLP does not identify which spans were rejected, the client ledgers nothing for a partially-rejected batch — the entire batch retries on the next push. This is safe: span IDs are deterministic (derived from the deduplication key), so servers that store by span ID treat re-sent spans as idempotent upserts.

Rate limiting (429)

A push runs to completion — there is no routine per-push cap (only a 50,000-call safety valve). Server rate limits are the intended brake:

• On HTTP 429 the client honors Retry-After (delta-seconds or HTTP-date), capped at 120 seconds per wait, defaulting to 5 seconds when the header is absent
• The same batch is retried up to 3 consecutive times; if the server is still rate-limiting after that, the push stops and the remaining (unledgered) calls are sent on the next push
• On 401 or 5xx the push stops immediately with the same resume-on-next-push behavior

Server contract

The backend must implement:

1. GET {baseUrl}/.well-known/codeburn-export.json — returns the discovery doc (public, no auth)
2. POST {baseUrl}{traces_path} — accepts OTLP/HTTP JSON with Bearer token
   • Validate JWT (issued by the configured IdP)
   • Derive developer identity from token's sub claim
   • Accept startTimeUnixNano up to 6 months in the past
   • Return standard OTLP response body

No PII is included in the payload. The server derives identity solely from the authenticated token.

Testing

Unit tests (tests/sync.test.ts)

26 tests covering pure functions: discovery parsing, PKCE generation, auth URL construction, scope resolution, callback server, config read/write. No network, no browser.

Mock IdP e2e (tests/sync-e2e.test.ts)

6 tests with a localhost mock IdP server. Exercises the full auth round-trip, token refresh, rotation, revocation — fully offline, runs in CI.

Headless browser e2e (tests/sync-headless-e2e.test.ts)

1 test with Playwright headless Chromium against real Cognito. Proves the actual browser PKCE flow works including Cognito Hosted UI form submission and localhost redirect.

Developer-only — requires:

• Deployed test backend (CDK stack at ../codeburn-sync-backend/)
• Cognito user with confirmed password
• Environment variables: CODEBURN_SYNC_URL, CODEBURN_SYNC_EMAIL, CODEBURN_SYNC_PASSWORD
• Playwright Chromium installed (PLAYWRIGHT_BROWSERS_PATH)

Skipped by default when env vars are not set. Never runs in CI.

Test CDK stack (codeburn-sync-backend/)

Minimal AWS backend for the headless e2e test:

• Cognito User Pool (PKCE, fixed callback ports)
• HTTP API with JWT authorizer
• Discovery Lambda (serves codeburn-export.json)
• Ingest Lambda (logs OTLP spans to CloudWatch)

Deploy: npx cdk deploy --profile andklee-dev Cost: ~$0/mo idle (pay-per-request)

This is a test fixture, not a production reference. Any OIDC provider + OTLP-accepting endpoint satisfies the server contract.