codeburn/docs/sync/DEVELOPER.md

# Sync — Developer Documentation

Architecture, protocol, server contract, and testing for `codeburn sync`.

## Architecture

```
Developer machine                          Remote backend
──────────────────                         ──────────────
~/.config/codeburn/sync.json  (config)
~/.config/codeburn/.sync-token (credential)
~/.cache/codeburn/sync-ledger.json (sent-ledger)

codeburn sync push
  │
  ├─ Read config → baseUrl, clientId, issuer, tracesPath
  ├─ Read refresh token from OS store
  ├─ POST {issuer}/oauth2/token (refresh_token grant) → access_token
  ├─ Collect ParsedProviderCall[] for window
  ├─ Filter against sent-ledger (only unsent calls)
  ├─ Build OTLP/HTTP JSON payload
  ├─ POST {baseUrl}{tracesPath} with Bearer token
  ├─ On success → append deduplicationKeys to ledger
  └─ Update lastSync in config
```

## Discovery Protocol

### Server discovery document

```
GET {baseUrl}/.well-known/codeburn-export.json
```

```json
{
  "version": 1,
  "issuer": "https://cognito-idp.us-west-2.amazonaws.com/us-west-2_XXXX",
  "client_id": "70e6sgst2ju6ff9dnrmv4l1tcb",
  "scopes": ["openid", "email"],
  "traces_path": "/v1/traces",
  "max_batch_size": 1000
}
```

| Field | Required | Default | Description |
|---|---|---|---|
| `version` | No | `1` | Client rejects `version > 1` |
| `issuer` | Yes | — | OIDC issuer URL. Client fetches `{issuer}/.well-known/openid-configuration` |
| `client_id` | Yes | — | OAuth client ID for this deployment |
| `scopes` | No | `["openid"]` | Scopes to request. `offline_access` added dynamically if IdP supports it |
| `traces_path` | No | `/v1/traces` | Path for OTLP POST |
| `max_batch_size` | No | `1000` | Max spans per HTTP request |

### Why not proxy `.well-known/openid-configuration`?

OIDC requires the `issuer` claim inside the discovery doc to match the URL it was fetched from. Serving Cognito's doc from a different domain violates this constraint. The `codeburn-export.json` doc decouples the metrics endpoint from the identity provider.

## OIDC Authentication

### Flow: Authorization Code + PKCE

1. Client generates `code_verifier` (32 random bytes, base64url) and `code_challenge` (SHA-256 of verifier, base64url)
2. Client starts callback server on `127.0.0.1:19876` (fallback: 19877, 19878)
3. Browser opens `{authorization_endpoint}?response_type=code&client_id=...&redirect_uri=http://127.0.0.1:{port}/callback&code_challenge=...&code_challenge_method=S256&state=...&scope=...`
4. User logs in at IdP → IdP redirects to `http://127.0.0.1:{port}/callback?code=...&state=...`
5. Callback server validates `state`, extracts `code`
6. Client POSTs to `{token_endpoint}` with `grant_type=authorization_code`, `code`, `code_verifier`, `redirect_uri`, `client_id`
7. IdP returns `access_token` + `refresh_token`

### Fixed ports

Cognito (and Okta) do exact string comparison on callback URLs. Ephemeral ports fail. We register three fixed ports: `19876`, `19877`, `19878`. The client tries in order, falling back if a port is in use.

RFC 8252 recommends `127.0.0.1` (IP literal) over `localhost` to avoid IPv6 `::1` resolution.

### Token refresh

On every `sync push`:
1. Read refresh token from OS store
2. POST `{token_endpoint}` with `grant_type=refresh_token`
3. Store whatever refresh token the server returns (handles rotation transparently)
4. On `invalid_grant` → stop, prompt user to re-run `sync setup`

### Scope resolution

- Request scopes from `codeburn-export.json`
- Add `offline_access` only if `scopes_supported` in OIDC discovery includes it
- Cognito rejects `offline_access` as `invalid_scope` — it issues refresh tokens without it

## Credential Storage

| Platform | Method | Implementation |
|---|---|---|
| macOS | Keychain | `security add-generic-password` / `find-generic-password` |
| Linux | libsecret | `secret-tool store` / `secret-tool lookup` |
| Windows | DPAPI | PowerShell `ConvertTo-SecureString` / `ConvertFrom-SecureString` |
| Fallback | File | `~/.config/codeburn/.sync-token` with `0600` permissions |

No native modules (`keytar` is archived). Shell out to OS CLIs. Fallback reported honestly in `sync status`.

## OTLP Encoding

Strict protobuf-JSON mapping of `ExportTraceServiceRequest`. lowerCamelCase fields, hex-encoded IDs, integer enums.

### Span identity (deterministic)

```
span_id   = first 8 bytes of SHA-256(deduplicationKey) → hex (16 chars)
trace_id  = first 16 bytes of SHA-256(sessionId)       → hex (32 chars)
```

Re-sends are byte-identical. Server-side dedup is defense-in-depth.

### Resource attributes

```json
{
  "resource": {
    "attributes": [
      { "key": "codeburn.device_id", "value": { "stringValue": "<SHA-256(hostname+username)[:16]>" } }
    ]
  }
}
```

### Span attributes

```json
{
  "attributes": [
    { "key": "ai.provider", "value": { "stringValue": "kiro" } },
    { "key": "ai.model", "value": { "stringValue": "claude-sonnet-4-6" } },
    { "key": "ai.input_tokens", "value": { "intValue": "12500" } },
    { "key": "ai.output_tokens", "value": { "intValue": "3200" } },
    { "key": "ai.cost_usd", "value": { "doubleValue": 0.085 } },
    { "key": "ai.project", "value": { "stringValue": "my-app" } },
    { "key": "ai.tools", "value": { "arrayValue": { "values": [{ "stringValue": "Edit" }] } } },
    { "key": "ai.speed", "value": { "stringValue": "standard" } },
    { "key": "ai.cost_estimated", "value": { "boolValue": true } }
  ]
}
```

## Sent-Ledger

Client-side deduplication source of truth at `~/.cache/codeburn/sync-ledger.json`.

Format: JSON array of `{ key: string, ts: string }` objects.

**Push logic**: collect all calls in window → subtract ledger entries → send remainder → append to ledger on success.

**Pruning**: entries older than 6 months removed on every push.

**Why not a watermark?** Timestamp watermarks silently skip late-arriving calls (long sessions, providers that update rows). The ledger is exact.

### Partial success

OTLP returns `partial_success.rejected_spans` in the response body. Because OTLP does not identify *which* spans were rejected, the client ledgers nothing for a partially-rejected batch — the entire batch retries on the next push. This is safe: span IDs are deterministic (derived from the deduplication key), so servers that store by span ID treat re-sent spans as idempotent upserts.

### Rate limiting (429)

A push runs to completion — there is no routine per-push cap (only a 50,000-call safety valve). Server rate limits are the intended brake:

- On HTTP 429 the client honors `Retry-After` (delta-seconds or HTTP-date), capped at 120 seconds per wait, defaulting to 5 seconds when the header is absent
- The same batch is retried up to 3 consecutive times; if the server is still rate-limiting after that, the push stops and the remaining (unledgered) calls are sent on the next push
- On 401 or 5xx the push stops immediately with the same resume-on-next-push behavior

### Server contract

The backend must implement:

1. `GET {baseUrl}/.well-known/codeburn-export.json` — returns the discovery doc (public, no auth)
2. `POST {baseUrl}{traces_path}` — accepts OTLP/HTTP JSON with Bearer token
   - Validate JWT (issued by the configured IdP)
   - Derive developer identity from token's `sub` claim
   - Accept `startTimeUnixNano` up to 6 months in the past
   - Return standard OTLP response body

No PII is included in the payload. The server derives identity solely from the authenticated token.

## Testing

### Unit tests (`tests/sync.test.ts`)

26 tests covering pure functions: discovery parsing, PKCE generation, auth URL construction, scope resolution, callback server, config read/write. No network, no browser.

### Mock IdP e2e (`tests/sync-e2e.test.ts`)

6 tests with a localhost mock IdP server. Exercises the full auth round-trip, token refresh, rotation, revocation — fully offline, runs in CI.

### Headless browser e2e (`tests/sync-headless-e2e.test.ts`)

1 test with Playwright headless Chromium against real Cognito. Proves the actual browser PKCE flow works including Cognito Hosted UI form submission and localhost redirect.

**Developer-only** — requires:
- Deployed test backend (CDK stack at `../codeburn-sync-backend/`)
- Cognito user with confirmed password
- Environment variables: `CODEBURN_SYNC_URL`, `CODEBURN_SYNC_EMAIL`, `CODEBURN_SYNC_PASSWORD`
- Playwright Chromium installed (`PLAYWRIGHT_BROWSERS_PATH`)

Skipped by default when env vars are not set. Never runs in CI.

### Test CDK stack (`codeburn-sync-backend/`)

Minimal AWS backend for the headless e2e test:
- Cognito User Pool (PKCE, fixed callback ports)
- HTTP API with JWT authorizer
- Discovery Lambda (serves `codeburn-export.json`)
- Ingest Lambda (logs OTLP spans to CloudWatch)

Deploy: `npx cdk deploy --profile andklee-dev`
Cost: ~$0/mo idle (pay-per-request)

This is a **test fixture**, not a production reference. Any OIDC provider + OTLP-accepting endpoint satisfies the server contract.