fix(telemetry): opt-in default + picker funnel events (#3308)
Some checks are pending
CLI Release / Build and release CLI (push) Waiting to run
Lint / macOS Compatibility (push) Waiting to run
Lint / Biome Lint (push) Waiting to run
Lint / ShellCheck (push) Waiting to run

Two bugs from the #3305 rollout:

1. Test pollution: orchestrate.test.ts imports runOrchestration directly
   and never calls initTelemetry, but _enabled defaulted to true in the
   module so captureEvent happily fired real events at PostHog tagged
   agent=testagent. The onboarding funnel filled up with CI fixture data.

2. Funnel started too late: funnel_* events fired inside runOrchestration,
   which is only called AFTER the interactive picker completes. Users who
   bail at the agent/cloud/setup-options/name prompts were invisible —
   yet that's exactly where real drop-off happens.

Fix 1 — telemetry.ts:
  - Default _enabled = false. Nothing fires until initTelemetry is
    explicitly called. Production (index.ts) calls it; tests that need
    telemetry (telemetry.test.ts) call it with BUN_ENV/NODE_ENV cleared.
  - Belt-and-suspenders: initTelemetry now short-circuits when
    BUN_ENV === "test" || NODE_ENV === "test", so even if future code
    calls it from a test context, events stay local.

Fix 2 — picker instrumentation:
  New events fired before runOrchestration in every entry path:

    spawn_launched         { mode: interactive | agent_interactive | direct | headless }
    menu_shown / menu_selected / menu_cancelled   (only when user has prior spawns)
    agent_picker_shown
    agent_selected         { agent }     — also sets telemetry context
    cloud_picker_shown
    cloud_selected         { cloud }     — also sets telemetry context
    preflight_passed
    setup_options_shown
    setup_options_selected { step_count }
    name_prompt_shown
    name_entered
    picker_completed

  Wired into:
    commands/interactive.ts  cmdInteractive + cmdAgentInteractive
    commands/run.ts          cmdRun (direct `spawn <agent> <cloud>`)
                             cmdRunHeadless (only spawn_launched)

  runOrchestration's existing funnel_* events continue to fire unchanged.
  The final funnel in PostHog:
    spawn_launched → agent_selected → cloud_selected → preflight_passed
    → setup_options_selected → name_entered → picker_completed
    → funnel_started → funnel_cloud_authed → funnel_credentials_ready
    → funnel_vm_ready → funnel_install_completed → funnel_configure_completed
    → funnel_prelaunch_completed → funnel_handoff

Tests:
- telemetry.test.ts: 2 new env-guard tests (BUN_ENV, NODE_ENV), plus
  updated beforeEach to clear both env vars so existing tests still
  exercise initTelemetry.
- Full suite: 2131/2131 pass, biome 0 errors.

Bumps 1.0.12 -> 1.0.13 (patch — auto-propagates under #3296 policy).
This commit is contained in:
Ahmed Abushagur 2026-04-15 01:43:30 -07:00 committed by GitHub
parent d1d51fb06d
commit a179fdbbab
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 164 additions and 3 deletions

View file

@ -1,6 +1,6 @@
{
"name": "@openrouter/spawn",
"version": "1.0.12",
"version": "1.0.13",
"type": "module",
"bin": {
"spawn": "cli.js"