fix(telemetry): suppress outbound pings while mock mode is enabled
Some checks are pending
Build and Test / Secret Scan (push) Waiting to run
Build and Test / Frontend & Backend (push) Waiting to run
Canonical Governance / governance (push) Waiting to run
Patrol Qualification Regression / Catalog, scorer, and replay regression (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 1/4) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 2/4) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 3/4) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 4/4) (push) Waiting to run
Core E2E Tests / E2E verdict (push) Blocked by required conditions
Unified Agent Native Verification / Linux ARM64 (push) Waiting to run
Unified Agent Native Verification / Linux x64 (push) Waiting to run
Unified Agent Native Verification / Windows x64 (push) Waiting to run
Unified Agent Native Verification / macOS ARM64 (push) Waiting to run
Unified Agent Native Verification / macOS Intel (push) Waiting to run
Unified Agent Native Verification / FreeBSD cross-build contract (push) Waiting to run

Every mock-mode boot (e2e/CI managed backends, qual runs, demo containers)
was sending startup/heartbeat pings describing the synthetic fixture fleet,
registering as real installs in the license-server telemetry DB and
dominating the VMware/Kubernetes/TrueNAS adoption aggregates (346 of 6,225
weekly-active installs matched the mock signature, incl. 120 of 227 on
6.1.0-rc.1).

- internal/telemetry: route startup and heartbeat sends through sendEvent,
  which drops the ping while mock.IsMockEnabled() is true; checked per event
  so runtime mock toggles take effect immediately. Disclosed in the package
  doc and both PRIVACY.md copies.
- scripts/telemetry_adoption_report.py: exclude historical mock-fleet rows
  (kubernetes_pods = 120×N with vmware_hosts = 7×N, the internal/mock
  fixture template) by default, report the excluded row/install counts, and
  add --include-mock-fleet to audit them. Versions through 6.1.0-rc.2 keep
  pinging the mock fleet until upgraded, so report-side filtering stays
  necessary.
- security-privacy contract: record the mock-mode suppression boundary and
  the report-side mock-fleet exclusion as subsystem invariants.
- Tests: Go sendEvent suppression/sending pair against an httptest endpoint;
  Python signature-matcher and summarize_rows exclusion coverage.
This commit is contained in:
rcourtman 2026-07-17 18:19:18 +01:00
parent 5abb2d8f48
commit 0f15de6a4a
7 changed files with 199 additions and 8 deletions

View file

@ -1250,6 +1250,55 @@ class TelemetryAdoptionReportTest(unittest.TestCase):
self.assertEqual(stages["approved_action_success"]["observed_signal_free_starts"], 1)
self.assertEqual(stages["approved_action_success"]["observed_signal_free_to_paid"], 1)
def test_is_mock_fleet_row_matches_scaled_fixture_signature(self) -> None:
self.assertTrue(report.is_mock_fleet_row({"kubernetes_pods": 120, "vmware_hosts": 7}))
self.assertTrue(report.is_mock_fleet_row({"kubernetes_pods": 600, "vmware_hosts": 35}))
self.assertFalse(report.is_mock_fleet_row({"kubernetes_pods": 120, "vmware_hosts": 4}))
self.assertFalse(report.is_mock_fleet_row({"kubernetes_pods": 119, "vmware_hosts": 7}))
self.assertFalse(report.is_mock_fleet_row({"kubernetes_pods": 0, "vmware_hosts": 0}))
self.assertFalse(report.is_mock_fleet_row({}))
def test_summarize_rows_excludes_mock_fleet_rows_by_default(self) -> None:
now = datetime.now(timezone.utc).replace(microsecond=0)
real_row = {
"install_id": "install-real",
"version": "v6.1.0-rc.2",
"platform": "binary",
"received_at": (now - timedelta(hours=2)).strftime("%Y-%m-%d %H:%M:%S"),
"event": "heartbeat",
"kubernetes_pods": 37,
"vmware_hosts": 2,
}
mock_row = {
"install_id": "install-mock",
"version": "v6.1.0-rc.2",
"platform": "docker",
"received_at": (now - timedelta(hours=3)).strftime("%Y-%m-%d %H:%M:%S"),
"event": "heartbeat",
"kubernetes_pods": 240,
"vmware_hosts": 14,
"pve_nodes": 10,
}
db_stats = {"latest_ping": real_row["received_at"], "total_rows": 2, "total_distinct_installs": 2}
summary = report.summarize_rows(db_stats, [real_row, mock_row], published_versions=set())
self.assertEqual(summary["active_latest"]["active_24h"], 1)
self.assertEqual(
summary["mock_fleet_exclusions"],
{"enabled": True, "rows": 1, "installs": 1},
)
text = report.format_text(summary, "rcourtman/Pulse", 7)
self.assertIn("mock fixture fleet excluded from window: 1 row(s) across 1 install(s)", text)
included = report.summarize_rows(
db_stats, [real_row, mock_row], published_versions=set(), include_mock_fleet=True
)
self.assertEqual(included["active_latest"]["active_24h"], 2)
self.assertEqual(
included["mock_fleet_exclusions"],
{"enabled": False, "rows": 0, "installs": 0},
)
def test_format_text_includes_latest_install_windows(self) -> None:
summary = {
"db_stats": {