diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 55badfaa0..032d808db 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -329,8 +329,12 @@ consequential, manually redacted error. transport; TrueNAS 26 removed the former `/api/v2.0` REST endpoints. #### TrueNAS pools/datasets not appearing -- TrueNAS data appears in the unified resource model and may take one polling cycle (30s) to appear. +- TrueNAS data appears in the unified resource model and may take one configured + polling cycle (60 seconds by default) to appear. - Check **Infrastructure** (TrueNAS host), **Storage** (pools/datasets), and **Recovery** (snapshots/replication). +- For data that stops refreshing, use the [TrueNAS polling checks](TRUENAS.md#stale-truenas-data) + before testing or restarting. A stale badge is not proof of an invalid key, + and a successful connection test is not proof that collection has recovered. ### Navigation (v6) diff --git a/docs/TRUENAS.md b/docs/TRUENAS.md index d587106b6..ddd5ea846 100644 --- a/docs/TRUENAS.md +++ b/docs/TRUENAS.md @@ -9,7 +9,7 @@ Pulse v6 includes first-class monitoring for **TrueNAS SCALE** and **TrueNAS COR 3. Enter the TrueNAS URL (e.g., `https://truenas.local`), the API key, and the username that owns the key. 4. Click **Test Connection** → **Save**. -5. Data appears within one polling cycle (~30 seconds). +5. Data appears within one configured polling cycle (60 seconds by default). ## Creating a TrueNAS API Key @@ -148,7 +148,8 @@ its trusted HTTPS origin without disabling certificate verification. current appliance through the removed `/api/v2.0` REST endpoints. ### No data appearing after adding connection -- Wait at least 30 seconds for the first poll cycle. +- Allow one configured polling cycle (60 seconds by default), not a fixed + 30-second wait. Check that the saved connection is enabled. - **Test Connection** checks authentication and the selected transport, not successful collection of every metric. The legacy REST diagnostic is expected for recognized CORE 13 systems; it is not itself a connection error. @@ -202,12 +203,53 @@ Do not upload a full browser network capture or paste code into the browser console to collect this evidence. ### Stale TrueNAS data -- If TrueNAS data stops updating, the source status transitions to `stale` after ~120 seconds. -- Check TrueNAS connectivity and API key validity. -- In your signed-in Pulse browser session, open `/api/resources` on the same - Pulse origin and inspect the entries with `platformType: "truenas"`. This - uses your existing session without copying a token or cookie. Do not post - the full response; use the sanitized export or a reviewed relevant excerpt. + +A **Pending** or **Stale** badge is not an authentication diagnosis. Keep the +connection and key unchanged while checking whether ordinary polling still +collects data; a working connection can have missing telemetry or a stalled +collection step. + +1. **Record the existing state before testing or restarting.** In your + signed-in Pulse admin browser session, open `/api/truenas/connections` on + the same Pulse origin. Inspect only the affected connection. This is a + read-only request using your existing session; do not copy a token or cookie. +2. Note `enabled`, `poll.intervalSeconds`, `poll.lastAttemptAt`, + `poll.lastSuccessAt`, `poll.consecutiveFailures`, `poll.lastError`, and + `observed.collectedAt`. When present, also note `transport.mode`, + `transport.reconnects` and `transport.lastError`. If your build does not + expose a field, record that rather than interpreting its absence as success. +3. Compare the timestamps after two configured polling cycles, without + pressing **Test Connection**, saving changes or restarting between reads. + Slow requests can take longer than the interval. `lastAttemptAt` records a + completed attempt, not an in-flight request; a frozen value alone cannot + distinguish a blocked request from polling that has not run. + +| Existing evidence | What it distinguishes and what to check next | +|---|---| +| `lastAttemptAt` advances, but `lastSuccessAt` and `observed.collectedAt` do not; failures increase | Polling returns failures. Use the actual error category and method to distinguish TLS, authentication, permissions and collection errors; a stale badge alone does not distinguish them. | +| `observed.collectedAt` advances, but some usage or History panels stay empty | Inventory is refreshing, not necessarily every metric. Follow the [missing-telemetry checks](#inventory-works-but-cpu-memory-or-history-is-missing); do not replace a working key to populate a chart. | +| No timestamps advance, or no completed attempt is recorded | Check that the connection and integration are enabled, then preserve the bounded local error above. This is not proof of an invalid key. | +| Reconnects or TrueNAS sign-ins increase while `observed.collectedAt` advances | Session turnover alone does not establish failed authentication or stopped polling. Record the timing and any existing close/error message separately from the data freshness. | + +**Test Connection** checks connectivity and authentication, not a complete +poll. Testing a saved connection can update `lastAttemptAt` and `lastSuccessAt` +and reset its failure count without refreshing `observed.collectedAt`. Record +any manual test and its time separately; a successful test is not evidence +that missing data or stopped polling has recovered. + +An `app.stats` error such as **Apps are not available** concerns Apps +collection, not necessarily the key. Check the existing Apps status in +TrueNAS, but do not stop Apps or reproduce a hang to gather evidence. Preserve +the existing error before changing anything. Do not repeatedly restart Pulse, +clear History, rotate keys or disable TLS verification as a diagnostic shortcut. + +For a report, include the running Pulse and TrueNAS versions, configured poll +interval, affected panels, before/after timestamps and a reviewed, bounded +error excerpt from the local logs above. Remove private names and addresses +from errors. Do not post the full connection response, credential files, +authentication messages, cookies or a full browser network capture. If safe +inspection is unavailable, describe what you could observe without repeating +the failure or collecting a larger dump. ### Disabling TrueNAS integration Set `PULSE_ENABLE_TRUENAS=false` and restart Pulse. Existing connection data is preserved but polling stops. diff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.md index 55badfaa0..032d808db 100644 --- a/frontend-modern/public/docs/TROUBLESHOOTING.md +++ b/frontend-modern/public/docs/TROUBLESHOOTING.md @@ -329,8 +329,12 @@ consequential, manually redacted error. transport; TrueNAS 26 removed the former `/api/v2.0` REST endpoints. #### TrueNAS pools/datasets not appearing -- TrueNAS data appears in the unified resource model and may take one polling cycle (30s) to appear. +- TrueNAS data appears in the unified resource model and may take one configured + polling cycle (60 seconds by default) to appear. - Check **Infrastructure** (TrueNAS host), **Storage** (pools/datasets), and **Recovery** (snapshots/replication). +- For data that stops refreshing, use the [TrueNAS polling checks](TRUENAS.md#stale-truenas-data) + before testing or restarting. A stale badge is not proof of an invalid key, + and a successful connection test is not proof that collection has recovered. ### Navigation (v6) diff --git a/frontend-modern/public/docs/TRUENAS.md b/frontend-modern/public/docs/TRUENAS.md index d587106b6..ddd5ea846 100644 --- a/frontend-modern/public/docs/TRUENAS.md +++ b/frontend-modern/public/docs/TRUENAS.md @@ -9,7 +9,7 @@ Pulse v6 includes first-class monitoring for **TrueNAS SCALE** and **TrueNAS COR 3. Enter the TrueNAS URL (e.g., `https://truenas.local`), the API key, and the username that owns the key. 4. Click **Test Connection** → **Save**. -5. Data appears within one polling cycle (~30 seconds). +5. Data appears within one configured polling cycle (60 seconds by default). ## Creating a TrueNAS API Key @@ -148,7 +148,8 @@ its trusted HTTPS origin without disabling certificate verification. current appliance through the removed `/api/v2.0` REST endpoints. ### No data appearing after adding connection -- Wait at least 30 seconds for the first poll cycle. +- Allow one configured polling cycle (60 seconds by default), not a fixed + 30-second wait. Check that the saved connection is enabled. - **Test Connection** checks authentication and the selected transport, not successful collection of every metric. The legacy REST diagnostic is expected for recognized CORE 13 systems; it is not itself a connection error. @@ -202,12 +203,53 @@ Do not upload a full browser network capture or paste code into the browser console to collect this evidence. ### Stale TrueNAS data -- If TrueNAS data stops updating, the source status transitions to `stale` after ~120 seconds. -- Check TrueNAS connectivity and API key validity. -- In your signed-in Pulse browser session, open `/api/resources` on the same - Pulse origin and inspect the entries with `platformType: "truenas"`. This - uses your existing session without copying a token or cookie. Do not post - the full response; use the sanitized export or a reviewed relevant excerpt. + +A **Pending** or **Stale** badge is not an authentication diagnosis. Keep the +connection and key unchanged while checking whether ordinary polling still +collects data; a working connection can have missing telemetry or a stalled +collection step. + +1. **Record the existing state before testing or restarting.** In your + signed-in Pulse admin browser session, open `/api/truenas/connections` on + the same Pulse origin. Inspect only the affected connection. This is a + read-only request using your existing session; do not copy a token or cookie. +2. Note `enabled`, `poll.intervalSeconds`, `poll.lastAttemptAt`, + `poll.lastSuccessAt`, `poll.consecutiveFailures`, `poll.lastError`, and + `observed.collectedAt`. When present, also note `transport.mode`, + `transport.reconnects` and `transport.lastError`. If your build does not + expose a field, record that rather than interpreting its absence as success. +3. Compare the timestamps after two configured polling cycles, without + pressing **Test Connection**, saving changes or restarting between reads. + Slow requests can take longer than the interval. `lastAttemptAt` records a + completed attempt, not an in-flight request; a frozen value alone cannot + distinguish a blocked request from polling that has not run. + +| Existing evidence | What it distinguishes and what to check next | +|---|---| +| `lastAttemptAt` advances, but `lastSuccessAt` and `observed.collectedAt` do not; failures increase | Polling returns failures. Use the actual error category and method to distinguish TLS, authentication, permissions and collection errors; a stale badge alone does not distinguish them. | +| `observed.collectedAt` advances, but some usage or History panels stay empty | Inventory is refreshing, not necessarily every metric. Follow the [missing-telemetry checks](#inventory-works-but-cpu-memory-or-history-is-missing); do not replace a working key to populate a chart. | +| No timestamps advance, or no completed attempt is recorded | Check that the connection and integration are enabled, then preserve the bounded local error above. This is not proof of an invalid key. | +| Reconnects or TrueNAS sign-ins increase while `observed.collectedAt` advances | Session turnover alone does not establish failed authentication or stopped polling. Record the timing and any existing close/error message separately from the data freshness. | + +**Test Connection** checks connectivity and authentication, not a complete +poll. Testing a saved connection can update `lastAttemptAt` and `lastSuccessAt` +and reset its failure count without refreshing `observed.collectedAt`. Record +any manual test and its time separately; a successful test is not evidence +that missing data or stopped polling has recovered. + +An `app.stats` error such as **Apps are not available** concerns Apps +collection, not necessarily the key. Check the existing Apps status in +TrueNAS, but do not stop Apps or reproduce a hang to gather evidence. Preserve +the existing error before changing anything. Do not repeatedly restart Pulse, +clear History, rotate keys or disable TLS verification as a diagnostic shortcut. + +For a report, include the running Pulse and TrueNAS versions, configured poll +interval, affected panels, before/after timestamps and a reviewed, bounded +error excerpt from the local logs above. Remove private names and addresses +from errors. Do not post the full connection response, credential files, +authentication messages, cookies or a full browser network capture. If safe +inspection is unavailable, describe what you could observe without repeating +the failure or collecting a larger dump. ### Disabling TrueNAS integration Set `PULSE_ENABLE_TRUENAS=false` and restart Pulse. Existing connection data is preserved but polling stops. diff --git a/scripts/tests/test_truenas_docs.py b/scripts/tests/test_truenas_docs.py index 09441b33d..385edb1a1 100644 --- a/scripts/tests/test_truenas_docs.py +++ b/scripts/tests/test_truenas_docs.py @@ -89,6 +89,57 @@ class TrueNASDocsTest(unittest.TestCase): self.assertIn(required, text) self.assertNotIn("```", text) + def test_polling_guidance_preserves_evidence_before_a_manual_test(self): + text = DOC.read_text().split("### Stale TrueNAS data", 1)[1].split("### Disabling", 1)[0] + for required in ( + "Record the existing state before testing or restarting", + "signed-in Pulse admin browser session", + "`/api/truenas/connections`", "read-only request", + "`poll.intervalSeconds`", "`poll.lastAttemptAt`", + "`poll.lastSuccessAt`", "`poll.consecutiveFailures`", "`poll.lastError`", + "`observed.collectedAt`", "`transport.reconnects`", + "two configured polling cycles", "completed attempt, not an in-flight request", + "without\n pressing **Test Connection**", + "without refreshing `observed.collectedAt`", + "a successful test is not evidence", + "Session turnover alone does not establish failed authentication", + "do not stop Apps or reproduce a hang", + "Do not post the full connection response", + ): + with self.subTest(required=required): + self.assertIn(required, text) + self.assertNotIn("```", text) + self.assertNotIn("Check TrueNAS connectivity and API key validity", text) + # Every suggested field belongs to the existing read-only projection, + # not a made-up diagnostic endpoint or a field in the secret payload. + schemas = { + "poll": (ROOT / "internal/monitoring/truenas_poller.go").read_text().split( + "type TrueNASConnectionPollStatus struct {", 1)[1].split("\n}", 1)[0], + "observed": (ROOT / "internal/monitoring/truenas_poller.go").read_text().split( + "type TrueNASConnectionObservedSummary struct {", 1)[1].split("\n}", 1)[0], + "transport": (ROOT / "internal/truenas/transport.go").read_text().split( + "type TransportStatus struct {", 1)[1].split("\n}", 1)[0], + } + fields = re.findall(r"`(poll|observed|transport)\.([A-Za-z]+)`", text) + self.assertGreater(len(fields), 8) + for projection, field in fields: + with self.subTest(field=f"{projection}.{field}"): + self.assertRegex(schemas[projection], rf'json:"{field}(?:,omitempty)?"') + + def test_general_help_links_to_polling_checks_and_uses_the_actual_default(self): + source = (ROOT / "internal/config/truenas.go").read_text() + default = re.search(r"const defaultTrueNASPollIntervalSecs = (\d+)", source).group(1) + general = (ROOT / "docs/TROUBLESHOOTING.md").read_text() + self.assertEqual((ROOT / "docs/TROUBLESHOOTING.md").read_bytes(), + (ROOT / "frontend-modern/public/docs/TROUBLESHOOTING.md").read_bytes()) + for text in (DOC.read_text(), general): + self.assertIn(f"{default} seconds by default", text) + self.assertIn("TRUENAS.md#stale-truenas-data", general) + self.assertIn("before testing or restarting", general) + self.assertIn("a successful connection test is not proof", general) + self.assertNotIn("(~30 seconds)", DOC.read_text()) + self.assertNotIn("cycle (30s)", general) + def test_preparation_protects_new_and_existing_payload(self): preparation, _ = commands() with tempfile.TemporaryDirectory() as temporary: