diff --git a/docs/PBS.md b/docs/PBS.md index 6093098e2..922ed7583 100644 --- a/docs/PBS.md +++ b/docs/PBS.md @@ -8,7 +8,8 @@ Pulse can monitor PBS backups in two ways: ### 1. Direct PBS Connection (Recommended) -Connect directly to your PBS server for full monitoring capabilities: +Connect to the PBS API with a dedicated read-only token. An agent is not +required for these API-backed readings: **Benefits:** - ✅ Deduplication factor and storage efficiency stats @@ -29,73 +30,100 @@ If your PVE cluster has PBS storage configured, Pulse automatically fetches back - ❌ Can be slow for encrypted PBS storage - ❌ Limited metadata per backup -**Recommendation:** If you see a banner in the Recovery page (formerly Backups) suggesting you add PBS directly, following this guide will significantly improve your monitoring experience. +**Recommendation:** Start with a direct API connection when you need PBS +datastore, job or server status beyond the PVE passthrough. Add a host agent +only for extra local telemetry such as SMART and temperatures; see +[Agent Security](AGENT_SECURITY.md#proxmox-deployment-choices). --- ## Setting Up Direct PBS Connection -### Method 1: Unified Agent Install (Recommended for Bare Metal) +### Method 1: API-Only Connection (Recommended) -Install the unified agent directly on your PBS server for automatic setup: +Use a dedicated PBS monitoring user and token, not a root password. On a new +setup, run these commands in an administrator shell **on the PBS server**: ```bash -# Run on your PBS server -curl -fsSL http://:7655/install.sh | \ - sudo bash -s -- --url http://:7655 --token --enable-proxmox --proxmox-type pbs -``` - -The agent will: -1. Detect it's running on a PBS server -2. Create a `pulse-monitor@pbs` user with read-only access -3. Generate an API token -4. Register the PBS node with Pulse automatically - -### Method 2: API-Only Setup Script (Best for PBS in Containers) ⭐ - -Use this when you can run a command on the PBS host but do not want to install the agent. - -From Pulse's Settings page: -1. Go to **Settings → Infrastructure**. -2. Click **Add infrastructure**. -3. Choose **Proxmox Backup Server**. -4. Use the API-only setup path and enter your PBS server's URL. -5. Click copy to get the setup command. -6. Run the command on your PBS server. - -Example (what the UI generates): -```bash -curl -fsSL "http://:7655/api/setup-script?type=pbs&host=https://:8007&pulse_url=http://:7655" | { if [ "$(id -u)" -eq 0 ]; then PULSE_SETUP_TOKEN="" bash; elif command -v sudo >/dev/null 2>&1; then sudo env PULSE_SETUP_TOKEN="" bash; else echo "Root privileges required. Run as root (su -) and retry." >&2; exit 1; fi; } -``` - -Pulse generates that full command for you from **Settings → Infrastructure**, including -the one-time setup token. The script creates a `pulse-monitor@pbs` user, -generates a scoped API token, and registers the server with Pulse. - -> **Note**: API-only mode does not include temperature monitoring or AI command execution. Use **Agent Install** for full functionality. - -> **Tip**: The installer now auto-detects Proxmox mode (`pve` or `pbs`) when possible, but keeping `--proxmox-type pbs` explicit is recommended for predictable PBS onboarding. - -### Method 3: Manual Token Creation - -If you prefer manual setup: - -```bash -# SSH into your PBS server - -# 1. Create a dedicated monitoring user proxmox-backup-manager user create pulse-monitor@pbs --comment "Pulse monitoring" - -# 2. Grant read-only access (Audit role) -proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs - -# 3. Generate an API token (save the output!) proxmox-backup-manager user generate-token pulse-monitor@pbs pulse-token +proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs +proxmox-backup-manager acl update / Audit --auth-id 'pulse-monitor@pbs!pulse-token' ``` -Copy the token value and enter it in Pulse: -- **Token ID:** `pulse-monitor@pbs!pulse-token` -- **Token Value:** The UUID shown after running the command +PBS tokens have their own permissions, limited by their owning user's +permissions. Grant `Audit` to **both the user and the token**. If either already +exists, inspect and reuse it instead of deleting it or rerunning token creation. +Do not disable privilege separation or grant `Admin` to work around a missing +permission. + +The token secret is shown once. Save it privately and enter it only in Pulse's +**Token Value** field; do not paste it into a shell command, URL, screenshot or +issue report. Keep any terminal recording containing that output private. + +In Pulse: +1. Open **Settings → Infrastructure → Add infrastructure** and choose + **Proxmox Backup Server**. +2. Enter the PBS HTTPS URL, normally `https://pbs.example.com:8007`. +3. Select **API Token** and **Manual Token Setup**. +4. Enter **Token ID** `pulse-monitor@pbs!pulse-token` and its secret in + **Token Value**. +5. Keep certificate verification enabled. For a self-signed certificate, use + an independently verified **SSL Fingerprint**; see the TLS guidance below. +6. Test and save the connection. Check that the expected datastores and backups + are visible, not just that the connection test succeeds. + +### Method 2: Optional Host Agent + +Use this only when you also need host-local telemetry. The Linux agent normally +runs as root; it is not required to fix API authentication or missing backup +permissions. Review [Agent Security](AGENT_SECURITY.md) first. + +Create a separate Pulse token in **API Access**, using the **Agent host** preset +for reporting, configuration reads and agent management. This is a **Pulse** +token, not the **PBS** token used above. Do not enable command execution just +for monitoring. + +On the PBS host, enter an administrator root shell before preparing the private +file below. In the editor, save only the Pulse agent token, with no header or +quotes; never put the secret in a command argument: + +```bash +umask 077 +mkdir -p "$HOME/.config/pulse" +chmod 700 "$HOME/.config/pulse" +touch "$HOME/.config/pulse/pbs-agent-token" +chmod 600 "$HOME/.config/pulse/pbs-agent-token" +vi "$HOME/.config/pulse/pbs-agent-token" +``` + +Download the agent installer from **your Pulse server's HTTPS address**, then +inspect the saved script before running it. Replace the example Pulse URL in +both commands. Stop if the download fails; run the next command only after a +successful download and inspection. Do not substitute GitHub's top-level `install.sh`: that installs +the Pulse server, not the agent. + +```bash +curl --fail --silent --show-error \ + --output "$HOME/.config/pulse/pbs-agent-install.sh" \ + https://pulse.example.com/install.sh +``` + +```bash +bash "$HOME/.config/pulse/pbs-agent-install.sh" \ + --url https://pulse.example.com \ + --token-file "$HOME/.config/pulse/pbs-agent-token" \ + --enable-proxmox --proxmox-type pbs --enable-docker=false +``` + +Do not bypass certificate checks to fetch or run the installer. Use a trusted +Pulse certificate or a CA file you verified separately. See +[Unified Agent Setup](UNIFIED_AGENT.md) for installer trust and other profiles. +After installation, check a fresh agent report and the host's +`pulse-agent --version`; installation or hardware capacity alone does not prove +that every CPU, memory or History reading is available. Keep the token file +private and remove the bootstrap copy when no longer needed; do not remove the +installed agent's runtime credential. --- @@ -133,18 +161,65 @@ If you have multiple PBS servers, add each one separately in Settings. Pulse wil ### "Connection Failed" Error -1. **Check URL:** Ensure the PBS URL is correct (default port is 8007) - - Format: `https://pbs.example.com:8007` +Check from the Pulse host or container's network, if possible. A request from +another machine does not prove that Pulse can reach PBS. -2. **Verify token:** Test authentication: - ```bash - curl -sk -H "Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token:YOUR_TOKEN" \ - https://your-pbs:8007/api2/json/version - ``` +- **Address/network:** use the PBS HTTPS URL and port `8007`; check DNS, routing + and the firewall before changing credentials. +- **TLS:** keep verification enabled. Use a certificate trusted by the Pulse + runtime, or set its **SSL Fingerprint** after verifying the SHA-256 value + through the PBS console or another already-trusted administrative channel. + Do not accept a fingerprint solely from the failed connection. For example, + on the PBS server you can inspect its public certificate (not its private key): -3. **Network access:** Ensure Pulse can reach PBS on port 8007 + ```bash + openssl x509 -in /etc/proxmox-backup/proxy.pem -noout -fingerprint -sha256 + ``` -4. **SSL verification:** If using self-signed certificates, disable SSL verification in the node settings +- **Authentication/permissions:** test a datastore request, not just `/version`. + Prepare a private header file on the machine running curl: + + ```bash + umask 077 + mkdir -p "$HOME/.config/pulse" + chmod 700 "$HOME/.config/pulse" + touch "$HOME/.config/pulse/pbs-header" + chmod 600 "$HOME/.config/pulse/pbs-header" + vi "$HOME/.config/pulse/pbs-header" + ``` + + In the editor, save this line, replacing `` with the secret + for the PBS token being tested: + + ```text + Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token: + ``` + + Then use curl 7.76 or later, with your PBS hostname: + + ```bash + curl --fail-with-body --silent --show-error --connect-timeout 5 --max-time 15 \ + --header "@$HOME/.config/pulse/pbs-header" \ + https://pbs.example.com:8007/api2/json/admin/datastore + ``` + + For curl with a private CA or self-signed certificate, add + `--cacert "$HOME/.config/pulse/pbs-ca.pem"` using a public certificate obtained + and verified through a trusted channel. Its hostname must still match. Do not + use `--insecure` or `-k`. + + `401` means authentication was rejected; `403` means the requested access was + refused. Both return a non-zero curl exit while retaining the error body. + Inspect the existing token and both `Audit` grants before replacing anything. + A `200` response listing the expected datastore establishes access to that + list, not successful collection of every backup, job or History graph. An + empty list is not proof that all permissions are correct. + +Keep the header file outside shared repositories and diagnostics. Do not share +verbose/trace curl output, full infrastructure responses, token values or +private keys. If help is needed, provide only the HTTP status, relevant redacted +error and which expected reading is missing. Do not clear History or recreate a +working connection to make missing data look resolved. ### Slow Backup Loading @@ -155,10 +230,12 @@ If you notice slow loading for PBS storage accessed via PVE: ### Duplicate Backups -If you see the same backup twice: -- This shouldn't happen—Pulse deduplicates by VMID and timestamp -- If it does occur, the direct PBS version takes priority -- Check console for debug logs: `localStorage.setItem('debug-pmg', 'true')` +Check the source, datastore, namespace, guest type/ID and backup time on each +entry. Independent PVE installations can reuse a guest ID; matching VMIDs alone +are not proof of a duplicate. Keep both records while checking their origin. +If the same backup remains listed twice, report those redacted distinctions and +whether each entry came from direct PBS or PVE passthrough. Do not delete backups +or change retention to hide a display problem. --- @@ -169,7 +246,9 @@ In the Recovery view, PBS backups show a data source indicator: - **"PBS"** badge alone = Direct PBS connection (full data) - **"PBS via PVE"** = Passthrough via PVE storage (limited data) -Adding your PBS server directly will remove the "via PVE" indicator and unlock full monitoring capabilities. +When the same backup is reconciled across both sources, Pulse prefers the +direct PBS observation. Check the actual source and expected readings after +adding the connection; the presence of a badge alone is not collection proof. --- diff --git a/frontend-modern/public/docs/PBS.md b/frontend-modern/public/docs/PBS.md index 6093098e2..922ed7583 100644 --- a/frontend-modern/public/docs/PBS.md +++ b/frontend-modern/public/docs/PBS.md @@ -8,7 +8,8 @@ Pulse can monitor PBS backups in two ways: ### 1. Direct PBS Connection (Recommended) -Connect directly to your PBS server for full monitoring capabilities: +Connect to the PBS API with a dedicated read-only token. An agent is not +required for these API-backed readings: **Benefits:** - ✅ Deduplication factor and storage efficiency stats @@ -29,73 +30,100 @@ If your PVE cluster has PBS storage configured, Pulse automatically fetches back - ❌ Can be slow for encrypted PBS storage - ❌ Limited metadata per backup -**Recommendation:** If you see a banner in the Recovery page (formerly Backups) suggesting you add PBS directly, following this guide will significantly improve your monitoring experience. +**Recommendation:** Start with a direct API connection when you need PBS +datastore, job or server status beyond the PVE passthrough. Add a host agent +only for extra local telemetry such as SMART and temperatures; see +[Agent Security](AGENT_SECURITY.md#proxmox-deployment-choices). --- ## Setting Up Direct PBS Connection -### Method 1: Unified Agent Install (Recommended for Bare Metal) +### Method 1: API-Only Connection (Recommended) -Install the unified agent directly on your PBS server for automatic setup: +Use a dedicated PBS monitoring user and token, not a root password. On a new +setup, run these commands in an administrator shell **on the PBS server**: ```bash -# Run on your PBS server -curl -fsSL http://:7655/install.sh | \ - sudo bash -s -- --url http://:7655 --token --enable-proxmox --proxmox-type pbs -``` - -The agent will: -1. Detect it's running on a PBS server -2. Create a `pulse-monitor@pbs` user with read-only access -3. Generate an API token -4. Register the PBS node with Pulse automatically - -### Method 2: API-Only Setup Script (Best for PBS in Containers) ⭐ - -Use this when you can run a command on the PBS host but do not want to install the agent. - -From Pulse's Settings page: -1. Go to **Settings → Infrastructure**. -2. Click **Add infrastructure**. -3. Choose **Proxmox Backup Server**. -4. Use the API-only setup path and enter your PBS server's URL. -5. Click copy to get the setup command. -6. Run the command on your PBS server. - -Example (what the UI generates): -```bash -curl -fsSL "http://:7655/api/setup-script?type=pbs&host=https://:8007&pulse_url=http://:7655" | { if [ "$(id -u)" -eq 0 ]; then PULSE_SETUP_TOKEN="" bash; elif command -v sudo >/dev/null 2>&1; then sudo env PULSE_SETUP_TOKEN="" bash; else echo "Root privileges required. Run as root (su -) and retry." >&2; exit 1; fi; } -``` - -Pulse generates that full command for you from **Settings → Infrastructure**, including -the one-time setup token. The script creates a `pulse-monitor@pbs` user, -generates a scoped API token, and registers the server with Pulse. - -> **Note**: API-only mode does not include temperature monitoring or AI command execution. Use **Agent Install** for full functionality. - -> **Tip**: The installer now auto-detects Proxmox mode (`pve` or `pbs`) when possible, but keeping `--proxmox-type pbs` explicit is recommended for predictable PBS onboarding. - -### Method 3: Manual Token Creation - -If you prefer manual setup: - -```bash -# SSH into your PBS server - -# 1. Create a dedicated monitoring user proxmox-backup-manager user create pulse-monitor@pbs --comment "Pulse monitoring" - -# 2. Grant read-only access (Audit role) -proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs - -# 3. Generate an API token (save the output!) proxmox-backup-manager user generate-token pulse-monitor@pbs pulse-token +proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs +proxmox-backup-manager acl update / Audit --auth-id 'pulse-monitor@pbs!pulse-token' ``` -Copy the token value and enter it in Pulse: -- **Token ID:** `pulse-monitor@pbs!pulse-token` -- **Token Value:** The UUID shown after running the command +PBS tokens have their own permissions, limited by their owning user's +permissions. Grant `Audit` to **both the user and the token**. If either already +exists, inspect and reuse it instead of deleting it or rerunning token creation. +Do not disable privilege separation or grant `Admin` to work around a missing +permission. + +The token secret is shown once. Save it privately and enter it only in Pulse's +**Token Value** field; do not paste it into a shell command, URL, screenshot or +issue report. Keep any terminal recording containing that output private. + +In Pulse: +1. Open **Settings → Infrastructure → Add infrastructure** and choose + **Proxmox Backup Server**. +2. Enter the PBS HTTPS URL, normally `https://pbs.example.com:8007`. +3. Select **API Token** and **Manual Token Setup**. +4. Enter **Token ID** `pulse-monitor@pbs!pulse-token` and its secret in + **Token Value**. +5. Keep certificate verification enabled. For a self-signed certificate, use + an independently verified **SSL Fingerprint**; see the TLS guidance below. +6. Test and save the connection. Check that the expected datastores and backups + are visible, not just that the connection test succeeds. + +### Method 2: Optional Host Agent + +Use this only when you also need host-local telemetry. The Linux agent normally +runs as root; it is not required to fix API authentication or missing backup +permissions. Review [Agent Security](AGENT_SECURITY.md) first. + +Create a separate Pulse token in **API Access**, using the **Agent host** preset +for reporting, configuration reads and agent management. This is a **Pulse** +token, not the **PBS** token used above. Do not enable command execution just +for monitoring. + +On the PBS host, enter an administrator root shell before preparing the private +file below. In the editor, save only the Pulse agent token, with no header or +quotes; never put the secret in a command argument: + +```bash +umask 077 +mkdir -p "$HOME/.config/pulse" +chmod 700 "$HOME/.config/pulse" +touch "$HOME/.config/pulse/pbs-agent-token" +chmod 600 "$HOME/.config/pulse/pbs-agent-token" +vi "$HOME/.config/pulse/pbs-agent-token" +``` + +Download the agent installer from **your Pulse server's HTTPS address**, then +inspect the saved script before running it. Replace the example Pulse URL in +both commands. Stop if the download fails; run the next command only after a +successful download and inspection. Do not substitute GitHub's top-level `install.sh`: that installs +the Pulse server, not the agent. + +```bash +curl --fail --silent --show-error \ + --output "$HOME/.config/pulse/pbs-agent-install.sh" \ + https://pulse.example.com/install.sh +``` + +```bash +bash "$HOME/.config/pulse/pbs-agent-install.sh" \ + --url https://pulse.example.com \ + --token-file "$HOME/.config/pulse/pbs-agent-token" \ + --enable-proxmox --proxmox-type pbs --enable-docker=false +``` + +Do not bypass certificate checks to fetch or run the installer. Use a trusted +Pulse certificate or a CA file you verified separately. See +[Unified Agent Setup](UNIFIED_AGENT.md) for installer trust and other profiles. +After installation, check a fresh agent report and the host's +`pulse-agent --version`; installation or hardware capacity alone does not prove +that every CPU, memory or History reading is available. Keep the token file +private and remove the bootstrap copy when no longer needed; do not remove the +installed agent's runtime credential. --- @@ -133,18 +161,65 @@ If you have multiple PBS servers, add each one separately in Settings. Pulse wil ### "Connection Failed" Error -1. **Check URL:** Ensure the PBS URL is correct (default port is 8007) - - Format: `https://pbs.example.com:8007` +Check from the Pulse host or container's network, if possible. A request from +another machine does not prove that Pulse can reach PBS. -2. **Verify token:** Test authentication: - ```bash - curl -sk -H "Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token:YOUR_TOKEN" \ - https://your-pbs:8007/api2/json/version - ``` +- **Address/network:** use the PBS HTTPS URL and port `8007`; check DNS, routing + and the firewall before changing credentials. +- **TLS:** keep verification enabled. Use a certificate trusted by the Pulse + runtime, or set its **SSL Fingerprint** after verifying the SHA-256 value + through the PBS console or another already-trusted administrative channel. + Do not accept a fingerprint solely from the failed connection. For example, + on the PBS server you can inspect its public certificate (not its private key): -3. **Network access:** Ensure Pulse can reach PBS on port 8007 + ```bash + openssl x509 -in /etc/proxmox-backup/proxy.pem -noout -fingerprint -sha256 + ``` -4. **SSL verification:** If using self-signed certificates, disable SSL verification in the node settings +- **Authentication/permissions:** test a datastore request, not just `/version`. + Prepare a private header file on the machine running curl: + + ```bash + umask 077 + mkdir -p "$HOME/.config/pulse" + chmod 700 "$HOME/.config/pulse" + touch "$HOME/.config/pulse/pbs-header" + chmod 600 "$HOME/.config/pulse/pbs-header" + vi "$HOME/.config/pulse/pbs-header" + ``` + + In the editor, save this line, replacing `` with the secret + for the PBS token being tested: + + ```text + Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token: + ``` + + Then use curl 7.76 or later, with your PBS hostname: + + ```bash + curl --fail-with-body --silent --show-error --connect-timeout 5 --max-time 15 \ + --header "@$HOME/.config/pulse/pbs-header" \ + https://pbs.example.com:8007/api2/json/admin/datastore + ``` + + For curl with a private CA or self-signed certificate, add + `--cacert "$HOME/.config/pulse/pbs-ca.pem"` using a public certificate obtained + and verified through a trusted channel. Its hostname must still match. Do not + use `--insecure` or `-k`. + + `401` means authentication was rejected; `403` means the requested access was + refused. Both return a non-zero curl exit while retaining the error body. + Inspect the existing token and both `Audit` grants before replacing anything. + A `200` response listing the expected datastore establishes access to that + list, not successful collection of every backup, job or History graph. An + empty list is not proof that all permissions are correct. + +Keep the header file outside shared repositories and diagnostics. Do not share +verbose/trace curl output, full infrastructure responses, token values or +private keys. If help is needed, provide only the HTTP status, relevant redacted +error and which expected reading is missing. Do not clear History or recreate a +working connection to make missing data look resolved. ### Slow Backup Loading @@ -155,10 +230,12 @@ If you notice slow loading for PBS storage accessed via PVE: ### Duplicate Backups -If you see the same backup twice: -- This shouldn't happen—Pulse deduplicates by VMID and timestamp -- If it does occur, the direct PBS version takes priority -- Check console for debug logs: `localStorage.setItem('debug-pmg', 'true')` +Check the source, datastore, namespace, guest type/ID and backup time on each +entry. Independent PVE installations can reuse a guest ID; matching VMIDs alone +are not proof of a duplicate. Keep both records while checking their origin. +If the same backup remains listed twice, report those redacted distinctions and +whether each entry came from direct PBS or PVE passthrough. Do not delete backups +or change retention to hide a display problem. --- @@ -169,7 +246,9 @@ In the Recovery view, PBS backups show a data source indicator: - **"PBS"** badge alone = Direct PBS connection (full data) - **"PBS via PVE"** = Passthrough via PVE storage (limited data) -Adding your PBS server directly will remove the "via PVE" indicator and unlock full monitoring capabilities. +When the same backup is reconciled across both sources, Pulse prefers the +direct PBS observation. Check the actual source and expected readings after +adding the connection; the presence of a badge alone is not collection proof. --- diff --git a/scripts/tests/test_pbs_docs.py b/scripts/tests/test_pbs_docs.py new file mode 100644 index 000000000..03ce29234 --- /dev/null +++ b/scripts/tests/test_pbs_docs.py @@ -0,0 +1,284 @@ +#!/usr/bin/env python3 +"""Execute the PBS guide's recipes with synthetic tokens and guest-local TLS. + +No real PBS/Pulse instance, installer, service or credential is used. Run the +loopback checks with pulse-worker-source-proof, not on the worker host. +""" + +from contextlib import contextmanager +from http.server import BaseHTTPRequestHandler, HTTPServer +import json +import os +from pathlib import Path +import re +import shutil +import ssl +import stat +import subprocess +import tempfile +import threading +import unittest + + +ROOT = Path(__file__).resolve().parents[2] +DOC = ROOT / "docs/PBS.md" +PBS_TOKEN = "synthetic-pbs-secret" +AGENT_TOKEN = "synthetic-agent-secret" +AUTH_HEADER = f"PBSAPIToken=pulse-monitor@pbs!pulse-token:{PBS_TOKEN}" + + +def blocks(): + # Nested list-item fences have two leading spaces, which are not shell + # input when copied from the rendered guide. + text = re.sub(r"^ ", "", DOC.read_text(), flags=re.MULTILINE) + return re.findall(r"```bash\n(.*?)```", text, re.DOTALL) + + +def recipe(needle): + matches = [block for block in blocks() if needle in block] + if len(matches) != 1: + raise AssertionError(f"expected one recipe containing {needle!r}") + return matches[0] + + +def fixture_environment(home): + env = dict(os.environ, HOME=str(home)) + # Keep all requests guest-local even if the proof runtime has proxy vars. + for key in list(env): + if key.lower().endswith("_proxy"): + del env[key] + return env + + +def certificate(directory, name): + certificate = directory / f"{name}.pem" + key = directory / f"{name}.key" + subprocess.run([ + "openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", + "-keyout", str(key), "-out", str(certificate), "-days", "1", + "-subj", "/CN=localhost", "-addext", "subjectAltName=DNS:localhost", + ], check=True, capture_output=True) + key.chmod(0o600) + return certificate, key + + +@contextmanager +def server(cert, key, status=200, installer=None): + requests = [] + + class Handler(BaseHTTPRequestHandler): + def do_GET(self): + requests.append((self.path, dict(self.headers))) + self.send_response(status) + self.end_headers() + body = installer if self.path == "/install.sh" else b'{"data":[]}\n' + self.wfile.write(body or b"fixture error\n") + + def log_message(self, *_args): + pass + + http = HTTPServer(("127.0.0.1", 0), Handler) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain(str(cert), str(key)) + http.socket = context.wrap_socket(http.socket, server_side=True) + thread = threading.Thread(target=http.serve_forever, daemon=True) + thread.start() + try: + yield http.server_address[1], requests + finally: + http.shutdown() + http.server_close() + thread.join(timeout=5) + + +class PBSDocsTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.temporary = tempfile.TemporaryDirectory() + cls.cert, cls.key = certificate(Path(cls.temporary.name), "server") + cls.other_cert, _ = certificate(Path(cls.temporary.name), "other") + + @classmethod + def tearDownClass(cls): + cls.temporary.cleanup() + + def test_shipped_copy_and_safe_scope(self): + self.assertEqual(DOC.read_bytes(), (ROOT / "frontend-modern/public/docs/PBS.md").read_bytes()) + text = DOC.read_text() + shell = "\n".join(blocks()) + self.assertNotRegex(shell, r"Authorization:|PBSAPIToken=|PULSE_SETUP_TOKEN=|--token(?:\s|=)|--insecure|curl\s+-[^\s]*k") + self.assertNotIn("/api2/json/version", shell) + self.assertNotIn("disable SSL verification", text) + for phrase in ("both the user and the token", "Manual Token Setup", + "not the **PBS** token", "independently verified", + "An agent is not", "empty list is not proof", + "Do not delete backups", "Do not clear History"): + self.assertIn(phrase, text) + + def test_every_documented_shell_recipe_parses(self): + for command in blocks(): + with self.subTest(command=command): + subprocess.run(["bash", "-n", "-c", command], check=True, capture_output=True) + + def test_new_setup_grants_audit_to_user_and_token(self): + command = recipe("proxmox-backup-manager user create") + with tempfile.TemporaryDirectory() as temporary: + home = Path(temporary) + tool = home / "proxmox-backup-manager" + tool.write_text( + "#!/usr/bin/env python3\nimport json, os, sys\n" + "with open(os.environ['CALLS'], 'a') as f: f.write(json.dumps(sys.argv[1:]) + '\\n')\n" + ) + tool.chmod(0o700) + env = fixture_environment(home) + env.update(PATH=f"{home}:{os.environ['PATH']}", CALLS=str(home / "calls.jsonl")) + subprocess.run(["bash", "-eu", "-c", command], env=env, check=True) + calls = [json.loads(line) for line in (home / "calls.jsonl").read_text().splitlines()] + self.assertEqual(calls, [ + ["user", "create", "pulse-monitor@pbs", "--comment", "Pulse monitoring"], + ["user", "generate-token", "pulse-monitor@pbs", "pulse-token"], + ["acl", "update", "/", "Audit", "--auth-id", "pulse-monitor@pbs"], + ["acl", "update", "/", "Audit", "--auth-id", "pulse-monitor@pbs!pulse-token"], + ]) + # Same permission shapes as the owning implementation, not a claim of + # running proxmox-backup-manager on a real appliance. + source = (ROOT / "internal/api/configapi/setup_script_render.go").read_text() + self.assertIn("proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs", source) + self.assertIn('proxmox-backup-manager acl update / Audit --auth-id "$PULSE_TOKEN_ID"', source) + + def test_private_preparation_preserves_existing_content(self): + for name in ("pbs-agent-token", "pbs-header"): + command = recipe(f'vi "$HOME/.config/pulse/{name}"') + with self.subTest(name=name), tempfile.TemporaryDirectory() as temporary: + home = Path(temporary) + tools = home / "tools" + tools.mkdir() + editor = tools / "vi" + editor.write_text('#!/bin/sh\n[ "$#" = 1 ] && [ -f "$1" ]\n') + editor.chmod(0o700) + env = fixture_environment(home) + env["PATH"] = f"{tools}:{os.environ['PATH']}" + path = home / ".config/pulse" / name + for existing in (False, True): + if existing: + path.write_text("synthetic-existing-content\n") + path.chmod(0o644) + path.parent.chmod(0o755) + subprocess.run(["bash", "-eu", "-c", command], env=env, check=True) + self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600) + self.assertEqual(stat.S_IMODE(path.parent.stat().st_mode), 0o700) + self.assertEqual(path.read_text(), "synthetic-existing-content\n" if existing else "") + + def run_curl(self, home, command, port, ca=None, hostname="localhost"): + private = home / ".config/pulse" + private.mkdir(parents=True, exist_ok=True, mode=0o700) + header = private / "pbs-header" + header.write_text(f"Authorization: {AUTH_HEADER}\n") + header.chmod(0o600) + real_curl = shutil.which("curl") + self.assertIsNotNone(real_curl) + tools = home / "tools" + tools.mkdir(exist_ok=True) + recorder = tools / "curl" + recorder.write_text( + "#!/usr/bin/env python3\nimport json, os, sys\nfrom pathlib import Path\n" + "Path(os.environ['ARGV_RECEIPT']).write_text(json.dumps(sys.argv[1:]))\n" + "os.execv(os.environ['REAL_CURL'], [os.environ['REAL_CURL'], *sys.argv[1:]])\n" + ) + recorder.chmod(0o700) + receipt = home / "argv.json" + env = fixture_environment(home) + env.update(PATH=f"{tools}:{os.environ['PATH']}", REAL_CURL=real_curl, ARGV_RECEIPT=str(receipt)) + command = command.replace("https://pbs.example.com:8007", f"https://{hostname}:{port}") + command = command.replace("https://pulse.example.com", f"https://{hostname}:{port}") + if ca is not None: + shutil.copyfile(ca, private / "pbs-ca.pem") + # This is the guide's optional, independently verified CA-file form. + command = command.replace("curl ", 'curl --cacert "$HOME/.config/pulse/pbs-ca.pem" ', 1) + result = subprocess.run(["bash", "-eu", "-c", command], env=env, capture_output=True, timeout=20) + argv = json.loads(receipt.read_text()) + for secret in (PBS_TOKEN, AGENT_TOKEN): + self.assertNotIn(secret, " ".join(argv)) + self.assertNotIn(secret.encode(), result.stdout + result.stderr) + self.assertNotIn("--insecure", argv) + self.assertNotIn("-k", argv) + return result, argv + + def test_datastore_request_transmits_auth_only_in_header(self): + with tempfile.TemporaryDirectory() as temporary, server(self.cert, self.key) as (port, requests): + result, argv = self.run_curl(Path(temporary), recipe("--fail-with-body"), port, self.cert) + self.assertEqual(result.returncode, 0, result.stderr.decode()) + self.assertIn("--max-time", argv) + self.assertIn("--connect-timeout", argv) + self.assertIn("@" + str(Path(temporary) / ".config/pulse/pbs-header"), argv) + self.assertEqual(requests, [("/api2/json/admin/datastore", { + **requests[0][1], "Authorization": AUTH_HEADER, + })]) + self.assertEqual(result.stdout, b'{"data":[]}\n') + + def test_auth_errors_retain_body_and_fail(self): + for status in (401, 403): + with self.subTest(status=status), tempfile.TemporaryDirectory() as temporary: + with server(self.cert, self.key, status) as (port, requests): + result, _ = self.run_curl(Path(temporary), recipe("--fail-with-body"), port, self.cert) + self.assertEqual(result.returncode, 22, result.stderr.decode()) + self.assertEqual(len(requests), 1) + self.assertEqual(result.stdout, b'{"data":[]}\n') + + def test_tls_rejects_untrusted_wrong_certificate_and_wrong_hostname(self): + for ca, hostname in ((None, "localhost"), (self.other_cert, "localhost"), (self.cert, "127.0.0.1")): + with self.subTest(ca=ca, hostname=hostname), tempfile.TemporaryDirectory() as temporary: + with server(self.cert, self.key) as (port, requests): + result, _ = self.run_curl(Path(temporary), recipe("--fail-with-body"), port, ca, hostname) + self.assertEqual(result.returncode, 60, result.stderr.decode()) + # Authentication is never sent over an unverified channel. + self.assertEqual(requests, []) + + def test_download_and_agent_file_handoff(self): + # The downloaded fixture deliberately cannot install or change a + # service. It verifies only the documented argument/file handoff. + installer = b'''#!/bin/bash +python3 - "$@" <<'PY' +import json, os, stat, sys +from pathlib import Path +args = sys.argv[1:] +path = Path(args[args.index('--token-file') + 1]) +assert stat.S_IMODE(path.stat().st_mode) == 0o600 +assert path.read_text().strip() == os.environ['EXPECTED_AGENT_TOKEN'] +assert os.environ['EXPECTED_AGENT_TOKEN'] not in ' '.join(args) +Path(os.environ['AGENT_RECEIPT']).write_text(json.dumps(args)) +PY +''' + with tempfile.TemporaryDirectory() as temporary, server(self.cert, self.key, installer=installer) as (port, requests): + home = Path(temporary) + result, _ = self.run_curl(home, recipe('--output "$HOME/.config/pulse/pbs-agent-install.sh"'), port, self.cert) + self.assertEqual(result.returncode, 0, result.stderr.decode()) + self.assertEqual(requests[0][0], "/install.sh") + self.assertNotIn("Authorization", requests[0][1]) + token = home / ".config/pulse/pbs-agent-token" + token.write_text(AGENT_TOKEN + "\n") + token.chmod(0o600) + command = recipe('bash "$HOME/.config/pulse/pbs-agent-install.sh"') + env = fixture_environment(home) + env.update(EXPECTED_AGENT_TOKEN=AGENT_TOKEN, AGENT_RECEIPT=str(home / "agent-argv.json")) + result = subprocess.run(["bash", "-eu", "-c", command], env=env, capture_output=True, timeout=10) + self.assertEqual(result.returncode, 0, result.stderr.decode()) + self.assertEqual(json.loads((home / "agent-argv.json").read_text()), [ + "--url", "https://pulse.example.com", "--token-file", str(token), + "--enable-proxmox", "--proxmox-type", "pbs", "--enable-docker=false", + ]) + self.assertNotIn(AGENT_TOKEN.encode(), result.stdout + result.stderr) + actual = (ROOT / "scripts/install.sh").read_text() + self.assertIn('--token-file) TOKEN_FILE_PATH="$2"; shift 2 ;;', actual) + self.assertIn('read_collector_token_file_safely "$TOKEN_FILE_PATH" true', actual) + + def test_failed_installer_download_is_nonzero(self): + with tempfile.TemporaryDirectory() as temporary, server(self.cert, self.key, 403) as (port, _requests): + home = Path(temporary) + result, _ = self.run_curl(home, recipe('--output "$HOME/.config/pulse/pbs-agent-install.sh"'), port, self.cert) + self.assertEqual(result.returncode, 22, result.stderr.decode()) + self.assertFalse((home / ".config/pulse/pbs-agent-install.sh").exists()) + + +if __name__ == "__main__": + unittest.main()