vrr/Pulse
2
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-10-03 04:38:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 17

Make PBS setup and diagnostics credential-safe

Browse source 
Prefer read-only API access, grant Audit to both PBS identities, keep tokens in private files and verify TLS before sending them. Exercise the documented recipes with synthetic tokens, HTTP failures and certificate rejection controls.

Contract-Neutral: Documentation and synthetic recipe checks only; no runtime API, installer, PBS identity or permission contract changes.
Change-source: pulse-maintainer
This commit is contained in:
pulse-triage[bot] 2026-10-01 04:11:04 +01:00
parent 920aa2f27c
commit a5c8b5d597
3 changed files with 582 additions and 140 deletions
Download patch file Download diff file Expand all files Collapse all files

219
docs/PBS.md
View file

@ -8,7 +8,8 @@ Pulse can monitor PBS backups in two ways:
### 1. Direct PBS Connection (Recommended)
Connect directly to your PBS server for full monitoring capabilities:
Connect to the PBS API with a dedicated read-only token. An agent is not
required for these API-backed readings:
**Benefits:**
- ✅ Deduplication factor and storage efficiency stats
@ -29,73 +30,100 @@ If your PVE cluster has PBS storage configured, Pulse automatically fetches back
- ❌ Can be slow for encrypted PBS storage
- ❌ Limited metadata per backup
**Recommendation:** If you see a banner in the Recovery page (formerly Backups) suggesting you add PBS directly, following this guide will significantly improve your monitoring experience.
**Recommendation:** Start with a direct API connection when you need PBS
datastore, job or server status beyond the PVE passthrough. Add a host agent
only for extra local telemetry such as SMART and temperatures; see
[Agent Security](AGENT_SECURITY.md#proxmox-deployment-choices).
---
## Setting Up Direct PBS Connection
### Method 1: Unified Agent Install (Recommended for Bare Metal)
### Method 1: API-Only Connection (Recommended)
Install the unified agent directly on your PBS server for automatic setup:
Use a dedicated PBS monitoring user and token, not a root password. On a new
setup, run these commands in an administrator shell **on the PBS server**:
```bash
# Run on your PBS server
curl -fsSL http://<pulse-ip>:7655/install.sh | \
sudo bash -s -- --url http://<pulse-ip>:7655 --token <api-token> --enable-proxmox --proxmox-type pbs
```
The agent will:
1. Detect it's running on a PBS server
2. Create a `pulse-monitor@pbs` user with read-only access
3. Generate an API token
4. Register the PBS node with Pulse automatically
### Method 2: API-Only Setup Script (Best for PBS in Containers) ⭐
Use this when you can run a command on the PBS host but do not want to install the agent.
From Pulse's Settings page:
1. Go to **Settings → Infrastructure**.
2. Click **Add infrastructure**.
3. Choose **Proxmox Backup Server**.
4. Use the API-only setup path and enter your PBS server's URL.
5. Click copy to get the setup command.
6. Run the command on your PBS server.
Example (what the UI generates):
```bash
curl -fsSL "http://<pulse-ip>:7655/api/setup-script?type=pbs&host=https://<pbs-ip>:8007&pulse_url=http://<pulse-ip>:7655" | { if [ "$(id -u)" -eq 0 ]; then PULSE_SETUP_TOKEN="<setup-token>" bash; elif command -v sudo >/dev/null 2>&1; then sudo env PULSE_SETUP_TOKEN="<setup-token>" bash; else echo "Root privileges required. Run as root (su -) and retry." >&2; exit 1; fi; }
```
Pulse generates that full command for you from **Settings → Infrastructure**, including
the one-time setup token. The script creates a `pulse-monitor@pbs` user,
generates a scoped API token, and registers the server with Pulse.
> **Note**: API-only mode does not include temperature monitoring or AI command execution. Use **Agent Install** for full functionality.
> **Tip**: The installer now auto-detects Proxmox mode (`pve` or `pbs`) when possible, but keeping `--proxmox-type pbs` explicit is recommended for predictable PBS onboarding.
### Method 3: Manual Token Creation
If you prefer manual setup:
```bash
# SSH into your PBS server
# 1. Create a dedicated monitoring user
proxmox-backup-manager user create pulse-monitor@pbs --comment "Pulse monitoring"
# 2. Grant read-only access (Audit role)
proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs
# 3. Generate an API token (save the output!)
proxmox-backup-manager user generate-token pulse-monitor@pbs pulse-token
proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs
proxmox-backup-manager acl update / Audit --auth-id 'pulse-monitor@pbs!pulse-token'
```
Copy the token value and enter it in Pulse:
- **Token ID:** `pulse-monitor@pbs!pulse-token`
- **Token Value:** The UUID shown after running the command
PBS tokens have their own permissions, limited by their owning user's
permissions. Grant `Audit` to **both the user and the token**. If either already
exists, inspect and reuse it instead of deleting it or rerunning token creation.
Do not disable privilege separation or grant `Admin` to work around a missing
permission.
The token secret is shown once. Save it privately and enter it only in Pulse's
**Token Value** field; do not paste it into a shell command, URL, screenshot or
issue report. Keep any terminal recording containing that output private.
In Pulse:
1. Open **Settings → Infrastructure → Add infrastructure** and choose
**Proxmox Backup Server**.
2. Enter the PBS HTTPS URL, normally `https://pbs.example.com:8007`.
3. Select **API Token** and **Manual Token Setup**.
4. Enter **Token ID** `pulse-monitor@pbs!pulse-token` and its secret in
**Token Value**.
5. Keep certificate verification enabled. For a self-signed certificate, use
an independently verified **SSL Fingerprint**; see the TLS guidance below.
6. Test and save the connection. Check that the expected datastores and backups
are visible, not just that the connection test succeeds.
### Method 2: Optional Host Agent
Use this only when you also need host-local telemetry. The Linux agent normally
runs as root; it is not required to fix API authentication or missing backup
permissions. Review [Agent Security](AGENT_SECURITY.md) first.
Create a separate Pulse token in **API Access**, using the **Agent host** preset
for reporting, configuration reads and agent management. This is a **Pulse**
token, not the **PBS** token used above. Do not enable command execution just
for monitoring.
On the PBS host, enter an administrator root shell before preparing the private
file below. In the editor, save only the Pulse agent token, with no header or
quotes; never put the secret in a command argument:
```bash
umask 077
mkdir -p "$HOME/.config/pulse"
chmod 700 "$HOME/.config/pulse"
touch "$HOME/.config/pulse/pbs-agent-token"
chmod 600 "$HOME/.config/pulse/pbs-agent-token"
vi "$HOME/.config/pulse/pbs-agent-token"
```
Download the agent installer from **your Pulse server's HTTPS address**, then
inspect the saved script before running it. Replace the example Pulse URL in
both commands. Stop if the download fails; run the next command only after a
successful download and inspection. Do not substitute GitHub's top-level `install.sh`: that installs
the Pulse server, not the agent.
```bash
curl --fail --silent --show-error \
--output "$HOME/.config/pulse/pbs-agent-install.sh" \
https://pulse.example.com/install.sh
```
```bash
bash "$HOME/.config/pulse/pbs-agent-install.sh" \
--url https://pulse.example.com \
--token-file "$HOME/.config/pulse/pbs-agent-token" \
--enable-proxmox --proxmox-type pbs --enable-docker=false
```
Do not bypass certificate checks to fetch or run the installer. Use a trusted
Pulse certificate or a CA file you verified separately. See
[Unified Agent Setup](UNIFIED_AGENT.md) for installer trust and other profiles.
After installation, check a fresh agent report and the host's
`pulse-agent --version`; installation or hardware capacity alone does not prove
that every CPU, memory or History reading is available. Keep the token file
private and remove the bootstrap copy when no longer needed; do not remove the
installed agent's runtime credential.
---
@ -133,18 +161,65 @@ If you have multiple PBS servers, add each one separately in Settings. Pulse wil
### "Connection Failed" Error
1. **Check URL:** Ensure the PBS URL is correct (default port is 8007)
- Format: `https://pbs.example.com:8007`
Check from the Pulse host or container's network, if possible. A request from
another machine does not prove that Pulse can reach PBS.
2. **Verify token:** Test authentication:
```bash
curl -sk -H "Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token:YOUR_TOKEN" \
https://your-pbs:8007/api2/json/version
```
- **Address/network:** use the PBS HTTPS URL and port `8007`; check DNS, routing
and the firewall before changing credentials.
- **TLS:** keep verification enabled. Use a certificate trusted by the Pulse
runtime, or set its **SSL Fingerprint** after verifying the SHA-256 value
through the PBS console or another already-trusted administrative channel.
Do not accept a fingerprint solely from the failed connection. For example,
on the PBS server you can inspect its public certificate (not its private key):
3. **Network access:** Ensure Pulse can reach PBS on port 8007
```bash
openssl x509 -in /etc/proxmox-backup/proxy.pem -noout -fingerprint -sha256
```
4. **SSL verification:** If using self-signed certificates, disable SSL verification in the node settings
- **Authentication/permissions:** test a datastore request, not just `/version`.
Prepare a private header file on the machine running curl:
```bash
umask 077
mkdir -p "$HOME/.config/pulse"
chmod 700 "$HOME/.config/pulse"
touch "$HOME/.config/pulse/pbs-header"
chmod 600 "$HOME/.config/pulse/pbs-header"
vi "$HOME/.config/pulse/pbs-header"
```
In the editor, save this line, replacing `<pbs-token-secret>` with the secret
for the PBS token being tested:
```text
Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token:<pbs-token-secret>
```
Then use curl 7.76 or later, with your PBS hostname:
```bash
curl --fail-with-body --silent --show-error --connect-timeout 5 --max-time 15 \
--header "@$HOME/.config/pulse/pbs-header" \
https://pbs.example.com:8007/api2/json/admin/datastore
```
For curl with a private CA or self-signed certificate, add
`--cacert "$HOME/.config/pulse/pbs-ca.pem"` using a public certificate obtained
and verified through a trusted channel. Its hostname must still match. Do not
use `--insecure` or `-k`.
`401` means authentication was rejected; `403` means the requested access was
refused. Both return a non-zero curl exit while retaining the error body.
Inspect the existing token and both `Audit` grants before replacing anything.
A `200` response listing the expected datastore establishes access to that
list, not successful collection of every backup, job or History graph. An
empty list is not proof that all permissions are correct.
Keep the header file outside shared repositories and diagnostics. Do not share
verbose/trace curl output, full infrastructure responses, token values or
private keys. If help is needed, provide only the HTTP status, relevant redacted
error and which expected reading is missing. Do not clear History or recreate a
working connection to make missing data look resolved.
### Slow Backup Loading
@ -155,10 +230,12 @@ If you notice slow loading for PBS storage accessed via PVE:
### Duplicate Backups
If you see the same backup twice:
- This shouldn't happen—Pulse deduplicates by VMID and timestamp
- If it does occur, the direct PBS version takes priority
- Check console for debug logs: `localStorage.setItem('debug-pmg', 'true')`
Check the source, datastore, namespace, guest type/ID and backup time on each
entry. Independent PVE installations can reuse a guest ID; matching VMIDs alone
are not proof of a duplicate. Keep both records while checking their origin.
If the same backup remains listed twice, report those redacted distinctions and
whether each entry came from direct PBS or PVE passthrough. Do not delete backups
or change retention to hide a display problem.
---
@ -169,7 +246,9 @@ In the Recovery view, PBS backups show a data source indicator:
- **"PBS"** badge alone = Direct PBS connection (full data)
- **"PBS via PVE"** = Passthrough via PVE storage (limited data)
Adding your PBS server directly will remove the "via PVE" indicator and unlock full monitoring capabilities.
When the same backup is reconciled across both sources, Pulse prefers the
direct PBS observation. Check the actual source and expected readings after
adding the connection; the presence of a badge alone is not collection proof.
---

219
frontend-modern/public/docs/PBS.md
View file

@ -8,7 +8,8 @@ Pulse can monitor PBS backups in two ways:
### 1. Direct PBS Connection (Recommended)
Connect directly to your PBS server for full monitoring capabilities:
Connect to the PBS API with a dedicated read-only token. An agent is not
required for these API-backed readings:
**Benefits:**
- ✅ Deduplication factor and storage efficiency stats
@ -29,73 +30,100 @@ If your PVE cluster has PBS storage configured, Pulse automatically fetches back
- ❌ Can be slow for encrypted PBS storage
- ❌ Limited metadata per backup
**Recommendation:** If you see a banner in the Recovery page (formerly Backups) suggesting you add PBS directly, following this guide will significantly improve your monitoring experience.
**Recommendation:** Start with a direct API connection when you need PBS
datastore, job or server status beyond the PVE passthrough. Add a host agent
only for extra local telemetry such as SMART and temperatures; see
[Agent Security](AGENT_SECURITY.md#proxmox-deployment-choices).
---
## Setting Up Direct PBS Connection
### Method 1: Unified Agent Install (Recommended for Bare Metal)
### Method 1: API-Only Connection (Recommended)
Install the unified agent directly on your PBS server for automatic setup:
Use a dedicated PBS monitoring user and token, not a root password. On a new
setup, run these commands in an administrator shell **on the PBS server**:
```bash
# Run on your PBS server
curl -fsSL http://<pulse-ip>:7655/install.sh | \
sudo bash -s -- --url http://<pulse-ip>:7655 --token <api-token> --enable-proxmox --proxmox-type pbs
```
The agent will:
1. Detect it's running on a PBS server
2. Create a `pulse-monitor@pbs` user with read-only access
3. Generate an API token
4. Register the PBS node with Pulse automatically
### Method 2: API-Only Setup Script (Best for PBS in Containers) ⭐
Use this when you can run a command on the PBS host but do not want to install the agent.
From Pulse's Settings page:
1. Go to **Settings → Infrastructure**.
2. Click **Add infrastructure**.
3. Choose **Proxmox Backup Server**.
4. Use the API-only setup path and enter your PBS server's URL.
5. Click copy to get the setup command.
6. Run the command on your PBS server.
Example (what the UI generates):
```bash
curl -fsSL "http://<pulse-ip>:7655/api/setup-script?type=pbs&host=https://<pbs-ip>:8007&pulse_url=http://<pulse-ip>:7655" | { if [ "$(id -u)" -eq 0 ]; then PULSE_SETUP_TOKEN="<setup-token>" bash; elif command -v sudo >/dev/null 2>&1; then sudo env PULSE_SETUP_TOKEN="<setup-token>" bash; else echo "Root privileges required. Run as root (su -) and retry." >&2; exit 1; fi; }
```
Pulse generates that full command for you from **Settings → Infrastructure**, including
the one-time setup token. The script creates a `pulse-monitor@pbs` user,
generates a scoped API token, and registers the server with Pulse.
> **Note**: API-only mode does not include temperature monitoring or AI command execution. Use **Agent Install** for full functionality.
> **Tip**: The installer now auto-detects Proxmox mode (`pve` or `pbs`) when possible, but keeping `--proxmox-type pbs` explicit is recommended for predictable PBS onboarding.
### Method 3: Manual Token Creation
If you prefer manual setup:
```bash
# SSH into your PBS server
# 1. Create a dedicated monitoring user
proxmox-backup-manager user create pulse-monitor@pbs --comment "Pulse monitoring"
# 2. Grant read-only access (Audit role)
proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs
# 3. Generate an API token (save the output!)
proxmox-backup-manager user generate-token pulse-monitor@pbs pulse-token
proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs
proxmox-backup-manager acl update / Audit --auth-id 'pulse-monitor@pbs!pulse-token'
```
Copy the token value and enter it in Pulse:
- **Token ID:** `pulse-monitor@pbs!pulse-token`
- **Token Value:** The UUID shown after running the command
PBS tokens have their own permissions, limited by their owning user's
permissions. Grant `Audit` to **both the user and the token**. If either already
exists, inspect and reuse it instead of deleting it or rerunning token creation.
Do not disable privilege separation or grant `Admin` to work around a missing
permission.
The token secret is shown once. Save it privately and enter it only in Pulse's
**Token Value** field; do not paste it into a shell command, URL, screenshot or
issue report. Keep any terminal recording containing that output private.
In Pulse:
1. Open **Settings → Infrastructure → Add infrastructure** and choose
**Proxmox Backup Server**.
2. Enter the PBS HTTPS URL, normally `https://pbs.example.com:8007`.
3. Select **API Token** and **Manual Token Setup**.
4. Enter **Token ID** `pulse-monitor@pbs!pulse-token` and its secret in
**Token Value**.
5. Keep certificate verification enabled. For a self-signed certificate, use
an independently verified **SSL Fingerprint**; see the TLS guidance below.
6. Test and save the connection. Check that the expected datastores and backups
are visible, not just that the connection test succeeds.
### Method 2: Optional Host Agent
Use this only when you also need host-local telemetry. The Linux agent normally
runs as root; it is not required to fix API authentication or missing backup
permissions. Review [Agent Security](AGENT_SECURITY.md) first.
Create a separate Pulse token in **API Access**, using the **Agent host** preset
for reporting, configuration reads and agent management. This is a **Pulse**
token, not the **PBS** token used above. Do not enable command execution just
for monitoring.
On the PBS host, enter an administrator root shell before preparing the private
file below. In the editor, save only the Pulse agent token, with no header or
quotes; never put the secret in a command argument:
```bash
umask 077
mkdir -p "$HOME/.config/pulse"
chmod 700 "$HOME/.config/pulse"
touch "$HOME/.config/pulse/pbs-agent-token"
chmod 600 "$HOME/.config/pulse/pbs-agent-token"
vi "$HOME/.config/pulse/pbs-agent-token"
```
Download the agent installer from **your Pulse server's HTTPS address**, then
inspect the saved script before running it. Replace the example Pulse URL in
both commands. Stop if the download fails; run the next command only after a
successful download and inspection. Do not substitute GitHub's top-level `install.sh`: that installs
the Pulse server, not the agent.
```bash
curl --fail --silent --show-error \
--output "$HOME/.config/pulse/pbs-agent-install.sh" \
https://pulse.example.com/install.sh
```
```bash
bash "$HOME/.config/pulse/pbs-agent-install.sh" \
--url https://pulse.example.com \
--token-file "$HOME/.config/pulse/pbs-agent-token" \
--enable-proxmox --proxmox-type pbs --enable-docker=false
```
Do not bypass certificate checks to fetch or run the installer. Use a trusted
Pulse certificate or a CA file you verified separately. See
[Unified Agent Setup](UNIFIED_AGENT.md) for installer trust and other profiles.
After installation, check a fresh agent report and the host's
`pulse-agent --version`; installation or hardware capacity alone does not prove
that every CPU, memory or History reading is available. Keep the token file
private and remove the bootstrap copy when no longer needed; do not remove the
installed agent's runtime credential.
---
@ -133,18 +161,65 @@ If you have multiple PBS servers, add each one separately in Settings. Pulse wil
### "Connection Failed" Error
1. **Check URL:** Ensure the PBS URL is correct (default port is 8007)
- Format: `https://pbs.example.com:8007`
Check from the Pulse host or container's network, if possible. A request from
another machine does not prove that Pulse can reach PBS.
2. **Verify token:** Test authentication:
```bash
curl -sk -H "Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token:YOUR_TOKEN" \
https://your-pbs:8007/api2/json/version
```
- **Address/network:** use the PBS HTTPS URL and port `8007`; check DNS, routing
and the firewall before changing credentials.
- **TLS:** keep verification enabled. Use a certificate trusted by the Pulse
runtime, or set its **SSL Fingerprint** after verifying the SHA-256 value
through the PBS console or another already-trusted administrative channel.
Do not accept a fingerprint solely from the failed connection. For example,
on the PBS server you can inspect its public certificate (not its private key):
3. **Network access:** Ensure Pulse can reach PBS on port 8007
```bash
openssl x509 -in /etc/proxmox-backup/proxy.pem -noout -fingerprint -sha256
```
4. **SSL verification:** If using self-signed certificates, disable SSL verification in the node settings
- **Authentication/permissions:** test a datastore request, not just `/version`.
Prepare a private header file on the machine running curl:
```bash
umask 077
mkdir -p "$HOME/.config/pulse"
chmod 700 "$HOME/.config/pulse"
touch "$HOME/.config/pulse/pbs-header"
chmod 600 "$HOME/.config/pulse/pbs-header"
vi "$HOME/.config/pulse/pbs-header"
```
In the editor, save this line, replacing `<pbs-token-secret>` with the secret
for the PBS token being tested:
```text
Authorization: PBSAPIToken=pulse-monitor@pbs!pulse-token:<pbs-token-secret>
```
Then use curl 7.76 or later, with your PBS hostname:
```bash
curl --fail-with-body --silent --show-error --connect-timeout 5 --max-time 15 \
--header "@$HOME/.config/pulse/pbs-header" \
https://pbs.example.com:8007/api2/json/admin/datastore
```
For curl with a private CA or self-signed certificate, add
`--cacert "$HOME/.config/pulse/pbs-ca.pem"` using a public certificate obtained
and verified through a trusted channel. Its hostname must still match. Do not
use `--insecure` or `-k`.
`401` means authentication was rejected; `403` means the requested access was
refused. Both return a non-zero curl exit while retaining the error body.
Inspect the existing token and both `Audit` grants before replacing anything.
A `200` response listing the expected datastore establishes access to that
list, not successful collection of every backup, job or History graph. An
empty list is not proof that all permissions are correct.
Keep the header file outside shared repositories and diagnostics. Do not share
verbose/trace curl output, full infrastructure responses, token values or
private keys. If help is needed, provide only the HTTP status, relevant redacted
error and which expected reading is missing. Do not clear History or recreate a
working connection to make missing data look resolved.
### Slow Backup Loading
@ -155,10 +230,12 @@ If you notice slow loading for PBS storage accessed via PVE:
### Duplicate Backups
If you see the same backup twice:
- This shouldn't happen—Pulse deduplicates by VMID and timestamp
- If it does occur, the direct PBS version takes priority
- Check console for debug logs: `localStorage.setItem('debug-pmg', 'true')`
Check the source, datastore, namespace, guest type/ID and backup time on each
entry. Independent PVE installations can reuse a guest ID; matching VMIDs alone
are not proof of a duplicate. Keep both records while checking their origin.
If the same backup remains listed twice, report those redacted distinctions and
whether each entry came from direct PBS or PVE passthrough. Do not delete backups
or change retention to hide a display problem.
---
@ -169,7 +246,9 @@ In the Recovery view, PBS backups show a data source indicator:
- **"PBS"** badge alone = Direct PBS connection (full data)
- **"PBS via PVE"** = Passthrough via PVE storage (limited data)
Adding your PBS server directly will remove the "via PVE" indicator and unlock full monitoring capabilities.
When the same backup is reconciled across both sources, Pulse prefers the
direct PBS observation. Check the actual source and expected readings after
adding the connection; the presence of a badge alone is not collection proof.
---

284
scripts/tests/test_pbs_docs.py Normal file
View file

@ -0,0 +1,284 @@
#!/usr/bin/env python3
"""Execute the PBS guide's recipes with synthetic tokens and guest-local TLS.
No real PBS/Pulse instance, installer, service or credential is used. Run the
loopback checks with pulse-worker-source-proof, not on the worker host.
"""
from contextlib import contextmanager
from http.server import BaseHTTPRequestHandler, HTTPServer
import json
import os
from pathlib import Path
import re
import shutil
import ssl
import stat
import subprocess
import tempfile
import threading
import unittest
ROOT = Path(__file__).resolve().parents[2]
DOC = ROOT / "docs/PBS.md"
PBS_TOKEN = "synthetic-pbs-secret"
AGENT_TOKEN = "synthetic-agent-secret"
AUTH_HEADER = f"PBSAPIToken=pulse-monitor@pbs!pulse-token:{PBS_TOKEN}"
def blocks():
# Nested list-item fences have two leading spaces, which are not shell
# input when copied from the rendered guide.
text = re.sub(r"^ ", "", DOC.read_text(), flags=re.MULTILINE)
return re.findall(r"```bash\n(.*?)```", text, re.DOTALL)
def recipe(needle):
matches = [block for block in blocks() if needle in block]
if len(matches) != 1:
raise AssertionError(f"expected one recipe containing {needle!r}")
return matches[0]
def fixture_environment(home):
env = dict(os.environ, HOME=str(home))
# Keep all requests guest-local even if the proof runtime has proxy vars.
for key in list(env):
if key.lower().endswith("_proxy"):
del env[key]
return env
def certificate(directory, name):
certificate = directory / f"{name}.pem"
key = directory / f"{name}.key"
subprocess.run([
"openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes",
"-keyout", str(key), "-out", str(certificate), "-days", "1",
"-subj", "/CN=localhost", "-addext", "subjectAltName=DNS:localhost",
], check=True, capture_output=True)
key.chmod(0o600)
return certificate, key
@contextmanager
def server(cert, key, status=200, installer=None):
requests = []
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
requests.append((self.path, dict(self.headers)))
self.send_response(status)
self.end_headers()
body = installer if self.path == "/install.sh" else b'{"data":[]}\n'
self.wfile.write(body or b"fixture error\n")
def log_message(self, *_args):
pass
http = HTTPServer(("127.0.0.1", 0), Handler)
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(str(cert), str(key))
http.socket = context.wrap_socket(http.socket, server_side=True)
thread = threading.Thread(target=http.serve_forever, daemon=True)
thread.start()
try:
yield http.server_address[1], requests
finally:
http.shutdown()
http.server_close()
thread.join(timeout=5)
class PBSDocsTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.temporary = tempfile.TemporaryDirectory()
cls.cert, cls.key = certificate(Path(cls.temporary.name), "server")
cls.other_cert, _ = certificate(Path(cls.temporary.name), "other")
@classmethod
def tearDownClass(cls):
cls.temporary.cleanup()
def test_shipped_copy_and_safe_scope(self):
self.assertEqual(DOC.read_bytes(), (ROOT / "frontend-modern/public/docs/PBS.md").read_bytes())
text = DOC.read_text()
shell = "\n".join(blocks())
self.assertNotRegex(shell, r"Authorization:|PBSAPIToken=|PULSE_SETUP_TOKEN=|--token(?:\s|=)|--insecure|curl\s+-[^\s]*k")
self.assertNotIn("/api2/json/version", shell)
self.assertNotIn("disable SSL verification", text)
for phrase in ("both the user and the token", "Manual Token Setup",
"not the **PBS** token", "independently verified",
"An agent is not", "empty list is not proof",
"Do not delete backups", "Do not clear History"):
self.assertIn(phrase, text)
def test_every_documented_shell_recipe_parses(self):
for command in blocks():
with self.subTest(command=command):
subprocess.run(["bash", "-n", "-c", command], check=True, capture_output=True)
def test_new_setup_grants_audit_to_user_and_token(self):
command = recipe("proxmox-backup-manager user create")
with tempfile.TemporaryDirectory() as temporary:
home = Path(temporary)
tool = home / "proxmox-backup-manager"
tool.write_text(
"#!/usr/bin/env python3\nimport json, os, sys\n"
"with open(os.environ['CALLS'], 'a') as f: f.write(json.dumps(sys.argv[1:]) + '\\n')\n"
)
tool.chmod(0o700)
env = fixture_environment(home)
env.update(PATH=f"{home}:{os.environ['PATH']}", CALLS=str(home / "calls.jsonl"))
subprocess.run(["bash", "-eu", "-c", command], env=env, check=True)
calls = [json.loads(line) for line in (home / "calls.jsonl").read_text().splitlines()]
self.assertEqual(calls, [
["user", "create", "pulse-monitor@pbs", "--comment", "Pulse monitoring"],
["user", "generate-token", "pulse-monitor@pbs", "pulse-token"],
["acl", "update", "/", "Audit", "--auth-id", "pulse-monitor@pbs"],
["acl", "update", "/", "Audit", "--auth-id", "pulse-monitor@pbs!pulse-token"],
])
# Same permission shapes as the owning implementation, not a claim of
# running proxmox-backup-manager on a real appliance.
source = (ROOT / "internal/api/configapi/setup_script_render.go").read_text()
self.assertIn("proxmox-backup-manager acl update / Audit --auth-id pulse-monitor@pbs", source)
self.assertIn('proxmox-backup-manager acl update / Audit --auth-id "$PULSE_TOKEN_ID"', source)
def test_private_preparation_preserves_existing_content(self):
for name in ("pbs-agent-token", "pbs-header"):
command = recipe(f'vi "$HOME/.config/pulse/{name}"')
with self.subTest(name=name), tempfile.TemporaryDirectory() as temporary:
home = Path(temporary)
tools = home / "tools"
tools.mkdir()
editor = tools / "vi"
editor.write_text('#!/bin/sh\n[ "$#" = 1 ] && [ -f "$1" ]\n')
editor.chmod(0o700)
env = fixture_environment(home)
env["PATH"] = f"{tools}:{os.environ['PATH']}"
path = home / ".config/pulse" / name
for existing in (False, True):
if existing:
path.write_text("synthetic-existing-content\n")
path.chmod(0o644)
path.parent.chmod(0o755)
subprocess.run(["bash", "-eu", "-c", command], env=env, check=True)
self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600)
self.assertEqual(stat.S_IMODE(path.parent.stat().st_mode), 0o700)
self.assertEqual(path.read_text(), "synthetic-existing-content\n" if existing else "")
def run_curl(self, home, command, port, ca=None, hostname="localhost"):
private = home / ".config/pulse"
private.mkdir(parents=True, exist_ok=True, mode=0o700)
header = private / "pbs-header"
header.write_text(f"Authorization: {AUTH_HEADER}\n")
header.chmod(0o600)
real_curl = shutil.which("curl")
self.assertIsNotNone(real_curl)
tools = home / "tools"
tools.mkdir(exist_ok=True)
recorder = tools / "curl"
recorder.write_text(
"#!/usr/bin/env python3\nimport json, os, sys\nfrom pathlib import Path\n"
"Path(os.environ['ARGV_RECEIPT']).write_text(json.dumps(sys.argv[1:]))\n"
"os.execv(os.environ['REAL_CURL'], [os.environ['REAL_CURL'], *sys.argv[1:]])\n"
)
recorder.chmod(0o700)
receipt = home / "argv.json"
env = fixture_environment(home)
env.update(PATH=f"{tools}:{os.environ['PATH']}", REAL_CURL=real_curl, ARGV_RECEIPT=str(receipt))
command = command.replace("https://pbs.example.com:8007", f"https://{hostname}:{port}")
command = command.replace("https://pulse.example.com", f"https://{hostname}:{port}")
if ca is not None:
shutil.copyfile(ca, private / "pbs-ca.pem")
# This is the guide's optional, independently verified CA-file form.
command = command.replace("curl ", 'curl --cacert "$HOME/.config/pulse/pbs-ca.pem" ', 1)
result = subprocess.run(["bash", "-eu", "-c", command], env=env, capture_output=True, timeout=20)
argv = json.loads(receipt.read_text())
for secret in (PBS_TOKEN, AGENT_TOKEN):
self.assertNotIn(secret, " ".join(argv))
self.assertNotIn(secret.encode(), result.stdout + result.stderr)
self.assertNotIn("--insecure", argv)
self.assertNotIn("-k", argv)
return result, argv
def test_datastore_request_transmits_auth_only_in_header(self):
with tempfile.TemporaryDirectory() as temporary, server(self.cert, self.key) as (port, requests):
result, argv = self.run_curl(Path(temporary), recipe("--fail-with-body"), port, self.cert)
self.assertEqual(result.returncode, 0, result.stderr.decode())
self.assertIn("--max-time", argv)
self.assertIn("--connect-timeout", argv)
self.assertIn("@" + str(Path(temporary) / ".config/pulse/pbs-header"), argv)
self.assertEqual(requests, [("/api2/json/admin/datastore", {
**requests[0][1], "Authorization": AUTH_HEADER,
})])
self.assertEqual(result.stdout, b'{"data":[]}\n')
def test_auth_errors_retain_body_and_fail(self):
for status in (401, 403):
with self.subTest(status=status), tempfile.TemporaryDirectory() as temporary:
with server(self.cert, self.key, status) as (port, requests):
result, _ = self.run_curl(Path(temporary), recipe("--fail-with-body"), port, self.cert)
self.assertEqual(result.returncode, 22, result.stderr.decode())
self.assertEqual(len(requests), 1)
self.assertEqual(result.stdout, b'{"data":[]}\n')
def test_tls_rejects_untrusted_wrong_certificate_and_wrong_hostname(self):
for ca, hostname in ((None, "localhost"), (self.other_cert, "localhost"), (self.cert, "127.0.0.1")):
with self.subTest(ca=ca, hostname=hostname), tempfile.TemporaryDirectory() as temporary:
with server(self.cert, self.key) as (port, requests):
result, _ = self.run_curl(Path(temporary), recipe("--fail-with-body"), port, ca, hostname)
self.assertEqual(result.returncode, 60, result.stderr.decode())
# Authentication is never sent over an unverified channel.
self.assertEqual(requests, [])
def test_download_and_agent_file_handoff(self):
# The downloaded fixture deliberately cannot install or change a
# service. It verifies only the documented argument/file handoff.
installer = b'''#!/bin/bash
python3 - "$@" <<'PY'
import json, os, stat, sys
from pathlib import Path
args = sys.argv[1:]
path = Path(args[args.index('--token-file') + 1])
assert stat.S_IMODE(path.stat().st_mode) == 0o600
assert path.read_text().strip() == os.environ['EXPECTED_AGENT_TOKEN']
assert os.environ['EXPECTED_AGENT_TOKEN'] not in ' '.join(args)
Path(os.environ['AGENT_RECEIPT']).write_text(json.dumps(args))
PY
'''
with tempfile.TemporaryDirectory() as temporary, server(self.cert, self.key, installer=installer) as (port, requests):
home = Path(temporary)
result, _ = self.run_curl(home, recipe('--output "$HOME/.config/pulse/pbs-agent-install.sh"'), port, self.cert)
self.assertEqual(result.returncode, 0, result.stderr.decode())
self.assertEqual(requests[0][0], "/install.sh")
self.assertNotIn("Authorization", requests[0][1])
token = home / ".config/pulse/pbs-agent-token"
token.write_text(AGENT_TOKEN + "\n")
token.chmod(0o600)
command = recipe('bash "$HOME/.config/pulse/pbs-agent-install.sh"')
env = fixture_environment(home)
env.update(EXPECTED_AGENT_TOKEN=AGENT_TOKEN, AGENT_RECEIPT=str(home / "agent-argv.json"))
result = subprocess.run(["bash", "-eu", "-c", command], env=env, capture_output=True, timeout=10)
self.assertEqual(result.returncode, 0, result.stderr.decode())
self.assertEqual(json.loads((home / "agent-argv.json").read_text()), [
"--url", "https://pulse.example.com", "--token-file", str(token),
"--enable-proxmox", "--proxmox-type", "pbs", "--enable-docker=false",
])
self.assertNotIn(AGENT_TOKEN.encode(), result.stdout + result.stderr)
actual = (ROOT / "scripts/install.sh").read_text()
self.assertIn('--token-file) TOKEN_FILE_PATH="$2"; shift 2 ;;', actual)
self.assertIn('read_collector_token_file_safely "$TOKEN_FILE_PATH" true', actual)
def test_failed_installer_download_is_nonzero(self):
with tempfile.TemporaryDirectory() as temporary, server(self.cert, self.key, 403) as (port, _requests):
home = Path(temporary)
result, _ = self.run_curl(home, recipe('--output "$HOME/.config/pulse/pbs-agent-install.sh"'), port, self.cert)
self.assertEqual(result.returncode, 22, result.stderr.decode())
self.assertFalse((home / ".config/pulse/pbs-agent-install.sh").exists())
if __name__ == "__main__":
unittest.main()