The Settings page was telling systemd/bare metal users to run install.sh
for upgrades, which is wrong - install.sh is for fresh installations only
and does nothing if Pulse is already installed.
Changes:
- Updated upgrade instructions to mention built-in "Install Update" button
- Added correct manual upgrade steps (download tarball, stop service, extract, start)
- Removed misleading "run install.sh" instruction
This fixes a critical UX issue where users would run install.sh and think
nothing happened, when they should either:
1. Use the built-in automatic update feature (Install Update button)
2. Manually download and extract the new binary
Related files:
- frontend-modern/src/components/Settings/Settings.tsx:4052-4072
Modern Proxmox LXC containers (cgroup v2 + systemd) don't expose the CTID
inside the guest namespace. The auto-detection in DetectLXCCTID() works
for older LXC setups and when hostname is numeric, but fails for most
production containers where users set custom hostnames.
Changes:
- Added PULSE_LXC_CTID environment variable override in router.go:490-495
- Graceful fallback: auto-detect first, then check env var, then show placeholder
- UI already handles missing CTID by showing "pct exec <ctid>" placeholder
This provides a robust solution for thousands of users:
- Stock Proxmox LXC: Shows `pct exec <ctid>` placeholder (user substitutes manually)
- Custom hostname containers: Can set PULSE_LXC_CTID=171 in compose/systemd
- Numeric hostname containers: Auto-detected (backwards compatible)
Related: FirstRunSetup.tsx already has graceful fallback (line 336-339)
The "SSH Fallback" label was confusing to users. Changed to "Proxy (SSH)"
to make it clearer that the proxy is using SSH to collect temperature data
from cluster nodes.
This appears in the Capabilities column on Settings → Nodes when:
- Temperature monitoring is enabled
- Socket proxy is not available/healthy
- HTTPS proxy is not available/reachable
- Added DetectDockerContainerName() to detect container name from hostname
- Extended /api/security/status to expose dockerContainerName field
- Updated FirstRunSetup to show actual container name when detected:
* Before: 'docker exec <container-name> cat /data/.bootstrap_token'
* After: 'docker exec pulse cat /data/.bootstrap_token'
This reduces friction for users - no need to look up the container name.
Works when Docker container is named (--name flag), falls back to
placeholder for auto-generated container IDs.
- Added DetectLXCCTID() to internal/system/container.go to detect Proxmox container ID
- Extended /api/security/status to expose inContainer and lxcCtid fields
- Updated FirstRunSetup to show most relevant command based on detected environment:
* LXC with CTID: Shows 'pct exec 171 -- cat /etc/pulse/.bootstrap_token'
* Docker: Shows 'docker exec <container-name> cat /data/.bootstrap_token'
* Bare metal: Shows 'cat /etc/pulse/.bootstrap_token'
- Collapsed alternative methods behind 'Show other retrieval methods' button
This addresses user feedback that showing all options was overwhelming.
Now users see the command most likely to work for their setup first,
with alternatives hidden but still accessible.
The installer was adding node hostnames (and accidentally the header "Name")
to allowed_nodes in addition to IPs. This caused:
1. Invalid entries like "Name", "minipc", "delly" in config
2. These are not valid for SSH temperature collection
Only IPs should be in allowed_nodes since that's what the proxy uses for SSH.
Removed the loop that added CLUSTER_NODE_NAMES to the array.
Also fixed: Removed extraction of CLUSTER_NODE_NAMES since it's no longer used.
When Pulse runs in Docker inside a Proxmox LXC container, users need
specific instructions to retrieve the bootstrap token. Added pct exec
and pct enter commands to the Docker instructions section.
Now shows three scenarios:
1. Direct Docker host: docker exec
2. Kubernetes: kubectl exec
3. Proxmox LXC with Docker: pct exec / pct enter
This makes first-time setup easier for users deploying Pulse in LXC
containers on Proxmox.
Problem:
Cleanup script uses systemd-run with both --wait and
--property="After=pulse-sensor-cleanup.service", creating a circular
dependency:
- cleanup.service runs and waits for uninstaller to complete
- uninstaller has After=cleanup.service, so it waits for cleanup to finish
- Result: Both services stuck waiting for each other
Fix:
Remove the --property="After=pulse-sensor-cleanup.service" line. The
Conflicts=pulse-sensor-proxy.service is sufficient to ensure the proxy
stops before uninstallation. The cleanup script doesn't need to finish
before the uninstaller starts.
Testing:
Cleanup now completes successfully, removing all artifacts:
- Systemd units removed
- Binaries deleted from /opt/pulse/sensor-proxy/
- Data directory /var/lib/pulse-sensor-proxy/ removed
- SSH keys cleaned from authorized_keys
- pulse-monitor user and API tokens deleted
- LXC bind mounts removed from container configs
Related to #605 (temperature monitoring cleanup)
**Missed Migration**:
- Line 2204 still used /usr/local/bin/pulse-sensor-wrapper.sh in fallback path
- Updated to use /opt/pulse/sensor-proxy/bin/pulse-sensor-wrapper.sh
**Backward Compatibility**:
- When pushing SSH keys to cluster nodes, installer now checks if remote node
has old installation (/usr/local/bin wrapper exists but /opt path doesn't)
- Automatically creates symlink on remote nodes to maintain compatibility
- Prevents temperature collection failures when cluster has mixed old/new installs
**Root Cause**:
When installer runs on upgraded node (delly), it pushes SSH keys with new forced
command path to all cluster nodes. If remote node (minipc) has old installation,
the forced command fails because wrapper doesn't exist at new path.
This fix ensures "it works straight out the box" by bridging old and new paths
automatically during SSH key deployment.
Rewrote AWK state machine to correctly handle:
- Multiple allowed_nodes sections (removes all of them)
- Comment lines immediately preceding allowed_nodes (discards them)
- Empty lines within allowed_nodes section
- Indented list items and comments
The function now:
1. Buffers comment lines that might precede allowed_nodes
2. When allowed_nodes: is detected, discards buffered comments
3. Skips all content until hitting a non-indented, non-comment line
4. Flushes buffered comments when hitting non-comment content
This ensures running the installer multiple times won't create duplicate
allowed_nodes sections in config.yaml.
Tested with script that verifies duplicate sections are removed correctly.
Updated CLEANUP_TODO.md with comprehensive documentation of all 8 critical
issues identified by Codex review (conv-1763166192078-1076) and their
resolutions.
Key updates:
- Added detailed problem/fix/impact for each issue
- Updated status to 'Codex review complete, ready for deployment testing'
- Documented all commits in implementation history
- Added Codex review summary section
- Marked phases 1-6 as complete, phase 7 (testing) as pending
This provides complete audit trail of cleanup implementation work.
**LXC Bind Mount Removal**:
- Changed from sed to `pct set -delete` for safer mount removal
- Validates syntax and prevents breaking container configs
- Finds mount points by grepping for pulse-sensor-proxy, extracts mp number
**API Token Parsing** (three-tier fallback):
1. Try `pveum --output-format json` with python3 JSON parsing
2. Fall back to `pvesh get /access/users/pulse-monitor@pam/token` JSON API
3. Last resort: parse table output with improved filtering (handles more Unicode chars)
**Retry Logic**:
- Rename cleanup-request.json to .processing instead of deleting immediately
- Allows retry on failure (processing file persists if script crashes)
- Remove .processing file only on successful completion
- Prevents loops while enabling failure recovery
These complete all 8 issues identified by Codex review.
**Host Detection**:
- Now detects localhost by hostname and FQDN, not just IP
- Fixes issue where nodes configured as https://hostname:8006 would skip
localhost cleanup (API tokens, bind mounts, service removal)
**Systemd Sandbox**:
- Added /etc/pve and /etc/systemd/system to ReadWritePaths
- Allows cleanup script to modify Proxmox configs and systemd units
**Uninstaller Improvements**:
- Use UUID for transient unit names (prevents same-second collisions)
- Added --purge flag for complete removal
- Added --wait and --collect flags to capture exit code
- Now fails cleanup if uninstaller exits non-zero
**Path Migration**:
- Fixed all /usr/local references to use /opt/pulse/sensor-proxy
- Updated forced command in SSH authorized_keys
- Updated self-heal script installer path
- Updated Go backend removal helpers (supports both new and legacy paths)
These fixes address Codex findings: hostname detection, sandbox permissions,
transient unit collisions, incomplete purging, and incomplete path migration.
Related to cleanup implementation testing.
After relocating binaries to INSTALL_ROOT, the SHARE_DIR variable was removed
but one reference remained in cache_installer_for_self_heal() causing
'unbound variable' error.
Changed to use INSTALL_ROOT directly since that's where the cached installer
is stored (STORED_INSTALLER=${INSTALL_ROOT}/install-sensor-proxy.sh).
The installer was trying to write binaries to /opt/pulse/sensor-proxy/bin/
before creating the directory structure, causing 'No such file or directory'
errors on fresh installs.
Moved directory creation for INSTALL_ROOT and bin/ to before binary installation
section (before line 657), ensuring directories exist before use.
Related to cleanup implementation testing.
All phases of full cleanup implementation are now complete:
- Phase 1: Binary relocation to /opt ✅
- Phase 2: Process isolation via systemd-run ✅
- Phase 3: flock serialization and immediate file deletion ✅
- Phase 4: Improved API token parsing ✅
- Phase 5: Testing (pending)
Updated status from 'requires architectural changes' to 'complete, ready for testing'.
Extends cleanup script to completely remove Pulse footprint from hosts
when nodes are removed, not just SSH keys. Now removes: SSH keys, proxy
service, binaries, API tokens, pulse-monitor user, and LXC bind mounts.
Key improvements:
1. **flock Serialization**: Prevents concurrent cleanup runs
- Acquires exclusive lock on cleanup.lock file
- Prevents race conditions and cleanup loops
2. **Immediate Request File Deletion**: Delete cleanup-request.json
before any long-running operations to prevent re-triggering
3. **API Token Cleanup**: Removes all pulse-monitor@pam API tokens
- Tries JSON output first (Proxmox 7.0+)
- Falls back to table parsing with proper filtering (no decoration chars)
- Deletes pulse-monitor@pam user after removing all tokens
4. **LXC Bind Mount Removal**: Scans all container configs and removes
pulse-sensor-proxy bind mount entries
5. **Process Isolation for Uninstaller**: Uses systemd-run to spawn
isolated transient unit that won't be killed when proxy service stops
- Unit name: pulse-uninstall-{timestamp}
- Properties: Type=oneshot, Conflicts=pulse-sensor-proxy.service
- Runs non-blocking so cleanup service can exit cleanly
- Falls back to direct call if systemd-run unavailable
6. **Complete Service/Binary Removal**: Calls installer's --uninstall
- Stops and disables pulse-sensor-proxy.service
- Removes all systemd units
- Deletes all binaries from /opt/pulse/sensor-proxy/
- Removes configuration files
- Cleans up directories
Changes to cleanup script logic:
- Added LOCKFILE and INSTALLER_PATH configuration
- Acquire flock before processing (prevents concurrent runs)
- Delete request file immediately after reading
- Full localhost cleanup: SSH keys → API tokens → bind mounts → uninstall
- Remote cleanup still SSH-key-only (can't orchestrate uninstall remotely)
- Better error handling with appropriate log levels
Updated cleanup service unit:
- ExecStart now uses ${CLEANUP_SCRIPT_PATH} variable (new /opt location)
- Changed heredoc from 'SERVICE_EOF' to SERVICE_EOF for variable expansion
Addresses all issues documented in CLEANUP_TODO.md:
- ✅ Read-only filesystem (binaries now in /opt, removable)
- ✅ Process isolation (systemd-run transient unit)
- ✅ Cleanup loops (flock + immediate file deletion)
- ✅ API token parsing (JSON first, filtered table fallback)
The UI message is now accurate: "Removing this proxmox ve node also
scrubs the Pulse footprint on the host — the proxy service, SSH key,
API token, and bind mount are all cleaned up automatically."
Part of: CLEANUP_TODO.md Phase 2-4
Supersedes: ed65fda74 (original cleanup attempt with process issues)
Depends on: b192c60e9 (binary relocation to /opt)
Moves all sensor-proxy binaries and scripts from /usr/local/bin to
/opt/pulse/sensor-proxy/bin to ensure they can be removed during cleanup
even on systems with read-only /usr (hardened Proxmox setups).
Changes:
- INSTALL_ROOT=/opt/pulse/sensor-proxy (new writable location)
- Binary path: /opt/pulse/sensor-proxy/bin/pulse-sensor-proxy
- Wrapper script: /opt/pulse/sensor-proxy/bin/pulse-sensor-wrapper.sh
- Cleanup script: /opt/pulse/sensor-proxy/bin/pulse-sensor-cleanup.sh
- Selfheal script: /opt/pulse/sensor-proxy/bin/pulse-sensor-proxy-selfheal.sh
- Installer storage: /opt/pulse/sensor-proxy/install-sensor-proxy.sh
Updated:
- Directory creation to include ${INSTALL_ROOT}/bin
- Systemd service ExecStart paths to use ${BINARY_PATH}
- Self-heal service ExecStart to use ${SELFHEAL_SCRIPT}
- Changed heredoc delimiters from 'EOF' to EOF for variable expansion
Rationale:
Proxmox VE can mount /usr as read-only in hardened configurations.
The previous /usr/local/bin location prevented complete uninstallation
on these systems, violating Pulse's correctness principle. The /opt
location is guaranteed writable and appropriate for third-party software.
This is Phase 1 of implementing full cleanup functionality per
CLEANUP_TODO.md. Subsequent commits will add process isolation,
API token deletion, and bind mount removal.
Part of: #CLEANUP_TODO.md Phase 1
Related: ed65fda74 (original cleanup attempt)
During cluster startup, nodes were temporarily using the primary cluster
endpoint for temperature collection before cluster metadata validation
completed. This caused all nodes to show the same (incorrect) temperature
values for ~4 minutes until validation finished and per-node endpoints
were established.
Example: minipc would show delly's temperature (90°C) instead of its own
(50°C) from startup until cluster validation completed.
Root cause:
- Temperature collection started immediately at startup
- Cluster endpoint validation happened asynchronously
- Code fell back to primary endpoint when ClusterEndpoints was empty
- All nodes used same endpoint, got same temperature data
Fix: Skip temperature collection for cluster nodes until:
1. ClusterEndpoints array is populated (validation complete)
2. Node's specific endpoint is found in the cluster metadata
This ensures correct temperature data from the very first collection,
maintaining data integrity during startup. When persisted config exists,
endpoints are available immediately so no delay occurs. For new clusters,
temperature collection begins once validation completes (~30s).
Preserves Pulse's correctness guarantee: users can trust metrics
immediately after restart without waiting for "warm-up" period.
The update_allowed_nodes function was changing ownership of the temp file
before all writes were complete, causing 'Permission denied' errors when
appending the allowed_nodes section.
Root cause:
- mktemp creates file owned by script runner (root)
- chown changed ownership to pulse-sensor-proxy:pulse-sensor-proxy
- Subsequent append (>>) failed because root can't write to the file
Fix: Defer chown until after all writes complete and file is moved to
final location. Ownership is still correctly set on the final config file.
The full cleanup implementation (ed65fda74) has architectural issues that need addressing:
1. Read-only /usr filesystems prevent binary removal
2. Process isolation issues cause cleanup service to be killed
3. Cleanup loops from improper request file handling
This TODO documents the required changes following Codex recommendations:
- Relocate binaries to /opt/pulse/sensor-proxy/
- Use transient systemd units for uninstall orchestration
- Add flock serialization and proper cleanup-request handling
Current state: SSH keys are removed (critical), full cleanup needs refactoring.
When a Proxmox node is removed from Pulse, the cleanup now performs full uninstallation:
- SSH keys removal (existing functionality)
- Uninstalls pulse-sensor-proxy service
- Removes LXC bind mounts from container configs
- Deletes Proxmox API tokens
- Removes pulse-monitor@pam user
This aligns with security best practices and user expectations - "remove node"
should completely sever trust with that machine, not leave credentials and
privileged services behind.
The cleanup script now calls the uninstaller (--uninstall) and uses pveum
to remove API tokens. This prevents leftover artifacts if the host is
repurposed or compromised.
Related: config_handlers.go triggerPVEHostCleanup() at node deletion
- Change mktemp to use /tmp/pulse-config.XXXXXXXXXX template
- Prevents conflicts with stale temp files from previous runs
- Fixes 'Permission denied' errors when script re-runs
- Add trap to remove temp file on function return (success or failure)
- Add error check for mv command with descriptive message
- Ensure config file has proper permissions after update
This prevents orphaned temp files when errors occur and provides
better diagnostics when file operations fail.
The all_nodes arrays were declared with 'local' keyword outside of
functions, causing bash syntax error:
'local: can only be used in a function'
Fixed by removing 'local' keyword - arrays in main script scope don't
need it and it's actually invalid syntax.
The awk logic was removing allowed_nodes sections but leaving their
comment headers behind. When multiple sections existed, comments would
accumulate.
New approach:
- Buffer all comment lines encountered outside sections
- When a non-comment line is found, flush buffered comments
- When allowed_nodes is found, discard buffered comments (they belonged
to the section we're removing)
- This cleanly removes section headers like:
'# Cluster nodes (auto-discovered during installation)'
'# These nodes are allowed to request...'
Tested with config containing duplicate allowed_nodes sections - now
correctly produces clean output with all duplicates and headers removed.
The installer was only creating base config.yaml in standalone mode,
but update_allowed_nodes() is also called in LXC mode. When the config
didn't exist, update_allowed_nodes() would create an empty file and only
add the allowed_nodes section, missing required fields like
allowed_peer_uids, metrics_address, rate_limit, etc.
This caused the proxy to fail when it tried to parse the incomplete config.
Now creates a proper base config with all required fields if the file
doesn't exist, before any mode-specific configuration is added.
The install-sensor-proxy.sh script was blindly appending allowed_nodes
sections to the config file without checking if they already existed.
When the script was re-run or if the initial config already had an
allowed_nodes section, this created duplicate YAML keys that caused
the proxy service to fail with parse errors.
Changes:
- Add update_allowed_nodes() helper function that safely updates the
allowed_nodes section by removing any existing ones first
- Replace all three cat >> config.yaml heredocs with calls to the
helper function (cluster nodes, standalone mode, pvecm fallback)
- Uses awk to properly parse and remove multi-line YAML sections
This makes the installer idempotent and prevents config corruption on
re-runs.
Fixes issue where proxy service crashed with:
'mapping key "allowed_nodes" already defined at line X'
