Fix auto-update leaving service down after installer failure
Some checks are pending
Build and Test / Secret Scan (push) Waiting to run
Build and Test / Frontend & Backend (push) Waiting to run
Canonical Governance / governance (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 1/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 2/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 3/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 4/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 5/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 6/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 7/8) (push) Waiting to run
Core E2E Tests / Playwright Core E2E (shard 8/8) (push) Waiting to run
Core E2E Tests / Agent registration lifecycle (push) Waiting to run
Core E2E Tests / E2E verdict (push) Blocked by required conditions
Unified Agent Native Verification / Linux ARM64 (push) Waiting to run
Unified Agent Native Verification / Linux x64 (push) Waiting to run
Unified Agent Native Verification / Windows x64 (push) Waiting to run
Unified Agent Native Verification / macOS ARM64 (push) Waiting to run
Unified Agent Native Verification / macOS Intel (push) Waiting to run
Unified Agent Native Verification / FreeBSD cross-build contract (push) Waiting to run

Two bugs from #1630 that combined to take installs down silently:

1. perform_update()'s install-failed rollback branch restored the backup
   but never restarted pulse.service. Since the generated
   pulse-update.service gates on ExecCondition=systemctl is-active,
   every later timer run was then skipped and the install stayed down
   until manual intervention. Restart is now guaranteed by a
   service_was_active-guarded restart in that branch plus an
   ensure_service_restarted RETURN trap so no exit path can miss it
   (re-fix of #1323, originally c0b3a0e66, lost in 778a2577b and only
   partially restored in 672e81985).

2. install.sh aborted under errexit when writing the /bin/update helper
   on a read-only filesystem - after the new binary was installed and
   the service stopped, landing in bug 1's no-restart branch. The stock
   pulse-update.service uses ProtectSystem=strict, so /bin and
   /usr/local/bin are read-only on stock unattended updates; transient
   read-only remounts hit the same path. The helper write, PATH
   appends, and the /usr/local/bin/pulse symlink are now idempotent and
   non-fatal with a warning (install_binary_symlink).

Contract: deployment-installability now pins fail-closed service
availability for unattended updates and non-fatal writes outside the
hardened unit's writable set, with proofs in pulse_auto_update_test.go
and root_install_sh_test.go plus shell regression coverage in
scripts/tests/test-pulse-auto-update.sh (installer-exits-nonzero path)
and scripts/tests/test-install-update-resilience.sh (read-only helper
and symlink paths, verified under set -e, root-safe via ENOTDIR).

Fixes #1630

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
courtmanr@gmail.com 2026-07-27 09:28:45 +01:00
parent bf24a9a9cf
commit 806cbe83dc
7 changed files with 569 additions and 15 deletions

View file

@ -604,8 +604,8 @@ func TestRootInstallScriptSupportsInstanceScopedServerInstalls(t *testing.T) {
`UPDATE_SERVICE_PATH="${PULSE_UPDATE_SERVICE_PATH:-$(default_update_service_path_for_service "$SERVICE_NAME")}"`,
`UPDATE_TIMER_PATH="${PULSE_UPDATE_TIMER_PATH:-$(default_update_timer_path_for_service "$SERVICE_NAME")}"`,
`if [[ "$SERVICE_NAME_EXPLICIT" == "true" ]]; then`,
`mkdir -p "$(dirname "$BINARY_LINK_PATH")"`,
`ln -sf "$INSTALL_DIR/bin/pulse" "$BINARY_LINK_PATH"`,
`install_binary_symlink "$INSTALL_DIR/bin/pulse" "$BINARY_LINK_PATH"`,
`ln -sf "$target" "$link_path"`,
`safe_systemctl enable "$update_timer_unit" || true`,
`safe_systemctl start "$update_timer_unit" || true`,
`Environment="PULSE_SERVICE_NAME=$service_name"`,
@ -1350,3 +1350,94 @@ func TestRootInstallServiceGrantsIcmpProbeCapability(t *testing.T) {
t.Fatal("AmbientCapabilities=CAP_NET_RAW is not in the unit's security hardening block")
}
}
// TestRootInstallScriptUpdateHelperWriteIsNonFatalOnReadOnlyPath asserts the
// #1630 guarantee: setup_update_command's write of the /bin/update helper
// (and its PATH appends) must not abort the installer under errexit when the
// destination is unwritable. The stock pulse-update.service runs the
// unattended updater with ProtectSystem=strict, leaving /bin and
// /usr/local/bin read-only; the old behavior killed the installer after the
// new binary was installed and the service stopped, and the auto-update
// rollback then left Pulse down. A regular file as the "parent directory"
// makes writes beneath it fail with ENOTDIR, which also fails when the test
// runs as root (unlike chmod 555).
func TestRootInstallScriptUpdateHelperWriteIsNonFatalOnReadOnlyPath(t *testing.T) {
script := `
set -euo pipefail
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
touch "$TMP/blocker"
GITHUB_REPO="rcourtman/Pulse"
INSTALL_SIGNATURE_IDENTITY="pulse-installer"
INSTALL_SIGNATURE_NAMESPACE="pulse-install"
PINNED_RELEASE_SSH_PUBLIC_KEY="test-key"
print_warn() { echo "WARN: $*"; }
print_success() { echo "OK: $*"; }
release_signature_key_available() { :; }
require_release_signature_verifier() { :; }
verify_release_signature() { :; }
` + extractRootInstallShellFunction(t, "setup_update_command") + `
UPDATE_HELPER_PATH="$TMP/blocker/update" \
PULSE_PROFILE_PATH="$TMP/profile" \
PULSE_BASHRC_PATH="$TMP/bashrc" \
setup_update_command
echo "SURVIVED_UNWRITABLE"
UPDATE_HELPER_PATH="$TMP/bin/update" \
PULSE_PROFILE_PATH="$TMP/profile" \
PULSE_BASHRC_PATH="$TMP/bashrc" \
setup_update_command
[[ -x "$TMP/bin/update" ]] && echo "HELPER_WRITTEN"
grep -q "Pulse update command" "$TMP/bin/update" && echo "HELPER_BODY_OK"
`
out, err := exec.Command("bash", "-c", script).CombinedOutput()
if err != nil {
t.Fatalf("setup_update_command aborted the installer on an unwritable helper path: %v\n%s", err, out)
}
got := string(out)
for _, want := range []string{"SURVIVED_UNWRITABLE", "WARN:", "HELPER_WRITTEN", "HELPER_BODY_OK"} {
if !strings.Contains(got, want) {
t.Fatalf("missing %q in setup_update_command output:\n%s", want, got)
}
}
}
// TestRootInstallScriptBinarySymlinkIsIdempotentAndNonFatal asserts the
// companion #1630 guarantee for the /usr/local/bin/pulse convenience
// symlink: an unwritable link path only warns, a writable one creates the
// link, and an already-correct link is kept without needing ln at all
// (the update-run case where the link survives but the fs is read-only).
func TestRootInstallScriptBinarySymlinkIsIdempotentAndNonFatal(t *testing.T) {
script := `
set -euo pipefail
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
touch "$TMP/blocker"
mkdir -p "$TMP/bin"
touch "$TMP/bin/pulse-binary"
print_warn() { echo "WARN: $*"; }
print_success() { echo "OK: $*"; }
` + extractRootInstallShellFunction(t, "install_binary_symlink") + `
install_binary_symlink "$TMP/bin/pulse-binary" "$TMP/blocker/pulse"
echo "SURVIVED_UNWRITABLE"
install_binary_symlink "$TMP/bin/pulse-binary" "$TMP/bin/pulse"
[[ "$(readlink "$TMP/bin/pulse")" == "$TMP/bin/pulse-binary" ]] && echo "LINK_CREATED"
ln() { echo "LN_CALLED_AGAIN"; return 1; }
install_binary_symlink "$TMP/bin/pulse-binary" "$TMP/bin/pulse"
echo "SURVIVED_EXISTING_LINK"
`
out, err := exec.Command("bash", "-c", script).CombinedOutput()
if err != nil {
t.Fatalf("install_binary_symlink aborted under errexit: %v\n%s", err, out)
}
got := string(out)
for _, want := range []string{"SURVIVED_UNWRITABLE", "WARN:", "LINK_CREATED", "already in place", "SURVIVED_EXISTING_LINK"} {
if !strings.Contains(got, want) {
t.Fatalf("missing %q in install_binary_symlink output:\n%s", want, got)
}
}
if strings.Contains(got, "LN_CALLED_AGAIN") {
t.Fatalf("install_binary_symlink should not invoke ln when the correct link already exists:\n%s", got)
}
}