vrr/Pulse
2
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-10-19 10:33:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

Harden outbound URLs and file-backed storage

Browse source
This commit is contained in:
rcourtman 2026-03-29 12:47:55 +01:00
parent 2918f5f994
commit d6536932fc
21 changed files with 1055 additions and 111 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/release-control/v6/internal/subsystems/agent-lifecycle.md
View file

@ -219,6 +219,12 @@ setup or router work touches shared `internal/api/` files, telemetry preview
and install-ID reset routes must keep reusing the canonical system-settings
trust boundary and server-owned telemetry runtime instead of borrowing agent
lifecycle proof or state ownership just because the same router surface moved.
That same shared `internal/api/` dependency also now assumes SSO test and
metadata-preview routes fail closed on validated outbound URL handling.
Lifecycle-adjacent setup and hosted bootstrap surfaces may depend on those
shared helpers, but they must not reintroduce raw URL concatenation,
userinfo-bearing fetch targets, or origin-root OIDC discovery drift when
operators validate identity configuration.
Agent lifecycle owns the install/register/update continuity surfaces, but it
does not own unified-resource history or control-plane timeline persistence.

5
docs/release-control/v6/internal/subsystems/ai-runtime.md
View file

@ -179,6 +179,11 @@ The same migration-only rule applies to guest knowledge under
knowledge files may only serve as migration input, and the knowledge store
must rewrite canonical encrypted-at-rest storage immediately on load instead
of leaving guest knowledge plaintext on disk until a future note update.
Chat-session and guest-knowledge persistence now also keep canonical on-disk
names opaque and machine-owned. Legacy identifier-derived filenames may be
discovered only by inspecting already-owned files for embedded record IDs, and
the next successful write must rewrite them to hashed canonical paths instead
of preserving user-controlled identifiers as filesystem path segments.
That trust boundary also applies when the store is constructed: if the
knowledge store cannot initialize encryption, construction must fail closed
instead of silently creating a plaintext-at-rest runtime store.

12
docs/release-control/v6/internal/subsystems/api-contracts.md
View file

@ -186,6 +186,7 @@ Own canonical runtime payload shapes between backend and frontend.
19. Governed frontend API clients open-coding `try/catch` wrappers around `apiFetchJSON(...)` just to map `402` or `404` into `[]`, `{ plans: [] }`, or `null` instead of using canonical shared API-error-status fallback helpers
20. Backend config/settings handlers pointing operator guidance at GitHub `main` docs when the running build already ships that guidance locally under `/docs/`
21. Telemetry preview or reset endpoints drifting from the exact server-owned telemetry runtime contract instead of reusing the same source-of-truth snapshot and install-ID state the background sender uses
22. Shared SSO test or metadata-preview handlers open-coding outbound metadata/discovery URLs, allowing userinfo-bearing HTTP(S) inputs, or rebuilding `/.well-known/openid-configuration` with origin-root string concatenation instead of the shared validated URL helpers before any outbound request
## Completion Obligations
@ -239,6 +240,11 @@ Own canonical runtime payload shapes between backend and frontend.
self-serve verification failures may render owned HTML, but they must
preserve originating Pulse context instead of collapsing into generic
control-plane failures.
22. Keep shared SSO test and metadata-preview transport fail-closed: SAML
metadata URLs and OIDC issuer URLs must reject non-HTTP or userinfo-bearing
inputs before any outbound request is attempted, and OIDC discovery must
append `/.well-known/openid-configuration` beneath the configured issuer
base path instead of resetting to the origin root.
## Current State
@ -261,6 +267,12 @@ the cloud proxy, the runtime must derive the effective same-origin
`https://<tenant-host>` boundary from trusted forwarded host/proto headers
only after the hosted container contract has injected explicit proxy CIDRs, so
live updates stay connected without weakening cross-site websocket checks.
That same shared boundary now also owns outbound SSO metadata and discovery
URL handling. SAML test/preview metadata fetches and OIDC issuer discovery
must normalize absolute HTTP(S) inputs through shared helpers, reject
userinfo-bearing URLs before any outbound request, and append the OIDC
well-known path relative to the issuer base instead of resetting discovery to
the origin root.
Commercial self-service actions in that shell must stay same-origin as well:
the frontend may only call the portal-owned `/api/portal/commercial/*` routes,
and `internal/cloudcp/portal/commercial_proxy.go` plus `internal/cloudcp/routes.go`

5
docs/release-control/v6/internal/subsystems/notifications.md
View file

@ -110,6 +110,11 @@ retry gate in `webhook_enhanced.go` must parse provider failures from both
`status 429`-style and `HTTP 429`-style error strings before it decides
whether to retry, so a non-retryable `HTTP 400` result cannot be retried just
because the transport changed its error wording.
That same notification transport boundary also owns outbound Apprise HTTP URL
normalization. Server URLs must be validated as absolute HTTP(S) endpoints
without userinfo before request construction, and the `/notify` plus optional
config-key path must append to the canonical server base instead of being
rebuilt through raw string concatenation in local delivery code.
`internal/api/notifications.go` and
`frontend-modern/src/api/notifications.ts` are shared boundaries with

6
docs/release-control/v6/internal/subsystems/storage-recovery.md
View file

@ -136,6 +136,12 @@ runtime data-dir authority being replaceable without hangs or leaked state,
and router teardown must close the exact session, CSRF, and recovery-token
workers that router initialized instead of assuming a later global auth-store
binding will clean them up.
That same runtime data-dir authority also assumes file-backed stores keep
canonical filenames opaque and machine-owned. Recovery-adjacent session,
knowledge, and discovery records may discover legacy identifier-derived files
only for migration, and the next successful write must replace those legacy
paths with hashed canonical names so operator-controlled identifiers do not
become durable filesystem path segments.
That same hosted handoff dependency also assumes the exchange path repairs
tenant org access before redirecting the browser into protected routes.
Recovery- and storage-adjacent hosted pages that open immediately after

87
internal/ai/chat/session.go
View file

@ -11,6 +11,7 @@ import (
"time"
"github.com/google/uuid"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rs/zerolog/log"
)
@ -71,7 +72,44 @@ func (s *SessionStore) sessionPath(id string) (string, error) {
if err := validateSessionID(id); err != nil {
return "", err
}
return filepath.Join(s.dataDir, id+".json"), nil
return filepath.Join(s.dataDir, securityutil.HashedStorageName(id)+".json"), nil
}
func (s *SessionStore) findLegacySessionPath(id string) (string, error) {
if err := validateSessionID(id); err != nil {
return "", err
}
canonicalName := securityutil.HashedStorageName(id) + ".json"
entries, err := os.ReadDir(s.dataDir)
if err != nil {
if os.IsNotExist(err) {
return "", nil
}
return "", fmt.Errorf("failed to scan session directory: %w", err)
}
for _, entry := range entries {
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" {
continue
}
if entry.Name() == canonicalName {
continue
}
path := filepath.Join(s.dataDir, entry.Name())
data, err := os.ReadFile(path)
if err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to read legacy session candidate")
continue
}
var session sessionData
if err := json.Unmarshal(data, &session); err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to parse legacy session candidate")
continue
}
if session.ID == id {
return path, nil
}
}
return "", nil
}
// List returns all sessions, sorted by updated_at descending
@ -90,11 +128,19 @@ func (s *SessionStore) List() ([]Session, error) {
continue
}
data, err := s.readSession(strings.TrimSuffix(entry.Name(), ".json"))
file, err := os.ReadFile(filepath.Join(s.dataDir, entry.Name()))
if err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to read session file")
continue
}
var data sessionData
if err := json.Unmarshal(file, &data); err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to parse session file")
continue
}
for i := range data.Messages {
data.Messages[i] = data.Messages[i].NormalizeCollections()
}
sessions = append(sessions, Session{
ID: data.ID,
@ -167,11 +213,26 @@ func (s *SessionStore) Delete(id string) error {
if err != nil {
return err
}
if err := os.Remove(path); err != nil {
if os.IsNotExist(err) {
return fmt.Errorf("session not found: %s", id)
var removed bool
candidates := []string{path}
legacyPath, err := s.findLegacySessionPath(id)
if err != nil {
return err
}
if legacyPath != "" && legacyPath != path {
candidates = append(candidates, legacyPath)
}
for _, candidate := range candidates {
if err := os.Remove(candidate); err != nil {
if os.IsNotExist(err) {
continue
}
return fmt.Errorf("failed to delete session: %w", err)
}
return fmt.Errorf("failed to delete session: %w", err)
removed = true
}
if !removed {
return fmt.Errorf("session not found: %s", id)
}
// Also clean up resolved context, FSM, and knowledge accumulator
@ -253,10 +314,17 @@ func (s *SessionStore) readSession(id string) (*sessionData, error) {
return nil, err
}
file, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
if os.IsNotExist(err) {
legacyPath, legacyErr := s.findLegacySessionPath(id)
if legacyErr != nil {
return nil, legacyErr
}
if legacyPath == "" {
return nil, fmt.Errorf("session not found: %s", id)
}
file, err = os.ReadFile(legacyPath)
}
if err != nil {
return nil, fmt.Errorf("failed to read session: %w", err)
}
@ -292,6 +360,9 @@ func (s *SessionStore) writeSession(data sessionData) error {
if err := os.WriteFile(path, file, 0600); err != nil {
return fmt.Errorf("failed to write session: %w", err)
}
if legacyPath, err := s.findLegacySessionPath(data.ID); err == nil && legacyPath != "" && legacyPath != path {
_ = os.Remove(legacyPath)
}
return nil
}

48
internal/ai/chat/session_test.go
View file

@ -104,6 +104,54 @@ func TestSessionStore(t *testing.T) {
})
}
func TestSessionStoreMigratesLegacySessionFileOnWrite(t *testing.T) {
store, err := NewSessionStore(t.TempDir())
require.NoError(t, err)
now := time.Now().UTC().Round(time.Second)
legacy := sessionData{
ID: "legacy-session",
Title: "",
Messages: []Message{},
CreatedAt: now,
UpdatedAt: now,
}
payload, err := json.MarshalIndent(legacy, "", " ")
require.NoError(t, err)
legacyPath := filepath.Join(store.dataDir, legacy.ID+".json")
require.NoError(t, os.WriteFile(legacyPath, payload, 0600))
canonicalPath, err := store.sessionPath(legacy.ID)
require.NoError(t, err)
_, err = os.Stat(canonicalPath)
require.ErrorIs(t, err, os.ErrNotExist)
err = store.AddMessage(legacy.ID, Message{
Role: "user",
Content: "hello from legacy storage",
})
require.NoError(t, err)
session, err := store.Get(legacy.ID)
require.NoError(t, err)
assert.Equal(t, legacy.ID, session.ID)
assert.Equal(t, "hello from legacy storage", session.Title)
messages, err := store.GetMessages(legacy.ID)
require.NoError(t, err)
require.Len(t, messages, 1)
assert.Equal(t, "hello from legacy storage", messages[0].Content)
_, err = os.Stat(canonicalPath)
require.NoError(t, err)
_, err = os.Stat(legacyPath)
assert.ErrorIs(t, err, os.ErrNotExist)
}
func TestMessage_UsesCanonicalEmptyCollections(t *testing.T) {
payload, err := json.Marshal(EmptyMessage())
require.NoError(t, err)

152
internal/ai/knowledge/store.go
View file

@ -11,6 +11,7 @@ import (
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/crypto"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/internal/utils"
"github.com/rs/zerolog/log"
@ -115,13 +116,83 @@ func NewStore(dataDir string) (*Store, error) {
// guestFilePath returns the file path for a guest's knowledge
func (s *Store) guestFilePath(guestID string) string {
// Sanitize guest ID for filesystem
safeID := filepath.Base(guestID) // Prevent path traversal
// Use .enc extension for encrypted files
fileBase := securityutil.HashedStorageName(guestID)
if s.crypto != nil {
return filepath.Join(s.dataDir, safeID+".enc")
return filepath.Join(s.dataDir, fileBase+".enc")
}
return filepath.Join(s.dataDir, safeID+".json")
return filepath.Join(s.dataDir, fileBase+".json")
}
func (s *Store) loadKnowledgeFromPath(path string) (*GuestKnowledge, bool, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, false, err
}
migratedPlaintext := false
if s.crypto != nil {
if decrypted, decErr := s.crypto.Decrypt(data); decErr == nil {
data = decrypted
} else {
migratedPlaintext = true
}
}
var knowledge GuestKnowledge
if err := json.Unmarshal(data, &knowledge); err != nil {
return nil, false, fmt.Errorf("failed to parse knowledge file: %w", err)
}
return &knowledge, migratedPlaintext, nil
}
func (s *Store) findLegacyKnowledgePath(guestID string) (string, error) {
canonicalEnc := securityutil.HashedStorageName(guestID) + ".enc"
canonicalJSON := securityutil.HashedStorageName(guestID) + ".json"
legacyEnc := filepath.Base(guestID) + ".enc"
legacyJSON := filepath.Base(guestID) + ".json"
entries, err := os.ReadDir(s.dataDir)
if err != nil {
if os.IsNotExist(err) {
return "", nil
}
return "", fmt.Errorf("failed to scan knowledge directory: %w", err)
}
for _, entry := range entries {
if entry.Name() == legacyEnc || entry.Name() == legacyJSON {
if entry.IsDir() {
return "", fmt.Errorf("legacy knowledge path %q is not a regular file", entry.Name())
}
}
if entry.IsDir() {
continue
}
switch filepath.Ext(entry.Name()) {
case ".json", ".enc":
default:
continue
}
if entry.Name() == canonicalEnc || entry.Name() == canonicalJSON {
continue
}
path := filepath.Join(s.dataDir, entry.Name())
knowledge, _, err := s.loadKnowledgeFromPath(path)
if err != nil {
if entry.Name() == legacyEnc || entry.Name() == legacyJSON {
return "", fmt.Errorf("failed to inspect legacy knowledge file %q: %w", entry.Name(), err)
}
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to inspect legacy knowledge candidate")
continue
}
if normalizeGuestID(knowledge.GuestID) == guestID {
return path, nil
}
}
return "", nil
}
// GetKnowledge retrieves knowledge for a guest
@ -151,13 +222,17 @@ func (s *Store) GetKnowledge(guestID string) (*GuestKnowledge, error) {
}
filePath := s.guestFilePath(guestID)
migratedPlaintext := false
data, err := os.ReadFile(filePath)
if os.IsNotExist(err) {
// Try legacy unencrypted file
legacyJSONPath := filepath.Join(s.dataDir, filepath.Base(guestID)+".json")
data, err = os.ReadFile(legacyJSONPath)
if os.IsNotExist(err) {
activePath := filePath
knowledge, migratedPlaintext, err := s.loadKnowledgeFromPath(filePath)
if err != nil {
if !os.IsNotExist(err) {
return nil, fmt.Errorf("failed to read knowledge file: %w", err)
}
activePath, err = s.findLegacyKnowledgePath(guestID)
if err != nil {
return nil, err
}
if activePath == "" {
// No knowledge yet, return empty
knowledge := &GuestKnowledge{
GuestID: guestID,
@ -166,35 +241,11 @@ func (s *Store) GetKnowledge(guestID string) (*GuestKnowledge, error) {
s.cache[guestID] = knowledge
return knowledge, nil
}
knowledge, migratedPlaintext, err = s.loadKnowledgeFromPath(activePath)
if err != nil {
return nil, fmt.Errorf("failed to read knowledge file: %w", err)
}
migratedPlaintext = true
log.Info().Str("guest_id", guestID).Msg("found unencrypted knowledge file, rewriting encrypted storage")
} else if err != nil {
return nil, fmt.Errorf("failed to read knowledge file: %w", err)
}
// Decrypt if crypto is available and file is encrypted
if s.crypto != nil && filepath.Ext(filePath) == ".enc" {
decrypted, err := s.crypto.Decrypt(data)
if err != nil {
// Try as plain JSON (migration case)
var plaintextKnowledge GuestKnowledge
if jsonErr := json.Unmarshal(data, &plaintextKnowledge); jsonErr == nil {
migratedPlaintext = true
log.Info().Str("guest_id", guestID).Msg("loaded unencrypted knowledge, rewriting encrypted storage")
} else {
return nil, fmt.Errorf("failed to decrypt knowledge: %w", err)
}
} else {
data = decrypted
}
}
var knowledge GuestKnowledge
if err := json.Unmarshal(data, &knowledge); err != nil {
return nil, fmt.Errorf("failed to parse knowledge file: %w", err)
log.Info().Str("guest_id", guestID).Str("path", activePath).Msg("loaded legacy knowledge file, rewriting canonical storage")
}
needsMigration := false
@ -207,15 +258,15 @@ func (s *Store) GetKnowledge(guestID string) (*GuestKnowledge, error) {
needsMigration = true
}
s.cache[guestID] = &knowledge
s.cache[guestID] = knowledge
if needsMigration || migratedPlaintext {
if err := s.saveToFile(guestID, &knowledge); err != nil {
if activePath != filePath || needsMigration || migratedPlaintext {
if err := s.saveToFile(guestID, knowledge); err != nil {
log.Warn().Err(err).Str("guest_id", guestID).Msg("failed to migrate canonical guest knowledge file")
}
}
return &knowledge, nil
return knowledge, nil
}
// SaveNote adds or updates a note for a guest
@ -428,12 +479,9 @@ func (s *Store) saveToFile(guestID string, knowledge *GuestKnowledge) error {
return fmt.Errorf("failed to write knowledge file: %w", err)
}
// Remove legacy unencrypted file if it exists
if s.crypto != nil {
legacyPath := filepath.Join(s.dataDir, filepath.Base(guestID)+".json")
if _, err := os.Stat(legacyPath); err == nil {
os.Remove(legacyPath)
log.Info().Str("guest_id", guestID).Msg("removed legacy unencrypted knowledge file")
if legacyPath, err := s.findLegacyKnowledgePath(guestID); err == nil && legacyPath != "" && legacyPath != filePath {
if err := os.Remove(legacyPath); err != nil && !os.IsNotExist(err) {
log.Warn().Err(err).Str("guest_id", guestID).Str("path", legacyPath).Msg("failed to remove legacy knowledge file")
}
}
@ -456,9 +504,17 @@ func (s *Store) ListGuests() ([]string, error) {
var guests []string
seen := make(map[string]struct{})
for _, file := range files {
if file.IsDir() {
continue
}
ext := filepath.Ext(file.Name())
if ext == ".json" || ext == ".enc" {
guestID := normalizeGuestID(file.Name()[:len(file.Name())-len(ext)])
knowledge, _, err := s.loadKnowledgeFromPath(filepath.Join(s.dataDir, file.Name()))
if err != nil {
log.Warn().Err(err).Str("file", file.Name()).Msg("failed to inspect knowledge file while listing guests")
continue
}
guestID := normalizeGuestID(knowledge.GuestID)
if guestID == "" {
continue
}

114
internal/api/contract_test.go
View file

@ -65,6 +65,120 @@ func TestContract_WebSocketTrustedProxyHostedOrigin(t *testing.T) {
conn.Close()
}
func TestContract_SSOTestRejectsMetadataURLWithUserinfo(t *testing.T) {
called := make(chan struct{}, 1)
metadataServer := newIPv4HTTPServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case called <- struct{}{}:
default:
}
w.Header().Set("Content-Type", "application/xml")
w.Write([]byte(testSAMLMetadata))
}))
defer metadataServer.Close()
metadataURL, err := url.Parse(metadataServer.URL)
if err != nil {
t.Fatalf("parse metadata server url: %v", err)
}
metadataURL.User = url.UserPassword("user", "pass")
reqBody := SSOTestRequest{
Type: "saml",
SAML: &SAMLTestConfig{
IDPMetadataURL: metadataURL.String(),
},
}
body, err := json.Marshal(reqBody)
if err != nil {
t.Fatalf("marshal request: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/security/sso/providers/test", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
setTestIP(req)
rec := httptest.NewRecorder()
router := &Router{}
router.handleTestSSOProvider(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status=%d, want %d: %s", rec.Code, http.StatusBadRequest, rec.Body.String())
}
var resp SSOTestResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("unmarshal response: %v", err)
}
if resp.Success {
t.Fatalf("expected failed response, got success: %+v", resp)
}
select {
case <-called:
t.Fatal("expected SAML metadata URL with userinfo to be rejected before outbound fetch")
default:
}
}
func TestContract_SSOTestOIDCDiscoveryKeepsIssuerBasePath(t *testing.T) {
var issuerURL string
discoveryServer := newIPv4HTTPServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/auth/realms/pulse/.well-known/openid-configuration" {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{
"issuer": %q,
"authorization_endpoint": %q,
"token_endpoint": %q,
"userinfo_endpoint": %q,
"jwks_uri": %q,
"scopes_supported": ["openid", "profile", "email"]
}`, issuerURL, issuerURL+"/protocol/openid-connect/auth", issuerURL+"/protocol/openid-connect/token", issuerURL+"/protocol/openid-connect/userinfo", issuerURL+"/protocol/openid-connect/certs")
}))
defer discoveryServer.Close()
issuerURL = discoveryServer.URL + "/auth/realms/pulse"
reqBody := SSOTestRequest{
Type: "oidc",
OIDC: &OIDCTestConfig{
IssuerURL: issuerURL,
},
}
body, err := json.Marshal(reqBody)
if err != nil {
t.Fatalf("marshal request: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/api/security/sso/providers/test", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
setTestIP(req)
rec := httptest.NewRecorder()
router := &Router{}
router.handleTestSSOProvider(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status=%d, want %d: %s", rec.Code, http.StatusOK, rec.Body.String())
}
var resp SSOTestResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("unmarshal response: %v", err)
}
if !resp.Success {
t.Fatalf("expected success response, got %+v", resp)
}
if resp.Details == nil {
t.Fatal("expected OIDC details in response")
}
if resp.Details.EntityID != issuerURL {
t.Fatalf("issuer=%q, want %q", resp.Details.EntityID, issuerURL)
}
}
func TestContract_FindingJSONSnapshot(t *testing.T) {
now := time.Date(2026, 2, 8, 13, 14, 15, 0, time.UTC)
lastSeen := now.Add(5 * time.Minute)

21
internal/api/identity_sso_handlers.go
View file

@ -19,6 +19,7 @@ import (
"github.com/crewjam/saml"
"github.com/google/uuid"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rs/zerolog/log"
)
@ -913,10 +914,19 @@ func (r *Router) testOIDCConnection(ctx context.Context, cfg *OIDCTestConfig) SS
}
// Fetch OIDC discovery document
discoveryURL := strings.TrimRight(cfg.IssuerURL, "/") + "/.well-known/openid-configuration"
issuerURL, err := securityutil.NormalizeHTTPBaseURL(cfg.IssuerURL, "")
if err != nil {
log.Error().Err(err).Msg("OIDC test: invalid issuer URL")
return SSOTestResponse{
Success: false,
Message: "Invalid issuer URL format",
Error: "invalid_url",
}
}
discoveryURL := securityutil.AppendURLPath(issuerURL, ".well-known", "openid-configuration")
httpClient := newTestHTTPClient()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, discoveryURL, nil)
req, err := securityutil.NewValidatedRequestWithContext(ctx, http.MethodGet, discoveryURL, nil)
if err != nil {
log.Error().Err(err).Msg("OIDC test: failed to create discovery request")
return SSOTestResponse{
@ -1187,7 +1197,12 @@ func newTestHTTPClient() *http.Client {
}
func fetchSAMLMetadataFromURL(ctx context.Context, client *http.Client, metadataURL string) ([]byte, *saml.EntityDescriptor, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, metadataURL, nil)
targetURL, err := securityutil.NormalizeAbsoluteHTTPURL(metadataURL)
if err != nil {
return nil, nil, err
}
req, err := securityutil.NewValidatedRequestWithContext(ctx, http.MethodGet, targetURL, nil)
if err != nil {
return nil, nil, err
}

8
internal/api/saml_service.go
View file

@ -19,6 +19,7 @@ import (
"github.com/crewjam/saml"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rs/zerolog/log"
)
@ -121,7 +122,12 @@ func (s *SAMLService) loadIDPMetadata(ctx context.Context) error {
}
func (s *SAMLService) fetchIDPMetadataFromURL(ctx context.Context, metadataURL string) (*saml.EntityDescriptor, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, metadataURL, nil)
targetURL, err := securityutil.NormalizeAbsoluteHTTPURL(metadataURL)
if err != nil {
return nil, err
}
req, err := securityutil.NewValidatedRequestWithContext(ctx, http.MethodGet, targetURL, nil)
if err != nil {
return nil, err
}

7
internal/notifications/notifications.go
View file

@ -19,6 +19,7 @@ import (
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rs/zerolog/log"
)
@ -1515,6 +1516,10 @@ func (n *NotificationManager) sendAppriseViaHTTP(cfg AppriseConfig, title, body,
}
requestURL := strings.TrimRight(serverURL, "/") + notifyEndpoint
targetURL, err := securityutil.NormalizeAbsoluteHTTPURL(requestURL)
if err != nil {
return fmt.Errorf("apprise server URL validation failed: %w", err)
}
payload := map[string]any{
"body": body,
@ -1532,7 +1537,7 @@ func (n *NotificationManager) sendAppriseViaHTTP(cfg AppriseConfig, title, body,
return fmt.Errorf("failed to marshal Apprise payload: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, requestURL, bytes.NewReader(payloadBytes))
req, err := securityutil.NewValidatedRequestWithContext(ctx, http.MethodPost, targetURL, bytes.NewReader(payloadBytes))
if err != nil {
return fmt.Errorf("failed to create Apprise request: %w", err)
}

41
internal/notifications/notifications_security_http_test.go
View file

@ -3,6 +3,7 @@ package notifications
import (
"encoding/json"
"net/http"
"net/url"
"strings"
"testing"
)
@ -27,6 +28,46 @@ func TestSendAppriseViaHTTPInvalidScheme(t *testing.T) {
}
}
func TestSendAppriseViaHTTPRejectsUserinfoServerURL(t *testing.T) {
nm := NewNotificationManager("")
defer nm.Stop()
called := make(chan struct{}, 1)
server := newIPv4HTTPServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case called <- struct{}{}:
default:
}
w.WriteHeader(http.StatusOK)
w.Write([]byte(`{"ok":true}`))
}))
defer server.Close()
if err := nm.UpdateAllowedPrivateCIDRs("127.0.0.1/32"); err != nil {
t.Fatalf("allowlist: %v", err)
}
serverURL, err := url.Parse(server.URL)
if err != nil {
t.Fatalf("parse server URL: %v", err)
}
serverURL.User = url.UserPassword("user", "pass")
err = nm.sendAppriseViaHTTP(AppriseConfig{
ServerURL: serverURL.String(),
TimeoutSeconds: 2,
}, "title", "body", "info")
if err == nil || !strings.Contains(err.Error(), "userinfo") {
t.Fatalf("expected userinfo validation error, got %v", err)
}
select {
case <-called:
t.Fatal("expected userinfo-bearing Apprise URL to be rejected before outbound delivery")
default:
}
}
func TestSendAppriseViaHTTPBlocksRedirectToLinkLocal(t *testing.T) {
nm := NewNotificationManager("")
defer nm.Stop()

201
internal/securityutil/httpurl.go Normal file
View file

@ -0,0 +1,201 @@
package securityutil
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"path"
"strings"
)
const requestPlaceholderURL = "http://pulse.invalid"
func cloneURL(u *url.URL) *url.URL {
if u == nil {
return nil
}
cloned := *u
return &cloned
}
func joinURLPath(basePath, relativePath string) string {
parts := []string{basePath}
if trimmed := strings.Trim(relativePath, "/"); trimmed != "" {
parts = append(parts, trimmed)
}
joined := path.Join(parts...)
switch joined {
case ".", "/":
return ""
default:
if strings.HasPrefix(joined, "/") {
return joined
}
return "/" + joined
}
}
// NormalizeAbsoluteHTTPURL validates a fully-qualified HTTP(S) URL.
func NormalizeAbsoluteHTTPURL(raw string) (*url.URL, error) {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return nil, fmt.Errorf("URL is required")
}
parsed, err := url.Parse(trimmed)
if err != nil {
return nil, fmt.Errorf("invalid URL: %w", err)
}
if parsed.Scheme != "http" && parsed.Scheme != "https" {
return nil, fmt.Errorf("URL scheme must be http or https")
}
if parsed.Host == "" {
return nil, fmt.Errorf("URL host is required")
}
if parsed.User != nil {
return nil, fmt.Errorf("URL userinfo is not allowed")
}
if parsed.Hostname() == "" {
return nil, fmt.Errorf("URL hostname is required")
}
return parsed, nil
}
// NormalizeHTTPBaseURL validates a base HTTP(S) URL and optionally adds a default scheme.
func NormalizeHTTPBaseURL(raw string, defaultScheme string) (*url.URL, error) {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return nil, fmt.Errorf("base URL is required")
}
if defaultScheme != "" && !strings.Contains(trimmed, "://") {
trimmed = defaultScheme + "://" + trimmed
}
parsed, err := NormalizeAbsoluteHTTPURL(trimmed)
if err != nil {
return nil, err
}
if parsed.RawQuery != "" || parsed.Fragment != "" {
return nil, fmt.Errorf("base URL must not include query or fragment")
}
cleanedPath := path.Clean(parsed.Path)
switch cleanedPath {
case ".", "/":
parsed.Path = ""
default:
if !strings.HasPrefix(cleanedPath, "/") {
cleanedPath = "/" + cleanedPath
}
parsed.Path = cleanedPath
}
parsed.RawPath = ""
return parsed, nil
}
// AppendURLPath appends path segments onto a validated base URL.
func AppendURLPath(base *url.URL, segments ...string) *url.URL {
cloned := cloneURL(base)
if cloned == nil {
return nil
}
parts := []string{cloned.Path}
for _, segment := range segments {
trimmed := strings.Trim(segment, "/")
if trimmed == "" {
continue
}
parts = append(parts, trimmed)
}
joined := path.Join(parts...)
if joined == "." || joined == "/" {
cloned.Path = ""
} else if strings.HasPrefix(joined, "/") {
cloned.Path = joined
} else {
cloned.Path = "/" + joined
}
cloned.RawPath = ""
cloned.Fragment = ""
return cloned
}
// ResolveRelativeURL validates a rooted relative path and resolves it against base.
func ResolveRelativeURL(base *url.URL, relativePath string) (*url.URL, error) {
if base == nil {
return nil, fmt.Errorf("base URL is required")
}
trimmed := strings.TrimSpace(relativePath)
if trimmed == "" {
return nil, fmt.Errorf("relative path is required")
}
if strings.Contains(trimmed, `\`) {
return nil, fmt.Errorf("relative path must not contain backslashes")
}
ref, err := url.Parse(trimmed)
if err != nil {
return nil, fmt.Errorf("invalid relative path: %w", err)
}
if ref.IsAbs() || ref.Host != "" || ref.User != nil {
return nil, fmt.Errorf("relative path must not include scheme or host")
}
if !strings.HasPrefix(ref.Path, "/") {
return nil, fmt.Errorf("relative path must start with '/'")
}
cleanedPath := path.Clean(ref.Path)
if !strings.HasPrefix(cleanedPath, "/") {
cleanedPath = "/" + cleanedPath
}
target := cloneURL(base)
if target == nil {
return nil, fmt.Errorf("base URL is required")
}
target.Path = joinURLPath(base.Path, cleanedPath)
escapedPath := path.Clean(ref.EscapedPath())
if !strings.HasPrefix(escapedPath, "/") {
escapedPath = "/" + escapedPath
}
target.RawPath = joinURLPath(base.EscapedPath(), escapedPath)
if target.RawPath == target.Path {
target.RawPath = ""
}
target.RawQuery = ref.RawQuery
target.Fragment = ""
return target, nil
}
// NewValidatedRequestWithContext builds an HTTP request from a pre-validated URL.
func NewValidatedRequestWithContext(ctx context.Context, method string, target *url.URL, body io.Reader) (*http.Request, error) {
if target == nil {
return nil, fmt.Errorf("target URL is required")
}
req, err := http.NewRequestWithContext(ctx, method, requestPlaceholderURL, body)
if err != nil {
return nil, err
}
req.URL = cloneURL(target)
req.Host = req.URL.Host
req.RequestURI = ""
return req, nil
}
// NewRelativeRequestWithContext validates a rooted relative path and builds a request from it.
func NewRelativeRequestWithContext(ctx context.Context, method string, base *url.URL, relativePath string, body io.Reader) (*http.Request, error) {
target, err := ResolveRelativeURL(base, relativePath)
if err != nil {
return nil, err
}
return NewValidatedRequestWithContext(ctx, method, target, body)
}

85
internal/securityutil/httpurl_test.go Normal file
View file

@ -0,0 +1,85 @@
package securityutil
import (
"context"
"strings"
"testing"
)
func TestNormalizeHTTPBaseURLAddsDefaultScheme(t *testing.T) {
parsed, err := NormalizeHTTPBaseURL("pbs.example.com:8007", "https")
if err != nil {
t.Fatalf("NormalizeHTTPBaseURL() error = %v", err)
}
if got := parsed.String(); got != "https://pbs.example.com:8007" {
t.Fatalf("NormalizeHTTPBaseURL() = %q", got)
}
}
func TestNormalizeHTTPBaseURLRejectsQuery(t *testing.T) {
if _, err := NormalizeHTTPBaseURL("https://example.com/path?x=1", ""); err == nil {
t.Fatal("expected query-bearing base URL to be rejected")
}
}
func TestResolveRelativeURLRejectsAbsoluteURL(t *testing.T) {
base, err := NormalizeHTTPBaseURL("https://example.com/api", "")
if err != nil {
t.Fatalf("NormalizeHTTPBaseURL() error = %v", err)
}
if _, err := ResolveRelativeURL(base, "https://evil.example.com"); err == nil {
t.Fatal("expected absolute URL to be rejected")
}
}
func TestAppendURLPathPreservesBasePath(t *testing.T) {
base, err := NormalizeHTTPBaseURL("https://issuer.example.com/realms/pulse", "")
if err != nil {
t.Fatalf("NormalizeHTTPBaseURL() error = %v", err)
}
appended := AppendURLPath(base, ".well-known", "openid-configuration")
if appended == nil {
t.Fatal("AppendURLPath() returned nil")
}
if got := appended.String(); got != "https://issuer.example.com/realms/pulse/.well-known/openid-configuration" {
t.Fatalf("AppendURLPath() = %q", got)
}
}
func TestNewRelativeRequestWithContextUsesValidatedURL(t *testing.T) {
base, err := NormalizeHTTPBaseURL("https://example.com/api2/json", "")
if err != nil {
t.Fatalf("NormalizeHTTPBaseURL() error = %v", err)
}
req, err := NewRelativeRequestWithContext(context.Background(), "GET", base, "/nodes", nil)
if err != nil {
t.Fatalf("NewRelativeRequestWithContext() error = %v", err)
}
if got := req.URL.String(); got != "https://example.com/api2/json/nodes" {
t.Fatalf("request URL = %q", got)
}
if strings.Contains(req.URL.String(), "pulse.invalid") {
t.Fatalf("placeholder URL leaked into request: %q", req.URL.String())
}
}
func TestNewRelativeRequestWithContextPreservesEscapedPath(t *testing.T) {
base, err := NormalizeHTTPBaseURL("https://example.com/api2/json", "")
if err != nil {
t.Fatalf("NormalizeHTTPBaseURL() error = %v", err)
}
req, err := NewRelativeRequestWithContext(context.Background(), "GET", base, "/nodes/node%2F1/backup", nil)
if err != nil {
t.Fatalf("NewRelativeRequestWithContext() error = %v", err)
}
if got := req.URL.Path; got != "/api2/json/nodes/node/1/backup" {
t.Fatalf("request path = %q", got)
}
if got := req.URL.EscapedPath(); got != "/api2/json/nodes/node%2F1/backup" {
t.Fatalf("request escaped path = %q", got)
}
}

12
internal/securityutil/storagepath.go Normal file
View file

@ -0,0 +1,12 @@
package securityutil
import (
"crypto/sha256"
"encoding/hex"
)
// HashedStorageName derives an opaque, fixed-width storage filename stem from an external identifier.
func HashedStorageName(id string) string {
sum := sha256.Sum256([]byte(id))
return hex.EncodeToString(sum[:])
}

205
internal/servicediscovery/store.go
View file

@ -11,6 +11,7 @@ import (
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/crypto"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rs/zerolog/log"
)
@ -207,10 +208,38 @@ func NewStore(dataDir string) (*Store, error) {
// getFilePath returns the file path for a resource ID.
func (s *Store) getFilePath(id string) string {
// Sanitize ID for filename: replace : with _
safeID := strings.ReplaceAll(id, ":", "_")
safeID = strings.ReplaceAll(safeID, "/", "_")
return filepath.Join(s.dataDir, safeID+".enc")
return filepath.Join(s.dataDir, securityutil.HashedStorageName(id)+".enc")
}
func (s *Store) findLegacyDiscoveryPath(id string) (string, error) {
canonicalName := securityutil.HashedStorageName(id) + ".enc"
entries, err := os.ReadDir(s.dataDir)
if err != nil {
if os.IsNotExist(err) {
return "", nil
}
return "", fmt.Errorf("failed to scan discovery directory: %w", err)
}
for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".enc") {
continue
}
if entry.Name() == canonicalName {
continue
}
path := filepath.Join(s.dataDir, entry.Name())
storedID, err := s.readDiscoveryIDFromPath(path)
if err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to inspect legacy discovery candidate")
continue
}
if storedID == id {
return path, nil
}
}
return "", nil
}
// readRegularFileWithLimit reads a file with a strict size cap and rejects non-regular files.
@ -307,6 +336,23 @@ func (s *Store) maybeRewritePlaintextDiscovery(filePath string, data []byte, mig
return nil
}
func (s *Store) marshalDiscoveryForStorage(discovery *ResourceDiscovery) ([]byte, error) {
data, err := marshalDiscovery(discovery)
if err != nil {
return nil, fmt.Errorf("failed to marshal discovery: %w", err)
}
if s.crypto != nil {
encrypted, err := s.crypto.Encrypt(data)
if err != nil {
return nil, fmt.Errorf("failed to encrypt discovery: %w", err)
}
data = encrypted
}
return data, nil
}
// Save persists a discovery to encrypted storage.
func (s *Store) Save(d *ResourceDiscovery) error {
s.mu.Lock()
@ -326,30 +372,24 @@ func (s *Store) Save(d *ResourceDiscovery) error {
toSave := cloneResourceDiscovery(d)
normalizeDiscovery(toSave)
data, err := marshalDiscovery(toSave)
data, err := s.marshalDiscoveryForStorage(toSave)
if err != nil {
return fmt.Errorf("failed to marshal discovery: %w", err)
}
// Encrypt if crypto is available
if s.crypto != nil {
encrypted, err := s.crypto.Encrypt(data)
if err != nil {
return fmt.Errorf("failed to encrypt discovery: %w", err)
}
data = encrypted
}
filePath := s.getFilePath(d.ID)
if err := s.persistDiscoveryBytes(filePath, data); err != nil {
return err
}
// Update cache
s.cache[d.ID] = toSave
s.cacheTime[d.ID] = time.Now()
filePath := s.getFilePath(toSave.ID)
if err := s.persistDiscoveryBytes(filePath, data); err != nil {
return err
}
if legacyPath, err := s.findLegacyDiscoveryPath(toSave.ID); err == nil && legacyPath != "" && legacyPath != filePath {
_ = os.Remove(legacyPath)
}
log.Debug().Str("id", d.ID).Str("service", d.ServiceType).Msg("discovery saved")
// Update cache
s.cache[toSave.ID] = toSave
s.cacheTime[toSave.ID] = time.Now()
log.Debug().Str("id", toSave.ID).Str("service", toSave.ServiceType).Msg("discovery saved")
return nil
}
@ -371,11 +411,20 @@ func (s *Store) Get(id string) (*ResourceDiscovery, error) {
defer s.mu.Unlock()
filePath := s.getFilePath(id)
activePath := filePath
data, migratedPlaintext, err := s.loadDiscoveryFileData(filePath, maxDiscoveryFileReadBytes)
if err != nil {
if os.IsNotExist(err) {
return nil, nil // Not found is not an error
} else {
activePath, err = s.findLegacyDiscoveryPath(id)
if err != nil {
return nil, err
}
if activePath == "" {
return nil, nil // Not found is not an error
}
data, migratedPlaintext, err = s.loadDiscoveryFileData(activePath, maxDiscoveryFileReadBytes)
}
if err != nil {
return nil, fmt.Errorf("failed to read discovery file: %w", err)
}
}
@ -384,12 +433,22 @@ func (s *Store) Get(id string) (*ResourceDiscovery, error) {
if err := unmarshalStoredDiscovery(data, &discovery); err != nil {
return nil, fmt.Errorf("failed to unmarshal discovery: %w", err)
}
if err := s.maybeRewritePlaintextDiscovery(filePath, data, migratedPlaintext); err != nil {
if err := s.maybeRewritePlaintextDiscovery(activePath, data, migratedPlaintext); err != nil {
return nil, err
}
// Canonicalize stored fields loaded from disk.
normalizeDiscovery(&discovery)
if activePath != filePath {
rewritten, err := s.marshalDiscoveryForStorage(&discovery)
if err != nil {
return nil, err
}
if err := s.persistDiscoveryBytes(filePath, rewritten); err != nil {
return nil, err
}
_ = os.Remove(activePath)
}
// Update cache
s.cache[id] = cloneResourceDiscovery(&discovery)
@ -410,11 +469,25 @@ func (s *Store) Delete(id string) error {
defer s.mu.Unlock()
filePath := s.getFilePath(id)
var removed bool
if err := os.Remove(filePath); err != nil {
if os.IsNotExist(err) {
return nil // Already deleted
if !os.IsNotExist(err) {
return fmt.Errorf("failed to delete discovery file: %w", err)
}
return fmt.Errorf("failed to delete discovery file: %w", err)
} else {
removed = true
}
if legacyPath, err := s.findLegacyDiscoveryPath(id); err == nil && legacyPath != "" && legacyPath != filePath {
if err := os.Remove(legacyPath); err != nil {
if !os.IsNotExist(err) {
return fmt.Errorf("failed to delete legacy discovery file: %w", err)
}
} else {
removed = true
}
}
if !removed {
return nil // Already deleted
}
// Remove from cache
@ -577,10 +650,17 @@ func (s *Store) Exists(id string) bool {
filePath := s.getFilePath(id)
_, err := os.Stat(filePath)
if err == nil {
return true
}
if err != nil && !os.IsNotExist(err) {
log.Warn().Err(err).Str("id", id).Str("file", filePath).Msg("Failed to stat discovery file")
}
return err == nil
legacyPath, findErr := s.findLegacyDiscoveryPath(id)
if findErr != nil {
log.Warn().Err(findErr).Str("id", id).Msg("Failed to scan for legacy discovery file")
}
return legacyPath != ""
}
// GetAge returns how old the discovery is, or -1 if not found.
@ -605,10 +685,46 @@ func (s *Store) NeedsRefresh(id string, maxAge time.Duration) bool {
// getFingerprintFilePath returns the file path for a fingerprint.
func (s *Store) getFingerprintFilePath(resourceID string) string {
// Sanitize ID for filename
safeID := strings.ReplaceAll(resourceID, ":", "_")
safeID = strings.ReplaceAll(safeID, "/", "_")
return filepath.Join(s.fingerprintDir, safeID+".json")
return filepath.Join(s.fingerprintDir, securityutil.HashedStorageName(resourceID)+".json")
}
func (s *Store) findLegacyFingerprintPath(resourceID string) (string, error) {
canonicalName := securityutil.HashedStorageName(resourceID) + ".json"
entries, err := os.ReadDir(s.fingerprintDir)
if err != nil {
if os.IsNotExist(err) {
return "", nil
}
return "", fmt.Errorf("failed to scan fingerprint directory: %w", err)
}
for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".json") {
continue
}
if entry.Name() == canonicalName {
continue
}
path := filepath.Join(s.fingerprintDir, entry.Name())
data, err := readRegularFileWithLimit(path, maxFingerprintFileReadBytes)
if err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to inspect legacy fingerprint candidate")
continue
}
var fp ContainerFingerprint
if err := unmarshalStoredFingerprint(data, &fp); err != nil {
log.Warn().Err(err).Str("file", entry.Name()).Msg("failed to parse legacy fingerprint candidate")
continue
}
canonicalizeFingerprint(&fp)
if fp.ResourceID == resourceID {
return path, nil
}
}
return "", nil
}
// loadFingerprints loads all fingerprints from disk into memory.
@ -681,6 +797,9 @@ func (s *Store) SaveFingerprint(fp *ContainerFingerprint) error {
}
return fmt.Errorf("failed to finalize fingerprint file: %w", err)
}
if legacyPath, err := s.findLegacyFingerprintPath(fpCopy.ResourceID); err == nil && legacyPath != "" && legacyPath != filePath {
_ = os.Remove(legacyPath)
}
return nil
}
@ -808,8 +927,17 @@ func (s *Store) CleanupOrphanedFingerprints(currentResourceIDs map[string]bool)
// Remove from disk
filePath := s.getFingerprintFilePath(fpID)
removeErr := error(nil)
if err := os.Remove(filePath); err != nil && !os.IsNotExist(err) {
log.Warn().Err(err).Str("id", fpID).Msg("failed to remove orphaned fingerprint file")
removeErr = err
}
if legacyPath, err := s.findLegacyFingerprintPath(fpID); err == nil && legacyPath != "" && legacyPath != filePath {
if err := os.Remove(legacyPath); err != nil && !os.IsNotExist(err) {
removeErr = err
}
}
if removeErr != nil {
log.Warn().Err(removeErr).Str("id", fpID).Msg("failed to remove orphaned fingerprint file")
} else {
log.Debug().Str("id", fpID).Msg("removed orphaned fingerprint")
}
@ -886,8 +1014,7 @@ func filenameToResourceID(filename string) string {
return resourceType + ":" + host + ":" + resourceID
}
func (s *Store) readDiscoveryIDFromFile(filename string) (string, error) {
filePath := filepath.Join(s.dataDir, filename)
func (s *Store) readDiscoveryIDFromPath(filePath string) (string, error) {
data, migratedPlaintext, err := s.loadDiscoveryFileData(filePath, maxDiscoveryFileReadBytes)
if err != nil {
return "", fmt.Errorf("failed to read discovery file: %w", err)
@ -909,6 +1036,10 @@ func (s *Store) readDiscoveryIDFromFile(filename string) (string, error) {
return discovery.ID, nil
}
func (s *Store) readDiscoveryIDFromFile(filename string) (string, error) {
return s.readDiscoveryIDFromPath(filepath.Join(s.dataDir, filename))
}
// ListDiscoveryIDs returns all discovery IDs currently stored.
func (s *Store) ListDiscoveryIDs() []string {
entries, err := os.ReadDir(s.dataDir)

7
pkg/audit/webhook.go
View file

@ -12,6 +12,7 @@ import (
"sync"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rs/zerolog/log"
)
@ -225,8 +226,12 @@ func (w *WebhookDelivery) deliver(url string, event Event) error {
if err := validateWebhookURL(ctx, url); err != nil {
return fmt.Errorf("webhook URL blocked: %w", err)
}
targetURL, err := securityutil.NormalizeAbsoluteHTTPURL(url)
if err != nil {
return fmt.Errorf("webhook URL blocked: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
req, err := securityutil.NewValidatedRequestWithContext(ctx, http.MethodPost, targetURL, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("failed to create webhook request: %w", err)
}

44
pkg/pbs/client.go
View file

@ -12,6 +12,7 @@ import (
"sync"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rcourtman/pulse-go-rewrite/pkg/tlsutil"
"github.com/rs/zerolog/log"
)
@ -32,11 +33,28 @@ func readResponseBodyLimited(r io.Reader) ([]byte, error) {
// Client represents a Proxmox Backup Server API client
type Client struct {
baseURL string
baseAPIURL *url.URL
httpClient *http.Client
auth auth
config ClientConfig
}
func (c *Client) apiBaseURL() (*url.URL, error) {
if c.baseAPIURL != nil {
return c.baseAPIURL, nil
}
baseURL := strings.TrimSpace(c.baseURL)
if baseURL == "" {
return nil, fmt.Errorf("base URL is required")
}
parsed, err := securityutil.NormalizeHTTPBaseURL(baseURL, "")
if err != nil {
return nil, fmt.Errorf("invalid base URL: %w", err)
}
c.baseAPIURL = parsed
return parsed, nil
}
// ClientConfig holds configuration for the PBS client
type ClientConfig struct {
Host string
@ -74,6 +92,11 @@ func NewClient(cfg ClientConfig) (*Client, error) {
if strings.HasPrefix(cfg.Host, "http://") {
log.Warn().Str("host", cfg.Host).Msg("Using HTTP for PBS connection. PBS typically requires HTTPS. If connection fails, try using https:// instead")
}
baseHostURL, err := securityutil.NormalizeHTTPBaseURL(cfg.Host, "")
if err != nil {
return nil, fmt.Errorf("invalid PBS host: %w", err)
}
baseAPIURL := securityutil.AppendURLPath(baseHostURL, "api2", "json")
var user, realm string
@ -129,7 +152,8 @@ func NewClient(cfg ClientConfig) (*Client, error) {
httpClient := tlsutil.CreateHTTPClientWithTimeout(cfg.VerifySSL, cfg.Fingerprint, timeout)
client := &Client{
baseURL: strings.TrimSuffix(cfg.Host, "/") + "/api2/json",
baseURL: baseAPIURL.String(),
baseAPIURL: baseAPIURL,
httpClient: httpClient,
config: cfg,
auth: auth{
@ -175,7 +199,11 @@ func (c *Client) authenticateJSON(ctx context.Context, username, password string
return err
}
req, err := http.NewRequestWithContext(ctx, "POST", c.baseURL+"/access/ticket", bytes.NewReader(body))
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, "POST", baseAPIURL, "/access/ticket", bytes.NewReader(body))
if err != nil {
return err
}
@ -196,7 +224,11 @@ func (c *Client) authenticateForm(ctx context.Context, username, password string
"password": {password},
}
req, err := http.NewRequestWithContext(ctx, "POST", c.baseURL+"/access/ticket", strings.NewReader(data.Encode()))
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, "POST", baseAPIURL, "/access/ticket", strings.NewReader(data.Encode()))
if err != nil {
return err
}
@ -274,7 +306,11 @@ func (c *Client) request(ctx context.Context, method, path string, data url.Valu
body = strings.NewReader(data.Encode())
}
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, body)
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return nil, err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, method, baseAPIURL, path, body)
if err != nil {
return nil, err
}

44
pkg/pmg/client.go
View file

@ -12,6 +12,7 @@ import (
"sync"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rcourtman/pulse-go-rewrite/pkg/tlsutil"
"github.com/rs/zerolog/log"
)
@ -31,12 +32,29 @@ func readResponseBodyLimited(r io.Reader) ([]byte, error) {
type Client struct {
baseURL string
baseAPIURL *url.URL
httpClient *http.Client
auth auth
config ClientConfig
mu sync.Mutex
}
func (c *Client) apiBaseURL() (*url.URL, error) {
if c.baseAPIURL != nil {
return c.baseAPIURL, nil
}
baseURL := strings.TrimSpace(c.baseURL)
if baseURL == "" {
return nil, fmt.Errorf("base URL is required")
}
parsed, err := securityutil.NormalizeHTTPBaseURL(baseURL, "")
if err != nil {
return nil, fmt.Errorf("invalid base URL: %w", err)
}
c.baseAPIURL = parsed
return parsed, nil
}
type ClientConfig struct {
Host string
User string
@ -170,6 +188,11 @@ func NewClient(cfg ClientConfig) (*Client, error) {
if strings.HasPrefix(cfg.Host, "http://") {
log.Warn().Str("host", cfg.Host).Msg("Using HTTP for PMG connection - consider enabling HTTPS")
}
baseHostURL, err := securityutil.NormalizeHTTPBaseURL(cfg.Host, "")
if err != nil {
return nil, fmt.Errorf("invalid PMG host: %w", err)
}
baseAPIURL := securityutil.AppendURLPath(baseHostURL, "api2", "json")
var user, realm string
@ -212,7 +235,8 @@ func NewClient(cfg ClientConfig) (*Client, error) {
httpClient := tlsutil.CreateHTTPClientWithTimeout(cfg.VerifySSL, cfg.Fingerprint, cfg.Timeout)
client := &Client{
baseURL: strings.TrimSuffix(cfg.Host, "/") + "/api2/json",
baseURL: baseAPIURL.String(),
baseAPIURL: baseAPIURL,
httpClient: httpClient,
config: cfg,
auth: auth{
@ -248,7 +272,11 @@ func (c *Client) authenticate(ctx context.Context) error {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/access/ticket", bytes.NewReader(body))
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, http.MethodPost, baseAPIURL, "/access/ticket", bytes.NewReader(body))
if err != nil {
return err
}
@ -276,7 +304,11 @@ func (c *Client) authenticateForm(ctx context.Context, username, password string
"password": {password},
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/access/ticket", strings.NewReader(data.Encode()))
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, http.MethodPost, baseAPIURL, "/access/ticket", strings.NewReader(data.Encode()))
if err != nil {
return err
}
@ -373,7 +405,11 @@ func (c *Client) request(ctx context.Context, method, path string, params url.Va
return nil, err
}
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, body)
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return nil, err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, method, baseAPIURL, path, body)
if err != nil {
return nil, err
}

56
pkg/proxmox/client.go
View file

@ -14,6 +14,7 @@ import (
"time"
"unicode"
"github.com/rcourtman/pulse-go-rewrite/internal/securityutil"
"github.com/rcourtman/pulse-go-rewrite/pkg/tlsutil"
"github.com/rs/zerolog/log"
)
@ -133,11 +134,28 @@ func coerceUint64(field string, value interface{}) (uint64, error) {
// Client represents a Proxmox VE API client
type Client struct {
baseURL string
baseAPIURL *url.URL
httpClient *http.Client
auth auth
config ClientConfig
}
func (c *Client) apiBaseURL() (*url.URL, error) {
if c.baseAPIURL != nil {
return c.baseAPIURL, nil
}
baseURL := strings.TrimSpace(c.baseURL)
if baseURL == "" {
return nil, fmt.Errorf("base URL is required")
}
parsed, err := securityutil.NormalizeHTTPBaseURL(baseURL, "")
if err != nil {
return nil, fmt.Errorf("invalid base URL: %w", err)
}
c.baseAPIURL = parsed
return parsed, nil
}
// ClientConfig holds configuration for the Proxmox client
type ClientConfig struct {
Host string
@ -206,6 +224,23 @@ func NewClient(cfg ClientConfig) (*Client, error) {
}
httpClient := tlsutil.CreateHTTPClientWithTimeout(cfg.VerifySSL, cfg.Fingerprint, timeout)
baseHost := strings.TrimSpace(cfg.Host)
if baseHost == "" {
return nil, fmt.Errorf("host is required")
}
if !strings.HasPrefix(baseHost, "http://") && !strings.HasPrefix(baseHost, "https://") {
baseHost = "https://" + baseHost
log.Debug().Str("host", baseHost).Msg("No protocol specified in Proxmox host, defaulting to HTTPS")
}
baseHostURL, err := securityutil.NormalizeHTTPBaseURL(baseHost, "")
if err != nil {
return nil, fmt.Errorf("invalid Proxmox host: %w", err)
}
if baseHostURL.Scheme == "http" {
log.Warn().Str("host", baseHostURL.String()).Msg("Using HTTP for Proxmox connection - consider enabling HTTPS")
}
baseAPIURL := securityutil.AppendURLPath(baseHostURL, "api2", "json")
// Extract just the token name part for API token authentication
tokenName := cfg.TokenName
if cfg.TokenName != "" && strings.Contains(cfg.TokenName, "!") {
@ -222,7 +257,8 @@ func NewClient(cfg ClientConfig) (*Client, error) {
Msg("Proxmox client configured")
client := &Client{
baseURL: strings.TrimSuffix(cfg.Host, "/") + "/api2/json",
baseURL: baseAPIURL.String(),
baseAPIURL: baseAPIURL,
httpClient: httpClient,
config: cfg,
auth: auth{
@ -268,7 +304,11 @@ func (c *Client) authenticateJSON(ctx context.Context, username, password string
return err
}
req, err := http.NewRequestWithContext(ctx, "POST", c.baseURL+"/access/ticket", bytes.NewReader(body))
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, "POST", baseAPIURL, "/access/ticket", bytes.NewReader(body))
if err != nil {
return err
}
@ -289,7 +329,11 @@ func (c *Client) authenticateForm(ctx context.Context, username, password string
"password": {password},
}
req, err := http.NewRequestWithContext(ctx, "POST", c.baseURL+"/access/ticket", strings.NewReader(data.Encode()))
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, "POST", baseAPIURL, "/access/ticket", strings.NewReader(data.Encode()))
if err != nil {
return err
}
@ -371,7 +415,11 @@ func (c *Client) requestWithRetry(ctx context.Context, method, path string, data
body = strings.NewReader(data.Encode())
}
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, body)
baseAPIURL, err := c.apiBaseURL()
if err != nil {
return nil, err
}
req, err := securityutil.NewRelativeRequestWithContext(ctx, method, baseAPIURL, path, body)
if err != nil {
return nil, err
}