Commit graph

349 commits

Author SHA1 Message Date
rcourtman
c8adbb7ae5 Add AI monitoring enhancements and host metadata features
- Add host metadata API for custom URL editing on hosts page
- Enhance AI routing with unified resource provider lookup
- Add encryption key watcher script for debugging key issues
- Improve AI service with better command timeout handling
- Update dev environment workflow with key monitoring docs
- Fix resource store deduplication logic
2025-12-09 16:27:46 +00:00
rcourtman
927ac76bad feat: AI integration, Docker metrics, RAID display, and infrastructure improvements
- Add Claude OAuth authentication support with hybrid API key/OAuth flow
- Implement Docker container historical metrics in backend and charts API
- Add CEPH cluster data collection and new Ceph page
- Enhance RAID status display with detailed tooltips and visual indicators
- Fix host deduplication logic with Docker bridge IP filtering
- Fix NVMe temperature collection in host agent
- Add comprehensive test coverage for new features
- Improve frontend sparklines and metrics history handling
- Fix navigation issues and frontend reload loops
2025-12-09 09:29:27 +00:00
rcourtman
e1e83c8295 fix: complete unified resources WebSocket integration
Backend:
- Call SetMonitor after router creation to inject resource store
- Add debug logging for resource population and broadcast

Frontend:
- Add resources array to WebSocket store initial state
- Handle resources in WebSocket message processing
- Use reconcile for efficient state updates

The unified resources are now properly:
1. Populated from StateSnapshot on each broadcast cycle
2. Converted to frontend format (ResourceFrontend)
3. Included in WebSocket state messages
4. Received and stored in frontend state
5. Consumed by migrated route components

Console now shows '[DashboardView] Using unified resources: VMs: X'
confirming the migration is working end-to-end.
2025-12-07 23:52:00 +00:00
rcourtman
f7735cca62 fix: Populate resources on-demand when /api/resources is called
The Resources page was showing 0 resources because the store was only
populated when /api/state was called (from the dashboard). Now the
resources are populated on-demand when /api/resources is accessed.

Changes:
- Added StateProvider interface to ResourceHandlers
- SetStateProvider() method for injecting the monitor
- HandleGetResources now calls PopulateFromSnapshot before querying
- Router injects monitor as state provider during SetMonitor()

This ensures the /resources page works even when accessed directly
without visiting the main dashboard first.
2025-12-07 14:47:29 +00:00
rcourtman
f34c28f970 feat: Complete Unified Resource Architecture (Phases 1-3)
This commit implements the Unified Resource Architecture for AI-first
infrastructure management. Key features:

Phase 1 - Backend Unification:
- New unified Resource type with 9 resource types, 7 platforms, 7 statuses
- Resource store with identity-based deduplication (hostname, machineID, IP)
- 8 converter functions (FromNode, FromVM, FromContainer, etc.)
- REST API endpoints: /api/resources, /api/resources/stats, /api/resources/{id}
- 28 comprehensive unit tests

Phase 2 - AI Context Enhancement:
- Unified context builder for AI system prompts
- Cross-platform query methods: GetTopByCPU, GetTopByMemory, GetTopByDisk
- Resource correlation: GetRelated (parent, children, siblings, cluster)
- Infrastructure summary: GetResourceSummary with health status counts
- AI context now includes top consumers and infrastructure overview

Phase 3 - Agent Preference & Hybrid Mode:
- Polling optimization methods in resource store
- ResourceStoreInterface added to Monitor
- SetResourceStore() and shouldSkipNodeMetrics() helper methods
- Store automatically wired into Monitor via Router.SetMonitor()
- Foundation ready for reduced API polling when agents are active

Files added:
- internal/resources/resource.go - Core Resource type
- internal/resources/store.go - Store with deduplication
- internal/resources/converters.go - Type converters
- internal/resources/platform_data.go - Platform-specific data
- internal/resources/store_test.go - 28 tests
- internal/resources/converters_test.go - Converter tests
- internal/api/resource_handlers.go - REST API handlers
- internal/ai/resource_context.go - AI context builder
- .gemini/docs/unified-resource-architecture.md - Architecture docs

All tests pass.
2025-12-07 13:49:00 +00:00
rcourtman
28ea85a0a0 Additional updates 2025-12-07 10:22:42 +00:00
rcourtman
bcd7b550d4 AI Problem Solver implementation and various fixes
- Implement 'Show Problems Only' toggle combining degraded status, high CPU/memory alerts, and needs backup filters
- Add 'Investigate with AI' button to filter bar for problematic guests
- Fix dashboard column sizing inconsistencies between bars and sparklines view modes
- Fix PBS backups display and polling
- Refine AI prompt for general-purpose usage
- Fix frontend flickering and reload loops during initial load
- Integrate persistent SQLite metrics store with Monitor
- Fortify AI command routing with improved validation and logging
- Fix CSRF token handling for note deletion
- Debug and fix AI command execution issues
- Various AI reliability improvements and command safety enhancements
2025-12-06 23:46:08 +00:00
rcourtman
8948e84fe5 feat: AI features, agent improvements, and host monitoring enhancements
AI Chat Integration:
- Multi-provider support (Anthropic, OpenAI, Ollama)
- Streaming responses with markdown rendering
- Agent command execution for remote troubleshooting
- Context-aware conversations with host/container metadata

Agent Updates:
- Add --enable-proxmox flag for automatic PVE/PBS token setup
- Improve auto-update with semver comparison (prevents downgrades)
- Add updatedFrom tracking to report previous version after update
- Reduce initial update check delay from 30s to 5s
- Add agent version column to Hosts page table

Host Metrics:
- Add DiskIO stats collection (read/write bytes, ops, time)
- Improve disk filtering to exclude Docker overlay mounts
- Add RAID array monitoring via mdadm
- Enhanced temperature sensor parsing

Frontend:
- New Agent Version column on Hosts overview table
- Improved node modal with agent-first installation flow
- Add DiskIO display in host drawer
- Better responsive handling for metric bars
2025-12-05 10:37:02 +00:00
rcourtman
53d7776d6b wip: AI chat integration with multi-provider support
- Add AI service with Anthropic, OpenAI, and Ollama providers
- Add AI chat UI component with streaming responses
- Add AI settings page for configuration
- Add agent exec framework for command execution
- Add API endpoints for AI chat and configuration
2025-12-04 20:16:53 +00:00
rcourtman
d0d989289a Refactor alert system: fix race conditions, memory leaks, and improve code quality
- Rename checkFlapping to checkFlappingLocked to clarify lock contract
- Replace goto statements with structured control flow
- Wire up unused recordAlertFired/recordAlertResolved metric hooks
- Add trackingMapCleanup goroutine to prevent memory leaks from stale entries
- Tighten alert ID validation to alphanumeric + safe punctuation
- Fix history save error handling to properly manage backup lifecycle
- Add auto-migration for deprecated GroupingWindow field
- Refactor 300+ line UpdateConfig into focused helper functions
- Unify duplicate evaluateVMCondition/evaluateContainerCondition
- Add constants for magic numbers (thresholds, timing, flapping)
- Update tests to match new backup behavior
2025-12-02 23:31:36 +00:00
rcourtman
bda8056e48 Add refresh-cluster button to detect new Proxmox cluster members
When new nodes are added to a Proxmox cluster after Pulse was
initially configured, they weren't showing up in Settings. The
existing "Refresh" button only triggered network discovery, not
cluster membership re-detection.

Changes:
- Add POST /api/config/nodes/{id}/refresh-cluster endpoint
- Add "Refresh" button in cluster node panel in Settings
- Re-detect cluster membership and update stored endpoints

Related to #799
2025-12-02 22:01:00 +00:00
rcourtman
4f824ab148 style: Apply gofmt to 37 files
Standardize code formatting across test files and monitor.go.
No functional changes.
2025-12-02 17:21:48 +00:00
rcourtman
cf26ed7f12 security: Add request body size limits to remaining API handlers
Add http.MaxBytesReader to 8 additional handlers to complete API
hardening against memory exhaustion attacks:

- guest_metadata.go: HandleUpdateMetadata (16KB)
- notification_queue.go: RetryDLQItem, DeleteDLQItem (8KB each)
- temperature_proxy.go: HandleRegister (8KB)
- host_agents.go: HandleReport (256KB)
- updates.go: HandleApplyUpdate (8KB)
- docker_metadata.go: HandleUpdateMetadata (16KB)
- system_settings.go: UpdateSystemSettings (64KB)

All API handlers that decode JSON request bodies now have size limits.
2025-12-02 16:47:13 +00:00
rcourtman
b4d497ce3b security: Add request body size limits to API handlers
Add http.MaxBytesReader to 16 additional handlers to prevent memory
exhaustion attacks via oversized request bodies:

- docker_agents.go: HandleReport (512KB), HandleCommandAck (8KB),
  HandleSetCustomDisplayName (8KB)
- alerts.go: UpdateAlertConfig (64KB), BulkAcknowledgeAlerts (32KB),
  BulkClearAlerts (32KB)
- config_handlers.go: HandleAddNode, HandleTestConnection,
  HandleUpdateNode, HandleTestNodeConfig (32KB each),
  HandleVerifyTemperatureSSH, HandleExportConfig, HandleDiscoverServers,
  HandleSetupScriptURL (8KB each), HandleImportConfig (1MB),
  HandleUpdateMockMode (16KB)
2025-12-02 16:43:13 +00:00
rcourtman
6eb7f06df1 security: Add request body size limits to notification handlers
Add http.MaxBytesReader limits to prevent memory exhaustion attacks:
- UpdateEmailConfig: 32KB limit
- UpdateAppriseConfig: 64KB limit
- CreateWebhook: 64KB limit
- UpdateWebhook: 64KB limit

This follows the pattern already used in system_settings.go for
SSH config validation.
2025-12-02 16:37:30 +00:00
rcourtman
c05817f9de docs: Add godoc comments to exported functions
Add missing godoc comments to:
- NewRateLimiter and Allow in ratelimit.go
- SnapshotSyncStatus in temperature_proxy.go
- NewClient and GetVersion in pkg/pmg/client.go
2025-12-02 15:58:59 +00:00
rcourtman
097976321b perf: Cache hostname lowercase in temperature proxy lookups
Pre-compute strings.ToLower(hostname) before loops that search for
matching PVE instances. Avoids repeated lowercasing in two functions.
2025-12-02 15:43:41 +00:00
rcourtman
98d170e087 perf: Cache err.Error() in cluster node validation
Cache err.Error() result in two locations in config_handlers.go:
- TLS mismatch detection (3x calls to 1)
- Standalone node detection (2x calls to 1)
2025-12-02 15:41:18 +00:00
rcourtman
158669296e refactor: Remove unreachable dead code branches
- firstForwardedValue: strings.Split always returns at least one element
- shouldRunBackupPoll: remaining is always >= 1 by math
- convertContainerDiskInfo: lowerLabel is never empty for non-rootfs

All three functions now at 100% coverage.
2025-12-02 14:41:53 +00:00
rcourtman
69bcd6ab0f test: Add SessionStore.load legacy format tests for API package 2025-12-02 14:12:32 +00:00
rcourtman
463d1087ba test: Add CSRFTokenStore.load format tests for API package
Cover legacy JSON format migration and current format with nil/expired
entries. Improves load function coverage from 67.9% to 100%.
2025-12-02 14:07:00 +00:00
rcourtman
9a40157aea test: Add CheckAuth tests for API package
Add tests for sliding expiration session validation and no-auth
configured scenarios. These test explicit paths for better coverage
documentation even though they were already exercised indirectly.
2025-12-02 13:57:23 +00:00
rcourtman
bbbeb45973 test: Add CheckCSRF valid token test for 100% coverage
Test the success path where a valid CSRF token is provided with a
matching session. This covers the final branch in CheckCSRF.
2025-12-02 13:51:27 +00:00
rcourtman
f2fdec9bd3 test: Add HandleSetupScript PBS path tests for API package
Cover the PBS script generation branch that was previously untested.
Verifies PBS-specific content, auth token handling, and placeholder host.
2025-12-02 13:36:23 +00:00
rcourtman
3970b9f9f5 test: Add CheckAuth tests for API package
Cover proxy auth headers, OIDC session validation, and session cookie
paths that were previously untested.
2025-12-02 13:32:21 +00:00
rcourtman
6065e9fbb0 test: Add CheckProxyAuth tests for API package
Add comprehensive direct tests for the CheckProxyAuth function covering:
- Not configured (returns false)
- Invalid secret (returns false)
- Missing secret header (returns false)
- Valid secret without user header configured (returns true, admin)
- Missing user header when configured (returns false)
- Valid auth with username (returns true with username)
- Role checking with empty roles header (defaults to admin)
- Role checking with admin role present (returns admin=true)
- Role checking without admin role (returns admin=false)
- Custom role separator (comma instead of pipe)
- Role with whitespace (trimmed correctly)

Coverage: CheckProxyAuth 89.3% → 100%
2025-12-02 13:28:02 +00:00
rcourtman
347f75541c test: Add ValidateSession tests for API package
Add comprehensive tests for the ValidateSession wrapper function covering:
- Non-existent token (returns false)
- Empty token (returns false)
- Valid token (returns true)
- Expired token (returns false)

The ValidateSession function is a simple wrapper around the SessionStore's
ValidateSession method, but having direct tests ensures the wrapper is
exercised and documents its expected behavior.

Coverage: ValidateSession 0% → 100%
2025-12-02 13:22:45 +00:00
rcourtman
0b5cbbe335 test: Add ensureScope tests for API package
Add comprehensive tests for the ensureScope function covering:
- Empty scope parameter (always allows access)
- No token in context (session-based request, allows access)
- Token with matching scope (allows access)
- Token with multiple scopes including required one (allows access)
- Token missing required scope (rejects with 403)
- Token with empty scopes (defaults to wildcard, allows access)
- Rejection returns proper JSON response format

Coverage: ensureScope 0% → 100%
Coverage: API package 32.1% → 32.2%
2025-12-02 13:19:11 +00:00
rcourtman
f7a0c2b055 test: Add RequireAuth tests for API package
Add comprehensive tests for the RequireAuth middleware covering:
- No auth configured (allows access by design)
- API-only mode (rejects requests without token)
- API-only mode (accepts valid X-API-Token)
- Basic auth with invalid credentials
- Basic auth JSON vs plain text error responses
- Valid basic auth (allowed)
- Proxy auth (allowed)
- Proxy auth with invalid secret (rejected)
- Bearer token with basic auth configured (allowed)
- Invalid Bearer token (rejected)

Coverage: RequireAuth 7.1% → 78.6%
Coverage: CheckAuth 66.9% → 69.1%
Coverage: API package 31.9% → 32.1%
2025-12-02 13:09:48 +00:00
rcourtman
d2f1cc21a7 test: Add RequireAdmin tests for API package
Add comprehensive tests for the RequireAdmin middleware covering:
- No auth configured (allows access by design)
- API-only mode (rejects requests without token)
- Basic auth with invalid credentials
- Proxy auth with admin role (allowed)
- Proxy auth with non-admin role (forbidden)
- Proxy auth with invalid secret (unauthorized)
- Proxy auth without role header (defaults to admin)
- Proxy auth with custom role separator
- Proxy auth with spaces in roles (trimmed)
- Basic auth authenticated users (allowed as admin)
- JSON vs plain text error responses based on path/Accept header

Also improves CheckProxyAuth coverage as a side effect.

Coverage: RequireAdmin 20.8% → 87.5%
Coverage: CheckProxyAuth 0.0% → 89.3%
Coverage: API package 30.9% → 31.9%
2025-12-02 13:06:06 +00:00
rcourtman
08e47c5849 test: Add isRequestAuthenticated tests for API package
Add comprehensive tests for the isRequestAuthenticated function covering:
- Nil inputs (config, request, both)
- Basic auth (valid, invalid password, invalid username, malformed base64, missing colon)
- API token via X-API-Token header
- API token via Bearer authorization header (case insensitive)
- Invalid/empty/whitespace API tokens
- No auth configured scenarios
- Empty session cookie handling

Coverage: isRequestAuthenticated 26.1% → 82.6%
Coverage: API package 30.7% → 30.9%
2025-12-02 12:59:18 +00:00
rcourtman
c82e3d5bb3 test: Add CheckCSRF tests for API package
Add comprehensive tests for the CheckCSRF function covering:
- Safe methods (GET, HEAD, OPTIONS) bypass
- API token authentication bypass
- Basic auth bypass
- No session cookie handling
- Missing CSRF token rejection with new token issuance
- Invalid CSRF token rejection with new token issuance
- CSRF token from FormValue
- Unsafe methods (POST, PUT, DELETE, PATCH) enforcement

Coverage: CheckCSRF 32.0% → 96.0%
Coverage: API package 30.5% → 30.7%
2025-12-02 12:53:32 +00:00
rcourtman
52e4e36504 test: Add resolvePublicURL tests for API package
Add comprehensive tests for the resolvePublicURL function covering:
- Configured PublicURL (simple, trailing slashes, ports, whitespace)
- Request-derived URL (HTTP, HTTPS via TLS, X-Forwarded-Proto)
- No host fallback (with/without frontend port)
- Nil request handling

Coverage: resolvePublicURL 12.5% → 100%
Coverage: API package 30.3% → 30.5%
2025-12-02 12:45:04 +00:00
rcourtman
59277343d5 fix: Use --ctid instead of --standalone --http-mode in quick-setup command
The quick-setup command for temperature monitoring was generating
--standalone --http-mode which is meant for Docker deployments. This
confused users trying to set up multi-server Proxmox monitoring.

Now uses --ctid which works for both local and remote Proxmox hosts.
The installer detects when the container doesn't exist locally and
installs in "host monitoring only" mode automatically.

If we can determine the actual CTID from the host proxy summary,
we use it; otherwise we show <PULSE_CTID> for the user to replace.

Related to #785
2025-12-02 11:38:47 +00:00
rcourtman
17c0254f43 test: Add diagnostics function tests for error handling
Add comprehensive tests for untested diagnostics functions:

- fingerprintPublicKey: 14 test cases covering empty/whitespace input,
  invalid key formats, truncated/malformed keys, and valid ED25519 keys

- countLegacySSHKeys: 8 test cases covering non-existent directories,
  empty directories, files without id_ prefix, multiple key types,
  and directory filtering (subdirectories not counted)

- resolveUserName: 4 test cases covering UID 0 (root), current user,
  non-existent UID fallback, and max uint32 boundary

- resolveGroupName: 4 test cases covering GID 0 (root/wheel), current
  group, non-existent GID fallback, and max uint32 boundary

Coverage: fingerprintPublicKey 0% -> 100%, countLegacySSHKeys 0% -> 100%,
resolveUserName 0% -> 100%, resolveGroupName 0% -> 100%
2025-12-02 03:39:52 +00:00
rcourtman
e248f2b895 fix: Update TestPublicURLDetectionUsesForwardedHeaders for proxy hardening
The test was failing after commit d6cbfc23 added security hardening
that requires authentication and trusted proxy configuration for
X-Forwarded-* headers to be read during public URL detection.

- Add API token authentication to the test request
- Configure 127.0.0.1 as trusted proxy for the test
- Add export_test.go with ResetTrustedProxyConfigForTests() to allow
  external tests to reset the trusted proxy configuration
2025-12-02 03:16:52 +00:00
rcourtman
24ae84671f test: Add handleHealth method tests
Add 2 tests for health endpoint:
- POST/PUT/DELETE/PATCH return 405 Method Not Allowed
- HEAD is allowed (same as GET)

Coverage: 50% → 83.3%
2025-12-02 02:09:04 +00:00
rcourtman
836303755f test: Add adminBypassEnabled tests
Add 5 tests to cover all branches:
- Not requested (ALLOW_ADMIN_BYPASS != "1")
- Enabled with PULSE_DEV=true
- Enabled with NODE_ENV=development
- Case-insensitive NODE_ENV check
- Declined when outside dev mode

Coverage: 40% → 100%
2025-12-02 01:53:55 +00:00
rcourtman
677d4417aa test: Add loadTrustedProxyCIDRs tests
Cover invalid CIDR, invalid IP, IPv6, and empty entry handling (48% to 100%)
2025-12-02 01:48:41 +00:00
rcourtman
daa11a072c test: Add LogAuditEvent tests
Cover success and failure logging branches (66.7% to 100%)
2025-12-02 01:46:27 +00:00
rcourtman
b9578b0665 test: Add SecurityHeadersWithConfig tests
Cover all CSP/X-Frame-Options embedding configurations (57.1% to 100%)
2025-12-02 01:44:22 +00:00
rcourtman
72fd885677 test: Add filesystem error tests for SessionStore
Cover saveUnsafe (57.9% to 89.5%) and load (55.6% to 74.1%) error paths
2025-12-02 01:40:54 +00:00
rcourtman
cd1420a085 test: Add filesystem error tests for CSRFTokenStore
Cover saveUnsafe (57.9% to 89.5%) and load (50.0% to 67.9%) error paths
2025-12-02 01:38:49 +00:00
rcourtman
eadd8442bb test: Add filesystem error tests for RecoveryTokenStore.saveUnsafe
Cover MkdirAll, WriteFile, and Rename error paths (46.7% to 86.7%)
2025-12-02 01:36:21 +00:00
rcourtman
a73dcd51a1 test: Add edge case tests for HandleSetupScript
Tests method not allowed, missing type parameter, and invalid host
parameter error paths (71.4% to 79.2% coverage).
2025-12-02 01:11:57 +00:00
rcourtman
13d05cbdc4 test: Add invalid PEM data test for newOIDCHTTPClient
Tests the error path when a CA bundle file contains non-PEM data
(81.8% to 86.4% coverage).
2025-12-02 01:00:06 +00:00
rcourtman
8eb195b9ff test: Add edge cases for UniversalRateLimitMiddleware
Cover nil config initialization and static asset bypass paths.
Coverage: UniversalRateLimitMiddleware 87.5% -> 100%
2025-12-02 00:54:37 +00:00
rcourtman
ba03551dba test: Add edge case for initializeBootstrapToken error path
Cover loadOrCreateBootstrapToken failure path.
Coverage: initializeBootstrapToken 92% -> 100%
2025-12-02 00:51:44 +00:00
rcourtman
475575212d test: Add edge cases for HandleLookup error paths
Cover method-not-allowed and missing lookup parameter paths.
Coverage: HandleLookup 88.4% -> 97.7%
2025-12-02 00:49:16 +00:00
rcourtman
4c7bcd12d0 test: Add edge cases for normalizeRequestedScopes
- Test blank scope identifier returns error
- Test wildcard-only input returns wildcard scope

Coverage: 89.7% → 96.6% (remaining 3.4% is defensive
unreachable code path)
2025-12-02 00:19:20 +00:00