vrr/Pulse
2
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-05-08 09:53:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
Pulse/internal/api
History
rcourtman 2786afdff0 feat: comprehensive diagnostics and observability improvements 
Upgrade diagnostics infrastructure from 5/10 to 8/10 production readiness
with enhanced metrics, logging, and request correlation capabilities.

**Request Correlation**
- Wire request IDs through context in middleware
- Return X-Request-ID header in all API responses
- Enable downstream log correlation across request lifecycle

**HTTP/API Metrics** (18 new Prometheus metrics)
- pulse_http_request_duration_seconds - API latency histogram
- pulse_http_requests_total - request counter by method/route/status
- pulse_http_request_errors_total - error counter by type
- Path normalization to control label cardinality

**Per-Node Poll Metrics**
- pulse_monitor_node_poll_duration_seconds - per-node timing
- pulse_monitor_node_poll_total - success/error counts per node
- pulse_monitor_node_poll_errors_total - error breakdown per node
- pulse_monitor_node_poll_last_success_timestamp - freshness tracking
- pulse_monitor_node_poll_staleness_seconds - age since last success
- Enables multi-node hotspot identification

**Scheduler Health Metrics**
- pulse_scheduler_queue_due_soon - ready queue depth
- pulse_scheduler_queue_depth - by instance type
- pulse_scheduler_queue_wait_seconds - time in queue histogram
- pulse_scheduler_dead_letter_depth - failed task tracking
- pulse_scheduler_breaker_state - circuit breaker state
- pulse_scheduler_breaker_failure_count - consecutive failures
- pulse_scheduler_breaker_retry_seconds - time until retry
- Enable alerting on DLQ spikes, breaker opens, queue backlogs

**Diagnostics Endpoint Caching**
- pulse_diagnostics_cache_hits_total - cache performance
- pulse_diagnostics_cache_misses_total - cache misses
- pulse_diagnostics_refresh_duration_seconds - probe timing
- 45-second TTL prevents thundering herd on /api/diagnostics
- Thread-safe with RWMutex
- X-Diagnostics-Cached-At header shows cache freshness

**Debug Log Performance**
- Gate high-frequency debug logs behind IsLevelEnabled() checks
- Reduces CPU waste in production when debug disabled
- Covers scheduler loops, poll cycles, API handlers

**Persistent Logging**
- File logging with automatic rotation
- LOG_FILE, LOG_MAX_SIZE, LOG_MAX_AGE, LOG_COMPRESS env vars
- MultiWriter sends logs to both stderr and file
- Gzip compression support for rotated logs

Files modified:
- internal/api/diagnostics.go (caching layer)
- internal/api/middleware.go (request IDs, HTTP metrics)
- internal/api/http_metrics.go (NEW - HTTP metric definitions)
- internal/logging/logging.go (file logging with rotation)
- internal/monitoring/metrics.go (node + scheduler metrics)
- internal/monitoring/monitor.go (instrumentation, debug gating)

Impact: Dramatically improved production troubleshooting with per-node
visibility, scheduler health metrics, persistent logs, and cached
diagnostics. Fast incident response now possible for multi-node deployments.
2025-10-21 12:37:39 +00:00
..
alerts.go feat: improve alert system performance, UX, and edge case handling 2025-10-21 11:05:45 +00:00
alerts_test.go Allow printable alert IDs for acknowledgements (#550) 2025-10-14 16:48:22 +00:00
auth.go feat: enhance sensor proxy with improved cluster discovery and SSH management 2025-10-17 11:43:26 +00:00
config_handlers.go fix: correct fmt.Sprintf argument alignment in PVE setup script 2025-10-20 21:58:37 +00:00
csrf_store.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
demo_middleware.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
diagnostics.go feat: comprehensive diagnostics and observability improvements 2025-10-21 12:37:39 +00:00
DO_NOT_EDIT_FRONTEND_HERE.md Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
docker_agents.go feat: add docker agent command handling 2025-10-15 19:27:19 +00:00
frontend_embed.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
guest_metadata.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
http_metrics.go feat: comprehensive diagnostics and observability improvements 2025-10-21 12:37:39 +00:00
middleware.go feat: comprehensive diagnostics and observability improvements 2025-10-21 12:37:39 +00:00
notifications.go Add Apprise notification integration (#570) 2025-10-18 16:39:39 +00:00
oidc_handlers.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
oidc_service.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
rate_limit_config.go Fix X-RateLimit-Limit header value 2025-10-19 11:43:03 +02:00
rate_limit_config_test.go test: add X-RateLimit-Limit header regression test 2025-10-20 15:10:59 +00:00
ratelimit.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
README.md Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
recovery_tokens.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
router.go fix: skip update check for source builds and show appropriate UI message 2025-10-21 10:08:00 +00:00
router_integration_test.go security: complete Phase 1 sensor proxy hardening 2025-10-20 15:13:37 +00:00
security.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
security_oidc.go feat: add professional logging with runtime configuration and performance optimization 2025-10-20 15:13:38 +00:00
security_setup_fix.go feat: enhance runtime configuration and system settings management 2025-10-20 17:41:19 +00:00
security_tokens.go feat: add docker agent command handling 2025-10-15 19:27:19 +00:00
session_store.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
settings.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00
system_settings.go feat: improve discovery with progress tracking, validation, and structured errors 2025-10-20 22:29:30 +00:00
types.go fix: skip update check for source builds and show appropriate UI message 2025-10-21 10:08:00 +00:00
updates.go Fix settings security tab navigation 2025-10-11 23:29:47 +00:00

README.md

Internal API Package

This directory contains the API server implementation for Pulse.

Important Note About frontend-modern/

The frontend-modern/ subdirectory that appears here is:

  • AUTO-GENERATED during builds
  • NOT the source code - just a build artifact
  • IN .gitignore - never committed
  • REQUIRED BY GO - The embed directive needs it here

Frontend Development Location

👉 Edit frontend files at: /opt/pulse/frontend-modern/src/

Why This Structure?

Go's //go:embed directive has limitations:

  1. Cannot use ../ paths to access parent directories
  2. Cannot follow symbolic links
  3. Must embed files within the Go module

This is a known Go limitation and our structure works around it.