Ruview/ui/index.html
rUv 7c1351fd5d
feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion
* feat: dual-modal WASM browser pose estimation demo (ADR-058)

Live webcam video + WiFi CSI fusion for real-time pose estimation.
Two parallel CNN pipelines (ruvector-cnn-wasm) with attention-weighted
fusion and dynamic confidence gating. Three modes: Dual, Video-only,
CSI-only. Includes pre-built WASM package (~52KB) for browser deployment.

- ADR-058: Dual-modal architecture design
- ui/pose-fusion.html: Main demo page with dark theme UI
- 7 JS modules: video-capture, csi-simulator, cnn-embedder, fusion-engine,
  pose-decoder, canvas-renderer, main orchestrator
- Pre-built ruvector-cnn-wasm WASM package for browser
- CSI heatmap, embedding space visualization, latency metrics
- WebSocket support for live ESP32 CSI data
- Navigation link added to main dashboard

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: motion-responsive skeleton + through-wall CSI tracking

- Pose decoder now uses per-cell motion grid to track actual arm/head
  positions — raising arms moves the skeleton's arms, head follows
  lateral movement
- Motion grid (10x8 cells) tracks intensity per body zone: head,
  left/right arm upper/mid, legs
- Through-wall mode: when person exits frame, CSI maintains presence
  with slow decay (~10s) and skeleton drifts in exit direction
- CSI simulator persists sensing after video loss, ghost pose renders
  with decreasing confidence
- Reduced temporal smoothing (0.45) for faster response to movement

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: video fills available space + correct WASM path resolution

- Remove fixed aspect-ratio and max-height from video panel so it
  fills the available viewport space without scrolling
- Grid uses 1fr row for content area, overflow:hidden on main grid
- Fix WASM path: resolve relative to JS module file using import.meta.url
  instead of hardcoded ./pkg/ which resolved incorrectly on gh-pages
- Responsive: mobile still gets aspect-ratio constraint

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: live ESP32 CSI pipeline + auto-connect WebSocket

- Add auto-connect to local sensing server WebSocket (ws://localhost:8765)
- Demo shows "Live ESP32" when connected to real CSI data
- Add build_firmware.ps1 for native Windows ESP-IDF builds (no Docker)
- Add read_serial.ps1 for ESP32 serial monitor

Pipeline: ESP32 → UDP:5005 → sensing-server → WS:8765 → browser demo

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs: add ADR-059 live ESP32 CSI pipeline + update README with demo links

- ADR-059: Documents end-to-end ESP32 → sensing server → browser pipeline
- README: Add dual-modal pose fusion demo link, update ADR count to 49
- References issue #245

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: RSSI visualization, RuVector attention WASM, cache-bust fixes

- Add animated RSSI Signal Strength panel with sparkline history
- Fix RuVector WasmMultiHeadAttention retptr calling convention
- Wire up RuVector Multi-Head + Flash Attention in CNN embedder
- Add ambient temporal drift to CSI simulator for visible heatmap animation
- Fix embedding space projection (sparse projection replaces cancelling sum)
- Add auto-scaling to embedding space renderer
- Add cache busters (?v=4) to all ES module imports to prevent stale caches
- Add diagnostic logging for module version verification
- Add RSSI tracking with quality labels and color-coded dBm display
- Includes ruvector-attention-wasm v2.0.5 browser ESM wrapper

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: 26-keypoint dexterous pose + full RuVector attention pipeline

Pose Decoder (17 → 26 keypoints):
- Add finger approximations: thumb, index, pinky per hand (6 new)
- Add toe tips: left/right foot index (2 new)
- Add neck keypoint (1 new)
- Hand openness driven by arm motion intensity
- Finger positions computed from wrist-elbow axis angles

CNN Embedder (full RuVector WASM pipeline):
- Stage 1: Multi-Head Attention (global spatial reasoning)
- Stage 2: Hyperbolic Attention (hierarchical body-part tree)
- Stage 3: MoE Attention (3 experts: upper/lower/extremities, top-2)
- Blended 40/30/30 weighting → final embedding projection

Canvas Renderer:
- Magenta finger joints with distinct glow
- Cyan toe tips
- White neck keypoint
- Thinner limb lines for hand/foot connections
- Joint count shown in overlay label

CSI Simulator:
- Skip synthetic person state when live ESP32 connected
- Only simulate CSI data in demo mode (was already correct)

Embedding Space:
- Fixed projection: sparse 8-dim projection replaces cancelling sum
- Auto-scaling normalizes point spread to fill canvas

Cache busters bumped to v=5 on all imports.

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: centroid-based pose tracking for responsive limb movement

Rewrites pose decoder from intensity-based to position-based tracking:
- Arms now track toward motion centroid in each body zone
- Elbow/wrist positions computed along shoulder→centroid vector
- Legs track toward lower-body zone centroids
- Smoothing reduced from 0.45 to 0.25 for responsiveness
- Zone centroids blend 30% old / 70% new each frame

6 body zones with overlapping coverage:
- Head (top 20%, center cols)
- Left/Right Arm (rows 10-60%, outer cols)
- Torso (rows 15-55%, center cols)
- Left/Right Leg (rows 50-100%, half cols each)

Hand openness now driven by arm spread distance + raise amount.
Cache busters v=6.

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: remove duplicate lAnkleX/rAnkleX declarations in pose-decoder

Stale code block from old intensity-based tracking was left behind,
re-declaring variables already defined by centroid-based tracking.

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion

- Add WasmLinearAttention and WasmLocalGlobalAttention to browser ESM wrapper
- Add 6 WASM utility functions (batch_normalize, pairwise_distances, etc.)
- Extend CnnEmbedder to 6-stage pipeline: Flash → MHA → Hyperbolic → Linear → MoE → L+G
- Use log-energy softmax blending across all 6 stages
- Wire WASM cosine_similarity and normalize into FusionEngine
- Add RuVector pipeline stats panel to UI (energy, refinement, pose impact)
- Compute embedding-to-joint mapping stats without modifying joint positions
- Center camera prompt with flexbox layout
- Add cache busters v=12

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-12 20:59:57 -04:00

513 lines
No EOL
24 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>WiFi DensePose: Human Tracking Through Walls</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<div class="container">
<!-- Header -->
<header class="header">
<h1>WiFi DensePose</h1>
<p class="subtitle">Human Tracking Through Walls Using WiFi Signals</p>
<div class="header-info">
<span class="api-version"></span>
<span class="api-environment"></span>
<span class="overall-health"></span>
</div>
</header>
<!-- Navigation -->
<nav class="nav-tabs">
<button class="nav-tab active" data-tab="dashboard">Dashboard</button>
<button class="nav-tab" data-tab="hardware">Hardware</button>
<button class="nav-tab" data-tab="demo">Live Demo</button>
<button class="nav-tab" data-tab="architecture">Architecture</button>
<button class="nav-tab" data-tab="performance">Performance</button>
<button class="nav-tab" data-tab="applications">Applications</button>
<button class="nav-tab" data-tab="sensing">Sensing</button>
<button class="nav-tab" data-tab="training">Training</button>
<a href="pose-fusion.html" class="nav-tab" style="text-decoration:none">Pose Fusion</a>
<a href="observatory.html" class="nav-tab" style="text-decoration:none">Observatory</a>
</nav>
<!-- Dashboard Tab -->
<section id="dashboard" class="tab-content active">
<div class="hero-section">
<h2>Revolutionary WiFi-Based Human Pose Detection</h2>
<p class="hero-description">
AI can track your full-body movement through walls using just WiFi signals.
Researchers at Carnegie Mellon have trained a neural network to turn basic WiFi
signals into detailed wireframe models of human bodies.
</p>
<!-- Error container -->
<div class="error-container" style="display: none;"></div>
<!-- Live Status Panel -->
<div class="live-status-panel">
<h3>System Status</h3>
<div class="status-grid">
<div class="component-status" data-component="api">
<span class="component-name">API Server</span>
<span class="status-text">-</span>
<span class="status-message"></span>
</div>
<div class="component-status" data-component="hardware">
<span class="component-name">Hardware</span>
<span class="status-text">-</span>
<span class="status-message"></span>
</div>
<div class="component-status" data-component="inference">
<span class="component-name">Inference</span>
<span class="status-text">-</span>
<span class="status-message"></span>
</div>
<div class="component-status" data-component="streaming">
<span class="component-name">Streaming</span>
<span class="status-text">-</span>
<span class="status-message"></span>
</div>
<div class="component-status" data-component="datasource" id="dashboard-datasource">
<span class="component-name">Data Source</span>
<span class="status-text">-</span>
<span class="status-message"></span>
</div>
</div>
</div>
<!-- System Metrics -->
<div class="system-metrics-panel">
<h3>System Metrics</h3>
<div class="metrics-grid">
<div class="metric-item">
<span class="metric-label">CPU Usage</span>
<div class="progress-bar" data-type="cpu">
<div class="progress-fill normal" style="width: 0%"></div>
</div>
<span class="cpu-usage">0%</span>
</div>
<div class="metric-item">
<span class="metric-label">Memory Usage</span>
<div class="progress-bar" data-type="memory">
<div class="progress-fill normal" style="width: 0%"></div>
</div>
<span class="memory-usage">0%</span>
</div>
<div class="metric-item">
<span class="metric-label">Disk Usage</span>
<div class="progress-bar" data-type="disk">
<div class="progress-fill normal" style="width: 0%"></div>
</div>
<span class="disk-usage">0%</span>
</div>
</div>
</div>
<!-- Features Status -->
<div class="features-panel">
<h3>Features</h3>
<div class="features-status"></div>
</div>
<!-- Live Statistics -->
<div class="live-stats-panel">
<h3>Live Statistics</h3>
<div class="stats-grid">
<div class="stat-item">
<span class="stat-label">Active Persons</span>
<span class="person-count">0</span>
</div>
<div class="stat-item">
<span class="stat-label">Avg Confidence</span>
<span class="avg-confidence">0%</span>
</div>
<div class="stat-item">
<span class="stat-label">Total Detections</span>
<span class="detection-count">0</span>
</div>
</div>
<div class="zones-panel">
<h4>Zone Occupancy</h4>
<div class="zones-summary"></div>
</div>
</div>
<div class="key-benefits">
<div class="benefit-card">
<div class="benefit-icon">🏠</div>
<h3>Through Walls</h3>
<p>Works through solid barriers with no line of sight required</p>
</div>
<div class="benefit-card">
<div class="benefit-icon">🔒</div>
<h3>Privacy-Preserving</h3>
<p>No cameras or visual recording - just WiFi signal analysis</p>
</div>
<div class="benefit-card">
<div class="benefit-icon"></div>
<h3>Real-Time</h3>
<p>Maps 24 body regions in real-time at 100Hz sampling rate</p>
</div>
<div class="benefit-card">
<div class="benefit-icon">💰</div>
<h3>Low Cost</h3>
<p>Built using $30 commercial WiFi hardware</p>
</div>
</div>
<div class="system-stats">
<div class="stat" data-stat="body-regions">
<span class="stat-value">24</span>
<span class="stat-label">Body Regions</span>
</div>
<div class="stat" data-stat="sampling-rate">
<span class="stat-value">100Hz</span>
<span class="stat-label">Sampling Rate</span>
</div>
<div class="stat" data-stat="accuracy">
<span class="stat-value">87.2%</span>
<span class="stat-label">Accuracy (AP@50)</span>
</div>
<div class="stat" data-stat="hardware-cost">
<span class="stat-value">$30</span>
<span class="stat-label">Hardware Cost</span>
</div>
</div>
</div>
</section>
<!-- Hardware Tab -->
<section id="hardware" class="tab-content">
<h2>Hardware Configuration</h2>
<div class="hardware-grid">
<div class="antenna-section">
<h3>3×3 Antenna Array</h3>
<p class="help-text">Click antennas to toggle their state</p>
<div class="antenna-array">
<div class="antenna-grid">
<div class="antenna tx active" data-type="TX1"></div>
<div class="antenna tx active" data-type="TX2"></div>
<div class="antenna tx active" data-type="TX3"></div>
<div class="antenna rx active" data-type="RX1"></div>
<div class="antenna rx active" data-type="RX2"></div>
<div class="antenna rx active" data-type="RX3"></div>
<div class="antenna rx active" data-type="RX4"></div>
<div class="antenna rx active" data-type="RX5"></div>
<div class="antenna rx active" data-type="RX6"></div>
</div>
<div class="antenna-legend">
<div class="legend-item">
<div class="legend-color tx"></div>
<span>Transmitters (3)</span>
</div>
<div class="legend-item">
<div class="legend-color rx"></div>
<span>Receivers (6)</span>
</div>
</div>
<div class="array-status"></div>
</div>
</div>
<div class="config-section">
<h3>WiFi Configuration</h3>
<div class="config-grid">
<div class="config-item">
<label>Frequency</label>
<div class="config-value">2.4GHz ± 20MHz</div>
</div>
<div class="config-item">
<label>Subcarriers</label>
<div class="config-value">30</div>
</div>
<div class="config-item">
<label>Sampling Rate</label>
<div class="config-value">100 Hz</div>
</div>
<div class="config-item">
<label>Total Cost</label>
<div class="config-value">$30</div>
</div>
</div>
<div class="csi-data">
<h4>Real-time CSI Data</h4>
<div class="csi-display">
<div class="csi-row">
<span>Amplitude:</span>
<div class="csi-bar">
<div class="csi-fill amplitude" style="width: 75%"></div>
</div>
<span class="csi-value">0.75</span>
</div>
<div class="csi-row">
<span>Phase:</span>
<div class="csi-bar">
<div class="csi-fill phase" style="width: 60%"></div>
</div>
<span class="csi-value">1.2π</span>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Demo Tab -->
<section id="demo" class="tab-content">
<h2>Live Demonstration</h2>
<div class="demo-controls">
<button id="startDemo" class="btn btn--primary">Start Stream</button>
<button id="stopDemo" class="btn btn--secondary" disabled>Stop Stream</button>
<div class="demo-status">
<span class="status status--info" id="demoStatus">Ready</span>
</div>
</div>
<div class="demo-grid">
<div class="signal-panel">
<h3>WiFi Signal Analysis</h3>
<div class="signal-display">
<canvas id="signalCanvas" width="400" height="200"></canvas>
</div>
<div class="signal-metrics">
<div class="metric">
<span>Signal Strength:</span>
<span id="signalStrength">-45 dBm</span>
</div>
<div class="metric">
<span>Processing Latency:</span>
<span id="latency">12 ms</span>
</div>
</div>
</div>
<div class="pose-panel">
<h3>Human Pose Detection</h3>
<div class="pose-display">
<canvas id="poseCanvas" width="400" height="300"></canvas>
</div>
<div class="detection-info">
<div class="info-item">
<span>Persons Detected:</span>
<span id="personCount">0</span>
</div>
<div class="info-item">
<span>Confidence:</span>
<span id="confidence">0.0%</span>
</div>
<div class="info-item">
<span>Keypoints:</span>
<span id="keypoints">0/0</span>
</div>
</div>
</div>
</div>
</section>
<!-- Architecture Tab -->
<section id="architecture" class="tab-content">
<h2>System Architecture</h2>
<div class="architecture-flow">
<img src="https://pplx-res.cloudinary.com/image/upload/v1748813853/gpt4o_images/m7zztcttnue7vaxclvuw.png"
alt="WiFi DensePose Architecture" class="architecture-image">
<div class="flow-steps">
<div class="step-card" data-step="1">
<div class="step-number">1</div>
<h3>CSI Input</h3>
<p>Channel State Information collected from WiFi antenna array</p>
</div>
<div class="step-card" data-step="2">
<div class="step-number">2</div>
<h3>Phase Sanitization</h3>
<p>Remove hardware-specific noise and normalize signal phase</p>
</div>
<div class="step-card" data-step="3">
<div class="step-number">3</div>
<h3>Modality Translation</h3>
<p>Convert WiFi signals to visual representation using CNN</p>
</div>
<div class="step-card" data-step="4">
<div class="step-number">4</div>
<h3>DensePose-RCNN</h3>
<p>Extract human pose keypoints and body part segmentation</p>
</div>
<div class="step-card" data-step="5">
<div class="step-number">5</div>
<h3>Wireframe Output</h3>
<p>Generate final human pose wireframe visualization</p>
</div>
</div>
</div>
</section>
<!-- Performance Tab -->
<section id="performance" class="tab-content">
<h2>Performance Analysis</h2>
<div class="performance-chart">
<img src="https://pplx-res.cloudinary.com/image/upload/v1748813924/pplx_code_interpreter/af6ef268_nsauu6.jpg"
alt="Performance Comparison Chart" class="chart-image">
</div>
<div class="performance-grid">
<div class="performance-card">
<h3>WiFi-based (Same Layout)</h3>
<div class="metric-list">
<div class="metric-item">
<span>Average Precision:</span>
<span class="metric-value">43.5%</span>
</div>
<div class="metric-item">
<span>AP@50:</span>
<span class="metric-value success">87.2%</span>
</div>
<div class="metric-item">
<span>AP@75:</span>
<span class="metric-value">44.6%</span>
</div>
</div>
</div>
<div class="performance-card">
<h3>Image-based (Reference)</h3>
<div class="metric-list">
<div class="metric-item">
<span>Average Precision:</span>
<span class="metric-value success">84.7%</span>
</div>
<div class="metric-item">
<span>AP@50:</span>
<span class="metric-value success">94.4%</span>
</div>
<div class="metric-item">
<span>AP@75:</span>
<span class="metric-value success">77.1%</span>
</div>
</div>
</div>
<div class="limitations-section">
<h3>Advantages & Limitations</h3>
<div class="pros-cons">
<div class="pros">
<h4>Advantages</h4>
<ul>
<li>Through-wall detection</li>
<li>Privacy preserving</li>
<li>Lighting independent</li>
<li>Low cost hardware</li>
<li>Uses existing WiFi</li>
</ul>
</div>
<div class="cons">
<h4>Limitations</h4>
<ul>
<li>Performance drops in different layouts</li>
<li>Requires WiFi-compatible devices</li>
<li>Training requires synchronized data</li>
</ul>
</div>
</div>
</div>
</div>
</section>
<!-- Applications Tab -->
<section id="applications" class="tab-content">
<h2>Real-World Applications</h2>
<div class="applications-grid">
<div class="app-card">
<div class="app-icon">👴</div>
<h3>Elderly Care Monitoring</h3>
<p>Monitor elderly individuals for falls or emergencies without invading privacy. Track movement patterns and detect anomalies in daily routines.</p>
<div class="app-features">
<span class="feature-tag">Fall Detection</span>
<span class="feature-tag">Activity Monitoring</span>
<span class="feature-tag">Emergency Alert</span>
</div>
</div>
<div class="app-card">
<div class="app-icon">🏠</div>
<h3>Home Security Systems</h3>
<p>Detect intruders and monitor home security without visible cameras. Track multiple persons and identify suspicious movement patterns.</p>
<div class="app-features">
<span class="feature-tag">Intrusion Detection</span>
<span class="feature-tag">Multi-person Tracking</span>
<span class="feature-tag">Invisible Monitoring</span>
</div>
</div>
<div class="app-card">
<div class="app-icon">🏥</div>
<h3>Healthcare Patient Monitoring</h3>
<p>Monitor patients in hospitals and care facilities. Track vital signs through movement analysis and detect health emergencies.</p>
<div class="app-features">
<span class="feature-tag">Vital Sign Analysis</span>
<span class="feature-tag">Movement Tracking</span>
<span class="feature-tag">Health Alerts</span>
</div>
</div>
<div class="app-card">
<div class="app-icon">🏢</div>
<h3>Smart Building Occupancy</h3>
<p>Optimize building energy consumption by tracking occupancy patterns. Control lighting, HVAC, and security systems automatically.</p>
<div class="app-features">
<span class="feature-tag">Energy Optimization</span>
<span class="feature-tag">Occupancy Tracking</span>
<span class="feature-tag">Smart Controls</span>
</div>
</div>
<div class="app-card">
<div class="app-icon">🥽</div>
<h3>AR/VR Applications</h3>
<p>Enable full-body tracking for virtual and augmented reality applications without wearing additional sensors or cameras.</p>
<div class="app-features">
<span class="feature-tag">Full Body Tracking</span>
<span class="feature-tag">Sensor-free</span>
<span class="feature-tag">Immersive Experience</span>
</div>
</div>
</div>
<div class="implementation-note">
<h3>Implementation Considerations</h3>
<p>While WiFi DensePose offers revolutionary capabilities, successful implementation requires careful consideration of environment setup, data privacy regulations, and system calibration for optimal performance.</p>
</div>
</section>
<!-- Sensing Tab -->
<section id="sensing" class="tab-content"></section>
<!-- Training Tab -->
<section id="training" class="tab-content">
<div class="tab-header">
<h2>Model Training</h2>
<p>Record CSI data, train pose estimation models, and manage .rvf files</p>
</div>
<div id="training-container" style="display: flex; gap: 20px; flex-wrap: wrap;">
<div id="training-panel-container" style="flex: 1; min-width: 400px;"></div>
<div id="model-panel-container" style="flex: 1; min-width: 350px; max-width: 450px;"></div>
</div>
</section>
</div>
<!-- Error Toast -->
<div id="globalErrorToast" class="error-toast"></div>
<!-- Load application scripts as modules -->
<script type="module" src="app.js"></script>
</body>
</html>