mirror of
https://github.com/ruvnet/RuView.git
synced 2026-08-20 22:14:56 +00:00
Merge pull request #1624 from ruvnet/claude/off-axis-sneaker-ruview-u398g0
Some checks failed
Bench Regression Guard / bench compile-verify (--no-run) (push) Has been cancelled
Continuous Integration / Code Quality & Security (push) Has been cancelled
Continuous Integration / Rust Workspace Tests (push) Has been cancelled
Continuous Integration / UI JavaScript Tests (push) Has been cancelled
Continuous Integration / Tests (push) Has been cancelled
Continuous Integration / Tests-1 (push) Has been cancelled
Continuous Integration / Tests-2 (push) Has been cancelled
wifi-densepose sensing-server → Docker Hub + ghcr.io / build · push · smoke-test (push) Has been cancelled
CSI data policy (ADR-299) / CSI data policy check (push) Has been cancelled
Fix-Marker Regression Guard / Verify fix markers (push) Has been cancelled
Security Scanning / Static Application Security Testing (push) Has been cancelled
Security Scanning / Dependency Vulnerability Scan (push) Has been cancelled
Security Scanning / Container Security Scan (push) Has been cancelled
Security Scanning / Infrastructure Security Scan (push) Has been cancelled
Security Scanning / Secret Scanning (push) Has been cancelled
Security Scanning / License Compliance Scan (push) Has been cancelled
Security Scanning / Security Policy Compliance (push) Has been cancelled
three.js demos → GitHub Pages / build-and-deploy (push) Has been cancelled
Bench Regression Guard / bench fast-run (informational, non-gating) (push) Has been cancelled
Continuous Integration / Performance Tests (push) Has been cancelled
Continuous Integration / Docker Build & Test (push) Has been cancelled
Continuous Integration / API Documentation (push) Has been cancelled
Continuous Integration / Notify (push) Has been cancelled
Security Scanning / Security Report (push) Has been cancelled
Some checks failed
Bench Regression Guard / bench compile-verify (--no-run) (push) Has been cancelled
Continuous Integration / Code Quality & Security (push) Has been cancelled
Continuous Integration / Rust Workspace Tests (push) Has been cancelled
Continuous Integration / UI JavaScript Tests (push) Has been cancelled
Continuous Integration / Tests (push) Has been cancelled
Continuous Integration / Tests-1 (push) Has been cancelled
Continuous Integration / Tests-2 (push) Has been cancelled
wifi-densepose sensing-server → Docker Hub + ghcr.io / build · push · smoke-test (push) Has been cancelled
CSI data policy (ADR-299) / CSI data policy check (push) Has been cancelled
Fix-Marker Regression Guard / Verify fix markers (push) Has been cancelled
Security Scanning / Static Application Security Testing (push) Has been cancelled
Security Scanning / Dependency Vulnerability Scan (push) Has been cancelled
Security Scanning / Container Security Scan (push) Has been cancelled
Security Scanning / Infrastructure Security Scan (push) Has been cancelled
Security Scanning / Secret Scanning (push) Has been cancelled
Security Scanning / License Compliance Scan (push) Has been cancelled
Security Scanning / Security Policy Compliance (push) Has been cancelled
three.js demos → GitHub Pages / build-and-deploy (push) Has been cancelled
Bench Regression Guard / bench fast-run (informational, non-gating) (push) Has been cancelled
Continuous Integration / Performance Tests (push) Has been cancelled
Continuous Integration / Docker Build & Test (push) Has been cancelled
Continuous Integration / API Documentation (push) Has been cancelled
Continuous Integration / Notify (push) Has been cancelled
Security Scanning / Security Report (push) Has been cancelled
ADR-324: off-axis-mode — clean-room Kooima projection in Rust/WASM + demo
This commit is contained in:
rUv
GitHub
commit
a76adc3c2f
16 changed files with 2522 additions and 1 deletions
|
|
@ -1 +1 @@
|
|||
{"sessionId":"d80c93c2-51b7-42e8-a0fc-dc47cff1200f","pid":45748,"acquiredAt":1779668018388}
|
||||
{"sessionId":"905385c4-b13f-5091-96df-5752fb109cf5","pid":509,"procStart":"527","acquiredAt":1786922977672}
|
||||
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -307,3 +307,7 @@ v2/crates/wifi-densepose-sensing-server/data/
|
|||
# file-specific so tracked datasets below v2/data remain visible.
|
||||
/v2/data/session-secret
|
||||
*.proptest-regressions
|
||||
|
||||
# ADR-324: wasm-bindgen output for ruview-offaxis is generated locally
|
||||
# (see the crate README); never commit generated artifacts.
|
||||
v2/crates/ruview-offaxis/pkg/
|
||||
|
|
|
|||
276
docs/adr/ADR-324-off-axis-head-coupled-perspective-demo.md
Normal file
276
docs/adr/ADR-324-off-axis-head-coupled-perspective-demo.md
Normal file
|
|
@ -0,0 +1,276 @@
|
|||
# ADR-324: off-axis-mode — RF-assisted head-coupled perspective for the three.js realtime demo
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed (core implemented — see §2.5) |
|
||||
| **Date** | 2026-08-16 |
|
||||
| **Deciders** | ruv |
|
||||
| **Codename** | **off-axis-mode** |
|
||||
| **Scope** | New `examples/three.js/demos/07-off-axis-window.html` (client-side only); no server changes |
|
||||
| **Relates to** | ADR-019 (sensing-only UI), ADR-035 (live sensing UI accuracy), ADR-169 (adam-mode), ADR-170 (yoga-mode), ADR-282 (L0–L5 evidence ladder), ADR-295 (source provenance), ADR-306 (spatial ontology), ADR-307 (persistent tracking), ADR-323 (pose refinement) |
|
||||
| **Prior art** | [`icurtis1/off-axis-sneaker`](https://github.com/icurtis1/off-axis-sneaker) (reference only — see §2.1 licensing) |
|
||||
| **Numbering note** | ADR-324 is the next free number in the authoring checkout (322 is unused, 323 is the latest on disk). Re-run the ADR index/collision check immediately before merge and rename if needed. |
|
||||
| **Tracking issue** | none yet |
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
### 1.1 The question this ADR answers
|
||||
|
||||
"Can we use [`icurtis1/off-axis-sneaker`](https://github.com/icurtis1/off-axis-sneaker)
|
||||
with RuView?" The answer is: **yes for the technique, no for the code, and
|
||||
only honestly for the RF part.** This ADR records the research behind each of
|
||||
those three clauses and defines the integration that is actually defensible.
|
||||
|
||||
### 1.2 What off-axis-sneaker is
|
||||
|
||||
`off-axis-sneaker` is a React + TypeScript + Vite web app that renders a GLB
|
||||
model (a sneaker) in three.js and creates a *head-coupled perspective*
|
||||
("fish-tank VR" / "window into the screen") illusion:
|
||||
|
||||
- **Tracking input**: MediaPipe Face Mesh (468 facial landmarks) from a
|
||||
webcam. Head (x, y) comes from the eye midpoint; depth (z) is proxied by
|
||||
inter-ocular distance. An exponential moving average (default factor 0.3)
|
||||
smooths jitter; sensitivity multipliers are `strengthX: 4`, `strengthY: 3`,
|
||||
`strengthZ: 2`.
|
||||
- **Projection**: `src/utils/offAxisCamera.ts` builds a **true asymmetric
|
||||
(off-axis) frustum** — `makePerspective(left, right, top, bottom, near, far)`
|
||||
with `left/right/top/bottom = (screenBound − eyePosition) · (near /
|
||||
viewerToScreenDistance)` — i.e. Kooima's generalized perspective projection,
|
||||
plus a matching camera translation. Constants: `nearPlane 0.05`,
|
||||
`farPlane 1000`, `worldScale 0.01` (cm → world units), `movementScale 1.5`.
|
||||
- **Calibration**: a wizard captures physical screen width/height (cm),
|
||||
typical viewing distance, and pixel density, stored locally, so eye position
|
||||
is computed relative to the *physical* display.
|
||||
|
||||
The technique descends from Johnny Chung Lee's 2007 Wii-remote desktop VR
|
||||
demo and the fish-tank VR literature (Ware, Arthur & Booth, CHI '93). The
|
||||
projection math is Robert Kooima's "Generalized Perspective Projection"
|
||||
(2008). Both are public, well-documented techniques independent of any one
|
||||
implementation.
|
||||
|
||||
### 1.3 What the illusion physically requires
|
||||
|
||||
The head-coupled illusion is only convincing when the tracked eye position is
|
||||
**accurate to roughly centimeters** and **low-latency**. The VR literature
|
||||
puts comfortable motion-to-photon latency below ~20 ms for head-mounted
|
||||
displays; desktop fish-tank VR tolerates more, but visible lag between head
|
||||
motion and parallax response is exactly what breaks the "window" illusion.
|
||||
`CLAIMED` (literature values; no RuView measurement exists for this demo yet).
|
||||
|
||||
### 1.4 What RuView RF sensing can actually supply today
|
||||
|
||||
This is where honesty is mandatory (repo rule: never present WiFi sensing as
|
||||
camera-grade).
|
||||
|
||||
- **Field-peak position, not metric localization.**
|
||||
`wifi-densepose-sensing-server/src/field_localize.rs` derives a position
|
||||
from the strongest peak of the 20×20 `signal_field` carried on
|
||||
`/ws/sensing` `sensing_update` frames. Its own module doc states the
|
||||
caveat: the subcarrier→angle mapping is a *representation*; "a single ESP32
|
||||
link cannot resolve a true (x, z) room position." The emitted position is
|
||||
"strongest field peak in the room model," mapped with `X_SCALE 0.6`,
|
||||
`Z_SCALE 0.5`, gated by `PEAK_THRESHOLD 0.35` — real, live, motion-tracking,
|
||||
but **not a calibrated person fix** and nowhere near eye-position precision.
|
||||
- **RF pose is 2-D, normalized, constant-confidence.** The committed Cog
|
||||
(ADR-101, restated by ADR-323) emits 17 COCO keypoints as normalized 2-D
|
||||
coordinates with a constant confidence and no per-joint uncertainty. A
|
||||
"nose" keypoint exists (COCO index 0), but it is not a metric 3-D head fix.
|
||||
- **Tracks are coarse and pseudonymous by design.** `ruview-track` (ADR-307)
|
||||
maintains `person_N` tracks with container-level ("kitchen → hallway")
|
||||
continuity, coarse non-reversible features, and asserts **no accuracy
|
||||
number** — outputs default to evidence level `L1`.
|
||||
- **Update cadence and latency are unmeasured for this purpose.** The demo
|
||||
pipeline runs at ~30 Hz on the MediaPipe side (ADR-170), but no end-to-end
|
||||
RF motion-to-photon latency has been measured. Any figure quoted for the RF
|
||||
path must be tagged `MEASURED` with a reproducer before it appears in docs
|
||||
or UI.
|
||||
|
||||
Conclusion of the capability match: **RF cannot drive a convincing fish-tank
|
||||
illusion by itself today**, and this ADR does not claim it can. RF *can*
|
||||
supply things a webcam cannot: camera-free presence, zone-level position,
|
||||
person count, approach direction, and pseudonymous continuity — including
|
||||
when the camera is off.
|
||||
|
||||
### 1.5 What this ADR is *not*
|
||||
|
||||
- Not a vendoring of `off-axis-sneaker` (see §2.1 — the repo has no license).
|
||||
- Not a claim of camera-grade RF head tracking, at any tier.
|
||||
- Not a backend change: no new server endpoints, no new auth surface, no
|
||||
schema changes. Purely additive client-side HTML/JS, like ADR-169/170.
|
||||
- Not a React/Vite/Tailwind adoption. The `examples/three.js/demos/*` are
|
||||
dependency-light single-file HTML demos and stay that way.
|
||||
|
||||
## 2. Decision
|
||||
|
||||
### 2.1 Licensing: adopt the technique, not the code
|
||||
|
||||
`off-axis-sneaker` publishes **no license**. Under default copyright, its
|
||||
source cannot be copied, vendored, or translated into this repository.
|
||||
Decision:
|
||||
|
||||
1. **No code, assets, or models from `off-axis-sneaker` enter this repo.**
|
||||
The GLB sneaker model is likewise unlicensed for reuse; demos use assets
|
||||
already present in `examples/`.
|
||||
2. The off-axis projection is implemented **clean-room from the public
|
||||
sources**: Kooima's "Generalized Perspective Projection" (2008) — the
|
||||
`pa/pb/pc` screen-corner formulation — and three.js's documented
|
||||
`PerspectiveCamera.projectionMatrix` override path. The repository is cited
|
||||
as prior art in this ADR only.
|
||||
3. If upstream later adds a permissive license, revisiting reuse requires a
|
||||
new ADR note, not silent copying.
|
||||
|
||||
### 2.2 Tiered integration — each tier labeled by what it really is
|
||||
|
||||
**Tier A (ships first): webcam-fine + RF-context hybrid.**
|
||||
`07-off-axis-window.html` uses MediaPipe Face Landmarker (already the pattern
|
||||
in demo 05) for fine head tracking and the Kooima frustum for rendering —
|
||||
functionally what off-axis-sneaker does, reimplemented. RuView RF adds the
|
||||
camera-free layer around it:
|
||||
|
||||
- **Presence-gated camera**: the webcam pipeline starts only when the RF
|
||||
presence signal (`/ws/sensing` `sensing_update`) says someone is in the
|
||||
zone, and stops after a configurable RF-vacancy timeout. The privacy
|
||||
posture improves: the camera is *off* until physics says there is someone
|
||||
to track.
|
||||
- **Multi-person arbitration**: when RF reports more than one person, the HUD
|
||||
says so and the demo holds the last stable perspective instead of jumping
|
||||
between faces.
|
||||
- **Pre-warm**: RF approach direction (field-peak trajectory) warms up
|
||||
MediaPipe and the scene before the person sits down.
|
||||
|
||||
**Tier B (demo mode, prominently labeled): RF-only coarse parallax.**
|
||||
A toggle drives the off-axis eye position from RF alone — field peak (x, z)
|
||||
plus the pose nose keypoint when present — through a one-euro filter, a
|
||||
deadband, and a hard gain clamp. The HUD labels it **"RF coarse body
|
||||
parallax — not head tracking"** and shows the live evidence level (`L1`
|
||||
heuristic unless a certificate says otherwise, per ADR-282/ADR-318). The
|
||||
expected experience is a slow, body-scale parallax sway — a demonstrative
|
||||
"the room model moves because *you* moved, with no camera" — not a stable
|
||||
fish-tank illusion. The demo must never present Tier B as equivalent to
|
||||
Tier A.
|
||||
|
||||
**Tier C (future, explicitly gated, not promised): metric RF head position.**
|
||||
Only a calibrated multistatic deployment (ADR-297 multi-node semantics,
|
||||
ADR-311 fusion, ADR-303 ground-truth sync) with an evidence-engine ledger
|
||||
entry (ADR-304) and a capability certificate (ADR-318) could justify feeding
|
||||
RF positions into the fine path. No current data supports this; Tier C exists
|
||||
in this ADR solely so nobody ships it informally without those gates.
|
||||
|
||||
### 2.3 Implementation surface
|
||||
|
||||
- New file `examples/three.js/demos/07-off-axis-window.html` (07, not 06 —
|
||||
ADR-170 reserves `06-yoga-mode.html`). Single-file demo following the 01–05
|
||||
conventions: same CSS custom properties, same HUD/helper-panel pattern,
|
||||
served from the existing static demo server
|
||||
(`http://127.0.0.1:8765/examples/three.js/demos/…`).
|
||||
- A small clean-room module (inline `<script type="module">` or
|
||||
`examples/three.js/lib/off-axis-camera.js` if shared later) that, given
|
||||
screen corners `pa, pb, pc` (from calibration) and eye point `pe`, sets
|
||||
`camera.projectionMatrix` via the Kooima formulation each frame.
|
||||
- Data inputs are the **existing** streams only: `/ws/sensing`
|
||||
(`sensing_update` → `signal_field` → field peak, using the same
|
||||
`X_SCALE`/`Z_SCALE`/`PEAK_THRESHOLD` mapping as `field_localize.rs`) and,
|
||||
when available, `/api/v1/stream/pose` for the nose keypoint. WebSocket
|
||||
access uses the existing ticket flow (`ws_ticket.rs` / `bearer_auth.rs`);
|
||||
no endpoint is exempted or added.
|
||||
- Calibration mirrors the sneaker app's concept without its code: screen
|
||||
width/height in cm, viewing distance, persisted in `localStorage` under a
|
||||
demo-scoped key. No calibration data leaves the browser.
|
||||
- Provenance discipline: if the demo is pointed at a synthetic or replayed
|
||||
source, the ADR-295 provenance state must surface in the HUD exactly as the
|
||||
Observatory does — synthetic can never present as live.
|
||||
|
||||
### 2.4 Honesty and evidence rules binding this feature
|
||||
|
||||
1. Every user-visible latency, accuracy, or precision statement in the demo,
|
||||
README, or docs carries a `MEASURED` (with reproducer), `CLAIMED`, or
|
||||
`SYNTHETIC` tag. This ADR itself contains no `MEASURED` claims.
|
||||
2. Tier B is labeled coarse body parallax in the HUD at all times; there is
|
||||
no configuration that hides the label while RF drives the camera.
|
||||
3. No PCK or pose-accuracy number may be quoted for the RF path without the
|
||||
mean-pose baseline and a leakage-free held-out split (repo rule).
|
||||
4. The webcam feed never leaves the browser; no frames, landmarks, or
|
||||
embeddings are sent to the server. RF data continues to obey ADR-307's
|
||||
privacy invariants (pseudonymous, coarse, rotatable).
|
||||
|
||||
### 2.5 Implementation status (2026-08-16 amendment)
|
||||
|
||||
The projection core shipped as a **Rust crate compiled to WASM** rather than
|
||||
the inline JS module §2.3 anticipated — a strict upgrade with the same
|
||||
surface: `v2/crates/ruview-offaxis` (dependency-free native core; wasm-bindgen
|
||||
only on wasm32) implements the Kooima projection, the one-euro filter, the
|
||||
field-peak mapping (constants mirroring `field_localize.rs`), and the Tier B
|
||||
coarse-parallax stage with its deadband/gain/clamp bounds enforced in Rust.
|
||||
`examples/three.js/demos/07-off-axis-window.html` consumes the wasm-bindgen
|
||||
output (built locally per the crate README; generated artifacts are not
|
||||
committed). Validation and `MEASURED` benchmarks live in the crate README.
|
||||
The demo ships with a `SYNTHETIC`-labeled mouse simulator and the labeled
|
||||
Tier B RF mode; a Tier A fine tracker connects through
|
||||
`OffAxisCamera.update_normalized` and remains host-provided.
|
||||
|
||||
## 3. Options considered
|
||||
|
||||
| Option | Verdict | Why |
|
||||
|---|---|---|
|
||||
| Vendor `off-axis-sneaker` (or fork + point at RuView) | **Rejected** | No license ⇒ no redistribution rights. Also React/Vite stack conflicts with the repo's single-file demo convention. |
|
||||
| Clean-room Kooima off-axis demo, webcam-fine + RF-context (Tier A/B) | **Chosen** | Legally clean, matches demo conventions, uses RF for what it is actually good at, and demonstrates camera-free presence value honestly. |
|
||||
| RF-only head-coupled perspective as the headline | **Rejected** | Over-claim. Single-link field peaks are a representation, not metric localization (`field_localize.rs` caveat); shipping this as "head tracking" violates the camera-grade rule. Survives only as the labeled Tier B toggle. |
|
||||
| Wait for multistatic metric localization (Tier C) before any demo | **Rejected** | Blocks a useful, honest demo on a phase-2/3 program (ADR-303/311/318) with no delivery date. The gates are recorded instead. |
|
||||
| Add a dedicated server endpoint for head position | **Rejected** | Unnecessary — existing `/ws/sensing` + `/api/v1/stream/pose` suffice; a new endpoint would expand the auth surface for no capability gain. |
|
||||
|
||||
## 4. Consequences
|
||||
|
||||
**Improves**
|
||||
|
||||
- A publicly legible demo of RF sensing's actual differentiator: the scene
|
||||
knows you are there, where you roughly are, and how many of you there are —
|
||||
before and without any camera.
|
||||
- Privacy posture of the head-tracking demo class: camera duty-cycle is
|
||||
bounded by RF presence instead of always-on.
|
||||
- Canonical, licensed off-axis projection code the Observatory or future UI
|
||||
can reuse.
|
||||
|
||||
**Costs / risks**
|
||||
|
||||
- Tier B can underwhelm viewers primed by webcam demos; the mitigation is the
|
||||
labeling and the side-by-side toggle, not inflated gain.
|
||||
- MediaPipe CDN dependency (same as demo 05) remains a network-availability
|
||||
risk for Tier A; the demo must degrade to Tier B with a visible notice.
|
||||
- Screen-calibration friction (cm measurements) may deter casual users; a
|
||||
"skip calibration (approximate)" path with degraded-accuracy labeling is
|
||||
acceptable.
|
||||
- Upstream `off-axis-sneaker` may change or add a license; tracking that is
|
||||
manual.
|
||||
|
||||
**Follow-ups (not in this ADR's scope)**
|
||||
|
||||
- Measure end-to-end RF motion-to-parallax latency with a reproducer and
|
||||
publish it `MEASURED`.
|
||||
- If/when ADR-303/311 land, evaluate Tier C against the ADR-318 certificate
|
||||
gate.
|
||||
- Consider promoting the off-axis camera module into the Observatory 3D view.
|
||||
|
||||
## 5. Validation
|
||||
|
||||
- Demo checklist (manual, per ADR-169/170 practice): loads from the static
|
||||
server; Tier A activates only on RF presence; Tier B label visible whenever
|
||||
RF drives the camera; provenance badge correct against a synthetic source;
|
||||
no network requests carrying webcam-derived data (verified in devtools).
|
||||
- `rg` gate before merge: no file under `examples/` contains code originating
|
||||
from `icurtis1/off-axis-sneaker`.
|
||||
- No workspace, harness, or firmware validation rows are triggered — the
|
||||
change is a static HTML demo plus this document.
|
||||
|
||||
## 6. References
|
||||
|
||||
- [`icurtis1/off-axis-sneaker`](https://github.com/icurtis1/off-axis-sneaker) — prior-art reference (unlicensed; technique only)
|
||||
- Robert Kooima, *Generalized Perspective Projection*, 2008 — off-axis frustum math
|
||||
- Johnny Chung Lee, *Head Tracking for Desktop VR Displays using the Wii Remote*, 2007
|
||||
- Ware, Arthur & Booth, *Fish Tank Virtual Reality*, CHI '93 — head coupling vs. stereo
|
||||
- `v2/crates/wifi-densepose-sensing-server/src/field_localize.rs` — field-peak honesty caveat and coordinate mapping
|
||||
- `v2/crates/wifi-densepose-sensing-server/src/ws_ticket.rs`, `bearer_auth.rs` — WebSocket auth pattern
|
||||
- `v2/crates/ruview-track/src/lib.rs` — ADR-307 privacy invariants and evidence discipline
|
||||
- ADR-169, ADR-170 — demo-scoped ADR pattern for `examples/three.js/demos/`
|
||||
- ADR-282 — L0–L5 evidence ladder; ADR-295 — provenance state machine
|
||||
|
|
@ -108,6 +108,7 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme
|
|||
| [ADR-115](ADR-115-home-assistant-integration.md) | Home Assistant integration via MQTT auto-discovery + Matter bridge (HA-DISCO + HA-FABRIC + HA-MIND) | Accepted (MQTT track) / Proposed (Matter SDK P8b) |
|
||||
| [ADR-169](ADR-169-adam-mode-light-theme.md) | adam-mode — light theme toggle for the three.js realtime demo | Proposed |
|
||||
| [ADR-170](ADR-170-yoga-mode-pose-system.md) | yoga-mode — yoga pose detection, classification, and scoring for the three.js realtime demo | Proposed |
|
||||
| [ADR-324](ADR-324-off-axis-head-coupled-perspective-demo.md) | off-axis-mode — RF-assisted head-coupled perspective demo (clean-room Kooima projection; RF presence gating) | Proposed |
|
||||
|
||||
### Architecture and infrastructure
|
||||
|
||||
|
|
|
|||
378
examples/three.js/demos/07-off-axis-window.html
Normal file
378
examples/three.js/demos/07-off-axis-window.html
Normal file
|
|
@ -0,0 +1,378 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>RuView · ADR-324 · off-axis window (ruview-offaxis WASM)</title>
|
||||
<link rel="icon" type="image/svg+xml" href="data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'><rect x='7' y='9' width='18' height='14' rx='2' fill='none' stroke='%23e8a634' stroke-width='2'/><circle cx='16' cy='16' r='3' fill='%23e8a634'/></svg>">
|
||||
<style>
|
||||
:root {
|
||||
--bg: #0a0a0a;
|
||||
--bg-panel: rgba(0, 0, 0, 0.88);
|
||||
--amber: #e8a634;
|
||||
--amber-dim: #4a3a1a;
|
||||
--amber-hot: #ffc04d;
|
||||
--grid-major: #444444;
|
||||
--grid-minor: #222222;
|
||||
--green: #4f4;
|
||||
--blue: #4cf;
|
||||
--red: #f66;
|
||||
--text-mute: #888;
|
||||
--border: #2a2a2a;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body {
|
||||
margin: 0;
|
||||
background: var(--bg);
|
||||
color: var(--amber);
|
||||
font-family: 'SF Mono', Monaco, 'Cascadia Code', Consolas, monospace;
|
||||
overflow: hidden;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
}
|
||||
canvas { display: block; }
|
||||
|
||||
#info {
|
||||
position: absolute;
|
||||
top: 16px;
|
||||
left: 16px;
|
||||
padding: 14px 16px;
|
||||
background: var(--bg-panel);
|
||||
border: 1px solid var(--amber);
|
||||
border-radius: 8px;
|
||||
min-width: 280px;
|
||||
max-width: 360px;
|
||||
font-size: 12px;
|
||||
line-height: 1.55;
|
||||
z-index: 10;
|
||||
backdrop-filter: blur(6px);
|
||||
box-shadow: 0 4px 24px rgba(232, 166, 52, 0.08);
|
||||
}
|
||||
#info h1 { margin: 0 0 2px 0; font-size: 14px; letter-spacing: 0.5px; }
|
||||
#info .sub { font-size: 11px; color: var(--text-mute); margin-bottom: 10px; }
|
||||
#info .row { display: flex; justify-content: space-between; gap: 12px; margin: 2px 0; }
|
||||
#info .row .k { color: var(--text-mute); }
|
||||
#info .row .v { color: var(--amber); font-variant-numeric: tabular-nums; }
|
||||
#info .row .v.live { color: var(--green); }
|
||||
#info .row .v.warn { color: var(--red); }
|
||||
|
||||
/* The mandatory ADR-324 §2.4 mode label: always visible while RF
|
||||
drives the camera; there is no configuration that hides it. */
|
||||
#mode-label {
|
||||
position: absolute;
|
||||
top: 16px;
|
||||
right: 16px;
|
||||
padding: 8px 14px;
|
||||
background: var(--bg-panel);
|
||||
border: 1px solid var(--blue);
|
||||
border-radius: 8px;
|
||||
color: var(--blue);
|
||||
font-size: 12px;
|
||||
z-index: 11;
|
||||
}
|
||||
#mode-label.rf { border-color: var(--red); color: var(--red); }
|
||||
|
||||
#controls {
|
||||
position: absolute;
|
||||
bottom: 16px;
|
||||
left: 16px;
|
||||
padding: 12px 16px;
|
||||
background: var(--bg-panel);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
font-size: 12px;
|
||||
z-index: 10;
|
||||
max-width: 340px;
|
||||
}
|
||||
#controls h2 { margin: 0 0 8px 0; font-size: 12px; color: var(--text-mute); }
|
||||
#controls label { display: flex; justify-content: space-between; gap: 8px; margin: 4px 0; align-items: center; }
|
||||
#controls input[type="number"] {
|
||||
width: 70px; background: #111; border: 1px solid var(--border);
|
||||
color: var(--amber); font-family: inherit; font-size: 12px; padding: 2px 6px; border-radius: 4px;
|
||||
}
|
||||
#controls input[type="text"] {
|
||||
width: 190px; background: #111; border: 1px solid var(--border);
|
||||
color: var(--amber); font-family: inherit; font-size: 11px; padding: 2px 6px; border-radius: 4px;
|
||||
}
|
||||
#controls button {
|
||||
background: var(--amber-dim); border: 1px solid var(--amber); color: var(--amber-hot);
|
||||
font-family: inherit; font-size: 12px; padding: 4px 10px; border-radius: 5px; cursor: pointer; margin-top: 6px;
|
||||
}
|
||||
#controls button:hover { background: var(--amber); color: #000; }
|
||||
|
||||
#wasm-missing {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
display: none;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: rgba(0, 0, 0, 0.92);
|
||||
z-index: 50;
|
||||
}
|
||||
#wasm-missing .box {
|
||||
max-width: 560px; border: 1px solid var(--amber); border-radius: 10px;
|
||||
background: var(--bg-panel); padding: 22px 26px; font-size: 13px; line-height: 1.7;
|
||||
}
|
||||
#wasm-missing code { color: var(--amber-hot); background: #151005; padding: 1px 5px; border-radius: 4px; display: block; margin: 4px 0; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="info">
|
||||
<h1>OFF-AXIS WINDOW</h1>
|
||||
<div class="sub">ADR-324 · clean-room Kooima projection · ruview-offaxis (Rust→WASM)</div>
|
||||
<div class="row"><span class="k">engine</span><span class="v" id="hud-engine">loading…</span></div>
|
||||
<div class="row"><span class="k">input</span><span class="v" id="hud-input">mouse (SYNTHETIC)</span></div>
|
||||
<div class="row"><span class="k">eye x/y/z (m)</span><span class="v" id="hud-eye">—</span></div>
|
||||
<div class="row"><span class="k">render fps</span><span class="v live" id="hud-fps">—</span></div>
|
||||
<div class="row"><span class="k">rf socket</span><span class="v" id="hud-ws">not connected</span></div>
|
||||
<div class="row"><span class="k">rf peak</span><span class="v" id="hud-peak">—</span></div>
|
||||
<div class="row" style="margin-top:8px"><span class="k" style="font-size:10px">
|
||||
keys: <b>M</b> mouse · <b>R</b> RF Tier B · wheel = distance</span></div>
|
||||
</div>
|
||||
|
||||
<div id="mode-label">MOUSE SIM — SYNTHETIC INPUT</div>
|
||||
|
||||
<div id="controls">
|
||||
<h2>PHYSICAL CALIBRATION (stored locally)</h2>
|
||||
<label>screen width (cm) <input id="cal-w" type="number" step="0.5" value="60"></label>
|
||||
<label>screen height (cm) <input id="cal-h" type="number" step="0.5" value="34"></label>
|
||||
<label>viewing distance (cm) <input id="cal-d" type="number" step="1" value="65"></label>
|
||||
<h2 style="margin-top:10px">RF SOURCE (Tier B)</h2>
|
||||
<label>ws url <input id="ws-url" type="text" value="ws://127.0.0.1:8080/ws/sensing"></label>
|
||||
<button id="apply">apply calibration</button>
|
||||
</div>
|
||||
|
||||
<div id="wasm-missing">
|
||||
<div class="box">
|
||||
<b>ruview-offaxis WASM module not found.</b><br><br>
|
||||
This demo loads the crate's wasm-bindgen output from
|
||||
<code>v2/crates/ruview-offaxis/pkg/</code>. Generated artifacts are not
|
||||
committed (repo rule); build them once:
|
||||
<code>cd v2 && cargo build -p ruview-offaxis --target wasm32-unknown-unknown --release</code>
|
||||
<code>wasm-bindgen --target web --out-dir crates/ruview-offaxis/pkg \
|
||||
target/wasm32-unknown-unknown/release/ruview_offaxis.wasm</code>
|
||||
(install the CLI with <code>cargo install wasm-bindgen-cli --version 0.2.114</code>)<br>
|
||||
then reload. Full steps: <code>v2/crates/ruview-offaxis/README.md</code>.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
|
||||
<script type="module">
|
||||
// ADR-324 demo: the projection math lives in Rust/WASM (ruview-offaxis).
|
||||
// This file only wires inputs (mouse SYNTHETIC sim, or /ws/sensing
|
||||
// signal_field for the labeled Tier B coarse-parallax mode) into the
|
||||
// WASM camera and copies its matrices onto a three.js camera.
|
||||
|
||||
const $ = (id) => document.getElementById(id);
|
||||
|
||||
// ---- Load the WASM module (not committed; user builds it once). ----
|
||||
let wasm;
|
||||
try {
|
||||
wasm = await import('../../../v2/crates/ruview-offaxis/pkg/ruview_offaxis.js');
|
||||
await wasm.default();
|
||||
$('hud-engine').textContent = 'ruview-offaxis wasm';
|
||||
} catch (e) {
|
||||
console.error('ruview-offaxis pkg not found', e);
|
||||
$('wasm-missing').style.display = 'flex';
|
||||
throw e;
|
||||
}
|
||||
const { OffAxisCamera, RfParallax } = wasm;
|
||||
|
||||
// ---- Calibration (persisted locally; never leaves the browser). ----
|
||||
const CAL_KEY = 'ruview-offaxis-demo-cal';
|
||||
const saved = JSON.parse(localStorage.getItem(CAL_KEY) || 'null');
|
||||
if (saved) { $('cal-w').value = saved.w; $('cal-h').value = saved.h; $('cal-d').value = saved.d; }
|
||||
const cal = () => ({ w: +$('cal-w').value || 60, h: +$('cal-h').value || 34, d: +$('cal-d').value || 65 });
|
||||
|
||||
let cam = new OffAxisCamera(cal().w, cal().h, cal().d, 0.05, 100.0);
|
||||
cam.set_filter(1.2, 0.4); // interactive: light smoothing, quick catch-up
|
||||
let rf = new RfParallax(cal().d / 100);
|
||||
|
||||
$('apply').onclick = () => {
|
||||
const c = cal();
|
||||
localStorage.setItem(CAL_KEY, JSON.stringify(c));
|
||||
cam = new OffAxisCamera(c.w, c.h, c.d, 0.05, 100.0);
|
||||
cam.set_filter(1.2, 0.4);
|
||||
rf = new RfParallax(c.d / 100);
|
||||
buildRoom(); // room proportions follow the physical screen
|
||||
};
|
||||
|
||||
// ---- three.js scene: a room extending behind the screen plane. ----
|
||||
const renderer = new THREE.WebGLRenderer({ antialias: true });
|
||||
renderer.setSize(window.innerWidth, window.innerHeight);
|
||||
renderer.setPixelRatio(window.devicePixelRatio);
|
||||
document.body.appendChild(renderer.domElement);
|
||||
|
||||
const scene = new THREE.Scene();
|
||||
scene.background = new THREE.Color(0x0a0a0a);
|
||||
|
||||
// Camera is fully driven by the WASM matrices.
|
||||
const camera = new THREE.PerspectiveCamera();
|
||||
camera.matrixAutoUpdate = false;
|
||||
|
||||
let room = new THREE.Group();
|
||||
function buildRoom() {
|
||||
scene.remove(room);
|
||||
room = new THREE.Group();
|
||||
const c = cal();
|
||||
const W = c.w / 100, H = c.h / 100, DEPTH = Math.max(W, 0.8) * 2.0;
|
||||
|
||||
// Wireframe box behind the screen: the classic "window" cue.
|
||||
const boxGeo = new THREE.BoxGeometry(W, H, DEPTH);
|
||||
const edges = new THREE.EdgesGeometry(boxGeo);
|
||||
const box = new THREE.LineSegments(edges, new THREE.LineBasicMaterial({ color: 0xe8a634 }));
|
||||
box.position.z = -DEPTH / 2; // screen plane is z = 0
|
||||
room.add(box);
|
||||
|
||||
// Depth rails: rows of columns receding into the box.
|
||||
const colMat = new THREE.MeshStandardMaterial({ color: 0x4a3a1a, emissive: 0x2a1f08 });
|
||||
for (let i = 1; i <= 6; i++) {
|
||||
for (const sx of [-1, 1]) {
|
||||
const col = new THREE.Mesh(new THREE.CylinderGeometry(0.008, 0.008, H * 0.9, 12), colMat);
|
||||
col.position.set(sx * W * 0.42, 0, -DEPTH * i / 7);
|
||||
room.add(col);
|
||||
}
|
||||
}
|
||||
|
||||
// Floating objects at staggered depths (parallax targets).
|
||||
const knotMat = new THREE.MeshStandardMaterial({ color: 0xe8a634, metalness: 0.4, roughness: 0.35 });
|
||||
const knot = new THREE.Mesh(new THREE.TorusKnotGeometry(H * 0.18, H * 0.05, 120, 16), knotMat);
|
||||
knot.position.set(0, 0, -DEPTH * 0.45);
|
||||
knot.name = 'knot';
|
||||
room.add(knot);
|
||||
|
||||
const orb = new THREE.Mesh(
|
||||
new THREE.IcosahedronGeometry(H * 0.08, 1),
|
||||
new THREE.MeshStandardMaterial({ color: 0x4cf0ff, emissive: 0x0a3540 })
|
||||
);
|
||||
orb.position.set(-W * 0.22, H * 0.18, -DEPTH * 0.18);
|
||||
room.add(orb);
|
||||
|
||||
// One object slightly IN FRONT of the screen plane — pops "out".
|
||||
const pop = new THREE.Mesh(
|
||||
new THREE.OctahedronGeometry(H * 0.05),
|
||||
new THREE.MeshStandardMaterial({ color: 0xffc04d, emissive: 0x604010 })
|
||||
);
|
||||
pop.position.set(W * 0.28, -H * 0.2, 0.06);
|
||||
room.add(pop);
|
||||
|
||||
room.add(new THREE.AmbientLight(0xffffff, 0.35));
|
||||
const key = new THREE.PointLight(0xffe0a0, 1.0);
|
||||
key.position.set(0.3, 0.4, 0.5);
|
||||
room.add(key);
|
||||
scene.add(room);
|
||||
}
|
||||
buildRoom();
|
||||
|
||||
// ---- Input modes. ----
|
||||
// 'mouse' — SYNTHETIC eye simulator (always available, no hardware).
|
||||
// 'rf' — Tier B: /ws/sensing signal_field → RfParallax.
|
||||
// Labeled coarse body parallax, NOT head tracking (ADR-324 §2.4).
|
||||
let mode = 'mouse';
|
||||
const modeLabel = $('mode-label');
|
||||
function setMode(m) {
|
||||
mode = m;
|
||||
if (m === 'rf') {
|
||||
modeLabel.textContent = 'RF COARSE BODY PARALLAX — NOT HEAD TRACKING';
|
||||
modeLabel.classList.add('rf');
|
||||
$('hud-input').textContent = 'rf field peak (Tier B)';
|
||||
connectWs();
|
||||
} else {
|
||||
modeLabel.textContent = 'MOUSE SIM — SYNTHETIC INPUT';
|
||||
modeLabel.classList.remove('rf');
|
||||
$('hud-input').textContent = 'mouse (SYNTHETIC)';
|
||||
}
|
||||
}
|
||||
window.addEventListener('keydown', (e) => {
|
||||
if (e.key === 'm' || e.key === 'M') setMode('mouse');
|
||||
if (e.key === 'r' || e.key === 'R') setMode('rf');
|
||||
});
|
||||
|
||||
// Mouse sim: pointer position maps to a ±0.3 m eye excursion;
|
||||
// wheel adjusts distance.
|
||||
let mouseEye = { x: 0, y: 0, d: cal().d / 100 };
|
||||
window.addEventListener('pointermove', (e) => {
|
||||
mouseEye.x = (e.clientX / window.innerWidth - 0.5) * 0.6;
|
||||
mouseEye.y = (0.5 - e.clientY / window.innerHeight) * 0.4;
|
||||
});
|
||||
window.addEventListener('wheel', (e) => {
|
||||
mouseEye.d = Math.min(2.5, Math.max(0.2, mouseEye.d + e.deltaY * 0.0005));
|
||||
}, { passive: true });
|
||||
|
||||
// ---- RF Tier B input: /ws/sensing sensing_update.signal_field. ----
|
||||
let ws = null;
|
||||
function connectWs() {
|
||||
if (ws) { try { ws.close(); } catch (_) {} }
|
||||
const url = $('ws-url').value;
|
||||
$('hud-ws').textContent = 'connecting…';
|
||||
try { ws = new WebSocket(url); } catch (e) {
|
||||
$('hud-ws').textContent = 'invalid url'; return;
|
||||
}
|
||||
ws.onopen = () => { $('hud-ws').textContent = 'connected'; };
|
||||
ws.onclose = () => { $('hud-ws').textContent = 'closed'; };
|
||||
ws.onerror = () => { $('hud-ws').textContent = 'error (server up? ticket needed?)'; };
|
||||
ws.onmessage = (ev) => {
|
||||
try {
|
||||
const msg = JSON.parse(ev.data);
|
||||
const field = msg.signal_field || (msg.data && msg.data.signal_field);
|
||||
if (!field || !field.values) return;
|
||||
const nx = field.grid_size || field.nx || 20;
|
||||
const nz = field.grid_size || field.nz || 20;
|
||||
const values = Float32Array.from(field.values);
|
||||
const found = rf.update(values, nx, nz, performance.now() / 1000);
|
||||
$('hud-peak').textContent = found
|
||||
? `value ${rf.peak_value().toFixed(2)} (≥ 0.35 gate)`
|
||||
: 'below 0.35 gate — holding';
|
||||
// Provenance surfaced verbatim (ADR-295: synthetic never
|
||||
// presents as live).
|
||||
if (msg.provenance || msg.source) {
|
||||
$('hud-ws').textContent = `connected · src: ${msg.provenance || msg.source}`;
|
||||
}
|
||||
} catch (_) { /* non-JSON frame */ }
|
||||
};
|
||||
}
|
||||
|
||||
// ---- Render loop: one WASM call, three matrix copies, render. ----
|
||||
const tmp = new THREE.Matrix4();
|
||||
let frames = 0, lastFps = performance.now();
|
||||
function animate() {
|
||||
requestAnimationFrame(animate);
|
||||
const t = performance.now() / 1000;
|
||||
|
||||
if (mode === 'mouse') {
|
||||
cam.update_eye(mouseEye.x, mouseEye.y, mouseEye.d, t);
|
||||
} else {
|
||||
const e = rf.eye(); // bounded coarse-parallax eye (metres)
|
||||
cam.update_eye(e[0], e[1], e[2], t);
|
||||
}
|
||||
|
||||
// Copy the Kooima matrices onto the three.js camera.
|
||||
camera.projectionMatrix.fromArray(cam.projection());
|
||||
camera.projectionMatrixInverse.copy(camera.projectionMatrix).invert();
|
||||
tmp.fromArray(cam.view());
|
||||
camera.matrixWorld.copy(tmp).invert(); // world = inverse(view)
|
||||
camera.matrixWorldInverse.copy(tmp);
|
||||
|
||||
const knot = room.getObjectByName('knot');
|
||||
if (knot) { knot.rotation.y += 0.003; knot.rotation.x += 0.001; }
|
||||
|
||||
renderer.render(scene, camera);
|
||||
|
||||
const eye = cam.eye();
|
||||
$('hud-eye').textContent = `${eye[0].toFixed(3)} / ${eye[1].toFixed(3)} / ${eye[2].toFixed(3)}`;
|
||||
frames++;
|
||||
const now = performance.now();
|
||||
if (now - lastFps > 1000) {
|
||||
$('hud-fps').textContent = String(frames);
|
||||
frames = 0; lastFps = now;
|
||||
}
|
||||
}
|
||||
animate();
|
||||
|
||||
window.addEventListener('resize', () => {
|
||||
renderer.setSize(window.innerWidth, window.innerHeight);
|
||||
// NOTE: no camera.aspect update — the frustum is fully determined
|
||||
// by the physical screen calibration, not the browser viewport.
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
8
v2/Cargo.lock
generated
8
v2/Cargo.lock
generated
|
|
@ -9770,6 +9770,14 @@ dependencies = [
|
|||
"thiserror 2.0.18",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruview-offaxis"
|
||||
version = "0.3.1"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ruview-ontology"
|
||||
version = "0.3.1"
|
||||
|
|
|
|||
|
|
@ -118,6 +118,9 @@ members = [
|
|||
"crates/ruview-counterfactual",# ADR-313 counterfactual spatial inference
|
||||
"crates/ruview-infogain", # ADR-314 information-gain scheduler
|
||||
"crates/ruview-active", # ADR-309 active sensing control
|
||||
# ADR-324 — clean-room Kooima off-axis (head-coupled perspective) projection.
|
||||
# Dependency-free native core; wasm-bindgen surface only on wasm32.
|
||||
"crates/ruview-offaxis",
|
||||
]
|
||||
# ADR-040: WASM edge crate targets wasm32-unknown-unknown (no_std),
|
||||
# excluded from workspace to avoid breaking `cargo test --workspace`.
|
||||
|
|
|
|||
26
v2/crates/ruview-offaxis/Cargo.toml
Normal file
26
v2/crates/ruview-offaxis/Cargo.toml
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
[package]
|
||||
name = "ruview-offaxis"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
authors.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
description = "Clean-room generalized off-axis (head-coupled) perspective projection for RuView demos (ADR-324)"
|
||||
readme = "README.md"
|
||||
|
||||
[lib]
|
||||
# cdylib for the wasm32 build (ADR-324 demo surface); rlib for native reuse.
|
||||
crate-type = ["cdylib", "rlib"]
|
||||
|
||||
# The native core is dependency-free by design (ADR-324 §2.3): the wasm-bindgen
|
||||
# surface exists only when compiling for wasm32, so the workspace test gate
|
||||
# (`cargo test --workspace --no-default-features`) never pulls JS-interop deps.
|
||||
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
||||
wasm-bindgen = "0.2"
|
||||
|
||||
[dev-dependencies]
|
||||
criterion = { workspace = true }
|
||||
|
||||
[[bench]]
|
||||
name = "offaxis"
|
||||
harness = false
|
||||
164
v2/crates/ruview-offaxis/README.md
Normal file
164
v2/crates/ruview-offaxis/README.md
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
# ruview-offaxis
|
||||
|
||||
Clean-room **off-axis (head-coupled) perspective projection** in Rust, with a
|
||||
wasm-bindgen surface for browser demos. Implements ADR-324.
|
||||
|
||||
The screen becomes a window: given the physical screen's corners and the
|
||||
viewer's eye position, the crate produces the asymmetric frustum and
|
||||
screen-aligned view matrix that keep the screen plane fixed while everything
|
||||
behind and in front of it moves with true parallax.
|
||||
|
||||
## Clean-room statement
|
||||
|
||||
ADR-324 records that the prior-art repository (`icurtis1/off-axis-sneaker`)
|
||||
publishes **no license**. No code, assets, or derived text from it appear in
|
||||
this crate. The implementation follows the published math only:
|
||||
|
||||
- Robert Kooima, *Generalized Perspective Projection* (2008) — the
|
||||
`pa`/`pb`/`pc` screen-corner frustum formulation.
|
||||
- Casiez, Roussel & Vogel, *1€ Filter* (CHI 2012) — adaptive tracking-noise
|
||||
smoothing.
|
||||
|
||||
## What's in the crate
|
||||
|
||||
| Module | Contents |
|
||||
|---|---|
|
||||
| `projection` | `Screen` (3 corners, any orientation), `off_axis()` → asymmetric `projection` + screen-aligned `view` matrix (column-major `f64`, the three.js `Matrix4.elements` layout). All failure modes are typed errors — never NaN matrices. |
|
||||
| `filter` | `OneEuro` / `OneEuro3` one-euro filter. Timestamps are injected; the crate never reads a clock. |
|
||||
| `rf` | `field_peak()` — strongest-cell extraction for `/ws/sensing` `signal_field` grids, mirroring the sensing server's `field_localize.rs` constants (`X_SCALE 0.6`, `Z_SCALE 0.5`, `PEAK_THRESHOLD 0.35`). `CoarseParallax` — the **Tier B** stage: deadband + gain + hard clamp + one-euro. `ScreenCalibration` — physical screen (cm) + normalized-head → metric eye mapping (**Tier A** input hook). |
|
||||
| `wasm` (wasm32 only) | `OffAxisCamera` and `RfParallax` wasm-bindgen classes. |
|
||||
|
||||
The native core is **dependency-free**; wasm-bindgen is pulled only when
|
||||
compiling for `wasm32`.
|
||||
|
||||
## Honesty contract (repo rule — read before demoing)
|
||||
|
||||
- A single-link CSI field peak is a *representation of field energy*, *not*
|
||||
metric localization and *never* a head position (see the caveat in
|
||||
`wifi-densepose-sensing-server/src/field_localize.rs`). The Tier B path is
|
||||
therefore **coarse body parallax by construction**: deadbanded,
|
||||
gain-limited, hard-clamped. Do not present it as head tracking; the demo
|
||||
labels it on screen at all times.
|
||||
- Numeric defaults (gains, deadbands, filter cutoffs) are interaction-design
|
||||
choices — `CLAIMED`, not measured performance.
|
||||
- The benchmark numbers below are `MEASURED` with the stated reproducer on
|
||||
the stated machine; re-run locally before relying on them.
|
||||
|
||||
## Native quick start
|
||||
|
||||
```rust
|
||||
use ruview_offaxis::{off_axis, Screen, Vec3};
|
||||
|
||||
// 60 cm × 34 cm screen centered at the origin; eye 65 cm out, 10 cm right.
|
||||
let screen = Screen::centered(0.60, 0.34)?;
|
||||
let oa = off_axis(&screen, Vec3::new(0.10, 0.0, 0.65), 0.05, 100.0)?;
|
||||
let mvp: [f64; 16] = oa.view_projection(); // column-major, GL/three.js layout
|
||||
# Ok::<(), ruview_offaxis::OffAxisError>(())
|
||||
```
|
||||
|
||||
Key invariant (unit-tested for a grid of eye positions and for tilted
|
||||
screens): the physical screen corners always project exactly to the NDC
|
||||
corners — `pa→(−1,−1)`, `pb→(1,−1)`, `pc→(−1,1)`, `pd→(1,1)` — and points on
|
||||
the screen plane are eye-invariant. That is the mathematical definition of
|
||||
"the screen is a window".
|
||||
|
||||
## Building the WASM package
|
||||
|
||||
Generated artifacts are not committed (repo rule). Build once:
|
||||
|
||||
```bash
|
||||
cd v2
|
||||
rustup target add wasm32-unknown-unknown
|
||||
cargo build -p ruview-offaxis --target wasm32-unknown-unknown --release
|
||||
|
||||
# Install the matching CLI once: cargo install wasm-bindgen-cli --version 0.2.114
|
||||
wasm-bindgen --target web --out-dir crates/ruview-offaxis/pkg \
|
||||
target/wasm32-unknown-unknown/release/ruview_offaxis.wasm
|
||||
```
|
||||
|
||||
Output: `pkg/ruview_offaxis.js` + `pkg/ruview_offaxis_bg.wasm`
|
||||
(≈54 KB wasm, `MEASURED` for this crate at wasm-bindgen 0.2.114; `wasm-opt -Oz`
|
||||
can shrink it further if you have binaryen). The demo at
|
||||
`examples/three.js/demos/07-off-axis-window.html` loads this path directly —
|
||||
build, then open the demo. A Node smoke test of the same flow lives in the
|
||||
PR's validation notes.
|
||||
|
||||
## three.js integration (the whole wiring)
|
||||
|
||||
```js
|
||||
import init, { OffAxisCamera, RfParallax } from './pkg/ruview_offaxis.js';
|
||||
await init();
|
||||
|
||||
// Physical calibration in cm — measure your actual screen.
|
||||
const cam = new OffAxisCamera(60, 34, 65, 0.05, 100.0);
|
||||
cam.set_filter(1.2, 0.4); // one-euro: min_cutoff Hz, beta
|
||||
|
||||
const camera = new THREE.PerspectiveCamera();
|
||||
camera.matrixAutoUpdate = false; // WASM owns every matrix
|
||||
|
||||
const view = new THREE.Matrix4();
|
||||
function onFrame(eyeX, eyeY, eyeZ) { // metres, screen space
|
||||
cam.update_eye(eyeX, eyeY, eyeZ, performance.now() / 1000);
|
||||
camera.projectionMatrix.fromArray(cam.projection());
|
||||
camera.projectionMatrixInverse.copy(camera.projectionMatrix).invert();
|
||||
view.fromArray(cam.view());
|
||||
camera.matrixWorld.copy(view).invert();
|
||||
camera.matrixWorldInverse.copy(view);
|
||||
}
|
||||
```
|
||||
|
||||
Scene convention: the screen plane is `z = 0`; content behind the screen has
|
||||
`z < 0`; content with `z > 0` "pops out". Do **not** update `camera.aspect`
|
||||
on resize — the frustum is determined by the physical screen, not the
|
||||
viewport.
|
||||
|
||||
### Input tiers (ADR-324)
|
||||
|
||||
- **Tier A (fine tracker)** — feed any head tracker through
|
||||
`cam.update_normalized(nx, ny, depthScale, lateralRangeM, t)`; `nx`/`ny`
|
||||
are normalized image coordinates from whatever fine tracker the host runs
|
||||
(which stays entirely in the browser). RF adds presence gating around it.
|
||||
- **Tier B (RF only, labeled)** — `RfParallax.update(Float32Array, nx, nz, t)`
|
||||
with `/ws/sensing` `signal_field` values, then `rf.eye()` →
|
||||
`cam.update_eye(...)`. Keep the on-screen "coarse body parallax — not head
|
||||
tracking" label; the clamps in the Rust core bound the excursion but the
|
||||
label is what keeps the demo honest.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
`MEASURED` — reproducer: `cd v2 && cargo bench -p ruview-offaxis`.
|
||||
Environment for the numbers below: Linux x86_64 container (shared/virtualized
|
||||
CPU), rustc 1.89.0, criterion 0.5, 2026-08-16. Treat them as order-of-
|
||||
magnitude; re-run on your hardware.
|
||||
|
||||
| Benchmark | Time (median) |
|
||||
|---|---|
|
||||
| `off_axis_projection` (frustum + view build) | ~76 ns |
|
||||
| `view_projection` combined (4×4 multiply) | ~27 ns |
|
||||
| `one_euro3_step` (3-axis filter step) | ~60 ns |
|
||||
| `field_peak_20x20` (live grid size) | ~488 ns |
|
||||
| `field_peak_100x100` | ~12.3 µs |
|
||||
| `tier_b_full_frame_20x20` (scan → parallax → projection) | ~598 ns |
|
||||
|
||||
The full Tier B per-frame path costs well under a microsecond — under 0.01%
|
||||
of a 60 Hz frame budget. The argmax scan was the only hot spot found; it was
|
||||
rewritten branch-light for a measured −18% (20×20) / −33% (100×100) before
|
||||
these numbers were taken. End-to-end *motion-to-photon* latency (RF capture →
|
||||
render) has **not** been measured and is dominated by the sensing pipeline,
|
||||
not this crate; no figure is claimed.
|
||||
|
||||
## Validation
|
||||
|
||||
```bash
|
||||
cd v2
|
||||
cargo test -p ruview-offaxis # 23 unit tests + doctest
|
||||
cargo clippy -p ruview-offaxis --all-targets # zero warnings
|
||||
cargo build -p ruview-offaxis --target wasm32-unknown-unknown --release
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- ADR-324 — decision record, tier definitions, licensing analysis
|
||||
- `docs/adr/ADR-282-*` — L0–L5 evidence ladder (labels used above)
|
||||
- `v2/crates/wifi-densepose-sensing-server/src/field_localize.rs` — the
|
||||
field-peak honesty caveat this crate inherits
|
||||
84
v2/crates/ruview-offaxis/benches/offaxis.rs
Normal file
84
v2/crates/ruview-offaxis/benches/offaxis.rs
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
//! Criterion benchmarks for the per-frame hot path (ADR-324).
|
||||
//!
|
||||
//! Reproducer: `cd v2 && cargo bench -p ruview-offaxis`
|
||||
//! Any numbers quoted from this bench are MEASURED on the machine that ran
|
||||
//! that command; re-run locally before relying on them.
|
||||
|
||||
use criterion::{black_box, criterion_group, criterion_main, Criterion};
|
||||
use ruview_offaxis::{
|
||||
field_peak, off_axis, CoarseParallax, CoarseParallaxConfig, OneEuro3, OneEuroConfig, Screen,
|
||||
Vec3,
|
||||
};
|
||||
|
||||
fn bench_projection(c: &mut Criterion) {
|
||||
let screen = Screen::centered(0.6, 0.34).unwrap();
|
||||
c.bench_function("off_axis_projection", |b| {
|
||||
let mut t = 0.0_f64;
|
||||
b.iter(|| {
|
||||
t += 0.016;
|
||||
let eye = Vec3::new(0.1 * t.sin(), 0.05 * t.cos(), 0.65);
|
||||
black_box(off_axis(black_box(&screen), black_box(eye), 0.05, 100.0).unwrap())
|
||||
})
|
||||
});
|
||||
|
||||
c.bench_function("off_axis_view_projection_combined", |b| {
|
||||
let eye = Vec3::new(0.1, -0.03, 0.65);
|
||||
let oa = off_axis(&screen, eye, 0.05, 100.0).unwrap();
|
||||
b.iter(|| black_box(black_box(&oa).view_projection()))
|
||||
});
|
||||
}
|
||||
|
||||
fn bench_filter(c: &mut Criterion) {
|
||||
c.bench_function("one_euro3_step", |b| {
|
||||
let mut f = OneEuro3::new(OneEuroConfig::default());
|
||||
let mut t = 0.0_f64;
|
||||
b.iter(|| {
|
||||
t += 0.016;
|
||||
black_box(f.filter(Vec3::new(t.sin(), t.cos(), 0.65), t))
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
fn make_grid(nx: usize, nz: usize) -> Vec<f32> {
|
||||
// Deterministic pseudo-field with one hot cell.
|
||||
let mut v: Vec<f32> = (0..nx * nz).map(|i| 0.05 + (i % 7) as f32 * 0.01).collect();
|
||||
v[(nz / 3) * nx + nx / 4] = 0.9;
|
||||
v
|
||||
}
|
||||
|
||||
fn bench_field_peak(c: &mut Criterion) {
|
||||
let g20 = make_grid(20, 20);
|
||||
c.bench_function("field_peak_20x20", |b| {
|
||||
b.iter(|| black_box(field_peak(black_box(&g20), 20, 20)))
|
||||
});
|
||||
|
||||
let g100 = make_grid(100, 100);
|
||||
c.bench_function("field_peak_100x100", |b| {
|
||||
b.iter(|| black_box(field_peak(black_box(&g100), 100, 100)))
|
||||
});
|
||||
}
|
||||
|
||||
fn bench_full_tier_b_frame(c: &mut Criterion) {
|
||||
// The whole Tier B per-frame path: grid scan → parallax → projection.
|
||||
let screen = Screen::centered(0.6, 0.34).unwrap();
|
||||
let g20 = make_grid(20, 20);
|
||||
c.bench_function("tier_b_full_frame_20x20", |b| {
|
||||
let mut cp = CoarseParallax::new(CoarseParallaxConfig::default());
|
||||
let mut t = 0.0_f64;
|
||||
b.iter(|| {
|
||||
t += 0.016;
|
||||
let peak = field_peak(black_box(&g20), 20, 20);
|
||||
let eye = cp.update(peak, t);
|
||||
black_box(off_axis(&screen, eye, 0.05, 100.0).unwrap())
|
||||
})
|
||||
});
|
||||
}
|
||||
|
||||
criterion_group!(
|
||||
benches,
|
||||
bench_projection,
|
||||
bench_filter,
|
||||
bench_field_peak,
|
||||
bench_full_tier_b_frame
|
||||
);
|
||||
criterion_main!(benches);
|
||||
240
v2/crates/ruview-offaxis/src/filter.rs
Normal file
240
v2/crates/ruview-offaxis/src/filter.rs
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
//! One-euro filter (Casiez, Roussel & Vogel, CHI 2012) for eye-position
|
||||
//! smoothing.
|
||||
//!
|
||||
//! Implemented from the published algorithm: an exponential low-pass whose
|
||||
//! cutoff adapts to the signal's speed — low cutoff (heavy smoothing) when
|
||||
//! nearly still, higher cutoff (low lag) when moving fast. This is the
|
||||
//! standard jitter/lag trade-off filter for interactive tracking.
|
||||
//!
|
||||
//! Timestamps are injected by the caller in seconds (monotonic). The crate
|
||||
//! never reads a clock — repo discipline, and it keeps the wasm build free
|
||||
//! of `performance.now()` assumptions.
|
||||
|
||||
use crate::math::Vec3;
|
||||
use core::f64::consts::PI;
|
||||
|
||||
/// One-euro filter parameters.
|
||||
///
|
||||
/// - `min_cutoff` (Hz): smoothing floor. Lower = smoother but laggier at rest.
|
||||
/// - `beta`: speed coefficient. Higher = less lag during fast motion.
|
||||
/// - `d_cutoff` (Hz): cutoff for the internal derivative estimate.
|
||||
///
|
||||
/// Defaults are the paper's recommended starting point (1.0, 0.0, 1.0);
|
||||
/// interactive head tracking typically tunes `min_cutoff` down and `beta` up.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct OneEuroConfig {
|
||||
/// Smoothing floor in Hz (lower = smoother but laggier at rest).
|
||||
pub min_cutoff: f64,
|
||||
/// Speed coefficient (higher = less lag during fast motion).
|
||||
pub beta: f64,
|
||||
/// Cutoff in Hz for the internal derivative estimate.
|
||||
pub d_cutoff: f64,
|
||||
}
|
||||
|
||||
impl Default for OneEuroConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
min_cutoff: 1.0,
|
||||
beta: 0.0,
|
||||
d_cutoff: 1.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Smoothing factor for an exponential low-pass at `cutoff` Hz sampled
|
||||
/// `dt` seconds apart.
|
||||
#[inline]
|
||||
fn alpha(cutoff: f64, dt: f64) -> f64 {
|
||||
let tau = 1.0 / (2.0 * PI * cutoff);
|
||||
1.0 / (1.0 + tau / dt)
|
||||
}
|
||||
|
||||
/// Scalar one-euro filter.
|
||||
#[derive(Clone, Copy, Debug, Default)]
|
||||
pub struct OneEuro {
|
||||
cfg: OneEuroConfig,
|
||||
/// `(t, x_hat, dx_hat)` from the previous accepted sample.
|
||||
state: Option<(f64, f64, f64)>,
|
||||
}
|
||||
|
||||
impl OneEuro {
|
||||
/// Create with the given parameters.
|
||||
pub fn new(cfg: OneEuroConfig) -> Self {
|
||||
Self { cfg, state: None }
|
||||
}
|
||||
|
||||
/// Replace the parameters, keeping filter state.
|
||||
pub fn set_config(&mut self, cfg: OneEuroConfig) {
|
||||
self.cfg = cfg;
|
||||
}
|
||||
|
||||
/// Clear state; the next sample passes through unfiltered.
|
||||
pub fn reset(&mut self) {
|
||||
self.state = None;
|
||||
}
|
||||
|
||||
/// Filter sample `x` taken at time `t_s` (seconds). Non-monotonic or
|
||||
/// non-finite input returns the previous estimate unchanged (never NaN).
|
||||
pub fn filter(&mut self, x: f64, t_s: f64) -> f64 {
|
||||
if !x.is_finite() || !t_s.is_finite() {
|
||||
return self.state.map_or(0.0, |(_, xh, _)| xh);
|
||||
}
|
||||
match self.state {
|
||||
None => {
|
||||
self.state = Some((t_s, x, 0.0));
|
||||
x
|
||||
}
|
||||
Some((t0, x0, dx0)) => {
|
||||
let dt = t_s - t0;
|
||||
if dt <= 0.0 {
|
||||
return x0;
|
||||
}
|
||||
let dx = (x - x0) / dt;
|
||||
let a_d = alpha(self.cfg.d_cutoff, dt);
|
||||
let dx_hat = a_d * dx + (1.0 - a_d) * dx0;
|
||||
let cutoff = self.cfg.min_cutoff + self.cfg.beta * dx_hat.abs();
|
||||
let a = alpha(cutoff, dt);
|
||||
let x_hat = a * x + (1.0 - a) * x0;
|
||||
self.state = Some((t_s, x_hat, dx_hat));
|
||||
x_hat
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Component-wise one-euro filter over a [`Vec3`].
|
||||
#[derive(Clone, Copy, Debug, Default)]
|
||||
pub struct OneEuro3 {
|
||||
x: OneEuro,
|
||||
y: OneEuro,
|
||||
z: OneEuro,
|
||||
}
|
||||
|
||||
impl OneEuro3 {
|
||||
/// Create with the same parameters on all three axes.
|
||||
pub fn new(cfg: OneEuroConfig) -> Self {
|
||||
Self {
|
||||
x: OneEuro::new(cfg),
|
||||
y: OneEuro::new(cfg),
|
||||
z: OneEuro::new(cfg),
|
||||
}
|
||||
}
|
||||
|
||||
/// Replace parameters on all axes, keeping state.
|
||||
pub fn set_config(&mut self, cfg: OneEuroConfig) {
|
||||
self.x.set_config(cfg);
|
||||
self.y.set_config(cfg);
|
||||
self.z.set_config(cfg);
|
||||
}
|
||||
|
||||
/// Clear state on all axes.
|
||||
pub fn reset(&mut self) {
|
||||
self.x.reset();
|
||||
self.y.reset();
|
||||
self.z.reset();
|
||||
}
|
||||
|
||||
/// Filter a 3-D sample taken at time `t_s` (seconds).
|
||||
pub fn filter(&mut self, v: Vec3, t_s: f64) -> Vec3 {
|
||||
Vec3::new(
|
||||
self.x.filter(v.x, t_s),
|
||||
self.y.filter(v.y, t_s),
|
||||
self.z.filter(v.z, t_s),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn first_sample_passes_through() {
|
||||
let mut f = OneEuro::new(OneEuroConfig::default());
|
||||
assert_eq!(f.filter(3.25, 0.0), 3.25);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn constant_input_stays_constant() {
|
||||
let mut f = OneEuro::new(OneEuroConfig::default());
|
||||
for i in 0..100 {
|
||||
let y = f.filter(1.5, i as f64 * 0.016);
|
||||
assert!((y - 1.5).abs() < 1e-12);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_response_converges_monotonically() {
|
||||
let mut f = OneEuro::new(OneEuroConfig {
|
||||
min_cutoff: 1.0,
|
||||
beta: 0.0,
|
||||
d_cutoff: 1.0,
|
||||
});
|
||||
f.filter(0.0, 0.0);
|
||||
let mut prev = 0.0;
|
||||
for i in 1..200 {
|
||||
let y = f.filter(1.0, i as f64 * 0.016);
|
||||
assert!(y > prev, "monotone rise");
|
||||
assert!(y <= 1.0 + 1e-12, "no overshoot");
|
||||
prev = y;
|
||||
}
|
||||
assert!(prev > 0.99, "converged near the step, got {prev}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn higher_beta_tracks_fast_motion_closer() {
|
||||
// A fast ramp: the adaptive filter (beta > 0) must lag less than the
|
||||
// pure low-pass (beta = 0).
|
||||
let slow_cfg = OneEuroConfig {
|
||||
min_cutoff: 0.5,
|
||||
beta: 0.0,
|
||||
d_cutoff: 1.0,
|
||||
};
|
||||
let fast_cfg = OneEuroConfig {
|
||||
min_cutoff: 0.5,
|
||||
beta: 1.0,
|
||||
d_cutoff: 1.0,
|
||||
};
|
||||
let mut slow = OneEuro::new(slow_cfg);
|
||||
let mut fast = OneEuro::new(fast_cfg);
|
||||
let mut last = (0.0, 0.0, 0.0);
|
||||
for i in 0..120 {
|
||||
let t = i as f64 * 0.016;
|
||||
let x = t * 2.0; // 2 m/s ramp
|
||||
last = (x, slow.filter(x, t), fast.filter(x, t));
|
||||
}
|
||||
let (x, s, f) = last;
|
||||
assert!(
|
||||
(x - f).abs() < (x - s).abs(),
|
||||
"beta reduces lag: |{x}-{f}| < |{x}-{s}|"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_non_monotonic_and_non_finite_samples() {
|
||||
let mut f = OneEuro::new(OneEuroConfig::default());
|
||||
f.filter(1.0, 1.0);
|
||||
let settled = f.filter(1.0, 2.0);
|
||||
// Time going backwards: hold the estimate.
|
||||
assert_eq!(f.filter(99.0, 0.5), settled);
|
||||
// NaN input: hold the estimate.
|
||||
assert_eq!(f.filter(f64::NAN, 3.0), settled);
|
||||
// NaN never propagates.
|
||||
let y = f.filter(1.0, 4.0);
|
||||
assert!(y.is_finite());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vec3_filter_is_componentwise() {
|
||||
let mut f3 = OneEuro3::new(OneEuroConfig::default());
|
||||
let mut fx = OneEuro::new(OneEuroConfig::default());
|
||||
for i in 0..10 {
|
||||
let t = i as f64 * 0.02;
|
||||
let v = Vec3::new(i as f64 * 0.1, -1.0, 2.0);
|
||||
let out = f3.filter(v, t);
|
||||
assert_eq!(out.x, fx.filter(v.x, t));
|
||||
assert_eq!(out.y, -1.0);
|
||||
assert_eq!(out.z, 2.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
67
v2/crates/ruview-offaxis/src/lib.rs
Normal file
67
v2/crates/ruview-offaxis/src/lib.rs
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
//! # `ruview-offaxis` — clean-room off-axis (head-coupled) perspective (ADR-324)
|
||||
//!
|
||||
//! Generalized perspective projection for "window into the screen"
|
||||
//! (fish-tank VR / head-coupled perspective) demos, implemented from the
|
||||
//! published math — Robert Kooima, *Generalized Perspective Projection*
|
||||
//! (2008) — plus the supporting input stages the ADR-324 demo needs:
|
||||
//!
|
||||
//! - [`Screen`] / [`off_axis`]: three physical screen corners + a tracked
|
||||
//! eye → asymmetric frustum and screen-aligned view matrix (column-major
|
||||
//! `f64`, the three.js `Matrix4.elements` layout).
|
||||
//! - [`OneEuro`] / [`OneEuro3`]: the one-euro filter (Casiez et al., CHI
|
||||
//! 2012) for tracking-noise smoothing with injected timestamps.
|
||||
//! - [`rf`]: RuView-specific input mapping — `/ws/sensing` signal-field
|
||||
//! peak extraction (constants mirroring the sensing server's
|
||||
//! `field_localize.rs`) and the bounded **Tier B** coarse-parallax stage.
|
||||
//! - A wasm-bindgen surface (wasm32 only) exposing [`wasm::OffAxisCamera`]
|
||||
//! and [`wasm::RfParallax`] to the browser demos.
|
||||
//!
|
||||
//! ## Clean-room statement
|
||||
//!
|
||||
//! ADR-324 records that the prior-art repository (`icurtis1/off-axis-sneaker`)
|
||||
//! is unlicensed: no code, assets, or derived text from it appear here. This
|
||||
//! crate is written solely from the published Kooima and Casiez papers and
|
||||
//! RuView's own source.
|
||||
//!
|
||||
//! ## Honesty contract (repo rule)
|
||||
//!
|
||||
//! Nothing in this crate asserts sensing accuracy. The Tier B RF path is
|
||||
//! **coarse body parallax by construction** — deadbanded, gain-limited,
|
||||
//! hard-clamped — because a single-link CSI field peak is a representation
|
||||
//! of field energy, not metric localization (see
|
||||
//! `wifi-densepose-sensing-server/src/field_localize.rs`). Numeric defaults
|
||||
//! are interaction-design choices (`CLAIMED`); benchmark numbers live in the
|
||||
//! README tagged `MEASURED` with their reproducer.
|
||||
//!
|
||||
//! ## Native quick start
|
||||
//!
|
||||
//! ```
|
||||
//! use ruview_offaxis::{off_axis, Screen, Vec3};
|
||||
//!
|
||||
//! // A 60 cm × 34 cm screen centered at the origin, eye 65 cm away and
|
||||
//! // 10 cm to the right.
|
||||
//! let screen = Screen::centered(0.60, 0.34)?;
|
||||
//! let oa = off_axis(&screen, Vec3::new(0.10, 0.0, 0.65), 0.05, 100.0)?;
|
||||
//! // Column-major, ready for three.js Matrix4.fromArray / any GL pipeline.
|
||||
//! let _m: [f64; 16] = oa.view_projection();
|
||||
//! # Ok::<(), ruview_offaxis::OffAxisError>(())
|
||||
//! ```
|
||||
|
||||
#![forbid(unsafe_code)]
|
||||
#![warn(missing_docs)]
|
||||
|
||||
pub mod filter;
|
||||
pub mod math;
|
||||
pub mod projection;
|
||||
pub mod rf;
|
||||
|
||||
#[cfg(target_arch = "wasm32")]
|
||||
pub mod wasm;
|
||||
|
||||
pub use filter::{OneEuro, OneEuro3, OneEuroConfig};
|
||||
pub use math::{Mat4, Vec3};
|
||||
pub use projection::{off_axis, OffAxis, OffAxisError, Screen, MIN_EYE_DISTANCE};
|
||||
pub use rf::{
|
||||
field_peak, CoarseParallax, CoarseParallaxConfig, FieldPeak, ScreenCalibration,
|
||||
FIELD_PEAK_THRESHOLD, FIELD_X_SCALE, FIELD_Z_SCALE,
|
||||
};
|
||||
177
v2/crates/ruview-offaxis/src/math.rs
Normal file
177
v2/crates/ruview-offaxis/src/math.rs
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
//! Minimal 3-vector and column-major 4×4 matrix helpers.
|
||||
//!
|
||||
//! Deliberately dependency-free: this crate targets wasm32 for the ADR-324
|
||||
//! demo surface, and a `nalgebra` pull would dominate the module size for
|
||||
//! what is a handful of fixed-size operations. Matrices use the OpenGL /
|
||||
//! three.js `Matrix4.elements` layout: column-major, `m[col * 4 + row]`.
|
||||
|
||||
/// A 3-component `f64` vector.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Default)]
|
||||
pub struct Vec3 {
|
||||
/// X component.
|
||||
pub x: f64,
|
||||
/// Y component.
|
||||
pub y: f64,
|
||||
/// Z component.
|
||||
pub z: f64,
|
||||
}
|
||||
|
||||
impl Vec3 {
|
||||
/// Construct from components.
|
||||
#[inline]
|
||||
pub const fn new(x: f64, y: f64, z: f64) -> Self {
|
||||
Self { x, y, z }
|
||||
}
|
||||
|
||||
/// Scale by `s`.
|
||||
#[inline]
|
||||
pub fn scale(self, s: f64) -> Vec3 {
|
||||
Vec3::new(self.x * s, self.y * s, self.z * s)
|
||||
}
|
||||
|
||||
/// Dot product.
|
||||
#[inline]
|
||||
pub fn dot(self, rhs: Vec3) -> f64 {
|
||||
self.x * rhs.x + self.y * rhs.y + self.z * rhs.z
|
||||
}
|
||||
|
||||
/// Cross product (right-handed).
|
||||
#[inline]
|
||||
pub fn cross(self, rhs: Vec3) -> Vec3 {
|
||||
Vec3::new(
|
||||
self.y * rhs.z - self.z * rhs.y,
|
||||
self.z * rhs.x - self.x * rhs.z,
|
||||
self.x * rhs.y - self.y * rhs.x,
|
||||
)
|
||||
}
|
||||
|
||||
/// Euclidean length.
|
||||
#[inline]
|
||||
pub fn length(self) -> f64 {
|
||||
self.dot(self).sqrt()
|
||||
}
|
||||
|
||||
/// Unit vector, or `None` when the length is (near) zero.
|
||||
#[inline]
|
||||
pub fn normalize(self) -> Option<Vec3> {
|
||||
let len = self.length();
|
||||
if len <= f64::EPSILON {
|
||||
None
|
||||
} else {
|
||||
Some(self.scale(1.0 / len))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl core::ops::Sub for Vec3 {
|
||||
type Output = Vec3;
|
||||
#[inline]
|
||||
fn sub(self, rhs: Vec3) -> Vec3 {
|
||||
Vec3::new(self.x - rhs.x, self.y - rhs.y, self.z - rhs.z)
|
||||
}
|
||||
}
|
||||
|
||||
impl core::ops::Add for Vec3 {
|
||||
type Output = Vec3;
|
||||
#[inline]
|
||||
fn add(self, rhs: Vec3) -> Vec3 {
|
||||
Vec3::new(self.x + rhs.x, self.y + rhs.y, self.z + rhs.z)
|
||||
}
|
||||
}
|
||||
|
||||
/// Column-major 4×4 matrix, `m[col * 4 + row]` — the exact layout of
|
||||
/// three.js `Matrix4.elements` / OpenGL, so the array can be passed to
|
||||
/// `Matrix4.fromArray` untouched.
|
||||
pub type Mat4 = [f64; 16];
|
||||
|
||||
/// The identity matrix.
|
||||
pub const IDENTITY: Mat4 = [
|
||||
1.0, 0.0, 0.0, 0.0, //
|
||||
0.0, 1.0, 0.0, 0.0, //
|
||||
0.0, 0.0, 1.0, 0.0, //
|
||||
0.0, 0.0, 0.0, 1.0,
|
||||
];
|
||||
|
||||
/// `a * b` (column-major).
|
||||
#[inline]
|
||||
pub fn mul(a: &Mat4, b: &Mat4) -> Mat4 {
|
||||
let mut out = [0.0; 16];
|
||||
for col in 0..4 {
|
||||
for row in 0..4 {
|
||||
let mut acc = 0.0;
|
||||
for k in 0..4 {
|
||||
acc += a[k * 4 + row] * b[col * 4 + k];
|
||||
}
|
||||
out[col * 4 + row] = acc;
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Transform a point (`w = 1`); returns the transformed `(x, y, z)` and `w`
|
||||
/// *before* the perspective divide, so callers can check clip-space signs.
|
||||
#[inline]
|
||||
pub fn transform_point(m: &Mat4, p: Vec3) -> (Vec3, f64) {
|
||||
let x = m[0] * p.x + m[4] * p.y + m[8] * p.z + m[12];
|
||||
let y = m[1] * p.x + m[5] * p.y + m[9] * p.z + m[13];
|
||||
let z = m[2] * p.x + m[6] * p.y + m[10] * p.z + m[14];
|
||||
let w = m[3] * p.x + m[7] * p.y + m[11] * p.z + m[15];
|
||||
(Vec3::new(x, y, z), w)
|
||||
}
|
||||
|
||||
/// Transform a point and apply the perspective divide, yielding normalized
|
||||
/// device coordinates. Returns `None` when `w` is (near) zero.
|
||||
#[inline]
|
||||
pub fn project_point(m: &Mat4, p: Vec3) -> Option<Vec3> {
|
||||
let (v, w) = transform_point(m, p);
|
||||
if w.abs() <= f64::EPSILON {
|
||||
None
|
||||
} else {
|
||||
Some(v.scale(1.0 / w))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn identity_transforms_are_noops() {
|
||||
let p = Vec3::new(1.5, -2.0, 3.25);
|
||||
let (q, w) = transform_point(&IDENTITY, p);
|
||||
assert_eq!(q, p);
|
||||
assert_eq!(w, 1.0);
|
||||
assert_eq!(mul(&IDENTITY, &IDENTITY), IDENTITY);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cross_is_right_handed() {
|
||||
let x = Vec3::new(1.0, 0.0, 0.0);
|
||||
let y = Vec3::new(0.0, 1.0, 0.0);
|
||||
assert_eq!(x.cross(y), Vec3::new(0.0, 0.0, 1.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_rejects_zero() {
|
||||
assert!(Vec3::new(0.0, 0.0, 0.0).normalize().is_none());
|
||||
let n = Vec3::new(0.0, 3.0, 4.0).normalize().unwrap();
|
||||
assert!((n.length() - 1.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mul_matches_manual_translation_composition() {
|
||||
// T(a) * T(b) == T(a + b) for translations.
|
||||
let mut ta = IDENTITY;
|
||||
ta[12] = 1.0;
|
||||
ta[13] = 2.0;
|
||||
ta[14] = 3.0;
|
||||
let mut tb = IDENTITY;
|
||||
tb[12] = -4.0;
|
||||
tb[13] = 0.5;
|
||||
tb[14] = 7.0;
|
||||
let tab = mul(&ta, &tb);
|
||||
assert_eq!(tab[12], -3.0);
|
||||
assert_eq!(tab[13], 2.5);
|
||||
assert_eq!(tab[14], 10.0);
|
||||
}
|
||||
}
|
||||
449
v2/crates/ruview-offaxis/src/projection.rs
Normal file
449
v2/crates/ruview-offaxis/src/projection.rs
Normal file
|
|
@ -0,0 +1,449 @@
|
|||
//! Clean-room generalized (off-axis) perspective projection.
|
||||
//!
|
||||
//! Implements the screen-corner formulation published by Robert Kooima,
|
||||
//! "Generalized Perspective Projection" (2008): given a physical screen
|
||||
//! described by three of its corners and a tracked eye position in the same
|
||||
//! tracker space, produce the asymmetric frustum and view transform that make
|
||||
//! the screen behave as a window into the virtual scene.
|
||||
//!
|
||||
//! No code from any existing implementation (including the unlicensed
|
||||
//! `icurtis1/off-axis-sneaker` prior art referenced by ADR-324) was copied or
|
||||
//! consulted while writing this module; the derivation follows the published
|
||||
//! math only.
|
||||
//!
|
||||
//! ## Conventions
|
||||
//!
|
||||
//! - Right-handed tracker space, metres.
|
||||
//! - `pa` = screen lower-left, `pb` = lower-right, `pc` = upper-left corner.
|
||||
//! - The screen normal `vn = vr × vu` points toward the viewer's side; the
|
||||
//! eye must be on that side (`EyeBehindScreen` otherwise).
|
||||
//! - Output matrices are column-major `f64` in the three.js / OpenGL layout
|
||||
//! (see [`crate::math::Mat4`]). NDC follows OpenGL: visible x/y/z in
|
||||
//! `[-1, 1]`, camera looking down `-z` in eye space.
|
||||
|
||||
use crate::math::{mul, Mat4, Vec3, IDENTITY};
|
||||
use core::fmt;
|
||||
|
||||
/// Minimum eye-to-screen-plane distance (metres). Below this the frustum
|
||||
/// degenerates (division by ~0); callers get a typed error instead of NaNs.
|
||||
pub const MIN_EYE_DISTANCE: f64 = 1e-6;
|
||||
|
||||
/// Errors from screen construction or projection evaluation.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum OffAxisError {
|
||||
/// The three corners do not span a plane (coincident or collinear).
|
||||
DegenerateScreen,
|
||||
/// The eye is on or behind the screen plane (`distance <= MIN_EYE_DISTANCE`).
|
||||
EyeBehindScreen,
|
||||
/// `near`/`far` are not `0 < near < far`, or not finite.
|
||||
InvalidClipPlanes,
|
||||
/// A non-finite input coordinate was supplied.
|
||||
NonFiniteInput,
|
||||
}
|
||||
|
||||
impl fmt::Display for OffAxisError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
OffAxisError::DegenerateScreen => {
|
||||
write!(f, "screen corners are coincident or collinear")
|
||||
}
|
||||
OffAxisError::EyeBehindScreen => {
|
||||
write!(f, "eye is on or behind the screen plane")
|
||||
}
|
||||
OffAxisError::InvalidClipPlanes => {
|
||||
write!(f, "clip planes must satisfy 0 < near < far and be finite")
|
||||
}
|
||||
OffAxisError::NonFiniteInput => write!(f, "input coordinate is not finite"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for OffAxisError {}
|
||||
|
||||
/// A physical screen described by three corners, with its orthonormal basis
|
||||
/// precomputed at construction (the basis is eye-independent, so caching it
|
||||
/// keeps the per-frame [`off_axis`] call to the eye-dependent work only).
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct Screen {
|
||||
pa: Vec3,
|
||||
pb: Vec3,
|
||||
pc: Vec3,
|
||||
vr: Vec3,
|
||||
vu: Vec3,
|
||||
vn: Vec3,
|
||||
}
|
||||
|
||||
impl Screen {
|
||||
/// Build a screen from its lower-left (`pa`), lower-right (`pb`) and
|
||||
/// upper-left (`pc`) corners, in metres, in any orientation.
|
||||
pub fn new(pa: Vec3, pb: Vec3, pc: Vec3) -> Result<Self, OffAxisError> {
|
||||
for v in [pa, pb, pc] {
|
||||
if !(v.x.is_finite() && v.y.is_finite() && v.z.is_finite()) {
|
||||
return Err(OffAxisError::NonFiniteInput);
|
||||
}
|
||||
}
|
||||
let vr = (pb - pa)
|
||||
.normalize()
|
||||
.ok_or(OffAxisError::DegenerateScreen)?;
|
||||
let vu = (pc - pa)
|
||||
.normalize()
|
||||
.ok_or(OffAxisError::DegenerateScreen)?;
|
||||
let vn = vr
|
||||
.cross(vu)
|
||||
.normalize()
|
||||
.ok_or(OffAxisError::DegenerateScreen)?;
|
||||
Ok(Self {
|
||||
pa,
|
||||
pb,
|
||||
pc,
|
||||
vr,
|
||||
vu,
|
||||
vn,
|
||||
})
|
||||
}
|
||||
|
||||
/// Convenience: an axis-aligned screen of `width_m × height_m` metres,
|
||||
/// centered at the origin in the `z = 0` plane, normal facing `+z`
|
||||
/// (the viewer side). This matches the usual desktop-demo setup where
|
||||
/// the tracker origin is the screen center.
|
||||
pub fn centered(width_m: f64, height_m: f64) -> Result<Self, OffAxisError> {
|
||||
if !(width_m.is_finite() && height_m.is_finite()) {
|
||||
return Err(OffAxisError::NonFiniteInput);
|
||||
}
|
||||
if width_m <= 0.0 || height_m <= 0.0 {
|
||||
return Err(OffAxisError::DegenerateScreen);
|
||||
}
|
||||
let hw = width_m / 2.0;
|
||||
let hh = height_m / 2.0;
|
||||
Screen::new(
|
||||
Vec3::new(-hw, -hh, 0.0),
|
||||
Vec3::new(hw, -hh, 0.0),
|
||||
Vec3::new(-hw, hh, 0.0),
|
||||
)
|
||||
}
|
||||
|
||||
/// Lower-left corner.
|
||||
pub fn pa(&self) -> Vec3 {
|
||||
self.pa
|
||||
}
|
||||
|
||||
/// Lower-right corner.
|
||||
pub fn pb(&self) -> Vec3 {
|
||||
self.pb
|
||||
}
|
||||
|
||||
/// Upper-left corner.
|
||||
pub fn pc(&self) -> Vec3 {
|
||||
self.pc
|
||||
}
|
||||
|
||||
/// The implied upper-right corner `pb + (pc - pa)`.
|
||||
pub fn pd(&self) -> Vec3 {
|
||||
self.pb + self.pc - self.pa
|
||||
}
|
||||
|
||||
/// Unit right vector along the screen's bottom edge.
|
||||
pub fn vr(&self) -> Vec3 {
|
||||
self.vr
|
||||
}
|
||||
|
||||
/// Unit up vector along the screen's left edge.
|
||||
pub fn vu(&self) -> Vec3 {
|
||||
self.vu
|
||||
}
|
||||
|
||||
/// Unit normal, pointing toward the viewer side.
|
||||
pub fn vn(&self) -> Vec3 {
|
||||
self.vn
|
||||
}
|
||||
|
||||
/// Signed distance from `eye` to the screen plane along the normal
|
||||
/// (positive when the eye is on the viewer side).
|
||||
pub fn eye_distance(&self, eye: Vec3) -> f64 {
|
||||
// va = pa - pe; d = -(va · vn)
|
||||
-(self.pa - eye).dot(self.vn)
|
||||
}
|
||||
}
|
||||
|
||||
/// The result of a generalized projection evaluation.
|
||||
///
|
||||
/// `projection` is the asymmetric frustum; `view` is the rigid transform
|
||||
/// (screen-basis rotation + eye translation) taking tracker space into eye
|
||||
/// space. For three.js, either:
|
||||
///
|
||||
/// - set `camera.projectionMatrix` from `projection` and position/orient the
|
||||
/// camera from the eye and screen basis yourself, or
|
||||
/// - use [`OffAxis::view_projection`] as a single combined matrix when you
|
||||
/// manage matrices manually.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct OffAxis {
|
||||
/// Asymmetric perspective frustum (column-major).
|
||||
pub projection: Mat4,
|
||||
/// Screen-aligned view matrix: rotation into the screen basis composed
|
||||
/// with translation by the negated eye position (column-major).
|
||||
pub view: Mat4,
|
||||
}
|
||||
|
||||
impl OffAxis {
|
||||
/// `projection * view` — the full tracker-space-to-clip-space matrix.
|
||||
pub fn view_projection(&self) -> Mat4 {
|
||||
mul(&self.projection, &self.view)
|
||||
}
|
||||
}
|
||||
|
||||
/// OpenGL-convention asymmetric frustum matrix (column-major).
|
||||
#[inline]
|
||||
fn frustum(l: f64, r: f64, b: f64, t: f64, n: f64, f: f64) -> Mat4 {
|
||||
let mut m = [0.0; 16];
|
||||
m[0] = 2.0 * n / (r - l);
|
||||
m[5] = 2.0 * n / (t - b);
|
||||
m[8] = (r + l) / (r - l);
|
||||
m[9] = (t + b) / (t - b);
|
||||
m[10] = -(f + n) / (f - n);
|
||||
m[11] = -1.0;
|
||||
m[14] = -2.0 * f * n / (f - n);
|
||||
m
|
||||
}
|
||||
|
||||
/// Evaluate the generalized off-axis projection for `screen` as seen from
|
||||
/// `eye`, with the given clip planes.
|
||||
///
|
||||
/// Errors when the eye is on/behind the screen plane, when clip planes are
|
||||
/// invalid, or when inputs are non-finite. Never returns NaN-bearing
|
||||
/// matrices: every failure mode is a typed error (renderer-facing code must
|
||||
/// hold the last good matrix on error, not draw garbage).
|
||||
pub fn off_axis(screen: &Screen, eye: Vec3, near: f64, far: f64) -> Result<OffAxis, OffAxisError> {
|
||||
if !(eye.x.is_finite() && eye.y.is_finite() && eye.z.is_finite()) {
|
||||
return Err(OffAxisError::NonFiniteInput);
|
||||
}
|
||||
if !(near.is_finite() && far.is_finite()) || near <= 0.0 || far <= near {
|
||||
return Err(OffAxisError::InvalidClipPlanes);
|
||||
}
|
||||
|
||||
let (vr, vu, vn) = (screen.vr, screen.vu, screen.vn);
|
||||
|
||||
// Vectors from the eye to each screen corner.
|
||||
let va = screen.pa - eye;
|
||||
let vb = screen.pb - eye;
|
||||
let vc = screen.pc - eye;
|
||||
|
||||
// Distance from the eye to the screen plane.
|
||||
let d = -va.dot(vn);
|
||||
if d <= MIN_EYE_DISTANCE {
|
||||
return Err(OffAxisError::EyeBehindScreen);
|
||||
}
|
||||
|
||||
// Frustum extents on the near plane.
|
||||
let nd = near / d;
|
||||
let l = vr.dot(va) * nd;
|
||||
let r = vr.dot(vb) * nd;
|
||||
let b = vu.dot(va) * nd;
|
||||
let t = vu.dot(vc) * nd;
|
||||
|
||||
let projection = frustum(l, r, b, t, near, far);
|
||||
|
||||
// View = screen-basis rotation (rows vr/vu/vn) * translation by -eye.
|
||||
// Composed directly: the translation column is -R^T-rotated eye.
|
||||
let mut view = IDENTITY;
|
||||
view[0] = vr.x;
|
||||
view[4] = vr.y;
|
||||
view[8] = vr.z;
|
||||
view[1] = vu.x;
|
||||
view[5] = vu.y;
|
||||
view[9] = vu.z;
|
||||
view[2] = vn.x;
|
||||
view[6] = vn.y;
|
||||
view[10] = vn.z;
|
||||
view[12] = -vr.dot(eye);
|
||||
view[13] = -vu.dot(eye);
|
||||
view[14] = -vn.dot(eye);
|
||||
|
||||
Ok(OffAxis { projection, view })
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::math::project_point;
|
||||
|
||||
const EPS: f64 = 1e-9;
|
||||
|
||||
fn assert_close(a: f64, b: f64, eps: f64, what: &str) {
|
||||
assert!((a - b).abs() < eps, "{what}: {a} vs {b}");
|
||||
}
|
||||
|
||||
/// The defining invariant of off-axis projection: for ANY valid eye
|
||||
/// position, the physical screen corners land exactly on the NDC corners
|
||||
/// — pa→(-1,-1), pb→(1,-1), pc→(-1,1), pd→(1,1).
|
||||
#[test]
|
||||
fn screen_corners_map_to_ndc_corners_for_all_eyes() {
|
||||
let screen = Screen::centered(0.6, 0.34).unwrap();
|
||||
let mut checked = 0;
|
||||
for xi in -4..=4 {
|
||||
for yi in -3..=3 {
|
||||
for zi in 1..=6 {
|
||||
let eye = Vec3::new(xi as f64 * 0.25, yi as f64 * 0.2, zi as f64 * 0.35);
|
||||
let oa = off_axis(&screen, eye, 0.05, 100.0).unwrap();
|
||||
let m = oa.view_projection();
|
||||
for (corner, ex, ey) in [
|
||||
(screen.pa(), -1.0, -1.0),
|
||||
(screen.pb(), 1.0, -1.0),
|
||||
(screen.pc(), -1.0, 1.0),
|
||||
(screen.pd(), 1.0, 1.0),
|
||||
] {
|
||||
let ndc = project_point(&m, corner).unwrap();
|
||||
assert_close(ndc.x, ex, EPS, "ndc.x");
|
||||
assert_close(ndc.y, ey, EPS, "ndc.y");
|
||||
}
|
||||
checked += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(checked, 9 * 7 * 6);
|
||||
}
|
||||
|
||||
/// The invariant survives an arbitrarily rotated + translated screen:
|
||||
/// the math is coordinate-frame independent.
|
||||
#[test]
|
||||
fn corner_invariant_holds_for_tilted_screen() {
|
||||
// A screen rotated ~30° about y and ~15° about x, shifted off origin.
|
||||
let (cy, sy) = (30f64.to_radians().cos(), 30f64.to_radians().sin());
|
||||
let (cx, sx) = (15f64.to_radians().cos(), 15f64.to_radians().sin());
|
||||
let rot = |v: Vec3| {
|
||||
// Ry then Rx.
|
||||
let v = Vec3::new(cy * v.x + sy * v.z, v.y, -sy * v.x + cy * v.z);
|
||||
Vec3::new(v.x, cx * v.y - sx * v.z, sx * v.y + cx * v.z)
|
||||
};
|
||||
let shift = Vec3::new(0.4, -0.2, 1.1);
|
||||
let pa = rot(Vec3::new(-0.3, -0.17, 0.0)) + shift;
|
||||
let pb = rot(Vec3::new(0.3, -0.17, 0.0)) + shift;
|
||||
let pc = rot(Vec3::new(-0.3, 0.17, 0.0)) + shift;
|
||||
let screen = Screen::new(pa, pb, pc).unwrap();
|
||||
|
||||
// An eye on the viewer side of the tilted plane.
|
||||
let eye = shift + rot(Vec3::new(0.1, 0.05, 0.8));
|
||||
let m = off_axis(&screen, eye, 0.01, 50.0)
|
||||
.unwrap()
|
||||
.view_projection();
|
||||
for (corner, ex, ey) in [
|
||||
(screen.pa(), -1.0, -1.0),
|
||||
(screen.pb(), 1.0, -1.0),
|
||||
(screen.pc(), -1.0, 1.0),
|
||||
(screen.pd(), 1.0, 1.0),
|
||||
] {
|
||||
let ndc = project_point(&m, corner).unwrap();
|
||||
assert_close(ndc.x, ex, EPS, "tilted ndc.x");
|
||||
assert_close(ndc.y, ey, EPS, "tilted ndc.y");
|
||||
}
|
||||
}
|
||||
|
||||
/// A centered eye must reduce to the ordinary symmetric perspective
|
||||
/// frustum: l == -r, b == -t, and the projection matrix has no skew
|
||||
/// terms (m[8] == m[9] == 0).
|
||||
#[test]
|
||||
fn centered_eye_is_symmetric_perspective() {
|
||||
let screen = Screen::centered(0.6, 0.34).unwrap();
|
||||
let d = 0.7;
|
||||
let near = 0.05;
|
||||
let oa = off_axis(&screen, Vec3::new(0.0, 0.0, d), near, 100.0).unwrap();
|
||||
assert_close(oa.projection[8], 0.0, EPS, "skew x");
|
||||
assert_close(oa.projection[9], 0.0, EPS, "skew y");
|
||||
// fovy check: m[5] == 1/tan(fovy/2), tan(fovy/2) = (h/2)/d.
|
||||
let expected_m5 = d / (0.34 / 2.0);
|
||||
assert_close(oa.projection[5], expected_m5, 1e-9, "m[5] focal");
|
||||
// aspect: m[0] == m[5]/aspect.
|
||||
let expected_m0 = d / (0.6 / 2.0);
|
||||
assert_close(oa.projection[0], expected_m0, 1e-9, "m[0] focal");
|
||||
}
|
||||
|
||||
/// Depth convention: a point at `near` in front of the eye maps to NDC
|
||||
/// z = -1; a point at `far` maps to +1 (OpenGL convention).
|
||||
#[test]
|
||||
fn near_and_far_map_to_ndc_depth_bounds() {
|
||||
let screen = Screen::centered(0.6, 0.34).unwrap();
|
||||
let eye = Vec3::new(0.12, -0.05, 0.65);
|
||||
let (near, far) = (0.05, 40.0);
|
||||
let m = off_axis(&screen, eye, near, far).unwrap().view_projection();
|
||||
// Looking direction in tracker space is -vn.
|
||||
let toward = screen.vn().scale(-1.0);
|
||||
let p_near = eye + toward.scale(near);
|
||||
let p_far = eye + toward.scale(far);
|
||||
assert_close(project_point(&m, p_near).unwrap().z, -1.0, 1e-7, "near z");
|
||||
assert_close(project_point(&m, p_far).unwrap().z, 1.0, 1e-7, "far z");
|
||||
}
|
||||
|
||||
/// A point exactly on the screen plane keeps the same NDC (x, y) for
|
||||
/// every eye position — this is the "window" property: the screen
|
||||
/// surface itself is the fixed point of the illusion.
|
||||
#[test]
|
||||
fn screen_plane_points_are_eye_invariant() {
|
||||
let screen = Screen::centered(0.5, 0.3).unwrap();
|
||||
// A point 30% right / 20% up from screen center, on the plane.
|
||||
let p = Vec3::new(0.5 * 0.3, 0.3 * 0.2, 0.0);
|
||||
let mut first: Option<(f64, f64)> = None;
|
||||
for eye in [
|
||||
Vec3::new(0.0, 0.0, 0.6),
|
||||
Vec3::new(0.3, 0.1, 0.4),
|
||||
Vec3::new(-0.5, -0.2, 1.2),
|
||||
] {
|
||||
let m = off_axis(&screen, eye, 0.05, 100.0)
|
||||
.unwrap()
|
||||
.view_projection();
|
||||
let ndc = project_point(&m, p).unwrap();
|
||||
match first {
|
||||
None => first = Some((ndc.x, ndc.y)),
|
||||
Some((fx, fy)) => {
|
||||
assert_close(ndc.x, fx, EPS, "plane-point ndc.x eye-invariance");
|
||||
assert_close(ndc.y, fy, EPS, "plane-point ndc.y eye-invariance");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn error_paths_are_typed() {
|
||||
let screen = Screen::centered(0.6, 0.34).unwrap();
|
||||
let ok_eye = Vec3::new(0.0, 0.0, 0.5);
|
||||
// Degenerate screens.
|
||||
let p = Vec3::new(0.0, 0.0, 0.0);
|
||||
assert_eq!(
|
||||
Screen::new(p, p, Vec3::new(0.0, 1.0, 0.0)).unwrap_err(),
|
||||
OffAxisError::DegenerateScreen
|
||||
);
|
||||
assert_eq!(
|
||||
Screen::new(p, Vec3::new(1.0, 0.0, 0.0), Vec3::new(2.0, 0.0, 0.0)).unwrap_err(),
|
||||
OffAxisError::DegenerateScreen
|
||||
);
|
||||
// Eye behind / on the plane.
|
||||
assert_eq!(
|
||||
off_axis(&screen, Vec3::new(0.0, 0.0, -0.5), 0.05, 10.0).unwrap_err(),
|
||||
OffAxisError::EyeBehindScreen
|
||||
);
|
||||
assert_eq!(
|
||||
off_axis(&screen, Vec3::new(0.2, 0.1, 0.0), 0.05, 10.0).unwrap_err(),
|
||||
OffAxisError::EyeBehindScreen
|
||||
);
|
||||
// Bad clip planes.
|
||||
assert_eq!(
|
||||
off_axis(&screen, ok_eye, 0.0, 10.0).unwrap_err(),
|
||||
OffAxisError::InvalidClipPlanes
|
||||
);
|
||||
assert_eq!(
|
||||
off_axis(&screen, ok_eye, 1.0, 1.0).unwrap_err(),
|
||||
OffAxisError::InvalidClipPlanes
|
||||
);
|
||||
assert_eq!(
|
||||
off_axis(&screen, ok_eye, -1.0, 10.0).unwrap_err(),
|
||||
OffAxisError::InvalidClipPlanes
|
||||
);
|
||||
// Non-finite input.
|
||||
assert_eq!(
|
||||
off_axis(&screen, Vec3::new(f64::NAN, 0.0, 0.5), 0.05, 10.0).unwrap_err(),
|
||||
OffAxisError::NonFiniteInput
|
||||
);
|
||||
assert_eq!(
|
||||
off_axis(&screen, ok_eye, f64::INFINITY, f64::INFINITY).unwrap_err(),
|
||||
OffAxisError::InvalidClipPlanes
|
||||
);
|
||||
}
|
||||
}
|
||||
442
v2/crates/ruview-offaxis/src/rf.rs
Normal file
442
v2/crates/ruview-offaxis/src/rf.rs
Normal file
|
|
@ -0,0 +1,442 @@
|
|||
//! RF-input mapping for the ADR-324 demo tiers.
|
||||
//!
|
||||
//! This module turns RuView's *existing* RF surfaces into a bounded,
|
||||
//! honestly-labeled eye position for the off-axis camera:
|
||||
//!
|
||||
//! - [`field_peak`] extracts the strongest cell of a `/ws/sensing`
|
||||
//! `signal_field` grid using the **same** grid→world mapping as the
|
||||
//! sensing server's `field_localize.rs` (constants mirrored below, with
|
||||
//! the same honesty caveat: a single-link field peak is a representation
|
||||
//! of where field energy concentrates, **not** calibrated metric
|
||||
//! localization, and never a head position).
|
||||
//! - [`CoarseParallax`] converts field-peak motion into a **Tier B** eye
|
||||
//! offset: deadbanded, gain-limited, hard-clamped, one-euro filtered.
|
||||
//! Its output is coarse body parallax by construction — the clamps make
|
||||
//! over-claiming impossible at the API level.
|
||||
//! - [`ScreenCalibration`] holds the physical screen measurements and maps
|
||||
//! a normalized head position (Tier A, from any fine tracker the host
|
||||
//! provides) into metric eye coordinates in screen space.
|
||||
//!
|
||||
//! Evidence discipline: nothing in this module asserts an accuracy number.
|
||||
//! All numeric defaults are interaction-design choices (`CLAIMED`), not
|
||||
//! measured performance.
|
||||
|
||||
use crate::filter::{OneEuro, OneEuroConfig};
|
||||
use crate::math::Vec3;
|
||||
|
||||
/// Grid-cell → world X scale (metres per cell), mirroring
|
||||
/// `wifi-densepose-sensing-server/src/field_localize.rs::X_SCALE`.
|
||||
pub const FIELD_X_SCALE: f64 = 0.6;
|
||||
/// Grid-cell → world Z scale (metres per cell), mirroring
|
||||
/// `field_localize.rs::Z_SCALE`.
|
||||
pub const FIELD_Z_SCALE: f64 = 0.5;
|
||||
/// Minimum normalized field value for a cell to count as a real peak,
|
||||
/// mirroring `field_localize.rs::PEAK_THRESHOLD`.
|
||||
pub const FIELD_PEAK_THRESHOLD: f64 = 0.35;
|
||||
|
||||
/// The strongest cell of a signal-field grid, in the demo's world mapping.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct FieldPeak {
|
||||
/// World X (metres): `(ix - nx/2) * FIELD_X_SCALE`.
|
||||
pub x: f64,
|
||||
/// World Z (metres): `(iz - nz/2) * FIELD_Z_SCALE`.
|
||||
pub z: f64,
|
||||
/// The peak's normalized field value in `[0, 1]`.
|
||||
pub value: f64,
|
||||
/// Grid column of the peak.
|
||||
pub ix: usize,
|
||||
/// Grid row of the peak.
|
||||
pub iz: usize,
|
||||
}
|
||||
|
||||
/// Find the strongest field cell at or above [`FIELD_PEAK_THRESHOLD`].
|
||||
///
|
||||
/// `values` is row-major with the sensing server's layout
|
||||
/// (`idx = iz * nx + ix`). Returns `None` when the slice length doesn't
|
||||
/// match `nx * nz`, when the grid is empty, or when no cell reaches the
|
||||
/// threshold (an honest "no localizable hotspot" outcome, mirroring the
|
||||
/// server's gating).
|
||||
pub fn field_peak(values: &[f32], nx: usize, nz: usize) -> Option<FieldPeak> {
|
||||
field_peak_with_threshold(values, nx, nz, FIELD_PEAK_THRESHOLD)
|
||||
}
|
||||
|
||||
/// [`field_peak`] with a caller-supplied threshold (used by tests and by
|
||||
/// demos that want to visualize sub-threshold energy without moving the
|
||||
/// camera).
|
||||
pub fn field_peak_with_threshold(
|
||||
values: &[f32],
|
||||
nx: usize,
|
||||
nz: usize,
|
||||
threshold: f64,
|
||||
) -> Option<FieldPeak> {
|
||||
if nx == 0 || nz == 0 || values.len() != nx * nz {
|
||||
return None;
|
||||
}
|
||||
// Branch-light argmax: a NaN never satisfies `v > best_v`, so non-finite
|
||||
// cells are skipped without an explicit is_finite() in the hot loop
|
||||
// (+inf is excluded by the finite check on the winner below).
|
||||
let mut best_idx = usize::MAX;
|
||||
let mut best_v = f32::NEG_INFINITY;
|
||||
for (idx, &v) in values.iter().enumerate() {
|
||||
if v > best_v {
|
||||
best_v = v;
|
||||
best_idx = idx;
|
||||
}
|
||||
}
|
||||
if best_idx == usize::MAX || !best_v.is_finite() {
|
||||
return None;
|
||||
}
|
||||
let (idx, v) = (best_idx, best_v);
|
||||
// Compare in f32: grid values arrive as f32 (Float32Array), and casting
|
||||
// 0.35_f32 up to f64 lands a hair below a 0.35_f64 threshold.
|
||||
if v < threshold as f32 {
|
||||
return None;
|
||||
}
|
||||
let ix = idx % nx;
|
||||
let iz = idx / nx;
|
||||
Some(FieldPeak {
|
||||
x: (ix as f64 - nx as f64 / 2.0) * FIELD_X_SCALE,
|
||||
z: (iz as f64 - nz as f64 / 2.0) * FIELD_Z_SCALE,
|
||||
value: v as f64,
|
||||
ix,
|
||||
iz,
|
||||
})
|
||||
}
|
||||
|
||||
/// Tier B parameters. Every default is an interaction-design choice
|
||||
/// (`CLAIMED`), deliberately conservative so the mode reads as what it is:
|
||||
/// slow body-scale parallax, not head tracking.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct CoarseParallaxConfig {
|
||||
/// Eye metres produced per body metre of peak movement (≤ 1 keeps the
|
||||
/// effect visibly sub-physical).
|
||||
pub gain: f64,
|
||||
/// Peak movement below this (metres, from the session origin) is
|
||||
/// ignored entirely — RF field peaks jitter, and the deadband keeps a
|
||||
/// still room visually still.
|
||||
pub deadband_m: f64,
|
||||
/// Hard clamp on the |x| eye offset (metres). The API cannot emit a
|
||||
/// larger excursion regardless of input.
|
||||
pub max_offset_m: f64,
|
||||
/// Nominal viewing distance (metres) used as the eye's Z when no depth
|
||||
/// modulation applies.
|
||||
pub base_distance_m: f64,
|
||||
/// One-euro parameters for the offset filter. Tier B wants heavy
|
||||
/// smoothing: default `min_cutoff` is well below the Tier A default.
|
||||
pub filter: OneEuroConfig,
|
||||
}
|
||||
|
||||
impl Default for CoarseParallaxConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
gain: 0.5,
|
||||
deadband_m: 0.15,
|
||||
max_offset_m: 0.35,
|
||||
base_distance_m: 0.65,
|
||||
filter: OneEuroConfig {
|
||||
min_cutoff: 0.4,
|
||||
beta: 0.2,
|
||||
d_cutoff: 1.0,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Tier B: field-peak motion → bounded, smoothed eye position.
|
||||
///
|
||||
/// The first accepted peak establishes a session origin; subsequent peaks
|
||||
/// move the eye relative to it. Peaks below threshold (i.e. `None` from
|
||||
/// [`field_peak`]) hold the last eye position — the camera never snaps.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct CoarseParallax {
|
||||
cfg: CoarseParallaxConfig,
|
||||
fx: OneEuro,
|
||||
fz: OneEuro,
|
||||
origin: Option<(f64, f64)>,
|
||||
eye: Vec3,
|
||||
}
|
||||
|
||||
impl CoarseParallax {
|
||||
/// Create with the given configuration.
|
||||
pub fn new(cfg: CoarseParallaxConfig) -> Self {
|
||||
Self {
|
||||
cfg,
|
||||
fx: OneEuro::new(cfg.filter),
|
||||
fz: OneEuro::new(cfg.filter),
|
||||
origin: None,
|
||||
eye: Vec3::new(0.0, 0.0, cfg.base_distance_m),
|
||||
}
|
||||
}
|
||||
|
||||
/// The current (last computed) eye position in screen space (metres).
|
||||
pub fn eye(&self) -> Vec3 {
|
||||
self.eye
|
||||
}
|
||||
|
||||
/// Forget the session origin and filter state; the eye returns to the
|
||||
/// centered rest position.
|
||||
pub fn reset(&mut self) {
|
||||
self.origin = None;
|
||||
self.fx.reset();
|
||||
self.fz.reset();
|
||||
self.eye = Vec3::new(0.0, 0.0, self.cfg.base_distance_m);
|
||||
}
|
||||
|
||||
/// Ingest a field peak observed at `t_s` seconds; returns the updated
|
||||
/// eye position. Call with the output of [`field_peak`]; pass `None`
|
||||
/// (below-threshold field) to hold the current position.
|
||||
pub fn update(&mut self, peak: Option<FieldPeak>, t_s: f64) -> Vec3 {
|
||||
let Some(p) = peak else { return self.eye };
|
||||
let (ox, oz) = *self.origin.get_or_insert((p.x, p.z));
|
||||
let dx = deadband(p.x - ox, self.cfg.deadband_m);
|
||||
let dz = deadband(p.z - oz, self.cfg.deadband_m);
|
||||
let raw_x = (dx * self.cfg.gain).clamp(-self.cfg.max_offset_m, self.cfg.max_offset_m);
|
||||
// Peak Z (room depth) modulates viewing distance, same bound.
|
||||
let raw_z = (dz * self.cfg.gain).clamp(-self.cfg.max_offset_m, self.cfg.max_offset_m);
|
||||
let x = self.fx.filter(raw_x, t_s);
|
||||
let z_off = self.fz.filter(raw_z, t_s);
|
||||
// Keep the eye strictly in front of the screen: distance floor at
|
||||
// half the nominal distance.
|
||||
let z = (self.cfg.base_distance_m + z_off).max(self.cfg.base_distance_m * 0.5);
|
||||
self.eye = Vec3::new(x, 0.0, z);
|
||||
self.eye
|
||||
}
|
||||
}
|
||||
|
||||
/// Zero inside `±band`, shifted toward zero outside it (continuous at the
|
||||
/// band edge, so motion doesn't jump when leaving the deadband).
|
||||
#[inline]
|
||||
fn deadband(v: f64, band: f64) -> f64 {
|
||||
if v > band {
|
||||
v - band
|
||||
} else if v < -band {
|
||||
v + band
|
||||
} else {
|
||||
0.0
|
||||
}
|
||||
}
|
||||
|
||||
/// Physical screen calibration (Tier A): metric screen size plus the
|
||||
/// viewer's nominal distance. Mirrors the concept of a measured-screen
|
||||
/// calibration wizard; values persist wherever the host keeps them.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct ScreenCalibration {
|
||||
/// Physical screen width in metres.
|
||||
pub width_m: f64,
|
||||
/// Physical screen height in metres.
|
||||
pub height_m: f64,
|
||||
/// Nominal eye-to-screen distance in metres.
|
||||
pub base_distance_m: f64,
|
||||
}
|
||||
|
||||
impl ScreenCalibration {
|
||||
/// Construct from centimetre measurements (how humans measure screens).
|
||||
pub fn from_cm(width_cm: f64, height_cm: f64, distance_cm: f64) -> Self {
|
||||
Self {
|
||||
width_m: width_cm / 100.0,
|
||||
height_m: height_cm / 100.0,
|
||||
base_distance_m: distance_cm / 100.0,
|
||||
}
|
||||
}
|
||||
|
||||
/// The screen this calibration describes, centered at the origin
|
||||
/// (see [`crate::Screen::centered`]).
|
||||
pub fn screen(&self) -> Result<crate::Screen, crate::OffAxisError> {
|
||||
crate::Screen::centered(self.width_m, self.height_m)
|
||||
}
|
||||
|
||||
/// Map a normalized head position from a fine tracker into metric eye
|
||||
/// coordinates in screen space.
|
||||
///
|
||||
/// - `nx`, `ny`: head position in normalized image coordinates
|
||||
/// (`[0, 1]`, origin top-left, x rightward — the usual camera-image
|
||||
/// convention). The image is assumed mirrored (selfie view), so a
|
||||
/// viewer moving to *their* left moves the eye left in screen space.
|
||||
/// - `depth_scale`: multiplier on the nominal distance (1.0 = at the
|
||||
/// calibrated distance; a fine tracker derives it from e.g. apparent
|
||||
/// inter-feature distance). Clamped to `[0.25, 4.0]`.
|
||||
/// - `lateral_range_m`: metres of eye travel represented by the full
|
||||
/// image width/height. Deployment-specific; the screen width is a
|
||||
/// reasonable default.
|
||||
pub fn eye_from_normalized(
|
||||
&self,
|
||||
nx: f64,
|
||||
ny: f64,
|
||||
depth_scale: f64,
|
||||
lateral_range_m: f64,
|
||||
) -> Vec3 {
|
||||
let cx = (nx.clamp(0.0, 1.0) - 0.5) * lateral_range_m;
|
||||
let cy = (0.5 - ny.clamp(0.0, 1.0)) * lateral_range_m * (self.height_m / self.width_m);
|
||||
let z = self.base_distance_m * depth_scale.clamp(0.25, 4.0);
|
||||
Vec3::new(cx, cy, z)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn grid(nx: usize, nz: usize, hot: &[(usize, usize, f32)]) -> Vec<f32> {
|
||||
let mut v = vec![0.05_f32; nx * nz];
|
||||
for &(ix, iz, val) in hot {
|
||||
v[iz * nx + ix] = val;
|
||||
}
|
||||
v
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn peak_mapping_matches_field_localize_layout() {
|
||||
// 20×20 grid, hot cell at (ix=15, iz=4).
|
||||
let g = grid(20, 20, &[(15, 4, 0.9)]);
|
||||
let p = field_peak(&g, 20, 20).unwrap();
|
||||
assert_eq!(p.ix, 15);
|
||||
assert_eq!(p.iz, 4);
|
||||
// world_x = (15 - 10) * 0.6, world_z = (4 - 10) * 0.5 — the exact
|
||||
// transform documented in field_localize.rs / the Observatory.
|
||||
assert!((p.x - 3.0).abs() < 1e-12);
|
||||
assert!((p.z - -3.0).abs() < 1e-12);
|
||||
assert!((p.value - 0.9).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn below_threshold_and_malformed_grids_yield_none() {
|
||||
let g = grid(20, 20, &[(3, 3, 0.34)]);
|
||||
assert_eq!(field_peak(&g, 20, 20), None, "0.34 < threshold 0.35");
|
||||
assert!(field_peak(&g, 19, 20).is_none(), "length mismatch");
|
||||
assert!(field_peak(&[], 0, 0).is_none(), "empty grid");
|
||||
// NaN cells are skipped, not propagated.
|
||||
let mut g = grid(4, 4, &[(1, 1, 0.8)]);
|
||||
g[0] = f32::NAN;
|
||||
let p = field_peak(&g, 4, 4).unwrap();
|
||||
assert_eq!((p.ix, p.iz), (1, 1));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn exact_threshold_is_accepted() {
|
||||
let g = grid(20, 20, &[(2, 2, 0.35)]);
|
||||
assert!(field_peak(&g, 20, 20).is_some(), "threshold is inclusive");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn coarse_parallax_is_deadbanded_gained_and_clamped() {
|
||||
let cfg = CoarseParallaxConfig::default();
|
||||
let mut cp = CoarseParallax::new(cfg);
|
||||
|
||||
// First peak sets the origin: eye stays at rest.
|
||||
let origin = FieldPeak {
|
||||
x: 1.2,
|
||||
z: -0.5,
|
||||
value: 0.8,
|
||||
ix: 12,
|
||||
iz: 9,
|
||||
};
|
||||
let eye0 = cp.update(Some(origin), 0.0);
|
||||
assert_eq!(eye0, Vec3::new(0.0, 0.0, cfg.base_distance_m));
|
||||
|
||||
// Movement inside the deadband: still at rest.
|
||||
let small = FieldPeak {
|
||||
x: 1.2 + 0.1,
|
||||
..origin
|
||||
};
|
||||
let eye1 = cp.update(Some(small), 0.5);
|
||||
assert_eq!(eye1.x, 0.0, "0.1 m < 0.15 m deadband");
|
||||
|
||||
// A huge excursion is clamped to max_offset regardless of gain.
|
||||
let huge = FieldPeak {
|
||||
x: 1.2 + 50.0,
|
||||
..origin
|
||||
};
|
||||
let mut eye = Vec3::default();
|
||||
for i in 0..600 {
|
||||
eye = cp.update(Some(huge), 1.0 + i as f64 * 0.05);
|
||||
}
|
||||
assert!(eye.x <= cfg.max_offset_m + 1e-9, "clamped: {}", eye.x);
|
||||
assert!(
|
||||
eye.x > cfg.max_offset_m * 0.9,
|
||||
"filter converged near clamp"
|
||||
);
|
||||
|
||||
// Below-threshold (None) holds position; never snaps back.
|
||||
let held = cp.update(None, 100.0);
|
||||
assert_eq!(held, eye);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn coarse_parallax_depth_never_crosses_the_screen() {
|
||||
let cfg = CoarseParallaxConfig::default();
|
||||
let mut cp = CoarseParallax::new(cfg);
|
||||
let origin = FieldPeak {
|
||||
x: 0.0,
|
||||
z: 0.0,
|
||||
value: 0.9,
|
||||
ix: 10,
|
||||
iz: 10,
|
||||
};
|
||||
cp.update(Some(origin), 0.0);
|
||||
// Walk far toward the screen (negative z offset).
|
||||
let close = FieldPeak { z: -50.0, ..origin };
|
||||
let mut eye = Vec3::default();
|
||||
for i in 0..600 {
|
||||
eye = cp.update(Some(close), 0.1 + i as f64 * 0.05);
|
||||
}
|
||||
assert!(
|
||||
eye.z >= cfg.base_distance_m * 0.5 - 1e-9,
|
||||
"distance floor: {}",
|
||||
eye.z
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reset_returns_to_rest() {
|
||||
let mut cp = CoarseParallax::new(CoarseParallaxConfig::default());
|
||||
cp.update(
|
||||
Some(FieldPeak {
|
||||
x: 0.0,
|
||||
z: 0.0,
|
||||
value: 0.9,
|
||||
ix: 0,
|
||||
iz: 0,
|
||||
}),
|
||||
0.0,
|
||||
);
|
||||
cp.update(
|
||||
Some(FieldPeak {
|
||||
x: 9.0,
|
||||
z: 0.0,
|
||||
value: 0.9,
|
||||
ix: 0,
|
||||
iz: 0,
|
||||
}),
|
||||
1.0,
|
||||
);
|
||||
cp.reset();
|
||||
assert_eq!(
|
||||
cp.eye(),
|
||||
Vec3::new(0.0, 0.0, CoarseParallaxConfig::default().base_distance_m)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn calibration_maps_normalized_head_to_metric_eye() {
|
||||
let cal = ScreenCalibration::from_cm(60.0, 34.0, 65.0);
|
||||
assert!((cal.width_m - 0.6).abs() < 1e-12);
|
||||
|
||||
// Centered head at nominal depth = centered eye at base distance.
|
||||
let c = cal.eye_from_normalized(0.5, 0.5, 1.0, cal.width_m);
|
||||
assert_eq!(c, Vec3::new(0.0, 0.0, 0.65));
|
||||
|
||||
// Head at image right edge → eye half the range to the right;
|
||||
// image y grows downward → ny=0 (top) is +y in screen space.
|
||||
let e = cal.eye_from_normalized(1.0, 0.0, 1.0, 0.6);
|
||||
assert!((e.x - 0.3).abs() < 1e-12);
|
||||
assert!(e.y > 0.0);
|
||||
|
||||
// Depth scale is clamped to a sane band.
|
||||
assert_eq!(cal.eye_from_normalized(0.5, 0.5, 100.0, 0.6).z, 0.65 * 4.0);
|
||||
assert_eq!(cal.eye_from_normalized(0.5, 0.5, 0.0, 0.6).z, 0.65 * 0.25);
|
||||
|
||||
// Out-of-range normalized coords are clamped, not extrapolated.
|
||||
let clamped = cal.eye_from_normalized(7.0, -3.0, 1.0, 0.6);
|
||||
assert!((clamped.x - 0.3).abs() < 1e-12);
|
||||
}
|
||||
}
|
||||
202
v2/crates/ruview-offaxis/src/wasm.rs
Normal file
202
v2/crates/ruview-offaxis/src/wasm.rs
Normal file
|
|
@ -0,0 +1,202 @@
|
|||
//! wasm-bindgen surface (compiled only for `wasm32`).
|
||||
//!
|
||||
//! Exposes two small classes to JavaScript:
|
||||
//!
|
||||
//! - [`OffAxisCamera`] — Tier A/B shared core: screen calibration + one-euro
|
||||
//! filtering + Kooima projection. Feed it an eye position each frame, read
|
||||
//! back column-major matrices ready for `THREE.Matrix4.fromArray`.
|
||||
//! - [`RfParallax`] — Tier B input stage: `/ws/sensing` `signal_field`
|
||||
//! grids in, bounded coarse-parallax eye position out. Its output is
|
||||
//! coarse body parallax by construction (deadband + gain + clamp), so a
|
||||
//! demo cannot accidentally present it as head tracking.
|
||||
//!
|
||||
//! All methods that can fail return `Result<_, JsError>` (thrown as JS
|
||||
//! exceptions); per-frame update methods instead hold the last good state so
|
||||
//! a render loop never has to try/catch.
|
||||
|
||||
use crate::filter::{OneEuro3, OneEuroConfig};
|
||||
use crate::math::Vec3;
|
||||
use crate::projection::{off_axis, OffAxis, Screen};
|
||||
use crate::rf::{field_peak, CoarseParallax, CoarseParallaxConfig, ScreenCalibration};
|
||||
use wasm_bindgen::prelude::*;
|
||||
|
||||
/// Head-coupled off-axis camera for a physically calibrated screen.
|
||||
#[wasm_bindgen]
|
||||
pub struct OffAxisCamera {
|
||||
screen: Screen,
|
||||
cal: ScreenCalibration,
|
||||
near: f64,
|
||||
far: f64,
|
||||
filter: OneEuro3,
|
||||
current: OffAxis,
|
||||
eye: Vec3,
|
||||
}
|
||||
|
||||
#[wasm_bindgen]
|
||||
impl OffAxisCamera {
|
||||
/// Create from physical screen measurements in **centimetres** (the
|
||||
/// units a person measures with) plus clip planes in metres.
|
||||
#[wasm_bindgen(constructor)]
|
||||
pub fn new(
|
||||
screen_width_cm: f64,
|
||||
screen_height_cm: f64,
|
||||
viewing_distance_cm: f64,
|
||||
near_m: f64,
|
||||
far_m: f64,
|
||||
) -> Result<OffAxisCamera, JsError> {
|
||||
let cal =
|
||||
ScreenCalibration::from_cm(screen_width_cm, screen_height_cm, viewing_distance_cm);
|
||||
let screen = cal.screen().map_err(|e| JsError::new(&e.to_string()))?;
|
||||
let eye = Vec3::new(0.0, 0.0, cal.base_distance_m);
|
||||
let current =
|
||||
off_axis(&screen, eye, near_m, far_m).map_err(|e| JsError::new(&e.to_string()))?;
|
||||
Ok(Self {
|
||||
screen,
|
||||
cal,
|
||||
near: near_m,
|
||||
far: far_m,
|
||||
filter: OneEuro3::new(OneEuroConfig {
|
||||
min_cutoff: 1.0,
|
||||
beta: 0.3,
|
||||
d_cutoff: 1.0,
|
||||
}),
|
||||
current,
|
||||
eye,
|
||||
})
|
||||
}
|
||||
|
||||
/// Tune the one-euro filter (`min_cutoff` Hz, `beta`). Lower
|
||||
/// `min_cutoff` = smoother at rest; higher `beta` = less lag in motion.
|
||||
pub fn set_filter(&mut self, min_cutoff: f64, beta: f64) {
|
||||
self.filter.set_config(OneEuroConfig {
|
||||
min_cutoff,
|
||||
beta,
|
||||
d_cutoff: 1.0,
|
||||
});
|
||||
}
|
||||
|
||||
/// Reset filter state (e.g. after tracking was lost).
|
||||
pub fn reset_filter(&mut self) {
|
||||
self.filter.reset();
|
||||
}
|
||||
|
||||
/// Update from a **metric eye position** in screen space (metres,
|
||||
/// origin = screen center, +x right, +y up, +z toward the viewer) at
|
||||
/// time `t_s` seconds. Returns `true` when the matrices were updated;
|
||||
/// `false` when the sample was rejected (eye behind screen / non-finite)
|
||||
/// and the previous matrices were held.
|
||||
pub fn update_eye(&mut self, x_m: f64, y_m: f64, z_m: f64, t_s: f64) -> bool {
|
||||
let eye = self.filter.filter(Vec3::new(x_m, y_m, z_m), t_s);
|
||||
match off_axis(&self.screen, eye, self.near, self.far) {
|
||||
Ok(oa) => {
|
||||
self.current = oa;
|
||||
self.eye = eye;
|
||||
true
|
||||
}
|
||||
Err(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Update from a **normalized head position** (`nx`, `ny` in `[0, 1]`,
|
||||
/// camera-image convention, mirrored/selfie view) with a depth scale
|
||||
/// (1.0 = at the calibrated distance) — the Tier A path fed by any fine
|
||||
/// tracker the host runs. `lateral_range_m` is how many metres of eye
|
||||
/// travel the full image spans (screen width is a reasonable start).
|
||||
pub fn update_normalized(
|
||||
&mut self,
|
||||
nx: f64,
|
||||
ny: f64,
|
||||
depth_scale: f64,
|
||||
lateral_range_m: f64,
|
||||
t_s: f64,
|
||||
) -> bool {
|
||||
let eye = self
|
||||
.cal
|
||||
.eye_from_normalized(nx, ny, depth_scale, lateral_range_m);
|
||||
self.update_eye(eye.x, eye.y, eye.z, t_s)
|
||||
}
|
||||
|
||||
/// The current asymmetric projection matrix, column-major, 16 elements
|
||||
/// (pass straight to `THREE.Matrix4.fromArray`).
|
||||
pub fn projection(&self) -> Vec<f64> {
|
||||
self.current.projection.to_vec()
|
||||
}
|
||||
|
||||
/// The current screen-aligned view matrix, column-major, 16 elements.
|
||||
pub fn view(&self) -> Vec<f64> {
|
||||
self.current.view.to_vec()
|
||||
}
|
||||
|
||||
/// `projection * view` as one matrix, column-major, 16 elements.
|
||||
pub fn view_projection(&self) -> Vec<f64> {
|
||||
self.current.view_projection().to_vec()
|
||||
}
|
||||
|
||||
/// The current (filtered) eye position `[x, y, z]` in metres.
|
||||
pub fn eye(&self) -> Vec<f64> {
|
||||
vec![self.eye.x, self.eye.y, self.eye.z]
|
||||
}
|
||||
}
|
||||
|
||||
/// Tier B input stage: signal-field grids → bounded coarse-parallax eye.
|
||||
#[wasm_bindgen]
|
||||
pub struct RfParallax {
|
||||
parallax: CoarseParallax,
|
||||
last_value: f64,
|
||||
has_peak: bool,
|
||||
}
|
||||
|
||||
#[wasm_bindgen]
|
||||
impl RfParallax {
|
||||
/// Create with the default conservative Tier B tuning and the given
|
||||
/// nominal viewing distance (metres).
|
||||
#[wasm_bindgen(constructor)]
|
||||
pub fn new(base_distance_m: f64) -> RfParallax {
|
||||
let cfg = CoarseParallaxConfig {
|
||||
base_distance_m,
|
||||
..CoarseParallaxConfig::default()
|
||||
};
|
||||
RfParallax {
|
||||
parallax: CoarseParallax::new(cfg),
|
||||
last_value: 0.0,
|
||||
has_peak: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Ingest one `signal_field` grid (`values.length == nx * nz`,
|
||||
/// `idx = iz * nx + ix` — the `/ws/sensing` layout) observed at `t_s`
|
||||
/// seconds. Returns `true` when a peak at/above the server's threshold
|
||||
/// was found (eye moved), `false` when the field had no localizable
|
||||
/// hotspot (eye held).
|
||||
pub fn update(&mut self, values: &[f32], nx: usize, nz: usize, t_s: f64) -> bool {
|
||||
let peak = field_peak(values, nx, nz);
|
||||
self.has_peak = peak.is_some();
|
||||
self.last_value = peak.map_or(self.last_value, |p| p.value);
|
||||
self.parallax.update(peak, t_s);
|
||||
self.has_peak
|
||||
}
|
||||
|
||||
/// The current eye position `[x, y, z]` in metres (screen space) —
|
||||
/// feed to `OffAxisCamera.update_eye`.
|
||||
pub fn eye(&self) -> Vec<f64> {
|
||||
let e = self.parallax.eye();
|
||||
vec![e.x, e.y, e.z]
|
||||
}
|
||||
|
||||
/// Whether the last grid had an at/above-threshold peak.
|
||||
pub fn has_peak(&self) -> bool {
|
||||
self.has_peak
|
||||
}
|
||||
|
||||
/// The last accepted peak's normalized field value (HUD display).
|
||||
pub fn peak_value(&self) -> f64 {
|
||||
self.last_value
|
||||
}
|
||||
|
||||
/// Forget the session origin (person left / demo reset).
|
||||
pub fn reset(&mut self) {
|
||||
self.parallax.reset();
|
||||
self.has_peak = false;
|
||||
self.last_value = 0.0;
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue