vrr/Ruview
2
0
Fork 0
mirror of https://github.com/ruvnet/RuView.git synced 2026-10-21 05:32:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Merge pull request #1858 from clonea1/contrib/gate-experiment-findings

Browse source 
docs: measured comparison of three CSI frame-selection gates, and the harness
This commit is contained in:
rUv 2026-09-09 13:10:02 +00:00 • committed by GitHub
commit e5422d52e4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 348 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -719,6 +719,7 @@ Start with the user, build, and calibration guides; expand for the full referenc
| [Semantic Primitives — Precision/Recall](docs/integrations/semantic-primitives-metrics.md) | Per-primitive F1 on the held-out paired-capture set: someone-sleeping, possible-distress, room-active, elderly-inactivity-anomaly, meeting, bathroom, fall-risk, bed-exit, no-movement, multi-room. |
| [Claude Code / Codex Plugin](plugins/ruview/README.md) | The `ruview` plugin + marketplace — skills, `/ruview-*` commands, agents, and the Codex prompt mirror |
| [Portable harness — `npx @ruvnet/ruview`](harness/ruview/README.md) | MetaHarness-minted, host-portable RuView operator harness — `ruview.*` MCP tools + the MEASURED-vs-CLAIMED honesty guardrail enforced in code ([ADR-182](docs/adr/ADR-182-npx-ruview-harness-via-metaharness.md)). A lighter, multi-host companion to the in-repo plugin. |
| [CSI Frame Selection — Measured](docs/csi-frame-selection-measured.md) | Which rate gate actually buys cross-node pairing, measured on a nine-node ESP32-C6 fleet as a 2x2 factorial in three Latin-square blocks. Selecting frames by 802.11 `rx_seq` roughly doubled pairing (+31.7 pp) while mesh-time alignment bought +0.5 pp, inside noise. Includes the trade it makes, a prediction that turned out wrong, the limits of the measurement, and `run_arms.py` so the comparison can be re-run rather than taken on trust. |
| [ESP32-C6 Firmware Runbook](firmware/esp32-csi-node/RUNBOOK.md) | What to run, in what order, and what to check: the three-file build that keeps OTA rollback compiled in, a five-key pre-flash gate, which flash regions OTA can and cannot reach, a single-board diagnostic flash with a revert path, and the tunables whose right value depends on your house rather than on the chip. |
| [Architecture Decisions](docs/adr/README.md) | 205 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Domain Models](docs/ddd/README.md) | 8 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI, rvCSI) — bounded contexts, aggregates, domain events, and ubiquitous language |

165
docs/csi-frame-selection-measured.md Normal file
View file

@ -0,0 +1,165 @@
# Which CSI frames a node keeps decides whether cross-node fusion works
A measured comparison of three frame-selection gates on a nine-node ESP32-C6
fleet, and a method for repeating it in your own environment.
**This is one house.** Nine boards, one floor plan, one RF environment, one
transmitter population. The numbers below are what we measured here; the part
worth reusing is the method, not our figures. `run_arms.py` is included so you
can measure yours.
---
## The problem
Cross-node fusion pairs frames by transmission identity: two nodes are fusing
usefully only when they hold measurements of *the same packet in the air*. The
metric is `paired_fraction` — of the transmissions the fleet reported, what
share were captured by two or more nodes.
Every node also rate-limits which CSI callbacks it keeps, because processing
every callback at 100–500 Hz destabilises the WiFi ISR. **How that limiting is
done turns out to matter far more than we expected**, because it decides
whether two nodes keep the *same* frames or merely the *same number* of frames.
Three mechanisms, all valid rate limits:
| gate | rule |
|---|---|
| `elapsed` | at least 20 ms since **this node's** last accepted frame |
| `mesh` | first frame heard in each shared 20 ms window of a mesh-aligned clock |
| `seq` | keep frames where the 802.11 `rx_seq` is divisible by N |
`elapsed` gives every node an independent phase. `mesh` makes nodes agree on
the *window*. `seq` makes them agree on the *frame*, using a number the
transmitter assigns — so it needs no clock, no leader and no synchronisation.
## Method
Four arms — the three gates plus `mesh`+`seq` stacked — run as a 2x2 factorial.
Three deliberate choices, each of which we got wrong first:
**Whole fleet per arm, never a subset.** An earlier attempt compared four gated
nodes against four ungated ones. Nodes differ enormously in how much traffic
they hear, so that comparison could not be read at all.
**Verify the gate is actually running.** Each switch is read back from every
node before sampling. "Config was stored" is not "gate is live", and an earlier
run could not distinguish a null result from a change that never applied.
**Frame rate is NOT a valid check that a gate is working.** A bucket gate needs
only one survivor per window, so at ~14 candidate frames per bucket even a
perfect 1-in-4 gate moves frame rate about 2%. Use instead
R_window = (delta early_drop + delta accepted) / delta accepted
computed from **differenced** counters. Never divide lifetime totals: they
carry a large post-boot burst. One node read R = 1126 moments after a reboot,
26.5 an hour later, and 19.2 windowed.
**Establish the noise floor first.** A null run — three hours, no changes —
gave drift of sd **0.45 pp** across 20-minute windows, about 2x the ±0.2 pp
binomial counting error. Without that number none of the results below can be
called significant.
Arms are 20 minutes in Latin-square blocks (`U M S MS / S MS M U / MS U S M`),
so slow drift lands on every arm equally rather than on whichever ran first.
## Results
Three blocks, twelve arms, every arm verified live on all nine nodes.
| arm | block 0 | block 1 | block 2 | mean |
|---|---|---|---|---|
| `elapsed` | 0.2881 | 0.2409 | 0.2012 | **0.2434** |
| `mesh` | 0.2692 | 0.2580 | 0.2167 | **0.2480** |
| `seq` (period 8) | 0.5819 | 0.5573 | 0.5422 | **0.5605** |
| `mesh`+`seq` | 0.5639 | 0.5623 | 0.5093 | 0.5452 |
**`seq` roughly doubles pairing: +31.7 pp, 2.30x.** That gap is ~70x the
measured drift floor and reproduces in all three blocks; within-block contrasts
sit at +0.153 to +0.175 every time.
**Mesh-time alignment bought essentially nothing here: +0.5 pp**, about one
standard deviation of the noise floor. We built that gate and expected it to be
the answer; it is indistinguishable from doing nothing. Stacking it on top of
`seq` is marginally *worse* than `seq` alone.
### It is a trade, not a free win
At period 8 the seq arms carried ~75–95k transmissions per 20 minutes against
~120–160k ungated. You are buying pairing with raw frame rate. Absolute paired
transmissions per minute still went **up** (2629 vs 2524), because the frames
it discards are disproportionately ones only one node would have kept — but
whether that trade is right depends on what else consumes the stream.
One measurement that made the trade easier here: on-device edge processing was
**unaffected**. Frames actually processed stayed ~1900 per node per 4 minutes
in both modes, because that pipeline is separately rate-limited. The cost falls
on the uplink, not on local sensing.
### The mesh arm was partly an elapsed arm
The mesh gate uses mesh-time buckets only while `c6_sync_espnow_is_valid()`. A
node without valid sync falls back to the elapsed gate rather than gating on a
meaningless epoch -- correct behaviour, but it weakens this comparison.
Three of the nine nodes never synchronised during these runs. For those three,
the `mesh` and `mesh`+`seq` arms silently ran the elapsed gate. So **+0.5 pp is
not a clean fleet-wide measurement of window alignment**; it is "mesh where sync
held, elapsed where it did not", and the true effect of alignment on a fully
synchronised fleet is not measured here.
We are reporting the number as observed rather than reconstructing it, because
the arms were assigned before we understood the fallback, and a post-hoc
per-node reanalysis would not be the experiment we ran.
The cause is worth naming because it is environmental, not a firmware defect.
Time sync is leader-elected and hub-and-spoke: one node broadcasts and the rest
follow. Whether every node hears that leader is a property of the building. A
larger or more divided house fragments into sync domains, and any option that
depends on shared time then degrades quietly for the nodes that fell out --
which is precisely the situation where someone would most want to trust this
measurement.
### A prediction we got wrong
Our mesh had split into two leader domains with three nodes never synchronising
at all. We predicted `seq` would win *most* on node pairs spanning that split.
Per-pair yield says otherwise:
| pair group | pairs | seq / mesh |
|---|---|---|
| same sync domain | 7 | 5.82x |
| spans the two domains | 8 | 6.24x |
| involves a never-synced node | 21 | **5.16x** |
Cross-domain is marginally best, but the never-synced nodes are the *lowest* —
the opposite of the prediction — and within-group spread (3.6x–8.1x) dwarfs
every between-group difference.
So `seq` is **not** a workaround for broken time sync. It wins roughly
uniformly, including where sync is healthy. The useful consequence is negative:
repairing our mesh split would probably not have closed the gap, and we would
have spent that effort for nothing.
## What we suggest
Not "use `seq`". **Make it selectable and measure your own environment.** The
right answer depends on your transmitter population, your node placement and
what consumes your CSI stream — none of which we can see from here.
`run_arms.py` runs the factorial unattended: it applies each arm fleet-wide,
verifies every node adopted it, samples `/api/v1/fusion`, and writes one row
per arm. Run the null pass first; without your own drift floor the contrasts
cannot be judged.
## Limits
- One environment, nine nodes. Treat the numbers as an existence proof, not a
constant.
- Only period 8 was characterised. 4 and 16 are unmeasured, so the
yield/pairing curve has a single point on it.
- `paired_fraction` measures **frame pairing**, which is a precondition for
cross-receiver work. Nothing here measures sensing accuracy, and none of it
should be read as an accuracy claim.

182
firmware/esp32-csi-node/run_arms.py Normal file
View file

@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""Run the gate factorial: U / M / S / MS, Latin-square blocked.
Arms (NVS gate_mode, applied live -- no reboot):
0 U elapsed-time gate, as upstream ships it
1 M mesh-aligned bucket gate (ours, 4b54ef8e)
2 S rx_seq selection + elapsed underneath
3 MS both stacked
Why it is built this way, and what each guard is for:
* Every arm is applied to the WHOLE fleet, never to a subset. The stopped
2026-09-04 run compared four gated nodes against four different control
nodes; nodes differ, and that comparison could not be read.
* After every switch the runner reads back `active.gate_mode` from EVERY node
and refuses to sample unless all nine agree. "Config was stored" is not
"gate is running" -- that gap is exactly what the previous run could not
distinguish from a null result.
* Arms are short and blocked in a Latin square rather than run in long
sequential batches, so slow drift lands on all arms equally instead of being
aliased onto whichever arm ran first. Measured drift over 20 min windows is
sd 0.0045; counting noise is +/-0.002.
* Metrics come from differenced cumulative counters, so the server is never
restarted and the fleet stays in production throughout.
Output: arms_<label>.csv, one row per arm, in the fleet-baselines directory
(outside git -- it carries presence information).
"""
import argparse, csv, json, os, sys, time, urllib.request
PORT = 8032
SERVER = "http://127.0.0.1:3000"
OUTDIR = "C:/temp/ruview/fleet-baselines/gate-experiment"
NODES = {0: "192.168.1.171", 1: "192.168.1.219", 2: "192.168.1.72",
3: "192.168.1.112", 4: "192.168.1.144", 5: "192.168.1.198",
6: "192.168.1.199", 7: "192.168.1.25", 8: "192.168.1.122"}
ARM_NAME = {0: "U_elapsed", 1: "M_mesh", 2: "S_seq", 3: "MS_both"}
# Latin square: every arm appears once per block, in a different position.
BLOCKS = [[0, 1, 2, 3], [2, 3, 1, 0], [3, 0, 2, 1]]
def key():
p = os.environ.get("RUVIEW_OTA_PSK_FILE")
if not p or not os.path.isfile(p):
sys.exit("set RUVIEW_OTA_PSK_FILE to the psk file path")
return open(p, encoding="utf-8").read().strip()
def node_get(ip, path, k, timeout=10):
r = urllib.request.Request(f"http://{ip}:{PORT}{path}")
r.add_header("Authorization", "Bearer " + k)
with urllib.request.urlopen(r, timeout=timeout) as resp:
return json.load(resp)
def node_post(ip, path, k, payload, timeout=15):
body = json.dumps(payload).encode()
r = urllib.request.Request(f"http://{ip}:{PORT}{path}", data=body)
r.add_header("Authorization", "Bearer " + k)
r.add_header("Content-Type", "application/json")
with urllib.request.urlopen(r, timeout=timeout) as resp:
return resp.status, resp.read().decode("utf-8", "replace")
def srv(path):
with urllib.request.urlopen(SERVER + path, timeout=15) as r:
return json.load(r)
def set_arm(mode, period, k):
"""Apply an arm fleet-wide and PROVE every node is running it."""
for nid, ip in sorted(NODES.items()):
try:
node_post(ip, "/config", k, {"gate_mode": mode, "gate_seq_period": period})
except Exception as e:
print(f" n{nid} set FAILED: {e}", file=sys.stderr)
time.sleep(5)
bad = []
for nid, ip in sorted(NODES.items()):
try:
act = node_get(ip, "/config", k).get("active", {})
if act.get("gate_mode") != mode or act.get("gate_seq_period") != period:
bad.append((nid, act))
except Exception as e:
bad.append((nid, str(e)))
return bad
def snapshot():
f = srv("/api/v1/fusion")
m = srv("/api/v1/mesh")["nodes"]
nodes = {}
for k2, v in m.items():
tx = v.get("health", {}).get("tx", {})
nodes[k2] = {"acc": v.get("csi_fps_samples"), "ed": tx.get("early_drop"),
"fps": v.get("csi_fps_ema"), "valid": v.get("is_valid")}
pairs = {f"{p['a']}-{p['b']}": p["common"] for p in f.get("pairs", [])}
return {"t": time.time(), "observations": f["observations"],
"transmissions": f["transmissions"], "paired": f["paired"],
"nodes": nodes, "pairs": pairs}
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--label", default="factorial")
ap.add_argument("--arm-minutes", type=float, default=20.0)
ap.add_argument("--settle-minutes", type=float, default=3.0)
ap.add_argument("--period", type=int, default=8)
ap.add_argument("--blocks", type=int, default=3)
a = ap.parse_args()
k = key()
os.makedirs(OUTDIR, exist_ok=True)
out = os.path.join(OUTDIR, f"arms_{a.label}.csv")
new = not os.path.exists(out)
fh = open(out, "a", newline="", encoding="utf-8")
w = csv.writer(fh)
if new:
w.writerow(["block", "arm", "arm_name", "period", "start_iso", "end_iso",
"dur_s", "d_observations", "d_transmissions", "d_paired",
"paired_fraction", "node_json", "pairs_json", "verified"])
fh.flush()
print(f"gate factorial: {a.blocks} blocks x 4 arms x {a.arm_minutes:g} min "
f"(+{a.settle_minutes:g} min settle), seq period {a.period}")
print(f"est. total {a.blocks*4*(a.arm_minutes+a.settle_minutes)/60:.1f} h -> {out}\n")
for bi in range(a.blocks):
order = BLOCKS[bi % len(BLOCKS)]
print(f"--- block {bi}: {' '.join(ARM_NAME[m] for m in order)} ---")
for mode in order:
print(f"[{time.strftime('%H:%M:%S')}] arm {ARM_NAME[mode]} ...", flush=True)
bad = set_arm(mode, a.period, k)
if bad:
print(f" WARNING not all nodes on arm: {bad}", file=sys.stderr)
# Settle: the gate changes instantly, but csi_fps_ema and the
# fusion pairing window carry the previous arm for a while.
time.sleep(a.settle_minutes * 60)
s0 = snapshot()
time.sleep(a.arm_minutes * 60)
s1 = snapshot()
dtr = s1["transmissions"] - s0["transmissions"]
dp = s1["paired"] - s0["paired"]
pf = dp / dtr if dtr > 0 else float("nan")
nodes = {}
for nid in s1["nodes"]:
if nid in s0["nodes"]:
dacc = (s1["nodes"][nid]["acc"] or 0) - (s0["nodes"][nid]["acc"] or 0)
ded = (s1["nodes"][nid]["ed"] or 0) - (s0["nodes"][nid]["ed"] or 0)
nodes[nid] = {"d_acc": dacc, "d_ed": ded,
"R": round((dacc + ded) / dacc, 3) if dacc > 0 else None,
"fps": round(s1["nodes"][nid]["fps"] or 0, 2),
"valid": s1["nodes"][nid]["valid"]}
dpairs = {kk: s1["pairs"].get(kk, 0) - s0["pairs"].get(kk, 0)
for kk in s1["pairs"]}
w.writerow([bi, mode, ARM_NAME[mode], a.period,
time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(s0["t"])),
time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(s1["t"])),
round(s1["t"] - s0["t"]),
s1["observations"] - s0["observations"], dtr, dp,
f"{pf:.6f}",
json.dumps(nodes, separators=(",", ":")),
json.dumps(dpairs, separators=(",", ":")),
"no" if bad else "yes"])
fh.flush()
meanR = [v["R"] for v in nodes.values() if v["R"]]
print(f" paired_fraction={pf:.4f} transmissions={dtr} "
f"meanR={sum(meanR)/len(meanR):.2f}" if meanR else "", flush=True)
fh.close()
print("done")
if __name__ == "__main__":
main()