mirror of
https://github.com/ruvnet/RuView.git
synced 2026-10-20 03:27:35 +00:00
feat(inference): validate explicit native ACF reduction runtime
Some checks failed
Security Scanning / Rust Dependency Audit (push) Has been cancelled
Continuous Integration / Code Quality & Security (push) Has been cancelled
Continuous Integration / Rust Workspace Tests (push) Has been cancelled
Continuous Integration / UI JavaScript Tests (push) Has been cancelled
Continuous Integration / Tests (push) Has been cancelled
Continuous Integration / Tests-1 (push) Has been cancelled
Continuous Integration / Tests-2 (push) Has been cancelled
Security Scanning / Static Application Security Testing (push) Has been cancelled
Security Scanning / Dependency Vulnerability Scan (push) Has been cancelled
Security Scanning / Container Security Scan (push) Has been cancelled
Security Scanning / Infrastructure Security Scan (push) Has been cancelled
Security Scanning / Secret Scanning (push) Has been cancelled
Security Scanning / License Compliance Scan (push) Has been cancelled
Security Scanning / Security Policy Compliance (push) Has been cancelled
Continuous Integration / Performance Tests (push) Has been cancelled
Continuous Integration / Docker Build & Test (push) Has been cancelled
Continuous Integration / API Documentation (push) Has been cancelled
Continuous Integration / Notify (push) Has been cancelled
Security Scanning / Security Report (push) Has been cancelled
Some checks failed
Security Scanning / Rust Dependency Audit (push) Has been cancelled
Continuous Integration / Code Quality & Security (push) Has been cancelled
Continuous Integration / Rust Workspace Tests (push) Has been cancelled
Continuous Integration / UI JavaScript Tests (push) Has been cancelled
Continuous Integration / Tests (push) Has been cancelled
Continuous Integration / Tests-1 (push) Has been cancelled
Continuous Integration / Tests-2 (push) Has been cancelled
Security Scanning / Static Application Security Testing (push) Has been cancelled
Security Scanning / Dependency Vulnerability Scan (push) Has been cancelled
Security Scanning / Container Security Scan (push) Has been cancelled
Security Scanning / Infrastructure Security Scan (push) Has been cancelled
Security Scanning / Secret Scanning (push) Has been cancelled
Security Scanning / License Compliance Scan (push) Has been cancelled
Security Scanning / Security Policy Compliance (push) Has been cancelled
Continuous Integration / Performance Tests (push) Has been cancelled
Continuous Integration / Docker Build & Test (push) Has been cancelled
Continuous Integration / API Documentation (push) Has been cancelled
Continuous Integration / Notify (push) Has been cancelled
Security Scanning / Security Report (push) Has been cancelled
This commit is contained in:
parent
ab25f5466d
commit
e2cff64cb3
18 changed files with 1737 additions and 0 deletions
8
.github/workflows/csi-encoder-training.yml
vendored
8
.github/workflows/csi-encoder-training.yml
vendored
|
|
@ -51,6 +51,14 @@ jobs:
|
|||
run: python -m unittest discover -s scripts/tests -p 'test_espfi*.py' -v
|
||||
- name: Enforce capture data policy
|
||||
run: bash scripts/csi-data-policy-check.sh --tracked
|
||||
- name: Validate native feature bounds with sanitizers
|
||||
run: |
|
||||
cc -std=c11 -O1 -g -fno-fast-math -ffp-contract=off \
|
||||
-fsanitize=address,undefined -fno-omit-frame-pointer \
|
||||
scripts/csi_training/espfi_motion_native.c \
|
||||
scripts/csi_training/espfi_motion_native_probe.c \
|
||||
-lm -o "$RUNNER_TEMP/espfi-native-probe"
|
||||
"$RUNNER_TEMP/espfi-native-probe"
|
||||
|
||||
field-context-contract:
|
||||
runs-on: ubuntu-latest
|
||||
|
|
|
|||
|
|
@ -174,3 +174,23 @@ The [measured mixed report](../espfi-motion-mixed.md) passes the memory gate,
|
|||
with 37.2239% fewer retained array bytes, but fails the speed gate. Preserve
|
||||
the second-round median and third-round p95 regressions. No default switch,
|
||||
training, new accuracy claim, final test reopening or deployment is implied.
|
||||
|
||||
## Explicit native reduction boundary, 2026-10-13
|
||||
|
||||
The reduced NumPy candidate preserves numerical parity but fails all declared
|
||||
speed gates. Preserve that outcome and evaluate the native followup separately.
|
||||
Keep amplitude flooring, logs and mean centering in the original NumPy float64
|
||||
path. A fixed C ABI may compute only the remaining centered reductions.
|
||||
|
||||
Native loading is explicit trusted code execution. Require a separately supplied
|
||||
library path and SHA-256, bounded regular-file reads, private exact-byte loading,
|
||||
strict contiguous float64 buffers and fixed dimensions. Model metadata cannot
|
||||
select code. Compile reviewed source explicitly without fast math or contraction;
|
||||
bind source, compiler, flags, build receipt and library hashes to measurements.
|
||||
Preserve the existing model loader, instance lock and owned output API.
|
||||
|
||||
Require <=1e-10 absolute logits, exact predictions on every admitted record,
|
||||
passing sanitizer/boundary tests and >=10% median and p95 improvement in every
|
||||
paired timing round. The [native report](../espfi-motion-native.md) passes these
|
||||
research gates. Keep the NumPy default and rollback path; target hardware,
|
||||
independent accuracy, mixed precision composition and deployment remain separate.
|
||||
|
|
|
|||
|
|
@ -540,3 +540,21 @@ the separate 10% speed gate, so this is retained only as an explicit memory
|
|||
option. Float64 remains the default. All 99 relevant Python tests pass.
|
||||
The unchanged weights, reused data and scoped array counts do not establish
|
||||
new accuracy, target device speed or total process memory improvement.
|
||||
|
||||
## Native centered reductions, 2026-10-13
|
||||
|
||||
The [NumPy reduction experiment](espfi-motion-reduced.md) preserves all 907
|
||||
admitted classes but misses the fixed speed gate in every round. Its source
|
||||
and negative measurement remain available for reproduction.
|
||||
|
||||
The distinct [native followup](espfi-motion-native.md) keeps amplitude floor,
|
||||
log and centering in NumPy float64, then computes only centered reductions in
|
||||
an explicit trusted C kernel. All three rounds improve median by 29.01% to
|
||||
30.57% and p95 by 25.78% to 29.81% versus buffered float64. Maximum logit error
|
||||
is 4.773960e-15 and every admitted prediction is unchanged.
|
||||
|
||||
All 113 relevant Python tests and the ruOS ASan/UBSan probe pass. The optional
|
||||
kernel adds a 15,736-byte compiled library, requires an explicit trusted hash
|
||||
and does not change model weights, the default runtime or measured accuracy.
|
||||
Its source, compiler recipe, library hash and both experimental receipts are
|
||||
recorded. Combining native execution with mixed precision remains unmeasured.
|
||||
|
|
|
|||
174
docs/espfi-motion-native.md
Normal file
174
docs/espfi-motion-native.md
Normal file
|
|
@ -0,0 +1,174 @@
|
|||
# Native centered-feature inference experiment
|
||||
|
||||
Protocol frozen on 2026-10-13 after the separately preserved NumPy reduction
|
||||
experiment missed every 10% speed gate. This is an adaptive implementation
|
||||
followup with unchanged numerical and timing thresholds, not an independent
|
||||
model-accuracy experiment.
|
||||
|
||||
## Boundary and hypothesis
|
||||
|
||||
The [NumPy experiment](espfi-motion-reduced.md) removes redundant work but
|
||||
retains multiple Python and NumPy dispatches. A small C kernel can fuse the
|
||||
remaining per-carrier reductions while retaining sensitive amplitude floor,
|
||||
logarithm and mean centering exactly in the existing NumPy float64 path.
|
||||
|
||||
The optional `NativeMotionBundle` inherits the existing model loader, bounded
|
||||
prediction API, float64 classifier, lock and owned output behavior. The C
|
||||
kernel receives only a centered 52 by 190 float64 window and writes 468
|
||||
float64 features. It computes unfloored carrier energies, global RMS, six
|
||||
normalized autocorrelations and three normalized mean absolute differences.
|
||||
The ABI fixes dimensions, checks pointers/lengths and rejects nonfinite input
|
||||
or output. There is no mutable native global state or heap allocation.
|
||||
|
||||
Compile repository-owned source explicitly with the system C compiler,
|
||||
optimization enabled, fast math disabled and floating point contraction
|
||||
disabled. Record exact source hash, compiler version, flags and library hash.
|
||||
Compilation is never automatic during model loading. The new runtime requires
|
||||
an explicit kernel path and expected SHA-256. Check bounded regular library
|
||||
bytes and load those exact bytes from a private temporary directory retained
|
||||
for the runtime lifetime. Model metadata cannot select an executable library.
|
||||
|
||||
Loading a shared library executes native code. The digest establishes code
|
||||
identity and integrity, not safety of arbitrary third-party code. Only use a
|
||||
trusted build of the reviewed repository source. Current validation targets
|
||||
Linux; it does not establish compatibility or performance on another system.
|
||||
|
||||
## Frozen acceptance
|
||||
|
||||
Require float64 output, maximum absolute logit error 1e-10 and identical
|
||||
classes against both retained buffered and original explicit references on
|
||||
all 837 training and 70 reused development records at batch sizes one and 32.
|
||||
The final test was opened once in the first pilot and is not reopened here.
|
||||
|
||||
Three fixed rounds each use 100 warmups and 1,000 alternating paired full
|
||||
prediction calls, rotating development inputs with seeds 42, 43 and 44.
|
||||
Both median and p95 must improve at least 10% in every round against the
|
||||
buffered float64 runtime. Include Python validation, NumPy preprocessing,
|
||||
native call overhead, feature kernel and classifier. Exclude compilation,
|
||||
library loading, model loading, acquisition, pooling and I/O.
|
||||
|
||||
Bind input, model, library, build receipt and source hashes before and after
|
||||
measurement. Reserve receipts exclusively, preserve failures, and do not
|
||||
repeat unchanged configurations to seek favorable timing. Test numerical
|
||||
edge cases, concurrency, output ownership, invalid inputs, library integrity,
|
||||
ABI mismatch and failed kernel status. Run an instrumented native bounds
|
||||
probe if the existing compiler supports it.
|
||||
|
||||
No production default, artifact, training split or weight changes. Retained
|
||||
array counts exclude the library, executable pages, native stack and Python
|
||||
overhead, so they do not prove total process memory improvement. Existing
|
||||
workspace retains transformed samples until overwritten.
|
||||
|
||||
Decision rule: retain an explicit native research option only if all parity,
|
||||
integrity and every-round speed gates pass. Next action is the same test on
|
||||
the intended device. Key risks are native memory safety, compiler-dependent
|
||||
rounding and unmeasured target latency. Acceptance requires the same numerical
|
||||
and speed thresholds with the prior NumPy runtime available for rollback.
|
||||
|
||||
## Measured outcome, 2026-10-13
|
||||
|
||||
Retain the explicit native research option. All three paired rounds pass
|
||||
both speed gates. Relative to the buffered float64 runtime, median latency
|
||||
falls 29.01% to 30.57% and p95 falls 25.78% to 29.81%.
|
||||
|
||||
| Round | Native p50 ms | Buffered p50 ms | Native p95 ms | Buffered p95 ms |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 1 | 0.270125 | 0.381550 | 0.405321 | 0.577428 |
|
||||
| 2 | 0.245240 | 0.353240 | 0.371675 | 0.525422 |
|
||||
| 3 | 0.265500 | 0.374005 | 0.402225 | 0.541913 |
|
||||
|
||||
Every admitted training and development class prediction is unchanged against
|
||||
both references at batch sizes one and 32. Maximum absolute logit error is
|
||||
4.773960e-15, below the unchanged 1e-10 limit. The shared ruOS host runs Linux
|
||||
x86_64, Python 3.12.3 and NumPy 2.4.4, with one configured OpenBLAS/OMP thread.
|
||||
This does not measure latency from physical sensor acquisition to a decision.
|
||||
|
||||
The GCC 13.3.0 build produces a 15,736-byte shared library with SHA-256
|
||||
`ae3da3fbc6d71dbdb9c605f1ac515d22e3fe108aee7b62ffd22d9751443626ad`.
|
||||
The unchanged compact weights remain 16,088 bytes. Model arrays remain
|
||||
487,480 bytes and workspace remains 163,736 bytes. Native stack, mapped
|
||||
library pages and temporary loading copies are additional costs outside those
|
||||
array counts. No total memory reduction is claimed. The preceding mixed
|
||||
precision option remains separate; this experiment does not combine their
|
||||
benefits or change a production default.
|
||||
|
||||
All 113 relevant Python tests pass on ruOS: 85 ESPFi tests and 28 original
|
||||
training tests. The eight focused native tests pass. The ASan and UBSan
|
||||
native probe passes on ruOS without disabling checks; CI now runs that probe.
|
||||
The local executor initially prevented LeakSanitizer from inspecting `/proc`,
|
||||
so the local attempt is not counted as passing sanitizer evidence. Python
|
||||
reports automatic temporary kernel directory cleanup at garbage collection
|
||||
as ResourceWarnings during tests; all test processes exit successfully.
|
||||
|
||||
Review found and fixed a benchmark intake issue before measurement: kernel
|
||||
and build receipt hashing must use bounded, nonblocking regular-file reads.
|
||||
Tests confirm FIFO rejection before model loading. The kernel path never
|
||||
comes from model metadata, and executable hashes remain integrity evidence
|
||||
rather than permission to load untrusted code.
|
||||
|
||||
The [reviewed aggregate](validation/espfi-motion-native-20261013.json) includes
|
||||
every round, gates, exact source/input/build hashes and validation scope.
|
||||
Full private receipt SHA-256:
|
||||
`cd9ac18713ca430b01fd0d2887f58b216a11ed78d344cb426047c3887acac556`.
|
||||
Raw signals, per-record results, native binaries and model weights remain
|
||||
outside Git. The failed NumPy experiment is preserved without reinterpretation
|
||||
in [its own report](espfi-motion-reduced.md).
|
||||
|
||||
## Build and reproduce
|
||||
|
||||
Build the reviewed source explicitly on Linux into a new directory:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=scripts python3 -m csi_training.espfi_motion_native_build \
|
||||
--output /path/to/new-kernel-build
|
||||
```
|
||||
|
||||
Record and review `build.json`, including the compiler, flags and library
|
||||
digest. Pass its `library_sha256` value explicitly to a fresh benchmark:
|
||||
|
||||
```bash
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
|
||||
python3 -m csi_training.espfi_motion_native_benchmark \
|
||||
--manifest /path/to/espfi-prepared/manifest.json \
|
||||
--model /path/to/espfi-motion-compact-001 \
|
||||
--baseline /path/to/espfi-motion-002-worst \
|
||||
--kernel /path/to/new-kernel-build/kernel.so \
|
||||
--kernel-sha256 REVIEWED_LIBRARY_SHA256 \
|
||||
--build-receipt /path/to/new-kernel-build/build.json \
|
||||
--output /path/to/new-native-receipt.json
|
||||
```
|
||||
|
||||
For explicit application use after trusting that local build:
|
||||
|
||||
```python
|
||||
from csi_training.espfi_motion_native import NativeMotionBundle
|
||||
|
||||
runtime = NativeMotionBundle(
|
||||
"/path/to/espfi-motion-compact-001",
|
||||
"/path/to/new-kernel-build/kernel.so",
|
||||
"REVIEWED_LIBRARY_SHA256",
|
||||
)
|
||||
logits = runtime.predict(amplitudes) # [N, 52, 190] to owned [N, 7] float64
|
||||
```
|
||||
|
||||
Validation commands from the repository root:
|
||||
|
||||
```bash
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
|
||||
python3 -m unittest discover -s scripts/tests -p 'test_espfi*.py' -q
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
|
||||
python3 -m unittest discover -s scripts/tests -p 'test_csi_training*.py' -q
|
||||
cc -std=c11 -O1 -g -fno-fast-math -ffp-contract=off \
|
||||
-fsanitize=address,undefined -fno-omit-frame-pointer \
|
||||
scripts/csi_training/espfi_motion_native.c \
|
||||
scripts/csi_training/espfi_motion_native_probe.c -lm -o /tmp/espfi-native-probe
|
||||
/tmp/espfi-native-probe
|
||||
```
|
||||
|
||||
Decision: retain the optional native path for the measured shared-CPU speed
|
||||
gain. Next action: build and validate on the intended device before adoption.
|
||||
Key risk: native code execution and compiler/platform differences. Acceptance:
|
||||
the same exact classes, <=1e-10 logits, passing sanitizer checks and >=10%
|
||||
median and p95 gain in every declared paired round on that device.
|
||||
Activity accuracy remains 39/70 on reused development data; no new accuracy
|
||||
or SOTA result follows from this inference optimization.
|
||||
106
docs/espfi-motion-reduced.md
Normal file
106
docs/espfi-motion-reduced.md
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
# Reduced-pass float64 motion inference
|
||||
|
||||
Protocol frozen on 2026-10-13 before real-data evaluation. This experiment
|
||||
changes execution of the retained seven-class activity classifier. It does not
|
||||
train a model, alter weights, reopen final test data or authorize deployment.
|
||||
|
||||
## Hypothesis
|
||||
|
||||
A synthetic diagnostic profile of 1,000 predictions on the ruOS x86_64 host
|
||||
attributes 0.374 of 0.449 seconds to feature extraction, approximately 83%.
|
||||
Profiling includes instrumentation and synthetic inputs, so these values are
|
||||
diagnostic costs rather than deployment latency evidence.
|
||||
|
||||
The buffered runtime computes a squared window, reduces it once for global
|
||||
RMS and again for per-carrier energy, then normalizes the entire centered
|
||||
window before computing three mean absolute differences. The experiment
|
||||
reuses the per-carrier energy reduction for global RMS and moves the common
|
||||
normalization after the three difference means.
|
||||
|
||||
The optional `ReducedMotionBundle` inherits validated artifact loading,
|
||||
bounded inputs, float64 classifier, locking and owned outputs from
|
||||
`BufferedMotionBundle`. Override only feature extraction:
|
||||
|
||||
1. Preserve explicit float64 amplitude flooring, log and per-carrier centering.
|
||||
2. Compute each carrier's squared energy with a direct dot reduction.
|
||||
3. Derive global RMS from the sum of those energies before applying the energy
|
||||
floor. Preserve the existing RMS and energy floors.
|
||||
4. Keep the six centered autocorrelation contractions and their normalization.
|
||||
5. Compute absolute differences on unnormalized centered values, reduce each
|
||||
lag to its mean, then divide those means by the common global RMS.
|
||||
|
||||
These reorderings are algebraically equivalent, but floating point results
|
||||
need not be bit identical. Preserve the existing absolute logit tolerance of
|
||||
1e-10. Never substitute a lower precision contract for this experiment.
|
||||
No native compiler or new runtime dependency is required.
|
||||
|
||||
## Frozen gates
|
||||
|
||||
Require identical predicted classes and at most 1e-10 absolute logit error
|
||||
against both the retained buffered float64 and original explicit references.
|
||||
Exercise all 837 admitted training and 70 reused development recordings at
|
||||
batch sizes one and 32. The original final test was opened once in the first
|
||||
pilot and is not reopened. This is numerical equivalence evidence on reused
|
||||
data, not independent accuracy confirmation.
|
||||
|
||||
Measure three rounds of 1,000 rotating and alternating paired full prediction
|
||||
calls, with 100 warmups for each runtime in each round. Use seeds 42, 43 and
|
||||
44. The primary timing reference is `BufferedMotionBundle`. Both median and
|
||||
p95 must improve by at least 10% in every round before retaining a speed
|
||||
improvement. Include input checks, features and classifier; exclude loading,
|
||||
acquisition, pooling and I/O. Do not repeat unchanged experiments to search
|
||||
for favorable timing.
|
||||
|
||||
Freeze source and artifact hashes, reserve a new receipt exclusively, record
|
||||
explicit failure, and reject any source or input change during measurement.
|
||||
Report retained model and workspace arrays separately from process and peak
|
||||
memory. Scratch retains transformed values until reuse; it is not erasure.
|
||||
|
||||
Test zero and mixed-floor inputs, very large amplitudes, nearly constant
|
||||
inputs, strided arrays, integer inputs, concurrency, output ownership and
|
||||
failure recovery. A third-round timing regression must reject the runtime
|
||||
even when earlier rounds pass. Preserve the previous runtime for rollback.
|
||||
|
||||
Decision rule: retain the optional runtime only if numerical and every-round
|
||||
speed gates pass. Next acceptance is the same comparison on the intended
|
||||
device and representative admitted inputs. The key risk is rounding near
|
||||
constant signals, so edge tests and real-record parity are both mandatory.
|
||||
|
||||
## Measured outcome
|
||||
|
||||
Reject this speed candidate. All 907 predictions match both references at
|
||||
batch sizes one and 32, with maximum absolute logit error 3.108625e-15.
|
||||
However, every round fails the frozen 10% median and p95 improvement gate.
|
||||
Do not switch the default or repeat this unchanged configuration for a more
|
||||
favorable result.
|
||||
|
||||
| Round | Reduced p50 ms | Buffered p50 ms | Reduced p95 ms | Buffered p95 ms |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 1 | 0.349145 | 0.363140 | 0.503569 | 0.527381 |
|
||||
| 2 | 0.363535 | 0.387440 | 0.528979 | 0.553136 |
|
||||
| 3 | 0.338075 | 0.361850 | 0.500688 | 0.520044 |
|
||||
|
||||
Both implementations retain 487,480 model-array bytes and 163,736 workspace
|
||||
bytes. Six focused tests pass with verified exit zero. This experiment does
|
||||
not improve model accuracy or establish a retained speed optimization.
|
||||
|
||||
The [reviewed aggregate](validation/espfi-motion-reduced-20261013.json) records
|
||||
all rounds and source/input hashes. Full receipt SHA-256:
|
||||
`e6d80614b66a599093e1e676f3094dca39a06d4d52a22a609c596c41b29fcf9b`.
|
||||
The full receipt and capture provenance remain outside Git.
|
||||
|
||||
Reproduce on admitted local artifacts with a new output path:
|
||||
|
||||
```bash
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
|
||||
python3 -m csi_training.espfi_motion_reduced_benchmark \
|
||||
--manifest /path/to/espfi-prepared/manifest.json \
|
||||
--model /path/to/espfi-motion-compact-001 \
|
||||
--baseline /path/to/espfi-motion-002-worst \
|
||||
--output /path/to/new-reduced-receipt.json
|
||||
```
|
||||
|
||||
A distinct native followup keeps NumPy floor/log/centering unchanged and
|
||||
moves only the remaining reductions to a compiled kernel. Its protocol and
|
||||
result are recorded separately; no threshold is relaxed to accommodate this
|
||||
failed experiment.
|
||||
221
docs/validation/espfi-motion-native-20261013.json
Normal file
221
docs/validation/espfi-motion-native-20261013.json
Normal file
|
|
@ -0,0 +1,221 @@
|
|||
{
|
||||
"status": "complete",
|
||||
"parity": {
|
||||
"train": {
|
||||
"samples": 837,
|
||||
"batch_sizes": [
|
||||
1,
|
||||
32
|
||||
],
|
||||
"references": {
|
||||
"buffered_reference": {
|
||||
"max_abs_logit_error": 4.218847493575595e-15,
|
||||
"identical_predictions": true
|
||||
},
|
||||
"original": {
|
||||
"max_abs_logit_error": 4.773959005888173e-15,
|
||||
"identical_predictions": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"validation": {
|
||||
"samples": 70,
|
||||
"batch_sizes": [
|
||||
1,
|
||||
32
|
||||
],
|
||||
"references": {
|
||||
"buffered_reference": {
|
||||
"max_abs_logit_error": 2.6645352591003757e-15,
|
||||
"identical_predictions": true
|
||||
},
|
||||
"original": {
|
||||
"max_abs_logit_error": 2.4980018054066022e-15,
|
||||
"identical_predictions": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"rounds": [
|
||||
{
|
||||
"native": {
|
||||
"min_ms": 0.21051,
|
||||
"p50_ms": 0.270125,
|
||||
"p95_ms": 0.405321,
|
||||
"derived_calls_per_second": 3501.0865902319724
|
||||
},
|
||||
"buffered_reference": {
|
||||
"min_ms": 0.29864,
|
||||
"p50_ms": 0.38155,
|
||||
"p95_ms": 0.5774275,
|
||||
"derived_calls_per_second": 2460.625816180356
|
||||
},
|
||||
"round": 1,
|
||||
"seed": 42,
|
||||
"gain_at_least_10percent": true
|
||||
},
|
||||
{
|
||||
"native": {
|
||||
"min_ms": 0.21061,
|
||||
"p50_ms": 0.24524,
|
||||
"p95_ms": 0.3716755,
|
||||
"derived_calls_per_second": 3793.373359871014
|
||||
},
|
||||
"buffered_reference": {
|
||||
"min_ms": 0.29499,
|
||||
"p50_ms": 0.3532395,
|
||||
"p95_ms": 0.5254215,
|
||||
"derived_calls_per_second": 2670.4977484940014
|
||||
},
|
||||
"round": 2,
|
||||
"seed": 43,
|
||||
"gain_at_least_10percent": true
|
||||
},
|
||||
{
|
||||
"native": {
|
||||
"min_ms": 0.21096,
|
||||
"p50_ms": 0.2655,
|
||||
"p95_ms": 0.402225,
|
||||
"derived_calls_per_second": 3535.2220643729856
|
||||
},
|
||||
"buffered_reference": {
|
||||
"min_ms": 0.29712,
|
||||
"p50_ms": 0.37400500000000003,
|
||||
"p95_ms": 0.5419134999999999,
|
||||
"derived_calls_per_second": 2532.7907308734148
|
||||
},
|
||||
"round": 3,
|
||||
"seed": 44,
|
||||
"gain_at_least_10percent": true
|
||||
}
|
||||
],
|
||||
"retained_runtime": true,
|
||||
"memory_bytes": {
|
||||
"native_model_arrays": 487480,
|
||||
"native_workspace": 163736,
|
||||
"reference_model_arrays": 487480,
|
||||
"reference_workspace": 163736
|
||||
},
|
||||
"receipt_sha256": "cd9ac18713ca430b01fd0d2887f58b216a11ed78d344cb426047c3887acac556",
|
||||
"gains_percent": [
|
||||
{
|
||||
"round": 1,
|
||||
"p50": 29.203249901716678,
|
||||
"p95": 29.805733187283256
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"p50": 30.574015646608032,
|
||||
"p95": 29.261459609094796
|
||||
},
|
||||
{
|
||||
"round": 3,
|
||||
"p50": 29.011644229355227,
|
||||
"p95": 25.776899818882526
|
||||
}
|
||||
],
|
||||
"hashes": {
|
||||
"manifest": "74e0fe039acbed1b287bd11ee6b779becbb8f9b66c3c249d260ed66f5583f02a",
|
||||
"compact_artifact": "87765a4da415465e3a4c5415b199da4593ef173e8fc0a2398dbe67faa454374d",
|
||||
"compact_metadata": "e80ceb9d6a68df18c35eb624fcadbfa9b715869998ee3f2fb39b504424df2372",
|
||||
"compact_status": "a437f3cb1b87019ab0af9cae510de8f811bdd6b12c9d3bcdce2e3482934a051d",
|
||||
"original_artifact": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
|
||||
"original_receipt": "f8a565cb031d3ce9b488432dfb97408a58cee8f511c0ddc89aa74edb90388b88",
|
||||
"original_status": "a437f3cb1b87019ab0af9cae510de8f811bdd6b12c9d3bcdce2e3482934a051d",
|
||||
"kernel": "ae3da3fbc6d71dbdb9c605f1ac515d22e3fe108aee7b62ffd22d9751443626ad",
|
||||
"build_receipt": "24eaa46d21ce411962313084fe9b3e553e5802048e4dcb3694dc2c4c56699bd3"
|
||||
},
|
||||
"source_sha256": {
|
||||
"espfi_motion_native_benchmark.py": "7c232af2cce2549c70c3a1c44cd8682fed3da95c370e4e710ab9794ac3637f60",
|
||||
"espfi_motion_native.py": "b1d9a7de11f5cf4e797e24bb3f83e4a164f4360a14f49bc2645bd114c946305b",
|
||||
"espfi_motion_native.c": "720523248507bd349788d58083181c5c1df1cf40b37fe92e9245635ca70e20ac",
|
||||
"espfi_motion_native_build.py": "2a2d1c5ac2a7daa88b1be85904d26d29b08201cc53ec2054ee622869a6a5c678",
|
||||
"espfi_motion_runtime.py": "1a468974a441045a852215529d73e3eaddf40c2c81842465b5952d0322490add",
|
||||
"espfi_motion_compact.py": "74b13890afd33fbdf40efce3e91f44ddeb505d0f6d84f2d9a0b03adcd92c6f52",
|
||||
"espfi_motion.py": "c6a8529ccf9d9f834383a2bdf5b14626764ec7ca2ade7ad5d4c1c14797d76054",
|
||||
"espfi_kernel.py": "ae837cb2e3fac7f052b4bd371cd496e9636e5d5dbaf3a341526a4fc5aef23478",
|
||||
"espfi_generalize.py": "956b3a5e918f7c263a2b9183b4a6a70c8f2157604262a4ec6cee2c324a69224f",
|
||||
"espfi_train.py": "f3692319b383c67d092776a7690b72963061efeaa3516f2d5e41b5dac1cfbd1e",
|
||||
"espfi.py": "1ea4f72b3bbb0fe68e9903e6f32663a887bd9c8eac3a659932ff97b7e3d87124"
|
||||
},
|
||||
"environment": {
|
||||
"python": "3.12.3",
|
||||
"numpy": "2.4.4",
|
||||
"platform": "Linux-6.12.105-fly-x86_64-with-glibc2.39",
|
||||
"machine": "x86_64",
|
||||
"thread_environment": {
|
||||
"OPENBLAS_NUM_THREADS": "1",
|
||||
"OMP_NUM_THREADS": "1",
|
||||
"MKL_NUM_THREADS": null
|
||||
}
|
||||
},
|
||||
"protocol": {
|
||||
"rounds": 3,
|
||||
"calls_per_runtime_per_round": 1000,
|
||||
"warmups_per_runtime_per_round": 100,
|
||||
"seed": 42,
|
||||
"batch_size": 1,
|
||||
"alternating_runtime_order": true,
|
||||
"output_dtype": "float64",
|
||||
"max_abs_logit_error": 1e-10,
|
||||
"exact_argmax_required": true,
|
||||
"max_p50_and_p95_ratio_every_round": 0.9,
|
||||
"rotating_seeded_development_order": true,
|
||||
"primary_reference": "BufferedMotionBundle retained float64",
|
||||
"original_reference": "parity only"
|
||||
},
|
||||
"elapsed_seconds": 4.57653488699998,
|
||||
"native_build": {
|
||||
"status": "complete",
|
||||
"promotionAuthorized": false,
|
||||
"abi": 1,
|
||||
"compiler": "/usr/bin/cc",
|
||||
"compiler_version": "cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0\nCopyright (C) 2023 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions. There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n",
|
||||
"flags": [
|
||||
"-std=c11",
|
||||
"-O3",
|
||||
"-fPIC",
|
||||
"-shared",
|
||||
"-fno-fast-math",
|
||||
"-ffp-contract=off",
|
||||
"-lm"
|
||||
],
|
||||
"source_sha256": "720523248507bd349788d58083181c5c1df1cf40b37fe92e9245635ca70e20ac",
|
||||
"library_sha256": "ae3da3fbc6d71dbdb9c605f1ac515d22e3fe108aee7b62ffd22d9751443626ad",
|
||||
"library_bytes": 15736,
|
||||
"build_implementation_sha256": "2a2d1c5ac2a7daa88b1be85904d26d29b08201cc53ec2054ee622869a6a5c678"
|
||||
},
|
||||
"reviewed_on": "2026-10-13",
|
||||
"evidence": "MEASURED_AGGREGATE",
|
||||
"promotionAuthorized": false,
|
||||
"float64_default_unchanged": true,
|
||||
"decision": "Retain explicit trusted native research option; unchanged weights and numerical contract. Failed NumPy reduction experiment preserved separately.",
|
||||
"tests": {
|
||||
"espfi": 85,
|
||||
"csi_training": 28,
|
||||
"total": 113,
|
||||
"focused_native": 8,
|
||||
"focused_reduction": 6,
|
||||
"verified_exit_code": 0,
|
||||
"sanitizers": {
|
||||
"host": "ruOS",
|
||||
"address": true,
|
||||
"undefined_behavior": true,
|
||||
"checks_disabled": false,
|
||||
"exit_code": 0,
|
||||
"probe_source_sha256": "a44bd4f70e4ce59b189932ac5f57a0c9e77782c62e4737cd9a346b63701ee484",
|
||||
"local_note": "Local LeakSanitizer could not inspect proc under the local executor. The separate ruOS sanitizer run passed without disabling checks."
|
||||
},
|
||||
"source_sha256": {
|
||||
"scripts/csi_training/espfi_motion_native.c": "720523248507bd349788d58083181c5c1df1cf40b37fe92e9245635ca70e20ac",
|
||||
"scripts/csi_training/espfi_motion_native.py": "b1d9a7de11f5cf4e797e24bb3f83e4a164f4360a14f49bc2645bd114c946305b",
|
||||
"scripts/csi_training/espfi_motion_native_build.py": "2a2d1c5ac2a7daa88b1be85904d26d29b08201cc53ec2054ee622869a6a5c678",
|
||||
"scripts/csi_training/espfi_motion_native_probe.c": "a44bd4f70e4ce59b189932ac5f57a0c9e77782c62e4737cd9a346b63701ee484",
|
||||
"scripts/csi_training/espfi_motion_native_benchmark.py": "7c232af2cce2549c70c3a1c44cd8682fed3da95c370e4e710ab9794ac3637f60",
|
||||
"scripts/tests/test_espfi_motion_native.py": "9b8b8dafc8bf18b74a28b26b3b4a1a211af489855e612136161df56f8f4111d2",
|
||||
"scripts/tests/test_espfi_motion_native_benchmark.py": "1c828ecb682b1940deb3ea09e7fae16e327d0932031e9ec2d469472a0fd8b2b9"
|
||||
},
|
||||
"notes": "Python ResourceWarnings report automatic temporary kernel directory cleanup on garbage collection; tests complete successfully."
|
||||
},
|
||||
"scope": "Full batch-one Python prediction versus BufferedMotionBundle float64, excluding compilation/loading/acquisition/pooling/IO. No mixed-precision composition measured. Retained NumPy array counts exclude native stack and mapped library. Reused development data is numerical evidence only; original final test was opened once in first pilot and not reopened. No new model accuracy, SOTA, target-device validation or deployment."
|
||||
}
|
||||
162
docs/validation/espfi-motion-reduced-20261013.json
Normal file
162
docs/validation/espfi-motion-reduced-20261013.json
Normal file
|
|
@ -0,0 +1,162 @@
|
|||
{
|
||||
"status": "complete",
|
||||
"parity": {
|
||||
"train": {
|
||||
"samples": 837,
|
||||
"batch_sizes": [
|
||||
1,
|
||||
32
|
||||
],
|
||||
"references": {
|
||||
"buffered_reference": {
|
||||
"max_abs_logit_error": 2.4424906541753444e-15,
|
||||
"identical_predictions": true
|
||||
},
|
||||
"original": {
|
||||
"max_abs_logit_error": 3.1086244689504383e-15,
|
||||
"identical_predictions": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"validation": {
|
||||
"samples": 70,
|
||||
"batch_sizes": [
|
||||
1,
|
||||
32
|
||||
],
|
||||
"references": {
|
||||
"buffered_reference": {
|
||||
"max_abs_logit_error": 1.7763568394002505e-15,
|
||||
"identical_predictions": true
|
||||
},
|
||||
"original": {
|
||||
"max_abs_logit_error": 2.42861286636753e-15,
|
||||
"identical_predictions": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"rounds": [
|
||||
{
|
||||
"reduced": {
|
||||
"min_ms": 0.28269,
|
||||
"p50_ms": 0.34914500000000004,
|
||||
"p95_ms": 0.503569,
|
||||
"derived_calls_per_second": 2708.965457442368
|
||||
},
|
||||
"buffered_reference": {
|
||||
"min_ms": 0.29914,
|
||||
"p50_ms": 0.36314,
|
||||
"p95_ms": 0.5273805,
|
||||
"derived_calls_per_second": 2596.825879472279
|
||||
},
|
||||
"round": 1,
|
||||
"seed": 42,
|
||||
"gain_at_least_10percent": false
|
||||
},
|
||||
{
|
||||
"reduced": {
|
||||
"min_ms": 0.28326,
|
||||
"p50_ms": 0.363535,
|
||||
"p95_ms": 0.5289794999999999,
|
||||
"derived_calls_per_second": 2597.7871508709154
|
||||
},
|
||||
"buffered_reference": {
|
||||
"min_ms": 0.29552,
|
||||
"p50_ms": 0.38744,
|
||||
"p95_ms": 0.5531364999999999,
|
||||
"derived_calls_per_second": 2477.1622055844723
|
||||
},
|
||||
"round": 2,
|
||||
"seed": 43,
|
||||
"gain_at_least_10percent": false
|
||||
},
|
||||
{
|
||||
"reduced": {
|
||||
"min_ms": 0.27682,
|
||||
"p50_ms": 0.3380745,
|
||||
"p95_ms": 0.5006879999999999,
|
||||
"derived_calls_per_second": 2774.0855546682396
|
||||
},
|
||||
"buffered_reference": {
|
||||
"min_ms": 0.29146,
|
||||
"p50_ms": 0.36185,
|
||||
"p95_ms": 0.520044,
|
||||
"derived_calls_per_second": 2617.8406847919396
|
||||
},
|
||||
"round": 3,
|
||||
"seed": 44,
|
||||
"gain_at_least_10percent": false
|
||||
}
|
||||
],
|
||||
"retained_runtime": false,
|
||||
"memory_bytes": {
|
||||
"reduced_model_arrays": 487480,
|
||||
"reduced_workspace": 163736,
|
||||
"reference_model_arrays": 487480,
|
||||
"reference_workspace": 163736
|
||||
},
|
||||
"receipt_sha256": "e6d80614b66a599093e1e676f3094dca39a06d4d52a22a609c596c41b29fcf9b",
|
||||
"hashes": {
|
||||
"manifest": "74e0fe039acbed1b287bd11ee6b779becbb8f9b66c3c249d260ed66f5583f02a",
|
||||
"compact_artifact": "87765a4da415465e3a4c5415b199da4593ef173e8fc0a2398dbe67faa454374d",
|
||||
"compact_metadata": "e80ceb9d6a68df18c35eb624fcadbfa9b715869998ee3f2fb39b504424df2372",
|
||||
"compact_status": "a437f3cb1b87019ab0af9cae510de8f811bdd6b12c9d3bcdce2e3482934a051d",
|
||||
"original_artifact": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
|
||||
"original_receipt": "f8a565cb031d3ce9b488432dfb97408a58cee8f511c0ddc89aa74edb90388b88",
|
||||
"original_status": "a437f3cb1b87019ab0af9cae510de8f811bdd6b12c9d3bcdce2e3482934a051d"
|
||||
},
|
||||
"source_sha256": {
|
||||
"espfi_motion_reduced_benchmark.py": "37eb030ec987bd1375f177f3226a07080fe869ed3cdc4193ce6b80a1e480df7e",
|
||||
"espfi_motion_reduced.py": "60b18e2c8f284860676ec36080ff559117689884959593b5c0ded4fc9fc9c476",
|
||||
"espfi_motion_runtime.py": "1a468974a441045a852215529d73e3eaddf40c2c81842465b5952d0322490add",
|
||||
"espfi_motion_compact.py": "74b13890afd33fbdf40efce3e91f44ddeb505d0f6d84f2d9a0b03adcd92c6f52",
|
||||
"espfi_motion.py": "c6a8529ccf9d9f834383a2bdf5b14626764ec7ca2ade7ad5d4c1c14797d76054",
|
||||
"espfi_kernel.py": "ae837cb2e3fac7f052b4bd371cd496e9636e5d5dbaf3a341526a4fc5aef23478",
|
||||
"espfi_generalize.py": "956b3a5e918f7c263a2b9183b4a6a70c8f2157604262a4ec6cee2c324a69224f",
|
||||
"espfi_train.py": "f3692319b383c67d092776a7690b72963061efeaa3516f2d5e41b5dac1cfbd1e",
|
||||
"espfi.py": "1ea4f72b3bbb0fe68e9903e6f32663a887bd9c8eac3a659932ff97b7e3d87124"
|
||||
},
|
||||
"environment": {
|
||||
"python": "3.12.3",
|
||||
"numpy": "2.4.4",
|
||||
"platform": "Linux-6.12.105-fly-x86_64-with-glibc2.39",
|
||||
"machine": "x86_64",
|
||||
"thread_environment": {
|
||||
"OPENBLAS_NUM_THREADS": "1",
|
||||
"OMP_NUM_THREADS": "1",
|
||||
"MKL_NUM_THREADS": null
|
||||
}
|
||||
},
|
||||
"protocol": {
|
||||
"rounds": 3,
|
||||
"calls_per_runtime_per_round": 1000,
|
||||
"warmups_per_runtime_per_round": 100,
|
||||
"seed": 42,
|
||||
"batch_size": 1,
|
||||
"alternating_runtime_order": true,
|
||||
"output_dtype": "float64",
|
||||
"max_abs_logit_error": 1e-10,
|
||||
"exact_argmax_required": true,
|
||||
"max_p50_and_p95_ratio_every_round": 0.9,
|
||||
"rotating_seeded_development_order": true,
|
||||
"primary_reference": "BufferedMotionBundle retained float64",
|
||||
"original_reference": "parity only"
|
||||
},
|
||||
"elapsed_seconds": 5.288693015000035,
|
||||
"reviewed_on": "2026-10-13",
|
||||
"evidence": "MEASURED_AGGREGATE",
|
||||
"promotionAuthorized": false,
|
||||
"decision": "Rejected speed candidate; preserve exact source and negative outcome. No unchanged repeat.",
|
||||
"tests": {
|
||||
"focused_reduction": 6,
|
||||
"verified_exit_code": 0,
|
||||
"source_sha256": {
|
||||
"scripts/csi_training/espfi_motion_reduced.py": "60b18e2c8f284860676ec36080ff559117689884959593b5c0ded4fc9fc9c476",
|
||||
"scripts/csi_training/espfi_motion_reduced_benchmark.py": "37eb030ec987bd1375f177f3226a07080fe869ed3cdc4193ce6b80a1e480df7e",
|
||||
"scripts/tests/test_espfi_motion_reduced.py": "6070b1e703c882ff2c4f47df0ef90ca0fe8e42788102e7efee5e5a54a16ed482",
|
||||
"scripts/tests/test_espfi_motion_reduced_benchmark.py": "7b9aa104e41f76c7fa0605ff02012ae19f4113f4ad445f6338222d210e0a38dd"
|
||||
}
|
||||
},
|
||||
"scope": "Reordered float64 reductions preserve admitted classes but miss frozen10% median/p95 speed gate in every round. Same artifact and retained array bytes. Original final test was opened once in first pilot, not reopened here. Reused development evidence is not independent accuracy confirmation."
|
||||
}
|
||||
46
scripts/csi_training/espfi_motion_native.c
Normal file
46
scripts/csi_training/espfi_motion_native.c
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
/* Fixed-shape centered ACF reductions. No input mutation, allocation or state. */
|
||||
#include <stddef.h>
|
||||
#include <math.h>
|
||||
|
||||
unsigned int espfi_motion_abi(void) { return 1u; }
|
||||
|
||||
int espfi_motion_features(const double *x, size_t input_count,
|
||||
double *out, size_t output_count) {
|
||||
static const size_t acf_lags[6] = {1,2,4,8,16,32};
|
||||
static const size_t diff_lags[3] = {1,4,16};
|
||||
double energy[52];
|
||||
double total = 0.0;
|
||||
if (!x || !out || input_count != 9880 || output_count != 468) return 1;
|
||||
for (size_t c=0;c<52;c++) {
|
||||
double sum=0.0;
|
||||
for (size_t t=0;t<190;t++) {
|
||||
double v=x[c*190+t];
|
||||
if (!isfinite(v)) return 2;
|
||||
sum += v*v;
|
||||
}
|
||||
if (!isfinite(sum)) return 2;
|
||||
energy[c]=sum;
|
||||
total += sum;
|
||||
}
|
||||
double scale=sqrt(total/9880.0);
|
||||
if (!isfinite(scale)) return 2;
|
||||
if (scale<1e-6) scale=1e-6;
|
||||
for (size_t c=0;c<52;c++) {
|
||||
const double *row=x+c*190;
|
||||
double denominator=energy[c]>190e-12?energy[c]:190e-12;
|
||||
for (size_t k=0;k<6;k++) {
|
||||
size_t lag=acf_lags[k];
|
||||
double sum=0.0;
|
||||
for (size_t t=lag;t<190;t++) sum += row[t]*row[t-lag];
|
||||
out[c*9+k]=sum/denominator;
|
||||
}
|
||||
for (size_t k=0;k<3;k++) {
|
||||
size_t lag=diff_lags[k];
|
||||
double sum=0.0;
|
||||
for (size_t t=lag;t<190;t++) sum += fabs(row[t]-row[t-lag]);
|
||||
out[c*9+6+k]=(sum/(double)(190-lag))/scale;
|
||||
}
|
||||
}
|
||||
for (size_t i=0;i<468;i++) if (!isfinite(out[i])) return 2;
|
||||
return 0;
|
||||
}
|
||||
68
scripts/csi_training/espfi_motion_native.py
Normal file
68
scripts/csi_training/espfi_motion_native.py
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
"""Explicit trusted native reductions; never loaded through model metadata.
|
||||
|
||||
The caller must supply a separately trusted kernel path and SHA-256. A digest
|
||||
verifies bytes, not code safety. Native code executes with process authority.
|
||||
Only Linux local shared libraries are supported. Default runtime is unchanged.
|
||||
"""
|
||||
import ctypes
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
import sys
|
||||
import tempfile
|
||||
import numpy as np
|
||||
from .espfi_motion_runtime import BufferedMotionBundle
|
||||
|
||||
|
||||
def _verified_kernel(path,expected):
|
||||
if not isinstance(expected,str) or re.fullmatch(r'[0-9a-fA-F]{64}',expected) is None:
|
||||
raise ValueError('Explicit trusted kernel SHA-256 required')
|
||||
if sys.platform != 'linux':
|
||||
raise ValueError('Native kernel currently supports Linux only')
|
||||
# O_NOFOLLOW rejects leaf symlinks atomically; fstat validates the opened file.
|
||||
fd=os.open(Path(path),os.O_RDONLY|os.O_NOFOLLOW|os.O_NONBLOCK)
|
||||
try:
|
||||
info=os.fstat(fd)
|
||||
if not stat.S_ISREG(info.st_mode) or not 0<info.st_size<=16*1024**2:
|
||||
raise ValueError('Expected bounded regular kernel')
|
||||
with os.fdopen(fd,'rb',closefd=False) as handle:
|
||||
data=handle.read(16*1024**2+1)
|
||||
finally:
|
||||
os.close(fd)
|
||||
if len(data)>16*1024**2 or hashlib.sha256(data).hexdigest()!=expected.lower():
|
||||
raise ValueError('Kernel hash or size mismatch')
|
||||
return data
|
||||
|
||||
|
||||
class NativeMotionBundle(BufferedMotionBundle):
|
||||
"""Verified explicit kernel; inherited locked prediction and owned output."""
|
||||
def __init__(self,model_directory,kernel_path,expected_kernel_sha256):
|
||||
super().__init__(model_directory)
|
||||
data=_verified_kernel(kernel_path,expected_kernel_sha256)
|
||||
self._kernel_directory=tempfile.TemporaryDirectory(prefix='espfi-native-')
|
||||
try:
|
||||
private=Path(self._kernel_directory.name)/'kernel.so'
|
||||
with private.open('xb') as handle: handle.write(data)
|
||||
self._library=ctypes.CDLL(str(private))
|
||||
abi=self._library.espfi_motion_abi
|
||||
abi.argtypes=[];abi.restype=ctypes.c_uint
|
||||
if abi()!=1: raise ValueError('Unsupported native kernel ABI')
|
||||
self._native=self._library.espfi_motion_features
|
||||
self._native.argtypes=[np.ctypeslib.ndpointer(dtype=np.float64,ndim=2,shape=(52,190),flags='C_CONTIGUOUS'),
|
||||
ctypes.c_size_t,np.ctypeslib.ndpointer(dtype=np.float64,ndim=2,shape=(1,468),flags=('C_CONTIGUOUS','WRITEABLE')),
|
||||
ctypes.c_size_t]
|
||||
self._native.restype=ctypes.c_int
|
||||
except Exception:
|
||||
self._kernel_directory.cleanup()
|
||||
raise
|
||||
|
||||
def _extract(self,sample):
|
||||
np.maximum(sample,1e-6,out=self._centered,dtype=np.float64)
|
||||
np.log(self._centered,out=self._centered)
|
||||
self._centered.mean(axis=-1,keepdims=True,out=self._mean)
|
||||
self._centered -= self._mean
|
||||
if self._native(self._centered,9880,self._features,468)!=0:
|
||||
raise ValueError('Native centered reduction rejected input')
|
||||
return self._features
|
||||
206
scripts/csi_training/espfi_motion_native_benchmark.py
Normal file
206
scripts/csi_training/espfi_motion_native_benchmark.py
Normal file
|
|
@ -0,0 +1,206 @@
|
|||
"""Frozen development-only native inference benchmark; never trains or promotes."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import stat
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import time
|
||||
|
||||
import numpy as np
|
||||
from .espfi import load_development
|
||||
from .espfi_motion import MotionBundle, _read, _validate
|
||||
from .espfi_motion_compact import BASELINE
|
||||
from .espfi_train import digest
|
||||
|
||||
ROUNDS, CALLS, WARMUPS, SEED = 3, 1000, 100, 42
|
||||
SOURCES = ('espfi_motion_native_benchmark.py', 'espfi_motion_native.py', 'espfi_motion_native.c',
|
||||
'espfi_motion_native_build.py', 'espfi_motion_runtime.py',
|
||||
'espfi_motion_compact.py', 'espfi_motion.py', 'espfi_kernel.py',
|
||||
'espfi_generalize.py', 'espfi_train.py', 'espfi.py')
|
||||
|
||||
|
||||
BUILD_FLAGS = ['-std=c11', '-O3', '-fPIC', '-shared', '-fno-fast-math', '-ffp-contract=off', '-lm']
|
||||
|
||||
|
||||
def _build_bytes(path):
|
||||
fd = os.open(Path(path), os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
||||
try:
|
||||
info = os.fstat(fd)
|
||||
if not stat.S_ISREG(info.st_mode) or not 0 < info.st_size <= 65536:
|
||||
raise ValueError('Expected bounded regular build receipt')
|
||||
with os.fdopen(fd, 'rb', closefd=False) as handle:
|
||||
data = handle.read(65537)
|
||||
if len(data) > 65536:
|
||||
raise ValueError('Oversized build receipt')
|
||||
return data
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def _validate_build(build, hashes, source_hashes, expected_kernel_sha256):
|
||||
if (not isinstance(build, dict) or build.get('status') != 'complete'
|
||||
or build.get('promotionAuthorized') is not False or build.get('abi') != 1
|
||||
or build.get('flags') != BUILD_FLAGS
|
||||
or build.get('source_sha256') != source_hashes['espfi_motion_native.c']
|
||||
or build.get('build_implementation_sha256') != source_hashes['espfi_motion_native_build.py']
|
||||
or build.get('library_sha256') != hashes['kernel']
|
||||
or expected_kernel_sha256 != hashes['kernel']
|
||||
or not isinstance(build.get('compiler'), str) or not build['compiler']
|
||||
or not isinstance(build.get('compiler_version'), str) or not build['compiler_version']):
|
||||
raise ValueError('Native build provenance mismatch')
|
||||
|
||||
|
||||
def _summary(values):
|
||||
return {'min_ms': float(np.min(values)), 'p50_ms': float(np.percentile(values, 50)),
|
||||
'p95_ms': float(np.percentile(values, 95)),
|
||||
'derived_calls_per_second': float(1000 / np.mean(values))}
|
||||
|
||||
|
||||
def _measure(candidate, reference, dev):
|
||||
rounds = []
|
||||
for number in range(ROUNDS):
|
||||
order = np.random.default_rng(SEED + number).permutation(len(dev))
|
||||
calls = (('native', candidate.predict), ('buffered_reference', reference.predict))
|
||||
for i in range(WARMUPS):
|
||||
x = dev[order[i % len(order)]:order[i % len(order)] + 1]
|
||||
for _, fn in calls[::1 if (i + number) % 2 == 0 else -1]:
|
||||
fn(x)
|
||||
values = {name: [] for name, _ in calls}
|
||||
for i in range(CALLS):
|
||||
x = dev[order[i % len(order)]:order[i % len(order)] + 1]
|
||||
for name, fn in calls[::1 if (i + number) % 2 == 0 else -1]:
|
||||
before = time.perf_counter_ns()
|
||||
fn(x)
|
||||
values[name].append((time.perf_counter_ns() - before) / 1e6)
|
||||
result = {name: _summary(v) for name, v in values.items()}
|
||||
result['round'] = number + 1
|
||||
result['seed'] = SEED + number
|
||||
result['gain_at_least_10percent'] = all(
|
||||
result['native'][q] <= .9 * result['buffered_reference'][q]
|
||||
for q in ('p50_ms', 'p95_ms'))
|
||||
rounds.append(result)
|
||||
return rounds
|
||||
|
||||
|
||||
def _parity(parts, candidate, references):
|
||||
result = {}
|
||||
for split, part in parts.items():
|
||||
checks = {name: {'max_abs_logit_error': 0., 'identical_predictions': True}
|
||||
for name in references}
|
||||
# Exercise both the timed batch-one path and bounded batch iteration.
|
||||
for size in (1, 32):
|
||||
for offset in range(0, len(part['x']), size):
|
||||
x = part['x'][offset:offset + size]
|
||||
actual = candidate.predict(x)
|
||||
for name, reference in references.items():
|
||||
expected = reference.predict(x)
|
||||
if (actual.shape != expected.shape or actual.dtype != np.float64 or expected.dtype != np.float64
|
||||
or not np.isfinite(actual).all() or not np.isfinite(expected).all()):
|
||||
raise ValueError('Invalid native/reference scores')
|
||||
error = float(np.max(np.abs(actual - expected)))
|
||||
equal = bool(np.array_equal(actual.argmax(1), expected.argmax(1)))
|
||||
checks[name]['max_abs_logit_error'] = max(checks[name]['max_abs_logit_error'], error)
|
||||
checks[name]['identical_predictions'] &= equal
|
||||
if error > 1e-10 or not equal:
|
||||
raise ValueError('Native runtime parity gate failed')
|
||||
result[split] = {'samples': len(part['x']), 'batch_sizes': [1, 32], 'references': checks}
|
||||
return result
|
||||
|
||||
|
||||
def benchmark(manifest, model, baseline, output, kernel, kernel_sha256, build_receipt):
|
||||
"""Reserve a new receipt before loading inputs; failure remains explicit."""
|
||||
output, manifest, model, baseline = map(Path, (output, manifest, model, baseline))
|
||||
# Exclusive creation never overwrites an existing receipt, including symlinks.
|
||||
with output.open('x') as handle:
|
||||
json.dump({'status': 'running', 'promotionAuthorized': False}, handle)
|
||||
def record(value):
|
||||
with output.open('w') as handle:
|
||||
json.dump(value, handle, indent=2, allow_nan=False)
|
||||
handle.write('\n')
|
||||
started = time.monotonic()
|
||||
try:
|
||||
from .espfi_motion_runtime import BufferedMotionBundle
|
||||
from .espfi_motion_native import NativeMotionBundle, _verified_kernel
|
||||
kernel_bytes = _verified_kernel(kernel, kernel_sha256)
|
||||
kernel_sha256 = kernel_sha256.lower()
|
||||
receipt_bytes = _build_bytes(build_receipt)
|
||||
paths = {'manifest': manifest, 'compact_artifact': model/'model.safetensors',
|
||||
'compact_metadata': model/'metadata.json', 'compact_status': model/'status.json',
|
||||
'original_artifact': baseline/'model.safetensors',
|
||||
'original_receipt': baseline/'metrics.json', 'original_status': baseline/'status.json'}
|
||||
hashes = {name: digest(path) for name, path in paths.items()}
|
||||
hashes['kernel'] = hashlib.sha256(kernel_bytes).hexdigest()
|
||||
hashes['build_receipt'] = hashlib.sha256(receipt_bytes).hexdigest()
|
||||
source_hashes = {name: digest(Path(__file__).with_name(name)) for name in SOURCES}
|
||||
build = json.loads(receipt_bytes)
|
||||
_validate_build(build, hashes, source_hashes, kernel_sha256)
|
||||
if build.get('library_bytes') != len(kernel_bytes):
|
||||
raise ValueError('Native build size mismatch')
|
||||
parts, provenance = load_development(manifest)
|
||||
_validate(parts)
|
||||
metadata = json.loads(_read(model/'metadata.json', 65536))
|
||||
prior = json.loads(_read(baseline/'metrics.json', 16*1024**2))
|
||||
if (provenance.get('manifest_sha256') != hashes['manifest']
|
||||
or metadata.get('provenance', {}).get('manifest_sha256') != hashes['manifest']
|
||||
or prior.get('provenance', {}).get('manifest_sha256') != hashes['manifest']
|
||||
or metadata.get('baseline_model_sha256') != hashes['original_artifact']
|
||||
or metadata.get('baseline_receipt_sha256') != hashes['original_receipt']):
|
||||
raise ValueError('Input provenance mismatch')
|
||||
if provenance.get('evidence') != 'SYNTHETIC' and {k: len(v['x']) for k, v in parts.items()} != {'train': 837, 'validation': 70}:
|
||||
raise ValueError('Unexpected admitted development population')
|
||||
candidate = NativeMotionBundle(model, kernel, kernel_sha256)
|
||||
reference = BufferedMotionBundle(model)
|
||||
original = MotionBundle(baseline)
|
||||
if original.candidate != BASELINE:
|
||||
raise ValueError('Unexpected original candidate')
|
||||
parity = _parity(parts, candidate, {'buffered_reference': reference, 'original': original})
|
||||
rounds = _measure(candidate, reference, parts['validation']['x'])
|
||||
if (hashlib.sha256(_verified_kernel(kernel, kernel_sha256)).hexdigest() != hashes['kernel']
|
||||
or hashlib.sha256(_build_bytes(build_receipt)).hexdigest() != hashes['build_receipt']):
|
||||
raise ValueError('Native inputs changed during benchmark')
|
||||
if any(digest(path) != hashes[name] for name, path in paths.items()):
|
||||
raise ValueError('Inputs changed during benchmark')
|
||||
if any(digest(Path(__file__).with_name(name)) != value for name, value in source_hashes.items()):
|
||||
raise ValueError('Sources changed during benchmark')
|
||||
report = {'status': 'complete', 'promotionAuthorized': False,
|
||||
'evidence': 'SYNTHETIC' if provenance.get('evidence') == 'SYNTHETIC' else 'MEASURED',
|
||||
'task': 'native_motion_inference_benchmark', 'hashes': hashes,
|
||||
'source_sha256': source_hashes, 'native_build': build, 'provenance': provenance, 'parity': parity,
|
||||
'rounds': rounds, 'retained_runtime': all(r['gain_at_least_10percent'] for r in rounds),
|
||||
'protocol': {'rounds': ROUNDS, 'calls_per_runtime_per_round': CALLS,
|
||||
'warmups_per_runtime_per_round': WARMUPS, 'seed': SEED,
|
||||
'batch_size': 1, 'alternating_runtime_order': True,
|
||||
'output_dtype': 'float64', 'max_abs_logit_error': 1e-10,
|
||||
'exact_argmax_required': True,
|
||||
'max_p50_and_p95_ratio_every_round': .9,
|
||||
'rotating_seeded_development_order': True,
|
||||
'primary_reference': 'BufferedMotionBundle retained float64',
|
||||
'original_reference': 'parity only'},
|
||||
'memory_bytes': {'native_model_arrays': int(candidate.model_state_bytes),
|
||||
'native_workspace': int(candidate.workspace_bytes),
|
||||
'reference_model_arrays': int(reference.model_state_bytes),
|
||||
'reference_workspace': int(reference.workspace_bytes)},
|
||||
'environment': {'python': platform.python_version(), 'numpy': np.__version__,
|
||||
'platform': platform.platform(), 'machine': platform.machine(),
|
||||
'thread_environment': {k: os.environ.get(k) for k in ('OPENBLAS_NUM_THREADS', 'OMP_NUM_THREADS', 'MKL_NUM_THREADS')}},
|
||||
'elapsed_seconds': time.monotonic() - started,
|
||||
'scope': 'Optional native runtime; unchanged BufferedMotionBundle default. Full predict includes input checks, features and classifier. Excludes model and kernel load, compilation, pooling, acquisition and IO. Throughput is derived from mean batch-one latency, not raw CSI throughput. No training or final-test array access; reused development data, not independent accuracy confirmation.'}
|
||||
record(report)
|
||||
return report
|
||||
except Exception as exc:
|
||||
record({'status': 'failed', 'promotionAuthorized': False, 'error_type': type(exc).__name__})
|
||||
raise
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
for name in ('manifest', 'model', 'baseline', 'output', 'kernel', 'kernel-sha256', 'build-receipt'):
|
||||
parser.add_argument('--' + name, required=True)
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(benchmark(args.manifest, args.model, args.baseline, args.output, args.kernel, args.kernel_sha256, args.build_receipt), indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
48
scripts/csi_training/espfi_motion_native_build.py
Normal file
48
scripts/csi_training/espfi_motion_native_build.py
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
"""Explicit local C11 build; no downloads, model writes or implicit compilation."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
FLAGS=['-std=c11','-O3','-fPIC','-shared','-fno-fast-math','-ffp-contract=off']
|
||||
|
||||
|
||||
def build(output):
|
||||
if sys.platform!='linux': raise ValueError('Linux build required')
|
||||
compiler=shutil.which('cc')
|
||||
if compiler is None: raise RuntimeError('System cc is required; no compiler is downloaded')
|
||||
root=Path(output).resolve()
|
||||
root.mkdir(parents=True,exist_ok=False)
|
||||
def write(name,value): (root/name).write_text(json.dumps(value,indent=2)+'\n')
|
||||
write('status.json',{'status':'running','promotionAuthorized':False})
|
||||
try:
|
||||
source=Path(__file__).with_suffix('.c')
|
||||
source=source.with_name('espfi_motion_native.c')
|
||||
source_bytes=source.read_bytes()
|
||||
# Compile an exclusive snapshot so the receipt describes the actual input.
|
||||
snapshot=root/'kernel.c';snapshot.write_bytes(source_bytes)
|
||||
version=subprocess.run([compiler,'--version'],check=True,capture_output=True,text=True,timeout=30).stdout
|
||||
command=[compiler,*FLAGS,str(snapshot),'-o',str(root/'kernel.so'),'-lm']
|
||||
result=subprocess.run(command,check=True,capture_output=True,text=True,timeout=60)
|
||||
library=(root/'kernel.so').read_bytes()
|
||||
if not 0<len(library)<=16*1024**2: raise ValueError('Unexpected kernel size')
|
||||
report={'status':'complete','promotionAuthorized':False,'abi':1,'compiler':compiler,
|
||||
'compiler_version':version[:8192],'flags':FLAGS+['-lm'],
|
||||
'source_sha256':hashlib.sha256(source_bytes).hexdigest(),
|
||||
'library_sha256':hashlib.sha256(library).hexdigest(),
|
||||
'library_bytes':len(library),'build_implementation_sha256':hashlib.sha256(Path(__file__).read_bytes()).hexdigest()}
|
||||
write('build.json',report);write('status.json',{'status':'complete','promotionAuthorized':False})
|
||||
return report
|
||||
except Exception:
|
||||
write('status.json',{'status':'failed','promotionAuthorized':False})
|
||||
raise
|
||||
|
||||
|
||||
def main():
|
||||
parser=argparse.ArgumentParser(description=__doc__);parser.add_argument('--output',required=True)
|
||||
print(json.dumps(build(parser.parse_args().output),indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
48
scripts/csi_training/espfi_motion_native_probe.c
Normal file
48
scripts/csi_training/espfi_motion_native_probe.c
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
/* Standalone sanitizer probe. Synthetic arrays only; no model or sensing input. */
|
||||
#include <float.h>
|
||||
#include <math.h>
|
||||
#include <stddef.h>
|
||||
#include <stdio.h>
|
||||
|
||||
unsigned int espfi_motion_abi(void);
|
||||
int espfi_motion_features(const double *, size_t, double *, size_t);
|
||||
|
||||
#define CHECK(condition) do { if (!(condition)) { \
|
||||
fprintf(stderr, "native probe failed at line %d\n", __LINE__); return 1; \
|
||||
} } while (0)
|
||||
|
||||
int main(void) {
|
||||
double input[9880] = {0};
|
||||
double guarded[470] = {0};
|
||||
double *output = guarded + 1;
|
||||
guarded[0] = 12345.0;
|
||||
guarded[469] = -12345.0;
|
||||
CHECK(espfi_motion_abi() == 1u);
|
||||
CHECK(espfi_motion_features(NULL, 9880, output, 468) != 0);
|
||||
CHECK(espfi_motion_features(input, 9880, NULL, 468) != 0);
|
||||
CHECK(espfi_motion_features(input, 9879, output, 468) != 0);
|
||||
CHECK(espfi_motion_features(input, 9881, output, 468) != 0);
|
||||
CHECK(espfi_motion_features(input, (size_t)-1, output, 468) != 0);
|
||||
CHECK(espfi_motion_features(input, 9880, output, 467) != 0);
|
||||
CHECK(espfi_motion_features(input, 9880, output, 469) != 0);
|
||||
CHECK(espfi_motion_features(input, 9880, output, 468) == 0);
|
||||
for (size_t i = 0; i < 468; i++) CHECK(output[i] == 0.0);
|
||||
input[0] = NAN;
|
||||
CHECK(espfi_motion_features(input, 9880, output, 468) != 0);
|
||||
input[0] = INFINITY;
|
||||
CHECK(espfi_motion_features(input, 9880, output, 468) != 0);
|
||||
input[0] = DBL_MAX;
|
||||
CHECK(espfi_motion_features(input, 9880, output, 468) != 0);
|
||||
for (size_t i = 0; i < 9880; i++) input[i] = i % 2 ? 1e-6 : -1e-6;
|
||||
CHECK(espfi_motion_features(input, 9880, output, 468) == 0);
|
||||
for (size_t c = 0; c < 52; c++) {
|
||||
CHECK(fabs(output[c*9 + 6] - 2.0) < 1e-10);
|
||||
CHECK(output[c*9 + 7] == 0.0);
|
||||
CHECK(output[c*9 + 8] == 0.0);
|
||||
}
|
||||
for (size_t i = 0; i < 468; i++) CHECK(isfinite(output[i]));
|
||||
for (size_t i = 0; i < 9880; i++) CHECK(input[i] == (i % 2 ? 1e-6 : -1e-6));
|
||||
CHECK(guarded[0] == 12345.0 && guarded[469] == -12345.0);
|
||||
puts("native sanitizer probe passed");
|
||||
return 0;
|
||||
}
|
||||
38
scripts/csi_training/espfi_motion_reduced.py
Normal file
38
scripts/csi_training/espfi_motion_reduced.py
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
"""Optional float64 reduction variant; artifact and default runtime unchanged.
|
||||
|
||||
Reordered reductions require measured <=1e-10 logit error and unchanged classes.
|
||||
No accuracy improvement or speed improvement is implied by this implementation.
|
||||
"""
|
||||
import numpy as np
|
||||
from .espfi_motion_runtime import BufferedMotionBundle
|
||||
|
||||
REDUCTION_CONTRACT = {'precision':'float64','max_abs_logit_error':1e-10,
|
||||
'identical_predictions_required':True,'bit_exact':False,
|
||||
'energy':'per-channel einsum before global RMS and energy floor',
|
||||
'differences':'sum absolute centered differences / length / global RMS',
|
||||
'promotionAuthorized':False}
|
||||
|
||||
|
||||
class ReducedMotionBundle(BufferedMotionBundle):
|
||||
"""Inherits bounded artifact loader, locked prediction and owned outputs."""
|
||||
def _extract(self,sample):
|
||||
np.maximum(sample,1e-6,out=self._centered,dtype=np.float64)
|
||||
np.log(self._centered,out=self._centered)
|
||||
self._centered.mean(axis=-1,keepdims=True,out=self._mean)
|
||||
self._centered -= self._mean
|
||||
np.einsum('ij,ij->i',self._centered,self._centered,out=self._energy,optimize=False)
|
||||
# Compute the recording RMS from original energy, before channel flooring.
|
||||
scale = max(float(np.sqrt(self._energy.sum()/9880)),1e-6)
|
||||
np.maximum(self._energy,190e-12,out=self._energy)
|
||||
for i,lag in enumerate((1,2,4,8,16,32)):
|
||||
np.einsum('ij,ij->i',self._centered[:,lag:],self._centered[:,:-lag],
|
||||
out=self._per_channel[:,i],optimize=False)
|
||||
self._per_channel[:,i] /= self._energy
|
||||
for i,lag in enumerate((1,4,16),6):
|
||||
scratch = self._scratch[:,:190-lag]
|
||||
np.subtract(self._centered[:,lag:],self._centered[:,:-lag],out=scratch)
|
||||
np.abs(scratch,out=scratch)
|
||||
np.add.reduce(scratch,axis=-1,out=self._per_channel[:,i])
|
||||
self._per_channel[:,i] /= 190-lag
|
||||
self._per_channel[:,i] /= scale
|
||||
return self._features
|
||||
160
scripts/csi_training/espfi_motion_reduced_benchmark.py
Normal file
160
scripts/csi_training/espfi_motion_reduced_benchmark.py
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
"""Frozen development-only reduction inference benchmark; never trains or promotes."""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import time
|
||||
|
||||
import numpy as np
|
||||
from .espfi import load_development
|
||||
from .espfi_motion import MotionBundle, _read, _validate
|
||||
from .espfi_motion_compact import BASELINE
|
||||
from .espfi_train import digest
|
||||
|
||||
ROUNDS, CALLS, WARMUPS, SEED = 3, 1000, 100, 42
|
||||
SOURCES = ('espfi_motion_reduced_benchmark.py', 'espfi_motion_reduced.py', 'espfi_motion_runtime.py',
|
||||
'espfi_motion_compact.py', 'espfi_motion.py', 'espfi_kernel.py',
|
||||
'espfi_generalize.py', 'espfi_train.py', 'espfi.py')
|
||||
|
||||
|
||||
def _summary(values):
|
||||
return {'min_ms': float(np.min(values)), 'p50_ms': float(np.percentile(values, 50)),
|
||||
'p95_ms': float(np.percentile(values, 95)),
|
||||
'derived_calls_per_second': float(1000 / np.mean(values))}
|
||||
|
||||
|
||||
def _measure(candidate, reference, dev):
|
||||
rounds = []
|
||||
for number in range(ROUNDS):
|
||||
order = np.random.default_rng(SEED + number).permutation(len(dev))
|
||||
calls = (('reduced', candidate.predict), ('buffered_reference', reference.predict))
|
||||
for i in range(WARMUPS):
|
||||
x = dev[order[i % len(order)]:order[i % len(order)] + 1]
|
||||
for _, fn in calls[::1 if (i + number) % 2 == 0 else -1]:
|
||||
fn(x)
|
||||
values = {name: [] for name, _ in calls}
|
||||
for i in range(CALLS):
|
||||
x = dev[order[i % len(order)]:order[i % len(order)] + 1]
|
||||
for name, fn in calls[::1 if (i + number) % 2 == 0 else -1]:
|
||||
before = time.perf_counter_ns()
|
||||
fn(x)
|
||||
values[name].append((time.perf_counter_ns() - before) / 1e6)
|
||||
result = {name: _summary(v) for name, v in values.items()}
|
||||
result['round'] = number + 1
|
||||
result['seed'] = SEED + number
|
||||
result['gain_at_least_10percent'] = all(
|
||||
result['reduced'][q] <= .9 * result['buffered_reference'][q]
|
||||
for q in ('p50_ms', 'p95_ms'))
|
||||
rounds.append(result)
|
||||
return rounds
|
||||
|
||||
|
||||
def _parity(parts, candidate, references):
|
||||
result = {}
|
||||
for split, part in parts.items():
|
||||
checks = {name: {'max_abs_logit_error': 0., 'identical_predictions': True}
|
||||
for name in references}
|
||||
# Exercise both the timed batch-one path and bounded batch iteration.
|
||||
for size in (1, 32):
|
||||
for offset in range(0, len(part['x']), size):
|
||||
x = part['x'][offset:offset + size]
|
||||
actual = candidate.predict(x)
|
||||
for name, reference in references.items():
|
||||
expected = reference.predict(x)
|
||||
if (actual.shape != expected.shape or actual.dtype != np.float64 or expected.dtype != np.float64
|
||||
or not np.isfinite(actual).all() or not np.isfinite(expected).all()):
|
||||
raise ValueError('Invalid reduced/reference scores')
|
||||
error = float(np.max(np.abs(actual - expected)))
|
||||
equal = bool(np.array_equal(actual.argmax(1), expected.argmax(1)))
|
||||
checks[name]['max_abs_logit_error'] = max(checks[name]['max_abs_logit_error'], error)
|
||||
checks[name]['identical_predictions'] &= equal
|
||||
if error > 1e-10 or not equal:
|
||||
raise ValueError('Reduced runtime parity gate failed')
|
||||
result[split] = {'samples': len(part['x']), 'batch_sizes': [1, 32], 'references': checks}
|
||||
return result
|
||||
|
||||
|
||||
def benchmark(manifest, model, baseline, output):
|
||||
"""Reserve a new receipt before loading inputs; failure remains explicit."""
|
||||
output, manifest, model, baseline = map(Path, (output, manifest, model, baseline))
|
||||
# Exclusive creation never overwrites an existing receipt, including symlinks.
|
||||
with output.open('x') as handle:
|
||||
json.dump({'status': 'running', 'promotionAuthorized': False}, handle)
|
||||
def record(value):
|
||||
with output.open('w') as handle:
|
||||
json.dump(value, handle, indent=2, allow_nan=False)
|
||||
handle.write('\n')
|
||||
started = time.monotonic()
|
||||
try:
|
||||
from .espfi_motion_runtime import BufferedMotionBundle
|
||||
from .espfi_motion_reduced import ReducedMotionBundle
|
||||
paths = {'manifest': manifest, 'compact_artifact': model/'model.safetensors',
|
||||
'compact_metadata': model/'metadata.json', 'compact_status': model/'status.json',
|
||||
'original_artifact': baseline/'model.safetensors',
|
||||
'original_receipt': baseline/'metrics.json', 'original_status': baseline/'status.json'}
|
||||
hashes = {name: digest(path) for name, path in paths.items()}
|
||||
source_hashes = {name: digest(Path(__file__).with_name(name)) for name in SOURCES}
|
||||
parts, provenance = load_development(manifest)
|
||||
_validate(parts)
|
||||
metadata = json.loads(_read(model/'metadata.json', 65536))
|
||||
prior = json.loads(_read(baseline/'metrics.json', 16*1024**2))
|
||||
if (provenance.get('manifest_sha256') != hashes['manifest']
|
||||
or metadata.get('provenance', {}).get('manifest_sha256') != hashes['manifest']
|
||||
or prior.get('provenance', {}).get('manifest_sha256') != hashes['manifest']
|
||||
or metadata.get('baseline_model_sha256') != hashes['original_artifact']
|
||||
or metadata.get('baseline_receipt_sha256') != hashes['original_receipt']):
|
||||
raise ValueError('Input provenance mismatch')
|
||||
if provenance.get('evidence') != 'SYNTHETIC' and {k: len(v['x']) for k, v in parts.items()} != {'train': 837, 'validation': 70}:
|
||||
raise ValueError('Unexpected admitted development population')
|
||||
candidate = ReducedMotionBundle(model)
|
||||
reference = BufferedMotionBundle(model)
|
||||
original = MotionBundle(baseline)
|
||||
if original.candidate != BASELINE:
|
||||
raise ValueError('Unexpected original candidate')
|
||||
parity = _parity(parts, candidate, {'buffered_reference': reference, 'original': original})
|
||||
rounds = _measure(candidate, reference, parts['validation']['x'])
|
||||
if any(digest(path) != hashes[name] for name, path in paths.items()):
|
||||
raise ValueError('Inputs changed during benchmark')
|
||||
if any(digest(Path(__file__).with_name(name)) != value for name, value in source_hashes.items()):
|
||||
raise ValueError('Sources changed during benchmark')
|
||||
report = {'status': 'complete', 'promotionAuthorized': False,
|
||||
'evidence': 'SYNTHETIC' if provenance.get('evidence') == 'SYNTHETIC' else 'MEASURED',
|
||||
'task': 'reduced_motion_inference_benchmark', 'hashes': hashes,
|
||||
'source_sha256': source_hashes, 'provenance': provenance, 'parity': parity,
|
||||
'rounds': rounds, 'retained_runtime': all(r['gain_at_least_10percent'] for r in rounds),
|
||||
'protocol': {'rounds': ROUNDS, 'calls_per_runtime_per_round': CALLS,
|
||||
'warmups_per_runtime_per_round': WARMUPS, 'seed': SEED,
|
||||
'batch_size': 1, 'alternating_runtime_order': True,
|
||||
'output_dtype': 'float64', 'max_abs_logit_error': 1e-10,
|
||||
'exact_argmax_required': True,
|
||||
'max_p50_and_p95_ratio_every_round': .9,
|
||||
'rotating_seeded_development_order': True,
|
||||
'primary_reference': 'BufferedMotionBundle retained float64',
|
||||
'original_reference': 'parity only'},
|
||||
'memory_bytes': {'reduced_model_arrays': int(candidate.model_state_bytes),
|
||||
'reduced_workspace': int(candidate.workspace_bytes),
|
||||
'reference_model_arrays': int(reference.model_state_bytes),
|
||||
'reference_workspace': int(reference.workspace_bytes)},
|
||||
'environment': {'python': platform.python_version(), 'numpy': np.__version__,
|
||||
'platform': platform.platform(), 'machine': platform.machine(),
|
||||
'thread_environment': {k: os.environ.get(k) for k in ('OPENBLAS_NUM_THREADS', 'OMP_NUM_THREADS', 'MKL_NUM_THREADS')}},
|
||||
'elapsed_seconds': time.monotonic() - started,
|
||||
'scope': 'Optional reduced runtime; unchanged BufferedMotionBundle default. Full predict includes input checks, features and classifier. Excludes model load, pooling, acquisition and IO. Throughput is derived from mean batch-one latency, not raw CSI throughput. No training or final-test array access; reused development data, not independent accuracy confirmation.'}
|
||||
record(report)
|
||||
return report
|
||||
except Exception as exc:
|
||||
record({'status': 'failed', 'promotionAuthorized': False, 'error_type': type(exc).__name__})
|
||||
raise
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
for name in ('manifest', 'model', 'baseline', 'output'):
|
||||
parser.add_argument('--' + name, required=True)
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(benchmark(args.manifest, args.model, args.baseline, args.output), indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
107
scripts/tests/test_espfi_motion_native.py
Normal file
107
scripts/tests/test_espfi_motion_native.py
Normal file
|
|
@ -0,0 +1,107 @@
|
|||
"""Explicit C boundary and synthetic parity checks; no sensing claims."""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import ctypes
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import numpy as np
|
||||
sys.path.insert(0,str(Path(__file__).resolve().parents[1]))
|
||||
from csi_training.espfi_motion_native import NativeMotionBundle, _verified_kernel
|
||||
from csi_training.espfi_motion_native_build import build
|
||||
from csi_training.espfi_motion_runtime import BufferedMotionBundle
|
||||
from csi_training.espfi_motion import motion_features
|
||||
from test_espfi_motion_runtime import make_models
|
||||
|
||||
|
||||
@unittest.skipUnless(sys.platform=='linux' and shutil.which('cc'),'Requires Linux system cc')
|
||||
class NativeMotionTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.tmp=tempfile.TemporaryDirectory();cls.root=Path(cls.tmp.name)
|
||||
cls.receipt=build(cls.root/'build');cls.kernel=cls.root/'build'/'kernel.so'
|
||||
cls.parts,cls.original,cls.model=make_models(cls.root)
|
||||
@classmethod
|
||||
def tearDownClass(cls):cls.tmp.cleanup()
|
||||
def bundle(self):return NativeMotionBundle(self.model,self.kernel,self.receipt['library_sha256'])
|
||||
|
||||
def test_parity_boundaries_strides_concurrency_ownership(self):
|
||||
native=self.bundle();reference=BufferedMotionBundle(self.model)
|
||||
rng=np.random.default_rng(39);floor=np.zeros((2,52,190),np.float32)
|
||||
floor[...,::2]=np.nextafter(np.float32(1e-6),np.float32(np.inf))
|
||||
sparse=np.ones((2,52,190));sparse[:,0]=np.exp(np.tile([-1.,1.],95)*1e-5)
|
||||
wave=np.tile([-1.,1.],95)
|
||||
boundaries=[np.broadcast_to(np.exp(wave*a),(2,52,190)).copy()
|
||||
for a in [np.nextafter(1e-6,0),1e-6,np.nextafter(1e-6,np.inf),.999e-6,1.001e-6]]
|
||||
for x in [self.parts['validation']['x'],self.parts['validation']['x'][...,::-1],floor,sparse,np.zeros((2,52,190)),
|
||||
np.ones((2,52,190),dtype=np.int64),np.full((2,52,190),1e100),
|
||||
1e50*np.exp(rng.normal(0,1e-9,(2,52,190))),
|
||||
np.exp(rng.normal(size=(2,52,380)))[...,::2]]+boundaries:
|
||||
features=motion_features(x,'motion_acf')
|
||||
for i,row in enumerate(x):np.testing.assert_allclose(native._extract(row),features[i:i+1],atol=1e-12,rtol=0)
|
||||
actual=native.predict(x)
|
||||
for expected in [reference.predict(x),self.original.predict(x)]:
|
||||
np.testing.assert_allclose(actual,expected,atol=1e-10,rtol=0)
|
||||
np.testing.assert_array_equal(actual.argmax(1),expected.argmax(1))
|
||||
x=self.parts['validation']['x'];expected=native.predict(x)
|
||||
kernel=native._native
|
||||
native._native=lambda *args:2
|
||||
try:
|
||||
with self.assertRaisesRegex(ValueError,'rejected input'):native.predict(x)
|
||||
finally:
|
||||
native._native=kernel
|
||||
np.testing.assert_array_equal(native.predict(x),expected)
|
||||
with ThreadPoolExecutor(max_workers=4) as pool:
|
||||
values=list(pool.map(lambda _:native.predict(x),range(8)))
|
||||
for v in values:np.testing.assert_array_equal(v,expected);self.assertTrue(v.flags.owndata)
|
||||
values[0][:]=999;np.testing.assert_array_equal(values[1],expected)
|
||||
for invalid in [np.ones((1,52,950)),np.full((1,52,190),np.nan),-np.ones((1,52,190))]:
|
||||
with self.assertRaises(ValueError):native.predict(invalid)
|
||||
|
||||
def test_abi_bounds_nonfinite_library_hash_and_private_copy(self):
|
||||
bundle=self.bundle()
|
||||
library=ctypes.CDLL(str(self.kernel));fn=library.espfi_motion_features
|
||||
fn.argtypes=[ctypes.POINTER(ctypes.c_double),ctypes.c_size_t,ctypes.POINTER(ctypes.c_double),ctypes.c_size_t];fn.restype=ctypes.c_int
|
||||
x=np.zeros((52,190));out=np.zeros(468);ptr=lambda a:a.ctypes.data_as(ctypes.POINTER(ctypes.c_double))
|
||||
self.assertNotEqual(fn(None,9880,ptr(out),468),0)
|
||||
self.assertNotEqual(fn(ptr(x),9879,ptr(out),468),0)
|
||||
self.assertNotEqual(fn(ptr(x),9880,None,468),0)
|
||||
self.assertNotEqual(fn(ptr(x),9880,ptr(out),467),0)
|
||||
x[0,0]=np.inf;self.assertNotEqual(fn(ptr(x),9880,ptr(out),468),0)
|
||||
with self.assertRaises(ValueError):self.bundle_with_hash('0'*64)
|
||||
with self.assertRaises(ValueError):self.bundle_with_hash('not-a-digest')
|
||||
link=self.root/'symlink.so';link.symlink_to(self.kernel)
|
||||
with self.assertRaises((ValueError,OSError)):_verified_kernel(link,self.receipt['library_sha256'])
|
||||
huge=self.root/'oversized.so'
|
||||
with huge.open('wb') as h:h.truncate(16*1024**2+1)
|
||||
with self.assertRaises(ValueError):_verified_kernel(huge,'0'*64)
|
||||
self.assertNotEqual(Path(bundle._library._name),self.kernel)
|
||||
self.assertEqual(hashlib.sha256(Path(bundle._library._name).read_bytes()).hexdigest(),self.receipt['library_sha256'])
|
||||
# Compile a deliberately incompatible ABI fixture, never a model-selected path.
|
||||
wrong=self.root/'wrong.c';wrong.write_text('unsigned int espfi_motion_abi(void){return 2;}')
|
||||
target=self.root/'wrong.so';subprocess.run([shutil.which('cc'),'-shared','-fPIC',str(wrong),'-o',str(target)],check=True,capture_output=True)
|
||||
with self.assertRaisesRegex(ValueError,'ABI'):
|
||||
NativeMotionBundle(self.model,target,hashlib.sha256(target.read_bytes()).hexdigest())
|
||||
def bundle_with_hash(self,value):return NativeMotionBundle(self.model,self.kernel,value)
|
||||
|
||||
def test_build_receipt_and_standalone_process(self):
|
||||
receipt=json.loads((self.root/'build'/'build.json').read_text())
|
||||
self.assertEqual(receipt['source_sha256'],hashlib.sha256((self.root/'build'/'kernel.c').read_bytes()).hexdigest())
|
||||
self.assertIn('-fno-fast-math',receipt['flags']);self.assertIn('-ffp-contract=off',receipt['flags'])
|
||||
with self.assertRaises(FileExistsError):build(self.root/'build')
|
||||
code="""import sys,numpy as np
|
||||
from csi_training.espfi_motion_native import NativeMotionBundle
|
||||
b=NativeMotionBundle(*sys.argv[1:]);assert b.predict(np.ones((1,52,190))).shape==(1,7)
|
||||
assert not any(n=='torch' or n.startswith('torch.') or n=='scipy' or n.startswith('scipy.') for n in sys.modules)
|
||||
assert 'csi_training.espfi_motion' not in sys.modules
|
||||
assert 'csi_training.espfi_motion_compact' not in sys.modules
|
||||
"""
|
||||
env=dict(os.environ,PYTHONPATH=str(Path(__file__).resolve().parents[1]))
|
||||
subprocess.run([sys.executable,'-c',code,str(self.model),str(self.kernel),receipt['library_sha256']],check=True,env=env,capture_output=True)
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
132
scripts/tests/test_espfi_motion_native_benchmark.py
Normal file
132
scripts/tests/test_espfi_motion_native_benchmark.py
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
"""Synthetic benchmark contracts; timing here is not sensing evidence."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
from csi_training import espfi_motion_native_benchmark as bench
|
||||
from csi_training.espfi_motion_compact import export
|
||||
from csi_training.espfi_motion_native_build import build
|
||||
from csi_training.espfi_train import digest
|
||||
from test_espfi_context import make_baseline
|
||||
from test_espfi_generalize import fixture
|
||||
|
||||
|
||||
class NativeBenchmarkTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.tmp = tempfile.TemporaryDirectory()
|
||||
cls.root = Path(cls.tmp.name)
|
||||
cls.build_info = build(cls.root/'kernel-build')
|
||||
cls.kernel = cls.root/'kernel-build'/'kernel.so'
|
||||
cls.build_receipt = cls.root/'kernel-build'/'build.json'
|
||||
cls.kernel_sha256 = cls.build_info['library_sha256']
|
||||
cls.manifest = cls.root/'manifest.json'
|
||||
cls.manifest.write_text('{"fixture":"SYNTHETIC"}')
|
||||
cls.parts, cls.provenance = fixture()
|
||||
cls.provenance['manifest_sha256'] = digest(cls.manifest)
|
||||
make_baseline(cls.parts, cls.provenance, cls.root/'baseline')
|
||||
export(cls.parts, cls.provenance, cls.root/'baseline', cls.root/'compact')
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.tmp.cleanup()
|
||||
|
||||
def test_real_runtime_parity_receipt_and_exclusive_output(self):
|
||||
output = self.root/'result.json'
|
||||
def bounded_digest(path):
|
||||
self.assertNotIn(Path(path), (self.kernel, self.build_receipt))
|
||||
return digest(path)
|
||||
with patch.object(bench, 'digest', side_effect=bounded_digest), \
|
||||
patch.object(bench, 'load_development', return_value=(self.parts, self.provenance)) as loader, \
|
||||
patch.object(bench, 'CALLS', 4), patch.object(bench, 'WARMUPS', 1):
|
||||
report = bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output, self.kernel, self.kernel_sha256.upper(), self.build_receipt)
|
||||
loader.assert_called_once_with(self.manifest)
|
||||
self.assertEqual(report['status'], 'complete')
|
||||
self.assertEqual(report['evidence'], 'SYNTHETIC')
|
||||
self.assertFalse(report['promotionAuthorized'])
|
||||
self.assertEqual(len(report['rounds']), 3)
|
||||
self.assertEqual(report['hashes']['compact_metadata'], digest(self.root/'compact'/'metadata.json'))
|
||||
self.assertIn('espfi_motion_native.py', report['source_sha256'])
|
||||
self.assertGreater(report['memory_bytes']['native_workspace'], 0)
|
||||
for split in ('train', 'validation'):
|
||||
for check in report['parity'][split]['references'].values():
|
||||
self.assertLessEqual(check['max_abs_logit_error'], 1e-10)
|
||||
self.assertTrue(check['identical_predictions'])
|
||||
saved = output.read_bytes()
|
||||
with self.assertRaises(FileExistsError):
|
||||
bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output, self.kernel, self.kernel_sha256, self.build_receipt)
|
||||
self.assertEqual(saved, output.read_bytes())
|
||||
|
||||
def test_provenance_mismatch_records_failed_lifecycle(self):
|
||||
output = self.root/'mismatch.json'
|
||||
wrong = dict(self.provenance, manifest_sha256='wrong')
|
||||
with patch.object(bench, 'load_development', return_value=(self.parts, wrong)), \
|
||||
self.assertRaisesRegex(ValueError, 'provenance'):
|
||||
bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output, self.kernel, self.kernel_sha256, self.build_receipt)
|
||||
receipt = json.loads(output.read_text())
|
||||
self.assertEqual(receipt['status'], 'failed')
|
||||
self.assertFalse(receipt['promotionAuthorized'])
|
||||
|
||||
def test_build_provenance_rejected_before_kernel_loading(self):
|
||||
bad = dict(self.build_info, flags=['-ffast-math'])
|
||||
receipt = self.root/'bad-build.json'
|
||||
receipt.write_text(json.dumps(bad))
|
||||
output = self.root/'bad-build-result.json'
|
||||
with patch.object(bench, 'load_development') as loader, self.assertRaisesRegex(ValueError, 'build provenance'):
|
||||
bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output,
|
||||
self.kernel, self.kernel_sha256, receipt)
|
||||
loader.assert_not_called()
|
||||
self.assertEqual(json.loads(output.read_text())['status'], 'failed')
|
||||
|
||||
def test_fifo_inputs_rejected_before_loading(self):
|
||||
fifo = self.root/'input.fifo'
|
||||
os.mkfifo(fifo)
|
||||
for name, kernel, receipt in (('kernel', fifo, self.build_receipt),
|
||||
('receipt', self.kernel, fifo)):
|
||||
output = self.root/('fifo-' + name + '.json')
|
||||
with self.subTest(name=name), patch.object(bench, 'load_development') as loader, \
|
||||
self.assertRaisesRegex(ValueError, 'regular'):
|
||||
bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output,
|
||||
kernel, self.kernel_sha256, receipt)
|
||||
loader.assert_not_called()
|
||||
self.assertEqual(json.loads(output.read_text())['status'], 'failed')
|
||||
|
||||
def test_frozen_measurement_order_and_every_round_gate(self):
|
||||
events = []
|
||||
class Bundle:
|
||||
def __init__(self, name): self.name = name
|
||||
def predict(self, x): events.append((self.name, int(x[0, 0, 0])))
|
||||
dev = np.arange(3).reshape(3, 1, 1)
|
||||
ticks = []
|
||||
current = 0
|
||||
# First two rounds pass; the third fails despite favorable earlier rounds.
|
||||
for round_index in range(3):
|
||||
for call in range(4):
|
||||
order = ['native', 'buffered_reference']
|
||||
if (call + round_index) % 2: order.reverse()
|
||||
for name in order:
|
||||
duration = (50 if round_index < 2 else 110) if name == 'native' else 100
|
||||
ticks.extend((current, current + duration))
|
||||
current += 1000
|
||||
with patch.object(bench, 'CALLS', 4), patch.object(bench, 'WARMUPS', 0), \
|
||||
patch.object(bench.time, 'perf_counter_ns', side_effect=ticks):
|
||||
rounds = bench._measure(Bundle('native'), Bundle('buffered_reference'), dev)
|
||||
self.assertEqual([r['gain_at_least_10percent'] for r in rounds], [True, True, False])
|
||||
self.assertFalse(all(r['gain_at_least_10percent'] for r in rounds))
|
||||
for round_index in range(3):
|
||||
order = np.random.default_rng(42 + round_index).permutation(3)
|
||||
for call in range(4):
|
||||
pair = events[round_index*8 + call*2:round_index*8 + call*2+2]
|
||||
self.assertEqual([p[1] for p in pair], [int(order[call % 3])]*2)
|
||||
expected = ['native', 'buffered_reference']
|
||||
self.assertEqual([p[0] for p in pair], expected[::1 if (call+round_index)%2 == 0 else -1])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
77
scripts/tests/test_espfi_motion_reduced.py
Normal file
77
scripts/tests/test_espfi_motion_reduced.py
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
"""Synthetic reordered-reduction parity, boundary and inherited safety checks."""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import numpy as np
|
||||
sys.path.insert(0,str(Path(__file__).resolve().parents[1]))
|
||||
from csi_training.espfi_motion_reduced import ReducedMotionBundle
|
||||
from csi_training.espfi_motion_runtime import BufferedMotionBundle
|
||||
from csi_training.espfi_motion import motion_features
|
||||
from test_espfi_motion_runtime import make_models
|
||||
|
||||
|
||||
class ReducedMotionTests(unittest.TestCase):
|
||||
def test_reordered_features_logits_and_rounding_boundaries(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
parts,original,path=make_models(Path(tmp))
|
||||
reference=BufferedMotionBundle(path);candidate=ReducedMotionBundle(path)
|
||||
self.assertIs(ReducedMotionBundle.__init__,BufferedMotionBundle.__init__)
|
||||
self.assertIs(ReducedMotionBundle.predict,BufferedMotionBundle.predict)
|
||||
rng=np.random.default_rng(44)
|
||||
floor=np.zeros((2,52,190),dtype=np.float32)
|
||||
floor[...,::2]=np.nextafter(np.float32(1e-6),np.float32(np.inf))
|
||||
# Centered log RMS straddles both the 1e-6 scale floor and 190e-12 energy floor.
|
||||
wave=np.tile([-1.,1.],95)
|
||||
boundaries=[np.broadcast_to(np.exp(wave*a),(2,52,190)).copy()
|
||||
for a in [np.nextafter(1e-6,0),1e-6,np.nextafter(1e-6,np.inf),.999e-6,1.001e-6]]
|
||||
sparse=np.ones((2,52,190),dtype=np.float64)
|
||||
sparse[:,0,:]=np.exp(wave*1e-5)
|
||||
sparse[:,1,:]=np.exp(wave*1e-9)
|
||||
# Constant carriers must not inflate the recording RMS via floored energy.
|
||||
inputs=[parts['validation']['x'],floor,sparse,np.zeros((2,52,190)),np.ones((2,52,190)),
|
||||
np.full((2,52,190),1e100),1e50*np.exp(rng.normal(0,1e-9,(2,52,190))),
|
||||
np.exp(rng.normal(size=(2,52,380)))[...,::2],parts['validation']['x'][...,::-1],
|
||||
np.ones((1,52,190),dtype=np.int64)] + boundaries
|
||||
for x in inputs:
|
||||
features=motion_features(x,'motion_acf')
|
||||
for i,sample in enumerate(x):
|
||||
np.testing.assert_allclose(candidate._extract(sample),features[i:i+1],atol=1e-12,rtol=0)
|
||||
actual=candidate.predict(x)
|
||||
for expected in [reference.predict(x),original.predict(x)]:
|
||||
np.testing.assert_allclose(actual,expected,atol=1e-10,rtol=0)
|
||||
np.testing.assert_array_equal(actual.argmax(1),expected.argmax(1))
|
||||
np.testing.assert_array_equal(actual,np.concatenate([candidate.predict(v[None]) for v in x]))
|
||||
self.assertEqual(candidate.workspace_bytes,reference.workspace_bytes)
|
||||
self.assertEqual(candidate.model_state_bytes,reference.model_state_bytes)
|
||||
|
||||
def test_concurrency_owned_outputs_and_invalid_inputs(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
parts,_,path=make_models(Path(tmp));bundle=ReducedMotionBundle(path)
|
||||
x=parts['validation']['x'];expected=bundle.predict(x)
|
||||
with ThreadPoolExecutor(max_workers=4) as pool:
|
||||
results=list(pool.map(lambda _:bundle.predict(x),range(8)))
|
||||
for actual in results:
|
||||
np.testing.assert_array_equal(actual,expected)
|
||||
self.assertTrue(actual.flags.owndata)
|
||||
results[0][:]=999
|
||||
np.testing.assert_array_equal(results[1],expected)
|
||||
for bad in [np.ones((0,52,190)),np.ones((1,52,950)),np.full((1,52,190),np.inf),
|
||||
-np.ones((1,52,190)),np.ones((1,52,190),dtype=complex)]:
|
||||
with self.assertRaises(ValueError):bundle.predict(bad)
|
||||
np.testing.assert_array_equal(bundle.predict(x),expected)
|
||||
|
||||
def test_inherited_verified_artifact_loader(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
_,_,path=make_models(Path(tmp))
|
||||
model=path/'model.safetensors';original=model.read_bytes()
|
||||
ReducedMotionBundle(path)
|
||||
self.assertEqual(model.read_bytes(),original)
|
||||
model.write_bytes(original+b'corruption')
|
||||
with self.assertRaisesRegex(ValueError,'hash mismatch'):ReducedMotionBundle(path)
|
||||
model.write_bytes(original)
|
||||
(path/'status.json').write_text('{"status":"failed","promotionAuthorized":false}')
|
||||
with self.assertRaisesRegex(ValueError,'Incomplete'):ReducedMotionBundle(path)
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
98
scripts/tests/test_espfi_motion_reduced_benchmark.py
Normal file
98
scripts/tests/test_espfi_motion_reduced_benchmark.py
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
"""Synthetic benchmark contracts; timing here is not sensing evidence."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
from csi_training import espfi_motion_reduced_benchmark as bench
|
||||
from csi_training.espfi_motion_compact import export
|
||||
from csi_training.espfi_train import digest
|
||||
from test_espfi_context import make_baseline
|
||||
from test_espfi_generalize import fixture
|
||||
|
||||
|
||||
class ReducedBenchmarkTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.tmp = tempfile.TemporaryDirectory()
|
||||
cls.root = Path(cls.tmp.name)
|
||||
cls.manifest = cls.root/'manifest.json'
|
||||
cls.manifest.write_text('{"fixture":"SYNTHETIC"}')
|
||||
cls.parts, cls.provenance = fixture()
|
||||
cls.provenance['manifest_sha256'] = digest(cls.manifest)
|
||||
make_baseline(cls.parts, cls.provenance, cls.root/'baseline')
|
||||
export(cls.parts, cls.provenance, cls.root/'baseline', cls.root/'compact')
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.tmp.cleanup()
|
||||
|
||||
def test_real_runtime_parity_receipt_and_exclusive_output(self):
|
||||
output = self.root/'result.json'
|
||||
with patch.object(bench, 'load_development', return_value=(self.parts, self.provenance)) as loader, \
|
||||
patch.object(bench, 'CALLS', 4), patch.object(bench, 'WARMUPS', 1):
|
||||
report = bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output)
|
||||
loader.assert_called_once_with(self.manifest)
|
||||
self.assertEqual(report['status'], 'complete')
|
||||
self.assertEqual(report['evidence'], 'SYNTHETIC')
|
||||
self.assertFalse(report['promotionAuthorized'])
|
||||
self.assertEqual(len(report['rounds']), 3)
|
||||
self.assertEqual(report['hashes']['compact_metadata'], digest(self.root/'compact'/'metadata.json'))
|
||||
self.assertIn('espfi_motion_reduced.py', report['source_sha256'])
|
||||
self.assertGreater(report['memory_bytes']['reduced_workspace'], 0)
|
||||
for split in ('train', 'validation'):
|
||||
for check in report['parity'][split]['references'].values():
|
||||
self.assertLessEqual(check['max_abs_logit_error'], 1e-10)
|
||||
self.assertTrue(check['identical_predictions'])
|
||||
saved = output.read_bytes()
|
||||
with self.assertRaises(FileExistsError):
|
||||
bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output)
|
||||
self.assertEqual(saved, output.read_bytes())
|
||||
|
||||
def test_provenance_mismatch_records_failed_lifecycle(self):
|
||||
output = self.root/'mismatch.json'
|
||||
wrong = dict(self.provenance, manifest_sha256='wrong')
|
||||
with patch.object(bench, 'load_development', return_value=(self.parts, wrong)), \
|
||||
self.assertRaisesRegex(ValueError, 'provenance'):
|
||||
bench.benchmark(self.manifest, self.root/'compact', self.root/'baseline', output)
|
||||
receipt = json.loads(output.read_text())
|
||||
self.assertEqual(receipt['status'], 'failed')
|
||||
self.assertFalse(receipt['promotionAuthorized'])
|
||||
|
||||
def test_frozen_measurement_order_and_every_round_gate(self):
|
||||
events = []
|
||||
class Bundle:
|
||||
def __init__(self, name): self.name = name
|
||||
def predict(self, x): events.append((self.name, int(x[0, 0, 0])))
|
||||
dev = np.arange(3).reshape(3, 1, 1)
|
||||
ticks = []
|
||||
current = 0
|
||||
# First two rounds pass; the third fails despite favorable earlier rounds.
|
||||
for round_index in range(3):
|
||||
for call in range(4):
|
||||
order = ['reduced', 'buffered_reference']
|
||||
if (call + round_index) % 2: order.reverse()
|
||||
for name in order:
|
||||
duration = (50 if round_index < 2 else 110) if name == 'reduced' else 100
|
||||
ticks.extend((current, current + duration))
|
||||
current += 1000
|
||||
with patch.object(bench, 'CALLS', 4), patch.object(bench, 'WARMUPS', 0), \
|
||||
patch.object(bench.time, 'perf_counter_ns', side_effect=ticks):
|
||||
rounds = bench._measure(Bundle('reduced'), Bundle('buffered_reference'), dev)
|
||||
self.assertEqual([r['gain_at_least_10percent'] for r in rounds], [True, True, False])
|
||||
self.assertFalse(all(r['gain_at_least_10percent'] for r in rounds))
|
||||
for round_index in range(3):
|
||||
order = np.random.default_rng(42 + round_index).permutation(3)
|
||||
for call in range(4):
|
||||
pair = events[round_index*8 + call*2:round_index*8 + call*2+2]
|
||||
self.assertEqual([p[1] for p in pair], [int(order[call % 3])]*2)
|
||||
expected = ['reduced', 'buffered_reference']
|
||||
self.assertEqual([p[0] for p in pair], expected[::1 if (call+round_index)%2 == 0 else -1])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue