mirror of
https://github.com/Helldez/BigMoeOnEdge.git
synced 2026-10-03 03:25:42 +00:00
b1f42cbdc7
scripts/ had drifted into a pile of near-duplicate one-off drivers. Four PS1
files each carried a byte-identical Run-Cfg (cooldown, adb into bench-run.sh,
filter the perf lines, pull the artifacts); three Python tools each re-derived
the same readers, one of them saying so out loud ("Mirrors scripts/
route-analyze.py's reader").
bench-lib.ps1 now holds the shared driver plumbing, so a driver is its config
matrix and nothing else. trace_io.py holds the reading contract every artifact
shares: `key=value` tokens on `#` lines, rows below, unknown keys kept.
The device model paths were hardcoded in scripts that were otherwise
parameterised; they are now defaults behind -Qwen/-Gemma params (PS1) and
${VAR:-default} (sh), which is what they always were in spirit.
Retired framing, pruned from the tools that still run:
- bench-analyze.py listed `sg_ov` (speculative gating — removed from the
engine, PR #15) and labelled --cache-mb auto "adaptive cache" with a
`resizes` column. The governor that resized is gone, so that count is 0 for
every run bmoe-cli can produce, and a column that is structurally always 0
reads as a finding rather than a blank. Dropped; auto is "auto-sized".
- bench-pr23-c2000.ps1 grepped for `moe-spec-gate:`, a line the engine no
longer emits. Its --prefetch A/B is still valid, so it moves to the lib.
bench-matrix-rework.ps1 and bench-pr23-summary.py are NOT retrofitted: they only
re-derive tables already published under docs/bench-data, and their spec-gate
cells cannot run against a current build. They are marked ARCHIVED with why —
deleting them would leave published numbers with no visible derivation.
bench-lib.ps1 also documents what the copies silently carried: the cooldown is a
timer, and a timer does not guarantee a thermal baseline (see the contaminated
matrices where tok/s tracked run order). Fixing that needs device calibration;
saying so beats leaving the next reader to rediscover it.
Verified against committed data, not just by inspection:
- route-analyze --view hot/reuse/overlap/cache and decode-analyze: output
byte-identical before and after (docs/bench-data/2026-07-15-route-trace).
- bench-analyze on docs/bench-data/2026-07-13: every figure matches the
committed summary.md digit for digit; only labels changed and sg_ov is gone.
- All five PS1 files parse; the dot-sourced Invoke-BenchCfg builds the exact
same adb command string the copies did.
34 lines
1.5 KiB
PowerShell
34 lines
1.5 KiB
PowerShell
# On-device benchmark matrix driver: the cache-size × read-lane grid, plus the overlap pair, per
|
||
# model. Invokes the instrumented device-side bench-run.sh via bench-lib.ps1.
|
||
param(
|
||
[string]$OutDir = (Join-Path (Split-Path $PSScriptRoot -Parent) ".bench"),
|
||
[int]$NPred = 256,
|
||
[int]$CooldownSec = 45, # let the SoC cool toward a similar baseline before each run
|
||
[string]$Qwen,
|
||
[string]$Gemma
|
||
)
|
||
$ErrorActionPreference = "Continue"
|
||
. "$PSScriptRoot\bench-lib.ps1"
|
||
New-Item -ItemType Directory -Force -Path $OutDir | Out-Null
|
||
|
||
if (-not $Qwen) { $Qwen = $BENCH_QWEN }
|
||
if (-not $Gemma) { $Gemma = $BENCH_GEMMA }
|
||
|
||
function Run-Cfg($tag, $model, $flags) {
|
||
Invoke-BenchCfg -Tag $tag -Model $model -Flags $flags -OutDir $OutDir -NPred $NPred -CooldownSec $CooldownSec
|
||
}
|
||
|
||
$models = @{ qwen = $Qwen; gemma = $Gemma }
|
||
foreach ($name in @("qwen","gemma")) {
|
||
$m = $models[$name]
|
||
Run-Cfg "${name}_mmap" $m ""
|
||
Run-Cfg "${name}_stream" $m "--moe-stream"
|
||
Run-Cfg "${name}_c2000_l2" $m "--moe-stream --cache-mb 2000 --io-threads 2"
|
||
Run-Cfg "${name}_c2000_l4" $m "--moe-stream --cache-mb 2000 --io-threads 4"
|
||
Run-Cfg "${name}_c4000_l2" $m "--moe-stream --cache-mb 4000 --io-threads 2"
|
||
Run-Cfg "${name}_c4000_l4" $m "--moe-stream --cache-mb 4000 --io-threads 4"
|
||
# intra-layer I/O–compute overlap: mirror the two lead configs with --overlap
|
||
Run-Cfg "${name}_c4000_l4_ov" $m "--moe-stream --cache-mb 4000 --io-threads 4 --overlap"
|
||
Run-Cfg "${name}_stream_ov" $m "--moe-stream --cache-mb 0 --io-threads 4 --overlap"
|
||
}
|
||
Write-Host "ALL DONE"
|