vrr/BigMoeOnEdge
2
0
Fork 0
mirror of https://github.com/Helldez/BigMoeOnEdge.git synced 2026-10-03 03:25:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs(app): say what each drop-cold-experts rung costs, not just what the knob does (#96)

Browse source 
The description explained the mechanism twice and the choice not at all,
which is backwards: a user opening this setting has already decided to
try it and only needs to know which rung to pick.

The rung labels now carry the trade ("barely bites" / "recommended" /
"fastest, roughest") and the blurb loses the paragraph that restated the
mechanism, gaining instead what separates 75 from 100: the measured
speeds, and that the top rung discards twice as much of the routing to
get there. Both numbers are marked as one model, since that is all that
has been measured.
This commit is contained in:
Helldez 2026-07-22 20:03:54 +02:00 • committed by GitHub
parent bea5a0b99e
commit 10f2bdd31d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 19 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

28
examples/android/app/src/main/java/io/bigmoeonedge/example/SettingsScreen.kt
View file

@ -146,21 +146,31 @@ fun SettingsScreen(current: AppSettings, onChange: (AppSettings) -> Unit, onBack
)
IntSetting(
"Drop cold experts (% of even share)", AppSettings.DROP_COLD_CHOICES, current.dropColdPct,
format = { if (it == 0) "off" else "$it%" },
// The rung labels carry the trade, so the blurb below does not have to repeat
// it: which value to pick is the only real question this setting poses.
format = {
when (it) {
0 -> "off"
50 -> "50% — barely bites"
75 -> "75% — recommended"
100 -> "100% — fastest, roughest"
else -> "$it%"
}
},
// Unlike top-k, this one asks the expert source what is resident, so it needs
// both the streamer and a live cache — the same condition prefetch is under.
enabled = !current.mmap &&
(current.cacheMb == AppSettings.CACHE_AUTO || current.cacheMb > 0),
) { onChange(current.copy(dropColdPct = it)) }
Text(
"Experimental. What slows a token down is reading an expert that is not already in RAM. " +
"This skips such an expert when the router barely wanted it anyway — below the chosen " +
"share of an even split. With 8 active experts an even split is 12.5% each, so 75% " +
"means \"skip it if it carries less than 9.4% of the routing\".\n\n" +
"Experts already in RAM always run, however small their weight: they cost no read. The " +
"strongest expert of each routing is never skipped.\n\n" +
"Higher is faster and rougher. Like Active experts, the reply changes — but unlike it, " +
"not the same way twice: what gets skipped depends on what the cache happened to hold.",
"Reading an expert that is not already in RAM is what slows a token down. This skips " +
"one — but only if the router barely wanted it, below the chosen share of an even " +
"split. With 8 active experts an even split is 12.5%, so 75% skips anything under " +
"9.4%. Experts already in RAM always run; the strongest is never skipped.\n\n" +
"50% barely fires and changes little. 75% measured +55% faster and 100% +84% on one " +
"model, but the top rung discards twice as much of the routing for that extra speed." +
"\n\nThe reply changes, and unlike Active experts not the same way twice: what gets " +
"skipped depends on what the cache happened to hold.",
fontSize = 12.sp, color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}