vrr/BigMoeOnEdge
2
0
Fork 0
mirror of https://github.com/Helldez/BigMoeOnEdge.git synced 2026-10-03 03:25:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(io): do not read in place under overlap on Windows

Browse source 
The combination produces wrong bytes with no error reported. Measured against a real model: same revision, same gguf, same flags, correct text on Android and garbage on Windows. Everything on this side is proven correct (requests, destination arithmetic, commit ranges, alignment, the partial-read retry), so the fault is below us; the difference is that in place the DMA target is freshly committed shared memory a compute thread is concurrently reading, where the bounce had a private buffer and a CPU copy ordered against the ready flag. Serial is correct on Windows, so only the overlapped path falls back, and it says so rather than degrading in silence. Tracked in issue #149.
This commit is contained in:
Helldez 2026-08-02 01:14:48 +02:00
parent 4a42df3c47
commit 5ff72ef086
1 changed files with 16 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
core/src/moe/expert_stream_source.cpp
View file

@ -51,6 +51,22 @@ bool ExpertStreamSource::init(const std::vector<std::string> & shard_paths,
// layer buffers are reserved before the reader opens; if O_DIRECT is then refused and the reader
// falls back to buffered, the placement is merely unused, never wrong.
zero_copy_ = cfg.odirect_zero_copy && cfg.o_direct;
#if defined(_WIN32)
// Windows + overlap + in-place produces WRONG BYTES, silently. Measured against a real model:
// the same revision, the same gguf and the same flags generate correct text on Android and
// garbage on Windows, with no read or commit error reported. Everything on this side is proven
// correct (requests, destination arithmetic, commit ranges, alignment, the partial-read retry),
// so the fault is below us in the Windows I/O or VM layer and the difference is that here the
// DMA target is freshly committed shared memory a compute thread is concurrently reading,
// where the bounce path had a private buffer and a CPU copy ordered against the ready flag.
// Serial is correct on Windows, so the bounce is only forced for the overlapped path.
// Tracked in https://github.com/Helldez/BigMoeOnEdge/issues/149.
if (zero_copy_ && overlap_) {
zero_copy_ = false;
std::fprintf(stderr, "bmoe: odirect-zero-copy disabled on Windows under --overlap (issue #149); "
"reads go through the bounce buffer\n");
}
#endif
cache_max_ = (size_t) std::max(0, cfg.cache_mb) * 1024ull * 1024ull;
io_threads_ = std::max(1, std::min(MoeStreamConfig::io_threads_max, cfg.io_threads));
page_ = pio::vm_page(); // the real OS page size, for the dense-residency probe in any cache mode